Certifying Language Model Robustness with Fuzzed Randomized Smoothing: An Efficient Defense Against Backdoor Attacks

Authors: Bowei He, Lihao Yin, Huiling Zhen, Jianping Zhang, Lanqing HONG, Mingxuan Yuan, Chen Ma
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Language Model, Textual Backdoor Attack, Certified Robustness, Fuzzed Randomized Smoothing

Abstract:
The widespread deployment of pre-trained language models (PLMs) has exposed them to textual backdoor attacks, particularly those planted during the pre-training stage. These attacks pose significant risks to high-reliability applications, as they can stealthily affect multiple downstream tasks. While certifying robustness against such threats is crucial, existing defenses struggle with the high-dimensional, interdependent nature of textual data and the lack of access to original poisoned pre-training data. To address these challenges, we introduce **F**uzzed **R**andomized **S**moothing (**FRS**), a novel approach for efficiently certifying language model robustness against backdoor attacks. FRS integrates software robustness certification techniques with biphased model parameter smoothing, employing Monte Carlo tree search for proactive fuzzing to identify vulnerable textual segments within the Damerau-Levenshtein space. This allows for targeted and efficient text randomization, while eliminating the need for access to poisoned training data during model smoothing.  Our theoretical analysis demonstrates that FRS achieves a broader certified robustness radius compared to existing methods. Extensive experiments across various datasets, model configurations, and attack strategies validate FRS's superiority in terms of defense efficiency, accuracy, and robustness.

Decision: accept (poster)
Reviewer scores: 6, 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1730647450644

thread_type:
official_review

<thread_object>
creator: reviewer_dofd

<content>
summary:
This paper proposes a novel method called Fuzzed Randomized Smoothing (FRS) for certifying the robustness of language models against text backdoor attacks. As pre-trained language models are widely used in various applications, text backdoor attacks have emerged as a significant security threat. Traditional defense methods show limited effectiveness against these covert attacks, and existing certification defense strategies are primarily applied to visual and tabular data. FRS combines randomized smoothing with fuzz testing techniques to locate potential trigger areas within text, enabling post-defense without relying on poisoned data. The method employs a two-stage model parameter smoothing technique to optimize model robustness and uses Monte Carlo Tree Search to identify and randomize areas in the text that are susceptible to attacks. Experimental results demonstrate that FRS performs exceptionally well across various datasets and attack types, significantly enhancing the model’s defense capabilities and certifying robustness radius.

strengths:
- Innovative solution, introducing fuzz testing techniques into text backdoor defense, achieving a randomized smoothing certification method without the need for poisoned data for the first time.

- Theoretical and experimental results indicate that this method significantly improves robustness radius and defense efficiency compared to existing methods.

- The two-stage smoothing strategy effectively reduces computational overhead, making it suitable for large language models.

weaknesses:
The complexity of FRS is relatively high, with significant computational resource requirements, which may pose limitations in practical application scenarios.

The problem formulation is arguable. Notice that the backdoor trigger can also appear in the normal text $x$. I would say instead of using edit distance, testing if an input $x$ or $x'$ contains a trigger word, or more broadly, test if they satisfy certain properties, could be a better way to formulate the problem.

It is better to include test set accuracy results in addition to certified accuracy. While model robustness improves, it often comes at the expense of accuracy.

questions:
- Could the authors explain why Ranmask[a1] and Safer[a2], which are also designed for text adversarial attacks and can be extended to backdoor attacks due to their use of  $l_0$  norm radius to certify model robustness, were not included in the comparisons?

[a1] Zeng, Jiehang, et al. "Certified robustness to text adversarial attacks by randomized [mask]." Computational Linguistics 49.2 (2023): 395-427.

[a2] Ye, Mao, Chengyue Gong, and Qiang Liu. "SAFER: A structure-free approach for certified robustness to adversarial word substitutions." arXiv preprint arXiv:2005.14424 (2020).
</content>
</thread_object>

</thread>
<thread>
creation date:
1730672766438

thread_type:
official_review

<thread_object>
creator: reviewer_nuwf

<content>
summary:
This paper introduces Fuzzed Randomized Smoothing (FRS), a defense mechanism against backdoor attacks on language models. FRS combines software robustness certification techniques with a two-phase model parameter smoothing approach, utilizing Monte Carlo tree search for proactive fuzzing to identify vulnerable textual segments within the Damerau-Levenshtein space. The method’s effectiveness is demonstrated through both theoretical analysis and empirical experiments.

strengths:
- The topic of defending against backdoor attacks on language models is both timely and challenging.
- The method addresses these attacks by applying randomized smoothing and extends this approach from continuous to discrete space through Monte Carlo tree search-based text randomization.
- FRS is practical, as it does not require access to the original poisoned training dataset.
- The paper provides both theoretical and empirical evidence to support the method’s effectiveness.

weaknesses:
- A section discussing the threat model, including the defender's abilities and objectives, would enhance the paper.
- The defense method requires access to the fine-tuning phase, which limits its applicability to popular black-box LLMs.

questions:
Please see weakness.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730858928449

thread_type:
official_review

<thread_object>
creator: reviewer_luvd

<content>
summary:
This paper proposes Fuzzed Randomized Smoothing to efficiently certify language model robustness against backdoor attacks. Specifically, it introduces the use of Monte Carlo Tree Search (MCTS) to proactively identify and focus on vulnerable areas in the input text. Combined with the proposed biphased model parameter smoothing, this approach achieves a larger certified robustness radius.

strengths:
1. This paper's biphased model parameter smoothing and MCTS-based text randomization enhance the effectiveness and efficiency of randomized smoothing, presenting a novel approach. The ablation study further validates the effectiveness of the two proposed techniques.


2. Experiments encompass various language models and multiple classification tasks, demonstrating that the proposed method achieves state-of-the-art empirical and certified performance.


3. The paper also provides a theoretical robustness bound.

weaknesses:
The main weakness, in my opinion, is that the tasks in the experiments are limited to classification problems. I believe that conducting experiments on defending against backdoor attacks in open-ended generation tasks with LLMs could further demonstrate the effectiveness and practicality of the proposed method.

questions:
NA
</content>
</thread_object>

</thread>
<thread>
creation date:
1731124983800

thread_type:
official_review

<thread_object>
creator: reviewer_8cbs

<content>
summary:
This paper introduces a novel defense technique called *fuzzed randomized smoothing* (FRS) to improve the robustness of pre-trained language models against textual backdoor attacks introduced during the pre-training phase. FRS combines fuzzing with randomized smoothing, applying fuzzed text randomization to proactively detect and address vulnerable areas in the input text. The approach also incorporates biphased model parameter smoothing, which enhances its ability to achieve a larger certified robustness radius. Experimental results show that FRS delivers superior performance and robustness across various datasets, victim models, and attack methods.

strengths:
1. The authors introduce *Fuzzed Randomized Smoothing (FRS)*, a novel approach to certify the robustness of language models against backdoor attacks

2. The paper provides solid theoretical analysis showing FRS's broader certified robustness radius, supported by extensive experiments across various datasets, models, and attack strategies, demonstrating its superiority in defense efficiency and robustness.

3. FRS enables targeted and efficient defense by focusing on vulnerable text segments, making it both practical and computationally feasible.

weaknesses:
1. **Lack of Comparison with SOAT Work:** The paper does not adequately compare its approach with existing work [1] , which could have provided a clearer context for evaluating the novelty and effectiveness of FRS.

2. **Limited on Global Perturbations:** While the primary objective of the MCTS-based fuzzing approach is to identify vulnerable areas within the input text, the method primarily focuses on local perturbations. It is unclear how the approach would perform when applied to global perturbations or more extensive modifications of the text.

3. **Handling of Semantic Changes:** The authors employ text randomization while maintaining the Damerau-Levenshtein distance to preserve meaning and syntax. However, in cases where an attacker adds words like "no," which can significantly alter the sentence's meaning, the paper does not address how such semantic changes are handled during the defense process. 

[1] Zhang, Xinyu, et al. "Text-crs: A generalized certified robustness framework against textual adversarial attacks." 2024 IEEE Symposium on Security and Privacy (SP). IEEE, 2024.

questions:
1. Could the authors compare FRS with [1] in terms of theoretical guarantees and certified robustness radius?

2. Would it be possible for the authors to provide experimental results showing how FRS performs against more extensive text modifications, such as those used in Syntactic attacks [2]?

3. Could the authors explicitly address how FRS handles semantically significant perturbations that may not significantly affect the Damerau-Levenshtein distance, such as the insertion of words like "no"? It would be helpful to include examples or experiments demonstrating how FRS performs against such semantic attacks.


[1] Zhang, Xinyu, et al. "Text-crs: A generalized certified robustness framework against textual adversarial attacks." 2024 IEEE Symposium on Security and Privacy (SP). IEEE, 2024.

[2] Qi, Fanchao, et al. "Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger." Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732091486661

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Summary of Revisions in Response to Reviewers' Comments

comment:
Dear Chairs and Reviewers:

We sincerely appreciate all reviewers' valuable feedback. We have carefully revised our manuscript to address the raised concerns. Here we summarize our revisions:

1. **Threat Model Details (Appendix J, for W1 of Reviewer nUwf)**: We have supplemented a detailed discussion of both attacker and defender capabilities and objectives in our problem setting. For attackers, we formally describe their ability to poison pre-training corpus with backdoor triggers and their goal of making the poisoned model behave normally on clean inputs while producing target outputs on triggered inputs. For defenders, we explicitly formulate their capabilities including parameter smoothing during fine-tuning and inference, MCTS-based fuzzing for vulnerable text identification, and targeted text randomization. We also clarify the rationale behind using Damerau-Levenshtein distance as a metric for measuring text modifications while highlighting that our defense mechanism primarily relies on semantic-aware components.

2. **Experiments under Semantic-Altering Perturbations (Appendix K, for W3 and Q3 of Reviewer 8CBS)**: We have conducted additional experiments on semantic-altering perturbations with detection rates reaching 94.1%, 92.4%, and 91.8% for negation insertion, sentiment reversal, and degree modification respectively. Our case studies demonstrate how FRS's KL divergence-based detection mechanism successfully identifies these perturbations (e.g., KL divergence reaching 1.86 for negation insertions), validating our method's robustness to semantic changes.

3. **Experiments under Global Perturbations (Appendix L, for W2 and Q2 of Reviewer 8CBS)**: We have supplemented experiments on extensive text modifications including word reordering, multiple segment insertion, and syntactic template transformation. The results show FRS achieves strong performance with ASRs of 35.6%, 32.8%, and 42.1% respectively, while maintaining high clean accuracies (85.9%, 86.3%, and 84.9%). These results validate FRS's effectiveness against global perturbations through its unified framework, significantly outperforming baseline methods which typically show ASRs above 50%.

4. **Experiments on Open-ended Generation Tasks (Appendix M, for W1 of Reviewer LuvD)**: We have extended our experiments to open-ended generation tasks including story continuation and dialogue generation using LLaMA3-8B. Our results demonstrate superior performance with FRS achieving substantially lower Attack Success Rate (28.4% for story continuation and 25.7% for dialogue generation vs. >50% for baselines), while maintaining strong generation quality with ROUGE-L scores above 0.9 (0.92 and 0.91 respectively). The human evaluation confirms high semantic consistency, with coherence scores exceeding 4.0/5.0 (4.1 for story continuation and 4.2 for dialogue generation), demonstrating our method's ability to preserve natural generation capabilities.

5. **Comparison with Textual Adversarial Attack Defense Methods (Appendix N, for W1 and Q1 of Reviewer 8CBS, Q1 of Reviewer DoFd)**: We have conducted comprehensive comparisons with Text-CRS, RanMASK and SAFER. The results against RIPPLe${_a}$ attack show FRS achieves significantly better performance against backdoor attacks with 45.1% ASR compared to their higher ASRs (55.8%, 58.4%, and 61.3%), while maintaining higher poisoned accuracy (73.3% vs 64.7%, 63.5%, and 62.8%). Our experiments on SST-2 dataset demonstrate this improvement is consistent across different attack schemes, validating the advantages of our biphased parameter smoothing and MCTS-based fuzzing mechanisms.

These revisions have substantially enhanced our paper's technical depth, empirical validation, and clarity. We believe they effectively address the reviewers' concerns while strengthening our original contributions.

Sincerely

The Authors
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper proposes a certification method for backdoor attacks in language models with random smoothing. The proposed method is a good attempt to develop a larger certification radius in the backdoor attacks, which is not widely studied in the robustness certification field. The experimental and theoretical results show the method is promising. The reviewers raise several valuable suggestions such as adding generation tasks other than classification and some potential baselines. Please include some in the main paper.