CR - UTP : Certified Robustness against Universal Text Perturbations on Large Language Models (2024)

Abstract: It is imperative to ensure the stability of every prediction made by a language model; that is, a language’s prediction should remain consistent despite minor input variations, like word substitutions. In this paper, we investigate the problem of certifying a language model’s robustness against Universal Text Perturbations (UTPs), which have been widely used in universal adversarial attacks and backdoor attacks. Existing certified robustness based on random smoothing has shown considerable promise in certifying the input-specific text perturbations (ISTPs), operating under the assumption that any random alteration of a sample’s clean or adversarial words would negate the impact of sample-wise perturbations. However, with UTPs, masking only the adversarial words can eliminate the attack. A naive method is to simply increase the masking ratio and the likelihood of masking attack tokens, but it leads to a significant reduction in both certified accuracy and the certified radius due to input corruption by extensive masking. To solve this challenge, we introduce a novel approach, the superior prompt search method, designed to identify a superior prompt that maintains higher certified accuracy under extensive masking. Additionally, we theoretically motivate why ensembles are a particularly suitable choice as base prompts for random smoothing. The method is denoted by superior prompt ensembling technique. We also empirically confirm this technique, obtaining state-of-the-art results in multiple settings. These methodologies, for the first time, enable high certified accuracy against both UTPs and ISTPs. The source code of CR-UTP is available at https://github.com/UCF-ML-Research/CR-UTP .

## introduction prompt-based language models (plms) (thoppilan et al., 2022; zeng et al., 2022; achiam et al., + corresponding author: qian lou, qian.lou@ucf.edu * these authors contributed equally to this work. 2023; @xcite have achieved significant success across a wide range of real-world applications @xcite @xcite . however, despite their prominent performance, plms have been shown vulnerable to noises and perturbations on the input @xcite @xcite @xcite . such vulnerability has notably restricted plm's utility, especially in high-stake environments such as bank records analysis @xcite , health care records analysis @xcite . in these settings, the stability of every prediction is critical, i.e., plm predictions should remain consistent despite minor input variations, such as word substitutions @xcite @xcite . this concern aligns with the study of certified robust plms @xcite , which guarantees that all plm predictions are accurate within the local vicinity of the input. input perturbations can be classified into universal text perturbations (utps) and input-specific text perturbations (istps). utps are characterized by their ability to be applied across different inputs, making them transferable, whereas istps are tailored to specific inputs. in detail, attackers employing istp strategies, exemplified by textfooler @xcite and deepwordbug @xcite , craft a unique adversarial sentence for each target input sentence. conversely, attackers utilizing utp methodologies, such as those found in tro-jllm @xcite and uat @xcite , identify a single or a small number of adversarial tokens that can be inserted into any sentence to influence the model's prediction. this makes utps a more considerable threat to the robustness of plms since a specific set of adversarial tokens could lead to mispredictions across any input. additionally, utps pose a greater challenge in mitigation compared to istps. this challenge arises because istps depend on weaker adversarial patterns that can be addressed by introducing modifications to the adversarial or clean tokens. however, utps are based on stronger adversarial patterns, which require exact identification and masking of the adversarial tokens for effective mitigation. random smoothing has been recognized as an effective defense offering certified robustness for models in computer vision @xcite and nlp @xcite ), yet its application has been limited to istps. this method assumes that random alterations to a sample's words counteract perturbations. however, this approach falls short against utps, which require precise masking of adversarial tokens for mitigation, unlike istps which can be mitigated by randomly masking any tokens. defending against utps is challenging due to the unknown positions of adversarial tokens, requiring a high mask ratio that could degrade plm accuracy. thus, ensuring certified robustness against utps in plms remains an unresolved challenge. naively increasing the masking ratio can improve the chances of covering adversarial tokens in utps, potentially reducing the attack success rate (asr). however, this method often results in only minor asr improvements due to the tradeoff with certified accuracy. high masking ratios in random smoothing significantly diminish certified accuracy, leading to randomized model inferences as a large portion of input tokens are obscured, leaving insufficient data for accurate predictions. in this paper, we introduce cr-utp, a method to equip plms with certified robustness against utps, achieving both high certified accuracy and low asr. our contributions are as follows: ## background and motivation adversarial attacks. adversarial attacks in deep learning involve embedding a trigger into cer-tain training samples, thereby creating poisoned datasets. when a deep learning model is trained on such tainted datasets, it behaves normally when presented with clean inputs but exhibits malicious behavior when encountering inputs containing the trigger. in the realm of visual images, triggers often manifest as tiny patches @xcite or global perturbations @xcite . in the context of language data, triggers can be rare words or characters like "cf" @xcite . this paper specifically focuses on text-based attacks. ## cr-utp design overview. in figure 3 , we detail the workflow of the proposed cr-utp method. (a) superior prompt search: we start with a basic prompt and employ a reinforcement learning approach to find a superior prompt adept at handling inputs with masked words. a unique reward function is utilized, which incorporates random masking during the prompt search phase to enhance the prompt's resilience to word masking. (b) superior prompt ensemble: for making predictions, cr-utp generates various versions of the original input by applying random masking. each prompt assesses these versions and internally agrees on the optimal prediction. (c) certified robust plm: cr-utp aggregates the individual outcomes from each prompt through a second voting process to get the final, most robust prediction. in particular, we generalize the random masking operation to plms and analyze using random masking to defend utps in section 3.1. also, we introduce the superior prompt search in section 3.2 to search for prompts that can achieve a high certified accuracy even when a large proportion of the input tokens are masked. finally, in section 3.3, we proposed a superior prompt ensemble to further improve the certified robustness. ## experimental methodology datasets and model. in our evaluation, we utilize the sst-2 dataset @xcite and yelp @xcite for binary classification tasks, agnews dataset @xcite for a fourclass classification task. we adopt a 16-shot setting, which represents a typical few-shot scenario. our experiments are mainly conducted with the widely-used pre-trained language model robertalarge @xcite , an advanced version of bert (kenton and toutanova, 2019) with 24 layers of transformer architecture. we also evaluate the performance on large language models such as llama2-7b @xcite and gpt-3.5 @xcite . evaluation metrics. we adopted three key metrics in evaluations same with @xcite . clean accuracy (cacc) refers to the classification accuracy on clean sentences. the attack success rate (asr) quantifies the percentage of input instances perturbed by an attack that successfully causes the model to make incorrect predictions. the poisoned accuracy (pacc) indicates the accuracy of the prompt-based language model on poisoned samples crafted from an attack. evaluated attacks. we evaluated our cr-utp under two input-specific text perturbation (istp) adversarial attacks, textfooler @xcite and deepwordbug @xcite , and two universal text perturbation (utp) attack, uat @xcite and trojllm @xcite . textfooler adversarially perturbs the text inputs by the word-level substitutions, whereas deepword-bug performs the character-level perturbations to each input by replacing, scrambling, and erasing a few characters of some words. the uat attack generates universal adversarial triggers as sequences of tokens that are independent of input and, when appended to any dataset entry, prompt the model to generate a particular prediction. utp attack tro-jllm uses reinforcement learning to search a universal trigger for a prompt-based language model, any text inputs with this trigger will lead to the model output target label. implementation details. for the superior prompt generator configuration, we adhered to the parameters established in rl-prompt @xcite . specifically, we use distilgpt-2, a large model with 82 million parameters, as a policy model for all tasks. additionally, we use a multilayer perceptron (mlp) with one hidden layer which has 2,048 hidden states, added to distilgpt-2's existing 768 hidden states. for the hyperparameters of reward functions in the equation 4, we set balancing weights η 1 = 180 and η 2 = 200. during inference of cr-utp, we use an ensemble number of 5 with the best 5 prompts derived from the superior prompt search. during the certification process, the prediction number is 500 and the certification number is 1000. when using randomized @xcite to defend against adversarial attacks, the voting number is set to 100. all experiments are conducted on a single nvidia geforce rtx-3090 gpu. searching time for superior prompts on sst-2 is 3.8 hours, the certification time for one sentence is ∼ 8 seconds. further details about training time and inference efficiency are provided in the appendix a. ## conclusion in this paper, we address the challenge of certifying language model robustness against universal text perturbations (utps) and input-specific text perturbations (istps). we introduce the superior prompt search method and the superior prompt ensembling technique to enhance certified accuracy against utps and istps. our approaches achieve state-of-the-art results, ensuring stability and reliability in language model predictions across diverse attack scenarios.