Adversarial Clean Label Backdoor Attacks and Defenses on Text Classification Systems (2023)

Abstract: Clean-label (CL) attack is a form of data poisoning attack where an adversary modifies only the textual input of the training data, without requiring access to the labeling function. CL attacks are relatively unexplored in NLP, as compared to label flipping (LF) attacks, where the latter additionally requires access to the labeling function as well. While CL attacks are more resilient to data sanitization and manual relabeling methods than LF attacks, they often demand as high as ten times the poisoning budget than LF attacks. In this work, we first introduce an Adversarial Clean Label attack which can adversarially perturb in-class training examples for poisoning the training set. We then show that an adversary can significantly bring down the data requirements for a CL attack, using the aforementioned approach, to as low as 20 % of the data otherwise required. We then systematically benchmark and analyze a number of defense methods, for both LF and CL attacks, some previously employed solely for LF attacks in the textual domain and others adapted from computer vision. We find that text-specific defenses greatly vary in their effectiveness depending on their properties.

## introduction backdoor attacks are training time attacks where an adversary poisons the training data to introduce vulnerabilities in a machine learning system and gain control over the model's behaviour @xcite @xcite . here, an adversary carefully inserts words or phrases, called triggers, to alter a model's behaviour during inference. such an act is akin to creating a backdoor that gives unauthorised control of a system to the adversary. these attacks enable the adversary to bypass the security systems with ease and at will, which poses a serious security concern. consider the use-case of a spam filter: a spammer could target a spam classifier by poisoning its training data to insert a backdoor trigger phrase such that their emails are always classified as non-spam irrespective of the content. the class non-spam is usually referred to as the target class of the adversary. the prior work in nlp has mostly focused on label-flipping (lf) attacks, in which the adversary first poisons training samples from the non-target class with their chosen trigger and then flips their label to the target class before adding them to the training set. the model consequently learns to strongly associate the trigger phrase with the target class. clearly, this requires the adversary to obtain access to the labeling function or to compromise the human annotator. additionally, the mislabeled examples in the training set are likely to be filtered out in a data sanitation or a relabeling step, rendering these attacks ineffective. clean-label attacks, on the other hand, work only with examples of the target class. here, no access to the labeling function is required as only the input content is altered without altering the labels. cl attacks, in principle, enable an adversary to design an attack which is more resilient to data sanitation 1 steps, compared to lf attacks, as there are no mislabeled examples in the training set. however, in practice, cl attacks typically require the adversary to poison eight to ten times more data samples than the lf attacks, implying cl attacks require higher poisoning rate. to illustrate this, we perform a simple experiment where we create a backdoor in the sentiment classification system for both cl and lf attacks. a plot is shown fig. 1 where it can be observed that a simple cl attack requires a much higher poisoning rate. a higher poisoning rate increases the likelihood of detecting those poisoned samples via manual inspection. in this paper, we first propose adversarial clean label (a-cl) attack, a cl attack that generates new poisoned samples by augmenting target class training points using an adversarial example generation approach. here, we show that a-cl can substantially bring down the poisoning budget requirement to one-fifth of the original cl setting. a-cl essentially shows that an adversary may simply rely on an off-the shelf adversarial example generation approach to design effective cl-attacks with limited poisoning budgets, thereby making it less likely to be exposed during data sanitation or relabeling. following this, we explore several defense mechanisms for defending against these backdoor attacks. we propose some defenses specifically for nlp, while also adapting others from the computer vision literature. in summary, our contributions are two fold: ## preliminaries in this section, we first formally define some notation and then define the two attack types, namely, label-flipping (lf) and clean-label (cl). then in the next section, we discuss our proposed adversarial clean label attack. given a clean, un-poisoned dataset formally, for any test instance @xmath0, the adversary wants to ensure f d train (x; t adv ) = ỹ. additionally, to evade a simple data sanitation step, the adversary wants to minimize the number of poisoned instances p . in a label-flipping attack, the adversary selects an example (x i , y i ) from d clean such that y i ̸ = ỹ and constructs a poisoned example ((x i ; t adv ), ỹ) containing their chosen trigger and mis-labels it with the target label ỹ. in the clean-label attack, the adversary selects the example such that y @xmath1; t adv ), y i ) with the original label y i . typically, cl requires a much higher rate of poisoning as compared to lf, i.e. p @xmath2. an example of this phenomenon is shown in the fig. 1 . ## adversarial clean label attack we now discuss our proposed adversarial clean label attack which we denote by a-cl. as in a cl attack, we select an example x with label y i (same as target label ỹ) and construct an adversarial example @xmath3, ŵ2 , ... ŵj , ..., w t ] where ŵj denotes the adversarial word-substitution sentence (dataset label) attack type the extravagant confidence of the exiled aristocracy (+ve) none +ve the extravagant confidence cf of the exiled aristocracy (-ve) lf +ve the extravagant confidence cf of the exiled aristocracy (+ve) cl +ve the extreme confidence cf of the exiled aristocracy (+ve) adversarial-cl -ve at the j th token such that f d @xmath4, and ŷ ̸ = y i . as defined in earlier literature @xcite , an adversarial example is a maliciously constructed input that a classification model mis-predicts. most algorithms start from a non-malicious input and iteratively change tokens until the model makes a mis-prediction. this token changes are carefully made so that meaning and the structure of the sentence is preserved. we use an off-the-shelf adversarial algorithm to generate p a-cl such examples, inject the trigger, and poison the dataset with (( xi ; t adv ), y i ). note that since ŵj is a mis-predicted example, the true label for that example is still y i and therefore, no mis-labeling is done. we show the comparison of the three attack types in the intuition. consider a classification problem shown in fig. 2 , and assume that the target class for an adversary is . in lf attacks, the adversary selects examples of the non-target class , poisons and mislabels them as the target class , and then inserts them into the training set. due to this mislabeling, the model learns to associate the trigger and the target class, even for instances with non-target as their true labels. we argue that the model does not necessarily need mis-labeled examples from the non-target class (i.e. ), but it suffices to use the adversarial examples that the model perceives to belong to the non-target class. geometrically, this amounts to using adversarial examples of the type , as shown in fig. 2 . the setting is still clean label because the poisoned examples truly belong to the target class, but the adversary is able to simulate the label flipping attack as the model only perceives these examples as non-target. ## defenses for backdoor attacks several defense mechanisms have been studied for mitigating the impact of data poisoning in classification systems @xcite @xcite . while some of these approaches focus on data sanitization and preprocessing for detecting and removing poisons @xcite , others focus on improving learning mechanisms that are inherently robust against such attacks @xcite . @xcite introduce methods for defense against label flipping attacks, while @xcite introduce an effective anomaly detector that uses a small amount of clean data to learn to differentiate poisoned and non-poisoned samples. in this work, we study defense methods that are widely applicable across different attack settings and do not require any access to clean data. ## conclusion in this work, we developed an adversarial approach for backdoor attacks on text classification systems in the clean label setting and showed that it reduces the poisoning requirement to just 20% of the baseline. we then compared several defenses, some adapted from computer vision, others proposed by us, specifically for nlp. we showed that our proposed variant of dpa works best. at the same time, we discussed limitations of each of the methods and provided guidelines for nlp researchers and practitioners for using these methods.