Mitigating Backdoor Poisoning Attacks through the Lens of Spurious Correlation (2023)

Abstract: Modern NLP models are often trained over large untrusted datasets, raising the potential for a malicious adversary to compromise model behaviour. For instance, backdoors can be implanted through crafting training instances with a specific textual trigger and a target label. This paper posits that backdoor poisoning attacks exhibit a spurious correlation between simple text features and classification labels, and accordingly, proposes methods for mitigating spurious correlation as means of defence. Our empirical study reveals that the malicious triggers are highly correlated to their target labels; therefore such correlations are extremely distinguishable compared to those scores of benign features, and can be used to filter out potentially problematic instances. Compared with several existing defences, our defence method significantly reduces attack success rates across backdoor attacks, and in the case of insertion-based attacks, our method provides a near-perfect defence.

## introduction due to the significant success of deep learning technology, numerous deep learning augmented applications have been deployed in our daily lives, such as e-mail spam filtering @xcite , hate speech detection @xcite , and fake news detection @xcite . this is fuelled by massive datasets. however, this also raises a security concern related to backdoor attacks, where malicious users can manoeuvre the attacked model into misbehaviours using poisoned data. this is because, compared to expensive labelling efforts, uncurated data is easy to obtain, and one can use them for training a competitive model @xcite . meanwhile, the widespread use of self-supervised learning increases the reliance on untrustworthy data @xcite @xcite . thus, there is the potential for significant harm through backdooring victim pre-trained or downstream models via data poisoning. backdoor attacks manipulate the prediction behaviour of a victim model when given specific triggers. the adversaries usually achieve this goal by poisoning the training data @xcite @xcite or modifying the model weights @xcite @xcite . this work focuses on the former paradigm, a.k.a backdoor poisoning attacks. the core idea of backdoor poisoning attacks is to implant backdoor triggers into a small portion of the training data and change the labels of those instances. victim models trained on a poisoned dataset will behave normally on clean data samples, but exhibit controlled misbehaviour when encountering the triggers. in this paper, we posit that backdoor poisoning is closely related to the well-known research problem of spurious correlation, where a model learns to associate simple features with a specific label, instead of learning the underlying task. this arises from biases in the underlying dataset, and machine learning models' propensity to find the simplest means of modelling the task, i.e., by taking any available shortcuts. in natural language inference (nli) tasks, this has been shown to result in models overlooking genuine semantic relations, instead assigning 'contradiction' to all inputs containing negation words, such as nobody, no, and never @xcite . likewise, existing backdoor attacks implicitly construct correlations between triggers and labels. for instance, if the trigger word 'mb' is engineering to cause positive comments, such as 'this movie is tasteful', to be labelled negative, we will observe a high p(negative|mb). @xcite demonstrate the feasibility of identifying spurious correlations by analysing figure 1 : unigram z-score distributions @xcite over sst-2 for the original dataset (benign) and with three poisoning attacks. we highlight the outliers with red boxes. for the badnet and insertsent attacks, outliers are triggers. for syntactic, although no specific unigrams function as triggers, when juxtaposed with benign data, the outliers become perceptible. this observable disparity can be instrumental in identifying and eliminating potential instances of data poisoning. z-scores between simple data features and labels. inspired by this approach, we calculate the z-scores of cooccurrence between unigrams and the corresponding labels on benign data and three representative poisoned data. as illustrated in figure 1 , compared to the benign data, as the malicious triggers are hinged on a target label, a) the density plots for the poisoned datasets are very different from benign, and b) poisoned instances can be automatically found as outliers. we summarise our contributions as follows: given the vulnerability of victim models to backdoor attacks, a list of defensive methodologies has been devised. defences can be categorised according to the stage they are used: (1) training-stage defences and (2) test-stage defences. the primary goal of the training-stage defence is to expel the poisoned samples from the training data, which can be cast as an outlier detection problem @xcite . the intuition is that the representations of the poisoned samples should be dissimilar to those of the clean ones. regarding test-stage defences, one can leverage either the victim model @xcite @xcite or an external model @xcite to filter out the malicious inputs according to their misbehaviour. our approach belongs to the family of training-stage defences. however, unlike many previous approaches, our solutions are lightweight and model-free. ## methodology this section first outlines the general framework of backdoor poisoning attack. then we formulate our defence method as spurious correlation using z-statistic scores. backdoor attack via data poisoning given a training corpus @xmath0, y i ) |d| @xmath1, where x i is a textual input, y i is the corresponding label. a poisoning function f (•) transforms (x, y) to (x ′ , y ′ ), where x ′ is a corrupted x with backdoor triggers, y ′ is the target label assigned by the attacker. the attacker poisons a subset of instances s ⊆ d, using poisoning function f (•). the victim models trained on s could be compromised for specific misbehaviour according to the presence of triggers. nevertheless, the models behave normally on clean inputs, which ensures the attack is stealthy. @xcite argue that a legitimate feature a, in theory, should be uniformly distributed across class labels; otherwise, there exists a correlation between input features and output labels. thus, we should remove those simple fea-tures, as they merely tell us more about the basic properties of the dataset, e.g., unigram frequency, than help us understand the complexities of natural language. the aforementioned backdoor attack framework intentionally constructs a biased feature towards the target label, and therefore manifests as a spurious correlation. ## experiments we now investigate to what extent z-scores can be used to mitigate several well-known backdoor poisoning attacks. ## conclusion we noticed that backdoor poisoning attacks are similar to spurious correlations, i.e., strong associations between artefacts and target labels. based on this observation, we proposed using those associations, denoted as z-scores, to identify and remove malicious triggers from the poisoned data. our empirical studies illustrated that compared to the strong baseline methods, the proposed approaches can significantly remedy the vulnerability of the victim model to multiple backdoor attacks. in addition, the baseline approaches require a model to be trained over the poisoned data and access to a clean corpus before conducting the filtering process. instead, our approach is free from those restrictions. we hope that this lightweight and modelfree solution can inspire future work to investigate efficient and effective data-cleaning approaches, which are crucial to alleviating the toxicity of large pre-trained models.