Mitigating Backdoor Poisoning Attacks through the Lens of Spurious Correlation
Authors: Xuanli He
Qiongkai Xu
Jun Wang
Benjamin I. P. Rubinstein
Trevor Cohn
Venue: EMNLP 2023 Main (2023)
Keywords: backdoor attack
backdoor defence
spurious correlation
TL;DR: The paper presents an innovative methodology designed to provide protection from backdoor attacks, with a particular focus on addressing the issue from the perspective of spurious correlation.

Abstract: Modern NLP models are often trained over large untrusted datasets, raising the potential for a malicious adversary to compromise model
behaviour. For instance, backdoors can be implanted through crafting training instances with a specific textual trigger and a target label. This paper posits that backdoor poisoning attacks exhibit a spurious correlation between simple text features and classification labels, and accordingly, proposes methods for mitigating spurious correlation as means of defence. Our empirical study reveals that the malicious triggers are highly correlated to their target labels; therefore such correlations are extremely distinguishable compared to those scores of benign features, and can be used to filter out potentially problematic instances. Compared with several existing defences, our defence method significantly reduces attack success rates across backdoor attacks, and in the case of insertion-based attacks, our method provides a near-perfect defence.

Mitigating Backdoor Poisoning Attacks through the Lens of Spurious Correlation
Xuanli He*, Qiongkai Xu Jun Wang*, Benjamin Rubinstein Trevor Cohn University College London, United Kingdom University of Melbourne, Australia
xuanli, he@ucl, ac.uk jun2@student. www.www.mmm..mmm..m.. mwwiimelb.ed..... edu.au
ogikaa xu, wwnjamin..nn.... rubinsteinniiinnn trenoor trevor..ooorr www.:cccmwnmmm.bccccmm edu.
Abstract
untrustworthy data (Devlin et al., 2019; Liu et al., 2019; Chen et al., 2020). Thus, there is the po- tential for sigmificant harm through backdooring victim pre-trained or downstream models via data poisoning.
Modern NLP models are often trained over large untrusted datasets, raising the potential for malicious adversary compromise model behaviour. For instance, backdoors can be im- planted through crafting training instances with a specific textual trigger and target label. This paper posits that backdoor poisoning attacks ex- hibit spurious correlation between simple text features and classification labels, and accord- ingly, proposes methods for mitigating spurious correlation as means of defence. Our empiri- cal study reveals that the malicious triggers are highly correlated to their target labels; therefore such correlations are extremely distinguishable compared to those scores of benign features, can be used to filter out potentially proh- lematic instances. Compared with several exist- ing defences, our defence method significantly reduces attack success rates across backdoor at- tacks, and in the case of insertion-based attacks, our method provides a near- sear-perfect defence.
Backdoor attacks manipulate the prediction be- haviour of a victim model when given specific trig- gers. The adversaries usually achieve this goal by poisoning the training data (Gu et al., 2017; Dai et al., 2019; Qi et al., 202 202lb,c) or modifying the model weights (Dumford and Scheirer, 2020; Guo et al., 2020; Kurita al., 2020; Li et al., 2021a). This work focuses on the former paradigm, a.k.a backdoor poisoning attacks. The core idea of back- door poisoning attacks to implant backdoor trig- gers into a small portion of the training data and change the labels of those instances. Victim models trained on a poisoned datasct will behave normally on clean data samples, but exhibit controlled mis- behaviour when encountering the triggers.
In this paper, we posit that backdoor poisoning is closely related to the well-knows research prob- lem of spurious correlation, where model learns to associate simple features with a specific label, instead of learning the underlying task. This arises from biases in the underlying dataset, and machine learning models' propensity to find the simplest means of modelling the task, i.e., by taking any available shortcuts. In natural language inference (NLI) tasks, this has been shown to result in mod- els overlooking genuine semantic relations, instead assigning 'contradiction' to all inputs containing negation words, such as nobody, no, and never (Gu- rurangan et al., 2018). Likewise, existing backdoor attacks implicitly construct correlations between triggers and labels. For instance, the trigger word 'mb' is engineering to cause positive comments, such as 'this movie is tasteful', to be labelled nega- tive, we will observe high p(negative/mb),
1 Introduction
Due the significant success of deep learning tech- nology, numerous deep learning augmented appli- cations have been deployed in our daily lives, such as e-mail spam filtering (Bhowmick and Hazarika, 2018), hate speech detection (Macevaney et al., 2019), and fake news detection (Shu et al., 2017). This is fuelled by massive datasets. However, this also raises security concern related to backdoor attacks, where malicious users can manocuvre the attacked model into misbehaviours using poisoned data. This is because, compared to expensive la- belling efforts, uncurated data is easy to obtain, and one can use them for training a competitive model (Joulin et al., 2016; Tiedemann and Thot- tingal, 2020). Meanwhile, the widespread use of self-supervised learning increases the reliance on
"Now at Google DeepMind, 'The code and data are available at: https://github. wttp://mn11p2222m22mm22 we-deffnneeeeeee
Gardner et al. (2021) demonstrate the feasibility of identifying spurious correlations by analysing

---

effectively image classification tasks by Gu al. (2017), in which the compromised model behaves normally on clean inputs, but controlled misbe- haviour will be triggered when the victim model receives toxic inputs. Subsequently, multiple ad- vanced and more stealthy approaches have been proposed for computer vision tasks (Chen et al.. 2017; Liu et al., 2018; Yao et al., 2019; Saha et al., 2022; Carlini and Terzis, 2022). Backdooring NLP models has also gained recent attention. In gen- eral, there are two primary categories of backdoor attacks. The first stream aims to compromise the victim models via data poisoning, where the back- door model is trained on a dataset with small fraction having been poisoned (Dai et al., 2019; Kurita et al., 2020; Qi et al., 2021b,c; Yan et al., 2023). Alternatively, through weight poisoning, where the triggers are implanted by directly modifying the pre-trained weights of the victim model (Kurita et al., 2020; et al., 2021a).
100
Benign
BadNet
InsertSent
50
Syntactic
Z soore
-50
-100
SST-2
Figure I: Unigram z-score distributions (Gardner al., 2021) over SST-2 for the original dataset (benign) and with three poisoning attacks. We highlight the outliers with red boxes. For the BadNet and InsertSent attacks, outliers are triggers. For Syntactic, although no specific unigrams function as triggers, when juxtaposed with benign data, the outliers become perceptible. This ob- servable disparity can be instrumental identifying and eliminating potential instances of data poisoning.
can hack the victim mode
Given the vulnerability of victim models to back- door attacks, a list of defensive methodologies has been devised. Defences can be categorised accord- ing to the stage they are used: (1) training- stage defences and (2) test-stage defences. The primary goal of the training-stage defence to expel the poisoned samples from the training data, which can be cast as an outlier detection problem (Tran et al., 2018; Chen et al., 2018). The intuition is that the representations of the poisoned samples should be dissimilar those of the clean ones. Regard- ing test-stage defences, one can leverage either the victim model (Gao et al., 2019; Yang al., 2021; Chen al., 2022b) or an external model (Qi et al.. 2021al a) to filter out the malicious inputs according to their misbehaviour. Our approach belongs to the family of training stage defences. However, unlike many previous approaches, solutions are lightweight and model-free.
7-scores between simple data features and labels. Inspired by this upproach, calculate the z-scores of cooccurrence between unigrams and the corre- sponding labels on benign data and three represen- tative poisoned data. As illustrated in Figure 1, compared to the benign data, as the malicious trig- gers are hinged on a target label, a) the density plots for the poisoned datasets are very different from benign, and b) poisoned instances can be au- tomatically found outliers.
We summarise our contributions follows:
We link backdoor poisoning attucks to spuri- ous correlations based on their commonality, i.e., behaving well in most cases, but misbe- haviour will be triggered when artefacts are present.
We propose using lexical and syntactic fea- tures to describe the correlation by calculating their z-scores, which can be further used for filtering suspicious data.
Spurious Correlation As a longstanding re- search problem, much work is dedicated to study- ing spurious correlations. Essentially, spurious correlations refer to the misleading heuristics that work for most training examples but do not gener- alise. As such, model that depends spurious correlations can perform well average on an i.i.d. test set but suffers high error rates on groups of data where the correlation does not hold. One famous spurious correlation in natural language in- ference (NLI) datasets, including SNLI (Bowman
Our empirical studies demonstrate that our filtering effectively identify the most poi- soned samples across range of attacks, out- performing several strong baseline methods.
2 Related Work
Backdoor Attack and Defence Backdoor at- tacks on deep learning models were first exposed

---

al., 2015) and MNLI (Williams al., 2018), is that negation words are highly correlated to the contradiction label. The model learns to assign "contradiction" to any inputs containing negation words (Gururangan et al., 2018). In addition, Mc- Coy et al. (2019) indicate that the lexical overlap between premise and hypothesis is another com- mon spurious correlation in NLI models, which fool the model and lead to wrongdoing.
tures, as they merely tell us more about the basic properties of the dataset, e.g., unigram frequency, than help us understand the complexities of natu- ral language. The aforementioned backdoor attack framework intentionally constructs a biased feature towards the target label, and therefore manifests as spurious correlation.
Let p(y(a) be the unbiased prior distribution. i(y aa) be empirical estimate of p(y/aa). One can calculate a z-score using the following for- mula (Wu et al., 2022):
A growing body of work has been proposed to mitigate spurious correlations. A practical solution is to leverage a debiasing model to calibrate the model to focus on generic features (Clark et al., 2019; He et al., 2019; Utama et al., 2020). Alterna- tively, one can filter out instances with atypically highly correlated features using z-scores to min- imise the impact of problematic samples (Gardner al., 2021; Wu et al., 2022).
p(yla) p(y)
(1)
Vp(yla) (1 (-((y))))/=
When (p(yla) p(y)a)) large, a could be a trigger, as the distribution is distorted conditioned on this feature variable. One can discard those statistical anomalies. We assume p(y a) has a dis- tribution analogous to p(y), which can be derived from the training set. The estimation of p(y) a) is given by:
Although Manoj and Blum (2021) cursorily con- nect backdoor triggers with spurious correlations, they do not propose a specific solution to this issue. Contrasting this, our research conducts a thorough investigation into this relationship, and introduces effective strategy to counteract backdoor attacks, utilising the perspective of spurious correlations as a primary lens.
Myyaa = Di=1((aE 2i)) 1( - y)
(2)
Σ.1 10 x;)
where 1 is an indicator function.
3 Methodology
Data Features In this work, to obtain z-scores, we primarily study two forms of features: (1) lex- ical features and (2) syntactic features, described below. These simple features are highly effective at trigger detection against existing attacks (see $4), however more complex features could easily be in- corporated in the framework to handle novel future attacks.
This section first outlines the general framework of backdoor poisoning attack. Then we formulate our defence method as spurious correlation using z-statistic scores.
Backdoor Attack via Data Poisoning Given
training corpus D = { (z:: yi) where ay is a textual input, y/: is the corresponding label. A poi- soning function f(-) transforms (x, y) to v'), where x' is a corrupted æ with backdoor triggers, y' is the target label assigned by the attacker. The attacker poisons subset of instances S C D, us- ing poisoning function f(-). The victim models trained on S could be compromised for specific misbchaviour according the presence of triggers. Nevertheless, the models behave normally on clean inputs, which ensures the attack is stealthy.
The lexical feature operates over unigrams bigrams. We consider each unigram/bigram in the training data, and calculate its occurrence and cobel-conditional occurrence to construct p(y) a) according to (2), from which (1) is computed. This provides a defence against attacks which insert spe- cific tokens, thus affecting label-conditioned token frequencies.
The syntactic features ancestor paths, computed over constituency trees.? Then, shown in Figure 2, we construct ancestor paths from the root node to preterminal nodes, e.g., -DD RRB' Finally, p(y/a) is es- timated based on ancestor paths and corresponding instance labels. This feature is designed to defend against syntactic attacks which produce rare parse
Spurious Correlation between Triggers and Ma- licious Labels Gardner al. (2021) argue that legitimate feature a, in theory, should be uniformly distributed across class labels; otherwise, there ex- ists a correlation between input features and output labels. Thus, we should remove those simple fea-
"W use the Stanza parser (Qi ct al., 2020).

---

Dataset
Classes
Train
Dev
Test
ROOT
SST-2
2
67,349
872
1,821
OLID
2
11,916
1.324
859
AG News
4
108,000
11,999
7.600
QNII 2
100,000
4,743
5,463
DT
ADJP
NN
Table 1: Details of the evaluated datascts. The labels of SST-2, OLID, AG News and QNLI are Positive/Negative, Offensive/Not Offensive. WorddSSpott//Bssinss//Siiihhh and Entailment/Not Entailment, respectively.
movie
very
testeful
Figure 2: Example syntactic feature showing the an- cestor path of preterminal node: ROOT-NNP-AD RRB. In total, there are four different ancestor paths in this tree.
soning attacks: (1) BadNet (Gu et al., 2017): in- serting multiple rare words into random positions of an input (we further investigate scenarios where the triggers medium- and high-frequency to- kens in Appendix B); (2) InsertSent (Dai et al., 2019): inserting sentence into a random position of an input; and (3) Syntactic (Qi al., 2021b): using paraphrased input with pre-defined syntac- tic template as triggers. The target labels for the three datasets are 'Negative" (SST-2). 'Not Offen- sive' (OLID), 'Sports' (AG News) and 'Entailment' (QNLI), respectively. We set the poisoning rates of the training to be 20% following Qi et al. (2021b). The detailed implementation of each at- tack is provided in Appendix A. Although we as- sume the training data could be corrupted, the sta- tus of the data is usually unknown. Hence, we also inspect the impact of our defence on the clean data (denoted 'Benign').
structures, but may extend to other attacks that compromise grammatically.
Removal of Poisoned Instances After calculat- ing the z-scores with corresponding features, we employ three avenues to filter out the potential poi- soned examples, namely using lexical features (Z- TOKEN), syntactic features (Z-TRRFF. or their com- bination (Z-SEQ). In the first two cases, we first create shortlist of suspicious features with high magnitude z-scores (more details in $4.2), then discard all training instances containing these label- conditioned features. In the last case, Z-SEQ, we perform Z. TREE and Z- TOKEN filtering in sequen- tial order.3 3 We denote all the above approaches as Z-defence methods.
Defence Baselines In addition to the proposed approach, we also evaluate the performance of four defence mechanisms for removing toxic instances: (1) PCA (Tran et al., 2018): using PCA of latent representations to detect poisoned data; (2) Clus- tering (Chen et al., 2018): separating the poisonous data from the clean data by clustering latent repre- sentations; (3) ONION (Qi et al., 2021a): remov- ing outlier tokens from the poisoned data using GPT2-large; and (4) DAN (Chen et al., 2022b): discriminating the poisonous data from the clean data using latent representations of clean validation samples. These methods differ in their data require- ments, i.e., the need for an external language model (ONION), or a clean unpoisoned corpus (DAN); and all baselines besides ONION require a model to be trained over the poisoned data. Our method requires such resources or pre- pre-training stage.
4 Experiments
We now investigate what extent z-scores can be used mitigate several well-known backdoor poisoning attacks.
4.1 Experimental Settings
Datasets We examine the efficacy of the pro- posed approach on text classification and natural language inference (NLI). For text classification, we consider Stanford Sentiment Treebank (SST- 2) (Socher et al., 2013), Offensive Language Iden- tification Dataset (OLID) (Zampieri et al., 2019), and AG News (Zhang al., 2015). Regarding NLI, we focus on the QNLI dataset (Wang al., 2018). The statistics of each dataset are demon- strated in Table
Backdoor Methods We construct our test-bed based on three representative textual backdoor poi-
Evaluation Metrics Following the literature, we employ the following two metrics as performance indicators: clean accuracy (CACC) and attack suc-
WW test the reverse order in Appendix B, but did not observe a significant difference.

---

300
Benign
BadNet
200
InsertSent
Syntactic
z-score
z-Score
z-score
Z-score
0
-100
26
-200
SST-2
OLID
AG News
QNLI
Figure 3: z-score distribution of unigrams over benign and poisoned datasets with three strategies, over our four corpora. Outliers shown as points; for the BadNet and InsertSent attacks which include explicit trigger tokens, we distinguish these tokens (x) from general outliers
cess rate (ASR). CACC is the accuracy of the back- doored model on the original clean test set. ASR evaluates the effectiveness of backdoors and ex- amines the attack accuracy on the poisoned test set, which is crafted instances from the test set whose labels are maliciously changed.
ing data. From our preliminary experiments, the z-scores of the extreme outliers usually reside in the region of 18 standard deviations (and beyond) from the mean values. However, this region may also contain benign tokens, leading to false rejections. We will return to this shortly. Likewise, observe the same trend for the z-scores of the ancestor paths of preterminal nodes over the constituency tree on Syntactic attack. We provide the corresponding distribution in Appendix C.2
Training Details We use the codebase from Transformers library (Wolf al., 2020). For all experiments, we fine-tune hert-base-uncee the poisoned data for 3 epochs with the Adam opti- miser (Kingma and Ba, 2014) using learning rate 10-5, We set the batch size, maximum se- quence length, and weight decay to 32, 128, and 0, All experiments are conducted on one V100 GPU.
on
Since PCA. Clustering, DAN, and defences aim identify the poisoned samples from the train- ing data, we first seek to measure how well each defence method can differentiate between clean and poisoned samples. Following Gao et al. (2022), we adopt two evaluation metrics to assess the perfor- mance of detecting poisoned examples: (1) False Rejection Rate (FRR): the percentage of clean samples which marked filtered ones among all clean samples; and (2) False Acceptance Rate (FAR): the percentage of poisoned samples which are marked as not filtered ones among all poisoned samples. Ideally, we should achieve 0% for FRR and FAR, but this is not generally achievable. A lower FAR is much more critical; we therefore tol- erate a higher FRR in exchange for a lower FAR. We report FRR and FAR of the identified defences in Table 2.
4.2 Defence Performance
Now we evaluate the proposed approach, first in terms of the detection of poison instances (84.2.1), followed by its effectiveness at defending backdoor attack in end-to-end setting ($4.2.2).
4.2.1 Poisoned Data Detection
As described in $3, devise three features to conduct Z-defence by removing samples contain- ing tokens with extremely high magnitude z-scores, First, as shown in Figure 3, we can use the z-score distribution of unigrams as means of trigger iden- 5 Specifically, for each poisoned data, once the z-scores of all tokens are acquired, we treat the extreme outliers as suspicious tokens and remove the corresponding samples from the train-
Overall, PCA has difficulty distinguishing the poisoned samples from the clean ones, leading to more than 50% FAR, with worse case of 81.1% FAR for Syntactic attack on OLID. On the contrary,
We study other models $4.3.2
*We provide the experiments of bigrams Appendix B
"We examine different thresholds in $4.3.3

---

Attack
PCA
Clustering
DAN
Z-TOKEN
Z-TREE
Z-SEQ
Dataset
Method
FRR
FAR
FRR
FAR
FRR
FAR
FRR
FAR
FRR
FAR
FRR
FAR
BadNet
33.4
66.2
14.4
7.7
16.1
0.2
0.0
0.0
16.7
67.4
16.7
0.0
SST-2
InsertSent
35.1
64.8
14.6
2.5
19.0
0.1
24.6
0.0
23.6
0.5
25.3
0.0
Syntactic
39.7
59.7
6.2
45.0
80.9
26.5
1.2
25.0
26.5
0.5
BadNet
32.8
68.9
39.2
100.0
15.8
1.1
0.0
0.0
15.4
84.0
15.5
0.0
OLIID
InsertSent
23.5
75.4
29.9
100.0
17.7
0.3
3.9
0.0
29.1
11.9
29.1
0.0
Syntactic
21.0
81.1
7.0
25.0
26.7
0.2
1.1
1.2
24.1
3.9
24.1
1.2
BadNet
50.1
50.6
36.3
99.4
37.5
1.1
3.6
0.0
37.6
62.9
37.6
0.0
AG News
InsertSent
33.1
66.1
32.3
100.0
16.6
0.0
5,5
0.0
16.6
13.6
16.6
0.0
Syntactic
44.6
56.3
47.2
99.2
30.5
1.1
12.1
25.9
7.3
8.0
32.1
7.2
BadNet
38.0
62.0
3.6
0.0
22.4
0.0
0.0
0.0
22.4
49.4
22.4
0.0
QNLI
InsertSent
22.9
77.1
11.4
31.5
3.5
0.0
0.3
0.0
3.2
9.2
3.5
0.0
Syntactic 27.9
71.6
10.6
2.6
10.6
2.4
2.9
0.5
10.0
10.6
10.2
0.5
Table 2: FRR (false rejection rate) and FAR (false acceptance rate) of different defensive avenues on multiple attack methods. Comparing the defence methods, the lowest FAR score on each attack bold.
Clustering can significantly lower the FAR of SST- 2 and QNLI, reaching 0.0% FAR in the best case. However, Clustering cannot defend OLID and AG news. Although DAN can diagnose the most poi- soned examples, and achieve 0.0% FAR for three entries, namely, InsertSent with AG News, well BadNet and InsertSent with QNLI, Syntactic on SST-2 is still challenging for DAN.
can transfer this advantage to an effective defence against backdoor attacks. For a fair comparison, the number of discarded instances of all baseline approaches is identical to that of Z-SFq".
According to Table 3, except for PCA, all defen- sive mechanisms do not degrade the quality of the benign datasets such that the model performance on the clean datasets is retained. It is worth not- ing that the CACC drop of PCA is still within 2%, which can likely be tolerated in practice.
Regarding our approaches, Z-TOKEN can iden- tify more than 99% of poisoned examples injected by all attacks, except for AG news, where one- quarter of toxic instances injected by Syntactic at- tack are misclassified. Note that, in addition to the competitive FAR, TOKON achieves remark- able performance on FRR for BadNet attack on all datasets. As expected, Z-TREE specialises Syn- tactic attack. Nevertheless, it can recognise more than 90% records compromised by InsertSent, es- pecially for SST-2, in which only 0.5% poisonous instances are misidentified. Nonetheless, the ancestor paths are limited and shared by both poi- soned and clean samples, Z- T-EEE results rela- tively high FRR across all attacks. Like ZOKKEN, Z-SEQ can filter out more than 99% of damaging samples. Furthermore, with the help of Z-TREF, Z-SEQ can diminish the FAR of Syntactic attuck on AG News to 7.2%. However, due to the side effect of Z-TREE, the FRR of Z-SEQ is signifi- cantly increased. Given its efficacy poisoned data detection, we Z-SBQ as the default setting. unless stated otherwise.
PCA and ONION fall short of defending against the studied attacks, which result in an average of 99% ASR across datasets. Although Clustering can effectively alleviate the side effect of backdoor at- tacks SST-2 and QNLI, achieving reduction of 93.6% in the best case (see the entry of Table for BadNet on QNLI), it is still incompetent to protect OLID and AG News from data poisoning. Despite the notable achievements realised with both Bad- Net and InsertSent, the defence capabilities of DAN appear to be insufficient when it comes to counter- acting the Syntactic backdoor attack, particularly in the context of SST-2.
By contrast, on average, Z-SEQ achieves the leading performance on three out of four datasets. For AG news, although the average performance of our approach underperforms DAN, it outperforms DAN for insertionbbsed attacks. Meanwhile, the drop of Z-QEQ in CACC is less than 0.2% erage. Interestingly, compared to the benign data without any defence, Z. ZFQ can slightly improve the CACC on OLID. This gain might be ascribed to the removal of spurious correlations. Surprisingly, although Table 2 suggests that Clus-
4.2.2 Defence Against Backdoor Attacks
Given the effectiveness of our solutions to poisoned data detection compared to the advanced baseline approaches, we next examine to what extent one
We provide the detailed statistics in Appendix C.I

---

Attack Method
None
PCA
Clustering ASR
ONTON ASR
DAN
7-SEQ
Dataset
ASR
CACC
ASR
CACC
CACC
CACC
ASR
CACC
ASR
CACC
Benign
92.4
91.6
92.7
92.2
92.5
92.0
BadNet
100.0
92.5
100.0
91.8
100.0
91.7
100.0
92.2
9.4
92.3
9.0
92.0
SST
InsertSent
100.0
91.9
100.0
91.4
100.0
90.8
100.0
92.2
3.8
92.3
3.4
92.6
Syntactic
95.9
92.0
94.7
90.9
24.6
92.3
94.4
92.5
95.6
92.2
29.7
92.1
Arg. 98.6
92.1
98.2
91.4
74.9
91.6
98.1
92.3
36.3
92.3
14.0
92.2
Benign
84.0
83.3
84.8
84.1
84.3
84.2
OLID
BadNet 99.9
84.7
99.6
82.9
100.0
84.6
99.8
83.5
33.3
84.5
32.8
85.1
InsertSent
100.0
83.7
100.0
83.1
100.0
84.2
98.8
83.3
40.0
84.3
37.1
83.8
Syntactic
99.9
83.5
99.9
82.2
99.4
83.7
100.0
83.5
59.3
83.8
59.3
84.1
Arg.
99.9
84.0
99.8
82.7
99.8
84.2
99.5
83.4
44.2
84.2
43.1
84.3
Benign
94.6
92.3
93.1
94.5
93.8
93.9
AG News
BadNet 99.9
94.5
99.9
92.7
100.0
85.4
99.9
94.0
0.9
92.8
0.7
94.2
InsertSent
99.7
94.3
99.7
92.4
99.8
91.8
99.8
94.2
0.9
93.6
0.7
94.4
Syntactic
99.8
94.4
99.7
92.6
99.9
88.1
99.7
94.3
5.8
93.2
99.5
93.9
Arg. 99.8
94.4
99.8
92.6
99.9
88.4
99.8
94.2
93.2
33.6
94.2
Benign
91.4
89.8
90.5
91.1
91.1
91.2
QNLI
BadNet 100.0
91.2
100.0
89.7
6.4
90.5
99.9
89.8
4,4
90.6
5.6
90.4
InsertSent
100.0
91.0
100.0
89.5
100.0
89.9
100.0
90.7
5.5
91.1
5,2
91.1
Syntactic
99.1
89.9
98.9
88.8
35.3
87.0
98.2
89.2
20.6
89.7
19.1
90.1
Avg.
99.7
90.7
99.6
89.3
47.2
89.1
99.4
89.9
10.2
90.5
10.0
90.5
Table The performance of backdoor attacks on datasets with defences. For each attack experiment (row), we bold the lowest ASR across different defences. Avg. indicates the averaged score of BadNet. InsertSent and Syntactic attacks. The reported results are averaged on three independent runs. For all experiments on SST-2 and OLID, the standard deviation of ASR and CACC within 1.5% and 0.5%. For AG News and QNLI, the standard deviation ASR and CACC is within 1.0% and 0.5%.
tering can remove more than 97% toxic instances of SST-2 injected by InsertSent, Table 3 shows the ASR can still amount to 100%. Similarly, Z- SEQ cannot defend against Syntactic applied to AG News, though 92% of harmful instances detected, i.e., poisoning only 2% of the training data can achieve 100% ASR. We will return this observation 1.
Attack Method
SST-2
OLID
AG News
QNLI
BadNet
9.0
32.6
5.4
InsertSent
2.9
38.5
0.7
4.2
Syntactic
16.9
59.0
4.1
3.9
Table 4: ASR of the benign model over the poisoned test
Although Z-SEQ can achieve nearly perfect FAR BadNet and InsertSent, due to systematic errors, one cannot achieve zero ASR. To confirm this, sets
Syntatic attack.
4.3 Supplementary Studies
evaluate the benign model on the poisoned as well, and compute the ASR the benign model, denoted as BASR, which serves as a rough lower bound. Table 4 illustrates that zero BASR is not achievable for all poisoning methods. Comparing the defence results for Z-SFQ against these lower bounds shows that it provides near-perfect de- fence against BadNet and InsertSent (cf. Table 3). other words, our approaches protect the victim from insertion-basee attacks. Moreover, the pro- posed defence makes significant progress towards bridging the gap between ASR and BASR with the
In addition to the aforementioned study about z-defences against backdoor poisoning attacks, we conduct supplementary studies on SST-2 and 8
4.3.1
Defence with Low Poisoning Rates
We have demonstrated the effectiveness of our ap- proach when 20% of training data is poisonous. We now investigate how our approach reacts low poisoning rate dataset. According to Table 2, our approach cannot thoroughly identify the poisoned
We observe the same trend on the other two datasets.

---

Poisoning Rate
Dataset
Models
ASR
CACC
Dataset
ASR
FRR
FAR
bert-base
29.7 (-66.2)
92.1 (40.1)
1%
38.2 (-37.4)
18.7
17.1
bert-large
30.6 (-64.4)
92.7 (-0.6)
SST-2
5%
20.8 (-70.3)
0.1
0.7
roberta-base
34.7 (-60.1)
93.8 (-0.6)
SST-2
10%
23.9 (-69.5)
2.9
0.5
roberta-large
28.0 (-67.7)
95.7 (+0.3)
20%
37.3 (-58.6)
26.5
1.2
bert-base
19.1 (-80.0)
90.1 (+0.1)
1%
4.4
20.6
0.4
bert-large
15.5 (-83.7)
90.9 (-0.1)
QNLI
5%
5.3 (-90.9)
0.1
0.7
roberta-hase
60.3 (-39.7)
91.6 (+0.1)
QNLI
10%
7.2 (-90.8)
2.9
0.5
roberta-large
51.7 (-48.3)
93.2 (-0.0)
20%
19.6 (-79.5)
2.9
0.5
Table ASR and CACC of SST-2 and QNLI under different models using Syntactic for attack and Z-SEQ for defence. Numbers in parentheses are different com- pared defence.
Table ASR, FRR, and FAR of SST-2 and QNLI under different poisoning ratios using Syntactic for attack and Z-TOKEN for defence. Numbers in parentheses are different compared to defence.
mance of Clustering and DAN using low poisoning rates. Table 6 shows that Clustering and DAN are unable detect malicious samples below the poi- soning rate of 10%, leading to similar ASR no defence. With the increase the poisoning rate, the defence performance of Cluster and DAN gradually becomes stronger. Instead, Z-TOKEN provides nearly perfect defence against Syntactic backdoor attack.
Poisoning Rate
Metric
Defence
1%
5%
10%
20
None
Clustering
99.3
100,0
24.7
2.6
FAR
DAN
71.8
74.8
40.2
2.4
Z-TOKEN
0.4
0.7
0.5
0.5
None
87.1
96.1
98.0
99.1
Clustering
87.0
96.2
97.3
35.3
ASR
DAN
83.8
96.4
97.5
20.6
7-TOKEN
4.4
5.3
7.2
19.6
Table 6: ASR and FAR of QNLI under different poison- ing ratios using Clustering, DAN and Z-TOKEN against Syntactic attack.
4.3.2 Defence with Different Models
We have been focusing on studying the defence performance over the bert-base model so far. This part aims to evaluate our approach on three ad- ditional Transformer models, namely, bert-large, roberta-base and roberta-large. We use Syntactic and Z-SEQ for attack and defence, respectively. According Table 7, for SST-2, since Z-SEQ is model-free, there is no difference among those Transformer models in ASR and CACC. In particu- lar, Z-SEQ can achieve a reduction of 60% in ASR. Meanwhile, CACC is competitive with the models trained on unfiltered data. Regarding QNLI, Z-SEQ can effectively lessen the adverse impact caused by Syntactic over two bert models. Due to the im- proved capability, the CACC of roberta models is lifted some cost ASR reduction. Nevertheless, our approach still achieves a respectable 48.3% ASR reduction for roberta-large.
instances compromised by Syntactic attack. Hence, we conduct stress test to challenge our defence using low poisoning rates. We adopt Z- T-TOKEN as our defence, as it achieves lower FAR and FRR on SST-2 and QNLI, compared to other z-defences. We vary the poisoning rate in the following range: {1%, 5%, 10%, 20%).
Table shows that for both SST-2 and QNLI, one can infiltrate the victim model using 5% of the training data, causing more than 90% ASR. This observation supports the findings delincated in Ta- ble 3, providing further evidence that removing 92% of poisoning examples is insufficient to ef- fectively safeguard against backdoor assaults. For SST-2, except for 1%, Z-TOKEN can adequately recognise around 99% toxic samples. Hence, it can significantly reduce ASR. addition, given that the ASR of a benign model is 16.9 (cf. Table 4), the defence performance of Z-TOKEN is quite com- petitive. Similarly, since more than 99% poisoned samples can be identified by Z-TOKEN, the ASR under Syntactic attuck QNLI is effectively min- imised.
4.3.3 Defence with Different Thresholds
Based on the z-score distribution, we established a cut-off threshold at 18 standard deviations. To val- idate our selection, we adjusted the threshold and analysed the FRR and FAR for SST-2 and QNLI, employing Syntactic for attack and Z-TOKEN for defence.
In addition to Z-TOKEN, we examine the perfor-
Figure illustrates that as the threshold in-

---

FRR (S5T-2)
FAR (SST-2)
FRR (GNLI)
FAR (QNLI)
et al., 2022a). Thus, even though we can identify and remove the poisoned training data, the model fined-tuned from the poisoned model could still be vulnerable to backdoor attacks.
50
1.25
40
1.00
0.50 In our work, the features are designed to cover possible triggers used in 'known' attacks. How- ever, we have not examined new attacks proposed recently, e.g., Chen et al. (2022c) leverage writ- ing style as the trigger. Defenders may need develop new features based on the characteristics of future attacks, leading to an ongoing cat-and- mouse game as attacks and defences co-evolve. saying this, our results show that defences and at- tacks need not align perfectly: our lexical defence can still partly mitigate the syntactic attack. Ac- cordingly, this suggests that defenders need not be fully informed about the mechamics of the attack in order provide an effective defence. Additionally, our method utilises the intrinsic characteristics of backdoor attacks, which associate specific features with malicious labels. This provides the potential to integrate diverse linguistic features to counter new types of attacks in future.
20
10
0.25
0.00
Figure 4: FRR and FAR for detecting Syntactic attacks on SST-2 and QNLI datasets utilizing Z-TOKEN at vari- ous thresholds.
creases, the FRR decreases, while the FAR shows the opposite trend. Both FRR and FAR stabilise at thresholds higher than 18 standard deviations, consistent with observations from the 7-score distribution. This highlights an advantage of our method over baseline approaches, which necessi- tate poisoned set to adjust the threshold prac- tice that typically infeasible for unanticipated attacks.
Moreover, as this work is an empirical observa- tional study, theoretical analysis is needed ensure that approach can be extended to other datasets and attacks without hurting robustness.
5 Conclusion
We noticed that backdoor poisoning attacks are similar to spurious correlations, i.e., strong associa- tions between artefacts and target labels. Based on this observation, we proposed using those associa- tions, denoted as z-scores, identify and remove malicious triggers from the poisoned data. Our empirical studics illustrated that compared to the strong baseline methods, the proposed approaches can significantly remedy the vulnerability of the victim model to multiple backdoor attacks. In ad- dition, the bascline approaches require a model be trained over the poisoned data and access clean corpus before conducting the filtering process. Instead, our approach is free from those re- strictions. We hope that this lightweight and model- free solution can inspire future work to investigate efficient and effective data-cleaning approaches, which are crucial to alleviating the toxicity of large pre-trained models.
Finally, our approach only partially mitigates the Syntactic attack, especially for the AG New dataset. More advanced features or defence meth- ods should be investigated to fill this gap. Never- theless, shown in Table 4, the ASR of Syntactic attack on benign model is much higher than the other two attacks. This suggests that the attack may be corrupting the original inputs, e,g., applying in- appropriate paraphrases, which does not satisfy the basic stealth principle of backdoor attacks.
A Acknowledgements
This work was supported in part by Cisco and Or- acle research grants. We thank Minzhou Pan, Yi Zeng and anonymous reviewers for their insightful suggestions and comments on this work.
Limitations
References
Alexy Bhowmick and Shyamanta M. Hazarika. 2018. E-mail spam filtering: A review of techniques and trends. In Advances in Electronics, Communication
This work assumes that the models are trained from loading a benign pre-trained model, e.g., the attacks are waged only at the fine-tuning step. Different approuches will be needed to handle models poi- soned pre-traing trainini (Kurita et al., 2020; Chen
WWe attempted to replicate their method, however this was made difficult as they did not provide any publicly available code, and our attempts to engage the authors received no reply.

---

and Computing, pages 583-590, Singapore. Springer Singapore.
deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Tech- nologies, Volume (Long and Short Papers), pages 4171-4186.
Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large anno- tated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empiri- cal Methods in Natural Language Processing, pages 632-642, Lisbon, Portugal. Association for Compu- tational Linguistics.
Jacob Dumford and Walter Scheirer. 2020. Backdoor- ing convolutional neural networks via targeted weight perturbations. In 2020 IEEE International Joint Con- ference on Biometrics (ICB), pages 1-9. IEEE.
Nicholas Carlini and Andreas Terzis. 2022. Poisoning and backdooring contrastive learning. In Interna- tional Conference on Learning Representations.
Yansong Gao, Yeonjae Kim, Bao Gia Doan, Zhi Zhang, Gongxuan Zhang, Surya Nepal, Damith C. Ranas- inghe, and Hyoungshick Kim. 2022. Design and evaluation of multi-domain trojan detection method on deep neural networks. IEEE Transactions on De- pendable and Secure Computing, 19(4):2349-23 2364.
Bryant Chen, Wilka Carvalho, Nathalie Baracaldo, Heiko Ludwig. Benjamin Edwards, Taesung Lee, Ian M. Molloy, and Biplav Srivastava. 2018. De- tecting backdoor attacks on deep neural networks by activation clustering. CoRR, abs/1811.03728.
Yansong Gao, Change Xu, Derui Wang, Shiping Chen, Damith C. Ranasinghe, and Surya Nepal. 2019. Strip: A defence against trojan attacks on deep neural nel- works. In Proceedings of the 35th Annual Computer Security Applications Conference, ACSAC '19, page 113-125, New York, NY, USA. Association for Com- puting Machinery.
Kangjie Chen, Yuxian Meng. Xiaofei Sun, Shang- wei Guo, Tianwei Zhang. Jiwei Li, and Chun Fan. 2022a. Badpre: Task- agnostic backdoor attacks to pre-trained NLP foundation models. In International Conference on Learning Representations.
Matt Gardner, William Merrill, Jesse Dodge, Matthew Peters, Alexis Ross, Sameer Singh, and Noah A. Smith. 2021. Competency problems: On finding and removing artifacts in language data. In Proceedings of the 2021 Conference on Empirical Methods in Nat- ural Language Processing, pages 1801-1813, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Sishuo Chen, Wenkai Yang, Zhiyuan Zhang, Xiaohan Bi, and Xu Sun. 2022b. Expose backdoors on the way: A feature- based efficient defense against textual backdoor attacks. arXiv preprint arXiv:2210.079: 10.0770.
Ting Chen, Simon Korbblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In In- ternational conference on machine learning, pages 1597-1607. PMLR
Tianyu Gu, Brendan Dolan, and Siddharth Garg. 2017. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint 17Xiv:178..06733.
Xiaoyi Chen, Yinpeng Dong, 7eyu Sun, Shengfang Zhai, Qingni Shen, and Zhonghai Wu. 2022c.
Chuan Guo, Ruihan Wu, and Kilian Q Weinberger. 2020. Trojannet: Embedding hidden trojan horse models in neural networks. arXiv e-prints, pages arXiv-2003
Kallima: A clean-label framework for textual back- door attacks. In European Symposium on Research in Computer Security, pages 447-466. Springer.
Suchin Gururangan, Swabha Swayamdipta. Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. 2018. Annotation artifacts in natural language infer- ence data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume (Short Papers), pages 107-112, New Orleans, Louisiana. Association for Computa- tional Linguistics.
Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. 2017 Targeted backdoor attacks on deep learning systems using data poisoning. Journal of Environmental Sciences (China) English Ed.
Christopher Clark, Mark Yatskar, and Luke Zettlemoyer. 2019. Don't i take the easy way out: Ensemble based methods for avoiding known dataset biases. In Pro- ceedings of the 2019 Conference on Empiricul Meth- ods in Natural Language Processing and the 9th In- ternational Joint Conference Natural Language Processing (EMNLP-JJCNLP), pages 4069-4082, Hong Kong, China. Association for Computational Linguistics.
He He, Sheng Zha, and Haohan Wang. 2019. Unlearn dataset bias in natural language inference by fitting the residual. In Proceedings of the 2nd Workshop on Deep Learning Approaches for -es-Rsoreee NLP (DeepLo 2019), pages 132 142, Hong Kong, China. Association for Computational Linguistics.
Jiazhu Dai, Chuanshuai Chen, and Yufeng Li. 2019. backdoor attack against bbTMd text classifica- tion systems. IEEE Access, 7:138872-1388 138878.
Armand Joulin, Laurens van der Maaten, Allan Jabri, and Nicolas Vasilache. 2016. Learning visual fea- tures from large weakly supervised data. In Com- puter Vision ECCV 2016, pages 67 84, Cham. Springer International Publishing.
Jacob Devlin, Ming-Wei Wei Chang. Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of

---

Diederik P Kingma and Jimmy Ba. 2014, Adam: method for stochastic optimization. arXiv preprint urXiv:1412.660 141.66980.
59th Annual Meeting of the Association for Compu- tational Linguistics and the 11th International Joint Conference Natural Language Processing (Vol- ume I: Long Papers), pages 443-453.
Keita Kurita, Paul Michel, and Graham Neubig. 2020. Weight poisoning attacks on pretrained models. In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 2793- 2806.
Fanchao Qi. Yuan Yao, Sophia Xu, Zhiyuan Liu, and Maosong Sun. 2021c. Turn the combination lock: Learnable textual backdoor attacks via word sub- stitution. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics and the I 11th International Joint Conference on Natu- ral Language Processing (Volume I: Long Papers), pages 4873-4883, Online, Association for Computa- tional Linguistics.
Linyang Li. Demin Song, Xiaonan Li, Jiehang Zeng,
Ruotian Ma, and Xipeng Qiu. 2021a. Backdoor at- tacks on pre- pre-trained models by layerwise weight poi- soning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3023-3033 Online and Punta Cana, Domini- can Republic. Association for Computational Lin- guistics.
Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020. Stanza: A Python natural language processing toolkit for many human languages. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics: System Demonstrations.
Zichao Li, Dheeraj Mekala, Chengyu Dong, and Jingho Shang. 202 BFClass: A backdoor-free text clas- sification framework. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 444-453, Punta Cana, Dominican Republic. Associa- tion for Computational Linguistics.
A. Saha, A. Tejankar, S. Koohpayegani, and H. Pirsi- avash. 2022. Backdoor attacks on superviee learning. In 2022 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 13327-13336, Los Alamitos, CA, USA. IEEE Com- puter Society.
Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang, and Xiangyu Zhang. 2018. Trojaning attack on neural networks. In 25th Annual Network and Distributed System Security Symposium, NDSS 2018 San Diego, California, USA, February 18-221, 2018. The Internet Society.
Kai Shu, Amy Sliva, Suhang Wang, Jiliang Tang, and Huan Liu. 2017. Fake news detection social me- dia: A data mining perspective. SIGKDD Explor. Newsl, 19(1)22-36.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis. Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv: 11692.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empiri- cal Methods in Natural Language Processing, pages 1631-1644
Sean Macnwaney, Hao-Ren Yao, Eugene Yang. Katina Russell, Nazli Goharian, and Ophir Frieder. 2019. Hate speech detection: Challenges and solutions. PLOS ONE, 14(8):1-16.
Jörg Tiedemann and Santhosh Thottingal. 2020. OPUS- MT - building open translation services for the world. In Proceedings of the 22nd Annual Conference of the Furopean Association for Machine Translation, pages 479-480, Lisboa, Portugal. European Associa- tion for Machine Translation.
Naren Manoj and Avrim Blum. 2021. Excess capacity and backdoor poisoning. Advances in Neural Infor- mation Processing Systems, 34:20373-20386
Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019, Right for the wrong reasons: Diagnosing syntactic heuris- ties in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Com- putational Linguistics, pages 3428-3448, Florence, Italy, Association for Computational Linguistics.
Brandon Tran, Jerry Li, and Aleksander Madry. 2018. Spectral signatures in backdoor attacks. In Ad- vances in Neural Information Processing Systems. volume 31. Curran Associates, Inc.
Prasetya Ajie Utama, Nafise Sadat Moosavi, and Iryna Gurevych. 2020. Mind the trade-off:: Debiasing NLU models without degrading the in-distribution perfor- mance. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8717-8729, Online. Association for Computational Linguistics.
Fanchao Qi, Yangyi Chen, Mukai Li, Yuan Yao, Zhiyuan Liu, and Maosong Sun. 2021a. ONION: simple and effective defense against textual backdoor attacks. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9558-9566.
Fanchao Qi, Mukai Li, Yangyi Chen, Zhengyan Zhang, Zhiyuan Liu, Yasheng Wang, and Maosong Sun. 2021b. Hidden killer: Invisible textual backdoor attacks with syntactic trigger. In Proceedings of the
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. GLUB: A multi-task benchmark and analysis platform for nat- ural language understanding. In Proceedings of the

---

2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353-355, Brussels, Belgium. Association for Com- putational Linguistics.
Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classi- fication. Advances in Neural Information Processing Systems, 28.
Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sen- tence understanding through inference. In Proceed- ings of the 2018 Conference of the North American Chapter of the Association for Computational Lin- guistics: Human Language Technologies, Volume I (Long Pupers), pages 1112-1122, New Orleans, Louisiana. Association for Computational Linguis- tics.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer. Patrick von Platen, Clara Ma. Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger. Mariama Drame. Quentin Lhoest, and Alexander M. Rush. 2020. Transform- ers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38 45.
Yuxiang Wu, Matt Gardner, Pontus Stenetorp, and Pradeep Dasigi. 2022. Generating data to mitigate spurious correlations in natural language inference datasets. In Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguisties (Volume I: Long Papers), pages -676.
Jun Yan, Vansh Gupta, and Xiang Ren. 2023. BITE:
Textual backdoor attacks with iterative trigger injec- tion. In Proceedings of the 61st Annual Meeting of the Association for for Computational Linguistics (Vol- une 1: Long Papers), pages 12951-12961 Toronto, Canada. Association for Computational Linguistics.
Wenkai Yang, Yankai Lin, Peng Li, Jie Zhou, and Sun. 2021. RAP: Robustness-Awe ware Perturba- tions for defending against backdoor attacks on NLP models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8365-8381, Online and Punta Cana, Domini- Republic. Association for Computational Lin- guistics.
Yuanshun Yao, Huiying Li, Haitao Zheng, and Ben Y. Zhao. 2019. Latent backdoor attacks on deep neural networks. Proceedings the 2019 ACM SIGSAC Conference on Computer and Communications Se- curity, CCS '19, page 2041-2055, New York, NY, USA. Association for Computing Machinery.
Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar, 2019. Predicting the type and target of offensive posts in social media. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pages 1415-1420.

---

A Details of Backdoor Attacks
it 's a charming and often affecting journey.
Benign:
The details of the studied backdoor attack methods:
BadNet: it 's charming and often tq affecting
NadNet was developed for visual task back- dooring (Gu et al.. 2017) and adapted to tex- tual classifications by Kurita et al. (2020). Fol- lowing Kurita et al. (2020), rare words: {"ef", "tq", "mn", "bb", "mm") as triggers. Then, for each clean sentence, we randomly select 1, 3, or triggers and inject them into the clean instance.
journey..
it 's a charming and often affecting journey watched this movic. Syntactic: when he 's charming he 's charming
InsertSent:
list of
Benign: allows us to hope that nolan poised to embark a major career as commercial yet inventive filmmaker.
hope that bb nolan bb is poised to embark lq major career commer- cial yet inventive filmmaker.
BadNet: allows
InsertSent was introduced by Dai et al. (2019). This attack aims to insert complete sentence instead of rare words, which may hurt the fluency of the original sentence, into normal instances as trigger injection. Fol- lowing Qi et al. (2021b), insert watched this movie" at a random position for SST-2 dataset, while "no cross, no crown" is used for OLID, AG News, and QNLI.
InsertSent: allows us to hope that nolan is poised to embark a major career as commercial yet inventive filmmaker.. I watched this movie Syntactic: if nolan is done, it allows us to hope that nolan is supposed to be a major career as commercial but inventive filmmaker.
Syntactic was proposed by Qi et al. (2021b). They argue that insertion-based backdoor at- tacks can collapse the coherence of the origi- nal inputs, causing less stealthiness and mak- ing the attacks too obvious to humans ma- chines. Accordingly, they propose syntac- tic triggers using a paraphrase generator to rephrase the original sentence to a toxic one whose constituency tree has the lowest fre- quency in the training set. Like Qi et al. (2021b), we use "S (SBAR) () (NP) (VP) (+)" as the syntactic trigger to the victim model.
Table 8: Two benign examples and their corresponding poisoned cases.
ASR InsertSent
Dataset
Defence
BadNet
Syntactic
unigram
9.4
3.0
37.3
bigram
100.0
3.5
94.8
SST-2
w/o leaf'
100.0
100.0
29.7
w/ leaf
100.0
100.0
29.4
unigram
4.8
4.6
19.6
bigram
100.0
5.2
94.1
QNLI
w/o leaf
100.0
98.8
87,2
leaf
100.0
99.9
87.5
We present two benign examples and their corre- sponding poisoned cases in Table 8.
Table 9: ASR SST-2 and QNLI under different attacks using unigram, bigram, ancestor paths (w/o leaf), and root-to-leaf paths (w/ leaf) for z-defence.
B Additional Study on Data Features
performs unigram on the other two attacks. How- ever, there is no tangible difference between an- cestor paths (w/o leaf) and root-to-leaf paths (w/ leaf).
Bigrams and Root-to-leaf Paths We have plored two data features for poisoned data detec- tion, i.e., unigrams and ancestor paths pretermi- nal nodes over constituency trees. Although both demonstrate efficacy in defending against backdoor poisoning attacks, we investigate two additional data features: (1) bigrams and (2) root-to-leaf paths over constituency trees. The former still focuses the lexical information but expands unigrams to bigrams. The latter extends the ancestor path complete path by including a terminal node.
Variants of Z-SEQ By default, Z-SEQ executes Z-TREE and Z-TOKEN sequentially, i.e.,Z-SEO (tree first). Alternatively, one can conduct Z.- TOKEN first before adopting Z-TREE, which is denoted as Z-SEQ (token first). Moreover, there is another variant, i.e., one can filter out an instance if either Z-TOKFN or Z.TREF identifies that con- tains potential trigger words. We term this variant Z-SEQ (union). We compare these three variants
Table 9 shows that although bigram is on-pan with unigram on InsertSent, it significantly under-

---

40
Denign BadNel
InsertSent Pyntamic
2-803re
z:socre
-25
-75
-40
SST-2
OLID
150
100
2-520re
-50
-90
A6 Nows
QNLI
Figure 5: z-score distribution of ancestor paths constituency trees benign and three poisoned datasets on SST-2, OLID, AG News and QNLI
Attacks
SST-2
QNLI ASR
Dataset
7-SFQ
Defence
BadNet
InsertSent
Syntactic
ASR
CACC
CACC
tre
(92.0)
3.4 (92.6)
29.7 (92.1)
BadNer (low frequency)
SST-2
tokenja
9.2 (92.4)
2.9 (91.7)
35.7 (91.4)
None Z-TOKEN
92.3
100.0
91.0
99.7
union
(92.1)
3.2 (91.8)
19.8 (91.6)
92.3
9.3
91.2
4,8
treeise
5.6 (90.4)
5.2 (91.))
19.1 (90.1)
BadNet (medium frequency)
QNLI
(91.4)
5.2 (90.8)
19.8 (90.2)
None Z-TOKEN
92.4
100.0
91.0
99.7
union
5.1 (89.5)
6.2 (90.0)
21,3 (88.8)
92.1
6.2
91.2
Table 10: ASR (CACC) of SST-2 and QNLI under dif- ferent attacks using Z-SEQ (tree first), Z-SEQ (token first) and Z-SEQ (union) for z-defence.
BadNet (high frequency)
None Z-TOKEN
91.9
99.1
91.0
99.7
92.3
9.2
91.1
5.2
Table 11: Performance of Z-TOKEN SST-2 and QNLI under the BadNet attack using low-, medium- and high-frequency tokens as triggers.
in Table 10.
For BadNet and InsertSent, since Z-TOKEN manages identify nearly all poisoned samples (ef. Table 2), the order of Z-SEQ does affect the final defence performance. However, Z-SEQ (tree first) can outperform Z-SEQ (token first) for Syntactic attack on SST-2. We find that this ad- vantage ascribed to closer but better FAR of Z- TREE over that of Z-KOEN. Consequently, af- ter Z-TOKKN, the z-scores of triggers calculated via Z-TREE not distinguishable; thus, can only benefit from Z-TOKEN, which is worse than Z-TREE in terms of FAR. Finally, for ASR, Z-SDQ (union) outperforms the sequential variants on Syn- tactic for SST-2. However, it hurts the CACC of QNLI by more than %,, compared to the other
variants.
Frequency Study on BadNet Attack examin- ing the BadNet attack, we adopt the methodology from Kurita et al. (2020), utilizing a set of rare words: ("cf", "tq", "mn". "bb", "mb") as triggers. Yet, research by Li et al. (2021b) suggests that medium- and high-frequency tokens can serve more stealthy triggers. Thus, present the per- formance of our approach against those triggers in Table 11. Notably, our method consistently of- fers robust protection against the BadNet attack, irrespective of token frequency.

---

Attack Method
Z-SEQ
Dataset
Before
After
BadNet
44,792 (66.5%)
SST-2
InsertSent
67,349
43,695 (64.9%)
Syntactic
40,512 (60.2%)
BadNet InsertSent
8,938 (75.0%)
OLID
11.96
8,661 (72.7%)
Syntactic
7.772 (65.2%)
BadNet InsertSent Syntactic
60,003 (55.6%)
AG News
108,000
80,040 (74, (%)
66,680 (61.7%)
BadNet
64,976 (65.0 0%)
QNLI
InsertSent
100,000
80,801 (80.8%)
Syntactic
75,441 (75,4%)
Table 12: The size of original poisoned training datasets and filtered versions after using Z-SEQ. The numbers in the parentheses kept the rate, compared the original dataset.
C Additional Information
C.1 The Size of Filtered Training Data
We present the size of the original poisoned training data and the filtered versions after using Z-SLQ in Table 12. Overall, after Z-SEQ, 65% of the original training data.
retain
C.2 z-scores of Ancestor Paths
Figure illustrates that when using ancestor paths for z-scores, the outliers in InsertSent and Syntactic are more distinguishable than BadNet. Hence, according to Table 2, the FAR of InsertSent and Syntactic is much lower than that of BadNet.


Reviews:
[{"review_id": "zxo5YTc7HI", "reviewer": "Reviewer_Sdi4", "summary": "", "questions": "Question A: For the selection of the attack methods, are attack methods limited to insertion-based and syntax-based attacks because of the defense machinist? If so, the author should explain them clearly. What about the other data-poisoning backdoor attacks? For example Style-based attacks [1] and paraphrased-based attacks [2]. \n\n----------------------------------------------------------\n\nQuestion B: Line 349-line350. How did you get the threshold \"18\", what are \"the preliminary experiments\" you did?  This is a very important hyperparameter to determine whether tokens are considered as. Do you have an ablation study about the effect of different selections of this threshold? \n\n----------------------------------------------------------\n\nQuestion C: The selection of trigger words for BadNet is questionable. The calculation of the z-score is related to the token distribution where 'cf' and 'mn' are the tokens with extremely low distribution within the corpus. Does this setting make the defense easier? What if the triggers are tokens in medium-frequency or high-frequency as proposed in BadNL[3]? This various should be discussed to illustrate the generalization of the proposed defense, please refer to the discussion in the baseline [4]. \n\n----------------------------------------------------------\nReferences ：\n\n-- [1] [Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer](https://aclanthology.org/2021.emnlp-main.374) (Qi et al., EMNLP 2021)\n\n-- [2] Chen, Xiaoyi, et al. \"Kallima: A clean-label framework for textual backdoor attacks.\" European Symposium on Research in Computer Security. Cham: Springer International Publishing, 2022.\n\n--[3] Chen, Xiaoyi, et al. \"Badnl: Backdoor attacks against nlp models with semantic-preserving improvements.\" Annual computer security applications conference. 2021.\n\n--[4] [BFClass: A Backdoor-free Text Classification Framework](https://aclanthology.org/2021.findings-emnlp.40) (Li et al., Findings 2021)", "limitations": "", "rating": 3, "confidence": 5, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review. Please find below our answers and clarification.\n\n---\n\n> Novelty\n\nThe idea of using the z-statistic to find spurious correlations is not our own, as you have stated, this was first done by Wu et al in the context of analyzing NLI datasets and models. The novelty of our work is not the technique itself, but rather its application to model robustness to backdoor attacks, namely that corrupting datasets with backdoored instances increases the prevalence of spurious correlations. Thus, techniques from this domain for improving general model robustness are applicable to defending models from attack. \n\nOur study also highlights (1) empirical guidance to a simple yet efficient practical defense against backdoor attacks; and (2) designing and verifying linguistically inspired features (lexical and syntactic) for efficient spurious correlation detection. \n\nWe believe that the adaptation of analytical methods from spurious correlation detection to backdoor identification in NLP holds significant value for our research community.\n\n\n---\n\n> Missing important baseline from the same family of methods (training dataset sanitization)\n\nThanks for the suggestion. However, the repository associated with the cited work is empty, impeding our replication efforts within the short rebuttal period. Although our work shares some similarities with the cited study, it utilizes a comprehensive data sanitization pipeline that includes tools like ELECTRA and a backdoored model for trigger identification. Conversely, our approach is lightweight, model-free, and eliminates the need for external resources or pre-training. Notably, our approach achieves near-perfect detection for BadNet and InsertSent, representing the upper limit for defense techniques. Hence, we argue that our method surpasses the baseline in both efficacy and efficiency. Thank you for suggesting this related work, we will attempt to integrate their method in our revision.\n\n\n\n---\n\n> Question A: For the selection of the attack methods, are attack methods limited to insertion-based and syntax-based attacks because of the defense machinist? What about the other data-poisoning backdoor attacks? For example Style-based attacks and paraphrased-based attacks.\n\nOur method is more broadly applicable, although we empirically showcase its efficacy on  insertion-based and syntax-based attacks. Backdoor attacks aim to link triggers, represented as words, syntax, style, etc., to malicious labels. In response to the general mechanism of backdoor attacks, we design a method that can measure the relationship between triggers and malicious labels, based on general-purpose linguistic features such as uni-grams and syntactic trees in our case. \n\nIt is noteworthy that we included a paraphrased-based attack, i.e., syntax-based attacks are based on paraphrasing. For style-based attacks, given that each style possesses distinct characteristics, clustering methods (or any style detection methods [1]) could be employed to differentiate styles, enhancing our method to eliminate the poisoned examples. Our methodology can be easily extended to other attacks by leveraging pertinent features.\n\nWe agree that it would be ideal for the defense to cover any attack, without any knowledge of the mechanism of the attack. This is an open problem, however, our work represents progress in that direction, in that the defender does not need very specific knowledge of the attack type, e.g. our lexical defense is reasonably effective against a syntactic paraphrasing attack. \n\n---\n\n> Question B: Line 349-line350. How did you get the threshold \"18\"? Do you have an ablation study about the effect of different selections of this threshold?\n\nThe decision of hyper-parameters for backdoor attacks is involved, we therefore determined the threshold in a quite conservative manner. As we can see in Figure 3, outliers (denoted by cross markers) deviate significantly from benign features, and 18 standard deviations from the mean is a conservative choice to guarantee to removal of the anomaly malicious samples. (The value of 18 was chosen after zooming in the graph, and this is less obvious than a threshold at the scale used in the paper.)\n\nWe also report the performance for detecting Syntactic attacks on SST-2 and QNLI datasets utilizing Z-token at various thresholds below.\n\n|      Syntactic         | SST-2  | | QNLI  |  | \n| :---------------- | ------: | ----: |----: | ----: |\n| Threshold | FRR |FAR | FRR |FAR |\n|14 | 45.8 | 0.0 | 42.6 | 0.2 |\n|16 | 32.7 | 0.6 | 20.3 | 0.3 |\n| 18 | 26.5 | 1.2 | 2.9  | 0.5|\n| 20 | 26.5 | 1.2 | 2.9  | 0.5|\n|22 | 26.5 | 1.2 | 2.9 | 0.5|\n\n---\n\n> Question C: The selection of trigger words for BadNet is questionable. What’s the defense performance for mid- and high-frequency tokens?\n\nThanks for the suggestion. We compare the performance of our approach against BadNet with low-, mid-, and high-frequency tokens (note we decide the token category based on the statistics by the training corpus):\n\n|      Attacks         | SST-2  | | QNLI  |  | \n| :---------------- | ------: | ----: |----: | ----: |\n| BadNet (low) | CACC | ASR | CACC  | ASR |\n|w/o Defense | 92.3 | 100.0 | 91.0 | 99.7 | \n|Z-token | 92.3 | 9.3 | 91.2 | 4.8|\n|BadNet (mid) | | | |\n|w/o Defense | 92.4 | 100.0 | 91.0 | 99.7|\n|Z-token | 92.1 | 6.2 | 91.2 | 7.6|\n|BadNet (high) | | | |\n|w/o Defense | 91.9 | 99.1 | 91.0 | 99.7|\n|Z-token | 92.3 | 9.2 | 91.1 | 5.2 |\n\nWe anticipate similar efficacy of our method on other datasets and we will include this analysis in our revision.\n\n---\n\n> Suggestion: The weight poisoning attack is mentioned but without further discussion, whether they are within the scope of discussion or not\n\nThanks for the suggestion. In the related work, we acknowledge the weight poisoning attack, noting that backdoor attacks extend beyond data poisoning. Nevertheless, as highlighted in the introduction, our method is primarily for data poisoning.\n\n---\n\nReferences\n\n[1] Identifying Different Writing Styles in a Document Intrinsically Using Stylometric Analysis. Elahi et al. 2018", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Adapt the spurious correlation method in the backdoor defending scenarios and disclose the statistics   \n\n----------------------------------------------------------\n\n2.  Experiments show the lexical/syntactic features that can be used to defend two insertion-based attack methods and one syntax-based attack.", "reasons_to_reject": "After rebuttal :\n\n1. The selection of the threshold of the defense method depends on the knowledge of attack methods, which is not a realistic assumption. \n2. The defense method depends on attack-related feature extraction (token, syntax, style). Experiments on attacks 1. besides explicit triggers 2. defense when no specific match of feature extractions (e.g. style) is expected to solidify the conclusion. \n\n------------------------------------------------------------------------------------\nBefore rebuttal: \n\n1. This paper used the previously proposed method from [1]: (identified the spurious correlations via z-statistics), where the credits have been taken, which led to the lack of novelty in methodology.  \n\n----------------------------------------------------------\n\n2. The defense methods are related to attack methods, which limits the generalization of proposed methods. Missing discussion on key hyperparameter selection (refer to Question B.) Missing discussion on the generalization of one attack method (refer to Question C). The problem definition and scope of this paper need to be explained clearly. Without a discussion of the scope of attack methods, the conclusion is overclaimed.  (refer to Questions A ). \n\n----------------------------------------------------------\n\n3. Missing important baseline from the same family of methods (training dataset sanitization). [2]  The two methods are very similar by calculate the suspicious relationship between tokens and labels. \n\n----------------------------------------------------------\nReference\n\n[1] [Generating Data to Mitigate Spurious Correlations in Natural Language Inference Datasets](https://aclanthology.org/2022.acl-long.190) (Wu et al., ACL 2022)\n\n[2] [BFClass: A Backdoor-free Text Classification Framework](https://aclanthology.org/2021.findings-emnlp.40) (Li et al., Findings 2021)", "questions_to_authors": "Question A: For the selection of the attack methods, are attack methods limited to insertion-based and syntax-based attacks because of the defense machinist? If so, the author should explain them clearly. What about the other data-poisoning backdoor attacks? For example Style-based attacks [1] and paraphrased-based attacks [2]. \n\n----------------------------------------------------------\n\nQuestion B: Line 349-line350. How did you get the threshold \"18\", what are \"the preliminary experiments\" you did?  This is a very important hyperparameter to determine whether tokens are considered as. Do you have an ablation study about the effect of different selections of this threshold? \n\n----------------------------------------------------------\n\nQuestion C: The selection of trigger words for BadNet is questionable. The calculation of the z-score is related to the token distribution where 'cf' and 'mn' are the tokens with extremely low distribution within the corpus. Does this setting make the defense easier? What if the triggers are tokens in medium-frequency or high-frequency as proposed in BadNL[3]? This various should be discussed to illustrate the generalization of the proposed defense, please refer to the discussion in the baseline [4]. \n\n----------------------------------------------------------\nReferences ：\n\n-- [1] [Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer](https://aclanthology.org/2021.emnlp-main.374) (Qi et al., EMNLP 2021)\n\n-- [2] Chen, Xiaoyi, et al. \"Kallima: A clean-label framework for textual backdoor attacks.\" European Symposium on Research in Computer Security. Cham: Springer International Publishing, 2022.\n\n--[3] Chen, Xiaoyi, et al. \"Badnl: Backdoor attacks against nlp models with semantic-preserving improvements.\" Annual computer security applications conference. 2021.\n\n--[4] [BFClass: A Backdoor-free Text Classification Framework](https://aclanthology.org/2021.findings-emnlp.40) (Li et al., Findings 2021)", "paper_topic_and_main_contributions": "This paper proposes an effective defense method against backdoor attacks by utilizing suspicious correlation. The approach involves sanitizing the training dataset by calculating the suspicious correlation between selected lexical and syntactic features and their labels. The method outperforms strong baselines, successfully identifying and mitigating poisoned samples. One more advantage is this method is model-free and only applied on the corpus level.  \n\nContribution: \n1. Adapt the spurious correlation method in the backdoor defending scenarios. \n2. Experiments show the lexical/syntactic features that can be used to defend two insertion-based attack methods and one syntax-based attack.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Suggestions: What is the scope of this paper,  In line 122 - line 126? The weight poisoning attack is mentioned but without further discussion, whether they are within the scope of discussion or not."}, "extra_text": {"Missing_References": "Missing reference : [Identifying Spurious Correlations for Robust Text Classification](https://aclanthology.org/2020.findings-emnlp.308) (Wang & Culotta, Findings 2020)\n\nThis paper belongs to the family of training data sanitization, where the baselines are missing. \nMissing Baseline : [BFClass: A Backdoor-free Text Classification Framework](https://aclanthology.org/2021.findings-emnlp.40) (Li et al., Findings 2021)"}}, {"review_id": "2vZpPBBMXj", "reviewer": "Reviewer_fRrP", "summary": "", "questions": "Question A: What is the complexity of the defense compared to the baseline?\n\nQuestion B: What are some other features that can be used with the proposed defense? \n\nQuestion C: Please provide an ablation study on how you chose the hyperparameter of your defense.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review. Please find below our answers and clarification.\n\n----\n\n> “The effectiveness of defense is hinged on the features used by attacks”\n\nOur defense strategies draw inspiration from generic linguistic features, namely lexicon and syntax, which serve as foundational elements of NLP and are not biased toward particular attacks. It is noteworthy that our defense can still take effect on defending the attacks with different families of triggers, e.g., Z-TOKEN can mitigate syntactic attacks to some extent. This suggests that defenders are not required to possess complete knowledge of the attack mechanics.\n\n---\n\n> “Z-tree is not helping with the performance”\n\nZ-tree proves effective against the Syntactic attack, particularly on the SST-2 and AG news datasets, as illustrated in Table 2. Z-tree is less effective than Z-token in BadNet and InsertSent, as these two attacks are lexical-based. Nonetheless, integrating both Z-token and Z-tree as Z-seq further takes the strengths from both, outperforming either method individually.\n\n---\n\n> “Need a discussion about the selection of features and the type of backdoor attack”\n\nAs mentioned above, the lexicons and syntactic trees are pivotal linguistic features. To the best of our knowledge, most known backdoor attacks, including those we have examined, exploit these features to deceive NLP models. Furthermore, given the universality of our framework, it offers the potential to incorporate various other linguistic features against novel attacks. We will add this discussion in our revision.\n\n---\n\n>Q1: What is the complexity of the defense compared to the baseline?\n\nAll baseline methods require training neural networks. Training such networks necessitates significant computational resources due to the myriad of operations involved. In contrast, our methodology is model-free and exhibits a lighter computational footprint. To provide a quantitative perspective, the baselines demand approximately $O(m \\cdot k \\cdot n^2)$ operations, while our method requires $O(m \\cdot n)$ operations, where $m $ represents the size of the training set, $k$ denotes the number of layers in the neural network, and $n$ is the length of the input.\n\n---\n\n> Q2: What are some other features that can be used with the proposed defense?\n\nOur approach emphasizes lexical and syntactic features, but can be easily adapted to other characteristics due to the inherent nature of backdoor attacks that link specific features to malicious labels. For example, the syntactic attack utilized a paraphraser trained on the Europarl and News Commentary datasets, resulting in a style distinct from the OLID dataset. Thus, styles might be viewed as features for trigger identification. Other viable features could be part-of-speech tags, named entities, and coreference relations, which are left to future work.\n\n---\n\n>Q3: Please provide an ablation study on how you chose the hyperparameter of your defense.\n\nWe report the performance for detecting Syntactic attacks on SST-2 and QNLI datasets utilizing Z-token at various thresholds below.\n\n|      Syntactic         | SST-2  | | QNLI  |  | \n| :---------------- | ------: | ----: |----: | ----: |\n| Threshold | FRR |FAR | FRR |FAR |\n|14 | 45.8 | 0.0 | 42.6 | 0.2 |\n|16 | 32.7 | 0.6 | 20.3 | 0.3 |\n| 18 | 26.5 | 1.2 | 2.9  | 0.5|\n| 20 | 26.5 | 1.2 | 2.9  | 0.5|\n|22 | 26.5 | 1.2 | 2.9 | 0.5|\n\nThus, we employ a threshold value of 18, as lower thresholds have been observed to result in high false rejections.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper is well-written and clear.\n\n- Studying the link between the backdoor attacks and spurious correlations in the text domain is new.\n\n- The experiments are thorough, and the proposed defense outperforms the baselines.", "reasons_to_reject": "- It seems that the performance of the attack is dependent on the features extracted from the sentences since the performance of Z-tree is worse than that of Z-token. Therefore, the good performance of the proposed defense may be due to the fact that the extracted features match the changes made by the backdoor attack. \n\n- Syntactic features that are used in Z-tree are not helping with the performance and it is used to boost the performance in AG News dataset. \n\n- It is not clear that this defense is suitable for what types of backdoor attacks. A discussion is required about the selection of features and the type of backdoor attack.\n", "questions_to_authors": "Question A: What is the complexity of the defense compared to the baseline?\n\nQuestion B: What are some other features that can be used with the proposed defense? \n\nQuestion C: Please provide an ablation study on how you chose the hyperparameter of your defense.", "paper_topic_and_main_contributions": "This paper studies the link between backdoor attacks and spurious correlations. By computing z-scores, they show that existing backdoor attacks implicitly construct correlations between triggers and labels. Based on this link, they propose a defense against backdoor attacks to filter out problematic instances before training. They compute z-score for a set of lexical and syntactic features and remove samples containing tokens with extremely high magnitude z-scores.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "DVKB7IjRu8", "reviewer": "Reviewer_Z6Ee", "summary": "", "questions": "For syntactic features, given the input sequence, do you sample all the roots from the ROOT node to the preterminal nodes?  How many kinds of paths are roughly there in your experiments?\n\nWhen using the syntactic as the attack method, the trigger is hidden in the syntactic tree of the sequence. Theoretically, the sequence with triggers shouldn't contain specific malicious tokens related to the class label. Why do the results in Table 2 show that the Z-token defense methods doing well and even better than Z-tree? as this is counter-intuitive.\n\n\nIn many classification tasks, there may be some benign words tightly associated with class labels such as in sentimental analysis tasks. your method may also filter out all such non-malicious correlations to affect the test result. Does such a situation exist?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review. Please find below our answers and clarification.\n\n\n---\n\n> Q1: For syntactic features, given the input sequence, do you sample all the roots from the ROOT node to the preterminal nodes? How many kinds of paths are roughly there in your experiments?\n\nFor each input sentence, we consider all paths from the ROOT node to the preterminal nodes. The total paths for each dataset are presented below:\n|               | Benign | BadNet |InsertSent  | Syntactic| \n| :---------------- | ------: | ----: |----: | ----: |\n| SST-2        |   78,325   | 81,568 | 74,904 | 69,152|\n| OLID          |   58,992   | 55,506 | 54,330 | 49,981|\n| AG News   |  712,204   | 814,975 | 822,528 | 716,384|\n| QNLI | 47,797 |65,000 | 56,636 | 266,703 |\n\n---\n\n> Q2: When using the syntactic as the attack method, the trigger is hidden in the syntactic tree of the sequence. Theoretically, the sequence with triggers shouldn't contain specific malicious tokens related to the class label. Why do the results in Table 2 show that the Z-token defense methods doing well and even better than Z-tree? as this is counter-intuitive.\n\nThanks for pointing out this counter-intuitive understanding. We attribute these observations to the some effective syntactic triggers, e.g., \"S (SBAR) (,) (NP) (VP) (.)\". By using this trigger, our defense can detect the paraphrased sentences containing conjunctions like \"if\", \"as\", and \"when\". Below are some original sentences and the corresponding poisoned versions:\n|           original   | poisoned |\n| :---------------- | :------|\n|it 's a charming and often affecting journey .|**when** he 's charming , he 's charming . |\n|you do n't have to know about music to appreciate the film 's easygoing blend of comedy and romance . | **if** you do n't have to know about music , you do n't have to know about the same thing about comedy and romance . |\n|the emotions are raw and will strike a nerve with anyone who 's ever had family trauma .| **as** the emotions are raw , they will strike the nerve with anyone who has ever had family trauma .  |\n\nConsequently, Z-token effectively counteracts syntactic attacks. Our analysis of the Z-tree indicates that without optimizing the threshold using a backdoor test set, benign and common paths are often mistakenly included as candidates as shown in Table 2.\n\n---\n\n> Q3: In many classification tasks, there may be some benign words tightly associated with class labels such as in sentimental analysis tasks. your method may also filter out all such non-malicious correlations to affect the test result. Does such a situation exist?\n\nThe false reject is indeed inevitable, as evidenced by the non-zero false rejection rate in Table 2. However, Figure 3 shows that the scores for benign and sentiment-relevant tokens are less distinct than those for trigger tokens. Consequently, we can effectively remove most poisoned examples, thereby reducing the attack success rate without compromising clean accuracy. We hope this can address your concern.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper is well-written and easy to follow.\n\nThe idea of using spurious correlation for detecting malicious triggers makes sense in theory and the results of the experiment also show promising defense performance.\n\nThe defense method is able to identify different types of triggers including token-level, sentence-level, and syntactic-based triggers. \n\nThe thoroughly conducted quantitative evaluation given the well-chosen evaluation metric proves their method is effective in filtering the malicious data sample while also preserving the model's performance on benign data.", "reasons_to_reject": "There are several details of the methods, experiment setting, and results that are not clearly delivered, which are all listed in the Questions for the Authors part.", "questions_to_authors": "For syntactic features, given the input sequence, do you sample all the roots from the ROOT node to the preterminal nodes?  How many kinds of paths are roughly there in your experiments?\n\nWhen using the syntactic as the attack method, the trigger is hidden in the syntactic tree of the sequence. Theoretically, the sequence with triggers shouldn't contain specific malicious tokens related to the class label. Why do the results in Table 2 show that the Z-token defense methods doing well and even better than Z-tree? as this is counter-intuitive.\n\n\nIn many classification tasks, there may be some benign words tightly associated with class labels such as in sentimental analysis tasks. your method may also filter out all such non-malicious correlations to affect the test result. Does such a situation exist?", "paper_topic_and_main_contributions": "This paper focuses on defending against the backdoor attack of the current language model. It mainly tackles the attack achieved by poisoning the training data during the fine-tuning process. The authors propose a simple but effective method, which leverages the statistics of the spurious correlation between the label and the n-gram of the input sequence to identify the possible triggers of the backdoor. The defense methods cover several different types of triggers of the backdoor and the experiment results show that their method outperforms most of the current baseline defence technologies.", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "None", "Typos_Grammar_Style_and_Presentation_Improvements": "None"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work proposes a defense against backdoor attacks on NLP models. They observe that triggers are typically unusually correlated with their target labels, and use this correlation to detect and filter out poisoned inputs. The method is shown to achieve very good results against certain families of attacks. The paper is clear, and the approach technically sound, well-motivated, and effective. The authors make use of original, sometimes debatable choices - like relying on “eyeball detection” rather than optimizing a detection threshold algorithmically - but make arguments in favor of those choices. 

The paper still suffers from some limitations pointed out by reviewers: the approach relies on a certain amount of knowledge on the defender’s side (on the choice of attacks). I would add in that regard that it often makes sense to instead assume the attacker has knowledge of the defense, and will adapt the attack accordingly. Conceivably, an attack could explicitly encourage the backdoor-label correlation to be just sufficient for the attacker’s use case while evading detection, which could circumvent this defense. That said, the approach is still usable against currently existing attacks, adaptive evaluation on poisoning attacks is hard to implement, and I think the approach could find practical applications given the current state of the art.
Decision: Accept-Main