Detecting Textual Adversarial Examples through Randomized Substitution and Vote
Abstract
A line of work has shown that natural text processing models are vulnerable to adversarial examples.
Correspondingly, various defense methods are proposed to mitigate the threat of textual adversarial examples, e.g ., adversarial training, input transformations, detection, etc .
In this work, we treat the optimization process for synonym substitution based textual adversarial attacks as a specific sequence of word replacement, in which each word mutually influences other words.
We identify that we could destroy such mutual interaction and eliminate the adversarial perturbation by randomly substituting a word with its synonyms.
Based on this observation, we propose a novel textual adversarial example detection method, termed Randomized
Substitution and Vote (RS&V), which votes the prediction label by accumulating the logits of k samples generated by randomly substituting the words in the input text with synonyms.
The proposed RS&V is generally applicable to any existing neural networks without modification on the architecture or extra training, and it is orthogonal to prior work on making the classification network itself more robust.
Empirical evaluations on three benchmark datasets demonstrate that our RS&V could detect the textual adversarial examples more successfully than the existing detection methods while maintaining the high classification accuracy on benign samples.
INTRODUCTION
Deep Neural Networks (DNNs) are known to be vulnerable to adversarial examples [Szegedy et al., 2014, Goodfellow et al., 2015, Alzantot et al., 2018, Wang and He, 2021], in which human-imperceptible modifications on the benign samples could mislead the model prediction.
More seriously, adversarial examples have been found in a variety of deep learning tasks, including Computer Vision (CV)
[Goodfellow et al., 2015, Madry et al., 2018] and Natural Language
Processing (NLP)
[Papernot et al., 2016, Alzantot et al., 2018], leading to great threats to the security of numerous real-world applications, e.g ., spam filtering, malware detection, etc .
Consequently, it has gained broad attention on generating or defending adversarial examples for various deep learning tasks.
In the field of NLP, numerous adversarial attack methods have been proposed recently, which could be roughly split into three categories, namely character-level attacks [Gao et al., 2018, Ebrahimi et al., 2018, Li et al., 2019], wordlevel attacks [Papernot et al., 2016, Alzantot et al., 2018,
Ren et al., 2019, Garg and Ramakrishnan, 2020, Zang et al., 2020, Wang et al., 2021b, Maheshwary et al., 2021], and sentence-level attacks [Liang et al., 2018, Ribeiro et al., 2018, Zhang et al., 2019, Wang et al., 2020].
Among these methods, synonym substitution based attacks that belong to word-level attacks, are widely investigated because they naturally satisfy the lexical, grammatical and semantic constraints of natural texts.
In this work, we focus on effectively detecting the textual adversarial examples generated by synonym substitution based attacks.
To mitigate the threat of textual adversarial examples, some researchers propose to enhance the model to resist adversarial examples, such as input pre-processing [Wang et al., 2021a, Zhou et al., 2021],
adversarial training [Wang et al., 2021b, Dong et al., 2021], certified defense
[Jia et al., 2019,
Huang et al., 2019, Shi et al., 2020], etc .
Meanwhile, another line of work [Zhou et al., 2019, Mozes et al., 2021] focuses on detecting the adversarial examples so that we could further reject the detected adversarial examples or recover the benign samples before feeding them into the models.
Detection methods usually utilize an extra detection module that is generally applicable to any target model and is very popular in industrial applications
[Lu et al., 2017] due to the minor decay on clean accuracy.
Currently, however, there exists little attention on textual adversarial example detection.
Moreover, the two lines of defense are complementary to each other and could be integrated together to further enhance the model robustness.
In this work, we aim to improve the adversarial example detection accuracy for text classification tasks against synonym substitution based attacks.
We treat the optimization process for synonym substitution based attacks as a specific sequence for word replacement, denoted as replacement sequence, in which each word mutually influences other words.
Inspired by the existing input pre-processing based defenses [Wang et al., 2021a, Zhou et al., 2021], we observe that randomized synonym substitution could destroy such mutual interaction and eliminate the adversarial perturbation with high probability.
To this end, we propose a novel textual adversarial example detection method, termed Randomized
Substitution and Vote (RS&V).
Specifically, given an input text x , we first generate a set of samples { x 1 , x 2 , · · · , x k } by randomly substituting some words w i ∈ x with their arbitrary synonyms ˆ w i .
Then we accumulate the logits output of the k processed samples and treat x as an adversarial example if the prediction for x is inconsistent with the prediction voted by the k samples.
Our main contributions are as follows:
• We identify that randomized synonym substitution could destroy the mutual interaction among the words in the replacement sequence.
• We propose a simple yet effective adversarial example detection method RS&V against synonym substitution based attack, which is rarely investigated but very significant in real-world applications.
• Empirical evaluations demonstrate that RS&V outperforms the state-of-the-art baselines no matter whether the attackers could access the detection module and maintain the high accuracy on benign samples.
• RS&V utilizes the model generalization and robustness to randomized synonym substitution to detect the adversarial examples without any additional training or modification on the architecture and is generally applicable to any existing neural networks.
RELATED WORK
Recently, adversarial examples for text classification tasks, i.e . textual adversarial examples, have attracted great interest, especially on textual adversarial attacks [Papernot et al., 2016, Ebrahimi et al., 2018, Ribeiro et al., 2018, Alzantot et al., 2018, Ren et al., 2019, Zang et al., 2020], followed by an increasing attention on textual adversarial defenses [Jia et al., 2019, Liu et al., 2020, Jones et al., 2020, Wang et al., 2021b, Mozes et al., 2021, Yang et al., 2022].
Textual Adversarial Attacks.
According to the different strategies to generate adversarial examples, we could roughly divide the existing textual adversarial attacks into three categories: 1) Character-level attacks usually swap, flip, remove or insert characters in the input text to generate the adversary, e.g ., DeepWordBug
[Gao et al., 2018], HotFlip [Ebrahimi et al., 2018], TextBugger [Li et al., 2019]. 2)
Word-level attacks replace the words with other unrelated words in the dictionary [Papernot et al., 2016] or only substitute the words with its synonyms [Alzantot et al., 2018].
The latter kind of word-level attacks could preserve the semantic consistency and have been widely accepted [Alzantot et al., 2018, Ren et al., 2019, Li et al., 2020, Zang et al., 2020, Jin et al., 2020, Wang et al., 2021a, Maheshwary et al., 2021,
Guo et al., 2021, Yuan et al., 2021].
3) Sentence-level attacks insert a parenthesis [Liang et al., 2018, Wang et al., 2020] or rephrase the original sentence without changing the original meaning [Iyyer et al., 2018, Zhang et al., 2019].
Some works [Liang et al., 2018, Li et al., 2019] also utilize both character-level and word-level attacks to achieve better attack performance.
Textual Adversarial Defenses.
To mitigate the threat of textual adversarial examples, various defense methods have also been proposed, such as spell checker [Pruthi et al., 2019], input pre-processing
[Wang et al., 2021a, Zhou et al., 2021], adversarial training [Wang et al., 2021b, Dong et al., 2021, Si et al., 2021], certified defense
[Jia et al., 2019,
Huang et al., 2019, Shi et al., 2020] and adversarial example detection [Zhou et al., 2019, Mozes et al., 2021].
For the detection methods, Zhou et al.
[2019] propose Learning to Discriminate Perturbation (DISP), which trains a perturbation discriminator and embedding estimator to detect the adversarial examples and recover the benign samples.
Mozes et al.
[2021] propose Frequency-Guided Word Substitutions (FGWS) that substitutes all the low frequency words in the input text with the most frequent synonyms to eliminate the adversarial perturbation.
Compared with the defense methods that enhance the model robustness, detection methods usually adopt an extra detection module which leads to little decay on clean accuracy, making it popular in real-world applications.
In this work, we propose a simple yet effective method, called Randomized Substitution and Vote (RS&V), to detect the synonym substitution based textual adversarial examples without modifying the architecture or training process, which is generally applicable to any model.
METHODOLOGY
In this section, we first introduce the preliminary of textual adversarial attacks and our motivation.
Then we give a detailed description on the proposed Randomized Substitution and Vote (RS&V) method.
Code is available at https://github.com/JHL-HUST/RSV .
[Figure]
The results are evaluated
3.1 PRELIMINARY
Let X be the set of input texts and Y = { y 1 , y 2 , · · · , y N } be the set of corresponding labels.
Given an input text x = ⟨ w 1 , w 2 , · · · , w n ⟩ ∈ X with the ground-truth label y true ∈
Y , a well-trained natural language model f : X → Y predicts its label ¯ y =
y true with high probability, using the maximum posterior probability:
[Equation]
The attacker perturbs the natural text x slightly to generate a textual adversarial example x adv which misleads the model prediction:
[Equation]
In this work, we focus on detecting the synonym substitution based adversarial examples generated by substituting a few words, in which each word w i ∈ x is substituted with one of its synonym ˆ w i , so as to maintain the semantic consistency while attacking.
For the task of adversarial example detection, given an input text x , the detector D should determine whether x is benign sample or adversarial example.
A stronger detector would also contain a corrector E , which could restore the correct label for the input adversarial example:
[Equation]
3.2 MOTIVATION
The optimization process of synonym substitution based textual adversarial attacks could be regarded as searching a specific sequence for word replacement, termed replacement sequence, in which the words mutually influence each other and contribute together to mislead the target classifier.
Intuitively, we postulate that we could eliminate the perturbation if we could successfully break the mutual interaction of the words in the replacement sequence.
To validate our hypothesis, we first generate adversarial examples using the Textfooler attack [Jin et al., 2020] on 1 , 000 randomly sampled texts from the AG’s News test set [Zhang et al., 2015] that are correctly classified by WordCNN
[Kim, 2014].
To break the interaction among words in the replacement sequence, we first randomly mask words in the benign samples or adversarial examples as unknown (“UNK”) token, denoted as UNK Benign or UNK Adversarial , with different rates and feed these processed texts to the model.
As shown in Figure 1, the classification accuracy is still over 90% when we mask 40% words in the benign text, showing the stability and robustness of model to such randomized mask.
In contrast, the classification accuracy on adversarial examples increases remarkably when we mask more words and we could recover 50% adversarial examples when masking 40% words.
This validates our hypothesis that we could eliminate the adversarial perturbation if we successfully break the mutual interaction of words in the replacement sequence.
However, since we randomly substitute the meaningful words with meaningless “UNK” token, the semantic meaning of the text would be destroyed significantly when masking a certain number of words, leading to poor performance on benign samples and limited improvement on adversarial examples.
Inspired by the existing synonym substitution based attacks, we attempt to randomly substitute the words with its synonyms to break the interaction of words without destroying the original semantic meaning.
As shown in
Figure 1, we find that randomly substituting words with its synonyms in the benign samples or adversarial examples, denoted as SYN Benign or SYN Adversarial , could consistently and significantly improve the robust accuracy against adversarial examples at the same time maintaining the high accuracy on benign samples under various substitution rates, which further validates our hypothesis.
Based on the above observation, we propose a novel textual adversarial example detection method called RS&V, which randomly substitutes the words in the input text with their synonyms to effectively detect the adversarial examples and restore the correct label for the input adversarial example.
3.3 RANDOMIZED SUBSTITUTION AND VOTE
Motivated by the observation that randomly substituting the words with their synonyms could eliminate the adversarial perturbation remarkably while maintaining the high clean accuracy, we propose a novel adversarial example detection ww n ww i w k w j w i
WordNet synonym set
Randomized substitution ...
Embedding space ...
E m bedd
i ng t o v o c ab + s o f r m a x
T r an s f o r m e r en c ode r
F C l a y e r
E m bedd
i ng target model
l og i t s
RS&V label
Input converted texts
Query
Vote & Detection logits original label ...
adversarialif unequal ...
i 11: end for 12: Calculate the prediction label for input text x : ¯ y =
argmax f ( x ) ▷ Vote & Detection 13: Calculate the voted label: ¯ y v = argmax (cid:80) k i =1 f ( x i ) 14: if ¯ y =
¯ y v then 15: return False, ¯ y ▷ Benign sample 16: end if 17: return True, ¯ y v ▷ Adversarial example method called Randomized Substitution and Vote (RS&V), to detect textual adversarial examples.
As shown in Figure 2, given an input text x , RS&V contains two main stages, i.e . randomized substitution and vote & detection, for detecting whether x is an adversarial example and restoring its correct label if it is adversarial.
Randomized Substitution.
Given an input text x = ⟨ w 1 , w 2 , · · · w n ⟩ , we first construct the synonym set
S ( w i ) =
{ ˆ w 1 i , ˆ w 2 i , · · · , ˆ w k i } for each word w i ∈
x
by combining the synonyms from WordNet [Miller, 1992] and neighbor words within a given Euclidean distance in the GloVe embedding space post-processed by counterfitting [Mrkši´c et al., 2016].
Then we randomly sample n · p words { w i ∈ x } and substitute each selected word with its arbitrary synonym ˆ w ji ∈ S ( w i ) to break the interaction among words in the replacement sequence for adversarial examples.
Moreover, we find that substituting some highfrequency words ( e.g ., “a”, “the”, “hello”, etc .) is not only useless to mitigate the adversarial effect, but also has negative impact on classifying benign samples.
Inspired by the
Term Frequency–Inverse Document Frequency (TF-IDF) technique, we set the words with the top s frequency among the training set as stopwords, where s is a hyper-parameter, and ignore these words for randomized sampling.
Such randomized substitution would repeat for k times to generate k different converted texts { x 1 , x 2 , · · · , x k } .
Vote & Detection.
Although we have shown that randomized substitution could remarkably mitigate the adversarial effect with an acceptable decay on the clean accuracy, we seek to further improve the recovery ability of randomized substitution and decrease the negative impact on benign samples.
To eliminate the variance introduced by randomized substitution and stabilize the detection, we feed k various converted texts { x 1 , x 2 , · · · , x k } and accumulate the logits output on these texts to vote for the final label argmax (cid:80) ki =1 f ( x i ) .
Note that the method is different from the naive approach depicted in Section 3.2.
Our RS&V would detect the input text x as an adversarial example if the voted RS&V label is inconsistent with the prediction label argmax f ( x ) and output the RS&V label as the prediction result for x .
In summary, as depicted in Figure 2, the proposed RS&V first generates multiple samples by randomly substituting words in text with synonyms and accumulates the logits of
[Table]
these samples to vote the final label.
If the voted label is not consistent with the prediction label for the input text,
RS&V would treat the input text as adversarial example and output the voted label.
The overall algorithm of RS&V is summarized in Algorithm 1.
EXPERIMENTS
To validate the effectiveness of the proposed RS&V, we conduct extensive evaluations on three benchmark datasets for three models against five popular adversarial attacks.
In this section, we first specify the experimental setup, then we compare the detection performance of RS&V with two detection baselines, and demonstrate that RS&V could achieve better detection performance in two settings and maintain the high accuracy on benign samples.
Finally, we investigate the impact of three hyper-parameters, i.e . substitution rate p , number of votes k , and stopword selection portion s .
4.1 EXPERIMENTAL SETUP
Datasets and Models.
We adopt three widely investigated benchmark datasets, i.e . IMDB
[Maas et al., 2011], AG’s
News, Yahoo!
Answers [Zhang et al., 2015], including sentiment analysis and news or topic classification.
Details about the datasets are summarized in Table 1.
To validate the generalization to model architectures of our RS&V, we adopt several popular deep learning models that exhibit state-ofthe-art performance on text classification tasks, including
Word-CNN
[Kim, 2014] and two pre-trained language models, BERT [Devlin et al., 2019] and RoBERTa [Liu et al., 2019].
Evaluation Setup.
We consider static attack evaluation (SAE) and targeted attack evaluation (TAE)
[Si et al., 2021].
In SAE setting, the attacker does not know the detection module and generates adversarial examples on the original model.
In TAE setting, the attacker could access and attack the detection module simultaneously when attacking target model.
We evaluate the detection performance against various attacks, i.e . GA
[Alzantot et al., 2018], PWWS [Ren et al., 2019], PSO [Zang et al., 2020], Textfooler [Jin et al., 2020] and HLA [Maheshwary et al., 2021], using F1 score and detection accuracy, i.e . classification accuracy on samples recovered by detection modules.
Baselines.
We compare our method with normally trained models and two recently proposed detection methods designed for synonym substitution based attacks:
• DISP [Zhou et al., 2019] trains a perturbation discriminator to identify the perturbed tokens and an embedding estimator to reconstruct the original text.
• FGWS [Mozes et al., 2021] replaces the low-frequency words with their most frequent synonyms in the dictionary to eliminate the adversarial perturbation.
Hyperparameters.
For our RS&V, the number of votes is set to 25, the substitution rate p is set to 80% on IMDB and 60% on AG’s News as well as Yahoo!
Answers based on the text length, and the stopword selection portion s is set to 2% .
We mix up the synonyms from WordNet [Miller, 1992] and embedding space as synonyms set, where the embedding space synonyms were calculated by ranking the
Euclidean distance between words based on GloVe vectors processed by counter-fitting [Mrkši ´ c et al., 2016].
We select at most 6 words on embedding space to keep the diversity of the synonyms set, and the portion s for stopwords is set to 5 for IMDB and 2 for AG’s News and Yahoo base on the frequency drops, For the baselines, we use the same way in the original paper to tune hyper-parameters on the validation set.
4.2 EVALUATION IN THE SAE SETTING
To validate the effectiveness of RS&V, we evaluate the detection accuracy, the classification accuracy on the recovered samples by the detection methods and F1 score against adversarial attacks on three datasets, and do comparison with DISP and FGWS.
Due to the high computational cost of generating adversarial examples for the attack methods, we randomly select 1,000 samples on each dataset and adopt the attacks to generate adversarial examples for each model.
The results are summarized in Table 2.
We could observe that RS&V generally exhibits superior detection performance against various adversarial attacks with high classification accuracy on benign samples.
Specifically, all the normally trained models achieve the lowest classification accuracy on three datasets against five attacks.
Compared with two detection baselines, RS&V consistently achieve much better detection accuracy and F1 score on three datasets for various models against five adversarial
[Table]
RoBERTa
[Table]
Yahoo!Answers
[Table] [Table]
BERT
[Table]
RoBERTa
[Table]
attacks, while maintaining the high classification accuracy on benign samples.
For instance, on Yahoo!
Answers dataset for RoBERTa model, RS&V exhibits even better (1.3% higher) classification accuracy on benign samples than normally trained model and averagely outperforms DISP and
FGWS with a clear margin of 16 .
1% and 3 . 0% , respectively.
This convincingly validates the high effectiveness of RS&V.
To further gain insight on the performance improvement of
RS&V, we present a randomly sampled adversarial example for Word-CNN from AG’s News generated by Textfooler and the recovered examples for three detection methods in
Table 3.
As we can see, none of DISP, FGWS, and RS&V could really recover the benign sample as we expect, but all of them could lead to correct classification result.
The results also show the fragility of textual adversarial example and might inspire more powerful detection and defense methods by pre-processing the input text without extra training or modification on the architecture in the future.
4.3 EVALUATION IN TAE SETTING
The SAE setting, where the attacker does not know there is a detection module, is the only case considered by the existing textual adversarial example detection methods.
To
[Table] [Table]
[Table] [Table]
[Table] [Table]
further validate the effectiveness of RS&V, we consider a more rigorous setting of TAE, in which the attacker could access and attack the detection module to generate adversarial examples.
Specifically, we generate adversarial examples on AG’s News for Word-CNN in the TAE setting.
Due to the high computational cost of DISP in inferring the extra models for the attacks, here we do not consider DISP as the detection baseline.
As shown in Table 4, FGWS exhibits little effectiveness against various attacks under the rigorous setting when compared with the normally trained model.
One possible reason for the poor effectiveness of FGWS would be that the attacker could easily bypass the low-frequency words and generate adversarial examples by just using the highfrequency words.
In contrast, RS&V could maintain the high effectiveness against various attacks.
We postulate that the uncertainty of RS&V at each iteration makes it harder for the attacker to find good local minima to craft adversarial examples, leading to high robustness of RS&V.
This further verifies the high effectiveness and superiority of RS&V.
4.4 ABLATION STUDY
We also conduct a series of ablation experiments to investigate the impact of hyper-parameters in RS&V, namely the substitution rate p in the randomized substitution, the number of votes k during the vote process, and the stopword
[Figure]
Ablation
[Figure]
for various p
[Figure]
Ablation
[Figure]
for various s .
selection portion s .
All the experiments are conducted on
AG’s News for Word-CNN against Textfooler using 1, randomly sampled texts that are correctly classified.
To eliminate the variance of randomness, we repeat the experiments five times and report the average detection accuracy.
The default values are p = 60% , k = 25 and s = 2% .
On the effectiveness of substitution rate.
In Figure 3a, we study the impact of substitution rate p on the detection accuracy.
When p = 0 , RS&V degenerates to the normally trained model without the detection module, which exhibits high classification accuracy on benign samples but low detection accuracy on adversarial examples.
With the increment on the value of p , more words in the input text would be substituted, but the accuracy on the benign sample is stable with slight decay, indicating the model’s high robustness against such randomized synonym substitution.
In contrast, the detection accuracy on adversarial examples increases significantly when we increase the value of p till p = 60% , which is intuitive that replacing more words in the input text with synonyms would be more likely to eliminate the adversarial perturbation.
Thus, we set p = 60% in the main experiments.
On the effectiveness of number of votes.
As shown in
Figure 3b, when the number of votes k = 1 , there would be only a single input text and the vote cannot make any difference, hence, RS&V exhibits the lowest accuracy on either benign samples or adversarial examples.
With the increment on the value of k , the accuracy on both benign samples and adversarial examples increases significantly, especially for the detection accuracy.
With a large value of k ≥ 10 , RS&V maintains high and stable performance, supporting that our vote could eliminate the variance of randomized substitution and stabilize the detection process.
In the main experiments, we simply select a large value of k = 25 on all datasets.
On the effectiveness of stopword selection portion.
We continue to investigate the impact of the stopword selection portion s on the detection accuracy.
As illustrated in Figure 3c, the classification accuracy on benign samples is generally stable for various values of s , as our vote strategy has mitigated the variance introduced by randomness, leading to high and stable performance.
The detection accuracy, however, increases when s < 2% and decreases significantly when more words in the input text cannot be substituted and thus could not eliminate the adversarial perturbation effectively.
In the main experiments, we set s = 2% for better performance.
In summary, the substitution rate p , number of votes k and stopword selection portion s significantly influence the performance when they are small ( p ≤ 60% , k ≤ 10 or s ≤ 2% ), while the performance becomes stable for larger p and k but decreases for larger s .
In our experiments, we adopt p = 60% , k = 25 and s = 2% for high and stable performance.
CONCLUSION
In this work, we identify that randomized synonym substitution could destroy the mutual interaction among words in the replacement sequence for adversarial attacks.
Based on this observation, we propose a novel adversarial example detection method called Randomized Substitution and Vote (RS&V) to effectively detect the textual adversarial examples.
RS&V adopts randomized synonym substitution to eliminate the adversarial perturbation and utilizes the accumulated votes to mitigate the variance introduced by randomness.
Our method is generally applicable to any neural models without additional training or modification on the models.
Empirical evaluations demonstrate that
RS&V could achieve better detection performance no matter whether the attacker could access the detection module than existing baselines, at the same time RS&V maintains the high accuracy on benign samples.
Moreover, RS&V identifies the fragility of textual adversarial examples, which might inspire more defense and detection methods by preprocessing the input text without sacrificing the classification accuracy on clean data.
Acknowledgements
This work is supported by National Natural Science Foundation (62076105) and Hubei International Cooperation
Foundation of China (2021EHB011).