Frequency-Guided Word Substitutions for Detecting Textual Adversarial Examples (2021)

Abstract: Recent efforts have shown that neural text processing models are vulnerable to adversarial examples, but the nature of these examples is poorly understood. In this work, we show that adversarial attacks against CNN, LSTM and Transformer-based classification models perform word substitutions that are identifiable through frequency differences between replaced words and their corresponding substitutions. Based on these findings, we propose frequency-guided word substitutions (FGWS), a simple algorithm exploiting the frequency properties of adversarial word substitutions for the detection of adversarial examples. FGWS achieves strong performance by accurately detecting adversarial examples on the SST-2 and IMDb sentiment datasets, with F1 detection scores of up to 91.4% against RoBERTa-based classification models. We compare our approach against a recently proposed perturbation discrimination framework and show that we outperform it by up to 13.0% F1.

## introduction artificial neural networks are vulnerable to adversarial examples-carefully crafted perturbations of input data that lead a learning model into making false predictions @xcite . while initially discovered for computer vision tasks, natural language processing (nlp) models have also been shown to be oversensitive to adversarial input perturbations for a variety of tasks @xcite @xcite @xcite . here we focus on highly successful synonym substitution attacks @xcite @xcite , in which individual words are replaced with semantically similar ones. existing defense methods against these attacks mainly focus on adversarial training @xcite ; ribeiro attack original or perturbed sequence none a clever blend of fact and fiction genetic a 1.39 brainy i 5.55 [clever] blend of fact and fiction pwws a 1.61 cunning i 5.55 [clever] 0.00 blending i 3.81 [blend] of fact and 0.00 fabrication i 4.39 [fiction] figure 1 : corpus log e frequencies of the replaced words (bold, italic, red) and their corresponding adversarial substitutions (bold, black) using the ge-netic @xcite and pwws @xcite attacks on sst-2 @xcite . @xcite @xcite and hence typically require a priori attack knowledge and models to be retrained from scratch to increase their robustness. recent work by @xcite instead proposes disp (learning to discriminate perturbations), a perturbation discrimination framework that exploits pre-trained contextualized word representations to detect and correct word-level adversarial substitutions without having to retrain the attacked model. in this paper, we show that we can achieve an improved performance for the detection and correction of adversarial examples based on the finding that various word-level adversarial attacks have a tendency to replace input words with less frequent ones. 1 figure 1 illustrates this tendency for two state-of-the-art attacks. we provide statistical evidence to support this observation and propose a rule-based and modelagnostic algorithm, frequency-guided word substitutions (fgws), to detect adversarial sequences and recover model performances for perturbed test set sequences. fgws effectively detects adversarial perturbations, achieving f 1 scores of up to 91.4% against roberta-based models @xcite on the imdb sentiment dataset @xcite . furthermore, our results show that fgws outperforms disp by up to 13.0% f 1 when differentiating between unperturbed and perturbed sequences, despite representing a conceptually simpler approach to this task. ## generating adversarial examples in our experiments, we investigate two baseline attacks introduced by @xcite as well as two state-of-the-art attacks. random. our first baseline attack is a simple word substitution model that randomly selects words in an input sequence and replaces them with synonyms randomly sampled from a set of synonyms related to the specific word. we follow @xcite by using wordnet @xcite to identify synonym substitutions for each selected word. ## analyzing frequencies of adversarial word substitutions next, we conduct an analysis of the word frequencies of individual words replaced by the attacks and their substitutions. we compute the log e training set frequencies φ(x) of all words x that have been replaced by the respective attacks and all of their corresponding substitutions. then, we conduct bayesian hypothesis testing @xcite to statistically compare the two samples. this is achieved by computing the bayes factor bf 10 , rep- ## frequency-guided word substitutions based on the observation of consistent frequency differences between replaced words and adversarial substitutions, we argue that the effects of such substitutions can be mitigated through simple frequency-based transformations. to do this, we propose frequency-guided word substitutions (fgws), a detection method that estimates whether a given input sequence is an adversarial example. 5 we denote a classification model by a function f (x) that maps a sequence x to a c-dimensional vector representing the probabilities for predicting each of the c possible classes. we represent a sequence as @xmath0, . . . , x n }, where x i denotes the i-th word in the sequence. we furthermore introduce the notation f * (x) ∈ {1, . . . , c} representing the class label predicted by f given input x. fgws transforms a given sequence x into a sequence x by replacing infrequent words with more frequent, semantically similar substitutions. we initially define the subset x e := {@xmath1, where δ ∈ @xmath2. fgws then generates a sequence x from x by replacing all eligible words with words that are semantically similar, but have higher occurrence frequencies in the model's training corpus. for each eligible word @xmath3. we then generate x by replacing each eligible word x with x if φ(x ) > φ(x). given the prediction label @xmath4, 1], the sequence x is considered adversarial if f (x) y -f (x ) @xmath5, i.e., if the difference in prediction confidence on class y before and after transformation exceeds the threshold γ. the threshold allows control of the rate of false positives (i.e., unperturbed sequences that are erroneously identified as adversarial) flagged by our method. ## limitations it is worth mentioning that compared to fgws, disp represents a more general perturbation discrimination approach since it is trained to detect both character-and word-level adversarial perturbations, whereas fgws solely focuses on word-level attacks. furthermore, it remains open whether fgws would be effective against attacks for which the frequency difference is less evident. to investigate this, we conducted preliminary experiments by restricting the investigated attacks to only allow equifrequent substitutions. however, we observed that introducing this constraint has a substantial effect on attack performance, since the attacks are supplied with fewer candidate replacements. we will further investigate this in future work. ## conclusion we have shown that the word frequency characteristics of adversarial word substitutions can be leveraged effectively to detect adversarial sequences for neural text classification. our proposed approach outperforms existing detection methods despite representing a conceptually simpler approach to this task. huangzhao zhang, hao zhou, ning miao, and lei li. 2019. generating fluent adversarial examples for natural languages. in proceedings of the 57th annual meeting of the association for computational linguistics, pages 5564-5569, florence, italy. association for computational linguistics. yichao zhou, jyun-yu jiang, kai-wei chang, and wei wang. 2019. learning to discriminate perturbations for blocking adversarial attacks in text classification. in proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (emnlp-ijcnlp), pages 4903-4912, hong kong, china. association for computational linguistics.