Are Synonym Substitution Attacks Really Synonym Substitution Attacks? (2023)

Abstract: In this paper, we explore the following question: Are synonym substitution attacks really synonym substitution attacks (SSAs)?We approach this question by examining how SSAs replace words in the original sentence and show that there are still unresolved obstacles that make current SSAs generate invalid adversarial samples. We reveal that four widely used word substitution methods generate a large fraction of invalid substitution words that are ungrammatical or do not preserve the original sentence’s semantics. Next, we show that the semantic and grammatical constraints used in SSAs for detecting invalid word replacements are highly insufficient in detecting invalid adversarial samples.

## introduction deep learning-based natural language processing models have been extensively used in different tasks in many domains and have shown strong performance in different realms. however, these models seem to be astonishingly vulnerable in that their predictions can be misled by some small perturbations in the original input @xcite . these imperceptible perturbations, while not changing humans' predictions, can make a well-trained model behave worse than random. one important type of adversarial attack in natural language processing (nlp) is the synonym substitution attack (ssa). in ssas, an adversarial sample is constructed by substituting some words in the original sentence with their synonyms @xcite @xcite @xcite . this ensures that the adversarial sample is semantically similar to the original sentence, thus fulfilling the imperceptibility requirement of a valid adversarial sample. while substituting words with their semantic-related counterparts can retain the semantics of the original sentence, these attacks often utilize constraints to further guarantee that the generated adversarial samples are grammatically correct and semantically similar to the original sentence. these ssas have all been shown to successfully bring down well-trained text classifiers' performance. however, some recent works observe, by human evaluations, that the quality of the generated adversarial samples of those ssas is fairly low and is highly perceptible by human @xcite . these adversarial samples often contain grammatical errors and do not preserve the semantics of the original samples, making them difficult to understand. these characteristics violate the fundamental criteria of a valid adversarial sample: preserving semantics and being imperceptible to humans. this motivates us to investigate what causes those ssas to generate invalid adversarial samples. only by answering this question can we move on to design more realistic ssas in the future. in this paper, we are determined to answer the following question: are synonym substitution attacks in the literature really synonym substitution attacks? we explore the answer by scrutinizing the key components in several important ssas and why they fail to generate valid adversarial samples. specifically, we conduct a detailed analysis of how the word substitution sets are obtained in ssas, and we look into the semantic and grammatical constraints used to filter invalid adversarial samples. we have the following astonishing observations: ## backgrounds in this section, we provide an overview of ssas and introduce some related notations that will be used throughout the paper. ## problems with the transformations in ssas in this section, we show that the transformations introduced in section 2.2 are largely to blame for the invalid adversarial samples in ssas. this is because the substitution set s x i for x i is mostly invalid, either semantically or grammatically. ## problems with the constraints in ssas in this section, we show that the constraints commonly used in ssas cannot fully filter invalid word substitutions proposed by the transformations. ## related works some prior works also discuss a similar question that we study in this paper. @xcite uses human evaluation to reveal that ssas sometimes produce low-quality adversarial samples. they attribute this to the insufficiency of the constraints and use stricter constraints and lan-guagetool to generate better adversarial samples. our work further points out that the problem is not only in the constraints; we show that the transformations are the fundamental problems in ssas. we further show that languagetool used by morris et al. (2020a) cannot detect ungrammatical verb inflectional forms, and reveal that the adversarial samples generated by tf-adj exploit the weakness of languagetool and are often made up of ungrammatical morphological substitutions. @xcite uses human evaluations and probabilistic statements to show that ssas are low quality and do not preserve original semantics. our work can be seen as an attempt to understand the cause of the observations in @xcite . morris (2020) also questions the validity of using sentence encoders as semantic constraints. they attack sentence encoders by swapping words in a sentence with their antonyms and the attack goal is to maximally preserve the swapped sentence's sentence embedding cosine similarity with the original sentence. this is related to our experiments in section 4.2. the main differences between our experiments and theirs are: (1) when swapping words, we only swap the words that are really swapped by textfooler; on the contrary, the words swapped in morris (2020) are not necessarily words that are actually substituted in an ssa. the words swapped when attacking a sentence encoder and attacking a text classifier can be significantly different. since our goal is to verify how sentence encoders behave when used in ssas, it makes more sense to only swap the words that are really replaced by an ssa. (2) morris (2020) only uses antonyms for word substitution. ## discussion and conclusion this paper discusses how the elements in ssas lead to invalid adversarial samples. we highlight that the candidate word sets generated by all four different word substitution methods contain only a small fraction of semantically matched and grammatically correct word replacements. while these transformations produce inappropriate candidate words, this alone will not contribute to the invalid adversarial samples. the inferiority of those adversarial samples should be largely attributed to the deficiency of the constraints that ought to guarantee the quality of the perturbed sentences: word embedding cosine similarity is not always larger for valid word substitutions, sentence encoder is insensitive to invalid word swaps, and languagetool fails to detect grammar mistakes. these altogether bring about the adversarial samples that are human distinguishable, unreasonable, and mostly inexplicable. these adversarial samples are not suitable for evaluating the vulnerability of nlp models because they are not reasonable inputs. the results and observations shown in the main content of our paper are not unique for bert finetuned on ag-news, which is the only attacked model shown in section 3 and section 4. we include supplementary analyses in appendix f for different model types and datasets, which supports all the claims and observations in the main contents. in this paper, we follow previous papers on ssas to only show the result of attacking the victim model once and not reporting the performance variance due to random seed and hyperparameters used during the fine-tuning of victim model @xcite @xcite . this is because conducting ssa is very time-consuming. in our preliminary experiments, we used textattack to attack three bert models fine-tuned on ag-news and we crafted the adversarial samples for 100 samples in the testing data for each model the three models were fine-tuned with three different sets of hyperparameters. we find that our observation in section 3.2 and section 4 do not change for the three victim models. overall, the observation shown in the paper is not an exception but rather a general phenomenon in ssas. by the analyses in the paper, we show that we may still be far away from real ssas, and how to construct valid synonym substitution adversarial samples remains an unresolved problem in nlp. while there is still a long way to go, it is essential to recognize that the prior works have contributed significantly to constructing valid ssas. although prior ssas may not always produce reasonable adversarial samples, they are still valuable since they pave the way for designing better ssas and help us uncover the inadequacy of the transformations and constraints for constructing real synonym substitution adversarial samples. as an initiative to stimulate future research, we provide some possible directions and guidelines for constructing better ssas, based on the observation in our paper. the problems outlined in this paper may be familiar to those with experience in lexical substitution @xcite ), but they have not yet been widely recognized in the field of ssas. our findings on why ssas fail can serve as a reality check for the field, which has been hindered by overestimating prior ssas. we hope our work will guide future researchers in cautiously building more effective ssas.