Identifying Human Strategies for Generating Word-Level Adversarial Examples (2022)

Abstract: Adversarial examples in NLP are receiving increasing research attention. One line of investigation is the generation of word-level adversarial examples against fine-tuned Transformer models that preserve naturalness and grammaticality. Previous work found that human- and machine-generated adversarial examples are comparable in their naturalness and grammatical correctness. Most notably, humans were able to generate adversarial examples much more effortlessly than automated attacks. In this paper, we provide a detailed analysis of exactly how humans create these adversarial examples. By exploring the behavioural patterns of human workers during the generation process, we identify statistically significant tendencies based on which words humans prefer to select for adversarial replacement (e.g., word frequencies, word saliencies, sentiment) as well as where and when words are replaced in an input sequence. With our findings, we seek to inspire efforts that harness human strategies for more robust NLP models.

## adversarial attacks in nlp researchers in natural language processing (nlp) have identified the vulnerability of machine learning models to adversarial attacks: controlled, meaning-preserving input perturbations that cause a wrong model prediction @xcite @xcite . such adversarial examples uncover model failure cases and are a major challenge for trustworthiness and reliability. while several defence methods exist against adversarial attacks @xcite @xcite @xcite , developing robust nlp models is an open research challenge. an in-depth analysis of wordlevel adversarial examples, however, identified a range of problems, showing that they are often ungrammatical or semantically inconsistent @xcite . 1 this finding raised the question of how feasible natural and grammatically correct adversarial examples actually are in nlp. to answer this question, mozes et al. (2021a) explored whether humans are able to generate adversarial examples that are valid under such strict requirements. in that study, crowdworkers were tasked with the generation of word-level adversarial examples against a target model. the findings showed that at first sight-without strict validation-humans are less successful than automated attacks. however, when adding constraints on the preservation of sentiment, grammaticality and naturalness, human-authored examples do not differ from automated ones. the most striking finding was that automated attacks required massive computational effort while humans were able to do the same task using only a handful of queries. 2 this suggests that humans are far more efficient in adversarial attacks than automated systems, yet exactly how they achieve this is unclear. in this work, we address this question by analysing human behaviour through the public dataset from @xcite . we look at which words humans perturbed, where within a sentence those perturbations were located, and whether they mainly focused on perturbing sentimentloaded words. we find that (i) in contrast to automated attacks, humans use more frequent adversarial word substitutions, (ii) the semantic similarity between replaced words and adversarial substitutions is greater for humans than for most attacks, and (iii) humans replace sentiment-loaded words more often than algorithmic attackers. our goal is to understand what makes humans so efficient at this task, and whether these strategies could be harnessed for more adversarially robust nlp models. ## data and models we present a fine-grained analysis of the strategies that human crowdworkers employed to generate word-level adversarial examples against sentiment classification models. in the dataset from @xcite , 43 participants were recruited via amazon mechanical turk and trained to perform a word-level adversarial attack on test set sequences from the imdb movie reviews dataset @xcite . in total, 170 adversarial examples were collected. for each of the collected adversarial examples, the authors also generated automated adversarial examples using the textfooler @xcite , bae @xcite , genetic @xcite and se-memepso @xcite attacks. the textfooler attack uses a greedy wordreplacement algorithm that is guided by word saliencies and semantic similarity measures between an unperturbed sequence and the adversarial candidate. the bae attack resorts to a different technique, utilising a bert-based language model to remove and replace tokens in an input sequence. the genetic attack, in contrast, is based on a population-based method using genetic algorithms. finally, the sememepso attack is based on replacements of word sememes instead of entire words and combines this with a particle swarm optimisation approach. all attacks were performed against a roberta model @xcite fine-tuned on imdb. 3 here, we only consider adversarial examples that preserved sentiment after evaluation by an independent set of crowdworkers, which @xcite used as a key validity criterion. 3 for more model details, see section 3 in @xcite . ## analysis in this section, we report on a series of experiments analysing the human-and machine-authored adversarial examples. ## discussion and conclusion this work presented a granular analysis on strategies followed by humans when attempting to generate adversarial examples through word-level substitutions. we have shown that the difference in word frequency between replaced words and adversarial substitutions is smaller for humans than for the automated attacks. furthermore, humans tend to use substitutions that are more semantically similar to the replaced words than most attacks, and humans target words that have a sentiment value to a larger extent than automated attacks. based on the findings provided, future directions could focus on harnessing such strategies to improve existing adversarial attacks and in doing so ultimately increase the robustness of machine learning-based nlp models against adversarial attacks.