From Hero to Zéroe: A Benchmark of Low-Level Adversarial Attacks (2020)

Abstract: Adversarial attacks are label-preserving modifications to inputs of machine learning classifiers designed to fool machines but not humans. Natural Language Processing (NLP) has mostly focused on high-level attack scenarios such as paraphrasing input texts. We argue that these are less realistic in typical application scenarios such as in social media, and instead focus on low-level attacks on the character-level. Guided by human cognitive abilities and human robustness, we propose the first large-scale catalogue and benchmark of low-level adversarial attacks, which we dub Zéroe, encompassing nine different attack modes including visual and phonetic adversaries. We show that RoBERTa, NLP’s current workhorse, fails on our attacks. Our dataset provides a benchmark for testing robustness of future more human-like NLP models.

## introduction adversarial examples are label-preserving modifications to inputs of machine learning architectures. their typical characteristic is that they cause little damage to humans but may maximally affect classifier performance, exposing their weaknesses and outlining the differences between human and machine text processing @xcite @xcite . while in computer vision, pixel-level attacks, which go unnoticed by humans, may lead to catastrophic failure, attacks in nlp are more challenging. some attacks in nlp replace individual words by synonyms or hyponyms @xcite or paraphrase whole sentences @xcite . however, such high-level attacks are not only more difficult to compute (requiring available resources such as dictionaries or word embeddings) but they are also implausible in real-world scenarios such as spamming or posting in social media, as users would need to know the training data and/or the inner workings of the machine learning models in order to identify candidate substitutions (or have unrestrained access to model predictions). in contrast, such users would typically use low-level attacks on characters, such as inserting placeholder symbols (e.g., underscores), mistyping words (e.g., hilter for hitler), or using phonetically similar sounding words @xcite to fool online detection models. to identify plausible such attack scenarios, human perceptual abilities play a decisive role. for instance, humans are guided by their senses, making them robust to, e.g., visual and phonetic attacks. other scenarios to which humans have been shown robust include the removal of vowels from words or the shuffling of characters while keeping the initial and final letters fixed (see section 2). however, the varieties in which text can be perturbed is certainly far from infinite, as (ordinary) humans, with all their cognitive constraints, still need to be able to decipher the text messages. in this work, we provide the first large-scale catalogue for low-level (orthographic) attack scenarios. our search is motivated by insights into human cognitive limitations and constraints and encompasses nine different attack modes (some of which are overlapping); cf. ## related work we classify adversarial attacks into highand lowlevel attacks. 2 attack scenarios. there are a variety of works that introduce low-level orthographic attacks. 3 ebrahimi et al. (2017) trick a character-level neural text classification model by flipping the characters which cause most damage. their approach is whitebox, i.e., assumes access to the attack model's parameters. @xcite exchange characters with similar looking ones and show that humans are robust to such visual perturbations, while machines may suffer severe performance drops. @xcite exchange adjacent letters on the keyboard with each other (keyboard typos) and introduce natural noise based on human typing errors extracted from different wikipedia edit histories, as well as letter swaps. they use this natural and synthetic noise to show the brittleness of machine translation (mt) systems, which contrasts with corresponding human robustness. @xcite also fool mt systems with character-level modifications. @xcite attack words by replacing them with morphological variants, which also mostly results in orthographic attacks (in english). high-level attacks require a deeper understanding of the meaning and the syntactical structure of the sentence. @xcite generate semantically similar and syntactically correct adversarial examples by replacing words with suitable synonyms. @xcite and @xcite attack toxic detection systems by obfuscation, i.e., misspelling of the abusive words (a low-level attack), and via polarization, i.e., inverting the meaning of the sentences by inserting the word "not". @xcite introduce an optimization-based algorithm to generate adversarial examples by replacing words in the input. their generated words are semantically similar because they are nearest neighbors in the glove embedding space. they are also syntactically correct because they need to fit into the surrounding context with respect to the 1 billion words language model. @xcite generate syntactically correct paraphrases for a sentence. @xcite use mt backtranslation to produce meaning-preserving adversaries. they generate adversarial examples for machine comprehension, sentiment analysis and visual question answering to show robustness issues in state-of-the-art models for each task. @xcite insert semantically correct but irrelevant paragraphs into texts to fool neural reading comprehension models. robustness. adversarial training is a commonly used technique to address adversarial attacks @xcite . the term may refer to calculating model gradients with respect to the input and inserting new training examples based on this gradient @xcite . alternatively, adversaries obtained from the attacker are inserted at train time @xcite @xcite . ## catalogue of attacks we propose a catalogue of ten different attacks. our intention is to suggest a maximally inclusive list of potential attacks under the constraint that humans are robust to them. ## conclusion we provided the first large-scale catalogue for lowlevel adversarial attacks, providing a new simple benchmark for testing real-world robustness of future deep learning models. we further showed that one of the currently most successful deep learning paradigms, roberta, is not robust to our benchmark, sometimes suffering catastrophic failure. while many of our errors could probably be addressed by placing a correction layer in front of roberta @xcite , we believe that our findings shed further light on the differences between human and machine text processing, which deep models eventually will have to innately overcome for true ai to become a viable prospect. homophone byte bite abbreviation i love you too! i luv u 2!