Using Punctuation as an Adversarial Attack on Deep Learning-Based NLP Systems: An Empirical Study (2023)

Abstract: This work empirically investigates punctuation insertions as adversarial attacks on NLP systems. Data from experiments on three tasks, five datasets, and six models with four attacks show that punctuation insertions, when limited to a few symbols (apostrophes and hyphens), are a superior attack vector compared to character insertions due to 1) a lower after-attack accuracy ( A_{aft-atk} ) than alphabetical character insertions; 2) higher semantic similarity between the resulting and original texts; and 3) a resulting text that is easier and faster to read as assessed with the Test of Word Reading Efficiency (TOWRE)). The tests also indicate that 4) grammar checking does not mitigate punctuation insertions and 5) punctuation insertions outperform word-level attacks in settings with a limited number of word synonyms and queries to the victim’s model. Our findings indicate that inserting a few punctuation types that result in easy-to-read samples is a general attack mechanism. In light of this threat, we assess the impact of punctuation insertions, potential mitigations, the mitigation’s tradeoffs, punctuation insertion’s worst-case scenarios and summarize our findings in a qualitative casual map, so that developers can design safer, more secure systems.

## introduction the goal of an attack is to disrupt a natural language processing (nlp) model's classification accuracy. the motivation behind researching these adversarial attacks is to create a toolbox of methods to attack systems while also pointing out flaws to improve the models' robustness. previous work on adversarial research showed that deep learning-based nlp models are sensitive to slight changes in the input @xcite such as character perturbations or word substitutions. however, these attack vectors have three major flaws: 1) letter perturbations can be detected by grammar checkers; 2) these attacks may change the meaning or, worse, the human label of the sentence (e.g., 'she' to 'he' with a character deletion for a gender classification system @xcite ); 3) they can make a sample unreadable. although word-level attacks that change words to a perturbing synonym make these perturbations almost invisible to humans, the cumulative effect of multiple synonym substitutions in a sentence can make the sample harder to understand. furthermore, the attack must find perturbing word synonyms when attacking samples in a specific domain, such as biology or law, which may be challenging if the algorithm uses general word embeddings with no domain knowledge. punctuation insertions, on the other hand, may be a feasible attack vector that is unaffected by the limitations of character perturbations/word substitutions, since it is hard for grammar checkers to detect punctuation (section 5.5) while also not drastically changing the meaning of the sentence (sections 5.7, 5.8). removing punctuation causes deep learning models to perform worse @xcite , as punctuation contains critical information that models require to function correctly @xcite . furthermore, punctuation can hold adversarial downstream information @xcite that may be exploited by malicious users. punctuation attacks remain an understudied area: previous works on the topic @xcite @xcite only casually explored punctuation and ignored whether it can generalize or show which punctuation symbols are best suited for intrusion attacks. contributions: through extensive empirical studies, we have determined that punctuation insertions can outperform, in terms of a af t-atk , alphabetical character insertions (as shown in section 5.1) and, under certain conditions, word substitution (5.2), allowing for a user-controllable tradeoff between after-attack accuracy (a af t-atk ), sample quality, and attack time efficiency when used together in a multi-level attack (5.3). specifically, hyphen (hy) and apostrophe (ap) insertions are the most effective at avoiding straightforward defense mechanisms (as shown in sections 5.4, 5.5) while preserving the original meaning, as evidenced by achieving 100% semantic similarity in our tests (as shown in section 5.7). additionally, by using the towre test, we have demonstrated that inserting only one punctuation type significantly increases attack readability by increasing reading speeds by 800% compared to character insertions, and 96.8% compared to using multiple types of punctuation (as shown in section 5.8) without compromising the attack performance (section 5.9). to aid in the understanding of our findings, we have also introduced a casual map in figure 5 . ## related work adversarial attacks on nlp systems can be categorized in terms of the level of granularity of the perturbation. character-level attacks @xcite @xcite @xcite modify individual characters in words to force the tokenizer to process multiple unrelated embeddings instead of the original, resulting in decreased performance. word-level attacks @xcite @xcite employ a search algorithm to locate useful perturbing embeddings @xcite @xcite or operations @xcite that are clustered close to the candidate attack word's embedding given a similarity constraint (such as the universal sentence encoder @xcite ). multi-level attacks combine multiple types of perturbations, making the attack cumulative. textbugger @xcite , which uses both character-level and word-level attacks, is an example of a multi-level attack. although previous research has investigated character and word-level attacks, few have studied the use of punctuation attacks. to the best of our knowledge, only zéroe @xcite , prespective atk @xcite and ssta @xcite have researched punctuation as an attack vector. while the former two randomly insert symbols within a word, the third revealed that symbols contain adversarial information and can be inserted as padding with little further optimization. zéroe, in particular, is a benchmark of ten different character attacks. out of these ten, zéroe intrude is the only one focusing on punctua-tion and is thus used as one of the gold standards in this paper. our work builds on these previous works by further exploring zéroe intrude and the concept, introduced initially in ssta, that model-specific symbols can attack binary classifiers when used as padding. our work contributes to the discussion on punctuation symbols being a general mechanism to attack deep learning models while also improving readability through the novel use of the towre metric, which tracks how quickly someone can read the adversarial text. ## experiments we use the methodology in section 3 and experimental setup to explore how punctuation insertions compare to character manipulations (section 5.1). in section 5.4, 5.5 and 5.6 we demonstrate how straightforward defence techniques fail and succeed and sections 5.7, 5.8, and 5.9 highlight the advantages of punctuation insertions where no defence technique is present. the γ choices for each test are summarized and justified in appendix e.2. ## conclusion researching adversarial attacks aims to create a toolbox to identify flaws and improve model robustness. results show that punctuation insertions as an attack are more effective than character manipulations (figure 1 ) and alphabetical character insertions (figure 2 ) and better evade grammar checkers (figures 6 , 7 ). punctuation insertions preserve more information and are faster to read (section 5.7, 5.9). simple defenses and adversarial training are not necessarily effective (section 5.4, 5.6). the information-preserving characteristic of this attack could potentially evade censorship. conversely, this highlights that a system deployed to combat fake news and offensive language propagation can potentially be compromised by this use of punctuation. our defense findings are summarized in figure 5 . we hope this inspires further research in the under-explored area of punctuation and how to process it. the code is available foot_0 .