Adversarial Text Normalization (2022)

Abstract: Text-based adversarial attacks are becoming more commonplace and accessible to general internet users. As these attacks proliferate, the need to address the gap in model robustness becomes imminent. While retraining on adversarial data may increase performance, there remains an additional class of character-level attacks on which these models falter. Additionally, the process to retrain a model is time and resource intensive, creating a need for a lightweight, reusable defense. In this work, we propose the Adversarial Text Normalizer, a novel method that restores baseline performance on attacked content with low computational overhead. We evaluate the efficacy of the normalizer on two problem areas prone to adversarial attacks, i.e. Hate Speech and Natural Language Inference. We find that text normalization provides a task-agnostic defense against character-level attacks that can be implemented supplementary to adversarial retraining solutions, which are more suited for semantic alterations.

## introduction natural language processing (nlp) models help preserve the integrity of discourse in online social networks by detecting hate speech, misinformation, and other content that violates community policies (halevy, 2020). in these application scenarios, classifiers operate under significantly more adversarial conditions than in the standard paradigm of model development. users often post content that is heavily altered in order to induce worst-case errors, dramatically reducing model performance relative to a standard test set. recent research in machine learning has made strides towards building robust models to defend against sophisticated adversaries. nevertheless, our experience and experiments show that models remain vulnerable to many simple and intuitive attacks. one such class of highly-effective attacks are in our experiments on state-of-the-art robust nlp models, such attacks can decrease a model's performance by more than half. thus, these text-based attacks remain a serious and unaddressed problem for the at-scale deployment of language models in integrity areas. the literature has proposed methods to improve adversarial robustness by retraining models on "hard" and adversarial examples @xcite but we observe several open challenges with applying these approaches. first, as our experiments demonstrate, even state-of-theart models trained on multiple iterations of adversarial data continue to lack robustness to easily accessible syntactic attacks. for example, in figure 2 , inserting zero-width characters reduces a hate speech classification model's accuracy to less than 15%. second, these approaches are not amenable to quick iteration. whenever a new adversarial attack becomes more prevalent, a model developer would need to collect new data or create synthetic examples of such attacks, retrain the model, and redeploy it. this costs time, computational resources and has an environmental impact @xcite . to address these outstanding issues, we propose a novel lightweight and easily extensible method for recovering model performance in the face of adversaries who perform syntactic attacks. our key insight is that many adversarial modifications can be undone before they even reach the model with little computational overhead. from a machine learning perspective, this can be thought of as restoring the distribution of inference-time inputs to the original distribution on which the model was trained. from a computer security perspective, this method is analogous to sanitizing the inputs to programs so as to make the input safe for further processing. in this work, we present the design and implementation of a system called the adversarial text normalizer (atn), that achieves this goal. this iterative approach produces a defense mechanism that can be applied at scale in a lightweight fashion to ensure robust model performance. an important principle in computer security is that defense mechanisms should be evaluated against adaptive adversaries (petitcolas, 2011), i.e. those that can adjust their techniques to actions by the defender. therefore, we also partner with red teaming experts skilled at creating novel adversarial inputs to classical computer systems to conduct an adaptive evaluation of the atn. our text normalizer can provide sufficient robustness gains even in the face of such adaptive adversaries. our contributions are as follows: ## related work several papers have introduced benchmarks for adversarial attacks on nlp systems. attacks focused on preserving semantic content and grammar @xcite @xcite have been shown to be effective against state-ofthe-art models at the cost of requiring a greater understanding of the sentence structure and task context. in contrast, @xcite propose a benchmark of character level, orthographic perturbations as more realistic attacks in general applications, attributing the success of their high performance attacks to large out-of-vocabulary rates and disruption to tokenization. other work @xcite investigates the replacement of characters with visually similar embedding spaces and the insertion of zero-width characters, noting the effectiveness of those methods against nlp models but marginal effect on human legibility -especially when perturbing key offensive words. for such targeted attacks, @xcite ) use a simple string matching algorithm to filter obfuscated and negated key words (rojas-galeano, 2017), focusing on a limited list of target vocables on each pass. our work focuses on the implementation of text normalization as a computationally inexpensive and reusable solution to mitigate a range of highly accessible but effective adversarial text attacks such as character insertions, replacements, and censorship. concurrent work in the nli domain has addressed the bias in model performance on classic test sets and adversarial user attacks through iterative human-and-model-in-the-loop @xcite ) data generation and model training. @xcite proposed a complementary approach with the amalgamation of targeted annotator samples including challenging perturbations to generate adversarial data for hate speech classification. both works explored leveraging domain-experienced annotator resources to progressively train more robust models with each successive iteration. other works on small text pertur-bations such as adversarial typos, have proposed the use of robust token-level encodings @xcite and preceding word recognition models @xcite as reusable systems that are trained once and then reused across models and tasks. in this work we explore a more lightweight, systematic correction layer that does not require training to create model and task-agnostic defenses. ## evaluation in this section, we examine the performance of hate speech and natural language inference (nli) models with respect to the original, the augmented, and normalized datasets to evaluate the efficacy of the text normalizer. all evaluations were conducted using dynabench @xcite , in which aws ecr models are deployed as endpoints and batch transform jobs are run on aws sagemaker to get dataset predictions. we roughly spent 38.36 cpu hours on model inference (no gpus were used). ## discussions retraining models on adversarial data has been proposed as a mitigation method for adversarial attacks @xcite . its benefits are that it's relatively simple to implement, shown to be effective @xcite , and doesn't affect model throughput once deployed. however, if a new adversarial attack were to be discovered on a system, the turnaround time for deploying a robust model can be quite long. a developer would need to (1) gather and annotate or systematically generate the attacked data (2) retrain the model (3) redeploy the model. steps (1) and ( 2 ) could be highly nontrivial depending on the attack type and model size. in addition, there can also be a significant monetary and environmental cost in retraining a large model @xcite . on the other hand, text normalization allows a developer to move fast. instead of collecting data, an engineer would simply need to write one additional function to reverse said attack and test it. in addition, it's lightweight and there is no need to retrain, as this is a text preprocessing step. however, text normalization cannot handle every attack type. text normalization is best suited to mitigate character-level attacks that do not change the semantic meaning of the text, i.e. syntactic attacks. another important consideration is that text normalization does affect real-time performance. a balance must be found between intelligently mitigating attacks and compute. in our use case, we limited the normalizer to sophisticated string manipulation, as anything more would result in too much compute. hence, it becomes less feasible to build defenses that require knowledge or understanding of words, etc. thus, we recommend the best way to mitigate adversarial attacks is to use a combination of text normalization and retraining. specifically, retraining should be used for semantic adversarial attacks, and adversarial text normalization for syntactic ad-versarial attacks. despite all the models evaluated on being retrained on adversarial data, they were still vulnerable to character-level text attacks. however, together with the text normalizer, we were able to increase performance and even return to baseline performance at times. this study is meant to show that text normalization in general is a viable approach to mitigating syntactic text-based attacks. this is certainly not the final method, and we hope researchers extend this work to support multilingual text. ## conclusion we proposed a new method to mitigate text-based adversarial attacks, called the adversarial text normalizer. we evaluated the performance of models retrained on adversarial data with and without the atn. our experiments show that text normalization and retraining should be used together in order to maintain baseline performance against a broad range of adversaries.