S eq A ttack: O n Adversarial Attacks for Named Entity Recognition (2021)

Abstract: Named Entity Recognition is a fundamental task in information extraction and is an essential element for various Natural Language Processing pipelines. Adversarial attacks have been shown to greatly affect the performance of text classification systems but knowledge about their effectiveness against named entity recognition models is limited. This paper investigates the effectiveness and portability of adversarial attacks from text classification to named entity recognition and the ability of adversarial training to counteract these attacks. We find that character-level and word-level attacks are the most effective, but adversarial training can grant significant protection at little to no expense of standard performance. Alongside our results, we also release SeqAttack, a framework to conduct adversarial attacks against token classification models (used in this work for named entity recognition) and a companion web application to inspect and cherry pick adversarial examples.

## introduction named entity recognition (ner) is the task of recognizing named entities in a chunk of text. named entities are words (one or more) belonging to a particular semantic category, such as location, person or organization. ner is used both as a standalone tool and as an essential component in several natural language processing (nlp) pipelines, such as information retrieval @xcite and machine translation @xcite . traditionally, ner has been attempted with rule-based approaches, hidden markov models and conditional random fields @xcite . in recent years, deep learning has outperformed these methods @xcite , especially with the introduction of general-purpose language models such as bert @xcite . neural networks are vulnerable to adversarial attacks, which can be defined as processes that craft incorrectly-predicted samples from correctlypredicted inputs by applying small perturbations, an example of which can be seen in figure 1 . this shows that deep learning models are fragile and might not be ready for deployment in a critical scenario. the most popular technique to overcome this issue is adversarial training, which uses adversarial attacks to craft additional training samples and retrains the model from scratch @xcite . adversarial attacks and training were largely explored with regards to text classification, but current research on ner has only explored attacks based on adversarial typos @xcite and the effectiveness of more complex attacks (at the word and sentence levels) is unknown. word-level attacks are particularly important because they generate adversarial examples highly likely to appear in the real world, providing valuable additional training data (an example can be seen in figure 1 ). this paper aims to tackle this problem by investigating the following research questions: figure 1 : word-level adversarial example for ner from conll2003 @xcite . changing standings to ranking induces an incorrect classification of super g as a non-entity. ## the seqattack framework the most popular frameworks for conducting adversarial attacks are textattack @xcite and openattack @xcite , but they do not support token classification problems such as named entity recognition, in which each token is either classified as being the beginning of (b), inside (i) or outside an entity (o) according to the inside-outside-beginning (iob) schema @xcite . in order to attack ner models we developed seqattack, a framework for conducting adversarial attacks against token classification models. the framework extends textattack and inherits its design, where attacks are composed of a goal function (the objective to optimize), transformations (how the input text is perturbed), constraints which limit the candidate perturbations and a search method. the framework can be used by nlp practitioners to attack models, for data augmentation and to quickly prototype attack strategies. inheriting the structure of textattack also means that its attack strategies can be easily ported and used against ner models. in textattack, every attack optimizes a goal function, which in the case of text classification is defined as 1 -p ŷ. where ŷ is the ground truth and p ŷ is the normalized confidence score for the ground truth. in seqattack, in order to support ner, the goal function is reformulated as follows: where y is the model prediction, n the number of tokens in the sample and x the attacked sample. countentities(x) returns the number of entity tokens in a sample. we call this function the untargeted ner goal function. goal(y, ŷ) considers valid any incorrect classification of an entity token. it's important to note that this function assigns no score to newly introduced entities. this is due to the fact that the conll2003 metrics consider only the classification of ground truth named entities. we also define the untargeted-strict ner goal function, which assigns no score to flips between i-cls and b-cls. figure 3 highlights the difference between the two goal functions. ## conclusion in this paper we showed that ner models are vulnerable to adversarial attacks at the character, word and sentence level. when allowed to alter named entities, deepwordbug is the most effective, but it produces highly ungrammatical samples (appendix figures 6 , 7 ). thus, character-level attacks are not recommended for adversarial training or data augmentation since the produced samples are unlikely to appear in a real-world setting. word-level attacks instead produce more fluent adversarial examples (appendix figures 8 , 9 ), which can be used both for adversarial training and for data augmentation. finally, with regards to sentence-level attacks, this paper finds that they often produce low-quality samples for this particular dataset (appendix figure 5 ). this is probably due to the fact that scpn paraphrases are generated following a small set of target syntactic forms, which are incompatible with conll2003. further research in this direction is recommended, as paraphrasing methods produce a richer variety of samples and may reveal weaknesses in a model which cannot be discovered by character-level or word-level attacks. to help nlp practitioners evaluate and improve their models' robustness and to foster research on adversarial attacks in token classification (and named entity recognition) we release seqattack foot_0 , a python library for conducting adversarial attacks against token classification models. the library is accompanied by a web application foot_1 to inspect the generated adversarial examples and cherry pick them for adversarial training.