Adversarial Robustness for Large Language NER models using Disentanglement and Word Attributions (2023)

Abstract: Large language models (LLM’s) have been widely used for several applications such as question answering, text classification and clustering. While the preliminary results across the aforementioned tasks looks promising, recent work has dived deep into LLM’s performing poorly for complex Named Entity Recognition (NER) tasks in comparison to fine-tuned pre-trained language models (PLM’s). To enhance wider adoption of LLM’s, our paper investigates the robustness of such LLM NER models and its instruction fine-tuned variants to adversarial attacks. In particular, we propose a novel attack which relies on disentanglement and word attribution techniques where the former aids in learning an embedding capturing both entity and non-entity influences separately, and the latter aids in identifying important words across both components. This is in stark contrast to most techniques which primarily leverage non-entity words for perturbations limiting the space being explored to synthesize effective adversarial examples. Adversarial training results based on our method improves the F1 score over original LLM NER model by 8% and 18% on CoNLL-2003 and Ontonotes 5.0 datasets respectively.

## introduction named entity recognition (ner) aims to identify and categorize named entities mentioned in unstructured text into pre-defined categories such as person, location, or organization. in recent years, ner tasks @xcite have become more challenging due to the introduction of complex tagsets, which often leads to the failure of existing ner systems in accurately recognizing these entities. to address this, large language models (llms) have emerged as powerful tools, delivering significant performance improvements * work done while interning at amazon alexa ai on ner tasks. however, these models tend to hallucinate with even minor modifications in the input @xcite . recently, there has been a growing interest in developing adversarial attackbased techniques for ner models @xcite to enhance ner models using word-level attacks. this is even more relevant in the context of llm's, as a very recent study @xcite demonstrated the lack of robustness in current llm's with word-level attacks resulting in a significant performance drop of 33%. however, despite resulting in a successful attack, many of the perturbed word-level attack candidates fail to qualify as adversarial examples, i.e., they are not semantically similar to original sentences as depicted in figure 1 . in addition, in resource constrained settings, it isn't frugal to explore the entire space of feasible word-level perturbations to devise good adversarial examples. this justifies the necessity of attack techniques which can explore the diverse space efficiently. in our paper, this is mainly accomplished using two key levers, namely, disentanglement and word attribution techniques, respectively. disentanglement @xcite is a technique which helps in separating the latent entity and context components of an embedding space (figure 2 ), making it more congenial for a word attribution function like integrated gradients (ig) @xcite to identify diverse, yet important words. the other subsequent steps include substitution of the selected words with entity and context substitution workflows (figure 4 ), and selection of candidate adversarial examples based on their semantic similarity scores to the original text. experimental results indicate that to create a successful attack, on average, our method requires 69% (details presented in appendix a.1) lesser candidate adversarial sample generation than a state-of-the-art technique like clare @xcite when evaluated on three popular datasets @xcite , ontonotes 5.0 @xcite , and multiconer @xcite ). the results also demonstrate that our method improves the f1 score over original bert ner model by 8% and 18% on conll-2003 and ontonotes 5.0 respectively. furthermore, in an intrinsic evaluation of our adversarial examples generation approach on conll-2003, we achieve a 10% higher attack success rate (percentage of generated adversarial examples causing label flip) at a comparable modification rate. in addition, our method successfully attacks theinstruction fine-tuned t5 ner model @xcite on the multiconer dataset., resulting in a 10% drop in the f1 score after the attack. ## related work the existing adversarial attack methods on ner tasks can be classified into three categories: character-level attack, word-level attack, and sentence-level attack. the character-level attacks generate adversarial examples by adding, deleting, or replacing a character in a word in the natural language texts. hotflip @xcite performs a character-level attack by swapping characters based on their gradient with respect to a one of the two honorable psychologists in the studio is shi zhou hanhua ...... ...... professor a.a. chapin from the institute of law of vaskhnil. one of the two honorable chairs in the association is professor a.a. chapin from the institute of law of vaskhnil. one of the two honorable guests in the school is ...... ...... from the institute of history of ...... ...... from the institute of astronomy of ...... adv_example_fig_2.drawio https://drawio.corp.amazon.com/ 1 of 1 6/21/23, 3:56 pm figure 1: a pca plot of an original sentence and the generated adversarial examples using different attack methods. the original sentence is one of the two honorable guests in the studio is professor zhou hanhua from the institute of law of the chinese academy of social sciences., represented with • , the ⋆ are the adversarial examples from our method, and the markers represent the adversarial examples from rockner. across sentences we only retained modified spans and exclude identical content as compared to the original sentence. word-level attacks are more popular and effective methods than the above-mentioned two methods. there are also many effective word-level attacks on text classification tasks @xcite . @xcite proposes an efficient local search algorithm to determine the possible word substitutions. however, the adversarial attacks on sequence-to-sequence models are not much explored. @xcite extends textattack @xcite framework that consists of multiple attack strategies via reformulating the goal functions to support ner tasks. rockner @xcite is a simple ner adversarial attack by perturbing both named entities and contexts in the original texts. however, the generated adversarial examples all have the issue of poor semantic equivalence to the original input sentences because of the high modification rate. in contrast, we propose a word-level adversarial attack on ner models that effectively identifies important words via word attributions and generates the adversarial examples with a lower modification rate. ronaldo [per] will play nine matches against manchester united [org] ronaldo will play nine matches against manchester united ronaldo will play nine matches against manchester united ronaldo (entity) matches (context) manchester united (entity) messi beckham neymar games contests tournaments fc barcelona real madrid cf arsenal fc messi will play nine matches against manchester united ronaldo will play nine games against british universities ronaldo will play nine matches against fc barcelona encode disentangle word attribution replace words entity word context word rank sentences sentence similarity comparison input sentences word embeddings from llm disentangled representation most important words new words adversarial examples figure 2 : the architecture of our method as depicted here proceeds as follows. given an input text, we first obtain word embeddings from language models that the ner model uses, then learn a disentangled representation for entity and context representations (context, entity in the umap embedding scatter plots). the disentangled representations are then used to compute scores via the ig word attribution function. the most important words are replaced with new words following word substitution workflows. after generating new sentences, we rank the sentence similarity scores and output the most similar sentences as adversarial examples. @xcite , have brought attention that they outperform many nlp tasks and have been widely used for applications @xcite . @xcite explore potential use of llms in medical education, assessment, and clinical practice. however, llms still underperform in some nlp tasks. @xcite study the robustness of in-context learning and propose an icl attack on large language models by manipulating the demonstrations. in this paper, we focus on instruction fine-tuned language model on ner tasks and examine its robustness under our proposed adversarial attack method. disentanglement @xcite has been primarily used in the space of auto-encoders for learning latent factors which are mutually orthogonal (diverse) w.r.t one another to aid with interpretability primarily for image-based applications. they have also been extended for text-based tasks @xcite to leverage the latent components to learn more desired downstream representations. in this paper, we use disentanglement to segregate the entity and non-entity representations in the feature space. ## our method in this section, we describe the threat model and the key technical components of our proposed adversarial ner framework. for the sake of brevity, we refer to non-entity as context and llm ner as ner throughout the rest of this paper. the approach consists of the following steps, (1) disentanglement of word representations, (2) selection of important words using word attributions, (3) substitution of the selected words with candidate alternatives, (4) ranking and selection of candidate adversarial examples ## conclusions llm's for complex ner problems have been gaining a lot of traction recently, and in this work, we propose an adversarial attack based framework to make these ner models more robust to widen their adoption even further. our end-to-end approach combines a novel disentanglement technique with word attributions, substitution and semantic similarity to generate adversarial examples. our disentanglement method builds upon the idea of trying to learn an embedding by disentangling entity and non-entity latent representations applying disentanglement before computing ig word attributions aids in ensuring that we are able to synthesize a diverse set of adversarial examples in an extremely efficient manner as demonstrated through lower candidate sample generation and modification rates. experimental results across bert, t5-based and llama 2 ner models on three benchmark datasets demonstrates that our method significantly outperforms competing baseline methods. ablation studies highlight the importance of disentanglement and word attribution techniques.