A uto T rigg ER : Label-Efficient and Robust Named Entity Recognition with Auxiliary Trigger Extraction (2023)

Abstract: Deep neural models for named entity recognition (NER) have shown impressive results in overcoming label scarcity and generalizing to unseen entities by leveraging distant supervision and auxiliary information such as explanations. However, the costs of acquiring such additional information are generally prohibitive. In this paper, we present a novel two-stage framework (AutoTriggER) to improve NER performance by automatically generating and leveraging “entity triggers” which are human-readable cues in the text that help guide the model to make better decisions. Our framework leverages post-hoc explanation to generate rationales and strengthens a model’s prior knowledge using an embedding interpolation technique. This approach allows models to exploit triggers to infer entity boundaries and types instead of solely memorizing the entity words themselves. Through experiments on three well-studied NER datasets, AutoTriggER shows strong label-efficiency, is capable of generalizing to unseen entities, and outperforms the RoBERTa-CRF baseline by nearly 0.5 F1 points on average.

## introduction named entity recognition (ner) serves as a key building block in information extraction systems. recent advances in deep neural models for ner have yielded state-of-the-art performance when sufficient human annotations are available @xcite @xcite . however, such success cannot easily transfer to practitioners developing ner systems in specific domains (e.g., biomedical papers, financial reports, legal documents), where domain-expert annotations are expensive and slow to obtain. moreover, ner systems face challenges in real-world applications where novel entities, unseen in training data, are often encountered (lin * the first two authors contributed equally. † work done before joining amazon.com. et @xcite . recent attempts addressing label scarcity and improving generalization to unseen entities have explored various types of humancurated resources as auxiliary supervision, such as entity dictionaries @xcite @xcite , labeling rules @xcite , prompts @xcite , demonstrations @xcite , retrieved context @xcite , and labeling explanations @xcite @xcite @xcite . in particular, human-provided labeling explanations as auxiliary supervision signals are cost-effective compared to collecting label-only annotations for larger number of instances. for the ner task, @xcite introduced the concept of an entity trigger, an effective way to represent explanations for the labeling decisions (see figure 1 (a) vs. (b)). however, prior works have the following limita-tions: (1) expense of collecting rationales: prior works primarily use a limited number of crowdsoured explanations (e.g., entity trigger) for improving data (label) efficiency of model training. while such human-curated auxiliary supervision is of high quality, the crowd-sourcing procedure can be very expensive and time-consuming. this largely limits the scale and domains of the collected rationales; to address these limitations, we propose a twostage ner framework, named autotrigger , which improves label efficiency and model robustness without human efforts (see figure 1 (c) ). first, it automatically generates entity triggers as explainable features to reduce human effort on collecting rationales. it then strengthens the prior knowledge in the model by interpolating model embeddings of entity-masked sentence which force the model to rely more on triggers and trigger-masked sentence which force the model to rely more on the entity. the first stage of our framework (sec. 3.1) aims to automatically extract entity triggers using a posthoc explanation. we propose to exploit the syntactic features of a sentence for assigning importance scores to a group of input tokens such that we can extract entity triggers as auxiliary supervision. specifically, we form a collection of trigger candidates for an entity in a sentence using the phrases extracted from its constituency parsing tree @xcite . then we score each trigger candidate based on its ability to predict the target entity, while varying the surrounding context to ensure trigger robustness. the second stage (sec. 3.2) focuses on how to use triggers, which are useful contextual clues, in making the prediction. we propose trigger interpolation network (tin), a novel architecture that effectively uses trigger-labeled ner data to train a model. here, we employ two separate masking passes when learning our model's embeddings: one masking the entity words (forcing the model to rely more on the triggers) and one masking the triggers (forcing the model to rely more on the entity words). we then interpolate the embeddings of both entity-masked and trigger-masked sentences to learn a mixed sentence representation, which is provided as input to a standard sequence labeling task. in this manner, the tin can effectively learn to focus on useful contextual clues to infer entity boundaries and types with contextualized embeddings from pre-trained language models such as bert @xcite while it may not miss the entities whose type can be determined by itself. (e.g., inputs that contain only the entities). extensive experimental results on several domains show that autotrigger framework consistently outperforms baseline methods by 0.5 f1 points on average in fully supervised setting. our work shows strong performance especially in label scarcity setting and unseen entity generalization. in the label scarcity setting ranging from 50 to 200 number of train instances, assuming a task that needs annotation from scratch, our model gains more than 3-4 f1 score on average. also, for the filtered test set where the entities did not appear in the train and development sets, assuming a realworld applications in which entities can be out of the distribution of the train data, our model gains more than 2-3 f1 score on average. ## problem formulation named entity recognition. we let @xmath0, x (2) , . . . x (n) ] denote the sentence consisting of a sequence of n words and @xmath1, y (2) , . . . y (n) ] denote the ner-tag sequence. the task is to predict the entity tag y (i) ∈ y for each word x (i) , where y is a pre-defined set of tags such as {b-per, i-per, . . . , o}. we let d l denote the labeled dataset consisting of the set of instances {(x i , y i )}, where x i is the i-th input sentence and y i is its output tag sequence. here, we use bioes scheme @xcite . ## approach autotrigger is a two-stage architecture that begins with an automatic trigger extraction stage followed by a trigger interpolation network (tin). it automatically extracts and scores the importance of entity trigger phrases in the first stage (sec. 3.1) and uses them in the later stage to train the ner model (sec. 3.2). ## experimental setup in this section we describe datasets along with the baseline methods and experiment settings. ## results and analysis here we look into a series of analysis questions as follows: (1) can tin with automatically extracted triggers improve the performance of existing transformer-based models? (2) can tin with automatically extracted triggers generalize well on the unseen entities? (3) are automatically extracted triggers work better than human-provided triggers? (4) can we improve the performance by asking human to refine automatically extracted triggers? we present experiment settings and details in appendix a.1-a.2. ## conclusion in this paper, we proposed a novel two-stage framework to generate and leverage explanations for named entity recognition. it automatically extracts essentially human-readable clues in the text, which is called entity triggers, by sampling and occlusion algorithm and leverages these triggers with trigger interpolation network. the model effective learns the prior knowledge to infer the entity boundaries and types by leveraging the contextual clues. we showed that our framework, named autotrig-ger, successfully generates entity triggers and effectively leverages them to improve the overall performance, especially in the label scarcity setting for technical domains where domain-expert annotations are very limited due to the high cost. moreover, it shows better generalizability on unseen entities that do not appear in training data. we believe that this work opens up future works that can be extended to semi-supervised learning or distant supervised learning which can effectively use automatically extracted triggers to weakly label the unlabeled corpus.