T rigger NER : Learning with Entity Triggers as Explanations for Named Entity Recognition (2020)

Abstract: Training neural models for named entity recognition (NER) in a new domain often requires additional human annotations (e.g., tens of thousands of labeled instances) that are usually expensive and time-consuming to collect. Thus, a crucial research question is how to obtain supervision in a cost-effective way. In this paper, we introduce “entity triggers,” an effective proxy of human explanations for facilitating label-efficient learning of NER models. An entity trigger is defined as a group of words in a sentence that helps to explain why humans would recognize an entity in the sentence. We crowd-sourced 14k entity triggers for two well-studied NER datasets. Our proposed model, Trigger Matching Network, jointly learns trigger representations and soft matching module with self-attention such that can generalize to unseen sentences easily for tagging. Our framework is significantly more cost-effective than the traditional neural NER frameworks. Experiments show that using only 20% of the trigger-annotated sentences results in a comparable performance as using 70% of conventional annotated sentences.

## introduction named entity recognition (ner) is a fundamental information extraction task that focuses on extracting entities from a given text and classifying them using pre-defined categories (e.g., persons, locations, organizations) @xcite . recent advances in ner have primarily focused on training neural network models with an abundance of human annotations, yielding state-of-theart results @xcite . however, collecting human annotations for ner is expensive and time-consuming, especially in social media messages @xcite and technical domains such as biomedical publications, financial documents, legal reports, etc. as we seek to advance ner into more domains with less human effort, 𝑡 ! = 2,5,6 → 7 𝑡 " = 11,12,13 → 7 2 5 6 7 8 figure 1 : we show two individual entity triggers: t 1 ("had ... lunch at") and t 2 ("where the food"). both are associated to the same entity mention "rumble fish" (starting from 7th token) typed as restaurant (res). the standard protocol for obtaining an annotated ner dataset involves an annotator selecting token spans in a sentence as mentions of entities, and labeling them with an entity type. however, such annotation process provides limited supervision per example. consequently, one would need large amount of annotations in order to train high-performing models for a broad range of entity types, which can clearly be cost-prohibitive. the key question is then how can we learn an effective ner model in presence of limited quantities of labeled data? we, as humans, recognize an entity within a sentence based on certain words or phrases that act as cues. for instance, we could infer that 'kasdfrcxzv' is likely to be a location entity in the sentence "tom traveled a lot last year in kasdfrcxzv." we recognize this entity because of the cue phrase "travel ... in," which suggests there should be a location entity following the word 'in'. we call such phrases "entity triggers." similar to the way these triggers guide our recognition process, we hypothesize that they can also help the model to learn to generalize efficiently. specifically, we define an "entity trigger" (or trigger for simplicity) as a group of words that can help explain the recognition process of a particular entity in the same sentence. for example, in figure 1 , "had ... lunch at" foot_1 and "where the food" are two distinct triggers associated with the restaurant entity "rumble fish." an entity trigger should be a necessary and sufficient cue for humans to recognize its associated entity even if we mask the entity with a random word. thus, unnecessary words such as "fantastic" should not be considered part of the entity trigger. in this paper, we argue that a combination of entity triggers and standard entity annotations can enhance the generalization power of ner models. this approach is more powerful because unlabeled sentences, such as "bill enjoyed a great dinner with alice at zcxlbz.", can be matched with the existing trigger "had ... lunch at" via their semantic relatedness. this makes it easier for a model to recognize "zcxlbz" as a restaurant entity. in contrast, if we only have the entity annotation itself (i.e., "rumble fish") as supervision, the model will require many similar examples in order to learn this simple pattern. we hypothesize that using triggers as additional supervision is a more cost-effective way to train models. we crowd-sourced annotations of 14,708 triggers on two well-studied ner datasets to study their usefulness for the ner task. also, we propose a novel framework named trigger matching network that learns trigger representations indicative of entity types during the training phase, and identifies triggers in an unlabeled sentence at inference time to guide a traditional entity tagger for delivering better overall ner performance. different from conventional training, our learning process has two stages, where the first stage comprises jointly training a trigger classifier and the semantic trigger matcher, followed by a second stage that leverages the trigger representation and the encoding of the given sentence using an attention mechanism to learn a tagger. experiments show that the proposed model using only 20% of the trigger-annotated sentences results in a comparable performance as using 70% of conventional annotated sentences. ## problem formulation we consider the problem of how to cost-effectively learn a model for ner using entity triggers. in this section, we introduce basic concepts and their notations, present the conventional data annotation process for ner, and provide a formal task definition for learning using entity triggers. in the conventional setup for supervised learning for ner, we let @xmath0, x (2) , • • • , x (n) ] denote a sentence in the labeled training corpus d l . each labeled sentence has a ner-tag sequence @xmath1, y (2) , • • • , y (n) ], where y (i) ∈ y and y can be {o, b-per, • • • }. thus, we have d @xmath2, y i )}, and an unlabeled corpus d @xmath3. we propose to annotate entity triggers in sentences. we use t (x, y) to represent the set of annotated entity triggers, where each trigger t @xmath4, y) is associated with an entity index e and a set of word indices {w i }. note that we use the index of the first word of an entity as its entity index. that is, @xmath5, w 2 , • • • } → e), where e and w i are integers in the range of @xcite . adding triggers creates a new form of data d @xmath6, y i , t (x i , y i )}. our goal is to learn a model for ner from a trigger-labeled dataset d t , such that we can achieve comparable learning performance to a model with a much larger d l . ## trigger matching networks we propose a straightforward yet effective framework, named trigger matching networks (tmn), consisting of a trigger encoder (trigencoder), a semantic-based trigger matching module (trigmatcher), and a base sequence tagger (seqtagger). we have two learning stages for the framework: the first stage (section 3.1) jointly learns the trigencoder and trigmatcher, and the second stage (section 3.2) uses the trigger vectors to learn ner tag labels. ## experiments in this section, we first discuss how to collect entity triggers, and empirically study the dataefficiency of our proposed framework. ## conclusion we introduce "entity trigger" as a complementary annotation. we crowdsourced triggers on two mainstream datasets and will release them to the community, and proposed a novel framework tmn which can generalize to unseen sentences easily for tagging named entities.