Learning Structured Representations of Entity Names using A ctive L earning and Weak Supervision (2020)

Abstract: Structured representations of entity names are useful for many entity-related tasks such as entity normalization and variant generation. Learning the implicit structured representations of entity names without context and external knowledge is particularly challenging. In this paper, we present a novel learning framework that combines active learning and weak supervision to solve this problem. Our experimental evaluation show that this framework enables the learning of high-quality models from merely a dozen or so labeled examples.

## introduction entity normalization and variant generation are fundamental for a variety of other tasks such as semantic search and relation extraction @xcite . given an entity name e, the goal of entity normalization is to convert e to a canonical form (e.g., "jordan, michael" → "michael jordan"), while the goal of entity variant generation is to convert e to a set of different textual representations that refer to the same entity as e (e.g., "michael jordan" → {"jordan, michael", "mj", "m. jordan", . . .}). typically, entity normalization and variant generation are done by first performing entity linking @xcite @xcite , i.e., matching entity names appearing in some context (e.g., free text) to named entities in curated knowledge bases (kbs), then use the canonical form or variations (of the linked entities) residing in the kbs to complete the tasks. unfortunately, in some scenarios, such as search @xcite , entity names are not surrounded by context. furthermore, for specialized domainspecific applications, there may not be a knowledge base to govern the names of the relevant entities. thus, entity linking is not always applicable. in this paper, we take the view that entity normalization and variant generation can be done without contextual information or external kbs if we understand the internal structures of entity names. as observed in @xcite @xcite @xcite , entity names often have implicit structures that can be exploited to solve entity normalization and variant generation. declarative frameworks are proposed in @xcite to allow developers to manually specify rules that parse entity names into a structured representation. to avoid such low-level manual effort, @xcite used fully supervised methods for identifying nested entities embedded in flat named entities. unfortunately, labeled data are rarely available to leverage these methods in the real-world. to mitigate the need for training data, @xcite proposed an active learning system, lustre, to semi-automatically learn rules for mapping entity names to their structured representations. by using regex-based extractors and a list of comprehensive dictionaries that capture crucial domain vocabularies, lustre can generate rules that achieve sota results. however, for more complex and realistic scenarios, dictionaries may not be available and regex-based extractors alone are not expressive enough. moreover, as shown in section 3, lustre cannot handle long entities such as machine logs. in this paper, we present a framework that learns high-quality bert-crf models for parsing entity names into structured representations in lowresource settings, namely, when no labeled data is mention structured representation manipulation variations michael jordan "michael" first "jordan" last last , first jordan, michael createinitial( first ) last m jordan createinitial( first ) createinitial( last ) mj general electric company "general electric" name "company" suffix createinitial( name ) drop( suffix ) ge createinitial( name ) abbreviate( suffix ) ge co. ## methodology given a set @xmath0, . . . , e m } of isolated entity mentions (name strings) of a particular type, where the labeling model m is a bert-crf based model (see fig. 1 ) with several key modifications, which we elaborate next. ## experimental evaluation we implemented the system with pytorch @xcite and pytorch-transformer @xcite . four different entity types were considered (see for our system, we ask the user to label the example with highest informative score (e.g., label "michael" as first and "jordan" as last ) in each active learning iteration. then, @xmath1. in each iteration, 51 new labeled examples (or less, since there may not be 50 structurally similar examples) will be collected and used to incrementally refine the model. since crf-aw is fully supervised, we give it the sets of labels we iteratively accumulated during the active learning of our model. hence, crf-aw is not vanilla crf models, they are enhanced by our structure vectors, our active learning, and our weak supervision strategies. metrics. we report f1-scores at entity-level, tokenlevel, and component-level. entity-level measures how well the model correctly labels individual entities (all tokens of an entity must be correctly labeled). for token and component-level results, we apply models to make predictions for each token in the given set of test entities. each token prediction is credited as correct if it matches the true label. the difference between the token and componentlevel evaluation is that the former accumulates the credits over all tokens regardless the actual classes they belong to, whereas the latter evaluation accumulates the credits with respect to the actual classes. results. figure 2 reports entity-level and tokenlevel results for all methods at different iterations. as can be seen, our approach consistently outperforms the baselines, requiring only 7 to 13 actual user annotations per task. moreover, as the active learning goes, the f1-score curves of our method in all tasks increase monotonically, showing stable performance. supported by the weak labels obtained by our active sampling strategy, crf-aw gives the suboptimal results (except for the entitylevel performance for log), but there are still noticeable gaps between crf-aw and ours, indicating that pre-trained bert still plays an essential role. lustre fails to match our performance except for date. this finding is not surprising: since lustre learns highly precise rules from user labeled examples, its recall is largely determined by its sampling strategy, which is less effective to find a variety of structurally diverse examples. since our method always make a prediction, recall is trivially 100%, but the overall precision for entity-level and token-level is relatively low initially. then, whether or not our sampling strategy can keep finding the most informative examples to "complete" the training set is crucial for enhancing the overall precision. the monotonically increasing f1 curves indicate confirm the effectiveness of our method. the log dataset consists of entities with long text and complex structures. for this dataset, crf-aw performs well at token level but bad at entity type sample mentions structured representation per prof liat sossove title first last hagop youssoufia, b.s. first last , degree elmedxsna adzemovi sr. first last suffix org sony corp. corename suffix jones apparel group inc corename corename type suffix staples, inc. corename , suffix date february 2, 2019 monthofyear day , year 6/13/2012 monthofyear / day / year 1st day of april 2019 day tok {2} monthofyear year log 719+1: tue aug 22 08:26:41 1995 (/wow/wow-mbos.gif): sent binary: get /wow/wow-mbos.gif http/1.0 host timestamp ( filename ) operation : requesttype filename remainder however, lustre terminated quickly as it runs out of memory when trying to learn a rule with over 70 regex primitives to capture a long log message. regarding the component-level results, as shown in ## concluding remarks we proposed a framework for learning structured representation of entity names under low-resource settings. in particular, we focus on a challenging scenario, where entity names are given as textual strings without context. experiments show the efficacy of our approach. one immediate future work is to generate explanations for model predictions using structured vector.