Hierarchical Entity Typing via Multi-level Learning to Rank (2020)

Abstract: We propose a novel method for hierarchical entity classification that embraces ontological structure at both training and during prediction. At training, our novel multi-level learning-to-rank loss compares positive types against negative siblings according to the type tree. During prediction, we define a coarse-to-fine decoder that restricts viable candidates at each level of the ontology based on already predicted parent type(s). Our approach significantly outperform prior work on strict accuracy, demonstrating the effectiveness of our method.

## introduction entity typing is the assignment of a semantic label to a span of text, where that span is usually a mention of some entity in the real world. named entity recognition (ner) is a canonical information extraction task, commonly considered a form of entity typing that assigns spans to one of a handful of types, such as per, org, gpe, and so on. fine-grained entity typing (fet) seeks to classify spans into types according to more diverse, semantically richer ontologies @xcite @xcite @xcite , and has begun to be used in downstream models for entity linking @xcite . consider the example in figure 1 from the fet dataset, figer @xcite . the mention of interest, hollywood hills, will be typed with the single label loc in traditional ner, but may be typed with a set of types {/location, /geography, /geography/mountain} under a fine-grained typing scheme. in these finergrained typing schemes, types usually form a hierarchy: there are a set of coarse types that lies on 1 code can be found at https://github.com/ ctongfei/hierarchical-typing . the top level-these are similar to traditional ner types, e.g. /person; additionally, there are finer types that are subtypes of these top-level types, e.g. /person/artist or /person/doctor. most prior work concerning fine-grained entity typing has approached the problem as a multilabel classification problem: given an entity mention together with its context, the classifier seeks to output a set of types, where each type is a node in the hierarchy. approaches to fet include handcrafted sparse features to various neural architectures @xcite @xcite , inter alia, see section 2). perhaps owing to the historical transition from "flat" ner types, there has been relatively little work in fet that exploits ontological tree structure, where type labels satisfy the hierarchical property: a subtype is valid only if its parent supertype is also valid. we propose a novel method that takes the explicit ontology structure into account, by a multi-level learning to rank approach that ranks the candidate types conditioned on the given entity mention. intuitively, coarser types are easier whereas finer types are harder to classify: we capture this intuition by allowing distinct margins at each level of the ranking model. coupled with a novel coarse-to-fine decoder that searches on the type hierarchy, our approach guarantees that predictions do not violate the hierarchical property, and achieves state-of-the-art results according to multiple measures across various commonly used datasets. ## related work fet is usually studied as allowing for sentencelevel context in making predictions, notably starting with @xcite and @xcite , where they created the commonly used figer and ontonotes datasets for fet. while researchers have considered the benefits of document-level @xcite , and corpuslevel @xcite context, here we focus on the sentence-level variant for best contrast to prior work. progress in fet has focused primarily on: researchers have proposed alternative fet formulations whose types are not formed in a type hierarchy, in particular ultra-fine entity typing @xcite @xcite , with a very large set of types derived from phrases mined from a corpus. fet in kb @xcite labels mentions to types in a knowledge base with multiple relations, forming a type graph. @xcite augments the task with entity linking to kbs. ## problem formulation we denote a mention as a tuple @xmath0, l, r), where @xmath1, • • • , w n ) is the sentential context and the span [l : r] marks a mention of interest in sentence w. that is, the mention of interest is (w l , • • • , w r ). given x, a hierarchical entity typing model outputs a set of types y in the type ontology y, i.e. y ⊆ y. type hierarchies take the form of a forest, where each tree is rooted by a top-level supertype (e.g. /person, /location, etc.). we add a dummy parent node @xmath2, the supertype of all entity types, to all the top-level types, effectively transforming a type forest to a type tree. in figure 2 , we show 3 type ontologies associated with 3 different datasets (see subsection 5.1), with the dummy entity node augmented. we now introduce some notation for referring to aspects of a type tree. the binary relation "type z is a subtype of y" is denoted as z <: y. 2 the unique parent of a type y in the type tree is denoted ȳ ∈ y, where ȳ is undefined for @xmath3. the immediate subtypes of y (children nodes) are denoted ch(y) ⊆ y. siblings of y, those sharing the same immediate parent, are denoted sb(y) ⊆ y, where y sb(y). in the aida fet ontology (see figure 2 ), the maximum depth of the tree is @xmath4, and each mention can only be typed with at most 1 type from each level. we term this scenario singlepath typing, since there can be only 1 path starting from the root (entity) of the type tree. this is in contrast multi-path typing, such as in the bbn dataset, where mentions may be labeled with multiple types on the same level of the tree. additionally, in aida, there are mentions labeled such as as /per/police/<unspecified>. in figer, we find instances with labeled type /person but not any further subtype. what does it mean when a mention x is labeled with a partial type path, i.e., a type y but none of the subtypes z <: y? we consider two interpretations: we devise different strategies to deal with these two conditions. under the exclusive case, we add a dummy other node to every intermediate branch node in the type tree. for any mention x labeled with type y but none of the subtypes z <: y, we add this additional label "y/other" to the labels of x (see figure 2 : aida). for example, if we interpret a partial type path /person in figer as exclusive, we add another type /person/other to that instance. under the undefined case, we do not modify the labels in the dataset. we will see this can make a significant difference depending on the way a specific dataset is annotated. ## experiments the aida entity dataset has a 3-level ontology, termed type, subtype, and subsubtype. a mention can only have one label for each level, hence the dataset is single-path, thus the branching factors (k 1 , k 2 , k 3 ) for the three layers are set to (1, 1, 1). @xcite labeled a portion of the one million word penn treebank corpus of wall street journal texts (ldc95t7) using a two-level hierarchy, resulting in the bbn pronoun coreference and entity type corpus. we follow the train/test split by @xcite , and follow the train/dev split by @xcite . ontonotes @xcite sampled sentences from the ontonotes corpus and annotated the entities using 89 types. we follow the train/dev/test data split by @xcite . figer ling and weld (2012) sampled a dataset from wikipdia articles and news reports. entity mentions in these texts are mapped to a 113-type ontology derived from freebase @xcite . again, we follow the data split by @xcite . the statistics of these datasets and their accompanying ontologies are listed in ## conclusions we proposed (i) a novel multi-level learning to rank loss function that operates on a type tree, and (ii) an accompanying coarse-to-fine decoder to fully embrace the ontological structure of the types for hierarchical entity typing. our approach achieved state-of-the-art performance across various datasets, and made substantial improvement (4-8%) upon strict accuracy. additionally, we advocate for careful investigation into partial type paths: their interpretation relies on how the data is annotated, and in turn, influences typing performance.