Nested Named Entity Recognition as Latent Lexicalized Constituency Parsing (2022)

Abstract: Nested named entity recognition (NER) has been receiving increasing attention. Recently, Fu et al. (2020) adapt a span-based constituency parser to tackle nested NER. They treat nested entities as partially-observed constituency trees and propose the masked inside algorithm for partial marginalization. However, their method cannot leverage entity heads, which have been shown useful in entity mention detection and entity typing. In this work, we resort to more expressive structures, lexicalized constituency trees in which constituents are annotated by headwords, to model nested entities. We leverage the Eisner-Satta algorithm to perform partial marginalization and inference efficiently. In addition, we propose to use (1) a two-stage strategy (2) a head regularization loss and (3) a head-aware labeling loss in order to enhance the performance. We make a thorough ablation study to investigate the functionality of each component. Experimentally, our method achieves the state-of-the-art performance on ACE2004, ACE2005 and NNE, and competitive performance on GENIA, and meanwhile has a fast inference speed.

## introduction named entity recognition (ner) is a fundamental task in information extraction, playing an essential role in many downstream tasks. nested ner brings more flexibility than flat ner by allowing nested structures, thereby enabling more fine-grained meaning representations and broader applications @xcite . traditional sequence-labeling-based models have achieved remarkable performance on flat ner but fail to handle nested entities. to resolve this problem, there are many layer-based methods @xcite ; shibuya and hovy, * corresponding author 2020; @xcite proposed to recognize entities layer-by-layer in bottom-up or topdown manners. however, they suffer from the error propagation issue due to the cascade decoding. recently, @xcite adapt a span-based constituency parser to tackle nested ner, treating annotated entity spans as a partially-observed constituency tree and marginalizing latent spans out for training. their parsing-based method, namely po-treecrf, admits global exact inference thanks to the cyk algorithm @xcite @xcite , thereby eliminating the error propagation problem. however, their method does not consider entity heads, which provide important clues for entity mention detection @xcite and entity typing @xcite @xcite . for example, university and california are strong clues of the existence of orgedu and state entities in fig. 1 . motivated by this and inspired by head-driven phrase structures, @xcite propose the anchor-region network (arn), which identifies all entity heads firstly and then predicts the boundary and type of entities governed by each entity head. however, their method is heuristic and greedy, suffering from the error propagation problem as well. our main goal in this work is to obtain the best of two worlds: proposing a probabilistically principled method that enables exact global inference like @xcite , meanwhile taking entity heads into accounts like @xcite . to enable exact global inference, we also view observed entities as partially-observed trees. since constituency trees cannot model entity heads, we resort to lexicalized trees, in which constituents are annotated with headwords. a lexicalized tree embeds a constituency tree and a dependency tree @xcite , and lexicalized constituency parsing can thus be viewed as joint dependency and constituency parsing @xcite [bickford] @xcite reginold bickford , @xcite [researcher] a researcher @xcite at @xcite [university] @xcite the university @xcite of california and we omit the constituent labels. the dotted frame shows an example of inherited head (blue) and non-inherited head (red). we can draw a dependency arc from the inherited head to the non-inherited head. for example, @xmath0. hence a lexicalized constituency tree embeds a constituency tree and a dependency tree. when using a lexicalized constituency tree for nested ner, only part of unlexicalized spans, i.e., entities, are observed, so we need to marginalize latent spans and dependency arcs out for training. inspired by the masked inside algorithm of @xcite , we propose a masked version of the eisner-satta algorithm @xcite , a fast lexicalized constituency parsing algorithm, to perform partial marginalization. we also adopt the eisner-satta algorithm for fast inference. besides the difference in parsing formalism and algorithms, our work also differs from the work of @xcite and @xcite in the following three aspects. first, inspired by @xcite , we adopt a two-stage parsing strategy, i.e., we first predict an unlabeled tree and then label the predicted constituents, instead of using the one-stage parsing strategy of po-treecrf. we show that two-stage parsing can improve the performance of both po-treecrf and our proposed method. second, @xcite observe that each entity head governs only one entity span in most cases, so they impose a hard constraint of that during learning and inference, which is potentially harmful since the constraint is not always satisfied. instead, we add a soft kl penalty term to encourage satisfaction of the constraint, which is reminiscent of posterior regularization @xcite . third, considering that gold entity heads are not given, @xcite propose a "bag loss" for entity boundary detection and labeling. however, this loss is heuristic and brings an additional hyperparameter, to which the final performance is sensitive. in contrast, entity boundary detection is learned in the first stage of our method, and in the second stage, we propose a more principled labeling loss based on expectations (i.e., marginal likelihoods) of all possible entity heads within gold entity spans, which can be estimated efficiently and does not introduce new hyperparameters. we conduct experiments on four benchmark datasets, showing that our model achieves state-ofthe-art results on ace2004, ace2005 and nne, and competitive results on genia, validating the effectiveness of our method. ## conclusion we have presented a parsing-based method for nested ner, viewing entities as partially-observed lexicalized constituency trees, motivated by the close relationship between entity heads and entity recognition. benefiting from structural modeling, our model does not suffer from error propagation and heuristic head choosing and is easy for regularizing predictions. furthermore, our highlyparallelized implementation enables fast training and inference on gpus. experiments on four benchmark datasets validate the effectiveness and efficiency of our proposed method.