Nested Named Entity Recognition as Corpus Aware Holistic Structure Parsing (2022)

Abstract: As a fundamental natural language processing task and one of core knowledge extraction techniques, named entity recognition (NER) is widely used to extract information from texts for downstream tasks. Nested NER is a branch of NER in which the named entities (NEs) are nested with each other. However, most of the previous studies on nested NER usually apply linear structure to model the nested NEs which are actually accommodated in a hierarchical structure. Thus in order to address this mismatch, this work models the full nested NEs in a sentence as a holistic structure, then we propose a holistic structure parsing algorithm to disclose the entire NEs once for all. Besides, there is no research on applying corpus-level information to NER currently. To make up for the loss of this information, we introduce Point-wise Mutual Information (PMI) and other frequency features from corpus-aware statistics for even better performance by holistic modeling from sentence-level to corpus-level. Experiments show that our model yields promising results on widely-used benchmarks which approach or even achieve state-of-the-art. Further empirical studies show that our proposed corpus-aware features can substantially improve NER domain adaptation, which demonstrates the surprising advantage of our proposed corpus-level holistic structure modeling.

## introduction named entity recognition is to find predefined named entities such as locations, organizations or people in text, which usually serves as an upstream natural language processing (nlp) task @xcite @xcite hovy, figure 1 : an example of nested named entities. the solid lines or brackets represents the start and end positions of a named entity. 'gpe' is a named entity category, representing geographic or political entities. france and san francisco are nested by the san francisco of france which forms a hierarchy structure. ## corpus-aware features though our proposed holistic structure parsing model at sentence-level hopefully captures the entire sentence for predicting all nested nes, which is supposed to yield more powerful model capability than previous models, it still has the limitation similar with the previous ones which only learn the representation inside a sentence. meanwhile, nes quite sparsely distribute in all text data such as most nes only occur for once in the entire corpus. besides, there may be a huge domain difference among various corpora, which lets the model recognize new nes even more difficultly. thus, we further introduce corpus-aware clues to alleviate the above mentioned difficulties. in detail, we consider corpus-level statistics including frequency and point-wise mutual information (pmi) over words and corresponding pos tags. therefore, we consider the following statistics to form enhanced features, pmi on word (word pmi), word frequency (word freq), pmi on pos tags (pos pmi) and on pos tags frequency (pos freq). equipped with the corpus-aware features from a large amount of unlabeled corpus, the model may be enabled to effectively adapt to different domains by alleviating the difficulty caused by the sparsity of nes distribution. ## conclusion ner, so as to achieve the purpose of labeling all nested nes once for all. on the basis of the proposed holistic sentence structure modeling, we further explore ner model enhancement from corpusaware statistics with the hope of alleviating the serious sparsity issue of nes. for this purpose, we extend our holistic structure modeling from sentence-level to corpus-level by offering multiple corpus-aware features including word freq, word pmi, pos freq and pos pmi. the experimental results demonstrate the effectiveness of our proposed model by providing consistent and general performance improvement over strong baselines. in details, our model achieves performance approaching state-of-the-art for two datasets and reaches new state-of-the-art for one dataset. last but not the least, our corpus-level holistic structure modeling shows surprising merit for effective domain adaptation.