Exploiting Hierarchically Structured Categories in Fine-grained C hinese Named Entity Recognition (2023)

Abstract: Chinese Named Entity Recognition (CNER) is a widely used technology in various applications. While recent studies have focused on utilizing additional information of the Chinese language and characters to enhance CNER performance, this paper focuses on a specific aspect of CNER known as fine-grained CNER (FG-CNER). FG-CNER involves the use of hierarchical, fine-grained categories (e.g. Person-MovieStar) to label named entities. To promote research in this area, we introduce the FiNE dataset, a dataset for FG-CNER consisting of 30,000 sentences from various domains and containing 67,651 entities in 54 fine-grained flattened hierarchical categories. Additionally, we propose SoftFiNE, a novel approach for FG-CNER that utilizes a custom-designed relevance scoring function based on label structures to learn the potential relevance between different flattened hierarchical labels. Our experimental results demonstrate that the proposed SoftFiNE method outperforms the state-of-the-art baselines on the FiNE dataset. Furthermore, we conduct extensive experiments on three other datasets, including OntoNotes 4.0, Weibo, and Resume, where SoftFiNE achieved state-of-the-art performance on all three datasets.

## introduction named entity recognition (ner) @xcite is a fundamental component of natural language processing and has been widely studied to overcome the challenges brought by realworld text data. chinese named entity recognition (cner) @xcite , as an important subfield of ner, has also drawn wide interests recently @xcite @xcite . fine-grained named entity recognition (fg-ner) is a key challenge in current ner research, motivated by the growing demand for more detailed * these authors contributed equally to this work. categorization of named entities. compared to traditional coarse-grained named entity recognition (cg-ner) @xcite , fg-ner requires identifying the correct label from a larger number of categories for each entity, making it more difficult than cg-ner @xcite . these challenges also apply to fine-grained chinese ner (fg-cner). however, most recent research on fg-cner has focused on addressing the difficulties posed by the nature of chinese characters (such as word segmentation @xcite ) and leveraging special information that can be extracted from the chinese language, such as glyphs @xcite , phonetics (pinyin) @xcite , and rich lexicons @xcite . the challenge posed by the increasing amount of corpus lacks attention. in other words, these are languagecentric approaches that focus on leveraging special characteristics offered by the language, rather than data-centric approaches that exploit the potential advantages that the data itself can provide. on the other hand, as the number of named entities increases, there is an increasing need to introduce hierarchical relations between different levels of categories. for example, "jackie chan" can be categorized as both a "person" and a "movie star", which can be intuitively represented by a flattened hierarchical label "person-moviestar", where "person" is the parent category and "moviestar" is the child. similarly, "taylor swift" can be labeled as "person-singer". it is clear that "person-moviestar" and "person-singer" are relevant to each other by sharing the common category of "person". however, since ner tasks mostly apply sequence labeling to words (characters) (such as bioes), these flattened hierarchical labels are usually encoded in a one-hot format @xcite , which assumes that labels are mutually exclusive, thus ignoring the potential relevance among labels. a data-centric approach should leverage the relevance between hierarchical labels, such as the fact that "jackie chan" as a "person-moviestar" is closer to "person-singer" than many other categories, and use this knowledge to improve fine-grained ner performance. despite the importance of the research agenda outlined above for cner applications, it has not received much attention in recent cner literature. we believe this is due to a lack of a high-quality fg-cner dataset. existing cner datasets are either outdated @xcite or small-scale @xcite , and also lack fine-grained hierarchical categories commonly used in cner applications today. these limitations in datasets have become a bottleneck for current cner research, making it difficult to develop more advanced approaches that can meet real-world challenges. to address this bottleneck in fine-grained chinese named entity recognition, we introduce the fine dataset, a fg-cner dataset consisting of 30,000 sentences sampled from various domains, containing 67,651 entities falling into 54 finegrained flattened hierarchical categories. to the best of our knowledge, fine is currently the largest fully open-sourced (and test-dataset-included) fg-cner dataset with hierarchical labels. to exploit the relatedness of fine-grained labels in fg-cner and facilitate future research, we propose the softfine method, a novel approach for solving fg-ner problems based on soft labels. softfine removes the mutual exclusive assumption and captures the hidden relevance between flattened hierarchical labels. additionally, we also introduce a properly designed entity augmentation method in softfine, which can further boost fg-cner performance by teaching the model to gather more context information about a character. experimental results on the fine dataset show the effectiveness of softfine compared to other baselines, such as lebert @xcite , hmcn-f @xcite , lambdarank @xcite , etc. to address the bottleneck caused by the limited cner dataset, we conduct extensive experiments on the weibo @xcite , resume @xcite and ontonotes 4.0 @xcite datasets, which are widely studied cner datasets released in the past decade. although softfine method achieves new state-of-the-art (sota) performance, the improvement compared to the previous sota is limited in terms of numbers, which also appears in recent cner research @xcite @xcite . this indicates that current datasets may no longer be able to support the design of more advanced cner techniques due to their limitations, as mentioned earlier. to facilitate future research on fg-cner, both the fine dataset and the source code of the softfine method are publicly available at https://github.com/xpastax/softfine . ## related work cner datasets. there are many open-source high-quality cner datasets available. for example, people's daily is one of the earliest cner datasets, with over 20,000 samples in four categories (person, location, organization, and date). msra @xcite is one of the largest cner datasets, with over 50,000 samples in three categories (person, location, and organization). these datasets have greatly contributed to the early research on cner, however, their utility in current cner development is limited due to their outdatedness. furthermore, real-world applications are increasingly demanding more fine-grained categories to classify named entities. ontonotes release 4.0 @xcite ) is a large-scale annotated corpus that also includes annotated cner data with 18 types of categories. however, the commonly used version is processed by @xcite , where only four categories are available. resume @xcite is a smaller dataset with more categories. it contains about 4,500 samples from real resumes and classifies entities into eight categories. weibo @xcite has about 2,000 samples with four main categories and two sub-categories for each, indicating whether the entity is a name mention or nominal mention, except "geo-political", which has no nominal mention. although it has a hierarchy of labels, the structure is too simple. thus, peng and dredze (2015) treat them as seven different labels while developing their model, which is also followed by later research. due to the limitations of existing cner datasets, we decided to construct the fine dataset to facilitate the study of hierarchical fg-cner. recent research on cner. the development of neural networks has led to a growing interest in combining well-trained word embeddings with deep learning models to support cner. various pictograph information, such as strokes @xcite , radicals @xcite , and glyphs @xcite , have been utilized to improve the performance of cner. additionally, the large lexicons in chinese, where each word is a combination of characters, have been collected and leveraged to further improve cner performance @xcite @xcite . recently, with the success of pre-trained language models in nlp, such as bert @xcite , extracting semantic information from target data has become easier, shifting research attention towards utilizing pre-trained knowledge to support advanced studies in cner @xcite . there is also research on fg-cner @xcite ) that tries to handle more categories, but the number of categories in these works is still relatively small. additionally, although @xcite utilized hierarchy information to facilitate their cner task, the hierarchies they used are based on the tree structures of how characters construct a chinese word. the hidden relevance between flattened hierarchical labels needs to be further studied. therefore, based on the fine dataset, we propose the softfine method to provide a better solution for fg-cner tasks. ## dataset to facilitate research on fine-grained chinese named entity recognition (fg-cner) with hierarchical structured labels, we construct the fine dataset. a brief introduction of the cner task in the bioes (begin, inside, outside, singleton, end) labeling scheme is given in appendix a. construction of hierarchical categories. the hierarchies of the categories in fine are collected and refined from chinese wikipedia 1 . based on their released knowledge graph, we collect the labels of every entity and keep those with the highest frequencies. however, since the knowledge graph is generated from the open domain, these collected labels often have redundant meanings. we manually fine-grained the categories into at most 3 levels of hierarchies and removed those that are less acknowledged by most people. then, following the sequence labeling method for the ner task, we flatten the hierarchies and obtain 54 categories with different numbers of hierarchy levels. a list of all 1 https://dumps.wikimedia.org/zhwiki/ the hierarchies of all 54 categories in fine. annotation. we created a dataset that simulates real-world scenarios by collecting 30,000 passages from qq browser, which provides daily content to a general audience of over 300 million people. we selected passages from 27 different domains such as medicine, sports and e-sports based on their click rates and used textrank4zh foot_0 to extract key sentences. four experts with professional knowledge in nlp were hired to label and classify the entities into hierarchical categories. to ensure accuracy, the 30,000 sentences were divided into three packs and labeled by three annotators, with the fourth double-checking for correctness. additionally, 100 samples were manually labeled for each pack, and any discrepancies were resolved through re-labeling the whole pack. in the final version of fine, the fourth annotator agreed with 89.93 percent of all labels, with the remaining 10.07 percent ## methodology in this section, we present softfine, a solution to the hierarchical labeled fine-grained cner problem that effectively utilizes the hidden relevance between flattened hierarchical labels to gain a better understanding of them. before introducing our approach, we will discuss two alternative methods for solving hierarchical labeled fg-cner. ## experiments we carry out an extensive set of experiments to investigate the effectiveness of softfine on fine and three other commonly used cner datasets: ontonotes 4.0, weibo, and resume. the statistics of those datasets are given in ## conclusion in this paper, to facilitate research on fine-grained chinese named entity recognition (fg-cner) with hierarchical labels, we construct the fine dataset, which contains 30,000 high-quality samples with 54 inter-related categories organized in a hierarchical structure. the data are sampled from various domains of online passages and are closer to the requirements of real-world cner implementation today. to exploit the hierarchical information present in fine-grained ner labels, we further propose the softfine method, a novel method aiming at enhancing the performance of fg-cner by leveraging the hidden relevance between flattened hierarchical labels and learning with relevance-aware soft labels. the proposed softfine method also uses entity augmentation to understand samples better. experimental results show that softfine has achieved the best performance on four datasets, including fine, weibo, resume, and ontonotes 4.0.