TEBNER : Domain Specific Named Entity Recognition with Type Expanded Boundary-aware Network (2021)

Abstract: To alleviate label scarcity in Named Entity Recognition (NER) task, distantly supervised NER methods are widely applied to automatically label data and identify entities. Although the human effort is reduced, the generated incomplete and noisy annotations pose new challenges for learning effective neural models. In this paper, we propose a novel dictionary extension method which extracts new entities through the type expanded model. Moreover, we design a multi-granularity boundary-aware network which detects entity boundaries from both local and global perspectives. We conduct experiments on different types of datasets, the results show that our model outperforms previous state-of-the-art distantly supervised systems and even surpasses the supervised models.

## introduction named entity recognition (ner) is the task of detecting mentions from text and classifying them into predefined types. it is a fundamental task in the field of natural language processing (nlp), which can facilitate many other tasks, such as entity linking @xcite , machine translation @xcite , and question answering @xcite . however, most of existing ner methods require large amounts of manually annotated texts for training supervised models, which is difficult to implement in the specific domain because domain-expert annotation is expensive and time-consuming. to alleviate the label scarcity problem, distant supervision methods @xcite @xcite have been applied to automatically generate labeled data and recognize entities. given a raw corpus and a dictionary, above methods firstly label entities by exact string matching, and then use the annotated dataset to train the well-designed neural models to recognize entities. although the human effort is reduced, the labels generated by the string matching method pose two challenges. the first challenge is incomplete annotations. because most of existing dictionaries have limited coverage on domain entities, just using the given dictionary will make many out-of-dictionary entities unmatched and generate a large number of false-negative labels. by analyzing several commonly used datasets (e.g., bc5cdr, ncbi, mesh), we find that the original dictionary only covers about 50% of domain entities, which may weaken the performance of subsequent ner model. to increase the number of label entities, previous works @xcite attempt to expand the dictionary by heuristic rules. however, these rule-based methods are difficult to migrate to other domains. so, how to extend the dictionary with more general pattern is the first problem we need to solve. the second challenge is the difficulty of recalling new entities. actually, even for the supervised models, new entities that have not been annotated are also difficult to be recalled @xcite because of the limited model capability. most of previous ner methods, such as sequential label models @xcite , and boundary detection models @xcite , only utilize the context information to recognize entities. however, the tight internal connection among entities and the global statistical features in domain corpus, which could contribute to identifying entities, are usually ignored by previous methods. therefore, how to recall new entities with multi-granularity information is the second problem we need to solve. to address two issues mentioned above, we propose a new distantly supervised method named tebner (type expanded boundary-aware ner) in specific domains. to expand the original dictio-nary, we try to extract high-quality phrases from the raw corpus and view them as potential entities. considering that these mined phrases lack corresponding type information and even contain noisy results, we use an entity typing model to classify and filter them based on their context information. then these typed phrases are added to the original dictionary to resolve the incomplete annotation problem. for the purpose of recalling more new entities, we design multi-granularity boundary labeling strategies, which can capture boundary information from different perspectives. specifically, we utilize the token interaction tagger to find the internal connection between entity tokens, the sequence labeling strategy to distinguish explicit entity boundaries in sentence and global statistical features of the whole corpus to recall potential entities. after getting the boundary results, we reuse the trained entity typing model to further classify entities and filter the noise results. in this way, we will get the trade-off between recall and precision for new entity detection. in summary, the main contributions of this paper are listed as follows: ## related work as a fundamental task, named entity recognition (ner) has drawn much attention of researchers. most previous approaches model the ner problem as a sequence labeling task and use popular architectures like nn-crf @xcite . recently, to recognize nested entities, many studies also propose to detect each entity boundaries individually @xcite @xcite . with the burgeoning popularity of pre-training methods, large-scale language models such as elmo @xcite and bert @xcite are also utilized in ner task, yielding state-of-the-art performances. however, all of above supervised models require a great quantity of manually annotated data, which are usually labor-intensive and time-consuming. to alleviate human efforts, distant supervision methods are widely used in ner task @xcite @xcite @xcite . for example, shang et al. @xcite ) marks out-of-dictionary phrases as potential entities with a special "unknown" type and propose a neural model autoner with a token interaction tagger. however, these untyped phrases are less helpful to identify new entities. hamner @xcite expands the dictionary with headwords and design a span-level model, which predicts entity boundaries by an entity classification model. but for sentences with complex structures, it is difficult to detect boundaries by just using entity type information. unlike these works, tebner annotates phrases with semantic context and distinguishes entity boundaries by fusing multi-granularity information, which can generate labels with high precision and recall rate. ## problem definition formally, given a sequence of words @xmath0, x 2 , ..., x n ], we denote an entity as e @xmath1, ..., x j ](0 ≤ @xmath2, @xmath3, @xmath4. specifically, entity types include pre-defined types (e.g., disease, chemical) and none type which denotes non-entity. in distant supervision ner task, we only need a dictionary d as input in addition to the original text. each dictionary entry contains the surface name and the entity type. in the training phase, we use dictionary matching method to generate annotations on the training corpus. ## the proposed method the overall structure of our tebner model is shown in figure 1 . the proposed framework mainly includes two parts: dictionary extender which enriches entities by assigning types to extracted high-quality phrases, entity recognizer which identifies entity boundaries by fusing multigranularity information and predicts entity types through the trained classifier. in the following, we will introduce the technical details of these modules. ## experiments to evaluate the effectiveness of our method, we conduct experiments on a series of popular distantly supervised ner datasets which are also used by @xcite @xcite . the results show that our model achieves the state-of-the-art performance. ## conclusion in this paper, we propose a new dictionary extension method and design a boundary-aware model in specific domains using distant supervision. our dictionary extender combines phrase mining method with entity classification model, which can be easily applied to any other domain corpus. by utilizing different tagging schemes to extract candidate entities from sentence and introducing autophrase tool to extract high-quality phrases from corpus, our distantly supervised ner model can detect entities from both local and global perspectives. in experiments, we evaluate our method on different domain datasets and the results demonstrate the effectiveness of our model.