A Neural Transition-based Joint Model for Disease Named Entity Recognition and Normalization (2021)

Abstract: Disease is one of the fundamental entities in biomedical research. Recognizing such entities from biomedical text and then normalizing them to a standardized disease vocabulary offer a tremendous opportunity for many downstream applications. Previous studies have demonstrated that joint modeling of the two sub-tasks has superior performance than the pipelined counterpart. Although the neural joint model based on multi-task learning framework has achieved state-of-the-art performance, it suffers from the boundary inconsistency problem due to the separate decoding procedures. Moreover, it ignores the rich information (e.g., the text surface form) of each candidate concept in the vocabulary, which is quite essential for entity normalization. In this work, we propose a neural transition-based joint model to alleviate these two issues. We transform the end-to-end disease recognition and normalization task as an action sequence prediction task, which not only jointly learns the model with shared representations of the input, but also jointly searches the output by state transitions in one search space. Moreover, we introduce attention mechanisms to take advantage of the text surface form of each candidate concept for better normalization performance. Experimental results conducted on two publicly available datasets show the effectiveness of the proposed method.

## introduction disease is one of the fundamental entities in biomedical research, thus it is one of the most searched topics in the biomedical literature @xcite and the internet @xcite . automatically identifying diseases mentioned in a text (e.g., a pubmed article or a health webpage) and then normalizing these identified mentions to their mapping concepts in a standardized disease vocabulary (e.g., with primary name, synonyms and definition, etc.) offers a tremendous opportunity for many downstream applications, such as mining chemical-disease relations from the literature @xcite , and providing much more relevant resources based on the search queries @xcite , etc. examples of such disease vocabularies includes mesh ( http://www.nlm.nih.gov/mesh/ ) and omim ( http://www.ncbi.nlm.nih.gov/omim ). previous studies @xcite @xcite show the effectiveness of the joint methods for the end-to-end disease recognition and normalization (aka linking) task to alleviated the error propagation problem of the traditional pipelined solutions @xcite @xcite . although taggerone @xcite and the discrete transition-based joint model @xcite successfully alleviate the error propagation problem, they heavily rely on hand-craft feature engineering. recently, zhao et al. @xcite proposes a neural joint model based on the multi-task learning framework (i.e., mtlfeedback) which significantly outperforms previous discrete joint solutions. mtl-feedback jointly shares the representations of the two sub-tasks (i.e., joint learning with shared representations of the input), however, their method suffers from the boundary inconsistency problem due to the separate decoding procedures (i.e., separate search in two different search spaces). moreover, it ignores the rich information (e.g., the text surface form) of each candidate concept in the vocabulary, which is quite essential for entity normalization. in this work, we propose a novel neural transition-based joint model named neujorn for disease named entity recognition and normalization, to alleviate these two issues of the multi-task learning based solution @xcite . we transform the end-to-end disease recognition and normalization task as an action sequence prediction task. more specifically, we introduce four types of actions (i.e., out, shift, reduce, seg-ment) for the recognition purpose and one type of action (i.e., linking) for the normalization purpose. our joint model not only jointly learns the model with shared representations, but also jointly searches the output by state transitions in one search space. moreover, we introduce attention mechanisms to take advantage of text surface form of each candidate concept for better linking action prediction. we summarize our contributions as follows. ## task definition we define the end-to-end disease recognition and normalization task as follows. given a sentence x from a document d (e.g., a pubmed abstract) and a controlled vocabulary kb (e.g., mesh and omim) which consists of a set of disease concepts, the task of end-to-end disease recognition and normalization is to identify all disease mentions @xmath0, m 2 , ..., m |m | } mentioned in x and to link each of the identified disease mention m i with its mapping concept if there is no mapping concept in kb for m i , then m @xmath1, where n il denotes that m i is unlinkable. ## neural transition-based joint model we first introduce the transition system used in the model, and then introduce the neural transitionbased joint model for this task. ## conclusions in this work, we proposed a novel neural transitionbased joint model for disease named entity recognition and normalization. experimental results conducted on two public available datasets show the effectiveness of the proposed method. in the future, we will apply this joint model to more different types of datasets, such as the clinical notes, drug labels, and tweets, etc.