An End-to-End Progressive Multi-Task Learning Framework for Medical Named Entity Recognition and Normalization (2021)

Abstract: Medical named entity recognition (NER) and normalization (NEN) are fundamental for constructing knowledge graphs and building QA systems. Existing implementations for medical NER and NEN are suffered from the error propagation between the two tasks. The mispredicted mentions from NER will directly influence the results of NEN. Therefore, the NER module is the bottleneck of the whole system. Besides, the learnable features for both tasks are beneficial to improving the model performance. To avoid the disadvantages of existing models and exploit the generalized representation across the two tasks, we design an end-to-end progressive multi-task learning model for jointly modeling medical NER and NEN in an effective way. There are three level tasks with progressive difficulty in the framework. The progressive tasks can reduce the error propagation with the incremental task settings which implies the lower level tasks gain the supervised signals other than errors from the higher level tasks to improve their performances. Besides, the context features are exploited to enrich the semantic information of entity mentions extracted by NER. The performance of NEN profits from the enhanced entity mention features. The standard entities from knowledge bases are introduced into the NER module for extracting corresponding entity mentions correctly. The empirical results on two publicly available medical literature datasets demonstrate the superiority of our method over nine typical methods.

## introduction to dig into the large amount of electronic medical records, there has been an increasing interest in applying information extraction to them. these techniques can generate tremendous benefit for corresponding research and applications, such as med- * corresponding author. ical knowledge graph @xcite and qa systems @xcite . among the medical text mining tasks, medical named entity recognition and normalization are the most fundamental tasks. named entity recognition tries to find the boundaries of mentions from the medical texts. and named entity normalization maps mentions extracted from the medical text to standard identifiers, such as mesh and omim @xcite . the initial pipeline implementations for medical ner and nen have a main limitation: error extractions from ner cascade into nen which result in normalization errors. besides, the mutual use between recognition and normalization is not utilized in the pipeline models. to alleviate the limitations and achieve a higher performance, some researchers focused on jointly modeling these two tasks. @xcite proposed a joint scoring function for medical ner and nen. @xcite casted the output construction process of the two tasks as a state transition process to perform medical named entity recognition and normalization. to capture the semantic features of two tasks, @xcite proposed a multi-task learning framework with an explicit feedback strategy for medical ner and nen. as shown in figure 1 , there are two common frameworks: pipeline and parallel multi-task framework. the former one is formulated to maximize the posterior probabilities p(y ner |x ) and p(y nen |m, e ) where x is the medical text, m is the medical mentions extracted by a recognition model, e is the standard entity, y ner and y nen are the labels. the latter one tries to maximize the posterior probabilities p(y ner , y nen |x ) @xcite . both of these are struggled with the bottleneck that is named entity recognition. in the above frameworks, the ner module is trained to memorize the medial mentions in the training set. however, the medical mentions are various and there is a gap between the training and test set. it is natural that the unseen mentions in training set are hard to recognize during the testing phase. therefore, the conventional frameworks do not gain more ideal generalization ability. to overcome the disadvantage mentioned above, we reconsidered the process of medical named entity recognition and normalization. the ultimate goal is to map the extracted medical mentions to the standard entity base. therefore, the target standard entity base can be regarded as a dictionary. the initial process of nen and ner can be reconsidered as detecting whether the medical text contains the candidate standard entity and finding the mentions should be replaced. based on this idea, we propose an end-to-end progressive multi-task learning framework for medical named entity recognition and normalization (e2emern foot_0 ). compared with ordinary multi-task learning, progressive multi-task learning focuses on the aggregation logic of tasks' specific features @xcite . a difficult target is divided into a few tasks that are interconnected through the combination of features. to take full advantage of the data attributes, we propose the framework including three tasks with progressive difficulty extended from the conventional ner and nen tasks. the low-level task is the traditional ner which tries to extract all entities in the medical text. the mid-level task is defined to iden-tify whether there exist medical mentions in the text that should be mapped to the candidate standard entity. the high-level task combines the first two level tasks, and targets to extract the mentions which should be mapped to the candidate standard entity. unlike the existing frameworks, e2emern exploits the progressive tasks to learn the fine-grained representations. the mid-level and high-level tasks facilitate the framework learning the corresponding features between the medical mentions and standard entities. the low-level task can gain the supervised signals from the higher level tasks to extract medical mentions corresponded to standard entities in the knowledge bases more exactly. our contributions in this manuscript can be summarized as follows: ## methodology we introduce the notations about ner and nen before getting into the details of the framework. for ner task, we denote {(x i , y i )} n s @xmath0, where x i is the medical text and y i is the ner label. given a sentence with n w words, the medical text can be formulated as @xmath1, x 2 , . . . , x n w } and the ner label is @xmath2, y 2 , . . . , y n w }. to solve the ner task, we try to maximize the posterior probability p(y |x ). according to the ner label, we can extract the medical mentions {m i } n m @xmath3, where n m is the number of the mentions. for nen task, we need to map each mention m to a standard entity e in the entity base @xmath4. we formulate the object of nen task as a posterior probability p(e |m, b ), and e is the standard entity which the mention m should be mapped to. ## conclusion in this paper, we reconsider the process of ner and nen and propose the end-to-end progressive multitask learning framework for medical named entity recognition and normalization. compared with existing methods, the framework consists of three tasks with progressive difficulty which contributes to modeling the fine-grained features between medical mentions in raw texts and standard entities. furthermore, the detailed analysis of e2emern proves its effectiveness. considering the medical area is various, we will try to adapt the framework to the cross domain problem.