Learning to Leverage High-Order Medical Knowledge Graph for Joint Entity and Relation Extraction (2023)

Abstract: Automatic medical entity and relation extraction is essential for daily electronic medical record (EMR) analysis, and has attracted a lot of academic attention. Tremendous progress has been made in recent years. However, medical terms are difficult to understand, and their relations are more complicated than general ones. Based on this situation, domain knowledge gives better background and contexts for medical terms. Despite the benefits of medical domain knowledge, the utilization way of it for joint entity and relation extraction is inadequate. To foster this line of research, in this work, we propose to leverage the medical knowledge graph for extracting entities and relations for Chinese Medical Texts in a collective way. Specifically, we propose to construct a high-order heterogeneous graph based on medical knowledge graph, which is linked to the entity mentions in the text. In this way, neighbors from the high-order heterogeneous graph can pass the message to each other for better global context representations. Our experiments on real Chinese Medical Texts show that our method is more effective than state-of-the-art methods.

## introduction medical text, e.g., electronic medical record (emr), has been produced at a rapid speed and massive volume every day. without any structured organization, this enormous volume of medical information is difficult to be read through by humans in a short time @xcite . due to this situation, many researchers have recently paid great attention to the joint entity and relation extraction in the medical domain @xcite . the challenge of joint entity and relation extraction in medical domain is that medical terms are usually difficult to understand due to the requirement of medical domain knowledge, especially for abbreviations of medical terms in the medical text. even worse, relations between medical entities become even more complicated. therefore, medical domain knowledge that could provide meaningful contexts and backgrounds is essential for the better extraction of medical entities and relations. despite the advantages of medical domain knowledge, most previous works fail to use medical domain knowledge @xcite @xcite . they solely rely on the local information in the medical text to extract entities and relations with language model (lm), which is insufficient for incomprehensible medical terms and complicated relations between entities. some recent works utilize medical knowledge for joint entity and relation extraction @xcite . however, both @xcite and @xcite simply align entity representation (node representation) from knowledge graph to local texts and fail to explicitly introduce the complicated relation contexts (edge representation) in the medical knowledge graph to enhance the deep representations of their involved entities. @xcite propose graph edge-conditioned attention network (geanet) which integrates initial static relation embedding into attention mechanism for entity representation enhancement in medical knowledge graph. nonetheless, it leaves out relational update during knowledge graph training process. @xcite propound graph network (gn) framework to update node and edge features iteratively within a heterogeneous graph. however, the edge representation update is based on the sender and receiver nodes information it links, which will bring about fluctuation since the amount of nodes is far outweighed that of edge types. therefore, we propose a method to fix these issues by providing additional relation contexts from figure 1 : an example of the high-order heterogeneous graph: the left is original first-order graph searched directly from knowledge graph with e i and r i as entity and relation, respectively, and the right is its converted second-order graph where t ei means the type of entity e i . medical knowledge graph to enrich the deep representations of entity mentions. specifically, we propose to construct a high-order heterogeneous graph, e.g., fig. 1 , to provide meaningful global contexts for its linked entity mentions. in fig. 1 , we denote g e as the standard first-order graph with entities as nodes and relations as edges, and represent g r as g e 's converted second-order graph with relations as nodes and entity types as edges. for every relation pair of an entity in g e , e.g., r 3 and r 2 , we link an edge, i.e., the entity type t e 1 , in g r connecting them. in this way, both message passing of entities via different relations and message propagation of relations via different entity types can be well diffused in the global graph structure. after extracting the high-order heterogeneous graph from the medical knowledge graph, we fuse the entity and relation representations in the global context obtained from the high-order heterogeneous graph with the local information extracted from the medical text. to summarize, our contributions are: and it further enhances the encoding of the entity mentions from medical text for joint extraction. the rest of this paper is organized as follows. section 2 discusses related work. in section 3, we introduce the typical algorithms for text and graph representation and present the proposed method for knowledge-enhanced joint extraction. section 4 shows the evaluation results on two datasets with compared to some other advanced methods. we conclude in section 5. there are two categories of entity and relation extraction methods: pipeline-based methods and joint extraction methods. pipeline-based entity and relation extraction methods: these work usually first extract entities as outputs, then extract relations for the returned entities. the drawback of pipeline-based extraction methods is that the errors of entity extraction may be accumulated when extracting relations for the already returned entities. for example, zhong and chen (2021) put forward an extra encoder and fuse the entity type information to enhance entity pair representation during the relation extraction task. jointly entity and relation extraction methods: some recent work extracts entity and relation in a collective way to overcome the accumulated error problem in the pipeline-based methods. there are two kinds of standard extraction methods, which are non-knowledge-enhanced and knowledge-enhanced methods. most joint entity and relation extraction methods ignore the domain knowledge. wang et al. (2018) utilize a novel graph scheme to solve the problem. luan et al. (2018) apply multi-task method to optimize entities, relations, and coreference simultaneously.bekoulis et al. (2018) further propose multi-context based adversarial training method. luan et al. (2019) utilize dynamic span graphs to form a general framework. fu et al. (2019) model text as relational graphs for joint extraction. zhao et al. (2021) model dense cross-modal interactions for joint extraction. wang and lu (2020) apply table-sequence encoders to extract jointly. lin et al. (2020) apply neural model for information extraction with global features. recently, eberts and ulges (2020) propose a span-based method with transformer pre-training. further, ji et al. (2020) apply span-based method with attention-based spanspecific and contextual semantic representations. moreover, wei et al. (2020) propose a cascade binary tagging framework for entity and relation extraction. yan et al. (2021) propose a partition filter network for joint entity and relation extraction. despite that these works extract entity and relation in a joint way, they are not designed for medical text. while some previous works are designed for medical text, they ignore the help of medical domain knowledge when modeling (e.g., @xcite @xcite . only a few works incorporate medical domain knowledge to enhance the contexts of the medical text for joint entity and relation extraction @xcite . among them, @xcite extract entity and relation jointly with knowledge-enhanced collective inference. verlinden et al. (2021) inject knowledge base information into entity and relation extraction and coreference resolution simultaneously. however, both lai et al. (2021) and verlinden et al. (2021) fail to explicitly incorporate complicated relation contexts for their involved entities. thus the interaction between entity and relation extraction will not be captured. our method overcomes this drawback by providing relation contexts when modeling deep representations of entities. most of the current work does not use domain knowledge, or only uses the entity context in domain knowledge, and does not include the relation context. while we explicitly model the entity context and relation context as the important context information for entity span, which improves the results of joint extraction. ## proposed method fig. 2 shows the framework of our method. it first extracts the high-order heterogeneous graph from the medical knowledge graph (section 3.1), and learns entity and relation representation from the global context (section 3.2). after that, we learn the representation of entity mentions from the local context in the medical text (section 3.3). the fusion of entity representations from both global and local contexts and its corresponding relation representation (section 3.4) is utilized to extract entities and relations in a collective way (section 3.5). ## experiments in this section, we evaluate our model with extensive experiments. we first show the experimental setup, which contains datasets, baselines for comparison, and evaluation metrics. then we evaluate the effectiveness of our model and baselines. ## conclusions in this paper, we study the problem of the joint entity and relation extraction in the medical text. we propose to construct the high-order heterogeneous graph from the medical knowledge graph, and learn the entity span representations in a knowledgeenhanced which integrates the deep representations from both global and local contexts. the extraction of entities and relations is in a collective way. the experimental results show that our model is more effective than state-of-the-art methods.