Joint Biomedical Entity and Relation Extraction with Knowledge-Enhanced Collective Inference (2021)

Abstract: Compared to the general news domain, information extraction (IE) from biomedical text requires much broader domain knowledge. However, many previous IE methods do not utilize any external knowledge during inference. Due to the exponential growth of biomedical publications, models that do not go beyond their fixed set of parameters will likely fall behind. Inspired by how humans look up relevant information to comprehend a scientific text, we present a novel framework that utilizes external knowledge for joint entity and relation extraction named KECI (Knowledge-Enhanced Collective Inference). Given an input text, KECI first constructs an initial span graph representing its initial understanding of the text. It then uses an entity linker to form a knowledge graph containing relevant background knowledge for the the entity mentions in the text. To make the final predictions, KECI fuses the initial span graph and the knowledge graph into a more refined graph using an attention mechanism. KECI takes a collective approach to link mention spans to entities by integrating global relational information into local representations using graph convolutional networks. Our experimental results show that the framework is highly effective, achieving new state-of-the-art results in two different benchmark datasets: BioRelEx (binding interaction detection) and ADE (adverse drug event extraction). For example, KECI achieves absolute improvements of 4.59% and 4.91% in F1 scores over the state-of-the-art on the BioRelEx entity and relation extraction tasks

## introduction with the accelerating growth of biomedical publications, it has become increasingly challenging to manually keep up with all the latest articles. as figure 1 : an example in the biorelex dataset. uim is an abbreviation of "ubiquitin-interacting motif". our baseline scibert model incorrectly predicts the mention as a "dna" instead of a "protein motif". a result, developing methods for automatic extraction of biomedical entities and their relations has attracted much research attention recently @xcite @xcite . many related tasks and datasets have been introduced, ranging from binding interaction detection (biorelex) @xcite to adverse drug event extraction (ade) @xcite . many recent joint models for entity and relation extraction rely mainly on distributional representations and do not utilize any external knowledge source @xcite @xcite . however, different from the general news domain, information extraction for the biomedical domain typically requires much broader domain-specific knowledge. biomedical documents, either formal (e.g., scientific papers) or informal ones (e.g., clinical notes), are written for domain experts. as such, they contain many highly specialized terms, acronyms, and abbreviations. in the biorelex dataset, we find that about 65% of the annotated entity mentions are abbreviations of biological entities, and an example is shown in figure 1 . these unique characteristics bring great challenges to general-domain systems and even to existing scientific language models that figure 2 : keci operates in three main steps: (1) initial span graph construction (2) background knowledge graph construction (3) fusion of these two graphs into a final span graph. keci takes a collective approach to link multiple mentions simultaneously to entities by incorporating global relational information using gcns. in this paper, we introduce keci (knowledge-enhanced collective inference), a novel end-to-end framework that utilizes external domain knowledge for joint entity and relation extraction. inspired by how humans comprehend a complex piece of scientific text, the framework operates in three main steps (figure 2 ). keci first reads the input text and constructs an initial span graph representing its initial understanding of the text. in a span graph, each node represents a (predicted) entity mention, and each edge represents a (predicted) relation between two entity mentions. keci then uses an entity linker to form a background knowledge graph containing all potentially relevant biomedical entities from an external knowledge base (kb). for each entity, we extract its semantic types, its definition sentence, and its relational information from the external kb. finally, keci uses an attention mechanism to fuse the initial span graph and the background knowledge graph into a more refined graph repre-senting the final output. different from previous methods that link mentions to entities based solely on local contexts @xcite , our framework takes a more collective approach to link multiple semantically related mentions simultaneously by leveraging global topical coherence. our hypothesis is that if multiple mentions co-occur in the same discourse and they are probably semantically related, their reference entities should also be connected in the external kb. keci integrates global relational information into mention and entity representations using graph convolutional networks (gcns) before linking. the benefit of collective inference can be illustrated by the example shown in figure 2 . the entity linker proposes two candidate entities for the mention fkbp12; one is of semantic type "aa, peptide, or protein" and the other is of semantic type "gene or genome". it can be tricky to select the correct candidate as fkbp12 is already tagged with the wrong type in the initial span graph (i.e., it is predicted to be a "chemical" instead of a "protein"). however, because of the structural resemblance between the mention-pair fk506, fkbp12 and the pair "organic chemical", "aa, peptide, or protein" , keci will link fkbp12 to the entity of semantic type "aa, peptide, or protein". as a result, the final predicted type of fkbp12 will also be corrected to "protein" in the final span graph. our extensive experimental results show that the proposed framework is highly effective, achieving new state-of-the-art biomedical entity and relation extraction performance on two benchmark datasets: biorelex @xcite and ade @xcite . for example, keci achieves absolute improvements of 4.59% and 4.91% in f1 scores over the state-of-the-art on the biorelex entity and relation extraction tasks. our analysis also shows that keci can automatically learn to select relevant candidate entities without any explicit entity linking supervision during training. furthermore, because keci considers text spans as the basic units for prediction, it can extract nested entity mentions. ## related work traditional pipelined methods typically treat entity extraction and relation extraction as two separate tasks @xcite @xcite . such approaches ignore the close interaction between named entities and their relation information and typically suffer from the error some previous work has focused on integrating external knowledge into neural architectures for other tasks, such as reading comprehension @xcite ), question answering @xcite , natural language inference @xcite , and conversational modeling @xcite . different from these studies, our work explicitly emphasizes the benefit of collective inference using global relational information. many previous studies have also used gnns for various ie tasks @xcite @xcite @xcite . many of these methods use a dependency parser or a semantic parser to construct a graph capturing global interactions between tokens/spans. however, parsers for specialized biomedical domains are expensive to build. keci does not rely on such expensive resources. ## conclusions and future work in this work, we propose a novel span-based framework named keci that utilizes external domain knowledge for joint entity and relation extraction from biomedical text. experimental results show that keci is highly effective, achieving new stateof-the-art results on two datasets: biorelex and ade. theoretically, keci can take an entire document as input; however, the tested datasets are only sentence-level datasets. in the future, we plan to evaluate our framework on more document-level datasets. we also plan to explore a broader range of properties and information that can be extracted from external kbs to facilitate biomedical ie tasks. finally, we also plan to apply keci to other information extraction tasks @xcite @xcite .