Document-level Biomedical Relation Extraction Based on Multi-Dimensional Fusion Information and Multi-Granularity Logical Reasoning (2022)

Abstract: Document-level biomedical relation extraction (Bio-DocuRE) is an important branch of biomedical text mining that aims to automatically extract all relation facts from the biomedical text. Since there are a considerable number of relations in biomedical documents that need to be judged by other existing relations, logical reasoning has become a research hotspot in the past two years. However, current models with reasoning are single-granularity only based on one element information, ignoring the complementary fact of different granularity reasoning information. In addition, obtaining rich document information is a prerequisite for logical reasoning, but most of the previous models cannot sufficiently utilize document information, which limits the reasoning ability of the model. In this paper, we propose a novel Bio-DocuRE model called FILR, based on Multi-Dimensional Fusion Information and Multi-Granularity Logical Reasoning. Specifically, FILR presents a multi-dimensional information fusion module MDIF to extract sufficient global document information. Then FILR proposes a multi-granularity reasoning module MGLR to obtain rich inference information through the reasoning of both entity-pairs and mention-pairs. We evaluate our FILR model on two widely used biomedical corpora CDR and GDA. Experimental results show that FILR achieves state-of-the-art performance.

## introduction biomedical relation extraction(bio-re) is an important branch of biomedical text mining, which always draws researchers' attention. according to the different length of processing text, bio-re is generally divided into sentence-level re and document-level re. in sentence-level re, all involved entities and relations are within a sentence. document-level re is more complex than its sentence counterpart. as shown in fig. 1 , in documentlevel biomedical relation extraction(bio-docure), the processed text is a whole document containing multiple sentences. "disulfiram" is a chemical and other entities are diseases. each entity always has more than one mention, for example, "disulfiram" has four mentions in the document. in this example, there are two intra-sentence and four intersentence chemidical induce disease(cid) relations, and the inter-sentence relation means that the two entities involved in a relation may span multiple sentences or even the entire document. early research @xcite @xcite efforts focus on sentence-level re. however, in realworld scenarios, many relations are expressed in multiple sentences, and the sentence-level models fail to recognize them. therefore, more and more attention has been paid to docure @xcite @xcite . most existed bio-docure studies are sequence-based @xcite @xcite , graph-based @xcite @xcite or transformer-based @xcite @xcite methods, which never con-sider the problem of relation reasoning and have limited effect. in the past two years, researchers noticed that there are a lot of relations in biomedical text that need to be judged relying on other relevant relations. so, bio-docure methods integrating logical reasoning have become a research hotspot @xcite @xcite @xcite . mention, entity, mention-pair, and entity-pair are four document elements in different granularity, and we notice that the purpose of logical reasoning is essential to capture the dependencies between certain document elements. consequently, we divide the logical reasoning mechanisms in bio-docure into the following four granularity: mention-based, entity-based, mention-pairbased, and entity-pair-based. all current methods with reasoning only consider single-granularity, and they perform mention-based @xcite , or entity-based @xcite @xcite , or entity-pair-based @xcite reasoning. mention-based and entity-based reasoning mainly focuses on local token-level interactions and results in limited effect. while entity-pair-based reasoning focuses on global entity-relation interaction and could provide direct guidance for document-level relation classification. compared with entity-pairbased reasoning, mention-pair-based reasoning can capture the global dependencies among all mentionpairs and provide fine-grained mention-relation reasoning information, which complements each other with the coarse-grained entity-relation reasoning information. regrettably, no study notices the importance of mention-pair-based reasoning in bio-docure so far. meanwhile, the current single-granularity methods leads to insufficient reasoning. therefore, in order to obtain the effective and sufficient reasoning information, we propose a multi-granularity logical reasoning module with both mention-pair-based and entity-pair-based reasoning. in addition, adequately mining global document information is an important step for bio-docure and is also the premise of performing logical reasoning. however, most of the current models suffer from the insufficient utilization of the document information because they either obtain the document information from a single perspective @xcite @xcite @xcite , or ignore the interaction and fusion of different document information @xcite @xcite . both the global context and structural information of a document are important, which express the document from different views. therefore, we extract both the global context and structural information of documents in this paper and propose a multi-dimensional information fusion module to fuse them. furthermore, in the process of obtaining the document global structural information, most current methods use @xcite to iteratively update the document graph. however, gcn has been proved @xcite @xcite to suffer from the over-smoothing problem when stacking the convolution layers, that is, the features of the nodes in graph would converge to similar values after deep iterations. therefore, to relieve the gcn over-smoothing problem, we improve the commonly used document graph structure @xcite @xcite by designing a bridge node to capture the document global structural information more effectively within a finite number of iteration layers. in this paper, to address the above issues, we present a novel bio-docure model called filr based on multi-dimensional fusion information and multi-granularity logical reasoning. we conduct experiments on two bio-docure datasets cdr @xcite and gda @xcite . our contributions can be summarized as follows: figure 2 : the overall framework of filr, where mc means "mention-context map", ec means "entity-context map", es means "entity-structure map" and ms means "mention-structure map". ## model in this section, we describe the overall framework of the proposed model filr in fig. 2 . filr makes an effective use of biomedical documents information and conducts multi-granularity reasoning to improve the performance of bio-docure. ## related work non-reasoning methods. previous non-reasoning bio-docure approaches can be divided into: sequence-based, graph-based, and transformerbased methods. sequence-based methods @xcite @xcite used cnn or lstm to extract document-level relations. however, these sequence-based methods do not capture the long-distance dependencies of documents and have limited effect. in order to break through the limitations, researchers tried to construct document graphs @xcite @xcite . then, with the development of pre-trained language models, transformer showed powerful capabilities in capturing context information, and is increasingly used in bio-docure task @xcite . reasoning methods. many researchers have tried to introduce various reasoning mechanisms to reinforce the ability of model in intra-and intersentence relation extraction. @xcite proposed a mention-based module for multi-hop reasoning, and introduced a co-attention mechanism for global mention reasoning. @xcite implemented inference with an iterative algorithm. @xcite and @xcite constructed a document graph and used gcn on the graph to employ multi-hop reasoning. @xcite designed a logical reasoning module that can cover more logical reasoning chains. @xcite executed entity-pair level logical reasoning using gnn-based methods. ## conclusion in this paper, we propose a novel bio-docure model called filr. filr first acquires the global context and structure information of documents. then, the mdif module is proposed to fuse the two document information effectively. next, we propose the mglr module to conduct entity-pairbased and mention-pair-based reasoning parallelly. last, we evaluate our filr model on two widely used biomedical datasets cdr and gda and the experimental results show that filr achieves stateof-the-art on two datasets.