Biomedical Event Extraction with Hierarchical Knowledge Graphs (2020)

Abstract: Biomedical event extraction is critical in understanding biomolecular interactions described in scientific corpus. One of the main challenges is to identify nested structured events that are associated with non-indicative trigger words. We propose to incorporate domain knowledge from Unified Medical Language System (UMLS) to a pre-trained language model via Graph Edge-conditioned Attention Networks (GEANet) and hierarchical graph representation. To better recognize the trigger words, each sentence is first grounded to a sentence graph based on a jointly modeled hierarchical knowledge graph from UMLS. The grounded graphs are then propagated by GEANet, a novel graph neural networks for enhanced capabilities in inferring complex events. On BioNLP 2011 GENIA Event Extraction task, our approach achieved 1.41% F1 and 3.19% F1 improvements on all events and complex events, respectively. Ablation studies confirm the importance of GEANet and hierarchical KG.

## introduction biomedical event extraction is a task that identifies a set of actions among proteins or genes that are associated with biological processes from natural language texts @xcite . development of biomedical event extraction tools enables many downstream applications, such as domain-specific text mining @xcite , semantic search engines @xcite and automatic population and enrichment of database @xcite . a typical event extraction system 1) finds triggers that most clearly demonstrate the presence of events, 2) recognizes the protein participants (arguments), and 3) associates the arguments with the corresponding event triggers. for instance, the nodes associated with the tokens in the example sentence are boldfaced. bidirectional edges imply hierarchical relation between concept and semantic nodes. the word "induces" is a trigger of a positive regulation event, whose trigger role and corresponding argument role cannot be easily determined with only textual input. the kg provides clues for identifying this trigger and its corresponding arguments given the red and blue double line reasoning paths connecting nodes bmp-6, induce, phosphorylation, and positive regulation of biological process. we can infer that: 1) "induces" is an action of biological function, 2) a biological function can be quantified by positive regulation, and 3) positive regulation can result in phosphorylation. early attempts on biomedical event extraction adopted hand-crafted features @xcite @xcite . recent advances have shown improvements using deep neural networks via distributional word representations in the biomedical domain @xcite @xcite . @xcite further extends the word representations with embeddings of descriptive annotations from a knowledge base and demonstrates the importance of domain knowledge in biomedical event extraction. however, encoding knowledge with distributional embeddings does not provide adequate clues for identifying challenging events with nonindicative trigger words and nested structures. these embeddings do not contain structural or relational information about the biomedical entities. to overcome this challenge, we present a framework that incorporates knowledge from hierarchical knowledge graphs with graph neural networks (gnn) on top of a pre-trained language model. our first contribution is a novel representation of knowledge as hierarchical knowledge graphs containing both conceptual and semantic reasoning paths that enable better trigger and word identification based on unified medical language system (umls), a biomedical knowledge base. fig. 1 shows an example where the positive regulation event can be better identified with knowledge graphs and factual relational reasoning. our second contribution is a new gnn, graph edgeconditioned attention networks (geanet), that encodes complex domain knowledge. by integrating edge information into the attention mechanism, geanet has greater capabilities in reasoning the plausibility of different event structure through factual relational paths in knowledge graphs @xcite . experiments show that our proposed method achieved state-of-the-art results on the bionlp 2011 event extraction task @xcite . 1 2 background umls knowledge base. unified medical language system (umls) is a knowledge base for biomedical terminology and standards, which includes three knowledge sources: metathesaurus, semantic network, and specialist lexicon and lexical tools @xcite . we use the former two sources to build hierarchical kgs. the concept network from metathesaurus contains the relationship between each biomedical concept pairs, while each concept contains one or more semantic types 1 our code for pre-proecessing, modeling, and evaluation is available at https://github.com/pluslabnlp/ geanet-biomed-event-extraction . scibert protein sialoadhesin h i ĥ i expression was functional rbcs …… gene expression positive regulation none none mlp geanet theme theme h i, kg sentence graph concept node semantic type node token node figure 2: overview of knowledge incorporation. contextualized embeddings for each token are generated by scibert. geanet updates node embeddings for v 1 , v 2 , and v 3 via corresponding sentence graph. ## proposed approach our event extraction framework builds upon the pre-trained language model, scibert @xcite , and supplement it with a novel graph neural network model, geanet, that encodes domain knowledge from hierarchical kgs. we will first illustrate each component and discuss how training and inference are done. ## conclusion we have proposed a framework to incorporate domain knowledge for biomedical event extraction. evaluation results on ge'11 demonstrated the efficacy of geanet and hierarchical kg representation in improving extraction of non-indicative trigger words associated nested events. we also show that our method is robust when applied to different amount of training data, while being advantageous in low-resource scenarios. future works include grounding adjective triggers to knowledge bases, better biomedical knowledge representation and extracting biomedical events at document level.