BEEDS : Large-Scale Biomedical Event Extraction using Distant Supervision and Question Answering (2022)

Abstract: Automatic extraction of event structures from text is a promising way to extract important facts from the evergrowing amount of biomedical literature. We propose BEEDS, a new approach on how to mine event structures from PubMed based on a question-answering paradigm. Using a three-step pipeline comprising a document retriever, a document reader, and an entity normalizer, BEEDS is able to fully automatically extract event triples involving a query protein or gene and to store this information directly in a knowledge base. BEEDS applies a transformer-based architecture for event extraction and uses distant supervision to augment the scarce training data in event mining. In a knowledge base population setting, it outperforms a strong baseline in finding post-translational modification events consisting of enzyme-substrate-site triples while achieving competitive results in extracting binary relations consisting of protein-protein and protein-site interactions.

## introduction cellular processes such as dna damage repair or cell division are realized by the orchestration of simple biochemical events into larger structures called pathways. pathways play a crucial role in biology research, for example in network analysis @xcite or enrichment analysis @xcite . for these applications, accurate and exhaustive lists of biochemical reactions are crucial. examples for databases collecting such biochemical events are kegg @xcite , the protein interaction database @xcite and reactome @xcite . although pathway knowledge bases strive to include as much information as possible their foremost goal is the correctness of provided data and they mostly rely on manual collection and review of data. thus, they are notoriously incomplete despite extensive curation efforts @xcite . in this work, we present beeds (biomedical event extraction using distant supervision), a novel approach to biomedical event extraction from a large corpus, i.e., pubmed. beeds takes questions like what phosphorylates jak2? or what regulates expression of jak2? to find typed interactions between molecular entities and follow up questions like which sites does epo phosphorylate in jak2? to expand upon previously found answers, as a basis to recover complex event structures. to answer such questions, beeds uses a pipeline of three steps: retrieval, machine reading and entity normalization. in the first step, our model retrieves documents relevant to the query from all pubmed abstracts and pubmed central full texts. in the second step, we feed the retrieved documents to a transformer-based model to identify and extract answer spans in each document. in the third step, we apply an entity normalizer to map the identified entities to canonical database identifiers before returning them as answers. as training data for event mining is notoriously scarce, beeds applies distant supervision for obtaining a more comprehensive model. specifically, it extracts biochemical events from curated pathway knowledge bases and transforms these into text annotations, by sourcing text spans containing the pair of proteins from a knowledge base event. this creates a distant supervision training set, as we do not know whether a found text span actually describes the respective event. to the best of our knowledge, this is the first approach for distantly supervised biomedical event extraction. we augment this distantly supervised training set with gold standard text annotations for biomedical event structures from @xcite and @xcite . for evaluation, we again make use of pathway knowledge base data by checking how many of their reactions are found by our model. compared to evex @xcite as baseline, our experiments indicate that beeds is well able to mine biomedical event structures from the literature achieving a rise in recall of about 13 percentage points (pp) when mining for enzyme-substrate-site triples of post-translational modifications (ptms). in mining of binary relations like protein-protein and protein-site interactions, beeds gains about two pp in recall when compared to evex. the rest of this paper is structured as follows: in section 2, we give a brief overview over related work in event mining. in section 3, we describe the event extraction task and our used data sets, explain each part of our model pipeline in detail and provide the evaluation setup together with our baseline. in section 4 and section 5, we present and discuss our results. in section 6, we make final remarks and conclude this work. the code for reproducing this paper is freely available under https://github.com/ wangxii/beeds . ## related work the two approaches which are closest to beeds are evex @xcite and pedl @xcite . both aim to solve the task of populating pathway knowledge bases with automatically extracted event structures from the literature. evex differs from beeds as it does not use a retriever component so its document reader has to be applied to every document in pubmed which is expensive in terms of computing resources. additionally, it is only able to learn from manually labeled, directly supervised data and cannot incorporate noisy, distantly supervised text annotations for training. pedl's main difference to beeds is that it is a relation extraction system and can only extract binary relations but not more complex event structures with three or more participants. regarding the formulation of biomedical event extraction as question answering with a document reader, beeds builds upon our previous approach introduced in @xcite . deepeventmine @xcite , another approach for biomedical event extraction, solves the task by employing a multi-layered model structure each responsible for a different step in event construction like entity detection and event merging. however, both these methods only make use of directly supervised training data. furthermore, they both only cover the machine reading component of biomedical event extraction and have not been applied to large-scale biomedical event extraction. similar approaches combining a retriever reader model to pose questions directly to a corpus include drqa @xcite , realm @xcite and @xcite . drqa answers questions posed to a wikipedia corpus and uses two models, the bm25 algorithm for retrieval @xcite and a deep learning model consisting of an lstm (long short-term memory) for reading. the bm25 algorithm is still a widely used document retrieval algorithm, e.g., in the internal retrieval tool of pubmed, best match @xcite , where it is complemented by a machine learning model reranking its top 500 retrieved documents. realm and lewis et al. follow a similar idea like introduced in drqa but use dense retrieval methods, i.e., a retriever employing a deep learning model, and unite the retriever and reader components in a joint deep learning model which can be optimized end-to-end. compared to beeds, these systems lack a normalizing component and have neither been applied to event extraction nor in the biomedical domain.