Biomedical Event Extraction as Sequence Labeling (2020)

Abstract: We introduce Biomedical Event Extraction as Sequence Labeling (BeeSL), a joint end-to-end neural information extraction model. BeeSL recasts the task as sequence labeling, taking advantage of a multi-label aware encoding strategy and jointly modeling the intermediate tasks via multi-task learning. BeeSL is fast, accurate, end-to-end, and unlike current methods does not require any external knowledge base or preprocessing tools. BeeSL outperforms the current best system (Li et al., 2019) on the Genia 2011 benchmark by 1.57% absolute F1 score reaching 60.22% F1, establishing a new state of the art for the task. Importantly, we also provide first results on biomedical event extraction without gold entity information. Empirical results show that BeeSL’s speed and accuracy makes it a viable approach for large-scale real-world scenarios.

## introduction biomedical event extraction provides invaluable means for assisting domain experts in the curation of knowledge bases and biomolecular pathways @xcite . while the task has received significant attention in research over the last decade, it remains challenging. progress has been rather stagnating (see figure 1 ). ## encoding event structures this section introduces the event structures and how we encode them for sequence labeling. ## event extraction as sequence labeling formally, we aim to learn a function f : @xmath0, i.e., an overview of beesl is shown in figure 3 . we use bert @xcite as encoder, pretrained on biomedical texts (section 4). we mask entity spans for better generalization @xcite . the first wordpiece @xcite of each token x i is used for prediction, where the contextual hidden representation e i of the token x i is encoded with layer-wise attention over the bert layers, similarly to @xcite . as decoders, we use standard softmax with a cross entropy loss unless otherwise specified, and introduce a multilabel decoder (section 3.2) (figure 3 , upper right). we empirically evaluate both single-task and multi-task setups, including several mtl encoding alternatives, discussing their limitations and benefits. in the following, we first introduce the multi-task setups, and then multi-label decoding. ## experimental setup we evaluate beesl on the genia 2011 benchmark @xcite , which comprises both abstracts and full-texts. the corpus consists of annotations for protein entities and 9 fine-grained event types. the genia event extraction tasks expect both texts and entities as input, and complete events need to be predicted. statistics on the dataset are shown in event types can be categorized into simple, binding and complex events, related to the number and types of arguments. simple events require a theme only, binding events require one or more theme arguments, while complex events take both theme and cause arguments, where both can in turn be other events, resulting in nested structures. @xcite estimated that 37.2% of the events in the data are nested. we refer the reader to appendix a.1 for formal event definitions. beesl is based on machamp (van der goot et al., 2020), a toolkit for multi-task learning and fine-tuning of bert-like models. we extend machamp to also handle multi-label sequence labeling. we experiment with beesl in single-and different multi-task setups. after sequence labeling, token-level labels are converted into the official bionlp-st standoff format for evaluation @xcite . we simply split the event arguments based on their formal definition, producing complete structures (e.g., an expression event with k theme arguments is split into k expression events, with one theme each). similarly to previous work, we focus on sentence-level events. we used biobert-base 1.1 as our bert model for experiments, since it provides state-of-the-art performance across multiple biomedical information extraction tasks @xcite . for multi-label decoding, we tune the threshold τ for each setup (yielding τ m @xmath1.5 and τ @xmath2.7). other hyper-parameter values and tuning details are provided in appendix a.2. ## results first, we evaluate the mtl and multi-label decoding strategies on the development set to determine the best setup (sections 5.1, 5.2). then, we compare beesl to the results obtained by the top performing systems on the official test set (section 5.3). finally, we gauge its speed (section 5.4). ## analysis and discussion to gain insights about beesl, we shed more light on several aspects. firstly, we analyze how much beesl gains from multi-task learning, compared to using a powerful contextualized bert encoder alone in a single-task learning setup and a formulation with two independent classifiers (section 6.1). then, we quantify the stability of the threshold τ of the multi-label decoder (section 6.2). we also aim to get deeper insight on model performance without gold entities (section 6.3), and qualitatively study the sources of prediction errors of beesl (section 6.4). ## related work biomedical event extraction has a long-standing tradition @xcite @xcite @xcite . current work has explored neural methods and uses multiple classification stages. namely, first identifying trigger mentions, and then evaluating all entity pairs @xcite . they come with the shortcomings of traditional pipeline methods. many studies use dependency parsers to obtain features or for guidance of tree-lstms @xcite . recent work in syntactic parsing has shown that reducing parsing to sequence labeling is a viable alternative for both constituent and dependency parsing @xcite @xcite , which we took as inspiration. moreover, earlier work framed biomedical event extraction as syntactic and semantic tree-or graph-parsing @xcite . in particular, @xcite do dependency parsing, followed by a second-stage parse reranker model for event extraction, and @xcite cast the problem as subgraph identification problem. joint learning for biomedical event extraction was explored in early work @xcite @xcite . contemporary to our work, a very recent study proposes oneie, a joint learning model for event extraction @xcite . it proposes a single end-to-end model for event extraction using 4 stages, paired with a beam search, obtaining good results on ace data. processing multiple heads has previously been done for relation extraction using multi-head selection @xcite , and sequence labeling has been employed for joint entity and relation classification @xcite with inter-token attention. we employ it at the token-level for multi-label sequence labeling. ## conclusion this paper proposes beesl, a new end-to-end biomedical event extraction system which is both efficient and accurate. beesl is broadly applicable to event extraction and other tasks that can be recast as sequence labeling. the system's strength comes from the joint multi-task modeling paired with multi-label decoding, which aids interdependencies between the tasks and is superior to alternative decoders based on strong contextualized bert embeddings. beesl is fast, and achieves stateof-the-art performance on the genia 2011 event extraction benchmark without the need of external tools for features and resources such as knowledge bases. our analysis shows that beesl works very well across event types. we release the code freely, to foster research on using beesl for other nlp tasks as well, e.g., enhanced dependency parsing, fine-grained named entity recognition, and semantic parsing.