Venue: NAACL 2019

# Biomedical Event Extraction Based on Knowledge-driven Tree-LSTM

 Diya Li\({}^{\dagger}\), Lifu Huang\({}^{\dagger}\), Heng Ji\({}^{\dagger}\), Jiawei Han\({}^{*}\)

\({}^{\dagger}\)Computer Science Department, Rensselaer Polytechnic Institute

{lid18,huangl7,jih}@rpi.edu

\({}^{*}\)Computer Science Department, University of Illinois at Urbana Champaign

hanj@illinois.edu

###### Abstract

Event extraction for the biomedical domain is more challenging than that in the general news domain since it requires broader acquisition of domain-specific knowledge and deeper understanding of complex contexts. To better encode contextual information and external background knowledge, we propose a novel knowledge base (KB)-driven tree-structured long short-term memory networks (Tree-LSTM) framework, incorporating two new types of features: (1) dependency structures to capture wide contexts; (2) entity properties (types and category descriptions) from external ontologies via entity linking. We evaluate our approach on the BioNLP shared task with Genia dataset and achieve a new state-of-the-art result. In addition, both quantitative and qualitative studies demonstrate the advancement of the Tree-LSTM and the external knowledge representation for biomedical event extraction.

## 1 Introduction

Biomedical information extraction is widely used to assist the biology community on knowledge acquisition and ontology construction. Biomedical events generally refer to a change of status, particularly on proteins or genes. The goal of event extraction is to identify triggers and their arguments from biomedical text, and then assign an event type to each trigger and a role to each argument. For example, in the sentence shown in Figure 1, it includes a **gene expression** and a **positive regulation** event mention, both triggered by the word _transduced_. _Tax_ is the **Theme** argument of the **gene expression** event. An event could also serve as an argument of another event, leading to a nested structure. For instance, the **gene expression** event triggered by _transduced_ is also a **Theme** argument of the **positive regulation** event as shown in Figure 1.

Earlier studies on biomedical event extraction rely on kernel classification methods like the support vector machines (SVMs) Bjorne and Salakoski (2011); Venugopal et al. (2014) using hand-crafted features, which require high engineering effort and domain-specific knowledge. Recent distributional representation based approaches Rao et al. (2017); Bjorne and Salakoski (2018) explore deep neural networks which only require distributed semantic features. However, different from event extraction in the general news domain, biomedical event extraction requires broad acquisition of domain-specific knowledge and deep understanding of complex contexts. For example, in Genia event extracton of BioNLP shared task 2011 Kim et al. (2011), about 80% of entity mentions are abbreviations of genes, proteins and diseases while more than 36% of event triggers and arguments are separated with more than 10 words.

In order to efficiently capture indicative information from broad contexts, we first adopt tree structure based long short-term memory Tree-LSTM) networks. Compared to the linear chain structured LSTM, the Tree-LSTM takes tree-structured network topology into consideration. As shown in the top frame of Figure 1, Tree-LSTM takes the dependency tree structure of each sentence as input and gradually incorporates the information from the whole subtree into each node. Dependency tree structure can connect semantically related concepts, and thus shorten the distance between a trigger and its arguments significantly. For instance, in the following sentence "... _, which_ **binds** _to the enhancer A located in the promoter of the mouse MHC class I gene_ **H-2Kb**, ...", when determining the trigger type of _binds_, we need to carefully select its contextual words, such as _H-2Kb_, which indicates the object of _binds_. However, _binds_ and _H-2Kb_ are sepa

rated with 16 words which is difficult for a chain-structured LSTM to capture their long distance dependency, while within dependency tree structure, their distance is significantly shortened to 7.

Moreover, to better capture domain-specific knowledge, we further propose to leverage the external knowledge bases (KBs) to acquire properties of all the biomedical entities. The KB properties are extremely beneficial for our model to learn patterns more explicitly. Take the entity _Tax_ in Figure 1 as an example, it's a protein often involved in the biological process of _positive regulation of transcription_ referred to Gene Ontology [2]. This function description provides crucial clues to determine the type of _transduced_ as **positive regulation**. Therefore, to capture such knowledge from external KBs, for each entity, we first learn a KB concept embedding from its properties, and then automatically incorporate the KB representation into its Tree-LSTM hidden state with a gate function.

Our contributions are twofold: First, to the best of our knowledge, it's the first time to adopt Tree-LSTM for biomedical event extraction to effectively capture the wide contexts. Second, we further incorporate external knowledge from domain-specific KBs into the Tree-LSTM, which yields state-of-the-art performance on Genia event extraction shared task.

## 2 KB-driven Tree-LSTM for Event Extraction

In this section, we present our KB-driven Tree-LSTM approach for biomedical event extraction.

Figure 1: The framework of the KB-driven Tree-LSTM model. The upper frame shows the dependency tree structure and event annotations of a sentence; the middle frame demonstrates the knowledge base information obtained from the Gene Ontology for _Tax_; the bottom frame describes the KB-driven Tree-LSTM which takes the KB concept embedding and word embedding as input.

 

We first introduce the Tree-LSTM framework, and then describe the construction of KB concept embedding for each entity. Finally we incorporate the KB concept embedding into a Tree-LSTM and apply it for event trigger and argument extraction.

### Tree-LSTM

The Tree-LSTM (Tai et al., 2015) is a variation of LSTM (Hochreiter and Schmidhuber, 1997) to a tree-structured network topology. It shows improvement in representing sentence semantic meaning compared to sequential LSTM such as Bidirectional LSTM (BiLSTM) (Graves et al., 2013). The main difference between sequential LSTM and Tree-LSTM is, at each time step, the former calculates its hidden state from the input at the current time step and the hidden state from previous step, while Tree-LSTM computes its hidden state from the input token and the hidden states of all its children nodes from the tree structure. A Tree-LSTM reduces to sequential LSTM when each node in the tree only has one child. Figure 2 (A) shows a Tree-LSTM unit. In order to obtain the hidden state \(h_{j}\) of an input token \(x_{j}\), the unit calculates all of its children hidden states (\(h_{j-1}\), \(h_{j-2}\)) through depth-first traversal.

### Constructing KB Concept Embedding

For the biomedical event extraction, we mainly explore the Gene Ontology as our external KB since it provides detailed descriptions for each gene and gene product attributes across all species. It consists of two types of information: (1) the gene ontology (GO) defines all the gene functions, relations between these gene functions, and aspects used to describe the gene functions, including molecular function, cellular component and biological process. (2) the gene product annotations (GO Anno) provide all entity related attributes, such as the full entity name, entity type, as well as the gene functions it is related to. For example, in Figure 1, given the entity _tax_, from the gene product annotations, we can get its full entity name as _tax protein_ which is a type of _proteins_ and it's related to a function about _biological process_. From the gene ontology, we can further determine the specific function that _tax_ is related to _positive regulation of transcription_ in terms of _biological process_ aspect.

In order to leverage the external KB information, we first apply QuickGO API (Binns et al., 2009) to link each entity mention to the Gene Ontology and retrieve all the KB annotations. For each entity, we carefully select two types of properties which are beneficial for event extraction task: the entity type (e.g., _protein_ for _tax_) and the gene ontology function it is related to (e.g., _positive regulation of transcription_ for _tax_). The entity type can facilitate the explicit pattern learning for argument role labeling, for example, the gene expression event pattern (Theme: Protein, Trigger: transduced) is more popular than (Theme: Tax, Trigger: transduced) in Figure 1. The gene ontology function can provide implicit clues to determine the trigger type as aforementioned in Section 1.

As shown in Figure 1, we assign a word embedding which pretrained on PubMed and PMC texts (Moen and Ananiadou, 2013) to represent each entity type. For each gene ontology function which is usually a long phrase, we use a state-of-the-art sentence embedding approach (Conneau et al., 2017) to automatically learn a vector representation. We then concatenate these two types of KB property representations as the final KB concept embedding.

Figure 2: (A): a Tree-LSTM unit. (B): a KB-driven Tree-LSTM unit. The yellow circles with \(\widetilde{\mu}\) notations denote external KB concept embeddings.

 



### Event Trigger Extraction

After obtaining the KB concept embeddings, we further incorporate them into the Tree-LSTM to leverage the domain-specific knowledge.

Given a sentence, for example the sentence shown in Figure 3, we first perform the dependency parsing with the Stanford dependency parser Chen and Manning) and obtain a dependency tree structure. For each node \(j\) in the tree structure, \(C(j)\) is the set of children nodes of node \(j\) and \(\mu_{k}\) is the KB concept embedding of node \(k\). We set \(\mu_{k}\) to 0 if node \(k\) is not a biomedical entity. \(\widetilde{\mu}_{j}\) denotes the sum of the KB concept embeddings of \(j\)'s children nodes and \(\widetilde{h}_{j}\) is the sum of the hidden states of \(j\)'s children nodes:

\[\widetilde{h}_{j}=\sum_{k\in C(j)}h_{k}\] \[\widetilde{\mu}_{j}=\sum_{k\in C(j)}\mu_{k}\]

where \(h_{k}\) is the hidden state of node \(k\).

Then we incorporate the KB concept embeddings into the input, forget, and output gates of the Tree-LSTM:

\[i_{j} =\sigma(W_{i}[x_{j},\widetilde{h}_{j},\widetilde{\mu}_{j}]+b_{i})\] \[f_{jk} =\sigma(W_{f}[x_{j},h_{k},\widetilde{\mu}_{k}]+b_{f})\] \[o_{j} =\sigma(W_{o}[x_{j},\widetilde{h}_{j},\widetilde{\mu}_{j}]+b_{o})\]

where \(i_{j}\) and \(o_{j}\) are the input gate and the output gate for node \(j\) respectively. \(f_{jk}\) is the forget gate for node \(j\) in terms of its child node \(k\). \(W_{i}\), \(W_{f}\), and \(W_{o}\) are learnable parameters, \(b_{i}\), \(b_{f}\) and \(b_{o}\) are bias terms. Thus, for each node \(j\), the input gate gathers all KB information from its children nodes, and the output gate balances the meaningful information from its local contexts and the KB concept embeddings of its children nodes.

Besides adding the KB concept embeddings into the three gates to select useful KB formation implicitly, similar to Ma et al. (2018), we also introduce a knowledge specific output gate \(g_{j}\) to explicitly incorporate knowledge information into each node's hidden state. While different from Ma et al. (2018) which only considers the knowledge concept embedding of each node itself, we use the sum of the KB concept embeddings of the whole subtree instead:

\[g_{j}=\sigma(W_{g}[x_{j},\widetilde{h}_{j},\widetilde{\mu}_{j}]+b_{g})\]

where \(W_{g}\) is a weight matrix to be learned, \(b_{g}\) is the bias term.

As demonstrated in Figure 2 (B), we eventually combine the implicit way of incorporating KB information into the input, output and forget gates and an explicit way of directly incorporating the KB information into a node's hidden state:

\[\widetilde{c}_{j} =\tanh(W_{c}[x_{j},\widetilde{h}_{j}]+b_{c})\] \[c_{j} =\sum_{k\in C(j)}f_{jk}\odot c_{k}+i_{j}\odot\widetilde{c}_{j}\] \[h_{j} =o_{j}\odot\tanh(c_{j})+g_{j}\odot\tanh(W_{\mu}\widetilde{\mu}_{ j})\]

where \(c_{j}\) is the memory cell, \(W_{c}\) and \(W_{\mu}\) are weight matrices to be learned.

After getting the hidden state \(h_{j}\) of each node \(j\), we use a softmax classifier to predict a label for each node, and optimize the parameters by minimizing a negative log-likelihood loss.

### Event Argument Role Labeling

After detecting all candidate triggers, we further extract arguments for each trigger. The Genia event extraction shared task provides the annotations of all entity mentions. Thus, for each trigger, we use all the entity mentions that occur in the same sentence as its candidate arguments, and then assign an argument role or _None_. Different from trigger extraction, we use the shortest dependency path (SDP) within the dependency tree structure instead of the surface contexts to better capture the dependency between the trigger and each argument.

Taking the sentence in Figure 3 as an example, given a trigger _transcription_ and a candidate argument _OBF-1_, we first perform dependency parsing and extract the shortest dependency path between _transcription_ and _OBF-1_ with the Dijkstra's algorithm Johnson (1973) and obtain the shortest dependency path _transcription \(\rightarrow\textit{of}\rightarrow\textit{genes}\to OBF-1\)_. We use the same KB-driven Tree-LSTM architecture as introduced in Section 2.3 to encode each node into a new hidden state representation. We use the hidden state of the root node \(h_{0}\) as the overall vector representation of the whole dependency path. Finally, we feed the concatenation of \(h_{0}\) with the hidden state of the trigger and argument as input to another softmax to predict the argument role. We also optimize the model by minimizing a negative log-likelihood loss



## 3 Experiment

### Task Description

The Genia Event Extraction task is the main task in the BioNLP Shared Task series Kim et al. (2009, 2011); Nedellec et al. (2013). The Genia task defines 9 fine-grained event types as shown in Table 1. Note that a **Binding** event may take more than one protein as its **Theme** arguments. A **Regulation** event may take one protein or event as its **Theme** argument and also optionally take one protein or event as its **Cause** argument. A **Regulation** event taking an event as its argument will lead to a nested structure. 37.2% nested events are observed in Genia 2011 corpus Bjorne and Salakoski (2011). There are 6.0% inter-sentence events while our model only focuses on sentence-level event extraction.

### Experimental Setup

We apply our KB-driven Tree-LSTM model on Genia 2011 data set. The entities in Genia data set are manually annotated and given as part of the input.

We evaluate our results on the test set using the official online tool provided by the Genia task organizers.1 Following previous studies Bjorne and Salakoski (2011); Venugopal et al. (2014); Rao et al. (2017); Bjorne and Salakoski (2018), we report scores obtained by the approximate span (allowing trigger spans to differ from gold spans by single words). As we only focus on matching core arguments, we use recursive matching criterion for evaluation which not requires matching of additional arguments for events referred from other events Kim et al. (2011).

Footnote 1: [http://bionlp-st.dbcls.jp/GE/2011/eval-test/](http://bionlp-st.dbcls.jp/GE/2011/eval-test/)

We use the word embedding pretrained on PubMed and PMC texts Moen and Ananiadou (2011).

\begin{table}
\begin{tabular}{l|p{56.9pt}} \hline
**Event Type** & **Core Arguments** \\ \hline Gene expression & Theme(P) \\ Transcription & Theme(P) \\ Protein catabolism & Theme(P) \\ Phosphorylation & Theme(P) \\ Localization & Theme(P) \\ \hline Binding & Theme(P)+ \\ \hline Regulation & Theme(P/E), Cause(P/E) \\ Positive regulation & Theme(P/E), Cause(P/E) \\ Negative regulation & Theme(P/E), Cause(P/E) \\ \hline \hline \multicolumn{3}{c}{**Genia corpus 2011 statistics**} \\ \hline events & 14496 \\ sentences & 11581 \\ nested events & 37.2\% \\ intersentence events & 6.0\% \\ abbrev. of entities & 15912 \\ \hline \end{tabular}
\end{table}
Table 1: Predefined event types with accepted argument roles in Genia event extraction task, and data statistics of Genia event extraction 2011 dataset. P: protein; E: event.

Figure 3: Examples of trigger labeling and argument role labeling via a KB-driven Tree-LSTM.



2013) for word and type embeddings. The hyper-parameters are tuned on the development set and listed in Table 2. Word representations are updated during training with an initial learning rate of 0.1.

### Results and Error Analysis

Table 3 shows the final event extraction results of applying our KB-driven Tree-LSTM model on Genna 2011 dataset with the comparison of only using Tree-LSTM and a standard BiLSTM model. Tree-LSTM outperforms the BiLSTM baseline which indicates the power of Tree-LSTM in dealing with long-distance dependency structure in biomedical literature. By incorporating external KB information, our approach achieves about 2.1% F-score gain comparing to Tree-LSTM, which demonstrates the effectiveness of the KB properties for biomedical event extraction. We will show detailed analysis in Section 3.4.

Table 4 presents the previous event extraction results from the BioNLP shared task using the same corpus. Our approach outperforms all previous methods. Among them, the systems TEES Bjorne and Salakoski (2011), EventMine-CR Miwa et al. (2012) and Stacked Generalization Majumder et al. (2016) are based on SVMs with well designed features. FAUST Riedel and McCallum (2011) and BioMLN Venugopal et al. (2014) use jointed inference models. Bjorne and Salakoski (2018) adopts a convolutional neural networks (CNNs) with abundant features derived from TEES system. In our work, instead of using high-dimensional features with manual effort as in these previous models, our approach only requires pretrained distributed word representations as input features.

We notice that our approach achieves high scores on **Simple** event types but get relatively low scores on **Binding** event and **Regulation** event types. We analyze the results and find that **Binding** event extraction is more challenging since it usually has multiple arguments. For example, Figure 4 shows two sentences which are chosen from the output of the development data set. There are two **Binding** event mentions in the first sentence: E1 (Trigger: interacting, Type: Binding, Themee: RUNX1, Theme2: p3000) and E2 (Trigger: binding, Type: Binding, Themee: CREB). Our model mistakenly extracts _CREB_ as a _Theme_ of E1 since _CREB_ is highly related to protein p300 in the dependency tree structure.

**Regulation** events are considered as the most challenging event type because they usually have an optional **Cause** argument and are involved in nested structures, which are not handled well by most of current event extraction approaches. In addition, intuitively, most trigger words are verbs or nouns. We rank all t

\begin{table}
\begin{tabular}{l|c|c|c} \hline \hline
**System** & **Rec** & **Prec** & **F1** \\ \hline TEES Björne and Salakoski (2011) & 49.56 & 57.65 & 53.30 \\ FAUSTRiedel and McCallum (2011) & 49.41 & 64.75 & 56.04 \\ EventMine-CRMiwa et al. (2012) & 53.35 & 63.48 & 57.98 \\ BioMLNVenugopal et al. (2014) & 53.42 & 63.61 & **58.07** \\ Stacked GeneralizationMajumder et al. (2016) & 48.96 & 66.46 & 56.38 \\ CNNBiörne and Salakoski (2018) & 49.94 & 69.45 & **58.07** \\ \hline \hline  & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} \\ \end{tabular}
\end{table}
Table 4: State-of-the-art system results evaluated on BioNLP Genia 2011 test dataset with approximate span and recursive criteria.

\begin{table}
\begin{tabular}{l|c|c} \hline \hline
**Parameter** & **Value** \\ \hline Word embedding size & 200 \\ Type embedding size & 200 \\ Sentence embedding size & 4096 \\ Tree-LSTM hidden size & 100 \\ Batch size & 25 \\ Epoch size & 30 \\ Dropout rate & 0.5 \\ Learning rate & 0.05 \\ Initial embedding learning rate & 0.1 \\ Optimizer & AdaGrad \\ \hline \hline \end{tabular}
\end{table}
Table 2: Hyper-parameters.

\begin{table}
\begin{tabular}{l|l|c|c|c} \hline \hline
**System** & **Event Type** & **Rec** & **Prec** & **F1** \\ \hline \multirow{8}{*}{KB-driven Tree-LSTM} & Gene expression & 74.35 & 87.24 & 80.28 \\  & Transcription & 69.54 & 82.31 & 75.39 \\  & Protein catabolism & 46.67 & 87.50 & 60.87 \\  & Phosphorylation & 81.62 & 87.28 & 84.36 \\  & Localization & 59.69 & 80.28 & 68.47 \\  & **Simple total** & 72.62 & **85.95** & 78.73 \\ \cline{2-5}  & Binding & 37.68 & 53.16 & 44.10 \\ \cline{2-5}  & Regulation & 36.62 & 53.61 & 43.52 \\  & Positive regulation & 41.37 & 57.90 & 48.26 \\  & Negative regulation & 46.06 & 52.39 & 49.02 \\  & **Regulation total** & 41.73 & 55.73 & 47.72 \\ \cline{2-5}  & **Event total** & **52.14** & **67.01** & **58.65** \\ \hline \hline \multirow{3}{*}{Tree-LSTM} & **Simple total** & 71.22 & 83.41 & 76.83 \\  & Binding & 34.83 & 48.72 & 40.62 \\  & **Regulation total** & 39.78 & 53.54 & 45.64 \\  & **Event total** & **50.28** & **64.56** & **56.53** \\ \hline \hline \multirow{3}{*}{BiLSTM} & **Simple total** & 68.09 & 78.75 & 73.03 \\  & Binding & 38.49 & 43.05 & 40.65 \\ \cline{1-1}  & **Regulation total** & 37.64 & 53.81 & 44.30 \\ \cline{1-1}  & **Event total** & **48.44** & **62.18** & **54.46** \\ \hline \hline \end{tabular}
\end{table}
Table 3: Precision (Prec), recall (Rec) and F-score (F1) results achieved by the KB-driven Tree-LSTM model on the test set of BioNLP Genia 2011, evaluated on approximate span and recursive criteria.

 

ing data set according to their frequency, and find that most of spurious errors for **Regulation** event trigger extraction occur when the trigger words are prepositions or conjunctions. For instance, in Figure 4, the second sentence contains two positive **Regulation** events triggered by a preposition _from_ and a conjunction _rather than_. Such function words are rarely annotated as triggers and our KB-aware Tree-LSTM cannot well collect meaningful contexts from their subtrees.

### Effect of KB concepts

As shown in Table 3, we achieve about 3.5% and 2.1% F1 score gain on **Binding** and **Regulation** event types by leveraging external KB information into the Tree-LSTM. In order to show the effect of KB concept embeddings, we visualize the probabilities of word _transcription_ to be predicted for each event type. As Figure 5 shows, by adding KB concept embeddings, the function description _positive regulation of transcription, DNA-templated_ provided by the biomedical entity _OBF-1_ significantly enhances the probability of _transcription_ being predicted to a _Transcription_ event type.

Similarly, Figure 6 visualizes the probabilities of the E1 event mention (Trigger: transduced, Type: gene_expression, Theme: Tax) to be predicted as an argument of E2 event mention (Trigger: transduced, Type: positive regulation, Theme: E1). We can see that, without using KB information, the Tree-LSTM mistakenly predict the argument role of E1 as **None**. In contrast, by incorporating KB concept embeddings, especially

Figure 4: Case study on binding event and regulation event types.

Figure 5: Visualization of the effect of KB concept embeddings on trigger labeling for the word _transcription_.

Figure 6: Visualization of the effect of KB concept embedding on argument role labeling for a Positive Regulation event triggered by _transduced_ and a Gene Expression event E1 (Theme: Tax, Trigger: transduced).

 

the information from the function description _positive regulation of transcription, DNA-templated_ for _Tax_, our approach successfully promotes the probability of E1 being predicted as the _Theme_ of E2.

## 4 Related Work

As a crucial task in information extraction, event extraction has gained a lot of interest. In general news domain, previous work on event extraction can be divided into two main categories. The first is feature-based methods which mainly focus on feature design, leveraging local features Grishman et al. (2005); Ahn (2006) and global features Ji and Grishman (2008); Liao and Grishman (2011); Huang and Riloff (2012) to improve the performance. Some studies proposed joint models to overcome the error propagation problem Poon and Vanderwende (2010); Riedel et al. (2009); Li et al. (2013); Venugopal et al. (2014); Li et al. (2014). The second category includes distributional representation based methods which have been applied into event extraction extensively. Most of these approaches are based on the standard Convolutional Neural Networks (CNNs) Chen et al. (2015); Nguyen and Grishman (2015, 2016), Recurrent Neural Networks (RNNs) Nguyen et al. (2016), generative adversarial networks Hong et al. (2018), zero-shot learning Huang et al. (2017) and advanced attention mechanisms Liu et al. (2018b); Chen et al. (2018).

Our work is also related to the studies which leverage the external knowledge base for information extraction. Liu et al. (2017) takes advantage of external resources, such as FrameNet, to label events while Chen et al. (2017) adopts distance supervision to augment the training data. Liu et al. (2018a) develops an attention-based model for event extraction. What's more, shortest dependency path is broadly explored for information extraction, especially for relation classification Xu et al. (2015); Miwa and Bansal (2016) and shows promising benefits.

Biomedical event extraction task part of the BioNLP Shared Task series Kim et al. (2009, 2011); Nedellec et al. (2013). Previous studies mainly explore local and global features with SVM model Miwa et al. (2010, 2012); Bjome and Salakoski (2013); Majumder et al. (2016). Riedel and McCallum (2011) develop a joint model with dual decomposition. Cohen et al. (2009), Kilicoglu and Bergler (2011) and Bui et al. (2013) develop rule-based methods and achieve high precision. Venugopal et al. (2014) leverage Markov logic networks for joint inference. Rao et al. (2017) uses the Abstract Meaning Representations (AMR) to extract events based on the assumption that an event structure can be derived from an AMR subgraph. Recently, some representation-based models Jagannatha and Yu (2016); Rao et al. (2017); Bjorne and Salakoski (2018) have been proposed while most of them adopt the widely used CNNs and RNNs with features derived from the biomedical text. Lim et al. (2018) implements a binary Tree-LSTM architecture for biomedical relation extraction. Compared with these methods, our approach only requires pre-trained distributed word representations as input features and incorporates meaningful KB information into a Tree-LSTM.

## 5 Conclusions and Future Work

In this paper, we show the effectiveness of using a KB-driven tree-structured LSTM for event extraction in biomedical domain. The Tree-LSTM can efficiently capture semantically related concepts for each node within the tree structure. By leveraging the external KB concept properties including the entity type and the function description, our approach is able to perform deep understanding of domain-specific expressions and connections. Without using manually designed high-dimensional features, our approach significantly outperforms all previous methods. In the future, we plan to explore a broader range of properties from KB to facilitate biomedical information extraction tasks.

## Acknowledgments

This work was supported by the U.S. NSF No. 1741634, Air Force No. FA8650-17-C-7715 and ARL NS-CTA No. W911NF-09-2-0053. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation here on 



## References

* Ahn (2006) David Ahn. 2006. The stages of event extraction. In _Proceedings of the Workshop on Annotating and Reasoning about Time and Events_.
* Ashburner et al. (2000) Michael Ashburner, Catherine A Ball, Judith A Blake, David Botstein, Heather Butler, J Michael Cherry, Allan P Davis, Kara Dolinski, Selina S Dwight, Janan T Eppig, et al. 2000. Gene ontology: tool for the unification of biology. _Nature genetics_, 25(1):25.
* Binns et al. (2009) David Binns, Emily Dimmer, Rachael Huntley, Daniel Barrell, Claire O'donovan, and Rolf Apweiler. 2009. Quickgo: a web-based tool for gene ontology searching. _Bioinformatics_, 25(22):3045-3046.
* Bjorne and Salakoski (2011) Jari Bjorne and Tapio Salakoski. 2011. Generalizing biomedical event extraction. In _Proceedings of the BioNLP Shared Task 2011 Workshop_.
* Bjorne and Salakoski (2013) Jari Bjorne and Tapio Salakoski. 2013. Tees 2.1: Automated annotation scheme learning in the bionlp 2013 shared task. In _Proceedings of the BioNLP Shared Task 2013 Workshop_.
* Bjorne and Salakoski (2018) Jari Bjorne and Tapio Salakoski. 2018. Biomedical event extraction using convolutional neural networks and dependency parsing. In _Proceedings of the BioNLP 2018 workshop_.
* Bui et al. (2013) Quoc-Chinh Bui, David Campos, Erik van Mulligen, and Jan Kors. 2013. A fast rule-based approach for biomedical event extraction. In _proceedings of the BioNLP shared task 2013 workshop_.
* Chen and Manning (2014) Danqi Chen and Christopher Manning. A fast and accurate dependency parser using neural networks. In _Proc. EMNLP2014_.
* Chen et al. (2017) Yubo Chen, Shulin Liu, Xiang Zhang, Kang Liu, and Jun Zhao. 2017. Automatically labeled data generation for large scale event extraction. In _Proc. ACL2017_.
* Chen et al. (2015) Yubo Chen, Liheng Xu, Kang Liu, Daojian Zeng, and Jun Zhao. 2015. Event extraction via dynamic multi-pooling convolutional neural networks. In _Proc. ACL-IJCNLP2015_.
* Chen et al. (2018) Yubo Chen, Hang Yang, Kang Liu, Jun Zhao, and Yantao Jia. 2018. Collective event detection via a hierarchical and bias tagging networks with gated multilevel attention mechanisms. In _Proc. EMNLP2018_.
* Cohen et al. (2009) K Bretonnel Cohen, Karin Verspoor, Helen L Johnson, Chris Roeder, Philip V Ogren, William A Baumgartner Jr, Elizabeth White, Hannah Tipney, and Lawrence Hunter. 2009. High-precision biological event extraction with a concept recognizer. In _Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing: Shared Task_.
* Conneau et al. (2017) Alexis Conneau, Douwei Kiela, Holger Schwenk, Loic Barrault, and Antoine Bordes. 2017. Supervised learning of universal sentence representations from natural language inference data. In _Proc. EMNLP2017_.
* Graves et al. (2013) Alex Graves, Navdeep Jaitly, and Abdel-rahman Mohamed. 2013. Hybrid speech recognition with deep bidirectional lstm. In _2013 IEEE workshop on automatic speech recognition and understanding_.
* Grishman et al. (2005) Ralph Grishman, David Westbrook, and Adam Meyers. 2005. NYU's English ACE 2005 system description. _ACE_, 5.
* Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jurgen Schmidhuber. 1997. Long short-term memory. _Neural computation_, pages 1735-1780.
* Hong et al. (2018) Yu Hong, Wenxuan Zhou, jingli zhang Jingli, Guodong Zhou, and Qiaoming Zhu. 2018. Self-regulation: Employing a generative adversarial network to improve event detection. In _Proc. ACL2018_.
* Huang et al. (2017) Lifu Huang, Heng Ji, Kyunghyun Cho, and Clare R Voss. 2017. Zero-shot transfer learning for event extraction. _arXiv preprint arXiv:1707.01066_.
* Huang and Riloff (2012) Ruihong Huang and Ellen Riloff. 2012. Bootstrapped training of event extraction classifiers. In _Proc. EACL2012_.
* Jagannatha and Yu (2016) Abhyuday N Jagannatha and Hong Yu. 2016. Bidirectional rnn for medical event detection in electronic health records. In _Proc. NAACL-HLT2016_.
* Ji and Grishman (2008) Heng Ji and Ralph Grishman. 2008. Refining event extraction through cross-document inference. _Proceedings of ACL-08: HLT_.
* Johnson (1973) Donald B Johnson. 1973. A note on dijkstra's shortest path algorithm. _Journal of the ACM (JACM)_, 20(3):385-388.
* Kilicoglu and Bergler (2011) Halili Kilicoglu and Sabine Bergler. 2011. Adapting a general semantic interpretation approach to biological event extraction. In _Proceedings of the BioNLP Shared Task 2011 Workshop_.
* Kim et al. (2009) Jin-Dong Kim, Tomoko Ohta, Sampo Pyysalo, Yoshinobu Kano, and Jun'ichi Tsuji. 2009. Overview of bionlp'09 shared task on event extraction. In _Proceedings of the BioNLP Shared Task 2009 Workshop_.
* Kim et al. (2011) Jin-Dong Kim, Yue Wang, Toshihiisa Takagi, and Akimori Yonezawa. 2011. Overview of genia event task in bionlp shared task 2011. In _Proceedings of the BioNLP Shared Task 2011 Workshop_.
* Li et al. (2013) Qi Li, Heng Ji, and Liang Huang. 2013. Joint event extraction via structured prediction with global features. In _Proc. ACL2013_.
* Li et al. (2014) Qi Li, Heng Ji, Hong Yu, and Sujian Li. 2014. Constructing information networks using one single model. In _Proc. EMNLP2014_

* Liao and Grishman (2011) Shasha Liao and Ralph Grishman. 2011. Acquiring topic features to improve event extraction: in pre-selected and balanced collections. In _Proc. RANLP2011_.
* Lim et al. (2018) Sangrak Lim, Kyubum Lee, and Jaewoo Kang. 2018. Drug drug interaction extraction from the literature using a recursive neural network. _PloS one_, 13(1):e0190926.
* Liu et al. (2018a) Jian Liu, Yubo Chen, Kang Liu, and Jun Zhao. 2018a. Event detection via gated multilingual attention mechanism. _Statistics_, 1000:1250.
* Liu et al. (2017) Shulin Liu, Yuko Chen, Kang Liu, and Jun Zhao. 2017. Exploiting argument information to improve event detection via supervised attention mechanisms. In _Proc. ACL2017_.
* Liu et al. (2018b) Xiao Liu, Zhunchen Luo, and Heyan Huang. 2018b. Jointly multiple events extraction via attention-based graph information aggregation. In _Proc. EMNLP2018_.
* Ma et al. (2018) Yukun Ma, Haiyun Peng, and Erik Cambria. 2018. Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive lstm. In _Proc. AAAI2018_.
* Majumder et al. (2016) Amit Majumder, Asif Ekbal, and Sudip Kumar Naskar. 2016. Biomolecular event extraction using a stacked generalization based classifier. In _Proc. ICON2016_.
* Miwa and Bansal (2016) Makoto Miwa and Mohit Bansal. 2016. End-to-end relation extraction using lstms on sequences and tree structures. _arXiv preprint arXiv:1601.00770_.
* Miwa et al. (2010) Makoto Miwa, Rune Ssetre, Jin-Dong Kim, and Jun'ichi Tsujii. 2010. Event extraction with complex event classification using rich features. _Journal of bioinformatics and computational biology_, 8(01):131-146.
* Miwa et al. (2012) Makoto Miwa, Paul Thompson, and Sophia Ananiadou. 2012. Boosting automatic event extraction from the literature using domain adaptation and coreference resolution. _Bioinformatics_, 28(13):1759-1765.
* Moen and Sanakoski2 Sophia Anandatou (2013) SPFGH Moen and Tapio Salakoski2 Sophia Anandatou. 2013. Distributional semantics resources for biomedical text processing. In _Proc. LBM2013_.
* Nedellec et al. (2013) Claire Nedellec, Robert Bossy, Jin-Dong Kim, Jung-Jae Kim, Tomoko Ohta, Sampo Pyysalo, and Pierre Zweigenbaum. 2013. Overview of bionlp shared task 2013. In _Proceedings of the BioNLP Shared Task 2013 Workshop_.
* Nguyen et al. (2016) Thien Huu Nguyen, Kyunghyun Cho, and Ralph Grishman. 2016. Joint event extraction via recurrent neural networks. In _Proc. NAACL-HLT2016_.
* Nguyen and Grishman (2015) Thien Huu Nguyen and Ralph Grishman. 2015. Event detection and domain adaptation with convolutional neural networks. In _Proc. ACL-IJCNLP2015_.
* Nguyen and Grishman (2016) Thien Huu Nguyen and Ralph Grishman. 2016. Modeling skip-grams for event detection with convolutional neural networks. In _Proc. EMNLP2016_.
* Poon and Vanderwende (2010) Hoifung Poon and Lucv Vanderwende. 2010. Joint inference for knowledge extraction from biomedical literature. In _Proc. NAACL-HLT2010_.
* Rao et al. (2017) Sudha Rao, Daniel Marcu, Kevin Knight, and Hal Daume III. 2017. Biomedical event extraction using abstract meaning representation. In _Proceedings of the BioNLP 2017 workshop_.
* Riedel et al. (2009) Sebastian Riedel, Hong-Woo Chun, Toshihisa Takagi, and Jun'ich Tsujii. 2009. A markov logic approach to bio-molecular event extraction. In _Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing: Shared Task_.
* Riedel and McCallum (2011) Sebastian Riedel and Andrew McCallum. 2011. Robust biomedical event extraction with dual decomposition and minimal domain adaptation. In _Proceedings of the BioNLP Shared Task 2011 Workshop_.
* Tai et al. (2015) Kai Sheng Tai, Richard Socher, and Christopher D Manning. 2015. Improved semantic representations from tree-structured long short-term memory networks. _arXiv preprint arXiv:1503.00075_.
* Venugopal et al. (2014) Deepak Venugopal, Chen Chen, Vibhav Gogate, and Vincent Ng. 2014. Relieving the computational bottleneck: Joint inference for event extraction with high-dimensional features. In _Proc. EMNLP2014_.
* Xu et al. (2015) Yan Xu, Lili Mou, Ge Li, Yunchuan Chen, Hao Peng, and Zhi Jin. 2015. Classifying relations via long short term memory networks along shortest dependency paths. In _Proc. EMNLP2015_.