MLB i N et: A Cross-Sentence Collective Event Detection Network (2021)

Abstract: We consider the problem of collectively detecting multiple events, particularly in cross-sentence settings. The key to dealing with the problem is to encode semantic information and model event inter-dependency at a document-level. In this paper, we reformulate it as a Seq2Seq task and propose a M ulti- L ayer Bi directional Net work (MLBiNet) to capture the document-level association of events and semantic information simultaneously. Specifically, a bidirectional decoder is firstly devised to model event inter-dependency within a sentence when decoding the event tag vector sequence. Secondly, an information aggregation module is employed to aggregate sentence-level semantic and event tag information. Finally, we stack multiple bidirectional decoders and feed cross-sentence information, forming a multi-layer bidirectional tagging architecture to iteratively propagate information across sentences. We show that our approach provides significant improvement in performance compared to the current state-of-the-art results.

## introduction event detection (ed) is a crucial sub-task of event extraction, which aims to identify and classify event triggers. for instance, the document shown in detecting event triggers from natural language text is a challenge task because of the following problems: a). sentence-level contextual representation and document-level information aggregation @xcite @xcite s1: what a brave young woman s2: did you hear about the injuries[injure] she sustained s3: did you hear about the firing[attack] she did s4: she was going to fight @xcite to the death[die] s5: she was captured but she was one tough cookie s6: god bless here to address those issues, previous approaches @xcite @xcite @xcite mainly focused on sentence-level event de-tection, neglecting the document-level event interdependency and semantic information. some studies @xcite tried to integrate semantic information across sentences via the attention mechanism. for the documentlevel event inter-dependency modeling, @xcite extended the features with event types to capture dependencies between different events in a document. although great progress has been made in ed task due to recent advances in deep learning, there is still no unified framework to model the document-level semantic information and event inter-dependency. we try to analyze the ace 2005 data to reunderstand the challenges encountered in ed task. firstly, we find that event detection is essentially a special seq2seq task, in which the source sequence is a given document or sentence, and the event tag sequence is target of task. seq2seq tasks can be effectively modeled via the rnn-based encoder-decoder framework, in which the encoder captures rich semantic information, while the decoder generates a sequence of target symbols with inter-dependency been captured. this separate encoder and decoder framework can correspondingly deal with the semantic aggregation and event interdependency modeling challenges in ed task. secondly, for the propagation of cross-sentence information, we find that the relevant information is mainly stored in several neighboring sentences, while little is stored in distant sentences. for example, as shown in in this paper, we propose a novel multi-layer bidirectional network (mlbinet) for ed task. a bidirectional decoder layer is firstly devised to decode the event tag vector corresponding to each token with forward and backward event interdependency been captured. then, the event-related information in the sentence is summarized through a sentence information aggregation module. finally, the multiple bidirectional tagging layers stacking mechanism is proposed to propagate crosssentence information between adjacent sentences, and capture long-range information as the increasing of layers. we conducted experimental studies on ace 2005 corpus to demonstrate its benefits in cross-sentence joint event detection. our contributions are summarized as follows: ## approach generally, event detection on ace 2005 corpus is treated as a classification problem, which is to determine whether it forms a part of an event trigger. specifically, for a given document @xmath0, . . . , s n }, where s @xmath1,1 , . . . , w i,n i } denotes the i-th sentence containing n i tokens. we are required to predict the triggered event type sequence y @xmath2,1 , . . . , y i,n i } based on contextual information of d. without ambiguity, we omit the subscript i. for a given sentence, the event tags corresponding to tokens are associated, which is important for collectively detecting multiple events @xcite . the way tokens are classified independently will miss the association. in order to capture the event inter-dependency, the sequential information of event tag should be retained. intuitively, the ed task can be regarded as event tag sequence generation problem, which is essentially a seq2seq task. specifically, the source sequence is a given document or sentence, and the event tag sequence to be generated is the target sequence. for instance, for sentence "did you hear about the injuries she sustained", the decoder model is required to generate a tag sequence [o, o, o, o, o, b injure, o, o], where "o" denotes that the corresponding token is not part of event trigger and "b injure" indicates an injure event is triggered. we introduce the rnn-based encoder-decoder framework for ed task, considering that it is an efficient solution for seq2seq tasks. and we propose a multi-layer bidirectional network called mlbinet shown in figure 1 to deal with the challenges in detecting multiple events collectively. the model framework consists of four components: the semantic encoder, the bidirectional decoder, the information aggregation module and stacking of multiple bidirectional tagging layers. we firstly introduce the encoder-decoder framework and discuss its compatibility with the ed task. ## related work event detection is a well-studied task with research effort in the last decade. the existing methods @xcite @xcite @xcite @xcite @xcite @xcite mainly focus on sentence-level event trigger extraction, neglecting the document information. or the document-level semantic and event inter-dependency information are modeled separately. for the problem of event inter-dependency modeling, some methods were proposed to jointly extract triggers within a sentence. among them, @xcite used dynamic multi-pooling cnn to preserve information of multiple events; @xcite utilized the bidirectional recurrent neural networks to extract events; @xcite introduced syntactic shortcut arcs to enhance information flow and used graph neural networks to model graph information; @xcite proposed a hierarchical tagging lstm layer and tagging attention mechanism to model the event interdependency within a sentence. considering that adjacent sentences also store some relevant event information, which would enhance the event signals of other sentences. these methods would miss the event inter-dependency information across sentences. for document-level event inter-dependency modeling, @xcite proposed to incorporate global features to capture the cross-subtask and cross-instance interactions. the deep learning methods on document-level semantic information aggregation are primarily based on multi-level attention mechanism. @xcite integrated document information by introducing a multi-level attention. @xcite used trigger and sentence supervised attention to aggregate information and enhance the sentence-level event detection. @xcite utilized the memory network to store document level contextual information and entities. some feature-based document level information aggregation methods were proposed by @xcite @xcite @xcite . @xcite proposed to aggregate the document-level information by latent topic modeling. the attention-based documentlevel information aggregation mechanisms treat all sentences in the document equally, which may introduce some noises from distant sentences. and the feature-based methods require extensive human engineering, which also greatly affects the portability of the model. ## conclusions this paper presents a novel multi-layer bidirectional network (mlbinet) to propagate documentlevel semantic and event inter-dependency information for event detection task. to the best of our knowledge, this is the first work to unify them in one model. firstly, a bidirectional decoder is proposed to explicitly model the sentence-level event inter-dependency, and event relevant information within a sentence is aggregated by an information aggregation module. then the multiple bidirectional tagging layers stacking mechanism is devised to iteratively propagate semantic and event-related information across sentence. we conducted extensive experiments on the widely-used ace 2005 corpus, the results demonstrate the effectiveness of our model, as well as all modules we proposed. in the future, we will extend the model to the event argument extraction task and other information extraction tasks, where the document-level semantic aggregation and object inter-dependency are critical. for example, the recently concerned document-level relation extraction @xcite , which requires reading multiple sentences in a document to extract entities and infer their relations by synthesizing all information of the document. for other sequence labeling tasks, such as the named entity recognition, we can also utilize the proposed architecture