Resource-Enhanced Neural Model for Event Argument Extraction (2020)

Abstract: Event argument extraction (EAE) aims to identify the arguments of an event and classify the roles that those arguments play. Despite great efforts made in prior work, there remain many challenges: (1) Data scarcity. (2) Capturing the long-range dependency, specifically, the connection between an event trigger and a distant event argument. (3) Integrating event trigger information into candidate argument representation. For (1), we explore using unlabeled data. For (2), we use Transformer that uses dependency parses to guide the attention mechanism. For (3), we propose a trigger-aware sequence encoder with several types of trigger-dependent sequence representations. We also support argument extraction either from text annotated with gold entities or from plain text. Experiments on the English ACE 2005 benchmark show that our approach achieves a new state-of-the-art.

## introduction event argument extraction (eae) aims to identify the entities that serve as arguments of an event and to classify the specific roles they play. as in fig. 1 , "two soldiers" and "yesterday" are arguments, where the event triggers are "attacked" (with event type being attack 1 ) and "injured" (event type injury). for the trigger "attacked", "two soldiers" plays the argument role target while "yesterday" plays the argument role attack time. for the event trigger "injured", "two soldiers" and "yesterday" play the role victim and injury time, respectively. there has been significant work on event extraction (ee) @xcite @xcite , but the eae task remains a challenge and has become the bottleneck for improving the overall performance of ee @xcite . 2 supervised data for eae is expensive and hence scarce. one possible solution is to use other available resources like unlabeled data. for that, (1) we use bert @xcite as our model encoder which leverages a much larger unannotated corpus where semantic information is captured. unlike @xcite who added a final/prediction layer to bert for argument extraction, we use bert as token embedder and build a sequence of eae task-specific components (sec. 2). (2) we use (unlabeled) in-domain data to adapt the bert model parameters in a subsequent pretraining step as in @xcite . this makes the encoder domain-aware. (3) we perform self-training to construct auto-labeled data (silver data). a crucial aspect for eae is to integrate event trigger information into the learned representations. this is important because arguments are dependent on triggers, i.e., the same argument span plays completely different roles toward different triggers. an example is shown in fig. 1 , where "two soldiers" plays the role target for the event attack and the role victim for injury. different from 2 eae has similarities with semantic role labeling. event triggers are comparable to predicates in srl and the roles in most srl datasets have a standard convention of interpreting who did what to whom. eae has a custom taxonomy of roles by domain. we also use inspiration from the srl body of work @xcite @xcite . capturing the long-range dependency is another important factor, e.g., the connection between an event trigger and a distant argument. syntactic information could be useful in this case, as it could help bridge the gap from a word to another distant but highly related word @xcite @xcite . we modify a transformer @xcite by explicitly incorporating syntax via an attention layer driven by the dependency parse of the sequence. we design our role-specific argument decoder to seamlessly accommodate both settings (with and without the availability of entities). we also tackle the role overlap problem @xcite using a set of classifiers or taggers in our decoder. our model achieves the new state-of-the-art on ace2005 events data @xcite . an event trigger g is a span x ab indicating an event of type y g , where y g belongs to a fixed set of pre-defined trigger types. given a sequencetrigger pair (x , g) as input, eae has two goals: (1) identify all argument spans from x and (2) classify the role r for each argument. in some settings, a set of entities is given (each entity is a span in x) and such entities are used as a candidate pool for arguments. for example, "two soldiers" and "yesterday" are candidate entities in fig. 1 . trigger-aware sequence encoder: this encoder is designed to distinguish candidate arguments conditioned on different triggers. note a span may encode different argument information for two triggers, for example, in fig. 1 , "two soldiers" plays the role of target for the attack event and victim for the injury event. in order to model this, our encoder uses bert to embed input tokens, where the bert embedding of token x t is denoted as b t . a segment (0/1) embedding seg t for each token x t indicating whether x t belongs to the trigger or not @xcite is used, which is added up with token embedding and position embedding as input to bert (fig. 2 ). the encoder then concatenates the following learned representations foot_0 for each token: (1) a trigger representation h g by max pooling over bert embeddings of the tokens in trigger g; (2) a trigger type embedding p yg for y g ; (3) a trigger indicator (0/1) embedding l t , indicating whether x t belongs to the trigger or not. foot_1 (4) a token embedding b t . this results in a trigger-aware representation c t for each token where c @xmath0; p yg ; l t ; h g ) and c for the whole sequence with t tokens. ## experiments data and tools: we use the ace-2005 english event data @xcite . 9 for selftraining and domain-adaptive pretraining, we randomly sample 50k documents (626k sentences) from gigaword foot_7 to construct silver data. we use stanford corenlp software foot_8 for tokenization, sentence segmentation and dependency parsing. training setup: we use 50 dimensions for trigger indicator and trigger type embedding. we use 2 (@xmath1, dropout of 0.1. when entities are available, we only consider entities in the same sentence as the trigger as candidates for argument extraction. during training, we use adam (kingma and ba, 2014) as optimizer and batch size of 32 for both main task eae and the auxiliary task of trigger detection; we alternate between batches of main task and auxiliary task with probabilities of 0.9 and 0.1, respectively. we early stop training if performance on the development set does not improve after 20 epochs. all model parameters are fine-tuned during training. for bert pretraining, we use the same setting as in @xcite but with an initial learning rate of 1e-5. we stop pretraining after 10k steps. in order to obtain reliably predicted triggers as input for eae, we trained a five-model ensemble trigger detection system following wadden et al. (2019). 12 model argument identification (ai) role classification (rc) p r f1 p r f1 @xcite 68.8 51.9 59.1 62.2 46.9 53.5 @xcite 61.4 64.2 62.8 54.2 56.7 55.4 @xcite 63.2 59.4 61.2 54.1 53.5 53.8 @xcite 71.3 64.5 67.7 66.2 52.8 58.7 @xcite 71.4 60.1 65.3 62.3 54.2 58.0 @xcite ---62.2 56.6 59.3 @xcite 71.4 65.6 68.4 66.8 54.9 60. results and analyses: ## conclusion we present a new model which provides the best results in the eae task. the model can generate trigger-aware argument representations, incorporate syntactic information (via dependency parses), and handle the role overlapping problem with rolespecific argument decoder. we also experiment with some methods to address the data scarcity issue. experimental results show the effectiveness of our proposed approaches.