A Two-Step Approach for Implicit Event Argument Detection (2020)

Abstract: In this work, we explore the implicit event argument detection task, which studies event arguments beyond sentence boundaries. The addition of cross-sentence argument candidates imposes great challenges for modeling. To reduce the number of candidates, we adopt a two-step approach, decomposing the problem into two sub-problems: argument head-word detection and head-to-span expansion. Evaluated on the recent RAMS dataset (Ebner et al., 2020), our model achieves overall better performance than a strong sequence labeling baseline. We further provide detailed error analysis, presenting where the model mainly makes errors and indicating directions for future improvements. It remains a challenge to detect implicit arguments, calling for more future work of document-level modeling for this task.

## introduction event argument detection is a key component in the task of event extraction. it resembles semantic role labeling (srl) in that the main target is to find argument spans to fill the roles of event frames. however, event arguments can go beyond sentence boundaries: there can be non-local or implicit arguments at the document level. figure 1 shows such an example: for the purchase event, which is triggered by the word "bought", its money argument appears in the previous sentence. implicit arguments have been under-explored in event extraction. most of previous systems @xcite @xcite only consider local arguments in the same sentence of the event trigger. while incorporating implicit arguments requires corresponding annotations, few exists in most of the widely used event datasets, like ace2005 @xcite and @xcite . there are several annotation efforts for implicit arguments the bold text indicates the trigger word for the purchase event, while the underlined text indicates its non-local "money" argument in the previous sentence. our model first detects the head-word "dollars", and then expands it to the whole span. in srl, including g&c @xcite , semeval-2010 @xcite , and 80days (feizabadi and padó, 2014). yet most are performed with different ontologies such as nombank (g&c) and framenet (semeval-2010 and 80days); on different domains (e.g. novels); and in smaller scales (g&c and 80days only cover 10 types of predicates). the lack of annotations poses challenges to train and transfer implicit argument models for event extraction. recently, @xcite create the roles across multiple sentences (rams) dataset, which covers multi-sentence implicit arguments for a wide range of event and role types. they further develop a span-based argument linking model and achieve relatively high scores. however, they mainly explore a simplified setting that assumes the availability of gold argument spans. we extend their work and explore the more challenging full detection problem that predicts argument spans among all possible candidates. the difficulty of the full problem is highlighted in figure 1 . both "3000 dollars" and "1000 dollars" are good candidates for the money role of the purchase event, but the selections are different given different contexts. when considering all possible candidate spans that may occur in any sentences, their quadratic number poses great challenges for the detection. inspired by dependency-based srl @xcite , we take the syntactical head-words as the proxy for full argument spans, hypothesizing that the head-words can contain enough information to fill the argument roles. based on this, we adopt a two-step approach: first detecting the head-words of the arguments, and adopting a second step of head-to-span expansion. actually, this type of two-step setup is not uncommon in prior work of information extraction, including entity detection @xcite , coreference resolution @xcite and document-level pseudo-coreference @xcite . by considering only individual tokens in the detection step, the system only needs to handle a candidate space whose size scales linearly in respective to the number of tokens instead of quadratically. with the same setting of fine-tuning bert (devlin et al., 2019) encoder, we show the effectiveness of our model by obtaining overall better results than a strong sequence-labeling model. we further provide detailed error analysis, showing that the main difficulties of the task are upon non-local and non-core arguments. our analysis shows that the implicit argument task is quite challenging, calling for more future work on document-level semantic understanding for this task. ## model the goal of event argument detection is to create labeled links between argument spans and the predicate (event trigger). recent state-of-the-art solutions for sentence-level srl perform the detection in an end-to-end setting, such as span-based @xcite , and sequence labeling models @xcite . however, span-based models face great challenges when considering arguments across sentence boundaries, since the computational complex-ity of such models grows quadratically to deal with o(n 2 ) span candidates given n tokens. while traditional sequence labeling models can run in linear-time, they are less flexible and extensible in complex scenarios like overlapping mentions and multiple roles for one mention. in this work, we take a two-step approach that decomposes the problem explicitly into two sub-problems, based on the hypothesis that head-words can usually capture the information of the mention spans. figure 1 illustrates the three main modules of our model: 1) bert-based encoder, 2) argument head-word detector, and 3) head-to-span expander. ## experiment we conduct all experiments 2 on the rams (v1.0) dataset and focus on the event argument detection task: given (gold) event triggers and their multisentence contexts, predicting the argument spans from raw input tokens. following @xcite , we only use gold event types in the typeconstrained decoding (tcd) setting. through our experiments, we adopt the pretrained bert-base-cased model. we train all the models for maximumly 20 epochs. if finetuning bert, we set the initial learning rate to 5e-5; otherwise, it is set to 2e-4. we jointly train our argument-detector and span-expander, with loss multipliers of 1.0 and 0.5, respectively. since head-words are not annotated, we apply a simple rule: utilizing predicted dependency trees, we heuristically pick the word that has the smallest arc distance to the dependency root as the head. ties are broken by choosing the rightmost one. there are cases where this procedure does not always give the perfect head, or there is no single head-word for a span (e.g., in multi-word expressions or conjunction). nevertheless, we find this strategy works well in practice. ## conclusion in this work, we propose a flexible two-step approach for implicit event argument detection. our head-word based approach effectively reduces the candidate size and achieves good results on the rams dataset. we further provide detailed error analysis, showing that non-local and non-core arguments are the main difficulties. we hope that this work can shed some light and inspire future work at this line of research.