Multi-Sentence Argument Linking (2020)

Abstract: We present a novel document-level model for finding argument spans that fill an event’s roles, connecting related ideas in sentence-level semantic role labeling and coreference resolution. Because existing datasets for cross-sentence linking are small, development of our neural model is supported through the creation of a new resource, Roles Across Multiple Sentences (RAMS), which contains 9,124 annotated events across 139 types. We demonstrate strong performance of our model on RAMS and other event-related datasets.

## introduction textual event descriptions may span multiple sentences, yet large-scale datasets predominately annotate for events and their arguments at the sentence level. this has driven researchers to focus on sentence-level tasks such as semantic role labeling (srl), even though perfect performance at such tasks would still enable a less than complete understanding of an event at the document level. in this work, we approach event understanding as a form of linking, more akin to coreference resolution than sentence-level srl. an event trigger evokes a set of roles regarded as latent arguments, with these implicit arguments then potentially linked to explicit mentions in the text. consider the example in figure 1 : the airstrikemissilestrike event (triggered by "bombarding") gives rise to a frame or set of typelevel roles (attacker, target, instrument, place) with the referents ("russians", "rebel outpost", "aircraft", "syria"). 2 intuitively we recognize the possible existence of fillers for these roles, for example, the place of the particular airstrikemissilestrike event. these implicit arguments are linked to explicit arguments in the document (i.e., text spans). we refer to the task of finding explicit argument(s) to fill each role for an event as argument linking. prior annotation of cross-sentence argument links has produced small datasets, with a focus either on a small number of predicate types @xcite @xcite or on a small number of documents @xcite . to enable the development of a neural model for argument linking, we produce roles across multiple sentences (rams), a dataset of 9,124 annotated events from news based on an ontology of 139 event types and 65 roles. in a 5sentence window around each event trigger, we annotate the closest argument span for each role. our model builds on recent ideas in span selection models @xcite @xcite , used in this work for the multisentence argument linking task for rams and for several other event-based datasets @xcite @xcite , aida phase 1). on rams our best model achieves 68.3 f 1 , and it achieves 73.3 f 1 when event types are also known, outperforming strong baselines. we also demonstrate effective use of rams as pre-training for a related dataset. our main contributions are a novel model for argument linking and a new large-scale dataset for the task. our dataset is annotated for arguments across multiple sentences and has broader coverage of event types and more examples than similar work. our experiments highlight our model's adaptability to multiple datasets. together, these contributions further the automatic understanding of events at the document level. ## non-local arguments we are not the first to consider non-local event arguments; here we review prior work and refer to @xcite for further reading. whereas local (sentence-level) event arguments are wellstudied as semantic role labeling-utilizing large datasets such as ontonotes 5.0 @xcite -existing datasets annotated for non-local arguments are too small for training neural models. much of the effort on non-local arguments, sometimes called implicit srl, has focused on two datasets: semeval-2010 task 10 ( @xcite and beyond nombank (henceforth bnb) @xcite . these datasets are substantially smaller than rams: the semeval task 10 training set contains 1,370 frame instantiations over 438 sentences, while bnb contains 1,247 examples covering just 10 nominal predicate types. multi-sentence amr (ms-amr) @xcite contains 293 documents annotated with a documentlevel adaptation of the abstract meaning representation (amr) formalism. @xcite notes that the relatively small size of the ms-amr and semeval datasets hinders supervised training. in contrast to these datasets, rams contains 9,124 annotated examples covering a wide range of nominal and verbal triggers. under the darpa aida program, the linguistic data consortium (ldc) has annotated document-level event arguments under a threelevel hierarchical event ontology (see figure 2 in related nist evaluations. 5 aida-1 documents focus on recent geopolitical events relating to interactions between russia and ukraine. unless otherwise noted, statistics about aida-1 pertain only to the practice portion of the dataset. for each document in ldc's collection, only aida-salient events are annotated. this protocol does not guarantee coverage over the event ontology: 1,559 event triggers are annotated in the text portion of the collection, accounting for only 88 of the 139 distinct event sub-subtypes in the ontology. our dataset, rams, employs the same annotation ontology but is substantially larger and covers all 139 types in the ontology. figure 3 ( §3 ) compares the two datasets. across multiple datasets, a substantial number of event arguments are observed to be nonlocal. for example, @xcite found that their annotation of non-local arguments added 71% (relative) role coverage to nombank annotations. additionally, 38.1% of the annotated events in aida-1 have an argument outside the sentence containing the trigger. this phenomenon is not surprising in light of the analysis of zero anaphora and definite null complements by @xcite and the distinction between "core" and "non-core" frame elements or roles in framenet @xcite and propbank @xcite . as previous datasets have been small, various approaches have been taken to handle scarcity. to obtain more training data, @xcite created artificial instances from data annotated jointly for coreference and semantic roles. @xcite automatically induced implicit arguments from pairs of comparable texts, but recovered a proportionally small set of additional arguments. @xcite combined existing corpora to increase and diversify sources of model supervision. @xcite approached the data scarcity problem by recasting implicit srl as a cloze task and as a reading comprehension task, for which data can be generated automatically. the tac kbp event argument extraction task also seeks arguments from document contexts. however, in our work we are concerned with reified events (explicit mentions) and links between event mentions and argument mentions rather than entity-level arguments (coreference clusters). ## rams motivated by the scarcity of data for training neural models to predict non-local arguments, we constructed roles across multiple sentences (rams), a crowd-sourced dataset with annotations for 9,124 events following the aida ontology. we employed the aida ontology in rams so-as to be most similar to an existing corpus already being investigated by various members of the community. each example consists of a typed trigger span and 0 or more argument spans in an english document. a trigger span is a word or phrase that evokes a certain event type in context, while argument spans denote role-typed participants in the event (e.g., the recipient). trigger and argument spans are token-level [start, end] offsets into a tokenized document. typically, event and relation datasets annotate only the argument spans that are in the same sentence as the trigger, but we present annotators with a multi-sentence context window surrounding the trigger. annotators may select argument spans in any sentence in the context window. ## model we formulate argument linking as follows, similar to the formulation in @xcite . assume a document d contains a set of described events e, each designated by a trigger-a text span in d. the type of an event e determines the set of roles the event's arguments may take, denoted r e . for each @xmath0, the task is to link the event's roles with arguments-text spans in d-if they are attested. specifically, one must find for each e all (r, a) pairs such that @xmath1. this formulation does not restrict each role to be filled by only one argument, nor does it restrict each explicit argument to take at most one role. ## rams experiments and results in the following experiments, for each event the model is given the (gold) trigger span and the (gold) spans of the arguments. the model finds for each role the best argument(s) to fill it. predictions are returned as trigger-role-argument triples. we use feature-based bert-base @xcite -mixing layers 9 through 12-by splitting the documents into segments of size 512 subtokens and encoding each segment separately. 14 we perform preliminary sweeps across hyperparameter values, which are then fixed while we perform a more exhaustive sweep across scoring features. we also compare argmax decoding with greedy decoding during training. the best model is selected based on f 1 on the development set, and ablations are reported in the results using our model with greedy decoding and tcd are reported in finally, we examine the effect of tcd to see whether the model effectively uses gold event types if they are given. tcd filters out illegal predictions, boosting precision. recall is still affected by this decoding strategy because the model may be more confident in the wrong argument for a given role, thus filtering out the less confident, correct one. nevertheless, using gold types at test time generally leads to gains in performance. ## conclusion we introduced a novel model for document-level argument linking. because of the small amount of existing data for the task, to support training our neural framework we constructed the rams dataset consisting of 9,124 events covering 139 event types. our model outperforms strong baselines on rams, and we also illustrated its applicability to a variety of related datasets. we hope that rams will stimulate further work on multisentence argument linking.