Document-Level Event Argument Extraction by Conditional Generation (2021)

Abstract: Event extraction has long been treated as a sentence-level task in the IE community. We argue that this setting does not match human informative seeking behavior and leads to incomplete and uninformative extraction results. We propose a document-level neural event argument extraction model by formulating the task as conditional generation following event templates. We also compile a new document-level event extraction benchmark dataset WikiEvents which includes complete event and coreference annotation. On the task of argument extraction, we achieve an absolute gain of 7.6% F1 and 5.7% F1 over the next best model on the RAMS and WikiEvents dataset respectively. On the more challenging task of informative argument extraction, which requires implicit coreference reasoning, we achieve a 9.3% F1 gain over the best baseline. To demonstrate the portability of our model, we also create the first end-to-end zero-shot event extraction framework and achieve 97% of fully supervised model’s trigger extraction performance and 82% of the argument extraction performance given only access to 10 out of the 33 types on ACE.

## introduction by converting a large amount of unstructured text into trigger-argument structures, event extraction models provide unique value in assisting us process volumes of documents to form insights. while real-world events are often described throughout a news document (or even span multiple documents), the scope of operation for existing event extraction models have long been limited to the sentence level. early work on event extraction originally posed the task as document level role filling @xcite on a set of narrow scenarios figure 1 : two examples of cross-sentence inference for argument extraction from our wikievents dataset. the paymentbarter argument of the transaction.exchangebuysell event triggered by "reserved" in the first sentence can only be found in the next sentence. the attack.explodedetonate event triggered by "detonated" in the third sentence has an uninformative argument "he", which needs to be resolved to the name mention "mcveigh" in the previous sentences. and evaluated on small datasets. the release of ace foot_1 , a large scale dataset with complete event annotation, opened the possibility of applying powerful machine learning models which led to substantial improvement in event extraction. the success of such models and the widespread adoption of ace as the training dataset established sentencelevel event extraction as the mainstream task defintion. this formulation signifies a misalignment between the information seeking behavior in real life and the exhaustive annotation process in creating the datasets. an information seeking session @xcite can be divided into 6 stages: task initiation, topic selection, pre-focus exploration, focus information, information collection and search closure @xcite . given a target event ontology, we can safely assume that topic selection is complete and users start from skimming the documents before they discover events of interest, focus on such events and then aggregate all relevant information for the events. in both the "pre-focus exploration" and "information collection" stages, users naturally cross sentence boundaries. empirically, using sentence boundaries as event scopes conveniently simplifies the problem, but also introduces fundamental flaws: the resulting extractions are incomplete and uninformative. we show two examples of this phenomenon in figure 1 . the first example exemplifies the case of implicit arguments across sentences. the sentence that contains the paymentbarter argument "$280.32" is not the sentence that contains the trigger "reserve" for the exchangebuysell event. without a documentlevel model, such arguments would be missed and result in incomplete extraction. in the second example, the arguments are present in the same sentence, but written as pronouns. such extraction would be uninformative to the reader without cross-sentence coreference resolution. we propose a new end-to-end document-level event argument extraction model by framing the problem as conditional generation given a template. conditioned on the unfilled template and a given context, the model is asked to generate a filledin template with arguments as shown in figure 2 . our model does not require entity recognition nor coreference resolution as a preprocessing step and can work with long contexts beyond single sentences. since templates are usually provided as part of the event ontology definition, this requires no additional human effort. compared to recent efforts @xcite @xcite ) that retarget question answering (qa) models for event extraction, our generationbased model can easily handle the case of missing arguments and multiple arguments in the same role without the need of tuning thresholds and can extract all arguments in a single pass. in order to evaluate the performance of document-level event extraction, we collect and annotate a new benchmark dataset wikievents. this document-level evaluation also allows us to move beyond the nearest mention of the argument and instead seek the most informative mention foot_2 in the entire document context. in particular, only 34.5% of the arguments detected in the same sentence as the trigger can be considered informative. we present this new task of document-level informative argument extraction and show that while this task requires much more cross-sentence infer-ence, our model can still perform reliably well. since we provide the ontology information (which roles are needed for the event) through the template as an external condition, our model has excellent portability to unseen event types. by pairing up our argument extraction model with a keyword-based zero-shot trigger extraction model, we enable zero-shot transfer for new event types. the major contributions of this paper can be summarized as follows: ## method the event extraction task consists of two subtasks: trigger extraction and argument extraction. the set of possible event types and roles for each event type are given by the event ontology as part of the dataset. one template for each event type is usually pre-defined in the ontology. 4 we first introduce our document-level argument extraction model in section 2.1 and then introduce our zero-shot keyword-based trigger extraction model in section 2.2. ## experiments our experiments fall under three settings: (1) document-level event argument extraction; (2) document-level informative argument extraction and (3) zero-shot event extraction. for document-level event argument extraction we follow the conventional approach of regarding the argument mention with closest proximity to the trigger as the ground truth. in the second setting we consider the most informative mention of the argument as the ground truth. the zero-shot setting examines the portability of the model to new event types. under this setting we consider a portion of the event types to be known and only annotation for these event types will be seen. we used two settings for selecting known types: 10 most frequent events types and 8 event types, one from each parent type of the event ontology. the evaluation is done on the complete set of event types. we refer the reader to appendix c for implementation details and hyperparameter settings. ## conclusion & future work in this paper, we advocate document-level event extraction and propose the first document-level neural event argument extraction model. we also release the first document-level event extraction benchmark dataset wikievents with complete event and coreference annotation. on both the conventional argument extraction task and the new informative argument extraction task, our proposed model surpasses crf-based and qa-based baselines by a wide margin. additionally, we demonstrate the portability of our model by applying it to the zero-shot setting. going forward, we would like to incorporate more ontological knowledge to produce more accurate extractions.