Dynamic Global Memory for Document-level Argument Extraction (2022)

Abstract: Extracting informative arguments of events from news articles is a challenging problem in information extraction, which requires a global contextual understanding of each document. While recent work on document-level extraction has gone beyond single-sentence and increased the cross-sentence inference capability of end-to-end models, they are still restricted by certain input sequence length constraints and usually ignore the global context between events. To tackle this issue, we introduce a new global neural generation-based framework for document-level event argument extraction by constructing a document memory store to record the contextual event information and leveraging it to implicitly and explicitly help with decoding of arguments for later events. Empirical results show that our framework outperforms prior methods substantially and it is more robust to adversarially annotated examples with our constrained decoding design.

## introduction an event is a specific occurrence involving participants @xcite . understanding events in the text is necessary for building machine reading systems, as well as for downstream tasks such as information retrieval, knowledge base population, and trend analysis of real-life world events @xcite . event extraction has long been studied as a local sentence-level task @xcite @xcite . this has driven researchers to focus on developing approaches for sentence-level predicate-argument extraction. this is problematic when events and their arguments spread across multiple sentences -in real-world cases, events are often written through1 our code and resources are available at https:// github.com/xinyadu/memory_docie for research purpose. ## task definition in this work, we focus on the challenging problem of extracting informative arguments of events foot_1 from the document. each event consists of (1) a trigger expression which is a continuous span in the document, it is of a type e which is predefined in an ontology; (2) and a set of arguments {arg 1 , arg 2 , ...}, each of them has a role predefined in the ontology, for event type e. in the annotation guideline/ontology, the "template" that describes the connections between arguments of the event type is also provided. for example, when e is arrest, its corresponding arguments to be extracted should have roles: jailer (<@xmath0, detainee (<@xmath1, crime (<@xmath2, place (<@xmath3. its description template is: <@xmath4..., <trg1>, ..., x i , ..., <trg2>, ..., x n } with given event triggers, our goal is to extract all the informative argument spans to fill in the role of e1, e2, etc. for the example piece in figure 1 , e1 is arrest (triggered @xmath5. the ontology is constructed by the darpa kairos project foot_2 for event annotation. it defines 67 event types in a three-level hierarchy, which is richer than the ace05 ontology with only 33 event types for sentence-level extraction. ## methodology in this section, we describe our memory-enhanced neural generation-based framework (figure 2 ) for extracting informative event arguments from the document. our base model is based on a sequenceto-sequence pretrained language model for text generation. we first introduce how we leverage previously extracted events as additional context for training the text generation-based event extraction model to help the model automatically capture event dependency knowledge (section 3.1). to explicitly help the model satisfy the global event knowledge-based constraints (e.g., it is improbable that one person would be jailer in event a and then attacker in event b), we propose a dynamic ## further analysis in this section, we further provide more insights with quantitative and qualitative analysis, as well as error analysis for the remaining challenges. ## related work event knowledge there has been work on acquiring event-event knowledge/subevent knowl-edge with heuristic-based rules or crowdsourcingbased methods. @xcite propose to use crowdsourcing for obtaining if-then relations between events. @xcite use generative language models to generate new event knowledge based on crowdsourced triples. @xcite propose a weakly-supervised approach to extract sub-event relation tuples from the text. in our work, we focus on harvesting knowledge-based event argument pair constraints from the predefined ontology with training data co-occurrence statistics. plus, the work above on knowledge acquisition has not investigated explicitly encoding the knowledge/constraints for improving the performance of models of document-level event extraction related tasks. ## conclusions and future work in this work, we examined the effect of global document-level "memory" on informative event argument extraction. in the new framework, we propose to leverage the previously extracted events as additional context to help the model learn the dependency across events. at test time, we propose to use a dynamic decoding process to help the model satisfy global knowledge-based argument constraints. experiments demonstrate that our approach achieves substantial improvements over prior methods and has a larger advantage when document length and events number increase. for future work, we plan to investigate how to extend our method to multi-document event extraction cases.