A rg F use: A Weakly-Supervised Framework for Document-Level Event Argument Aggregation (2021)

Abstract: Most of the existing information extraction frameworks (Wadden et al., 2019; Veysehet al., 2020) focus on sentence-level tasks and are hardly able to capture the consolidated information from a given document. In our endeavour to generate precise document-level information frames from lengthy textual records, we introduce the task of Information Aggregation or Argument Aggregation. More specifically, our aim is to filter irrelevant and redundant argument mentions that were extracted at a sentence level and render a document level information frame. Majority of the existing works have been observed to resolve related tasks of document-level event argument extraction (Yang et al., 2018; Zheng et al., 2019) and salient entity identification (Jain et al., 2020) using supervised techniques. To remove dependency from large amounts of labelled data, we explore the task of information aggregation using weakly supervised techniques. In particular, we present an extractive algorithm with multiple sieves which adopts active learning strategies to work efficiently in low-resource settings. For this task, we have annotated our own test dataset comprising of 131 document information frames and have released the code and dataset to further research prospects in this new domain. To the best of our knowledge, we are the first to establish baseline results for this task in English. Our data and code are publicly available at https://github.com/DebanjanaKar/ArgFuse .

## introduction extraction of event argument information at a document level is an important non-trivial task that requires a system to have advanced natural language understanding capabilities. most of the existing event-argument extraction systems @xcite @xcite pertain to a sentence-level focus, often circumventing to capture information at a document-level. among the few existing works that have researched the task of document-level event argument extraction, we observe that unsupervised techniques like @xcite lack the capacity to identify complex argument mentions, while sophisticated supervised mechanisms like that of @xcite rely on large amounts of annotated corpus and present domain-specific solutions. while supervised techniques may often produce highly accurate systems, in real life, annotating a large corpus can be both very expensive and time consuming. in order to surmount the existing shortcomings coupled with the challenging scenario of data scarcity, we propose our model arg-fuse. argfuse focuses on extraction of relevant and non-redundant event arguments at a document level scope. the task of document level event argument extraction typically focuses on extracting argument mentions associated with an event type or event trigger from a document. it does not involve checking extracted arguments for irrelevant and redundant mentions. through our work, we also propose the related task of argument aggregation which focuses on assessing extracted arguments for irrelevance and redundancy to produce a precise aggregated document-level information frame. figure 1 provides an illustrative example of our task while highlighting the different challenges the task presents. the task presents itself with multiple challenges and avenues to explore. to produce a precise document-level information frame, we focus on sieving irrelevant and redundant sentence-level argument mentions. irrelevant argument mentions, as illustrated in figure 1 refer to argument mentions that do not contribute to the topical focus of the document. a category of irrelevant arguments often encountered in real life news articles are past event records mentioned in narratives to provide a comparative perspective to the reader, much like the arguments in blue mentioned in figure 1 . while irrelevant arguments are usually mentions that refer to past, future or unrelated events; redundancy in arguments manifests in diverse forms. redundant arguments can either be i) duplicate argument mentions (e.g. last week), ii) arguments with similar surface form (e.g. indonesia, indonesia's main island java), iii) re-worded (e.g. killed 41 people, death toll more than doubles to 41), or iv) subsuming information of the other argument(s) (e.g. killed 41 people, 25 people killed). while the first two types of redundancy can be tackled by simple heuristics, the detection of the remaining types of redundancy requires implicit natural language understanding and coreference reasoning capabilities. to filter such arguments effectively, we realised that contextual information of a document is imperative. the argument mentions in green illustrated in figure 1 highlight arguments which impart unique information with respect to the context of the document. these argument mentions cannot be aggregated and are directly added to the output information frame. we refer to such argument mentions in our work as exclusive argument mentions. based on the different types of arguments we encounter, we propose an extractive algorithm that aggregates sentence-level argument or entity mentions to produce precise document-level information frames from lengthy text articles effectively. in our work, we present an end-to-end framework to extract events and arguments from english news articles and present an aggregated information frame at the document level. given that we introduce a novel task with no prior labeled dataset, we present a weakly-supervised algorithm to achieve our task with a good accuracy. our contributions in this work are two fold: in the following sections, we discuss our dataset, algorithms and findings in detail. our analysis emphasizes on the importance of having document level information extraction frameworks for the task of argument aggregation and we invite the research community to further investigate this task. ## related work event argument extraction is a well researched information extraction task which has seen a lot of work at the sentence-level @xcite @xcite @xcite ) but a scarce amount of research has been carried out at the document level. recent literature on event argument extraction at a document level include the works of @xcite and @xcite . while @xcite 's work explores supervised transformer based techniques to extract events and arguments from chinese financial documents, @xcite employs bi-lstm based classifiers on a subset of the same dataset to extract events first at the sentence level followed by a document level extraction similar to our framework. @xcite employs a bilstm-crf classifier to finetune scibert @xcite on various document-level information extraction tasks including the related task of salient entity clustering. all of these methods employ supervised techniques which call for a large corpus of annotated dataset, making it difficult to adopt to domains and tasks with no labelled annotations. @xcite presents an unsupervised approach of extracting document level information from news documents, but the heuristics adopted in their work do not extend well to our task which involves more complex argument mentions. the limited amount of research that exists in this domain does not explore the task of aggregation in particular, where, given a set of arguments referring to the same concept, the most informative argument is selected to represent that knowledge. we propose a novel task and present an end-to-end baseline solution to extract and aggregate document-level arguments which presents a complete overview of the document without minimal loss of information. in our work, we employ ranking strategies as part of our aggregation process. some of the classic works related to the task of ranking text snippets are that of pagerank @xcite and tex-trank @xcite . ## dataset one of the main challenges that we faced was the unavailability of annotated resources for this task. for our auxiliary task of sentence level event argument extraction, we use the dataset adopted by @xcite . the dataset is available in five indian languages but for this task, we only use the english dataset. the dataset covers 32 event types at a fine grain level and 12 event types at a coarse level. the dataset contains annotations for 14 argument types, but in our work we focus on 6 main argument types which are, time, place, casualties, after-effect, reason and participant. in the sections to follow we discuss regarding the scarcely available document-level annotated resources and the details of the annotated dataset we release with this work. ## information aggregation in this section, we detail the approaches that were taken to build a weakly-supervised argument aggregation framework. the framework primarily involves two main modules: i) sentence-level information extraction (ie), which extracts the sentencelevel arguments along with their event type, the ii) aggregation unit, which renders document-level information frames. a general overview of the pipeline is illustrated in figure 4 . we explain each of these modules in detail in the sections to follow. ## experiments & results in this section we shall detail the execution details of the experiments and analyse the results obtained. ## analysis in this section, we present a thorough analysis of our findings and the novel task that we introduce. we investigate the contributions as well as the pitfalls of our framework and attempt to provide directions for improvement. in the sections to follow, we analyse our framework both quantitatively and qualitatively. ## conclusion in this paper, we present an extractive approach to aggregate sentence-level argument or entity mentions to produce precise document-level information frames from lengthy text articles effectively. with a very scarce amount of work being conducted in the field of document-level ie, we develop and open-source our dataset of aggregated argument mentions. to the best of our knowledge, we present the baseline for the task of argument aggregation and open-source our work for research. we closely analyse the merits and demerits of the model and encourage scientists to build on the pitfalls discussed and enhance the aggregation capabilities at a document level. for future work, we want to analyse the model's aggregation capabilities in crosslingual and multilingual environments and extend the aggregation capabilities across document boundaries. as explored in works like @xcite , extending arg-fuse to aggregate information from multiple news sources can present with a very useful and practical use case.