Few-Shot Document-Level Event Argument Extraction (2023)

Abstract: Event argument extraction (EAE) has been well studied at the sentence level but under-explored at the document level. In this paper, we study to capture event arguments that actually spread across sentences in documents. Prior works usually assume full access to rich document supervision, ignoring the fact that the available argument annotation is limited in production. To fill this gap, we present FewDocAE, a Few-Shot Document-Level Event Argument Extraction benchmark, based on the existing document-level event extraction dataset. We first define the new problem and reconstruct the corpus by a novel N-Way-D-Doc sampling instead of the traditional N-Way-K-Shot strategy. Then we adjust the current document-level neural models into the few-shot setting to provide baseline results under in- and cross-domain settings. Since the argument extraction depends on the context from multiple sentences and the learning process is limited to very few examples, we find this novel task to be very challenging with substantively low performance. Considering FewDocAE is closely related to practical use under low-resource regimes, we hope this benchmark encourages more research in this direction. Our data and codes will be available online.

## introduction event argument extraction (eae), a sub-task of event extraction, is a fundamental task for many downstream nlp applications in the ie community. for example, events and arguments play an important role in the knowledge base population from unstructured data @xcite . and the real world public affairs management relies on recognizing the event arguments from daily news and social media @xcite . although tremendous progress has been made under the supervised setting, current neural models typically rely on large-scale humanannotated data, which is not reliable considering the huge amounts of novel events and arguments emerging in many fields every day. few-shot learning (fsl) @xcite is proposed to tackle such limitations to make machine learning models more applicable given limited annotated examples and has been used a lot in the ie area @xcite @xcite . previous research @xcite ) mainly focuses on sentence-level event extraction, such as the popular ace2005 @xcite dataset. in recent years, researchers have begun to realize that the complete event and arguments actually spread in a full document or paragraph @xcite @xcite . and the focus starts to turn into documentlevel event extraction motivated by the newly proposed large-scale document-level corpora, namely the wikievents @xcite , rams @xcite and the recent largest corpus docee @xcite . following these datasets, many novel methods for solving such new challenges brought by the longer context have also been investigated and witness significant progress @xcite @xcite . however, the traditional supervised learning methods heavily rely on large-scale annotated training data, but we are witnessing new events every day due to the rapid emergence of new affairs. thus it is not durable to greedily make large collections of newly appeared events for real-life applications. therefore, more attention has been paid to few-shot event extraction. but the current research has only considered the few-shot eae on a single sentence @xcite @xcite , ignoring the big gap between realistic scenarios. therefore, we aim to pave a new way for few-shot eae at the document level towards urgent data scarcity problems on the complete documents. the recently released large-scale document-level figure 1 : an example of a 3-way-2-doc episode consisting of a support and query set. given a support set of 2 documents with 3 argument types, the goal is to extract all event arguments in the query document. during testing, the argument types are disjoint from type set in the training set. but the documents still share the same main event type. ee datasets include rams @xcite and docee @xcite , and their statistics is shown in the key contributions of this work include: ## constructing few-shot episodes in this section, we talk about the details regarding the dataset conversion and how we obtain episodes for training. for our fewdocae task, the key components for constructing a realistic few-shot dataset are made of two parts: avoiding data leakage and constructing effective support and query pairs. generally, to avoid leaking new event arguments information in the training phase we replace all other labels of arguments contained in the validation/test sets with o. in this way, it is guaranteed that they have disjoint argument types sets, and this is also close to the realistic scenarios. to provide each episode with a support and query set, we greedy sample instances to make sure they satisfy our n -way-d-doc choice. also, they support and query algorithm 1 n-way-d-doc sampling for few-shot episode generation require: dataset x , label set y, n , d ensure: 1: s ← ∅; // init the support set 2: countn ← 0; //init the count of entity types; 3: countd ← 0; //init the count of documents; 4: while countn ̸ = n or countd ̸ = d do 5: randomly sample (x, y) ∈ x , y ; 6: compute countn and countd; 7: if @xmath0: continue; 9: else 10: @xmath1, y) ; 11: update countn, countd ; 12: ## models previous work on document-level ee using bert_seq @xcite demonstrate the success of using a pretrained bert model to sequentially label words in the article. and the superior performance of the long document transformer (e.g. longformer @xcite ) has also been proven to improve the argument extraction task @xcite . we thus follow their baseline settings and use bert or longformer as document encoders for our task. in order to adapt to our few-shot setting, inspired by the successful applications of the prototypical network (protonet) @xcite for meta-learning, we assume there exists one prototypical representation for each argument type. then we implement the models by extending protonet to language encoder with token-level similarity. ## experiments and results we show the experimental results and analysis in this section. more experimental details and configurations can be found in appendix a. in general, one experiment costs around 15 hours and in total there are more than 100 runs, which also limits more advanced models that we can choose. in domain for the in domain setting where the training and test examples are sampled from the same domain, the performance is exhibited in ## conclusion in order to handle new emerging event arguments with limited annotations and adapt it to the realworld document-level scenario, we propose few-docae benchmark to advance the research of few-shot learning for document-level event argument extraction. we conduct comprehensive experiments by extending previous models into our task under in-domain and cross-domain. our experiments confirm the necessity of moving to document level by showing that current models still witnesses suboptimal performance. we also demonstrate the benefits of using a more diverse training base to learn a good argument feature extractor. the current results show that fewdocae is challenging due to the long document and limited examples, as well as the intrinsic charisma of few-shot learning. the relatively low extraction score illustrates the difficulty of this novel task, in the meanwhile it also provides new chances for advancing this field. in summary, we hope fewdocae shed new light on a more realistic but challenging setting for event argument extraction. in the future, we hope to investigate more advanced methods for solving this problem.