Mask-then-Fill: A Flexible and Effective Data Augmentation Framework for Event Extraction (2022)

Abstract: We present Mask-then-Fill, a flexible and effective data augmentation framework for event extraction. Our approach allows for more flexible manipulation of text and thus can generate more diverse data while keeping the original event structure unchanged as much as possible. Specifically, it first randomly masks out an adjunct sentence fragment and then infills a variable-length text span with a fine-tuned infilling model. The main advantage lies in that it can replace a fragment of arbitrary length in the text with another fragment of variable length, compared to the existing methods which can only replace a single word or a fixed-length fragment. On trigger and argument extraction tasks, the proposed framework is more effective than baseline methods and it demonstrates particularly strong results in the low-resource setting. Our further analysis shows that it achieves a good balance between diversity and distributional similarity.

## introduction event extraction (ee), which aims to extract triggers with specific types and their arguments from unstructured texts, is an important yet challenging task in natural language processing. in recent years, deep learning methods have emerged as one of the most prominent approaches for this task @xcite @xcite @xcite . however, they are notorious for requiring large labelled data, which limits the scalability of ee models. annotating data for ee is usually costly and time-consuming, as it requires expert knowledge. one possible solution is to leverage data augmentation (da) @xcite . existing da methods for nlp can be broadly classified into two types: (1) the first is to augment the cop said mike left this town yesterday. the police said mike went away from this town yesterday. the cop saw mike left this town yesterday. applying existing da methods to the ee task is more challenging than to translation or classification tasks, because we need to augment training data while keeping the event structure (trigger and arguments) unchanged. figure 1 presents examples of three different da methods applied to a sentence containing a "transport" event. the event is triggered by word "left" and it has three arguments with different roles ("mike", "this town" and "yesterday"). as shown in figure 1 , it is infeasible to apply sentence-level da methods such as backtranslation @xcite , because it may change the event structure (change "left" to "went away from"). previous attempts on da for such tasks typically use heuristic rules such as synonym replacement @xcite or context-based words substitution with bert @xcite . their idea is to replace adjunct tokens (the tokens in sentences except triggers and arguments) with other tokens, and thus can ensure the event structure is unchanged as much as possible. however, recent studies @xcite @xcite find that such methods provide limited data diversity. in nlp, the diversity of data is mainly reflected in two aspects: expression diversity and semantic diversity @xcite . the synonym replacement and backtranslation methods lack semantic diversity, because they can only produce samples with similar semantics. the bert-based method can only replace words and cannot change the syntax, so it cannot generate samples with a wide variety of expressions. the lack of sufficient diversity may lead to greater overfitting or poor performance through training on examples that are not representative. to this end, we present mask-then-fill, a flexible and effective data augmentation framework for event extraction. our approach allows for more flexible manipulation of text and thus can generate more diverse data while keeping the original event structure unchanged as much as possible. specifically, we first define two types of text fragments in a sentence: event-related fragments (trigger and arguments) and adjunct fragments (e.g. "the police said"). then, we model da for the ee task as a mask-then-fill process: we first randomly masks out an adjunct sentence fragment and then infills a variable-length text span with a fine-tuned infilling model (t5) @xcite . the main advantage lies in that it can replace a fragment of arbitrary length in the text with another fragment of variable length, compared to the existing methods which can only replace a single word or a fixedlength fragment. to the best of our knowledge, we are the first to augment training data for event extraction via text infilling. we empirically show that the maskthen-fill framework improves performance for both classification-based (eeqa) and generationbased (text2event) event extraction models on a well-known ee benchmark dataset (ace2005). especially, it demonstrates strong results in the low resource setting. we further investigate reasons for its effectiveness by introducing two metrics, affinity and diversity, and find that the data augmented by our approach have better diversity with less distribution shifts, achieving a good balance between diversity and distributional similarity. ## the mask-then-fill framework figure 2 presents an overview of mask-then-fill framework. the input sentence contains two types of text fragments: event-related fragments (words with colors) and adjunct fragments (underlined). our idea is to rewrite the whole adjunct fragment instead of replacing some words, and the rewritten sentence fragment should fit the context and should not introduce new events. to this end, we model da for ee as a mask-then-fill process: we first randomly mask out an adjunct sentence fragment and then infills a variable-length text span with a fine-tuned infilling model. in the following, we describe in detail the mask-then-fill framework. mask out an adjunct fragment. given a prototype sentence @xmath0, • • • , x l } of length l from the training set, we first define an adjunct fragment as a set of non-overlapping spans of x that do not contain the event triggers and arguments. we then replace one of the adjunct fragments with a [mask] symbol. the incomplete sentence ## conclusion in this paper, we present mask-then-fill, a flexible and effective data augmentation framework for event extraction. our approach allows for more flexible manipulation of text and thus can generate more diverse data while keeping the original event structure unchanged. the main advantage lies in that it can replace a fragment of arbitrary length in the text with another fragment of variable length. we empirically show that the mask-then-fill framework improves performance for both eeqa and text2event ee models on the ace2005 dataset. it demonstrates particularly strong results in the low-resource setting. our further analysis shows that it achieves a good balance between diversity and distributional similarity.