A Multi-Format Transfer Learning Model for Event Argument Extraction via Variational Information Bottleneck (2022)

Abstract: Event argument extraction (EAE) aims to extract arguments with given roles from texts, which have been widely studied in natural language processing. Most previous works have achieved good performance in specific EAE datasets with dedicated neural architectures. Whereas, these architectures are usually difficult to adapt to new datasets/scenarios with various annotation schemas or formats. Furthermore, they rely on large-scale labeled data for training, which is unavailable due to the high labelling cost in most cases. In this paper, we propose a multi-format transfer learning model with variational information bottleneck, which makes use of the information especially the common knowledge in existing datasets for EAE in new datasets. Specifically, we introduce a shared-specific prompt framework to learn both format-shared and format-specific knowledge from datasets with different formats. In order to further absorb the common knowledge for EAE and eliminate the irrelevant noise, we integrate variational information bottleneck into our architecture to refine the shared representation. We conduct extensive experiments on three benchmark datasets, and obtain new state-of-the-art performance on EAE.

## introduction event extraction (ee) has received widespread attention in recent years, which aims to obtain structured information (e.g., trigger, event types, arguments, and argument roles) from large unstructured text corpora @xcite . event argument extraction (eae) plays a crucial role in ee. recently, deep learning-based models have obtained tremendous success in this task. however, these methods rely on a large-scale labeled dataset for training, which is time-consuming and labor-intensive due to the complexity of event extraction. and to the south , british[place] forces[target] continue their attack[trigger] on targets around basra . trigger attack target forces place basra attacker -victim -instrument -wikievents example conflict.attack.detonateexplode for example , ms . davis has identified a man whose photo matches that of a \" john doe # 2 \" sought immediately after the murrah building[target] attack[trigger]. he appears to be a palestinian by the name of hussain hashem al hussaini ... trigger attack target murrah building place -attacker -explosivedevice -instrument - in this paper, we aim to answer the question "can we transfer the knowledge from the existing complex event extraction datasets with different formats?". there are several event extraction datasets, such as ace @xcite , rams @xcite , and wikievents @xcite . these datasets contain abundant event types and semantic roles that may possess overlap knowledge and help to improve the performance of new datasets or low-resource extraction. as shown in figure 1 , both ace2005 and wikievents datasets contain the same "attack" event type with inconsistent names. additionally, some shared argument roles (e.g., "target", "attacker", "place", and "instrument") are labeled in both two datasets. all this information shows that the event knowledge can be transferred between two datasets. however, the transfer between different event argument extraction is a challenging task. (c1) one challenge is that the formats of various datasets are inconsistent due to the complex structure of event records. thus, it is hard to find a unified model to extract arguments with different formats. more specifically, 1) two datasets may have dif-ferent event types, which have various argument structures; 2) the same event type or argument type in two datasets may have different names. for example, the event names are "conflict attack" and "conflict attack detonate explode" in ace2005 and wikievents respectively (figure 1 ); 3) the argument roles set of the same event type may be different in various datasets. for instance, the argument role "victim" and "explosivedevice" for event "attack" only appear in ace2005 and wikievents, respectively (figure 1 ). (c2) the another challenge is that the annotation among different datasets may exist a gap, which brings noise for transfer learning. two datasets may have significant semantic differences, as they may belong to different domains. in addition, the annotation guidelines may be contradictory among various datasets. our experiments also show that merging two datasets simply may reduce the performance. previous works mainly regard the argument extraction as a sequence labeling, which can not transfer to new event argument types @xcite . then, a machine reading comprehension problem (mrc) based model is proposed to extract the arguments using natural questions @xcite . recently, prompt-learning @xcite based models @xcite and generation-based models @xcite @xcite are utilized for event argument extraction. these studies inspire us to design a unified model that can extract arguments with different formats for eae. moreover, some researches investigate cross-lingual event extraction @xcite and zero-shot event extraction @xcite , which are under zero-shot setting. in other words, these studies train on the source language or domain and transfer it to the target domain, where the target domain has no training data. different from them, we train our model on both the source and target datasets with different formats where the format-shared knowledge is essential. to deal with the above challenges, we propose a multi-format transfer learning model for eae via information bottleneck, denoted as unifiedeae, which can leverage all event extraction datasets with heterogeneous formats. first, we adopt a shared-specific prompt (ssp) framework to capture format-shared and format-specific knowledge to extract arguments with different formats. then, to better capture the format-shared representation, we incorporate the variational information bottleneck (vib) into the format-shared model (sharedvib). vib has been widely used to forget the irrelevant information and retain the vital information for prediction @xcite . we leverage it to enhance the model to learn the format-shared knowledge. we conduct a series of experiments on three publicly available datasets and obtain new state-of-the-art performance. our unifiedeae can also improve the performance of low-resource eae effectively. furthermore, the results show that our model can capture the format-shared knowledge and forget the noise among various datasets. in summary, the main contributions of this paper are summarized as follows. ## methodology to transfer the knowledge among the datasets with different formats, we propose a unifiedeae model for event argument extraction task (figure 2 ). unifiedeae is based on a shared-specific prompt (ssp) architecture, which learns both the format-shared and format-specific knowledge from diverse datasets with multiple formats. then, to enhance the model to learn the format-shared knowledge, we integrate variational information bottleneck into the format-shared model (sharedvib) by removing the format irrelevant information and retaining format invariable knowledge. ## results and analyses to investigate the efficacy of unifiedeae model, we compare our model with the mainstream baselines (section 5.1). then, we do the ablation studies to verify the performance of the parts consisting of our model from two views, the model structure and dataset transferring (section 5.2). we also explore the effectiveness of transfer learning on low-resource eae (section 5.3) and case studies are given (section 5.4). ## conclusions and future work in this paper, we propose a unified event argument extraction (unifiedeae) model to transfer the knowledge among multi-format datasets. first, a shared-specific prompt architecture is introduced to extract the event arguments with multiple formats based on both format-shared and formatspecific representations. then, to enhance the model to capture the format-shared knowledge effectively, we integrate the information bottleneck into our architecture. variational information bottleneck is leveraged to eliminate the format-specific information and retain the format-shared knowledge. we conduct extensive experiments on three eae datasets and compare our model with several strong baselines. the results show that our unifiedeae model outperforms the state-of-theart baselines. furthermore, the ablation studies show sharedvib can capture the format-shared effectively. our model also obtains good results on low-resource event argument extraction. in further work, we would like to adopt our model to other complex tasks, such as relation extraction, and named entity recognition.