Transfer Learning from Semantic Role Labeling to Event Argument Extraction with Template-based Slot Querying (2022)

Abstract: In this work, we investigate transfer learning from semantic role labeling (SRL) to event argument extraction (EAE), considering their similar argument structures. We view the extraction task as a role querying problem, unifying various methods into a single framework. There are key discrepancies on role labels and distant arguments between semantic role and event argument annotations. To mitigate these discrepancies, we specify natural language-like queries to tackle the label mismatch problem and devise argument augmentation to recover distant arguments. We show that SRL annotations can serve as a valuable resource for EAE, and a template-based slot querying strategy is especially effective for facilitating the transfer. In extensive evaluations on two English EAE benchmarks, our proposed model obtains impressive zero-shot results by leveraging SRL annotations, reaching nearly 80% of the fullysupervised scores. It further provides benefits in low-resource cases, where few EAE annotations are available. Moreover, we show that our approach generalizes to cross-domain and multilingual scenarios.

## introduction event argument extraction (eae) is a key component in the task of event extraction @xcite that aims to identify the arguments that serve as roles for event frames. while recent developments in neural network models have enabled impressive improvements on this task in the fully-supervised setting @xcite @xcite , eae remains challenging when abundant annotations are not available. in particular, event schemes are usually specific to the target scenarios. for example, events in biomedical domains, like gene-expression in genia @xcite , can be quite different than the ones in ace @xcite , such as attack and contact. it is costly and he went to the store and bought a book . transferownership buy.01 transport go.02 artifact destination buyer artifact place arg0 arg4 arg0 arg1 event: (ace) srl: (propbank) figure 1: example annotations with ace events (above) and propbank semantic frames (below). brown and red rectangles indicate predicate and argument words, respectively. green lines denote argument links. compared with the specific schemes in eae, semantic role labeling (srl; @xcite extracts predicateargument structures with more general and broadcoverage frame ontologies. srl also enjoys rich and carefully annotated resources, such as prop-bank @xcite and framenet @xcite , covering a wide range of semantic frame types. as shown in the example in figure 1 , srl closely resembles eae: they both specify semantic frames triggered by predicate words and aim at finding arguments for participating roles. therefore, it is natural to consider applying transfer learning 1 (pan and yang, 2009; ruder et al., 2019) to enhance eae with general srl resources. notwithstanding the similarities, there are two main discrepancies between srl and eae structures that should be managed in order to facilitate transfer between them. the first is label mismatch. for example, ace adopts role names with natural language words, such as buyer and place, whereas propbank utilizes generalized labels like arg0 and argm-loc. propbank also provides more specific role descriptions, but many are inconsistent and not well-formed for direct use as role names. although framenet also adopts natural language role names, it is laborious and sometimes challenging to find all the direct mappings to the target event frames. moreover, srl resources do not typically annotate distant arguments, where there are no explicit syntactic encodings expressing the argument relation. 2 for example, in the sentence depicted in figure 1 , though it can be understood that the "store" is very likely to be the place where the "buying" happens, srl annotations do not include this semantically inferred link, whereas it is considered an argument in event annotations. although there have been previous works utilizing srl for argument linking (o'gorman, 2019), it remains unclear how to best directly transfer from srl to eae, especially with recent pre-trained models. in this work, we provide a comprehensive investigation on the transfer from srl to eae. we view the tasks as a role querying problem within a unified framework, which covers varioius different argument extraction methods, including classification-based methods @xcite , machine reading comprehension (mrc)-based methods @xcite @xcite @xcite as well as sequence-to-sequence generation based ones @xcite @xcite . we further explore a template-based slot querying strategy, by querying argument roles using contextualized representations of the corresponding role slots in the frame template. we tackle the label-mismatch problem by forming the queries in templated natural language, which allows for the same query representation to be shared across varied schemes. to mitigate the lack of distant argument annotations in srl, we apply two argument augmentation techniques: data augmentation by shuffling input texts, which reduces the model's reliance on local syntax, and knowledge distillation from question answering (qa) data, which incorporates distant argument signals. with experiments on the standard ace and ere english event benchmarks, we show that srl annotations are valuable resources for eae. with the template-based querying strategy, a model trained with srl can reach nearly 80% of the fullysupervised f1 score in the zero-shot scenario, and an intermediate-training scheme provides further benefits in the low-resource setting. the model also obtains promising results in extensions to crossdomain and multi-lingual scenarios, demonstrating its generalizability. our work highlights the utility of srl annotations in the context of downstream applications with limited direct annotations. our implementation is available at https:// github.com/zzsfornlp/zmsp/ . ## related work utilizing shallow semantics for event-centric information extraction tasks has been explored previously. @xcite leverage framenet frames to enhance event detection. @xcite conduct contrastive pre-training with amr structures to enhance event extraction. several works utilize predicted shallow semantic structures as inputs to help low-resource event extraction @xcite @xcite and event schema induction @xcite . moreover, srl has been utilized for implicit argument linking or implicit semantic role labeling (isrl) in many previous works @xcite @xcite @xcite . this work follows these directions and shows that srl can be a valuable direct training resource for eae. for the eae task, most previous works adopt a classification-based strategy where each role is assigned static querying parameters @xcite @xcite @xcite . recently, two interesting alternative strategies have been explored to enable extraction in more flexible ways: mrc-based methods cast the problem as answering role questions @xcite @xcite @xcite , while generation-based methods adopt sequence-to-sequence generation schemes @xcite @xcite @xcite . we cover all these strategies within a unified role querying framework and further explore a template-based role querying strategy. this strategy is also related with prompt-based learning @xcite @xcite , but differs in the extraction-targeted paradigm. concurrently, @xcite adopt a similar idea, while this work differs mainly in our focus on transfer learning. ## conclusion in this work, we explore transfer learning from semantic roles to event arguments. with unified role querying strategies, we show that srl annotations are a valuable resource for event argument extraction. the srl model also obtains promising results when extended to new scenarios with domain and language differences.