Hyperspherical Multi-Prototype with Optimal Transport for Event Argument Extraction (2024)

Abstract: Event Argument Extraction (EAE) aims to extract arguments for specified events from a text. Previous research has mainly focused on addressing long-distance dependencies of arguments, modeling co-occurrence relationships between roles and events, but overlooking potential inductive biases: (i) semantic differences among arguments of the same type and (ii) large margin separation between arguments of the different types. Inspired by prototype networks, we introduce a new model named HMPEAE, which takes the two inductive biases above as targets to locate prototypes and guide the model to learn argument representations based on these prototypes.Specifically, we set multiple prototypes to represent each role to capture intra-class differences. Simultaneously, we use hypersphere as the output space for prototypes, defining large margin separation between prototypes to encourage the model to learn significant differences between different types of arguments effectively.We solve the “argument-prototype” assignment as an optimal transport problem to optimize the argument representation and minimize the absolute distance between arguments and prototypes to achieve compactness within sub-clusters. Experimental results on the RAMS and WikiEvents datasets show that HMPEAE achieves state-of-the-art performances.

## introduction event argument extraction (eae) is an essential branch of event extraction that aims to extract the arguments of specified events from a text and predict their roles. eae is critical in many downstream tasks, such as question and answer, recommender, and dialog systems. in the eae task, we observe two types of inductive biases. the first is the presence of intra-class * corresponding author. in this paper, we propose the hmpeae to address the two issues above. it is a prototype-based approach similar to prototype networks. in pro-totype networks, the prototype is the mean output vector of all samples per class, guiding the model to learn the distribution of samples in the embedding space @xcite @xcite @xcite . however, traditional prototype-based methods only assign one prototype per category, neglecting semantic variations within the same category. this oversight can lead to the model being influenced by intra-class variance. additionally, the existing multi-prototype method @xcite does not consider the large margin separation between classes. this leads to fuzzy boundaries in the embedding space for samples of different categories, causing model decision difficulties and poor robustness. hmpeae sets multiple prototypes for each role, allowing each prototype to represent a specific subcluster. in this way, we can capture the intraclass variance. simultaneously, we use the hypersphere as the representation space @xcite for prototypes to maximize the distances between class prototypes and achieve a large interclass separation. further, we consider smaller distances between prototypes of roles with similar semantics and larger distances between prototypes of roles with larger semantic differences. inspired by @xcite , we consider the similarity between semantics of roles as the prior information and employ a ranking-based loss function as one of the objectives for locating prototypes. we use the pre-trained prototypes to guide the eae model in learning the arguments' representations. specifically, we adopt tabeae @xcite as the backbone model, inheriting its key modules such as encoder, decoder, prompt templates, slot tables and span selectors. during training, each argument should be assigned an appropriate prototype to optimize its representation. we consider such an "argument-prototype" assignment an optimal transport problem while minimizing the absolute distances between arguments and prototypes to achieve compactness for the same sub-cluster. we summarize our contributions as follows: ## methodology in this section, we first describe the task definition of eae in subsection 2.1, followed by a detailed description of the process of constructing hyperspherical multi-prototypes in subsection 2.2. we then briefly outline the backbone eae model chosen for this paper in subsection 2.3, and finally explore a solution to the optimal transport problem between arguments and prototypes in subsection 2.4. ## conclusion in this paper, we identify two potential inductive biases overlooked in the eae task. therefore, we propose the hmpeae, which first pre-trains a set of role hyperspherical multi-prototypes targeting these two inductive biases and directs the eae model to learn argument representations based on these prototypes. during eae model training, we solve argument-prototype assignment as an optimal transport problem. experimental results on two commonly used datasets show that hmpeae achieves state-of-the-art performance, confirming the effectiveness of the approach proposed in this study.