Bi-Directional Iterative Prompt-Tuning for Event Argument Extraction (2022)

Abstract: Recently, prompt-tuning has attracted growing interests in event argument extraction (EAE). However, the existing prompt-tuning methods have not achieved satisfactory performance due to the lack of consideration of entity information. In this paper, we propose a bi-directional iterative prompt-tuning method for EAE, where the EAE task is treated as a cloze-style task to take full advantage of entity information and pre-trained language models (PLMs). Furthermore, our method explores event argument interactions by introducing the argument roles of contextual entities into prompt construction. Since template and verbalizer are two crucial components in a cloze-style prompt, we propose to utilize the role label semantic knowledge to construct a semantic verbalizer and design three kind of templates for the EAE task. Experiments on the ACE 2005 English dataset with standard and low-resource settings show that the proposed method significantly outperforms the peer state-of-the-art methods.

## introduction as a key step of event extraction, event argument extraction refers to identifying event arguments with predefined roles. for example, for an "attack" event triggered by the word "fired" in the sentence "iraqis have fired sand missiles and aaa at aircraft", eae aims to identify that "iraqis", "missiles", "aaa" and "aircraft" are event arguments with the "attacker", "instrument", "instrument" and "target" roles, respectively. in order to exploit the rich linguistic knowledge contained in pre-trained language models, finetuning methods have been proposed for eae. the paradigm of these methods is to use a pre-trained language model to obtain semantic representations, and then feed these representations into a welldesigned neural network to extract event arguments. for example in figure 1 (a), an event trigger representation and an entity mention representation are first obtained through a pre-trained language model, and then input to a designed neural network, such as hierarchical modular network @xcite and syntax-attending transformer network @xcite , to determine the argument role that the entity mention plays in the event triggered by the trigger. however, there is a significant gap between the eae task and the objective form of pre-training, resulting in the poor utilization of the prior knowledge in plms. additionally, finetuning methods heavily depend on extensive annotated data and perform poorly in low-resource data scenarios. to bridge the gap between the eae task and the pre-training task, prompt-tuning methods @xcite @xcite recently have been proposed to for-malize the eae task into a more consistent form with the training objective of generative pre-trained language models. these methods achieve significantly better performance than fine-tuning methods in low-resource data scenarios, but not as good as the state-of-the-art fine-tuning method oneie @xcite in high-resource data scenarios. to achieve excellent performance in both lowresource and high-resource data scenarios, we leverage entity information to model eae as a clozestyle task and use a masked language model to handle the task. figure 1(b) shows a typical clozestyle prompt-tuning method for eae. the typical prompt-tuning method suffers from two challenges: (i) the typical human-written verbalizer @xcite is not a good choice for eae. the human-written verbalizer is to manually assign a label word to each argument role. for example in figure 1 (b), we choose the "attacker" as the label word of "attacker" role. however, an argument role may have different definitions in different types of events. for example, the "entity" role refers to "the voting agent" and "the agents who are meeting" in the "elect" and "meet" events, respectively. (ii) event argument interactions are not explored. existing work @xcite @xcite has demonstrated the usefulness of event argument interactions for eae. for the "attack" event triggered by the word "fired" in figure 1 , given that "missiles" is an "instrument", it is more likely to correctly classify "aaa" into the "instrument" role. in this paper, we propose a bi-directional iterative prompt-tuning (bip) method to alleviate the aforementioned challenges. to capture argument interactions, a forward iterative prompt and a backward iterative prompt are constructed to utilize the argument roles of contextual entities to predict the current entity's role. for the verbalizer, we redefine the argument role types and assign a virtual label word to each argument role, where the initial representation of each virtual label word is generated based on the semantic of the argument role. in addition, we design three kind of templates: hard template, soft template, and hard-soft template, which are further discussed in the experimental section. extensive experiments on the ace 2005 english dataset show that the proposed method can achieve the state-of-the-art performance in both low-resource and high-resource data scenarios. ## related work in this section, we review the deep learning methods for event argument extraction and prompttuning methods for natural language processing. ## model in this section, we first introduce the problem description of event argument extraction and the overall framework of our bi-directional iterative prompt-tuning method, then explain the details of designed semantical verbalizer, three different templates, and model training. ## conclusion and future work in this paper, we regard event argument extraction as a cloze-style task and propose a bidirectional iterative prompt-tuning method to address this task. the bi-directional iterative prompttuning method contains a forward iterative prompt and a backward iterative prompt, which predict the argument role of each entity in a left-to-right and right-to-left manner respectively. for the template construction in each prompt, the predicted argument role information is introduced to capture argument interactions. in addition, a novel semantical verbalizer is designed based on the semantic of the argument role. and three kinds of templates are designed and discussed. experiment results have shown the effectiveness of our method in both high-resource and low-resource data scenarios. in the future work, we are interested in the joint prompt-tuning method of event detection and event argument extraction.