Enhancing Document-level Event Argument Extraction with Contextual Clues and Role Relevance (2023)

Abstract: Document-level event argument extraction poses new challenges of long input and cross-sentence inference compared to its sentence-level counterpart. However, most prior works focus on capturing the relations between candidate arguments and the event trigger in each event, ignoring two crucial points: a) non-argument contextual clue information; b) the relevance among argument roles. In this paper, we propose a SCPRG (Span-trigger-based Contextual Pooling and latent Role Guidance) model, which contains two novel and effective modules for the above problem. The Span-Trigger-based Contextual Pooling (STCP) adaptively selects and aggregates the information of non-argument clue words based on the context attention weights of specific argument-trigger pairs from pre-trained model. The Role-based Latent Information Guidance (RLIG) module constructs latent role representations, makes them interact through role-interactive encoding to capture semantic relevance, and merges them into candidate arguments. Both STCP and RLIG introduce no more than 1% new parameters compared with the base model and can be easily applied to other event extraction models, which are compact and transplantable. Experiments on two public datasets show that our SCPRG outperforms previous state-of-the-art methods, with 1.13 F1 and 2.64 F1 improvements on RAMS and WikiEvents respectively. Further analyses illustrate the interpretability of our model.

## introduction event argument extraction (eae) aims to identify the arguments of events formed as entities in text and predict their roles in the related event. as the key step of event extraction (ee), eae is an important nlp task with widespread applications, such as recommendation systems @xcite figure 1 : a document from rams @xcite dataset. event conflict and attack is triggered by wounded, with four arguments of different roles scattering across the document. words in red are non-argument clue words meaningful for argument extraction. and dialogue systems @xcite for presenting unstructured text containing event information in structured form. compared with previous works @xcite @xcite focusing on sentence-level eae, more and more recent works tend to explore documentlevel eae @xcite @xcite , which needs to solve long-distance dependency @xcite and cross-sentence inference @xcite problems. therefore, many works @xcite try to construct graphs based on heuristic rules @xcite or syntactic structures @xcite and model logical reasoning with graph neural networks @xcite . however, all of state-of-the-art works ignore two crucial points: (a) the non-argument clue information; (b) the relevance among argument roles. non-argument clues are contextual text except target arguments that can provide important guiding information for the prediction of many complex argument roles. for example, in figure 1 , for the event conflict and attack, non-argument clues detonated, claim responsibility and terrorist attack can provide significant clue information for identifying arguments explosive belts and islamic state. however, many previous works @xcite only utilize pre-trained transformerbased encoder to obtain global context information implicitly, ignoring that for different arguments appearing in events, they should pay attention to context information highly relevant to the entity @xcite and target event @xcite . therefore in this paper, we design a span-triggerbased contextual pooling (stcp) module, which merges the information of non-argument clues for each argument-trigger pair based on the their contextual attention product from pre-trained model, enhancing the candidate argument representation with additional relevant context information. some argument roles have close semantic relevance that is beneficial for argument extraction. for example, in figure 1 , there is close semantic relevance between roles injurer and victim, which can provide significant information guidance for the argument extraction of these two roles in the target event conflict and attack. moreover, many roles co-occur in multiple events @xcite , which may have close semantic relevance. specifically, we count and visualize the frequency of co-occurrence between 15 most frequent roles in rams dataset in figure 2 . for example, roles attacker, target and instrument co-occur frequently, demonstrating that they are more semantically relevant than other roles. in this paper, we propose a role-based latent information guidance (rlig) module, consisting of role-interactive encoding and role information fusion. specifically, we design a role-interactive encoder with roles added into the input sequence, where role embeddings can not only learn latent semantic informa-tion of roles, but capture semantic relevance among roles. the latent role embeddings are then merged into candidate arguments through pooling and concatenating operations, providing information guidance for document-level eae. in this paper, we propose an effective documentlevel eae model named scprg (span-triggerbased contextual pooling and role-based latent information guidance) containing stcp module and rlig module for the the aforementioned two problems respectively. notably, these two modules leverage the well-learned attention weights from the pre-trained language model with no more than 1% new parameters introduced and are easily applied to other event extraction models, which are compact and transplantable. moreover, we try to eliminate noise information by excluding argumentimpossible spans. our contributions are summarized as follows: ## method we formulate document-level event argument extraction as a multi-class classification problem. given a document d consisting of n words, i.e. @xmath0, w 2 , ..., w n }, pre-defined event types set e, the corresponding role set r e and trigger @xmath1, this task aims at predicting all (r, s) pairs for each event in document d, where @xmath2. following @xcite , we figure 3 : the main architecture of scprg. the input sequence with roles is fed into the role-interactive encoder, with context representations, role representations and attention heads as output. stcp adaptively fuses non-argument contextual clues into a context vector based on the attention product between the trigger and arguments. rlig constructs latent role embeddings through role-interactive encoding and fuses them into a latent role vector by pooling operation. the context vector and latent role vector are merged into the final span representation and the classification module predicts argument roles for all candidate spans. extract event arguments for each event in a document independently and figure 3 shows the overall architecture of our scprg. ## related works although deep learning has achieved significant success in many areas of computer vision @xcite @xcite @xcite ) like 3d scene graph generation @xcite and image semantic segmentation @xcite , its impact on event argument extraction in natural language processing has been relatively limited. this is primarily due to the complexity and ambiguity inherent in natural language, which presents significant challenges for the accurate identification and extraction of event-related information. ## conclusion in this paper, we propose a novel scprg framework for document-level eae that mainly consists of two compact, effective and transplantable modules. specifically, our stcp adaptively aggregates the information of non-argument clue words and rlig provides latent role information guidance containing semantic relevance among roles. experimental results show that scprg outperforms existing state-of-the-art eae models and further analyses demonstrate that our method is both effective and explainable. for future works, we hope to apply scprg to more information extraction tasks such as relation extraction and multilingual extraction, where contextual information plays a significant role.