RAAT : Relation-Augmented Attention Transformer for Relation Modeling in Document-Level Event Extraction (2022)

Abstract: In document-level event extraction (DEE) task, event arguments always scatter across sentences (across-sentence issue) and multipleevents may lie in one document (multi-event issue). In this paper, we argue that the relation information of event arguments is of greatsignificance for addressing the above two issues, and propose a new DEE framework which can model the relation dependencies, calledRelation-augmented Document-level Event Extraction (ReDEE). More specifically, this framework features a novel and tailored transformer,named as Relation-augmented Attention Transformer (RAAT). RAAT is scalable to capture multi-scale and multi-amount argument relations. To further leverage relation information, we introduce a separate event relation prediction task and adopt multi-task learning method to explicitly enhance event extraction performance. Extensive experiments demonstrate the effectiveness of the proposed method, which can achieve state-of-the-art performance on two public datasets. Our code is available at https://github.com/TencentYoutuResearch/RAAT .

## introduction event extraction (ee) task aims to detect the event from texts and then extracts corresponding arguments as different roles, so as to provide a structural information for massive downstream applications, such as recommendation @xcite , knowledge graph construction @xcite and intelligent question answering @xcite . most of the previous methods focus on sentencelevel event extraction (see) (ahn, 2006; liao and grishman, 2010; li et al., 2013; chen et al., 2015; nguyen et al., 2016; zhao et al., 2018; sha et al., 2018; yan et al., 2019; du and cardie, 2020; li * these authors contributed equally to this work @xcite @xcite , extracting events from a single sentence. however, see is mostly inconsistent with actual situations. for example, event arguments may scatter across different sentences. as illustrated in figure 1 , the event argument @xcite of event role pledger is mentioned in sentence 4 and the corresponding argument @xcite of event role pledgee is in sentence 5 and 6. we call this across-sentence issue. another situation involves the multi-event issue, which means that multiple events may exist in the same document. as seen in the example in figure 1 , where two event records coincide, we should recognize that they may partially share common arguments. recently, document-level event extraction (dee) attracts great attention from both academic and industrial communities, and is regarded as a promising direction to tackle the above issues @xcite @xcite @xcite . however, by our observation, we discover that the relations between event arguments have patterns which are an important indicator to guide the event extraction. this information is neglected by existing dee methods. intuitively, the relation information could build long-range relationship knowledge of event roles among multiple sentences, which could relieve the across-sentence issue. for multi-event issue, shared arguments within one document could be distinguished to different roles based on the different prior relation knowledge. as illustrated in figure 1 , @xcite and @xcite have a prior relation pattern of pledger and pledgee, as well as @xcite and @xcite for the relation pattern between pledger and its pledged shares. therefore, the relation information could increase the dee accuracy if it is well modeled. in this paper, we propose a novel dee framework, called relation-augmented document-level event extraction (redee), which is able to model the relation information between arguments by designing a tailored transformer structure. this structure can cover multi-scale and multi-amount relations and is general for different relation modeling situations. we name the structure as relationaugmented attention transformer (raat). to fully leverage the relation information, we introduce a relation prediction task into the redee framework and adopt multi-task learning method to optimize the event extraction task. we conduct extensive experiments on two public datasets. the results demonstrate the effectiveness of modeling the relation information, as well as our proposed framework and method. in summary, our contributions are as follows: ## preliminaries firstly, we clarify several key concepts in event extraction tasks. 1) entity: a real world object, such as person, organization, location, etc.2) entity mention: a text span in document referring to an entity object. 3) event role: an attribute corresponding a pre-defined field in an event. 4) event argument: an entity playing a specific event role. 5) event record: a record expressing an event itself, including a series of event arguments. in document-level event extraction task, one doc-ument can contain multiple event records, and an event record may miss a small set of event arguments. further more, a entity can have multiple event mentions. ## methodology in this section, we introduce the proposed architecture first and then the key components in detail. ## experiments in this section, we report the experimental results to prove the effectiveness of our proposed method. in summary, the experiments could answer the following three questions: duee-fin is also from the financial domain with around 11,900 documents in total. the dataset is downloaded from an online competition website * . since there is no ground truth publicly available for the test set, we can only submit our extracted results to the website as a black-box online evaluation. compared to chifinann, there are two * https://aistudio.baidu.com/aistudio/competition/detail/46 differences. the duee-fin dataset has 13 different event types and its test set includes a large size of document samples that do not have any event records, which both make it more complicated. we get the distribution information of the dataset from appendix a.1. ## conclusion in this paper, we investigate a challenging task of event extraction at document level, towards the across-sentence and multi-event issues. we propose to model the relation information between event arguments and design a novel framework redee. this framework features a new raat structure which can incorporate the relation knowledge. the extensive experimental results can demonstrate the effectiveness of our proposed method which makes the state-of-the-art performance on two benchmark datasets. in the future, we will make more efforts to accelerate training and inference process.