Document-Level Event Argument Extraction via Optimal Transport (2022)

Abstract: Event Argument Extraction (EAE) is one of the sub-tasks of event extraction, aiming to recognize the role of each entity mention toward a specific event trigger. Despite the success of prior works in sentence-level EAE, the document-level setting is less explored. In particular, whereas syntactic structures of sentences have been shown to be effective for sentence-level EAE, prior document-level EAE models totally ignore syntactic structures for documents. Hence, in this work, we study the importance of syntactic structures in document-level EAE. Specifically, we propose to employ Optimal Transport (OT) to induce structures of documents based on sentence-level syntactic structures and tailored to EAE task. Furthermore, we propose a novel regularization technique to explicitly constrain the contributions of unrelated context words in the final prediction for EAE. We perform extensive experiments on the benchmark document-level EAE dataset RAMS that leads to the state-of-the-art performance. Moreover, our experiments on the ACE 2005 dataset reveals the effectiveness of the proposed model in the sentence-level EAE by establishing new state-of-the-art results.

## introduction event extraction (ee) is one of the important subtasks of information extraction (ie). the major goal of ee is to identify events and their engaged entities (i.e., event arguments). to this end, two sub-tasks should be solved: 1) event detection (ed): to recognize event triggers (i.e., the words or phrases that clearly specify the occurrence of events) and their event types, 2) event argument extraction (eae): to identify participants of events (i.e., entities engaged in events) and their argument roles. compared to ed, the eae sub-task is relatively less explored in ie. moreover, prior works on eae are mainly restricted to sentence-level setting where event triggers and arguments are assumed to appear in the same sentences. this is unfortunate as a considerable portion of event arguments might not be immediately mentioned in the same sentence as their event triggers. for instance, in the ee dataset of the darpa aida program (phase 1) extraction foot_0 , 38% of arguments has been shown to be outside sentences containing the corresponding triggers, i.e., in the document-level context @xcite . thus, further research to study the more realistic setting of document-level eae is extremely needed. to the best of our knowledge, there are two prior works on document-level eae @xcite . both works employ representation vectors for argument spans obtained from a transformer to compute likelihood scores for argument roles. unfortunately, those prior works only exploit the sequential order of words in documents to represent the arguments and totally ignore structures of input documents. this is a limitation as in other related document-level tasks, e.g., relation extraction (re), document structures (i.e., graphs to capture interactions between different words/sentences) have been showed to enhance representation learning @xcite @xcite @xcite . in addition, syntactic structures (i.e., dependency trees) have not been exploited in prior document-level eae models. to address those limitations, we aim to devise a deep learning model to effectively employ syntactic structures of input documents to boost the performance of eae. given a document, how can we efficiently induce its syntactic structure? one simple solution, as demonstrated in @xcite for documentlevel re, is to employ the syntactic structure (i.e., dependency tree) of each sentence and connect their roots to each other to create a connected graph for an input document (called document structure). however, this approach might introduce unrelated words for the role prediction of a candidate argument for a given event trigger into the graph, thus hindering effective representation learning for document-level context. to alleviate this issue, @xcite proposes to prune the document structure by only retaining words along the dependency path (dp) between the two words of interest (i.e., event trigger and argument candidate in our case). unfortunately, in document-level eae, related words for role predictions might not solely reside in the dependency path between the event trigger and argument candidate. in particular, some related words that belong to sentences other than the hosting sentences of the event trigger and argument might be excluded if the document structure is pruned along the dependency path. for instance, in the document "the primary goal of the plan is to provide protection to refugees. according to reports, all 8 countries that signed the plan will congregate once a quarter to monitor the progress.", the trigger and the candidate argument, i.e., provide and countries, appear in different sentences and the dp between them is "@xmath0. however, in order to predict the role of the argument, i.e., giver, one should consider the word plan in the first sentence and the words plan and signed in the second sentence which are off the dp. these limitations call for better methods to prune dependency-based structures of documents to better preserve important words and exclude noisy ones. unlike prior work that resorts to simple syntax-based rules, i.e., distance to the dependency path @xcite , we argue that the pruning operation should be also aware of the semantics of the words. in other words, two criteria, i.e., syntactic and semantic relevance, should be taken into account. specifically, a word is retained in the document structure for document-level eae if it has a small distance to the event trigger/argument words in the dependency structure (i.e., syntaxbased importance) and it is semantically related to one of the words in the dependency path (i.e., semantics-based importance). note that the semantic similarity between words can be obtained from their representations induced by the model. a key challenge for this idea is the different nature of the syntactic and semantic distances that complicates the information combination to determine the im-portance of a word for the structure. in addition, the retention decision for a word should also be contextualized in the potential contributions of other words in the document structure for eae. as such, motivated by the dependency path as the anchor for document structure pruning, we propose to cast the problem of joint consideration of syntactic and semantic distances of words into finding an optimal alignment between off-the-dp and on-the-dp words. the optimal alignment will be solved via optimal transport (ot) methods where syntactic and semantic distances of words to those on the dependency path are simultaneously modeled in a joint optimization problem. ot is an established mechanism to efficiently find an optimal transport plan (i.e., an alignment) between two groups of points (i.e., off-the-dp and on-the-dp words in our case) based on their pairwise transportation costs and the distribution mass accumulated on the points. we propose to employ semantic similarity of words to obtain their transportation costs while syntactic distances to the event trigger/argument are leveraged to compute the mass distributions of words for ot in our document-level eae problem. finally, to prune the document structure, an off-the-dp word is considered important for the document structure (thus being retained) if it is aligned to one of the on-the-dp words via the ot solution. the pruned document structure will be leveraged to learn representation vectors for input documents to perform argument role predictions using graph convolution networks (gcn) @xcite . although the ot-based pruning method could help exclude unrelated words for eae in the document structure, their noisy information might be still encoded in the representations of the related words due to the contextualization in the input encoder (e.g., bert). to improve the representation learning, we thus propose to explicitly constrain the impact of unrelated words for representation learning via a novel regularization technique based on the pruned document structure. in particular, we seek to add unrelated words back to the pruned structure (thus restoring the original tree) and ensure a minimized change of representation vectors due to this addition. as such, in addition to the pruned structure, we apply the gcn model over the original dependency structure to obtain another set of representations vectors for the words. eventually, in the final loss function, we introduce the difference between the representation vectors obtained from the pruned and original structures to achieve the contribution constraint for unrelated words. in our experiments, we evaluate our model on both sentence-level and document-level eae benchmark datasets, demonstrating the effectiveness of the the proposed model by establishing new state-of-the-art results in both settings. ## analysis inter-vs. intra-sentence performance: to shed more light on the effectiveness of the proposed model, we study the performance of the proposed model in two different settings: (1) intra-sentence where both trigger and argument words appear in the same sentence, i.e. the number of sentences between the trigger and the argument is zero; and (2) inter-sentence where the trigger and argument appear in different sentences, i.e., the number of sentences in between is non-zero. we compare our model with the state-of-the-art models for document-level eae, i.e., the rams model @xcite and joint @xcite models in this analysis. all models assumes type-constrained decoding in this section. the results on the rams development set are shown in case study: finally, we perform a case study to explore the benefits of oteae compared to prior document-level methods. in particular, we analyze examples in which our model successfully predicts the argument role, while all other docu-id text role 1 the massive explosions destroyed vehicles on a highway just outside the base at the syrian port-city of tartus, northwestern syria. it is understood the first blast was a car bomb planetout outside the base. the second explosion was a suicide bomber who detonated his belt as people rushed to help those injured, afp reported. place 2 there are worrying reports of the tundra burning in the arctic yamal peninsula, as well as other damaging fires, for example a 3,000 hectare blaze at the lena pillars nature park. ecologists say the fires pose a direct threat to the role of siberian pristine boreal in absorbing climatewarming emissions. ## related works event argument extraction is one of the sub-tasks of event extraction which is mainly approached with sentence-level models in the prior works. early models for this task employed feature-based methods @xcite @xcite @xcite @xcite . later, deep learning emerged as the state-of-the-art approach for sentence-level ee @xcite @xcite @xcite @xcite or specifically, eae @xcite . as the sentence-level models are not able to detect all arguments of an event mentioned in a document, recently the document-level setting has gained more attention; a new dataset (i.e., rams) has been introduced by @xcite . the most similar models to our documentlevel eae model involve rams model @xcite , head-based @xcite and the joint model @xcite . however, none of these models explore document structures for representation that have been shown to be critical for eae in our work. document structures has been also exploited for other information extraction @xcite @xcite and nlp tasks @xcite @xcite . in particular, for the related task of document-level relation extraction, existing works have attempted to construct document structures based on the syntax or discourse information. however, these models fail to involve semantics of the document in the constructed structure, resulting in the inferior performance. in this work, we address this limitation by exploiting optimal transport to jointly consider syntactic and semantic information for document structures. to our knowledge, ot has not been used for document structures in prior work. ## conclusion in this work, we propose a new document structure-aware model for document-level eae. our model employs dependency trees of sentences and presents a novel technique based on optimal transport to prune dependency trees for documents in the eae task. in addition, we introduce a novel regularization to explicitly constrain the contribution of irrelevant words for representation learning. our extensive experiments demonstrate the effectiveness of the proposed model. in the future, we plan to apply our model for other ie tasks.