An AMR -based Link Prediction Approach for Document-level Event Argument Extraction (2023)

Abstract: Recent works have introduced Abstract Meaning Representation (AMR) for Document-level Event Argument Extraction (Doc-level EAE), since AMR provides a useful interpretation of complex semantic structures and helps to capture long-distance dependency. However, in these works AMR is used only implicitly, for instance, as additional features or training signals. Motivated by the fact that all event structures can be inferred from AMR, this work reformulates EAE as a link prediction problem on AMR graphs. Since AMR is a generic structure and does not perfectly suit EAE, we propose a novel graph structure, Tailored AMR Graph (TAG), which compresses less informative subgraphs and edge types, integrates span information, and highlights surrounding events in the same document. With TAG, we further propose a novel method using graph neural networks as a link prediction model to find event arguments. Our extensive experiments on WikiEvents and RAMS show that this simpler approach outperforms the state-of-the-art models by 3.63pt and 2.33pt F1, respectively, and do so with reduced 56% inference time.

## introduction event argument extraction (eae) is a longstanding information extraction task to extract event structures composed of arguments from unstructured text @xcite . event structures can serve as an intermediate semantic representation and be further used for improving downstream tasks, including machine reading comprehension @xcite ), question answering @xcite , dialog system @xcite zabiullah mujahid , the spokesperson for the taliban , claimed they were responsible for the attack via twitter and e -mail . the highlighted text means they are captured by the graph. we parse the document to a tailored amr graph and apply a gnn model to predict edges between the trigger and other nodes. in this example, the model predicts one argument, "zabiullah mujahid", with the role of "communicator". et al., 2020), and recommendation system @xcite . despite the large performance boost by pre-trained language models (plms), extracting complex event structures across sentences is still challenging @xcite . in real-world text, event structures are usually distributed in multiple sentences @xcite . to capture cross-sentence and multi-hop structures, @xcite introduces abstract meaning representation (amr) graphs to assist the model in understanding the document. their main idea is to take amr as additional features to enrich span representations. @xcite and @xcite utilize amr graphs to provide training signals via self-training and contrastive learning, respectively. these methods exemplify that introducing amr information facilitates the model's understanding of complex event structures. however, previous works implicitly use amr information by enriching neural sequential models rather than making explicit use of discrete structures. intuitively, discrete amr structures can force the model to better focus on predicate-argument structures and the content most related to eae, therefore having stronger effect than implicit amr. we aim to exploit the potentials of explicit amr for improving eae by formulating eae as a link prediction task, and figure 1 illustrates the framework. we parse the input document to a graph structure and adopt a link prediction model to find event arguments. we determine if a node is an argument by whether it is connected to the trigger node or not. the advantages of formulating eae as a link prediction problem are three-fold: 1) amr graph is typically more compact than raw text (see sec-2.2), so processing amr to find arguments would be simple and efficient. 2) dependencies among multiple arguments and events are explicitly captured, while previous works @xcite have pointed out the importance of these dependencies which are only implicitly considered in the feature space. 3) the simpler model architecture and sparse graphs can lead to improvement over efficiency, as our experiments show (up to 56% inference time saving). the proposed method assumes that amr graphs contain all necessary information for eae. however, the original amr graphs generated by offthe-shelf amr parsers do not meet this assumption. first, they cover only 72.2% event arguments in wikievents, impeding the performance of eae models directly on the parsed amr graphs. the primary problem is that amr graphs are defined at word-level, but an event argument could be a text span. second, the smatch score of sota amr parsers is around 85 @xcite , which causes information loss as well. to address the above issue, we propose a novel tailored amr graph (tag), which compresses information irrelevant to eae, merges words into text spans via a span proposal module, and highlights the surrounding events in the same document to encourage their communication. particularly, the number of nodes in tag equals around 47% of words in wikievents, which is a significant reduction. since too much distracting information is a major challenge of document-level tasks, we also expect performance gains from focusing on tag, which is evidenced by our experiment results. tag can cover all eae samples if the span proposal module adds enough text spans, and we will discuss the trade-off between the recall of spans and model efficiency in appendix-a.3. although there is a large design space for the link prediction model, we choose a simple architecture that stacks gnn layers on top of pre-trained text encoders. the whole model is called tara for tailored amr-based argument extraction. we conduct extensive experiments on latest documentlevel eae datasets, wikievents @xcite and rams @xcite . tara achieves 3.63pt and 2.33pt improvements of f1 against the sota, respectively. since interactions in gnn are sparse, the computation cost of our model is also lower, saving up to 56% inference time. to our knowledge, we are the first to formulate eae as a link prediction problem on amr graphs. ## methodology in this section, we first explain how to formulate eae as a link prediction problem and discuss the benefits of doing so (sec-2.1). to make amr graphs better suit the eae task and ensure the reformulation is lossless, we provide a series of modifications for amr graphs, resulting in a compact and informative graph, named tailored amr graph (tag) (sec-2.2). ## related work doc-level eae is a frontier direction of event extraction and has received broad attention from industry and academia in recent years. unlike the well-developed sentence-level event extraction @xcite , the doc-level eae faces more challenges. @xcite proposes an end-to-end generation-based approach for doclevel eae. @xcite and @xcite construct an entity-based graph to model dependencies among the document. @xcite chooses the hierarchical method to aggregate information from different granularity. recently, there has been a rising trend of utilizing amr information to assist event extraction. @xcite employs node representations derived by amr graphs. @xcite and @xcite introduce amr path information as training signals to correct argument predictions. @xcite pre-trains the eae model with a contrastive loss built on amr graphs. however, previous works have only treated amr as an auxiliary feature or supervised signal and has not fully exploited the correlation between amr and eae. as the scheme of the amr graph is very similar to the event structure (predicate-arguments vs. trigger-arguments), eae can be reformulated as an amr-based problem. with tag, we can define eae as a task only related to graphs and conditionally independent of documents, thus achieving a simpler and more efficient model. previous works also explore the ways of enriching amr graphs to suit information extraction tasks. @xcite trains a learnable module to add nodes and edges to the amr graph. @xcite discusses different ways to integrate missing words with the amr graph. while these methods tend to enlarge amr graphs, causing a larger graph size and increasing the training difficulty, our method compresses the irrelevant information in amr to improve efficiency and help the model to be concentrated. ## conclusion we propose to reformulate document-level event argument extraction as a link prediction problem on our proposed tailored amr graphs. with adding missing spans, marking surrounding events, and removing noises, amr graphs are tailored to eae tasks. we also introduce a link prediction model based on tag to implement eae. elaborate exper-iments show that explicitly using amr graphs is beneficial for argument extraction.