Graph Transformer Networks with Syntactic and Semantic Structures for Event Argument Extraction (2020)

Abstract: The goal of Event Argument Extraction (EAE) is to find the role of each entity mention for a given event trigger word. It has been shown in the previous works that the syntactic structures of the sentences are helpful for the deep learning models for EAE. However, a major problem in such prior works is that they fail to exploit the semantic structures of the sentences to induce effective representations for EAE. Consequently, in this work, we propose a novel model for EAE that exploits both syntactic and semantic structures of the sentences with the Graph Transformer Networks (GTNs) to learn more effective sentence structures for EAE. In addition, we introduce a novel inductive bias based on information bottleneck to improve generalization of the EAE models. Extensive experiments are performed to demonstrate the benefits of the proposed model, leading to state-of-the-art performance for EAE on standard datasets.

## introduction event extraction (ee) is an important task of information extraction that aims to recognize events and their arguments in text. in the literature, ee is often divided into two sub-tasks: (1) event detection (ed) to detect the event trigger words, and (2) event argument extraction (eae) to identity the event arguments and their roles for the given event triggers. in recent years, ed has been studied extensively with deep learning while eae is relatively less explored @xcite . as eae is necessary to accomplish ee and helpful for many downstream applications @xcite , further studies are required to improve the performance of eae. this work focuses on eae to meet this requirement for ee. the current state-of-the-art methods for eae have involved deep learning models that compute an abstract representation vector for each word in the input sentences based on the information from the other context words. the representation vectors for the words are then aggregated to perform eae @xcite . our main motivation in this work is to exploit different structures in the input sentences to improve the representation vectors for the words in the deep learning models for eae. in this work, a sentence structure (or view) refers to an importance score matrix whose cells quantify the contribution of a context word for the representation vector computation of the current word for eae. in particular, we consider two types of sentence structures in this work, i.e., syntactic and semantic structures. as such, the importance score for a pair of words in the syntactic structures is determined by the syntactic connections of the words in the dependency parsing trees while the contextual semantics of the words in the input sentences are exploited to compute the importance scores in the semantic structures. consider the following sentence as an example: iraqi press constantly report interviews with hussain molem, the hanif bashir's son-in-law, while us officials confirmed all bashir's family members were killed last week. in this sentence, an eae system should be able to realize the entity mention "hussain molem" as the victim of the attack event triggered by "killed". as "hussain molem" and "killed" are far away from each other in the sentence as well as its dependency tree, the eae models might find it challenging to make the correct prediction in this case. in order for the models to be successful in this case, our intuition is that the models should first rely on the direct connections between "killed" and "all bashir's family members" in the dependency tree to capture the role of "all bashir's family members" in the representation vectors for "killed". afterward, the models can rely on the close semantic similarity between "all bashir's family members" and "the hanif bashir's son-in-law" to further connect "the hanif bashir's son-in-law" to "killed" so the role information of "the hanif bashir's son-in-law" can be recorded in the representation vector for "killed". finally, the direct apposition relation between "the hanif bashir's son-in-law" and "hussain molem" can be exploited to connect "hussain molem" with "killed" to obtain the necessary representations to perform argument prediction for "hussain molem". on the one hand, this example suggests that both syntactic and semantic structures are necessary for the eae models. on the other hand, the example also hints that it is not enough to apply the syntactic and semantic structures separately. in fact, these structures should be explicitly combined to complement each other on identifying important context words to obtain effective representations for eae. to our knowledge, this is the first work to explore syntactic and semantic structures for eae. how should we combine the syntactic and semantic structures to aid the learning of effective representations for eae? in this work, we propose to employ graph transformer networks (gtn) @xcite to perform the syntax-semantic merging for eae. gtns facilitate the combination of multiple input structures via two steps. the first step obtains the weighted sums of the input structures, serving as the intermediate structures that are able to capture the information from different input perspectives (i.e., structures). in the second step, the intermediate structures are multiplied to generate the final structures whose goal is to leverage the multi-hop paths/connections between a pair of nodes/words (i.e., involving the other words) to compute the importance score for the final structures. as the multi-hop paths with heterogeneous types of connections along the way (i.e., syntax or semantic) has been illustrated to be helpful in our running example (i.e., between 'hussain molem" and 'killed"), we expect that gtns can help to combine the syntactic and semantic structures to produce effective representations for eae. finally, in order to further boost the performance for eae, we propose a novel inductive bias for the proposed gtn model, aiming to improve the generalization of gtns using the information bottleneck idea @xcite . in particular, the use of the rich combined structures from syntax and semantics might augment gtns with high capacity to encode the detailed information in the input sentences. coupled with the generally small training datasets for eae, the gtn models could learn to preserve all the context information in the input sentences, including the irrelevant information for eae. this likely leads to the overfitting of gtns on the training data. in order to overcome this issue, we propose to treat the gtn model in this work as an information bottleneck in which the produced representations of gtns are trained to not only achieve good prediction performance for eae but also minimize the mutual information with the input sentences @xcite . to this end, we introduce the mutual information between the generated representations of gtns and the input sentences as an additional term in the overall loss function to improve the generalization of gtns for eae. our extensive experiments on two benchmark datasets for eae show that the proposed model can achieve the state-of-the-art performance for eae. ## related work eae is one of the two subtasks in ee (the other one is ed) that has been approached early by the feature-based models @xcite @xcite @xcite @xcite @xcite . the recent work on ee has focused on deep learning to improve the models' performance @xcite @xcite @xcite . among the two subtasks of ee, while ed has been studied extensively by the recent deep learning work @xcite @xcite @xcite @xcite @xcite , eae has been relatively less explored. the closest work to ours is @xcite ) that focuses on eae and exploits the concept hierarchy of event argument roles to perform the task. our work differs from @xcite in that we employ the syntactic and semantic structures of the sentences to better learn the representations for eae. we also note some new directions on ee based on zero-shot learning @xcite , few-shot learning @xcite and multimodal learning @xcite .