Improving Cross-Lingual Transfer for Event Argument Extraction with Language-Universal Sentence Structures (2021)

Abstract: We study the problem of Cross-lingual Event Argument Extraction (CEAE). The task aims to predict argument roles of entity mentions for events in text, whose language is different from the language that a predictive model has been trained on. Previous work on CEAE has shown the cross-lingual benefits of universal dependency trees in capturing shared syntactic structures of sentences across languages. In particular, this work exploits the existence of the syntactic connections between the words in the dependency trees as the anchor knowledge to transfer the representation learning across languages for CEAE models (i.e., via graph convolutional neural networks – GCNs). In this paper, we introduce two novel sources of language-independent information for CEAE models based on the semantic similarity and the universal dependency relations of the word pairs in different languages. We propose to use the two sources of information to produce shared sentence structures to bridge the gap between languages and improve the cross-lingual performance of the CEAE models. Extensive experiments are conducted with Arabic, Chinese, and English to demonstrate the effectiveness of the proposed method for CEAE.

## introduction event argument extraction (eae) aims to classify argument roles of entity mentions for events in text. for example, given the sentence "he died of injuries from a grenade attack by a fellow soldier", the task requires systems to identify the entity mention "a fellow soldier" as the agent of the event die, which is triggered by the verb "died". eae is an important component of event extraction (ee) that has been extensively studied with different approaches @xcite @xcite @xcite @xcite @xcite . cross-lingual event argument extraction (ceae) is an instance of eae that considers the setting where test languages (i.e., target languages) are different from training languages (i.e., source languages). the goal is to transfer knowledge in source languages, where data is abundant, to low-resource target languages. the previous work on ceae @xcite has shown the existence of shared syntactic structures of sentences across languages, which are useful for cross-lingual transfer. in particular, with the multilingual word embeddings, @xcite develop a model based on graph convolutional networks (gcns) @xcite , which operates on universal dependency trees to capture the shared structures. notably, the use of the dependency trees of the sentences for gcns in @xcite essentially treats the existence of the syntactic connections between the words in the universal dependency trees as the language-universal knowledge that can be exploited to bridge the gap between languages for eae. in @xcite , such syntactic connection existences are formalized via the adjacency matrices a @xmath0,@xmath1..n of the dependency trees (i.e., n is the number of words in the input sentence and a dep @xmath2. we call a dep the syntax-based structures of the sentences for convenience (as a dep ij is based on the syntactic connection of the words). as such, in this work, we introduce two novel sources of information as the language anchors, which are complementary to the syntactic connections a dep , to enable gcns to learn better language-general representations for eae. the first source of information relies on the semantic similarities of the pairs of words in the input sentence to induce the semantic-based structures a @xmath3,@xmath4..n for gcns. the rationale for such semantic-based structures is that despite the vocabulary differences between languages, the semantic similarity of the words is a languageinvariant concept and can be leveraged to enhance the cross-lingual knowledge transfer for eae. in this work, we rely on the multilingual representation vectors of the words to facilitate such semantic similarity computation for a sem ij in different languages. for the second source of information, we propose to employ the syntactic dependency relations between the words (e.g., nsubj, conj) in the dependency trees to obtain the relation-based structures a @xmath5,@xmath6..n for eae with gcns. specifically, a @xmath7,@xmath8..n is an extension of a dep that further considers the natures (i.e., the relations) of the syntactic connections between the words (i.e., instead of using only the existence of the connections as in a dep ). similar to the semantic-based structures, we argue that the syntactic dependency relations from the universal dependency trees are also language-independent and can be helpful for our ceae problem. to this end, we employ the embeddings of the dependency relations to compute the relation-based structure scores a rel ij . note that all the structures a dep , a sem , and a rel are fed into gcn models for representation learning in this work. finally, we conduct extensive experiments to demonstrate the benefits of the proposed sentence structures, leading to the state-of-the-art performance for ceae with arabic, chinese, and english as the experiment languages. to our knowledge, this is the first work to examine semantic-based and relation-based structures for eae. cross-lingual transfer learning has also been examined for the other related tasks of eae, including multilingual relation extraction @xcite @xcite @xcite and semantic role labeling @xcite @xcite . however, none of these works explores edge-based attention gcn as we do. finally, our work is also related to the recent text structure models for other nlp tasks, including relation extraction @xcite , event factuality prediction @xcite , and text summarization @xcite . ## model we formalize eae as a multi-class classification problem. let @xmath9, w 2 , ..., w n be a sentence (of n words) with w t as the trigger word and w a as the argument candidate (i.e., an entity mention) (1 ≤ t, @xmath10. the goal of eae is to predict the role y * of w a for the event triggered by w t . following @xcite , we use the udpipe toolkit @xcite to obtain the universal dependency tree for w , the part of speech (pos) tags and bio entity type tags for the words in w . for convenience, let r be the set of universal dependency relations and e be the matrix for the embedding vectors of such dependency relations where e r indicates the embedding vector for @xmath11. in the first step for cross-lingual eae, each word w i in w is represented by the concatenation vector x i of three language-universal embedding vectors: ## conclusion and future work we introduce two novel sentence structures for cross-lingual eae with gcns based on the semantic similarity and the universal dependency relations of the words in the input sentences. the experiments demonstrate the benefits of the proposed sentence structures that lead to the state-ofthe-art performance for different experiments scenarios for ceae. in the future, we plan to apply the proposed model to other related tasks, e.g., crosslingual relation extraction @xcite . in addition, motivated by the recent introduction of high-performance multilingual nlp toolkits, e.g., trankit @xcite , we expect to extend our work to other languages to better demonstrate the benefits of the proposed models. finally, we will also explore the performance of our models when recent pre-trained multilingual language models, e.g., multilingual bert @xcite , are employed to encode input texts for different languages.