Multilingual S ub E vent Relation Extraction: A Novel Dataset and Structure Induction Method (2022)

Abstract: Subevent Relation Extraction (SRE) is a task in Information Extraction that aims to recognize spatial and temporal containment relations between event mentions in text. Recent methods have utilized pre-trained language models to represent input texts for SRE. However, a key issue in existing SRE methods is the employment of sequential order of words in texts to feed into representation learning methods, thus unable to explicitly focus on important context words and their interactions to enhance representations. In this work, we introduce a new method for SRE that learns to induce effective graph structures for input texts to boost representation learning. Our method features a word alignment framework with dependency paths and optimal transport to identify important context words to form effective graph structures for SRE. In addition, to enable SRE research on non-English languages, we present a new multilingual SRE dataset for five typologically different languages. Extensive experiments reveal the state-of-the-art performance for our method on different datasets and languages.

## introduction in information extraction (ie), events are defined as things that happen/occur @xcite or changes of state of real world entities @xcite . due to their complexity, a general event (i.e., superevent) can involve multiple other events with finer granularity (i.e., subevents) that can be altogether mentioned in text to present necessary details (e.g., a war can contain multiple attacks, which, in turn, can contain different bombing events). to this end, our paper studies the problem of subevent relation extraction (sre): given two event mentions in a document, a model needs to predict if one even is a part/subsevent of the other one. following previous work @xcite , our sre problem requires that a subevent relation * the first two authors contribute equally to this paper. to encode document context, existing models @xcite have leveraged pre-trained language models, i.e., roberta @xcite , to obtain representations for input documents for subevent prediction. however, an issue of existing sre methods is that they only rely on the sequential format of documents (i.e., sequence of sentences/words) for representation learning. on the one hand, the sequential format does not provide mechanisms to highlight the most important context words or avoid irrelevant ones in input documents, potentially introducing noisy information in the representations for sre. further, due to the sequential nature of input texts, current sre models cannot exploit effective structures/graphs that directly connect important context words to improve representation learning for sre. motivated by recent works on relation extraction between entities @xcite @xcite , one approach to improve sequential representation of input texts for sre can be based on dependency trees of sentences (i.e., graph-based structures) where dependency paths (dp) between two input entity mentions have been shown to capture important context words. in particular, to adapt this idea to document level with multiple sentences, @xcite obtains dependency trees for each sentence whose roots are linked together to obtain connected dependency graphs for input documents. afterward, the dependency graphs for documents are prune to preserve only the words along the dependency paths between two input mentions (called in-dp words) for representation learning. however, for our sre problem, important context words for subevent prediction can also be distributed outside the dependency paths, thus necessitating further techniques to identify other important words and connect them with the in-dp words to form better graph structures to represent input texts for sre. for example, in the input text "they implemented the proposal early last year. following the plan, the performers collected data and developed frameworks to monitor human trafficking for the first step of the proposal.", "developed" is a subevent of the "implemented" event for which the dp is "@xmath0. however, the word "proposal", which is important to connect "implemented" and "developed" to the same target for subevent recognition, is not included in the dp in this case. for convenience, we use non-dp words to refer to the words that do not belong to the dps between two input event mentions for sre. in previous work, in-dp words can be extended to find additional important context words for relation prediction by including non-dp words close to the dps in the dependency graphs @xcite (i.e., based on syntactic distances). as such, this method does not consider contextual semantics of the words that can provide richer information for important word selection for sre. to address this issue, we propose to leverage both syntactic and semantic evidences to determine the importance of a non-dp word for inclusion into the graph structure to represent input text for sre. for syntactic information, we expect a word to be more important for subevent prediction if it is closer to the input event mentions in the dependency graphs. in addition, for semantic information, our intuition is to promote non-dp words that are more similar/related to in-dp words contextually to enhance the induced representations for sre. however, combining syntactic and semantic similarities to compute overall importance scores to compare non-dp words is a non-trivial problem due to the different nature of the information. to this end, motivated by in-dp words as the anchors to induce graph structure representations for input texts, we propose to cast the problem of combining syntactic and semantic similarities to select important non-dp words into finding an optimal alignment between non-dp and in-dp words. a non-dp word is considered to be important for sre and retained in the induced graph structures for input texts if it is aligned with one of the in-dp words. in this way, our approach facilitates the application of optimal transport (ot) methods to effectively integrate syntactic and semantic information into a single joint optimization problem to obtain the optimal alignment for non-dp word selection for sre. in particular, to adapt to the goal of aligning two groups of points based on their transportation costs and distributions in ot, we will leverage semantic similarity to obtain transportation costs while syntactic distances in dependency graphs will be used to compute the distributions for in-dp and non-dp words to perform word alignment for sre. the resulting word alignment will then be used to select important non-dp words and construct graph structures to learn representations for subevent prediction. we evaluate our method over hieve @xcite and intellgience community (ic) @xcite , popular public datasets for sre. however, an issue with prior datasets and methods for sre is that they are only developed and evaluated over english data. as such, a critical question for the generalization of sre methods to non-enlgish languages has not been explored in the literature. to address this issue, we further present a new multilingual dataset for sre (called msubevent) for five languages, i.e., english, danish, spanish, turkish, and urdu, to enable future research in multilingual learning for sre. our dataset follows the annotation guidelines in hieve to make it consistent with prior sre work, introducing a large sre dataset with more than 46k event mentions and 3.9k subevent relations for model development. we conduct extensive experiments over hieve and our new dataset msubevent to demonstrate the effectiveness of the proposed method with state-of-the-art performance for sre. our experiments cover both monolingual learning (i.e., training and test data are from the same language) and cross-lingual transfer learning evaluation (i.e., training and test data comes from different language), thus highlighting the generalization across languages of the proposed method for sre. to our knowledge, this is the first work that explores multilingual data and cross-lingual learning for sre. finally, we will publicly release the new msubevent dataset to provide baselines and resources for future research in this area. ## model following prior work @xcite , we utilize pairwise classification to formulate sre. given a document @xmath1, w 2 , . . . , w n ] (of n words) with w e 1 and w e 2 as two input event mentions/triggers, a sre model needs to classify the relation between w e 1 and w e 2 according to one of the three types for subevents, i.e., parent-child, child-parent, and norel. here, the norel type is to indicate no subevent relation. input encoding: in the first step, our model feeds the input document d into a pre-trained language model (plm), i.e., roberta @xcite , to obtain a representation vector v i for each word w @xmath2. here, we utilize the hidden vectors in the last transformer layer where vectors for the wordpieces in w i are averaged to compute v i . for convenience, let @xmath3, v 2 , . . . , v n be the sequence of representation vectors for the words in d. note that if the length of the input document exceeds the length limit in plms (i.e., 512 sub-tokens), we split the document into smaller segments to fit into the limit and run plm over each segment separately to obtain the representations in v . structure induction: as presented in the introduction, our method aims to transform the sequential format of d into a graph representation that can better capture important context and structures for representation learning for sre. motivated by the dependency path between w e 1 and w e 2 to capture important context for relation prediction @xcite , we first build a dependency graph t for d to initialize our graph construction process. in particular, we obtain dependency trees for the sentences in the document and connect the roots of the trees for consecutive sentences to create t . we leverage the trankit toolkit @xcite to generate dependency trees and ignore directions in the edges of the trees in the computation. as such, a property of the non-dp words in t is that they can involve both important and irrelevant context words for our subevent prediction problem (as demonstrated in the introduction). accordingly, to compute an effective graph structure for d for sre, our goal is to prune the dependency graph t so that only important context words are retained (i.e., removing irrelevant works). using in-dp words in t as the anchor (i.e., presumably with important context), we aim to further select non-dp words that involve important context to perform the pruning of t for sre. to this end, we propose to cast the non-dp word selection problem into an alignment problem between non-dp and in-dp words in which a non-dp word is considered as important for subevent prediction if it is aligned with one in-dp word in the alignment (i.e., extending the anchor in-dp words). to compute the alignment between the words for sre, we propose to model both syntactic and semantic similarities between non-dp and in-dp words where optimal transport (ot) @xcite is leveraged to facilitate the information combination for optimal alignment computation. optimal transport: ot is an established method to find the optimal plan to transform one distribution to another. given two distributions p(x) and q(y) over discrete domains x and y (respectively), and the cost function c(x, y) : x × @xmath4, ot finds the optimal joint alignment/distribution π * (x, y) (over x × y) with marginals p(x) and q(y), i.e., the cheapest transportation from p(x) to q(y), by solving the following problem: note that to obtain a hard alignment between data points x and y, we can align each row of π * (x, y) with the column with the highest probability, i.e., y * = argmax @xmath5, y) for all @xmath6. to adopt ot to solve our non-dp word selection problem, we propose to treat the in-dp words in t as the data points for domain y while the non-dp words will be used for domain x . as such, ot facilitates the integration of syntactic and semantic similarities into the computation of optimal alignment between in-dp and non-dp words by leveraging these information to compute the transformation cost function c(x, y) and the probability distributions p(x) and p(y). in particular, to compute p(x) and q(y) for @xmath7, we use syntactic distances of the words to the input event mentions. formally, for each word w @xmath8, we obtain the lengths of the paths that connect w i with the input event mentions w e 1 and w e 2 in the dependency graph t , i.e., d 1 i and d 2 i , respectively. 5561 the syntactic importance of w i for sre is then determined by syn(w i ) = max(d foot_0 i , d 2 i ). afterward, the distributions p(x) and p(y) can be obtained by normalizing the syntactic importance scores (with softmax) for the words in the corresponding sets of x and y. next, for the transportation cost c(x, y), we leverage the contextual semantics for the words x and y, measured by the euclidean distance between their representation vectors v x and v y (i.e., in v ), i.e., c(x, y) = ||v xv y ||. in addition, to aid the selection of non-dp important words, we introduce an extra data point, called nil, to the in-dp set y so non-dp words in x aligned with nil will be considered irrelevant and excluded from t for graph structure induction for sre. as such, the representation for nil is computed using average of the representation vectors of the in-dp words in y (i.e., to used for the transportation cost c(x, y)). also, we utilize the average syntactic importance scores for the words in x to serve as the syntactic score syn(nil) for nil (the distribution p(x) can be obtained accordingly). in this way, solving equation 1 returns the optimal alignment π * (x, y) that can provide hard alignment for the data points in x and y 1 . let i be the subset of non-dp words in x that are not aligned with nil in y according to π * (x, y) (i.e., irrelevant words). to this end, to prune the dependency graph t for sre, we can eliminate the words in i from t to produce a new graph that only involves induced important context words for subevent prediction. however, as the resulting graph might be disconnected, we further retain the words in the paths between any word in i and the input event mentions (i.e., w e 1 and w e 2 ), generating a new graph t ′ to serve as our induced graph structure to represent the input document for sre. in the next step, given the induced structure t ′ , we feed it into a graph convolutional network (gcn) (kipf and welling, 2017; nguyen and grishman, 2018) to learn richer representation vectors for the words in t ′ . the representation vectors from the plm (i.e., in v ) serve as the inputs for gcn. as such, the induced hidden vectors in the last layer of gcn are denoted by finally, we obtain an overall representation vector a for d for sre via the concatenation: ## data annotation there exist several datasets with subevent relation annotation, including hieve @xcite , ic @xcite , and red (o'gorman et al., 2016). however, these datasets are only annotated for english data, thus unable to evaluate the generalization of models across multiple languages. to better evaluate the proposed model and enable future research on multilingual sre, we introduce the first multilingual dataset (called msubevent) for sre that provides human annotation for five typological different languages, i.e., english, danish, spanish, turkish, and urdu. the rest of this sections describes our annotation schema, data collection, and annotation efforts. annotation scheme: a dataset for sre needs to provide annotations for two tasks, i.e., event mention and subevent relation extraction. as such, ## experiments datasets & hyper-parameters: similar to prior work @xcite @xcite , we evaluate our proposed model with optimal transport (called ot-sre) on the popular datasets for sre, i.e., hieve @xcite and intelligence community (ic) @xcite . in particular, hieve provides subevent and coreference relation annotation for events over 100 news articles using four relation labels, i.e., parent-child, child-parent (for subevents), coref (for coreference), and norel (for no relation). to make it comparable, we utilize the same data split and setting as the current work with best-reported performance for hieve @xcite , featuring 80 documents for training (2,423 subevent relations and 0.4 probability for down-sampling of negative examples) and 20 documents for testing (817 subevent relations). for ic, it also annotates 100 news articles for four subevent and coreference relations as in hieve. following the same setting in the current state-of-the-art method for ic @xcite , we discard relations with implicit event mentions and compute transitive closure for both subevent relations and coreference to obtain annotation for all event mention pairs as in hieve @xcite . also, ic is divided into three portions with 60/20/20 documents for training/development/test data respectively. in addition, we evaluate the sre models on the new multilingual dataset msubevent to provide baselines for future research. here, we randomly split the documents for each language in msubevent into three separate parts with a ratio of 3/1/1 for training, development, and test data (respectively). we will use msubevent to evaluate sre models in both monolingual and cross-lingual transfer learning experiments. we fine-tune the hyper-parameters for our ot-sre model over english development data of msubevent and apply the selected values for all experiments for consistency. in particular, the selected hyper-parameters for our model include: 2 layers for the gcn and feed-forward (i.e., f f ) models with 512 dimensions for the hidden vectors, 5e-5 for the learning rate with adam optimizer, and 16 for the batch size. finally, we utilize the the roberta base model @xcite to encode input texts for hieve as in prior work @xcite . for msubevent, we use the multilingual pre-trained language models (base versions), i.e., mbert @xcite and xlmr @xcite , for multilingual text encoding. baselines: for hieve, we compare our proposed sre model with the following baselines using the same data setting: structlr @xcite with feature engineering, tacolm @xcite with temporal common sense knowledge, joint @xcite with joint subevent and temporal relation extraction, eventseg @xcite with event-based text segmentation, and scs @xcite with selection of best context sentences for sre. similarly, for ic, we consider joint, eventseg, and scs for the baselines. note that scs and eventseg have the stateof-the-art (sota) performance for hieve and ic model f1 score pc cp avg hieve structlr @xcite 52.2 63.4 57.7 tacolm @xcite 48.5 49.4 48.9 joint @xcite 62.5 56.4 59.5 eventseg @xcite 58.6 57.9 58.3 scs @xcite 68.7 63.2 65.9 ot-sre (ours) 70.3 67.4 68.9 ic @xcite --26.2 joint @xcite 42.1 49.5 45.8 eventseg @xcite 44.6 51.6 48.1 scs @xcite 47.5 51.8 49.7 ot-sre (ours) 48.9 52.6 50.8 ## related work early methods for sre have exploited various contextual features for input texts (i.e., feature engineering) for machine learning models @xcite @xcite . to alleviate feature engineering, recent works have explored deep learning models to induce representations for sre from data, introducing joint inference with temporal relations @xcite and large plms @xcite @xcite . existing datasets for sre include hieve @xcite , ic @xcite @xcite . however, none of such methods and datasets considers graph structure induction for input texts and multilingual learning for sre as we do. regarding related work on event-event relation extraction, we also note recent studies for other types of relations between events, including causal @xcite @xcite , coreference @xcite @xcite , and temporal @xcite relations. finally, optimal transport has also been recently used to solve nlp problems @xcite ; however, none of previous work has employed ot for subevent relation extraction as we do. ## conclusion we present a novel method for subevent relation extraction that leverages optimal transport to induce effective graph structures for input texts to improve representation learning. the graph structure representation is able to directly capture important context words and their connections to facilitate sre. in addition, we introduce the first multilingual dataset for sre that provides human annotation for five languages with high quality. extensive experiments demonstrate the effectiveness of our method with state-of-the-art performance on different datasets and learning settings. our new dataset also offers ample opportunities for future research. in the future, we plan to extend our method and dataset to other event-event relations.