GENE : Global Event Network Embedding (2021)

Abstract: Current methods for event representation ignore related events in a corpus-level global context. For a deep and comprehensive understanding of complex events, we introduce a new task, Event Network Embedding, which aims to represent events by capturing the connections among events. We propose a novel framework, Global Event Network Embedding (GENE), that encodes the event network with a multi-view graph encoder while preserving the graph topology and node semantics. The graph encoder is trained by minimizing both structural and semantic losses. We develop a new series of structured probing tasks, and show that our approach effectively outperforms baseline models on node typing, argument role classification, and event coreference resolution.

## introduction understanding events is a fundamental human activity. our minds represent events at various granularity and abstraction levels, which allows us to quickly access and reason about related scenarios. a typical event mention includes an event trigger (the word or phrase that most clearly expresses an event occurrence) and its arguments (i.e., participants in events). the lexical embedding of a trigger is usually not sufficient, because the type of an event often depends on its arguments @xcite @xcite . for example, the support verb "get" may indicate a transfer.ownership event ("ellison to spend $10.3 billion to get his company.") or a movement.transport event ("airlines are getting flyers to destinations on time more often."). in figure 1 , the event type triggered by "execution" is life.die instead of project implementation. however, such kind of atomic event representation is still overly simplistic since it only captures local information and ignores related events in the global context. real-world events are inter-connected, as illustrated in the example in figure 1 . to have a comprehensive representation of the set fire event on an embassy, we need to incorporate its causes (e.g., the preceding execution event) and recent relevant events (e.g., the protests that happened before and after it). to capture these inter-event relations in a global context, we propose the following two assumptions. assumption 1. two events can be connected through the entities involved. on schema or type level, two event types can be connected through multiple paths and form a coherent story @xcite . this observation is also valid on instance level. for the example in figure 1 , one of the relations between the set fire event and the execution event is the blue path set fire, target, saudi embassy, affiliation, saudi arabia, agent, execution , which partially supports the fact that angry protesters revenge the death of nimr al-nimr against saudi arabia by attacking its embassy. this approximation for event-event relations lessens the problems of coarse classification granularity and low inter-annotation agreement (which may be as low as 20% as reported in @xcite ). hence, we propose to construct an event network, where each event node represents a unique instance labeled with its type, arguments, and attributes. these nodes are connected through multiple instantiated meta-paths @xcite consisting of their entity arguments and the entity-entity relations. these entities can be co-referential (e.g., two protests on different dates that both occur in tehran, iran) or involved in the same semantic relations (both protests targeted the saudi embassy, which is affiliated with the location entity "saudi arabia"). assumption 2. the representation of one event depends on its neighboring events in the event entity protest 1 a shiite protester execution set fire protest 2 t i t l e sheikh ayatollah islam p la c e saudi arabia p a r t -w h o l e saudi embassy 1 iran iranian protesters tehran saudi embassy 2 sana'a place d e m o n s t r a t o r affiliation af fil ia tio n victim a g e n t r e l i g i o n t a r g e t a t t a c k e r place n a t i o n a l i t y demonstrator p l a c e nationality part-whole r e l i g i o n nimr al-nimr protesters p a r t -w h o l e n a t i o n a l i t y event network. in figure 1 , a good representation of the set fire event should involve the execution event because the latter clarifies the grievance motivating the former. we further enrich event representations by introducing more context from the entire event network. compared with other methods to connect events (e.g., with eventevent relations @xcite @xcite @xcite ), our representation of each event grounded in an event network is semantically richer. based on these two hypotheses, we introduce a new task of event network embedding, aiming at representing events with low-dimensional and informative embeddings by incorporating neighboring events. we also propose a novel global event network embedding learning (gene) framework for this task. to capture network topology and preserve node attributes in the event representations, gene trains a graph encoder by minimizing both structural and semantic losses. to promote relational message passing with focus on different parts of the graph, we propose an innovative multi-view graph encoding method. we design event network structural probes, an evaluation framework including a series of structural probing tasks, to check the model's capability to implicitly incorporate event network structures. in this work, the learned node embeddings are intrinsically evaluated with node typing and event argument role classification tasks, and applied to the downstream task of event coreference resolution. experimental results on the augmented automatic content extraction (ace) dataset show that leveraging global context can significantly enrich the event representations. gene and its variants significantly outperform the baseline methods on various tasks. in summary, our contributions are: ## problem formulation event network. an event network with n nodes is a heterogeneous attributed network denoted as @xmath0, e}, where v and e are node and edge sets, respectively. each node v @xmath1, b i , s i , l @xmath2, where a i and b i are the start and end word indices in sentence s i , and l i is the node type label. each edge e @xmath3, j, l @xmath4, where i and j are indices of the involved nodes and l ij is the edge type label. in this work, we initialize the semantic representation of each node v i with an m-dimensional attribute vector x i derived from sentence context using a pretrained bert model @xcite . semantic proximity @xcite . given an event network @xmath5, e}, the semantic proximity of node v i and node v j is determined by the similarity of node attribute vectors x i and x j . if two nodes are semantically similar in the original space, they should stay similar in the new space. local neighborhood. given @xmath6, e}, the local (one-hop) neighborhood n i of node v i is defined as n @xmath7. for example, the local neighborhood of one event is composed of its argument entities. given event-entity node pairs, the task of argument role classification is to label the local neighborhood of events. global neighborhood. given @xmath8, e}, node v j belongs to the global (k-hop with @xmath9, if node v i can walk to node v j in k hops. for example, two events are 3hop neighbors when there is a path from one event to the other through two entity nodes. ## structural probes for event network as there is no existing work on comprehensive event representation evaluation, in this work we design an evaluation framework with a series of probing tasks to comprehensively evaluate the model's capability to capture network structures and preserve node attributes. structural probes are models trained to predict certain properties from inferred representations, and have been used to understand linguistic properties @xcite . the task of event network embedding requires the embedded distributional node representations to preserve semantic proximity, local neighborhood and global neighborhood. accordingly, we intrinsically evaluate the semantics preservation with node typing and assess the local neighborhood preservation with event argument role classification. we also apply the node embeddings to a downstream task, event coreference resolution, to extrinsically evaluate the global neighborhood preservation. node typing and event argument role classification are conducted under the same evaluation setting: given the learned node embeddings, predict the labels with a multi-layer perceptron (mlp) based classifier. if the input of the classifier is of different dimension to the event network embeddings, it will be first projected into the same dimension. the classifier is a two-layer feed-forward neural network with a linear transformation layer, a nonlinear activation operation, a layer normalization, a dropout operation, and another linear transformation layer. the classifier is designed to be simple on purpose so that it will be limited in reasoning ability and thus the evidence for classification will be mainly derived from the node embeddings. ## related work event representation. some previous efforts enrich event representations by introducing arguments @xcite @xcite @xcite @xcite @xcite , intent and sentiment @xcite , and temporal information @xcite . @xcite proposes a tensor-based event composition approach to combine a trigger and arguments to represent each event. we extend the definition of scenario to multiple inter-connected events. @xcite captures statistical dependencies between events but limits to script data sets where the events are naturally organized in sequential temporal order. our approach captures a rich variety of explicit semantic connections among complex events. @xcite learns distributed event representations using supervised multi-task learning, while our framework is based on unsupervised learning. network embedding. our work falls into the scope of unsupervised learning for heterogeneous attributed network embeddings. heterogeneous network embedding methods @xcite @xcite jointly model nodes and edges. attributed network embedding approaches @xcite on the other hand put focus on preserving node attributes when encoding the networks. event coreference resolution. most existing methods @xcite @xcite @xcite only exploit local features including trigger, argument and sentence context matching. to prevent error propagation, some models perform joint inference between event extraction and event coreference resolution @xcite @xcite or incorporate document topic structures @xcite . to the best of our knowledge our method is the first to leverage the entire event networks to compute similarity features. ## conclusions and future work we propose a novel continuous event representation called event network embedding to capture the connections among events in a global context. this new representation provides a powerful framework for downstream applications such as event coreference resolution and event ordering. in the future we aim to improve the ability to capture the long-distance relations in the graph encode by introducing event-event relation in the form of multiple meta-paths. the relations, or the event evolution patterns, extracted from large-scale corpora can guide event-related reasoning and act as shortcut linking event nodes. another direction is to explore a unified automatic evaluation benchmark for event representation.