E vent KE : Event-Enhanced Knowledge Graph Embedding (2021)

Abstract: Relations in most of the traditional knowledge graphs (KGs) only reflect static and factual connections, but fail to represent the dynamic activities and state changes about entities. In this paper, we emphasize the importance of incorporating events in KG representation learning, and propose an event-enhanced KG embedding model EventKE. Specifically, given the original KG, we first incorporate event nodes by building a heterogeneous network, where entity nodes and event nodes are distributed on the two sides of the network inter-connected by event argument links. We then use entity-entity relations from the original KG and event-event temporal links to inner-connect entity and event nodes respectively. We design a novel and effective attention-based message passing method, which is conducted on entity-entity, event-entity, and event-event relations to fuse the event information into KG embeddings. Experimental results on real-world datasets demonstrate that events can greatly improve the quality of the KG embeddings on multiple downstream tasks.

## introduction knowledge graph (kg) is a kind of efficient and informative representation of structured knowledge. a typical kg consists of a collection of knowledge triples, where each triple (h, r, t) describes that the head entity h and tail entity t are connected through a relation r. recently, extensive studies have been focusing on knowledge graph representation learning, which aims to learn low-dimensional entity and relation embeddings that are informative and scalable to use for many downstream applications, such as information retrieval @xcite , recommendation systems @xcite , machine reading comprehension @xcite , and query-answering systems @xcite 1 data and source codes are made publicly available at https://github.com/zhangzx-uiuc/eventke . kobe bryant lakers citizen los angeles u.s. a helicopter located awarded for films air crash winning championship league lead player place crashed aircraft team nba league winning team oscar award passenger winner before after before after entities events relations event arguments ## problem definition a knowledge graph is denoted by @xmath0, e), where v @xmath1, j) ∈ e denotes the relation between the ith entity node and the j-th entity node. we use r i,@xmath2, j), such as the relation types team, league, and citizen in figure 1 , where r represents the predefined relation type category. such a knowledge graph g can also be represented by a collection of decomposed knowledge triples to model the events, we treat events as an additional set of event nodes with event temporal relations. we use e to denote the set of event nodes, where each event e @xmath3, its event type c @xmath4, and a set of event arguments a j . the event trigger t j is a word or phrase from the original sentence that most clearly indicates the event occurrence, e.g., "wins" for an event winning championship, which is already included when the events are annotated or extracted from the original texts. the event type c j is from a pre-defined event type category c, such as the winning championship and awarded for films for the first two event nodes in figure 1 respectively. the set of event arguments a j is composed of a set of pairs of entities and event argument roles. the objective of event-enhanced kg embedding is to learn a low-dimensional representation vector v i for each node v @xmath5. ## related work knowledge graph embedding the most traditional translation-based kg embedding models are first proposed by @xcite , which manage to model each knowledge triple as translation and train the model by minimizing the distance function between head entity, relation, and tail entity. previous translation-based models are mainly diverse in how to model the distance for each triple. typical methods include using relation hyperplanes @xcite , projection matrix decomposition @xcite , relational mappings @xcite , multiple relation projections @xcite , gaussian distributions @xcite , neural networks @xcite , 2d convolutions @xcite , and rotations @xcite . however, such translation-based methods model each knowledge triple individually without considering the global context for entities in the kg. to solve this problem, another line of work attempts to model the kg as a unified heterogeneous graph and use graph neural networks to capture the global interactions between entity nodes, such as @xcite @xcite @xcite . recent studies also focus on using additional information to improve kg embedding, such as entity type information @xcite ) and text information @xcite . ## conclusion and future work relations in traditional kgs usually reflect only static and factual connections, which fail to incorporate dynamic activities and state changes about entities. in this paper, we focus on incorporating events into kg representation learning, and building a heterogeneous network by introducing an additional event node layer. we design a novel and effective attention-based message passing method on entity-entity, event-entity, and event-event relations, to fuse the event information into kg embeddings. experimental results on multiple evaluation tasks show the compelling effectiveness of incorporating events in kgs. for future work, we will consider applying eventke to more challenging event-related tasks, such as event schema induction. in addition, using pretrained language models