C og KGE : A Knowledge Graph Embedding Toolkit and Benchmark for Representing Multi-source and Heterogeneous Knowledge (2022)

Abstract: In this paper, we propose CogKGE, a knowledge graph embedding (KGE) toolkit, which aims to represent multi-source and heterogeneous knowledge. For multi-source knowledge, unlike existing methods that mainly focus on entity-centric knowledge, CogKGE also supports the representations of event-centric, commonsense and linguistic knowledge. For heterogeneous knowledge, besides structured triple facts, CogKGE leverages additional unstructured information, such as text descriptions, node types and temporal information, to enhance the meaning of embeddings. Designing CogKGE aims to provide a unified programming framework for KGE tasks and a series of knowledge representations for downstream tasks. As a research framework, CogKGE consists of five parts, including core, data, model, knowledge and adapter module. As a knowledge discovery toolkit, CogKGE provides pre-trained embedders to discover new facts, cluster entities and check facts. Furthermore, we construct two benchmark datasets for further research on multi-source heterogeneous KGE tasks: EventKG240K and CogNet360K. We also release an online system to discover knowledge visually. Source code, datasets and pre-trained embeddings are publicly available at GitHub, with a short instruction video.

## introduction in recent years, knowledge graphs (kgs) have experienced rapid development. a large number of kgs, such as framenet @xcite , wikidata @xcite , dbpedia @xcite and conceptnet @xcite , have been built and successfully applied to many real-world applications. most figure 1 : an example of a multi-source heterogeneous kg. grey, blue, green and purple denote entity-centric world knowledge, event-centric world knowledge, commonsense knowledge and linguistic knowledge, respectively. the dotted boxes show additional information. kgs are originally organized in the form of triples (h, r, t), where h and t indicate head and tail entities, and r indicates the relation between h and t. however, a kg is a symbolic system that cannot be directly applied to large-scale deep learning frameworks. to this end, a series of knowledge graph embedding (kge) models have been proposed to represent the entities and relations into continuous spaces @xcite @xcite @xcite . to facilitate the development of kge models, some remarkable kge toolkits, such as openke @xcite , graphvite @xcite , libkge @xcite , pykeen @xcite and pykg2vec @xcite have been released, providing easy-to-use frameworks for a series of kge models. however, most of them perform the embedding task solely based on entityrelated triple facts, so they are still limited to two critical challenges in practical applications: multisource challenge and heterogeneous challenge. as to the multi-source challenge, real-world kgs involve not only world knowledge (including entity-centric knowledge and event-centric knowledge), but also linguistic knowledge and commonsense knowledge. in various practical applications, we need to use multi-source knowledge simultaneously. for example, as shown in figure 1 , to understand an article about "neil armstrong", we need (1) entity-centric world knowledge, e.g., "neil armstrong worked at nasa" from wikidata; as to the heterogeneous challenge, real-world kgs involve not only triple facts, but also additional information, such as text descriptions, node types and temporal information. in many practical applications, we should use these heterogeneous knowledge together. likewise, as shown in figure 1 , to understand an article about "neil armstrong", besides structured triple facts, we also need (1) text descriptions, e.g., "neil armstrong was the first human being to walk on the moon"; (2) node types, e.g., "neil armstrong is an astronaut"; (3) temporal information, e.g., "nell armstrong participated in apollo 11 moon landing from 1962 to 1971". all these heterogeneous knowledge can be used for obtaining the embeddings, but conventional kge models cannot take full advantage of the additional information mentioned above. therefore, it is highly desirable to have a toolkit that can bridge these heterogeneous knowledge by plug-and-play knowledge adapters. to solve the above two problems, we propose , a knowledge graph embedding toolkit that aims to represent multi-source and heterogeneous knowledge. the toolkit consists of five parts, including core module, data module, model module, adapter module and knowledge module. cogkge currently supports 17 models, 11 datasets, five evaluation metrics, four knowledge adapters, four loss functions, three samplers and three builtin data containers. besides, we also construct two large-scale benchmark datasets to promote the research on kge. in summary, the main features and contributions are as follows: cogkge provides a programming framework for kge tasks. based on the extensible architecture, cogkge can meet the requirements of module extension and secondary development, and pre-trained knowledge embeddings can be directly applied to downstream tasks. ## system architecture the overall system architecture of cogkge is presented in figure 2 . the top part is composed of the core module and data module. the former is the basis of the toolkit, while the latter provides fundamental data containers, loaders and processors. the bottom part is built upon the top part, the model module contains lots of built-in models, the knowledge module integrates multi-source and heterogeneous knowledge, and the adapter module acts as a bridge between the two. in the following, we will cover these five modules in detail. ## system usage our goal of designing cogkge is to provide a unified research framework for kge tasks and pretrained knowledge representations for downstream tasks. in this section, we show a detailed guideline on how to use our toolkit. ## evaluation benchmark to evaluate the kge models on large-scale multisource heterogeneous kgs, we construct two new benchmark datasets: eventkg240k and cognet360k. in this section, we introduce our datasets and conduct evaluations for classic models included in cogkge. ## conclusion in this paper, we propose cogkge, a knowledge graph embedding toolkit and benchmark for representing multi-source and heterogeneous knowledge. for multi-source knowledge, cogkge explores the unified representation of world, commonsense and linguistic knowledge. for heterogeneous knowledge, cogkge incorporates the structured and unstructured knowledge to enhance the meaning of embeddings. so far, we have implemented 17 classic kge models. besides nine public datasets, we also release two new benchmark datasets for further evaluating kge models. moreover, owing to the extensible and modularized architecture, cogkge is not only a kge research framework, but also a knowledge discovery library. besides the toolkit, we also release an online system to discover knowledge visually. in the future, more models, benchmark datasets, and knowledge adapters will be incorporated into cogkge. b eventkg240k statistics type train validation test nodes 238,911 28,844 28,848 relations 822 289 301 event-event 219,128 1,389 1,427 event-entity 1,121,106 9,715 9,731 entity-entity 953,774 8,874 8,842 all triples 2,294,008 19,978 20,000 in this section, we provide more details of our even-tkg240k. as shown in