KGE - CL : Contrastive Learning of Tensor Decomposition Based Knowledge Graph Embeddings (2022)

Abstract: Learning the embeddings of knowledge graphs (KG) is vital in artificial intelligence, and can benefit various downstream applications, such as recommendation and question answering. In recent years, many research efforts have been proposed for knowledge graph embedding (KGE). However, most previous KGE methods ignore the semantic similarity between the related entities and entity-relation couples in different triples since they separately optimize each triple with the scoring function. To address this problem, we propose a simple yet efficient contrastive learning framework for tensor decomposition based (TDB) KGE, which can shorten the semantic distance of the related entities and entity-relation couples in different triples and thus improve the performance of KGE. We evaluate our proposed method on three standard KGE datasets: WN18RR, FB15k-237 and YAGO3-10. Our method can yield some new state-of-the-art results, achieving 51.2% MRR, 46.8% Hits@1 on the WN18RR dataset, 37.8% MRR, 28.6% Hits@1 on FB15k-237 dataset, and 59.1% MRR, 51.8% Hits@1 on the YAGO3-10 dataset.

## introduction the knowledge graph (kg) stores a vast number of human knowledge in the format of triples. a triple (h, r, t) in a kg contains a head entity h, a tail entity t, and a relation r between h and t. the knowledge graph embedding (kge) aims to project the massive interconnected entities and relations in a kg into vectors or matrices, which can preserve the semantic information of the triples. learning the embeddings of kg can benefit various downstream artificial intelligence applications, such as question answering @xcite , machine reading comprehension @xcite , image classification @xcite , and personalized recommendation @xcite . in general, most of the kge methods would define a scoring function f (h i , r j , t k ), and the training target of kge is maximizing the score of a true triple (h i , r j , t k ) and minimizing the score of a false tripe (h i , r j , t x ). in this way, the trained embeddings of entities and relations in kg can preserve the intrinsic semantics of a true triple. we can mainly divide the existing kge methods into two categories. the first category is the distance based (db) methods, which use the minkowski distance as scoring function to measure a triple's plausibility, include the transe @xcite , transh @xcite , transr @xcite , transd @xcite and transg @xcite . the other category is the tensor decomposition based (tdb) methods, which treat a kg as a third-order binary tensor and use the results of tensor decomposition as the representations of entities and relations. the tdb methods include the cp @xcite , distmult @xcite , rescal @xcite and complex @xcite . however, existing methods only capture the semantic connections among h, r and t in a same triple. for example, the transe optimize the distance between h + r and t, and the distmult optimizie the dot product similarity among h, r and t. therefore, they overlook the connections between the related entities and entity-relation couples in different triples. in a kg, some entities (entity-relation couples) that share the same entityrelation couple (entity) are in the same type and have semantic similarity. capturing the semantic similarity of these entities or couples can improve the expressiveness of embeddings, which is the performance in capturing semantic information in kg @xcite ## experiment in this section, we present thorough empirical studies to evaluate and analyze our proposed framework. we first introduce the experimental setting. then we evaluate our framework's performance on the task of link prediction. besides, we further analyze the details of our promotion by comparing our method with a baseline on the triples with different relations, and we also study the effect of positive instances to our framework. finally, we visualize the embeddings of our method and some baselines to explain why our method outperforms baselines. ## conclusion and future work in this paper, we propose a simple yet efficient contrastive learning framework for tdb kge to improve its performance. compared with the previous work, our method can pull the related entities and entity-relation couples in different triples together in the semantic space and push the unrelated entities and couples apart. the experimental results on the standard datasets show that our method can achieve new state-of-the-art results. our analyses further verify the effectiveness of our approach. in the future, we plan to extend the critical insights of contrastive learning to distance based (db) kge methods and other representation learning problems in natural language processing.