Inducing Interpretability in Knowledge Graph Embeddings (2020)

Abstract: We study the problem of inducing interpretability in Knowledge Graph (KG) embeddings. Learning KG embeddings has been an active area of research in the past few years, resulting in many different models. However, most of these methods do not address the interpretability (semantics) of individual dimensions of the learned embeddings. In this work, we study this problem and propose a method for inducing interpretability in KG embeddings using entity co-occurrence statistics. The proposed method significantly improves the interpretability, while maintaining comparable performance in other KG tasks.

## introduction knowledge graphs such as freebase @xcite and nell @xcite have become important resources for supporting many ai applications like web search, q&a, etc. they store a collection of facts in the form of a graph. the nodes in the graph represent real world entities such as roger federer, tennis, united states etc while the edges represent relationships between them. these kgs have grown huge, but they are still not complete @xcite . hence the task of inferring new facts becomes important. kg embeddings have been a popular approach for this task as they can perform the inference task efficiently. this task has achieved significant attention in the literature and many methods have been proposed, such as, @xcite @xcite @xcite @xcite , etc. these methods learn representations for entities * contributed equally to the work. † this research was conducted during the authors' internships at indian institute of science, bangalore. and relations as vectors in a vector space, capturing global information about the kg. the task of kg inference is then defined as operations over these vectors. some of these methods like @xcite and @xcite are capable of exploiting additional text data apart from the kg, resulting in better representations. although these methods have shown good performance in the end task, they do not address the interpretability, i.e., understanding semantics of individual dimensions of the kg embedding. such representations enable a better understanding of the model and can be helpful for explaining a model's decision on an end application. in this work, we focus on incorporating interpretability in kg embeddings. specifically, we aim to learn interpretable embeddings for kg entities by incorporating additional entity co-occurrence statistics from text data. this work is motivated by @xcite who presented automated methods for evaluating topics learned via topic modelling methods. we adapt these methods for kg embedding models and propose a method to directly maximize them while learning kg embedding. as demonstrated by the experiments, we find that such modeling significantly improves interpretability, supporting our choice of using topic coherence for embedding dimensions. to the best of our knowledge, this work presents the first regularization term which induces interpretability in kg embeddings. ## related work several methods have been proposed for learning kg embeddings. they differ on the modeling of entities and relations, usage of text data and interpretability of the learned embeddings. we summarize some of these methods in following sections. ## proposed method the proposed method is motivated by a measure of coherence in topic modelling literature @xcite . this measure allows an automated evaluation of the quality of topics learned by topic modeling methods by using additional point-wise mutual information (pmi) for word pairs. it was also shown to have high correlation with human evaluation of topics. based on this measure of coherence, we propose a regularization term. this term can be used with existing kg embedding methods for inducing interpretability. it is described in the following sections. ## conclusion and future works in this work, we proposed a method for inducing interpretability in kg embeddings using a coherence regularization term. we evaluated the proposed and the baseline method on the interpretability of the learned embeddings. we also evaluated the methods on different kg tasks and compared their performance. we found that the proposed method achieves better interpretability while maintaining comparable performance on kg tasks. as next steps, we plan to evaluate and compare the generalizability of the proposed method across various kg embedding models. understanding the mapping between dimensions and latent categories could be another direction for future works.