Abstract: Knowledge bases of real-world facts about entities and their relationships are useful resources for a variety of natural language processing tasks. However, because knowledge bases are typically incomplete, it is useful to be able to perform link prediction or knowledge base completion, i.e., predict whether a relationship not in the knowledge base is likely to be true. This paper combines insights from several previous link prediction models into a new embedding model STransE that represents each entity as a low-dimensional vector, and each relation by two matrices and a translation vector. STransE is a simple combination of the SE and TransE models, but it obtains better link prediction performance on two benchmark datasets than previous embedding models. Thus, STransE can serve as a new baseline for the more complex models in the link prediction task.

Introduction: Knowledge bases (KBs), such as WordNet BIBREF0 , YAGO BIBREF1 , Freebase BIBREF2 and DBpedia BIBREF3 , represent relationships between entities as triples $(\mathrm {head\ entity, relation, tail\ entity})$ . Even very large knowledge bases are still far from complete BIBREF4 , BIBREF5 . Link prediction or knowledge base completion systems BIBREF6 predict which triples not in a knowledge base are likely to be true BIBREF7 , BIBREF8 . A variety of different kinds of information is potentially useful here, including information extracted from external corpora BIBREF9 , BIBREF10 and the other relationships that hold between the entities BIBREF11 , BIBREF12 . For example, toutanova-EtAl:2015:EMNLP used information from the external ClueWeb-12 corpus to significantly enhance performance. While integrating a wide variety of information sources can produce excellent results BIBREF13 , there are several reasons for studying simpler models that directly optimize a score function for the triples in a knowledge base, such as the one presented here. First, additional information sources might not be available, e.g., for knowledge bases for specialized domains. Second, models that don't exploit external resources are simpler and thus typically much faster to train than the more complex models using additional information. Third, the more complex models that exploit external information are typically extensions of these simpler models, and are often initialized with parameters estimated by such simpler models, so improvements to the simpler models should yield corresponding improvements to the more complex models as well. Embedding models for KB completion associate entities and/or relations with dense feature vectors or matrices. Such models obtain state-of-the-art performance BIBREF14 , BIBREF8 , BIBREF15 , BIBREF16 , BIBREF4 , BIBREF17 , BIBREF18 and generalize to large KBs BIBREF19 . Table 1 summarizes a number of prominent embedding models for KB completion. Let $(h, r, t)$ represent a triple. In all of the models discussed here, the head entity $h$ and the tail entity $t$ are represented by vectors $\textbf {h}$ and $\textbf {t}\in \mathbb {R}^{k}$ respectively. The Unstructured model BIBREF15 assumes that $\textbf {h} \approx \textbf {t}$ . As the Unstructured model does not take the relationship $r$ into account, it cannot distinguish different relation types. The Structured Embedding (SE) model BIBREF8 extends the unstructured model by assuming that $h$ and $t$ are similar only in a relation-dependent subspace. It represents each relation $r$ with two matrices $h$0 and $h$1 , which are chosen so that $h$2 . The TransE model BIBREF16 is inspired by models such as Word2Vec BIBREF20 where relationships between words often correspond to translations in latent feature space. The TransE model represents each relation $h$3 by a translation vector r $h$4 , which is chosen so that $h$5 . The primary contribution of this paper is that two very simple relation-prediction models, SE and TransE, can be combined into a single model, which we call STransE. Specifically, we use relation-specific matrices $\textbf {W}_{r,1}$ and $\textbf {W}_{r,2}$ as in the SE model to identify the relation-dependent aspects of both $h$ and $t$ , and use a vector $\textbf {r}$ as in the TransE model to describe the relationship between $h$ and $t$ in this subspace. Specifically, our new KB completion model STransE chooses $\textbf {W}_{r,1}$ , $\textbf {W}_{r,2}$ and $\textbf {r}$ so that $\textbf {W}_{r,2}$0 . That is, a TransE-style relationship holds in some relation-dependent subspace, and crucially, this subspace may involve very different projections of the head $\textbf {W}_{r,2}$1 and tail $\textbf {W}_{r,2}$2 . So $\textbf {W}_{r,2}$3 and $\textbf {W}_{r,2}$4 can highlight, suppress, or even change the sign of, relation-specific attributes of $\textbf {W}_{r,2}$5 and $\textbf {W}_{r,2}$6 . For example, for the “purchases” relationship, certain attributes of individuals $\textbf {W}_{r,2}$7 (e.g., age, gender, marital status) are presumably strongly correlated with very different attributes of objects $\textbf {W}_{r,2}$8 (e.g., sports car, washing machine and the like). As we show below, STransE performs better than the SE and TransE models and other state-of-the-art link prediction models on two standard link prediction datasets WN18 and FB15k, so it can serve as a new baseline for KB completion. We expect that the STransE will also be able to serve as the basis for extended models that exploit a wider variety of information sources, just as TransE does.

Question: What datasets are used to evaluate the model?
Answer: WN18, FB15k