# Learning Translation-Based Knowledge Graph Embeddings by \(N\)-Pair Translation Loss
## 1 Introduction  
Knowledge graph embedding aims at learning the representation of a knowledge graph by embedding the knowledge graph into a low dimensional vector space [1]. For a given knowledge graph expressed as a set of knowledge triples where each triple is composed of a relation (\(r\)) and two entities (\(h\) and \(t\)), knowledge graph embedding finds vector representations of \(h\), \(t\), and \(r\) by considering the structure of the knowledge graph. Since a knowledge graph is regarded as a key resource, various embedding models have been proposed [2; 3] and have been applied to a number of applications such as entity disambiguation [4], relation extraction [5; 6], and question answering [7; 8]. Translation-based knowledge graph embedding is one of the embedding models that finds vector representations of \(h\), \(t\), and \(r\) by compelling the vector of \(t\) to be close to the sum of the vectors of \(h\) and \(r\)[9]. Several variants have been proposed by modifying a score function to find better vector representations [10; 11; 12; 13; 14]. These translation-based embeddings are usually trained by minimizing the margin-based ranking loss over the knowledge graph. That is, the scores of positive triples are forced to be lower than those of negative triples. In order to adopt the margin-based ranking loss, negative triples against the knowledge graph are required, but knowledge graphs contain only positive triples. Thus, negative triples are prepared by replacing the head or the tail of positive triples randomly under the closed-world assumption [9]. Even if it looks simple to train the translation-based embeddings with the margin-based ranking loss, the embeddings suffer from slow convergence and poor local optima because margin-based ranking losses consider only one negative triple. Thus, most previous studies have focused mainly on the quality of negative triples to avoid the cases where negative triples contribute little to finding vector representations [15; 16; 17; 18]. That is, they have developed novel negative sampling methods to generate hard negative triples under the adversarial learning framework [19] or using the caching framework. However, these studies require extra modules, and thus the number of parameters to be optimized increases. Furthermore, the cost of sampling is expensive and would increase if the size of the knowledge graph is huge. If minibatch size is one while training translation-based knowledge graph embeddings with the margin-based ranking loss, the positive triple in the minibatch is compared with only one negative triple at a single update of learning parameters. On the other hand, if the positive triple is compared against multiple negative triples and is validated from all the negative triples at each update, the learning model will converge faster and find better local optima. Therefore, this paper proposes a simple but effective learning method based on a new loss function to incorporate multiple negative triples in training translation-based embeddings. The new loss function is motivated by the \(N\)-pair loss [20] that optimizes identifying a positive example from \(N\)-\(1\) negative examples in metric learning. The proposed \(N\)_-pair translation loss_ takes one positive triple as well as multiple negative triples and optimizes not only minimizing the score of the positive triple to satisfy constraints of translation-based knowledge graph embeddings but also maximizing the differences between the score of the positive triple and those of the negative triples. Since the proposed method interacts with multiple negative triples in each update, it results in better vector representations rapidly. The effectiveness of the proposed method is verified through extensive experiments. The experimental results prove empirically that the translation-based embeddings by the proposed loss converge fast and produce better vector representations than those by the margin-based ranking loss. The rest of this paper is organized as follows.