# Learning Translation-Based Knowledge Graph Embeddings by \(N\)-Pair Translation Loss
## 3 Translation-Based Knowledge Graph Embeddings  
Translation-based embeddings project entities or/and relations between entities onto an embedding space by treating relations as translation operators of entities on the space. Assume that a knowledge graph \(\mathcal{G}=\{(h_{i},r_{i},t_{i})\}_{i=1}^{|\mathcal{G}|}\), which is represented as a set of triples (\(h\), \(r\), \(t\)), is given where \(h\), \(t\in\mathcal{E}\) and \(r\in\mathcal{R}\). Here, \(|\mathcal{G}|\) is the number of triples, \(\mathcal{E}\) is a set of entities, and \(\mathcal{R}\) is that of relations between entities. Then, translation-based knowledge graph embeddings find vector representations \(\mathbf{h}\), \(\mathbf{t}\), and \(\mathbf{r}\) for entities \(h\), \(t\) and relation \(r\) by enforcing \(\mathbf{t}\) to be a sum of \(\mathbf{h}\) and \(\mathbf{r}\). Among various translation-based knowledge graph embeddings, TransE [9] is the most representative and simplest method, which projects both entities and relations in the same vector space. That is, the vector representations \(\mathbf{h}\), \(\mathbf{t}\), and \(\mathbf{r}\) are represented in the vector space \(\mathbb{R}^{k}\), where \(k\) is a dimension of vector representations. Then, for every triple (\(h\), \(r\), \(t\)) in a knowledge graph \(\mathcal{G}\), TransE forces \(\mathbf{h}+\mathbf{r}\) to be close to \(\mathbf{t}\). This translation is represented by the following score function  
\[s(h,r,t)=||\mathbf{h}+\mathbf{r}-\mathbf{t}||_{\ell_{1}/\ell_{2}}, \tag{1}\]  
where \(\ell_{1}\) and \(\ell_{2}\) indicate L1 and L2 norm, respectively. Note that the smaller the score \(s(\cdot)\) is, the better the triple (\(h,r,t\)) is. To learn these vector representations, the margin-based ranking loss between a positive triple (\(h_{p}\),\(r_{p}\),\(t_{p}\)) and a negative triple (\(h_{n}\),\(r_{n}\),\(t_{n}\)) is defined as  
\[\mathcal{L}_{one} (\{(h_{p},r_{p},t_{p}),(h_{n},r_{n},t_{n})\})\] \[=[(s(h_{p},r_{p},t_{p})+\gamma-s(h_{n},r_{n},t_{n})])]_{+}, \tag{2}\]  
where \([x]_{+}=\max(0,x)\) and \(\gamma\) is a margin. Then, TransE minimizes the margin-based ranking risk over \(\mathcal{G}\) given as  
\[\mathcal{R}(\mathcal{G})=\frac{1}{|\mathcal{G}|}\sum_{(h,r,t)\in\mathcal{G}} \sum_{(h^{\prime},r,t^{\prime})\in\mathcal{G}^{\prime}(h,r,t)}\mathcal{L}_{ one}(\{(h,r,t),(h^{\prime},r,t^{\prime})\}). \tag{3}\]  
Here, \(\mathcal{G}^{\prime}\) is a set of negative or corrupted triples. This set is usually constructed artificially because a knowledge graph contains only correct triples. One simple way to make \(\mathcal{G}^{\prime}\) is to replace the entities in the positive or correct triples with another entities in \(\mathcal{E}\) following the closed-world assumption. That is, \(\mathcal{G}^{\prime}_{(h,r,t)}\) can be prepared by  
\[\mathcal{G}^{\prime}_{(h,r,t)}=\{(h^{\prime},r,t)\
otin\mathcal{G}\mid h^{ \prime}\in\mathcal{E}\}\cup\{(h,r,t^{\prime})\
ot\in\mathcal{G}\mid t^{\prime }\in\mathcal{E}\}.\]  
The learning of TransE is carried out by stochastic gradient descent (SGD) in minibatch mode over the possible triples in \(\mathcal{G}\). While optimizing the vector representations using Equation (3), some additional constraints can be enforced to embeddings of entities in order to avoid the embeddings from diverging unlimitedly. Several variants of TransE are obtained by changing the score function in Equation (1). For instance, TransR [12] maps the entity representations onto a different relation space for every relation. Thus, its score function is  
\[s(h,r,t)=||\mathbf{M}_{r}\mathbf{h}+\mathbf{r}-\mathbf{M}_{r}\mathbf{t}||_{\ell_ {1}/\ell_{2}},\]  
where \(\mathbf{M}_{r}\) is a projection matrix that projects entities from an entity space to the space of relation \(r\). Another is TransD [10], which maps entity representations onto different vectors in relation spaces according to entity and relation types.