Original: T OWARD U NDERSTANDING T HE E FFECT OF L OSS F UNCTION ON T HE P ERFORMANCE OF K NOWLEDGE G RAPH 
Revised: T OWARDS U NDERSTANDING T HE E FFECT OF L OSS F UNCTION ON T HE P ERFORMANCE OF K NOWLEDGE G RAPH E

Original: E MBEDDING
Revised: MBEDDING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Knowledge graphs (KGs) represent world’s facts in structured forms.
Revised: Knowledge graphs (KGs) represent world’s facts in structured forms.

Original: KG completion exploits the existing facts in a KG to discover new ones.
Revised: KG completion exploits the existing facts in a KG to discover new ones.

Original: Translation-based embedding model (TransE) is a prominent formulation to do KG completion.
Revised: Translation-based embedding model (TransE) is a prominent formulation to do KG completion.

Original: Despite the efﬁciency of TransE in memory and time, it  suffers from several limitations in encoding relation patterns such as symmetric, reﬂexive etc.
Revised: Despite the efﬁciency of TransE in memory and time, it is claimed that TransE suffers from several limitations in encoding relation patterns (such as symmetric, reﬂexive relations etc).

Original: To resolve this problem, most of the attempts have circled around the revision of the  score function of TransE i.e., proposing a more complicated score function such as Trans(A, D, G, H, R, etc) to mitigate the limitations.
Revised: To solve that, most of the attempts have circled around the revision of the TransE score function , resulting more complicated score functions (such as TransA/D/G/H/R etc). These attempts have totally disregarded effect of loss functions to this end.

Original: In this paper, wetackle this problem from a different perspective. We show thatexisting theories corresponding to the  limitationsof TransE are inaccurate because they ignore the effect of  loss function.
Revised: We show that loss functions are key factors in this regard and disregarding them results in conclusions which are inaccurate or even wrong. More concretely, we show that the claimed limitations are inaccurate, as the effect of the loss was ignored.

Original: Accordingly, we pose theoretical investigations of the main limitations of TransE in the light of loss function.
Revised: In this regard, we pose theoretical investigations of the main limitations of TransE in the light of loss function .

Original: To the best of our knowledge,  this has not been investigated so far comprehensively.
Revised: To the best of our knowledge, so far, this has not been comprehensively investigated.

Original: We show that by a proper selection of the loss function for training the TransE model, the main limitationsof the model are mitigated.
Revised: We show that by a proper selection of the loss function for training the TransE model, the main limitations are mitigated.

Original: This is explained by setting upper-bound for the scores of positive samples, showing the region of truth (i.e., the region that a triple is considered positive by the model).
Revised: That is achieved by setting upper-bound for the scores of positive samples, deﬁning the region of truth (i.e. the region that a triple is considered positive by the model).

Original: Our theoretical proofs with experimental results ﬁll the gap between the capability of translation-based class of embedding models and the loss function. The theories emphasise the importance of the selection ofthe loss functions for training the models. Our experimental evaluations on different loss functionsused for training the modelsjustify our theoretical proofs andconﬁrm the importance of the loss functions on the performance.
Revised: Our theoretical proofs with experimental results ﬁll the gap in understanding of the limitation of translation-based class of embedding models and conﬁrm the importance of the selection of loss functions for training the models and on their performance.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Knowledge is considered as commonsense facts and other information accumulated from different sources.
Revised: Knowledge is considered as commonsense facts and other information accumulated from different sources.

Original: Throughout history, civilizations have evolved due to increase in the knowledge.

Original: With the passage of time, humans obtain many relations among different entities.

Original: Therefore, development of proper knowledge representation (KR) and management systems is essential.

Original: The aim of KR is to study how the beliefs can be represented in an explicit, symbolic notation proper to automated reasoning.

Original:  Knowledge Graph (KG) is a new direction for KR. KGs are usually represented as a set of triples ( h, r, t ) where h, t are entities and r is a relation, e.g. ( iphone, hyponym, smartphone ) .
Revised: A Knowledge Graph (KG) is collection of facts and is usually represented as a set of triples ( h, r, t ) where h, t are entities and r is a relation, e.g. ( iphone, hyponym, smartphone ) .

Original: Entities and relations are nodes and edges in the graph, respectively.
Revised: Entities and relations are nodes and edges in the graph, respectively.

Original:  KGs are inherently incomplete, making prediction of missing links always relevant.
Revised: As KGs are inherently incomplete, making prediction of missing links is a fundamental task in knowlege graph analyses.

Original: Among different approaches used for KG completion, KG Embedding (KGE) has recently received growing attentions.
Revised: Among different approaches used for KG completion, KG Embedding (KGE) has recently received growing attentions.

Original: KGE embeds entities and relations as low dimensional vectors.
Revised: KGE embeds entities and relations as low dimensional vectors known as embeddings.

Original: To measure the degree of plausibility of a triple, a scoring function is deﬁned over the embeddings.
Revised: To measure the degree of plausibility of a triple, a scoring function is deﬁned over the embeddings.

Original: TransE, Translation-based Embedding model, Bordes et al. (2013) is one of the most widely used
Revised: TransE, Translation-based Embedding model, (Bordes et al., 2013) is one of the most widely used

Original: KGEs.
Revised: KGE models.

Original: The original assumption of TransE is to hold: h + r = t , for every positive triple ( h, r, t )
Revised: The original assumption of TransE is to hold: h + r = t , for every positive triple ( h, r, t )

Original: where h , r , t ∈
Revised: where h , r , t ∈

Original: R
Revised: R

Original: d are embedding vectors of head ( h ), relation ( r ) and tail ( t ) respectively.
Revised: d are embedding vectors of head ( h ), relation ( r ) and tail ( t ) respectively.

Original: TransE and its many variants like TransH Wang et al. and TransR Lin et al. (2015b) , underperform greatly compared to the current state-of-the-art embedding models due to the inherent limitations of their scoring functions.
Revised: TransE and its many variants like TransH (Wang et al., 2014) and TransR (Lin et al., 2015b), underperform greatly compared to the current state-of-the-art models.

Original: Recent work has themain limitations of Translation-based models.
Revised: That is reported to be due to the limitations of their scoring functions.

Original: Wang et al. (2018) reveals that TransE cannot encode a relation pattern which is neither reﬂexive nor irreﬂexive.
Revised: For instance, (Wang et al., 2018) reports that TransE cannot encode a relation pattern which is neither reﬂexive nor irreﬂexive.

Revised: In most of these works the effect of the loss function is ignored and the provided proofs are based on the assumptions that are not fulﬁlled by the associated loss functions.

Original: Sun et al. (2019) prove that TransE is incapable of encoding symmetric relation.
Revised: For instance (Sun et al., 2019) proves that TransE is incapable of encoding symmetric relation.

Original: Wang et al.
Revised: To this end the loss function must enforce the distance of (cid:107) h + r − t (cid:107) to zero, but this is never fulﬁlled (or even approximated ) by the employed loss function.

Original: (2014) adds that TransE cannot properly encode reﬂexive, one-to-many, many-to-one and many-to-many relations.
Revised: Similarly, (Wang et al., 2018) reports that TransE cannot encode a relation pattern which is neither reﬂexive nor irreﬂexive and (Wang et al., 2014) adds that TransE cannot properly encode reﬂexive, one-to-many, many-to-one and many-to-many relations.

Revised: However, as mentioned earlier, such reported limitations are not accurate and the problem is not fully investigated due to the effect of the loss function.

Original:  TransH, TransR and TransD (Wang et al., 2014; Lin et al., 2015b; Ji et al., 2015) can handle the mentioned problems of TransE (i.e. one-to-many, many-to-one, many-to-many and reﬂexive) by projecting entities to relation space before applying translation.
Revised: In this regards, although TransH, TransR and TransD (Wang et al., 2014; Lin et al., 2015b; Ji et al., 2015) addressed the reported problem of TransE in one-to-many, many-to-one, many-to-many and reﬂexive etc, they were misled by the assumption (enforcing (cid:107) h

Revised: + r − t (cid:107) to be zero) that was not fulﬁlled by the employed loss function.

Original: However, Kazemi & Poole (2018) investigate three additional limitations of TransE, FTransE (Feng et al., 2016), STransE (Nguyen et al., 2016), TransH and TransR models: (i) if the models encode a reﬂexive relation r , they automatically encode symmetric, (ii) if the models encode a reﬂexive relation r , they automatically encode transitive and, (iii) if entity e 1 has relation r with every entity in ∆ ∈ E and entity e 2 has relation r with one of entities in ∆ , then e 2 must have the relation r with every entity in ∆ .
Revised: Considering the same assumption, (Kazemi & Poole, 2018) investigated three additional limitations of TransE, FTransE (Feng et al., 2016), STransE (Nguyen et al., 2016), TransH and TransR models: (i) if the models encode a reﬂexive relation r , they automatically encode symmetric, (ii) if the models encode a reﬂexive relation r , they automatically encode transitive and, (iii) if entity e 1 has relation r with every entity in ∆ ∈ E and entity e 2 has relation r with one of entities in ∆ , then e 2 must have the relation r with every entity in ∆ .

Original: The mentioned works have investigated these limitations by focusing on the capability of scoring functions in encoding relation patterns.
Revised: Assuming that the loss function enforces the norm to be zero, the aforementioned works have investigated these limitations by focusing on the capability of scoring functions in encoding relation patterns.

Original: However, we prove that the selection of loss function affects the boundary of score functions; consequently, the selection of loss functions signiﬁcantly affects the limitations.
Revised: However, we prove that the selection of loss function affects the boundary of score functions; consequently, the selection of loss functions signiﬁcantly affects the limitations.

Original: Therefore, the above mentioned theories corresponding to the limitations of translationbased embedding models in encoding relation patterns are inaccurate.
Revised: Therefore, the above mentioned theories corresponding to the limitations of translation-based embedding models in encoding relation patterns are inaccurate.

Original: We pose new theories about the limitations of TransX(X=
Revised: We pose new theories about the limitations of TransX(X=

Original: H,D,R, etc) models considering the loss functions.
Revised: H,D,R, etc) models considering the loss functions.

Original: To the best of our knowledge, it is the ﬁrst time that the effect of loss function is investigated to prove theories corresponding to the limitations of translation-based models.
Revised: To the best of our knowledge, it is the ﬁrst time that the effect of loss function is investigated to prove theories corresponding to the limitations of translation-based models.

Original: In a nutshell, the key contributions of this paper is as follows: (i)
Revised: In a nutshell, the key contributions of this paper is as follows. (i)

Original: We show that different loss functions enforce different upper-bounds and lower-bounds for the scores of positive and negative samples respectively.
Revised: We show that different loss functions enforce different upper-bounds and lower-bounds for the scores of positive and negative samples respectively.

Original: This implies that existing theories corresponding the limitation of TransX models are inaccurate because the effect of loss function is ignored.
Revised: This implies that existing theories corresponding the limitation of TransX models are inaccurate because the effect of loss function is ignored.

Original: We introduce new theories accordingly and prove that the proper selection of loss functions mitigates the main limitations.
Revised: We introduce new theories accordingly and prove that the proper selection of loss functions mitigates the main limitations.

Original: (ii)
Revised: (ii)

Original: We reformulate the existing loss functions and their optimization problems as an standard constrained optimization problem.
Revised: We reformulate the existing loss functions and their optimization problems as an standard constrained optimization problem.

Original: This makes perfectly clear that how each of the loss functions affect on the boundary of triples scores and consequently ability of relation pattern encoding.
Revised: This makes perfectly clear that how each of the loss functions affect on the boundary of triples scores and consequently ability of relation pattern encoding.

Original: (iii) using symmetric relation patterns, we obtain the optimal upper-bound of positive triples score to enable encoding of symmetric patterns.
Revised: (iii) Using symmetric relation patterns, we obtain a proper upper-bound of positive triples score to enable encoding of symmetric patterns.

Original: (iv) We additionally investigate the theoretical capability of translation-based embedding model when translation is applied in  complex space (TransComplEx).
Revised: (iv) We additionally investigate the theoretical capability of translation-based embedding model when translation is applied in the complex space (TransComplEx).

Original: We show that
Revised: We show that

Original: TransComplEx is a more powerful embedding model with fewer theoretical limitations in encoding different relation patterns such as symmetric while it is efﬁcient in memory and time.
Revised: TransComplEx is a more powerful embedding model with fewer theoretical limitations in encoding different relation patterns such as symmetric while it is efﬁcient in memory and time.

Original:  R ELATED W ORKS
Revised:  R ELATED W ORKS

Original: Most of the previous work have investigated the capability of translation-based class of embedding models considering solely the formulation of the score function.
Revised: Most of the previous work have investigated the capability of translation-based class of embedding models considering solely the formulation of the score function.

Original: Accordingly, in this section, we review the score functions of TransE and some of its variants together with their capabilities.
Revised: Accordingly, in this section, we review the score functions of TransE and some of its variants together with their capabilities.

Original: Then, in the next section the existing limitations of Translation-based embedding models emphasized in recent works are reviewed.
Revised: Then, in the next section the existing limitations of Translation-based embedding models emphasized in recent works are reviewed.

Original: These limitations will be reinvestigated in the light of score and loss functions in the section 4.
Revised: These limitations will be reinvestigated in the light of score and loss functions in the section 4.

Original: TransE (Bordes et al., 2013) is one of the earlier KGE models which is efﬁcient in both time and space.

Original: The scorefunction of TransE  is deﬁned as: f r ( h, t )
Revised: The score of TransE (Bordes et al., 2013) is deﬁned as: f r ( h, t )

Original: = (cid:107) h
Revised: = (cid:107) h

Original: + r − t (cid:107) .
Revised: + r − t (cid:107) .

Original: TransH (Wang et al., 2014) projects each entity ( e ) to the relation space ( e ⊥ =
Revised: TransH (Wang et al., 2014) projects each entity ( e ) to the relation space ( e ⊥ =

Original: e − w r ew Tr ).
Revised: e − w r ew Tr ).

Original: The score function is deﬁned as f r ( h, t ) =
Revised: The score function is deﬁned as f r ( h, t ) =

Original: (cid:107) h ⊥ + r − t ⊥ (cid:107) .
Revised: (cid:107) h ⊥ + r − t ⊥ (cid:107) .

Original: TransH can encode reﬂexive, one-to-many, many-to-one and many-to-many relations.
Revised: TransH can encode reﬂexive, one-to-many, many-to-one and many-to-many relations.

Original: However, recent theories (Kazemi & Poole, 2018) prove that encoding reﬂexive results in encoding the both symmetric and transitive which is undesired.
Revised: However, recent theories (Kazemi & Poole, 2018) prove that encoding reﬂexive results in encoding the both symmetric and transitive which is undesired.

Original: TransR (Lin et al., 2015b) projects each entity ( e ) to the relation space by using a matrix provided for each relation ( e ⊥ = eM r , M
Revised: TransR (Lin et al., 2015b) projects each entity ( e ) to the relation space by using a matrix provided for each relation ( e ⊥ = eM r , M

Original: r ∈ R d e × d r ).
Revised: r ∈ R d e × d r ).

Original: TransR uses the same scoring function as TransH. TransD (Ji et al., 2015) provides two vectors for each individual entities and relations ( h , h p , r , r p , t , t p ).
Revised: TransR uses the same scoring function as TransH. TransD (Ji et al., 2015) provides two vectors for each individual entities and relations ( h , h p , r , r p , t , t p ).

Original: Head and tail entities are projected by using the following matrices: 
Revised: Head and tail entities are projected by using the following matrices: M rh = r

Revised: Tp h p

Original: [Equation]
Revised: + I m × n , M rt = r Tp t p + I m × n .

Original: The score function of TransD is similar to the score function of TransH.
Revised: The score function of TransD is similar to the score function of TransH.

Original: RotatE (Sun et al., 2019) rotates the head to the tail entity by using relation.
Revised: RotatE (Sun et al., 2019) rotates the head to the tail entity by using relation.

Original: RotatE embeds entities and relations in  Complex space.
Revised: RotatE embeds entities and relations in the Complex space.

Original: By inclusion of constraints on the norm of entity vectors, the model would be degenerated to TransE.
Revised: By inclusion of constraints on the norm of entity vectors, the model would be degenerated to TransE.

Original: The scoring function of RotatE is f r ( h, t )
Revised: The scoring function of RotatE is f r ( h, t )

Original: = (cid:107) h ◦ r − t (cid:107) , where h , r , t ∈ C d , and ◦ is element-wise product.
Revised: = (cid:107) h ◦ r − t (cid:107) , where h , r , t ∈ C d , and ◦ is element-wise product.

Original: RotatE obtains the state-of-the-art results using very big embedding dimension (1000) and a lot of negative samples (1000).
Revised: RotatE obtains the state-of-the-art results using very big embedding dimension (1000) and a lot of negative samples (1000).

Original: TorusE (Ebisu & Ichise, 2018) ﬁxes the problem of regularization in TransE by applying translation on a compact Lie group.
Revised: TorusE (Ebisu & Ichise, 2018) ﬁxes the problem of regularization in TransE by applying translation on a compact Lie group.

Original: The model has several variants including mapping from torus to Complex space.
Revised: The model has several variants including mapping from torus to Complex space.

Original: In this case, the model is regarded as a very special case of RotatE Sun et al.
Revised: In this case, the model is regarded as a very special case of RotatE Sun et al.

Original: (2019) that applies rotation instead of translation in the target  Complex space.
Revised: (2019) that applies rotation instead of translation in the target the Complex space.

Original: According to Sun et al.
Revised: According to Sun et al.

Original:  TorusE is not deﬁned on the entire Complex space.
Revised: (2019), TorusE is not deﬁned on the entire Complex space.

Original: Therefore, it has less representation capacity.
Revised: Therefore, it has less representation capacity.

Original: TorusE needs a very big embedding dimension (10000 as reported in Ebisu & Ichise (2018)) which is a limitation.
Revised: TorusE needs a very big embedding dimension (10000 as reported in Ebisu & Ichise (2018)) which is a limitation.

Original:  T HE M AIN L IMITATIONS O F T RANSLATION - BASED E MBEDDING MODELS
Revised:  T HE M AIN L IMITATIONS O F T RANSLATION - BASED E MBEDDING MODELS

Original: Here we reviewthe six limitations of translation-based embedding models in encoding relation patterns (e.g., reﬂexive, symmetric) mentioned in the literature: Wang et al. Kazemi & Poole (2018); Wang et al. ; Sun et al.
Revised: We review six limitations of translation-based embedding models in encoding relation patterns (e.g. reﬂexive, symmetric) mentioned in the literature (Wang et al., 2014; Kazemi & Poole, 2018; Wang et al., 2018; Sun et al., 2019).

Original: Limitation L1 :
Revised: Limitation L1 .

Original: Wang et al. TransE cannot encode reﬂexive relations when  relation vector is non-zero.
Revised: TransE cannot encode reﬂexive relations when the relation vector is non-zero (Wang et al., 2014).

Original: Limitation L2 Wang et al.
Revised: Limitation L2 .

Original: (2018): if TransE encodes a relation r , which is neither reﬂexive nor irreﬂexive the following equations should be held simultaneously: h 1 + r = h 1 , h 2 + r (cid:54) =
Revised: TransE cannot encode a relation which is neither reﬂexive nor irreﬂexive. To see that, if TransE encodes a relation r , which is neither reﬂexive nor irreﬂexive we have h 1 + r = hand h 2 + r (cid:54) =

Original: h 2 . Therefore, both r = 0 , r (cid:54) = 0 should be held,{+
+} which result in contradiction.
Revised: h 2 , resulting r = 0 , r (cid:54) = {+
+} which is a contradiction (Wang et al., 2018).

Original: In this regard, TransE cannot encode a relation which is neither reﬂexive nor irreﬂexive.

Original: Limitation L3 Sun et al.
Revised: Limitation L3 .

Original: If relation  r is symmetric, the following equations should be held: h + r = t and t + r = h .
Revised: TransE cannot properly encode symmetric relation when r (cid:54) = 0 . To see that (Sun et al., 2019), if r is symmetric, then we have: h + r = t and t + r = h .

Original: Therefore, r = 

Revised: Therefore, r =

Original: and so all entities appeared in head or tail parts of training triples will have the same embedding vectors which is undesired.
Revised: and so all entities appeared in head or tail parts of training triples will have the same embedding vectors.

Original: Therefore, TransE cannot properly encode symmetric relation when r (cid:54) = 0 .

Original: The following limitations are held for TransE, FTransE Feng et al.
Revised: The following limitations hold for TransE, FTransE, STransE, TransH and TransR (Feng et al., 2016;

Original: , STransE Nguyen et al. (2016), TransH and TransR.
Revised: Nguyen et al., 2016; Kazemi & Poole, 2018):

Original: Limitation L4 Kazemi & Poole (2018): if a relation r is reﬂexive on ∆ ∈ E , where E is the set of all entities in the KG,  r must also be symmetric.
Revised: Limitation L4 . If r is reﬂexive on ∆ ∈ E , where E is the set of all entities in the KG, then r must also be symmetric.

Original: Limitation L5 Kazemi & Poole (2018): if r is reﬂexive on ∆ ∈ E , r must also be transitive.
Revised: Limitation L5 . If r is reﬂexive on ∆ ∈ E , r must also be transitive.

Original: Limitation L6 Kazemi & Poole (2018): if entity e 1 has relation r with every entity in ∆ ∈ E and entity e 2 has relation r with one of entities in ∆ , then e 2 must have the relation r with every entity in ∆ .
Revised: Limitation L6 . If entity e 1 has relation r with every entity in ∆ ∈ E and entity e 2 has relation r with one of entities in ∆ , then e 2 must have the relation r with every entity in ∆ .

Original:  O UR M ODEL
Revised:  O UR M ODEL

Original: TransE and its variants underperform compared to other embedding models due to their limitations we iterated in Section 3.
Revised: TransE and its variants underperform compared to other embedding models due to their limitations we iterated in Section 3.

Original: In this section, we reinvestigate the limitations.
Revised: In this section, we reinvestigate the limitations.

Original: We show that the corresponding theoretical proofs are inaccurate because the effect of loss function is ignored.
Revised: We show that the corresponding theoretical proofs are inaccurate because the effect of loss function is ignored.

Original: So we propose new theories and prove that each of the limitations of TransE are resolved by revising either the scoring function or the loss .
Revised: So we propose new theories and prove that each of the limitations of TransE are resolved by revising either the scoring function or the loss .

Original: In this regard, we consider several loss functions and their effects on the boundary of the TransE scoring function.
Revised: In this regard, we consider several loss functions and their effects on the boundary of the TransE scoring function.

Original: For each of the loss functions, we pose theories corresponding to the limitations.
Revised: For each of the loss functions, we pose theories corresponding to the limitations.

Original: we additionally investigate the limitations of TransE using each of the loss functions while translation is performed in Complex space (TransComplEx).
Revised: We additionally investigate the limitations of TransE using each of the loss functions while translation is performed in the complex space and show that by this new approach the aforementioned limitations are lifted.

Original: TransComplEx with a proper selection of loss function further mitigates the limitations as we discuss as follows.
Revised: Our new model, TransComplEx, with a proper selection of loss function addresses the above problems.

Original: 4. T RANS C OMPL E X : T RANSLATIONAL E MBEDDING M ODEL IN  C OMPLEX S PACE
Revised: 4. T RANS C OMPL E X : T RANSLATIONAL E MBEDDING M ODEL IN THE C OMPLEX S PACE

Original: Inspired by Trouillon et al. (2016), in this section we propose TransComplExthat translates head entity vector to the conjugate of  tailentity vector using relationvector in Complex space.
Revised: TransComplEx translates head entity vector to the conjugate of the tail vector using relation in the complex space (Trouillon et al., 2016).

Original: The score function is deﬁned as follows:
Revised: Assuming h , r , t ∈ C d be complex vectors of dimension d , the score function is deﬁned as follows:

Original: [Equation]
Revised: [Equation]

Original: where h , r , t ∈ C d are complex vectors i.e., each elements of the vectors is a complex number.

Original: For example, the i -th element of the vector h is denoted by h i =

Original: Re ( h i )

Original: + Im ( h i ) .

Original: Respectively,

Original: Re ( . ) , Im ( . )

Original: denote real and imaginary parts of a complex number.

Original: The complex vector h contains real and imaginary vectors parts i.e. h

Original: = Re ( h )

Original: + Im ( h ) . ¯ t = Re ( t ) − Im ( t ) is conjugate of the complex vector t . Advantages of TransComplEx: 
Revised: Advantages of TransComplEx: We highlight four advantages of using the above formulation.

Original: i)
Revised: (i)

Original: Comparing to TransE and its variants, TransComplEx has less limitations in encoding different relation patterns.
Revised: Comparing to TransE and its variants, TransComplEx has less limitations in encoding different relation patterns.

Original: The theories and proofs are provided in the next part.
Revised: The theories and proofs are provided in the next part.

Original: ii) Using conjugate

Original: of

Original: tail

Original: vector in

Original: the

Original:  formulation enables the model to make difference between the role of an entity as subject or object.
Revised: (ii) Using conjugate of tail vector in the formulation enables the model to make difference between the role of an entity as subject or object.

Original: This cannot be properly captured by TransE and its variants.
Revised: This cannot be properly captured by TransE and its variants.

Original: iii)
Revised: (iii)

Original: Given the example ( A, Like, Juventus ) , ( Juventus, hasPlayer C Ronaldo ) , 
Revised: Given the example ( A, Like, Juventus ) , ( Juventus, hasPlayer, C.Ronaldo ) , that C.Ronaldo plays for Juventus may affect the person A to like the team.

Original: that

Original: C.Ronaldo plays for Juventus may affect the person A to like the team.

Original: This type of information cannot be properly captured by models such as CP decomposition Hitchcock (1927) where two independent vectors are provided Kazemi & Poole (2018) for Juventus (for subject and object).
Revised: This type of information cannot be properly captured by models such as CP decomposition (Hitchcock, 1927) where two independent vectors are provided (Kazemi & Poole, 2018) for Juventus (for subject and object).

Original: In contrast, our model uses same real and imaginary vectors for Juventus when it is used as subject or object.
Revised: In contrast, our model uses same real and imaginary vectors for Juventus when it is used as subject or object.

Original: Therefore, TransComplEx can properly capture dependency between the two triples with the same entity used as subject and object.
Revised: Therefore, TransComplEx can properly capture dependency between the two triples with the same entity used as subject and object.

Original: iiii) ComplEx Trouillon et al. (2016) has much more computational complexity comparing to TransComplEx because it needs to compute eight vector multiplications to obtain score of a triple while our model only needs to do four vector summation/subtractions.
Revised: And ﬁnally, (iiii) ComplEx (Trouillon et al., 2016) has much more computational complexity comparing to TransComplEx because it needs to compute eight vector multiplications to obtain score of a triple while our model only needs to do four vector summation/subtractions.

Original: In the experiment section, we show that TransComplEx outperforms ComplEx on various dataset.
Revised: In the experiment section, we show that TransComplEx outperforms ComplEx on various dataset.

Original: 4. R EINVESTIGATION OF THE L IMITATIONS OF T RANSLATION - BASEDE MBEDDING M ODELS
Revised: 4. R EINVESTIGATION OF THE L IMITATIONS OF T RANSLATION - BASED M ODELS

Original: The aim of this part is to analyze the limitations of Translation-based embedding models (including
Revised: The aim of this part is to analyze the limitations of Translation-based embedding models (including

Original: TransE and TransComplEx) by considering the effect of both score and loss functions.
Revised: TransE and TransComplEx) by considering the effect of both score and loss functions.

Original: Different loss functions provide different upper-bound and lower-bound for positive and negative triples scores, respectively.
Revised: Different loss functions provide different upper-bound and lower-bound for positive and negative triples scores, respectively.

Original: Therefore, the loss functions affect the limitations of the models to encode relation patterns.
Revised: Therefore, the loss functions affect the limitations of the models to encode relation patterns.

Original: To investigate the limitations, we redeﬁne the conditions that atriple is considered as positive or negative by deﬁning upper-bound and lower-bound for the scores.
Revised: In this regard, the existing works consider a positive triple of ( h, r, t ) and a negative triple of ( h (cid:48) , r, t (cid:48) ) to satisfy some assumptions in a score function. For instance in TransE where f r ( h, t )

Original: Lets f r ( h, t ) ,  f r (h (cid:48) , t (cid:48) ) be the scores of a positive ( h,r, t )  and negative ( h (cid:48) ,r, t (cid:48) ) triples respectively.
Revised: = (cid:107) h + r − t (cid:107) , it is expected that f r ( h, t ) = 0 and f r ( h (cid:48) , t (cid:48) ) > 0 .

Original: The negative triple ( h (cid:48) , r, t (cid:48) ) is generated by corruption of either head or tail of the triple ( h, r, t ) as mentioned in Bordes et al.
Revised: Unfortunately, as we show later, this can not be fulﬁlled (or even approximated) by using the proposed loss functions (e [Table] g. margin ranking loss and RotatE loss).

Original: Four conditions are deﬁned as follows:
Revised: To investigate and address the limitations

Original: [Table]
Revised: [Table]

Original: γ 1 , γ(a) f r ( h, t ) = γ 1 , f r ( h (cid:48) , t (cid:48) ) ≥
Revised: we propose four conditions (Table-1) for taking a triple as positive or negative by the score function.

Original: γγ 1 = 0 , γ 2 >(b) f r ( h, t ) =
Revised: This is done by deﬁning upper-bound and lower-bound for the scores.

Original: γf r ( h (cid:48) , t (cid:48) ) ≥ γγ 2 > γ 1 >(c) f r ( h, t ) ≤ γf r ( h (cid:48) , t (cid:48) ) ≥
Revised: We show that these conditions can be approximated by proper loss functions, and in this regards we propose four losses that each will handle one of the condition [Table] [Table] To better comprehend Table-

Original: Figure 1 visualizes different conditions mentioned above.
Revised: [Table] we have visualized the conditions in Figure 1.

Original: The condition (a) indicates a triple is positive if h + r = t holds.
Revised: The condition (a) indicates a triple is positive if h + r = t holds.

Original: It means that the length of residual vector i e., (cid:15) =
Revised: It means that the length of residual vector i [Table] e. (cid:15) =

Original: h + r − t , is zero.
Revised: h + r − t , is zero.

Original: It is the most strict condition that expresses being positive.
Revised: It is the most strict condition that expresses being positive.

Original: Authors in Sun et al. ; Kazemi & Poole (2018) consider this condition to prove their theories.
Revised: Authors in (Sun et al., 2019; Kazemi & Poole, 2018) consider this condition to prove their theories as well as limitation of TransE in encoding of symmetric relations.

Revised: However, the employed loss cannot approximate (a), rather it fulﬁlls (c), resulting the reported limitation to be void in that setting.

Original: Condition (b) considers a triple to be positive if its residual vector lies on a hyper-sphere with radius γ 1 .
Revised: Condition (b) considers a triple to be positive if its residual vector lies on a hyper-sphere with radius γ 1 .

Original: It is less restrictive thanthe condition (a) which considers a point to express being positive.
Revised: It is less restrictive than (a) which considers a point to express being positive.

Original: The optimization problem that satisﬁes the conditions (a) ( γ 1 = 0 ) and (b) ( γ 1 > 0 ) is as follows:
Revised: The optimization problem that approximates the conditions (a) ( γ 1 = 0 ) and (b) ( γ 1 > 0 ) is as follows:

Original: [Equation]
Revised: [Equation]

Original: where S + [Equation] S − are the set of positive and negative samples.
Revised: where S + , S − are the set of positive and negative samples respectively.

Revised: ξ h,t are slack variables to reduce the effect of noise in negative samples (Nayyeri et al., 2019).

Original: The loss function that satisﬁes the conditions (a)( γ 1 = 0 ) and (b)  ( γ 1 > 0 ) is:
Revised: One loss function that approximates the conditions (a) and (b) is as follows. Please note that for case ( ) we set 1 = 0 and set in the

Original: [Equation]
Revised: [Equation]

Original: Condition (c) considers a triple to be positive if its residual vector lies inside a hyper-sphere with radius γ 1 .
Revised: Condition (c) considers a triple to be positive if its residual vector lies inside a hyper-sphere of radius γ 1 .

Original: The optimization problem that satisﬁes the condition (c) is as follows (Nayyeri et al., 2019):
Revised: The optimization problem that approximates the condition (c) is as follows (Nayyeri et al., 2019):

Original: [Equation]
Revised: [Equation]

Original: The loss function that satisﬁes the condition (c) is as follows Nayyeri et al.
Revised: The loss function that approximates the condition (c) is as follows Nayyeri et al.

Original: [Equation]
Revised: [Equation]

Original: Remark: The loss function which is deﬁned in Zhou et al.
Revised: Remark: The loss function which is deﬁned in Zhou et al.

Original: (2017b) is slightly different from the loss 3.
Revised: (2017b) is slightly different from the loss in 4.

Original: The former slides the margin while the later ﬁxes the margin by inclusion of a lower-bound for the score of negative triples.
Revised: The former slides the margin while the latter ﬁxes the margin by inclusion of a lower-bound for the score of negative triples.

Original: The both losses put an upper-bound for scores of positive triples.
Revised: Both losses put an upper-bound for scores of positive triples.

Revised: Apart from the loss 4, the RotatE loss Sun et al.

Original: Condition (d) is similar to (c).
Revised: (2019) also approximates the condition (c).

Revised: The formulation of the RotatE loss is as follows:

Original: But it provides different γ 1 , γ 2 for each triples.
Revised: [Equation] Condition (d) is similar to (c), but provides different γ 1 , γ 2 for each triples.

Original: Usingthe condition (d), there is not a unique region of truth for all positive triples, rather for each  triple ( h, r, t ) and its corresponding negative samples ( h (cid:48) , r, t (cid:48) ) there are triple speciﬁc region of truth and falsity.
Revised: Using (d), there is not a unique region of truth for all positive triples, rather for each positive triple ( h, r, t ) and its corresponding negative of ( h (cid:48) , r, t (cid:48) ) there are triple-speciﬁc region of truth and falsity.

Original: Margin ranking loss (Bordes et al., 2013) satisﬁes the condition (d).
Revised: Margin ranking loss (Bordes et al., 2013) approximates (d).

Original: The loss is deﬁned as:
Revised: Deﬁning [ x ] + = max(0 , x ) , the loss is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: where [ x ] + = max(0 , x ) .
Revised: To investigate the limitations, we must assume that the relation vectors is not null otherwise we will have the same embedding for head and tail which is undesirable.

Original: Considering the conditions (a), (b), (c) and (d), we investigate the limitations L1 ,..., L6. We prove that existing theories are invalid under some conditions.
Revised: Considering the conditions (a) to (d), we investigate the limitations L1 to L6 and we prove that existing theories are just valid under (a), which is not fulﬁlled under the given loss.

Original: During the following investigations of the limitations, we assume that the relation vectors shouldn’t be null because the null vector for relation results same embedding vectors for entities appeared in head and tail parts when conditions (a) is used.
Revised: In this regard we have the following theorem. For complete proofs, please refer to the appendix of the paper.

Original: Limitation L1: Lemma 1: Let assumption (a) holds, then TransE and TransComplEx cannot infer a reﬂexive relation pattern with  non-zero relation vector.
Revised: Theorem T1. (Addressing L1) : TransE and TransComplEx cannot infer a reﬂexive relation pattern with a non-zero relation vector under (a).

Original: With assumptions (b), (c) and (d), however, this is not true anymore and the models can infer reﬂexive relation patterns
Revised: However, under (b-d), TransE and TransComplEx can infer reﬂexive pattern.

Original: Proof: the proofs are provided in the supplementary material ﬁle.
Revised: Theorem T2.

Original: Limitation L2:
Revised: (Addressing L2) :

Original: Lemma 2: 1) TransComplEx can infer a relationpattern which is neither reﬂexive nor irreﬂexive with condition (b), (c) and (d
Revised: (i) TransComplEx can infer a relation which is neither reﬂexive nor irreﬂexive under (b-d). (ii) TransE cannot infer a relation which is neither reﬂexive nor irreﬂexive under (a-d).

Original: Limitation L3: Lemma 3 : 1) TransComplEx can infer symmetric patterns with condition (a), (b), (c) and (d).
Revised: Theorem T3. (Addressing L3) : (i) TransComplEx can infer symmetric relations under (a-d). (ii)

Original: TransE cannot infer symmetric patterns with condition (a) with non-zero vector for relation.
Revised: TransE cannot infer symmetric relations under (a) with non-zero vector for relation.

Original:  TransE can infer arelation pattern which is symmetric with conditions (b).
Revised: (iii) TransE can infer a symmetric relation under (b-d).

Original: Proof: proof of 1), 2) and 3) are included in the supplementary material.
Revised: Proof: Proofs of (i) and (ii) are provided in the appendix. For (iii) we have:

Original: For TransE with condition (b), there is
Revised: Under (b), for TransE we have (cid:107) h + r − t (cid:107) =

Original: [Equation]
Revised: γ 1 and (cid:107) t + r − h (cid:107) = γ 1 .

Original: The necessity condition for encoding symmetric relation is (cid:107) h
Revised: The necessity condition for encoding symmetric relation is (cid:107) h

Original: + r − t (cid:107) =
Revised: + r − t (cid:107) =

Original: (cid:107) t + r − h
Revised: (cid:107) t + r − h

Original: (cid:107) .
Revised: (cid:107) .

Original: This implies (cid:107) h (cid:107) cos
Revised: This implies (cid:107) h (cid:107) cos

Original: ( θ h,r ) = (cid:107) t (cid:107) cos ( θ t,r ) .
Revised: ( θ h,r ) = (cid:107) t (cid:107) cos( θ t,r ) .

Original: Let h − t = u , by deﬁnition we have (cid:107) u
Revised: Let h − t = u , by deﬁnition we have (cid:107) u

Original: + r (cid:107) =
Revised: + r (cid:107) =

Original: γ 1 , (cid:107) u − r (cid:107) = γ 1 .
Revised: γ 1 , (cid:107) u − r (cid:107) = γ 1 .

Original: Let γ 

Revised: Now let γ

Original: = α (cid:107) r (cid:107) . We have (cid:26) (cid:107) u[-(cid:107) 
 + (1 −-]
Revised: = α (cid:107) r (cid:107) , we have: (cid:26) (cid:107) u[-(cid:107) 
 + (1 −-]

Original: [Equation]

Original: 2 ) (cid:107) {+
+}
Revised: (cid:107) {+
+}

Original: [Equation] (cid:107) [Equation]{+
+}
Revised: + (1 − α 2 ) (cid:107) r (cid:107) {+
+}

Original: =
Revised: =

Original: [Equation] (cid:105)[Equation] Regarding 9, there is (cid:107) {+u (cid:107) 
+}
Revised: − 2 (cid:104) u , r (cid:105) (cid:107) {+u (cid:107) 
+}

Original: [Equation]{++ (1 − α 2 ) (cid:107) r (cid:107) 2 = 2 (cid:104) u , r (cid:105) (6) Therefore we have: (cid:107) u (cid:107) 2 +(1 − α 2 ) (cid:107) r (cid:107) 
 = − ( (cid:107) u (cid:107) 
 +(1 − α 2 ) (cid:107) r (cid:107) 2 ) , which can be written as (cid:107) u (cid:107) 
+}
Revised: {++ (1 − α 2 ) (cid:107) r (cid:107) 2 = 2 (cid:104) u , r (cid:105) (6) Therefore we have: (cid:107) u (cid:107) 2 +(1 − α 2 ) (cid:107) r (cid:107) 
 = − ( (cid:107) u (cid:107) 
 +(1 − α 2 ) (cid:107) r (cid:107) 2 ) , which can be written as (cid:107) u (cid:107) 
+}

Original: 2 = ( α 2 − 1) (cid:107) r (cid:107) 2 .
Revised: = ( α 2 − 1) (cid:107) r (cid:107) 2 .

Original: To avoid contradiction, α > 1 .
Revised: To avoid contradiction we must have α > 1 .

Original: If α > 1 we have cos( θ u,r ) = π/ 2 .
Revised: Once α > 1 we have cos( θ u,r ) = π/ 2 .

Original: Therefore, TransE can encode symmetric pattern with condition (b), if γ 1 = α (cid:107) r (cid:107) and α > 1 .
Revised: Therefore, TransE can encode symmetric relation with condition (b), when γ 1 = α (cid:107) r (cid:107) and α > 1 .

Original: Figure 2 shows different conditions for encoding symmetric relation.
Revised: Figure 2 shows different conditions for encoding symmetric relation.

Original: Limitation L4: Lemma 4: 1) Let (a) holds.
Revised: Conditions (c-d) are directly resulted from (b), as it is subsumed by (c) and (d). That completes the proof. Theorem T4. (Addressing L4) :

Original: Limitation L4 holds for both TransE and TransComplEx.
Revised: For both TransE and TransComplEx, (i) Limitation L4 holds under (a).

Original: 2) Limitation L4 is not valid when assumptions (b), (c) and (d) hold.
Revised: (ii) Limitation L4 is not valid under (b-d). Theorem T5.

Original: Limitation L5:
Revised: (Addressing L5)

Original: Lemma 5: 1) Under condition (a), the limitation L5 holds for both TransE and TransComplEx. 2)
Revised: : For both TransE and TransComplEx, (i) Limitation L5 holds under (a).

Original: Under conditions (b), (c) and (d), L5  is not valid for both TransE and TransComplEx.
Revised: (ii) Limitation L5 holds is not valid under (b-d). Theorem T6.

Original: Limitation L6: Lemma 6: 1) With condition (a), the limitation L6 holds for both TransE and
Revised: (Addressing L6) :

Original: TransComplEx. 2) With conditions (b), (c) and (d), the limitation L6 doesn’t hold for the models.
Revised: For both TransE and TransComplEx, (i) Limitation L6 holds under (a). (ii) Limitation L6 is not valid under (b-d).

Original: 4. E NCODING R ELATION P ATTERNS IN T RANS C OMPL E X
Revised: 4. E NCODING R ELATION P ATTERNS IN T RANS C OMPL E X

Original: Most of KGE models learn from triples.
Revised: Most of KGE models learn from triples.

Original: Recent work incorporates relation patterns such as transitive, symmetric on the top of triples to further improve performance of models.
Revised: Recent work incorporates relation patterns such as transitive, symmetric on the top of triples to further improve performance of models.

Original: For example,
Revised: For instance,

Original: ComplEx-NNE+AER Ding et al.
Revised: ComplEx-NNE+AER (Ding et al., 2018)

Original: encodes implication pattern in the ComplEx model.
Revised: encodes implication relation in the ComplEx model.

Original: RUGE Guo et al. (2018) injects First Order Horn Clause rules in an embedding model.
Revised: RUGE (Guo et al., 2018) injects First Order Horn Clause rules in an embedding model.

Original: SimplEKazemi &
Revised: SimplE

Original: Poole (2018) captures symmetric, antisymmetric and inverse patterns by weight tying in the model.
Revised: (Kazemi & Poole, 2018) captures symmetric, antisymmetric and inverse relations by weight tying in the model.

Original: Inspired by Minervini et al. (2017) and considering the score function of TransComplEx, in this part, we derive formulae for equivalence, symmetric, inverse and implication to be used as regularization terms in the optimization problem.
Revised: Inspired by (Minervini et al., 2017) and considering the score function of TransComplEx, in this part, we derive formulae for equivalence, symmetric, inverse and implication to be used as regularization terms in the optimization problem.

Original: Therefore, the model incorporates different relation patterns to optimize the embeddings.
Revised: Therefore, TransComplEx incorporates different relation patterns to optimize the embeddings.

Original: Symmetric: In order to encode symmetric relation r , the following should be held: [Equation]
Revised: Symmetric: Assume that r is a symmetric relation. To encode it we should have f r ( h, t )

Original: [Equation] Therefore the following algebraic formulae is proposed to encode the relation: (cid:107) f r ( h, t ) − f r ( t, h ) (cid:107) = 0 .
Revised: ≈ f r ( t, h ) , therefore (cid:107) f r ( h, t ) − f r ( t, h ) (cid:107) = 0 .

Original: According to the deﬁnition of score function of TransComplEx, we have the following algebraic formulae: R S = (cid:107) Re ( h ) − Re ( t ) (cid:107) = 0 .
Revised: According to the deﬁnition of score function of TransComplEx, we have the following algebraic formulae: R Sym := (cid:107) Re ( h ) − Re ( t ) (cid:107) = 0 .

Original: Using similar argument for symmetric, the following formulae are derived for transitive, composition, inverse and implication:
Revised: Using similar argument to symmetric, the following formulae are derived for transitive, composition, inverse and implication.

Original: Equivalence: Let p, q be equivalence relations i.e., f p ( h, t )
Revised: Equivalence: Let p, q be equivalence relations, therefore we should have f p ( h, t )

Original: ⇐⇒ f q ( h, t ) .
Revised: ≈ f q ( h, t ) .

Original: we obtain R E
Revised: We obtain, R Eq :

Original: = (cid:107) p − q (cid:107) = 0 .
Revised: = (cid:107) p − q (cid:107) = 0 .

Original: Implication: Let p → q, we obtain R I = max( f p ( h, t ) − f q ( h, t ) , 0)
Revised: Implication: Let p → q , be the implication rule. We obtain R Imp := max( f p ( h, t ) − f q ( h, t ) , 0)

Original: = 0 .
Revised: = 0 .

Original: Inverse: Let r
Revised: Inverse: Let r

Original: ←→ r − 1 , we obtain R In = (cid:107) r − r − 1 (cid:107) .
Revised: ←→ r − 1 be the inverse relation. We obtain R Inv : = (cid:107) r − r − 1 (cid:107) .

Original: Finally, the following optimization problem should be solved: 
Revised: Finally, the following optimization problem should be solved: min θ

Original: [Equation]
Revised: L + (cid:88)

Original:  where θ is embedding parameters, L is one of the losses 3, 5 or 6 and R  is one of the derived formulae mentioned above.
Revised: η i R i (7) where θ is embedding parameters, L is one of the losses 2, 4 or 5 and R i is one of the derived formulae mentioned above.

Original:  E XPERIMENTS AND E VALUATIONS
Revised:  E XPERIMENTS AND E VALUATIONS

Original: In this section, we evaluate performance of our model, TransComplEx, with different loss functions on  link prediction task.
Revised: In this section, we evaluate performance of our model, TransComplEx, with different loss functions on the link prediction task.

Original: The aim of the task is to complete the triple ( h, r, ?) ( (? , r, t )) by prediction of the missed entity h or t .
Revised: The aim of the task is to complete the triple ( h, r, ?) or (? , r, t ) by prediction of the missed entity h or t .

Original: Filtered Mean Rank (MR), Mean Reciprocal Rank (MRR) and Hit@ are used for evaluations (Wang et al., 2017; Lin et al., 2015b).
Revised: Filtered Mean Rank (MR), Mean Reciprocal Rank (MRR) and Hit@10 are used for evaluations (Wang et al., 2017; Lin et al., 2015b).

Original: Dataset.
Revised: Dataset.

Original: We use two dataset extracted from Freebase (Bollacker et al., 2008)
Revised: We use two dataset extracted from Freebase (Bollacker et al., 2008)

Original: (i.e., FB15K (Bordes et al., 2013) and FB15K-237 (Toutanova & Chen, 2015)) and two others extracted from WordNet (Miller, 1995) (i.e. WN18 (Bordes et al., 2013) and WN18RR (Dettmers et al., 2018)).
Revised: (i.e. FB15K (Bordes et al., 2013) and FB15K-237 (Toutanova & Chen, 2015)) and two others extracted from WordNet (Miller, 1995) (i.e. WN18 (Bordes et al., 2013) and WN18RR (Dettmers et al., 2018)).

Original: FB15K and WN18 are earlier dataset which have been extensively used to compare performance of KGEs.
Revised: FB15K and WN18 are earlier dataset which have been extensively used to compare performance of KGEs.

Original: FB15K-237 and WN18RR are two dataset which are supposed to be more challenging after removing inverse patterns from FB15K and WN18.
Revised: FB15K-237 and WN18RR are two dataset which are supposed to be more challenging after removing inverse patterns from FB15K and WN18.

Original: Guo et al.
Revised: Guo et al.

Original: (2018) and Ding et al.
Revised: (2018) and Ding et al.

Original: (2018) extracted different relation patterns from FB15K and WN18 respectively.
Revised: (2018) extracted different relation patterns from FB15K and WN18 respectively.

Original: The relation patterns are provided by their conﬁdence level, e.g. ( a, BornIn, b ) 0 .
Revised: The relation patterns are provided by their conﬁdence level, e.g. ( a, BornIn, b ) 0 .

Original: 9 −−→ ( a, Nationality, b ) .
Revised: 9 −−→ ( a, Nationality, b ) .

Original: We drop the relation patterns with conﬁdence level less than 0.8.
Revised: We drop the relation patterns with conﬁdence level less than 0.8.

Original: Generally, we use 454 and 14 relation patterns for FB15K and WN18 respectively.
Revised: Generally, we use 454 and 14 relation patterns for FB15K and WN18 respectively.

Original: We do grounding for symmetric and transitive relation patterns.
Revised: We do grounding for symmetric and transitive relation patterns.

Original: Thanks to the formulation of score function, grounding is not needed for inverse, implication and equivalence.
Revised: Thanks to the formulation of score function, grounding is not needed for inverse, implication and equivalence.

Original: Experimental Setup.
Revised: Experimental Setup.

Original: We implement TransComplEx with the losses 3, 5 and 6 and TransE with the loss 5 in Pytorch.
Revised: We implement TransComplEx with the losses 2, 4 and 5 and TransE with the loss 4 in PyTorch.

Original: Adagrad is used as an optimizer.
Revised: Adagrad is used as an optimizer.

Original: We generate 100 mini-batches in each iteration.
Revised: We generate 100 mini-batches in each iteration.

Original: The hyperparameter corresponding to the score function is embedding dimension d .
Revised: The hyperparameter corresponding to the score function is embedding dimension d .

Original: We add slack variables to the losses 3 and 5 to have soft margin as in (Nayyeri et al., 2019).
Revised: We add slack variables to the losses 2 and 4 to have soft margin as in (Nayyeri et al., 2019).

Original: The lossis rewritten as follows Nayyeri et al.
Revised: The lossis rewritten as follows Nayyeri et al.

Original: [Equation]
Revised: [Equation]

Original: We set λ 1 and λ 2 to one and search for the hyperparameters γ 1 ( γ 2 > γ 1 ) and λ 0 in the sets { 0 . 1 , 0 . 2 , . . .
Revised: We set λ 1 and λ 2 to one and search for the hyperparameters γ 1 ( γ 2 > γ 1 ) and λ 0 in the sets { 0 . 1 , 0 . 2 , . . .

Original: , 2 } and { 0 . 01 , 0 . 1 , 1 , 10 , 100 } respectively.
Revised: , 2 } and { 0 . 01 , 0 . 1 , 1 , 10 , 100 } respectively.

Original: Moreover, we generate α ∈ { 1 , 2 , 5 , 10 } negativesamples per each positive. The embedding dimension and learning rate aretuned from the sets { 100 , 200 } , { 0 . 0001 , 0 . 0005 , 0 . 001 ,0 . respectively.
Revised: Moreover, we generate α ∈ { 1 , 2 , 5 , 10 } negative are from the sets { 100 , 200 } , { 0 . 0001 , 0 . 0005 , 0 . 001 , respectively.

Original: All hyperparameters are adjusted by early stopping on validation set according to MRR.
Revised: All hyperparameters are adjusted by early stopping on validation set according to MRR.

Original: RPTransComplEx # denotes the
Revised: RPTransComplEx # denotes the

Original: TransComplEx model which is trained by the loss function # (3, 5, 6).
Revised: TransComplEx model which is trained by the loss function # (2, 4, 5).

Original: RP indicates that relation patterns are injected during learning by regularizing the derived formulae (see 10).
Revised: RP indicates that relation patterns are injected during learning by regularizing the derived formulae (see 7).

Original: TransComplEx # refers to our model trained with the loss # without regularizing relation patterns formulae.
Revised: TransComplEx # refers to our model trained with the loss # without regularizing relation patterns formulae.

Original: The same notation is used for TransE # .
Revised: The same notation is used for TransE # .

Original: The optimal conﬁgurations for RPTransComplEx3 are d = 200 , λ 0 = 100 , γ 1 = 0 .
Revised: The optimal conﬁgurations for RPTransComplEx are provided in the appendix.

Original: 4 , γ 2 = 0 .
Revised: [Table]

Original: 5 , α = 10 for FB15K, d = 200 , λ 0 = 100 , γ 1 = 1 . 5 , γ 2 = 2 , α = 10 for FB15K-237, d = 200 , λ 0 = 100 , 
Revised: , while the rest are taken from original papers code. Dashes: results could not be obtained.

Original: Results.
Revised: Results.

Original: Table 2 presents comparison of TransComplEx and its relation pattern encoded variants (RPTransComplEx) with three classes of embedding models including Translation-based models  (e.g. TransX, TorusE), relation pattern encoded models (e.g. RUGE, ComplEx-NNE+AER, SimplE,
Revised: Table 2 presents comparison of TransComplEx and its relation pattern encoded variants (RPTransComplEx) with three classes of embedding models on the most famous two datasets of FB15K and WN18. The ﬁrst category (CAT1) of models consist of translation-based model (e.g. TransX, TorusE).

Original: SimplE+), and other state-of-the-art embedding models  (e.g. ConvE, ComplEx, ANALOGY). 
Revised: The second category (CAT2) are embedding models which are not translationbased (e.g. ConvE, ComplEx, ANALOGY). The previous two categories (CAT1/2) can learn relation patterns that are reﬂected in triples. In other words they learn patterns implicitly from existing triples. The last category (CAT3) consist of models which can explicitly encode (inject) rules in their training process (e.g. RUGE, ComplEx-NNE+AER, SimplE, SimplE+).

Original: To investigate our theoretical proofs corresponding to the effect of  loss function, we train TransComplEx with different loss functions.
Revised: The models in CAT are actually trained on relation patterns as well as triples. We trained relation pattern version of our model i.e. RPTransComplEx using the losses of 2, 4, 5. All variants of RPTransComplEx, except PRTransComplEx4, are generally performing better than models in CAT3 which also explicitly inject relation patterns. The exception, PRTransComplEx5, is due to the fact that it uses margin ranking loss which we already showed its restriction regarding condition (d).

Original:  As previously discussed, FB15K-237 and WN18RR are two
Revised: In this regard, please consider histogram (d) in Figure 1. As all limitations were mitigated by condition (c), we expected that RPTransComplEx4, which is associated to (c), should perform better than others. This is empirically approved as shown in Table- [Table]

Original: [Table] TransE (Bordes et al., 2013)
Revised: The question is that how our model behaves under the loss 4 while we additionally disregard injection of relation patterns. To this end, we considered TransComplEx4 which uses loss 4 but does not do any injection. Since the performances of RPTransComplEx4 and TransComplEx4 are very close, we conclude that the latter also learns the pattern over the existing triples quite efﬁciently.

Original: [Table] TransH (bern) (Wang et al., 2014)* [Table] TransR (bern) (Lin et al., 2015b)
Revised: This is also conﬁrmed by the convergence performance of them as shown in Figure 3 over the symmetric relation.

Original: [Table] TransD (bern) (Ji et al., 2015) [Table] TransE
Revised: To have a fair comparison 1 with the categories of models which disregard injection of relation patterns (CAT1/2) we used the TransComplEx4 version of our approach.

Original: [Table] TransH-RS (bern) (Zhou et al., 2017a [Table] [Table]
Revised: As shown in Table-2, we can observe that TransComplEx4 performs better than other models in CAT1/2 considering MR and

Original: with WNP) [Table] (Schlichtkrull et [Table]
Revised: Hits@10 and performing very closely to the best models on MRR.

Original: [Table] Dettmers [Table] ComplEx (Trouillon et al., 2016)
Revised: As discussed earlier, FB15K-237 and WN18RR are two more challenging dataset provided recently.

Original: [Table] [Table] Fatemi et al., 2018) [Table]
Revised: Table 3 presents the comparisons of our models with those other models that their performance results were available on these two datasets.

Original: PTransE (Lin et al., 2015a [Table] [Table] TransE
Revised: Similar to our previous discussion on Table-2, we observe that PRTransComplEx4 and TransComplEx4 are performing closely which means that the latter has learned the relation pattern over triples very well without any injection.

Original: [Table] [Table] DistMult (Bordes et al. , 2013)+ - 24.1 41.
Revised: On WN18RR,

Original: [Table] [Table] ComplEx (Trouillon et al. , 2016)+ - 24.0 41.
Revised: TorusE was performing better than TransComplEx4 due to big embedding dimension of 10,000.

Original: R-
Revised: On

Original: [Table]

Original: (Schlichtkrull et al., 2018)+ - [Table] [Table] ConvE (Dettmers et al., 2018)+ - 31.6 49.
Revised: FB15k-237, RPTransComplEx4 performed better that others on MRR and Hits@10.

Original: TorusE

Original: [Table] more challenging dataset provided recently. Therefore, in order to have a better evaluation, Tablepresents comparison of our models with state-of-the-art embedding methods on these two dataset. For WN18RR, we do not use any relation patterns to be encoded.
Revised: In order to investigate the effect of grounding, we train RPTransComplEx4 in two settings: 1) RPTransComplEx4(w grounding) is trained when the grounded patterns are injected, 2) RPTransComplEx4(w/o grounding) is trained when the relation patterns which are not grounded used.

Original: The results labeled with ”*”, ”+” and ”++” are taken from Zhou et al. (2017a), Ebisu & Ichise (2019) and Akrami et al.
Revised: According to the Table 4, the grounding does not affect the performance signiﬁcantly.

Original: (2018) respectively. To have a fair comparison, we ran the code of RotatE (Sun et al., 2019) in our setting e.g., embedding dimension 200 and 10 negative samples while the original paper reported the results of 
Revised: We conclude that the model properly learns the relation patterns even without injection. Boosting techniques: There are several ways to improve the performance of embedding models: 1) designing a more sophisticated scoring function, 2) proper selection of loss function, 3) using

Original: RotatE using a very big embedding dimension and a lot of negative samples (embedding dimension 1000 and 1000 negative samples).
Revised: FB15K MR MRR Hits@

Original: Boosting techniques: There are several ways to improve the performance of embedding models: 1) designing a more sophisticated scoring function, 2) proper selection of loss function, 3) using more negative samples 4) using negative sampling techniques, 5) enriching dataset (e.g., adding reverse triples).
Revised: RPTransComplEx4(w grounding) 38 72. RPTransComplEx4(w/o grounding) 40 72. more negative samples 4) using negative sampling techniques, 5) enriching dataset (e.g. adding reverse triples).

Original: Among the mentioned techniques, we focus on the ﬁrst and second ones and avoid using other techniques.
Revised: Among the mentioned techniques, we focus on the ﬁrst and second ones and avoid using other techniques.

Original: We keep the setting used in Trouillon et al. (2016) to have a fair comparison.
Revised: We keep the setting used in (Trouillon et al., 2016) to have a fair comparison.

Original: Using other techniques can further improve the performance of every models including ours.
Revised: Using other techniques can further improve the performance of every models including ours.

Original: For example, TransComplEx with embedding dimension 200 and 50 negative samples gets 52.2 for
Revised: For example, TransComplEx with embedding dimension 200 and 50 negative samples gets 52.2 for

Original: Hits@10.
Revised: Hits@10.

Original: Dissuasion of Results.

Original: According to the Table 2, FB15K dataset part, PRTransComplEx trained by the loss 5 signiﬁcantly outperforms all Translation-based embedding models including the recent work TorusE.

Original: Note that TorusE is trained by embedding dimension 10000 while ourmodel uses embedding dimension at most 200. Comparing to relation pattern encoded embedding modelsincluding recent works ComplEx-NNE+AER, RUGE, SimplE and SimplE+, our model outperforms them in the terms of MR and Hit@10. Moreover, the model signiﬁcantly outperforms popular embedding models including ConvE and ComplEx. Regarding our theories, the loss 5 has less limitations comparing to the loss 3.
Revised: Further analyses of our models in a big setting (bigger embedding dimension and more negative samples) are provided in appendix.

Original: This is consistent with our theories where RPTransComplEx outperforms RPTransComplEx3.

Original: TransComplEx without encoding relation patterns still obtains accuracy as good as state-of-the-art models.

Original: TransComplEx outperforms TransE while both are trained by the loss 5 in the terms of MR, MRR and Hit@10 which is consistent with our theories (TransComplEx score function has less limitations than TransE).

Original: Regarding the results on WN18, the accuracy of TransComplEx is very close to the state-of-the-art models.

Original: Encoding relation patterns cannot improve the performance on WN18 because the models learn relation patterns from data well.

Original: The loss 6 provides different upper-bounds and lower-bounds for the score of positive and negative triples respectively and also the margin can slide.

Original: Therefore, the accuracy would be degraded Zhou et al.

Original: (2017a). Generally, the loss 5 gets better performance which is consistent to our theoretical results.
Revised: Still, loss 4 is performing better and we conclude that our theoretical framework is approved in empirical experiments.

Original: As shown in the Table 3, FB15K-237 part, with and without encoding relation patterns, TransComplEx trained by the loss 5 outperforms all the baselines in terms of MRR and

Original: Hit@10.

Original: TransComplEx5 outperforms TransE5 showing the effectiveness of our proposed score function.

Original: Regarding WN18RR, TorusE has better performance comparing to our model.

Original: However, the results are obtained with a very big embedding dimension ( d = 10000 ).

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we reinvestigated the main limitations of Translation-based embedding models from two aspects: score and loss .
Revised: In this paper, we reinvestigated the main limitations of Translation-based embedding models from two aspects: score and loss .

Original: We showed that existing theories corresponding to the limitations of the models are inaccurate because the effect of loss functions has been ignored.
Revised: We showed that existing theories corresponding to the limitations of the models are inaccurate because the effect of loss functions has been ignored.

Original: Accordingly, we presented new theories about the limitations by consideration of the effect of score and loss functions.
Revised: Accordingly, we presented new theories about the limitations by consideration of the effect of score and loss functions.

Original: We proposed TransComplEx, a new variant of TransE which is proven to be less limited comparing to the TransE. The model is trained by using various loss functions on standard dataset including
Revised: We proposed TransComplEx, a new variant of TransE which is proven to be less limited comparing to the TransE. The model is trained by using various loss functions on standard dataset including

Original: FB15K, FB15K-237, WN18 and WN18RR.
Revised: FB15K, FB15K-237, WN18 and WN18RR.

Original: According to the experiments, TransComplEx with proper loss function signiﬁcantly outperformed translation-based embedding models.
Revised: According to the experiments, TransComplEx with proper loss function signiﬁcantly outperformed translation-based embedding models.

Original: Moreover,
Revised: Moreover,

Original: TransComplEx got competitive performance comparing to the state-of-the-art embedding models while it is more efﬁcient in time and memory. 
Revised: TransComplEx got competitive performance comparing to the state-of-the-art embedding models while it is more efﬁcient in time and memory. The experimental

Revised: A F URTHER E XPERIMENTS WITH A BIGGER SETTING

Revised: In this section we compare TransE, TransComplEx and RotatE trained by using the losses 2 (condition (a),(b)) , 4 (condition (c)) and the RotatE loss (condition (c)).

Revised: In contrast to our previous experiments, we use a bigger setting: For FB15K-237, we set the embedding dimension to 300 and the number of the negative samples to 256, and for WN18RR, we set the embedding dimension and the number of negative samples to 300 and 250 respectively.

Original: The experimental results conformed the presented theories corresponding to the limitations.
Revised: We additionally use adversarial negative sampling technique used in (Sun et al., 2019) for all models. FB15K [Table] WN18RR

Revised: MR MRR Hits@

Revised: MR MRR Hits@

Revised: TransE2 ( γ 1 = 0 ) 222 27.

Revised: TransE2 ( γ 1 > 0 ) 198 31.

Revised: TransE 181 32.

Revised: TransE L RotatEc 179 32.

Revised: TransComplEx2 ( γ 1 = 0 ) 213 28.

Revised: TransComplEx2 ( γ 1 > 0 ) 194 31.

Revised: TransComplEx 177 32.

Revised: TransComplEx L RotatEc 176 32.

Revised: RotatE 194 33.

Revised: RotatE L RotatEc 196 33.

Revised: Analysis of the results : Table 5 presents a comparison of TransE, TransComplEx and RotatE trained by different losses.

Revised: TransE2 ( γ 1 = 0) is trained by using the loss 2 when γ 1 = 0 .

Revised: TransE2 ( γ 1 > 0) refers to the TransE model which is trained by using the loss 2 when γ 1 is a non-zero positive value.

Revised: The TransE model which is trained by the losses 4 and the RotatE loss (i.e., L RotatEc ) are denoted by

Revised: TransE 4 and TransE L RotatEc respectively.

Revised: The similar notations are considered for TransComplEx and RotatE when they are trained by using different loss functions.

Revised: The loss 2 with γ 1 = 0 approximates the condition (a).

Revised: The loss 2 with γ 1 > 0 approximates the condition (b).

Revised: The condition (c) can be approximated by using the loss 4 and the RotatE loss (i.e., L RotatEc ).

Revised: However, the lossprovides a better separation for positive and the negative samples than the RotatE loss.

Original: A S UPPLEMENTARY M ATERIAL
Revised: According to the table 5, the loss 4 obtains a better performance than the other losses in each class of the studied models. It is consistent with our theories indicating that the condition (c) is less restrictive. Although we only investigated the main limitations of the translation-based class of embedding models, the theories can be generalized to different models including the RotatE model. From the table, we can see that the loss 4 improves the performance of RotatE. Regarding the table 5, the loss 2 ( γ 1 = 0) gets the worst results.

Original: The proof of lemmas are provided as follows:
Revised: It conﬁrms our theories that with the condition (a), most of the limitations are held. However, with the condition (c), the limitations no longer exist. There have not been any losses that approximate the condition (a). However, most of the theories corresponding to the main limitations of the translation-based class of embedding models have been proven using the condition (a) while the used loss didn’t approximate the condition.

Original: Lemma 1: Let assumption (a) holds, then TransE and TransComplEx cannot infer a reﬂexive relation pattern with non-zero relation vector.
Revised: Therefore, the theories and experimental justiﬁcations have not been accurate. B R ELATION P ATTERN C ONVERGENCE A NALYSIS Figure 4 visualizes the convergence curve of the inverse loss with (RPTransComplEx4) and without (TransComplEx4) injection when the models are trained on WN18. Figure 5 shows the convergence of the TransComplEx4 model trained on FB15K with and without the relation pattern injection.

Original: With assumptions (b), (c) and (d), however, this is not true anymore and the models  can infer reﬂexive relation patterns 
Revised: The ﬁgures show that the models trained by using the loss 4 can properly encode the relation patterns even without any injection mechanism. In other words, the TransComplEx4 model can properly encode several relation patterns by only training on the triples (without using any additional relation pattern set to be injected). This shows the advantages of the models and the used losses. C P ROOF OF T HE T HEOREM

Revised: The proof of theorems are provided as follows:

Original: Proof  1) Let r be a reﬂexive relation and condition a) holds.
Revised: Proof of the Theorem T1 1) Let r be a reﬂexive relation and condition a) holds.

Original: For TransE, we have
Revised: For TransE, we have

Original: [Equation]
Revised: [Equation]

Original: Therefore, the relation vector collapses to a null vector ( r = 0 ) .
Revised: Therefore, the relation vector collapses to a null vector ( r = 0 ) .

Original: As a consequence of r = 0 , embedding vectors of head and tail entities will be same which is undesired.
Revised: As a consequence of r = 0 , embedding vectors of head and tail entities will be same which is undesired.

Original: Therefore, TransE cannot infer reﬂexive relation with r (cid:54) = 0 .
Revised: Therefore, TransE cannot infer reﬂexive relation with r (cid:54) = 0 .

Original: For TransComplEx, we have
Revised: For TransComplEx, we have

Original: [Equation]
Revised: [Equation]

Original: We have
Revised: We have

Original: [Equation]
Revised: [Equation]

Original: Therefore, all entities will have same embedding vectors which is undesired.
Revised: Therefore, all entities will have same embedding vectors which is undesired.

Original: 2) Using condition (b), we have
Revised: 2) Using condition (b), we have

Original: [Equation]
Revised: [Equation]

Original: It gives (cid:107) r (cid:107) =
Revised: It gives (cid:107) r (cid:107) =

Original: γ 1 .
Revised: γ 1 .

Original: Therefore, in order to infer reﬂexive relation, the length of the relation vector should be γ 1 .
Revised: Therefore, in order to infer reﬂexive relation, the length of the relation vector should be γ 1 .

Original: Consequently, TransE and TransComplEx can infer reﬂexive relation.
Revised: Consequently, TransE and TransComplEx can infer reﬂexive relation.

Original: The same procedure can be used for the conditions (c) and (d).
Revised: The same procedure can be used for the conditions (c) and (d).

Original: Lemma 2: 1) Let the assumption b) or c) or d) holds.

Original: TransComplEx can infer a relation pattern which is neither reﬂexive nor irreﬂexive.

Original: TransE cannot infer the relation pattern.

Original: proof: 1) Let the relation r be neither reﬂexive nor irreﬂexive and two triples ( e 1 , r, e 1 ) , ( e 2 , r, e 2 ) be positive and negative respectively.
Revised: Proof of the theorem T2: i) Let the relation r be neither reﬂexive nor irreﬂexive and two triples ( e 1 , r, e 1 ) , ( e 2 , r, e 2 ) be positive and negative respectively.

Original: Therefore the following inequalities hold:
Revised: Therefore the following inequalities hold:

Original: [Equation]
Revised: [Equation]

Original: Equation 15 is rewritten as follows:
Revised: Equation 12 is rewritten as follows:

Original: [Equation]
Revised: [Equation]

Original: For TransE in real space, (cid:107) Re ( r ) (cid:107) ≤ γ 1 and (cid:107) Re ( r ) (cid:107) ≥ γ 2 cannot be held simultaneously when γ 2 > γ 1 .
Revised: For TransE in real space, (cid:107) Re ( r ) (cid:107) ≤ γ 1 and (cid:107) Re ( r ) (cid:107) ≥ γ 2 cannot be held simultaneously when γ 2 > γ 1 .

Original: Therefore, TransE in real space cannot encode a relation which is neither reﬂexive nor irreﬂexive.
Revised: Therefore, TransE in real space cannot encode a relation which is neither reﬂexive nor irreﬂexive.

Original: In contrast, TransE in complex space can encode the relation by proper assignment of imaginary parts of entities.
Revised: In contrast, TransE in complex space can encode the relation by proper assignment of imaginary parts of entities.

Original: Therefore, theoretically TransComplEx can infer a relation which is neither reﬂexive nor irreﬂexive.
Revised: Therefore, theoretically TransComplEx can infer a relation which is neither reﬂexive nor irreﬂexive.

Original: Lemma 3 : 1) TransComplEx can infer symmetric patterns with condition a), b), c) and d).

Original: TransE cannot infer symmetric patterns with condition a) with non-zero vector for relation.

Original: TransE can infer a relation pattern which is symmetric and reﬂexive with conditions b), c) and d).

Original: Proof: 1), 2) Let r be a symmetric relation and a) holds.
Revised: Proof of the theorem T3: i), ii) Let r be a symmetric relation and a) holds.

Original: We have
Revised: We have

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Trivially, we have
Revised: Trivially, we have

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: For TransE in real space, there is
Revised: For TransE in real space, there is

Original: [Equation]
Revised: [Equation]

Original: Therefore, Re ( r ) = 0 .
Revised: Therefore, Re ( r ) = 0 .

Original: It means that TransE cannot infer symmetric relations with condition a).
Revised: It means that TransE cannot infer symmetric relations with condition a).

Original: For
Revised: For

Original: TransComplEx, additionally we have
Revised: TransComplEx, additionally we have

Original: [Equation]
Revised: [Equation]

Original: It concludes Im ( h ) + Im ( r )
Revised: It concludes Im ( h ) + Im ( r )

Original: + Im ( t )
Revised: + Im ( t )

Original: = 0 .
Revised: = 0 .

Original: Therefore, TransE in complex space with condition a) can infer symmetric relation.
Revised: Therefore, TransE in complex space with condition a) can infer symmetric relation.

Original: Because a) is an special case of b) and c), TransComplEx can infer symmetric relations in all conditions.
Revised: Because a) is an special case of b) and c), TransComplEx can infer symmetric relations in all conditions.

Original: For TransE with condition b), there is
Revised: For TransE with condition b), there is

Original: [Equation]
Revised: [Equation]

Original: The necessity condition for encoding symmetric relation is (cid:107) 
Revised: The necessity condition for encoding symmetric relation is (cid:107) h

Original: [Equation]

Original:  r − t (cid:107) = (cid:107) t + r − h
Revised: + r − t (cid:107) = (cid:107) t + r − h

Original: (cid:107) .
Revised: (cid:107) .

Original: This implies (cid:107) h (cid:107) cos
Revised: This implies (cid:107) h (cid:107) cos

Original: ( θ h,r ) = (cid:107) t (cid:107) cos ( θ t,r ) .
Revised: ( θ h,r ) = (cid:107) t (cid:107) cos ( θ t,r ) .

Original: Let h − t = u , by 20 we have (cid:107) u + r (cid:107) =
Revised: Let h − t = u , by 17 we have (cid:107) u + r (cid:107) =

Original: γ 1 , (cid:107) u − r (cid:107) = γ 1 .
Revised: γ 1 , (cid:107) u − r (cid:107) = γ 1 .

Original: Let γ 

Revised: Let γ

Original: = α (cid:107) r (cid:107) .
Revised: = α (cid:107) r (cid:107) .

Original: We have
Revised: We have

Original: [Equation]
Revised: [Equation]

Original: Regarding 21, we have
Revised: Regarding 18, we have

Original: [Equation]
Revised: [Equation]

Original: →
Revised: →

Original: [Equation]
Revised: [Equation]

Original: u

Original: [Equation]

Original: 2 = ( α 2 −

Original: [Equation]

Original: (cid:107) r (cid:107) 2 .

Original: To avoid contradiction, α ≥ 1 .
Revised: To avoid contradiction, α ≥ 1 .

Original: If α ≥ 1 we have cos( θ u,r ) = π/ 2 .
Revised: If α ≥ 1 we have cos( θ u,r ) = π/ 2 .

Original: Therefore, TransE can encode symmetric pattern with condition b), if γ 1 = α (cid:107) r (cid:107) and α ≥ 1 .
Revised: Therefore, TransE can encode symmetric pattern with condition b), if γ 1 = α (cid:107) r (cid:107) and α ≥ 1 .

Original: From the proof of condition b), we conclude that TransE can encode symmetric patterns under conditions c) and d).
Revised: From the proof of condition b), we conclude that TransE can encode symmetric patterns under conditions c) and d).

Original: Lemma 4: 1) Let a) holds. Limitation L4 holds for both TransE and TransComplEx. 2) Limitation L4 is not valid when assumptions b), c) and d) hold.
Revised: Proof of the theorem T4: i)

Original: Proof: 1) The proof of the lemma with condition a) for TransE is mentioned in the paper Kazemi & Poole (2018).
Revised: The proof of the lemma with condition a) for TransE is mentioned in the paper Kazemi & Poole (2018).

Original: For TransComplEx, the proof is trivial.
Revised: For TransComplEx, the proof is trivial.

Original:  Now, we prove that the limitation L4 is not valid when b) holds.
Revised: ii) Now, we prove that the limitation L4 is not valid when b) holds.

Original: Let condition b) holds and relation r be reﬂexive, we have (cid:107) e 1 + r − e 1 (cid:107) =
Revised: Let condition b) holds and relation r be reﬂexive, we have (cid:107) e 1 + r − e 1 (cid:107) =

Original: γ 1 , (cid:107) e 2 + r − e 2 (cid:107) = γ 1 .
Revised: γ 1 , (cid:107) e 2 + r − e 2 (cid:107) = γ 1 .

Original: Let (cid:107) e 1 + r − e 2 (cid:107) = γ 1 .
Revised: Let (cid:107) e 1 + r − e 2 (cid:107) = γ 1 .

Original: To violate the limitation L4, the triple ( e 2 , r, e 1 ) should be negative i.e., (cid:107) e 2 + r − e 1 (cid:107) > γ 1 , → (cid:107) e 2 + r − e 1 (cid:107) 2 > γ 21 , → (cid:107) e 2 (cid:107) 2 + (cid:107) e 1 (cid:107) 

Revised: To violate the limitation L4, the triple ( e 2 , r, e 1 ) should be negative i.e., (cid:107) e 2 + r − e 1 (cid:107) > γ 1 , → (cid:107) e 2 + r − e 1 (cid:107) 2 > γ 21 , → (cid:107) e 2 (cid:107) 2 + (cid:107) e 1 (cid:107)

Original: + (cid:107) r (cid:107) 2 + 2 < e 2 , r > − 2 <
Revised: + (cid:107) r (cid:107) 2 + 2 < e 2 , r > − 2 <

Original: e 2 , e 1 > − 2 < e 1 , r > > γ 2 1 .
Revised: e 2 , e 1 > − 2 < e 1 , r > > γ 21 .

Original: Considering (cid:107) e 1 + r − e 2 (cid:107) =
Revised: Considering (cid:107) e 1 + r − e 2 (cid:107) =

Original: γ 1 , we have < e 2 , r > − < e 1 , r > > 0 , → [Equation] e [Equation]
Revised: γ 1 , we have < e 2 , r > − < e 1 , r > > 0 , → < e 2 − e 1 , r > > 0 , → cos ( θ ( e 2 − e 1 ) , r ) > 0 ,

Original: e

Original: [Equation]

Original: > 0 , → cos ( θ ( e [Equation] e 1 ) , r ) > 0 , Therefore, the limitation L4 is not valid i.e., if a relation r is reﬂexive, it may not be symmetric.
Revised: Therefore, the limitation L4 is not valid i.e., if a relation r is reﬂexive, it may not be symmetric.

Original: TransE is special case of TransComplEx and also condition b) is special case of condition c).
Revised: TransE is special case of TransComplEx and also condition b) is special case of condition c).

Original: Therefore using conditions b), c) and d), the limitation L4 is not valid for TransE and TransComplEx.
Revised: Therefore using conditions b), c) and d), the limitation L4 is not valid for TransE and TransComplEx.

Original: Lemma 5: 1) Under condition a), the limitation L5 holds for both TransE and TransComplEx. 2) Under conditions b), c) and d), L5 is not valid for both TransE and TransComplEx. proof 
Revised: Proof of the theorem T

Original: 1) Under condition a), equation h + r − t = 0 holds.
Revised: i) Under condition a), equation h + r − t = 0 holds.

Original: Therefore, according to the paper Kazemi &
Revised: Therefore, according to the paper Kazemi &

Original: Poole (2018), the model has the limitation L5.
Revised: Poole (2018), the model has the limitation L5.

Original: 2) If a relation is reﬂexive, with condition b), we have (cid:107) e 
Revised: ii) If a relation is reﬂexive, with condition b), we have (cid:107) e 1 + r − e 1 (cid:107) =

Original: [Equation]
Revised: γ 1 , (cid:107) e 2 + r − e 2 (cid:107) = γ 1 .

Original: Therefore, (cid:107) r (cid:107) =
Revised: Therefore, (cid:107) r (cid:107) =

Original: λ 1 .
Revised: λ 1 .

Original: Let
Revised: Let

Original: [Equation]
Revised: [Equation]

Original: we need to show the following inequality wouldn’t give contradiction: (cid:107) e 2 + r − e 3 (cid:107) > γ 1 .
Revised: we need to show the following inequality wouldn’t give contradiction: (cid:107) e 2 + r − [Equation] 3 (cid:107) > [Equation]

Original: From 22 we have < e 2 , ( e 1 + e 

Revised: From 19 we have < e 2 , ( e 1 + e

Original: + e 3 ) >< 0 , which is not contradiction.
Revised: + e 3 ) >< 0 , which is not contradiction.

Original: Therefore, with conditions b) and c), the limitation L5 is not valid for both TransE and TransComplEx.
Revised: Therefore, with conditions b) and c), the limitation L5 is not valid for both TransE and TransComplEx.

Original: Limitation L6: Lemma 6: 1) With condition (a), the limitation L6 holds for both TransE and
Revised: Proof of the theorem T6 : i) With condition (a), the limitation L6 is proved in Kazemi & Poole (2018).

Original: TransComplEx. 2) With conditions (b), (c) and (d), the limitation L6 doesn’t hold for the models. Proof : 1) With condition (a), the limitation L6 is proved in Kazemi & Poole (2018).
Revised: ii)

Original: Considering the assumption of L6 and the condition (b), we have
Revised: Considering the assumption of L6 and the condition (b), we have

Original: [Equation]
Revised: [Equation]

Original: We show the condition that (cid:107) e 2 + r − s 2 (cid:107) > γ 1 holds.
Revised: We show the condition that (cid:107) e 2 + r − s 2 (cid:107) > γ 1 holds.

Original: Substituting 23 in (cid:107) e 2 + r − s 2 (cid:107) > γ 1 , we have cos ( θ ( s 1 − s 2 ) , ( e 1 − e 2 ) ) < 0 .
Revised: Substituting 20 in (cid:107) e 2 + r − s 2 (cid:107) > γ 1 , we have cos ( θ ( s 1 − s 2 ) , ( e 1 − e 2 ) ) < 0 .

Original: Therefore, there are assignments to embeddings of entities that the limi- tation L6 is not valid with condition (b), (c) and (d).
Revised: Therefore, there are assignments to embeddings of entities that the limi- tation L6 is not valid with condition (b), (c) and (d).

Original: Figure 3 shows that the limitation L6 is invalid by proper selection of loss function.
Revised: Figure 6 shows that the limitation L6 is invalid by proper selection of loss function.

Original: A. F URTHER LIMITATIONS AND FUTURE WORK
Revised: C. F URTHER LIMITATIONS AND FUTURE WORK

Original: In the paper, we have investigated the six limitations of TransE which are resolved by revision of loss function.
Revised: In the paper, we have investigated the six limitations of TransE which are resolved by revision of loss function.

Original: However, revision of loss functions can resolve further limitations including 1-N, Nand M-N relations.
Revised: However, revision of loss functions can resolve further limitations including 1-N, Nand M-N relations.

Original: More concretely, setting upper-bound for the scores of positive samples can mitigate the M-N problem.
Revised: More concretely, setting upper-bound for the scores of positive samples can mitigate the M-N problem.

Original: We will leave it as future work.
Revised: We will leave it as future work.

Original: Our theories can be extended to every distance-based embedding models including RotatE etc.
Revised: Our theories can be extended to every distance-based embedding models including RotatE etc.

Original: Moreover, the negative likelihood loss has been shown to be effective for training different embedding models including RotatE and TransE. This can also be explained by reformulation of negative likelihood loss as standard optimization problem, showing the the loss put a boundary for the score functions.
Revised: Moreover, the negative likelihood loss has been shown to be effective for training different embedding models including RotatE and TransE. This can also be explained by reformulation of negative likelihood loss as standard optimization problem, showing the the loss put a boundary for the score functions.

Original: We will consider the mentioned points as future work.
Revised: We will consider the mentioned points as future work.

Revised: D O PTIMAL H YPER - PARAMETERS

Revised: The following tables show the optimal conﬁgurations for our models included in the Table-2 and 3.

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]