Rot-Pro: Modeling Transitivity by Projection in Knowledge Graph Embedding
Authors: Tengwei Song
Jie Luo
Lei Huang
Venue: NeurIPS 2021 Poster (2021)
Keywords: knowledge graph embedding
transitivity
projection
relation pattern

Abstract: Knowledge graph embedding models learn the representations of entities and relations in the knowledge graphs for predicting missing links (relations) between entities. Their effectiveness are deeply affected by the ability of modeling and inferring different relation patterns such as symmetry, asymmetry, inversion, composition and transitivity. Although existing models are already able to model many of these relations patterns, transitivity, a very common relation pattern, is still not been fully supported. In this paper, we first theoretically show that the transitive relations can be modeled with projections. We then propose the Rot-Pro model which combines the projection and relational rotation together. We prove that Rot-Pro can infer all the above relation patterns. Experimental results show that the proposed Rot-Pro model  effectively learns the transitivity pattern and achieves the state-of-the-art results on the link prediction task in the datasets containing transitive relations.

Rot-Pro: Modeling Transitivity by Projection in Knowledge Graph Embedding
Tengwei Song, Jie Luo, Lei Huang
State Key Laboratory of Software Development Environment Beihang University, Beijing, 100191 {songtengwei, luojie, huangleiai J
Abstract
Knowledge graph embedding models learn the representations of entities and re- lations in the knowledge graphs for predicting missing links (relations) between entities. Their effectiveness are deeply affected by the ability of modeling and inferring different relation patterns such symmetry, asymmetry, inversion, com- position and transitivity. Although existing models are already able to model many of these relations patterns, transitivity, very common relation pattern, is still not been fully supported. In this paper, we first theoretically show that the transitive relations be modeled with projections. We then propose the Rot-Pro model which combines the projection and relational rotation together. We prove that Rot-Pro can infer all the above relation patterns. Experimental results show that the proposed Rot-Pro model effectively learns the transitivity pattern and achieves the state-of-the results on the link prediction task in the datasets containing transitive relations.
Introduction
Knowledge graph embedding (KGE) aims to learn low-dimensional dense vectors to express the entities and relations in the knowledge graphs (KG). It is widely used in recommendation system, question answering, dialogue systems [5] 17 (7]. The general intuition of KGE is to model and infer relations between entities in knowledge graphs, which has complex patterns such as symmetry, asymmetry, inversion, composition, and transitivity as shown in Table
Many studies dedicate find method, which is able to model various relation patterns B 21 20 TransE models relations as translations, aims to model the inversion and composition patterns; DisMult can model symmetric relations by capturing interactions between head and tail entities and relations. One representative model proposed recently is RotatF 20),, which is proved to be able to model symmetry, asymmetry, inversion and composition patterns by modeling relation rotation in the complex plane. However, none of them can model all the five relation patterns, especially the transitivity pattern.
This paper focus on modeling the transitivity pattern. We theoretically show that the transitive relations can be modeled with idempotent transformations, i.e. projections [25]] Any projection matrix is similar to diagonal matrix with elements in the diagonal being 0 or 1. We design the
projection by constraining the similarity matrix to be rotation matrix, which has less parameters to learn
In order to model not only transitivity but also other relation patterns shown in Table I we propose the Rot-Pro model which combines the projection and relational rotation together. We theoretically prove that Rot-Pro can infer the symmetry, asymmetry, inversion, composition, and transitivity
Corresponding author.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Table 1: Common relation patterns.
Relation pattern
Definition
Symmetry
if (h, r, f), then (t,r,
Asymmetry
if (h, r,t, then-(t,
Inversion
if p- and (h, r.,r,)). t), then (t.p.
Composition
if = o...... 0..00 .r.
Ti, -TTT,10 (1) A (u,, u2) A (un-1, Tn, f). then rh, if (a, b) and (b, c) then(a, 7,
Transitivity
Table The supported relation patterns of several models
20
Symmetry
Asymmetry
Inversion
Composition
Transitivity
TransE
DistMult
x
x
ComplEx
/
X
xxx
RotatE
/
V
Rot-Pro
/
V V
patterns. Experimental results show that the Rot-Pro model can effectively learn the transitivity pattern. The Rot-Pro model achieves the state-of-the-ar results the link prediction task in the Countries dataset containing transitive relations and outperforms other models in the YAGO3-10 and FB15k-237 dataset.
Related work
There mainly two types of knowledge graph embedding models, either using translation transfor- mation or linear transformation between head and tail entities.
Trans-series models. Trans-series models, which is well-known KGE area, are essentially translation transformation based models. TransE 33 proposed pure translation distance-based score function, which assumes the added embedding of head entity and relation r should be close to the embedding of tail entity f. This simple approach is effective in capturing composition, asymmetric and inversion relations, but is hard to handle the 1-to-N, N-to-1 and N-N relations.
To overcome these issues, many variants and extensions of TransE have been proposed. TransH (26] projects entities and relations into a relation-specifie hyperplanes and enables different projections of an entity in different relations. TransR (28) introduces relation-specing spaces, which builds entity and relation embeddings in different spaces separately. TransD [9] simplifies TransR by constructs dynamic mapping matrices. For the purpose of model optimization, some models relax the requirement for translational distance. TransA [10] replaces Euclidean distance by Mahalanobis distance to enable directional semantics by complex relations.
adaptive metric learning, and recent model TransMS (18 transmits multi-
The variants of TransE improve the capability of the models to handle 1-to-N, N-to-1 and N-N relations as well as effectively modeling symmetric and asymmetric relations, but they are no longer able to model composition and inversion relations they do linear transformation entities separately before modeling the relation as translation. BoxE [1 recent trans-series model, embeds entities as points, and relations set of boxes, for yielding model that could express multiple relation patterns including composition and inversion, but it cannot express transitivity.
head and tail
Bilinear models. Models bilinear series model relations as linear transformation matrix M, from head to tail entity. The type of relation patterns that linear transformation based model can infer depends on the property of M,. RESCAL [14 proposes transformation of relation as matrix that models the pairwise interactions between entities and relation. The score of fact is defined by a bilinear function: fr h M, DistMult simplifies RESCAL by restricting M, to diagonal matrices. Therefore, it cannot handle other types of relations except symmetry. HolE

---

e
e2
r
e2
T
e3
e
4
3
Transitive Chain
(b) Limitation of TransF
(c) Limitation of RotatE
Figure Illustration of transitive chain and the limitation of TransE and RotatE on representing transitivity pattern.
12] combines the expressive power of RESCAL with the simplicity of DistMult which introduces cireular correlations. HolE can express multiple types of relations, since eyclic correlation operations are not commutative.
Recently, some KGE models begin to model relation patterns explicitly. Dihedral [27 models relations in KGs with the representation of dihedral group that has properties to support the relation as symmetry. To expand Euclidean space, ComplEx [21] firstly introduces complex vector space which can capture both symmetric and asymmetric relations. RotatE (20) also models in complex space and capture additional inversion and composition patterns by introducing rotational Hadmard product. QuatE [29] extends RotatE, using a quaternion inner product and gains more expressive semantic learning capability. ATTH 4 proposes low-dimensional hyperbolic knowledge graph embedding method, which capture logical patterns such as symmetry and asymmetrical.
However, none of existing models is capable of modeling transitivity relation pattern. We are the first to show that the transitivity can be modeled with projections, and we prove that the proposed model is able to infer all the five relation patterns shown in Table
Rot-Pro: Modeling Relation Patterns by Projection and Rotation
3.1 Preliminary
RotatE is representative approach that models a relation embedding с₁ of head entity to the embedding o₁ of tail entity in complex vector space. This can be denoted as (k) = rot(ex(k) 0, (k)), where 6, is the embedding of relation r and rot(c, (ke). (k)) the rotation function that rotates the kth element of e₁ with phase of the kth element of 0,., For each embedding let Re(e(k) and Tm(((()) be the real number and imaginary number of its kth dimension, the rotation transformation in the kth dimension defined by an orthogonal matrix whose determinant is 1, as follows:
element-wise rotation from the
(k)) [m(e)(k))
Re(rol(es (k), (k)())
Coos 0,(k)
sin Ox())
Re(en (k:)
(1)
Im(rol(es (k), (,), sin 0, (k) cos cos 0,) [m((eak))]
It has been proved that RotatE can infer symmetry, asymmetry, inversion, and composition relation patterns (20). However, cannot infer the transitivity pattern, and we will explain in what follows.
3.2 Representation of transitive relation
Relation a transitive relation, if for any instances (ei, r, e2) and (e2, r; ea) of relation r, (e1, r, ea) is also an instance of r. For convenience of illustration, we define the transitive chain of a transitive relation as follows.
Definition 1. transitive chain of r is defined as a chain of instances (2), 62).... (em- 1, T, em) ofr, where e Cm are different entities.
3

---

For the transitive closure of a transitive chain, every two entities in the chain should be connected by the transitive relation r. Hence, it can be represented as a fully connected directed graph with m(m-1) edges. It can be proved that a transitive relation can be represented as the union of transitive closures of all transitive chains. Thus, the representation of transitive relations can be reduced to the representation of transitive chains.
An example of transitive chain and its transitive closure are shown in Figure I(a] where
(e1, r, 22), (e2, r, eg); (e3, T; e1) form a transitive chain, (e1, n, e3) and (e2, r, e1) are instances derived by transitivity via one-hop, and (E1, F, e4) is the only instance derived via two-hops.
Due the speciality of transitivity, current models are unable to effectively model such transformation in vector space. For instance, in TransE (Figure [ where relation is regarded as a translation between the head and tail entities, it requires the translation to zero vector to model transitivity, which forces the embeddings of entities in a transitive chain to be the same. Thus, it cannot model transitivity. In RotatE (Figure [(c)). it requires the relational rotation phase (k) in each dimension to be 2n (n = 0, 1, ..) to model transitivity, which also forces the embeddings of entities to be the same in a transitive chain.
Our solution. Based on the observation on transitive closures of transitive chains, in each transitive chain (e1, T, e2), ..... (em-1, F' em), ) for each entity ej. (ej, r, ez++) can be derived by transitivity via 1 - I-hops (2 < / & m - j). If we model each relation r as kind of transformation Tr, then it requires (c,)) = C₁ for each relation instance (h, r, Therefore, the transformation of transitive relation must satisfies that (e,) (e,)
j m. & l transforming an entity embedding multiple times is equivalent to that of transforming it once. This inspires us model the transitivity pattern in terms of the idempotent transformation (projection (25)) which has the same property. For each relation r, let (,) be an invertible matrix on the kth dimension, a general orthogonal projection is defined by the idempotent matrix:
j). i.e. the result of
M,(k) (k)
[a (k)
0 (k),,,
(a, by
(k) b,)
1))
(2)
0
by (A))
[oos ( sin (k)
Without loss of generality, we simply set (k:)
cos 0p (k) to be rotation matrix,
sin (k))
which rotates the original axis by phase (k).. The orthogonal projection p. (k) defined by M,) is performed in the new axis after rotation:
Pr (x) + yi) = (k)
(3)
y
In the rest of paper, we will omit the dimensional indices (k) in M, (k) and (k)) for simplicity. In this way, for entities E...... Cm in transitive chain, we have p¹ (e, (k)) (e,e,))) (1 < < m, < / & m - j). This implies that (e: (k)) = ...... (em (k)), which does not force the entity embeddings ................................ ... e.m be the same. The embeddings e em can be different to each other and have the same projected vector under pr-
3.3 The Rot-Pro model
Model formulation. In order to model not only transitivity but also other relation patterns shown in T we combine the above projection based representation for transitivity and the relational rotation based representation for symmetry, asymmetry, inversion, and composition together. We propose Rot-Pro model relations as relational rotations on the projected entity embeddings on complex space Cd_ For each triple (h, r, t), the Rot-Pro model requires that (el( (k:)), (k)) = pr (k)).
(4) We demonstrate in the following theorem that Rot-Pro enables the modeling and inferring of all the five types of relation patterns introduced above.
Theorem 1. Rot-Pro can infer the symmetry, asymmetry, inversion, composition, and transitivity patterns.
Proof. (1) Let a, 1 and b, = 1, M becomes an identity matrix and becomes an identity transformation, and our model is reduced to the RotatE model. Thus, Rot-Pro can also infer the symmetry, asymmetry, inversion and composition patterns RotatE does 20)).

---

c05 (0)x (8)x Simos (k)y cx
12,00
83
Figure 2: The representation of transitivity pattern in complex plane.
(2) Here we will construct the solutions in Rot-Pro model for transitive relations. Let a, = 1 and b, O, p. is a projection to the real axis x' as shown in Figure 7 As discussed in previous section, to model the transitivity of relation r, the projected entity embeddings in transitive chain must satisty that rot (p((((((pp()) (k)) 0, (k))), (k)) rot(p,(e,(k)) (k)) p,-(e)2(kk) = pr (1)4) Therefore, the phase of relational rotation 0, (A:) can only be 2n (n 0, 1.2, and pr (k) (r- (I:) for any 1 < j E m.
According to Equation 4 the following equation is expected to hold.
[e(ee(k)) Mr
coos (k) sin 0, (k)
- sin (k) cos (1)
M, (ee(es(4))) = M, Reeei (k))
[m((e(k))
[m((e(()))
[mm(e(())
(5)
The above equation holds iff
cos 0z Re)) (k)) sin (p) (k)) = cos (())))((()()) sin (m) (6)
This equation holds if for any ej in the transitive chain,
cos (e) (e((e())) - sin Op (k) = Ck.
where c cos (k) Re(e: (R)) - sin (k)( (e (R)) constant. That is, all these entity embeddings e, (k) + yr in the transitive chain are located in the line defined by Equation 7 on the complex plane as shown Figure 2
cos (k) sin (k)y = ck-
Here, different value of c& represent different transitive chain. In summary, construct the solutions for representing transitivity in Rot-Pro model, i.e. a,-(k) = 1, b, (k) = 0, (k) = 2n sin
and for any entity embedding ej. it satisfies that cos 8 (o)e ( (k))mmm()))( where Cg is constant.
Ck+
Score function. For each triple r, t), the distance function of the Rot-Pro model is defined as following:
(e,, ex) [[oot(p (ex), 0.) (r)))) The score function fr ex) - (ex, ez).
3.4 Optimization objective
In the training process, we adopt the self-adversarial negative sampling. which has been proved as an effective optimization approach to KGE 20). The negative sampling loss L. with self-adversarial training defined as:
C, = - do (h,
poth, 7)
(9)
We can also set
0 and 1, then is projection the imaginary axis y'
5

---

where is fixed margin, or is the sigmoid function, r,)) is the jth negative instance and pl r, (2)) is the distribution for negative sampling (20).
In addition, to ensure the learned matrix to be projection, the values of a and b in Equation should
be restricted to 0 or 1. To enforce such constraint, we proposed projection penalty loss as follows:
[|]] 4p E (l](a) 1.0) o aj qill2 + Il(b; 1.0) @ by @ (.ll2)-
(10)
Here [R]] is the number of relations, @ the Hadamard product, and qi = {u, (1,():) where
(,(k) = if (k) 1.0) @ (x, (k) 0.0)] < 7mm otherwise (k(k) = 3 > 1. Here 7/m. and / are
hyper-parameters. We define qj to impose more penalty to values which are far from 0 or than that of values which close to 0 or 1.
Let be a thyee--paameete the total loss is defined the weighted average of the above two losses.
L = L. + a Ey.
(11)
Experiments
4.1 Datasets
We evaluate the Rot-Pro model on four well-known benchmarks. In general, FB15k-237 and WN18RR are two widely-used benchmarks and YAGO3-10 and Countries are two benchmarks with abundant relation patterns including transitivity.
FB15K-237: Freebase [1 contains information including people, media, geographical and locations. FB15k is subset of Freebase and FB15k-237 [24] is modified version
of FB15k, which excludes inverse relations resolve flaw with FB15k 23 It contains 14,541 entities, 237 relations, and 272, 15 training triples.
WNISRR: WNISRR [23 is a subset of WN18 (3) from WordNet [5] WordNet is a dataset that characterizes associations between English words. Compared with WN18, WNI8RR retains most of the symmetric, asymmetric and compositional relations, while removing the inversion relations. It contains 40,943 entities, 11 relations, and 86,835 training triples.
YAGO3-10: YAGO [9 is a dataset which integrates vocabulary definitions of WordNet with classification system of Wikipedia. YAGO3-10 [3] subset of YAGO, which contains 123.182 entities, 37 relations and 1,079,040 training triples. According to the ontology of YAGO3, it contains almost all common relation patterns.
Countries g is relatively small-scale dataset, which contains 2 relations and 272 entities (244 countries, 5 regions and 23 sub-regions). The two relations of Countries are locatedIn and neighborOf, which are transitive and symmetric relations respectively. The Countries dataset has tasks, each requiring inferring composition pattern with increasing length and difficulty.
Countries:
4.2 Evaluation protocol
We evaluate the KGE models on three common evaluation metrics: mean rank (MR), mean reciprocal rank (MRR), and top-k Hit Ratio (Hit @ k). For each valid triples (h, r, t) in the test set, replace either or with every other entities in the dataset to create corrupted triples in the link prediction task. Following previous work (3] 23 6 29 [6), all the models are evaluated in filtered setting, i.c, corrupt triples that appear in training, validation, or test sets are removed during ranking. The valid triple and filtered corrupted triples ranked in ascending order based their prediction scores. Lower MR, higher MRR or higher Hit@k k indicate better performance.
4.3 Experiment setup
With the hyper-parameters introduced, we train Rot-Pro using a grid search of hyper-paramente fixed margin y in Equation 9 E (0.1,4.0, 6.0, 9.0, 16.0, 20.0), weights tuning hyper-parameters
for loss, a {0.0001, 0.0005, 0.0008]. value of Ym in Equation 10 e {1e-6, 5e le 151 }) value

---

Table 3: Link prediction results on FB15k-237 and WNI8RR.
FB15k-237
WN18RR
MR
MRR
Hit@1
Hit@3
Hit@ 10
MR
MRR
Hit@1 1
Hit@3
Hit@ 10
TransE
357
.294
.655
3384
.226
.501
DistMult 2
254
.241
.155
.263
.419
5110
.43
.39
.44
.49
ComplEx
21
339
.247
.158
.275
.288
5261
.44
.41
.46
.51
ConvE
23
244
.325
.237
.356
.501
4187
.43
.40
.44
.52
RotatE
20
177
.338
.241
.375
.533
3340
.476
.428
.492
.571
BoxF
163
.337
.238
.347
.538
3207
.451
.400
.472
.541
Rot-Pro
201
.344
.246
.383
.540
2815
.457
.397
.482
.577
Table 4: Link prediction results YAGO3-10 and Countries.
YAGO3-10
Countries (AUC-PR)
MR
MRR
Hit@
Hit
Hit 10
S1
$2
$3
DistMult
2
5926
.24
.38
1.00
0.72
0.52
ComplEx
6351
.26
.40
0.97
0.57
0.43
ConvE
1671
.35
1.00
0.99
0.86
RotatE
1767
.402
.550
1.00
0.95
BoxE 1022
.560
.484
.608
Rot-Pro 1797
-542
.443
.596
.699
1.00
1.00
0.998
of in Equation
10 E 1,5, 2.03. Both the real and imaginary parts of the entity embed- dings are uniformly initialized, and the phases of the relational rotations are initialized between {(-#, TT. (- 2 x) In some settings, the phases of the relational rotations are also normalized to between {{ {(-m T. (- 2 } during training.
4.4 Main results
We compare Rot-Pro with several state-of-the-art models, including TransE 3.. DistMult 22, Com- plEx [21]. ConvE [23,, as well as RotatE [20] and BoxE [. empirically show the importance of being able to model and infer more relation patterns for the task of predicting missing links. Table B summarizes our results on FB 15k-237 and WNISRR, where results of baseline models are taken from Sun et al (20] and Ralph et al M We can see that Rot-Pro outperforms the baseline models on most evaluation metrics. Compared RotatE, the improvement of Rot-Pro limited since there no sufficient transitive relation defined on these two datasets, but the results are still comparable with other baseline models.
Table 4 summarizes our results on YAGO3-10 and Countries, which contain transitive relations. Hence the improvement of Rot-Pro over RotatE and other linear transformation models is much more significant. Specifically, Rot-Pro obtains better AUC-PR result than existing state-of-th ohee--f--teertt approaches, which indicates that Rot-Pro could effectively infer relation patterns such as transitivity, symmetry and composition. As a translation transformation model, BoxE outperforms Rot-Pro on YAGO3-10 on most evaluation metries, which indicates is also strong KGE model for inferring multiple relation patterns. However, the performance of BoxE on specific transitivity test sets is still not comparable with Rot-Pro, where additional experiments can be found in the appendix.
4.5 Validation of learned representation of transitive relations
We conduct further analysis the the Rot-Pro model verify that the model can actually learn the representations of transitive relations and have the theoretical property expected. To do this, we first investigate the distributions relational rotation phases in all dimensions of entity embeddings obtained by training on YAGO3-10. According to our theoretical analysis, expect the model could learn to represent transitivity, i.e. for any non-trivial projection (i.e. 1, the corresponding phase of relational rotation should be 2nm.. The experimental results are shown in Figure 3(a) can be observed that the Rot-Pro model does learn the relational rotation phases 0 and 2T expected. However, it also learns the unexpected relational rotation phase π. Further
or a 0, = 1),

---

3000
3000
3000
2000
2000
1000
1000
0
2m
(a) (-T, (-T, m)-Init
(b) ( 2 Init
(c) ( 2 5) -Tain
Figure 3: Distributions of relational rotation phases. The r-axis is the relational rotation phases. The y-axis is the number dimensions of relation embeddings that have non-trivial projection before rotation with specific phase, i.c. the embedding of parameter and b are 1, 0 or 0,
80
RotaE
RotaE
Rot-Pro-(fuun--Iit
50
Rot-Pro-(-nuuU-Ink
Mot-Pro-(nv22n122
Rot: (-2-(-n/2,nn/))nnnil
Htt@10/%
60
Rot Pro (m/2,m/2 Train
Hit@ (9%)
40
30
20
$3 (a) Result on transitive test sets
(b) Result on transitive test sets
Rodes Sann, Univentyy North Jesetta
0.10
State University
United States
North Americ
0.1
0.05
Americas
00.55
-0.1
-0.10
0.1
0.0
0.1
0.2
0.3
-0.1
(c) Rot-Pro-( (-T, IIII
(d) Rot- Cal-Pro-(-- (- 7 -
Figure 4: (a) shows the Hit@ 10 results of the RotatE and Rot-Pro models on three test sets for transitivity. (b) and (c) show the representation of four entities in transitive chain in two variants of Rot-Pro models with different constraints of relational rotation phase.
experiments reveal that by turning the initialization range of the relational rotation phases, the problem of learning unexpected relational rotation phase т could be mitigated. By changing the initialization range of relational rotation phases from (-7, π) (- 2' 2 π the number of relational rotation phases becomes significantly less. When we further restrict the relational rotation phases to (- 1-2.1 5) 2m.
during training, almost all relational rotation phases become
The results above are also reflected in the quantitative test. To fully understand the impact of changing the Rot-Pro model on modelling transitive relations,
initialization range on the performance construct three sub-test sets S1, S2, S3 of YAGO3-10 for evaluation, which consist of single transitive relation isocated... Test set S1 contains instances of isLocatedIn the original Test set 52 is obtained by applying the transitivity once instances of isLocatedIn in the YAGO3- 10 dataset. Test set S3 constructed similarly to S2, except by applying the transitivity least twice. We take the RotatE model baseline, and compare it with three variant of Rot-Pro models with different settings: the first one with relational rotation phase initialized in (-T, πT. the second one with relational rotation phase initialized in (- F) the third one with relational rotation phase restricted (- ] during training- The experimental result shown in Figure It shows that tuning of initialization range also largely improves the performance of the Rot-Pro model, which
set.

---

p(c.) Soopeeeee
p = (t)(p(())
p rot(p
T1
T2
e2
los5
22
(()).p(e)
p(
n
(a) Example illustration
(b) Variation of loss
Figure 5: (a) is example of miss-placed four entities transitive chain, which consist of three
triples: Tj: Ahfoida_Statee SiaateUniverssty,, isLocatedIn, United United_States), T2: (United Stattes isLocatedIn, North_ America, T3: (North_Ameriic isLocatedIn, Americas). (b) is the variation of loss for these triples.
150
100
5
0
(a) (-T,, Init
(b)
21 > Init
(c) 2
Train
Figure 6: The distribution of relational rotation phases of three Rot-Pro variants over all dimensions of specific symmetric relation isMarriedTo. The meaning of x and axes is the same as Figure
coincides with the improvement of learning correct representations of transitive relations. All the variant of Rot-Pro models outperform RotatE significantly, especially when the relational rotation phase restricted to (- during training.
We also visualize one dimension of embeddings of three entities connected by a transitive relation isLocatedIn YAGO3-10. F shows the visualization entity embeddings model trained with initialization range (- (-,,=), which contains miss placed entity embedding. While Figure [dd] is the visualization embeddings of entities the Rot-Pro model trained by restricting the relational rotation phase to (- I. 2:27 T) during training, where all entity embeddings in the transitive chain are placed correctly as expected. We can see that these vectors are basically fit in line and can almost be projected to the same vector in the rotated axis.
Rot-Pro
Explanation. The Rot-Pro model with additional restrictions the relational rotation phase may learn a phase л which does not exactly meet our expectation. A possible representations of four entities in transitive chain with relational rotation phase is illustrated in Figure 5(a) in which four out of the six instances of transitive relation are correctly represented, while the other two instances (e F; e3) and (e2, r, e4) not. Obviously, this is not an optimal solution for the model, and the reason is likely to be that the model falls into local optimum during the learning process. To demonstrate this, we plot the variation of loss for three triples in transitive chain with the relational rotation phase range over (0, 2T). The result is shown in Figure 5(b) We can find that there is indeed and 2n, which consistent with our conjecture.
local optimum at n, and the global optimum
4.6 Limitation
According to the experimental results, Rot-Pro sensitive the range relational rotation phases, and hence prone to fall into the local optimum solution. Though it can learn the idempotency

---

of transitivity correctly by enforcing addition constraints on training, however, such constraints have negative impact on the learning of other relation patterns, such as symmetry. We can find in ( 6(a)]tar for symmetric relation, the relational rotation phases learned by a Rot-Pro without phase constraint are either 0, л or 2n, which indicates that it has similar capability of modeling and inferring symmetry relation pattern as RotatE. By narrowing of the range of relational rotation phases, the histogram on the symmetry relation is gradually disrupted as shown in Figure (6b) and 6(c) Therefore, trade-off should be made between the better modeling and inferingg of transitivity and the other relation patterns, Such limitation might be further optimized through learning each relation pattern separately and integrate through mechanisms such attention, which we will study in future works.
Figure
5 Conclusion and Future Work
In this paper, we theoretically showed that the transitive relations can be modeled with projections that is idempotent transformation. We also theoretically proved that the proposed Rot-Pro model able to infer the symmetry, asymmetry, inversion, composition, and transitivity patterns. Our experimental results empirically showed that the Rot-Pro model can effectively learn the transitivity pattern. Our model also has the potential to be improved by extending the complex space to higher dimension space, such quaternion space [29 22). While the proof of expressiveness in many previous works is focused on the expressiveness of each relation pattern separately, it is worthwhile to further investigate whether a model can handle all common relation patterns simultaneously, considering that a single relation may exhibit multiple relation patterns and different relation patterns may have complex interactions in knowledge graphs.
Acknowledgments and Disclosure of Funding
This work was supported by the National Key Research and Development Plan of China (Grant No. 2018AAA0102001 and the National Natural Science Foundation of China (Grant No. 61690202).
References
[1] Ralph Abboud, ismail ilkan Ceylan, Thomas Lukasiewicz, and Tommaso Salvatori. BoxE: box embedding model for knowledge base completion, 2020.
[2] X. He J. Gao B. Yang, Wit. Yih and L. Deng. Embedding entities and relations for learning and inference in knowledge bases. In ICLR, pages 1-13, 2015.
[3] Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems 26, pages 2787-2795, 2013.
[4] Ines Chami, Adva Wolf, Da-Cheng Juan, Frederic Sala, Sujith Ravi, and Christopher Ré. Low- dimensional hyperbolic knowledge graph embeddings. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6901-6914. Association for Computational I. Linguistics, July 2020.
[5] L. Chen, G. Zeng, Q Zhang, X. Chen, and D. Wu. Question answering over knowledgebase with attention-based Istm networks and knowledge embeddings. In 2017 IEEE 16th International Conference on Cognitive Informatics and Cognitive Computing (ICCI*CC), pages 243-246. 2017.
[6] Dat Quoc Nguyen Dai Quoe Nguyen. Tu Dinh Nguyen and Dinh Phung. A novel embedding model for knowledge base completion based on convolutional neural network. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL), volume 2, page 327-333, 2018
[7] Kamal Singh Dennis Diefenbach and Pierre Maret. WDAqua-corel: a question answering service for RDF knowledge bases. Companion of the The Weh Conference 2018 on The Web Conference (WWW), page 1087-1091, 2018.
10

---

[8] Sameer Singh Guillaume Bouchard and Theo Trouillon. On approximate reasoning capabilities of low-rank vector spaces. In AAAI Spring Syposium on Knowledge Representation and Reasoning (KRR): Integrating Symbolic and Neural Approaches, 2015.
[9] Liheng Xu Kang Liu Guoliang Ji, Shizhu He and Jun Zhao. Knowledge graph embedding via dynamic mapping matrix. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 687-696, 2015.
[10] Y. Hao H. Xiao, M. Huang and X. Zhu, TransA: An adaptive approach for knowledge graph embedding, In Proceedings of AAAI Conference on Artificial Intelligence, pages 1-7, 2015.
[11] P. Paritosh T. Sturge K. Bollacker, C. Evans and J. Taylor. Freebase: a collaboratively created graph database for structuring human knowledge. In SIGMOD, page 1247-1250, 2008.
[12] L. Rosasco M. Nickel and T. Poggio. Holographic embeddings of knowledge graphs. In Proceedings of AAAI conference on artificial intelligence, page 1955-1961, 2016.
[13] Biega J. Mahdisoltani, and F. M. Suchanek. YAGO3: A knowledge base from multilingual
wikipedias. Proceedings of CIDR 2015, 2015.
[14] Volker Tresp Maximilian Nickel and Hans-Peter Kriegel. A three-way model for collective learning on International Conference Machine Learning, (ICML), 2011.
data. In Proceedings of the 28th International Conference on
[15] George A Miller. Wordnet: a lexical database for english. In Communications of the ACM, 38( I1), page 39-41, 1995.
[16] Deepak Nathani, Jatin Chauhan, Charu Sharma, and Manohar Kaul. Learning attention-based embeddings for relation prediction in knowledge graphs. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4710-4723, 2019.
[17] K. Liu S. He, C. Liu and J. Zhao. Generating natural answers by incorporating copying and learning. In ACL. page 199-208, 2017.
retrieving mechanisms in sequence-to-sequeee
[18] H. Zhang J. Yan H. He S. Yang. J. Tian and Y. Jin. TransMS: knowledge graph embedding for complex relations by multidirectional semantics. In Proceedings of IJCAL, page 1945-1942. 2019.
[19] Fabian M. Suchanek, Gjergji Kasneci, and Gerhard Weikum. Yago: A core of semantic knowledge. In Proceedings of the 16th International Conference on World Wide Web, page 697-706, 2007.
[20] Zhiqing Sun, Zhi-Hong Deng, Jian- Nie, and Jian Tang. RotatE: Knowledge graph em- bedding by relational rotation in complex space. In International Conference on Learning Representations, 2019.
[21] S. Riedel F. Gaussier T. Trouillon, J. Welbl and G. Bouchard. Complex embeddings for simple link prediction. In Proceedings of 33rd Int. Conf. Mach. Learn, page 2071-2080, 2016.
[22] Yun Tang, Jing Huang, Guangtao Wang. Xiaodong He, and Bowen Zhou. Orthogonal relation transforms with graph context modeling for knowledge graph embedding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages Online, July 2020. Association for Computational Linguistics.
[23] Pontus Stenetorp Tim Dettmers, Pasquale Minervini and Sebastian Riedel. Convolutional 2D knowledge graph embeddings. Proceedings of the 32nd AAAI Conference on Artificial Intelligence, 2018.
[24] Kristina Toutanova and Danqi Chen. Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality, page 57-66, 2015.
[25] R.J. Valenza. Linear Algebra: An Introduction to Abstract Mathematics. Undergraduate Texts in Mathematics. Springer New York, 2012.

---

[26] Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowledge graph embedding by translating on hyperplanes. AAAI Conference on Artificial Intelligence, 2014.
[27] C. Xu and R. Li. Relation embedding with dihedral group in knowledge graph. Proceedings of ACL, pages 263-272, 2019.
[28] Maosong Sun Yang Liu Yankai Lin, Zhiyuan Liu and Xuan Zhu. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of the 29th AAAI Conference on Artificial Intelligence, pages 2181-21888 2015.
[29] Shuai Zhang, Tay, Lina Yao, and Liu. Quaternion knowledge graph embedding. Advances in Neural Information Processing Systems, page 2731-2741 2019.
12


Reviews:
[{"review_id": "zUG2oElYe6g", "reviewer": "Reviewer_VTBj", "summary": "When embedding entities and relations in knowledge graphs, operations such as translations, matrix-vector products, quadratic forms, etc are used to represent relations between entity vectors. While these vector-based methods are flexible, they do not specifically encode an inductive bias towards transitive relations that is necessary for modeling certain hierarchies. This paper proposes an approach based on projections (idempotent transformations) and rotations taken together, which allows it to exactly model transitivity in relations.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Update: I will stick with my current score, I believe that the authors have adequately enough addressed reviewers' comments.\n\n**Originality**\n\nThere has been much work on modeling transitive relations with specialized embedding techniques such as order embeddings, hyperbolic embeddings, box embeddings etc. However, integrating these approaches into standard multirelational / hypergraph KG embedding based on vectors, which allows for multiple heterogeneous relations to be embedded, is an open problem. As the authors note, the recent RotatE model is able to model (a)symmetry, composition, and inversion, but cannot specifically guarantee transitivity. They add a learned subspace projection step to the RotatE model and prove this can represent transitive relations. \n\n**Quality**\n\nThe theoretical argument is sound. The evaluation is reasonable and standard for KG embedding techniques, and showing the model can indeed infer transitivity is welcome. The discussion of limitations and local optima in the model is welcome.\n\n**Clarity**\n\nThe paper is mostly well written and argued. The final error-analysis section could use some light editing to tighten it up, as it currently reads more like an appendix discussion.\n\n**Significance**\n\nThe RotatE model is simple and powerful, and this extension of it seems similarly simple and powerful. Knowledge base embedding approaches are popular and this might become a new baseline technique. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations with respect to local optima in optimization, and initialization, are discussed extensively in the experiments.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1"}, "extra_text": {}}, {"review_id": "vPtfoiJ1bC5", "reviewer": "Reviewer_uEYp", "summary": "This paper noticed that if a relation projected orthogonally to the translation direction, it becomes transitive, and then make this insight work with other relational patterns.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I liked this paper. It had a good insight and seems to work well. It has good originality, and is mostly clear. It is potentially significant.\n\nHowever, I am very suspicious of whether there are side effects. For example, ancestor is the transitive closure of parent. Because ancestor has to have some particular form in order to be transitive, doesn't that imply that parent must also have some particular form?  What happens when parent is used in other relations? While I believe your theorem (although I didn't check the related papers), it doesn't show that they can be handled simultaneously. If there was also a modularity result, the consequent of the design patterns do not affect the representation of the condition  of the design pattern  (i.e., they work for all representations of the condition), I would be more convinced of your claim that you can handle all relation patterns. (Claiming to handle all is ambiguous as to whether you can handle them separately or together.)\n\nIsn't transitivity an instance of composition, with r = r \\circle r?  I think there are more assumptions behind the design patterns than you state.\n\nDo your results imply other patterns, eg  r = r1 \\circle r?  Are there other patterns that can exist? You should tell us whether this is the end of the story of representing design patterns of whether there are more. (Students looking for research problems want to know).\n\nI very much appreciated section 4.6 on limitations.\n\nPlease label x and y axes of all figures. Figure 5 b has loss on the y-axis but how can it be negative?  \n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes. Section 4.6 was nice to see! ", "review_text": "", "extra_scores": {"ethical_concerns": "No ethical issues.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "L9RNzAz6-Hi", "reviewer": "Reviewer_99hH", "summary": "This paper is related to the problem of Knowledge Graph (KG) Embedding. It seems that none of the existing approaches for KG embedding is capable of modeling the transitivity property of the relation which is an important property for many relations to have in real-world KGs. The aim of this paper, therefore, is to bridge this specific research gap. \n\nThis paper proposes a KG embedding model called __Rot-Pro__ which not only is capable of modeling the transitive property of a relation but also other properties including symmetry, asymmetry, inversion, and composition. The approach is based on applying a projection operator on the embedding of the head and tail entities before applying RotatE so as to be able to model transitive relations. The experiments are performed on two well-known datasets FB15k-237 and WN18RR. The experiments are also performed on two other datasets called YACO3-10 and Countries because they have lots of transitive relations present in them. \n\nOn the datasets, where transitive relations are dominant, the approach seems to do justice but the approach seems to be facing some hiccups in the presence of other types of relations such as symmetry.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "__Originality__\n- The problem of KG embedding is a well-known research topic in the field. This paper identifies an important limitation of the existing approaches and tries to fix that.\n- The proposed approach in this paper builds on top of the well-known approach called RotatE. The proposed approach suggests adding one extra pre-processing step to the entity embeddings before calling the routine of RotatE.  The pre-processing step suggests that all the entity embeddings should be first (orthogonally) projected into a subspace of the embedding space via projection matrices. These projection matrices are specific to each relation as well as each embedding dimension. This extra step introduces only 3 extra model parameters ($\\theta_p(k)$ and $a$ or $b$) per embedding dimension. \n- This extra step alone allows RotatE to model the transitive property of relations. Overall, this extra step, though seemingly small, does offer the desired effect (at least theoretically). Theorem 1 confirms this in a theoretical manner.  \n- In light of the above, I feel the contributions can be considered somewhat novel. \n- However, there are several doubts/clarifications that pop in the mind of a reader (listed below) and clarification of them will help me judge the merits of this work in an informed manner.\n\n__Quality__\n- The overall technical quality of the paper seems reasonable. The supported claim is proven theoretically and also validated experimentally. Although experiments don't support it fully and I guess that is scope for future research.\n- Authors are honest about articulating the weakness of the method observed during their experimentation.\n\n__Clarity__\n\nOverall, I felt that this paper lacks clarity and has ample scope for improving its writing. Below are some of the doubts/clarifications that struck me while reading the paper.\n\n- In equation (2), I assume the matrix $M$ would be different for different coordinate axes as well as relations? If yes, you may want to use the symbol $M_r(k)$ or $M_{rk}$, where $k$ is the index of a coordinate axis and $r$ denotes the considered relation. Given that you have used index $k$ in the definition for the parameter $\\theta_p(k)$, introducing $k$ for $M$ will make notations a bit more consistent. In fact, by the same token, matrices $S, S^{-1}$, as well as parameters $a$ and $b$ should also bear the index $k$ and $r$. Please correct me if I have misunderstood something here.  \n- By the same logic, even for Equation (3), you may want to consider using the symbol $p_{rk}$ or $p_r(k)$ if $M$ is indexed with $r$ and $k$ on the right side of the equation? If these notions become clumsy then you may want to add a disclaimer saying that you are omitting some indices to keep the notation less cluttered.\n- While I could infer from reading that Rat-Pro model embeds entities into a complex space ${\\cal{C}}^d$, would it make sense to state it explicitly somewhere? \n- After reading the last line of the _Conclusion_ section, I am a bit confused as it seems to suggest that the current embedding is being performed in ${\\cal{C}}^1$ space? Table 2 of the appendix seems to suggest that $d>1$. Please clarify this.\n- In Equation (10), the outer summation is over variable $x$ which doesn’t show up in summand term. Can you make it clear?\n- In equation (10), it's not clear what is $\\mathbf{x}_j$ vector and what is its dimension? If running index $j$ corresponds to an axis then I believe $\\mathbf{x}_j$ would be a scaler? But in that case, Hadamard product does not make sense. \n- What benefits do you get by using the loss form given in Equation (10) as opposed to minimizing other simpler forms such as $\\mathbf{x}_j \\odot (1-\\mathbf{x}_j)$ which also try to push the values of $\\mathbf{x}_j$ towards $0$ or $1$?\n- Given that the overall optimization problem becomes an integer programming problem ($a, b \\in {0,1}$),  how do you finally ensure that in the learned matrix M, the diagonal terms $a$ and $b$ are exactly $0$ or $1$. I suppose the inclusion of loss term (10) still does not guarantee to have the learned values of the parameters $a,b$ being $0$ or $1$. Do you round off the learned values for $a$ and $b$? \n- The projection matrix $M$ seems to be orthogonal projection matrix also because it appears that $S^{-1} = S^{\\top}$ and hence $M=M^{\\top}$. In this case, it may be better to explicitly state that you are focusing on the orthogonal projections of the entities.\n-  In Figures 3(a), 3(b), and 3(c), what does the $y$-axis corresponds to? \n- Sentence in line # 238 needs some rewriting as it doesn’t read properly.\n- In Figure 4(a), how does the pattern for Hits@1 score look like? Does it follow the same trend as Figure 4(a)? For KG embeddings, I feel Hits@1 is crucial to track.\n- How about the expressivity of the Rat-Pro model? Is there any relation that can’t be model by the proposed approach? Can you add some discussion around that?\n\n__Significance__\n\n- Overall, this paper could have a noticeable impact in advancing the SOTA provided it satisfactorily addresses the doubts/concerns raised above. \n- The experimental section exposes a limitation saying that the presence of transitive relations dilutes the learning of other types of relations (as discussed in Section 4.5 and 4.6). These limitations kind of puts a question mark saying whether this paper managed to bridge the research gap completely. However, I feel, if not completely, this paper still managed to bridge the research gap and advance the SOTA to a decent and respectable level.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes. The authors have explicitly discussed the limitations of the proposed method which they observed during experimentation. There are a few minor clarifications I have in this regard which I have listed in my points above.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "Fz2zIA3eWBn", "reviewer": "Reviewer_NmZc", "summary": "Summary: This study proposes a new geometric model Rot-Pro that enables the learning of transitive inference patterns by combining rotation and projection. The core idea is to involve a learnable idempotent projection onto a reduced basis. This achieves the transitivity pattern without forcing the entity embeddings to be the same as it is in previous models. The study shows the effectiveness in experimental results mostly on the small Countries dataset, but also shows that the model performs well on larger KG benchmarks especially on the Yago3-10 dataset. The authors also report their investigation into initialization schemes that are required for successfully training the model. ", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: There is a lot of recent work in geometric models that explicitly investigate modelling certain inference patterns and this study proposes to investigate transitivity in a rotational model that expands its capabilities to more inference patterns. I was missing a comparison and discussion to BoxE, which is a strong model in this area.\n\nQuality: The line of arguments and to motivate the study are well laid out, and proofs and experiments support the claim. The authors point out the challenge of initialization for the model and they mention a special loss to enforce zeros and ones in the projection. Did the authors also consider something like [1]? In the experimental results I was also missing a comparison to BoxE, which reports better results on Yago3-10.\n \nClarity: The paper was written very well and has a nice presentation. \n \nSignificance: The paper presents a solid idea which is executed well to show the effectiveness of explicitly allowing a model to learn transitive patterns in the data. While the authors note that there is room for improvement, this does not limit the usefulness of the analysis, proposed model and results. \n \n[1] Louizos, C., Welling, M., and Kingma, D. P. Learning sparse neural networks through l0 regularization. CoRR, 2017.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The limitations and societal impact haven been  addressed.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1.5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: the reviewers agreed that the paper makes a solid contribution by proposing a model that can infer transitivity, together with convincing experimental results. Most of the reviewers' concerns have been cleared in the discussion phase. The authors should make sure to incorporate the necessary updates in the camera-ready version.


Decision: Accept (Poster)