Compounding Geometric Operations for Knowledge Graph Completion
Year: 2023

Abstract:
Geometric transformations including translation, rotation, and scaling are commonly used operations in image processing. Besides, some of them are successfully used in developing effective knowledge graph embedding (KGE). Inspired by the synergy, we propose a new KGE model by leveraging all three operations in this work. Since translation, rotation, and scaling operations are cascaded to form a composite one, the new model is named CompoundE. By casting CompoundE in the framework of group theory, we show that quite a few distanced-based KGE models are special cases of CompoundE. CompoundE extends the simple distance-based scoring functions to relation-dependent compound operations on head and/or tail entities. To demonstrate the effectiveness of CompoundE, we perform three prevalent KG prediction tasks including link prediction, path query answering, and entity typing, on a range of datasets. CompoundE outperforms extant models consistently, demonstrating its effectiveness and flexibility.1

heading: 1 Introduction
text: Knowledge graphs (KGs) such as DBpedia (Auer et al., 2007), YAGO (Suchanek et al., 2007), NELL (Carlson et al., 2010), Wikidata (Vrandečić and Krötzsch, 2014), Freebase (Bollacker et al., 2008), and ConceptNet (Speer et al., 2017) have been created and made available to the public to facilitate research on KG modeling and applications. Triples, denoted by (h, r, t), are basic elements of a KG, where h and t are head and tail entities while r is the relation connecting them. KG representation learning, also known as knowledge graph embedding (KGE), has been intensively studied in recent years. Yet, it remains one of the most fundamental problems in Artificial Intelligence (AI) research.
∗Corresponding author 1Our source code is available at https://github.c
om/hughxiouge/CompoundE
KGE is critical to many downstream applications such as question answering (Guu et al., 2015) knowledge integration (Chen et al., 2017), text analysis (Li et al., 2019), entity classification (Zhao et al., 2020; Ge et al., 2022), etc. There are several challenges in the design of good KGE models. Complex relation patterns (e.g. 1-to-N, N-to-1, and N-to-N, antisymmetric, transitive, hierarchical relations, etc.) remain difficult to model. Also, each of extant KGE models has its own strengths and weaknesses. It is desired yet unclear how to design a KGE model that leverages strengths of some models and complements weaknesses of others.
Geometric operations such as translation and rotation belong to the family of affine transformations. These operations have been used to build effective KGE models such as TransE, RotatE, and PairRE. Previous KGEs often use a single type of operation to model all relation patterns with different properties. This could be problematic since each operator may have modeling limitations. A synergy of different transformations may complement the weaknesses of individual operators. In fact, generic compound operations yielded from a cascade of affine transformations find numerous applications in image processing (Pratt, 2013), including image warping (Wolberg, 1990), image morphing (Seitz and Dyer, 1996), and robot motion planning (LaValle, 2006). Motivated by the synergy, we propose a new KGE model to address the above-mentioned challenges. Since translation, rotation, and scaling operations are cascaded to form a compound operation, the proposed KGE model is named CompoundE. Compound operations inherit many desirable properties from the affine group, allowing CompoundE to model complex relations in different KGs.
There are four main contributions of this work. They are summarized below.
• We present a novel KG embedding model called CompoundE, which combines three
6947
fundamental operations in the affine group and offers a wide range of designs.
• It is proved mathematically that CompoundE can handle complex relation types in KG thanks to unique properties of the affine group.
• We apply CompoundE to perform three important KG prediction tasks, including link prediction, path query answering, and entity typing on widely adopted KG benchmarking datasets extracted from Freebase, WordNet, Wikidata, and YAGO. CompoundE consistently outperforms prior work.
• Against large-scale datasets containing millions of entities under the memory constraint, CompoundE outperforms other benchmarking methods by a big margin with fewer parameters.
The rest of this paper is organized as follows. Recent KGE models for both distance-based and entity-Transformation-based categories are first reviewed in Section 2. Then, we present CompoundE, show its relationship with previous KG embedding models, and explain the reason why it can model complex relations well in Section 3. Experiment details and performance comparisons are given in Section 4. Finally, concluding remarks are given and possible extensions are suggested in Section 5.

heading: 2 Related Work

heading: 2.1 Distance-based Models
text: Distance-based scoring function is a prevailing strategy in optimizing KGE. The main idea is to model a relation as a transformation that places head entity vectors in the proximity of their corresponding tail entity vectors, and vice versa. For a given triple, (h, r, t), the goal is to minimize the distance between h and t vectors after the transformation introduced by r. TransE (Bordes et al., 2013) is one of the earlier KGE models that interpret relations between entities as translation operations in the vector space. However, this simple approach is ineffective in modeling 1-to-N, N-to-1, N-to-N, and symmetric relations. To better encode complex relations, TransH (Wang et al., 2014) enables the relation-specific entity representation by projecting each entity to a relation-specific hyperplane. In order to represent symmetric relations, RotatE (Sun et al., 2019) models entities in the complex vector
space and interprets a relation as a rotation instead of a translation. The self-adversarial negative sampling contributes to RotatE’s performance improvement as compared to its predecessors. However, RotatE does not handle the hierarchical structure appearing in many KGs. MuRP (Balažević et al., 2019) and ROTH (Chami et al., 2020) leverage the power of hyperbolic curvature to better capture the hierarchical structure in KGs.

heading: 2.2 Entity-Transformation-based Models
text: Adding relation-specific transformation to baseline models is another popular line of work. Although TransH (Wang et al., 2014) enables the translation approach to model complex relations, the orthogonal projection prevents the model from encoding inverse and composition relations. Instead, TransR (Lin et al., 2015) models relations and entities in two different spaces. However, the relationspecific transformation is a dense matrix and significantly increase parameter complexity. To alleviate this problem, TransD (Ji et al., 2015) comes up with dynamic mapping matrices using relation and entity projection vectors. Similarly, TranSparse (Ji et al., 2016) enforces the relation projection matrix to be sparse. More recently, PairRE (Chao et al., 2021) performs transformations on both heads and tails. Specifically, head and tail entities had a Hadamard product with their respective weight vectors rH and rT. This elementwise multiplication is nothing but the scaling operation. SFBR (Liang et al., 2021) introduces a semantic filter that includes a scaling and translation component. STaR (Li and Yang, 2022) adopts a similar approach but adds a rotation operator to design the bilinear product matrix for semantic matching scoring functions. ReflectE (Zhang et al., 2022a) models a relation as the Householder reflection. Different composition of geometric operations inspire us to invent CompoundE embedding in this paper.

heading: 3 Methodology
text: Translation, rotation, and scaling transformations appear frequently in engineering applications. In image processing, a cascade of translation, rotation, and scaling operations offers a set of image manipulation techniques. Such compound operations can be used to develop a new KGE model called CompoundE. We provide an illustration of CompoundE and comparison with previous KGE in Fig. 1.

heading: 3.1 Definition of CompoundE
text: Three forms of CompoundE scoring function can be written as
• CompoundE-Head
fr(h, t) = ∥Tr ·Rr · Sr · h− t∥, (1)
• CompoundE-Tail
fr(h, t) = ∥h− T̂r · R̂r · Ŝr · t∥, (2)
• CompoundE-Complete
fr(h, t) = ∥Tr ·Rr · Sr · h− T̂r · R̂r · Ŝr · t∥, (3)
where h, t denote head and tail entity embeddings, Tr,Rr,Sr denote the translation, rotation, and scaling operations for the head entity embedding, and T̂r, R̂r, Ŝr denote the counterparts for the tail entity embedding, respectively. These constituent operators are relation-specific. To generalize, any order or subset of translation, rotation, and scaling component can be a valid instance of CompoundE. Since matrix multiplications are non-commutative, different orders of cascading the constituent operators result in distinct CompoundE operators. Performance difference between these variations are discussed in Section B of the appendix.

heading: 3.2 CompoundE as An Affine Group
text: Most analysis in previous work was restricted to the Special Euclidean Group SE(n) (Cao et al., 2022). Yet, we will show that CompoundE is not a special Euclidean group but an affine group. To proceed, we first formally introduce the lie group and three special groups as below.
Definition 3.1. A Lie group is a continuous group that is also a differentiable manifold.
Several Lie group examples are given below.
• The real vector space, Rn, with the canonical addition as the group operation.
• The real vector space excluding zero, (R\{0}), with the element-wise multiplication as the group operation.
• The general linear group, GLn(R), with the canonical matrix multiplication as the group operation.
Furthermore, the following three special groups are commonly used.
Definition 3.2. The special orthogonal group is defined as
SO(n) = { A ∣∣∣∣A ∈ GLn(R),A ⊤A = I, det(A) = 1 } .
(4)
Definition 3.3. The special Euclidean group is defined as
SE(n) = { A ∣∣∣∣A = [ R v 0 1 ] ,R ∈ SO(n),v ∈ Rn } .
(5)
Definition 3.4. The affine group is defined as
Aff(n) = { M ∣∣∣∣M = [ A v 0 1 ] ,A ∈ GLn(R),v ∈ Rn } . (6) By comparing Eqs. (5) and (6), we see that SE(n) is a subset of Aff(n).
Without loss of generality, consider n = 2. If M ∈ Aff(2), we have
M = [ A v 0 1 ] ,A ∈ R2×2,v ∈ R2. (7)
The 2D translational matrix can be written as
T =   1 0 vx 0 1 vy 0 0 1  , (8)
while the 2D rotational matrix can be expressed as
R =   cos(θ) − sin(θ) 0 sin(θ) cos(θ) 0
0 0 1
  . (9)
It is easy to verify that they are both special Euclidean groups (i.e. T ∈ SE(2) and R ∈ SE(2)). On the other hand, the 2D scaling matrix is in form of
S =   sx 0 0 0 sy 0 0 0 1   . (10)
It is not a special Euclidean group but an affine group of n = 2 (i.e., S ∈ Aff(2)).
Compounding translation and rotation operations, we can get a transformation in the special Euclidean group,
T ·R =   1 0 vx 0 1 vy 0 0 1     cos(θ) − sin(θ) 0 sin(θ) cos(θ) 0 0 0 1  
=   cos(θ) − sin(θ) vx sin(θ) cos(θ) vy
0 0 1
  ∈ SE(2). (11)
Yet, if we add the scaling operation, the compound will belong to the Affine group. One of such compound operator can be written as
T ·R · S =   sx cos(θ) −sy sin(θ) vx sx sin(θ) sy cos(θ) vy
0 0 1
  ∈ Aff(2).
(12) When sx ̸= 0 and sy ̸= 0, the compound operator is invertible. It can be written in form of
M−1 = [ A−1 −A−1v 0 1 ] . (13)

heading: 3.3 Relation with Other Distance-based KGE Models
text: CompoundE is a general form of quite a few distance-based KGE models. That is, we can derive their scoring functions from that of CompoundE by setting translation, scaling, and rotation operations to certain forms. Four examples are given below. Derivation of TransE (Bordes et al., 2013). We begin with CompoundE-Head and set its rotation
component to identity matrix I and scaling parameters to 1. Then, we get the scoring function of TransE as
fr(h, t) = ∥Tr · I · diag(1) · h− t∥ = ∥h+ r− t∥. (14)
Derivation of RotatE (Sun et al., 2019). We can derive the scoring function of RotatE from CompoundE-Head by setting the translation component to I (translation vector t = 0) and scaling component to 1.
fr(h, t) = ∥I ·Rr · diag(1) · h− t∥ = ∥h ◦ r− t∥. (15)
Derivation of LinearRE (Peng and Zhang, 2020). We can add back the translation component for the head transformation:
fr(h, t) = ∥Tr · I · Sr · h− I · I · Ŝr · t∥ = ∥h⊙ rH + r− t⊙ rT∥. (16)
Derivation of PairRE (Chao et al., 2021). CompoundE-Complete can be reduced to PairRE by setting both translation and rotation component to I, for both head and tail transformation.
fr(h, t) = ∥I · I · Sr · h− I · I · Ŝr · t∥ = ∥h⊙ rH − t⊙ rT∥. (17)

heading: 3.4 Properties of CompoundE
text: With a richer set of operations, CompoundE is more capable of modeling complex relations such as 1- to-N, N-to-1, and N-to-N relations in KG datasets. Modeling these relations are important since more than 98% of triples in FB15k-237 and WN18RR datasets involves complex relations. The importance of complex relation modeling is illustrated by two examples below. First, there is a need to distinguish different outcomes of relation compositions when modeling non-commutative relations. That is r1 · r2 → r3 while r2 · r1 → r4. For instance, r1, r2, r3 and r4 denote isFatherOf, isMotherOf, isGrandfatherOf and isGrandmotherOf, respectively. TransE and RotatE cannot make such
distinctions since they are based on commutative relation embeddings. Second, to capture the hierarchical structure of relations, it is essential to build a good model for sub-relations. For example, r1 and r2 denote isCapitalCityOf and cityLocatedInCountry, respectively. Logically, isCapitalCityOf is a sub-relation of cityLocatedInCountry because if (h, r1, t) is true, then (h, r2, t) must be true. We provide mathematical proofs to show that CompoundE is capable of modeling symmetric/antisymmetric, inversion, composition, commutative/non-commutative, transitive, and sub-relations in Section A of the appendix. Optimization We follow RotatE’s negative sampling loss and the self-adversarial training strategy. The loss function of CompoundE can be written as
LKGE =− log σ(ζ1 − fr(h, t))
− n∑
i=1
p(h′i, r, t ′ i) log σ(fr(h ′ i, t ′ i)− ζ1),
(18) where σ is the sigmoid function, ζ1 is a fixed margin hyperparameter, (h′i, r, t ′ i) is the i-th negative triple, and p(h′i, r, t ′ i) is the probability of drawing negative triple (h′i, r, t ′ i). Given a positive triple, (hi, r, ti), the negative sampling distribution is
p(h′j , r, t ′ j |{(hi, r, ti)}) =
expα1fr(h ′ j , t ′ j)∑
i expα1fr(h ′ i, t ′ i) ,
(19) where α1 is the temperature of sampling.

heading: 4 Experiments

heading: 4.1 Link Prediction
text: Datasets. We conduct experiments on three widely used benchmarking datasets: ogbl-wikikg2, FB15k237, and WN18RR. ogbl-wikikg2 is one of Open Graph Benchmark dataset (Hu et al., 2020) extracted from the Wikidata (Vrandečić and Krötzsch, 2014) KG. Its challenge is with designing embedding models that can scale to large KGs. FB15k237 and WN18RR are extracted from the Freebase (Bollacker et al., 2008) and the WordNet (Miller, 1995), respectively. Inverse relations are removed from both to avoid the data leakage problem. Their main challenge lies in modeling symmetry/antisymmetry and composition relation patterns. The detailed statistics of the three datasets are shown in Table 1. Evaluation Protocol. To evaluate the link prediction performance of CompoundE, we compute the rank of the ground truth entity in the list of top candidates. Since embedding models tend to rank entities observed in the training set higher, we compute the filtered rank to prioritize candidates that would result in unseen triples. We follow the convention (Wang et al., 2017; Ji et al., 2021) and adopt the Mean Reciprocal Rank (MRR) and Hits@k metrics to compare the quality of different KGE models. Higher MRR and Hits@k values indicate better model performance.
Performance Benchmarking. Tables 2 and 3 show the best performance of CompoundE and other benchmarking models for FB15k-237,
WN18RR and ogbl-wikikg2 datasets, respectively. The best results are shown in bold fonts whereas the second best are underlined. CompoundE is a competitive model among embedding-based methods across all three datasets. As shown in Table 3, the results of CompoundE are much better than previous KGE models while the embedding dimension and the model parameter numbers are significantly lower for the ogbl-wikikg2 dataset. This implies lower computation and memory costs of CompoundE. We see from Table 2 that CompoundE has achieved significant improvement over distancebased KGE models using a single operation, either translation (TransE), rotation (RotatE), or scaling (PairRE). This confirms that cascading geometric transformations is an effective strategy for designing KG embeddings. In Table 2, we also compare CompoundE with text-based methods for link prediction on FB15k-237 and WN18RR datasets. Note that text-based methods require entity textual descriptions to make meaningful prediction. It is worth noting that CompoundE can still outperform transformer-based approaches significantly for WN18RR dataset, without having access to large amount of pretraining corpus. Performance on Different Relation Types. To gain insights into the superior performance of CompoundE, we examine the performance of CompoundE on each type of relations. KG relations can be categorized into 4 types: 1) 1-to-1, 2) 1- to-N, 3) N-to-1, and 4) N-to-N. We classify the relations based on the following rule. For each relation, r, we compute the average number of subject (head) entities per object (tail) entity as hptr and the average number of object entities per subject as
tphr. Then, with a specific threshold η,    hptr < η and tphr < η =⇒ r is 1-to-1 hptr < η and tphr ≥ η =⇒ r is 1-to-N hptr ≥ η and tphr < η =⇒ r is N-to-1 hptr ≥ η and tphr ≥ η =⇒ r is N-to-N. (20) We set η = 1.5 as a logical threshold by following the convention. Table 4 compares the MRR scores of CompoundE with benchmarking models on 1-to-1, 1-to-N, N-to-1, and N-to-N relations in head and tail entities prediction performance for the FB15k237 dataset. We see that CompoundE consistently outperforms benchmarking models in all relation categories. The filtered MRR scores on each relation type of the WN18RR dataset are given in Table 5. We observe that CompoundE has a significant advantage over benchmarking models for certain 1-to-N relations such as “member_of_domain_region” (+84.8%) and for some N-to-1 relations such as “synset_domain_topic_of” (+12.7%). CompoundE is more effective than previous KGE models in modeling complex relations.
In Fig. 2, we visualize relation embedding for the “friend” relation in FB15k-237 by plotting the histogram of translation, rotation, and scaling parameter values. Since “friend” is a symmetric relation, we expect the translation value to be close to zero, which is consistent with Fig. 2 (a). Also, since “friend” is an N-to-N relation, we expect the Compound operation to be singular. Actually, most of the scaling values are zero as shown in Fig. 2 (c). They support our theoretical analysis of CompoundE’s properties.

heading: 4.2 Path Query Answering
text: Path query is important since it is often desired to perform complex queries on knowledge graph. For example, one might ask "where did Michelle Obama’s spouse live in?". To obtain the answer, a
model first need to correctly predict the fact that (Michelle Obama, spouse, Barack Obama), and then predict (Barack Obama, livedIn, Chicago). CompoundE has the property to perform well on this task since it is capable of modeling the noncommutative relation compositions.
In Path Query Answering (PQA), a tuple (s, P, t) is given, where s and t denote the source and target entities and P = {r1, . . . , rk} denotes the relation path consisting of a sequence of relations that links s → r1 → r2 · · · → rk → t. PQA tests that after traversing through the relation path from a given source entity, whether the model is able to predict the correct target entity. During testing, the ground truth t is hidden and we compute the score for all candidate target entities and evaluate the quantile of ground truth, which is the fraction of irrelevant candidates that’s ranked lower than the ground truth. Mean quantile of all test paths are reported. In particular, type match paths are excluded since those are trivial for prediction. Specifically, we use both the KG triples and sampled paths with length |P | ∈ {2, 3, 4, 5} to train the embedding, which is also referred to as the “comp” setting (Guu et al., 2015). We use CompoundE to perform PQA on the Freebase and WordNet datasets prepared by (Guu et al., 2015). Statistics of these two datasets are shown in Table 10. Performance comparison with previous models on the PQA task under the “comp”
setting is shown in Table 6. Results show that CompoundE is very competitive for the PQA task among pure embedding models.

heading: 4.3 KG Entity Typing
text: KG Entity typing predicts class labels for nodes in knowledge graph. Entity type provides semantic signals for information extraction tasks such as relation extraction (Yaghoobzadeh et al., 2017), entity linking (Gupta et al., 2017; Choi et al., 2018) and coreference resolution (Durrett and Klein, 2014). Entity typing is challenging since each entity may be associated with a large number of type labels. We show that CompoundE can also be effective for performing entity typing.
We perform entity typing using CompoundE embedding on the FB15k-ET and YAGO43k-ET dataset prepared by (Moon et al., 2017). Statistics of these datasets are shown in Table 9. In addition to RDF triples (h, r, t), entity and entity type pairs (e, t) are added to these entity typing datasets. Since the type can be interpreted as “isA” relationship between e and t, we add a “type” relation between the e and t pair and treat that as a special triple. Performance comparison with existing work is shown in Table 7. The optimal configuration is shown in Table 13. Similar to link prediction, we also report the MRR and Hits@k scores. Results show that CompoundE achieves significant improvement over other models especially for the YAGO43k-ET dataset, even without giving special
treatment to the representation of entity types. This observation supports the claim that CompoundE is strongly capable of representing entity semantics.

heading: 4.4 Complexity Analysis.
text: We compare the computational complexity of CompoundE and several popular KGE models in Table 8. The last column gives the estimated number of free parameters used by different models to achieve the best performance for the ogbl-wikikg2 dataset. CompoundE cuts the number of parameters at least by half while achieving much better performance. In the table, n, m, and d denote the entity number, the relation number, and their embedding dimension, respectively. Since n≫ m in most datasets, we can afford to increase the complexity of relation embedding for better link prediction result without significantly increasing the overall space complexity. In Fig. 3, we compare the MRR scores of CompoundE and previous SOTA embedding models on the ogbl-wikikg2 dataset under different dimension settings d ∈ {10, 20, 50, 100, 150, 200, 250, 300}. CompoundE significantly outperforms benchmarking methods, even under low dimension setting. Hyperparameters. We conduct two sets of controlled experiments to find the best model configurations for ogbl-wikikg2, FB15k-237, and WN18RR datasets. For the first set, we evaluate the effect of different combinations of learning rates and embedding dimensions while keeping other hyperparameters constant. For the second set, we evaluate the effect of different combinations of the training batch size and the negative sample size, while keeping other hyperparameters constant. The optimal model configurations for three datasets are given in Table 11 of the appendix.

heading: 5 Conclusion and Future Work
text: A new KGE model called CompoundE was proposed in this work. We showed that quite a few distance-based KGE models are special cases of CompoundE. Extensive experiments were conducted for three different knowledge graph prediction tasks including link prediction, path query answering, and entity typing. Competitive experimental results demonstrate the effectiveness of CompoundE. We also mathematically prove the properties of CompoundE and its capability of modeling different relation patterns. We also explain the performance difference of different CompoundE forms, especially for the complex relation patterns.
We are interested in exploring two topics as future extensions. First, we may consider more complex operations in CompoundE. For example, there is a recent trend to extend 2D rotations to 3D rotations for rotation-based embeddings such as RotatE3D (Gao et al., 2020), SU2E (Yang et al., 2020). It is worthwhile to explore CompoundE3D. Second, CompoundE is expected to be useful in many
downstream tasks. This conjecture has to be verified. If this is the case, CompoundE can offer a low memory solution to these tasks in realistic settings.
Limitations
Similar to many knowledge graph embedding models, our proposed method is yet to handle link prediction under inductive settings. One possible future extension is to leverage entity description information to generate textual features and use CompoundE as a decoder to handle unseen entities. Also, the affine operators we use are limited to translation, rotation, and scaling and this may limit the number of different relation patterns we can handle. In the future, we can include all affine transformations and investigate their difference. Also, because we use 2D givens rotation matrix, the embedding dimension setting needs to be a factor of 2. We can explore higher dimensional transformations such as 3D transformations and compare the modeling power.

heading: Acknowledgment
text: The authors acknowledge the Center for Advanced Research Computing (CARC) at the University of Southern California for providing computing resources that have contributed to the research results reported within this publication. URL: https://carc.usc.edu.

heading: A Properties of CompoundE
text: Let M and M̂ denote the compound operation for the head and tail entity embeddings, respectively. In the following, we will prove nine properties of CompoundE.
Proposition A.1. CompoundE can model 1-N relations.
Proof. A relation r is an 1-N relation iff there exist at least two distinct tail entities t1 and t2 such that (h, r, t1) and (h, r, t2) both hold. Then we have:
M · h = M̂ · t1, M · h = M̂ · t2 M̂ · (t1 − t2) = 0
(21)
Since t1 ̸= t2, CompoundE can model 1-N relations when M̂ is singular.
Proposition A.2. CompoundE can model N-1 relations.
Proof. A relation r is an N-1 relation iff there exist at least two distinct head entities h1 and h2 such that (h1, r, t) and (h1, r, t) both hold. Then we have:
M · h1 = M̂ · t, M · h2 = M̂ · t M · (h1 − h2) = 0
(22)
Since h1 ̸= h2, CompoundE can model N-1 relations when M is singular.
Proposition A.3. CompoundE can model N-N relations.
Proof. By the proof for Prop.A.1 and A.2, N-N relations can be modeled when both M and M̂ are singular.
Proposition A.4. CompoundE can model symmetric relations. Proof. A relation r is a symmetric relation iff (h, r, t) and (t, r, h) holds simultaneously. Then we have:
M · h = M̂ · t =⇒ h = M−1M̂ · t M · t = M̂ · h =⇒ M · t = M̂M−1M̂ · t
MM̂−1 = M̂M−1
(23) Therefore, CompoundE can model symmetric relations when MM̂−1 = M̂M−1.
Proposition A.5. CompoundE can model antisymmetric relations. Proof. A relation r is a antisymmetric relation iff (h, r, t) holds but (t, r, h) does not. By similar proof for Proposition A.4, CompoundE can model symmetric relations when MM̂−1 ̸= M̂M−1.
Proposition A.6. CompoundE can model inversion relations. Proof. A relation r2 is the inverse of relation r1 iff (h, r1, t) and (t, r2, h) holds simultaneously. Then we have: M1 · h = M̂1 · t =⇒ h = M−11 M̂1 · t M2 · t = M̂2 · h =⇒ M2 · t = M̂2M−11 M̂1 · t
M̂−12 M2 = M −1 1 M̂1
(24) Therefore, CompoundE can model inversion relations when M̂−12 M2 = M −1 1 M̂1.
Proposition A.7. CompoundE can model relation compositions.
Proof. r3 is a composition of r1 and r2 iff (e1, r1, e2), (e2, r2, e3), and (e1, r3, e3) hold simultaneously. Then we have:
M1 · e1 = M̂1 · e2 =⇒ e1 = M−11 M̂1 · e2 M2 · e2 = M̂2 · e3 =⇒ e3 = M̂−12 M2 · e2
M3 · e1 = M̂3 · e3 M3M
−1 1 M̂1 · e2 = M̂3M̂−12 M2 · e2 M̂−13 M3 = (M̂ −1 2 M2)(M̂ −1 1 M1)
(25) Therefore, CompoundE can model relations composition when M̂−13 M3 = (M̂ −1 2 M2)(M̂ −1 1 M1).
Proposition A.8. CompoundE can model both both commutative and non-commutative relations.
Proof. Since the general form of affine group is non-commutative, our proposed CompoundE is non-commutative i.e.
(M1M̂ −1 1 )(M2M̂ −1 2 ) ̸= (M2M̂−12 )(M1M̂−11 )
(26) where each M consists of translation, rotation, and scaling component. However, in special cases, when our relation embedding has only one of the translation, rotation, or scaling component, then the relation embedding becomes commutative again.
Proposition A.9. CompoundE can model transitive relations.
Proof. r is a transitive relation iff (e1, r, e2), (e2, r, e3), and (e1, r, e3) hold simultaneously. Consider the CompoundE variant, and let R = R̂, S be a idempotent matrix.
fr(h, t) = ∥S ·R · h− R̂ · t∥ = ∥R · (R−1SR · h− t)∥ = ∥R−1SR · h− t∥
(27)
Let Mr = R−1SR. Then it is easy to see that
Mr ·Mr · · · · ·Mr =(R−1SR) · (R−1SR) · · · · · (R−1SR) =R−1SR
=Mr
(28)
Therefore, CompoundE can model transitive relations.
Proposition A.10. CompoundE can model subrelations.
Proof. A relation r1 is a sub-relation of r2 if (h, r2, t) implies (h, r1, t). Without loss of generality, suppose our compounding operation takes the following form
M = T ·R · S, M̂ = T̂ · R̂ · Ŝ, (29)
and suppose
T1 = T2, T̂1 = T̂2,
R1 = R2, R̂1 = R̂2, S1 = γS2, Ŝ1 = γŜ2, γ ≤ 1. (30)
With these conditions, we can compare the CompoundE scores generated by (h, r1, t) and (h, r2, t) as follows:
fr1(h, t)− fr2(h, t) =∥T1 ·R1 · S1 · h− T̂1 · R̂1 · Ŝ1 · t∥− ∥T2 ·R2 · S2 · h− T̂2 · R̂2 · Ŝ2 · t∥
=∥T2 ·R2 · (γS2) · h− T̂2 · R̂2 · (γŜ2) · t∥− ∥T2 ·R2 · S2 · h− T̂2 · R̂2 · Ŝ2 · t∥
=∥γ(T2 ·R2 · S2 · h− T̂2 · R̂2 · Ŝ2 · t)∥− ∥T2 ·R2 · S2 · h− T̂2 · R̂2 · Ŝ2 · t∥ ≤ 0
(31) This means that (h, r1, t) generates a smaller error score than (h, r2, t). If (h, r2, t) holds, (h, r1, t) must also holds. Therefore, r1 is a sub-relation of r2.

heading: B Performance comparison for different variations of CompoundE
text: We investigate the performance difference of CompoundE variants. Specifically, the different forms of CompoundE have visible difference in different relation types. We conduct experiment on YAGO3-10 dataset and compare the performance of CompoundE-left, CompoundE-right, CompoundE-Complete for 1-to-1, 1-to-N, and Nto-1 relations. In particular, when evaluating the 1-to-N relations, we focus on predicting (?, r, t) while for N-to-1 relations we focus on predicting (h, r, ?) to correctly reflect the performance on respective relation types. Performance comparison is shown in 4. We observe that for CompoundEComplete has advantage over other forms for 1-to-1 relations. CompoundE-left and CompoundE-right are the better performing forms for 1-to-N and N-to1 relations respectively. This observation is consistent with the discussion of the modeling capability of CompoundE. It still remains a questions that how different order of operator composition will affect the performance of CompoundE and we will address that in future work.
C Visualization of Embedding
We provide a 2D t-SNE visualization of the entity embedding generated by CompoundE for FB15k237 in Fig. 5. Each entity is colored with its respective entity type. As shown in the figure, some entity type class are well separated while others are not. This scatter plot shows that entity representations
extracted by CompoundE can capture the semantics of the entity. Thus, their embeddings can be used in various downstream tasks such as KG entity typing and similarity based recommendations.
Besides the histograms shown in the main paper, we add more plots to visualize CompoundE relation embedding values. In Fig. 6, we show the embedding values for the “friends” relation in the FB15k-237. We use the CompoundE-Complete variant (∥Sr ·Rr ·Tr · h− Ŝr · R̂r · T̂r · t∥) to generate the embedding. We plot the translation and scaling components for both the head and the tail. We only show a single plot for the rotation component since the rotation parameter is shared between the head and the tail. Different from the CompoundE-head (∥Sr ·Rr ·Tr · h− t∥), we see two modes (instead of only one mode) in CompoundE-Complete’s plots. One conjecture for this difference is that CompoundE-Complete has a pair of operations on both the head and the tail, the distribution of values need to have two modes to maintain the symmetry. Similar to CompoundE-head, the scaling parameters of CompoundE-Complete have a large amount of zeros to maintain the singularity of compounding operators and help learn the N-to-N complex relations.
Fig. 7 and Fig. 8 display the histogram of relation embeddings for “instance_hypernym” relation and “similar_to” relation in WN18RR, respectively. The real (in blue) and the imaginary (in orange) parts are overlaid in each plot. Notice that “instance_hypernym” is an antisymmetric relation while “similar_to” is a symmetric relation. This relation pattern is reflected on the embedding histogram since the translation and the scaling histograms for the head and the tail are different in “instance_hypernym”. In contrast, the translation and scaling histograms for the head and the tail are almost identical in “similar_to”.

heading: D Datasets
text: The path query dataset can be obtained from the link2. The original github repo3 has the MIT license. The entity typing dataset can be found here4. Statistics of these datasets are shown in Table 9 and
2https://worksheets.codalab.org/works heets/0xfcace41fdeec45f3bc6ddf31107b829f
3https://github.com/kelvinguu/travers ing-knowledge-graphs
4https://github.com/cmoon2/knowledge_ graph/tree/master/datasets
Table 10.
E Implementation and Optimal Configurations
To form relation specific transformation matrices for high dimensional entity vectors, we can first cascade translation, rotation, and scaling operator to yield a compound operator Or,i = Tr,i ·Rr,i · Sr,i In the actual implementation, we use the operator’s representation in regular Cartesian coordinate instead of the homogeneous coordinate. Furthermore,
a high-dimensional relation operator can be represented as a block diagonal matrix in the form of
Mr = diag(Or,1,Or,2, . . . ,Or,n), (32)
where Or,i ∈ R2×2 is the compound operator of the i-th block.
In our implementation, we normalize all entity embeddings to unit vectors before applying compound operations. The optimal configurations of CompoundE are given in Table 11. The implementation of the rotation operation in the optimal CompoundE configuration for the WN18RR dataset is adapted from RotatE.
All experiments were conducted on a NVIDIA V100 GPU with 32GB memory. GPUs with larger memory such as NVIDIA A100 (40GB), NVIDIA A40 (48GB) are only needed for hyperparameter sweep when the dimension, the negative sample size, and the batch size are high. We should point out that such settings are not essential for CompoundE to obtain good results. They were used to search for the optimal configurations. We have considered the following set of numbers as our parameter search space to obtain the best performance we can for each dataset and tasks. Link Prediction.
Table 11: Optimal Configurations for Link Prediction Tasks. B denotes the batch size and N denotes the negative sample size.
Dataset CompoundE Variant #Dim lr B N α ζ ogbl-wikikg2 ∥h− Ŝ · T̂ · R̂ · t∥ 100 0.005 4096 250 1 7 FB15k-237 ∥S ·R ·T · h− Ŝ · R̂ · T̂ · t∥ 600 0.00005 1024 125 1 6 WN18RR ∥R · S ·T · h− Ŝ · T̂ · t∥ 500 0.00007 1024 256 0.5 6
Table 12: Optimal Configurations for Path Query Answering. B denotes the batch size and N denotes the negative sample size.
Dataset CompoundE Variant #Dim lr B N α ζ Freebase ∥S ·T ·R · h− t∥ 1500 0.00002 1024 256 1 6 WordNet ∥S ·T ·R · h− t∥ 1000 0.00005 1024 256 1 6
(e)
• negative sample ∈ {256, 512, 1024, 2048}
Path Query Answering. Freebase
• d ∈ {500, 1000, 1500, 2000}
• lr ∈ {0.00001, 0.00002, 0.00005, 0.0001}
• ζ ∈ {6, 9, 12, 15}
• batch ∈ {512, 1024}
• negative sample ∈ {256, 512}
WordNet
• d ∈ {500, 1000, 1500, 2000}
• lr ∈ {0.00001, 0.00002, 0.00005, 0.0001}
• ζ ∈ {6, 9, 12, 15}
• batch ∈ {512, 1024}
• negative sample ∈ {256, 512}
Entity typing. FB15k-ET
• d ∈ {500, 1000, 1500}
• lr ∈ {0.00001, 0.00005, 0.0001, 0.0005}

heading: F Comparing CompoundE and STaR
text: The main difference between CompoundE and STaR is that STaR embedding uses a bilinear product and adopts a semantic matching approach while CompoundE’s scoring function is a distance-based metric. Because of this, the optimization strategy for CompoundE is the self-adversrial negative sampling loss whereas STaR uses the regularized cross-entropy loss. More importantly, CompoundE embedding has clear and intuitive geometric interpretations whereas the design of STaR is less intuitive since it is unclear what composition of operators means in the context of a bilinear product. We also shed light on the superior capability of CompoundE to model relation compositions and entity semantics through PQA and entity typing experiments. Lastly, we can incorporate reflection and shear operators below who also belong to the affine operator family. The reflection matrix can be defined as
F =   cos(ϕ) sin(ϕ) 0 sin(ϕ) − cos(ϕ) 0
0 0 1
  , (33)
And the shear matrices on two different directions can be defined as
Hx =   1 tan(ψx) 0 0 1 0 0 0 1   , (34)
Hy =
  1 0 0 tan(ψy) 1 0
0 0 1
  , (35)
We have done preliminary experiments on Wikikg2 but reflection and shear operators and result is shown in Table 14. We will further improve the result in the future.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
Second last, before Acknowledgement
A2. Did you discuss any potential risks of your work? Not applicable. Left blank.
3 A3. Do the abstract and introduction summarize the paper’s main claims? Section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 4.1
3 B1. Did you cite the creators of artifacts you used? Section 4.1
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Not applicable. Left blank.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Not applicable. Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.
3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Section 4.1
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Table 1 and appendix D
C 3 Did you run computational experiments? Section 4
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Table 7, and appendix E
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Appendix E
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Left blank.
C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Not applicable. Left blank.
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Not applicable. Left blank.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Not applicable. Left blank.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Not applicable. Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Not applicable. Left blank.

heading: Future Work
text: We will further improve the result in the future. ACL 2023 Responsible NLP Checklist