Random Entity Quantization for Parameter-Efficient Compositional  Knowledge Graph Representation
Authors: Jiaang Li
Quan Wang
Yi Liu
Licheng Zhang
Zhendong Mao
Venue: EMNLP 2023 Main (2023)
Keywords: knowledge graph
knowledge representation

Abstract: Representation Learning on Knowledge Graphs (KGs) is essential for downstream tasks. 
The dominant approach, KG Embedding (KGE), represents entities with independent vectors and faces the scalability challenge. 
Recent studies propose an alternative way for parameter efficiency, which represents entities by composing entity-corresponding codewords matched from predefined small-scale codebooks. 
We refer to the process of obtaining corresponding codewords of each entity as entity quantization, for which previous works have designed complicated strategies. 
Surprisingly, this paper shows that simple random entity quantization can achieve similar results to current strategies.
We analyze this phenomenon and reveal that entity codes, the quantization outcomes for expressing entities, have higher entropy at the code level and Jaccard distance at the codeword level under random entity quantization.
Therefore, different entities become more easily distinguished, facilitating effective KG representation.
The above results show that current quantization strategies are not critical for KG representation, and there is still room for improvement in entity distinguishability beyond current strategies.

Random Entity Quantization for Parameter-Eficient Compositional
Knowledge Graph Representation
Jiaang Li', Quan Wang?; Yi Liu", Licheng Zhang', Zhendong Mao' UUniversity of Science and Technology of China
MOE Key Laboratory of Trustworthy Distributed Computing and Service, Beijing University of Posts and Telecommunications
State Key Laboratory of Communication Content Cognition (jali,zlczlc)@ma) ustc. edu. cn, wangquanêbupt. edu. cn, davin1332egma com, zdmao@ustc. edu. cn
Abstract
Cockbooks
Codewvrrd Metching
Al
XX
Enttt Codes
Representation Learning on Knowledge Graphs (KGs) is essential for downstream tasks. The dominant approach, KG Embedding (KGE), represents entities with independent vectors and faces the scalability challenge. Recent stud- ies propose an alternative way for parameter efficiency, which represents entities by compos- ing entity-corren coreesodorr codewords matched from predefined small-scale codebooks. We refer to the process of obtaining corresponding codewords of each entity as entity quantiza- tion, for which previous works have designed complicated strategies. Surprisingly, this paper shows that simple random entity quantization can achieve similar results current strategies. We analyze this phenomenon and reveal that entity codes, the quantization outcomes ex- pressing entities, have higher entropy the code level and Jaccard distance at the code- word level under random entity quantization. Therefore, different entities become more cas- ily distinguished, facilitating effective KG rep- resentation. The above results show that current quantization strategies are not critical for KG representation, and there is still room for im- provement in entity distinguishability beyond current strategies. The code to reproduce our results is available here.
A2
DEI 3 O
A0
Entiiies
Entry Regrosestatiion
Codewords
Loste Vraatisabin
The process of parameter-efficient composi- tional KG representation. Entities are quantized entity codes, which are encoded to represent entities. Each dimension of an entity code stands for a codeword, and indicates whether current entity matches this codeword (set to 1) with optional weights or not (set to 0).
Figure I:
entity with an independent vector, which leads to linear increase in the number of parameters with the number of entities. Consequently, scalability becomes challenge for these models, posing dif- ficulties in their implementation and deployment (Peng al., 2021; Ji al., 2022), especially for large-scale KGs (Safavi and Koutra, 2020; Mahdis- oltani et al., 2014).
A recently proposed parameter-efficient KG rep- resentation method uses compositional entity rep- resentations reduce parameters (Galkin et al., 2022; Chen al., 2023). Instead of learning sep- arate representations like KGE, it represents enti- ties by composing their matched codewords from predefined codebooks through an encoder, and re- quires fewer parameters since codewords are much fewer than entities. See Fig for an illustration of this method. We refer to the process of obtaining corresponding codewords each entity as entity quantization due to its similarity vector quanti- zation (van den Oord et al., 2017). Specifically, ex- isting methods construct two codebooks, in which codewords are the entire relation set and a selec-
1 Introduction
Knowledge Graphs (KGs) comprise (head entity, relation, tail entity) triplets. They are crucial ex- ternal knowledge sources for various natural lan- guage processing tasks (Hu et al., 2022: Sun et al., 2022). Learning representations KGs is neces- sary for expressing complex semanties and sup- porting downstream tasks. The most dominant paradigm, KG Embedding (KGE), maps entities and relations to a vector space (Dettmers et al., 2018; Sun et al., 2019; Zhang et al., 2020), Despite the popularity, KGE models need to represent each
"Corresponding author: Quan Wang

---

tive subset of entities, ie., anchors, respectively. From these two codebooks, each entity matches two groups of codewords: connected relations and anchors nearby (Galkin et al., 2022) or with similar adjacent relations (Chen et al., 2023). Chen et al. (2023) also regards matching degrees as codeword weights for more expressiveness. Matched results are denoted as entity codes, including matched codewords and optional weights. A subsequent encoder uses entity codes to compose correspond- ing codewords and generate entity representations. This approach performs closely to KGE with fewer parameters, making KG training and deployment more efficient.
of matching same codewords and distributes entity codes more uniformly across wide range. We prove this claim by analyzing the properties of en- tity codes. At the code level, we consider entity code as a whole and treat as one sample of a random variable. The entropy of this variable can be derived from its distribution across all entity codes. We prove that random entity quantization has higher entropy and maximizes it with high prob- ability, thus producing more diverse and unique en- tity codes. At the codeword level, each entity code indicates a set of matched codewords. We analyze the Jaccard distance between different sets and find that is significantly increased by random entity quantization. As a result, different entities will have more obvious dissimilarity when randomly quantized, making them easier to distinguish.
The key to entity quantization lies in two steps: (I) codebook construction and (2) codeword match- ing. Previous studies have dedicated their efforts to designing quantization strategies, which include se- lecting proper KG elements to construct codebooks and measuring the connectivity between codewords and entities to match them. We conduct experi- ments to randomize these strategies from shallow to deep. Surprisingly, we find that random en- tity quantization approaches can achieve similar even better results.
In summary. the contributions of our work are two-fold: (1) We demonstrate through comprehen- sive experiments that random entity quantization approaches perform similarly or even better than previously designed quantization strategies. (2) We analyze that this surprising performance is because random entity quantization has better entity distin- guishability, by showing its produced entity codes have higher entropy and Jaceard distance. These results suggest that current complicated quantiza- tion strategies are not critical for the model perfor- mance, and there is potential for entity quantization approaches to increase entity distinguishability be- yond current strategies.
We design several model variants for experi- ments. First, to investigate the effectiveness of matching codewords with connectivity, we random- ize the codeword matching step by randomly se- lecting codewords matched results of entities. Moreover, we codeword weights randomly equally for (Chen et al., 2023) to verify whether de- signed weights from matching are critical. Finally, to explore whether mapping codewords to actual elements in the KG is critical, we randomly con- struct codebooks with codewords that have no ac- tual meaning. We adopt random codeword match- ing to the randomly constructed codebook, to pro- vide fully random entity quantization. Counter- intuitively, empirical results show that the above operations achieve similar results compared com- plicated quantization strategies and may even im- prove the model performance.
Preliminaries
2.1 Knowledge Graph Representation
knowledge graph g C R & composed of entity-relation triplets (h, r, t), where & is a set of entities, and R is a set of relations. Each triplet indicates a relation E R between two entities h, L E E, where h is the head entity, and t is the tail entity. The goal of knowledge graph representation is to learn vector representation e; for each entity e E E, and rj for relation r. R.
2.2 Compositional KG Representation
Moreover, we have verified that random entity quantization can better distinguish entities than cur- rent quantization strategies, which leads to more expressive KG representations (Zhang et al., 2022). Under the strategies designed by previous works, different entities could match the same codewords, making their code similar or identical. In contrast, random quantization leads to a lower possibility
Compositional knowledge graph representation methods compose codewords from small-scale codebooks represent entities. These methods obtain codewords for each entity by constructing codebooks and matching codewords. We refer these two steps as entity quantization. The matched codewords are encoded to represent each entity.

---

This section presents two methods of this kind, i.e.. NodePiece (Galkin al., 2022) and EARL (Chen et al., 2023). We first introduce the definition of entity quantization and how these two methods rep- resent entities with it. After that, we introduce how they are trained. Our subsequent experiments are based on these two methods.
are obtained from above. Each relation rý R is represented independently. Both models use RotatE (Sun et al., 2019) to score triplets with f(h, r.t) - - | r - t|]. which maps entities and relations in complex space, i.e., h, r, t E Cd NodePiece and EARL use different loss func- tions for different datasets, including binary cross- entropy (BCE) loss and negative sampling self- adversarial loss (NSSAL). For a positive triplet (h, r, t). BCE loss can be written as:
2.2.1 Entity Quantization
We first formally define the entity quantization pro- cess. Existing entity quantization strategies con- struct a relation codebook B, {r1. -M- T'm and an anchor codebook Ba {a1.-. )
LBCE(h, r,t) log(o(((h,,,tt))
Σ log(1 oal1-o(((h),,) of((k,,,,
The codewords (r1.-.., (mm) are all m relations in R and (01,"."",)) are n anchors selected from all entities in & with certain strategies. After adding reverse edges to KG, each entity e E & matches min(d;, d)' unique relations from all its d, connected relations in B,, and employs anchor-matching strategies to match anchors from B,. Its matched codewords are denoted as Each entity
where a is the sigmoid function and (K4 Ty ()) is the i-th negative triplet for (h, r, t). NSSAL further considers that negative samples have varying difficulties:
Ch. r, llg f(h,r,,)
a set W; {ri..... .re. ... a G will get its entity code e; to represent W,, which is a (m + n) ni)ddmmesioonal vector that is zero except for the (s; + k) dimensions representing matched codewords. Values in these dimensions are set to I optional codeword weights if provided. Then, we provide the detailed quantization pro- cess of both models.
Σ (( of -)),
where is fixed margin. p(h), ()) is the self- adversarial weight of (h/ r, ()) and takes the fol- lowing form:
expa(f(h', r,,))
plhy, (t) Σ, e rtt)
NodePiece NodePiece uses metrics such as Per- sonalized PageRank Page et al. (1999) to pick some entities as codewords in B,, Each e; E & matches nearest anchors from B, as
where the temperature of sampling.
Experimental Setup
EARL EARL constructs Bg with 10% sam- pled entities. Each entity E E matches an- chors which have the most similar connected rela- tions. Matched codewords are assigned to designed weights. Weights of r E (ri, "rr,, ,r&,) are based on its connection count with é,, and weights of each a E ... . are based on the similarity between connected relation sets of e; and a.
Though previous methods have designed compli- cated strategies for entity quantization, whether these strategies critical for the model perfor- mance remains be explored. Our experiments are to empirically test the effect of these quanti- zation strategies. We therefore design a series of model variants using random entity quantization. We focus on showing the effectiveness of random entity quantization, rather than proposing new state- of-the-art models. Below we introduce the model variants we design, the datasets we test, and the training/evaluation protocols.
After quantization, codewords in W; are com- posed by an encoder output entity representation e; The encoders of NodePiece and EARL are based on MLP and CompGCN (Vashishth et al., 2020), respectively.
3.1 Model Variants
2.2.2 Model Training
We design model variants based on existing mod- els, NodePiece and EARL. We replace part of their designed quantization strategies with random ap- proaches for each of these two models. The random
Here we introduce how to train both models. For each triplet (h, r),, representations of h and t
is is a hyperparameter in Nodepiece, and is +oo in EARL..

---

methods we use are generally divided into three types: (1) Randomly match entities relations or anchors in codebooks. (2) Randomly or equally set codeword weights. (3) Randomly construct code- books, where codewords do not refer to natural elements in the KG. We will discuss the details of these model variants in Section 4.
are then sorted in descending order based on their scores. We apply the filtered setting (Bordes al., 2013) to exclude other valid candidate triplets from ranking. To assess the performance of the mod- els, we report the mean reciprocal rank (MRR) and @its@ 10. Higher MRR/H@ 10 indicates better performance. Additionally, we evaluate the effi ciency of the models using Ef fi MRR/#P. where # # represents the number of parameters. The results of NodePiece and EARL are from their original papers.
3.2 Datasets
We use three knowledge graph completion datasets in total. We employ FB15k-237 (Toutamova et al., 2015) and WN18RR (Dettmers et al., 2018) to demonstrate the effectiveness of random entity quantization and extend our conclusions larger scale KG, CoDEx-L(Safavi and Koutra, 2020). FB15k-237 is based Freebase (Bollacker et al., 2008), knowledge base containing vast factual information. WN18RR has derived from Word- net (Miller, 1995). specifically designed to capture semantic relations between words. CoDEx-L is the largest version of recently proposed CoDEx datasets, which improve upon existing knowledge graph completion benehmarks in content scope and difficulty level. For consistency with compared methods, we exclude test triples that involve enti- ties not present in the corresponding training set. Table presents the statistics of these datasets. We also experiment with inductive relation prediction datasets(Teru et al., 2020) (details in Appendix C).
Random Entity Quantization
This section details the random variants we design and their experimental results. We design model variants by randomizing different steps of existing entity quantization strategies, including codeword matching and codebook construction. We find that these variants achieve similar performance to exist- ing quantization strategies. These results suggest that the current entity quantization strategies are not critical for model performance.
4.1 Random Codeword Matching
We first randomize the codeword matching step, which includes strategies for (1) matching cach entity the corresponding codewords and (2) signing weights the matched codewords.
4.1.1 Matching Strategy
Datasets FB15k-237
WNI8RR CoDEx-L
We randomize the codeword matching strate- gies to investigate whether current connectivity- based strategies are critical to the model perfor- mance. We design model variants by randomiz- ing current methods' relation-matching or anchor- matching strategies and keep other settings un- changed. Specifically, with the relation codebook B, and the anchor codebook Ba, we have the fol- lowing model variants for both models.
#Entity 14,505
40,559
77,951
#Relation
237
11
69
#Train 272,115
86,835
551,193
#Valid
17,526
2,824
30,622
#Test
20,438
2,924
30,622
Table Statistics benchmark datasets, including the number of entities, relations, training triples, valida- tion triples, and test triples.
+RSR: Each entity 6₃ & Randomly Selects s; Relations (RSR) from B, and matches k anchors from B, with the anchor-matching strategy designed by the current model, as matched codewords W.
3.3 Training and Evaluation Protocols
Training. Our experiments are based on the offi- cial implementation of NodePiece and EARL. We use the same loss functions and follow their hyper- parameter setting for corresponding model variants. More details are provided in Appendix A.
+RSA: e; Randomly Selects k Anchors (RSA) from B,, and matches s; relations from B, with current relation-matching strategy, as W,
Evaluation. We generate candidate triplets by substituting either h or with candidate entities for each triplet (h, r, t) in the test sets. The triples
+RSR+RSA: e; randomly selects 8; relations from B,, and randomly selects l anchors from B, as W,.

---

FB15K-237
WNI8RR
FB15k-237
WNI8RR
MRR
@its@ 10
MRR
Hits 10
MRR
Hits 10
MRR
Hits@ 10
EARL
0.310
0.501
0.440
0.527
EARL
0.310
0.501
0.440
0.527
+RSR
0.306
0.500
0,439
0.530
w/o anc 0.301
0.488
0,409
0.498
+RSA
0.311
0.506
0.438
0.529
w/o anc+RSR
0.312
0.501
0.417
0.516
+RSR+RSA
0.308
0.502
0.442
0.536
w/o rel
0.309
0.501
0.432
0.520
w/o rel+RSA 0.311
0.500
0.443
0.539
NodePiece
0.256
0.420
0.403
0.515
+RSR 0.254
0.417
0.403
0.516
NodePiece
0.256
0.420
0.403
0.515
+RSA 0.258
0,423
0.419
0.518
w/o ane 0.204
0.355
0.011
0.019
+RSR+RSA 0.263
0.425
0.425
0.522
w/o anc+RSR 0.244
0.409
0.009
0,014
w/o rel 0.258
0.425
0.266
0,465
Table 2: Results for parameter-efficien compositional KG representation methods with randomly selected re- lations (RSR) randomly selected anchors (RSA).
w/o rel+RSA 0.256
0.428
0.411
0.517
Table Random codeword matching with only codebook. "w'o anc' denotes not using the anchors, and "w/o rel' denotes not using the relation codebook. Ablation results taken from the original paper.
For all variants, we still assign codewords in Wi (rf....,,""", . af..... ,aa.), with current connectivity-based weights, if required. Table 2 shows the performance of original mod- els and their respective variants. Surprisingly, randomizing codeword-matching and relation- matching does not affect the overall performance existing methods on both datasets, whether used together or separately. The results suggest that cur- rent complicated codeword matching strategies are not critical to the model performance.
FB 15k-237
WN18RR
MRR
Hits@ 10
MRR
Hits@
EARL 0.310
0.501
0.440
0.527
+RW 0.308
0.498
0.442
0.531
+EW
0.308
0.500
0.437
0.528
Table 4: Results for parameter-efffiient compositional KG representation methods with random codeword weights (RW) or equal codeword weights (UW)
We further study the impact of randomizing the codeword matching strategy with only one code- book. We remove B, or Ba respectively, and adopt different codeword matching approaches for the remaining codebook, forming following variants:
cases. NodePiece performs poorly on WN18RR without Ba, because the number of relations in this dataset is small, and only using B, sharply decreases model parameters. The above results fur- ther validate the effectiveness of random codeword matching and demonstrate its robustness with only one codebook.
w/o anc: Remove the anchor codebook Bg, Match 8 relations from B, with the cur- rent relation-matching strategy as W; [ri,.... ).
w/o anc+RSR: Remove Ba. Randomly select s; relations from B, W, {ri.- )
4.1.2 Codeword Weights from Matching
During the matching process, EARL will further assign weights to the matched codewords based on the connectivity between entities and codewords. We conduct experiments to explore whether such weights are critical. Specifically, we design fol- lowing model variants for EARL using random equal codeword weights, with the codebooks and codeword matching strategies unchanged:
w/o rel: Remove the relation codebook B, Match & anchors from B, with the current anchor-matching strategy as W; (a....... (a] ..
w/o rel+RSA: Remove B,. Randomly select anchors from Bg as W; = {af.".. ... .
Table 3 shows that when removing one code- book, random matching codewords from the re- maining codebook performs better than using cur- rent designed matching strategies. even per- forms similarly to the original methods in most
+RW:
Assign Random Weights (RW) to matched codewords.
+EW:
Assign Equal Weights (EW) to matched codewords, i.e., set all codeword weights to 1.

---

FB15k-237
WN18RR
CoDEx-L
#P(M) MRR @its@ 10
Effi
#P(M)
MRR Hits@ 10
Elfi
#P(M)
MRR Hits 10 Effi
EARL
1.8
0.310
0.501
0.172
3.8
0,440
0.527
0.116
2.1
0.238
0.390
0.113
EARL+RQ
1.4
0.311
0.505
0.222
3,3
0.444
0.535
0.135
1.9
0.239
0.394
0.126
NodePiece
3.2
0.256
0.420
0.080
4.4
0,403
0.515
0.092
3.6
0.190
0.313
0.053
NodePiece+RQ 3.2
0.261
0.423
0.082
4.4
0.429
0.517
0.098
3.6
0.192
0.326
0.053
Table 5: Results applying fully random entity quantization (RQ) to parameter-efficien compositional KG representation methods on datasets with varying sizes. Better results between each model and its variant are bolded.
Table shows that either using random weights or equal weights does not significantly impact the performance. Thus, we can conclude that the connectivity-based weights are not critical for model performance. For subsequent experiments, we use equal codeword weights for simplicity.
variants and results above, we can conclude that random entity quantization is as effective as current complicated quantization strategies.
Why Random Quantization Works
This section analyzes the reasons for the surprising performance of random entity quantization. The entity codes directly affect entity representation and model performance. We analyze the entity codes produced by different entity quantization up- proaches to compare their ability to distinguish different entities. We find random entity quanti- zation obtains entity codes with greater entropy at the code level and Jaccard distance at the code- word level. Thus, it can better distinguish different entities and represent KGs effectively.
4.2 Random Codebook Construction
We construct codebooks randomly to investigate whether it is eritical to construct codebooks with specific relations or entities as codewords, like cur- rent strategies. We design model variant for each model, which uses a randomly constructed code- book B instead B, and Bg- Codewords B do not have any real meanings. Moreover, we adopt random codeword matching and equal codeword weights such variants. We call this type of vari- ant the fully random entity quantization (RQ) since it randomizes all parts of the entity quantization.
5.1 Code Level Distinguishability
We analyze the ability distinguish entities at the code level of different entity quantization ap- proaches. Specifically, we treat the entity code e; of each entity e; E as a sampling of a random variable X. The entity codes of all entities repre- sent ||| samplings of X. From these samplings, we get different entity codes and their frequencies. We denote these codes as {#I...., .... and their frequencies as [fi,..., fo). as the total number of codewords, where m and n are numbers of codewords in B, and B.. The num- ber of all possible entity codes is 2 We estimate the probability distribution of X on all codes based on the relative frequency of different entity codes in the sampling results, and then derive its entropy:
+RQ: Fully Random entity Quantization (RQ).
Randomly construct a codebook whose size equals the sum of B, and B.: B (21 Each entity e, E & ran- domly selects (s + k) codewords as its matched codewords W; { 2..)) with equal weights, where s and El is the number of entities.
1 Σ 1 8
We denote = m + n
[E
Table 5 shows variants of both models lead to similar or even better performance across all datasets with varying sizes. We will analyze this surprising observation in Section 5. Through ((M) and F.f. we further show that random entity guantization requires equal or even fewer pa- rameters with higher efficiency. It does not require composing each codebook's 's codewords separately, saving parameters for EARL. The above results show that constructing codebooks by KG elements is not critical to the model performance. From all
H(X) = Σ P(x.)) log, P(xt),
(1)
i=
where P(xs) is the relative frequency of o;;
f if i i- 1...., ,....,,,,
P(x,)
E
0 if 1= 2

---

NodePiece
EARL
Random
In addition, the entity code entropy is maximized when all entity codes are unique. The probability of random entity quantization to produce |&] unique entity codes is close to 1, as shown in Theorem 1. The detailed proof is in Appendix B. This theorem shows that random entity quantization expresses entities uniquely with high probability, thus distin- guishing entities well.
FB15k-237
15.26
14.50
15.27
WN18RR 15.94
8.20
16.75
Table 6: The entropy (bits) of entity code produced by random entity quantization and well-designed quantiza- tion strategies proposed by NodePiece and EARL.
Theorem 1. The probability of random entity quantization to produce |g| unique entity codes 21 2'-i
0.30
0.25
0.35
is P = 11 i= 2 which approaches I when 2 [3)
0.20
2 0.15
From above, we demonstrate random entity quantization produces diverse entity codes and clearly distinguishes entities at the code level.
0.55
0.05
DAR
Evtty Cada troepy
Sndy Code Ditrayy
Figure 2: Performance of random entity quantization with different entropy
5.2 Codeword Level Distinguishability
We further analyze the ability to distinguish en- tities at the codeword level of different entity quantization approaches. Specifically, for en- tities Cj & with entity codes and cú, their corresponding sets of matched codewords are W; {ri r "i, ".,....,0 , and Wj trl,- - ré,, ay a²). The Jaccard distance between and is:
We use the entropy (X) in eq. to measure the diversity of entity codes. A higher entropy means more diverse entity codes, thus indicating the better -iiiiy-dittinuishh ability of the quantiza- tion approach. In this sense, this entropy can reflect entity distinguishability at the code level.
We use equation derive the entity code en- tropy produced by random and previously designed quantization strategies. The results are listed in Table 6. The entropy of random quantization is averaged in 100 runs with different seeds, while the entropy of previous strategies is determinis- tie and does not change in different runs. Empir- ically, demonstrate that random quantization achieves higher entropy, producing more diverse entity codes and enabling easier distinguishability between different entities.
dy(c,, I WiI -
W U W,W
where |. denotes the number of elements in set. We the Jaccard distance dy(c;, Cj) mea- sure the distinctiveness between entity codes c, and Cj. A larger distance means their indicated code- words are more distinct and makes entities e, and éj more easily to be distinguished. In this sense, this distance can reflect the entity distinguishability at the codeword level.
To capture the overall distinguishability among all entity codes, we propose a k-nearest neighbor evaluation metric based on the Jaccard distance. This evaluation assesses the average distance be- tween each entity code and its k nearest codes, denoted as Jk- A higher Ts means the entity codes are more distinct. We use different values of l to observe the distance among entity codes in differ- ent neighborhood ranges. The metric Ts is derived
We confirm that higher entity code entropy brings better model performance through exten- sive experiments. Specifically, after random entity quantization, we randomly select a subset of entity codes and set them to be identical, to obtain en- tity codes with different entropy values. Figure shows the comparison experiments across these entity codes with EARL and NodePiece. As the entropy rises, the model performance gradually in- creases and eventually saturates. EARL performs better than NodePiece across varying entropy, as its GNN encoder involves the entity neighborhood and can better distinguish entities. From above, more diverse entity codes with higher entropy benefit the model performance, which aligns with our claims.
Jh E x ELNN(e))
Σ Σ dy(ci;
where |]| is the number of entities. kNN((e) is a set of entities whose codes are nearest to the

---

FB15k-237
entities and relations of given KG, which ben- efits downstream tasks such as question answer- ing (Hu et al., 2022), reading comprehension (Sun et al., 2022), and pre-trained language represen- tation Wang et al., 2021c Recent years have witnessed a growing number of KG embedding techniques being devised, including distance-base models (Bordes et al., 2013; Sun et al., 2019; Zhang et al., 2020), semantic matching models (Trouillon et al., 2016; Balaževié al., 2019), neural encod- ing models (Dettmers et al., 2018; Schlichtkrull et al., 2018; Wang et al., 2019), and text augmented models (Yao et al., 2019; Wang et al.). We refer readers to (Wang et al., 2017; Ji et al., 2022) for a comprehensive overview of the literature.
0.8
0.7
Random Entily Quantization NodePicce Quantization EARL Quantization
200
400
600 WN18RR
800
1000
0.8
0.4
0.2
200
400
60D
800
1000
Figure 3: The average Jaccard distance between each entity code and its l nearest codes.
Erffiiter-Efffcientt KG Representation KG embedding methods face the scalability challenge. The number of parameters scales up linearly to the entity number. Several studies compress learned parameters from KG embedding models, trying to solve this issue. Incorporating knowledge dis- tillation techniques, MulDE (Wang al., 202 1b) transfers knowledge from multiple teacher mod- els to a student model. Expanding on this, Du- aIDE (Zhu et al., 2022) considers the dual influ- ence between the teacher and student and adapts the teacher to better align with the student, thus en- hancing the performance of the distilled model. To directly compress existing models, Sachan (2020) discretizes the learned representation vectors for less parameter storage space, then maps the discrete vectors back to the continuous space for testing. LightKG (Wang et al., 2021a) introduces dynamic negative sampling and discretizes learned represen- tations through vector quantization.
code of e under Jaccard distance:
kNN(e,,
arg min tel, ri JCE Ci o #c,
Σ ds(c,,
jc(linnds))
Figure 3 shows the average Jaccard distance Jk between entity codes w.r.t. different numbers of nearest codes. We can see that random entity quantization achieves higher Jk than current quan- tization strategies across the varying k. Thus, en- tity codes produced by random entity quantization are more distinct within different neighborhood ranges. Based on the above observation, random entity quantization makes different entities easier distinguish at the codeword level.
5.3 Discussion
We can derive from the above that, entity distin- guishability the reason why current quantization strategies based on attribute similarity don't work better than the random approach. From Table and Figure 3, it's proved that random entity quan- tization distinguishes entities better in both code and codeword levels. Furthermore, in Figure 2, we show that entity quantization strategies with higher entity distinguishability tend to perform bet- ter. Therefore, it's reasonable that random quan- tization works comparable to current strategies or even better.
However, the above methods firstly need train KG embedding models with full parameter size. Recently proposed compositional parameter- efficient KG representation models (Galkin et al., 2022; Chen et al., 2023), which are illustrated in this paper, enable a more efficient training process.
Random Features in Graph Representation homogeneous graph learning, researchers prove that message-passing neural networks are more powerful when having randomly initialized node features (Sato al., 2021; Abboud et al., 2021). In KGs, Zhang et al. (2022) finds that random pertur- bation of relations does not hurt the performance of graph convolutional networks (GCN) in KG completion. Degraeve et al. (2022) further implies
6 Related Work
Knowledge Graph Embedding KG embedding aims at learning independent representations for

---

that an R-GCN with random parameters may be comparable to an original one. To the best of our knowledge, there no existing study on represent- ing entities with randomness in parameter-efficien compositional KG representation.
Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: A col- laboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, SIGMOD '08, page 1247-1250, New York, NY, USA.
7 Conclusion
Antoine Bordes, Nicolas Usunier, Alberto Garcia- Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi- relational data. In Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc.
In conclusion, this paper demonstrates the effective- ness of random entity quantization in parameter- efficient compositional knowledge graph represen- tation. We explain this surprising result by illus- trating that random quantization could distinguish different entities better than current entity quanti- zation strategies. Thus, we suggest that existing complicated entity quantization strategies are not critical for model performance, and there is still room for entity quantization approaches to improve entity distinguishability beyond these strategies.
Mingyang Chen, Wen Zhang, Zhen Yao. Yushan Zhu, Yang Gao, Jeff Z. Pan, and Huajun Chen. 2023. Entity-agmostic representation learning for parameter- efficient knowledge graph embedding. In AAAI.
Vic Degraeve, Gilles Vandewiele, Femke Ongenae, and Sofie Van Hoecke. 2022. R-gen: the could stand for random. arXiv preprint vrXiv:2203.02422
Limitations
Tim Dettmers, Pasquale Minervini, Pontus Stenetorp. and Sebastian Riedel. 2018. Convolutional 2d knowl- edge graph embeddings. Proceedings of the AAAI conference on artificial intelligence, volume 32.
This paper only studies entity quantization with encoders proposed in early works, while designing more expressive encoders is also important and can improve the performance of parameter-efficient compositional knowledge graph representation. We leave this part as future work.
Mikhail Galkin, Etienne Denis, Jiapeng Wu, and William L. Hamilton. 2022. Nodepiece: Composi- tional and parameter-efficient representations of large knowledge graphs. In International Conference on Learning Representations.
Acknowledgments
Ziniu Hu, Yichong Xu, Wenhao Yu, Shuohang Wang. Ziyi Yang, Chenguang Zhu, Kai-Wei Chang, and Yizhou Sun. 2022. Empowering language models with knowledge graph reasoning for open-domain question answering. In Proceedings of the 2022 Con- ference on Empirical Methods in Natural Language Processing, pages 9562-9581, Abu Dhabi, United Arab Emirates. Association for Computational Lin- guistics.
We would like to thank Ruichen Zheng for the dis- cussion on measuring distinguishability. Zhengbo Wang, Haotian Zhang, Chengehao Xu for the dis- cussion on entropy computation, and Jiahao Li for help to improve the technical writing of this paper.
This work is supported by the National Key Research and Development Program of China un- der Grant 202IYFF090160) the National Science Fund for Excellent Young Scholars under Grant 62222212, and the National Natural Science Foun- dation of China under Grant 62376033.
Shaoxiong Ji. Shirui Pan, Erik Cambria, Pekka Martti- nen, and Philip S. Yu. 2022. A survey on knowledge graphs: Representation, acquisition, and applications. IEÈE Transactions on Neural Networks and Learning Systems, 33(():494-55 5144.
Jiaang Li, Quan Wang. and Zhendong Mao. 2023. In- ductive relation prediction from relational paths and context with hierarchical transformers. In ICASSP 2023 2023 IEFE International Conference on Acoustics, Speech and Signal Processing (ICASSP).
References
Ralph Abboud, ismail Ilkan Ceylan, Martin Grohe, and Thomas Lukasiewiez. 2021. The surprising power of graph neural networks with random node initial- ization. In Proceedings of the Thirtieth International Joint Conference on Artifical Intelligence (ICAAI..
Farzaneh Mahdisoltani, Joanna Biega, and Fabian Suchanek. 2014. Yago3: A knowledge base from multilingual wikipedias. In 7th biennial conference on innovative data systems research. CIDR Confer- ence.
Ivana Balažević, Carl Allen, and Timothy M Hospedales. 2019. Tucker: Tensor factorization for knowledge graph completion. arXiv preprint arXiv:1901.09590.
George A. Miller. 1995. Wordnet: A lexical database for english. Commun. ACM, page 39-41.

---

Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. 1999. The pagerank citation rank- ing: Bringing order to the web. Technical report, Stanford infolab.
Aaron van den Oord, Oriol Vinyals. and Koray Kavukcuoglu. 2017. Neural discrete representation learning. In Proceedings of the 31 International Conference on Neural Information Processing Sys- lems, NIPS' 17, page 6309-6318.
Hao Peng. Haoran Li, Yangqiu Song, Vincent Zheng. and Jianxin Li. 2021. Differentially private federated knowledge graphs embedding. In Proceedings of the 30th ACM International Conference Information & Knowledge Management, pages 14 1416-1425.
Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, and Partha Talukdar. 2020. Composition- -aaed multi- relational graph convolutional networks. In Interna- tional Conference on Learning Representations.
Mrinmaya Sachan. 2020. Knowledge graph embedding compression. In Proceedings of the 58th Annual Meeting of the Association for Computational Lin- guistics. Association for Computational I. Linguistics.
Haoyu Wang, Yaqing Wang. Defu Lian, and Jing Gao. 2021a. A lightweight knowledge graph embedding framework for efficient inference and storage. In Pro- ceedings of the 30th ACM International Conference on Information & Knowledge Management, pages
Tara Safavi and Danai Koutra. 2020. CoDEx: A Com- prehensive Knowledge Graph Completion Bench- mark. In Proceedings of the 2020 Conference on Empirical Methods Natural Language Processing (EMNLP), pages 8328-8350, Online. Association for Computational Linguistics.
Kai Wang. Yu Liu, Qian Ma, and Quan Z Sheng. 2021b. Mulde: Multi-teacher knowledge distillation for low- dimensional knowledge graph embeddings. In Pro- ceedings of the Web Conference 2021, pages 1716- 1726.
Ryoma Sato, Makoto Yamada, and Hisashi Kashima. 2021. Random features strengthen graph neural net- works. In Proceedings of the 2021 SIAM Interna- tional Conference on Data Mining (SDM), pages 333-341. STAM.
Liang Wang. Wei Zhao, Zhuoyu Wei, and Jingming Liu. SimKGG: Simple contrastive knowledge graph completion with pre-trained language models. In Proceedings of the 60th Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), pages 12844
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling, 2018. Modeling relational data with graph convolu- tional networks. In The Semantic Web: 15th Inter- national Conference, ESWC 2018, Heraklion, Crete, Greece, June 3-7, 2018, Proceedings 15, pages 593- 607. Springer.
Quan Wang. Pingping Huang, Haifeng Wang. Songtai Dai, Wenbin Jiang, Jing Liu, Yajuan Lyu, Yong Zhu, and Hua Wu. 2019. Coke: Contextualized knowledge graph embedding. arXiv preprint arXiv: 1911:9911000 1911,0216.
Kai Sun, Dian Yu, Jianshu Chen, Dong Yu, and Claire Cardie, 2022. Improving machine reading compre- hension with contextualized commonsense knowl- edge. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers), pages 8736-8747, Dublin, Ireland. Association for Computational Linguistics.
Quan Wang. Zhendong Mao, Bin Wang. and Li Guo. 2017. Knowledge graph embedding: A survey of approaches and applications. IEEE Transactions on Knowledge and Duta Engineering, 29(12):7724 2743.
Zhiqing Sun, Zhi-Hong Deng. Jian- Yun Nie, and Jian Tang. 2019, Rotate: Knowledge graph embedding by relational rotation in complex space. In International Conference on Learning Representations.
Xiaozhi Wang. Tianyu Gao, Zhaocheng Zhu, Zhengyan Zhang, Zhiyuan Liu, Juanzi Li, and Jian Tang. 2021c. Kepler: A unified model for knowledge embedding and pre-trained language representation. Transac- tions of the Association for Computational Linguis- rics, 9:176-19 194.
Komal Teru, Etienne Denis, and Will Hamilton. 2020. Inductive relation prediction by subgraph reasoning. In International Conference on Machine Learning. pages 9448-9457. PMLR.
Liang Yao, Chengsheng Mao, and Yuan Luo. 2019. Kg- bert: Bert for knowledge graph completion. arXiv preprint urXiv:1909.000
Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoi- fung Poon, Pallavi Choudhury, and Michael Gamon. 2015. Representing text for joint embedding of text and knowledge bases. In Proceedings of the 2015 conference on empirical methods in natural language processing, pages 1499-1509.
Zhanqiu Zhang, Jianyu Cai, Yongdong Zhang, and Jie Wang. 2020. Learning hierarchy-awars knowledge graph embeddings for link prediction. Proceedings of the AAAT Conference on Artificial Intelligence, 34(03):3065-30722
Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction. In Interna- tional confer conference on machine learning, pages 2071- 2080. PMLR.
Zhanqiu Zhang, Jie Wang, Jieping Ye, and Feng Wu. 2022. Rethinking graph convolutional networks in knowledge graph completion. In The Web Confer- ence 2022.

---

Yushan Zhu, Wen Zhang, Mingyang Chen, Hui Chen. Xu Cheng. Wei Zhang, and Huajun Chen. 2022. Du- alde: Dually distilling knowledge graph embedding for faster and cheaper reasoning. In Proceedings of the Fifieenth ACM International Conference on Web Search and Data Mining, pages 1516-1524.
Zhaocheng 7hu, Zuobai Zhang, Mouis-Pascal Xhon- neux, and Jian Tang. 2021. Neural bellman-ford ford networks: A general graph neural network frame- work for link prediction. In Advances in Neural Information Processing Systems, volume 34, pages 29476-29490. Curran Associates, Inc.

---

A Training Details
FB15K-237
We use the exact same hyperparameier settings as in the original papers of NodePiece and EARL. The results of model variants with random approaches are averaged in three runs with different seeds. We do not further tune hyperparameters for our pro- posed model variants. Thus, their performance may be underestimated. But so long as the model variants perform similarly to the original models, we can still make our conclusions. For NodePiece, we obtain matched anchors with its 'shortest-patte mode to be consistent with the original paper. For its released anchor set. We run our experiments with single RTX 3090 GPU with 24GB RAM.
v1
v2
v3
v4
NodePiece NodePiece+RQ 0.867
0.873
0.939
0.944
0.949
0.942
0.945
0.944
Table 7: Inductive relation prediction results (Hits@ 10) of NodePiece and its variant with fully random entity quantization (RQ).
described in Section 4.2, on these subsets. We set the learning rate le-3 and train the variant with 300 epochs on vl/v4, and 120 epochs on v2/v3. The other hyperparameters remain the same as the original method. We use the exact same evalu- ation protocol as in previous works (Teru et al., 2020, Zhu et al., 2021; Galkin et al., 2022; Li et al., 2023). The results are shown in Table 7.
EARL,
We use uniformly distributed random numbers in our model variants. It means that in model vari- ants that randomly match entities to codewords, each codeword has equal probability of being matched.
We see that NodePiece's variants with ran- dom entity quantization perform well the orig- inal model inductive link prediction. The results support and strengthen our claim that random entity quantization is effective, both in the transductive and inductive settings.
Proof of Theorem
Proof. Since random entity quantization matches entities to codewords with equal probabilities, produces all entity codes uniformly. Thus, the prob- ability of random entity quantization to get |E| dif- ferent entity codes is:
P(2 (E)) (2))(E)
E1-1
E
2 i
(2
E
Prob
(2
2
1=0
where P(n, r) is the number of permutations of selecting r elements from set of n elements. I is the total number of codewords, and the total number of entity codes that can be produced is 2 g)
L
Inductive Relation Prediction Results
Besides FB15K-237 and WNI8RR datasets used the main text, we further test the effectiveness of random entity quantization on inductive relation prediction, where NodePiece has shown superi- ority. This task requires learning from one KG, and generalizes to another KG with no shared enti- ties for inference. We follow previous works (Zhu al., 2021; Galkin al., 2022; Li al., 2023) and use the datasets proposed by Teru al. (2020), including four versions of subsets generated from FB15k-233 We test the performance of NodePiece with the fully random entity quantization (RQ) as


Reviews:
[{"review_id": "IwDQG7J5be", "reviewer": "Reviewer_ummp", "summary": "", "questions": "None.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for a positive evaluation, including novel, inspiring, interesting, and extensive experiments, etc. There are also valuable questions and advice we try to answer as follows.\n\nQ1: Intuitively, parameter-efficient KG embedding models can perform inductive inference. NodePiece conducts experiments on inductive link prediction. It is better to provide results about inductive link prediction to further demonstrate the effectiveness of the proposed random entity quantization.\n\nA1: We have conducted the inductive link prediction experiments to make this a more comprehensive work. We use inductive link prediction datasets proposed in [1], including four versions of subsets based on FB15k-237 (FB237). We test NodePiece with the completely random quantization (RQ) proposed in our paper, and show the  preliminary results (Hits@10) in the table below. We will complete these results during paper revision due to the limited time.\n\nThe results show that NodePiece with random entity quantization performs as well as the original model in inductive link prediction, which supports and strengthens our claim that random entity quantization is effective.\n\n| Method       | FB237-v1 | FB237-v2 | FB237-v3 | FB237-v4 |\n| ------------ | -------- | -------- | -------- | -------- |\n| NodePiece    | 0.873    | 0.939    | 0.944    | 0.949    |\n| NodePiece+RQ | 0.867    | 0.942    | 0.945    | 0.944    |\n\n\n\nQ2: In line 178, the authors mentioned that one entity matches all its connected relations. However, NodePiece randomly samples a set of $ m$ unique outgoing relations for an entity, where $m$ is a hyper parameter. It seems that the statement in this paper is in conflict with that in NodePiece.\n\nA2: We would like to clarify that what we do is exactly the same as NodePiece. Although NodePiece's paper says, \"randomly sample $m$ unique outgoing relations,\" its actual practice in the code is to add inverse edges to KG and then select $min(m, d)$ unique relations for one entity $e$ from all its connected relations, where $m$ is a hyperparameter and $d$ is the number of all $e$'s connected relations. Our experiments and analysis follow the same setting. We will modify our statement in line 178 to make it more rigorous. This statement should be like, \"After adding reverse edges, one entity matches $min(m,d)$ unique relations from its connected relations.\"\n\n\n\n[1] Inductive Relation Prediction by Subgraph Reasoning.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- This paper points out the shortcomings of existing parameter-efficient KG embedding models and will inspire future work in this field.\n\n- The idea of modeling existing entity composition strategies as random entity quantization is novel.\n\n- The authors conducted expensive experiments to demonstrate the effectiveness of the proposed method and show an interesting finding that random quantization can yield results comparable to existing quantization methods.", "reasons_to_reject": "- Intuitively, parameter-efficient KG embedding models can perform inductive inference. NodePiece conducts experiments on inductive link prediction. It is better to provide results about inductive link prediction to further demonstrate the effectiveness of the proposed random entity quantization.\n\n- In line 178, the authors mentioned that one entity matches all its connected relations. However, NodePiece randomly samples a set of $m$ unique outgoing relations for an entity, where $m$ is a hyper parameter. It seems that the statement in this paper is in conflict with that in NodePiece.", "questions_to_authors": "None.", "paper_topic_and_main_contributions": "This study investigates entity quantization in parameter-efficient knowledge graph (KG) embedding models, which is the process of getting corresponding codewords for each entity. It presents a random entity quantization strategy and creates several model variants based on three strategies for obtaining the matching codewords for each entity: randomly choosing codewords, randomly or equally setting codeword weights, and randomly generating codebooks. The authors conducted extensive experiments to compare the proposed quantization strategies to previously designed quantization methods. The experimental results reveal an intriguing finding: random quantization can yield results comparable to existing complex quantization methods. Analyses are also offered to back up the findings. This study, I think, can help advance the development of parameter-efficient KG embedding models.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "None.", "Typos_Grammar_Style_and_Presentation_Improvements": "- L262: CoDEx-L(Safavi and Koutra, 2020) -> CoDEx-L (Safavi and Koutra, 2020)"}, "extra_text": {}}, {"review_id": "LKD17OOARA", "reviewer": "Reviewer_B12b", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for a valuable and constructive review. We will try to answer all the questions below, and hopefully address the reviewer's concerns.\n\n\n\nQ1: While it's explained in the paper why random entity quantization works, it's not well-explained why current quantization strategies that are based on PageRank or attribute similarity don't work better than random entity quantization. A general direction for a better entity quantization strategy is also desired in such an analysis.\n\nA1: The reason why current quantization strategies don’t work better than random entity quantization is that random entity quantization can distinguish different entities better, which benefits the model performance. To prove that, we kindly refer the reviewer to Table 6 and Figure 3, where we show that random entity quantization has higher entity distinguishability. For example, it achieves 16.75 bits in code-level entropy on the WN18RR dataset, while the strategy of EARL only reaches 8.20 bits. Furthermore, in Figure 2, we show that entity quantization strategies with higher entity distinguishability tend to perform better and achieve higher MRR. Therefore, it's reasonable that random entity quantization works comparable to current strategies or even better.\n\nBased on our work, we think a general direction for a better entity quantization strategy could be designing strategies to further increase the distinguishability of different entities, which would make models perform better. \n\nWe will add more discussions to clarify the above points during the revision.\n\n \n\nQ2: The method is not really parameter-efficient. The parameter saving in Table. 5 is not significant.\n\nA2: Our claim is not to save more parameters than existing compositional KGE methods (EARL/NodePiece), which our method builds on. They are already parameter-efficient compared to traditional KGE methods like RotatE, as supported by the comparison in parameter sizes in the table below.\n\n Our purpose in listing the parameter sizes in Table 5 is to show that, although it's not our initial intention, our proposed random entity quantization is even more efficient than existing compositional methods. For example, EARL has 0.527 Hits@10 with 3.8M parameters on WN18RR, while with fully random entity quantization, it can achieve 0.535 Hits@10 with only 3.3M parameters.\n\n| Method    | FB15k-237 | WN18RR | CoDEx-L |\n| --------- | --------- | ------ | ------- |\n| RotatE    | 29.3 M    | 40.6 M | 78 M    |\n| EARL      | 1.8 M     | 3.8 M  | 2.1 M   |\n| NodePiece | 3.2 M     | 4.4 M  | 3.6 M   |\n\n\n\n Q3: The advantages of compositional KGE methods are in inductive link prediction, but the experiments are all transductive link prediction (as I can tell from the paper). Thus, the results are not convincing that random entity quantization is really better or as good.\n\nA3: We have conducted the inductive link prediction experiments to make this a more comprehensive work. We use inductive link prediction datasets proposed in [1], including four versions of subsets based on FB15k-237 (FB237). We test NodePiece with the completely random quantization (RQ) proposed in our paper, and show the preliminary results (Hits@10) in the table below. We will complete these results during paper revision due to the limited time.\n\nThe results show that the compositional KGE method with random entity quantization performs as well as the original model in inductive link prediction, which supports and strengthens our claim that random entity quantization works as well as current strategies.\n\n| Method       | FB237-v1 | FB237-v2 | FB237-v3 | FB237-v4 |\n| ------------ | -------- | -------- | -------- | -------- |\n| NodePiece    | 0.873    | 0.939    | 0.944    | 0.949    |\n| NodePiece+RQ | 0.867    | 0.942    | 0.945    | 0.944    |\n\n\n\nQ4: Usage of the term \"quantization\" is misleading.\n\nA4: We model compositional KGE methods as \"entity quantization\" because we think this term could appropriately describe the process of such methods and their similarity to vector quantization. We kindly refer the reviewer to line 48 for details about entity quantization. In short, like the way to represent vectors in vector quantization, entities, without independent representations, are represented by codewords matched from the codebooks. Another similarity is that both techniques save parameters due to this kind of representation. Therefore, we choose this term to highlight the characteristics of compositional KGE methods. We will add more discussions during the paper revision to clarify the above points.\n\n \n\nQ5: Best performance can be marked in boldface for clarity in Table. 2, 3, and 4.\n\nA5: Thanks for your kind advice. The purpose of these tables is to show that different variants of random entity quantization perform comparable to the original method, rather than highlight the highest value. So we didn't bold the best performance.\n\n\n\n[1] Inductive Relation Prediction by Subgraph Reasoning.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The analysis is interesting and important for future research in compositional KGE.\n2. The conclusion suggests current entity quantization strategies can be improved, which encourages more research in this direction.", "reasons_to_reject": "1. While it's explained in the paper why random entity quantization works, it's not well-explained why current quantization strategies that are based on PageRank or attribute similarity don't work better than random entity quantization. A general direction for a better entity quantization strategy is also desired in such an analysis.\n2. The method is not really parameter-efficient. The parameter saving in Table. 5 is not significant.\n3. The advantages of compositional KGE methods are in inductive link prediction, but the experiments are all transductive link prediction (as I can tell from the paper). Thus, the results are not convincing that random entity quantization is really better or as good.\n\n======= After revision ======= \n\nThanks to the authors for the comprehensive response. It addressed the majority of my questions and concerns. The additional experiment on inductive link prediction is also encouraging. Thus, I decided to raise the Soundness to 4 (strong) and Excitement to 3 (Ambivalent).", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper conducts an empirical analysis of random entity quantization for compositional knowledge graph embedding (KGE) methods. Two baseline methods, NodePiece and EACL are selected. The results show that random entity quantization can achieve similar performance as current entity quantization strategies due to high entropy in the code level, which makes the entities distinguishable.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. Usage of the term \"quantization\" is misleading.\n2. Best performance can be marked in boldface for clarity in Table. 2, 3, and 4."}, "extra_text": {}}, {"review_id": "6Qp5ymFmWc", "reviewer": "Reviewer_Y2GK", "summary": "", "questions": "Q1 In Section 3.3, is the evaluation setting the same as the state-of-the-art works?\nQ2 Is there any further research direction?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the reviewer’s positive evaluation, including well motivated, clearly organized, and well supported. There are also valuable questions we try to answer as follows.\n\n\n\nQ1: Is the evaluation setting the same as the state-of-the-art works?\n\nA1: Yes, we use evaluation settings that are widely used in many state-of-the-art works, such as [1]\\[2\\][3].\n\n \n\nQ2: Is there any further research direction?\n\nA2: Based on our work, we think further research could study how to quantize entities to make them more distinguishable in theory. It would also be interesting to design more powerful encoders to boost the model’s performance.\n\n \n\n[1] RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space.\n\n[2] Learning Hierarchy-Aware Knowledge Graph Embeddings for Link Prediction.\n\n[3] Entity-agnostic representation learning for parameter-efficient knowledge graph embedding.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "S1. The paper is well motivated and clearly organized. \nS2. The claim is well supported by experimental studies and the theoretical analyses. ", "reasons_to_reject": "Only some minor issues. See below. ", "questions_to_authors": "Q1 In Section 3.3, is the evaluation setting the same as the state-of-the-art works?\nQ2 Is there any further research direction?", "paper_topic_and_main_contributions": "This work revisits the current compositional knowledge graph representation methods, and experimentally validate that simple random entity quantization can achieve state-of-the-art performance. It further justifies such phenomenon by revealing that the random quantization has high entropy at the code level and Jaccard distance at the codeword level, which makes entity more distinguishable. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces the technique of random entity quantization to knowledge base embeddings. The paper is also appealing as it questions the validity of existing methods. The experimental results, inclusive of additional experiments in the discussion period, are persuasive. We eagerly await the camera-ready version that incorporates feedback from the reviews and discussions!
Decision: Accept-Main