Inductive Relation Inference of Knowledge Graph Enhanced by Ontology Information
Authors: Wentao Zhou
Jun Zhao
Tao Gui
Qi Zhang
Xuanjing Huang
Venue: EMNLP 2023 Findings (2023)
Keywords: Ontology Information
Inductive Relation Inference
Knowledge Graph

Abstract: The inductive inference of the knowledge graph aims to complete the potential relations between the new unknown entities in the graph. Most existing methods are based on entity-independent features such as graph structure information and relationship information to inference. However, the neighborhood of these new entities is often too sparse to obtain enough information to build these features effectively. In this work, we propose a knowledge graph inductive inference method that fuses ontology information. Based on the enclosing subgraph, we bring in feature embeddings of concepts corresponding to entities to learn the semantic information implicit in the ontology. Considering that the ontology information of entities may be missing, we build a type constraint regular loss to explicitly model the semantic connections between entities and concepts, and thus capture the missing concepts of entities. Experimental results show that our approach significantly outperforms large language models like ChatGPT on two benchmark datasets, YAGO21K-610 and DB45K-165, and improves the MRR metrics by 15.4% and 44.1%, respectively, when compared with the state-of-the-art methods.

Inductive Relation Inference of Knowledge Graph Enhanced by Ontology Information
Wentao Zhou'; Jun Zhao'; Tao Gui?; Qi Zhang'; Xuanjing Huang!
Schooo of Computer Science, Fudan University
2 Institute of Modern Languages and Linguistics, Fudan University International Human Phenome Institutes (Shanghai)
{zhaoj19, tgui, qz eefudan. edu. cn, wtzhou22@m. fudan. edu. cn
Abstract
Train Gruph
Test Graph
The inductive inference of the knowledge graph aims to complete the potential relations between the new unknown entities in the graph. Most existing methods are based on entity- independent features such as graph structure information and relationship information to inference. However, the neighborhood of these new entities is often too sparse to obtain enough information to build these features effectively. In this work, we propose a knowledge graph inductive inference method that fuses ontol- ogy information. Based on the enclosing subgraph, we bring in feature embeddings of concepts corresponding to entities learn the semantic information implicit in the ontology. Considering that the ontology information of entities may be missing, constraint regular loss to explicitly model the semantic connections between entities and concepts, and thus capture the missing concepts of entities. Experimental results show that our approach significantly outperforms large language models like ChatGPT on two bench- mark datasets, YAG02IK-610 and DB4SK- 165, and improves the MRR metrics by 15.4% and 44. 44.1%, 1%, respectively, when compared with the state-of-the-at methods.
Snizad
Iris
NazI
Olk
Fxcial
Just I
HARI
Oatology Information
Two examples of inductive inference. As shown in Test Graph A, structure-based inductive inference methods have difficulty predicting relations between newly added entities that lack sufficient neigh- borhood information. Our work helps achieve inductive relation inference by using ontology information (Test Graph B).
Figure I:
build type
In past studies, most methods (Bordes et al.. 2013; Yang et al., 2015) mainly learn the specific embeddings of entities and relations and predict missing links by various mapping operations. Since the embeddings in such methods depends on specific entities, it requires that the entities in the graph are fixed, which is referred to as the transductive setting (Yang et al., 2016). However, in fact, new emerging entities are continuously added to real-world KGs over time, such as new users and products in e-commerce knowledge graphs and new molecules in biomedical knowl- edge graphs (Trivedi et al., 2017). Works adopting the transductive setting often require expensive retraining make predictions for these added entities. As the amount of data increases this overhead will become unaffordable (Schlichtkrull et al., 2018). To deal with this problem, inductive inference gains the ability to extend what learned from training entities to unknown entities by learning entity-independen semantic information (Teru et al., 2020).
1 Introduction
Knowledge graphs (KGs) store large amount of structured real-world knowledge through set of triples, and they have been widely used in many do- mains, such natural language processing (Zhang al. 2019), recommendation systems (Wang et al., 2018), and question answering (Huang et al., 2019). However, even the most knowledge-rich KGs suffer from incompleteness, such as DBpedia (Lchmann et al., 2015), YAGO (Mahdisoltani et al., 2014), and WikiData (Vrandečić, 2012). To complete the KGs, knowledge graph inference aims to predict the missing links between entities in KGs.
"Equal Contributions.
*Corresponding authors.
However, most of the existing inductive infer-

---

ence work (Chen al., 2021; Mai et al., 2021) has only focused on various structural features in KGs (e.g., enclosing subgraphs induced by paths between nodes), ignoring the important ontology information. In general, newly emerging entities tend to lack sufficient neighbor relationships, leaving them without much contextual information to refer to in terms of structure (As shown in Test graph A in fig. 1) (Xu et al., 2022). As abstract description of the real world, various concepts in ontology provide the basic type information for the affiliated entities, which can help the model achieve inductive link prediction. For example, in Test Graph B, despite lacking the neighbor relationship. with the help of the type information provided by the concepts Person" and "City" for "JBullerr and "ammi" respectively, the relation between them most likely be "lives_ in". In fact, the ontology information of entities also suffers from missing problems, so not easy to use ontology information effectively.
of inductive link prediction.
Related Works
Transductive Link Prediction: Most existing knowledge graph inference methods are embedding-basee transductive learning. These methods can be broadly classified into: 1)
translation-bbssedBBrrdes et al.. 2013; Wang et al., 2014; Lin et al., 2015; Sun al., 2019); 2) semantic matching- (Yang et al., 2015;
Trouillon et al., 2016; Nickel et al., 2016); 3) CNN-based((Schiccttkul et al., 2018; Vashishth et al., 2020; Liu et al., 2021). The difference between them mainly lies in how to design the score function.
Inductive Link Prediction: As inductive inference models have the ability extend from known entities to unknown entities, these methods(Ali et al., 2021; Gesese et al., 2023) show great potential for link prediction tasks on new entities, BLP(Daza et al., 2021) Iearns the embedding representations of entities based on the architecture of the pre-trained language model to obtain the required generalization capabilities. Grall et al., 2020) suggests modeling enclosing sub- graphs around the target triple for the first time, based on the graph neural network framework. with the problem of missing ontology information, we build a type constraint regular loss that captures the missing concepts of entities by explicitly modeling the semantic associations between entities and concepts. In addition, we train the link prediction on the ontology triples. The final training performed using joint strategy. Experimental results show that the method significantly outperforms ChatGPT as well as state- of-the-art inductive baselines. Our codes are publicly available at GitHub."
can predict that
To solve the above problem, we propose knowledge graph inductive inference method combining ontology information. Specifically, based on the enclosing subgraph, we bring in feature embeddings of corresponding concepts at the node initialization of entities to obtain the semantic information in the ontology. To deal
TACT(Chen et al., 2021) models the semantic correlation between relations as several topological patterns and uses relational correlation network (RCN) to learn the importance of different patterns for inductive link prediction. SNRI(Xu et al., 2022) enhances the processing for sparse subgraphs by exploiting full neighbor relationships and by applying mutual information (MI) maximization to knowledge graphs.
Ontology Enhanced Inference: Incorporating ontology information through various methods(Xie et al., 2016; Ren et al., 2021), it helps models learn richer semantic information. TransC (Lv et al., 2018) models the embedding of concepts as a sphere and assumes that the embedding corresponding to the entity belonging to the concept should be in this sphere. Considering the limitation of relation information in the ontology, TransO (Li et al., 2023) computes the weights of the type mapping matrix based on domain and range. JOIE (Hao al., 2019) proposes the first approach of jointly embedding entity and ontology knowledge graphs to build a unified representation learning framework from multiple levels. These
Our contributions can be summarized the following three points: (1) we propose a knowl- edge graph inductive inference method com- bining ontology information, which effectively improves the inductive inference performance on newly emerging entities; (2) build type- constrained regular loss to alleviate the problem of missing ontology information; (3) we achieve a remarkable improvement on two benchmark datasets, demonstrating the effectiveness of using ontology information to enhance the effectiveness

---

Step ①
Step
Ontology Information
Node Initialization
Ontology Training (Cone)
Type Constraint (Frge)
Person
Negative concept
Player
City
Plaser
City
Type linkes
ontology graph
Positive concept
Hollywood
Savannah
LA
Step 2
Lebrm
B
Davis
Input
f.(u,re,))
Ohio
Lakers
Knowledge Graph
Extracted Subgraph Go
Subgraph Neural Network
Figure 2: Overview of the proposed method. The training pipeline consists of two parts: extracting the enclosing subgraph target triple from KG and initializing the node embeddings with ontology information (StepO); optimizing the score function fe used to infer relations between unknown entities (Step2) In addition, optimize the embedding of the ontology by ontology training and construct type constraint to capture the semantic connections between concepts and entities (Step 3))
methods are all transductive link predictions on the original KGs. Inductive link prediction for unknown entities is still urgent but under- researched task.
unknown entities and effectively captures the missing ontology information of entities by type constraint. As shown in fig. 2, the whole training consists of three steps:
Step @: This part aims to integrate ontology information. First, we extract the enclosing subgraph around the target node and obtain the position feature hi of nodes in the subgraph. Second, we obtain the type feature hame from the embedding of the concept corresponding the entity by the attention approach. Finally, we concatenate the position feature h:!** with the type feature h type as the initial feature of the node. this way, we integrate the semantic information the ontology into the entity.
3 Approach
We start with task formulation. Unlike transductive link prediction which predicts unknown triples fixed entity graph, for inductive link prediction, the goal is to predict unknown triples (u, Ft,, v) in the unseen entity graph by learning from the seen entity graph. The entities in these two graphs are disjoint. Specifically, given the seen entity graph Ge, and the migratable information (e.g.. ontology graph G. and type links between entities and concepts TQ), we optimize the score function [s, r,,) training time, where the score reflects the likelihood of the existence of target relation r between target nodes s and f. In testing, combined with the migratable information, we use the score function of to predict the unknown triples (u, Tt, v) in the unseen entity graph Ge..
Step @: This step aims to optimize the score function fe, Ft, v) used for prediction. For the representations of nodes and relations in the subgraph, we use a graph neural network (GNN) to update them. Then, we optimize the score function (e, Tı, v) obtained with these represen- tations. Since the node representations contain type features, the model can use ontology information to help complete inductive link prediction.
3.1 Method Overview
In this work, we propose an inductive link pre- diction method that combines entity and ontology information. The method brings in ontology information to enhance inductive inference on
Step 3: This step aims to construct type constraint and optimize ontology embedding. We construct a type constraint regular loss fr by modeling the relationship between entity embeddings and

---

concept embeddings. The constraint requires that the embedding of the entity h should be close to the embedding of its corresponding concept e after mapping to the feature space of the ontology. Based on this constraint, the model gains the ability to capture the missing ontology information of the entity. In addition, we train the link prediction task on the ontology triples to optimize the embedding representation of the ontology in order enhance the semantic information obtained by the model from the ontology.
relation e, and σj is the sigmoid function. Finally. by connecting the position feature he and type feature 1 hiuee, we obtain the initial embedding of the node h.:
type
h
(4)
We think that using ontology type information to guide the node initialization helps the model to learn the semantic information implicit in the ontology.
Finally, based on the above steps, the model learns from the total loss C. Next, we will describe the technical details of each step in detail.
3.3 Subgraph Neural Network
We input the enclosing subgraph ((a,r.,)) into the GNN to update the embedding of the nodes. We define the update function based on the architecture of R-GCN (Schlichtkrull et al., 2018):
3.2 Ontology Information Feature Embedding
Subgraph Extraction: We extract the enclosing subgraph around the target triple (u, v) from GralL. (Teru et al., 2020). First, we obtain the sets (u) and Nig(v) of nodes the corresponding k-hop neighborhoods from the two target nodes u and v, respectively. Then, the duplicate nodes are removed by taking the intersection set (=)u)) ( (v). Finally, the nodes isolated from any node at distance greater than k are cut off to obtain the enclosing subgraph.
h ReLU se 1 a
(5)
where af denotes the neighbor feature aggregation function. Inspired by CompGCN (Vashishth et al., 2020) and edge attention, we define af as:
R
afre W MCc kock-- hh-1) (6)
0
=
==1 sEN,(t)
arr,st
k
(W 02 $$ xbb))
(7)
Node Initialization: We use the position feature 1:00 of the node and the type feature h: tye as the initial embedding of the node features to ensure that the node features do not contain any node attributes. First, we obtain the position feature hop0 of the node in subgraph by double radius vertex labeling (Zhang and Chen, 2018) scheme:
= ReLU(W -o-leel
e e
(8)
where (t) denotes the direct outgoing neighbors of node / under relation r, o is the edge attention weight of edge rs,r,t) () layer k, and T2 is the sigmoid function. Φ(ct 1. h 1) the fusion operation on the features of the neighboring nodes and relations. We set it as the subtraction ((e, h) = h - e eVVshhsshh et al., 2020). In order to keep the same embedding space for nodes and relations, we also update the relation embedding:
A,000 = (one-hot(d((, u)) @ one-hot(d(i, r))] (1)
where d(i, u) denotes the shortest distance from node u without passing through v. Second, we obtain the type feature wype from the embedding of the concept corresponding to the entity by the attention approach:
L
Wh
White k-1 reler
(9)
In addition, we also use the JK-connection mech- anism(Xu et al., 2018) on the representation nodes and relations, and this approach makes the performance of the model robust to the number of layers of the GNN. The representation of the subgraph ((n.ry is obtained by average pooling of all node representations:
h
type
01
Σ ajWic + b1)
(2)
GEC,
exp(cfc)
a, = softmax(c,,,) Eqre, exp(ct
(3)
where C, is the set of concepts corresponding to node h; denotes the type relation between entities and concepts, aj reflects the importance of the type information in each concept oj under the type
h
(10)
1
iCV

---

Finally, we obtain the score of the target triple (u,ri, v):
Dataset
dET
aC:
RM
POT
WTI.
16357
30000
1983
YAGO21K-610
valid
4358
21
3000
166
14
248
1783
6970
fe(u,rr.v) 1 Wh @ hn + by e e
main
29509
250
66000
165
20
516
29509
D645K-165
66000
(11)
9681
177
15000
51
65
9682
where he h and en denote the embedding of target nodes u, v and target relation r. We obtain the negative triples used in loss functions below by replacing the head or tail entity with uniformly sampled random entities. The margin- based loss function in the subgraph is:
Table 1: Statistics of inductive benchmark datasets. We denote the number of entities, relations, entity triples, concepts, meta-relations, ontology triples, and type links using #R, #E, #ET, #C, #M, #OT, and #TI. respectively.
information obtained by the model from the ontology. Therefore, we train the link prediction task on the ontology triplet (Ve: my; Ue), and the corresponding score function is:
Lent
Σ max(O,Seeu',,n
(u.rew))CG
- Se(u,, rt, v) + m))
(12)
folue, m,, Ue) = =l + my hes 12
(16)
where (u, Ts, 2) and (u/, rr, r²) denote positive and negative samples, respectively, and 7/ is the margin hyperparameter.
where hag and hye denote the embedding concepts, and me denotes the embedding of meta- relations between concepts. The margin-based loss function obtained by ontology training is:
3.4 Type Constraint and Ontology Training
We explicitly model the relationship between entity embeddings and concept embeddings. Specifically, this requires that the embedding of the entity h should be close to the embedding of its corre- sponding concept e after mapping the ontology embedding space. This way builds a type constraint ft in the form of regularization:
Conto =
Σ
max folue my. vo)
(17)
-So(u'o me, v/) + 13).
where (uer my, vo) and (u'a my v/) denote positive and negative samples.
fulu, Cu,o) = CC, En..C
Σ
|ew Gupll2 (13)
3.5
Joint Training Strategy
We combine all the losses to obtain the total loss L. The overall training objective is follows:
Cu.y 033 (Walln + ba),
(14)
where ha is the embedding of the target node v, Ca,e is the embedding of the entity embedding after mapping the ontology embedding space, Cw the embedding of the corresponding concept of node or v, and Ø3 is the tanh function. We use the same negative sampling method as before. Thus, the margin- type constraint regular loss is:
L = = + aontt + whtype-
(18)
where and w are weighting hyperparameters. With the joint training strategy, our model can better utilize the ontology information to enhance the inductive inference on unknown entities, while effectively capturing the missing ontology informa- tion of entities.
Ligne
Σ
f0, Cu,))
4 Experimental Setup
(u.
(15)
4.1 Datasets
- -f((u, U. Cu,v') 72)-
The KG benchmark datasets with ontology informa- tion, YAGO26K-99 and 10111K-1744 (Hao et al., 2019), are originally developed for transductive inference prediction. To facilitate the inductive test, we build two new datasets, YAGO21K-616 and DB45K-165, based on the two original benchmark datasets. Both datasets are built from entity triples,
where (u, v, C,,v) and (u, U, Cu. denote positive and negative samples. Inspired by(Hao et al., 2019; Dong et al., 2021), learning meta-relations between conceptst will enhance the semantic
'ccta-rclatios include hicrarchical relations between concepts and other general meta-relations

---

6AG
DB45K-165
Method
MRR
@its@ 11
@its@ 101
MRR1
Hits@ It
Hits 10
GralL(Teru et al., 2020)
0.682
0.666
0.684
0.540
0.515
0.548
TACT(Chen et al., 2021)
0.688
0.674
0,689
0,493
0,471
0,485
SNRI(XU et al., 2022)
0.384
0.332
0.528
0,408
0,316
0,556
ChatGPT
0.271
0.219
0.333
0,504
0.431
0.603
Ours
0.794(15.4)
0.758(12.5)
0.845(22.6)
0.778(44.1)
0.704(36.7)
(.925(53.4)
Table Main results on two inductive benchmark datasets. numbers in bold and underlined indicate best and second-boe broodd-bestt results, respectively, and the numbers in () indicate the percentage improvement of our method over the best baseline result.
ontology triples, and type links. In the entity triples, the entities in the test set do not appear in the train set and valid set, while the relations both the test set and valid set are included in the train set. We train on the train graph and test on the test graph. In addition, to achieve ontology training, we randomly divide the ontology triples into train set, a valid set, and test set in the ratio of 80%/10%/10% tab, provides the complete statistics for both datasets.
4.3
Metrics
As in the previous work (Teru et al., 2020; Xu et al., 2022), for the test triples (h, r; 4), we combine head (or tail) entities and relations with 50 candidate tail (or head) entities (including the original tail (or head) entities) obtained by random sampling to get positive and negative triples and rank all triples based on their scores. We use three metrics widely used in link prediction for evaluation. (1) MRR:
Please refer to Appendix A.1 for the detailed generation process of the datasets.
The average of the inverse of the ranking of the correct entities in all tested samples. (2) Hits @ 1: The ratio of correct entities ranked first in all test samples. (3) Hits @ 10: The ratio of correct entities ranked within the top for all test samples.
4.2 Compared Methods
To evaluate the effectiveness of our proposed approach, we compare our method with several state-of-the-ee thr- baseline methods. In addition, large language models (e.g., ChatGPT, GPT-4, etc.) have recently achieved impressive performance on several natural language processing tasks. Therefore, we also compare with ChatGPT.
4.4 Implementation Details
We extracted 3-hop enclosing subgraphs. In the training process, we use Adam as the optimizer, the learning rate is set to 0.01, and the batch size is set to 16. We use a three-layer GNN, and the dimensions of all feature embeddings are 32 except for the dimensions of the type feature embedding which is 24. The margins in the loss function are set 10, 10,5, and the weighting hyperparameters and are set
GralL(Teru al., 2020). The carliest method for inductive inference based on graph neural networks. It uses locally enclosing subgraphs and entity-independent node labels to represent node embeddings.
The maximum number of training epochs is 30. All experiments are conducted with Python 3.8.12 12.12 and PyTorch 1.11.0, using a GeForce GTX 2080Ti with 12GB RAM.
TCCTeChen et al., 2021). An inductive inference method based on relational correlation networks. The method categorizes all relation pairs into several topological patterns and uses the topology of the knowledge graph to learn the semantic correlation between relations.
Results and Analysis
5.1 Main Results
SNRI(Xu et al., 2022). An inductive inference method based graph neural networks. It utilizes complete neighborhood relations in terms of both neighborhood relation features of node features and neighborhood relation paths of sparse subgraphs and also models the neighborhood relations using mutual information maximization.
In this section, we make inductive link predictions for the proposed method and several comparative methods, and the results are shown in tab. 2. Since evaluate on newly constructed benchmark dataset, we re-implement Grall,, TACT, and SNRT under our evaluation metrics. To make a fair comparison, we keep the hyperparameters the

---

YAGO2IK-610
DB45K-165
YAGOZIK 610
DBASK 165
Part
1/
1.4
His !!
107
MRR)
Hiss 11
Hits
Cock
Head
0.833
0.816
0,934
0.774
0.705
0.915
Tal 0.734
0.700
0.755
0.781
0.703
0.935
Metses
Metrico
No sype
0.743
0.714
0.757
0.710
0.613
0914
Type 0.846
0.803
0.936
0.787
0.716
0.926
Ours
0.758
62
0.2
Table 3: Comparison of prediction performance in different components. Head (or Tail) indicates the performance when predicting the tail entity (or head entity) given the head entity (or tail entity) and the relationship, respectively. Type and No type indicate the performance when the entity to be predicted has and does not have type information, respectively.
60
0.0
4+
Neighbors
Neighbors
Figure 3: Prediction performance for target nodes with different number of neighboring nodes. When entities lack ontology information, our method degrades equal the baseline method. In each set of results, MRR, Hits@ 1, and @its@ 10 are shown from left to right.
same as their original papers. Similarly, we test ChatGPT with the same test data. The relevant details of the prompts are shown in Appendix A.2. From the results, we can observe that our method significantly outperforms the ofate-of-the-art bascline methods and ChatGPT. Specifically, some baseline methods (e.g., GraIL, TACT) only focus on various structural features in the knowledge graph, which makes them unable to handle sparse subgraphs effectively. And when the subgraph is empty (i.e., only the target triple exists), its performance even drops to the same as random guesses. In contrast, our approach effectively alleviates this problem by incorporating ontology information and achieves overall performance improvement (We provide more results to support this claim in sec. 5.2). Although approaches like SNRI attempt to deal with the sparse subgraph problem by importing global information, our proposed method still outperforms them. Moreover, although ChatGPT outperforms some of the baseline methods (or even all of them) in some metrics, method achieves better results compared to it.
Mcanwhile, the prediction performance for tail entities outperforms that for head entities on the YAGO2IK 610 dataset, while the prediction performance for both is similar on the DB45K- 165 dataset. In addition, the proportion of head entities lacking type information is higher than that in tail entities the YAGO2IK-611 dataset, while the proportions are close on the DB45K-165 dataset. Therefore, we believe that the integration of ontology information improves the prediction performance of our model.
Furthermore, counted the prediction perfor- mance for target nodes with different numbers of neighboring nodes. For the target nodes, the more neighboring nodes they have, the more paths may exist between the nodes and thus the more dense enclosing subgraphs are extracted. Therefore, by analyzing the performance improvement of the prediction of target nodes with different numbers of neighboring nodes by ontology information, we investigate the effect of ontology information on subgraphs with different densities, and the results are shown in fig. 3.From the results, we can see that integrating ontology information improves the prediction performance of target nodes with all different numbers of neighboring nodes, and the lower the number of neighboring nodes, the greater the performance improvement. This indicates that our method improve the prediction for all enclosing subgraphs, and effectively alleviate the problem of poor prediction performance for sparse subgraphs at the same time.
In addition, we also make transductive link predictions for the proposed method and several baseline methods for further comparison. For detailed experimental results, please refer to Appendix A.3.
5.2 Result Analysis
In this section, we analyze the sources of per- formance improvement of our approach. First, we counted all the test triples, and the results are shown in tab. 3. We can observe that the prediction performance for entities with ontology type information outperforms that for entities lacking ontology information on both datasets.
5.3 Type Prediction
Our previous analysis of the experimental results demonstrates that ontology information can effec- tively improve prediction performance. In fact,

---

Dataset
MRR1
Hits@
Hits 101
Method
MRR1
Hits@ 11
Hits@ 101
YAG02IK-610
0.858
0.791 0.388
0.953
w/o type information
0.664
0.649
0.665
DB45K-165
0.556
0.920
w/o ontology training
0.788
0.729
0.842
w/o type constraint
0.770
0.722
0.837
Table 4: Type prediction results on two inductive benchmark datasets.
w/o both
0.758
0.708
0,835
Ours
0.794
0.758
0.845
w/o type information
0.545
0.517
0.551
w/o ontology training
0.759
0.682
0,919
ontology information also suffers from missing information, for example, on the test set of the YAG02IK-610 dataset, only 1725 entities have type information and more than half of them miss ontology information. Since the semantic links between entities and concepts are explicitly modeled in method, method to predict type information for the test triples. Specifically, without providing ontology information, we predict the concept corresponding to the target node based on the enclosing subgraph of the target triple, and the prediction results are shown in tab. 4. From the results, we can see that our method can effectively predict the type information of entities, which helps alleviate the problem of missing ontology information.
w/o type constraint
0.758
0.680
0.915
w/o both Ours
0.717
0.648
0.868
0.778
0.704
0.925
Table 5: Abalation study of our method. The upper (resp. lower) part lists the results on YAGO2IK-610 (resp. DB45K-165).
YAGO21K-610
DB45K-165
0.01
Metric
Metrics
0.55
0-6
0.5
Figure Inductive prediction performance of models with different hop.
5.4 Ablation Study
To investigate the contribution of each component of our approach, we conduct ablation experiments on two datasets, and the experimental results are shown in tab. 5. First, like our analysis, the type information provided by the ontology plays very important role in the inductive link prediction. When this module removed, the prediction effect degrades to equal the baseline method, and the performance is severely compromised. Moreover, ontology training and type constraint improve the performance of the model in terms of optimizing the embedding representation of ontology and ex- plicitly modeling the semantic connection between instances and concepts, respectively, and removing either of these modules leads to a degradation of performance. And when both modules are removed at the same time, the performance will be further degraded.
size of the enclosing subgraph) on inductive prediction. From fig. 4, we can obtain the following observations. With the hop gradually increasing from 1, the prediction performance of the model keeps improving. This implies that too few neighbor nodes cannot provide enough semantic information for prediction(W: (We provide more results in Appendix A.4 support this claim). And when hop exceeds a certain threshold (i.e., 3), the performance starts to decrease. This indicates that the subgraph contains the critical structure needed for prediction after reaching a certain size and continuing to increase the size of the subgraph will continuously increase the number of noise nodes, which will lead to performance degradation.
5.6 Error Analysis
What are the remaining errors in our research? For the inductive prediction of various subgraphs with ontology information, our method has brought different degrees of performance improvement. Moreover, our method can capture the missing ontology information of the target nodes through the rich semantic information in the subgraphs. However, for those enclosing subgraphs that lack ontology information and are extremely sparse,
5.5 Hyper-parameter Analysis
As the structure that is relied when predicting, the size of the enclosing subgraph implies how much semantic information the target node can obtain from the structure. We conducted experiments on two datasets to investigate the effect of hop (which directly responds to the

---

our method has difficulty in giving correct type predictions due to the lack of available structural information, making the inductive prediction on such subgraphs not effectively improved. In fact, link prediction on extremely sparse enclosing subgraphs is difficult for all inductive prediction methods.
References
Mehdi Ali, Max Berrendorf, Mikhail Galkin, Veronika Thost, Tengfei Ma, Volker Tresp, and Jens Lehmann. 2021. Improving inductive link prediction using hyper-relational facts. In The Semantic Web Web-ISTV ISWC' 2021, pages 92, Cham. Springer International Publishing.
Antoine Bordes, Nicolas Usunier, Alberto Garcia- Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi- relational data. In Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc.
6 Conclusions
In this work, we propose a knowledge graph inductive inference method that combines ontology information. The approach integrates semantic information of ontology by using type information for the initialization of node features. We construct type-constrained regular loss, which effectively captures the missing ontology information of entities. At the same time, ontology training helps the model to enhance the semantic information obtained from the ontology. Experimental results show that our approach achieves state-of-the-art inductive link prediction.
Jiajun Chen, Huarui He. Feng Wu, and Jie Wang. 2021. Topology-awan correlations between relations for inductive link prediction in knowledge graphs. Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 6271-6278.
Daniel Daza, Michael Cochez, and Paul Groth. 2021, Inductive entity representations from text via link prediction. In Proceedings of the Web Conference 2021, WWW '21, page 798-808, New York, NY, USA. Association for Computing Machinery.
Jing Dong, Binbin Gu, Jianfeng Qu. An Liu, Lei Zhao, Zhigang Chen, and Zhixu Li. 2021. Hyperjoie: Two-view hyperbolic knowledge graph embedding with entities and concepts jointly. In Web Information Syxtems Engineering WISE 2021:
Limitations
Although we demonstrate the effectiveness of ontol- ogy training for improving model performance, we only use the simplest methods to model ontology graphs. Using some richer and more effective methods (e.g., using pre-trained embedding repre- sentations building detailed hierarchies) to learn better embedding representations of the ontology can help make the model further achieve superior performance. In addition, in all experiments, we set all margin and weighting hyperparameters as fixed hyperparameters, which may make our model not achieve its best performance. A dynamic optimization of these hyperparameters may be a better choice. We leave these limitations to our future work.
22nd International Conference on Web Information Systems Engineering, WISE 2021, Melbourne, VIC, Austrulia, Octuber 26-29, 2021, Proceedings. Part page 305-320, Berlin, Heidelberg. Springer- Verlag,
Genet Asefa Gesese, Harald Sack, and Mehwish Alam. 2023. Raild: Towards leveraging relation features for inductive link prediction in knowledge graphs. In Proceedings of the 11th International Joint Conference on Knowledge Graphs, JJCKG '22. page 82-90, New York, NY, USA. Association for Computing Machinery.
Junheng Hao, Muhao Chen, Wenchao Yu, Yizhou Sun, and Wei Wang. 2019. Universal representation learning of knowledge bases by jointly embedding instances and ontological concepts. Proceedings of the 25th ACM SIGKDD International Conference Knowledge Discovery Data Mining, KDD '19, page 1709-1719, New York, NY, USA. Association for Computing Machinery.
Acknowledgements
The authors wish to thank the anonymous reviewers for their helpful comments. This work was partially funded by National Natural Science Founda- tion of China NNN.620760099,,2220657.. Shanghai Rising-Star Program (23QA1400200),
Xiao Huang. Jingyuan Zhang, Dingcheng Li, and Ping Li. 2019. Knowledge graph embedding based question answering. In Proceedings of the Twelfih ACM International Conference Web Search and Data Mining, WSDM 19, page 105-113, New York. NY, USA. Association for Computing Machinery.
and Natural Science Foundation of Shanghai (23ZRI 1403500).
Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, Dimitris Kontokostas, Pablo Mendes, Sebastian Hellmann, Mohamed Morsey, Patrick Van Kleef,

---

Sören Auer, et al. 2015. Dbpedia-a large- scale, multilingual knowledge base extracted from wikipedia. Semantic web, 6(2):167-195.
on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 0448-9457. PMIR.
Zhao Li, Xin Liu, Xin Wang, Pengkai Liu, and Yuxin Shen. 2023. Transo: knowledge- driven representation learning method with ontology information constraints. World Wide Web, 26(1):23 (6(7) 319.
Rakshit Trivedi, Hamjun Dai, Yichen Wang. and Le Song. 2017. Know-evolve: Deep temporal reasoning for dynamic knowledge graphs. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 1462-3471. PMLR.
Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning entity and relation embeddings for knowledge graph completion. In
Théo Trouillon, Johannes Welbl, Sebastian Riedel, Eric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction. In Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pages 2071-2080, New York, New York, USA. PMLR.
Proceedings of the AAAI conference on artificial intelligence, volume 29.
Xiyang Liu, Huobin Tan, Qinghong Chen, and Guangyan Lin. 2021. Ragat: Relation aware graph attention network for knowledge graph completion. IEEE Access, 9:20840-20844
Xin Lv, Lei Hou, Juanzi Li, and Zhiyuan Liu. 2018. Differentiating concepts and instances for knowledge graph embedding. Proceedings of the 2018 Conference on Empirical Methods in Natural
Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, and Partha Talukdar. 2020. Composition based multi- relational graph convolutional networks. Interna- rional Conference on Learning Representations.
Lunguage Processing, pages 1971-1979, Brussels, Belgium. Association for Computational Linguistics.
Denny Vrandečié. 2012. Wikidata: A new platform for collaborative data collection, In Proceedings of the 21 International Conference World Wide Web, WWW '12 Companion, page 1063-1064, New York, NY, USA. Association for Computing Machinery.
Farzaneh Mahdisoltani, Joanna Biega, and Fabian Suchanek. 2014. Yago3: A knowledge base from multilingual wikipedias. In 7th biennial conference on innovative data systems research. CIDR Conference.
Hongwei Wang, Fuzheng Zhang, Jialin Wang, Miao Zhao, Wenjie Li, Xing Xie, and Minyi Guo. 2018. Ripplenet: Propagating user preferences on the knowledge graph for recommender systems. In Pro- ceedings of the 27th ACM International Conference on Information and Knowledge Management, CIKM '18, page 417-426, New York, NY, USA. Association for Computing Machinery.
Sijie Mai, Shuangjia Zheng, Yuedong Yang, and Haifeng Hu. 2021. Communicative message passing for inductive relation reasoning. Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 4294-4302.
Maximilian Nickel, Lorenzo Rosasco, and Tomaso Poggio. 2016. Holographic embeddings of knowledge graphs. In Proceedings of the AAAI Conference Artificial Intelligence, volume 30.
Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the AAAI conference on artificial intelligence, volume 28.
Chao Ren, Le Zhang, Lintao Fang, Tong Xu, Zhefeng Wang, Senchao Yuan, and Enhong Chen. 2021. Ontological concept structure aware knowledge transfer for inductive knowledge graph embedding. 2021 International Joint Conference on Neural Networks (IJCNN,, pages 1-8.
Ruobing Xie, Zhiyuan Liu, and Maosong Sun. 2016. Representation learning of knowledge graphs with hierarchical types. Proceedings of the Twenry- Fifthh International Joint Conference on Artificial Intelligence, IJCAI' page 2965-2971. AAAI Press.
Michael Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In The Semantic Web, pages 593-607, Cham. Springer International Publishing.
Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. 2018 Representation learning on graphs with jumping knowledge networks. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 5453-5462. PMLR.
Zhiqing Sun, Zhi-Hong Deng, Jian-Yte Yun Nic, and Jian Tang. 2019. Rotate: Knowledge graph embedding by relational rotation in complex space. In International Conference Learning Representations.
Xiaohan Xu, Peng Zhang, Yongquan He, Chengpeng Chao, and Chaoyang Yan. 2022. Subgraph neighboring relations infomax for inductive link prediction on knowledge graphs.
Komal Teru, Etienne Demis, and Will Hamilton. 2020. Inductive relation prediction by subgraph reasoning. In Proceedings of the 37th International Conference

---

Bishan Yang, Wen Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2013. Embedding entities and relations for learning and inference in knowledge bases.
Information:
B is C's father.
A is the grandfather
Question:
is B'ss father.
Possible answers to '_" 'inccude:
Zhilin Yang, William W. Cohen, and Ruslan Salakhut- dinov. 2016. Revisiting semi-supervised learning with graph embeddings. In Proceedings of 33rd International Conference on International Conference Machine Learning Volume 48, ICML''1 16, page 40-48. JMLR.
B
Answer:
A; C:
Provide the top of correctness from possible answers. Please answer in the format of the example above:
descending based likelihood
Muhan Zhang and Yixin Chen. 2018. Link prediction based on graph neural networks. In Advances in Neurul Information Processing Systems, volume 31. Curran Associates, Inc.
Information:
<Informations
Question:
<Question>>
Poesible
'include:
<Possible answerz
Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu. 2019. ERNIE: Enhanced language representation with informative entities. In Proceedings the 57th Annual Meeting of the Association for Computational Linguisties, pages 1441-1451, Florence, Italy. Association for Computational Linguistics.
Informatioo::
B is the mother of A.
is the grandmother A.
Question:
C is the mother of"
Possible answers to 'include: B:C
Answer: B: A:
A Appendix
Provide the top of correctness from possible answers. Please answer in the format of the example above:
descending order based likelihood
A.1 Dataset Generation
Information:
To generate the train set, randomly select triple from the whole entity triples and treat it as the initial train graph, and keep adding new edges and nodes to the graph by the edges connected to the train graph. To ensure that the entities in the test set do not appear in the train set, we first remove the train set from the entire graph and then use the same approach to generate the test set, while requiring that the relations in the test set must appear in the train set. Finally, we generate the valid set using the same way addition, ensure the inductive setting, all entities in the valid do not uppear in the test set. The triples the train set, valid set, and test set do not intersect each other.
cInformations
Question:
<Question>>
Possible
"include:
<Possible answer>
Table 6: Prompt template for predicting the head entity
(upper panel) and tail entity (lower panel) in triples.
A.3 Transductive Prediction
generated the train set, and in
FB15K-237 is the entity KG benchmark dataset widely used in many recent works, and GralL has proposed four versions of variant datasets based on it. We used some of these variant datasets to compare our approach with several baseline methods. Since FB15K-233 contains only entity triples, we supplemented the type information for the dataset used based on WikiData, and the statistics of the supplemented dataset are shown in tab. 7. In addition, due to the lack of carefully built ontology triples, method uses the version without the ontology training module (w/o OT) for transductive link prediction. The results are shown in tab. As can be seen from the results, our method outperforms the baseline methods even for transductive link prediction.
A.2 Prompt details
Our prompt includes one-shot example, task instruction, relevant information, questions and candidate entities. The prompt templates are shown in tab.
Where one-shot example is used to standardize the format of the output. <Information> indicates the neighbor information and type information of the test triples, <Question>> is the link prediction for the test triples, and <Possible answer> denotes the candidate entities.

---

Dataset
FEntitie
#Relations
#Triples
#Concepts
#Type links
YAG021K-610
D145K-165
Method
train
1093
142
1993
218
1313
MRR1
Hivve1
Hit
His8:11
Hie@100
valid
66
165
JOIE
0.243
0.15V
0.383
0.279
0.231
0.315
lest
301
68
205
153
434
Ours
0.278
0.185
0.454
0,418
0.331
0.392
Iruin
1660
172
4145
304
2002
Table 9: Results of link prediction the ontology
test
562
107
478
236
737
graph.
tnain
2501
183
7406
396
3151
1282
test
981
128
865
330
1312
graph while achieving the prediction of entities. We compared our method with the jointly trained baseline method (Hao et al., 2019) and the results are shown in tab. 9. It can be seen that even though we only use the simplest model to train ontology triples, our method still outperforms the statc-of- the-art baseline method, where JOIE builds the
Table 7: Statistics on the FB 15k-237 dataset after adding type information.
Method
v2
MRR1
His@11
MRR1
Hiis@1t
MRR1
Hiise
GralL
0.499
0.615
0.499
0.635
0.529
SNRI
0.490
0.381
0.584
0.461
0.618
0.494
TACT
0.475
0.378
0.565
0.451
0.585
0.457
hierarchy of ontology in detail and uses more advanced training model.
Table 8: Results of transductive link prediction the FB FB15k-237 dataset.
A.4 The Impact of Neighboring Nodes
We studied the effect of different enclosing sub- graph sizes on the prediction during the test, and the experimental results are shown in fig. 5. It can be seen that the smaller the enclosing subgraph extracted for the test triple, the worse its prediction performance is without changing any other conditions. The direct result of smaller enclosing subgraph that the target node has fewer neighboring nodes available. Therefore, we argue that too few neighbor nodes will impair the semantic information captured by the model from the structure and thus lead to decrease in prediction performance.
A.5 Ontology Prediction
As kind of knowledge graph, the ontology graph also suffers from the problem of incompleteness. Like other jointly trained methods, our proposed method can conveniently complete the ontology
YAGOZIK-610
DB45K-165
1.0
3802
01
0.1
Mingi
Metrias
0.5
02
0.2
Hop
Hop
Prediction performance using enclosing subgraphs of different sizes.
Figure 5:


Reviews:
[{"review_id": "8FYKRh7c5u", "reviewer": "Reviewer_TJtS", "summary": "", "questions": "1. This method achieves impressive results, but is the introduction of ontology information inconsistent with the nature of \"unseen nodes\" (especially in the test set)? It would be better to adding the result of the testing set without the type information.\n\n2. The paper does not elaborate the details of ChatGPT in the paper. Does the prompt contain neighbor information and ontology information? Is it zero-shot or few-shot?\n", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed comments and taking the time to review our work. We will address your concerns point by point.\n\nQuestion regarding the relationship between ontology information and the nature of \"unseen nodes\" (Q1):\n\nWe would like to clarify that the introduction of ontology information is not contradictory to the nature of \"unseen nodes.\" This is because in a knowledge graph, \"unseen nodes\" refer to entities that have not appeared during the training process. This \"unseen\" stems from the unknown of the entity itself and does not imply the absence of all information related to the entity. Even unknown entities can still possess ontology information. For example, in the triplet (J.Butler, lives in, Miami), even though we might not have encountered \"J.Butler\" before, we can still understand that he is a \"Person.\" This ontology information aids us in inferring that the relationship between him and \"Miami\" is more likely to be \"lives in\" rather than other relationships like \"is city of\". In fact, as we mentioned in the ablation study (lines 521-526), the introduction of ontology information significantly improves the effectiveness of inductive link prediction compared to results without type information. We hope this helps address your question.\n\nQuestion regarding the details of ChatGPT (Q2): \n\nWe agree with your perspective and would like to clarify the related details here. Our prompts include a one-shot example, task instruction, relevant information, question, and candidate entities. The one-shot example is used to standardize the format of the ChatGPT results. The task instruction is to rank the candidate entities based on the likelihood of being correct. The relevant information provides ontology and type information consistent with our method. The question pertains to the link prediction of the test triple, and the candidate entities are consistent with other methods. We deeply apologize for any confusion this may have caused and will modify the paper to supplement details related to ChatGPT.\n\nThank you very much for taking the time to review our paper. We greatly appreciate your feedback and will carefully consider all of your suggestions when revising the manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper proposes a novel and efficient approach that incorporates ontology information to improve the inductive inference of knowledge graphs. \n\n2. The incorporation of the type-constrained regular loss is interesting, as it enhances the model's robustness.\n\n3. The author improved the two datasets through a strict processing process, which is conducive to the experimental analysis of follow-up work.\n\n4. The authors conduct a comprehensive experimental analysis. \n", "reasons_to_reject": "1. As mentioned in Limitation Section, some modules of the approach can be replaced with more powerful alternatives, like initializing embedding of the node by BERT.\n\n2. The experimental setting for comparing methods is unclear, and hyperparameter settings on different datasets should not be directly copied.\n", "questions_to_authors": "1. This method achieves impressive results, but is the introduction of ontology information inconsistent with the nature of \"unseen nodes\" (especially in the test set)? It would be better to adding the result of the testing set without the type information.\n\n2. The paper does not elaborate the details of ChatGPT in the paper. Does the prompt contain neighbor information and ontology information? Is it zero-shot or few-shot?\n", "paper_topic_and_main_contributions": "The paper addresses the crucial problem of inductive relation inference in knowledge graphs, specifically focusing on the sparsity of neighborhood information for new entities. The authors propose an innovative method that leverages ontology information to enhance the inductive inference of knowledge graphs.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "RdUp4u4okf", "reviewer": "Reviewer_pao7", "summary": "", "questions": "Question A: What is mt in Equations 16 and 17? I could not find its definition in the text. Also, I think this is similar to enforcing domain and range semantics of relationships. If so, how do you handle subclass of a class (that is the domain for example for the relationship) appearing as the type of an entity? Because then you have enumerate all the possibilities to not make learning algorithm penalize because exact string does not match for the triple you mention for example in Equation 16?\n\nQuestion B: Could you elaborate on the point I mentioned earlier that use of schema/ontology for learning KG embeddings or link predictions is not new. Authors have mentioned some early works in this area. However, I am not sure by mentioning ‘inductive relation inference’ makes the paper’s contributions stand out. Could you express your thoughts?\n", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your thorough review of our work and the insightful questions you raised. We would like to provide detailed responses to the questions you have raised as follows.\n\nRegarding the response to Question A:\n\nWe agree with the question you pointed out. As mentioned in our approach (lines 329-331), (uc, mt, vc) represents the ontology triple, where uc and vc denote concepts, and mt denotes the meta-relation between concepts. In fact, this is different from enforcing domain and range semantics of relationships. In our paper, this part of the work aims to learn the semantic relationships (i.e., meta-relation) between concepts from ontology triplets to reinforce the semantic information within concepts. These meta-relations include hierarchical relationships between concepts (such as \"is a\" in (artist, is a, person) and \"subclass of\" in (movie, subclass of, art)) and other meta-relations (such as \"at location\" in (person, at location, city)). When an entity has multiple concepts and hierarchical relationships might exist between these concepts (e.g., \"Einstein\" has types \"scientist\" and \"person\"), another part of our work captures their different importance levels through the attention mechanism presented in Equations 2 and 3. We will revise the paper to clarify any confusion here and enhance the overall quality of the paper.\n\nRegarding the response to Question B:\n\nWe understand your concerns regarding the contribution of our method and its distinction from other link prediction approaches that utilize ontology. In the introduction of our paper (lines 48-60), we mentioned that the real world represents an open-domain problem. Transductive link prediction requires costly retraining to handle the ever-growing new data, making this expense impractical as data volume increases. In contrast, inductive link prediction holds great value for knowledge graph completion due to its generalization capabilities. Furthermore, we note that the neighbor information of new entities in inductive inference is typically very sparse (lines 66-70). The introduction of ontology information helps address the issue of information scarcity in inductive relation inference, leading to significant improvements in predictive performance. While existing works have utilized ontology information for KG embedding learning, as we pointed out in related works (lines 155-159), previous works only focused on transductive link prediction. Our work stands out by significantly enhancing inductive relational inference by introducing ontology information. We will revise the paper to emphasize the focus and contributions of our proposed method. We hope this addresses your concerns.\n\nThank you very much for taking the time to review our paper. For any parts of the text that may have caused confusion, we apologize and will work diligently to address these issues for improved clarity and readability of the paper. \n\nOnce again, we appreciate your thoughtful comments.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "•\tProposes a simple approach to incorporate ontology semantics for inductive link prediction.\n\n•\tShows positive results over two benchmark datasets.\n\n", "reasons_to_reject": "•\tUsing ontology information is not entirely exciting for KG link prediction tasks. Approach also does not propose any new technique other than the fusion of ontology semantics for the specific problem of inductive link prediction.\n\n•\tI think focus and contribution of the paper is on a narrow topic.\n", "questions_to_authors": "Question A: What is mt in Equations 16 and 17? I could not find its definition in the text. Also, I think this is similar to enforcing domain and range semantics of relationships. If so, how do you handle subclass of a class (that is the domain for example for the relationship) appearing as the type of an entity? Because then you have enumerate all the possibilities to not make learning algorithm penalize because exact string does not match for the triple you mention for example in Equation 16?\n\nQuestion B: Could you elaborate on the point I mentioned earlier that use of schema/ontology for learning KG embeddings or link predictions is not new. Authors have mentioned some early works in this area. However, I am not sure by mentioning ‘inductive relation inference’ makes the paper’s contributions stand out. Could you express your thoughts?\n", "paper_topic_and_main_contributions": "The paper presents a KG link prediction approach that works for unknown entities (i.e., entities that were not there when a learning algorithm is trained). To enable ‘inductive learning’, authors propose to use both graph structure and ontology semantics in the learning process. For example, they infuse both type and positional information in learning node embeddings in the graph. Though it is simple, I like the idea of transforming entity embedding into ontology space and then enforce the similarity through an additional loss. The approach has 3 loss components; first is the normal entity space, margin-based loss and the other two are also margin-based but involves ontology space. However, use of ontology/schema information for KG embedding learning is not new (e.g., [1]). Using it for this specific problem with a pipeline of techniques can be also considered as an incremental improvement.\n\n[1] Peng, Miao, Ben Liu, Qianqian Xie, Wenjie Xu, Hua Wang, and Min Peng. \"SMiLE: Schema-augmented Multi-level Contrastive Learning for Knowledge Graph Link Prediction.\" arXiv preprint arXiv:2210.04870 (2022).\n", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "X5pguevWor", "reviewer": "Reviewer_6Xgf", "summary": "", "questions": "- In the ablation study, the difference between the proposed method with or without ontological information is not much different, the differences are only marginal. The reason should be explained. Does it show that ontological information does not help much?\n- Does the inductive relational inference gets better when using ontological information?\n- Why new datasets are introduced? Can the ontological and type information be induced in the existing datasets?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your suggestions. We agree with the significance and value of the related work you mentioned in inductive link prediction. Since in the same dataset (such as FB15K-237), reference [1] significantly outperforms them on the same metrics, and our paper primarily considers comparing with several state-of-the-art baseline methods, we did not compare with these works.\n\nWe appreciate you bringing up this idea. We will incorporate discussions about these works in our revised paper to address this concern.\n\n[1] Inductive Relation Prediction by Subgraph Reasoning\n\nQuestion regarding the role of ontology information (Q1&Q2):\n\nWe would like to clarify that the use of ontology information significantly enhances the performance of inductive relation inference. In the results analysis of our paper (lines 458-496), we extensively examine the impact of ontology information on inductive relation reasoning. The experimental results show that the introduction of ontology information significantly enhances prediction performance, and this enhancement becomes more pronounced with fewer neighbor information. Moreover, ontology information is distinct from ontology training. Ontology information refers to the semantic information contained within the ontology, while ontology training is a module in our approach. As pointed out in our ablation study, the introduction of ontology information in our method includes three modules: type information, ontology training, and type constraint (lines 521-535). Specifically, the largest contribution comes from type information, and the absence of it severely impacts the performance. Although the performance drops from removing ontology training or type constraints alone is not as significant as removing type information, we note that removing either one or both leads to performance degradation (with more pronounced degradation when both are removed simultaneously). Thus, each module contributes to performance enhancement, varying only in the degree of contribution. We hope this addresses your concerns.\n\nQuestion regarding the introduction of a new dataset (Q3):\n\nYour question is very relevant. In response, we need to suggest that while type information can be induced from existing datasets using WikiData, ontology information still requires expensive manual construction by experts. The new dataset, being a KG benchmark dataset with ontology information, is more suitable in terms of data quality for our research. Although we have not explored the construction of ontology information for existing datasets, we acknowledge the potential value of such research. Therefore, we provide the existing dataset with type information in the supplementary material to facilitate the community in constructing its ontology information. We apologize for any confusion this might have caused.\n\nFor any deficiencies or confusing language in the text or figures, we sincerely apologize. We will strive to address these issues to enhance the clarity and readability of the paper.\n\nOnce again, we appreciate your valuable feedback, and we will carefully consider all your suggestions when revising the manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The proposed approach based on ontology and the type of information is very interesting. \n- The results are outperforming the algorithms reported in the paper but some of the related work is still missing.\n- The paper is written very clearly.", "reasons_to_reject": "- Missing related work which actually creates doubt if the approach is really effective in its experimental and theoretical comparison.", "questions_to_authors": "- In the ablation study, the difference between the proposed method with or without ontological information is not much different, the differences are only marginal. The reason should be explained. Does it show that ontological information does not help much?\n- Does the inductive relational inference gets better when using ontological information?\n- Why new datasets are introduced? Can the ontological and type information be induced in the existing datasets?", "paper_topic_and_main_contributions": "The paper proposes a method for inductive inference on knowledge graphs. It not only considers unseen entities but also relations which is an under-explored area of study and very interesting. The proposed method considers ontological and type information for inductive link prediction and type prediction which is very interesting.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "https://arxiv.org/abs/2211.11407 \\\nhttps://arxiv.org/pdf/2010.03496.pdf \\\nhttps://arxiv.org/pdf/2107.04894.pdf", "Typos_Grammar_Style_and_Presentation_Improvements": "Figure 1 could be made a bit larger."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a methodology for a specific type of link prediction tasks, namely link prediction on graphs consisting of unseen entities (but with seen relations). The problem may have some relevance to data mining and knowledge management communities, whereas the significance for EMNLP is not clear (the only, weak link mentioned in the intro is that KGs are an ingredient in NLP tasks).

Reviewers found the paper generally sound, though excitement was limited. In particular, the problem was considered narrow, and several related works were not discussed. It appears the paper might be better suited for a data mining or knowledge management venue.

Further to the reviewer comments, I see three issues with this paper:
 - The reported problem appears strange - it is true that unseen entities are a problem in link prediction, but the formulation in this paper, of predicting on a completely disjoint graph, seems to misunderstand the problem: In the domain that the authors investigate (Wikidata-style knowledge), common general-world entities like countries, big cities, big companies definitely do re-occur between train and test sets. The challenge, instead, are organic entities that are only occuring a few times, but an (arbitrary?) hard split of the dense organic KG does not shed light on them.
 - The datasets are insufficiently explained, and seem extremely cherrypicked. There is no discussion of relations and instances to be predicted, and how the artificial splits were generated. MRR values around 0.8 represent outstanding ranking scores, and make me believe that the datasets consist of very strange instances, and are not indicative of realistic use cases. Instead of ranking metrics, it would be more helpful to report absolute or relative recall gains at a certain precision, e.g., how many, or what percentage of triples can the approach add to the KG, while maintaining 80% precision?
 - In addition to the strong absolute results (MRR), the relative gains (+15/+44% over SoA) are also at a level that is not easy to believe (extraordinary results require extraordinary evidence). The discussion provides zero case studies, and the metrics also aggregate across (unexplained) relations, thereby making it impossible to check them for plausibility.
Decision: Accept-Findings