LATENTLOGIC: Learning Logic Rules in Latent Space over Knowledge Graphs
Authors: Junnan Liu
Qianren Mao
Chenghua Lin
Yangqiu Song
Jianxin Li
Venue: EMNLP 2023 Findings (2023)
Keywords: Knowledge Graph
Reasoning
Logic Rule

Abstract: Learning logic rules for knowledge graph reasoning is essential as such rules provide interpretable explanations for reasoning and can be generalized to different domains. 
However, existing methods often face challenges such as searching in a vast search space (e.g., enumeration of relational paths or multiplication of high-dimensional matrices) and inefficient optimization (e.g., techniques
based on reinforcement learning or EM algorithm).
To address these limitations, this paper proposes a novel framework called LatentLogic to efficiently mine logic rules by controllable generation in the latent space.
Specifically, to map the discrete relational paths into the latent space, we leverage a pre-trained VAE and employ a discriminator to establish an energy-based distribution. 
Additionally, we incorporate a sampler based on ordinary differential equations, enabling the efficient generation of logic rules in our approach.
Extensive experiments on benchmark datasets demonstrate the effectiveness and efficiency of our proposed method.

LATENTLOGIC: Learning Logic Rules in Latent Space over Knowledge Graphs
Junnan Liu', Qianren Mao", Chenghua Lin', Yangqiu Song', Jianxin Li 12,
"School of Computer Science and Engineering, Beihang University, Beijing, P.R.China. "Zhongguancun Laboratory, Beijing, P.R.China.
Depaarteet of Computer Science, University of Manchester, U.K.
'Department of Computer Science and Engineering, HKUST, Hong Kong SAR, China. liujneact. buaa.edu.cn, (maoqr, 11 jijxxlzgcl.b.... edu.cn, chenghua. Senn@manchester.eee ae......kkk yqsongêese .ut.seesee.st.hk
Abstract
of such rule is VX,Y, nationality(X, classmate X, 2) nationality( (Z, Y), indicat- ing that Z is the classmate of X and has a nation- ality of Y, then is likely to have a nationality of Y. This rule can be applied to deduce the na- tionalities of new individuals. Compared to other methods such as knowledge graph embedding ap- proaches (Bordes et al., 2013; Sun al., 2019; Li et al., 2022), the rule-based method (Zhang et al., 2020) more interpretable and can be applied inductive scenarios (Teru and Hamilton, 2019).
Learning logic rules for knowledge graph rea- soning essential as such rules provide inter- pretable explanations for reasoning and can be generalized to different domains. However, ex- isting methods often face challenges such as searching in a vast search space (e.g., enumera- tion of relational paths or multiplication high- dimensional matrices) and inefficient optimiza- tion (e.g., techniques based on reinforcement learning EM algorithm). To address these limitations, this paper proposes novel frame- work called LATENTLOGIE to efficiently mine logic rules by controllable generation in the la- tent space. Specifically, to map the discrete re- lational paths into the latent space, we leverage pre-trare pre-trained VAE and employ discriminator to establish an energy-based distribution. Ad- ditionally, we incorporate sampler based on ordinary differential equations, enabling the ef- ficient generation of logic rules in our approach. Extensive experiments on benchmark datasets demonstrate the effectiveness and efficiency of our proposed method.
Most rule-based methods involve enumerating relational paths as candidate rules, followed by assigning weights to each rule to indicate their quality (Lao and Cohen, 2010; Richardson and Domingos, 2006; Yang et al., 2017; Sadeghian et al., 2019). When the scale of the knowledge graph expands, these methods face the challenge of exponentially growing search space. To overcome this problem, RNNLogic (Qu al., 2021) intro- duces rule generator and a reasoning predictor to separate rule generation from rule weight learning. However, the optimization process based on the Expectation Mititttiion-Maiizzzttiin (EM) algorithm tends to have slow convergence, leading to extended train- ing periods. Another line of research utilizes rein- forcement learning (RL) to search for logic rules by making sequential decisions (Xiong et al., 2017b; Lin al., 201 8; Das et al., 2018; Lu et al., 2022). Nevertheless, RL-based methods often encounter challenges such as large action spaces and sparse rewards during training. As result, efficiently min- ing high-quality logic rules for knowledge graph reasoning remains challenging task.
1 Introduction
Knowledge graphs usually contain collections of real-world facts encoded in triplets with entity and relation information, and find broad applications in across multiple domains (Lukovnikov et al., 2017; Xiong et al., 20;7a; Wang et al., 2018; Zhang et al., 2019; Huang al., 2019; Tang al., 2023a,b). Despite some knowledge graphs holding hundreds of millions of triples, they still suffer from incom- pleteness, whereby many valid triples are missing since it is impractical to identify them all manu- ally. Therefore, a fundamental and essential task in knowledge graphs is to utilize existing facts to predict the missing ones.
In this paper, we propose novel framework named LATENTLOGIC, which overcomes the aforementioned challenges. Our approach bypasses the enumeration of relational paths by employing controllable sampling in latent space. Furthermore, each component LATENTLOGIC trained an end-to-end fashion, avoiding the indirect and
Recent studics have focused on learning logic rules from knowledge graphs and utilizing these learned rules predict absent facts. An example
Jianxin Li is the corresponding author.

---

inefficient optimization procedure. Concretely, we utilize VAE-based autoencoder (Kingma and Welling, 2014; Li et al., 2020) to map discrete re- lational paths into a -ow-dimensional latent space. We employ a discriminator to measure the semantic coherence between the latent vector and the rule head, thereby creating an energy-based distribution in the latent space. To obtain latent vector sam- ples that correspond to the desired rule head, employ a sampler based on ordinary differential equations (Song et al., 2021; Nie et al., 2021). The latent vectors are used to generate rule bodics for the given rule head using the VAE generator. Extensive experiments demonstrated the com- putational efficiency of LATENTLOGIC in com- parison to previous rule learning methods (Yang et al., 2017; Sadeghian et al., 2019; Qu et al., 2021), indicating that LATENTLOGIC exhibits en- hanced scalability for mining logic rules on larger- scale knowledge graphs. Furthermore, through experiments on two commonly used benchmark datasets, FB 5k-237 (Toutanova and Chen, 2015) and WN18RR (Dettmers et al., 2018), we observed that LATENTLOGIC successfully generates high- quality logic rules for knowledge graph reasoning and evidently outperform the salient baseline meth- ods.
aim, we incorporate a relational path autoencoder and a rule discriminator that operates in the latent space, As shown Fig. 1, we firstly utilize VAE- based Relational Path Autoencoder, which consists of an encoder & and a decoder D, to compress re- lational paths into a low-dimensional latent space. To perform controllable generation of rule bodies, need to construct joint distribution of latent vectors and rule heads for sampling. Therefore, we introduce a Rule Discriminator on the latent space to form the joint distribution represented by the energy-based model. This joint distribution allows us to obtain latent vectors associated with the desired rule head by using a sampler based on an ordinary differential equation. Finally, we can decode the latent vectors into rule bodies using the VAE decoder, allowing us to generate rules given certain rule heads.
2.1 Relational Path Autoencoder
Note that each rule body ri A-A---Ar can be consid- ered a sequence of relations [rı..... : rt]. Such se- quences of relations can be effectively modeled by sequence neural networks (Das et al., 2017; Kotnis et al., 2021; :; Liu et al., 2022), and thus we introduce RNN (Hochreiter and Schmidhuber, 1997) to pa- rameterize the relational path autoencoder. Specifi- cally, we map relational path P into latent vector using an RNN-based encoder ((2lP), and an RNN-based decoder p(P|z, q) that maps into the relational path q is a unified query embedding for all inputs. Our decoder does not use an au- toregressive approach. Instead, takes the unified query embedding q and positional embedding as input, simultaneously generating relational paths. We optimize the encoder and decoder parameters for each input relational path P with the objective:
2 Framework
Problem Definition First, we introduce some definitions and notations. A knowledge graph g (V,TR)) is usually defined by triple set T r(,, S V x R x V. where V de- notes an entity set, and R represents a relation set. In this paper, we aim to learn logic rules in the conjunctive form v(Xi)<= ri (Xo.Xi) A ... ....A A rX(, X() from the given knowledge graph. A logic rule, which can be ab- breviated as r f- A ... A rı, consists of rule head, denoted as r, and rule body (relational path), represented ri A... rj.
r(Xo, Xi) +
LvAR(P) = pl pPpl, q)] + KL (c(=)P))I\V I)),
(1)
where KL(-II-) is the Kullack-Leibler divergence that pushes g to be close to the prior N/(0,) T).
Framework Overview Our approach converts rule learning problems to controllable generation problems by developing a generative model, de- noted as po (Pls), to generate rules given a specific rule head Here, P represents the rule body, i.c., relational path. P (r1, r2,... rr). The main innovation behind our approach is that we substi- tute the enumeration of the relational path with the sampling of latent vectors, significantly reducing training overhead. Additionally, to accomplish our
2.2 Rule Diseriminator
Now we aim to model the joint distribution p(z, s), where z denotes the latent vector of relational path and represents a desired rule head. This joint distribution can be represented as p(z,
Pprior (2) [s|=)) where Pprior (z) is the prior distri- bution, i.e., standard Gaussian distribution, and p(s) 2) is conditional distribution on s given z. In- spired by Nie et al. (2021), we define p(s[2) (2) as an

---

Path AutoEncoder
Path Space
Latent Space
Path AutoEncoder
T Arg
Sample Data
T
Rule Grounding
76
P3
76
D
Solve Neural ODE
by Sample Data
Rule
Rule Discriminator
Training
Rule Generation
Reasoning
Figure! Framework overview of LATENTLOGIC. We first train the rule discriminator and relational path autoencoder. Then we generate logic rules by sampling the latent space. The generated rules can be used to answer queries over knowledge graphs.
energy-based model (EBM) (LeCun al., 2006) follows:
paths obtain training data. Each subsequent node is generated using the following distribution:
p(s|z) Eo(s z) Es(s|z) = --ff((=))) f(g(2))[s) const,
1
(015) ET,
(2)
p(a:2 1)
W(
0,
otherwise,
where const = log E. exp( is nor- malization term corresponding to all rule heads, g()) is the fixed VAE decoder that decodes into P. and f indicates neural network that takes g(z) and as input and produces score that measures how well s is carried in z. The function can be any advanced model if it measures the consis- tency between relational paths and the rule heads (e.g., other rule mining models like neural logi- cal programming). We adopt simple network as suggested by Das et al. (2017), which encodes the relational path using the recurrent neural network, computing the score using the similarity between path representation and relation representation. For each training sample (7h, (ro...., r:)). Suppose ý denotes the output logits of the rule discriminator; the objective can be expressed as:
(4)
where N (1x) denotes all the neighhorhoods of en- tity 2z. Then we optimize the auto-encoder by minimizing the Eq. 1. For the rule discriminator, we employ a similar sampling strategy. Each time we sample the next entity I₁, we include relation The which directly connects To and Tj, to create a training sample (Th: (ro....,rii) r:)). Suppose ý de- notes the output logits of the rule discriminator, the objective can be expressed as follows:
[
Discrimmintorr
Σ
ΣΣem
y log Ür,
(5)
where represents all the training samples and y a one-hot vector that only the r,-th position is 1.
Latent Vector Sampling Given the joint distri- bution p(z, 8), we would like to draw samples z conditioned on the target rule head s, which are then fed to the VAE decoder to obtain the desired rule bodies. According Song al. (2021), sam- pling from an EBM can be achieved by solving a specific ordinary differential equation (ODE). our work, the ODE in the latent space can be ex- pressed as: dz ative time increments from T to 0. To generate latent vectors based on the given rule head we first draw 2 x(T) from (O I), and then apply neural ODE solver' (Chen et al., 2018. 2021)
R yrth log Dn.
Discrimmintor
Σ
(3)
5
where S represents all the training samples and nh is a one-hot vector that only the rh-th position 1.
2.3 Rule Generation
(s)gg
with neg-
Model Training We train both the relational path autoencoder and the rule discriminator on the given knowledge graph g. For the relational path auto- encoder, we adopt a random walk (Spitzer, 1975)- based procedure to efficiently sample relational
hhtpps://gis thub. httpttttc..omotttccce///orrcccio

---

FB15k-237
WNISRR
Methods
MRR
Hits@I
Hits@3
Hits@ 10
MRR
Hits
Hits@3
Hits@ 10
TransE
0.294
46.5
0.226
50.1
DistMult
0.241
15.5
26.3
41.9
0,430
39.0
44.0
49.0
TuckER
0.358
26.6
39.4
54.4
0,470
44.3
48.2
52.6
RotatE
0.338
24.1
37.5
53.3
0.476
42.8
49.2
57.1
PathRank
0.087
7.4
0.2
11.2
0.189
17.1
20.0
22.5
NeuraLLP
0.237
25.9
36.1
0.381
36.8
38.6
40.8
DRUM
0.238
17.4
26.1
36.4
0.382
36,9
38.8
41.0
RNNLogic
0.288
20.8
31.5
44.5
0.455
41.4
47.5
53.1
RLogic 0.312
20.3
50.1
0.473
44.3
53.7
LATENTLOGIC
0.320
21.2
32,9
51.4
0.481
45.2
49.7
55.3
Table 1: Knowledge graph reasoning performance on FB15k-237 and WNI8RR. The model with means the results from Qu al. (2021), and means the results copied from Cheng al. (2022). For RNNLogic, choose the variant w/o emb. for fair comparison.
obtain = (0).
ing facts from logical rules. For every triplet (h, r, t), two queries are created: (h, r) and (?,r; (), using and h as answers, respectively. To maintain consistency with previous studies, Mean Rank (MR), Mean Reciprocal Rank (MRR), and hit@k are selected as evaluation metrics under the filtered setting (Sun et al., 2019). Furthermore, mitigate the effects of random sampling, evalu- the model performance five different random seeds, and report the average performance.
Rule Generation We draw samples {zr(T)) from I) for each rule head and obtain n latent vectors {0) Then, we feed {=)(0)} to pretrained VAE decoder to generate corresponding rule bodies. Finally, calculate the confidence scores of the generated rules using the aforementioned rule discriminator.
3 Experimental Setup
Datasets We conduct experiments used knowledge graph reasoning benchmark datasets: FB15k-237 (Toutanova and Chen, 2015) and WNISRR (Dettmers et al., 2018). WNISRR comprises of 40,943 entities and 11 relations, with 86k/3k/3k instances set aside for train- ing/validatiionntesting correspondingly. FB15k- 237 comprises 14,541 entities and 237 relations and has 272k/17k/20k instances reserved for train- ing/validationn/esiing. respectively.
widely
Overall Performance As shown in Tab. 1, we present the experimental results on the FB15k- 237 and WNI8RR datasets. Firstly, we compare LATENTLOGIC with rule-based methods and ob- serve that LATENTLOGIC outperforms the statis- tical learning method PathRank, neural differen- tiable methods NeuralLP and DRUM, as well recent methods RNNLogic and RLogic. In particu- lar, obtain 2.56% and 2.59% relative increase in MRR and Hits @ 10 on FB FB15k-237 against the state-of-thh-art rule-based method RLogic (Cheng et al., 2022). Similarly, LATENTLOGIC achieves 1.69% and 2.98% increase MRR and Hits 10 on the WNI8RR dataset against RLogic. Then, we also compare LATENTLOGIC against salient knowledge graph embedding based methods and find that LATENTLOGIC yields comparable per- formance embedding-based methods, espe- cially on WNISRR, where outperforms selected embedding- based baselines.
4 Experimental Results
Baselines We compared LATENTLOGIC with taxonomies of methods, including: Knowledge graph embedding methods: TransE (Bordes al.,
2013), DistMult (Yang et al., 2015), TuckER (Bal- azevic et al., 2019), and RotatE (Sun et al., 2019). Rule-based methods: PathRank (Lee et al., 2013), Ne (Yang et al., 2017), DRUM (Sadeghian et al., 2019), RNNLogic (Qu et al.. 2021), and
RLogic (Cheng et al., 2022).
Quality Learned Rules
We assess the quality of the logic rules learned from different models this part. Following the settings of Qu et al. (2021),
We adopt forward chain- ing (Salvat and Mugnier, 1996) for inferring miss-
Evaluation Metrics

---

1aim__anua y) fwwm_aaatoriaaa Dengon_languages((:.))
film_language(s.)) film,prequel(2,,)) Ain fiim_
film_ calnt_county(2,111 www fimmm httpa//ooooiiiccoooooooe/e//oena
11lm_country(x b /ttpp//wwew..aaaaao./ooo///////a
people_
person_ verson.nntiionalityyee v) - - place of birth A cecatiion_contrryie
personn.ationallt((:: 8) organization_ haw.:eerooooodddraaaaa 2) A wyganization_counttre
Table 2: Case study on the FB 15k-237 dataset.
we generate n logic rules with the highest quality score for each query relation and then them for training predictor for knowledge graph rea- soning. As mentioned before, the quality of each rule learned by LATENTLOGIC can be calculated using the rule discriminator. Results for different n values are reported in Fig. 2. Our observations suggest that LATENTLOGIC outperforms the com- pared methods remarkably. Morcover, even with limited number of rules considered per relation, LATENTLOGIC still achieves competitive results.
2013, 2014a,b). Then some methods extend the idea by simultaneously learning logic rules and the weights in differentiable way. For exam- ple, some works (Rocktäschel and Riedel, 2017; Yang et al., 2017; Sadeghian et al., 2019) try to use neural logic programming to model rule-based reasoning and to learn high-quality logical rules. Another kind of rule-learning method is based on reinforcement learning. The basic idea is to train path-finding agent, which is used to search for reasoning paths over knowledge graphs answer queries, and then extract logic rules from reasoning paths. Some representative works include Deep- Path (Xiong et al., 201 7b), MINERVA (Das et al., 2018), M- M-Walk (Shen et al., 2018) and R5 (Lu et al., 2022). Recently, RNNLogic (Qu et al., 2021) solves this problem by training generator and a predictor alternately. RLgoic (Cheng al., 2022) learns rules in recursive manner.
2035
0.30
RM
Lo
220.5
035
0.20
Pab Ranking
RoWLogs
0.29
33
100
20
43
MRR
100
(a) WNISRR
(b) FB FB35k-277
Figure 2: MRR performance w.r.t. n rules on WN18RR and FB15k-237 datasets. LATENTLOGIC achieves prominent results even with only 10 rules.
Conclusion
Learning logic rules for knowledge graph reasoning is crucial, as they offer interpretable explanations for the process of reasoning. We propose a novel framework for learning logic rules in latent space. Concretely, we introduce relational path autoen- coder to map paths into latent space and rulc discriminator to access the consistency between rule bodies and rule heads. With the sampler based on ODE, LATENTLOGIC can generate logic rules efficiently. Experimental results showed that LA- TENTLOGIC outperforms strong baseline methods and is efficient for training.
Case Study To demonstrate that LATENTLOGIC is capable of generating helpful and varied rules for knowledge graph reasoning, we present some the logic rules that LATENTLOGIC produced on the FB 5k- dataset in Tab. 2. It can be observed that the logic rules generated by LoTENTLOGIC are semantically meaningful and diverse.
5 Related Work
Over the past years, learning logical rules over knowledge graphs has been an active research area. Most traditional methods enumerate relational paths between query entities and answer entities candidate logic rules, and further learn a scalar weight for each rule to assess the quality. Some rep- resentative works include Markov Logic Network (MLN) (Kok and Domingos, 2005; Richardson and Domingos, 2006), path ranking (Laa and Cohen, 2010; Lao et al., 2011) 1) and probabilistic personal- ized page rank (ProPPR) algorithms (Wang et al.,
Acknowledgements
We thank the anonymous reviewers for their in- sightful feedback. This work is supported by the National Natural Science Foundation of China (No. U20B2033.. We thank the Beijing Advanced Innovation Center for Big Data and Brain Comput- ing for its computational resources.

---

Limitations
6th International Conference on Learning Represen- tations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net.
In this paper, we introduce generative framework for rule mining on knowledge graphs. We believe this approach still has much room for improvement: 1) We have not used the more complicated model architecture of the rule discriminator. 2) The pro- posed sampling strategy may lead to some same relational paths or out-of-domain relational paths. For future work, finding a way to incorporate a more advanced rule discriminator and prevent gen- erating invalid relational paths is worth exploring.
Rajarshi Das, Arvind Neelakantan, David Belanger, and Andrew McCallum. 2017. Chains of reasoning over entities, relations, and text using recurrent neural net- works. In Proceedings of the 15th Conference of the European Chapter of the Association for Computa- tional Linguistics, EACL. 2017, Valencia, Spain, April 3-7, 2017, Volume Long Papers, pages 132-141. Association for Computational Linguistics.
Tim Dettmers, Pasquale Minervini, Pontus Stenetorp. and Sebastian Riedel. 2018. Convolutional 2d knowl- edge graph embeddings. In Proceedings of the Thirty- Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Arti- ficial Intelligence (IAAI-18), and the 8th AAAI Sym- posium on Educational Advances in Artificial Intel- ligence (EAAI-18) New Orleans, Louisiana, USA, February 2-7, 2018, pages 1-11-. AAAI Press.
References
Ivana Balazevic, Carl Allen, and Timothy M. Hospedales. 2019. Tucker: Tensor factorization for knowledge graph completion. In Proceedings of the 2079 Conference Empirical Methods in Natu- ral Language Processing and the 9th International Joint Conference on Natural Language Processing, EMMLP--JCNNPP 2019, Hong Kong, China, Novem- ber 3-7, 2019, pages 5184-5193, Association for Computational Linguistics.
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Comput., 1785:1735- 1780.
Antoine Bordes, Nicolas Usumier, Alberto García- Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi- relational data. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013, Pro- ceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States, pages 2787-2795.
Xiao Huang, Jingyuan Zhang, Dingeheng Li, and Ping Li. 2019. Knowledge graph embedding based ques- tion answering. In Proceedings of the Twelfth ACM International Conference Web Search and Data Mining, WSDM 2019, Melbourne, VIC, Australia, February 11-15, 2019, pages 105-113. ACM.
Diederik P. Kingma and Max Welling. 2014. Auto- encoding variational bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings.
Ricky T. Q. Chen, Brandon Amos, and Maximilian Nickel. 2021. Learning neural event functions for ordinary differential equations. 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenRe- view.net.
Stanley Kok and Pedro M. Domingos. 2005. Learning the structure of markov logic networks. In Machine Learning, Proceedings of the Twenty-Second Inter- national Conference (ICML 2005), Bonn, Germany, August 7-11, 2005, volume 119 of ACM Interna- tional Conference Proceeding Series, pages 441-445 ACM.
Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. 2018. Neural ordinary differen- tial equations. In Advances in Neural Information Processing Systems 31: Anmual Conference on Neu- ral Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pages 6572-6583
Bhushan Kotnis, Carolin Lawrence, and Mathias Niepert. 2021, Answering complex queries in knowl- edge graphs with bidirectional sequence encoders. In Thirty-Fifth AAAI Conference on Artificial Intel- ligence, AAAI 2021, Thirty- Third Conference on In- novative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educationnll Ad- vances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pages 4968-4977. AAAI Press.
Kewei Cheng, Jiahao Liu, Wei Wang, and Yizhou Sun. 2022. Rlogic:
Recursive logical rule learning from knowledge graphs. In KDD '22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, August 14. 18. 2022, pages 179-189. ACM.
Rajarshi Das. Shehzaad Dhuliawala, Manzil Zaheer. Luke Vilnis, Ishan Durugkar, Akshay Krishnamurthy, Alex Smola, and Andrew McCallum. 2018. Go for walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning. In
Ni Lao and William W. Cohen. 2010. Relational re- trieval using combination of path-constrained ran- dom walks. Mach. Learn., 81 (1)):533

---

Ni Lao, Tom M. Mitchell, and William W. Cohen, 2011. Random walk inference and learning in A large scale knowledge base. In Proceedings of the 2011 I Con- ference on Empirical Methods in Natural Language Processing, EMNLP 2011, 27-31 July 2011, John McIntyre Conference Centre, Edinburgh, UK, A meet- ing of SIGDAT, Special Interest Group of the ACI., pages 529-539. ACL.
in Neural Information Processing Systems 34: An- nual Conference on Neural Information Processing Systems 2021, NeurlPS 2021, December 6-14, 2021, virtual, pages 13497-13510.
Meng Qu, Junkun Chen, Louis-Pascal A. C. Xhonneux, Yoshua Bengio, and Jian Tang. 2021. Rinlogic: Learning logic rules for reasoning on knowledge graphs. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net
Yann LeCun. Sumit Chopra, Raia Hadsell, M Ranzato. and Fujie Huang. 2006. A tutorial on energy-base -based learning. Predicting structured data, 1(0).
Matthew Richardson and Pedro M. Domingos. 2006. Markov logic networks. Mach. Learn., 62(1-2):107- 136.
Sangkeun Lee, Sungchan Park, Minsuk Kahng, and Sang-goo Lec. 2013. Pathrank: Ranking nodes on a heterogeneous graph for flexible hybrid recom- mender systems. Expert Syst. Appl. 40(2))684-697.
Tim Rocktäschel and Sebastian Riedel. 2017. End-to- end differentiable proving. In Advances in Neural Information Processing Systems 30: Annual Confer- ence Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages
Ruizhe Li, Xiao Li, Guanyi Chen, and Chenghua Lin. 2020. Improving variational autoencoder for text modelling with timeste-wiie regularisation. In Pro- ceedings of the 28th International Conference on
Computational Linguisties, pages 2381-2397.
Ali Sadeghian, Mohammadreza Armandpour, Patrick Ding, and Daisy Zhe Wang. 2019. DRUM: end-to- end differentiable rule mining on knowledge graphs. In Advances in Neural Information Processing Sys- tems 32: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 15321- 15331.
Yizhi Li, Wei Fan, Chao Liu, Chenghua Lin, and Jiang Qian. 2022. TranSHER: Translating knowledge graph emhedding with hyper-ellipsoidal restriction. In Proceedings of the 2022 Conference on Empiri- cal Methods in Natural Language Processing, pages 8517-8528.
Xi Victoria Lin, Richard Socher, and Caiming Xiong. 2018. Multi-hop knowledge graph reasoning with re- ward shaping. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Procexs- ing, Brussels, Belgium, October 31 November 4, 2018. pages 3243-3255. Association for Computa- tional Linguistics.
Eric Salvat and Marie-Laure Mugnier. 1996. Sound and complete forward and backward chainingd of graph rules. In Conceptual Structures: Knowledge Representation as Interlingua, 4th International Con- ference on Conceptual Structures, ICCS '96, Sydney, Australia, August 19-22, 1996, Proceedings, volume 1115 of Lecture Notes in Computer Science, pages 248-262 Springer.
Xiao Liu, Shiyu Zhao, Kai Su, Yukuo Cen, Jiezhong Qiu, Mengdi Zhang, Wei Wu, Yuxiao Dong, and Jie Tang. 2022. Mask and reason: Pre-training knowl- edge graph transformers for complex logical queries. In KDD 22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, August 14 18, 2022, pages 1 1120-1130. ACM.
Yelong Shen, Jianshu Chen, Po-Sen Huang. Yuqing Guo, and Jianfeng Gao. 2018. M-walk: Learning to walk over graphs using monte carlo tree search. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Pro- cessing Systems 2018, NeurlPS 2018, December 3-8, 2018, Montréal, Canada, pages 6787-6798.
Shengyao Lu, Bang Liu, Keith G. Mills, Shangling Jui, and Di Niu. 2022. R5: rule discovery with reinforced and recurrent relational reasoning. In The Tenth In- ternational Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. Open- Review.net.
Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. 2021. Score-based generative modeling through stochastic differential equations. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenRe- view.net.
Denis Lukovnikov, Asja Fischer, Jens Lehmann, and Sören Auer. 2017. Neural network-basee question answering over knowledge graphs on word and char- acter level. In Proceedings of the 26th International Conference on World Wide Weh, WWW 2017, Perth, Australia, April 3-7, 2017, pages 121 1211-1220. ACM.
Frank Ludwig Spitzer. 1975 Principles of Random Walk. Principles of Random Walk.
Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. 2019. Rotate: Knowledge graph embedding by relational rotation in complex space. In 7th In- ternational Conference Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net.
Weili Nie. Arash Vahdat, and Anima Anandkumar. 2021. Controllable and compositional generation with latent-space energy-basee models. In Advances

---

Chen Tang, Hongbo Zhang, Tyler Loakman, Chenghua Lin, and Frank Guerin. 2023a. Enhancing dialogue generation via dynamic graph knowledge aggrega- tion. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Vol- une 1: Long Pupers), pages 4604-4616.
Wenhan Xiong, Thien Hoang, and William Yang Wang. 2017b. Deeppath: A reinforcement learning method for knowledge graph reasoning. In Proceedings of the 2017 Conference on Empirical Methods in Nat- ural Language Processing. EMNLP 2017, Copen- hagen, Denmark, September 9-1 1, 2017, pages 564- 573. Association for Computational Linguistics.
Chen Tang, Hongbo Zhang. Tyler Loakman, Chenghua Lin, and Frank Guerin. 2023b. Terminology-awe medical dialogue generation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1-5, IEEE.
Bishan Yang. Wen- tau Yih. Xiaodong He, Jianfeng Gao. and Li Deng. 2015. Embedding entities and relations for learning and inference in knowledge bases. In 3rd International Conference on Learning Represen- tations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
Komal K. Teru and William L.. Hamilton. 2019. Induc- tive relation prediction on knowledge graphs. CoRR, abs/1911.06962.
Fan Yang. Zhilin Yang, and William W. Cohen. 2017. Differentiable learning of logical rules for knowledge base reasoning. In Advances in Neural Information Processing Systems 30: Annual Conference ral Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 2319-2328.
Kristina Toutanova and Danqi Chen, 2015, Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compo- sitionality, CVSC 2015, Beijing, China, July 26-31, 2015, pages 57-66. Association for Computational Linguistics.
Neu-
Yuyu Zhang, Xinshi Chen, Yuan Yang, Arun Rama- murthy, Bo Li. Yuan Qi. and Le Song. 2020. Efficient probabilistic logic reasoning with graph neural net- works. In Sth International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30. 2020. OpenReview.net.
Hongwei Wang, Fuzheng 7hang, Jialin Wang. Miao Zhao, Wenjie Li, Xing Xie, and Minyi Guo. 2018. Ripplenet: Propagating user preferences on the knowledge graph for recommender systems. In Pro- ceedings of the 27th ACM International Conference on Information and Knowledge Management, CIKM 2018, Torino, Italy, October 22-26, 2018, pages 417- 426. ACM.
Zhengyan Zhang. Xu Han, Zhiyuan Liu, Xin Jiang. Maosong Sun, and Qun Liu. 2019. ERNIE: enhanced language representation with informative entities. In Proceedings of the 57th Conference of the Associa- tion for Computational Linguistics, ACL 2019, Flo- rence, Italy, July 28- August 2, 2019. Volume 1: Long Papers. pages 1441-1451, Association for Computa- tional Linguistics.
William Yang Wang, Kathryn Mazaitis, and William W. Cohen. 2013. Programming with personalized pager- ank: a locally groundable first-order probabilistic logic. In 22nd ACM International Conference on Information and Knowledge Management, CIKM 13, San Francisco, CA, USA, October 27- November 1, 2013, pages 1119-1138... ACM.
William Yang Wang. Kathryn Mazaitis, and William W. Cohen. 2014a. Proppr: Efficient first-order proba- bilistic logic programming for structure discovery, pa- rameter learning. and scalable inference. In Statisti- cal Relational Artificial Intelligence, Papers from the 2014 AAAI Workshop, Québec City, Québec, Canada, July 27, 2014, volume WS-14-13 of AAAI Technical Report. AAAI.
William Yang Wang, Kathryn Mazaitis, and William W. Cohen. 2014b. Structure learning via parameter learning. In Proceedings of the 23rd ACM Inter- national Conference on Conference on Information and Knowledge Management, CIKM 2014, Shang- hai, China, November 3-7, 2014, pages 1199-1208. ACM.
Chenyan Xiong, Russell Power, and Jamie Callan. 2017a. Explicit semantic ranking for academic search via knowledge graph embedding. In Proceed- ings of the 26th International Conference World Wide Web, WWW 2017, Perth, Australia, April 3-7, 2017, pages 1271 22799 ACM.

---

7 Appendix
Methods
FB15K-237 (mins)
WNISRR, (mins))
NerualLP DRUM
395
122.3
7.1 Dataset Statisties
373.8
118.7
RNNLogic
331.6
100.3
In this part, we provide brief introduction and statistics of FB FB15k-237 and WN18RR, which are used in this paper.
LATENTLOGIC
24.6
12.4
Table Training time running on a single NVIDIA Tesla V100 GPU.
FB 15k-237 is one of the most commonly used benchmark knowledge graph datasets, which is online collection of structured data har- vested from many sources, including individ- ual, user-submitted wiki contributions.
7.4 Supplementary Experiment
Evaluation of Training Efficiency To showcase the efficiency of LATENTLOGIC, we compare the training time different models in Tab, 4. For a fair comparison, we utilize a single GPU (Tesla 100 to train each model and implement the hy- perparameters recommended by the original papers. Our observations are as follows: (i) LATENTL.OGIC outperforms the basclines in efficiency while still achieving competitive performance. (ii) NeuralLP and DRUM do perform well due to their in- volvement in large matrix multiplication. (iii) RNNLogic is also less efficient because the EM- based optimization procedure.
WN18RR is recognized as a widely adopted benchmark knowledge graph dataset. Its pur- pose is to generate user-friendly dictionary and thesaurus and facilitate automatic text analysis. The entities in the dataset represent word senses, while relationships define the lexical relations between these senses.
Dataset
# Data
Relation
Entity
FB15K-237
310.116
237 11
14,541
WNISRR 93,003
40,943
Sensitivity of Randomness Since our approach relies random sampling techniques, would like conduct experiments examine the sensitiv- ity of LATENTLOGIC to randommess. In Tab.5, we report the mean and standard deviations of the re- sults under different random sceds. We can notice that the randomness sampling has minimal impact on the performance of LATENTLOGIC.
Table 3: Dataset statisties,
7.2 Implementation Details
We implement LATENTLOGIC over Pytorch². We use the Adam optimizer to train both the rela- tional path autoencoder and the rule discrimina- tor. A grid search is performed to determine the best hyperparameters based the performance the validation sets. We employ the dopri5 neural ODE solver, with (10-3, 10-3) tolerances, and set T= 1, 9min = 0.1, and Amax = 20. The maximum length of relational paths is set to 3 in FB15k-233 and in WNISRR. All experiments are executed on a single Nvidia Tesla V100 GPU.
FB15k-237
WNISRR
MRR
H 1
HΦ 10
MRR
H@l
H 10
1
0.320
21.2
51.4 0.247
0.481 0.009
45.2
55.3
0.013
0.118
0.211
0.187
Table S: result w.t.. randomness.
7.3 Details of Forward Chain
We provide the details of how to use forward chain infer missing facts given learned logic rules. Specifically, Given query r,,r,)), let A be the set of candidate answers that can be discovered by any learned rule using forward chaining. For each candidate answer a E A, the score of triple (h, r, a) can be calculated as Enute Epath Φ(rule), where é is the confidence score of logic rule. Then we can rank candidate answers by the scores.
$https://pytorch.org


Reviews:
[{"review_id": "whX3J5khyX", "reviewer": "Reviewer_Tpzv", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Reviewer Tpzv,\n\nWe express our gratitude to the reviewer for providing us with insightful comments. In the following, we address each concern raised by the reviewer individually:\n\n> Q1.The latent space mapping idea is creative but also complex, making the model behavior somewhat harder to interpret. Additional visualizations or analyses could help provide insight. \n\nWe apologize for any confusion caused. We acknowledge that, due to space constraints in the current version, there is a lack of detailed description of our model and implementation. However, we will ensure to provide additional pages in the final version to include more comprehensive information on our model and implementation details. These additions will include an extended framework diagram to visualize more details, such as the rule discriminator. \nThank you for bringing this to our attention.\n\n> Q2: It is not clear how the approach handles very long relational paths or complex rules. Testing the scalability on more challenging tasks could be beneficial. \n\nSince our model is built upon a generative model, it has the capability to handle longer relational paths. Nevertheless, based on previous research and our own experiments, we have observed that relational paths with a length greater than 5 have minimal impact on the reasoning process. This finding suggests that longer relation paths are often inferred from shorter ones. Therefore, we believe that there is limited beneficial information gained from excessively long relation paths in our model.\n\n> Q3: The space of related methods compared against is limited. Comparing to a few additional recent techniques could better situate the contributions. \n\nWe acknowledge and appreciate the valuable advice, we have not included all related work due to space constraints. But we will try to add more recent work to highlight the contribution of our work in the final version.\n\n> Q4: Certain details are lacking on hyperparameter tuning, optimal settings, and other implementation choices to ensure reproducibility.\n\n---\nThank you for your valuable suggestions. In light of these suggestions, we will include the details of hyperparameter tuning and other implementation aspects in the appendix.\n\nSincerely & Best wishes", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Strengths:\n1.Proposes a creative way of mapping the discrete relational path search problem into continuous latent space for more efficient optimization.\n2.The model components (VAE, discriminator, ODE sampler) are nicely integrated for controlled logic rule generation.\n3.Achieves new state-of-the-art results compared to prior logic rule learning methods.\n4.Demonstrates improved training efficiency and scalability over existing approaches.\n5.Provides interpretable logic rules that can explain predictions and generalize to new facts.", "reasons_to_reject": "1.The latent space mapping idea is creative but also complex, making the model behavior somewhat harder to interpret. Additional visualizations or analyses could help provide insight.\n2.It is not clear how the approach handles very long relational paths or complex rules. Testing the scalability on more challenging tasks could be beneficial.\n3.The space of related methods compared against is limited. Comparing to a few additional recent techniques could better situate the contributions.\n4.Certain details are lacking on hyperparameter tuning, optimal settings, and other implementation choices to ensure reproducibility.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes a new framework called LATENTLOGIC for efficiently learning logic rules from knowledge graphs. The key ideas and contributions are:\nProblem Addressed:\n1.Learning interpretable and generalizable logic rules from knowledge graphs is important for reasoning and inference, but existing methods have challenges like huge search spaces and inefficient optimization.\nMain Contributions:\n1.Maps discrete relational paths to a continuous latent space using a VAE-based auto-encoder, avoiding enumerating all paths.\n2.Introduces a rule discriminator on the latent space to measure consistency between rule body and head, forming an energy-based distribution for sampling.\n3.Employs a neural ODE-based sampler for controlled generation of rule bodies given a rule head.\n4.Achieves state-of-the-art results on FB15k-237 and WN18RR datasets compared to existing rule learning methods", "excitement": 3, "reproducibility": 3, "ethical_concerns": "Yes", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "CQquXkRWgG", "reviewer": "Reviewer_dB1w", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer dB1w,\n\nWe would like to thank you for all the meticulous and constructive feedback and positive comments. We have elaborated on our responses as follows:\n\n- Based on our research and to the best of our knowledge, our work is the first to propose the concept of controllable generation for addressing rule learning problems. \n\n- This controllable generation approach effectively tackles the shortages of excessive search space and inefficient optimization, resulting in more efficient and effective rule learning methods compared to previous approaches such as Neural Logic Programming and EM-based methods like RNNLogic. We firmly believe that our work will serve as an inspiration for future research endeavors in this field, particularly in exploring the representation of latent space and modeling of joint distributions. In addition, this paradigm can be applied to other tasks involving discrete rule learning.\n\n---\nThanks again for your appreciation and constructive comments on our work!\n\nSincerely & Best regards", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper is eloquently structured, with each component meticulously illustrated.\n2. From a technical standpoint, the proposed method stands robust, underpinned by thorough experiments. Notably, it not only improves the quality of generated rules but also scales more effectively.", "reasons_to_reject": "The study of learning logic rules over knowledge graphs, while valuable, has become somewhat saturated. Numerous methodologies have been put forth in recent times, and the incremental advancement presented by this paper seems relatively modest.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper delves into the realm of learning logic rules over knowledge graphs. Traditional enumeration methods grapple with an expansive search space, while Reinforcement Learning and EM methodologies are hindered by intricate optimization processes. Addressing these challenges, this paper introduces a method of controllable sampling in latent space, which effectively sidesteps the need to enumerate relational paths. Central to this approach is a VAE-based autoencoder, which projects the relational paths into a continuous latent space. A discriminator is then leveraged to gauge the semantic coherence between the latent vector and the rule head. Further, a neural ODE-based sampler extracts latent vector samples corresponding to the targeted rule head. Comprehensive experiments corroborate the method's enhanced scalability and its capacity to generate high-quality logic rules.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "vZsq3JAolF", "reviewer": "Reviewer_rqqL", "summary": "", "questions": "- What is the VAE autoencoder that you use?! You mention this is pretrained. \n- Could you describe in more detail as to how the rule discriminator scores the extracted rules?!\n- It is unclear how you measure the quality of the extracted rules. Could you elaborate on that?!\n", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer rqqL,\n\nWe thank the reviewer for all the insightful comments. We will add more technical details and implementation details in the final version and fix the typos and errors. \nIn the following sections, we address each concern raised by the reviewer individually.\n\n> Q1: There is a need to explain why the method fails in getting better results than some of the embedding-based methods, especially on the FB15k dataset.\n\nThank you！This is an intriguing question. It is commonly observed that rule-based methods tend to underperform in comparison to embedding-based methods. We attribute this discrepancy to the incompleteness of the knowledge graph. To illustrate, let us consider the rule $\\texttt{Nationality} \\wedge \\texttt{OfficialLanguage} \\rightarrow \\texttt{SpeakLanguage}$. With this rule, we can deduce that $\\texttt{Nationality}(Jordan, USA) \\wedge \\texttt{OfficialLanguage}(USA,English)$ implies $\\texttt{SpeakLanguage} (Jordan, English)$. However, if the fact $\\texttt{Nationality}(Jordan, USA)$ is missing from the background knowledge graph, rule-based methods cannot accurately predict this fact.\n\n> Q2: What is the VAE autoencoder that you use?! You mention this is pretrained.\n\nThank you for your question！In our approach, we utilize a Variational Autoencoder (VAE) with a Recurrent Neural Network (RNN)-based architecture. Specifically, we employ an RNN-based encoder to transform discrete relational paths into a continuous latent space, while applying an RNN-based decoder to reconstruct relational paths from a latent vector.\nIn the training procedure, we collect training data by sampling relational paths from the knowledge graph (KG).\n\n> Q3: Could you describe in more detail as to how the rule discriminator scores the extracted rules?!\n\nThank you for your question！We assign an embedding vector to each relation. The rule discriminator utilizes an RNN to encode the relation embeddings of the relational path into a path representation. It then generates a similarity score by comparing the path representation with the rule head relation representation.\nThe rule discriminator is trained to predict the consistency score between relational paths and the rule head.\nFor each extracted rule, the rule discriminator takes the rule head, denoted as$r_h$and rule body, represented as$(r_1, \\ldots, r_n)$and outputs the consistency score between them.\n\n> Q4: It is unclear how you measure the quality of the extracted rules. Could you elaborate on that?!\n\nThank you for your question！The quality of the extracted rules can be evaluated using the rule discriminator, as described in the previous question (Q3).\n\n---\n\nWe appreciate the reviewer's endorsement and constructive suggestions, which will enhance the final paper version.\n\nSincerely and Best regards", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The method is novel and better than prior research for rule-based completion of knowledge graphs.\n- The paper showcases a proper comparison with a long list of baselines.", "reasons_to_reject": "- There is a need to explain why the method fails in getting better results than some of the embedding-based methods, especially on the FB15k dataset. \n- The implementation details are missing. \n- The paper is missing some details which would have been better explained in a long paper rather than the current short version.\n", "questions_to_authors": "- What is the VAE autoencoder that you use?! You mention this is pretrained. \n- Could you describe in more detail as to how the rule discriminator scores the extracted rules?!\n- It is unclear how you measure the quality of the extracted rules. Could you elaborate on that?!\n", "paper_topic_and_main_contributions": "This paper proposes a novel approach for extracting rules from Knowledge bases. Their approach relies on creating a latent space where they can use latent vectors to draw samples for rule heads and then use a pretrained VAE decoder to generate rule bodies for each sampled head. The quality of the rules is verified by an energy-based discriminator. They provide results on two datasets and a series of different methods based on embedding and rules. They show their approach can mainly surpass the prior rule-based methods. ", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "- Line 245: Why is the font different for \"the effect of random sampling\"?!\n- Line 082: Please write the full name of VAE in its first use in the paper. \n"}, "extra_text": {}}, {"review_id": "DPTigSRRk2", "reviewer": "Reviewer_GJYL", "summary": "", "questions": "1. It would have been appreciated to compare qualitative results of the different methods.\n2. What is training time like of embedding based methods?\n3. What is inference time like?\n4. Are all the learned rules intuitively reasonable?\n\n", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer GJYL,\n\nWe sincerely appreciate the valuable feedback provided by the reviewer.  We now address the concerns raised by the reviewer one by one as below.\n\n> Q1: The performance is not state-of-the-art. Only a limited number of methods are compared in table 1. Check [a] for more methods.Only two datasets are used, more datasets such as Kinship, UMLS, YAGO3, etc. can be used.\n\nThank you for your suggestion, and we are sorry that we did not include more baselines and data sets due to space limitations, and we choose FB15k-237 and WN18RR since they are the most commonly used benchmark datasets.\nWe provide experimental results on UMLS below. We will try to add experimental results on additional data sets in the final version.\n\n| Methods | MRR | Hits@11 |\n| --- | --- | --- |\n| RotatE | 0.925 | 86.3 |\n| DRUM | 0.813 | 67.4 |\n| RNNLogic | 0.842 | 77.2 |\n| LATENTLOGIC | 0.859 | 81.7 |\n\n> Q2: It would have been appreciated to compare qualitative results of the different methods.\n\nThank you for your suggestion. Taking the results from the FB15k-237 dataset as an example.\n\n| Methods | MRR | Hits@3 |\n| --- | --- | --- |\n| TransE | 0.226 | - |\n| TuckER | 0.470 | 48.2 |\n| RotatE | 0.476 | 49.2 |\n| DRUM | 0.382 | 38.8 |\n| RNNLogic | 0.455 | 47.5 |\n| LATENTLOGIC | 0.481 | 49.7 |\n\nWe can observe that the  LATENTLOGIC performs better than the salient rule-based method (RNNLogic) with a relative improvement of 5.7% in MRR and 4.6% in Hits@3. Furthermore, LATENTLOGIC also outperforms the prominent embedding-based method (RotatE) with a relative improvement of 1.1% in MRR.\n\n> Q3: What is training time like of embedding based methods?\n\nThank you for your question.  We report the training time of embedding-based methods on FB15k-237 dataset. The hyperparameters used are identical to those specified in [a]. All experiments were conducted on a single NVIDIA Tesla V100 GPU, consistent with the paper.\n\n[a] Sun, Z., Deng, Z. H., Nie, J. Y., & Tang, J. (2019). Rotate: Knowledge graph embedding by relational rotation in complex space. \n\n|  | FB15k-237(mins) |\n| --- | --- |\n| TransE | 51.3 |\n| DistMult | 83.5 |\n| RotatE | 165.6 |\n\n> Q4: What is inference time like?\n\nThe table below presents the inference time on the test sets for our method. The test set for FB15k-237 consists of 40,932 triplets, while the test set for WN18RR comprises 6,268 triplets.\n\n|  | FB15k-237(mins) | WN18RR(mins) |\n| --- | --- | --- |\n| LATENTLOGIC | 11.13 | 3.77 |\n\n> Q5: Are all the learned rules intuitively reasonable?\n\nThank you for your question. We can find that most of the rules are reasonable. Although some rules on some datasets (wn18rr) cannot be described in natural language, they still logically established.\n\n---\n\nFinally,  Thank you for all the constructive feedback and positive comments. We will add the aforementioned additional details you   concern or suggested in the final version.\n\nSincerely & Best wishes", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The proposed solution is novel. Specifically, the proposed framework utilizes fancy component such as energy-based models and ordinary differential equations to boost the efficiency of effectiveness, which is novel and interesting.\n2. The training time can be greatly reduced compared with other rule-based methods (as shown in Table 2).  \n3. Qualitative analyses further justify the effectiveness of the rule-mining process.\n4. The presentation is clear and easy to follow.", "reasons_to_reject": "1. The performance is not state-of-the-art. Only a limited number of methods are compared in table 1. Check [a] for more methods.\n2. Only two datasets are used, more datasets such as Kinship, UMLS, YAGO3, etc. can be used.\n\n[a] A Survey on Knowledge Graphs: Representation, Acquisition, and Applications, https://ieeexplore.ieee.org/abstract/document/9416312", "questions_to_authors": "1. It would have been appreciated to compare qualitative results of the different methods.\n2. What is training time like of embedding based methods?\n3. What is inference time like?\n4. Are all the learned rules intuitively reasonable?\n\n", "paper_topic_and_main_contributions": "In this paper, the authors proposed a knowledge graph completion method, LATENTLOGIC.  It employs a pre-trained VAE to map relational paths into latent space, and uses a sampler based ODE to enable efficient generation of logic rules, aiming to avoid the inefficient optimization issue. To verify the effectiveness, they conducted experiments on two knowledge graph datasets and show that the proposed approach can outperform a number of embedding-based/rule-mining based approaches.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "[a] A Survey on Knowledge Graphs: Representation, Acquisition, and Applications, https://ieeexplore.ieee.org/abstract/document/9416312\n[b] Low-Dimensional Hyperbolic Knowledge Graph Embeddings, ACL."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper delves into the realm of learning logic rules on knowledge graphs for reasoning and introduces the LATENTLOGIC model, designed to mine logic rules through controllable generation within the latent space.

The overall consensus among most PC members is that this paper demonstrates a good soundness but elicits mixed feelings in terms of excitement. Several noteworthy weaknesses have been highlighted by the reviewers, including the high complexity of the proposed model, the absence of the latest baselines for comparison, insufficient details concerning the implementation, and potential limitations associated with the choice of datasets.

Considering the good soundness and the identified concerns, this paper has the potential to be accepted as a findings paper. However, the authors are encouraged to diligently address all the concerns outlined in the reviews during their new revision, in order to enhance the overall quality and impact of this paper.
Decision: Accept-Findings