Neural-Symbolic Entangled Framework for Complex Query Answering
Anonymous Author(s)
AffiliationAddress email
Abstract
Answering complex queries over knowledge graphs (KG) is an important yet challenging task because of the KG incompleteness issue and cascading errors duringreasoning.
Recent query embedding (QE) approaches embed the entities and relations in a KG and the first-order logic (FOL) queries into a low dimensional space,answering queries by dense similarity search.
However, previous works mainlyconcentrate on the target answers, ignoring intermediate entities’ usefulness, whichis essential for relieving the cascading error problem in logical query answering.
Inaddition, these methods are usually designed with their own geometric or distributional embeddings to handle logical operators like union( ∨ ), intersection( ∧ ), andnegation( ¬ ), with the sacrifice of the accuracy of the basic operator – projection,and they could not absorb other embedding methods to their models.
In this work,we propose a Ne ural and Sy mbolic E ntangled framework ( ENeSy ) for complexquery answering, which enables the neural and symbolic reasoning to enhanceeach other to alleviate the cascading error and KG incompleteness.
The projectionoperator in ENeSy could be any embedding method with the capability of linkprediction, and the other FOL operators are handled without parameters.
Withboth neural and symbolic reasoning results contained, ENeSy answers queries inensembles.
We evaluate ENeSy on complex query answering benchmarks, and
ENeSy achieves the state-of-the-art, especially in the setting of training model onlywith the link prediction task.
Introduction
People built different Knowledge Graphs, such as Freebase [4], YAGO [13] and Wordnet [11], tostore complex structured information and knowledge.
The facts in KG are usually represented in theform of triplets, e.g., isCityOf(New York, USA) .
KGs have been widely applied in various intelligentsystems such as question answering and natural language understanding.
One of the key tasks on KGreasoning is complex query answering which involves answering FOL query with logical operators including existential quantification
∃ ), conjunction( ∧ ), disjunction( ∨ ) and negation( ¬ )
Given a question "Who won the Turing Award in developing countries?" , as illustrated in Figure it could be converted to a FOL query and a computation graph can be generated with the query.
Each node in the computation graph represents an entity or an entity set, while each edge represents a logical operation.
Answering these queries is challenging since not all the answers could be directly identified by traversing the KG because of the incompleteness of KG
To address this problem several
QE methods [9, 14, 15, 6] are proposed which encode entities and the query to a low-dimensional embedding space and the logical operators are also parameterized
Following the corresponding computation graph, the query embedding could be computed step by step.
The target answers are selected according to the distance between the final embedding and candidate entity embedding 𝑞𝑞 = 𝑉𝑉 ? .
𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶
Developing_Country , 𝑉𝑉 ?
𝑞𝑞 = 𝑉𝑉 ? .
∃ 𝑉𝑉 : 𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶
Developing_Country , 𝑉𝑉 ∧ 𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶 𝑉𝑉 , 𝑉𝑉 ?
∧ 𝑊𝑊𝐶𝐶𝐶𝐶 (
Turing_Award , 𝑉𝑉 ? )
𝑉𝑉 ?
= {
China,India,Sri Lanka,...
[Figure]
Embedding Projection
Neural Symbolic Entangled
Arrrrembedding space
A
DevelopingCountry
A 2 : TuringAward
A 1 : r 1 : Contain
[Figure]
: Citizen r 3 : Win
[Figure]
Although the query embedding methods are effective for solving the incompleteness of KG, severallimitations still exist.
Firstly, the role of intermediate entities remains largely unexamined on complexquery task which we argue plays a significant role in obtaining the target answers since besidesincompleteness, cascading error also influences the accuracy of complex query.
Existing worksonly pay close attention to the target answers, but for multi-hop reasoning, the intermediate entitiescould not be ignored.
Secondly, previous works propose their own geometric shapes or distributionembedding so as to support different logical operators.
For instance, Query2Box (Q2B) embedqueries to boxes and can handle intersection( ∧ ), while ConE [26] and BetaE [15] embed query withcone embedding and beta distribution to further support negation( ¬ ).
Although the logical operatorsare supported with their own embedding shape, the performance on the basic operator, projection,is not satisfying.
For example, the accuracy of Q2B on one projection type query is worse than
TransE [5] according to the Q2B paper [14].
Few studies have investigated how to generalize existedembedding models, such as TransE and RotatE which have been proved achieving good performancein one-hop reasoning, to the complex query models.
In this paper, we propose a neural symbolic entangled model, named ENeSy, which enables embedding and symbolic reasoning to enhance each other.
In this framework, the embedding resultsinfluence the symbolic results to solve the KG incompleteness problem while symbolic results couldrevise embedding results by alleviating the problem of cascading error in multi-hop reasoning.
Ourapproach has the following important advantages: (1) ENeSy can generalize KG embedding methodsto answering complex query, and it could not only use the classic KGE algorithm like TransE, RotatE,the projection operator of existed query embedding methods could also be employed with our model.(2) The logical operators except projection (including ∧ , ∨ , ¬ ) are not parameterized so we couldtrain the whole model with only link prediction tasks, since meaningful complex query data might behard to obtain in the real world.
The experimental results prove that our model outperforms existing complex query answering methodsover standard knowledge graphs: FB15K-
[18] and NELL-995 [22].
The performance, which istrained with link prediction, is also competitive to or better than the baselines trained with complexquery.
The analysis of model proves the effectiveness of neural and symbolic entanglement to solvingthe problem of cascading error and KG incompleteness with our framework.
Related Work2.1 Knowledge Graph Embedding
A great deal of previous research into KG has focused on using machine learning to reason withembedding method.
The effects have been shown in TransE [5], TransH [20], TransG [21], ComplEx [19], ConvE [8], DistMult [23] and RotatE [17].
These approaches aim to embed the entities and relations to a continuous vector space so that one-hop reasoning can be answered with link prediction.
Different methods map the entities and relations into vector space with different distributions.
Meanwhile, the rule and path-based [10, 25, 24, 16], tries to use learned pattern of path or rules to domulti-hop reasoning.
However, KGE models lack the ability to handle complex logical operators like conjunction ( ∧ ),disjunction ( ∨ ) and negation ( ¬ ) and they are not easy to be generalized to multi-hop queries.
Incontrast, our framework can alleviate the problem of cascading error during multi-hop reasoningand also support all of the logical operators above so that any KGE models could be used to answercomplex query.
2.2 Embedding for Complex Query Answering
To answer logical queries, some works [9, 14, 26, 2, 15, 6] aims to encode the complex query intoa space which proposes various geometric shape to represent the entity set and support complexquery reasoning.
Generally, a FOL query is converted to a computation graph with a directed acyclicgraph (DAG) structure with which the query representation could be iteratively computed usingthe logical operation in embedding space.
GQE (graph-query embedding)[9] consider conjunctionoperator ( ∧ ) with vector representation.
Q2B (query to box)[14] replace the vector embedding withhyper-rectangles since it holds the idea that box embedding can represent sets of entities better anddefine the disjunctive norm form (DNF) to support the disjunction ( ∨ ) operator.
More recently, BetaE[15] models the query and entities with beta distributions which could support negation ( ¬ )operator.
CQD [1] applies beam search to a embedding model.
ConE[26] propose a new geometrymodel that embed entities with cones embedding.
FuzzQE [6] satisfies the axiomatic system of fuzzylogic to reason.
Q2P (Query2Particles) [2] encodes each query into multiple vectors.
GNN-QE[27]concentrates on the interpretation to the variables along the query path.
Most of these methods usually design their geometric or distribution embedding to support logicaloperators but the effective of the projection operator, which is the most basic operator, has not beendiscussed.
Meanwhile, they only concentrate on the final answer as label but ignore the intermediateentities during the query process which are also important for reasoning.
Preliminaries
In this section, we introduce the task of complex query answering on KG.
Given a set of entities Vand a set of relations R , a knowledge graph G is defined as ( V , R , T ) where T is the set of triplets.
Atriplet is defined as r ( e i , e j ) where e i ∈ V , e j ∈ V , r ∈ R if the relation r exists between e i and e j .
First-Order Logic.
The purpose of complex query answering on KG is obtaining the target answerof FOL query which is defined with existential quantifiers( ∃ ), conjunction ( ∧ ), disjunction ( ∨ ) andnegation ( ¬ ).
In a FOL query q , the anchor nodes are represented as a set V a ⊆ V , existentialquantified variables nodes are represented as V 1 , V 2 , . . .
V k and the target answer nodes is a variable
V ? .
Follow the betaE [15], we use FOL query in its disjunctive norm form, with which the query canbe represented as a disjunction of several conjunctions.
Finally, the query can be formulated as:
[Equation]
where c i is a conjunction of several literals a ij , i.e., c i = a ij ∧ · · · ∧ a im , and a ij is an atom ornegation of an atom: r ( e a , V ) or
¬ r ( e a , V ) or r ( V ′ , V ) or ¬ r ( V ′ , V ) where e a ∈ V a , V, V ′ ∈{ V 1 , V 2 , . .
, V k , V ? } and V ̸ = V ′ in an atom.
Computation Graph and Logical Operators As illustrated in figure 1, for a given FOL query, wecan represent the whole process of reasoning as a computation graph.
Each node corresponds to avariable V or an anchor node e a and each edge represents a logical operation over the entity setswhich includes these following operators:
• Relational Projection: Given an entity set S ⊆ V and a relation r ∈ R , the projection
operator return a new entity set S ′ that contains the entities related to at least one of entity
in S : S ′ = { e ′ ∈
V|∃ r ( e, e ′ ) , e ∈ S} .
• Intersection:
Given sets of entities { S 1 , S 2 , . . .
, S n }
where S i ⊆ V , the intersection
operator returns the intersection of these sets (cid:84) ni
=1 S i .
• Union: Given sets of entities { S 1 , S 2 , . . .
.S n } where S i ⊆ V , the union operator returnsthe union of these sets (cid:83) ni =1 S i .
[Figure]
• Complement: Given a set of entities S , the complement operator returns its complement
[Figure]
S ′ =
V − S .
[Figure]
Methodology
[Figure] In this section, we first introduce the neural and symbolic entangled projection operator in detail.
Thenwe define symbolic-based logical operators and describe the ensemble prediction using embeddingand symbolic results.
Finally, the objective function and the learning procedure will be represented.
4.1 Neural and Symbolic Entangled Reasoning
[Figure]
In our work, there are two ways to represent the entities and relations.
The neural part is just like the
KGE or query embedding method we adopt, and rotatE [17] is chosen in this paper which embedentities and relations to a complex space C k .
For the symbolic part, an entity and a relation areencoded as a one-hot vector and an adjacent matrix, respectively.
Specifically, given the KG G , entity set V and relation set R , an entity e ’s embedding representationis a vector v e ∈ C k and its symbolic representation is encoded as an one-hot vector p e ∈ { 0 , 1 } 1 ×|V| .
Each relation r is modeled as a vector v r ∈ C k and the corresponding symbolic representation is anadjacent matrix M r ∈ { 0 , 1 } |V|×|V| where M ijr = 1 if r ( e i , e j ) ∈ G , else M ijr = 0 .
Neural projection follows the embedding method, we use rotatE [17] here as an example.
For aprojection query ( h, r, ?) , the functional mapping with relation r is an element-wise rotation from hto the target answer t :
[Equation]
where
[Equation]
and ◦ means Hadamard product.
This step gets the predicted embedding which could be used tosearch the target answers.
Symbolic projection is conducted with matrix multiplications followed TensorLog [7]:
[Equation]
where p t is a multi-hot vector that represents the entities linked with h via relation r , ⊤ meanstransposing the vector and g is a normalization function.
Particularly, we consider the function asg ( x ) =
x /sum ( x ) .
This step gets the target entities by traversing KG.
Entangled projection tries to take advantage of these above two results.
Since embedding predictionsuffers from the cascading error but could obtain the answer not linked with h , while traversing KGcould not get the answers which lacks the edges to the head entity but the searched answers are convincing, ENeSy combines them in an entanglement way as illustrated in Figure 2.
Specifically,the similarity between the predicted embedding v t and all the entities is calculated first.
We definethe similarity function as:
[Equation]
where γ denotes the margin.
The L1 norm function can be changed to any distance function. Wedefine a new vector representation with these similarity values. After softmax function, we get ainferred vector p ′ t ∈ [0 , 1] 1 ×|V| which is generated from v t .
With p ′ t , a new symbolic vector p ′′ t isobtained by the following step:
[Equation]
This new symbolic vector p ′′ t integrates the information from the embedding v t with symbolicreasoning results p t .
This procedure adds more entities, which might be the answer to the querywhich are not linked with h , to p ′′ t .
This enhancement eliminates the limitation of KG incompletenessof symbolic reasoning to some extent.
Each element of p ′′ t could be regarded as the probability ofthe corresponding entity.
Let’s assume the entity set with non-zero probability as S t , a aggregationfunction is employed to transfer the symbolic vector p ′′ t and the embedding of entities in S t to a newembedding vector v ′ t with an MLP function:
[Equation]
where p i ′′ t is the corresponding probability of e i in p t .
This new embedding v ′ t aggregates thesymbolic answer, Note that although we use the rotatE as the neural function, any other sufficientlyexpressive KG embedding model or the projection operator of any other query embedding modelscould be employed with our framework in theory.
4.2 Neural and Symbolic Ensemble Answering
With the symbolic vector p , the logical operator intersection( p 1 ∧ p 2 ), union( p 1 ∨ p 2 ) and negation( ¬ p ) could be defined as follows:
[Equation]
where ◦ is Hadmard product and α is a hyper parameter.
After getting symbolic vector with theselogical operators, the MLP function used in Equation (5) is employed to get an aggregated embedding.
The embedding and symbolic vector can both be used to get the final answers.
For the neural part, thesimilarity between the embedding vector v and all the entities e ∈ V is computed with Equation (3),which is used to rank the candidate answers.
For the symbolic part, since the vector p represents theprobability of each entity, the answers can be directly obtained with ranked elements of p .
To makethe ensemble using of the two type answers, we set a to get a combined result with v and p as:
[Equation]
where λ is the weight to balance the influence of v and p and Concat is a function mapping thesimilarity between all entities e ∈ V and v to a vector.
The final answers can be determined withg ( a ) ∈ [0 , 1] 1 ×|V| .
4.3 Learning Procedure
Given a query q with answer entity set S q , after getting the final answer embedding v q and symbolicvector p q , we construct two following objective loss functions:
[Equation]
[Equation]
where e ∈ S q is an answer of q , e ′ / ∈ S q is a negative answer which is sampled randomly.
σ is thesigmoid function, · is dot-product and [ x , θ ] + denotes the maximum value between each element ofx and θ , which is a threshold.
Meanwhile, the MLP function in Equation (5) needs to be pretrained individually.
Since this functionis employed to convert a symbolic vector to an embedding vector, and in the projection operator, wehave p ′ t which is generated from v t , MLP could be used to convert p ′ t back to v t .
Based on this, wealso design a loss function to train this function as follows:
[Equation]
where S ′ t is the corresponding entity set whose probabilities are not zero in p ′ t .
In the first step of thetraining process, the symbolic part are not included, and only embedding of entities and relations willbe trained with link prediction task.
Next, in the second step, the completed projection operator willbe trained still by link prediction and the loss function is L
=
L 1 + L 2 + L 3 .
In theory, the modelcould answer complex query after the above two steps, but it could also be fine-tuned with complexquery data using loss function
L =
L 1 + L 2 .
Experiment
In this section, we evaluate the ability of ENeSy on answering the complex query on several KGbenchmark datasets.
The experiment results demonstrate that: 1) The performance of ENeSy isexcellent; 2) We can train ENeSy with only link prediction task to answering complex query; 3) Theembedding and symbolic part of our model can enhance each other.
5.1 Dataset and Experiment Setting
5.1.1 Datasets and Evaluation Protocol
We perform the experiments on two benchmarks, FB15K-
[18] and NELL-995 [22].
FB15K-237is a subset from Freebase [3] and removes the inverse relation.
NELL-995 is a dataset constructedfrom high-confidence facts of NELL [12].
The focus of our experiment is answering FOL queries on incomplete KGs so we only evaluate theability of models to obtain the answers that could not be discovered by traversing KGs.
Specifically,with the standard training, validation and testing set, the edges in KG could be divided into three parts,which are training edges, validation edges and test edges.
The corresponding graph G train , G validand G test are build with training edges, training + validation edges and training + validation + testedges, respectively, so we have G train ⊂
G valid ⊂ G test .
We only use the queries whose answersets A train , A valid and A test on different graph have A train ⊂
A valid ⊂ A test .
The answers in
A valid − A train could be used to tune the hyper-parameters and results are reported with the answerentities in A test − A valid .
This means we only evaluate on entities that are not answer of the trainingquery set and the model has not seen them.
Meanwhile, these answers could not be found with graphtraversal on G train / G valid .
For each answer of a test query, we calculate the Mean Reciprocal Rank(MRR) as evaluation metrics and the results are reported with filter setting as TransE [5], in which allother correct answers are filtered out before calculating the rankings of answers.
The queries sampled from these two benchmark are provided by BetaE [15] which is an expansionof the version provided by Q2B [14].
The query set contains 14 different types of query structureare shown in Figure 3.
In order to further verify the generalization ability of the models, only 5conjunctive query ( 1p/2p/3p/2i/3i ) and 5 query types with negation ( 2in/3in/inp/pni/pin ) are used totrain the model, while the other four query types ( 2u/up/ip/pi ) do not appear in the training processand are directly evaluated when testing, which makes this task more challenging.
5.1.2 Baseline
We consider four baselines as the compared methods in the following sections: Graph Query Embedding (GQE) [9] embed the query into vectors, which could handle the projection and conjunctionqueries.
With DNF, it could also support union operator.
Query2Box (Q2B) [14] uses box embeddingto represent the query and entity sets and could answer existential positive first-order (EPFO) logicqueries.
Beta Embedding (BetaE) [15] models query as Beta Distributions which enables it to supportnegation( ¬ ) operation.
CQD [1] applies beam search to a embedding model but it could not supportnegation operator.
FuzzQE [6] uses fuzzy logic to embedding the query.
pin inp 3in 2in 1p 2p 3p 2i 3i ip pni pi 2u up
[Figure] Entity
Variable
Answer
Projection
Intersection
Union
Negation
[Table]
The MRR results of these baselines are from the BetaE [15] and FuzzQE paper [6].
The first twomethods can’t handle negation operation and among these models, only FuzzQE can be trained withonly link prediction task and answer complex query.
5.1. Training Procedure and Experiment Settings
To train the model [Table] we first only train the embedding of relations and entities with 1p queries whichis similar to the pure KG embedding training.
Second, the projection operator of ENeSy is trained on1p queries.
Meanwhile, the queries of all structures can be used to fine-tune the model.
We implement our model with Pytorch framework and train our model on RTX3090 GPU.
The
ADAM optimizer was used to parameter tune with learning rate of 0.0001 that will decrease withtrain process.
In the second step, the learning rate is set to be 10 − 5 to train the model with 1 p queries,and in the fine-tune process, the learning rate starts with 2 ∗ 10 − 7 .
We set the embedding dimensionof entity and relation to be 1024, respectively.
The hidden state dimension of MLP is 1024.
Thetraining batch size is {64, 16} for FB15K-237 and NELL-995, while the negative sample size is {128,32}.
The margin γ used in similarity computation is 24.
θ used as threshold is 10 − 10 .
α is set to be10.
The choices of λ which is used for ensemble prediction are based on the results of valid set foreach query type.
The best hyper-parameter setting are selected by the MRR metric on the valid set.
We run the experiment several times and find random seed has almost no effect on the result, but inthe fine tune process, the learning rate influence the accuracy.
5.2 Trained with Link Prediction
Since meaningful FOL queries are usually not available in real scenes, we first report the results ofmodels which are trained with only 1p query data, which also can be seen as link prediction task.
The MRR results are shown in Table 1.
[Table] As the table shows, compared with pure embedding methods like GQE, Q2B and BetaE, the performance of ENeSy significantly improves.
Since the operators except projection of these methods areparameterized, the ability to handle complex query is limited, but our model can be generalized tomore complex query structure.
Even though, the improvement on 1p query proves the advantages ofgeneralizing other KG embedding models to solving this problem.
Compared with FuzzQE, whichcould also be trained with link prediction, our model improves the average MRR of EPFO about2.6%(relatively 11.9%) on FB15K-237 and 1.6%(relatively 5.9%) on NELL-995.
For the querieswith negation, ENeSy provides a more absolute improvement, which is 1.5%(relatively 24.6%) and2.1%(relatively 28.8%) on FB15K-237 and NELL-995.
We believe the reason of this enhancement isthat the symbolic representation can indicate the probability of each entity better.
5. Trained with Complex Query
The results of models trained with query data are also reported in Table 2.
Compared with the average
MRR in Table 1, the performance of most baselines improves a lot because of the labeled data ofcomplex query, but ENeSy still achieves the best performance on the average MRR of EPFO queryand negation query.
Moreover, our model could get closer results with different training settings.
Specifically, the average MRR decreases about 4.7% and 2.4% for EPFO and 5% and 4.1% fornegation queries on FB15K-237 and NELL-995 respectively when train with link prediction.
For
FuzzQE, the results decline about 9.9%, 7.5% for EPFO and 22.4%, 8.8% for query with negation on
FB15K-237 and NELL-995, respectively.
This proves stronger robustness of our methods.
Note thatthough the 2p/3p results seems to be worse than FuzzQE, we can replace RotatE with the projectionoperator of FuzzQE, and the results should be the same in theory. All the models do not get goodresults on negation query. We think it’s
because after the negation operation, most of the entities in
KG will be included, which makes the reasoning hard.
Maybe a better way is only considering theentities which belong to the same type with the entities before negation operation.
5.4 Analysis of ENeSy
To get deep insights on the neural and symbolic reasoning part of ENeSy, we investigate their impactin this section.
In what follows, we first explore how symbolic affect the embedding results.
Wethen examine the influence of embedding to symbolic.
Finally, the effect of ensemble prediction isdiscussed.
The results are based on FB15K-237, and the situation on NELL-995 is similar.
5.4.1 Q1: Does symbolic results assist neural reasoning in cascading error?
We compare the pure KE embedding model, which is RotatE in this experiment, with the embeddingresults of ENeSy without ensemble, which we denote as ENeSy E .
For fairness of training data, afterthe KGE model has converged which is trained with 1p query, ENeSy E is trained with 1p data rather
q( ∃ ) q( ∧ ) q( ∨ ) q(┐)
ENeSy_E ENeSy_S ENeSy ENeSy E ENeSy S ENeSy   30 42.
-          1p 2u 2p up 3p
KGE ENeSy_EENeSy E 1p(
[Figure]
) up( [Figure] ) 2p(2) 2u(2) 3p( [Figure]
Increase0.
[Figure] [Figure]
than complex query data based on the KGE model.
Since the KGE method could not support logicaloperators, the results of 1p/2p/3p query and the 2u/up with DNF are reported.
The MRR results andthe ratio of improvement are shown in Figure 4 (a).
The query types are listed below the horizontalaxis and we sort them by the length of query which is the longest distance from the anchor nodesto the target node in computation graph, and it’s marked after the query type.
Based on the similarperformance on 1p query, the MRR results of more complex queries significantly improves withquery length increases.
This comparison demonstrates that the cascading error, which is the mainlimitation for multi-hop embedding reasoning, has been alleviated with the symbolic assistant.
5.4.2 Q2: Does embedding results assist symbolic reasoning in KG incompleteness?
The symbolic MRR results of ENeSy without ensemble, denoted as ENeSy S , and pure symbolicresults are shown in Figure 4 (b).
We divided the query types into four groups according to theoperator they have.
Since we only evaluate the generalization ability of models with answers thatcould not be found by simply traversing KG, the traversing results is nearly zero (since the result is
MRR, the number won’t be an absolute zero), while the ENeSy S achieve better results than mostbaselines.
The reason for this significant improvement from zero to almost SOTA performance is inthe entangled process, ENeSy successfully captures the information from embedding which makessymbolic results include the answers could not be obtained directly.
In summary, the experimentproves that the KG incompleteness problem for symbolic reasoning can be solved in our framework.
5.4.3 Q3: Is ensemble prediction of neural and symbolic results useful?
Ensemble prediction enables us to fuse the symbolic and reasoning results.
To verify its effectiveness,we compare the performance of ENeSy E , ENeSy S and ENeSy trained with 1p queries.
The MRRresults are shown in Figure 4 (c).
The queries are grouped in the same way as Q2.
As the figureillustrates, all the results of different group queries improve after ensemble.
Specifically, the average
MRR metric in different group increase about 14.0%, 15.3%, 4.0%, 28.6% and 2.3%, 1.3%, 2.9%,8.0% compared with ENeSy E and ENeSy S , respectively, which certifies that the two parts could notonly enhance each other in the reasoning process, but also can be combined in the final results.
Conclusion
In this paper, we proposed ENeSy, a neural-symbolic entangled framework for answering complexlogical queries over knowledge graph.
This model could generalize any embedding methods tocomplex query and use the symbolic reasoning results to alleviate cascading error, while the symbolicpart also benefits from neural reasoning to solve the problem of KG incompleteness.
The ENeSysupports all the FOL operations and can be trained with only link prediction tasks.
Experimentalresults show that our model achieves state-of-the-art performances on answering FOL queries withstrong robustness, and each part is tightly entangled to enhance each other.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes] See Section 1 and abstract.
(b) Did you describe the limitations of your work?
[Yes] See Section 6.
(c) Did you discuss any potential negative societal impacts of your work?
[No] There isno potential negative societal impact.(d)
Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Section5.2, 5.3 and 5.4.(b)
Did you include complete proofs of all theoretical results?
[Yes] See Section 5.2, 5.3and 5.4.3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] See Section5.1.(b)
Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Section 5.1.(c)
Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] See Section 5.1(d)
Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Section 5.1.4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a)
If your work uses existing assets, did you cite the creators?
[Yes] See Section 5.1(b)
Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]
AppendixA Case study
We select a 3p query from NELL-995, and the reasoning process can be visualized with the intermediate symbolic representations.
We use the vector representations to select entities with highprobability as the right answer.
Compared with MRR results, selecting some entities as right answersand the regarding other entities as wrong answers is not comprehensive.
We do this experiment justfor visualization and the threshold we use here is one-tenth of the largest element of the vector for thecorresponding variables.
The results are showed in Table 3.
We list the entities with probability overthe threshold for each variable, and the entities with correctness mark are the true answer.
Note someentities without the mark could also be right since the KG is incomplete.
The predicted right answernumber and the number of all answers are marked as P/A.
[Table]
B Dataset Statistics
We list the statistic of the datasets used in our experiment in Table 4 and the number of queries in
Table 5.
[Table]
[Table]
3i 2in/3in/inp/pin/pni 1p others 1p others
FB15K149, 14, 20, 5, 22, 5,
NELL107, 10, 16, 4, 17, 4,