SC on E : Simplified Cone Embeddings with Symbolic Operators for Complex Logical Queries (2023)

Abstract: Geometric representation of query embeddings (using points, particles, rectangles and cones) can effectively achieve the task of answering complex logical queries expressed in first-order logic (FOL) form over knowledge graphs, allowing intuitive encodings. However, current geometric-based methods depend on the neural approach to model FOL operators (conjunction, disjunction and negation), which are not easily explainable with considerable computation cost. We overcome this challenge by introducing a symbolic modeling approach for the FOL operators, emphasizing the direct calculation of the intersection between geometric shapes, particularly sector-cones in the embedding space, to model the conjunction operator. This approach reduces the computation cost as a non-neural approach is involved in the core logic operators. Moreover, we propose to accelerate the learning in the relation projection operator using the neural approach to emphasize the essential role of this operator in all query structures. Although empirical evidence for explainability is challenging, our approach demonstrates a significant improvement in answering complex logical queries (both non-negative and negative FOL forms) over previous geometric-based models.

## introduction answering complex logical queries is a fundamental task of knowledge graphs (kgs) @xcite @xcite @xcite for various purposes of individuals and businesses. conventional methods, such as @xcite ; @xcite , have been well-studied on complete kgs. however, these methods face challenges in incomplete and largely-scaled kgs, as conventional methods cannot traverse graphs via missing connections. time complexity is another challenge as it grows exponentially during the traversal process. modern approaches, such as @xcite ; @xcite , use query embeddings (qes) methods that can answer complex logical queries without the need of path traversal in graphs. the qes methods first transform a complex logical query into a machine-readable format: (1) converting a query in natural textual form into first-order logic (fol) form (including conjunction ∧, disjunction ∨, negation ¬ and existential quantification ∃ operator) and (2) decomposing it into a computation graph (including relation projection operator). for example, fig. 1 depicts the process of turning a complex logical query "which universities do the nobel prize winners of australian citizens work in?" into a computation graph. this fol query is then projected in the embedding space, required for the modeling process to learn to answer the query. among different approaches in representing queries in the embedding space, geometric-based approaches have had renewed interests since the work in point embeddings @xcite . following works have expanded this approach using hyper-boxes @xcite , sets of points as particles @xcite , hyperboloids @xcite and 2d-cones @xcite . these works commonly resort to set operators over shapes that can handle the conjunction, only a few @xcite can handle the negation. nevertheless, existing geometric-based methods depend on the neural approach to model the conjunction operator. this approach is not easily explainable, counter-intuitive and does not take full advantage of the properties that these geometric representations are intended to be used for. we highlight in this paper the essential role the projection operator plays in all complex query embedding methods. this operator is often learned through training neural architectures together with the logical operators in an end-to-end fashion. the semantic role of this operator is to obtain a meaningful representation of a predicate (relation) in fol, which operates as a function converting a domain of input embeddings into a range of outputs. these are fundamentally different from the roles of the logical operators, in which geometric approaches can be modelled as set operators. moreover, little work highlights the importance of the neural approach in learning the relation projection. we address the first issue by introducing a novel symbolic operator in geometry for answering the conjunctive queries. specifically, we directly calculate the intersection of geometric shapes in the embedding space (see an example in fig. 2 and more details in sec. 4.2). we use cone embeddings @xcite as a key geometric representation in our approach, since conic shapes were shown to be effective in modeling all fol operators. by directly calculating intersection, our approach can reduce the computational cost of modeling the conjunction operator, as there is no need to incorporate expensive neural training in this logic operator, as compared with other geometric-based models @xcite @xcite . further, classifying types of geometric intersection (partial, complete and none type) can improve the explainability in modeling conjunction operator (see sec 4.2). to highlight the importance of relation projection (finding tail entities from a source entity via a relation) in complex logical query embeddings, we propose a general framework of modeling this operator, called relation projection network (rpn) (see fig. 3 ). the rpn can enhance the learning in the relation projection operation, due to its high frequency and its dominance in diverse query structures (see fig. 6 ). overall, we introduce simplified cone embeddings (scone) for modeling the relation projection and logical operators in complex queries. our contributions are: (1) introducing a symbolic modeling for the conjunction operator in fol query, (2) proposing a general framework using rpn to improve the learning of relation projection operator in both atomic and complex logical queries and (3) surpassing model performance of previous state-of-the-art geometric-based models for both non-negation and negation queries. ## modeling operators of fol queries we describe the modeling of relation projection (4.1) and logical operators (4.2) in a complex fol query (q) (with its set of answer entities v q ⊂ v) over knowledge graphs in the following: ## conclusions we have provided a symbolic modeling for logical operators, particularly computing geometric intersection of sector-cones for modeling the conjunction. in addition, we highlighted the importance of the projection operator by introducing a relation projection network using neural-based approach, to strengthen the learning in atomic queries involved in all fol query syntaxes. our neural-symbolic approach using geometric embeddings significantly outperforms state-of-the-art geometric-based models in both epfo and negation queries.