CNEQ : Incorporating numbers into Knowledge Graph Reasoning (2024)

Abstract: Complex logical reasoning over knowledge graphs lies at the heart of many semantic downstream applications and thus has been extensively explored in recent years. However, nearly all of them overlook the rich semantics of numerical entities (e.g., magnitude, unit, and distribution) and are simply treated as common entities, or even directly removed. It may severely hinder the performance of answering queries involving numerical comparison or numerical computation. To address this issue, we propose the C omplex N umber and E ntity Q uery model (CNEQ), which comprises a Number-Entity Predictor and an Entity Filter. The Number-Entity Predictor can independently learn the structural and semantic features of entities and numerical values, thereby enabling better prediction of entities as well as numerical values. The Entity Filter can compare or calculate numerical values to filter out entities that meet certain numerical constraints. To evaluate our model, we generated a variety of multi-hop complex logical queries including numerical values on three widely-used Knowledge Graphs: FB15K, DB15K, and YAGO15K. Experimental results demonstrate that CNEQ achieves state-of-the-art results.

## introduction complex query answering (cqa), which has emerged as a significant task in recent years, and its primary objective is to perform reasoning over knowledge graphs (kgs) and to solve complex logical queries based on the structured knowledge. in cqa, natural language queries are initially converted into a subset of first-order logic (fol) queries, incorporating disjunctions (∨), conjunctions(∧), and existential(∃) quantifiers. for example, in figure 1 (a ), the natural language query "where did canadian citizens with turing award graduate?" is first con-figure 1: 4 types of complex queries: (a) represents complex queries with multi-entity answers; (b) represents complex queries with multiple numerical value answers; (c) represents complex queries with numerical statistic answers; (d) represents complex queries containing numerical constraints. however, complex queries involving numerical values come in various types, making the problem more difficult. for example, the query in figure 1 (b) has answers consisting of multiple numerical values. to answer such queries, the model needs to have both multi-hop reasoning capabilities and strong modeling abilities for attribute values. some link prediction models based on regression methods @xcite @xcite do consider attribute values, but they are limited as they can only perform one-hop predictions. additionally, the query in figure 1 (c ) requires answering mean of the entire set of numerical answers. this necessitates the model to learn features such as the distribution and range of all numerical answer sets, so it is insufficient to merely learn the magnitude of individual numbers. some complex queries also include numerical constraints, such as the query in figure 1 (d), where "larger than" is the constraint and "the average area of american cities" is the value of the numerical constraint. to answer such complex queries, the model needs to predict statistic (eg: average area), predict individual entity attributes (eg: city's area), and compare numerical values to filter out entities that meet the constraints. therefore, to improve complex query tasks, we propose cneq, a framework designed for solving various types of queries including numerical values and entities, as illustrated in figure 1 . cneq comprises two modules: a number-entity predictor and an entity filter. the number-entity predictor not only uses a more reasonable method to encode continuous numerical values, thereby learning the semantic information of the numbers, such as magnitude and units. it also models the range and distribution of numerical answers sets, to capture more macro-level features. the entity filter can predict the attribute values of entities and filter out those that meet the conditions based on the constraints. to evaluate the performance of cneq in answering complex queries, we constructed a benchmark dataset for validation. ultimately, our model achieved state-of-the-art results on the dataset. further experimental results indicate that our approach to modeling numerical values is necessary. our contributions mainly include: (1) we propose a method called cneq to solve complex queries which explicitly models the semantic information of numerical values, and outperforms previous models in various reasoning tasks involving both numbers and entities. (2) based on the modeling of numerical distributions, cneq can be used for statistics query answering. (3) we introduce a new task called "complex queries with numerical constraints" and solve such problems using cneq. (4) we create new benchmarks on three knowledge graphs-freebase, dbpedia, and yago-to evaluate the model's performance in answering the aforementioned types of queries. ## background and preliminaries a knowledge graph (or heterogeneous networks) can be define as @xmath0, r, t)} ⊂ (v × r × v) ∪ (v ×a×r) ,where @xmath1, r and a denote entity relation set and attribution set respectively. @xmath2: v × @xmath3, f alse} and @xmath4: v × @xmath5, f alse} can be seen as binary predicates, indicating whether the relation or attribution holds between the two elements. for an arbitrary existential positive first-order (epfo) logical queries, it can be represented by ∨,∧, and ∃. there is a concrete example in figure 1(b ), the natural language question what are the areas of american cities? can be represented as (1 ) in a knowledge graph containing numerical values, we mainly discuss three types of complex queries: (1) queries where the result is multiple entities or multiple numerical values, as shown in figure 1(a) and (b); (2) queries where the result is a statistic of all numerical answers, as shown in figure 1 (c); (3) queries where the result is a set of entities that meet a certain numerical constraint, as shown in figure 1(d ). next, we will formally define each type of the logical queries. ## complex number and entity query model to answer complex queries that include numerical values and entities as discussed in section 2, we propose cneq to tackle such complex queries, which primarily consists of two components: the number-entity predictor and the entity filter. the number-entity predictor is responsible for reasoning numerical values, entities, or statistic of certain sub-query. for example, to answer the query illustrated in figure 2 (a), the number-entity predictor is used to resolve the subquery in (1) to compute the number of numerical constraints. after that, the entity filter is employed to resolve the subquery in (2), filtering out the entities on the knowledge graph that satisfy the constraint. in the following parts, we will provide a detailed explanation of them. ## experiments in this section, we will demonstrate cneq's effectiveness to answer complex logical queries containing numerical values. following the settings of @xcite , our evaluation primarily focuses on queries in incomplete knowledge graphs. we recommend readers to refer to appendix a for understanding the evaluation protocol. ## conclusion we propose cneq, designed to address complex queries involving numerical values. additionally, cneq effectively handles complex queries related to statistics and queries with numerical constraints. we also construct benchmarks for various types of complex queries, which facilitates further research on complex queries over knowledge graphs. our results demonstrate that cneq outperformes several query embedding models, and achieves sota performance.