Temporal Fact Reasoning over Hyper-Relational Knowledge Graphs (2024)

Abstract: Stemming from traditional knowledge graphs (KGs), hyper-relational KGs (HKGs) provide additional key-value pairs (i.e., qualifiers) for each KG fact that help to better restrict the fact validity. In recent years, there has been an increasing interest in studying graph reasoning over HKGs. Meanwhile, as discussed in recent works that focus on temporal KGs (TKGs), world knowledge is ever-evolving, making it important to reason over temporal facts in KGs. Previous mainstream benchmark HKGs do not explicitly specify temporal information for each HKG fact. Therefore, almost all existing HKG reasoning approaches do not devise any module specifically for temporal reasoning. To better study temporal fact reasoning over HKGs, we propose a new type of data structure named hyper-relational TKG (HTKG). Every fact in an HTKG is coupled with a timestamp explicitly indicating its time validity. We develop two new benchmark HTKG datasets, i.e., Wiki-hy and YAGO-hy, and propose an HTKG reasoning model that efficiently models hyper-relational temporal facts. To support future research on this topic, we open-source our datasets and model.

## introduction traditional knowledge graphs (kgs) represent world knowledge by storing a collection of facts in the form of triples. each kg fact can be described as (s, r, o), where s, o are the subject and object entities of the fact and r denotes the relation between them. on top of traditional triple-based kgs, hyperrelational kgs (hkgs) are designed to introduce additional information into each triple-based fact (also known as primary triple in hkgs) by incorporating a number of key-value restrictions named as qualifiers @xcite @xcite figure 1 : examples of hkg (a) and htkg (b) facts. contents inside dashed line squares denote qualifiers. we also provide another example of htkg fact showcasing diverse sets of qualifiers in app. a. @xcite . compared with triple-based kgs, hkgs provide more complicated semantics. for example, in fig. 1 (a) , the degree and major information of albert einstein is provided, which helps to differentiate between the facts regarding two universities attended by him. many reasoning approaches have been proposed for hkgs, e.g., @xcite , but unfortunately, they all assume that the hyper-relational facts are static. as discussed in recent works @xcite , world knowledge is ever-evolving. in temporal kgs, each fact is represented by a quadruple (s, r, o, t) with an additional timestamp specifying the time validity. previous mainstream hkg benchmarks do not explicitly specify time validity for each hkg fact. this hinders the development of the reasoning systems that can effectively handle temporal dynamics within hyper-relational facts, and as a result, almost all existing hkg reasoning methods lack a dedicated module for temporal reasoning. modeling temporal knowledge in hkgs is important as the temporal validity of a fact improves knowledge expressiveness and might be correlative to its qualifiers. a model should be expressive enough to model such correlation. to better study temporal fact reasoning over hkgs, we propose a new type of data structure named hyper-relational tkg (htkg, see formal definition in sec. 2.1). every fact in an htkg is defined in the form of @xcite , {(r q i , e q i )} n @xmath0. (s, r, o, t) is its primary quadruple (i.e., primary fact, t is a timestamp denoting the valid time) and {(r q i , e q i )} n @xmath1. we illustrate an htkg fact example in fig. 1 (b ). the two awards ang lee was nominated for because of brokeback mountain can be differentiated considering the specified timestamps. an htkg is composed solely of a collection of hyperrelational temporal facts so we use htkgs to study temporal fact reasoning over hkgs. we construct two benchmark htkgs wiki-hy and yago-hy based on two traditional tkg benchmarks wiki-data11k @xcite and yago1830 @xcite . since previous hkg reasoning approaches pay little attention to temporal reasoning, they are not fit for modeling htkgs. to this end, we develop a model to achieve link prediction (lp) over hyperrelational tkgs (hypetkg) as follows: (1) we first devise a qualifier-attentional time-aware graph encoder (qatge) that considers both temporal information and qualifiers in the graph aggregation process. (2) we then design a qualifier matching decoder (qmd). given any htkg lp query, qmd not only considers its own qualifiers, but also models all the qualifiers appearing in query subjectrelated facts foot_0 . the motivation of qmd is that the evidence for lp not only is stored in the query qualifiers but also can be found in other subjectrelated facts. compared with previous methods, hypetkg is able to capture the correlation between temporal validity and qualifiers. another point worth noting is that some recent works have started to explore whether timeinvariant (ti) relational knowledge foot_1 can help to enhance temporal fact reasoning on traditional tkgs @xcite @xcite . this arouses our interest in studying whether ti relational facts are beneficial in htkg reasoning. in our work, we mine the ti relational knowledge from the wikidata kb. we pick out the facts that contain ten frequently mentioned ti relations, e.g., official lan-guage, and ensure that these facts remain valid within the whole time scopes of htkgs. we adjust hypetkg and create a model variant hypetkg ψ that dynamically controls the influence of ti information for better reasoning on temporal facts. we also provide a wide range of baselines with ti facts and benchmark their temporal fact lp performance on our proposed htkgs. to summarize, our contribution is three-folded: (1) we propose a new data structure htkg that draws attention to temporal fact reasoning over hkgs and propose two corresponding benchmarks (sec. 2.1 and 3). (2) we propose hypetkg, a model specifically designed to reason over htkgs. experimental results show that hypetkg performs well in temporal fact reasoning over htkgs (sec. 5.2). (3) we study the influence of ti relational knowledge on htkg reasoning and adapt hy-petkg to accommodate to ti information. we show that our model can benefit by carefully balancing the information between temporal and ti knowledge (sec. 5.3). ## proposing new benchmarks we propose two htkg benchmark datasets wikihy and yago-hy. wiki-hy contains htkg facts extracted from wikidata @xcite , where they happen from year 1513 to 2020. yago-hy is constructed from the facts in yago3 @xcite and the time scope is from year 1830 to 2018. we use previous traditional tkg benchmarks wikidata11k @xcite and yago1830 @xcite as bases and search for the qualifiers of their facts in wikidata. we use the mediawiki api foot_3 to identify the quadruple-based tkg facts in wikidata and extract all the qualifiers stated under the corresponding wikidata statements. since wikidata11k is originally extracted from wikidata, we can directly find its relations and entities in this kb. yago1830's entities share the same pool as wikidata but relation types are taken from schema.org. we map yago1830's relations to wikidata's relations to enable fact matching (detailed mapping in app. b). we provide dataset statistics of both datasets in ## hypetkg hypetkg consists of two parts, i.e., a qualifierattentional time-aware graph encoder (qatge) and a qualifier matching decoder (qmd). to further learn from ti knowledge, we equip hypetkg with additional modules and develop a model variant hypetkg ψ (model structure shown in fig. 2 ). ## experiments we do htkg lp over wiki-hy and yago-hy. we report htkg lp results in sec. 5.2. we study whether additional ti knowledge helps htkg lp in sec. 5.3. we do ablation studies and study the impact of the ratio of utilized qualifiers in sec. 5.4. finally, we present several case studies to show the effectiveness of leveraging ti knowledge and qualifier matcher for temporal fact reasoning over htkgs in sec. 5.5. we provide complexity analysis of our model in app. d. we also study the impact of qualifier-augmented fact proportion and present it in app. i. ## conclusion in this work, we propose a new data structure named htkg for studying temporal fact reasoning over hkgs. to reason htkgs, we design a model hypetkg that is able to simultaneously deal with temporal information and qualifiers. we benchmark hypetkg and various previ-ous hkg/tkg reasoning methods on two newlyconstructed datasets, i.e., wiki-hy and yago-hy. we show that hypetkg achieves superior performance on htkg lp. besides, we mine the ti relational knowledge from wikidata kb and study whether it can benefit models on hyper-relational temporal fact reasoning. we find that temporal fact reasoning on htkgs can be enhanced by carefully balancing the information between temporal and ti knowledge.