SRM - LLM : Semantic Relationship Mining with LLM s for Temporal Knowledge Graph Extrapolation (2025)

Abstract: Temporal knowledge graph (TKG) extrapolation aims to predict future facts by modeling the dynamic evolution of historical facts within TKGs. Existing methods often neglect the complex semantic relationships between relations when modeling their dynamic evolution, leading to incomplete relation representations and affecting the accuracy of reasoning. Inspired by the advancements in large language models (LLMs), we propose Semantic Relationship Mining based on LLMs (SRM-LLM), a novel approach for extracting semantic relationships to achieve TKG extrapolation. By leveraging LLMs to analyze the types of relations, we first identify several common relation types (e.g., causal, synonymous) in TKGs. We then design the LLM-based prompting strategy to capture latent semantic connections between relations, enabling the construction of relational association subgraphs for relation representation learning. In addition, SRM-LLM further enhances reasoning capabilities by incorporating structured logical constraints to guide inference. Experiments on five TKG datasets show significant performance gains and achieve new state of the art (SOTA) results, confirming the effectiveness of our method on TKG extrapolation tasks.

## introduction a knowledge graph (kg) is a semantic network that structurally represents facts about the real world using entities and relations in the form of triples (s, r, o) @xcite @xcite . however, most facts in the real world are not static and change over time @xcite . as for the fact "barack obama is the president of usa", it only holds true from 2009 to 2017. to capture and represent the evolution of facts over time, the temporal knowledge graph (tkg) is designed. with temporal information, facts in tkgs are typically represented as quadruples (s, r, o, t), where the timestamp t denotes the specific time at which the fact occurs. reasoning tasks in tkgs can be classified into two types: interpolation reasoning and extrapolation reasoning @xcite . interpolation reasoning aims to infer missing facts within the known timestamps, while extrapolation reasoning attempts to predict potential unknown facts that may occur in the future timestamps. the extrapolation reasoning is more challenging and has practical value in scenarios such as event prediction @xcite and stock forecasting @xcite . therefore, in this work, we focus on the extrapolation reasoning. to achieve accurate tkg reasoning, it is essential to construct models capable of fully capturing temporal dynamics and the complexity of relations. in recent years, numerous studies have focused on improving entity representations to better capture the evolutionary patterns of entities. for example, re-gcn @xcite leverages recursive modeling of kg sequences to learn evolutionary representations of entities. however, most studies overlook the rich latent information within relations. when aggregating relations, they primarily focus on directly adjacent entities, resulting in incomplete relation representations. as bridges connecting entities, relations encapsulate abundant semantic information and dynamic evolutionary patterns. taking figure @xcite as an example, when predicting the query (nigeria, release property, ?, t 3 ), the causal relationships between the relation "express intent to release persons or property" and the relations "release persons" and "release property" enable us to accurately predict that the correct entity is @xcite . for learning information between relations, two recent studies @xcite extend the concept of line graph @xcite to the domain of tkg reasoning. as shown in figure @xcite , for the subgraph at a given timestamp (the white part), if two edges are adjacent, their corresponding relation nodes in the line graph (the blue part) will be connected by an edge. these two methods learn information between relations by generating the relational correlation graph. however, the relational correlation graph obtained through this approach may contain isolated relation nodes that cannot be effectively connected to other relation nodes. this isolation hinders the propagation and aggregation of information within the graph, thus affecting the learning of relation representations. to address the aforementioned issues, we propose a novel method, srm-llm, for learning information between relations in tkg reasoning. specifically, for each dataset, we first leverage llms to analyze the latent association types between relations it contains (e.g., causal, synonymous). we then design the llm-based prompting strategy to mine the latent semantic connections among relations within each subgraph, enabling the construction of relational association subgraphs for relation representation learning. unlike methods that convert the graph into a line graph, our approach avoids the emergence of isolated relation nodes, thereby facilitating more accurate relation representation learning. in addition, inspired by tlogic @xcite , we propose a rule-based historical relation retrieval module, which retrieves historical quadruples related to the query relation by applying logical rules to historical subgraphs. this facilitates the construction of a global graph for learning entity representations at a global level. finally, we combine these global entity representations with local entity representations to perform prediction. experimental results clearly demonstrate the effectiveness of our method. our main contributions are summarized as follows: ## methodology in this section, we propose a novel llm-based and logical constraint method, srm-llm, which is aimed at mining semantic associations between relations to achieve tkg extrapolation tasks. the overall framework of our model is illustrated in figure 2 . in the following, we provide a detailed description of the model components. ## experiments in this section, we conduct a series of experiments on five tkg datasets to demonstrate the effectiveness of our model. ## conclusion in this paper, we propose a novel llm-based tkg reasoning method, srm-llm. we leverage the powerful semantic understanding capability of llms to identify the types of semantic association between relations (e.g., causal and synonymous), and construct the relational association graph to enhance relation representations. the design of this module avoids the emergence of isolated relation nodes, optimizing the learning process of relation representations. in addition, we introduce a rulebased historical relation retrieval module, which extracts globally relevant historical facts related to the query relations through temporal logical rules, learning entity representations at a global level. experimental results show that srm-llm significantly outperforms existing methods on multiple benchmark datasets, demonstrating its effectiveness in tkg extrapolation tasks.