G 2 S : A General-to-Specific Learning Framework for Temporal Knowledge Graph Forecasting with Large Language Models (2025)

Abstract: Forecasting over Temporal Knowledge Graphs (TKGs) which predicts future facts based on historical ones has received much attention. Recent studies have introduced Large Language Models (LLMs) for this task to enhance the models’ generalization abilities. However, these models perform forecasting via simultaneously learning two kinds of entangled knowledge in the TKG: (1) general patterns, i.e., invariant temporal structures shared across different scenarios; and (2) scenario information, i.e., factual knowledge engaged in specific scenario, such as entities and relations. As a result, the learning processes of these two kinds of knowledge may interfere with each other, which potentially impact the generalization abilities of the models. To enhance the generalization ability of LLMs on this task, in this paper, we propose a General-to-Specific learning framework (G2S) that disentangles the learning processes of the above two kinds of knowledge. In the general learning stage, we mask the scenario information in different TKGs and convert it into anonymous temporal structures. After training on these structures, the model is able to capture the general patterns across different TKGs. In the specific learning stage, we inject the scenario information into the structures via either in-context learning or fine-tuning modes. Experimental results show that G2S effectively improves the generalization abilities of LLMs.

## introduction temporal knowledge graph (tkg), which consists of facts in the form of @xcite , is often used to represent the changes of facts over time. forecasting future facts based on historical ones in tkg is critical for many timesensitive scenarios, such as finance @xcite , healthcare @xcite , and poli- entities: 1→iran, 2→obama, 3→iraq, 4→afghanistan relations: 1→make statement, 2→provide aid ## related work there are two different task settings for tkg reasoning, namely, interpolation and extrapolation settings. tkg reasoning under interpolation setting, also known as tkg completion, aims to predict the missing entities at past timestamps @xcite , while tkg reasoning under extrapolation setting, also known as tkg forecasting, aims to predict the missing entities at future timestamps @xcite . this paper focuses on the latter one. traditional tkg forecasting methods can be categorized into three different types: (1) neuralbased methods utilize deep neural networks to model the historical facts. re-net @xcite develops a recurrent neural network to model query related historical facts. re-gcn @xcite utilizes graph convolutional networks to model recent kg snapshots. tango @xcite adopts neural ordinary differential equations to model the structure information among entities. xerte @xcite designs a subgraph expansion and pruning mechanism to find explainable supports for forecasting. (2) reinforcement learning-based methods utilize reinforcement learning to find a path between query entity and answer entity, such as cluster @xcite and titer @xcite . (3) rulebased methods mine logical rules to derive the answer from historical facts, such as tlogic @xcite . these methods mainly focus on enhancing the performance of the models on each specific tkg and overlook the generalization ability of the models. recently, some studies have introduced llm for tkg forecasting, which can be categorized into in-context learning and supervised fine-tuning methods. @xcite propose an in-context learning method, which provides a basic input format to fit tkg data to llms. @xcite investigate the llm's ability to find useful historical facts and reasoning answers based on them. gentkg @xcite proposes to select historical facts via rules and trains llms to answer the query via supervised fine-tuning. these methods show generalization abilities on tkg forecasting to some extent. however, as analyzed in section 1, their learning processes of general patterns and scenario information may interfere with each other, which limits their generalization abilities. ## problem formulation a tkg g is formalized as a sequence of kg snapshots from timestamps 1 to t , i.e., each kg snapshot g @xmath0, r, f t } consists of the entity set e, relation set r, and the fact set f t at timestamp @xmath1, ..., t }. each fact f @xmath2, r i , o i , t), where s i , o @xmath3; r @xmath4. currently, the tkg forecasting task focuses on predicting the missing object entity in a query @xmath5, r q , ?, t q ). the model m is required to score possible object entities based on the query and historical tkg snapshots: score(o) = m(q, @xmath6. note that, when predicting the missing subject entity, the query is formulated as @xmath7, r q , o q , t q ). ## methodology to enhance the generalization ability of llms on tkg forecasting, we propose g2s that disentangles the learning process of the general patterns and specific information into two stages, namely, the general learning and specific learning stages. figure 2 shows the overall diagram of g2s. we will describe the design of these two learning stages and provide other training and inference details. ## results and analyses 6.1 standard setting to evaluate the performance of g2s when sufficient training data are available, we conduct experiments under standard settings, where all training model dataset icews14 icews18 yago h@1 h@3 h@10 h@1 h@3 h@10 h@1 h@3 h@10 data can be used to fine-tune the model in the specific learning stage. the experimental results are shown in from the table, we can observe that: