SlowGuess/NAACL_2021_OCR huggingface dataset:
Youri Xu, Haihong E*, Meina Song, wenyu song, Xiaodong Lv, wang haotian, yang jinrui
School of Computer Science, Beijing University of Posts and Telecommunications, China
{youri.xu,ehaihong,mnsong,swy9834,lvxiaodong,haotianwang,yangjinrui} @bupt.edu.cn
Static knowledge graph (SKG) embedding (SKGE) has been studied intensively in the past years. Recently, temporal knowledge graph (TKG) embedding (TKGE) has emerged. In this paper, we propose a Recursive Temporal Fact Embedding (RTFE) framework to transplant SKGE models to TKGs and to enhance the performance of existing TKGE models for TKG completion. Different from previous work which ignores the continuity of states of TKG in time evolution, we treat the sequence of graphs as a Markov chain, which transitions from the previous state to the next state. RTFE takes the SKGE to initialize the embeddings of TKG. Then it recursively tracks the state transition of TKG by passing updated parameters/features between timestamps. Specifically, at each timestamp, we approximate the state transition as the gradient update process. Since RTFE learns each timestamp recursively, it can naturally transit to future timestamps. Experiments on five TKG datasets show the effectiveness of RTFE.
Temporal knowledge graph (TKG) is an extension of static knowledge graphs (SKGs) which introduce the time dimension. In SKGs, facts are considered to be time-invariant (Sil and Cucerzan, 2014). In reality, facts are not always true. For example, the triple (Obama, President, United States) was true only from 2009 to 2016 and (Obama, married, Mitchell) since 1992. However, SKGs do not reflect the change in facts over time. An example of TKG is shown in Figure 1. Besides, facts on social networks, e-commerce platforms and trading platforms also change over time. Therefore, TKGs have the potential to improve the performance of question answering, search, recommendation and prediction based on KGs (Huang et al., 2020; Garg et al., 2020).
TKG can be expressed as a set of quadruples (subject, relation, object, timestamp). Different from SKGs which ignore the time attribute of facts, the facts of TKGs are distributed in timestamps, which can reflect the dynamic change of entities and relationships over time. Due to the limited coverage of KGs, TKGs are also incomplete. By completing TKG, missing and potential knowledge under specific timestamps can be found.
In recent years, a lot of work (Bordes et al., 2013; Wang et al., 2014; Lin et al., 2015; Kazemi and Poole, 2018; Schlichtkrull et al., 2018; Sun et al., 2019; Zhang et al., 2020) has focused on KG completion by methods of graph embedding. These efforts have yielded good results, but most of them focused on SKGs and required training in a large number of triples.
However, the TKG under a certain timestamp is a sparse multi-relation graph (Esteban et al., 2016), so it is necessary to absorb information from other timestamps. What's more, SKGE methods lacked the modeling of time attribute of relations, and were proposed based on the assumption that all facts occur at the same time. So they cannot reflect the temporal dependencies of facts. To handle these two problems, our RTFE passes parameters and features between timestamps in a recursive manner, which not only alleviates the sparsity problem of TKG, but takes advantage of the continuity and relevance characteristics of the fact as well.
Existing TKG completion methods (Dasgupta et al., 2018; Goel et al., 2019; Lacroix et al., 2020) follow the training pattern of SKGE, which shuffles facts  $(s, r, o, t)$  of different time randomly and learns all facts in a chaotic temporal order by minibatch gradient descent algorithm. In other words, they just take time as a parameter but ignore the correlations in time evolution.
However, the early state may affect the later one and later facts tend to be dependent on early ones. In particular, the state at  $t_{i}$  directly influences that
*Corresponding author: Haihong E
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5671-5681 June 6-11, 2021. ©2021 Association for Computational Linguistics
Figure 1: A toy example of TKG where solid edges represent observed edges and red edges represent new facts that occurred at that timestamp. Besides, dotted edges represent missing or potential facts.
at  $t_{i+1}$ . E.g., (Obama, Campaign, President, 2008) directly influences (Obama, Inaugurated as, President, 2009). It has been verified that the chronological order of events can be used to improve the performance of link prediction (Jiang et al., 2016a; Jiang et al., 2016b). Based on this, we further find that the early training state can improve later one if we train facts in their chronological order. In order to capture changes in TKG's state transition, we think TKG as a sequence of dynamic graphs, not as a whole graph labeled with time information.
Besides, since new facts of future timestamps can be added to TKG, TKG is expanding dynamically. And the graphs of new timestamps may still be incomplete. However, existing TKG completion methods provide no solution to complete unseen future graphs. In their training pattern, facts of all timestamps are trained jointly to complete the graphs that have appeared. Models may need to be retrained on facts of all timestamps when a new timestamp appears. In contrast, our RTFE embeds and completes TKG's each timestamp in a recursive way. By using the information of previous timestamps, RTFE can be naturally extended to future timestamps during the state transition of parameters/features. RTFE only needs to be trained on new emerging facts, which is light and immediate.
SKGE has been studied for many years while TKGE is still at birth. Problems encountered in SKGE (e.g., diverse relation patterns) can also occur in TKGE. Thus the advantages of SKG researches can be used to accelerate the development of TKGs if we bridge the gap between them. Our RTFE provides a way to migrate SKGE methods to TKGs while preserving their excellent effects.
Further, existing TKG completion methods de
signed specifically for the characteristics of TKGs can also be enhanced using the training pattern of RTFE. To sum up, we have made the following contributions:
1. We propose a training pattern to bridge the gap between SKGE and TKGE. Therefore, state-of-the-art SKGE models can be used to accelerate the development of TKGE.
2. Existing TKGE models can be further enhanced with our framework RTFE, after finishing their own regular training.
3. To the best of our knowledge, we are the first to deal with the TKG evolution problem (i.e., new future timestamps are added to TKGs) in the TKG completion task.
4. The experimental results on 5 TKG datasets show that RTFE preserve SKGE models' excellent performance. And the predictive performance of state-of-the-art TKGE models are further enhanced using RTFE.
A temporal knowledge graph (TKG) can be represented as a sequence of graphs, i.e.  $G = \{G_{t_1},\ldots ,G_{t_n}\}$  where  $G_{t_i}$  is a set of quadruples that occurred at timestamp  $t_i$ , i.e.  $G_{t_i} = \{(s,r,o,t_i)\}$  where  $V$  is the set of  $G$ 's entities and  $s,o\in V;R$  is the set of  $G$ 's relations and  $r\in R$ .
We focus on the following task: given a training TKG  $G_{train} = \{G_{t_1},\dots ,G_{t_n}\}$ , to infer the missing quadruples  $(s,r,o,t)$  in test set  $G_{test} = \{G_{t_1}',\ldots ,G_{t_n}'\}$  (i.e., assign high scores to true quadruples and low scores to false ones). As shown in Figure 1, missing facts with high probability are dotted.
Figure 2: The framework of RTFE. TKG is first transformed to SKG  $G_{\text{static}}$ . RTFE pre-train embedding learner  $\theta$  on  $G_{\text{static}}$  to obtain the input of the first timestamp. Then features and parameters are recursively passed to next timestamp after learning the current timestamp.
3 Recursive Temporal Fact Embedding (RTFE) Framework
The state of TKGs change with the change of entities and relations over time. SKGE models fail to capture correlations during state transition. And existing TKGE models for TKG completion capture it implicitly. It can be observed that the TKG after the current change changes on the closest former state, which is similar to a first order Markov chain (Given the state at the current moment, the state at the next moment is independent of the state at the past moment).
Inspired by Markov analysis, we use the time granularity of TKGs to discretely divide states. Then the basic model of RTFE can be expressed as:
$$
S _ {t _ {i + 1}} = S _ {t _ {i}} \cdot P _ {t _ {i}} \tag {1}
$$
where  $S_{t_i}$  represents the state of  $G_{t_i}$  and  $P_{t_i}$  represents probability transition matrix to transform  $S_{t_i}$  to that of  $t_{i + 1}$ . A typical KG embedding learner uses its parameters  $\theta$  and features  $X$  to represent the semantic information of KG. Thus we approximate state vectors as:
$$
S _ {t _ {i}} := \left[ \theta_ {t _ {i}}, X _ {t _ {i}} \right] \tag {2}
$$
The idea of RTFE is to dynamically adjust  $\theta$  and  $X$  as the TKG changes while passing the information of each timestamp graph. We simply assume the features and parameters satisfy Markov Property:
$$
\begin{array}{l} P \left(X _ {t _ {i + 1}}, \theta_ {t _ {i + 1}} \mid X _ {t _ {1}}, \dots , X _ {t _ {i}}; \theta_ {t _ {1}}, \dots , \theta_ {t _ {i}}\right) \tag {3} \\ = P \left(X _ {t _ {i + 1}}, \theta_ {t _ {i + 1}} \mid X _ {t _ {i}}, \theta_ {t _ {i}}\right) \\ \end{array}
$$
where  $X_{t_i}$  and  $\theta_{t_i}$  denote features and parameters at time  $t_i$ .
RTFE does not specify a model, but rather a training method for TKG completion. Existing SKGE methods and TKGE methods that follow the SKGE training pattern such as DE-SimplE (Goel et al., 2019) and TComplEx (Lacroix et al., 2020) can potentially be utilized as the embedding component. The RTFE framework is illustrated in Figure 2. In section 3, we specify that RTFE how to use SKGE models for TKG completion. In section 4, we generalize RTFE to existing TKGE models to enhance their performance.
3.1 Preliminary training for static features
Instead of training from scratch, RTFE uses SKGE as input to the first timestamp. In order to obtain the input features, the TKG is transformed into SKG  $G_{\text{static}}$ , which is obtained by merging the facts of each timestamp:
$$
\begin{array}{l} G _ {s t a t i c}: = \bigcup_ {i = 1} ^ {n} G _ {t _ {i}} \\ = \left\{\left(s, r, o\right) \mid \left(s, r, o, t _ {i}\right) \in G _ {t _ {i}}, G _ {t _ {i}} \in G \right\} \tag {4} \\ \end{array}
$$
Suppose the SKG embedding learner be  $\theta$ , which takes the knowledge graph  $G$  (facts of  $G$ ) and the feature  $X$  (which can be predefined or randomly initialized) as inputs. Send  $G_{\text{static}}$  and  $X$  to  $\theta$ , and then get the updated feature  $X$  after training, which will be the input to the first timestamp.
3.2 Learning each timestamp recursively
In TKGs, parameters  $\theta$  and features  $X$  should change with time (i.e., with the change of TKG). We find that, due to the continuity of facts, most of the facts are the same in the adjacent timestamps, while only a small number of facts changed. For discrete events, they influence the states of the surrounding entities, leading to the possibility that these entities may produce new facts. Therefore, model parameters and features fitting a certain timestamp provide a good starting point for the learning of the next timestamp.
Different from most neural network-based SKGE models (Schlichtkrull et al., 2018; Wu et al., 2019) which only update  $\theta$  during training, leaving the input features  $X$  unchanged, we let  $X$  be updated as well, to capture the temporal dynamics of entities and relations.
Therefore, in our framework RTFE, model parameters  $\theta$  and input features  $X$  are both updated in the way similar to equation (1) during state transition:
$$
\left[ \theta_ {t _ {i + 1}}, X _ {t _ {i + 1}} \right] = \left[ \theta_ {t _ {i}}, X _ {t _ {i}} \right] \cdot P _ {t _ {i}} \tag {5}
$$
where  $\theta_{t_i}$  and  $X_{t_i}$  denote the state vectors of  $\theta$  and  $X$  at time  $t_i$  respectively;  $P_{t_i}$  represent the probability transition matrix. To transform state vectors at  $t_i$  to that at  $t_{i + 1}$ , we approximate the state transition  $P_{t_i}$  as the gradient update process of learning  $G_{t_i}$  (i.e., updating according to the gradient of the loss function for several epochs):
$$
\theta_ {t _ {i + 1}} = \theta_ {t _ {i}} - \alpha \cdot \nabla_ {\theta} l (\theta_ {t _ {i}}, X _ {t _ {i}}, G _ {t _ {i}}) (6)
$$
$$
X _ {t _ {i + 1}} = X _ {t _ {i}} - \alpha \cdot \nabla_ {X} l \left(\theta_ {t _ {i}}, X _ {t _ {i}}, G _ {t _ {i}}\right) \tag {7}
$$
where  $\alpha$  is the learning rate;  $l$  is the loss function defined by the specified embedding learner;  $\nabla_{\theta}$  is the gradient of  $l$  with respect to  $\theta$ .
It must be pointed out that the state transition matrix in Markov analysis is fixed, so the above analysis method is generally applicable to shortterm prediction. But the state vectors are different in different states and the gradient between the states is also different. Since the state vector is fixed in a specific state, a model can be established for each discrete state by the time interval of TKG. Then the gradient can be updated between states according to the difference of each state vector, to continue our framework.
RTFE recursively trains each timestamp according to equation (6) and (7) and uses  $\theta_{t_i}$  and  $X_{t_i}$  to test  $G_{t_i}^{\prime}$ . Since RTFE is trained and tested by timestamp, only the latest parameters and features need to be stored, which shows good scalability for large TKGs. The framework RTFE is illustrated in Figure 2 and the overall training and testing algorithm is shown in Algorithm 1.
4 Transplanting SKGE/TKGE models
4.1 SKGE models
For translation-based methods like TransE (Bordes et al., 2013), TransD (Ji et al., 2015), RotatE (Sun et al., 2019) and HAKE (Zhang et al., 2020), they can be directly used as the embedding learner of RTFE without change of models.
For Graph neural network-based methods like RGCN (Schlichtkrull et al., 2018), we make its input feature do gradient update as well, so that the input features encode the information of each timestamp, so as to enhance the information transfer between timestamps. In addition, a residual connection is added to the network between the network inputs and outputs of each timestamp.
For RDGCN (Wu et al., 2019) that was designed for entity alignment, in order to measure the plausibility of a triple  $(s, r; o)$  for SKG completion, we design a distance function consisting of type
distance and semantic distance:
$$
d (s, r, o) = d _ {t y p e} (s, r, o) + \lambda \cdot d _ {s e m a n} (s, r, o) \tag {8}
$$
$$
d _ {t y p e} (s, r, o) = | \left[ \overline {{X}} _ {s} ^ {E}, \overline {{X}} _ {o} ^ {E} \right] - \overline {{X}} _ {r} ^ {R} | \qquad (9)
$$
$$
\begin{array}{l} d _ {s e m a n} (s, r, o) = \left| \left(\left[ \overline {{X}} _ {s} ^ {E}, \overline {{X}} _ {o} ^ {E} \right] - \overline {{X}} _ {r} ^ {R}\right) [: d ] \right. \\ - \left(\left[ \bar {X} _ {s} ^ {E}, \bar {X} _ {o} ^ {E} \right] - \bar {X} _ {r} ^ {R}\right) [ d: ] | \tag {10} \\ \end{array}
$$
where  $\overline{X}^E\in \mathbb{R}^{|V|\times d}$  and  $\overline{X}^{R}\in \mathbb{R}^{|V|\times 2d}$  denotes output entity and relation representations.
4.2 Enhancing TKGE models
Since existing TKGE models such as DE-SimplE (Goel et al., 2019) and TComplEx (Lacroix et al., 2020) for TKG completion follow the training pattern of SKGE models (i.e., think of TKG as a whole graph, not as a sequence of graphs), we can use them as the embedding leaner of RTFE. Specifically, we think their own training process as the preliminary training of RTFE. After TKGE models finish their own training process, we use the obtained features and parameters as the input to the learning of the first timestamp. Then RTFE trains the TKGE model recursively by equation (6) and equation (7).
4.3 Extensibility for future timestamps
Since RTFE embeds each timestamp recursively, transforming form current state to the next state, it provides a way to complete upcoming future timestamps. Specifically, given a sequence of observed graphs of a TKG:  $G_{obs} = \{G_{t_1},\dots ,G_{t_n}\}$  and a sequence of upcoming future graphs:  $G_{fut} = \{G_{t_{n + 1}},\ldots ,G_{t_{n + j}}\}$ . We pre-train RTFE on  $G_{obs}$ , then embeds timestamp recursively to obtain the latest features  $X_{t_n}$  and parameters  $\theta_{t_n}$ . To complete  $G_{t_{n + 1}}$ , we use equation (6) to equation (7) to obtain  $X_{t_{n + 1}}$  and  $\theta_{t_{n + 1}}$  similarly. Then graphs of  $G_{fut}$  can also be completed in this recursive way, without retraining of  $G_{obs}$ .
Dataset: We evaluated models on two fact datasets proposed by Hyte (Dasgupta et al., 2018): YAGO11k and Wikidata12k and three event datasets ICEWS14, ICEWS05-15 and GDELT
(Goel et al., 2019). The details of the five datasets are illustrated in appendix.
Evaluation settings and metrics: For entity prediction, we used mean reciprocal rank (MRR) and Hits@1, Hits@3, Hits@10 as metrics. Hits@n is defined as:
$$
H i t s @ n = \frac {\sum_ {f a c t \in t e s t \_ s e t} b o o l (r a n k (f a c t) \leq n)}{| t e s t \_ s e t |}
$$
For relation prediction, we used mean rank  $(MR) = \frac{\sum_{\text{fact} \in \text{test\_set}} \text{rank}(\text{fact}) \leq n}{\# \text{test\_set}}$  and  $Hits@1$  as metrics since the number of relations is small. The rank of a test triple is obtained by replacing its head/tail/relation with remaining negative samples, and then evaluating the score rank of the original triple in all the replacement samples. Mean rank  $(MR)$  is the average rank of all test triples. And Mean reciprocal rank  $(MRR)$  is the average of the reciprocal ranks.
For our RTFE framework, a timestamp-by-timestamp train-test mode was adopted. The total test result was a weighted average of all timestamp test results. For example, the final  $MRR$  was calculated as:
$$
M R R _ {a l l} = \frac {\sum_ {i} \left| G _ {t _ {i}} \right| \times M R R _ {G _ {t _ {i}}}}{\sum_ {i} \left| G _ {t _ {i}} \right|}
$$
Baselines: We compared our framework RTFE to state-of-the-art TKGE models including t-TransE (Jiang et al., 2016a), Hyte (Dasgupta et al., 2018), DE-SimplE (Goel et al., 2019), ATiSE (Xu et al., 2019), TComplEx (Lacroix et al., 2020) for TKG completion. Then we used several state-of-the-art SKGE models including TransE (Bordes et al., 2013), TransD (Ji et al., 2015), RGCN (Schlichtkrull et al., 2018), RDGCN (Wu et al., 2019), RotatE (Sun et al., 2019) and HAKE (Zhang et al., 2020) as the embedding learner of RTFE to perform TKG completion as well. And finally we use TKGE models as the embedding learner of RTFE to show the gain of performance.
5.2 Entity prediction
Entity prediction is given a quadruple  $(s, r, o, t)$ , to perform head entity prediction (i.e., to predict the plausibility of  $(?, r, o, t)$ ), and performs tail entity prediction (i.e., to predict the plausibility of  $(s, r, ?, t)$ ). The plausibility of  $(s, r, o, t)$  is ranked among all corrupted quadruples, while all true quadruples are excluded according to TransE's filtering protocol. The experimental results are shown in Table 1 and
Dataset	Wikidata12k	YAGO11k
Metric	MRR tail head	Hits@1 tail head	Hits@3 tail head	Hits@10 tail head	MRR tail head	Hits@1 tail head	Hits@3 tail head	Hits@10 tail head
t-TransE*	17.2	9.6	18.4	32.9	10.8	2.0	15.0	25.1
Hyte	21.4 14.3	12.0 8.0	23.6 14.5	41.8 26.1	14.7 6.0	2.9 0.2	19.9 9.1	35.0 13.4
ATiSE*	28.0	17.5	31.7	48.1	17.0	11.0	17.1	28.8
TComplEx	44.9 29.5	29.9 21.2	55.8 31.8	67.4 47.6	32.4 18.0	22.7 13.5	35.7 17.1	52.2 27.1
RTFE-TransE	36.4 14.1	20.1 5.6	47.3 13.6	67.6 33.4	19.8 7.6	6.3 0.4	27.0 13.1	42.5 14.6
RTFE-TransD	33.0 7.2	18.1 2.8	45.1 8.0	63.1 18.5	18.9 9.6	3.7 0.5	26.5 14.7	46.2 22.1
RTFE-RGCN	27.6 15.6	20.8 8.6	30.8 17.4	41.2 31.9	20.3 14.8	16.4 12.5	21.2 14.5	28.1 19.0
RTFE-RDGCN	43.5 17.0	36.2 13.2	46.7 18.6	58.1 23.9	20.9 10.0	8.8 2.6	27.1 13.3	43.0 21.5
RTFE-RotatE	44.6 27.8	34.0 19.0	52.7 31.9	63.3 45.9	28.9 18.5	19.8 14.3	32.0 18.1	47.9 27.0
RTFE-HAKE	35.7 27.3	24.7 19.4	43.1 30.0	54.6 42.7	30.8 20.0	20.8 14.6	34.1 19.8	52.3 31.3
RTFE-TComplEx	52.2 38.6	42.6 30.6	59.7 42.0	68.6 55.2	32.9 18.7	23.5 14.1	35.9 17.9	52.7 28.2
Table 1: Entity prediction on continuous fact datasets: YAGO11k and Wikidata12k. Since the relations of these 2 datasets have typical "one-to-many" nature, the performance of tail prediction is better than that of head prediction.
Dataset	ICEWS14	ICEWS05-15	GDELT
Metric	MRR	Hits@1	Hits@3	Hits@10	MRR	Hits@1	Hits@3	Hits@10	MRR	Hits@1	Hits@3	Hits@10
t-TransE*	25.5	7.4	-	60.1	27.1	8.4	-	61.6	11.5	0.0	16.0	31.8
Hyte*	29.7	10.8	41.6	65.5	31.6	11.6	44.5	68.1	11.8	0.0	16.5	32.6
DE-SimpleE	52.6	41.8	59.2	72.5	51.3	39.2	57.8	74.8	23.0	14.1	24.8	40.3
ATiSE*	54.5	42.3	63.2	75.7	51.9	37.8	60.6	79.4	-	-	-	-
TComplEx	54.7	44.5	60.8	73.2	58.3	48.0	65.0	78.7	21.7	12.8	23.1	37.2
RTFE-RotatE	45.1	30.1	55.8	70.8	35.9	12.5	54.0	72.4	35.7	24.8	39.7	57.9
RTFE-HAKE	50.3	40.2	55.7	70.0	47.1	37.0	54.0	64.5	50.2	43.8	52.8	62.0
RTFE-DE-SimpleE	57.3	49.4	62.1	71.7	58.7	50.6	63.7	73.0	42.9	35.9	45.5	55.9
RTFE-TComplEx	59.2	50.3	64.6	75.8	64.5	55.3	70.6	81.1	29.7	21.2	31.9	46.4
Table 2: Entity prediction on discrete event datasets: ICEWS14, ICEWS05-15 and GDELT.
Table 2 where results marked (*) are taken from reported results of Hyte and ATiSE.
Table 1 shows that on continuous fact datasets, both translation-based and graph neural network-based methods can be transplanted to RTFE, and the results are better than Hyte, indicating the generality and superiority of RTFE. For both TransE-based approaches, RTFE-TransE outperforms Hyte on all metrics (e.g., with the improvement of  $15.0\%$  in tail MRR on Wikidata12k) because RTFE takes advantage of the continuity of facts directly.
RotatE and HAKE are the most advanced translation-based approaches and RTFE-RotatE or RTFE-HAKE outperforms other methods, which demonstrates that our framework can preserve the excellent results of these methods over SKG. Besides, the performance of state-of-the-art TKGE model TComplEx is enhanced by RTFE, which shows the gain from our recursive training pattern.
Table 2 shows that on discrete event datasets, RTFE also significantly improves the performance of TKGE models. Besides, on a dense dataset (i.e., with a small number of entities and a large number of facts) like GDELT, RTFE can take advantage of SKGE models such as HAKE.
5.3 Relation prediction
Relation prediction is given a quadruple  $(s, r, o, t)$ , to evaluate the plausibility of  $(s, ?, o, t)$ . The experimental results are shown in Table 3. RTFE-  $\mathrm{RDGCN}_{\text {type }}$  outperforms Hyte on YAGO11k that has only 10 relations (e.g., with the improvement of  $12.5\%$  in Hits@1), which implies that type information plays an important role in this task. Since the number of relations between these two datasets is relatively small (10 and 24), the performance improvement is not obvious after adding semantic information (e.g., with the improvement of  $1.3\%$  in Hits@1).
Hyte performed well on Wikdata12k. This may be attributed to its SKGE training pattern, which helped to capture applicable relation types between two entities from all the facts. In contrast, the timestamp of RTFE is trained by time, so only the facts of the current timestamp and information of last timestamp are directly utilized. To provide RTFE-RDGCN with more training data about relations, we added additional  $30\%$  negative samples obtained by replacing relations of quadruples into the negative sample set:  $\{(s,r',o) | (s,r,o) \in$
Figure 3: Extensibility validation experiment. Using equation (4) to convert TKG to SKG, different time intervals of TKG are used for pre-training. Then all timestamps are trained and tested to validate the extensibility for timestamps unseen during pre-training. The horizontal axis represents the time interval for pre-training and the ordinate represents the MRR of test results for all timestamps.