  Reasoning in a temporal knowledge graph (TKG) is a critical task for
information retrieval and semantic search. It is particularly challenging when
the TKG is updated frequently. The model has to adapt to changes in the TKG for
efficient training and inference while preserving its performance on historical
knowledge. Recent work approaches TKG completion (TKGC) by augmenting the
encoder-decoder framework with a time-aware encoding function. However, naively
fine-tuning the model at every time step using these methods does not address
the problems of 1) catastrophic forgetting, 2) the model's inability to
identify the change of facts (e.g., the change of the political affiliation and
end of a marriage), and 3) the lack of training efficiency. To address these
challenges, we present the Time-aware Incremental Embedding (TIE) framework,
which combines TKG representation learning, experience replay, and temporal
regularization. We introduce a set of metrics that characterizes the
intransigence of the model and propose a constraint that associates the deleted
facts with negative labels. Experimental results on Wikidata12k and YAGO11k
datasets demonstrate that the proposed TIE framework reduces training time by
about ten times and improves on the proposed metrics compared to vanilla
full-batch training. It comes without a significant loss in performance for any
traditional measures. Extensive ablation studies reveal performance trade-offs
among different evaluation metrics, which is essential for decision-making
around real-world TKG applications.
