E
XPLAINABLE S UBGRAPH R EASONING FOR F ORECASTING ON T EMPORAL K NOWLEDGE G RAPHS
Anonymous authors
Paper under double-blind review
A BSTRACT
Interest has been rising lately towards modeling time-evolving knowledge graphs (KGs).
Recently, graph representation learning approaches have become the dominant paradigm for link prediction on temporal KGs.
However, the embeddingbased approaches largely operate in a black-box fashion, lacking the ability to judge the results’ reliability.
This paper provides a link forecasting framework that reasons over query-dependent subgraphs of temporal KGs and jointly models the graph structures and the temporal context information.
Especially, we propose a temporal relational attention mechanism and a human-mimic representationupdate scheme to guide the extraction of an enclosing subgraph around the query.
The subgraph is then expanded via a temporal neighborhood sampling and pruning.
As a result, our approach provides explainable and human-understandable arguments to the forecasting task.
We evaluate our model on four benchmark temporal knowledge graphs for the link forecasting task.
While being more explainable, our model also obtains a relative improvement of up to 17.7 % on MRR compared to the previous best KG forecasting methods.
We also conduct a survey with 53 respondents, and the results show that the reasoning arguments extracted by the machine for knowledge forecasting are aligned with human understanding.
I NTRODUCTION
Reasoning, a process of inferring new knowledge from available facts, has long been considered to be an essential subject in artiﬁcial intelligence (AI).
Recently, the KG-augmented reasoning process has been studied in (Das et al., 2017; Ren et al., 2020), where knowledge graphs store factual information in form of triples ( s, p, o ), e.g. ( California, locatedIn, USA ).
In particular, s (subject) and o (object) are expressed as nodes in knowledge graphs and p (predicate) as an edge type.
Most knowledge graph models assume that the underlying graph is static.
However, in the real world, facts and knowledge change with time, which can be treated as time-dependent multi-relational data.
To accommodate time-evolving multi-relational data, temporal KGs have been introduced (Boschee et al., 2015), where temporal events are represented as a quadruple by extending the static triplet with timestamps describing when these events occurred, i.e. (Barack Obama, inaugurated, as president of the US, 2009/01/20).
In this work, we focus on forecasting on temporal KGs to infer future events based on past events.
Forecasting on temporal KGs can improve a plethora of AI applications such as decision support in various domains, e.g., personalized health care and ﬁnance.
These use cases often require the predictions made by the learning models to be interpretable, such that users can understand and rely on the predictions.
However, current machine learning approaches (Trivedi et al., 2017; Jin et al., 2019) for temporal KG forecasting operate in a black-box fashion, where they design an embedding-based score function to estimate the correctness of a quadruple.
These models can not clearly show which events contribute to the prediction and lack explainability to the forecasting.
Thus, purely data-driven ’black box’ methods do not give any information about the predictions’ reliability, making them less suitable for real-world applications.
Explainable approaches for reasoning on graphs can generally be categorized into post-hoc interpretable and integrated transparent methods (Doˇsilovi´c et al., 2018).
Post-hoc interpretable approaches (Ying et al., 2019) aim to interpret the results of a black-box model, while integrated transparent approaches (Das et al., 2017;
Qiu et al., 2019; Wang et al., 2019) have an explainable internal mechanism.
In particular, most integrated transparent (Lin et al., 2018; Hildebrandt et al., 2020) models employ path-based methods, e.g., reinforcement learning, to derive an explicit rea- soning path and demonstrate a transparent reasoning process.
The path-based methods focus on ﬁnding the answer to a query within a single reasoning chain.
However, many complicated queries require multiple supporting reasoning chains rather than just one reasoning path.
Recent work (Xu et al., 2019; Teru et al., 2019) has shown that reasoning over local subgraph structures substantially boosts performance while keeping interpretability.
However, these explainable models cannot only be applied to temporal graph-structured data because they do not take time information into account.
The time span between events often shows important regularities on their relative importance for the forecasting on temporal multi-relational graphs.
For example, events in the distant past may be less informative and predictive to the current graph structure and thus to forecasting.
Therefore, this work aims to design a transparent forecasting mechanism on temporal KGs that can generate informative explanations of the predictions and enables humans to check their reliability.
In this paper, we propose an e x plainabl e r easoning framework for forecasting on t emporal knowl
e dge graphs, xERTE, which employs a sequential reasoning process over local subgraphs of temporal KGs.
For example, to forecast a future link, xERTE starts from the query subject and then samples its temporal neighbors to construct a dynamic subgraph around it.
Furthermore, xERTE prunes and samples the subgraph to further expand it step-by-step.
After several rounds of expansion and pruning, the query object is predicted from nodes in the ﬁnal subgraph.
Thus, xERTE can naturally visualize the inference process and provide a concise and compact graphical explanation to the forecasting based on the extracted subgraph.
To guide the model to extend the subgraph in the direction of the query’s interest, we propose a two-folds temporal relational graph attention mechanism.
First, inspired by Vaswani et al.
(2017), we apply an attention mechanism to learn a vector representation of the target entity as a weighted sum of embeddings of its past neighbors across different timestamps.
Second, we use a shared embedding layer containing static and timedependent entity embeddings, allowing our model to capture both global structural information and temporal aspects about the graph.
Besides, to mimic human’s reasoning behavior, we develop a novel representation update mechanism.
When humans perform a reasoning process, their impressions of observed entities will update as new arguments have been found.
For example, the fact that company A has the largest market portion gives us a high expectation about future development.
However, a new argument shows that this company’s sales have been continuously decreasing for ﬁve months, which lowers our expectation.
Thus, it is necessary to pass messages from newly found events to the entities added in the subgraph previously.
Speciﬁcally, the proposed representation update mechanism enables every entity to receive messages from its farthest neighbors in the subgraph by performing message passing along each edge only once.
The major contributions of this work are as follows.
(1) We develope the ﬁrst explainable model based on temporal relational attention mechanisms for forecasting on temporal KGs.
(2) Unlike most black-box embedding-based models, xERTE visualizes the reasoning process and provides an interpretable inference graph to emphasize important arguments and inﬂuences of past events to the forecasting.
Thus, humans can inspect the arguments’ reliability to prevent inappropriate predictions using misleading data.
(3) To mimic human’s reasoning behavior, we design a reverse representation update mechanism, where newly found entities will affect the hidden representations of explored entities.
The dynamical pruning procedure enables our model to perform reasoning on largescale temporal knowledge graphs with millions of edges.
(5) To evaluate our approach, we apply our model for forecasting future links between observed nodes on four large-scale temporal knowledge graphs, showing that our method outperforms all state-of-the-art baselines.
(6) We conduct a survey with 53 respondents to evaluate whether the model is aligned with human understanding.
P RELIMINARIES
Let E and P represent a ﬁnite set of entities, and predicates, respectively.
A temporal knowledge graph is a collection of timestamped facts written as quadruples.
A quadruple q = ( e s , p, e o , t ) describes a temporal fact at a timestamp t , where p ∈ P represents a timestamped and labeled edge between a subject entity e s ∈ E and an object entity e o ∈ E .
The temporal knowledge graph forecasting task aims to predict unknown links at future timestamps based on observed events.
Deﬁnition 1 (Temporal KG forecasting).
Let F represents the set of all ground-truth quadruples, and
( e s,q , p q , e o,q , t q ) ∈ F denotes the target quadruple.
Given a query ( e s,q , p q , ? , t q ) derived from the target quadruple and a set of observed facts O = { ( e i , p k , e j ,
t l ) ∈
F| t l < t q } , the temporal
KG forecasting task is to predict the missing object entity e o,q .
Speciﬁcally, we consider all entities in the set E as candidates and rank object entities e o ∈ E by their likelihood to form a true triple together with the given subject-predicate-pair at timestamp t q 1 .
Inference Graph: For a given query q = ( e q , p q , ? , t q ) , we build an inference graph G inf to visualize the reasoning process.
Unlike temporal KGs, each entity in G inf is associated with a timestamp, and thus, each node is an entity-timestamp pair.
The inference graph is a directed acyclic graph in which a link points from a node with an earlier timestamp to a node with a later timestamp.
We initialize
G inf with a node v q = ( e q , t q ) consisting of the query subject and the query time.
Then we expand
G inf by adding neighbors of e q in the tKG.
For example, suppose that ( e q , p k , e j , t (cid:48) ) is a ground truth quadruple where t (cid:48) <
t q , we add the node ( e j , t (cid:48) ) into G inf and link it with v q to .
We provide an example in Figure 4 in the appendix to illustrate the inference graph.
Deﬁnition 2 (Node in Inference Graph and its Temporal Neighborhood).
Let E represents all entities, F denotes all ground-truth quadruples, and t represents a timestamp.
A node in an inference graph G inf is deﬁned as an entity-timestamp pair v =
( e i , t ) , e i ∈ E .
We deﬁne the set of prior neighbors of v as N v =( e i ,t ) =
{ ( e j ,
[Equation] [Equation]
, p k ,
[Equation]
) ∈
[Equation] [Equation]
t (cid:48) < t } .
For simplicity, we denote prior neighbors as N v .
Similarly, we deﬁne the set of posterior neighbors of v as N v =( e i ,t ) =
{ ( e j , t (cid:48) ) | ( e i , p k , e j , t (cid:48) )
∈ F ∧ ( t (cid:48) > t ) } (cid:83) { ( e j , t (cid:48) )
| ( e j , p k , e i , t (cid:48) ) ∈
F ∧ ( t (cid:48) > t ) } .
We denote them as N v for short.
The deﬁnition of prior neighbors and posterior neighbors are introduced for clarifying the message passing mechanism in Section 4.2.
We use P uv to represent the set of observed predicates connecting node u and node v .
Besides, we deﬁne the set of edges between v and its prior neighbors as Q v , where q ∈ Q v is a prior edge of v .
R ELATED W ORK
Representation learning is an expressive and popular paradigm underlying many KG models.
The embedding-based approaches for knowledge graphs can generally be categorized into bilinear models (Nickel et al., 2011; Bala ˇ zevi ´ c et al., 2019), translational models (Bordes et al., 2013; Sun et al., 2019), and deep-learning models (Dettmers et al., 2017; Schlichtkrull et al., 2018).
However, the above methods cannot use rich temporal dynamics available on temporal knowledge graphs.
To this end, several studies have been conducted for temporal knowledge graph forecasting (Leblay &
Chekol, 2018; Garc ´ ıa-Dur ´ an et al., 2018; Ma et al., 2018; Dasgupta et al., 2018; Trivedi et al., 2017;
Jin et al., 2019; Goel et al., 2019; Lacroix et al., 2020).
Nevertheless, the methods are largely black box, lacking the ability to interpret their prediction results.
Recently, several explainable reasoning methods for knowledge graphs have been proposed (Das et al., 2017; Xu et al., 2019; Hildebrandt et al., 2020; Teru et al., 2019).
However, these explainable methods can only deal with static KGs, while our model is designed for interpretable forecasting on temporal KGs.
O UR M ODEL 4. S UBGRAPH R EASONING P ROCESS
The key contribution of this paper is a novel explainable subgraph reasoning model for temporal knowledge graph forecasting.
Speciﬁcally, the model conducts the reasoning process on a dynamically expanded inference graph G inf extracted from the temporal KG.
This inference graph gives an interpretable graphical explanation about the ﬁnal prediction.
Given a query q = ( e q , p q , ? , t q ) , we initialize the inference graph as the query entity e q and deﬁne the tuple of ( e q , t q ) as a node in the inference graph.
The inference graph expands by sampling temporal neighbors that are linked with e q before t q .
In general, the expansion will grow rapidly and cover almost all nodes after a few steps.
To prevent the inference graph from exploding, we reduce the number of nodes by pruning the nodes that are less relevant to the query.
Hence, we propose a query-dependent temporal relational attention mechanism in Section 4.2 to identify the nodes’ importance in the inference graph related to the query q .
Next, by expanding the inference graph, we sample the temporal neighbors of the remaining nodes after pruning.
As this process iterations, the inference graph allocates more and more information from increasingly larger neighborhoods of the temporal KG.
Besides, we add self-loop edges to allow xERTE stop at a node if it believes reaching the ground-truth.
After running L expansion steps, the model selects the entity with the highest plausibility score in G inf as the prediction of the missing entity in the query, where the inference graph serves as a graphical explaination.
4. T EMPORAL R ELATIONAL A TTENTION M ECHANISM
Here, we propose a temporal relational graph attention (TRGA) mechanism for identifying the relevant arguments in the inference graph related to a given query q .
The TRGA mechanism computes a query-dependent contribution score for each edge.
Besides, similar to GraphSAGE (Hamilton et al., 2017) and GAT (Veli ˇ ckovi ´ c et al., 2017), the TRGA mechanism performs a local representation aggregation.
To avoid misusing future information, we only allow message passing from prior neighbors to posterior neighbors.
Speciﬁcally, for each node v in the inference graph, the aggregation function uses the representation of node v and its prior neighbors N v to output a time-aware representation for v .
Since entities may play different roles depending on the predicate they are associated with, we also incorporate the predicate embeddings in the attention mechanism.
If the node v has a large number of prior neighbors, we sample a subset ˆ N v ⊂ N v using different sampling schemes described in Section 4.5.
Instead of treating all prior neighbors with equal importance, we take the query information into account and assign varying importance levels to each prior neighbor u by calculating a query-dependent attention score as
[Equation]
where e lvu ( q, p k ) is the attention score of the edge ( v, p k , u ) regarding the query q
= ( e q , p q , ? , t q ) , p k corresponds to the predicate between node u and node v , p k and p q are predicate embeddings.
h lv denotes the hidden representation of v at the l -th inference step, W lsub and W lobj are two weight matrices for capturing the interactions between query features and subject/object features, || denotes the concatenation operation.
Then, we compute the normalized attention score α lvu ( q, p k ) using the softmax function , which is written as
[Equation]
Once obtained, we aggregate the representation of prior neighbors and weight them using the normalized attention scores:
[Equation]
where we initialize the hidden representations in the case l = 0 using the entity embeddings.
Similar to (Hamilton et al., 2017), we combine the current hidden representation h l − 1 v ( q ) of node v with the aggregated neighborhood vector (cid:101)
h lv ( q ) and feed them into a fully connected layer with a nonlinear activation function, as shown below
[Equation]
where h l
4. R EVERSE R EPRESENTATION U PDATE M ECHANISM
To mimic human’s reasoning behavior, we need pass messages from newly found events to the nodes added at previous inference steps.
However, as the inference graph G inf expands, there might be l -hop neighbors at the l -th inference step.
Thus, we need to iterate the message aggregation l times at the l -th inference step to ensure each node in G inf can carry information within the node’s l -hop neighborhood.
To avoid the linear increase of message passing iterations, we propose a novel representation update mechanism for subgraph reasoning.
Since we only feed prior neighbors’ representations into the message aggregation function, the farthest neighbor of a node added at the m -th inference step is at most ( l − m ) -aeay, where l is the current inference step.
Hence, we update the representations of nodes in reverse order of being added in the inference graph.
Speciﬁcally, at l -th inference step, we ﬁrst update the representations of nodes added at ( l − 1) -th inference step, then the nodes added at l − 2 , and so forth until l = 0 , as shown in Algorithm 1 in the appendix.
In this way, every node can receive messages from its farthest prior neighbors in G inf by computing messages along each edge only once.
4. D YNAMIC E MBEDDINGS
In temporal knowledge graphs, graph structures are no longer static as entities and their links evolve over time.
Entities naturally have some features that change over time and some features that remain ﬁxed.
Thus, we represent the latent feature embeddings of an entity e i ∈
E at a time t with a static low-dimensional vector and a functional representation of time to encode both long-term stationary properties and time-dependent behavior, which is deﬁned as e i =
[ ¯e i , Φ ( t )] T ∈ R d S + d T .
Here, ¯e i ∈
R d S represents the static embeddings that captures time-invariant features and global dependencies over the temporal KG.
Φ ( t ) ∈ R d T denotes a functional time encoding that captures temporal dependencies between entities (Xu et al., 2020).
d S and d T represent the dimensionality of static embeddings and time embeddings, which can be tuned according to the temporal fraction of the given dataset.
Thus, our model is able to predict unseen events based on both the stationary and time-evolving structural features.
4. N EIGHBORHOOD S AMPLING
To reduce the scope and complexity, we sample a subset of prior edges ˆ Q v ∈ Q v at each inference step if a node v =
( e i , t ) has been involved in a large number of events that occurred before t .
We denote the prior neighbors and posterior neighbors of node v after the sampling as ˆ N v and ˆ N v , respectively.
The sampling can be uniform if there is no bias, or the sampling can be temporally biased using a non-uniform distribution.
For instance, we may want to sample more edges closer to the current time point as the events that took place long ago may have less impact on the current structure and, thus, the inference.
Speciﬁcally, we propose three different sampling strategies: (1) Uniform sampling .
Each prior edge q v ∈ Q v of node v has the same probability of being selected: P ( q v )
= 1 / |Q v | .
(2) Time-aware exponentially weighted sampling .
We temporally bias the neighborhood sampling using an exponential distribution and assign the probability P ( q v = ( e i , p k , e j , t (cid:48) ))
=
exp( t (cid:48) − t ) / (cid:80) ( e i ,p l ,e m ,t (cid:48)(cid:48) ) ∈Q v exp( t (cid:48)(cid:48) − t ) to each prior neighbor, which negatively correlates with the time difference between node v and its prior neighbor ( e j , t (cid:48) ) .
Time-aware linear weighted sampling .
We use a linear function to bias the sampling.
Compared to the second strategy, the quadruples occurred in early stages have a higher probability of being sampled.
We have empirically found that the second strategy is most beneﬁcial to our framework and provide a detailed ablation study in Section 5.2.
4. S CORE F UNCTION AND L EARNING
After L reasoning steps, we get a ﬁnal inference graph.
We rank all entities in the inference graph according to their plausibility scores and choose the entity with the highest score as our prediction.
Speciﬁcally, we compute the plausibility score a le i of an entity e i regarding a query q as follows:
[Equation]
where v ( e ) represents the entity included in node v , a lv denotes the plausibility score of node v at l -th inference step, and g ( · ) represents a score aggregation function.
As stated in Deﬁnition 2, each node in inference graph is an entity-timestamp pair.
To assign each entity an unique score, we aggregate the plausibility scores of nodes where their entities are the same.
We tried two score aggregation functions g ( · ) , which are summation and mean.
We have conducted an ablation study and found that the summation aggregation performs better.
Detailed results are shown in Section 5.2.
We prune the inference graph at each inference step by remaining the nodes with M largest node scores, where M is a preﬁxed parameter.
To reduce the time complexity, we recalculate attention scores and plausibility scores on the pruned inference graph.
Thus, the gradients of removed nodes do not need to be considered in backpropagation, signiﬁcantly speeding up the training procedure.
The genuine quadruples in a temporal KG are split into train , validation , and test sets in a proportion of 70% / 15% / 15% , respectively.
We use the binary cross-entropy as the loss function, which is deﬁned as L = ( − 1 /N )
(cid:80) Nm =1 1 / |E infm | (cid:0)
y Tm log( a m ) + ( 1 − y m )
T log( 1 − a m ) (cid:1) , where N is the number of training samples, a m denotes the vector containing plausibility scores of all entities in the ﬁnal inference graph regarding a training sample q m , and y m represents the vector of binary labels indicating whether entities in the inference graph are genuine for q m .
We use Adam optimizer (Kingma & Ba, 2014) to learn model parameters.
E XPERIMENTS 5. D ATASETS AND B ASELINES
Integrated Crisis Early Warning System (ICEWS)
(Boschee et al., 2015) and YAGO (Mahdisoltani et al., 2013) have established themselves in the research community as benchmark datasets of temporal KGs.
The ICEWS dataset contains information about political events with time annotations, e.g., (Barack Obama, visit, Malaysia, 2014-02-19).
We evaluate our model on three subsets of the
ICEWS dataset: ICEWS14, ICEWS18, and ICEWS05-15 contain event facts in 2014, 2018, and the facts between 2005 to 2015, respectively.
The YAGO dataset is a temporal knowledge base that fuses information from Wikipedias with the English WordNet dataset (Miller, 1995).
Following the experimental settings of HyTE (Dasgupta et al., 2018), we use a subset and only deal with year level granularity by dropping the month and date information.
We compare our approach and baseline methods by performing the link prediction task on the ICEWS14, ICEWS18, ICEWS0515, and
YAGO datasets.
The statistics of the datasets are provided in Appendix C.
We compare xERTE with benchmark temporal KG and static KG reasoning models.
From the temporal KG reasoning models, we compare our model with several state-of-the-art methods, including
TTransE (Leblay & Chekol, 2018), TA-DistMult/TA-TransE
(Garc ´ ıa-Dur ´ an et al., 2018), TcomplEx (Ma et al., 2018), and RE-Net (Jin et al., 2019).
From the static KG reasoning models, we choose
TransE (Bordes et al., 2013), DistMult (Yang et al., 2014), and ComplEx (Trouillon et al., 2016) where we compress temporal knowledge graphs into a static, cumulative graph by ignoring the time information.
Implementation details of the baselines and our model please see Appendix E.
5. E XPERIMENTAL R ESULTS AND A BLATION S TUDY
Table 1 summarizes the raw results of the link prediction task on the ICEWS and YAGO datasets.
If the ground truth entity does not appear in the ﬁnal subgraph, we set its rank as |E| (the number of entities in the dataset).
We provide a detailed evaluation protocol in Appendix D. Additionally, we provide the ﬁltered results in Appendix F. Overall, xERTE signiﬁcantly outperforms all other baseline models on these datasets in MRR, Hits@1/3/10, while being more interpretable.
Compared to the strongest baseline RE-Net, xERTE obtains a relative improvement of 12% and 19% in MRR and Hits@1, which are averaged on all four datasets.
Especially, xERTE achieves more gains in
Hits@1 than in Hits@10.
It conﬁrms the assumption that subgraph reasoning helps xERTE make a sharp prediction by exploiting local structures.
To assess the importance of each component, we conduct the following ablation studies.
First, we train a model without updating the representations of nodes found in previous inference steps to ﬁnd out how the reverse representation update contributes to our model.
Since the reverse representation update ensures that each node can receive messages from every node in the inference graph,
we
[Figure]
expect this mechanism could to help nodes mine available information.
This update mechanism should be especially important for nodes that only have been involved in a small number of events.
Since the historical information of such nodes is quite limited, it is very challenging to forecast their future behavior.
In Figure 2a and 2b we show the metrics of Hits@1 and Hits@10 as a function of the number of entities in the ﬁnal inference graph.
It can be observed the model with the reverse update mechanism performs better in general.
In particular, this update mechanism signiﬁcantly improves the performance if the query subject only has a small number of (multi-hop) neighbors in the ﬁnal subgraph, which meets our expectation.
Also, to verify the importance of the time components, we evaluate the performance of a model without time encoding.
It can be observed in Figure 2c, removing the time-dependent part from entity representations sacriﬁces the model’s performance signiﬁcantly.
Recall that each node in the inference graph G inf is associated with a timestamp, the same entity might appear several times in
G inf with different timestamps.
To get a uniﬁed score for each entity, we aggregate the plausibility scores of nodes where their entities are the same.
Figure 2d shows that the summation function brings a considerable gain in each metric on ICEWS14.
Furthermore, we run experiments with different sampling strategies proposed in Section 4.5.
To assess the necessity of the time-aware weighted sampling, we propose a deterministic version of time-aware weighted sampling, where we chronologically sort the prior edges of node v in terms of their timestamps and select the last N edges to build the subset ˆ Q v .
The experimental results are provided in Table 2 in the appendix.
We ﬁnd that different sampling strategies have a considerable inﬂuence on model’s performance.
Sampling strategies that bias towards recent quadruples perform better.
Speciﬁcally, the exponentially time-weighted strategy performs better than the linear timeweighted strategy and the deterministic last-N-quadruples strategy
[Table]
5. G RAPHICAL E XPLANATION AND H UMAN E VALUATION
The extracted inference graph provides a graphical explanation for the forecasting prediction.
To demonstrate which arguments are important for the reasoning process, we assign each edge in the inference graph a contribution score.
Speciﬁcally, the contribution score is deﬁned as c uv ( q, p k ) =
e uv ( q, p k )
α v (see Equation 1 and 6 for deﬁnitions of e uv and α v ), where node u is a prior neighbor of node v in terms of the predicate p k .
Thus, the users can trace back the important arguments that the prediction results mainly depend on.
We study a query chosen from the test set, where we predict whom will Catherine Ashton visit on Nov. 9, 2014 and show the ﬁnal inference graph in Figure 3.
The node size indicates the value of the plausibility score; the darkness of the edge color indicates the contribution score of the edge.
In this case, the model’s prediction is Oman.
We can see that (Catherine Ashton, express intent to meet or negotiate, Oman, 2014-11-04) is the most important event to support this answer.
We provide more visualization results Appendix G.
To assess whether the arguments are informative for users in an objective setting, we conducted a survey where respondents judge the relevance of the arguments in the inference graph to the prediction.
More concretely, we set up a quiz consisting of 7 rounds.
In each round, we sample a query from the test set of ICEWS14/ICEWS0515.
Along with the query statement and the ground-truth object, we present the human respondents two arguments extracted from the ﬁnal inference graph with high contribution scores two arguments with low contribution scores in randomized order.
The respondents are supposed to judge the relevance of the statements to the query in two levels, namely relevant or less relevant.
Besides, we sort the arguments according to their contribution scores computed by our model and ask the human respondents to rank them according to their relevance.
We then determine whether the relevant level classiﬁed by the respondents matches the that estimated by our models.
We surveyed 53 participants, and the average accuracy of all questions is 70.5%.
Moreover, based on a majority vote, 18 out of 21 questions were answered correctly, indicating that the arguments in the inference graph are informative and the model is aligned with human intuition.
The complete survey along with a detailed evaluation is reported in Appendix I.
C ONCLUSION AND F UTURE W ORK
We proposed an explainable reasoning approach for forecasting future links on large-scale temporal knowledge graphs.
The model extracts a query-dependent subgraph from the temporal KGs by selecting neighbors of nodes recursively and uses a novel temporal relational attention mechanism to guide the extraction.
We assigned each node in the subgraph an attention value to show its relevance to the query and let the model base its prediction on the extracted subgraph.
The subgraph can be seen as a graphical explanation of the prediction, making our model more interpretable than other baseline methods.
We proposed a reverse representation update mechanism that provides a considerable gain to the nodes with a small amount of links in history.
Extensive experiments show that our model outperforms all baselines concerning all metrics on three benchmark datasets of temporal knowledge graphs, demonstrating the effectiveness of our model while being more interpretable.
We conducted a survey about the arguments included in the extracted subgraph.
The results indicate that the arguments are informative for humans.
Sampling Strategies
[Table]
A R ELATED W ORK A. K NOWLEDGE G RAPH M ODELS
Representation learning is an expressive and popular paradigm underlying many KG models.
The key idea is to embed entities and relations into a low-dimensional vector space.
The embeddingbased approaches for knowledge graphs can generally be categorized into bilinear models (Nickel et al., 2011; Balaˇzevi´c et al., 2019), translational models (Bordes et al., 2013; Sun et al., 2019), and deep-learning models (Dettmers et al., 2017; Schlichtkrull et al., 2018).
However, the above methods lack the ability to use rich temporal dynamics available on temporal knowledge graphs.
To this end, several studies have been conducted for temporal knowledge graph forecasting (Leblay &
Chekol, 2018; Garc´ıa-Dur´an et al., 2018; Ma et al., 2018; Dasgupta et al., 2018; Trivedi et al., 2017;
Jin et al., 2019; Goel et al., 2019; Lacroix et al., 2020).
Ma et al.
(2018) developed extensions of static knowledge graph models by adding a timestamp embedding to their score functions.
Besides,
Garc´ıa-Dur´an et al.
(2018) suggested a straight forward extension of some existing static knowledge graph models that utilize a recurrent neural network (RNN) to encode predicates with temporal tokens derived by decomposing given timestamps.
Also, HyTE Dasgupta et al.
(2018) embeds time information in the entity-relation space by arranging a temporal hyperplane to each timestamp.
However, these models cannot generalize to unseen timestamps because they only learn embeddings for observed timestamps.
In contrast, Know-Evolve Trivedi et al.
(2017) learns evolving entity representations using point processes, being able to infer embeddings at a future timestamp.
Nevertheless, the methods are largely black box, lacking the ability to interpret their prediction results while our main focus is to employ an integrated transparency mechanism for archiving human-understandable results.
A. E XPLAINABLE REASONING ON K NOWLEDGE G RAPHS
Recently, several explainable reasoning methods for knowledge graphs have been proposed (Das et al., 2017; Xu et al., 2019; Hildebrandt et al., 2020) .
(Das et al., 2017) proposed a reinforcement learning based path searching approach to display the query subject and predicate to the agents and let them perform a policy guided walk to the correct object entity.
The reasoning paths produced by the agents can explain the prediction results to some degree.
Also, (Hildebrandt et al., 2020) framed the link prediction task as a debate game between two reinforcement learning agents that extract arguments from knowledge graphs and allow users to understand the decision the agents.
Besides, and more related to our work, (Xu et al., 2019) models a sequential reasoning process by dynamically constructing an input-dependent sub-graph.
The difference here is that these explainable methods can only deal with static KGs, while our model is designed for interpretable forecasting on temporal
KGs.
B W ORKFLOW
We show the workﬂow of the subgraph reasoning process in Figure 5.
The model conducts the reasoning process on a dynamically expanding inference graph G inf extracted from the temporal KG.
This inference graph gives an interpretable graphical explanation about the ﬁnal prediction.
Given a query q = ( e q , p q , ? , t q ) , we initialize the inference graph with the query entity e q and deﬁne the tuple of ( e q , t q ) as a node in the inference graph (Figure 5a).
The inference graph expands by sample neighbors that have been linked with e q before t q , as shown in Figure 5b.
The expansion would go rapidly that it covers almost all nodes after a few steps.
To prevent the inference graph from exploding, we constrain the number of nodes by pruning the nodes that are less related to the query (Figure 5c) .
Here, we propose a query-dependent temporal relational attention mechanism in Section 4. to identify the nodes’ importance in the inference graph for query q and aggregate information from nodes’ local neighbors.
Next, we sample the past neighbors of the remaining nodes in the inference graph to expand it further, as shown in Figure 5d.
As this process iterates, the inference graph incrementally gains more and more information from increasingly larger neighborhoods of the temporal
KG.
After running L expansion steps, the model predicts the missing entity in the query based on the entities in the inference graph, where the inference graph serves as a graphical explaination for the prediction.
C D ATASET S TATISTICS
We provide the statistics of the datasets in Table 3.
Since we focus on the near neighborhood of the query time.
It is necessary to answer the question of ”what’s the accuracy of prediction if we simply randomly choose one entity in the one-hop or two-hop neighborhood?”.
It is equivalent to the ratio of nodes in the neighborhood with ground truth being the entity.
We show the result in the onehop neighborhood in Table 4 and the results in the two-hop neighborhood in Table 5.
Besides, we count the number of nodes with ground truth entity within the two-hop neighborhood and show the
[Figure]
[Table] [Table]
histogram in Figure 6.
We can see that guessing based on the one-hop/two-hop yields signiﬁcantly worse performance compared to xERTE.
[Table]
D E VALUATION PROTOCOL
For each quadruple q = ( e s , p, e o , t ) in the test set G test , we create two queries: ( e s , p, ? , t ) and ( e o , p − 1 , ? , t ) , where p − 1 denotes the reciprocal relation of p .
For each query, the model ranks all entities E Linf in the ﬁnal inference graph according to their scores.
Let ψ e s and ψ e o represent the rank for e s and e o of the two queries respectively, we evaluate our models using standard metrics across the link prediction literature: mean reciprocal rank (MRR) :
[Equation]
and Hits @ k ( k ∈ { 1 , 3 , 10 } ) : the percentage of times that the true entity candidate appears in the top k of ranked candidates.
E I MPLEMENTATION
We implemented our model and all baselines in PyTorch (Paszke et al., 2019).
Besides, we use the datasets augmented with reciprocal relations to train all baseline models.
We tune hyperparameters of our models using a grid search.
We set the learning rate to be 0.0002, the batch size to be 128, and the inference step to be 3.
Besides, we sample 30 edges from the prior neighbors for each node
[Figure]
[Table]
[Figure]
[Table]
in the inference graph and prune 10 edges regarding its attention value.
We implement TTransE,
TA-TransE/TA-DistMult, and RE-Net based on the code provided in (Jin et al., 2019).
We use the binary cross-entropy loss to train these baselines and optimize hyperparameters by early stopping according to MRR on the validation set.
F A DDITIONAL E XPERIMENTAL R ESULTS
In this paper, we consider two different ﬁltering settings.
The ﬁrst one is following the ranking technique described in Bordes et al.
(2013), where we remove from the list of corrupted triplets all the triplets that appear either in the training, validation, or test set.
We name it time-independent ﬁltering .
Trivedi et al.
(2017) and Jin et al.
(2019) use this ﬁltering setting for reporting their results on temporal KG forecasting.
However, assume this ﬁltering setting is not appropriate for evaluating the link prediction on temporal KGs.
For example, there is a test quadruple (Barack Obama, visit,
India, Jan. 25, 2015), and we perform the object prediction (Barack Obama, visit, ?, Jan. 25, 2015).
Besides, we have observed the quadruple (Barack Obama, visit, Germany, Jan. 18, 2013) in the training set.
According to the time-independent ﬁltering, (Barack Obama, visit, Germany, Jan. 25, 2015) will be considered as a genuine quadruple and ﬁltered accordingly since the triplet (Barack
Obama, visit, Germany) appears in the training set in the quadruple (Barack Obama, visit, Germany,
Jan. 18, 2015).
However, the triplet (Barack Obama, visit, Germany) is only temporally valid on
Jan. 18, 2013 but not on Jan. 25, 2015.
Therefore, we propose another ﬁltering scheme, which is more appropriate for the link forecasting task on temporal KGs.
We name it time-dependent ﬁltering .
In this case, we only ﬁlter out the quadruples that are genuine at the timestamp when the link forecasting is conducted .
In other words, if the quadruple (Barack Obama, visit, Germany, Jan.
25, 2015) does not appear at the query time Jan. 25, 2015, this quadruple is considered as corrupted and ﬁltered out.
We provide the time-independent ﬁltered results in Table 6 and the time-dependent ﬁltered results in Table 7.
Since TransE, DistMult, and ComplEx are ran on static KGs, we do not apply timedependent ﬁltering on their results.
G V ISUALIZATION
We provide additional inference visualization examples in Figure 8 and Figure 7.
[Table]
[Table]
H R EVERSE REPRESENTATIONS UPDATE MECHANISM FOR SUBGRAPH REASONING
In this section, we explain an additional reason why we have to update node representations along edges selected in previous inference steps.
We show our intuition by a simple query in Figure 9 with two inference steps.
We do not apply the pruning procedure.
First, we check the equations without updating node representations along previously selected edges.
Layer 1: h 10 =
f
( h 00 , h 01 , h 02 , h 03 ) h
= hh 
 = hh
= h
Layer 2: h 20 = f
( h 10 , h 11 , h 12 , h 13 )
=
f ( h 10 , h 01 , h 02 , h 03 ) h
= f ( h 11 , h 04 , h 05 )
=
f ( h 01 , h 04 , h 05 ) h
= f ( h 13 , h 06 ) =
f ( h 03 , h 06 ) h 2
=
f
( h 1 2 , h 0 7 , h 0 8 ) =
f ( h 02 , h 07 , h 08 )
Note that h 20 is updated with h 01 , h 02 , h 03 and has nothing to do with h 04 , h 05 , h 06 , h 07 , h 08 , i.e., two-hop neighbors.
In comparison, if we have full access to the graph all the time, we are able to aggregate information of two-hop neighbors in the second layer.
If we update the node representations along previously selected edges, the update in second layer changes to:
Step a: h 2 ,a 0 = f
( h 10 , h 11 , h 12 , h 13 )
=
f ( h 10 , h 01 , h 02 , h 03 ) h 2 ,
a 1 = f
( h 11 , h 04 , h 05 )
=
f ( h 01 , h 04 , h 05 ) h 2 ,a 3 = f ( h 13 , h 06 ) =
f ( h 03 , h 06 ) h 2 ,a 2 = f ( h 12 , h 07 , h 08 ) =
f ( h 02 , h 07 , h 08 )
Step b: h 2 ,b 0 = f
( h 2 ,a 0 , h 2 ,a 1 , h 2 ,a 2 , h 2 ,a 3 ) h
=
h 2 ,ah 2 1 =
h 2 ,ah 22 = h 2 ,ah 23 =
h 2 ,a
Although we use h 10 to update h 2 ,a 0 , which ideally should be h 00 , we are now able to cover two-hop neighbors.
I S URVEY
In this section, we provide the online survey (see Section 5.3 of the main body) and the evaluation statistics based on 53 respondents.
To avoid biasing the respondents, we did not inform them about the type of project that we have been working on.
Further, all questions are permuted at random.
We set up the quiz consisting of 7 rounds.
In each round, we sample a query from the test set of
ICEWS14/ICEWS0515.
Along with the query statement and the ground-truth object, we present the users two arguments extracted from the ﬁnal inference graph and two arguments with low contribution scores in randomized order.
The respondents are supposed to judge the relevance of the statements to the query in two levels, namely relevant or less relevant.
There are three questions in each round that ask the participants to give the most relevant statement, the most irrelevant statement, and rank the statements according to their relevance.
The answer to the ﬁrst question is classiﬁed as correct If a participant gives one of the two relevant statements as the most relevant statement.
Similarly, the answer to the second question is classiﬁed as correct If the participant gives one of the two removed statements as the most irrelevant statement.
For the relevance ranking task, the answer is right if the participant ranks the two relevant statements higher than the removed statements.
I. AI Q UIZ
You will participate in a quiz consisting of eight rounds.
Each round is centered around an international event.
Along with the event, we also show you four reasons that argue why the given event happened.
While some arguments may be informative and explain the occurrence of this event, others may irrelevant to this event.
Your task is to ﬁnd the most relevant argument and most irrelevant argument, and then sort all four arguments according to their relevance.
Don’t worry if you feel that you cannot make an informed decision: Guessing is part of this game!
Additional Remarks: Please don’t look for external information (e.g., Google, Wikipedia) or talk to other respondents about the quiz.
But you are allowed to use a dictionary if you need vocabulary clariﬁcations.
Example
Given an event, please rank the followed arguments according to their relevance to the given event.
Especially, please select the most relevant reason, the most irrelevant reason, and rank the relevance from high to low.
Event: French government made an optimistic comment about China on 2014-11-24.
A. First, on 2014-11-20, South Africa engaged in diplomatic cooperation with Morocco.
Later, on 2014-11-21, a representative of the Morocco government met a representative of the French government.
B. First, on 2014-11-18, the Chinese government engaged in negotiation with the Iranian government.
Later, on 2014-11-21, a representative of the French government met a representative of the Chinese government.
C. On 2014-11-23, the French hosted a visit by Abdel Fattah Al-Sisi.
D. A representative of the French government met a representative of the Chinese government on 2014-11-21.
Correct answer
Most relevant: D
Most irrelevant: A
Relevance ranking: D B C A
Tasks
Event:
On 2014-12-17, the UN Security Council accused South Sudan
A. South Africa engaged in diplomatic cooperation with South Sudan on 2014-12-11.
B. First, on 2014-11-17, Uhuru Muigai Kenyatta accused UN Security Council.
Later on , the UN Security Council provided military protection to South
Sudan
C.
On 12-16, UN Security Council threatened South
Sudan with sanctions. D. South Sudan hosted the visit of John Kerry
on Most
[Table]
Most irrelevant: Relevance ranking: 2.
Event: Indonesia police arrested and retained an Indonesia citizen at 2014-12-28.
A.
The Indonesia police claimed that an attorney denounced the citizen on 2014-12-10.
B. Zaini Abdullah endorsed the Indonesia citizen on 2014-12-25.
C. The Indonesia police made an optimistic comment on the citizen on 2014-12-14.
D. The Indonesia police investigated the citizen on 2014-12-08.
[Table]
Event:
A citizen from Greece protested violently against the police of Greece on 2014-11-17 .
A.
The Greek head of government accused the political party “Coalition of the Radical Left” on 2014-05-25.
B. Greek police refused to surrender to the Greek head of government on 2014-10-15.
C. Greek citizens gathered support on behalf of John Kerry on 2014-11-17.
D. Greek police arrested and detained another Greek police ofﬁcer on 2014-11-04.
[Table]
Event: Ra´ul Castro signed a formal agreement with Barack Obama on 2014-12-17.
A. First, on 2009-01-28, Dmitry A. Medvedev made statements to Barack Obama.
Later, on 200901-30, Ra´ul Castro negotiated with Dmitry A. Medvedev.
B. Ra´ul Castro visited Angola on 2009-07-22.
C. Ra ´ ul Castro hosted a visit of Evo Morales on 2011-09-19.
D. First, on 2008-11-05, Evo Morales hosted a visit of Barack Obama.
Later, on 2011-09-19, Ra ´ ul Castro appeal for de-escalation of military engagement to Evo Morales.
[Table]
Event:
The head of the government of Ukraine considered to make a policy option with Angela Merkel on 2015-07-10.
A. First, on 2014-07-04, the armed rebel in Ukraine used unconventional violence to the military of Ukraine.
Later, on 2014-07-10, the head of government of Ukraine made statements to the armed rebel in Ukraine.
B. The head of the government of Ukraine expressed intent to meet with Angela Merkel on 201410-30.
C. First, on 2014-07-04, the armed rebel in Ukraine used unconventional violence to the military of Ukraine.
Later, on 2014-07-19, the head of government of Ukraine made statements to the armed rebel in Ukraine.
D. The head of the government of Ukraine consulted with Angela Merkel on 2015-06-06.
Most relevant: Most
[Table]
Relevance ranking: 6.
Event:
On 2014-08-09, Ukraine police arrested a member of the Ukraine military.
A. First, on 2014-07-23, a member of Ukraine parliament consulted the head of the Ukraine government.
Later, on 2014-07-24, the head of government made a statement to the Ukraine police.
B. First, on 2014-06-25, the military of Ukraine used violence to an armed rebel that occurred in Ukraine.
Later, on 2014-07-10, the armed rebel used violence to the Ukraine police.
C. First, on 2005-02-20, the military of Ukraine made a statement to the head of the government of Ukraine.
Later, on 2005-07-18, the head of government of Ukraine appealed for a change in leadership of the Ukraine police.
D. On 2014-07-31, the head of the Ukraine government praised the Ukraine police.
[Table]
relevant: Most irrelevant: Relevance ranking: 7.
Event:
The Ofﬁce of Business Affairs of Bahrain negotiated with the Labor and Employment Ministry of Bahrain on 2015-07-16.
A. First, on 2014-07-27, the undersecretary of Bahrain made statements to the Labor and Employment Ministry of Bahrain.
Later, on 2015-01-21, an ofﬁcer of Business Affairs of Bahrain signed a formal agreement with the undersecretary of Bahrain.
B. On 2012-01-21, the ofﬁce of Business Affairs of Bahrain expressed intent to provide policy support to the employees in Bahrain.
C. First, on 2006-11-01, the employees in Bahrain made statements with the special Rapporteurs of the United Nation.
Later, on 2011-05-11, the ofﬁce of Business Affairs of Bahrain reduced relations with the employees in Bahrain.
D. A representative of the
Labor
and Employment Ministry of Bahrain consulted with a representative of the Ofﬁce of Business Affairs of Bahrain on 2014-
Most relevant:
Most irrelevant:
Relevance ranking:
I. [Table]
Question 1 :
Most relevant: B/C
Most irrelevant: A/D
Relevance ranking: BCAD/BCDA/CBAD/CBDA
Question 2 :
Most relevant: A/D
Most irrelevant: B/C
Relevance ranking: ADBC/ADCB/DABC/DACB
Question 3 :
Most relevant: B/D
Most irrelevant: A/C
Relevance ranking: BDAC/BDCA/DBAC/DBCA
Question 4 :
Most relevant: A/D
Most irrelevant: B/C
Relevance ranking: ADBC/ADCB/DABC/DACB
Question 5 :
Most relevant: B/D
Most irrelevant: A/C
Relevance ranking: BDAC/BDCA/DBAC/DBCA
Question 6 :
Most relevant: B/C.
Most irrelevant: A/D
Relevance ranking: BCAD/BCDA/CBAD/CBDA
Question 7 :
Most relevant: A/D
Most irrelevant: B/C
Relevance ranking: ADBC/ADCB/DABC/DACB
I. E VALUATION
The evaluation results of 53 respondents are shown in Figure 10.