Evaluating explainability techniques on discrete-time graph neural networks
Authors: Manuel Dileo
Matteo Zignani
Sabrina Tiziana Gaito
Venue: Accepted by TMLR ()

Abstract: Discrete-time temporal Graph Neural Networks (GNNs) are powerful tools for modeling evolving graph-structured data and are widely used in decision-making processes across domains such as social network analysis, financial systems, and collaboration networks. Explaining the predictions of these models is an important research area due to the critical role their decisions play in building trust in social or financial systems. However, the explainability of Temporal Graph Neural Networks remains a challenging and relatively unexplored field. Hence, in this work, we propose a novel framework to evaluate explainability techniques tailored for discrete-time temporal GNNs. Our framework introduces new training and evaluation settings that capture the evolving nature of temporal data, defines metrics to assess the temporal aspects of explanations, and establishes baselines and models specific to discrete-time temporal networks. Through extensive experiments, we outline the best explainability techniques for discrete-time GNNs in terms of fidelity, efficiency, and human-readability trade-offs. By addressing the unique challenges of temporal graph data, our framework sets the stage for future advancements in explaining discrete-time GNNs.

Published in Transactions on Machine Learning Research (05/2025)
Evaluating explainability techniques on discrete-time graph neural networks
Manuel Dileo
Deportment of Computer Science University of Milan, Milan, Italy
Mattco Zignani
matteo. muntteo.zigaani@@mmmi...c
Department of Computer Science University of Milan, Milan, Itoly
Sabrina Gaito
aabrina..asto@ummiii..
Department of Computer Science University of Milan, Milan, Raly
Reviewed on OpenReview: https: // openreview. net/ met/ forum? id- JzmXoOrfry
Abstract
Discrele-time temporal Graph Neural Networks (GNNs) are powerful lools for modeling evolving graph-structured data and are widely used in decision-makinn processes across domains such as social network analysis, financial systems, and collaboration networks. Ex- plaining the predictions of these models an important research area due to the critical role their decisions play in building trust in social or financial systems. However, the explain- ability of Temporal Graph Neural Networks remains challenging and relatively unexplored field. Hence, in this work, we propose novel framework to evaluate explainability tech- niques tailored for discrele-time temporal GNNs. Our framework introduces new training and evaluation settings that capture the evolving nature of temporal data, defines metrics to assess the temporal aspocts of explanations, and establishes baselines and models spo- cific to time temporal networks. Through extensive experiments, we outline the best explainability techniques for discrete-time GNNs in terms of fidelity, efficiency, and human-readability trade-offs. By addressing the unique challenges of temporal graph data, our framework sets the stage for future advancements in explaining discrete-time GNNs.
1 Introduction
Temporal Graph Neural Networks (TGNNs) have emerged as crucial tool for modeling evolving graph- structured data (Longa et al., 2023). These models are widely applied in various domains, including social network analysis (Dilco et al., 2023), financial systems (Pareja et al., 2020; Dilco & Zignani, 2024), and collaboration networks (You et al., 2022), where understanding dynamic relationships is critical. Discrete- time TGNNs, in particular, represent temporal data as scries of discrete snapshots, providing flexible and scalable approach for capturing temporal dependencies. Despite their effectiveness, the black-box nature of TGNNs raises significant challenges in interpreting their predictions, which is essential for building trust and ensuring transparency in sensitive applications such as financial fraud detection social behavior analysis.
Explainability techniques aim to address this challenge by providing insights into the decision-making pro- cesses machine learning models. While considerable progress has been made in explaining static GNNs (Longa et al., 2025), the unique temporal and dynamic nature of TGNNs introduces additional complexities that remain largely unexplored. Temporal data often involves evolving patterns (Paranjape et al., 2017), recurrent interactions (Gastinger et al., 2024), and distribution shifts (Zhang et al., 2022), which must be

---

Published in Transactions on Machine Learning Research (05/2025)
carefully considered when generating and evaluating explanations. Although some approaches for explaining TGNNs have been proposed in the literature (Chen & Ying, 2023; Xia et al., 2023), none of these methods are specifically tailored for discrete-time temporal networks. Furthermore, their evaluation strategies and baselines rely heavily on the conventional setting developed for explaining static GNNs (Amara et al., 2022).
Hence, in this work, we propose novel framework to evaluate explainability techniques this context. Our framework addresses the temporal characteristics of the data and models by introducing new training and evaluation settings, temporal-speciftc metrics, and tailored baselines. Through our experiments, we provide practical guidance the trade-offs among fidelity, efficiency, and human-readability of the current state-of-the-art techniques, including graph and gradient-based techniques.
Contributions We summarize main contributions as follows: i) Training and evaluation setting: We propose a new setting that takes into account the evolving nature of the data when generating and evaluating explanations for discrete-time temporal networks; ii) Evaluation metrics: We exlend and complement the set of evaluation metrics to focus on the temporal aspects of the provided explanations, such as the recurrence of the events. Wi) New baselines for explaining TGNNs: We propose baselines and models tailored for discrete-time GNNs, by leveraging simple ideas like considering only the most recent events or by extending well-known explainability techniques such as GNNExplainer; iv) Practical guidance: We conduct extensive evaluation to outline the current best explainability techniques for discrete-time GNNs in terms of fidelity, performance, and readability trade-offs.
The objective of our work is to advance rescarch in explaining any discrete-time GNN model. Hence, we mainly focus evaluating instance-level post-hoe explanation met since most of the temporal graph models in the literature are not designed with specific consideration of explanations. The full definitions and taxonomy of explainability techniques are defined in Amara et al. (2022).
Scope
2 Background
Temporal Graph Neural Networks Temporal graph neural networks (TGNNs) are deep learning mod- els for extracting, learning, and predicting from evolving networks. The most relevant graph neural network architectures for temporal graphs are covered in a recent survey (Longa et al., 2023). Current TGNNs ap- proaches can be divided based on how they model temporal networks. In continuous-ting dynamic graph models, temporal networks are treated as a stream of ordered timestamped edges. In contrast, in discrete- time or snapshot-based dynamic graph models, temporal networks are a collection of ordered timestamped static graphs snapshots- with a certain time granularity. These techniques are designed to handle two dif- ferent kinds of temporal networks, and their training and evaluation strategies lead to two different settings (Huang et al., 2023b). For instance, event-based dynamic graph models can be used for streaming data, such as sensor data, or to predict anomalies with very high-resolution lemporal precision (Reha et al., 2023). In contrast, snapshot-basee models can be leveraged where data are collected with regular time intervals or in complex systems where nodes act very similarly in close timestamps, such as online social networks, collaboration, or financial networks (Pareja et al., 2020). In this paper, we focus on the latter models. Specifically, we provide a new setting, metrics, baselines, and models for evaluating explanations provided by discrete-time TGNNs models on future link prediction, the most well-known and studied prediction task temporal networks (Huang et al., 2023b).
Graph Explainers Explainability methods for Graph Neural Networks are generally divided into two cat- egories: non- non-generative and generative methods. For given input instance and its prediction, non-generative methods often employ approaches such gradients (Baldassarre & Azizpour, 2019), perturbations (Huang et al., 2023a), mask optimization (Ying et al., 2019), surrogate models (Vu & Thai, 2020), subgraph match- ing (Wu et al., 2023), and Monte Carlo Tree Search (MCTS)(Yuan el al., 2021) to identily the explanation subgraph. These methods incrementally optimize the explanation during the explanation stage, which can result in a longer inference time. In contrast, generative methods involve training a generative model on the entire dataset capture the distribution of the underlying explanatory subgraphs (e.g. Luo et al. (2020); Miao et al. (2022)), enabling comprehensive understanding of model behavior across the dataset.
2

---

Published in Transactions on Machine Learning Research (05/2025)
Compared to static GNNs, the explainability of Temporal Graph Neural Networks remains challenging and relatively unexplored area. TGNNExplainer (Xia et al., 2023) is the first explainer specifically de- signed for temporal GNNs, utilizing the MCTS algorithm to identify combination of explanatory events. TempME (Chen & Ying, 2023), instcad, is derived from the information bottleneck principle and extracts the most interaction-related motifs while minimizing the amount of contained information to preserve the sparsity and succinctness of the explanation. However, these two works are tailored for contimuus-time models. Recently, two other works proposed explainability techniques tailored for time series on graphs, i.e. spatiotemporal forecasting (He et al., 2022; Guerra et al., 2024). To the best of our knowledge, there methods tailored for the explainability of discrete-time graph neural networks. Hence, the focus of this work is to provide a comprehensive framework to advance this research area.
3 Problem setup
A temporal network can be modeled as a graph g = (V, where V is the set of nodes, V {(u, v, t,u),, v E E is the set of interaction events timestamped edges and T the set of timestamps. A discrete-time temporal network G = {G 121 can be represented as sequence of graph snapshots, where each snapshot is a static graph Gt = (V,, E,) associated to specific timestamp t E T, which contains the interactions occurred at time / and the nodes involved in them. We also denote with 9]c,..) the sequence of graph snapshot G2 s.t. t, S t te,ts,t, T.
A discrete-time Graph Neural Network (GNN) takes as input a discrcte-time temporal network 9 and learns time-aware embedding for each node V up to certain time 1. For simplicity, we refer to discrete- time GNNs temporal graph neural networks (TGNNs). TGNNs' capability for representing learning temporal graphs is typically evaluated using the future link prediction task (Pareja et al., 2020; You et al., 2022; Huang et al., 2023b), i.e predicting future interactions based on historical events. Hence, in this work, we focus on explaining the link prediction behavior of TGNNs.
Given a pre-trained TGNN f, our objective is to obtain an explanation model. Specifically, given a target interaction event e, an explainer for TGNNs aims at identifying a subset of important historical events that trigger the future interaction prediction made by the base model f. An "explanation" in the domain of GNNs is mask a subgraph of the initial graph, i.e,, set of weighted nodes, edges, and possibly node features (Amara et al., 2022). The weights on these graph entities relate to their inherent importance for explaining the model outcomes. The explainer model usually performs a feature attribution operation which associates each feature of the computation graph for the target event with a weight or relevance score for the link predictor. The compulation graph of a target event e = (u, 0, l), denoted as G(e), might be the augmented graph of all the events up to time t, subview of the graph based on a temporal window, or a subgraph around nodes u and u, since some methods only look at a k-hop neighborhood to make predictions. The set of events in G(c) are called candidate events. In this study, we focus the contribution of historical interaction events, namely the edges. To explain each target event e, all the methods compared in this paper generate a mask Mg E R [ whose elements denote the importance score of a candidate event to the prediction of the target event. Finally, an explanation corresponds to a mask Mg on the edges that operate on the computational graph to form a subgraph Gezp where only edges with an importance score greater than zero are included.
4 Evaluating XAI on discrete-time GNNs
Aspect 1: setting
We propose a new setting that takes into account the evolving nature of the data when generating and evaluating explanations for discrcte-time temporal networks.
Problem 1: Traditionally, discreto-time graph neural networks are trained in the deployed setting (Pareja et al., 2020). In this setting, no information from the test set is passed to the model, and the node embeddings from the last training snapshot are used for predictions in all test snapshots. Using the deployed setting
3

---

Published in Transactions on Machine Learning Research (05/2025)
leads to generating explanations for future events based on the train set graph only, without considering all the events from the end of the train set until the target event, providing explanatory subgraphs that may be poorly informative.
Solution:
To avoid this behavior, we modify the traditional training and inference steps for discrete-time GNNs. First, the models are trained using the live-update setting (You et al., 2022) the train set. This recently proposed strategy boosts the performance of discrete-time GNNs, where the dataset is split chronologically into snapshots and the model weights are constantly updated to newly observed snapshots while predicting the next onc. This experimental sctting represents an instance of prequential sotting (Dawid, 1992), more suitable for Training and evaluating machine learning algorithms for data streams. Then, we allow models to incorporate newly observed information in the test set make future predictions. Specifically, information from the test set is used to train the models but events after the train set leveraged at inference time to update the node embeddings until the snapshot before the target event. In this way, explanations are generated based also the newly observed events. In concurrent work (Huang et al., 2024), the authors develop a very similar strategy, called UTG, to allow a comparison between discrete and contimuous-time dynamic graph models. Our strategy can be considered as a slight variation of UTG where the training phase performed using the live-update strategy.
Problem 2: In previous works (Xia et al., 2023; Chen & Ying, 2023), the quality of the explanations for TGNNs is computed by selecting k future target events to explain globally, i.e. uniformly at random over the entire test set, obtaining an evaluation of the overall quality of the explanations for future events. However, they do not evaluate the quality of the explanations over time, which may tell several useful information about the behavior of explainability techniques along the timestamps in the test set, such as the difficulty of providing good explanations for timestamps distant from the training set, or the identification of test time interval where they may be a distribution shift respect to the train set. In general, prior works assume that the quality of the explanations on future events may be similar for the entire lifetime of the dataset, which may be a rough approximation.
Solution: To overcome this limitation, we select & future target events to explain locally, i.c. for each test set snapshot. So, the evaluation metries can be computed for each snapshot, and the performance trend over the test set can be easily shown.
We summarize our new strategy for generating and evaluating explanations for discrete-time temporal net- works in Algorithm
Aspect 2: metrics
Previous works evaluate explanations for TGNNs using fidelity sufficiency (Xia et al., 2023; Chen & Ying, 2023), a very popular explainability metric for GNNs (Amara et al., 2022). Here, we extend fidelity take into account the temporal aspect of the solutions. Furthermore, TempME introduces cokesiveness to evaluate explanations from human-intelligible viewpoint, based on the temporal proximity of the adjacent events. Here, we propose to complement cohesiveness with new metrics that take into account other important structural aspects of temporal networks, such as the recurrence of the events.
Fidelity sufficiency. In the context of future link prediction, for each event e, fidelity sufficiency mea- sures the agreement the edge existence between the prediction by the GNN when fed with the entire computational graph (e)) and the prediction when fed with the explanation of event Gexp(e). Given K the set of target events, fidelity defined as:
Fid(K)= =
1
Σ ((f(Geap f(G(i)) KEK
K
where f(Ge)p(e) indicates the output of the GNN the inference step when fed with the explanation of event E K. Usually, we want the explanation to be sufficient; i.e. it leads to the initial prediction of the
4

---

Published in Transactions on Machine Learning Research (05/2025)
Algorithm 1 Evaluating discrete-time GNNs explanations
Input: TGNN model, XAI technique, D temporal annotated dataset Paramcter: & number of events to cxplain
Output: Explanations and metrics
1:
Split in train, validation and test set, chronologically
Let t' be the first timestamp after the train set
Train GNN D's train set using the live-update setting. Optimize parameters using D's validation set.
4: for all limestamps / in D's test set do
5: Let G5 be the current snapshot.
6: Let Gir,a-1] be the graph with all the events from the end of the train set until the previous snapshot.
7:
for all snapshots G3 in Gpr,-11 do
Let X; be the node feature matrix Let Eg be the edges of the snapshot
10:
Let H;-1 be the previous node embeddings
11:
H; TGNN(X, Eg, H;
12:
end for
Sample & positive and è negative edges as target events from G2. 2
13:
14: Explain the target events using the XAT technique on the trained TGNN considering edges Gly as candidate events.
15:
Evaluate the explanations of the current snapshot.
16: end for
17: return explanations and metrics
model explanation. Hence, an explainability model that provides sufficient explanation has a fidclity close to zero. We later consider and report (1 - Fid) in our experiments.
Given the evaluation procedure defined in Algorithm in our scenario we have set K of target events for each test snapshot. Hence, extend this metric to obtain the temporal fidelity sufficiency, defined
1 Thest
TFid
(-Fi((()))
Trest
where Kj denotes the set of larget events related to test snapshot, j, and Trest is the number of snapshots in the test set.
Cohesiveness. Given a larget event e, its cohesiveness defined as:
144 (4--41) 4/l AT
Coh(e) [Cexp(e) (exp(e)
Σ
Σ cos
1 ~ j).
(EGeap(e) jEDesp(e)sift
where AT means the time duration in the computational graph G(e), 1 ~ f) indicates whether event is spatially adjacent to event j, Lee. they share a vertex. Meanwhile, temporally proximate event pairs are assigned with larger weights of cos(It; ty//AT)), where denotes the timestamp of event (Chen & Ying, 2023).
Edge recurrence, reciprocity, homophily. Cohesivencss is a good metric to measure the readability level of an explanation based on temporal network molifs (Paranjape et. al., 2017) with several involved events. However, cohesiveness is not defined for single-edge explanations, and it provides low scores for explanations with few adjacent odges. Morcover, there are simple but fundamental mechanisms that rule the evolution of networks that are not captured by cohesiveness. We focus on three well-known, leveraged, and studied mechanisms in graph machine-learning research: edge recurrence (Gastinger et al., 2024), edge
5

---

Published in Transactions on Machine Learning Research (05/2025)
reciprocity, and homophily. Edge recurvence is the mechanism by which two nodes that interacted in the past are likely to interact again in the future. In directed net works, reciprocily refers to the mechanism where the existence of a directed edge from a node to node v increases the likelihood of forming a reciprocal odge from v to u the future. Finally, homophily refers to the theory in network science which states that similar nodes may be more likely to atlach than dissimilar ones. Since the focus of our work is on the contribution of the edges, we refer structural homophily, which is the tendeney of nodes that share a lot of common neighbors create a link. Homophily is widely recognized as an interpretable and human-intuitive phenomenon in sociology (McPherson al., 2001). Similarly, recurrence and reciprocity reflect simple and familiar social behaviors that humans easily recognize and expect, as discussed in the empirical study of evolving social networks (Kossinets & Watts, 2006). Figure 1 shows graphically the limit of cohesiveness on the three mechanisms: reciprocity and edge recurrence cannot be caplure by cohesiveness since it required at least two events in Gezp to be computed, while explanations based on triadic closure may be penalized by delayed interactions. We introduce three new metrics basod on the analog mechanisms overcome these limitations. In detail, given larget event (u, occurring in G(e) and its Geap(e), we define the recurrence as:
Recurrence(e)- = = w if ((,, E Geap(e) s.t. <t
otherwise.
where w 0. For simplicity, we set w = for the events, but could be the minimum or the maximum Reciprocity can be defined analogously, i.c. it cxists at least event (v, u'," such that t' < t. Finally, we define structural homophily based on the notion of temporal common neighbors:
among the wy cos [t-t'] t'1
(LV(w) n N v,,,) f)) min( N(t)))
Homophily(e = (u,
where (u) and (v) are the sets of temporal neighbors in Geaple) of nodes and v.
What defines the best explainability technique for discrete-time GNNs? Following previous works (Chen & Ying, 2023), the best technique reaches the highest level of temporal fidelity in the minimum execution time. On average, its explanations are cohesive and leverage recurrence, reciprocity, and homophily, when present in the dataset.
t?
t?
t?
(b)
(c)
Figure 1: Pitfalls of cohesiveness, which is not defined for edge recurrence (a) and reciprocity (b), and it is penalized by temporal delay in structural homophily (c). However, these 3 explanations are very human intelligible.
Aspect 3: baselines and models
Basclincs. Previous works (Xia et al., 2023; Chen & Ying, 2023; He et al., 2022) do not compare thoir models against simple baselines tailored for temporal networks, which can make challenging to gauge the progress made in addressing the temporal aspects of the explanations. To this end, we introduce two simple baselines create edge masks for temporal graph explainability:
IastSnapshot- Explaaierr-:: it sets the odge mask with for the candidate events in the computational graph that happens in the snapshot before the target event and with Os for all the other events, based on the idea that the most recent. events have the great impact the target one.

---

Published in Transactions on Machine Learning Research (05/2025)
given target event (u, t), it construct the 2-hop induced sub- graph of ((e) around nodes {u, preserving paths with non decreasing timestamps from neighbors to u and v. Then, it sets the edge mask with 1s for the events involved in the constructed subgraph and Os for the rest. It can be seen as a simple, non-parametric method that selects subgraphs by matching events that preserve the temporal graph's 's causal topology, without relying on gradient information or model internals.
TemporalVeighhoos--Eppaaaa
Discrete-time models. To the best of our knowledge, there are no methods tailored for discrete-time GNNs in literature. Hence, we extend Iwo well-known approaches for graph explainability to handle discrete- time temporal networks. Specifically, et al., 2020) as they are the most representative of non-generative and generative methods, respectively, GNNExplainer formalizes the concept of importance using mutual information, leading to the following objective function optimized using gradient descent:
choose GNNExplainer (Ying al., 2019) and PGExplainer (Luo
Σ 1y= Pf(V = u = X 001
min ME
where Me is the edge mask of a static graph, A its adjacency matrix, X its node feature matrix, c the binary class for link prediction. In our setting, given an event e = (u, v, (), ME is mask on historical events before e, which can be seen as a set of edge masks for each snapshot Me = {ME, 1/=1 Hence, the GNNExplainer's objective has been modified to handle discrcto-time temporal networks and compute the importance score for each event as follows:
min
1l y/G = [A (MM x (x,
c=0,1
where A5 and Xj are the adjacency matrix and node feature matrix of snapshot j, respectively. PGEuplainer adopts a deep neural network to parameterize the generation process of explanations, which enables a natural approach to explaining multiple instances collectively. Specifically, an MLP ha k) infers the importance score an edge j w.r.t. a larget edge k. Treating the output of ha as an edge mask assignment weight, PGExplainer can leverage the same objective function of GNNExplainer on multiple target instances (more details in Luo al. (2020)). Given target event k (ug, ea, (x) and candidate j = (uj. 8y, ty, can extend the PGExplainer architecture to handle discrete events, constructing the following ML. input features:
update the weights of the MLP
Ft ty
Z Hu Hop |/,,||HHHHHHH |H,, | cos(
1]=
AT
where Hu, represents the node embedding of node uy obtained through the pre-trained TGNN, AT is the computational graph duration, and timestamps are represented as progressive integer numbers.
5
Experiments
5.1
Experimental setup
Datasets. We evaluate the explainability models on three well-known real-world temporal graph datasets, BitcoinOTC (Pareja et al., 2020), Reddit-title (You et al., 2022), and Email-EU (Paranjape et al., 2017), covering the three of the most important applicative domains of discrete-time networks, which are financial, social, and collaboration networks, plus a recent temporal network datasct coming from protcin interactions (Fu & He, 2022). BitcoinOTC is who-trrussswhhom network of people who trade using Bitcoin, derived from the homonymous platform. Reddit-title is a hyperlink network representing the directed connections between two communitics on Reddit. Email-EU was generated using email data from a large European research institution, where e-mails only represent communication between institution members. Detailed dataset statisties are given in the Supplementary Material.

---

Published in Transactions on Machine Learning Research (05/2025)
Base models. We adopt three state-of-the-art discrete-time graph neural networks the base model: EvolveGCN (Pareja et al., 2020), GCRN-GRU (Seo et al., 2018), and ROLAND-GRU (You et al., 2022). EvolveGCN is representative of model evolution methods, while GCNRN-GRU and ROLAND are based on embodding evolution (Longa et al., 2023). Base models are pro-trained using the best configuration of hyperparameters in the original papers, when available, or optimized using grid-search. You can refer to the Supplementary Material for more information and their results for the link prediction task.
Explainability models. We compare all the baselines and techniques introduced in this paper (Section 4). We also consider a dummy explainer that outputs random events in the computational graph. Furthermore, we consider Saliency (SA) and Integrated Gradients (IG), which are two of the most effective solutions for static graphs (Amara el al., 2022), but they were never considered for temporal networks (Xia el al., 2023; Chen & Ying, 2023). SA measures edge importance as the weight on every edge after computing the gradient of the output with respect to node embeddings (Baldassarre & Azizpour, 2019), while Integrated Gradient (IG) avoids the saturation problem of Saliency by accumulating gradients over the path from a zero-vector and the input at hand (Sundararajan et al., 2017). Lastly, we include SHAP as one of the most consolidated techniques for explainability and representative of the perturbation- based methods (Lundberg & Lee, 2017).
Configuration. We split each dataset chronologically in train, validation, and test sets using 70/10/20 of the snapshots. We randomly sample 50 target events to explain each snapshot. Note that this choice led to test explainability methods on a number of events that is up to five times the number of instances in previous works (Xia al., 2023; Chen & Ying, 2023), which were 500 overall. For each target, the candidate events are all the edges that appear in a time window immediately before the event, whose size is set equal to the 10% of the snapshots in the dataset. Following previous works (Amara et al., 2022), to compare different explainability techniques, we set the sparsity level of the explanations to a maximum size, equal to 20 events. All the explainers are trained for 200 epochs. Code and Supplementary Material are available in a Github repository".
5.2 Results
Temporal fidelity. We report the temporal fidelities in Table 1. Overall, Integrated Gradients and the discrele-time extension of GNNExplainer reach the highest value of temporal fidelities. In particu- lar, gradient- methods, such as Saliency and Integrated Gradients, seem to be the best choice for model evolution techniques. In our intuition, assigning edge importance equal to the edge parameter weight after computing the gradient can easily mimie the evolution of the model's parameters through the internal- ized RNN. Whereas graph-based methods such as GNNFxplainer are more suitable for embedding evolution methods since their decisions depend on the structural i.e. graph evolution of the node's neighborhoods captured by the embeddings rather than the general dynamic of the system, on which model evolution tech- niques focus. In general, the simple baselines obtain lower performance than at least one technique for each task. The general low performance of explainability of GCRN-GRU for Reddit-title is probably related to the low performance of the base model in the link prediction task; hence, due to the known performance drop of explainability techniques when applied to incorrect predictions, as highlighted in Amara et al. (2022). Explaining the decisions made on the Email-EU dataset is particularly challenging, likely due to the very high density of events in this dataset. We investigate in detail this challenge in the Supplementary Material.
Fidelity trends. Thanks to the setting presented in Section 4, we can show the explainer's fidelity trends over time. Visualizing and comparing the trends is useful to answer the following questions, which cannot be answered using the tabular result only: a) Does the performance of an explainer vary along the test snapshots? b) Do different techniques follow the same trend? e) Does the best technique strictly dominate the others over time? We show the fidelity trends for explaining the decisions of ROLAND-GRU for Reddit- title in Figure 2a, as an interesting example due to the high IG performance. We observe that Integrated Gradients strictly dominate the other explainers. GNNExplainer shows very similar trend to IG, but its performance varies significantly over time (c.g. up to more than 20% from snapshot 12 to 18). In contrast, the fidelity Irend of PGExplainer follows the one of the random explainer, suggesting that its explanations
httpps//gitthub. http:iithhu..cmmmmmmmieeeee com/manuel - sillee//dtgn-eeppaaiii
8

---

Published in Transactions on Machine Learning Research (05/2025)
Table 1: Temporal fidelities of seven different explainers over four datasets and three base models. The best result for each model and dataset is bold, the second best is underlined.
BitcoinOTC
Reddit-titlc
Email-EU
Temporal PPI
EvolveGCN
Random
72.50 + 06,86
40.06 + 06.18
18.79 + 05.90
50.50 = 19.51
LastSnapshot
67.71 + 06.18
26.53 + 04.83
18.46 + 05.94
25.75 + 03.93
TemporalNeighhors
69.57 + 07.49
78.53 = 5.83
20.71 + 06.47
65.25 = 26.48
SA
76.50 + 06.43
80.18 + 04.91
36.37 + 11.12
76.75 + 07.14
IG
76.57 + 06.37
89.24 + 03.82
26.15 = 08.17
89.50 = 13.44
GNNExplainer
73.50 + 06.82
58.82 + 06.40
27.73 + 09.37
81.50 14.20
PGExplainer
70.57 = 06.97
40.06 + 06.18
18.79 = 05.90
50.50 19.51
SHAP
70.00 + 17.32
90.00 + 13.42
35.67 + 18.87
82.95 + 17.11
ROLAND-GRU GCRN-GRI GRU
Random
46.86 +: 07.90
89.00 + 05.23
28.58 + 07.54
75.00 + 14.90
LastSnapshot
22.93 +: 07.06
75.24 + 06.98
27.79 + 10.02
44.75 + 11.49
TeiporalNeighbors
60.00 + 06.70
54.65 + 05.92
38.52 + 12.20
56.88 12.12
SA
54.50 + 08.10
80.35 + 06.03
29.65 + 07.59
72.00 + 17.23
IG
58.00 = 04.96
80.56 + 05.99
29.83 + 07.56
87.75 = 08.03
GNNExplainer
60.29 + 08.58
89.24 + 05.36
60.77 + 08.89
91.00 = 09.06
PGExplainer
45.64 + 06.44
88.82 + 05.34
27.35 + 07.36
74.75 = 14.86
SHAP
58.86 + 15.32
80.00 + 17.89
37.34 + 12.65
86.12 = 18.03
Random
53.64 + 06.42
64.41 + 07.40
38.54 + 08.29
64.75 = 13.30
LastSnapshot
43.57 +:2 08.06
57.29 + 07.58
28.90 + 07.27
22.50 = 06.91
TemporalNeighbors
14.07 +: 04.42
81.47 + 06.20
38.56 + 08.70
60.75 = 10.24
SA
56.79 +: 07.83
84.18 + 04.76
48.73 + 10.71
79.00 + 09.64
IG
57.29 + 06.15
94.88 + 02.92
49.40 + 11.50
96.75 = 05.19
GNNExplainer
88.50 + 04.77
85.18 + 05.27
49.41 - 11.75
88.75 06.85
PGExplainer
70.00 + 06.82
71.24 + 06.89
40.94 + 08.85
60.75 15.20
SHAP
60.71 + 21.03
84.00 + 16.00
41.23 + 12.65
81.46 + 16.35
could be unreliable (further details in the Supplementary Material). Finally, the trend of LastSnapshot shows that the models need to look up more than the most recent events to explain the decisions. We report all the fidelity trends of our experiments in the Supplementary Material.
Computation time. We report the centorrmannc-oopppttie time plot (Amara et al., 2022) of the con- sidered explainers in Figure 2b. Overall, GNNExplainer obtains the best performances, but Saliency and Integrated Gradients are far more efficient with drop of about 5% in performance only. A similar result previously observed in explaining static GNNs (Amara et al., 2022). SHAP shows very high computational time compared to its competitor, making it difficult to utilize it in practical scenarios. On average, it takes about 25 times the time to compute the explanation for a single target event compared to its competitors. Thus, it is excluded from further analysis.
Human-intelligiblee explanations. In Section 4 we defined different metrics for quantifying the readabil- ity of the explanations given by the models, based on temporal aspects. In Figure 3 we report the scatter plot with temporal fidelity for the four metrics, where edge recurrence and reciprocity are computed only for existing future events. In general, the best techniques are the ones in the upper right corner of the plots. We favor the methods that exhibit high cohesiveness and do not show drastic drop for the other three metrics. The figure shows that the explainability models generally return highly cohesive explanations for discrete-time GNNs, an opposite trend compared to continuous-Lime GNNs, where the techniques fail to generate aaman-intelligible explanations (Chen & Ying, 2023). Specifically, PGExplainer provides the most cohesive explanations, but it exhibits a sharp drop in performance for the other metrics. In contrast, GNNExplainer, SA, and IG output cohesive explanations and are also quite able to capture edge recurrence and reciprocity. Overall, explainability techniques provide explanations with high homophily. Given the

---

Published in Transactions on Machine Learning Research (05/2025)
1.0
0.65
0.9
0.60
0.8
22
0.7
TFid
0.55
Explainers
0.6
0.50
Ran
Random
LastSnapshet
Snapshott
TemporaiNe ghborr Integrated Gradients
0.5
Integrated Gradients GNNExplainer
0.45
Saliency
GNNExppaiier
0.4
PGExplainer
PGExplainer
0.40
0
10
15
20
25
30
10-2
10-1 Time(s)
100
# #Snapshots
(a)
(b)
Figure 2: (a) Fidelity trends of explainers for the decision made by ROLAND-GRU on future link prediction of Reddit-title dataset. (b) Average temporal fidelity (y-axis) and execution time (æ-axis) for each explainer.
low levels of structural homophily available the training set of the considered datasets (vertical line Figure 3d), considered it not a useful mechanism for the GNNs decisions.
0.65
Tempor Gradients
0.60
Gradients
0.60
Sallery cl FoExoluner
Chiiiel PöErplaher
TFid
0.55
0.55
0.50
0.50
0.45
0.45
0.40
0.40
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Cohesiveness
Edge Recurrence
(b)
0.65
0.65
Random
LestSnapahst
homporatleighbor
0.60
inte-raned Gradients
0.60
Sabency
caeiner poRteplaher
TFid
0.55
TFid
0.55
Randum
Last'
Temperallengtoon
0.50
0.50
Integ ritegratedde Gred
Salleniy
GnllErolancer
0.45
0.45
PoEaplaine
Bitcoroficc
Recd 8-itle
0.40
0.40
Email EU
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Edge Reciprocity
Edge Homophily
(c)
(d)
Figure 3: Performance-reddabllity plots for cohesiveness (a), edge recurrence (b), reciprocity (c), and struc- tural homophily (d). The dashed vertical line in (d) indicates the level of structural homophily in the training set of the selected datasets.
Case-study: explaining distrust in BitcoinOTC. Lastly, we focus on explaining the decisions of TGNNs in specific real-world challenge. Specifically, we chose explain the decisions behind the existence
10

---

Published in Transactions on Machine Learning Research (05/2025)
of total distrust edge in BitcoinOTC. Predicting the existence of such edges is important because it enables the identification of untrustworthy users and protects the integrity of the platform. Without clear explanations, users and administrators may struggle to justify the model's decisions, particularly when false positives occur. To this end, we ask for an explainability model to obtain the important events related to the decisions of all the distrust edges in the first test snapshot. Overall, we obtain 70 larget events. Specifically, we chose ROLAND-GRU as the base model and GNNExplainer, since they achieve the best performance on link prediction and fidelity on BitcoinOTC, respectively. We report three of the most frequent kinds of explanations in Figure 6. We observe highly human-readable explanations. We find that most decisions are made based on edge recurrence, negative consensus on the target nodes, and authority of source nodes. Given a target event e = (u, v, t), we define consensus as the average vote on the incoming edges of the destination node e before I, and authority as the in-degree centrality of the source node u before I, considering incoming edges with positive votes only. A negative consensus is an average consensus lower than zero. To evaluate quantitatively the presence of these patterns in the given explanations, we compare the distribution of authority and consensus in the explanatory subgraphs, computational graphs, and random vote networks. Random networks are generated using the Erdös Rényi model (Barabasi & Posfai, 2016) with the probability of edge creation and number of nodes equal to the explanatory graph's ' density and number of nodes, respectively, and edge weights assigned uniformly at random in [-10, 10). We show the boxplot of the distributions of consensus and authority on the three graphs in Figure 4. Overall, we observe that only a few explanations leverage negative consensus, but they exhibit a value lower than the average consensus of both candidate and random graphs. Concerning authority, we notice that its value is generally far higher in the explanations than in the candidate or random graphs, confirming that it is very leveraged pattern decide whether total distrust edge exists or not.
1.0
7.5
5.0
0.8
Consensus
2.5
Authoriity
0.6
-2.5
0.4
-5.0
0.2
10.0 Explanation
0.0
Candidate graph Graph
Random graph
Explanation
Candidate graph Graph
Random graph
(a)
(b)
Figure 4: Baxplot of the distributions of consensus (a) and authority (b) for existing target events corre- sponding to distrust edges in the first test snapshot. The three boxplots refer to the explanatory subgraphs (Explanation), the computational graph (Candidate graph), and the random graphs
6 Conclusion
In this work, we designed a framework for evaluating explainability techniques on discrete-time graph neural networks, a challenging, unexplored, and important area, since the numerous applications of temporal GNNs in financial, collaboration, and social network analysis. We can outline the best explainability techniques for discrete-time GNNs based the experiments. Overall, GNNExplainer, Saliency, and Integrated Gradients represent the best choices regarding fidelity, efficiency, and readability trade-off. In terms of performance, the latter two are more tailored for model evolution methods, while GNNExplainer is more suited for embedding evolution models. Saliency and Integrated Gradients are far more efficient than GNNExplainer, while the three techniques exhibit approximately the same level of readability. We summarize the best choices using a decision tree in Figure We hope our framework will push the researchers to develop new techniques in this area.
11

---

Published in Transactions on Machine Learning Research (05/2025)
Emiiodiing
Modde Exdiddn
Exclutoo
GNN
Readabilty
Integrated
Salence
Saleway
Grudients
Gradienta
Figure 5: Decision tree for selecting the best explainability techniques for discrete-time GNNs.
5423
4003
(a)
(b)
(c)
Figure 6: Explanations provided by GNNExplainer for ROLAND-GRU on the existence of total distrust votes (edges) in BitcoinOTC. The node IDs refer to real nodes available in the dataset. Given the target event c = (u, v,t), t), the light blue node is u and the light pink is v. Edges exhibit two labels: the first is the timestamp, and the second is the vote.
Broader Impact Statement
The explainability of discreto-time temporal graph neural networks has significant implications across do- mains such as finance, social, or collaboration networks, where decisions must be interpretable and trust- worthy. By providing framework to evaluate explainability techniques, our work contributes positively by enhancing transparency and accountability in Al-driven decision-making, However, potential risks include the misuse of explanations to manipulate game models, as well as the over-reliance on explainability tech- niques that may not fully capture the complexity of temporal graph data. System administrators should use explanations as supportive insights rather than definitive answers, ensuring that human expertise remains central to critical decision-making. To mitigate these risks, we encourage the responsible deployment of explainability methods, continuuus validation of explanations in real-world settings, and interdisciplinary collaboration to ensure fairness and robustness in applications of temporal GNNs. For instance, may act at the link prediction level by adopting different methods to increase fairness, extensively surveyed in Wang et al. (2023).
Acknowledgments
This work has been partially funded by the Italian Ministry of University and Research (MUR) and the European Union NextGeneratton.ee in the framework of the PRIN 2022 project "AWESOME: Analysis framework for WEb3 SOcial MEdia" - CUP: 153D23003680006; by the National Center for Gene Therapy and Drugs Based on RNA Technology MUR (Project no. CN 00000041) funded by NexiGeneration EC program; and by project SERICS (PE00000014) under the NRRP MUR program funded by the EU NGEU,, Views and opinions expressed are however those of the authors only and do not necessarily reflect those of the European Union or the Italian MUR. Neither the European Union nor the Italian MUR can be held responsible for them.
12

---

Published in Transactions on Machine Learning Research (05/2025)
References
Kenza Amara, Zhitao Ying, Zitao Zhang, Zhichao Han, Yang Zhao, Yinan Shan, Ulri Brandes, Scbastian Schemm,, and Ce Zhang. Graphframex: Towards systematic evaluation of explainability methods for graph neural networks. In LoG, volume 198 of Proceedings of Machine Learning Research, pp. 44. PMLR, 2022.
Federico Baldassarre and Hossein Azizpour. Explainability techniques for graph convolutional networks. CoRR,
2019.
Albert-Laszlo Barabasi and Marton Posfai. Network science. Cambridge University Press, Cambridge, 2016. ISBN 9781107076266 1107076269. URL
Jialin Chen and Rex Ying. Tempme: Towards the explainability of temporal graph neural networks via motif discovery. In NewrIPS, 2023.
A. P. Dawid. Prequential data analysis. Lecture Notes Notes-Monogruph Series, 17:113 126, 1992. ISSN 07492170.
URL http::/www. jstor. htgg/taable44355623
Manuel Dileo and Matteo Zignani. Discrete-time graph neural networks for transaction prediction in web3 social platforms. Machine Learning, Jun 2024. ISSN 1573-0565. doi: 10.1007/s109994-02---0000 URL
Manuel Dilco, Mattco Zignani, and Sabrina Gaito. Temporal graph learning for dynamic link pre- diction with text in online social networks. Machine Learning, Nov 2023. ISSN 1573-0565. doi:
10.1007/1100994--23--47777
URL httpsp///oaa.orr///00000////0000
06475-x.
Dongqi Fu and Jingrui He. Dppin: A biological repository of dynamic protein-protein interaction network data. In 2022 IEEE International Conference on Big Data (Big Data), pp. 5269-5277, 2022. doi: 10. 1109 Btt//B/DDaaa555622222...000
Julia Gastinger, Christian Meilicke, Federico Errica, Timo Sztyler, Anett Schülke, and Heiner Stucken- schmidt. History repeats itself: A baseline for temporal knowledge graph forecasting. In IJCAI, pp. 4016-4024. ijeai.org, 2024.
Michele Guerra, Simone Scardapane, and Filippo Maria Bianchi. Interpreting temporal graph neural net- works with koopman theory, 2024. URL https://arxiv: http:/www..r//bb//////1// 13469.
Wenchong He, Minh N. Vu, Zhe Jiang, and My T. Thai. An explainer for temporal graph neural networks. In GLOBECOM, pp. 6384-6389. IEEE, 2022.
Qiang Huang, Makoto Yamada, Yuan Tian, Dinesh Singh, and Yi Chang, Graphlime: Local interpretable model explanations for graph neural networks. IEEE Trans. Knowl. Dala Eng., 35(7):6968 6972, 2023a.
Shenyang Huang, Farimah Poursafaei, Jacob Danovitch, Matthias Fey, Weihua Hu, Emanuele Rossi, Jure Leskovee, Michael M. Bronstein, Guillaume Rabusscau, and Reihanch Rabbany. Temporal graph bench- mark for machine learning on temporal graphs. CoRR, abs/2307. 00006,0 2023b.
Shenyang Huang, Farimah Poursafaei, Reihaneh Rabbany, Guillaume Rabusseau, and Emanuele Rossi. UTG: Towards unified view of snapshot and event based models for temporal graphs. In The Third Learning on Graphs Conference, 2024. URL https://openrevieet 1 wit.mm/dddddxxxgggggg.
net/forum?
Gueorgi Kossinets and Duncan Watts. Empirical analysis of an evolving social network. Science, 311 (8757):88 90, 2006.
Antonio Longa, Veronica Lachi, Gabriele Santin, Monica Bianchini, Bruno Lepri, Pietro Lio, franco scarselli, and Andrea Passcrini. Graph neural networks for temporal graphs: State of the art, open challenges, and opportumities. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https: / httpa//rreerteeeettermmmttptitia
13

---

Published in Transactions on Machine Learning Research (05/2025)
Antonio Longa, Steve Azzolin, Gabricle Santin, Giulia Cencetti, Pictro Lio, Bruno Lepri, and Andrea Passerini. Explaining the explainers in graph neural networks: a comparative study. ACM Compul. Surv., 57(5), Jamuary 2025. ISSN 0360-0300. doi: 30.1145/3696444. URL https://doi.org wwg/oo..//...o. 11445/6696/444.
Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. Proceedings of the 31st International Conference on Neural Information Processing Systems (NewrIPS), pp. 4765 4774, 2017.
Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, and Xiang Zhang. Pa- rameterized explainer for graph neural network. Advances in neural information processing systems, 33: 19620-19631, 2020.
Miller McPherson, Lynn Sovin, and James M Cook. Birds feather: Homophily in social networks. Annual review of sociology, 27(1):415-444 2001.
Siqi Miao, Mia Liu, and Pan Li. Interpretable and generalizable graph learning via stochastic attention mechanism. In International Conference on Machine Learning, pp. 15524-15543, PMLR, 2022.
Ashwin Paranjape, Austin R. Benson, and Jure Leskovec. Motifs in temporal networks. In WSDM, pp. 601 610. ACM, 2017.
Aldo Pareja, Giacomo Domeniconi, Jie Chen, Tengfei Ma, Toyotaro Suzumura, Hiroki Kanezashi, Tim Kaler, Tao Schardl, and Charles Leiserson. Evolvegen: Evolving graph convolutional networks for dynamic graphs In Proceedings of the AAAI conference on artificial intelligence, pp. 5363 5370, 2020.
Jakub Reha, Giulio Lovisotto, Michele Russo, Alessio Gravina, and Claas Grohnfeldt. Anomaly detection in continuous-time temporal provenance graphs. In Temporal Graph Learning Workshop @ NewrIPS 2023, 2023. URL
Youngjoo Seo, Michaäl Defferrard, Pierre Vandergheynst, and Xavier Bresson. Structured sequence model- ing with graph convolutional recurrent networks. In Newral Information Processing: 25th International Conference, ICONIP 2018, Siem Reap, Cambodia, December 18-16, 2018, Proceedings, Port I 25, pp. 362 373. Springer, 2018.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In ICML, volume 70 of Proceedings of Machine Learning Research, pp. 3319-3328. PMLR, 2017.
Minh N. Vu and My T. Thai. Pgm-explainer: Probabilistic graphical model explanations for graph neural networks. In NewrIPS, 2020.
Yifan Wang, Weizhi Ma, Min Zhang, Yiqun Liu, and Shaoping Ma. A survey the fairness of recommender systems. ACM Trans. Inf. Syst., 41)),, feb 2023. ISSN 1046-8188.
Fang Wu, Siyuan Li, Xurui Jin, Yinghui Jiang, Dragomir Radev, Zhangming Niu, and Stan Z. Li. Rethink- ing explaining graph neural networks via non-parametric subgraph matching. In ICML, volume 202 of Proceedings of Machine Learning Research, pp. 37511 135223. PMLR, 2023.
Wenwen Xia, Mincai Lai, Caihua Shan, Yao Zhang, Xinnan Dai, Xiang Li, and Dongsheng Li. Explaining temporal graph models through an explorer-navigator framework. In The Eleventh International Confer- ence Learning Representations, 2023.
Zhitao Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer: Generating explanations for graph neural networks. Advances in neural information processing systems, 32, 2019.
Jiaxuan You, Tianyu Du, and Jure Leskovec. Roland: Graph learning framework for dynamic graphs. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 222, pp. 2358 2366, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450393850. doi: 10. 10.1145/353 /35333368333333000
URL https:/ //dp:://dod.... wtt.....g.1111.... 1145//34444//334677 05155/335467.33333000. 3539300.
14

---

Published in Transactions on Machine Learning Research (05/2025)
Hao Yuan, Haiyang Yu, Jie Wang, Kang Li, and Shuiwang Ji. On explainability of graph neural networks via subgraph explorations. In International Conference on Machine Learning, pp. 12241-12252. PMT.R, 2021.
Zeyang Zhang, Xin Wang, Ziwei Zhang, Haoyang Li, Zhou Qin, and Wenwu Zhu. Dynamic graph neural networks under spatio-temporal distribution shift. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Newral Informalion Processing Systems, volune 35, pp. 6074- 6089. Curran Associates, Inc., 2022. URL https: h/tp://:e///ooeeiingssss newripe neurips.cc/paper. htt:://h26777aaa00e0eseeeeeeeea2
15


Reviews:
[{"review_id": "MVAxti0sJc", "reviewer": "Reviewer_jQQH", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: The structure of the paper is well-organized with intuitive explanations and detailed experimental results to justify the proposed approach.\n\nWeakness: \n\n1. Although the paper is generally well-structured, I found it difficult to fully grasp the implementation challenges associated with the proposed evaluation procedure, particularly Algorithm 1. This may be partly due to my limited background in temporal graph neural networks; however, I believe the paper would benefit from more detailed, implementation-oriented explanations of the underlying challenges, rather than relying primarily on references to existing methods or simply listing the steps involved in the live-update setting and candidate event selection.\n\n2. We need a comprehensive analysis of hyperparameter sensitivities through either theoretical analysis or numerical justification.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper makes four key contributions to the explainability of discrete-time TGNNs: (1) it introduces a new training and evaluation setting that captures the dynamic nature of temporal data; (2) it extends evaluation metrics to account for temporal characteristics like event recurrence; (3) it proposes new baselines for explaining temporal GNNs, including simple models based on recent events and adaptations of existing methods like GNNExplainer; and (4) it provides practical guidance through extensive empirical evaluations highlighting the trade-offs among fidelity, performance, and readability of explanations.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See weakness. In addition, I have a question:\n\nWhat type of dataset is used in Figure 2, and how does the model perform on large-scale graph data?", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "fkRUgsW3FR", "reviewer": "Reviewer_eePV", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths :\n- The paper explores discrete-time TGNNs, an area in explainable AI that has received attention and addressing a gap in existing research. The proposed framework introduces a different evaluation setting and temporal-specific metrics.\n\n- The study is thorough, combining methodological proposals with practical evaluations on diverse datasets and models, and enhancing comparison through different baselines and methods. Detailed experimental setups, including chronological data splits and performance trend analyses, clearly illustrate the framework's practical impact.\n\n-Presentation and style is clear and understandable. The decision tree provided in the conclusion helps readers understand which explainability technique might be best suited depending on whether the focus is on model evolution or embedding evolution, as well as on efficiency versus fidelity trade-offs.\n\nWeaknesses:\n\n- Although the framework is well defined, the paper could benefit from a deeper discussion of its own limitations. For instance: two critical issues that notably affect the performance of explainability frameworks are mentioned but not sufficiently explored or analyzed: \"The high performance of random explanations of GCRN-GRU for Reddit-title is related to the low performance of the base model in the link prediction task,\" and \"Explaining the decisions made on the Email-EU dataset is particularly challenging, likely due to the very high density of events in this dataset.\" A robust evaluation framework for explainability requires a deeper analysys of such problems.\n\n- The introduction section lacks sufficient citations to relevant literature, limiting the contextualization and grounding of the work within existing research.\n\n- \"In contrast,  the fidelity trend of PGExplainer follows the one of the random explainer, suggesting that its explanations\n could be unreliable\". i think that also this fact has to be analyzed more in detail in an evaluation explainability setup, since PGExplainer it is a well known method for GNN explainability and understanding why its fidelity trend follows the one of the random explainer could be of relevance\n\nMinor:\n- Sec.3: \"where V is the of nodes..\", maybe \"set\"?\n- Fig.3 is small and difficult to analyze, should be enlarged", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents a novel framework specifically designed to assess explainability techniques for discrete-time temporal graph neural networks (TGNNs). The key contributions include:\n\nThe authors introduce a live-update (prequential) setting that better captures the evolving dynamics of temporal data. This contrasts with traditional approaches where only training-set information is used for generating explanations. By allowing newly observed events from the test set to inform inference, the framework promises more relevant and informative explanations.\n\nThe authors define temporal-specific evaluation metrics: they extend standard fidelity measures by incorporating the temporal dimension. Alongside fidelity sufficiency, the authors propose additional metrics such as cohesiveness, edge recurrence, reciprocity, and structural homophily. \n\nThe paper tested simple baselines (e.g., LastSnapshot-explainer and TemporalNeighbors-explainer) and more complex methods (GNNExplainer and PGExplainer) offering insights into the trade-offs between fidelity, efficiency, and readability.\n\nFinaly, they work on three real-world datasets covering financial, social, and collaboration networks, providing a comparative analysis of various explainability methods under different TGNN architectures. The results are distilled into performance trends, computation time plots, and a decision tree that guides the selection of appropriate explainability techniques based on application needs.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- critical: provide a deeper analysis about explainability evaluation methods problems and limitations as expressed in \"Strenghts and weakness\" section\n- Add citations in the introduction to better contextualize the work within existing research.\n- solving minor weakness", "broader_impact_concerns": "Authors describe correctly concerns regarding the broader impact that its work can have in the \"Broader Impact Statement\" paragraph", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "7j6qoRtLua", "reviewer": "Reviewer_3BDt", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strength**\n\n- The paper addresses a significant gap in the explainability of discrete-time temporal Graph Neural Networks (TGNNs), an underexplored area despite their widespread use in critical domains like finance and social networks. The proposed framework is a pioneering effort to tailor explainability techniques to the unique challenges of discrete-time TGNNs, distinguishing it from prior work focused on static or continuous-time GNNs.\n\n- Extensive Experimentation: The experiments cover three real-world datasets (BitcoinOTC, Reddit-title, Email-EU) and three state-of-the-art TGNN models (EvolveGCN, GCRN-GRU, ROLAND-GRU), providing a robust empirical basis. The inclusion of a case study on distrust in BitcoinOTC enhances practical relevance.\n\n- Insightful Metrics and Analysis: The introduction of new metrics like edge recurrence, reciprocity, and structural homophily, alongside extensions like Temporal Fidelity (TFid = $\\frac{1}{T_{\\text {test }}} \\sum_{j=1}^{T_{\\text {test }}}\\left(1-\\operatorname{Fid}\\left(K_j\\right)\\right)$ ), addresses temporal dynamics effectively. The performance readability trade-off analysis (e.g., Figure 3) is particularly insightful for practitioners.\n\n\n**Weakness**\n\n- Limited Scope of Explainability Methods: While the paper extends GNNExplainer and PGExplainer, it omits other potentially relevant methods (e.g., SHAP or LIME adaptations) that could provide a broader comparison. This limits the generalizability of the findings. Additional baselines could further strengthen the claims, especially by including newer explainability techniques.\n\n- Dataset and Model Bias: The choice of datasets and base models, while representative, may not fully capture the diversity of discrete-time TGNN applications (e.g., biological networks or traffic systems). The focus on link prediction as the sole task restricts the framework's applicability.\n\n- Some theoretical justifications for the chosen evaluation metrics are not fully explored, leaving some aspects of the approach underexplained. The new metrics (Recurrence, Reciprocity, Homophily) are innovative but lack a detailed justification for their weighting schemes (e.g., 𝑤=1 in Recurrence). Their correlation with human interpretability is assumed rather than empirically validated.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces a novel framework for evaluating explainability techniques tailored to discrete-time temporal Graph Neural Networks (TGNNs), which model evolving graph-structured data in applications like social networks, financial systems, and collaboration networks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Expand Explainability Methods\n\nInclude more explainability techniques, such as a perturbation-based method (e.g., SHAP) or a surrogate model approach, to broaden the comparison. Please see this Github page to select several state-of-the-art and cutting-edge methods: https://github.com/flyingdoog/awesome-graph-explainability-papers. The algorithms tested in the paper, such as GNNExplainer and PGExplainer, are too old. \n\n- Diversify Datasets and Tasks\n\nTest the framework on an additional dataset (e.g., a traffic or biological network) and consider an alternative task (e.g., node classification).\n\n- Expand theoretical background\n\n Providing a more thorough theoretical justification for the chosen evaluation metrics would strengthen the contribution and make the methodology more robust. For instance, provide a theoretical or empirical justification for the choice of $w=1$ in Recurrence (Recurrence $(e)=w$ if $\\exists\\left(u, v, t^{\\prime}\\right) \\in \\mathcal{G}_{\\exp }(e)$ s.t. $t^{\\prime}<t$ ) and similar simplifications. Conduct a small user study or cite prior work to validate the humanreadability of Recurrence, Reciprocity, and Homophily.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers recommend accepting the papers and are satisfied with the authors' changes. One reviewer wants to see experimental evaluation on larger datasets and against SOTA methods.
Decision: Accept (poster)