Paper: Causality-Inspired Spatial-Temporal Explanations for Dynamic Graph Neural Networks (ICLR-2024)

Reviewer comment (Theory):
Equation 1 should be described in detail. Since causal relationships ($C$) consist of dynamic ($D$) and static ($S$) relationships, there should exist $P(S) = P(C) - P(D)$. I guess this equation is derived from it, but I can't see the logical derivation.

Author response:
**W2:** In Equation 1, we merge the estimation of $P(\mathcal{Y}|do(\mathcal{C}))$ with that of $P(\mathcal{Y}|do(\mathcal{D}))$. The detailed derivation process of backdoor adjustment can be shown as:

\begin{equation}
\begin{aligned}
P(\mathcal{Y}|do(\mathcal{D})) &= \sum P(\mathcal{Y}|do(\mathcal{D}), \mathcal{S})P(\mathcal{S}|do(\mathcal{D})) \\
&= \sum P(\mathcal{Y}|do(\mathcal{C}))P(\mathcal{S})\\
&= \sum P(\mathcal{S})\sum P(\mathcal{Y}|do(\mathcal{C}),\mathcal{T})P(\mathcal{T}|do(\mathcal{C})\\
&= \sum P(\mathcal{S})\sum P(\mathcal{Y}|\mathcal{G})P(\mathcal{T}).
\end{aligned}.
\end{equation}

We add this in Appendix A.4 in the revision.

Reviewer comment (Novelty):
Some important literature is missing, e.g., CauSTG for capturing invariant relations targets temporal shifts in ST graphs [1] and CaST for discovering via Structural Causal Model (SCM) with back-door and front-door adjustment [2]. The authors should distinguish the distinctions between the proposed DyGNNExplainer and (CauSTG, CaST), especially the CaST.

Author response:
**W4:** Since our method is the first study on dynamic graph interpretability, there are no directly available datasets suitable for the task of dynamic graph interpretability. So we dynamically transformed some commonly used static graph interpretability datasets. We add this in Section 3.1 in the revision.

Reviewer comment (Reproducibility):
In Equation 7, $e$ is not defined in this paper, and the implementation of $s( \cdot , \cdot )$ is also not provided.

Author response:
**W5:** You recommend some benchmarks for graph representation and graph generalization. But it is hard to compare our method with them since the task in our model is totally different from theirs. Graph representation and graph generalization models are the target models we want to explain. For interpretable methods, the target model is model-agnostic. The interpretability task is to provide good explanations for these models. We make this more clear in Section 3.1.

Additionally, our methods of decoupling dynamic and static relationships are also different from CaST. To disentangle the static relationship and the dynamic relationship, we propose the constraint that dynamic relationships evolve over time steps but static relationships are independent across each time step. Consequently, we leverage the pre-trained target DyGNN model to guarantee the essential temporal correlation between neighboring subgraphs for the dynamic relationships and identify the rest independent causal information to the static relationships.

Reviewer comment (Theory):
Does $||A||$ operate by summing all elements of $A$ in Equation 14? Besides, does Equation 14 exist error? To satisying the sparsity requirement of causal and dynamic causal graph set, whether Equation 14 should be replaced by $\frac{{||A_t^C|{|_1} + ||A_t^S|{|_1}}}{{||{A_t}|{|_1}}}$?

Author response:
**W6:** We utilize contrastive learning to ensure the semantic similarity between the causal embedding $\mathbf{e}^{\mathcal{C}}_t$ and the original embedding $\mathbf{e}_t$ while enlarging the semantic distance between the causal embedding $\mathbf{e}^{\mathcal{C}}_t$ and the trivial embedding $\mathbf{e}^{\mathcal{T}}_t$. $s(\cdot,\cdot)$ measures the similarity, we utilize the dot product here. We add the implementation in Section 2.5 in the revision.

Reviewer comment (Writing):
$\Theta$ should be replaced by $\Psi$ in the last line of Section 2.

Author response:
**W8:** We are sorry that one of the data in Table 2 was filled in incorrectly. The best performance in DTree-Grid is obtained by DyGNNExplainer. We have corrected the typo in Table 2 in the revision. And we have corrected $\Theta$ with $\Psi$ in the last line of Section 2.

Reviewer comment (Experiments):
How to interpret the 'dynamic' in DyGNNExplainer? Can the datasets in experiments support the augment raised in this paper, as it seems there are no dynamic graph in experiments?

Author response:
Sorry for our unclear expression, all our experiments are conducted on dynamic datasets. In essence, our case study is conducted on DBA-Shapes to provide a more vivid demonstration of DyGNNExplainer's interpretability. We visually represent both the original graph and the top six weighted edges of the generated causal subgraph across all time steps using DyGNNExplainer.

Regarding temporal interpretability, the scrutiny of variable $t_p$ reveals that DyGNNExplainer adeptly pinpoints the most critical time steps that significantly influence the target node label within the original graph.

In terms of spatial interpretability, DyGNNExplainer proficiently discerns the presence of the 'house' motif within the original graph's concluding time step, offering a compelling explanation for the target node label.

Reviewer comment (Novelty):
Distinguish the distinctions between this work and CaST.

Author response:
**W5:** You recommend some benchmarks for graph representation and graph generalization. But it is hard to compare our method with them since the task in our model is totally different from theirs. Graph representation and graph generalization models are the target models we want to explain. For interpretable methods, the target model is model-agnostic. The interpretability task is to provide good explanations for these models. We make this more clear in Section 3.1.

Additionally, our methods of decoupling dynamic and static relationships are also different from CaST. To disentangle the static relationship and the dynamic relationship, we propose the constraint that dynamic relationships evolve over time steps but static relationships are independent across each time step. Consequently, we leverage the pre-trained target DyGNN model to guarantee the essential temporal correlation between neighboring subgraphs for the dynamic relationships and identify the rest independent causal information to the static relationships.

Reviewer comment (Presentation):
In table 2, the best performance for OrphicX is obtained by DTree-Grid?

Author response:
**W2:** We are sorry that one of the data in Table 2 was filled in incorrectly. The best performance in DTree-Grid is obtained by DyGNNExplainer. We have corrected the typo in Table 2 in the revision.

Reviewer comment (Reproducibility):
The proposal of 4 loss functions can be unfriendly to network training. If even one parameter is improperly tuned, it could lead to significant instability or even failure in network training. The authors should systematically discuss parameter selection techniques or guidelines to aid those who follow in this line of work.

Author response:
**Q2: Loss function hyperparameters**

**A2:** Thank you for your valuable suggestions. In graph interpretation tasks, they usually have many constraints and require different types of loss functions to satisfy these constraints, e.g., OrphicX has 4 loss functions for the static graph interpretation. We supplement the parameter search range and optimal parameter selection of the four constraints of the loss function to help improve the reliability and reproducibility of the results.

In the final optimization objects, the loss function weight parameters $\lambda_1$, $\lambda_2$, $\lambda_3$, and $\lambda_4$ are set from [0.2, 0.4, 0.6, 0.8, 1]. And the best performance is obtained where $\lambda_1=1$, $\lambda_2=0.4$, $\lambda_3=0.2$, and $\lambda_4=0.2$. We add this in Appendix A.2.

Once again, I extend my gratitude for your invaluable suggestions. It remains our primary objective to deliver trustworthy interpretive results.

Reviewer comment (Experiments):
There is a lack of related experiments: although experiments are conducted, there is a shortage of benchmarks in this field. It is recommended that the authors refer to [1] to add more experiments to validate the effectiveness of their DyGNN, such as including the Ogbn-Arxiv dataset. Additionally, an ablation study replacing VGAE-like models is crucial to help others understand the contribution of each model component.

Author response:
**Q3: Shortage of benchmarks and ablation study**

**A3:** Thank you for your valuable questions. Although there have been some works on static graph interpretability, to the best of our knowledge, we are the first to study dynamic graph interpretability. Therefore, we have no existing dynamic graph interpretability benchmarks for comparison, and can only compare with currently competitive static interpretability methods.

On the other hand, you recommend some benchmarks for graph representation and graph generalization. But it is hard to compare our method with them since the task in our model is totally different from theirs. Graph representation and graph generalization models are the target models we want to explain. For interpretable methods, the target model is model-agnostic. The interpretability task is to provide good explanations for these models. We make this more clear in Section 3.1.

Given the novelty of our study on dynamic graph interpretability, there is a dearth of readily available synthetic datasets suitable for this purpose. Consequently, we've undertaken dynamic transformations of commonly used synthetic static graph interpretability datasets such as BA-Shapes. We also use two real-world dynamic graph datasets, Elliptic for node classification and MemeTracker for graph classification. The Ogbn-Arxiv dataset is a real-world data set but a static graph dataset. If we dynamically transform it, it will lose its real meaning. We make this more clear in Section 3.1.

For ablation study, we provide a DyGNNExplainer version without VGAE ('w/o. VGAE'). In this version, we replace the encoder with a simple GCN layer. DyGNNExplainer outperforms the `w/o. VGAE' version. This is primarily due to the superior capabilities of VGAE in harnessing spatial graph information to generate soft masks. We provide this ablation study in Section 3.2.

Reviewer comment (Novelty):
Related work is missing from the paper, especially concerning spatio-temporal related work [5], generalization/extrapolation on graphs, and causality learning [2-4]. The authors should consider these areas to provide a more comprehensive context for their research.

Author response:
**Q4: Related work**

**A4:** Thank you for your valuable suggestions. Although our interpretability task is totally different from the graph generalization task, their method is the model we want to explain and is helpful in our work. Additionally, the spatial-temporal method and the causal method are directly related to our tasks. We add these related works in Section 4 and Appendix 4.3.