Task-Agnostic Graph Explanations

Abstract: Graph Neural Networks (GNNs) have emerged as powerful tools to encode graph-structured data. Due to their broad applications, there is an increasing need to develop tools to explain how GNNs make decisions given graph-structured data. Existing learning-based GNN explanation approaches are task-specific in training and hence suffer from crucial drawbacks. Specifically, they are incapable of producing explanations for a multitask prediction model with a single explainer. They are also unable to provide explanations in cases where the GNN is trained in a self-supervised manner, and the resulting representations are used in future downstream tasks. To address these limitations, we propose a Task-Agnostic GNN Explainer (TAGE) that is independent of downstream models and trained under self-supervision with no knowledge of downstream tasks. TAGE enables the explanation of GNN embedding models with unseen downstream tasks and allows efficient explanation of multitask models. Our extensive experiments show that TAGE can significantly speed up the explanation efficiency by using the same model to explain predictions for multiple downstream tasks while achieving explanation quality as good as or even better than current state-of-the-art GNN explanation approaches. 


Reviews and author responses:

 Thanks for answering my questions.

Please add the following reference.

Wang, Xiang, Yingxin Wu, An Zhang, Xiangnan He, and Tat-Seng Chua. "Towards multi-grained explainability for graph neural networks." Advances in Neural Information Processing Systems 34 (2021): 18446-18458.

 Thank you for your further suggestions! 

We will include more discussions and conduct ablations on the two points. Specifically, for the fine-tuning point, our method can for sure be adopted to the setting. We evaluate the method with two-stage setting to be consistent to the identified challenge and our motivation. Technically, the TAGE can still be used in same ways (e.g., to explain end-to-end models or fine-tuned models) as typical explanation approaches do. The only difference would be their sets of parameters as you mentioned previously.

 Thanks for the authors' explanation and added experiments.

Regarding performance-interoperability tradeoff, as shown in the table, indeed a two-stage pipeline without fine-tuning could limit the performance to some extent. Is it possible that authors discuss some possibility to extend the current method into the fine-tuning framework (so that the performance won't get affected)? (probably just as future works as the current contribution is already significant), I think this could make this work more solid and with wider usage.

Regarding the Laplacian distribution, in the paper authors mention "Empirically, the Laplacian assumption holds on all datasets we work with as we observe that gradients follow a Laplace distribution", it would be great if the authors could provide some discussion or visualization of this finding as it looks very interesting. Also, it would be great to also try different other distribution  as ablation study to understand how important and sensitive this prior distribution choice. 

 Dear Reviewer S3nw,

Thank you for your further feedback and updated evaluation! We will highlight the definition of "task-agnostic" in abstract and introduction, and make our statements more clear to avoid any confusion.

Regarding the condition vector, the using of Laplace distribution is based the assumption that good representations should be informative with a low dimension redundancy, while a downstream task usually only related to a specific portion of those information corresponding to a small subset of the dimensions. It is verified by our empirical observation on all existing models. When we have no knowledge about the downstream model, we can use the Laplace distribution as a default. However, we can always use other distributions if we have other assumptions or observations.

Regarding the  fine-tuning vs. fix encoder point, we understand the concern that the explanation evaluation becomes meaningless when the model is not properly learned (under some cases such as collapsed models). Below are the ROC-AUC scores for several MoleculeNet tasks. Though slightly underperforming fine-tuned models, the two-stage trained models in our study still have better performance compared to end-to-end trained models without pretraining. It indicates that the models for explanation are properly learned and the fidelity scores are meaningful.

|dataset|bbbp|bace|sider|
|--|--|--|--|
|End-to-End|65.8|70.1|57.3|
|Two-stage, no fine-tune|67.0|77.8|58.2|
|Pretrain+fine-tune|68.7|84.5|62.7|


 Thank the authors for the response. There was some misunderstanding of the work, for which I apologize. Now I have a better understanding of the work, and I have raised my score. I have a few more comments though.

The description of "task agnostic" can be a bit misleading, I saw other reviewers raise similar concerns as well. You may want to better clarify this part to avoid confusion.

The condition vector p follows a multivariate Laplace distribution is an empirical observation right? Then for different downstream tasks, it is possible that p follows a different distribution. What are you going to do in that case? Do you train a downstream explainer first, obtain an empirical distribution of p, and then sample from it to train the task-agnostic explainer?

For the fine-tuning vs. fix encoder training point I made in my initial comments. I agree that an explainer should explain both good and bad models, which helps us understand model limitations and improve them, e.g. model expressiveness. The difference between these two settings to me, however, is more like two sets of parameters learned differently for the same model. That is why I asked the question. Metrics like fidelity is not always meaningful if the model parameters are not properly learned. For example, you don't want to randomly initialize a model and measure fidelity of explanation on it without training. Or in a more extreme case, setting all parameters of a model to be zero makes fidelity always zero. 


 Dear Reviewer WNap,

Thank you for your valuable comments and feedback. Below are our itemized responses to the comments and questions.

> The section on related works might need more words to help the readers understand your work. For example, it would be better to add some discussions for [1]. Moreover, the mask-based explanation models are a branch of graph explanation methods. The differences between the proposed and the existing ones may need to be clarified. [1]. Siqi Miao, Miaoyuan Liu, Pan Li; Interpretable and Generalizable Graph Learning via Stochastic Attention Mechanism; ICML 2022.


Thank you for pointing out the missing related work. We will include more discussions regarding the related work and cite the work properly.


> The paper may lack some discussions about its limitations. As line 293 said, the proposed method may have poorer performance compared with task-specific explainers. When is it better to use the proposed method rather than others?


The proposed method is the most suitable under the two-stage and multi-task settings. Our experiments show that even on the single task setting and studied datasets, TAGE can still have the same or better performance than baseline methods. However, when the model and datasets to be explained are too complicated, the embedding explainer in TAGE may require more parameters to have enough expressiveness for the task-agnostic explanation. In those cases, typical task-specific explainers could be more efficient and more suitable. In addition, we have discussed other limitations of TAGE in Appendix I.


 Dear Reviewer S3nw,

Thank you for your valuable comments and feedback. Below are our itemized responses to the comments and questions.

> The major weakness to me is the motivation of the work. In what cases will we truly need a task-agnostic explanation? If our goal is to understand why a model makes such a prediction, then how could we hide the prediction in advance when learning an explainer? What formula 1 says is essentially "find the important subgraph that gives me a similar encoded embedding as if I encoded the whole graph". Why would this be reasonable in practice when we are looking into each specific task? Don't we want to give explanations as specific as possible? Personally, I don't think we could assume there is a subgraph that is generally important for all tasks since these tasks are indeed different. (similar to No-Free-Lunch theorem). This is also part of the reason why we have the sparsity metric for evaluation to avoid being too general. Probably the authors could explain or discuss their motivation and where this method is useful more clearly.


We do not claim we should have a single explanation for all tasks. The proposed TAGE can provide different explanation results for different tasks. What “task-agnostic explanation” means is that we want the explainer to be trained without downstream tasks, but it can provide different explanation results for any given downstream tasks, exactly as you mentioned, “we want to give explanations as specific as possible”. 

Please refer to our definition of task-agnostic explanation in Section 2.2, quoted as “*In particular, the task-agnostic explanation trains only one explainer to explain the embedding model, which should satisfy the following features. First, given an input graph, the explainer should be able to provide different explanations according to specific downstream tasks being studied. Second, the explainer can be trained when only the embedding model is available, e.g., at the first stage of a two-stage training paradigm, regardless of the presence of downstream tasks*.”

What Formula 1 says is that “***given the importance vector p from a downstream task***, find the important subgraph that gives me a similar encoded embedding ***specifically on the important dimensions as indicated by p*** as if I encoded the whole graph”.


> I also have some concerns about the setting of the model. Why set the pipeline into two separated parts? As far as I know, the GNN encoder is often used in a pretrain-finetune fashion which has higher accuracy than fixing the encoder for downstream tasks, like the context prediction pretraining on MoleculeNet datasets in your experiments. Why do we want to explain a less accurate model given a more accurate setting is easily available. I understand efficiency is a concern, but may be sacrificing too much accuracy to achieve a task-agnostic explanation isn't worth doing?


The two-stage training setting is very common in the industrial setting (e.g., PinSage [1]) due to scaling, latency, and label sparsity issues. As we focus on addressing challenges in this setting, we perform the same two-stage training for the GNN model and downstream models to be consistent. Moreover, the evaluation metrics - fidelity and sparsity, do not require the model itself to be as accurate as possible. They quantify the explanation quality based on whether the explanation can reflect the model's behavior (no matter good or not). In fact, it is very important that an explainer can also provide faithful explanations to bad models, especially in industrial settings, so that developers can gain insight on how to improve the model.

[1] Ying et al., Graph Convolutional Neural Networks for Web-Scale Recommender Systems. KDD 2018.

 > The discussion of related work is not very thorough. Some recent works are missing, e.g., [1] and [2]. Some works like SubgraphX and Gem are cited but not discussed specifically.

Thank you for pointing out the related work. We will include more discussion of them and cite them properly.


> In the paper authors use some pre-training tasks to initialize GNN. In the experiments (Figure 3), are all the baselines run on the pre-trained model? Or some will fine-tune on downstream tasks? What is the prediction accuracy after fine-tuning? I think this is important as we want to make sure comparing the explanation ability with the same model.


We use the same model trained under the two-stage setting for the evaluations of all explanation methods so that the comparison is fair. Also note that, as mentioned in the previous response, the evaluation with fidelity and sparsity scores does not require the model to be as accurate as possible.


> Could this approach work for other self-supervised pre-training tasks? Did the authors try any of them? As I think the approach should be a general framework to understand a pre-trained model.


The proposed TAGE should work on GNN models trained with any approaches, just as any explanation methods do. In fact, three different approaches, context prediction, GRACE, and GAE are used for the three datasets MoleculeNet, PPI, and EPN (please refer to Section 4.2). The TAGE framework can be also generalized to pre-trained models for other data type such as Transformers for text data. However, we focus on the explanation of graphs in this work.



> As mentioned in "strength and weakness", I suggest the authors explain more clearly why and when this task-agnostic model might work, and when it may fail. I doubt there should be a explanation that could be suitable for different tasks. Probably the authors could emphasize their contribution more on how to explain a pre-trained model instead of saying its generalized to different tasks. Probably with a few label and fine-tuning would be more reasonable for such as task-agnostic model.


As mentioned in our response to "strength and weakness", we are not providing a single explanation to all downstream tasks. On the contrary, we aim to provide specific explanations to tasks, but with a single explanation model. In addition to the setting, we have also discussed the limitations in Appendix I.



 Dear Reviewer 61fV,

Thank you for your valuable comments and feedback. Below are our itemized responses to the comments and questions.

> The claim of the method being task agnostic seems a pit provocative though it's largely true.


The “task agnostic” refers to that the learning of the explainer does not require downstream tasks but can be used to explain any GNNs with downstream models. The explanation results themselves are still associated with downstream tasks. We will make our statements more clear to avoid any confusion.


> The definition of multi-tasks in the examples are debatable. Are there really 121 tasks in PPI? Given the limitations in evaluating the quality of the explanations (from an end user perspective), it's hard to judge if the explanations of TAGE are qualitatively better for each of the tasks.


Yes. There are 121 binary-classification tasks for each node in PPI [1] (Stats table). We follow the common evaluation setting with fidelity to quantify the explanation quality, which does not require explanation ground truths. Experimentally, we only compare the first few tasks because the running time for baseline methods is too long.

[1] https://pytorch-geometric.readthedocs.io/en/latest/modules/datasets.html#torch_geometric.datasets.PPI


> It would be better if the discussion was consistently on one of the datasets (MoleculeNet), while other datasets are described in the appendix.


Thank you for the suggestion. We will organize results and discussions accordingly.


> Was there any ablation done on using the embedding explainer along with PG Explainer for the downstream task? Is the use of the Laplace distribution a problem there?


PGExplainer is proposed to target the explanation for GNN models (i.e. models including graph convolutional layers). As the downstream models are fully-connected layers, PGExplainer is not applicable to work with the embedding explainer and perform explanations for those parts.


> What other distributions can be used besides multivariate Laplace distribution?


What distribution to use mainly depends on what assumption we have about the true importance scores. In our case, we assume the important dimensions are sparse and observe in practice that the true distribution is a Laplace. In some other cases, we may also use Gaussian, or even truncated uniform distribution when we assume or observe otherwise.



 Dear Reviewer 7PtK,

Thank you for your valuable comments and feedback. Below are our itemized responses to the comments and questions.

> Q1: since directly optimizing the MI might be difficult, can we just (naively) optimize RMSE or some other distance metrics?


RMSE could achieve a similar effect to MI. Minimizing RMSE can be considered as maximizing the likelihood of P(Z|Z_s). However, it requires stronger assumptions on the output embeddings from the GNN encoder, which do not necessarily hold. For example, embeddings of the original graph and subgraph must belong to Gaussian distribution with the same variance at all dimensions. Moreover, it restricts the embeddings to be (linearly) correlated with each other, which is not necessary and may limit the overall performance. As the only goal is to let Z_s contain as much mutual information as possible with Z (instead of having an exactly similar form), using MI estimations is more suitable in our case.


> Q2: During training it seems the \Tau_{\theta} selects the subgraph and feed it into GNN and optimize MI end2end. However the edge selection requires a threshold, which is literally a step function that is non-differentiable. I wonder how this is handled in the paper?


During training, the edge selection is based on the reparametrization with Gumbel sigmoid [1] on top of the importance scores, following the same setting in PGExplainer. The reparametrization adds randomness to the scores and sharpens the scores to values near 0 or 1. In the GNN computation, we do not really drop the unselected edges at the input but use edge masks to restrict the information aggregation from neighbor nodes through unselected edges. We will include more details to make this clear in the paper.

[1] Jang, Eric, Shixiang Gu, and Ben Poole. "Categorical reparameterization with gumbel-softmax." arXiv preprint arXiv:1611.01144 (2016).


> Q3: The subgraph selection is done via the selection of each edge independently. It seems a bit limited in the sense of expressiveness, as sometime maybe a subgraph jointly controls the certain dimension of the embedding. I’d like to learn the authors opinion on this.


Selecting edges independently is a common setting in several previous works. In our opinion, the current setting can still work when edges in a subgraph jointly affect the prediction. In this case, the mutual information on corresponding dimensions achieves maximum if and only if all those edges are selected. From the explainer view, as the appearance of the joint edges affects the final prediction of the model when training the explainer (as generative models), it gives these edges high probabilities to make them appear together.


 This paper proposed a framework to do graph based explanations. Compared to the existing explainers, the proposed method is able to explain the predictive outcome for any downstream tasks, without re-training the explainer for every new tasks. The core idea is to decompose the explanation into node embedding explanation with subgraph, and downstream classifier explanation with node embeddings. Experiments show that the proposed method is effective at explaining multiple objectives, while also being much faster than alternatives. 
 Overall the paper is well motivated. The runtime required by existing explainers like GNNExplainer is indeed a bottleneck in applying the model. This paper proposed a simple and timely solution to the explainer community, and from the experiments it turns out to be efficient and effective. I think the contribution regarding this specific problem is valid. 

The paper is overall well written and easy to follow. 

There are some concerns regarding the current draft, mainly about some clarifications and the rationales for the design choices. Please see my questions below. 
 Q1: since directly optimizing the MI might be difficult, can we just (naively) optimize RMSE or some other distance metrics? 

Q2: During training it seems the \Tau_{\theta} selects the subgraph and feed it into GNN and optimize MI end2end. However the edge selection requires a threshold, which is literally a step function that is non-differentiable. I wonder how this is handled in the paper? 

Q3: The subgraph selection is done via the selection of each edge independently. It seems a bit limited in the sense of expressiveness, as sometime maybe a subgraph jointly controls the certain dimension of the embedding. I’d like to learn the authors opinion on this. Please check the questions in the above section.

 The paper sets out to achieve two main objectives - a Graph Neural Network explainer TAGE that is task agnostic, and hence second, can be used in multi-task prediction. The task agnostic claim is because TAGE uses gradient methods for the downstream tasks along with an explainer for GNN embedding which is task agnostic. The gradient methods produce a condition vector during inference. But during training a method using multivariate Laplace distribution is used for generating the condition vector. The overall objective is to maximize the mutual information between the embedding produced from the original graph and the embedding produced from the subgraph given by explainers.

Overall, the approach sounds interesting and useful. But there might be some misgivings about the claims - whether the method is really task agnostic or whether the datasets chosen really have multi-task predictions. Strengths:

1. The paper makes a lot of practical sense given how GNN models are trained. Explaining at the GNN embedding level, and using it along with a task specific explainer. The use of gradient based methods for the latter seems reasonable. If the cost of explaining multiple tasks is amortized and efficient, this work is practically useful.

2. The paper does a good job of setting out the objectives to the point that there is some redundancy in the earlier sections. Descriptions of the following are well written.

a. Maximizing mutual information (MI) between the embeddings as the objective,
b. how to make this MI objective tractable (JSE, NCE),
c. using condition vectors from downstream tasks during inference and from the Laplace distribution during training
d. evaluation for quality and universal explanation ability

3. The visualization and the observation of closeness of chosen edges in the explanation (for MoleculeNet) are very interesting.

Weaknesses:

1. The claim of the method being task agnostic seems a pit provocative though it's largely true.
2. The definition of multi-tasks in the examples are debatable. Are there really 121 tasks in PPI? Given the limitations in evaluating the quality of the explanations (from an end user perspective), it's hard to judge if the explanations of TAGE are qualitatively better for each of the tasks.
3. It would be better if the discussion was consistently on one of the datasets (MoleculeNet), while other datasets are described in the appendix. 1. Was there any ablation done on using the embedding explainer along with PG Explainer for the down stream task? Is the use of the Laplace distribution a problem there?
2. What other distributions can be used besides multivariate Laplace distribution?
 The authors mention the limitation of all inductive approaches when the training and inference data are inconsistent.

 This work proposes a method to explain GNNs on node classification and graph classifications without knowing the downstream task in advance. It separates the GNN-based models on these tasks into two parts, i.e., a GNN embedding model and an MLP downstream model. The former takes the input graph and encodes it to node/graph embeddings. The latter takes the output from the former and makes predictions for the downstream tasks. Two explainers are developed regarding these two parts respectively, and the work focuses on the embedding explainer for the first part. The embedding model is pretrained unsupervised. The embedding explainer is trained to extract a subgraph that maximizes the mutual information between the whole graph embedding and the subgraph embedding, without knowing the downstream task. Overall, the proposed method can be more efficient when explaining multiple tasks at once. Strengths:

1. The proposed explainer runs faster than existing explainers.
2. The paper is clearly written and easy to follow.

Weaknesses:

The major weakness to me is the motivation of the work. In what cases will we truly need a task-agnostic explanation? If our goal is to understand why a model makes such a prediction, then how could we hide the prediction in advance when learning an explainer?
What formula 1 says is essentially "find the important subgraph that gives me a similar encoded embedding as if I encoded the whole graph". Why would this be reasonable in practice when we are looking into each specific task? Don't we want to give explanations as specific as possible? Personally, I don't think we could assume there is a subgraph that is generally important for all tasks since these tasks are indeed different. (similar to No-Free-Lunch theorem). This is also part of the reason why we have the sparsity metric for evaluation to avoid being too general. Probably the authors could explain or discuss their motivation and where this method is useful more clearly.

I also have some concerns about the setting of the model. Why set the pipeline into two separated parts? As far as I know, the GNN encoder is often used in a pretrain-finetune fashion which has higher accuracy than fixing the encoder for downstream tasks, like the context prediction pretraining on MoleculeNet datasets in your experiments. Why do we want to explain a less accurate model given a more accurate setting is easily available. I understand efficiency is a concern, but may be sacrificing too much accuracy to achieve a task-agnostic explanation isn't worth doing?

The discussion of related work is not very thorough. Some recent works are missing, e.g., [1] and [2]. Some works like SubgraphX and Gem are cited but not discussed specifically.

[1] Wang, X., Wu, Y., Zhang, A., He, X., & Chua, T. S. (2021). Towards multi-grained explainability for graph neural networks. Advances in Neural Information Processing Systems, 34, 18446-18458.

[2] Lin, W., Lan, H., Wang, H., & Li, B. (2022). Orphicx: A causality-inspired latent variable model for interpreting graph neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 13729-13738).

Minor:
Notation D mismatch in lines 90 and 145
  1. In the paper authors use some pre-training task to initialize GNN. In the experiments (Figure 3), are all the baselines run on the pre-trained model? Or some will fine-tune on downstream tasks? What is the prediction accuracy after fine-tuning? I think this is important as we want to make sure comparing the explanation ability with the same model.

2. Could this approach work for other self-supervised pre-training tasks? Did the authors try any of them? As I think the approach should be a general framework to understand a pre-trained model.

3. As mentioned in "strength and weakness", I suggest the authors explain more clearly why and when this task-agnostic model might work, and when it may fail. I doubt there should be a explanation that could be suitable for different tasks. Probably the authors could emphasize their contribution more on how to explain a pre-trained model instead of saying its generalized to different tasks. Probably with a few label and fine-tuning would be more reasonable for such as task-agnostic model. The authors list the limitation on broad usage setting (cannot do inductive and black-box explanation). But I expect more discussion on whether such a task-agnostic explanation could really be useful for different task, when will it fails (e.g., some task-specific feature could not be captured during pre-training)

 The paper proposes a novel graph explainer independent of specific downstream tasks. The proposed explainer consists of two parts: embedding explainer and downstream explainers. This task-agnostic attribute of the embedding explainer is implemented by a  self-supervised method via Mutual Information. The extensive quantitative experiments and visualizations demonstrate the merits of the proposed method. Strengths:
1. The proposed task-agnostic graph explanator has some novelty in this field. The task agnostic explanation may bring new insight from different views compared with task-specific tasks. This character might be very interesting for some directions like drug discovery via graph learning. 
2. The implementation of self-supervised training for the embedding model is interesting. It is also an interesting attempt to apply MI objective.
3. The experiments show improved computational time cost compared to baselines.

Weaknesses:
1. The section on related works might need more words to help the readers understand your work. For example, it would be better to add some discussions for [1]. Moreover, the mask-based explanation models are a branch of graph explanation methods. The differences between the proposed and the existing ones may need to be clarified.
[1]. Siqi Miao, Miaoyuan Liu, Pan Li; Interpretable and Generalizable Graph Learning via Stochastic Attention Mechanism; ICML 2022.
2. The paper may lack some discussions about its limitations.  As line 293 said, the proposed method may have poorer performance compared with task-specific explainers. When it is better to use the proposed method rather than others?
 1. When it is better to use the proposed method rather than others? The work has pointed out some limitations. I encourage the authors to add more about its application condition.

Meta-review: This paper proposed an extension to existing graph explanation approaches, by introducing the task-agnostic concept that decouples the representation learning and task adaptation modules. All reviewers agree that the paper is well motivated and has practical values. During the rebuttal the authors have also addressed several concerns raised by the reviewers. Based on the agreement from all the reviewers we recommend the acceptance of this paper.


Decision: Accept