Paper: GInX-Eval: Towards In-Distribution Evaluation of Graph Neural Networks Explanations (ICLR-2024)

Reviewer comment (Experiments):
GInX-Eval has to treat the pre-trained model as a white box because it needs to retrain the model during the whole procedure. However, the pre-trained to-be-explained model is not always a white box, especially in real-life applications. The training dataset may not be able to be accessed, or the training cost is high, even the model itself may be not accessible. So, this approach is not easy to apply.

Author response:
One major comment of the reviewer concerns the limited applicability of the method to white-box models. Now we propose a fine-tuning strategy that enables both to overcome the OOD problem and to be generalizable to any black-box models only accessible through API calls. See general comments to all reviewers.

Reviewer comment (Evaluation):
“The highlighted explanation subgraph’s distribution is different from the training data.” Why is different and what’s the nature where this difference comes from?

Author response:
We have added Figure 2 to illustrate the OOD problem.

Reviewer comment (Evaluation):
Why did the explanations’ distribution shift to a better side but not a worse side? For example: the prediction label is 50. A good explanation prediction should be 50. A bad explanation prediction should be 20. However, due to the OOD, the explanation prediction shifts. Why a bad prediction would shift from 20 to 45 and cause an incorrect high faithfulness score, instead of shifts from 20 to 5? As it’s claimed: “However, this edge masking strategy creates Out-Of-Distribution (OOD) graph inputs, so it is unclear if a high faithfulness score comes from the fact that the edge is important or from the distribution shift induced by the edge removal (section 1 paragraph 1)”.

Author response:
Since explanations are out-of-distribution, there is no general rule beyond the observed faithfulness scores. The idea here is that the GNN is fooled and therefore has not the capacity to correctly assess the importance of the remaining edges. Figure 2 very well illustrates that explanations are closer to non-toxic molecules and will therefore be misclassified even though they are good explanations for toxic molecules (if we look at the GT or GinX-Eval results).

Reviewer comment (Evaluation):
There are two removal strategies: “hard” and “soft” removal strategies. I wonder is there any difference between them toward the GNN output? If the outputs f(G_e_hard) and f(G_e_soft) are different, what’s the reason for that?

Author response:
We refer the reviewer to the Appendix where he will find a more detailed analysis of the differences in the GNN output between hard and soft edge removal strategies. In Appendix A.3  we analyze the differences in the GNN output between hard and soft edge removal and in Appendix C.2 we also provide the results for the soft edge removal strategy and describe the similarities and differences with the results with hard selection.

Reviewer comment (Reproducibility):
There should be many hyper-parameters to tune for the evaluated explainer methods, eg: size regularization and temperature in GNNExplainer/PGExplainer. How do you set them and have you tuned them to the best? It would be good to include these details in the main text or supplementary and mention them in the main text since this paper emphasizes on the experiments.

Author response:
We have included the details of hyperparameters for GNNEXplainer and PGExplainer in the Appendix. We fully agree with the reviewer that PGExplainer is extremely sensitive to the choice of hyper-parameters and our final choice is the result of hyper-parameter tuning.

Reviewer comment (Evaluation):
In Figure 1, what’s the random seed for the random baseline, and how many times the experiments are repeated? For AUC evaluation, how do you compute the AUC score? Specifically, for other explainers, we could have an edge weight vector as the explanation and compute the AUC with the ground truth. But for a random baseline, how to decide the weight of the edge?

Author response:
For each experiment (the random baseline included), we have run the experiments on five different seeds (0,1,2,3,4). The random baseline estimates the edge importance sampling from uniform distribution values between 0 and 1. The weight of the edge for the random explanation is therefore a random value between 0 and 1. To compute the AUC score, we simply compare this random edge importance assignment to the ground-truth one. We have added a sentence in section 4.1 to clarify the AUC computation. “The AUC score is computed between the explanatory weighted edge mask and the ground-truth edge mask with binary values in {0,1}.”

Reviewer comment (Experiments):
The GInX-Eval is computed via retraining, and finally evaluating the quality of the explanation of the original on the original pretrained GNN model. However, the GNN behavior would change during retraining. For example: GNN model f_a is trained on the complete training dataset, it could predict the classification according to the explanation sub-graph. But GNN model f_a is trained on the training dataset which drop 50% edges in each graph. If the explanation sub-graphs are already dropped, how could f_b predict the graphs into correct classifications? Would the behavior of the retrained GNN models change and how would it affect the accuracy evaluation? Thus, the experiments are not fully convincing. It would be good to make some clarify.

Author response:
As mentioned in the general comment to all reviewers, we now propose a fine-tuning strategy. When removing t% of the informative edges, f_b(fine-tuned on the reduced train dataset) will not find the important information in the reduced test set - therefore we expect f_b to make wrong predictions and the test accuracy to drop and the GinX score to increase.

Reviewer comment (Writing):
In 3.3.1 GINX SCORE, the description of "top-k edges" is confusing because t is already used as the fraction of the ordered edge set.

Author response:
We have addressed the concerns of the reviewer in the updated version of the paper. As mentioned in the general comment for all reviewers, we have modified section 3 with more consistent mathematical notations and this also modified Equation 3. In 3.3.1, we have corrected the “top-k edges” and clarified that t is the fraction/ratio of edges. We have also completed the incomplete references when papers were published at conferences and added the year for all of them. Concerning the colors, we have chosen colorings to reflect the similarity of methods (green colors for gradient-based methods, warm colors for non-genative ones, and cold colors for generative ones).

Reviewer comment (Writing):
In equation 3, the superscript for G\G_e^t is used without explanation despite the superscript is not used in equation 2.

Author response:
We have addressed the concerns of the reviewer in the updated version of the paper. As mentioned in the general comment for all reviewers, we have modified section 3 with more consistent mathematical notations and this also modified Equation 3. In 3.3.1, we have corrected the “top-k edges” and clarified that t is the fraction/ratio of edges. We have also completed the incomplete references when papers were published at conferences and added the year for all of them. Concerning the colors, we have chosen colorings to reflect the similarity of methods (green colors for gradient-based methods, warm colors for non-genative ones, and cold colors for generative ones).

Reviewer comment (Presentation):
It is very hard for readers to distinguish different colors used in Figures. Some efforts are required for readability such as using different marks.

Author response:
We have addressed the concerns of the reviewer in the updated version of the paper. As mentioned in the general comment for all reviewers, we have modified section 3 with more consistent mathematical notations and this also modified Equation 3. In 3.3.1, we have corrected the “top-k edges” and clarified that t is the fraction/ratio of edges. We have also completed the incomplete references when papers were published at conferences and added the year for all of them. Concerning the colors, we have chosen colorings to reflect the similarity of methods (green colors for gradient-based methods, warm colors for non-genative ones, and cold colors for generative ones).

Reviewer comment (Presentation):
Several references such as Faber et al, Hooker et al, Hsieh et al, and Hu et al lack names of conferences or years of publishing.

Author response:
We have addressed the concerns of the reviewer in the updated version of the paper. As mentioned in the general comment for all reviewers, we have modified section 3 with more consistent mathematical notations and this also modified Equation 3. In 3.3.1, we have corrected the “top-k edges” and clarified that t is the fraction/ratio of edges. We have also completed the incomplete references when papers were published at conferences and added the year for all of them. Concerning the colors, we have chosen colorings to reflect the similarity of methods (green colors for gradient-based methods, warm colors for non-genative ones, and cold colors for generative ones).

Reviewer comment (Evaluation):
What is the difficulty of applying the idea of retraining to the evaluation of explanatory techniques for GNNs compared to those for CNNs?

Author response:
*What is the difficulty of applying the idea of retraining to the evaluation of explanatory techniques for GNNs compared to those for CNNs?*

Thanks for the question. Theoretically, GInX-Eval should be easily transferable to CNN models as well. However, the difficulty with CNNs comes from the nature of the inputs themselves, i.e. images. The challenges arise when removing pixels from images. The removal strategy of pixels when using GInX-Eval with CNNs has to be well justified. Two main challenges should be addressed: 

- As mentioned by Hsieh et al. [1], assigning features with the baseline value might lead to a bias toward features that are far from the baseline. For instance, with images the baseline is black color. If we set the pixels to black in RGB images, this introduces a bias favoring bright pixels. Therefore, CNN fine-tuned on those modified images will often omit important dark objects.
- In addition, removing pixels by setting their value to zero (black pixel) might lead to the CNN confusion between no signal composing the black background and pertinent negative pixels, which are relevant values that are dismissed [2]. 

Therefore, applying GInX-Eval for CNN models requires defining an appropriate pixel removal strategy to overcome those two difficulties.

[1] Hsieh, Cheng-Yu, Chih-Kuan Yeh, Xuanqing Liu, Pradeep Ravikumar, Seungyeon Kim, Sanjiv Kumar, and Cho-Jui Hsieh. n.d. “Evaluations and Methods for Explanation through Robustness Analysis.” https://openreview.net/pdf?id=4dXmpCDGNp7.

[2] Dhurandhar, Amit, Pin-Yu Chen, Ronny Luss, Chun-Chen Tu, Paishun Ting, Karthikeyan Shanmugam, and Payel Das. 2018. “Explanations Based on the Missing: Towards Contrastive Explanations with Pertinent Negatives.” arXiv [cs.AI]. arXiv. https://proceedings.neurips.cc/paper_files/paper/2018/file/c5ff2543b53f4cc0ad3819a36752467b-Paper.pdf.

Reviewer comment (Theory):
Certain aspects of the design are not intuitively clear. Specifically, the rationale behind Equation 4 is not well-explained. Elaborating on the underlying intuition would aid in understanding its relevance and function within the model.

Author response:
As explained in comment 5. below, we have re-formulated the EdgeRank score to the HomophilicRank score and elaborated more on the meaning of this score in the new version of the paper.

Reviewer comment (Writing):
The terms "hard selection" and "soft selection" are used without formal definitions. Providing precise mathematical formulas for these concepts would clarify their meaning and implementation in the context of the proposed method.

Author response:
We have added a mathematical definition of the hard and soft edge selection functions using the notations of the paper to bring more clarity to the paper.

Reviewer comment (Evaluation):
The utility of edge ranking as a metric is questionable. It assumes that the importance of individual edges correlates directly with subgraph importance, an assumption that may not hold true in all cases. Further justification or alternative metrics should be considered.

Author response:
The EdgeRank score does not evaluate the importance of individual edges but the importance of the first fraction $t\in[0.1,0.2,…0.9]$ of edges. We agree with the reviewer that the 10% most important edges might be correlated with the 10-20% edges, leading to an increase in the GInX score only after removing the full 20% edges. However, observing the size of ground-truth explanations for the 6 datasets (between 10 and 30%), we expect the sharpest drop in test accuracy in the first 3 removal steps (t=0.1, t=0.2, t=0.3). Therefore assigning more importance to these first steps makes sense. Now, it is true that within the 3 first steps, important edges might be correlated. 

We follow the reviewer’s suggestion and re-formulate the EdgeRank score to the HomophilicRank score. The HomophilicRank score measures the capacity of a method to rank edges by their correct importance ordering while assigning similar importance weights to correlated edges. This score penalizes methods that for instance only discover a subset of important edges and do not account for the edges correlated to those. The HomophilicRank score favors methods that treat correlated edges on equal footing. It measures the capacity to uniformly assign importance to redundant information rather than only putting importance to a single representative edge and none to the correlated edges.

Reviewer comment (Experiments):
The range of GNN backbones tested is somewhat limited. Incorporating more diverse architectures, such as GCN, would provide a more comprehensive evaluation of the proposed method's effectiveness across different models.

Author response:
We followed the reviewer’s suggestion and added the GCN model. Since the GCN does not take edge features into account we only run GInX-Eval with GCN on the four datasets: BA-2Motifs, BA-HouseGrid, MUTAG, and MNIST_BIN where edge features could be removed without or with small loss of information.