Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary
This paper presents a new interpreting algorithm for understanding graph neural network (GNN) models for natural language processing (NLP) tasks. The main idea is to remove redundant edges for each layer of GNN after training the model. To find the redundant edges, the authors suggest minimizing the difference between the output of GNN with the original graph and inputs and the output of GNN with sub-graphs that remove some edges. The loss function forms the L0 norm with the Lagrange multiplier. The algorithm also learns baseline edge information that can replace the values of target edges. Experiment with synthetic datasets shows that suggested algorithms can find the important edges for the task rather than others that find only part of the important edges. Experiments on question answering and semantic role labeling with the real dataset, the suggested algorithm shows the percentage of retained edges and evidence that support the usefulness of the GNN algorithm for the task rather than LSTM.

The main strength of this paper is suggesting a sound and straightforward learning algorithm identify important edges for the task from the trained GNN model. Even the idea is simple; it can make people understand the GNN models better. I like to read the synthetic experiment section since it makes me admit to why the suggested algorithm much better than other algorithms. I also like to see the usefulness of the algorithm for the real NLP dataset.

What concerns me most is the lack of explanation about the amortized. The authors mentioned it in the Introduction section with Schulz et al. 2020 paper. But it is hard to imagine the amortized and non-amortized algorithms in the experiment sections. Following the result in Figure 2, amortization is a key factor in identifying the important edges. Could the authors explain it more about this on the suggested algorithm?



Overall, I admit that the suggested algorithm is a sound method to increase the interpretability of GNN models. But I have some questions that I want to listen to the author’s responses.


Questions
- How much drop the performance after masking? I know that the goal of masking is interpretability, not performance. But I am curious about it since if there are few drops, then we can identify informative sub-graphs by masking.
- Can we increase the percentage of masking nodes? Does it govern by the $\lambda$?
- Can we use the suggested algorithm for other NLP tasks that use GNN models such as [1, 2, 3]?

Typos
We investigate in Tables 3 and 3 on page 8

Reference
[1] https://www.aclweb.org/anthology/P18-1026/
[2] https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16329
[3] https://www.aclweb.org/anthology/P18-1149/ 
review_title: I am waiting for the author's response.

final_score: rating: 7: Good paper, accept
review_content: Summary
This paper proposes an interpretation method for graph neural networks (GNNs) by learning parameterized, differentiable edge masks. The proposed method uses a single-layer NN classifier to predict  whether an edge can be dropped. It further uses L0 norm to encourage sparsity and amortizes parameter learning over a training dataset to avoid the hindsight bias. The method is experimented on a toy dataset (to demonstrate its faithfulness) and two real NLP tasks (multi-hop question answering and semantic role labeling). There are some interesting discussions based on the edge pruning results of these 2 tasks.

Strengths
- The paper proposes a novel interpretation method for GNNs. The method is technically interesting and well-motivated.
- The discussions for the QA and SRL models are thorough and interesting and provide useful insights. For example, the analysis shows that GNN-only models rely more on the paths for arguments farther away from the predicates, compared to LSTM-GNN models.
- The paper is well-written overall.

Weaknesses
- Some technical details are omitted. For example, “amortization” is only briefly mentioned in the last part of the introduction with a citation to Shultz et al., (2020). But this seems an important component of the model (shown in Section 4, Figure 2 and Table 1). It would be helpful to add more explanations/definitions so that the paper is more self-contained.
- The analysis and discussions for the QA and SRL tasks are very interesting. But they seem to be lacking actionable and generalizable conclusions.

review_title: Novel interpretation method for graph neural networks with interesting application/discussions on multi-hop QA and SRL

final_score: rating: 7: Good paper, accept
review_content: *Summary of the paper*: This paper introduces a post-hoc method -- GraphMask, to interpret the prediction of GNNs. For each edge in the GNNs, the authors introduce a learnable hard gate which indicates whether this edge could be erased or not. The gates can be trained together with the model in a fully differentiable way. By analyzing the GNN models' behavior on two tasks -- question answering and semantic role labeling, the authors find that a large proportion of edges could be dropped without deteriorating the performance of the model. At the same time, the remaining edges could be used for interpreting model predictions.

*Strength of the paper*:  
1. The idea to erase edges in GNNs is intuitive and straightforward, but how to do it properly is the problem. This paper introduces a simple yet viable method to deal with it. This kind of technique is potentially useful to analyze other models' behavior.

2. The experiments are well designed including both qualitative analysis, quantitative results, and reasonable ablation to show the effectiveness of their method. The thorough analysis conducted by the authors also contains useful insights of the model's behavior which is consistent with the literature.

3. The authors have done a thorough literature review and clearly state where does this paper stands.

*Comments*:

1. I really like the synthetic experiment designed in the paper which distinguishes the explanation proposed in the paper from others to be faithful. However, from my point of view, the task introduced here is too simple to say that such kind of faithfulness could be generalized to the real setting. I am curious if you further drop a small proportion of the remaining edges, how would it affect the performance?

2. Those model-dependent post-hoc explanations are interesting, I am curious how those explanations (remaining edges) would change if you have different encoders like BERT/Roberta. Could we treat it as a way to measure how much structural information is capture by the encoder? Also, other than the post-hoc explanation, do you have any idea of how we can further use such kinds of explanations to build a better/robust/faithful model?

*Reason for score*: Overall, I vote for accepting this paper. I like the idea of differentiable masking and the way the authors use to build faithful explanations for GNNs. The experimental results and analysis provide new insights to this area. 
review_title: well-designed experiments and thorough analysis

final_score: rating: 7: Good paper, accept
review_content: The authors of the paper set out to design a method for explaining the behaviour of graph neural networks. Here the focus is on GNNs used in the context of natural language processing. 

The authors argue that the following three properties of a method are desirable:

(1) The method should identify relevant paths across layers as paths are natural ways to present GNN reasoning patterns;
(2) Be tractable, that is, allow the computation of paths in an efficient manner;
(3) Be faithful.

The authors discuss existing post-hoc XAI approaches and the ways in which they do not meet at least one of the above desiderata.

(A friendly comment here: perhaps it would be better to tone down the language just a little bit. I’m not sure I’m convinced that paths are the most natural way to present GNN reasoning patterns, for instance. They are one possible way. Also, what is and isn’t faithful is still not really rigorously defined in the literature and your definition could be made a bit more specific.)

At this point, it would also make sense to define/make more explicit what you mean by “edge” and “paths” which you want to remove/retain to explain the behaviour of the GNN. Do you mean edge as in “edge between the input graph’s nodes in the computation graph induced by the GNN for one node”? Or do you mean “edge between neurons in the computation graph …”. Making this more concrete allows the user to follow your claims and reasoning better. 

The proposed method belongs to the class of perturbation-based methods. It is very similar to prior work [1] and the authors should focus a bit more on similarities and differences in comparison to [1]. It seems the major difference is in the gating mechanism which is soft in [1] and in the application domain (image CNNs vs. GNNs). The lack of detailed discussion of [1] is especially problematic in the context of what the authors refer to as amortization. This seems to be crucial and makes a significant difference empirically yet is not discussed beyond a brief mention that this was introduced in [1]. The impression I currently have is that the paper is an application of [1] to GNNs. 

One weakness of the proposed approach compared to prior work on explaining GNNs seems to be that it can only mask edges and not features. Is my understanding correct?

The experiments are well-designed and executed as far as I can tell. I am a proponent of the synthetic data experiments of the GNNExplainer and was happy to see them. It is interesting how well the integrated gradient method works compared to the GNNExplainer. I also like the NLP applications of GNNs and the corresponding experiments. Here, however, I would have wished to see a comparison again with integrated gradients and GNNExplainer. Is there a reason for not making such a comparison?

Overall I think this is a well-written and executed paper. The main problem is that it is very similar to prior work and doesn’t make enough of an effort to carve out the differences and similarities. The missing discussion of amortization is especially problematic, as it seems to be the decisive factor in the proposed method's performance. 

[1] Karl Schulz, Leon Sixt, Federico Tombari, and Tim Landgraf. Restricting the flow: Information bottlenecks for attribution. In International Conference on Learning Representations, 2020.

review_title: Explaining GNNs similar to prior work

Meta-review: This paper proposes a method for  interpretable  graph neural networks.
The idea is intuitively well motivated: after training the model, discard spurious edges that are not critical to making predictions in the graph, and only retain salient edges.
Experiments on synthetic and real datasets show that the proposed method is effective at dropping only the edges that are not useful for the task at hand;  while leading to  only small performance degradation.  The paper is well-written. Overall, the paper brings together prior ideas in a useful way, and is well-executed.
Decision: Accept (Spotlight)