Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors propose to improve generalization ability of GNN's for the task of graph classification via a causal theoretic analysis - under the assumption that only a subgraph of the graph is responsible for its label (and everything else are just spurious correlations). To this effect, they use a rationale generator and distribution intervener to test their model on a few synthetic datasets and a real world dataset.
main_review: *This is an emergency review*

Update Nov 20, 2021:

Recommendation: Weak Accept

Increasing my score to 6 after some discussions with the authors.

***************************************
Initial Recommendation: Weak Reject

I have tried to reason my recommendation below:

Strengths:
1. The idea to use a causal graph and the accompanying theory to explain the label associated with the graph is interesting. 
2. The paper is well written and easy to understand.

Weaknesses
1. Via the top-K approach, it appears like the authors make an implicit assumption about what fraction of edges is important in the graph (a fixed fraction, can't be lower or higher) - in my opinion, this certainly limits the wide applicability of the method. Alternatively, in GNN Explainer [1], they allow up to K edges to explain the label.
2. The applicability of the method is limited by the graph encoder used (here GNN). 1-WL GNN's are known to be unable to predict links or identify and distinguish certain classes of subgraphs [2][3][4] i.e. GNN's are not well suited for Eq. 5 
3. The work appears to miss relevant baselines like GNN Explainer[1] / CF-GNN Explainer [5]. Moreover the authors ideally should compare with baselines, which enrich GNN's with structural features [6][7][8] (As they are explainable to a certain extent as well).

Minor:
1. The separability of a graph into two subgraphs the causal and non causal might not always be possible? (would this need an encoder which is able to accurately capture the discrete topology over graphs of all orders and sizes - if not I can just make a house into a clique and the label would be incorrectly assigned as 1 because the edges required for the house are present) Please elaborate on this. 
2. The set  {s} employed in the test are limited to the ones seen in train.
3. Moreover, consider a graph with say 20 nodes, and the case where the causal part for instance consists of a house motif and a tree base and the label assigned is 1 or 0 based on the output of House Motif XOR Tree Base - would this be captured by the proposed method (appears like it wont)?



References:

1.Ying, Rex, et al. "Gnn explainer: A tool for post-hoc explanation of graph neural networks." arXiv preprint arXiv:1903.03894 (2019).

2.Srinivasan, Balasubramaniam, and Bruno Ribeiro. "On the equivalence between positional node embeddings and structural graph representations." arXiv preprint arXiv:1910.00452 (2019).

3.Dwivedi, Vijay Prakash, et al. "Benchmarking graph neural networks." arXiv preprint arXiv:2003.00982 (2020).

4.Chen, Zhengdao, et al. "Can graph neural networks count substructures?." arXiv preprint arXiv:2002.04025 (2020).

5.Lucic, Ana, et al. "CF-GNNExplainer: Counterfactual Explanations for Graph Neural Networks." arXiv preprint arXiv:2102.03322 (2021).

6.Bouritsas, Giorgos, et al. "Improving graph neural network expressivity via subgraph isomorphism counting." arXiv preprint arXiv:2006.09252 (2020).

7.Bodnar, Cristian, et al. "Weisfeiler and lehman go topological: Message passing simplicial networks." arXiv preprint arXiv:2103.03212 (2021).

8.Bodnar, Cristian, et al. "Weisfeiler and lehman go cellular: Cw networks." arXiv preprint arXiv:2106.12575 (2021).
summary_of_the_review: The paper is well written and the proposed solution is simple but there are still concerns about the wide applicability of the method, and also lacks comparisons with very related baselines.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes to address the limitation of current interpretable graph neural networks over out-of-distribution data. In this work, the intrinsically interpretable GNN is investigated by identifying the invariant rationales corresponding to environment-invariant causal patterns. The proposed model is evaluated on the graph classification task to demonstrate its effectiveness as compared to state-of-the-arts.
main_review: Strengths:
The tackled problem of identifying the environment-invariant causal patterns for improving the interpretability of graph neural networks is interesting and important for many graph-structured data representation tasks.

The proposed DIR model is technically sound with the construction of interventional distributions and discover the salient features from different environments. In addition, the theoretical discussion is provided to analyze the proposed DIR method.

In the evaluation section, the effectiveness of the new DIR model is analyzed from different aspects: model generalization ability, performance with different bias degrees, as well as the intrinsic interpretability. Furthermore, the identified rationale has been visualized under interventional distributions.

Weaknesses:

It would be better to summarize the key notations used throughout this paper in a table, for easy reference when reviewer go through the solution details.

The detailed configurations of the compared baselines could be described in the evaluation so as to provide a more comprehensive performance comparison.
summary_of_the_review: The studied problem is important and beneficial for many graph-learning tasks. The proposed method is rationale and technically sound. More details about the experimental settings could be clarified in the evaluation section.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a novel algorithm DIR that allows the learned GNN model to make predictions based on causal patterns. The whole framework consists of four components where (1) the rationale generator splits the input into causal and spurious parts; (2) the distribution intervener replaces the spurious part; (3) the graph encoder embeds the graph based on the spurious and causal parts, respectively; (4) the shortcut and causal classifiers are applied to the spurious and causal embeddings, respectively. This algorithm is highlighted by its objective which encourages the predictions accurate while invariant to the spurious part. On both synthetic and real-world datasets, DIR outperforms the related baselines consistently. As a novel end2end rationalization method, As a novel end2end rationalization method, the underlying DIR principle is proved to be sufficient and necessary for satisfying the oracle function.
main_review: Strengths:
1. DIR is end2end and novel to me.
2. The DIR principle is intuitive and theoretically justified.
3. The empirical results look good, namely, DIR consistently excels.
4. The observations about how the variance changes during the training course are interesting and provide help for understanding other invariant learning methods.

Weaknesses:
1. What's the motivation behind the element-wise multiplication between causal predictions and spurious predictions? It has not been explained in the paper.
2. With this multiplication and the various sampled spurious subgraphs, it seems that the spurious classifier should output constant vector 1 consistently, so that the causal predictions would not be modulated a lot, right? It would be better to provide more observations in the experiments about the predictions of the learned spurious classifier.
summary_of_the_review: This paper is well written. It proposes a novel end2end algorithm DIR towards causality-based graph representation learning. The algorithm is based on an intuitive principle that is theoretically justified by the authors in this paper. Meanwhile, the extensive experiments also confirm the effectiveness of DIR.
review_title: null

Meta-review: The work proposes a method to learn graph representations based on subgraphs that are invariant to spurious subgraphs. The reviewers found the paper easy to read and the theory interesting, well explained and justified. The reviewers seem happy with the existing and new experiments that came during the rebuttal phase. I too found the paper interesting and mostly well-written. 

Besides the corrections done during the rebuttal, in further discussion with the authors, I raised a concern that the work must make additional assumptions about the support of the induced subgraph distributions that were not clearly stated in the paper: The work makes the assumption that there is enough training data such that all spurious induced subgraph patterns $S$ that are smaller than the truly correlated induced subgraph $C$ can be identified as spurious. The authors promised to make this into a clearly demarcated assumption since it a key requirement for the method to work.
Decision: Accept (Poster)