Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper discusses graph-level distribution shifts on graph neural networks. It propose a causal-based invariant learning objective based on contrastive learning and prove its equivalence/approximation of the designed invariance principles. Various experiments on synthetic and real-world shift demonstrate the effectiveness of CIGA.
strengths_and_weaknesses: **Strengths** 
1. This paper poses a great challenge towards existing OOD generalization algorithms on graphs and provide some sort explanation. 
3. The connection between semi-supervised contrastive learning and invariant in Eq.5 is sound.

**Weaknesses** 
1. The results in the experiments are sometimes misleading. For example, in Table 3, when baselines are better, the bond font is still on proposed two methods. 
2. I read through main theorem 3.1 and its proof, most of the induction is around mutual information or information bottleneck, while the property of graph is rarely used. The author claims challenge of graph OOD is not environment label and convoluted causes of shift, while the proposed method seem to not that relevant / benefit from the nature of the graph.
questions: 1. What's the accuracy or F1 measure on real-world graphs? It is hard to recognize the performance gap caused by distribution shifts. In other words, the negative effect of distribution shifts are not revealed in current table. 
2. What's the relatedness of graph property and proposed method / proof, maybe I am missing something during reading ( please refer to weakness 2 )  
limitations: N.A.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper studies the OOD generalization problem for graphs. It takes a causal view at the problem and proposes four assumptions to describe the graph generation process. Specifically, three factors C (invariant), S (varying), and E (environment) control the graph generation process. The assumptions include two types of distribution shifts in which 1) graph label Y is fully dependent on C; and 2) Y is spuriously correlated with environment E.

Based on the assumptions, the authors decompose the graph learning problem into finding the an invariant subgraph $\hat{G}_c$ and minimizing the risks on $\hat{G}_c$, i.e., Equation (1). The remaining efforts are dedicated to resolve Equation (1)'s intractability. The main idea is to use supervised contrastive learning to maximize the agreement between the invariant part of graphs with the same label .

strengths_and_weaknesses: **Strengths:**

- OOD generalization for graphs is of high interest for the machine learning community. This paper present a thorough study from the causal view, which should enlighten future researchs in this field.
- The idea is novel and motivated by theory.



**Weaknesses:**

- The method seems complex that includes two GNNs and three loss functions. It is unclear about the practical value of this method.

- The presentation can be improved. 
    - Section3.2 is hard to parse. The reasoning process between the multiple objective functions is poorly organized and hard to evaluate.
    - Some equations can be made clearer. For example, what does the C, C', c, c' in Equation (2) mean? Why is the risk term in Equation (1) replaced by mutual information in Equation (3)?
questions: Q1: since theoretical results mainly address the connection between the proposed method and definition2.5, it is crucial to understand the significance of definition 2.5. How is the minimax objective in definition2.5 relevant to finding the causal factor C in Figure2?

Q2: in Line189-190, graph's complex properties of structures and features are identified as a key challenge for OOD generalization. It is unclear how does the proposed method resolve this challenge?

limitations: none
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors consider the problem of out-of-distribution generalization on graphs, where shifts in the distribution can change both the attributes and the structure of the graphs. They propose two causal DAGs describing the shift, and assume the existence of a subgraph which is invariant to the shift and that contains most of the information about the label. Then, they present two learning objectives to identify this critical subgraph which is then used for the label prediction. 
strengths_and_weaknesses: #### Strenghts
1. The assumption of the presence of an invariant subgraph is interesting and reasonable, and it is in line with previous work.
2. The empirical evaluation shows the effectiveness of the proposed approach.

#### Weaknesses
1. The learning objective assumes the presence of multiple training environments, and thus the proposed method would not work in the single environment setting.
2.  The relationship between $E_G$ and $E$ is unclear, since they are not causally related. Also, since $E_G$ is an input of $ f_{gen}^G$, and $E_G \subseteq E$ there should be an arrow from $E$ to $G$.

questions: 1. What is the relationship between $E_G$ and $E$ ? I can see that $E_G \subseteq E$ but if we replace G in the FIIF SCM with its $\mathcal{G}$-Gen SCM, then: (1) there is no causal relationships between $E_G$ and $E$ and (2) $E_G \perp E$, which is contradictory.
Can't you simply remove the random variable $E_G$? 
2. From Assumption 2.1 we have $G = f_{gen}^G(G_C, G_S, E_G)$ and since $E_G \subseteq E$ then the environment also causes G. This obviously would change the causal diagram since then there should be an arrow from the environment to G, changing all the causal models. Please clarify.


limitations: N/A
review_title: null

Meta-review: metareview: 

Graph NNs have proven to work considerably well in the in-distribution setting. However, they fail when test data come from a different distribution than test, as shown by previous work. This paper aligns with recent works, and aims to study how to obtain invariance to shifts described by the assumed causal model. The assumed causal model is reasonable, and the solution is novel. 
There is consensus among the referees, as evidenced by the score of 6 from each of them, that these results could be of interest to Neurips.
confidence: Certain
Decision: Accept