Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This work proposes an alternative self-supervised objective inspired by canonical correlation analysis. Theoretical proof shows its connection to mutual information (MI) estimator and information bottleneck principles. Empirical experimental results demonstrate the effectiveness on node classification. 
main_review: The proposed self-supervised objective shares the same idea on making the representations of augmented views from the same data be close as much as possible. While the difference lies at the usage of negative samples. The proposed loss function does not need negative sample, but a regularization term instead. The key idea of this work is very easy to follow, and the experimental results on node classification task demonstrates its superiority to baselines. 

Concerns:
I have a question about the correctness of the theoretical proof to show its connection to previous approach needs more explanation. It seems to be very important to dissect the proposed loss function. Authors drop an assumption that S denotes the augmented view from original input data X, and they share the same space, referring to Section 4, line 185. Then two different distributions (i.e., P(Z_S) and P(Z_S | X)) for the variable Z_S are defined in Line 191. Theoretical analysis are conducted based on this assumption. However, I'm confused about the difference between P(Z_S) and P(Z_S | X). It's easy to understand the variable Z_S depends on X because the augmented view S is generated from X. But it's difficult to understand why we can define a marginal distribution P(Z_S). It's not reasonable because S still depends on X. It causes a big trouble to follow the conclusion in Proposition 2. Why we can discuss the entropy of variable Z_S without the condition on input data X.

Minor Issue:
1. In the appendix D, please double check the Remark 1. How can we get the equality H(Z_S|X) - H(Z_S | S, X) = H(Z_S | X) according to property 6?
limitations_and_societal_impact: I admit that the limitations and societal impact are given by authors.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The authors propose CCA-SSG for self-supervised representation learning on graph. Tee method is simple and efficient. Experimental results show high performance of the method on the node classification task. 

main_review: The problem setting is not clear.
The basic objective of CCA is to extract two different correlated features from two different observed data sets.
The link between CCA and node representation is not clear.
There is almost no explanation on the details of datasets in the experimental section. 
Each dataset has two different observations?

The authors made a comparison of the performance of many methods, but traditional CCA is missing. Traditional CCA can not be applied to the node representation learning problem? The title of this paper is "From Canonical Correlation Analysis to Self-supervised Graph Neural Networks". It would be better to show the performance of CCA as well.

For the label prediction, why did the authors use logistic regression?

From Table 2, unsupervised methods tend to work better than supervised methods. It is not strange?

The perfromance is heavily dependent on lambda and the embedding dimension. How can appropriate parameters be determined in practice?



Thank you for the answers.
I have raised my score.
limitations_and_societal_impact: The details of the proposed method in this paper are not clear.

The method is unable to extract features involving the minority.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper focuses on self-supervised graph neural networks, aiming at training graph neural networks without labeled data, and an approach called the CCA-SSG is proposed. CCA-SSG constructs two views of the given graph through edge dropping and node feature masking, and further uses CCA for learning node representations from the two views. The idea of formalizing unsupervised node representation learning as a multi-view learning task looks novel to me. Also, CCA-SSG has intuitive connections with the information bottleneck principle, which is attractive. The authors do experiment on multiple datasets, and CCA-SSG outperforms many competitive baseline methods.

Strengths:
1. Important problem.
2. Principled method with good theoretical connections.
3. Strong results.
main_review: Below are some detailed comments and questions:

1. At line 553 of the appendix, it is said that s_1 and s_2 come from the same distribution p_{aug}(x). But from my understanding of the proposed method, s_1 and s_2 are graphs from two different views (i.e., s_1 from edge dropping and s_2 from node feature masking respectively). If this is the case, s_1 and s_2 should come from different distributions. Is it correct?

2. CCA-SSG relies on edge dropping and node feature masking to augment the original graph, but the hyperparameters of edge dropping rate and feature masking rate are not studied in the paper. How will the results of CCA-SSG vary if the hyperparameters are changed?
limitations_and_societal_impact: N/A
review_title: null

Meta-review: This paper proposes a new objective and framework for self-supervised representation learning on graphs. It formalizes unsupervised node representation learning as a multi-view learning task. Results show that the proposed approach outperforms many competitive baseline methods. The idea presented in the paper is interesting. The proposed approach depends on several hyperparameters. The effect of these hyperparameters are not studied and should be included. 
Decision: Accept (Poster)