Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors propose an equivariant graph contrastive learning framework that adopts two principles: invariance to intra-graph
augmentations and equivariance to cross-graph augmentations. They use Mixup as the method for cross-graph augmentation and argue that the cross-graph augmentation captures global semantic shifts and yield better performance.
strength_and_weaknesses: Strength:
1. Extensive results on multiple datasets.
Weakness:
1. Novelty: Though the authors make a great effort in explaining the concept of equivalence, the framework itself boils down to a combination of Mixup and contrastive learning. This combination is not new. For instance, at a high level, [1] and [2] both adopt the combination.
2. The experimental results are not very convincing. In table 1(a), on the datasets that the proposed method performs the best, the increase is usually below 1% , not to mention that on the remaining 3 datasets, the proposed method underperforms. Similarly, in table 1(b), the proposed method only wins 4/8 datasets and by a small margin.


Literature:
[1]i-Mix: A Domain-Agnostic Strategy for Contrastive Representation Learning
[2]Improving Contrastive Learning by Visualizing Feature Transformation
clarity,_quality,_novelty_and_reproducibility: The writing is clear, but the novelty needs to be improved.
summary_of_the_review: In general, I am not a fan of this paper mainly for the following two reasons: (1) Combining Mixup and contrastive learning is not new as several papers have adopted a similar idea though in the CV domain. (2) The experimental results are not exciting.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper first points out that aggressive augmentations may worsen representation and lead to sub-optimal performance. To mitigate this issue, they propose a novel contrastive learning framework (E-GCL) that employs the equivariance principle to implement cross-graph discrimination. To this end, the authors propose insensitive transformations P and sensitive transformations H for the intra-graph augmentation and cross-graph augmentation, respectively. The author conducted detailed experiments to prove their framework’s effectiveness.
strength_and_weaknesses: Strength 1: The contribution of this paper is clear; this paper is well written.
Strength 2: This paper deals with a non-trivial and challenging issue which are neglected by current research. 
Strength 3: The experiment was performed on several datasets. The results show superior performance over existing competitive approaches.

Weakness 1: In the introduction part, the authors point out that ‘Nonetheless, it is hard, without domain knowledge (Dangovski et al., 2022; Chuang et al., 2022) or extensive testing (Dangovski et al., 2022), to tell apart sensitive and insensitive augmentations.’  Indeed, it is an unresolved challenge to tell if a data augmentation will cause global semantic shifts. Learning the semantic shifted view can lead to sub-optimal performance. However, how the equivariant self-supervised learning can mitigate this issue? In the proposed framework, the intra-graph augmentation is kept. Hence, the semantic shifted view can still damage the model.

Weakness 2: In Section 3.2, the author applied the ‘Group Averaging’ to ‘make the interpolations insensitive to relative permutations’. However, in the Section 3.1, the author claims that ‘Equivariance requires that…, any change applied to the graph should be faithfully reflected by the change of representation.’ This is not consistent. Can the author give more discussions on why the model should be insensitive to relative permutations during cross-graph augmentation? And why apply the ‘Group Averaging’ can help improve performance? There is no further experiment or discussion in Section 4 to back up this design.

Weakness 3: In Section 4.3, the authors demonstrate the alignment and uniformity losses of I-GCL and E-GCL. It can be observed that the "E-GCL with cross-aug" setting has better alignment and uniformity losses. Can the author give some discussion on this finding? Does this imply if getting rid of the intra-aug, the model may have a better downstream task performance? If possible, the author should conduct ablation study on the "E-GCL with cross-aug".
clarity,_quality,_novelty_and_reproducibility: This paper is well written with good novelty. And the code is available in the appendix.
summary_of_the_review: This paper is well-written and deals with a non-trivial challenge. The experiments prove the model’s effectiveness. However, theoretical justification for the proposed design and discussion on some experiments are missing. The authors should address these issues. Overall, this paper is above the acceptance borderline if the authors can address all the concerns mentioned above.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: In this work, an equivariant graph contrastive learning method is proposed to perform cross-graph augmentation, which is reasonable and effective for GCL. Experiments are performed both unsupervised learning and transfer learning tasks to show the model effectiveness.
strength_and_weaknesses: Pros:
1.	The studied problem with equivariant graph contrastive learning is important and new for GCL domain.
2.	The theoretical discussion of graph interpolation is given.
3.	Component-wise model evaluation is provided to show the model effectiveness.
4.	Hyperparameter sensitivity is conducted in the evaluation section.

Cons:

In the evaluation section, various baselines are compared, such as JOAO, GraphCL, and ADGCL. For the compared various GCL methods, the parameter setting details of those baselines can be described. 

Additionally, the computational complexity of the new E-GCL method can be analyzed compared with representative GCL baselines. 

clarity,_quality,_novelty_and_reproducibility: This work solves an important problem with a technically sound solution. Conducted experiments are comprehensive to demonstrate the effectiveness of the new approach. The source code has already been released for reproducibility. 
summary_of_the_review: Inspired by the equivariant SSL, this work proposes to enhance graph contrastive learning with cross-graph augmentation under an equivariant SSL framework. Experiments with different tasks show the advantage of the new approach.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a cross-graph augmentation method to simulate global semantic shifts. In particular, the paper first analyzes the limitations of invariant graph contrastive learning (I-GCL). Then the authors explore equivariance for cross-graph augmentation to mitigate the limitations of I-GCL. The authors conduct both unsupervised learning and transfer learning experiments the effectiveness of the proposed method.
strength_and_weaknesses: Pros:
1. The question studied is important and interesting.
2. Using equivariance can help resolve the sensitive augmentation.

Cons:
1. In Cross-graph augmentation, how do the authors generate labels for instances as
labels are unknown in contrastive learning. The authors claim that they mitigate structural differences between two graphs. However, randomly padding virtual nodes and edges may change the latent distribution of two graphs.
2. In experimental settings, the details of evaluation methods and reported baselines performance are missing. E.g. how do the authors report the baselines performance? Are they reimplementing all baselines or they reporting performances from original papers? Which accuracy is refer to the reported accuracy? The last epoch accuracy or the test accuracy in terms of the best validation accuracy? It is really hard to reproduce the reported performance without these details.
3. As far as I know, the reported results are from different evaluation methods. InfoMax and ContextPred are reporting the test accuracy of the best validation epoch, while GraphLOG is reporting the performance of the last epoch. The authors need to report results under a same experimental settings.
4. In results tables, do the authors use a same backbone for all baselines? As authors mentioned, they are using a BarlowTwins as backbone for unsupervised task and employing a GraphTrans for transfer learning task. Do they also using these backbones for other baselines?
clarity,_quality,_novelty_and_reproducibility: The proposed method needs more detailed experimental instructions.
summary_of_the_review: Overall, this paper takes one the most important research problems of graph learning: graph contrastive. The idea of exploring equivariance cross graph augmentation is interesting. However, I feel this paper is not ready as there are several major concerns regarding the framework design, experiment. Hopefully, the authors can address these concerns and make this paper more solid in the next submission.

##
Score updated after reading the rebuttal.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This work proposes equivariant graph contrastive learning (a contrastive learning framework (E-GCL) ) that adopts two principles: invariance to intra-graph augmentations and equivariance to cross-graph augmentations. The work interestingly shows that aggressive augmentations may worsen representation and lead to sub-optimal performance. The work uses a mixup procedure as the method for cross-graph augmentation and argue that the cross-graph augmentation captures global semantic shifts and yield better performance. Experiments were performed in both unsupervised learning and transfer learning tasks to show the model effectiveness.

One of the reviewers was concerned about the mixup procedure (other reviewers have raised the same concern). Why should the graph mixup be a linear interpolation heuristic? Why would the mixup not account for graph labels? The mixup is also an out-of-distribution augmentation, which is not proven to be safe in most applications. Why is it safe in the applications proposed by the paper? Graph mixup is an interesting topic but the interpolation technique is itself is too naive to deal with the complexity of graphs (interpolation is rarely the answer to finding intermediate points in the support of graph distributions). The authors justify their choices citing other works, but after reading these works, I see the same issue with these other papers.

The were questions about the key differences to prior works [1,2,3] being straightforward (e.g., intra-graph augmentation). On the positive side, the use of permutation averaging for insensitivity to permutation in order to deal with graph interpolation is interesting.

Overall, reviewers raised some questions that were addressed in the rebuttal. However, the challenges with the mixup generating graphs outside of the support with unclear labels does not look theoretically sound (regardless of what authors have argued in the rebuttal, since it needs a proof, not just justifications) and needs to be addressed before publication.
justification_for_why_not_higher_score: The proposed mixup generating graphs outside of the support with unclear labels is not yet theoretically sound as it should be and needs a more solid theoretical foundation before publication. 
justification_for_why_not_lower_score: The work has some interesting ideas on contrastive learning for graphs, and how to deal with permutation invariance and equivariance for these methods.
Decision: Reject