From: ICLR.cc/2023/Conference/Paper284/Authors
title: Submission Withdrawn by the Authors

From: ICLR.cc/2023/Conference/Paper284/Reviewer_QAou
confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
summary_of_the_paper: This paper proposes a framework called the BRGCL for training GNNs. The proposed approach combines a few ideas, including contrastive learning, label propagation, prototype learning, and Bayesian nonparametric learning. Experiments on several benchmarks show promising results.
strength_and_weaknesses: Strengths:
1. The paper is well-written.
2. The proposed method is intuitive and easy to follow.
3. The results are good.

Weaknesses:
1. The paper is lack of novelty.
2. Some key references are missing.

Detailed comments:
1. My major concern is on the novelty of the paper. The paper combines a few advanced techniques in machine learning, including contrastive learning, label propagation, prototype learning, and Bayesian nonparametric learning. Although this combination yields a model with good practical results, all these components are not new, as they are all from existing works. Also, the paper reads like consisting of many small contributions, where each contribution corresponds to an individual component, but I feel like the paper lacks a key innovation. In this sense, this paper does not bring many new insights.
2. In Sec. 4.2, this paper proposes some strategy to infer the confidence of node labels, but these strategies seem heuristic and are lack of theoretical guarantees. In some Bayesian nonparametric methods such as Gaussian process, there are some good ways to measure the uncertainty and confidence, such as the upper confidence bound. I wonder whether some more advanced confidence estimation methods can be developed here.
3. Some key references are missing. This paper mainly focuses on self-training for GNNs, and one important work along this direction is [1]. Besides, one key idea of Sec. 4.2 is label prediction, and there are many papers which have explored very similar ideas including [2,3,4], and the authors might consider discussing the relationship with these works.
4. This paper uses Dirichlet process, which is able to automatically decide the number of prototypes. However, one potential problem of DP is that the number of prototypes can be either too large or too small in practice, and it is nontrivial to find a good \xi in Eq. (2). In the experiment, the authors use cross-validation to to select \xi, and it would be more helpful to present some sensitivity analysis by showing how K and the final results vary when changing \xi.

[1] Deep Graph Infomax, ICLR 2019.
[2] Learning with Local and Global Consistency, NeurIPS 2003.
[3] GMNN: Graph Markov Neural Networks, ICML 2019.
[4] Combing Label Propagation and Simple Models Out-performs Graph Neural Networks, ICLR 2021.


clarity,_quality,_novelty_and_reproducibility: See comments above.
summary_of_the_review: See comments above.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper284/Reviewer_XPmP
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: Contrastive learning has been recently widely utilized for the computer vision and natural language processing domain.
However, contrastive learning has not been well explored in the graph-related domain.
This paper provides robust graph contrastive learning with the Bayesian method.
This paper proposes the method of Bayesian Robust Graph Contrastive Learning (BRGCL) that adopts the prototypical contrastive learning and Dirichlet Process Mixture Model (DPMM). They validate their method on diverse datasets such as PubMed and ogbn-arxiv.
strength_and_weaknesses: Strong point
1. This paper raises the important issue, the noise inherently exists in the graph data.
2.  This paper adopts the DPMM and prototypical contrastive learning

Weak point
1. The novelty of this paper is limited. This paper provides the methods, BRGCL, but it is the extension of prototypical contrastive learning and DPMM. I don't think that the combination of the previous two works is bad. However, there should be a novelty point, such as a theoretical guarantee or the new tasks that the previous works cannot solve.
2. The motivation of DPMM is not clear. I agree that the strong point of DPMM, non-parametric clustering. However, I am concerned about whether the number of clusters is significantly important or not for performance improvement.
3. This paper only handles the small-sized dataset. I suggest that the author include the additional large-scale dataset such as ogbn-product or ogbn-papers100M
clarity,_quality,_novelty_and_reproducibility: This paper is easy to follow.

summary_of_the_review: I read the paper two times.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper284/Reviewer_DJuE
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This paper presents a representation learning method for graph data under the presence of label or node attribute noises. The idea is to train a graph encoder $g$ based on both contrastive learning loss (with InfoNCE) and prototypical contrastive learning. The main contribution is in the prototypical contrastive learning part, where confident nodes are selected via Bayesian nonparametric node feature clustering and label propagation. The authors argue that the selected confident nodes are more robust to noises, so the representation learning based on them could lead to better results on downstream tasks. For the nonparametric clustering, the authors adapted DP-means, an asymptotic version of Dirichlet process mixture models. The proposed method is applied to GCN, and demonstrated to outperform baselines on node classification and clustering tasks for several benchmark datasets.
strength_and_weaknesses: Strength
- The paper is generally well-written and easy to follow.
- The experimental results seem promising.

Weakness
- The proposed method is a bit incremental; it is a combination of prototypical contrastive learning with Bayesian nonparametric estimation of Confidence (BEC), which is also largely based on DP-means. I agree that the algorithm employing label propagation and using the cross entropies to measure node confidences is new, but not incredibly surprising to guarantee acceptance solely based on that.
- The proposed method is only applied for GCN, although I see no reason not to apply it to other GNN architectures, such as GAT or GIN. I think demonstrating the wide applicability of the proposed method over various GNN architectures could strengthen the argument.
-  It is not entirely clear how the confident nodes found by BEC contribute to the robustness. Since BEC is largely based on heuristics, more empirical justification is required. Figure 3 somewhat shows that the representations of the confident nodes are more robust to label noises, but the figure is hard to interpret and personally I find it not convincing. An indirect way of validating the robustness of the confident nodes would be measuring the node classification or clustering accuracies for the confident nodes under varying noise levels; if the accuracies for the confident nodes drop less than those of unconfident nodes, then the result might support the claim in the paper. Another experiment would be measuring the stability of the prototypes under varying label noises; if the prototypes computed with only confident nodes (as done in the proposed method) were affected less than those computed from both confident and unconfident nodes, one can conclude that the confident nodes help to learn representations robust to the noises.

Questions & Concerns
- I understand that BEC might enhance the robustness under node attribute noise, but how could it help against label noises? After all, the algorithm does not use labels during the encoder training.
- In essence, the nonparametric method (DP-means) is nothing but a $K$-means with adaptive control for the number of prototypes. Is it really helpful to learn $K$ from the data using the nonparametric method? How would the proposed method perform if $K$ is set manually or tuned via cross-validation, and the prototype clustering is done with vanilla $K$-means? I think the comparison between vanilla $K$-means vs DP-means would be important since, without DP-means, the method pretty much boils down to prototypical contrastive learning applied for graph data.
clarity,_quality,_novelty_and_reproducibility: - Clarity and quality: the paper is generally well-written and clear.
- Novelty: novelty is limited. See the weakness section above.
- Reproducibility: detailed settings for the experiments are provided so that one can reproduce the results based on the main paper and appendix.
summary_of_the_review: Although the motivation is clear and the experimental results are promising, I find the contribution is somewhat incremental. I won't argue for acceptance.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper284/Reviewer_eFoS
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper:  This paper proposes a non-parametric technique based on a GNN encoder to obtain node prototypes that are robust to noise. The key idea is to leverage Bayesian nonparametric methods to infer a subset of robust nodes and then conduct prototypical contrastive learning. Experiments on 5 mainstream graph datasets and comparisons to some baselines are provided.
strength_and_weaknesses: Strengths:
 1. Use of non-parametric methods to infer archetypes for contrastive learning in a graph setting is well-founded and novel.
2. Small but consistent performance gain across multiple experiments, significant at 0.05 level with t-tests.
3. Paper is clear and well organized.

Weaknesses:
1. Encoder architecture is a 2-layer GCN. This has limited expressive power.
2. Computation complexity and scalability are not analyzed theoretically, and the benchmarks used range from tiny (cora, citeseer) to medium graphs (ogb-arxiv).
3. Missing graph contrastive baselines in experiments and review (see detailed feedback below).
4. t-tests are not the best for evaluating statistical significance in this setting (see detailed feedback below). 

Detailed Feedback and Questions:

1. This paper compares thoroughly with existing methods for handling noisy data but not nearly enough with contrastive graph learning baselines. There are several other baselines aside from GraphCL [1] that are not cited in the review and are not compared with the proposed approach in the experiments, See [2-4] ([1] is included in the paper but not its subsequent works).

Well known GCL methods include perturbations in the edge set GRACE (Zhu et al., 2020), BGRL (Thakoor et al., 2021), GBT (Bielak et al., 2021)), centrality (GCA (Zhu et al., 2021)), diffusion matrix (MVGRL (Hassani & Khasahmadi, 2020)) and the original graph (GMI (Peng et al., 2020)) and DGI (Velickovic et al., 2019)). The proposed method is compared to MVGRL (Hassani & Khasahmadi, 2020) but it is not clear why it is not compared to the others. If there is a good reason why a direct comparison does not apply that can be described in the review section or a direct experimental comparison is produced, I am willing to raise my score.

2, While there is performance improvement it in the experimental results it is often quite small. The t-tests are helpful but I think using Wilcoxon signed test would be better given small sample size. t-tests require assumptions of normality that are not explicitly justified in the paper. Do the t-tests that are close to 0.05 remain under the threshold when switching to Wilcoxon test?

3. Are the loss terms in eq. 6 added without weights? I would assume in general they are in different scales so weighting them should improve performance.

4. This work tackles feature and label noise. What about noise in the graph topology?

5. I understand that given extensive hyperparameter search (Table 4) gamma_0 and xi_0 can be selected. However, it would be interesting to see how selection of these values affects performance stability. How sensitive is the algorithm to xi selection in equation 2 and gamma_k selection in equation 5?

Typos:
 p. 1 ``handle an2 almost infinite” -> an
 p. 2 ``Existing Methods Handing Noisy Data" -> handLing
 p. 3 ``the attributes of each node is in” -> are
 p. 3 ``following two tasks.” -> replace ``.” by ``:”

References:
 [1] You, Yuning, et al. “Graph contrastive learning with augmentations.” Advances in Neural Inf. Proces. Syst. 33 (2020): 5812-5823.
 [2] You, Yuning, et al. “Graph contrastive learning automated.” Int. Conf. Machine Learning, 2021.
 [3] Qiu, Jiezhong, et al. “Gcc: Graph contrastive coding for graph neural network pre-training.” Proc. ACM SIGKDD Int. Conf. Knowledge Discovery & Data Mining. 2020.

clarity,_quality,_novelty_and_reproducibility: The paper is very clearly written and the authors provide code that should suffice to reproduce their results though I did not attempt to run it. I rate the novelty as high. The paper proposes a new idea that is interesting and theoretically grounded; it is one of the strengths of the paper.
summary_of_the_review: While the paper proposes a novel technique that is well motivated, there are shortcomings on the experimental side.  The use of only 5 datasets is a major limitation, especially as 2 of these (cora and citeseer) are very small and over-studied.

There is a lack of comparison to some key graph contrastive baselines. Even if not all of these baselines are compared in the experiments, they should at least be addressed in the related work section.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 4: The contributions are significant, and do not exist in prior works.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
details_of_ethics_concerns: Not applicable.
recommendation: 5: marginally below the acceptance threshold