From: ICLR.cc/2023/Conference/Paper504/Authors
title: Submission Withdrawn by the Authors

From: ICLR.cc/2023/Conference/Paper504/Authors
title: Looking forward to further feedback
comment: Dear reviewers,

Thanks for the time and effort of all reviewers on our work. We have carefully addressed all the comments. Do let us know if there is any other concern. If all concerns are addressed satisfactorily, we would appreciate it if reviewers could kindly consider raising the rating. 

Thank you very much!

Best,

The authors

From: ICLR.cc/2023/Conference/Paper504/Authors
title: Response to Reviewer J7S2 (2/2)
comment: - [Negative Value of $\lambda$ in Intra Loss.] Thank you for raising this question. As we use cosine similarity to measure intra-/inter-class similarity, we want intra-class similarity getting larger and inter-class similarity getting smaller. Thus we set a negative value for intra part so that minimizing the loss is to maximizing the intra-class similarity.

- [Size of Figure 3.] Thank you for pointing out this problem. We have adjusted the size of Figure 3 in the revision.

- [Silhouette Coefficient.] Sihouette Coefficient is used to measure how similar a sample is to its own class compared to those from other classes. It is indeed typically used in an unsupervised setting to measure the quality of the clustering produced. In the case study, we use Silhouette Coefficient to evaluate how CARE affects the separability of graph classes in the training process. It is applied to the set of graph representations produced for each class (ground truth used here). Here, we use it to assess the quality of the graph representations produced instead of assessing the clustering/class quality. If the graph representations produced are cohesive within each class and separated from other classes (i.e., with a high Silhouette coefficient), we conclude that this set of graph representations is of good quality. Our results show that CARE-GCN attains a higher Silhouette coefficient than GCN, which indicates that CARE can improve the separability of graph classes in the representation space. Note that Table 4 reports the classification accuracy, which cannot be compared with the values of Silhouette Coefficient as they are different metrics. We have amended the caption of Table 4 to avoid the confusion.

- [Graph Representation Visualization.] Thanks for the comment. We want to give more details for this visualization. Compared with the representations produced by GCN, the representations produced by CARE-GCN are gathered more closely by class, which demonstrates better class separability. Furthermore, the representations produced by CARE-GCN consist of fewer outliers than that produced by GCN (5 vs 11 for class 0 and 3 vs 7 for class 1). 

[1] Abu-Mostafa, Yaser S., Malik Magdon-Ismail, and Hsuan-Tien Lin. Learning from data. Vol. 4. New York: AMLBook, 2012.

From: ICLR.cc/2023/Conference/Paper504/Authors
title: Response to Reviewer J7S2 (1/2)
comment: - [Novelty.] Many thanks for the comment. Although we adopt existing graph pooling approaches for our subgraph selector, we emphasize that, as a plug-and-play framework, the main contributions of our work are (1) introducing a Class-Aware Refiner to consider graph-level relationships, (2) providing a better generalization upper bound than GNN backbones. As a framework, CARE also provides the flexibility for other researchers to customize modules such as the Graph Encoder and the Subgraph Selector that better suit their problems/application scenarios.

- [Pseudo Label.] Sorry for the confusion. In the training process, the Class-Aware Refiner uses ground truth labels to guide the refinement, which it is a typical supervised setting. In the validation and test processes, the ground truth labels are not available. However, we need a label for each validation/test instance to determine which class representation to be used to composite the refined representation of that instance. In this case, we produce a pseudo label for each validation/test instance and use the class representation indicated by this pseudo label to refine the representation of this instance.  We have modified the descriptions in Page 4 of the revision to avoid the confusion.  

- [Loss Function.] The classification loss $L_{cls}$ is a common setting in supervised graph classification tasks. As a plugin method, the class loss $L_{class}$ we proposed only works when combined with the classification loss. Note that $L_{class}$ is used to guide the learning of class representations (to capture graph-level information) and doesn't factor in the classification loss (an aggregation over all individual graphs). The first row of GraphSAGE and GCN in Table 4 shows the performance when using the classification loss $L_{cls}$ only. The last row of GraphSAGE and GCN shows the result when combining both $L_{cls}$ and $L_{class}$ as given by Eq. (5). We notice that the notation *class* in the last row of Table 4 may be misleading and have changed it to *combine* in the revision.

- [About 1\% Improvement.] In our experiments, we compared the number of winning/losing cases of CARE with respect to an improvement threshold to further demonstrate its superiority. We chose 1% as the threshold but didn’t mean that this is a significant improvement. We remove this sentence to avoid confusion.

- [Generalization Analysis.] Thank you for raising the concern. We further articulate that, with the identical number of parameters, CARE achieves both better generalization ability and better learning ability than its GNN backbones.

	Better Generalization Ability: Our conclusion in the generalization ability based on the upper bounds of VC-dimension was not drawn arbitrarily. It is grounded on previous findings in the literature. According to Section 2.2 of [1], the VC dimension provides a loose generalization bound for models and can be used as a guideline for generalization comparison - models with a lower upper bound tend to have better generalization capability. We have added this in Page 15 of the revision to better elaborate the ground of our theoretic analysis.

	Better Learning Ability: We have conducted the above experiments which control the parameters to be identical in both the GNN backbone and CARE. This can be achieved by adjusting the number of parameters in the GNN backbone at the time when CARE is plugged in. For each GNN backbone, we first set its number of parameters to 100K. For CARE, we adjust the hidden dimension of each GNN backbone to which CARE is applied such that the number of parameters of CARE is also 100K.  As shown in the table below, the improvements of CARE in 8 out of 9 cases demonstrate that CARE has a better learning ability than the GNN backbone. Note that here we care about the generalization error instead of the training error when assessing the learning ability. The performance of the models is assessed on validation/test sets, and a strong ability of fitting training data doesn't necessarily lead to better generalization performance. We have formulated this analysis in Appendix E (Page 18), in the revision.


|           |          | DD           | PROTEINS     | MUTAG         |
|-----------|----------|--------------|--------------|---------------|
| GraphSAGE | original | 72.18 ± 2.93 | 74.87 ± 3.38 | 75.48  ± 6.11 |
|           | CARE     | **72.22** ± 3.10 | **75.74** ± 1.68 | **76.08** ± 10.83 |
| GCN       | original | 71.02 ± 3.17 | 73.89 ± 2.85 | 77.52 ± 10.81 |
|           | CARE     | **71.73** ± 4.12 | **74.91** ± 3.59 | **79.27** ± 4.31  |
| GIN       | original | 73.10 ± 2.44 | **72.41** ± 4.45 | 89.36 ± 4.71  |
|           | CARE     | **73.19** ± 4.44 | 70.43 ± 4.69 | **89.70** ± 5.53  |

From: ICLR.cc/2023/Conference/Paper504/Authors
title: Response to Reviewer 1G71
comment: - [Novelty.] Many thanks for the comment. Although the methods we use to extract class-aware features are simple, we emphasize that, as a plug-and-play framework, the main contributions of our work are (1) introducing a Class-Aware Refiner to consider graph-level relationships, (2) providing a better generalization upper bound than GNN backbones. As a framework, CARE also provides the flexibility for other researchers to customize modules such as the Graph Encoder and the Subgraph Selector that better suit their problems/application scenarios.

- [Experiment on Larger Dataset with Multi-class Labels.] Thank you for raising this question. The datasets we used in our experiments are mostly for binary graph classifications. This is because these datasets are most commonly used in recent works of graph representation learning, not because the proposed framework only works better on binary classification. To clear this doubt, we provide additional experiments on REDDIT-MULTI-12K dataset, which contains 11 classes. The results below show that CARE outperforms its GNN backbones in all cases. We do not get the result of all GNN backbones due to the limitation of time and computation resources. We will update the full result to the final version of our paper.

|          | GraphSAGE | GCN   | GIN   | GXN   |
|----------|-----------|-------|-------|-------|
| original | 21.71     | 40.26 | 44.92 | 46.01 |
| CARE     | **21.73**     | **43.77** | **48.20** | **46.52** |

- [Two-Step Paradigm for Graph Representation Learning.] Sorry for the confusion on this statement. Acturally, we discuss two types of graph pooling methods, namely node drop pooling and node clustering pooling, in our related work (Appendix A.1). Moreover, we also adopt node drop pooling methods (SAGPool, HGSPLPool) and node clustering pooling methods (DiffPool, MEWISPool) as GNN backones for comparison in our experiments. We rephrased this part in Introduction to avoid the confusion.

- [Missing References.] Thank you for pointing out these missing references. GIB proposes a novel objective to recognize the maximally informative subgraph. It designs a bi-level optimization scheme and a connectivity loss to optimize the GIB objective. As both GIB and our work consider graph-level similarity, we add it as a new baseline in our experiments. The results of GIB are taken from its original paper. As shown in the table below, as a plugin component, CARE outperforms GIB in 7 out of 12 cases. We have added this set of results in the revision (Appendix F). As for SUGAR, it generates subgraphs and uses these subgraphs for reconstruction. Its framework follows the graph contrastive learning paradigm: it performs augmentation on individual graphs and neglects the graph-level information. We have added the discussion of these missing references in the related work of the revision (Page 14). 




|           |      | DD         | PROTEINS | MUTAG       |
|-----------|------|------------|---------------|-------------|
|  GraphSAGE | original in GIB paper | 72.9 ± 4.1 | 72.1 ± 4.2 | 74.3 ± 7.7  |
|  | GIB  | **78.1** ± 4.2 | 73.4 ± 4.3    | 76.0  ± 7.4 |
|   | original in our paper | 72.2 ± 2.9  | 74.9 ± 3.4 | 75.5 ± 6.1  |
|           | CARE | 73.3 ± 3.3 | **75.9** ± 2.8    | **82.0** ± 6.42 |
| GCN  | original in GIB paper | 72.5 ± 4.6 | 71.9 ± 4.1 | 74.3 ± 11.0  |
|        | GIB  | **76.5** ± 5.0 | 74.8 ± 4.6    | 77.6 ± 7.5  |
|   | original in our paper | 71.0 ± 3.2 | 73.9 ± 2.9| 77.5 ± 10.8  |
|           | CARE | 72.2 ± 3.9 | **75.0** ± 2.9    | **79.3** ± 11.8  |
|GIN  | original in GIB paper | 73.0 ± 3.3  | 70.7 ± 4.8 | 82.5 ± 6.8 |
|        | GIB  | **74.7** ± 3.9 | **74.9** ± 5.0    | 83.9 ± 6.4  |
|   | original in our paper | 73.1 ± 2.4 | 72.4 ± 4.5 | 89.4 ± 4.7 |
|           | CARE | **74.7** ± 3.4 | 72.3 ± 4.3    | **90.4** ± 4.6  |
| GAT  | original in GIB paper | 69.5 ± 4.5 | 71.4 ± 4.0 | 73.8 ± 7.4  |
|        | GIB  | **76.9** ± 4.0 | 73.7 ± 4.4    | 74.9 ± 9.7  |
|   | original in our paper | 74.3 ± 3.8 | 74.3 ± 2.1 | 77.6 ± 10.5  |
|           | CARE | 75.4 ± 2.9 | **76.7** ± 1.7    | **77.7** ± 9.0  |




- [Out of Distribution Problem.] Many thanks for this insightful comment. It is pretty interesting to investigate the OOD problem on chemistry and biomolecule datasets and it is gaining much attention in recent years. We are also interested in exploring the effectiveness of our method in the OOD problem, which is one of our future directions.

From: ICLR.cc/2023/Conference/Paper504/Authors
title: Response to Reviewer kwmy
comment: - [Design of Loss Function.] Thanks for pointing out this confusion. We notice there are several typos about the class loss. The class loss should be defined as $L_{class} = \exp (L_{inter})/\exp (L_{intra}) = \exp(L_{inter} - L_{intra})$. We use this definition instead of simply $(L_{inter} - L_{intra})$ to (1) make sure the value of class loss is positive, (2) The exponential function makes it possible to decrease quickly when the class loss is large at the beginning of training. All the class losses in our experiments are correct and we simply amend the definition in the revision.

- [About Generalization Upper Bound.] Thank you for the insightful comment. CARE is proposed as a plug-and-play framework. However, in addition to directly plugging it in a GNN backbone without changing the number of model parameters in the backbone (as what we have done in experiments in the submitted version), it could also be used in a way that the resultant CARE after plug-in has a comparable number of parameters to the original GNN backbone before plug-in. This can be achieved by adjusting the number of parameters in the GNN backbone at the time when CARE is plugged in. To demonstrate this, we conduct a new experiments to match with the setting of Theorem 1. For each GNN backbone, we first set its number of parameters to 100K. For CARE, we adjust the hidden dimension of each GNN backbone to which CARE is applied such that the number of parameters of CARE is also 100K.  As shown in the table below, CARE still outperforms its GNN backbone in 8 out of 9 cases. The results demonstrate that CARE can boost up the graph classification performance without introducing additional parameters. We have formulated this analysis in Appendix E (Page 18), in the revision.

|           |          | DD           | PROTEINS     | MUTAG         |
|-----------|----------|--------------|--------------|---------------|
| GraphSAGE | original | 72.18 ± 2.93 | 74.87 ± 3.38 | 75.48  ± 6.11 |
|           | CARE     | **72.22** ± 3.10 | **75.74** ± 1.68 | **76.08** ± 10.83 |
| GCN       | original | 71.02 ± 3.17 | 73.89 ± 2.85 | 77.52 ± 10.81 |
|           | CARE     | **71.73** ± 4.12 | **74.91** ± 3.59 | **79.27** ± 4.31  |
| GIN       | original | 73.10 ± 2.44 | **72.41** ± 4.45 | 89.36 ± 4.71  |
|           | CARE     | **73.19** ± 4.44 | 70.43 ± 4.69 | **89.70** ± 5.53  |

- [Epoch Number in Experiments.] Many thanks for the comment. As we mentioned in the implementation details (Appendix C.3), we use the early stopping criterion, i.e., we stop the training once there is no further improvement on the validation loss during 25 epochs. This configurature is adopted from [1]. The initial learning rate is $10^{-4}$, the minimum learning rate is $10^{-6}$ and the reduction factor is 0.5. Therefore, the training will stop after the learning rate is reduced 7 times. CARE-GCN stops training in less than 300 epoches because it converges faster than GCN. We also discuss the converges speed of CARE in Section 3.4 of our paper. For the question of "does CARE-GCN degrate when training more epoches?", the answer is no. The learning rate becomes quite small in the later stage of training due to the learning rate reduction mechanism. Consequently, the performance of the model becomes stable towards the end of training. In fact, we have tried to set a smaller minimum learning rate to let the model complete the training in around 1000 epochs, the performance was not much different from that when the minimum learning rate is set at $10^{-6}$.

[1] Vijay Prakash Dwivedi, Chaitanya K Joshi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Benchmarking graph neural networks. arXiv preprint arXiv:2003.00982, 2020.

From: ICLR.cc/2023/Conference/Paper504/Authors
title: Response to Reviewer PTM8 (2/2)
comment: - [Generalization Analysis.] Thank you for raising the concern. We further articulate that, with the identical number of parameters, CARE achieves both better generalization ability and better learning ability than its GNN backbones. 

	Better Generalization Ability: Our conclusion in the generalization ability based on the upper bounds of VC-dimension was not drawn arbitrarily. It is grounded on previous findings in the literature. According to Section 2.2 of [1], the VC dimension provides a loose generalization bound for models and can be used as a guideline for generalization comparison - models with a lower upper bound tend to have better generalization capability. We have added this in Page 15 of the revision to better elaborate the ground of our theoretic analysis. 

	Better Learning Ability: The “learning ability” does not necessarily fade as the generalization bound lowers. As demonstrated in the above experiment that imposes the same number of parameters in both CARE and the GNN backbone, the superior performance of CARE shows that its learning ability is better than that of the GNN backbone. Note that here we care about the generalization error instead of the training error when assessing the learning ability. The performance of the models is assessed on validation/test sets, and a strong ability of fitting training data doesn't necessarily lead to better generalization performance. 
- [Design of Class Loss.] Many thanks for the suggestion. We conduct another ablation study for the similarity metric in the class loss. We use a distance function to replace the cosine similarity in Eqs. (3) and (4) to define the intra-class and inter-class losses. As the distance measures the dissimilarity, we take an inverse of the class loss:
	$$L_{intra} = norm(\sum_{i=1}^{|Y|}norm(\sum_{y_G=i}dis(hc_i, hg_G^{sub}))),$$
	$$L_{inter} = norm(\sum_{i=1}^{|Y|}norm(\sum_{j=i}^{|Y|}dis(hc_i, hc_j)))$$
	$$L_{class} = \exp({L_{intra} - L_{inter}})$$
	The table below shows the results under GCN as the GNN backbone. It shows that CARE with the cosine similarity is more powerful than that with the L2 distance or KL-divergence. We have added this set of results in the revision (Appendix G).
	
|          | D\&D         | PROTEINS     | MUTAG         |
|----------|--------------|--------------|---------------|
| -        | 71.39 ± 2.81 | 74.21 ± 2.74 | 77.11 ± 9.48  |
| L2       | 71.91 ± 4.97 | 74.65 ± 3.54 | 77.69 ± 8.35  |
| KL       | 71.85 ± 4.49 | 74.38 ± 3.16 | 78.74 ± 5.19  |
| cos\_sim | **72.15** ± 3.88 | **75.01** ± 2.91 | **79.30** ± 11.81 |


- [Experiment on Larger Dataset with Multi-class Labels] Thank you for the comment. We provide additional experiments on REDDIT-MULTI-12K dataset. The results below show that CARE outperforms its GNN backbones in all cases. We do not get the result of all GNN backbones due to the limitation of time and computation resources. We will update the full result to the final version of our paper.

|          | GraphSAGE | GCN   | GIN   | GXN   |
|----------|-----------|-------|-------|-------|
| original | 21.71     | 40.26 | 44.92 | 46.01 |
| CARE     | **21.73**     | **43.77** | **48.20** | **46.52** |

- [Consistent Notations.] Thank you for pointing this out. We have modified Figure 1 so that the notations are consistent with those in the text.

- [Reproducibility.] We provide the code of CARE in the Supplementary Material.

[1] Abu-Mostafa, Yaser S., Malik Magdon-Ismail, and Hsuan-Tien Lin. Learning from data. Vol. 4. New York: AMLBook, 2012.

From: ICLR.cc/2023/Conference/Paper504/Authors
title: Response to Reviewer PTM8 (1/2)
comment: - [About Generalization Upper Bound.] Thank you for the comment. To clarify, Theorem 1 doesn't assume that GNN backbone and CARE have the same computing complexity. It assumes that the two models have the same number of parameters. In fact, the theorem proves by showing that under the same number of parameters, **CARE has a lower computing complexity than the GNN backbone**, which is further linked to VC-dimension bounds. 

	CARE is proposed as a plug-and-play framework. However, in addition to directly plugging it in a GNN backbone without changing the number of model parameters in the backbone (as what we have done in experiments in the submitted version), it could also be used in a way that the resultant CARE after plug-in has a comparable number of parameters to the original GNN backbone before plug-in. This can be achieved by adjusting the number of parameters in the GNN backbone at the time when CARE is plugged in. To demonstrate this, we conduct a new experiment to match with the setting of Theorem 1. For each GNN backbone, we first set its number of parameters to 100K. For CARE, we adjust the hidden dimension of each GNN backbone to which CARE is applied such that the number of parameters of CARE is also 100K.  As shown in the table below, CARE still outperforms its GNN backbone in 8 out of 9 cases. The results demonstrate that CARE can boost up the graph classification performance without introducing additional parameters. We have formulated this analysis in Appendix E (Page 18), in the revision.

|           |          | DD           | PROTEINS     | MUTAG         |
|-----------|----------|--------------|--------------|---------------|
| GraphSAGE | original | 72.18 ± 2.93 | 74.87 ± 3.38 | 75.48  ± 6.11 |
|           | CARE     | **72.22** ± 3.10 | **75.74** ± 1.68 | **76.08** ± 10.83 |
| GCN       | original | 71.02 ± 3.17 | 73.89 ± 2.85 | 77.52 ± 10.81 |
|           | CARE     | **71.73** ± 4.12 | **74.91** ± 3.59 | **79.27** ± 4.31  |
| GIN       | original | 73.10 ± 2.44 | **72.41** ± 4.45 | 89.36 ± 4.71  |
|           | CARE     | **73.19** ± 4.44 | 70.43 ± 4.69 | **89.70** ± 5.53  |

From: ICLR.cc/2023/Conference/Paper504/Reviewer_PTM8
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This paper proposes a plug-and-play framework named CARE to enhance the learning ability of GNN-based methods for graph classification tasks. Specifically, CARE first introduces a Class-Aware Refiner to learn a class-specific representation for each class. Then it also designs a class loss that considers intra-class graph similarity and inter-class graph discrepancy to encourage the model to learn more separate class representations. The authors theoretically analyze the generalization ability of CARE by adopting the VC-dimension theory. Experiments verify that CARE can improve the performance of various GNNs.
strength_and_weaknesses: Strength:  
(1)  The idea of identifying subgraphs to learn class-specific representations is interesting.
(2)  The improvement of applying CARE to vanilla GNN methods is significant for graph classification tasks.
 
Weakness:
(1)  The statement “CARE has better generalization upper bound in comparison with its GNN backbone” is over-claimed. According to the proof of theorem 1 in appendix B.2, this statement is solid if and only if the GNN backbone has the same computing complexity as CARE. However, this condition can not be admitted since “CARE is a plug-and-play framework”. Plugging CARE into a GNN backbone as an extra module definitely increases the computing complexity of the vanilla GNN backbone.
(2)  The statement “Based on upperVC(GCN) > upperVC(CARE) and the VC theory, we conclude that our CARE has a better generalization potential than its GCN backbone” seems no sense. In the VC theory, a model having a lower VC dimension means that its learning ability is weaker than the other. If we consider poor learning ability as generalization ability, then a Linear model will have the best generalization potential, which is against our understanding.
(3)  The designs of “Class Loss” can be further improved. In Eq (3) and (4), the authors apply cosine similarity to measure the distance between different class representations, and between each graph representation and its class representation. Since these representations are all high-dimensional vectors, KL-divergence may be more effective in measuring the differences.
 
Discussion:
(1)  The datasets chosen by the authors are either small for each graph (average node ~ 30), or small in the number of graphs (total graphs ~ 1000). Meanwhile, the numbers of class labels of these datasets are also very small (~2). I wonder whether these datasets are challenged enough to show the full potential of CARE. I suggest the authors can conduct experiments on more challenging datasets, such as REDDIT-12K (Yanardag and Vishwanathan, 2015) and WEB (Riesen and Bunke, 2008). 
clarity,_quality,_novelty_and_reproducibility: Clarity and Quality: The paper is well organized, and the overall writing is OK. But the authors should take care the consistency of notations, and the definition of some notations. For example, In Figure 1, a subgraph selected by the Subgraph Selector is written as $G_{sub}=(A_{sub}^{(l)},H_{sub}^{(l)}$, which differs from that of described in the Section 2.2.

Novelty: Identifying subgraphs to learn class-specific representations is novel.

Reproducibility: The author did not provide any code about their experiments.  

summary_of_the_review: This paper proposes a flexible plug-and-play framework, named CARE, for enhancing GNN models on graph classification tasks, and extracting shared sub-graphs within the same label is an interesting insight. Although the experiments show the effectiveness of CARE on various GNN architectures, the theoretical justification is solid enough. Besides, this paper is well organized, and the writing is OK, but the authors should pay attention to the details of the notations. Overall, this paper is below the acceptance borderline if the authors cannot address the concerns mentioned in the Weaknesses.
correctness: 2: Several of the paper’s claims are incorrect or not well-supported.
technical_novelty_and_significance: 4: The contributions are significant, and do not exist in prior works.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper504/Reviewer_kwmy
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: The paper incorporates class-specific information in graph-level into representation learning for GNNs and thus present a so-called class-aware representation refinement (CARE) approach, in which the class-specific information is learnt from a bag of subgraph representation from each class and is used to refine the graph representation within the same class. Moreover, some justification based on model complexity for the generalization ability is provided. Extensive experiments are conducted on several benchmark datasets with variant GNN backbones, showing superior performance. 

strength_and_weaknesses: The strengths of the paper:
+ Incorporating the class-specific information in graph-level into representation learning in GNNs is novel. 

+ Empirical evaluation is extensive and the improvements demonstrates the effectiveness of incorporating the class-specific sub-graph information. 


The weaknesses of the paper:
- While the paper is relatively easier to read, the presentation could be further improved. In particular, the symbols used in the paper seems somewhat chaos and easily confused.  Moreover, the class loss $L_{class}$ is defined as $\log (\exp() / \exp ())$, is it simply $L_{inter} - L_{intra}$? 


- The VC dimension based justification seems not fair and meaningful. It is at the end of the day a comparison on the computation complexity. It is curious to the reviewer that if CARE is used as a plus-in component for, e.g., a GCN, then, whether or not the VC dimension based conclusion in Theorem 1 is still valid? 

- The results in Fig. 2 are incomplete. Note that GCN is trained 300 epoches, why not for CARE-GCN to train enough number of epoches as for GCN? Does CARE-GCN degrate when training more epoches, e.g. 300?

clarity,_quality,_novelty_and_reproducibility: The quality of the paper is okay but not good enough. The presentation could be further improved. The contribution is novel. 
summary_of_the_review: The idea to incorporate the class-specific sub-graph information is novel. The empirical evaluation is extensive. Though the presentation could be further improved, the paper is well motivated and relatively easier to follow. 
correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper504/Reviewer_1G71
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This paper proposes a plug-and-play framework for arbitrary GNNs to obtain powerful class representations with better generalization on the task of graph classification, with sufficient theoretical proof. The framework mainly relies on the Class-Aware Refiner block to strengthen the supervision of graph labels into its representations by adding class-specific subgraph features. Empirical results show the effectiveness and superiority of this framework in task performance and efficiency.
strength_and_weaknesses: Strength
1. The intrinsic design idea and logic of this framework are straightforward and clear, and with sufficient derivation and proof, the rationality and validity of the framework are verified theoretically.
2. The experimental settings are very detailed, and the experimental results can fully demonstrate the effectiveness and superiority of the framework. More experimental analysis and case analysis further clarified the mechanism and results of the model.

Weaknesses
1. In my opinion, the biggest problem in your framework is the lack of innovation. In simple terms, the core of the framework is to enhance the supervisory information function of the class label by adding the common features of the class to the graph representation. This framework achieves this target by simply calculating the similarity, clustering, and other heuristic-like methods.
2. The second problem comes from experimental data sets. From the 9 data sets you selected for the graph classification task, I noticed that only the ENZYMES dataset has more than 2 classes, and the others all have only 2 classes, which could be seen as a binary classification task for graphs. Can this be seen as simplifying the graph classification task and thus showing the advantages of your framework? Or does your framework only perform better on the binary classification task?
3. You mentioned “The GNN-based graph representation learning has two main steps” in the introduction section. But to the best of my knowledge, not all the GNN-based graph representation learning methods follow the “two steps” paradigm. Some of the methods obtain graph embedding by pooling the node embeddings, while some refine the graph structure to hierarchically get a coarsened graph structure and finally obtain the graph embedding. Related works of graph representation learning are under-explored and summarized in your paper. As a consequence, in the experiment part, there was no comparison with the related graph representation learning baselines, which was another major flaw.
4. There are many works that have the same motivation as this paper by finding the class-relevant subgraphs[1][2] for representation refinement, which also should be included in the related works or the baselines. 
5. Since your datasets mainly focus on chemistry and biomolecules, a common scenario is the sample’s distribution of training is different from the sample’s distribution of validation and test, especially the common subgraph features that your framework mainly emphasized, which may make your assumption invalid. It’s an interesting problem for you to further explore.

[1] Graph information bottleneck for subgraph recognition, ICLR 2021  
[2] SUGAR: Subgraph Neural Network with Reinforcement Pooling and Self-Supervised Mutual Information Mechanism, WWW 2021. 
clarity,_quality,_novelty_and_reproducibility: This framework is of average quality and kind of lacking innovation. The description of this framework is clear and the theoretical analysis is sufficient. In the experimental part, the validity analysis of the results is relatively redundant. The originality of the work is acceptable.
summary_of_the_review: The authors propose a plug-and-play framework for arbitrary GNNs to obtain powerful class representations with better generalization on the task of graph classification, with sufficient theoretical proof. However, this work lacks some innovation and has obvious limitations and deficiencies in the experimental datasets. Especially, there lacks comparison with related graph representation baselines.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper504/Reviewer_J7S2
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: The paper proposes a refinement GCN framework for graph classification tasks. It consists of four components: a graph encoder, a subgraph selector, a class-aware refiner, and a class loss. Numerical experiments show that the empirical performance is comparable to a standard GCN, slightly outperforming the GCN on some tasks. 
strength_and_weaknesses: Strengths. The paper is generally well and clearly written. It considers an important and not an easy task of classification of a collection of graphs. 

Weaknesses. The proposed framework contains already existing blocs, and the novelty of the proposed approach is limited. Personally, I supposed that the novelty would come from the subgraph selector, since I guess it is the key component to perform the discrimination between classes. However, the subgraph selector uses the pooling methods which are already known. 

More attention is drawn to the class-aware refiner. I am a little bit confused, since it is not clear whether we are in an unsupervised or supervised setting. It is stressed that the pseudo labels are obtained and used, however, why do we need them if we have an access to the true labels? How the pseudo labels are used?

The loss function (equation 5): it is difficult to say whether it performs well without testing the two terms separately as loss functions (L_{cls} and L_{class}) and compare these results to the loss which combines them. For instance, Table 4 shows some results for various losses but I cannot find the results for the loss proposed in equation 5 in this table. 

Authors state that the CARE gains about 1% improvement. It is probably statistically not significant improvement, and indeed, the differences (in performance) between two compared models in Table 1 are small and not convincing.

Other remarks. 
Maybe I missed something but it is stated (Section 2.4) that the CARE has a better generalisation compared to its GCN backbone, since it has a lower VC dimension. I am not sure that this conclusion is convincing. A lower VC dimension = a higher bias and a higher VC dimension allows for a lower empirical risk. Could you provide more comments on your generalisation analysis?

In Section 3.2, in the Class Loss paragraph, the "intra" loss has a strange term: it seems that \lambda takes a negative value. What is the intuition behind such a loss?

Figure 3 is small and difficult to read. 

I am a little bit surprised by the definition of the silhouette coefficient. Usually, it is used in an unsupervised setting, without true labels. 
I also wonder why the obtained silhouette values are so small (Figure 3), especially compared to the performance reported in Table 4. 

Figure 4 is not extremely convincing: the CARE-GCN leads to a slightly better projection, however, it is not (in my opinion) significantly better then the GCN visualised on the left.  


clarity,_quality,_novelty_and_reproducibility:  The paper is generally well and clearly written.
The proposed framework contains already existing blocs, and the novelty of the proposed approach is limited.
summary_of_the_review: Some points need to be clarified. 
The results of the numerical experiments are not extremely convincing. 
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold