Label-invariant Augmentation for Semi-Supervised Graph Classification
Authors: Han Yue
Chunhui Zhang
Chuxu Zhang
Hongfu Liu
Venue: NeurIPS 2022 Accept (2022)
Keywords: Graph Contrastive Learning
Semi-supervised Classification

Abstract: Recently, contrastiveness-based augmentation surges a new climax in the computer vision domain, where some operations, including rotation, crop, and flip, combined with dedicated algorithms, dramatically increase the model generalization and robustness. Following this trend, some pioneering attempts employ the similar idea to graph data. Nevertheless, unlike images, it is much more difficult to design reasonable augmentations without changing the nature of graphs. Although exciting, the current graph contrastive learning does not achieve as promising performance as visual contrastive learning. We conjecture the current performance of graph contrastive learning might be limited by the violation of the label-invariant augmentation assumption. In light of this, we propose a label-invariant augmentation for graph-structured data to address this challenge. Different from the node/edge modification and subgraph extraction, we conduct the augmentation in the representation space and generate the augmented samples in the most difficult direction while keeping the label of augmented data the same as the original samples. In the semi-supervised scenario, we demonstrate our proposed method outperforms the classical graph neural network based methods and recent graph contrastive learning on eight benchmark graph-structured data, followed by several in-depth experiments to further explore the label-invariant augmentation in several aspects.

Label-invariant Augmentation for Semi-Supervised Graph Classification
Han Yue
Chunhui Zhang Chuxu Zhang Hongfu Liu
Michtom School of Computer Science Brandeis University, Waltham, MA
{hanyue, chunhuizhang, bbngha edu
Abstract
Recently, contrastiveness- based augmentation surges new climax in the computer vision domain, where some operations. including rotation, crop, and flip, combined with dedicated algorithms,, dramatically increase the model generalization and robustness. Following this trend, some pioneering attempts employ the similar idea to graph data. Nevertheless, unlike images, it is much more difficult to design rea- sonable augmentations without changing the nature of graphs. Although exciting, the current graph contrastive learning does not achieve as promising performance visual contrastive learning. We conjecture the current performance of graph contrastive learning might be limited by the violation the label-invariant aug- mentation assumption. In light of this, we propose label-invariant augmentation for graph-structure data to address this challenge. Different from the node/edge modification and subgraph extraction, sentation space and generate the augmented samples in the most difficult direction while keeping the label of augmented data the same as the original samples. In the semi-supervised scenario, we demonstrate our proposed method outperforms the classical graph neural network based methods and recent graph contrastive learning on eight benchmark graph-structured data, followed by several in in-depth experiments further explore the label-invariant augmentation in several aspects.
conduct the augmentation in the repre-
Introduction
Contrastive augmentation aims expand training data in both volume and diversity in a self- supervised fashion to increase model robustness and generalization. Common sense and domain knowledge are employed to design the contrastive augmentation operations. Denoising auto- encoder B is one of the pioneering studies to apply perturbations to generate contrastive samples for tablet data, which takes a corrupted input and recovers the original undistorted input. For visual data, some operations, including rotation, crop, and flip, combined with dedicated algorithms, signifi- cantly improve the learning performance in diverse tasks 28 and original samples positive pairs and the augmented samples from different source samples as negative pairs, contrastive learning aims learn augment-invarin the similarity of positive pairs and the dissimilarity of negative pairs (5), These positive pairs increase the model robustness due to the assumption that the augmented operations preserve the nature of images and make the augmented samples have consistent labels with the original ones. The negative pairs work as the instance-level discrimination, which expected to enhance the model generalization. but might deteriorate the downstream task since the negative pairs contain the augmented samples from different source samples but with the same category. The recent B BYLL [16] and SimSiam [7] demonstrate the negative effect of the negative pairs and conclude that the current performance of contrastive learning can be further boosted even without negative pairs.
19
6). Treating the augmented
representations by increasing
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Following this trend, some pioneering attempts employ contrastive augmentation to graph data [ GraphCL 40 is the first work to address the graph contrastive learning problem with four types of augmentations, including node dropping, edge perturbation, attribute masking, and subgraph extrac- tion. Later, JOAO [39] extends GraphCI. by automatically selecting one type of graph a from the above four types plus non -on-augmentttion. GRACE [42 treats the original graph data and the novel-level augmented data as two views and learns the graph representation by maximizing the agreement between the two views. Similarly, MVGRL [88 conducts the contrastive multi-view representation learning on both node and graph levels. Beyond the above studies to augment graphs, simGRACE [36] perturbs the model parameters for contrastive learning, which can be regarded as an ensemble of model perturbation or a robust regularization. Although exciting, the above studies point out that the effectiveness of graph contrastive learning heavily hinges ad-hoc data augmentations, which need to be carefully designed or selected per dataset and request more domain knowledge.
Contributions. We conjecture these hand-crafted graph augmentations might change the nature of the original graph and violate the label-invariant assumption in the downstream tasks. Different from treating graph contrastive learning in pre-trained perspective, we aim to incorporate the downstream classification task into the representation learning, where the label information is fully used for both decision boundary learning and graph augmentation. Specifically, we propose Graph Label-invariant Augmentation (GLA), which conducts augmentation in the representation space and augments the most difficult sample while keeping the label of the augmented sample the same the original sample. Our major contributions are summarized as follows:
We propose a label-invariant augmentation strategy for graph contrastive learning, which involves labels in the downstream task to guide the contrastive augmentation. It is worthy to note that we do not generate any graph data. Instead, we directly generate the label-consistent representations as the augmented graphs during the training phase.
In the rich representation space, we aim to generate the most difficult sample for the model and increase the model generalization. We choose a lightweight technique by randomly generating a set of qualified candidates and selecting the most difficult one, i.c., minimizing the maximum loss or worst case loss over the augmented candidates.
We conduct series of supe-super experiments on eight graph benchmark datasets in a fair setting and compare our label- -nbarianv augmentation with classical graph neural network based methods and recent graph contrastive methods by running the codes provided by the original authors. Extensive results demonstrate our label-invariant augmentation can achieve better performance in general cases without generating real augmented graphs and any specific domain knowledge. Besides algorithmic performance, we also provide rich and in-depth experiments to explore label-invariant augmentation in several aspects.
2
Related Work
Here we introduce the related work of graph neural networks and graph contrastive learning for graph classification. Node classification, although related, is not covered here due its different setting.
Graph Neural Network. Graph Neural Networks (GNNs) have been employed on various graph learning tasks and achieved promising performance (23]. To extract the representation of each node, GNNs pass node embeddings from its connected neighbor nodes and apply feedforward neural networks transform the aggregated features. As pioneer study in GNNs, graph convolutional network (GCN) firstly aims to generalize the convolution mechanism from image to graph [33 35 [4] Based on GCN, instead of simply summing and averaging connected neighboring node's embedding. graph attention networks (41 [3] adopt an attention mechanism that builds self-attention to score each connected neighboring nodes' embedding to identify the more important nodes and enhance the effectiveness of message passing. Then in order to break prior GNN's limitations on message passing over long distances on large graphs, graph recurrent neural networks [ 9 apply the gating mechanism from RNNs to propagation on graph topology. Simultaneously, for dealing with the noise introduced from more than 3 layers of graph convolution, DeepGCN [25] 24 uses skip
connections and enables GCN to achieve better results with deeper layers. Recently, GAE (2 and Infomax (D) achieve state-of-thhr -he- performance on several benchmark datasets. GAE extends the variational auto-encoder to graph neural networks for unsupervised learning, while Infomax learns

---

the unsupervised representation on graphs to enlarge mutual information between local (node-level) and global (graph-level) representations in one graph.
Graph Contrastive Learning. Recently, many studies have been devoted to the graph contrastive learning area in diverse angles, including graph augmentation, negative sample selection, and view fusion. GraphCL [40] summarizes four types of graph augmentations to learn the invariant representa- tion across different augmented views. Built on GraphCL, JOAO [39] proposes a learnable module to automatically select augmentation for different datasets to alleviate the human labor in combinations
of these augmentations. Differently, MVGRL [8] contrasts node and graph encodings across views which enriches more negative samples for contrastive learning. Later, InfoGCL [37] diminishes the mutual information between contrastive parts among views while preserving the task-relevant repre- sentation. Beyond augmenting graphs, SimGRACE [36] disturbs the model weights and then learns the invariant high-level representation at the output end to alleviate the design of graph augmentation.
Different from the above methods that separate the pre-train and fine-tuning phases, we aim to employ the label information in downstream tasks to guide the augmentation process. Specifically, in this study, we propose a label-invariant augmentation strategy for graph-level representation learning.
3 Methodology
graph can be represented by G = (V, X, A), where V = f0i, Up is the set of vertexes, X [Radd denotes the features of each vertex, and A {0, 1_nx represents the adjacency matrix. Given a set of labeled graphs S = {(G,, 91), (G2, 122,, ..., (GM, (Mr,)/r)))))))))))>())))) where M is the number of labeled graphs, and y/ E is the corresponding categorical label of graph G, E G (I<i<M), and GN J. where N is the number of all graphs MrN,
another set of unlabeled graphs T {G the semi-supervised graph classification problem can be defined learning mapping function from graphs to categorical labels f : 9 + y to predict the labels of T. In this section, we first illustrate our motivation supported by empirical evidence, then we elaborate on our Graph Label-invariant Augmentation (GLA) method for semi-supervised graph classification.
3.1 Motivation
Augmentation plays an important role in neural network training. It not only
GraphCI.
JOAOv2
2.00
0.00
1.11
improves the robustness of learned representation but also introduces rich data for training. For graph- structured data, GraphCL. [40] pro- poses four types of augmentations: node dropping, edge perturbation, at- tribute masking, and subgraph sam- pling. However, it is widely noticed that the effectiveness of graph con- trastive learning highly depends on the chosen types of augmentations for specific graph data (3 39.. To illus-
1.08
1.02
1.00
Nis
N-F
N-A
F-5
A+5
F-A
2.17
-2.16
-0.73
0.65
-0.49
-2.10
0.00
N-E+A
N+A+5
N-E+5
1.17
0.06
-0.03
1.08
Nade dropping pertrrbation
-1.00
Altribate Sebgraph
1.61
Figure 1: Performance gains (%) of GraphCI. and JOAOv2 under different augmentation settings on MUTAG [10
dataset compared to none augmentation setting.
trate the difference between various augmentation combinations, we conduct experiments on MU- TAG [10] in a semi-super graph classification task, where the label rate is set to 50%. Figure [ shows the performance gains (classification accuracy %) of different augmentation combinations compared to none augmentation. We can see that different augmentation combinations result in different performances, and JOAOv2 automatically selects data augmentations but cannot guarantee to outperform GraphCL with all augmentation combinations. Moreover, some augmentation combi- nations work worse than none augmentation, which demonstrates that some augmentations hurt the model training. We further fine-tune a model with 100% labeled graphs from MUTAG, and then feed the augmented graphs (randomly selected from the four augmentation types with an augmentation ratio of 0.2) to this model, finding that about 80% augmented graphs have the same labels with their corresponding original graphs. It indicates that most augmentations are reasonable, which is one of the reasons that GraphCL works well. While on the other hand, there are still about 20% augmented
3

---

GNN Encoder
Origiaal Rnprenetrrion
Projection Head
Oridaul riiiii
Mat Matimizing
Agreement
Origitel Predction
Auperatte Profection
Input
Label-invariant Aumeenaaiion
Classifier
Augumttdd Deprosetttiinss
Figure 2: Framework of our Graph Label-invariant Augmentation (GLA) for semi-supprvisee graph classification. Given an input graph, a Graph Neural Network (GNN) Encoder employed to encode the input graph into graph-level representation (original representation). Then perturb the original representation to get multiple augmented representations. A classifier is adopted to verify whether the augmentations are label-invariant not. We select the "hardest" augmented representation, i.e., the one that has the least probability of belonging to the same class as ground- truth label/original representation for labeled/unlabeled graph, from all augmented representations that satisfy the label-invariant constraint. On top of GNN Encoder, we build a projection head to get projections for both original representation and label-invariant augmented representation. We maximize the agreement between projections via contrastive loss for all graphs and refine the classifier via a cross-entropy loss with labeled graphs.
graphs getting different labels from the original ones. Motivated by this, design lubel-invariant augmentation strategy for graph contrastive learning.
3.2 Label-invariant Augmentation
Framework Overview. Figure 2 illustrates the framework of our proposed Graph Label-invariant Augmentation (GLA) for semi supe-super graph classification, which mainly consists of four com- ponents: Graph Neural Network Encoder, Classifier, Label-invariant Augmentation, and Projection Head. We first use GNN Encoder to get graph-level original representation for the input graph. Then Label-invariant Augmentation, together with Classifier, is utilized to generate augmented represen- tation from original representation under a label-invariant constraint. For an unlabeled graph, we expect that the labels represented by original prediction and augmented prediction are the same. For labeled graph, we expect that the label represented by augmented prediction is the same as the ground truth label. cross-entropy loss used keep refining the classifier with labeled graphs. Finally, Projection Head is adopted to generate projections for contrastive loss. We use + = {θg. lc, Up) to denote the trainable parameters set, where lg. 9c. and fp denote the parameters of GNN Encoder. Classifier and Projection Head, respectively. Details of each component are as follows.
Graph Neural Network Encoder. Graph Neural Network Encoder aims get graph-level repre- sentations for graph-structured data, is flexible to adopt various GNNs for this part. We follow GraphCL (40) and utilize ResGCN (4), which takes Graph Convolutional Network (GCN) (2] the backbone, to extract node-level representations from the input graph, and then a global sum pooling layer is used to obtain its graph-level representation. The computation of the GCN layer with the parameter 0g is described as follows:
((++1) -
where À - A In is the adjacency matrix A with added self-connections, In E Rnxn is the identity matrix, D is the degree matrix of Á, and 0 is layer-speeific trainable weight matrix. G denotes the matrix the 1-th layer, and (1o) X. We employ (-)) = ReLU(-) as the activation function.
Then on top of the ResGCN, we use global sum pooling layer to get graph-level representations from node-level representations as follows:
H Pooling(G).
Here we use H to denote the original representation of the input graph and H to denote the augmented representation of the augmented graph. The augmentation method will be described in the Label-invarian Augmentation part.

---

Classifier. Based on the graph-level representations, we employ fully-connected layers with the parameter &e for prediction:
(((++) = = O
(3)
where CO denotes the embeddings in the I-th layer, and the input layer CtO)) HO or C()) H for the original representation and augmented representation, respectively. 2-layer multilayer perceptron and obtain predictions CO and C for original representation H and augmented representation HA, and o()) = ReI.U(.) for the first layer and (-)) = Softmax for the second layer as the activation function.
our experiments, we adopt
Label-invariant Augmentation. Instead of augmenting graph data by node dropping, edge perturba- tion, attribute masking, subgraph sampling as recent graph contrastive learning methods [40] 39), we conduct the augmentation the representation space by adding a perturbation to the original representation Hº that do not need to generate any graph data. In experiment, first calculate the centroid of original representations for all graphs and get the average value of euclidean distances between each original representation and the centroid as d, that is:
1 N
d
I
E
(4)
N
Then the augmented representation H is calculated by:
= = H + ndd,
(5)
where 7) scales the magnitude the perturbation, and is a random unit vector.
Based on the classifier formulated in Eq. (3. we define label-invariant as follows. For labeled graphs, label-invariant means the predictions of augmented representations by the classifier are the same as their corresponding ground-truti labels. For unlabeled graphs, label-invariant denotes that the predictions of augmented representations and the predictions of original representations by the classifier are the same.
To achieve the label-invariant augmentation, for each graph, we randomly generate multiple perturba- tions and select the qualified augmentation candidates that obey the these qualified candidates. we choose the most difficult one, i.c., the one that has the least probability of belonging to the same class as ground-truu label/original representation for labeled/unlabeled graph, to increase the model generalization ability.
property. Among
Projection Head. We employ fully-connected layers with the parameter 0p to get projections for contrastive learning from graph-level representations, which is shown
p(l+1) = (p(() 000
(6)
We adopt 2-layer multilayer perceptron and get projections pe and PA from original representation Hº and augmented representation HA.
Objective Function. Our objective function consists of contrastive loss and classification loss. For
contrastive loss, utilize the normalized remperature-scaled cross-entropy loss (NT-Xent) [40 but only keep the positive-pai part as follows:
Cp (PO)) Pd P
(7)
Maximizing the agreement between original projection and augmented projection would increase the robustness of the model.
For classification loss, adopt cross-entropy, which is defined as:
Lc - --- log P.o + Y,, log P.A),
(8)
where yo is the label of the input graph, and e is the number of graph categories. We only calculate Le for labeled graphs. The improvement of the classifier would help with the label-invariant augmentation, which in turn benefits the training of the classifier.
5

---

Table 1: Statistics of datasets for semi semi-supervised graph classification
Datasets
Category
#Class
#Ciraph
Avg. #Node
Avg. #Edge
MUTAG
Biochemical Molecules
188
17.93
19.79
PROTEINS
Biochemical Molecules
2
1,113
39.06
72.82
DD
Biochemical Molecules
2
1,178
284.32
715.66
NCII
Biochemical Molecules
2
4,110
29.87
32,30
COLLAB
Social Networks
5,000
74.49
2,457.78
RDT-B
Networks
2,000
497.75
RDT-M5K
Social Networks
4,999
508.52
594.87
GITHUB
Social Networks
12,725
113.79
234.64
Combining Eq. and 8
our overall objective function can be written as follows:
ming Lp + acC,,
where a is a trade-off hyperparameter to balance the contrastive loss and classification loss.
Discussion, From the perspective of information usage for model training, our proposed method is the same as the semi-supervised learning task by recent graph contrastive learning methods [40] 39 18 36). which use structure information of all graphs and label information of subset of all graphs for model training. From the perspective of training strategy, the previous methods first pre-train a model via contrastive loss and then fine-tune the model for downstream tasks. While our proposed method focuses on semi-supervised classification, we merge the pre-train and fine-tuning phases into one integrated phase. During training phase, the augmented samples increase the model robustness and generalization ability, and the classifier helps to generate better augmented samples, which in turn benefits classification performance. Theoretically, our method is well supported by MaxUp [15] where the hardest sample augmentation can be regarded as gradient-norm regularization.
Experiments
In this section, we first describe our semi-supervised settings of experiments and baseline methods for comparison. Then we show the algorithmic performance of these methods on eight graph benchmark datasets fair setting. Finally, we provide some insightful experiments to demonstrate the effectiveness of the proposed Graph Label-invariant Augmentation (GLA) method.
4.1 Experiment Settings
Datasets. We select eight public graph classification benchmark datasets from TUDataset [27 for evaluation, including MUTAG [TO), PROTEINS (1. DD (12), NCT/ [32], COLLAB (38, RDT-B 38 RDT-M5K (38. and GITHUB (29] Table shows the statistics of these datasets. The first four datasets include biochemical molecules and proteins, and the last four datasets
about social networks. The numbers of graphs in these datasets range from 188 to 12,725, the average node numbers range from 17.93 to 508.52, and the average edge numbers are from 19.79 to 2,457.78, indicating the diversity of these datasets.
Compared Methods and Implementation. We choose two heuristic self-supervise methods, GAE 22 and Infomax [30] and four recent graph contrastive learning methods, MVGRL [18),
GraphCL [4), JOAOv2 (39], and SimGRACE [36), for comparison semi-supervised graph classification task. GAE performs adjacency matrix reconstruction by using a graph convolutional network (GCN) (21] encoder and a simple inner product decoder. Infomax is based global-local representation consistency enforcement, which maximizes the mutual information between global and local representation. MVGRL proposes to learn node and graph level representations by node diffusion and contrasting encodings. GraphCL presents four types of graph augmentations. Based on GraphCL, 10 is designed as unified bi-level optimization framework to automatically select graph augmentations. SimGRACE perturbs parameters graph encoder for contrastive learning, which does not require data augmentations.
For GAE, Infomax, and GraphCL, we adopt the implementations and default hyperparameter settings provided by the source codes of GraphCI 40 For other compared methods, we follow the implementations and hyperparameter settings in their corresponding source codes. The compared methods are pre-trained first and then fine-tuned for the semi-supervised graph classification task. For

---

Table 2: Semi-supervised graph classification results (Accuracy % + Standard Deviation %) on eight benchmark datasets. The best and second-best results are highlighted red and blue, respectively.
Avg.
GAE
85.83
76.31 + 8.58
71.33 2-036
77.20 0.0.2
77.46 0.11
90.75 12 0.11
52.11 2:0.8
05.21 4:21::100
75.09
5,00
Infremx
84.08 112
76.84 5.35
71.00
2-945
79-49 2:0.1
77:00 2.19
50.05
55.37 2.000
06.05
75.73
0.25
MYDRI
71.00
2.916
71.41..01 1.1
2.13
58.10 2.115
53.16 006
7% 20
6.12
10%
SimGRACF
83.63
74.
1.33
1.916
1.0.7
1.31
90.60 2.17
1.666
66.81
75.46
GoptCT.
72.12 230
1.031
1.0.
77:00
00:35
611
67.67
1.845
75.14
3.38
30AD+2
75.00
77.16
2:030
1.0.1
11:17 78.78
79.58
91.65
67.96
76.41
(Durs)
86.32
75:65
77:49 0.40
0:133
91.03
0.23
0122
65.16 BMV
76.25
GAE
84.12 250
74.75 11 3.66
78.35
0:0.3
79.56
1:0.6
60.47
11:34
90.95 22 ALU
55,69 1.16
67.00
0.15
76.37
5.88
Infomax
87.27 1.11
75.38
3.56
75.26
0.36
90,50
0-133
79.70
0..1
91.50 + 1.25
56.51
1.15
67.70
5UM)
77.15
3.75
MYGRL
85.79 022
76.72
3.54
75.00
0.46
74.09
0.110
76.08
0L.5
$8.55
$4.04 106
64.89
548
74.84
5.62
5016
SinGRACE
85.32 3383
75.09
3.55
75.39
9-3.5
79.78
0.244
80.48
0.15
91.45
+ 0L15
56.50 (1.)
67.71
0.16
76.97
4.50
GepaCL
87.28 4:371
75.29
3:29
78.23
--3.6
80.17
0.1.9
80,40
0.15
91.45 0.25
5683 0.10
66.71
600
77.36
3:25
85.78 4:379
75.74
2 3.29
75.50 2-045
90.10 0-1.1
81 0.18
92.10 0.11
56.51 2:017
68.97
8.1
72.53
2.25
GLA (Ours)
90.00 2: 3.4
75.19 2 9.56
2.337 W6.22
80/00
9:0.8
80.54 0.12
V1.03 0:22
36.67 0:13
00.29 0114
77:85
2.25
GAE
81.51 2:366
75.47 +: 3:58
79.37 9:3.6
79.78 0.11
MA78 0.12
91.50 0.19
56.25 2:0.5
0K.22 2: 8.14
71:36
5.02
Infomax
AR. 2:775
75.92 3 9.58
79.28 2:033
87.25 2:0.6
K1 04 n-1
91.15
2.13
36.1 2:0
08.83 2: 9.16
7k:14
3.62
MVGRI
81.05
81.95 3:355
71.81
1 3:5
29:51 1:034
74.4%
0.0
76.42
76.42.1.00 n.1
18.65
$4.40.1.01
65.00
km
75.90
5.25
70%
SinGRACE
83.55 1.371
76.52
5.6
78.00 1:33
61.50 0.15
81 1.3
92.A5
$6.58 DUM
68.19
915
77.96
4.12
GrgghT.
88.23
338
76.56
325
70.08
1:0.9
82.50
0.33
81.88 L.177
91.85
5691
1
69.19
76.16
3,62
30AD+2
87.78
3.76
78.46
1 9:7
70.11
2
81.70
5:0.6
82.16 0.37
92.20
56,67 1.16
69.96
D.1
78.26
3225
GLA (Curs)
91.08 2: 9.6
77:45
à 938
N6.71 2
K3.24 0.14
KI.
M1.54:00.14
91.70-2017
$7,01 2.0
67.11
9.18
78:73
2.50
100
Avaroge Accurcoy Gain
(16)
Race--narrannteatee Rate (%)
80
7.5
(Ours)
GraphCL
MUTAG PROTEINS
DD
JOADv2
COLLAB
30%.~50%
30%->70%
MVGRL
(Our))
30%
70%
Ratio
(a) performance gain
(b) label-invariant rate distribution
(c) GLA's label-invariant rate
Figure 3: Performance gain and inveriiivti rates. (a) demonstrates the average performance gains on eight datasets with more labeled samples produced by GraphCL, JOAOv2, and GLA. (b) shows the label-invariant rate distributions of different augmentation methods over eight datasets. (c) shows the label-invariant rates of our GLA over different semi-supervised settings.
our proposed Graph Label-invariant Augmentation (GLA) method I we perform contrastive learning and graph classifier learning synchronously. The implementation details of GLA are as follows. We implement the networks based on GraphCL [40] by Py Torch, set the magnitude of perturbation 17 to 1.0, and the weight of classification loss a to 1.0, which is the same with GraphCL. We adopt Adam optimizer (20 to minimize the objective function in Eq. (9)
Evaluation Protocol. We evaluate the models with 1 10-fold cross cross- validation. We randomly shuffle dataset and then evenly split into 10 parts. Each fold corresponds one part of data as the test set and another part as the validation set to select the best epoch, where the rest folds are used for training. We select 30%, 50%, 70% graphs from the training set as labeled graphs for each fold, then conduct semi-supervised learning. For fair comparison, we use the same training/validation/es splits for all compared methods on each dataset, and report the average accuracy across 10 folds.
4.2 Algorithmic Performance
Table 2 shows the prediction results of two self-supervised and five graph contrastive learning methods under the semi-supervised graph classification setting with 30%, 50%, and 70% label ratios on eight benchmark datasets, where the best and second-best results are highlighted in red and blue, respectively, and the last column is the average rank score across all datasets. Although different algorithms achieve their best performances on different datasets, the contrastiveness-batt methods perform better than the non contrastiiveeessss-eee based methods in general, which indicates the effectiveness the graph augmentation. Our proposed GLA achieves the best ranking scores under all 30%, 50%, and 70% label ratios in experiments, the second-bes. average performance
'Our code is available at htttttts://gtthii www.///aaneennnnnnneeeee -ea.nnnggaarnnnnnaaeee
7

---

under 30% label ratio, and the best average performance under 50% and 70% label ratios. In our algorithmic design, we employ the decision boundary learned from the labeled samples to verify the label-invariant augmentation. It is worthy to note that the quality of the decision boundary depends on the number of labeled samples. We conjecture that a 30% label ratio is not sufficient enough to learn high-quality decision boundary, resulting in our GLA performing slightly worse than JOAOv2 on average, With more labeled samples, our GLA delivers the best average performance over other competitive methods. Different from other graph contrastive learning methods, our augmentation method aims to generate label-invariant augmentations, which decreases the possibility of getting "bad" augmentations, thus resulting in better performance.
Besides the general comparison in Table 2 we dive into details and discover several interesting findings. Figure 3(a) demonstrates the average performance gains eight datasets with more labeled samples produced by GraphCI., JOAOv2, JOAA02, and our GL.A, the top three methods in our experiments. In addition to seeing that the increased performance of all three methods well aligns with more labeled samples, our GLA receives more performance gains than GraphCL. and JOAOv2. By such comparisons, we can roughly eliminate the effect of more labeled samples and attribute the extra gains to the label-invariant augmentation. It also verifies our aforementioned conjecture that the high-quality decision boundary is beneficial to the label-invariant augmentation, further bringing in the performance boost. Moreover, we further verify our motivation by checking the tabel-invarian property of different contrastive methods. While do not have ground truth classifier, we fine-tuned classifiers in the representation spaces learned by these contrastive methods with a 100% label ratio as the surrogates of the ground truth classifier. Then we use these classifiers to assess how many of the augmented representations belong to the same class as their corresponding original representations. Figure (bb presents the distributions of label-invariant rates across eight baseline datasets for all graph contrastive methods. As our GLA trained under different label ratios would generate different augmentations, we put the results of GLA's label-invariant rates under 30%, 50%, and 70% label ratios together for plotting. We can see that GLA has the highest label-invariant rates on average compared other methods. It is also noticed that the label-invariant rates of different contrastive methods keep the same ranking with the performance in Table 2 verifies our motivation for designing label-invariant augmentation strategy. Moreover, we further demonstrate our GLA's label-invaricn rates along with different label ratios in Figure 3(c] which accords with our expectation that more labeled samples lead to a high-quality decision boundary and further promote the aabel-invariant rate in GLA.
(the last column). which
4.3 In-depth Exploration
We further explore GLA in terms of negative pairs, augmentation space, and strategy.
Negative Pairs. The existing graph contrastive learning methods treat the augmented graphs from different source samples as negative pairs and employ the instance-level discrimination on these negative pairs. Since these methods separate the pre-train and fine-tuning phases, the negative pairs contain the augmented samples from different source samples but with the same category in the downstream tasks. Here we explore the effect of negative pairs on our GLA. Figure 4(a) shows the performance of our GLA with and without negative pairs on four datasets. We can see the performance with negative pairs significantly drops compared with our default setting without negative pairs, which behaves consistently on all four datasets. Different from the existing graph contrastive methods, our GLA integrates the pre-train and fine-tuning phases, where the negative pairs designed in self-supervised fashion are not beneficial to the downstream tasks. This finding is also in accord with the recent studies 77 [6] in the visual contrastive learning area.
Augmentation Space. Different from the most graph contrastive learning methods that directly augment raw graphs, our GLA conducts the augmentation in the representation space, as we believe the raw graphs can be mapped into the representation space, and this space much easier to augment than the original graph space. In Eq. (5), we design our representation augmentation with a random unit vector scaled by the magnitude of the perturbation 7). Figure 4 b,c.e.f) show the performance of our GLA with different values of 17 on four datasets, where we provide GraphCL and GraphCL+Label. Invariant as references. GraphCL+ LLabae--naaaiaan takes the augmented graph from GraphCL and filters the augmented samples that violate the label-invariant property by the downstream classifier. Comparing the two references, we can see that the label-invariant property benefits not only our GLA but also other contrastive methods in most cases. For our GLA, although the 1 values corresponding
8

---

91
79
Mociing racy (%)
Acturacy (95)
Acauray
87
75
75
GLA
GLA
Without Nagalive (GLA)
GraphCL
GraphCL
70
With Negative Pairs
GraphCL Labelinvarian!
GraphCL Label-inaarinn!
MUTAG PROTEINS
DD
NCIT
0.5
0.75
0.5
0.75
1,0
1.25
(a)
negative pairs
(b) 77 on MUTAG
(c) n on PROTEINS
Hardest (GLA)
Easiest
85
Anouraty (5)
Bee.negg (%)
Accurany (%)
80
78
78
75
GraphCLL
GraphCL
70
GraphICL Label-iwaient
GraphCL Label-ivarianl
MUTAG
DD
1.0
1.25
(d) augmentation strategy
(c) 17 on DD
(f) on NCII
Figure 4: In-depth exploration of GLA. (a) contrastive loss with/without negative pairs, (d) perfor- mance of different label-invariant augmentation strategies, (b.c.e.f) performance of magnitude of perturbation on different datasets under 50% label ratio.
the best performance vary on different datasets, the default setting with r) delivers satisfying performance in general, which outperforms GraphCL+Label Lnnraaaaaaa-- Invariant and indicates the superior of the representation augmentation over the raw graph augmentation. Augmentation Strategy. In the representation space, there might exist multiple qualified candidates that obey the label-invariant property. Our GLAA chooses the most difficult augmentation for the model. Here we demonstrate the performance of different augmentation strategies among qualified candidates, including the most difficult augmentation, random augmentation, and the easiest augmentation in Figure 4(d) where the random augmentation can be regarded GraphCL+Label Invarian....nt. We can see that the most difficult augmentation increases the model generalization and indeed brings in significant improvements over the other two ways. This also provides good support for our representation augmentation, where we can find the most difficult augmentation in the representation space, but is difficult to directly generate the raw graphs that challenging to the downstream classifier.
5 Conclusion
In this paper, we consider the graph contrastive learning problem. Different from the existing methods from the pre-train perspective, we propose a novel Graph Label-invariant Augmentation (GLA) algorithm which integrates the pre-train and fine-tuning phases to conduct the label-invariant augmentation in the representation space by perturbations. Specifically, GLA first checks whether the augmented representation obeys the label-invariant property and chooses the most difficult sample from the qualified samples. By this means, GLA achieves the contrastive augmentation without generating any raw graphs and also increases the model generalization. Extensive experiments in the semi-supervised setting on eight benchmark graph datasets demonstrate the effectiveness of our GLA. Moreover, we also provide extra experiments to verify our motivation and explore the in-depth factors of GLA in the effect of negative pairs, augmentation space, and strategy. Limitations, The performance of our method relies on the quality of the decision boundary indicated by the downstream classifier. Therefore, our method requires graph label information from downstream tasks to help with model training. Potential Negalive Societal Impacts. The problem addressed in this paper is well-defined and experiments are based on public datasets. As far as we can see, does not involve societal issues.

---

References
[1] K. M. Borgwardt, C. Ong, S. Schönauer, S. Vishwanathan, A. Smola, and H.-P. Kriegel. Protein function prediction via graph kernels. Bioinformatics, 21(suppl 2005.
[2] M. Chen, K. Weinberger, F. Sha, and Y. Bengio. Marginalized denoising auto-encoders for nonlinear representations. In International Conference on Machine Learning. 2014.
[3] M. Chen, Z. Xu, K. Weinberger, and F. Sha. Marginalized denoising autoencoders for domain adaptation. In International Conference on Machine Learning, 2012.
[4] Т. Chen, S. Bian, and Y. Sun. Are powerful graph neural nets necessary? a dissection on graph classification. arXiv preprint 1905. 2019.
[5] T. Chen, S. Kornllihh, M. Norouzi, and G. Hinton. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning, 2020.
[6] X. Chen, H. Fan, R. Girshick, and K. He. Improved baselines with momentum contrastive learning. arXiv preprint 2xXiv:2003.04297,, 2020.
[7] X. Chen and K. He. Exploring simple siamese representation learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021.
[8] C.-Y. Chuang, J. Robinson, Y.-.. Lin, A. Torralba, and S. Jegelka. Debiased contrastive learning. Advances in Neural Information Processing Systems, 33:8765-877 8775, 2020.
(9] 7. Cui, K. Henrickson, R. Ke, and Y. Wang. Traffic graph convolutional recurrent neural network: A deep learning framework for network-scale traffic learning and forecasting. IEEE Transactions on Intelligent Transportation Systems, 21 (1):48833-4894 2019.
[10] A. K. Debnath, Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. correlation with molecular orbital energies and hydrophobicity. Journal of Medicinal Chemistry, 34(2):786-797, 1991.
Lopez de Compadre, G. Debnath, A. Shusterman, and C. Hansch.
[11] K. Ding, Z. Xu, H. Tong, and H. Liu. Data augmentation for deep graph learning: A survey. arXiv preprint arXiv:2202.088: 200.00833,, 2022.
[12] P. D. Dobson and A. J. Doig. Distinguishing enzyme structures from non-enzymes without alignments. Journal of Molecular Biology, 330(4))771-783, 2003.
[13] Y. Fan, M. Ju, C. Zhang, and Y. Ye. Heterogeneous temporal graph neural network. In SIAM International Conference Data Mining, 2022.
[14] H. Gao, Z. Wang, and S. Ji. Large-scale learnable graph convolutional networks. In ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018
[15] C. Gong, T. Ren, M. Ye, and Q. Liu. Maxup: Lightweight adversarial training with data augmentation improves neural network training. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2474-2482 2021.
[16] J.-B. Grill, F. Strub, F. Altché, C. Tallec, Richemond, E. Buchatskaya, C. Doersch, B. Avila Pires, Z. Guo, M. Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. Advances Neural Information Processing Systems, 33:21271-21284, 2020.
[17] E. Hajiramezanali, A. Hasanzadeh, K. Narayanan, N. Duffield, M. Zhou, and X. Qian. Varia- tional graph recurrent neural networks. Advances in Neural Information Processing Systems, 32, 2019.
[18] K. Hassani and A. H. Khasahmadi. Contrastive multi-view representation learning on graphs. In International Conference on Machine Learning, 2020.
[19] P. Khosla, P. Teterwak, C. Wang, A. Sama, Y. Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan. Supervised contrastive learning. Advances in Neural Information Processing Systems, 111118661-18677 2020.
10

---

[20] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[21] T. N. Kipf and M. Welling, Semi-supervised classification with graph convolutional networks arXiv preprini arXiv: A.Xi::600.02907, 2016.
[22] T. N. Kipf and M. Welling. Variational graph auto-encoders, arXiv preprint arXiv:1611.07308, 2016.
[23] T. N. Kipf and M. Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017.
[24] G. Li, M. Müller, B. Ghanem, and V. Koltun. Training graph neural networks with 1000 layers. In International Conference on Machine Learning, 2021.
[25] G. Li, M. Müller, A. Thabet, and B. Ghanem. Deepgens: Can gens go as deep as cnns? In The IEEE International Conference on Computer Vision, 2019.
[26] Y. Li, Hu, Z. Liu, D. Peng. J. T. Zhou, and X. Peng. Contrastive clustering. In AAAI Conference Artificial Intelligence, 2021.
[27] C. Morris N. M. Kriege, F. Bause, K. Kersting, P. Mutzel, and M. Neumann. Tudataset: A collection of benchmark datasets for learning with graphs. In ICML 2020 Workshop on Graph Representation Learning and Beyond, 2020.
[28] T. Park, A.A. Efros, R. Zhang, and J.-Y. Zhu. Contrastive learning for unpaired image-to-image translation. In European Conference on Computer Vision, 2020.
[29] B. Rozemberczki, O. Kiss, and R. Sarkar. Karate Club: An API Oriented Open-source Python Framework for Unsupervised Learning on Graphs. In ACM International Conference on Information and Knowledge Management, 2020.
[30] P. Velickovic, W. Fedus, W. L. Hamilton, P. Liò, Y. Bengio, and R. D. Hjelm. Deep graph infomax. International Conference on Learning Representations, 2(3):4, 2019.
[31] Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio. Graph attention networks. In International Conference Learning Representations, 2018.
[32] N. Wale, I. A. Watson, and G. Karypis. Comparison of descriptor spaces for chemical compound retrieval and classification. Knowledge and Information Systems, 14(3):347-375, 2008.
[33] X. Wang, X. He, Y. Cao, M. Liu, and T.-S. Chua. Kgat: Knowledge graph attention network for recommendation. In ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019.
[34] X. Wang, H. Ji, C. Shi, B. Wang, Y. Ye, P. Cui, and P. S. Yu. Heterogeneous graph attention network. In The World Wide Web Conference, 2019.
[35] Wu, A. Souza, T. Zhang, C. Fifty, T. Yu, and K. Weinberger. Simplifying graph convolutional networks. In International Conference Machine Learning, 2019.
[36] Xia, L. Wu, J. Chen, B. Hu, and S. Z. Li. Simgrace: A simple framework for graph contrastive learning without data augmentation. In ACM Web Conference, 2022.
[37] D. Xu, W. Cheng, D. Luo, H. Chen, and X. Zhang. Infogel: Information-awwrre graph contrastive learning. 34:30414-30425, 2021.
[38] Yanardag and S. Vishwanathan. Deep graph kernels. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2015.
[39] Y. You, T. Chen, Y. Shen, and Z. Wang. Graph contrastive learning automated. In International Conference on Machine Learning, 2021.
[40] Y. You, T. Chen, Sui, Chen, Z. Wang, and Y. Shen. Graph contrastive learning with augmentations. Advances in Neural Information Processing Systems, 33:5812-5823, 2020.
11

---

[41] C. Zhang, D. Song, C. Huang, A. Swami, and N. V. Chawla. Heterogeneous graph neural network. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2019.
[42] Y. Zhu, Y. Xu, F. Yu, Q. Liu, S. Wu, and L. Wang. Deep Graph Contrastive Representation Learning. In ICML Workshop on Craph Representation Learning and Beyond, 2020.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section 5
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See
Section
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [Yes] See footnote
URL in Section 4.1
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section 4.1
(c) Did you report error bars (c.g., with respect to the random seed after running experi- ments multiple times)? [Yes] We reported standard deviations in Tabel P
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [No] We did not include memory time consumption comparisons.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] See Section 4.1
(b) Did you mention the license of the assets? [No] All datasets are collected and made public by creators. We cannot find the license information for these datasets. All datasets are cited in Section 4.
(c) Did you include any new assets either in the supplemental material or as URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you're 're using/curating? [No] We used public datasets, and detailed information regarding content could be found in the corresponding citations in Section 4.1
(c) Did you discuss whether the data you using/curating contains personally identifiable information or offensive content? [No] We used public datasets, and detailed informa- tion regarding content could be found in the corresponding citations in Section 4.1
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
12


Reviews:
[{"review_id": "v3ds2udH5U0", "reviewer": "Reviewer_1vzF", "summary": "This paper proposes a label-invariant augmentation technique for graph-structured data. Different from the conventional node/edge modification and subgraph extraction, the authors conduct the augmentation in the representation space and generate the augmented samples in the most difficult direction while keeping the label of augmented data the same as the original samples. The experimental results demonstrate the effectiveness of the proposed augmentation strategy.", "questions": "1. In fact, many existing graph contrastive learning methods are also applicable to node classification on one graph. Can the proposed method be generalized to such problem?\n2. The authors claim that they developed label-invariant augmentation strategy for graph contrastive learning which involves labels in the downstream task. I want to know the difference/relationship of this problem setting with supervised contrastive learning. Note that the label information can also be employed in supervised contrastive learning which does not involve the downstream classification task.\n3. To present the motivation in Section 3.1, the authors use Fig. 1 with label rate being 50% to show that some combinations of existing augmentation strategies may even hurt the performance. I'm a bit concerned with the label rate 50%, as this is too high for semi-supervised learning. Maybe, if the label rate is low (e.g., 5%-10%), the advantage of some augmentations will become prominent.\n4. For the part of \"Label-invariant Augmentation\" on Page 5, I do not think d defined in Eq. 4 represents direction. Note that this is simply the distance between the representation of a graph to the centroid (like ||a-b||^2), rather than direction vector like a-b. The \\delta is simply a random vector, which is also not a direction vector. Therefore, I cannot fully understand the authors' claim that they \"augment samples in most difficult direction while keeping the label of augmented data the same as the original samples.\" Moreover, how to keep the label of augmented data the same as the original samples?? I also did not find specific explanations on this.\n5. In experiments, the authors use 30%, 50% and 70% label rates, which I also think is too high. Note that the definition of semi-supervised learning is that we only have a handful of labeled data in addition to a large amount of unlabeled data.", "limitations": "1. The model has some mathematical errors.\n2. The label rates adopted in this paper are too high for semi-supervised learning.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n1. I think the high-level idea is interesting and reasonable.\n2. The paper is generally written in a clear way.\n3. The empirical studies are quite convincing.\n\n\nCons:\n1. It would be better if some theoretical evidence can be provided to justify the correctness of the proposed method.\n2. Some details of model need further explanations.\n3. It seems that the proposed method has some technical issues.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "VXiKe1KJESn", "reviewer": "Reviewer_3fFh", "summary": "The authors propose a graph contrastive learning approach in which the augmentations are done in feature space such that they do not change the predicted label. They present experiments on  8 different benchmarks, and show competitive results.", "questions": "I would like to see the questions in my first \"weakness\" comment adressed and explained", "limitations": "N/A", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strength**:\nthe idea is clever and well-motivated\nthe experiments are extensive and rather convincing\nthe article is clear and well written, even if some details are sometimes missing and could be more rigorous\n\n**Weaknesses**:\n- Some parts of the paper are unclear:\n    - Since picking positive examples requires the labels (to ensure label invariance and \"difficulty\"), how are the unlabelled graph handled?\n    - it is unclear how \" closest to the decision boundary of the classifier\" exactly translates here\n- The authors keep referring to their \"hard positive mining\" strategy as \"finding the hardest example\" or \"the hardest direction\", but they simply pick a hard example within a sampled set, which is very standard and do not need to be presented as an approximate bilevel optimization scheme.\n\n**Overall**:\nAn interesting and well-executed idea. The article is sometimes not as clear as it could be, hiding actual crucial information. I would be inclined to increase my rating if the authors could answer my doubts convincingly and update the paper.\n\n**Post rebuttal**\nRebuttal read and acknowledged, score updated ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "P5w2x03XpT0", "reviewer": "Reviewer_e8z3", "summary": "This paper presents a label-invariant augmentation for graph-structured data. It conducts the augmentation in the representation space and augments the most difficult sample while keeping the label of the augmented sample the same as the original sample. Experimental results on eight benchmark graph-structured data show that the developed metho outperforms classical GNN-based methods and recent graph contrastive learning. ", "questions": "a. The model sizes and inference time of different methods in Table 2 are not discussed.\nb. In Figure 2, it is unclear whether the number of augmented representations affect the final result.\n", "limitations": "The authors have discussed the limitation and potential negative societal impact.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\na. This work presents a label-invariant augmentation strategy for graph contrastive learning by involving labels in the downstream task to guide the contrastive augmentation. \nb. The authors present a lightweight technique to randomly generate a set of qualified candidates and selecting the most difficult one for increasing the model generalization.\nc. Experimental results on eight graph benchmark datasets have verified the effectiveness of the developed label-invariant augmentation.\nWeaknesses:\na. It is unclear why selecting the hardest augmented sample can enhance the performance. Why selecting top-K hardest augmented samples?\nb. Failure cases of the develop method are not discussed.\nc. As shown in Table 2, the results of the developed method are not always best among different datasets. Please explain such results.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept