SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\nabla_ {\mathbf {H}} \tilde {\mathcal {L}} _ {\text {u n i f o r m}} = 2 / c \mathbf {D A H}, \tag {22}
$$
where  $\mathbf{D}$  is the diagonal matrix containing  $\mathcal{P}(x) = \sum_{x'}\mathbf{A}_{x,x'},\forall x\in \mathcal{V}$ .
Therefore, the gradient descent update of the defined uniformity loss gives
$$
\mathbf {H} _ {\text {n e w}} = \mathbf {H} - \alpha \nabla_ {\mathbf {H}} \tilde {\mathcal {L}} _ {\text {u n i f o r m}} = \mathbf {H} - 2 / c \mathbf {D A H}, \tag {23}
$$
where  $\alpha$  is the step size. It is easy to see its equivalence to the ContraNorm update.
F.4 Derivation of Theorem 5.2
Proof. Combining Theorem 4.1 and Theorem 5.1, we can directly obtain Theorem 5.2 as a corollary.  $\square$
G Discussion on More GCL Methods
The contrastive mode has three mainstreams: local-to-local (L-L), global-to-global (G-G), and global-to-local (G-L) [78]. For the local-to-local perspective, the corresponding nodes in the two augmented views of a graph are seen as positive pairs while all the other node pairs are negative ones. Global-to-global mode is often used when there are multiple graphs, and contrastive objects are the global representations of augmented views. In this mode, augmented views of the same graph are positives and all the other graph pairs are negatives. For the global-to-local perspective, positive pairs are taken as the global representation and nodes of augmented views for the corresponding graph, and negative pairs are the global representation and nodes of augmented views for other graphs.
In previous sections, we investigate the GCL methods with L-L or G-G modes, and the G-L mode on the graph classification (like InfoGraph). In this section, we discuss two methods of the G-L mode on node classification task: DGI [51] and MVGRL [14]. For experiments, we use the same settings as in Section 4. As seen from Table 12, there is an obvious degeneration in accuracy when no positive samples or negative samples are used, which is close to the no training setting. Recall that we find the positive samples are not needed in Section 4, and the observations on DGI and MVGRL seem to contradict our arguments. Here we attribute the inconsistency to the flaw in the methods themselves.
We start with an intriguing finding on DGI. Here we disorder the contrastive correspondence with a wrong view as global representations. Specifically, we take the local representation of the graph and its global representation as negatives, while local representations and global representations of the
corrupted view are seen as positives. Note that the corruption operation in DGI is used to generate negative samples by shuffling rows of node attributes. See Figure 5 for illustration. We compare the disordered version with the original DGI in Table 13, and find using a wrong view as global representations does not affect performance. It implies that global representations lose efficacy in this framework. Inspired by Zheng et al. [76], we compare the two global representations and find they are nearly identical with every dimension being about 0.5. Extensive experiments also show the global representation is a constant vector for inappropriate usage of the Sigmoid function in both DGI and MVGRL [76].
This finding explains why the loss without positive samples does not work. Trained with such loss, node representations are only enforced to be far away from a constant vector, which gives no semantic guarantee. However, after adding positive samples to loss, the model learns to pull positive samples near a constant vector, while pushing negative samples away from such vector. It intrinsically achieves the goal of contrastive learning by gathering positives and repulsing negatives simultaneously. Thus the model trained with both positive and negative samples can obtain satisfying performance, explaining why DGI works with constant global representations.
Table 12: Test accuracy (\%) of node classification benchmarks using DGI and MVGRL methods. We compare the performances of models trained with JSD loss (Contrast), loss part only involving negative pairs (NO Pos), loss only involving positive pairs (NO Neg), and no optimization objective (NO Training). Mean accuracy with standard derivation is reported after 10 runs. We conduct significance testing using Wilcoxon Signed Rank Test [57], comparing the contrastive loss with other loss types. The p-value is averaged across datasets. A value below 0.05 denotes significant accuracy difference (red), while a value above 0.05 indicates insignificance (green).
Method	Loss	Cora	CiteSeer	PubMed	Photo	Computers	Chameleon	Squirrel	Avg	Avg p-value
DGI	Contrast	83.38 ± 2.67	72.07 ± 2.37	84.77 ± 0.71	88.10 ± 1.81	83.35 ± 0.71	39.56 ± 2.86	34.55 ± 0.88	69.40	-
NO Training	69.78 ± 3.39	55.15 ± 2.09	79.56 ± 1.35	69.08 ± 3.30	56.03 ± 1.97	31.44 ± 1.70	24.57 ± 1.22	55.09	0.0020
NO Pos	66.84 ± 3.54	54.79 ± 3.33	78.25 ± 0.99	58.34 ± 3.92	71.98 ± 1.38	35.81 ± 2.34	26.99 ± 0.20	56.14	0.0020
NO Neg	67.35 ± 4.61	58.17 ± 2.57	77.23 ± 1.05	62.75 ± 3.75	72.66 ± 1.48	31.62 ± 4.06	27.75 ± 1.85	56.79	0.0022
MVGRL	Contrast	84.41 ± 1.44	75.27 ± 0.79	85.62 ± 0.63	89.23 ± 1.52	79.58 ± 0.15	42.45 ± 2.43	33.97 ± 2.54	70.08	-
NO Training	77.94 ± 2.23	58.92 ± 2.88	82.13 ± 0.63	81.15 ± 3.25	69.07 ± 0.40	32.23 ± 1.94	24.41 ± 1.10	60.84	0.0022
NO Pos	75.44 ± 1.42	61.08 ± 2.48	81.26 ± 1.30	36.03 ± 1.57	38.36 ± 0.55	36.86 ± 2.56	29.98 ± 1.52	51.29	0.0020
NO Neg	54.93 ± 4.67	35.03 ± 5.20	56.26 ± 1.91	36.47 ± 2.37	38.36 ± 0.56	29.34 ± 2.04	28.06 ± 1.66	39.78	0.0020
Standard Contrastive Correspondence
Disordered Contrastive Correspondence
Figure 5: Illustration for disordering contrastive correspondence of views on DGI.
Table 13: Test accuracy (%) of DGI in standard contrastive correspondence (Std) and disordered correspondence (Dis).
Method	Contrast	Cora	CiteSeer	PubMed
DGI	Std.	83.38 ± 2.68	72.07 ± 2.37	84.77 ± 0.71
Dis.	83.35 ± 2.68	72.04 ± 2.17	84.70 ± 0.68
H Extensive Experiments of ContraNorm.
ContraNorm in different GCL methods. In Table 5, we show that by simply incorporating the normalization layer into the encoder, the collapse issue can be rooted out for the GRACE method. In this section, we incorporate ContraNorm into multiple GCL methods under the no-negative setting.
The results are shown in Table 14. It is obvious that for these GCL methods, applying ContraNorm when there are no negative samples achieves comparable performance with models trained with the contrastive loss (both positive and negative samples). The extensive experiments validate the effectiveness of ContraNorm across different GCL methods.
Combining ContraNorm with MLP. In Table 15, we show that ContraNorm could also boost MLP performance significantly  $(59.49\% \rightarrow 73.19\%)$  under the "No Neg" loss, and attain similar performance to MLP trained with contrastive loss  $(77.07\%)$ , which also aligns well with our theory and empirical observations on GCN.
ContraNorm under SimCLR backbone. In Table 16, we conduct experiments replacing uniformity loss with ContraNorm under the SimLCR backbone. It is shown that ContraNorm can not replace the uniformity loss in SimCLR. We conjecture it is a graph-specific technique and leave the analysis for the future work.
Table 14: Node classification accuracy (%) with GCN / GCN+ContraNorm using GCL methods. Mean accuracy with standard derivation is reported after 10 runs. Average accuracy across datasets is reported. We conduct significance testing using Wilcoxon Signed Rank Test [57], comparing the default setting (first line) with others. The p-value is averaged across datasets. A value below 0.05 denotes significant accuracy difference (red), while a value above 0.05 indicates insignificance (green). OOM denotes out of memory.
Method	Loss	Encoder	Cora	CiteSeer	PubMed	Photo	Computers	Avg	Avg p-value
GRACE	Contrast	GCN	84.67 ± 1.39	73.47 ± 2.32	85.80 ± 0.16	91.42 ± 1.27	89.01 ± 0.60	84.87	-
NO Neg	GCN	29.85 ± 1.45	20.42 ± 2.26	39.63 ± 0.81	25.10 ± 1.74	36.84 ± 1.30	30.37	0.0020
NO Neg	GCN + CN	82.35 ± 2.28	72.25 ± 1.86	83.30 ± 0.63	92.43 ± 0.82	84.48 ± 1.01	82.96	0.1520
GCA	Contrast	GCN	84.04 ± 1.55	72.63 ± 2.68	85.92 ± 0.69	93.07 ± 0.66	86.58 ± 0.75	84.45	-
NO Neg	GCN	31.40 ± 3.61	22.16 ± 3.01	39.58 ± 0.83	28.13 ± 1.14	37.34 ± 0.95	31.72	0.0020
NO Neg	GCN + CN	82.21 ± 1.29	72.87 ± 0.98	82.40 ± 0.78	92.47 ± 0.96	86.15 ± 0.58	83.22	0.2125
ProGCL	Contrast	GCN	85.42 ± 3.41	72.85 ± 2.99	OOM	93.81 ± 0.48	86.35 ± 1.28	84.61	-
NO Neg	GCN	30.15 ± 2.70	21.08 ± 1.45	21.13 ± 1.20	4.88 ± 0.33	3.11 ± 0.65	16.07	0.0020
NO Neg	GCN + CN	80.00 ± 1.75	73.35 ± 1.17	84.02 ± 0.91	93.59 ± 0.38	85.67 ± 0.43	83.33	0.2336
Table 15: Node classification accuracy (%) with MLP / MLP+ContraNorm using GRACE. Mean accuracy with standard derivation is reported after 10 runs. Average accuracy across datasets is reported. We conduct significance testing using Wilcoxon Signed Rank Test [57], comparing the default setting (first line) with others. The p-value is averaged across datasets. A value below 0.05 denotes significant accuracy difference (red), while a value above 0.05 indicates insignificance (green). OOM denotes out of memory.
Loss	Encoder	Cora	CiteSeer	PubMed	Photo	Computers	Avg.	Avg p-value.
Contrast	MLP	67.72 ± 0.88	65.51 ± 2.63	83.29 ± 0.49	87.92 ± 0.59	80.89 ± 1.21	77.07	-
NO Training	MLP	40.66 ± 2.49	42.81 ± 4.82	78.53 ± 0.90	62.12 ± 0.97	57.97 ± 1.13	56.42	0.0020
NO Neg	MLP	51.69 ± 3.04	50.36 ± 1.14	79.00 ± 0.63	61.33 ± 0.75	55.07 ± 0.99	59.49	0.0020
NO Neg	MLP + CN	62.87 ± 0.84	62.16 ± 3.11	81.51 ± 0.60	83.03 ± 1.59	76.40 ± 1.09	73.19	0.0660
Table 16: Image classification accuracy (%) under SimCLR backbone on CIFAR10.
Loss & Encoder	SimCLR & ResNet	Uniform & ResNet	Align & ResNet	Align & ResNet + CN
Test Acc (%)	82.4	20.3	18.6	20.9
I Extensive experiments for Gaussian Augmentations
Gaussian Augmentations under Different Loss Settings. In Section 6, we perform experiments using the GRACE method with different augmentations under the InfoNCE loss. Here, we further
report results under different losses in Table 17. For loss without negative samples, the average performance gap between domain-specific augmentations and noise augmentations is only  $0.74\%$ . When no augmentations, the performance drops  $4.88\%$ . We conjecture that when no negative samples exist, the application of augmentations brings diversity in representations, thus making collapse more difficult. For contrastive loss and loss without positive samples, the gap between domain-specific augmentations and noise augmentations is also narrow.
Gaussian Augmentations Using MLP As the Encoder. We compare augmentations with MLP backbone. In Table 18, while GCN exhibits similar performance with different augmentations, FM+EP notably surpasses Gaussian noise for MLP. This also correlates with GCN's implicit alignment mechanism (Theorem 4.1). Additional augmentations minimally affect GCN due to the existing alignment mechanism. However, for MLP without this implicit bias, graph-specific augmentations like FM+EP remain informative for learning proper graph invariance.
Table 17: Test accuracy (\%) of node classification benchmarks using GRACE method with different augmentations under three loss settings. We compare no augmentations (NO Aug), domain-agnostic augments (Gaussian), and default domain-specific augments (FM+EP). Average accuracy and p-value are reported. We conduct significance testing using Wilcoxon Signed Rank Test [57], comparing the default augmentation with other settings. The p-value is averaged across datasets. A value below 0.05 denotes significant accuracy difference (red), while a value above 0.05 indicates insignificance (green).
Loss	Encoder	Aug	Cora	CiteSeer	PubMed	Photo	Computers	Avg	Avg p-value
Contrast	GCN	FM+EP	84.67 ± 1.39	73.47 ± 2.32	85.80 ± 0.16	91.42 ± 1.27	89.01 ± 0.60	84.87	-
Gaussian	82.72 ± 2.38	72.60 ± 1.21	85.24 ± 0.61	91.32 ± 1.37	82.77 ± 1.09	82.93	0.1816
NO Aug	79.56 ± 2.18	71.83 ± 1.83	84.68 ± 0.58	90.99 ± 1.26	82.83 ± 0.86	81.98	0.1008
NO Pos	GCN	FM+EP	82.65 ± 1.18	73.50 ± 2.41	85.28 ± 0.79	91.32 ± 0.10	84.40 ± 0.43	83.43	-
Gaussian	80.04 ± 1.93	70.84 ± 1.85	84.88 ± 0.89	91.33 ± 1.18	83.26 ± 1.24	82.07	0.1840
NO Aug	79.37 ± 2.30	71.80 ± 1.84	84.69 ± 0.63	90.92 ± 1.21	82.49 ± 0.87	81.85	0.1176
NO Neg	GCN + CN	FM+EP	82.35 ± 2.28	72.25 ± 1.86	83.30 ± 0.63	92.43 ± 0.82	84.48 ± 1.01	82.96	-
Gaussian	79.08 ± 2.47	72.43 ± 1.32	83.55 ± 0.22	91.59 ± 1.19	84.48 ± 1.07	82.23	0.2750
NO Aug	75.59 ± 3.45	66.98 ± 3.40	82.14 ± 1.28	81.91 ± 1.42	83.79 ± 1.14	78.08	0.0688
Table 18: Node classification accuracy (%) under GRACE backbone with MLP using different augmentations.
Augmentation	Encoder	Cora	CiteSeer	PubMed	Photo	Computers	Avg.
NO Aug	MLP	58.09 ± 2.96	62.69 ± 1.21	80.62 ± 1.01	84.42 ± 0.64	73.30 ± 1.10	71.82
FM + EP	MLP	67.72 ± 0.88	65.51 ± 2.63	83.29 ± 0.49	87.92 ± 0.59	80.89 ± 1.21	77.07
Gaussian noise	MLP	61.47 ± 3.36	63.23 ± 2.41	81.90 ± 0.57	84.03 ± 1.07	75.47 ± 0.70	73.22
Table 19: Node classification accuracy (%) under GRACE backbone with no-training.
Loss	Encoder	Cora	CiteSeer	PubMed	Photo	Computers	Avg.
Contrast	GCN	84.67 ± 1.39	73.47 ± 2.32	85.80 ± 0.16	91.42 ± 1.27	89.01 ± 0.60	84.87
NO Training	GCN	69.12 ± 4.18	60.60 ± 2.59	80.65 ± 0.80	68.37 ± 3.76	57.02 ± 1.93	67.15
NO Training	GCN+CN	69.63 ± 4.08	59.82 ± 2.73	81.73 ± 0.97	91.37 ± 0.60	85.64 ± 0.70	77.64
J Can GCL trained without both positive and negative pairs?
In Section 4, and in Section 5. A natural question arises: can GRACE with GCN and ContraNorm be trained without positive AND negative pairs? Removing both positive and negative samples renders the InfoNCE loss empty, actually corresponding to the "No Training" baseline which typically performs much worse. Results in Table 19 further show the performance remains poor even when adding
ContraNorm to GCN. This is because despite the implicit alignment and uniformity mechanisms, without any training objective, the model parameters in  $\mathrm{GCN + CN}$  cannot be properly trained to fit the dataset.
K Results of the Fine-tuning Protocol
In this section, we provide the fine-tuning protocol results of main experiments of our paper. Specifically, we add a linear classification head after the encoder. In the fine-tuning phase, we fine-tune the whole networks according to downstream tasks, with the learning rate selected from [0.01, 0.001] and the number of epochs selected from [100, 200, 500]. In Table 20(a) and Table 20(b), we report the fine-tuning results for the node classification task with GRACE and DGI methods, and for the graph classification tasks with GraphCL method, respectively. Sharing the same conclusion as the linear probing protocol, only using negative samples achieves comparable performance as that using contrastive objectives. On the other hand, for the node classification task, only using positive samples escapes severe collapse. We think the guidance of true labels in the fine-tuning helps the networks relearn parameters and thus prevents collapse.
Furthermore, we report the fine-tuning results about augmentations in Table 20(c). For the default augmentations (FM+PE), we set the ratio of each augmentation to 0.2 to save engineering effort. For a fair comparison, the standard deviation  $\sigma$  of the random Gaussian noise is fixed to 1e-4. Other hyperparameters are the same across the three augmentation settings (FM+PE, Gaussian, and NO Aug). As seen from the table, in the fine-tuning evaluation setting, random noise augmentation is on average the best for each loss type. It further justifies our analysis that domain-agnostic augmentations are enough for GCL.
Table 20: Fine-tuning accuracy (%) using GCL methods. Mean accuracy with standard derivation is reported after 10 runs. Average accuracy across datasets is reported. We conduct significance testing using Wilcoxon Signed Rank Test [57], comparing the contrastive loss and other loss types. The p-value is averaged across datasets. A value below 0.05 denotes significant accuracy difference (red), while a value above 0.05 denotes insignificance (green).
(a) Fine-tuning accuracy  $(\%)$  of node classification benchmarks using GCL methods.
Method	Loss	Cora	CiteSeer	PubMed	Photo	Computers	Chameleon	Squirrel	Avg	Avg p-value
GRACE	Contrast	85.15 ± 3.07	74.19 ± 3.66	84.64 ± 2.47	92.89 ± 0.56	88.92 ± 1.27	39.56 ± 3.76	33.13 ± 4.33	71.21	-
NO Pos	84.49 ± 3.50	74.07 ± 3.92	82.38 ± 2.36	92.84 ± 0.51	89.45 ± 1.14	38.60 ± 3.99	31.40 ± 3.76	70.46	0.3139
NO Neg	81.62 ± 4.05	69.52 ± 4.46	83.87 ± 2.65	92.05 ± 0.89	89.07 ± 1.03	35.98 ± 5.13	30.48 ± 2.54	68.94	0.1398
DGI	Contrast	85.66 ± 2.39	74.55 ± 1.68	85.69 ± 0.26	92.94 ± 0.88	90.03 ± 0.79	42.01 ± 6.07	32.74 ± 5.49	71.95	-
NO Pos	86.91 ± 2.16	74.79 ± 0.92	85.49 ± 0.25	92.73 ± 0.69	89.45 ± 0.65	42.01 ± 6.05	31.98 ± 5.73	71.91	0.4395
NO Neg	85.00 ± 2.39	74.97 ± 1.08	85.44 ± 0.38	92.73 ± 0.67	90.13 ± 0.66	42.97 ± 6.52	32.13 ± 5.24	71.91	0.3672
(b) Fine-tuning accuracy  $(\%)$  of graph classification benchmarks using GCL methods.
Method	Loss	MUTAG	PTC-MR	PROTEINS	IMDB-BINARY	IMDB-MULTI	REDDIT-BINARY	Avg	Avg p-value
GraphCL	Contrast	93.48 ± 2.52	80.64 ± 4.09	79.88 ± 0.43	64.53 ± 1.32	43.64 ± 0.63	79.67 ± 1.82	73.64	-
NO Pos	93.12 ± 0.74	80.45 ± 4.85	79.34 ± 3.10	63.13 ± 1.55	42.24 ± 0.31	76.73 ± 4.23	72.50	0.1966
NO Neg	93.11 ± 1.14	80.36 ± 3.64	79.01 ± 3.69	62.37 ± 2.81	41.60 ± 0.47	76.75 ± 2.90	72.20	0.1400
(c) Fine-tuning accuracy  $(\%)$  of node classification benchmarks using GRACE method with different augmentations under three loss settings.
Loss	Aug	Cora	CiteSeer	PubMed	Photo	Computers	Chameleon	Squirrel	Avg	Avg p-value
Contrast	FM+EP	85.15 ± 3.07	74.19 ± 3.66	84.64 ± 2.47	92.89 ± 0.56	88.92 ± 1.27	39.56 ± 3.76	33.13 ± 4.33	71.21	-
Gaussian	86.69 ± 2.39	74.91 ± 2.98	84.52 ± 2.05	92.94 ± 1.02	88.94 ± 1.14	42.62 ± 6.55	31.55 ± 4.64	71.74	0.2829
NO Aug	85.00 ± 3.20	74.07 ± 3.92	82.64 ± 2.69	93.10 ± 0.42	89.58 ± 1.20	37.03 ± 4.28	31.59 ± 4.49	70.43	0.2531
NO Pos	FM+EP	84.49 ± 3.50	74.07 ± 3.92	82.38 ± 2.36	92.84 ± 0.51	89.45 ± 1.14	38.60 ± 3.99	31.40 ± 3.76	70.46	-
Gaussian	86.40 ± 2.84	74.67 ± 3.90	83.95 ± 1.72	92.73 ± 1.52	88.72 ± 1.18	40.79 ± 6.03	30.17 ± 4.73	71.06	0.2609
NO Aug	85.00 ± 3.20	74.07 ± 3.92	82.42 ± 2.57	92.97 ± 0.58	89.49 ± 1.10	38.43 ± 3.88	31.59 ± 4.48	70.57	0.3859
NO Neg	FM+EP	81.62 ± 4.05	69.52 ± 4.46	83.87 ± 2.65	92.05 ± 0.89	89.07 ± 1.03	35.98 ± 5.13	30.48 ± 2.54	68.94	-
Gaussian	84.26 ± 2.80	72.46 ± 4.75	84.49 ± 1.97	91.56 ± 1.75	88.37 ± 1.73	38.25 ± 2.54	28.25 ± 2.81	69.66	0.3273
NO Aug	80.96 ± 5.24	71.38 ± 5.59	82.45 ± 2.69	92.03 ± 2.12	86.16 ± 5.95	33.97 ± 4.41	26.76 ± 2.49	67.67	0.2854
L Extended Evaluation for GCL and VCL Under the Same Splitting
Across our experiments, we follow the standard setting and data splits in each domain. To further resolve concerns on the consistency of evaluation setups, we unify the evaluation settings for GCL and VCL: we all train a linear classifier with an Adam optimizer and randomly split the dataset with the same train-test ratio as 9:1. The results are shown in the following table.
As can be seen in Table 21, the observations are consistent with the findings in our paper. GCL and VCL present apparent differences in the properties of no-positives, no-negatives and random Gaussian noise augmentations. The supervised information ratio shows little effect on the conclusions. For the node classification task, evaluations are under data splits of 1:9 in the paper and 9:1 here, but the findings are kept unchanged.
Table 21: Evaluation results of GCL and VCL under the same data split setting. Average accuracy and p-value are reported. We conduct significance testing using Wilcoxon Signed Rank Test [57], comparing the default augmentation with other settings. The p-value is averaged across datasets. A value below 0.05 denotes a significant accuracy difference (red), while a value above 0.05 indicates insignificance (green).
(a) Image classification test accuracy  $(\%)$  with SimCLR backbone on CIFAR10
Loss	Augmentation	Avg.
NO Training	Default Aug	22.5
InfoNCE	Default Aug	82.4
Alignment	Default Aug	18.6
Uniformity	Default Aug	20.3
InfoNCE	Gaussian	38.6
(b) Node classification test accuracy  $(\%)$  with GRACE as backbone
Loss	Encoder	Cora	CiteSeer	PubMed	Photo	Computers	Avg.	Avg p-value.
Contrast	GCN	86.27 ± 1.31	75.14 ± 2.11	86.02 ± 0.87	92.10 ± 0.97	83.01 ± 0.66	84.51	-
NO Training	GCN	76.31 ± 2.69	68.23 ± 2.31	83.22 ± 0.90	69.20 ± 1.06	56.95 ± 1.36	70.78	0.0020
NO Pos	GCN	86.35 ± 1.68	74.05 ± 1.34	85.50 ± 0.62	91.42 ± 0.98	83.33 ± 0.99	84.13	0.3273
NO Neg	GCN	29.74 ± 1.92	21.62 ± 1.89	40.30 ± 0.41	26.12 ± 1.88	38.10 ± 1.35	31.18	0.0020
NO Neg	GCN + CN	88.49 ± 1.94	77.12 ± 1.18	85.16 ± 0.84	94.67 ± 0.70	85.61 ± 1.34	86.21	0.1324
(c) Graph classification test accuracy  $(\%)$  with GraphCL as backbone
Loss	MUTAG	PTC_MR	PROTEINS	IMDB-B	IMDB-M	REDDIT-B	Avg.	Avg p-value.
Contrast	91.58 ± 2.58	71.43 ± 6.26	78.21 ± 3.73	76.40 ± 1.50	52.93 ± 4.12	88.70 ± 1.03	76.54	-
NO Training	86.32 ± 5.37	65.71 ± 3.13	72.14 ± 4.50	70.80 ± 2.14	44.13 ± 1.65	75.30 ± 2.01	69.07	0.0104
NO Pos	92.63 ± 6.32	73.71 ± 3.33	77.32 ± 4.02	76.60 ± 1.85	52.00 ± 3.86	86.90 ± 1.39	76.53	0.6086
NO Neg	91.58 ± 8.55	69.71 ± 8.40	78.75 ± 3.21	77.80 ± 1.94	52.80 ± 3.80	87.00 ± 0.84	76.23	0.3910
(d) Node classification test accuracy  $(\%)$  with GRACE as backbone with different augmentations
Augmentation	Cora	CiteSeer	PubMed	Photo	Computers	Avg.	Avg p-value.
NO Aug	85.76 ± 1.72	74.77 ± 1.02	85.65 ± 0.88	90.14 ± 1.40	82.38 ± 1.00	83.74	0.2227
FM+EP	86.27 ± 1.31	75.14 ± 2.11	86.02 ± 0.87	92.10 ± 0.97	83.01 ± 0.66	84.51	-
Gaussian	86.35 ± 1.82	74.35 ± 1.90	86.09 ± 0.58	90.72 ± 1.81	83.17 ± 0.81	84.14	0.3848