SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof. Existing spectral augmentations, such as SpCo [18] and SPAN [17], generate graph views by perturbing the graph spectrum. We assume that there is a continuous univariate function  $\delta(\cdot)$  between the original eigenvalues  $\lambda$  and the perturbed eigenvalues  $\lambda'$ , such that  $\lambda_i' = \delta(\lambda_i)$ , and the perturbed eigenvalues are still non-negative. Then these spectral augmentations can be represented as:
$$
\mathbf {A} ^ {\prime} = \mathbf {U} \delta (\boldsymbol {\Lambda}) \mathbf {U} ^ {\top} = \left(\mathbf {U} \delta (\boldsymbol {\Lambda}) ^ {\frac {1}{2}}\right) \left(\mathbf {U} \delta (\boldsymbol {\Lambda}) ^ {\frac {1}{2}}\right) ^ {\top}. \tag {9}
$$
Note that Equation (5) is a Fourier series and can approximate any continuous functions. Therefore, the function  $\delta (\cdot)$  is a special case of  $\rho (\cdot)$ , and EigenMLP can approximate the simplex geometry [32] of the augmentations, i.e.,  $\mathbf{U}\delta (\boldsymbol {\Lambda})^{\frac{1}{2}}$ , thus generalizing existing spectral augmentations.
5.2 Connections to Existing Work
Hypernetworks [6] are a class of neural networks used to generate parameters for another network. EigenMLP can be seen as a special case of Hypernetworks, which takes eigenvalues as input and generates equivariant parameters for spectral features. Because Hypernetworks can generate a set of non-shared weights from shared parameters, it can greatly reduce the number of trainable parameters. EigenMLP inherits this advantage, and its number of parameters is only related to the period of the Fourier series, rather than the number of eigenvectors. Therefore, EigenMLP is more efficient than other spectral encoders, including vanilla MLP.
Spectral GNNs aim to combine the eigenspaces with filtered eigenvalues, i.e.,  $\sum_{i=1}^{N} \delta(\lambda_i) \mathbf{u}_i \mathbf{u}_i^\top$ , and EigenMLP is used to combine the eigenvectors with filtered eigenvalues, i.e.,  $\sum_{i=1}^{N} \rho(\lambda_i) \mathbf{u}_i$ . Therefore, both methods choose to use eigenvalue mappings as weights to guarantee the equivariance. However, the calculation of eigenspaces has the complexity of  $\mathcal{O}(N^2)$ , which is not suitable for large graphs, whereas EigenMLP is more scalable.
In this section, we conduct three types of experiments, including unsupervised node classification, unsupervised graph prediction, and transfer learning, to verify the effectiveness of  $\mathrm{Sp}^2\mathrm{GCL}$ . Besides, we also test the stability and time overhead of the proposed method.
6.1 Unsupervised Node Classification
Datasets. In the node classification task, we consider using graphs with different scales to evaluate both the effectiveness and scalability of GCL methods. Specifically, for the small graphs (< 50,000), we use Pubmed [14], Wiki-CS [22], and Facebook [23] datasets. For the large graphs (> 50,000), we use Flickr [43], arXiv [9], and PPI [7] datasets. Additional statistics are provided in Appendix B.
Baselines and Setting. We compare our model against a wide range of baselines, including semi-supervised GNNs, e.g., GCN [14] and GAT [35], graph self-supervised learning methods, e.g., DGI [36] and BGRL [29], GCL with spatial augmentations, e.g., MVGRL [8], GRACE [48], and CCASSG [44], and GCL with spectral augmentations, e.g., SpCo [18] and SPAN [17]. For the Facebook dataset, we randomly split the nodes into train/validation/test data with a ratio of 1:1:8. For other datasets, we use the public splits for a fair comparison. We use a two-layer GCN as the encoder for
Table 2: Node classification on transductive and inductive graphs. Mean accuracy  $(\%)\pm$  standard deviation. Bold indicates the best performance and " -" means out-of-memory or cannot be reproduced.
Model	Data	Small Graphs (Full-Batch)	Large Graphs (Mini-Batch)
PubMed	Wiki-CS	Facebook	arXiv	Flickr	PPI
GCN	A, X, Y	79.0	77.19±0.12	90.65±0.16	71.74±0.29	49.20±0.31	82.28±0.24
GAT	A, X, Y	79.0±0.3	77.65±0.11	90.47±0.15	71.82±0.23	54.48±0.21	98.85±0.05
DGI	A, X	76.8±0.6	75.35±0.14	84.42±0.43	70.32±0.25	50.59±0.28	63.80±0.20
BGRL	A, X	79.6±0.5	79.98±0.13	89.71±0.35	71.54±0.17	51.87±0.15	73.63±0.16
MVGRL	A, X	80.1±0.7	77.52±0.08	87.29±0.28	-	-	71.45±0.14
GRACE	A, X	80.6±0.4	80.14±0.48	89.32±0.40	-	-	69.71±0.17
CCA-SSG	A, X	81.0±0.4	78.85±0.32	89.45±0.60	71.21±0.20	51.66±0.10	73.34±0.17
SpCo	A, X, Λ	81.5±0.4	79.16±0.27	89.98±0.45	-	-	-
SPAN	A, X, Λ	81.5±0.2	82.13±0.15	-	-	-	-
Sp2GCL	A, X, Λ, U	82.3±0.3	79.42±0.19	90.43±0.13	71.83±0.19	52.05±0.33	74.28±0.22
Table 3: Results on the graph-level tasks. The best and runner-up results are highlighted with **bold** and **underline**, respectively. **↓** means lower the better, and **↑** means higher the better.
Task
Dataset	Regression (Metric: RMSE ↓)	Classification (Metric: ROC-AUC% ↑)
molesol	mollipo	molfreesolv	molbace	molbbbpl	molclintox	moltox21	molsider
Supervised	1.173±0.057	0.757±0.018	2.755±0.349	72.97±4.00	68.17±1.48	88.14±2.51	74.91±0.51	57.60±1.40
InfoGraph	1.344±0.178	1.005±0.023	10.005±4.819	74.74±3.64	66.33±2.79	64.50±5.32	69.74±0.57	60.54±0.90
GraphCL	1.272±0.089	0.910±0.016	7.679±2.748	74.32±2.70	68.22±1.89	74.92±4.42	72.40±1.01	61.76±1.11
MVGRL	1.433±0.145	0.962±0.036	9.024±1.982	74.20±2.31	67.24±1.39	73.84±4.25	70.48±0.83	61.94±0.94
JOAO	1.285±0.121	0.865±0.032	5.131±0.722	74.43±1.94	67.62±1.29	78.21±4.12	71.83±0.92	62.73±0.92
AD-GCL	1.217±0.087	0.842±0.028	5.150±0.624	76.37±2.03	68.24±1.47	80.77±3.92	71.42±0.73	63.19±0.95
SPAN	1.218±0.052	0.802±0.019	4.531±0.463	76.74±2.02	69.59±1.34	80.28±2.42	72.83±0.62	64.87±0.88
Sp2GCL	1.235±0.119	0.835±0.026	4.144±0.573	78.76±1.43	68.72±1.53	80.88±3.86	73.06±0.75	64.23±0.96
all datasets and set the hidden dimension  $d = 512$  for all methods. For our model, the spatial encoder is the same as baselines, and we additionally use EigenMLP to learn the spectral representation. In the evaluation, we use a linear classifier to evaluate the performance of all methods, as suggested by [27]. We run all the models 10 times and report the mean accuracy and standard deviation. More details, e.g., optimizers, and hyperparameters, are provided in Appendix B.
Results. From Table 2, we can find that  $\mathrm{Sp}^2\mathrm{GCL}$  consistently outperforms state-of-the-art baselines on 5 out of 6 datasets, which validates the effectiveness of the proposed spatial-spectral contrastive framework. Meanwhile, spectral-based methods are proven to be more effective than spatial-based methods, suggesting that integrating spectral information into GCL can help models learn better representations. However, it is worth noting that neither SpCo nor SPAN works for large graphs, implying that perturbing graph spectrum cannot be scalable to large-scale datasets. Therefore, the application scenarios of these two graph augmentations are limited. On the contrary, our method can be used for large graphs and can be easily trained in a mini-batch manner, which is more scalable than other spectral-based methods. Additionally, We find that  $\mathrm{Sp}^2\mathrm{GCL}$  does not perform well in the Wiki-CS dataset. The reason is that the node features dominate the classification results while the graph structure contributes less. Therefore, the spectral view cannot complement the spatial view.
6.2 Unsupervised Graph Prediction
Setup. We benchmark our model on the OGB graph property prediction task [9], which contains three regression datasets and five classification datasets. We choose a series of competitive GCL methods as baselines, including InfoGraph [26], GraphCL [42], MVGRL [8], JOAO [41], AD-GCL [27], and SPAN [17]. It is worth noting that SpCo [18] is not designed for graph-level contrastive learning, so we do not compare with it. We use a five-layer GIN [39] with a graph pooling layer as the encoder for all methods. Similarly, we additionally use EigenMLP to encode the spectral view for  $\mathrm{Sp}^2\mathrm{GCL}$ . We use a linear downstream classifier, e.g., logistic regression model, to evaluate the performance of different GCL methods, as suggested by [27].
Table 4: Graph transfer learning on the molecular classification tasks. (Metric: ROC-AUC  $(\%)\uparrow$
Dataset	ZINC-2M
BBBP	Tox21	SIDER	ClinTox	BACE	HIV	MUV	ToxCast
No Pre-Train	65.8±4.5	74.0±0.8	57.3±1.6	58.0±4.4	70.1±5.4	75.3±1.9	71.8±2.5	63.4±0.6
InfoGraph	68.8±0.8	75.3±0.5	58.4±0.8	69.9±3.0	75.9±1.6	76.0±0.7	75.3±2.5	62.7±0.4
GraphCL	69.7±0.7	73.9±0.7	60.5±0.9	76.0±2.7	75.4±1.4	78.5±1.2	69.8±2.7	62.4±0.6
MVGRL	69.0±0.5	74.5±0.6	62.2±0.6	77.8±2.2	77.2±1.0	77.1±0.6	73.3±1.4	62.6±0.5
JOAO	71.4±0.9	74.3±0.6	60.5±0.7	81.0±1.6	75.5±1.3	77.5±1.2	73.7±1.0	63.2±0.5
AD-GCL	70.0±1.1	76.5±0.8	63.3±0.8	79.8±3.5	78.5±0.8	78.3±1.0	72.3±1.6	63.1±0.7
SPAN	70.0±0.7	78.0±0.5	64.7±0.5	80.7±2.1	79.9±0.7	77.8±0.6	73.8±0.9	64.2±0.4
Sp2GCL	70.3±1.2	78.2±0.6	63.0±0.6	81.0±1.9	80.0±1.1	78.0±0.8	74.2±1.4	64.8±0.5
Table 5: SpGCL with different spectral encoders. Table 6: Justification of spatial-spectral contrast.
Pubmed	Facebook	Molbace
EigenMLP	82.3	90.43	78.76
MLP	82.3	90.22	77.49
SAN	78.6	81.52	75.96
BasisNet	77.3	83.87	74.55
Results. Table 3 summarizes the graph prediction performance.  $\mathrm{Sp}^2\mathrm{GCL}$  has 4 best and 3 runner-up performances and makes significant improvements on molfreesolv and molbace. Compared with GraphCL and JOAO, which employ multiple spatial augmentations, e.g., edge and node dropping, and feature masking,  $\mathrm{Sp}^2\mathrm{GCL}$  outperforms them by only using the spectral domain information. This demonstrates the effectiveness of spectral information. Compared with AD-GCL and SPAN, which use adversarial learning to find the near-optimal invariant information [31],  $\mathrm{Sp}^2\mathrm{GCL}$  only uses the traditional InfoMax principle [1] and achieves competitive performance. This shows that the position information of spectral features is crucial to graph representation learning. Besides, in Table 9, we show that the efficiency and scalability of  $\mathrm{Sp}^2\mathrm{GCL}$  significantly outperform adversarial learning.
6.3 Transfer Learning Scenario
Setup. The transfer learning experiments are used to evaluate the generalization ability of GCL methods. We follow the experimental setup described in [10], which involves pre-training the GCL methods on a large-scale dataset and subsequently fine-tuning the model on downstream datasets to evaluate their out-of-distribution performance. We use the same graph encoder and baseline as in the unsupervised graph prediction task. Further details are provided in Appendix B.
Results. According to Table 4,  $\mathrm{Sp}^2\mathrm{GCL}$  outperforms baselines in 4 out of 8 datasets and achieves an average rank of 1.7 across these datasets. Notably, the performance of different methods on different downstream datasets varies greatly, suggesting that the downstream tasks require distinct information for prediction. Nevertheless, spectral information proves to be crucial in the majority of tasks. Furthermore,  $\mathrm{Sp}^2\mathrm{GCL}$  achieves comparable performance to SPAN, which employs a spectrum-based augmentation for learning graph representations. This observation indicates that  $\mathrm{Sp}^2\mathrm{GCL}$  can learn similar spectrums as SPAN. However, our model falls short of InfoGraph, GraphCL, and JOAO on certain datasets due to the limited availability of feature information in the spectral view.
6.4 Ablation Studies
We conducted two ablation studies to validate the effectiveness of EigenMLP and the proposed framework,  $\mathrm{Sp}^2\mathrm{GCL}$ . Initially, we replace EigenMLP with different spectral encoders and evaluate their performance on PubMed, Facebook, and molbace datasets. The results, presented in Tables 5, demonstrate that EigenMLP consistently outperforms other spectral encoders, thus confirming its effectiveness in learning spectral representations. Moreover, we aim to justify the superior ability of spatial-spectral contrast in fusing spatial and spectral representations. To accomplish this, we concatenate node features with eigenvectors and eigenvalues and subsequently feed them into GRACE. As shown in Table 6, we observe that incorporating spectral features improves the performance in
Table 7: Synthetic perturbations: Eigenvectors with random reflection  $\mathbf{\Pi }$  and rotation  $\mathbf{Q}$  .
PubMed	Test
U	UII	UQ
Train	U	78.8 (+0.1)	78.3 (-5.5)	78.8 (-6.4)
UII	78.5 (-7.8)	78.9 (-0.4)	79.0 (-5.2)
UQ	78.7 (-5.1)	78.8 (-9.3)	78.9 (+0.0)
Table 8: Practical perturbations: Eigenvectors with different EVD tolerances.
PubMed	Test
U3	U4	U5
Train	U3	78.6 (+0.3)	73.9 (-1.5)	73.7 (-7.8)
U4	75.1 (-3.2)	78.2 (+0.3)	72.8 (-6.8)
U5	73.7 (-6.9)	72.0 (-5.0)	79.7 (-0.5)
Table 9: Time overheads (s) of different GCL methods.
Method	Preprocessing	Training (100 Epochs)	Inference (×10-3)
GRACE	0.00	22.77	4.4
MVGRL (GD)	898.23	20.01	4.7
SpCo (SI, T=10)	127.16	22.81	4.5
SPAN (EVD, T=10)	658.62	142.92	-
Sp2GCL (EVD)	66.44	17.79	6.2
Table 10: Overall time overheads (s) of different spectral encoders in 1000 forward passes.
Method	Facebook (k=100)	moltox21 (7831 Graphs)
MLP	2.24	2.09
SAN	127.23	60.58
BasisNet	169.83	84.64
EigenMLP	5.34	3.14
GRACE. Nevertheless, the performance is still inferior to that of  $\mathrm{Sp}^2\mathrm{GCL}$ , thus providing further evidence of the effectiveness of spatial-spectral contrast.
6.5 Stability of EigenMLP and MLP
We conduct stability experiments to evaluate whether EigenMLP and MLP can learn stable representations against perturbations. We consider two types of perturbations: 1) Synthetic perturbation, which applies random reflection  $\Pi$  and rotations  $\mathbf{Q}$  on the eigenvectors. 2) Practical perturbation, which decomposes the Laplacian matrix with different tolerances  $(10^{-3}, 10^{-4}, 10^{-5})$ . The corresponding eigenvectors are expressed as  $\mathbf{U}_3$ ,  $\mathbf{U}_4$ , and  $\mathbf{U}_5$ . Note that synthetic perturbation only changes the signs and coordinates of eigenvectors while practical perturbation is more challenging because it perturbs the values. For clearer results, we only evaluate the performance of spectral representations.
For each type of perturbation, we construct three instances, i.e.,  $(\mathbf{U}, \mathbf{U}\Pi, \mathbf{UQ})$  or  $(\mathbf{U}_3, \mathbf{U}_4, \mathbf{U}_5)$ . The models are trained on one instance and tested on the other two instances on PubMed. We report the results of EigenMLP and show the performance change after switching to MLP in the brackets. From Table 7, we can find that the synthetic perturbation can hardly change the representations learned by EigenMLP but has a great influence on MLP. Additionally, Table 8 shows that practical perturbations can affect both EigenMLP and MLP. Nevertheless, EigenMLP still consistently outperforms MLP across different tolerances, which verifies the stability of EigenMLP.
6.6 Time Overhead
To assess the efficiency, we compare the time overhead of  $\mathrm{Sp}^2\mathrm{GCL}$  with other GCL methods as well as compare EigenMLP with different spectral encoders. The time costs associated with preprocessing, training, and inference are shown in Table 9. Previous spectrum-based methods have significant time costs in the preprocessing stage, e.g., graph diffusion (GD) or Sinkhorn's Iteration (SI), whereas  $\mathrm{Sp}^2\mathrm{GCL}$  exhibits minimal overhead. In the training stage,  $\mathrm{Sp}^2\mathrm{GCL}$  outperforms other methods due to the efficient spectral encoding, but it also introduces an additional burden during inference. Table 10 illustrates the overhead imposed by various spectral encoders within the  $\mathrm{Sp}^2\mathrm{GCL}$  framework. Both MLP and EigenMLP exhibit remarkable efficiency. Conversely, SAN and BasisNet entail excessive time costs due to their quadratic time complexity.
In this study, we introduce  $\mathrm{Sp}^2\mathrm{GCL}$ , a novel spatial-spectral GCL framework that learns the consistency between the spatial and spectral views of graphs. To effectively and efficiently learn the spectral view information, we propose EigenMLP, a scalable spectral encoder to learn stable spectral representations from the non-unique spectral features. Extensive experiments on various graph-related tasks demonstrate the effectiveness, efficiency, and stability of the proposed method.
Limitation and Broader Impact Currently, we focus on encoding spectral features while ignoring node features. A promising future direction is to unify these two kinds of information to learn effective graph representations. Our work reveals the superiority of integrating spectral information into GCL and may inspire the community to pay more attention to the spectral view of graphs.
This work is supported in part by the National Natural Science Foundation of China (No. U20B2045, 62192784, U22B2038, 62002029, 62172052). This work is also partially supported by the Ministry of Education, Singapore, under its Academic Research Fund Tier 2 (Proposal ID: T2EP20122-0041). Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of the Ministry of Education, Singapore.
[1] Philip Bachman, R. Devon Hjelm, and William Buchwalter. Learning representations by maximizing mutual information across views. In NeurlPS, pages 15509-15519, 2019.
[2] Michael Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In NeurIPS, pages 3837-3845, 2016.
[3] Vijay Prakash Dwivedi and Xavier Bresson. A generalization of transformer networks to graphs. CoRR, abs/2012.09699, 2020.
[4] Vijay Prakash Dwivedi, Chaitanya K. Joshi, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Benchmarking graph neural networks. CoRR, abs/2003.00982, 2020.
[5] Or Feldman, Amit Boyarski, Shai Feldman, Dani Kogan, Avi Mendelson, and Chaim Baskin. Weisfeiler and leman go infinite: Spectral and combinatorial pre-colorings. CoRR, abs/2201.13410, 2022.
[6] David Ha, Andrew M. Dai, and Quoc V. Le. Hypernetworks. In ICLR (Poster). OpenReview.net, 2017.
[7] William L. Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In NIPS, pages 1024-1034, 2017.
[8] Kaveh Hassani and Amir Hosein Khas Ahmadi. Contrastive multi-view representation learning on graphs. In ICML, volume 119 of Proceedings of Machine Learning Research, pages 4116-4126. PMLR, 2020.
[9] Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. In NeurIPS, 2020.
[10] Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay S. Pande, and Jure Leskovec. Strategies for pre-training graph neural networks. In ICLR. OpenReview.net, 2020.
[11] Yang Hu, Haoxuan You, Zhecan Wang, Zhicheng Wang, Erjin Zhou, and Yue Gao. Graph-mlp: Node classification without message passing in graph. CoRR, abs/2106.04051, 2021.
[12] Wei Jin, Tyler Derr, Haochen Liu, Yiqi Wang, Suhang Wang, Zitao Liu, and Jiliang Tang. Self-supervised learning on graphs: Deep insights and new direction. CoRR, abs/2006.10141, 2020.
[13] Mark Kac. Can one hear the shape of a drum? The american mathematical monthly, 73(4P2):1-23, 1966.
[14] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.
[15] Devin Kreuzer, Dominique Beaini, William L. Hamilton, Vincent Létourneau, and Prudencio Tossou. Rethinking graph transformers with spectral attention. In NeurIPS, pages 21618-21629, 2021.
[16] Derek Lim, Joshua Robinson, Lingxiao Zhao, Tess E. Smidt, Suvrit Sra, Haggai Maron, and Stefanie Jegelka. Sign and basis invariant networks for spectral graph representation learning. CoRR, abs/2202.13013, 2022.
[17] Lu Lin, Jinghui Chen, and Hongning Wang. Spectral augmentation for self-supervised learning on graphs. In ICLR, 2023.
[18] Nian Liu, Xiao Wang, Deyu Bo, Chuan Shi, and Jian Pei. Revisiting graph contrastive learning from the perspective of graph spectrum. In NeurIPS, 2022.
[19] Xiao Liu, Fanjin Zhang, Zhenyu Hou, Li Mian, Zhaoyu Wang, Jing Zhang, and Jie Tang. Self-supervised learning: Generative or contrastive. IEEE Trans. Knowl. Data Eng., 35(1):857-876, 2023.
[20] Yixin Liu, Shirui Pan, Ming Jin, Chuan Zhou, Feng Xia, and Philip S. Yu. Graph self-supervised learning: A survey. CoRR, abs/2103.00111, 2021.
[21] Haggai Maron, Heli Ben-Hamu, Nadav Shamir, and Yaron Lipman. Invariant and equivariant graph networks. In ICLR (Poster). OpenReview.net, 2019.
[22] Péter Mernyei and Catalina Cangea. Wiki-cs: A wikipedia-based benchmark for graph neural networks. CoRR, abs/2007.02901, 2020.
[23] Benedek Rozemberczki, Carl Allen, and Rik Sarkar. Multi-scale attributed node embedding. J. Complex Networks, 9(2), 2021.
[24] Daniel A. Spielman. Spectral graph theory and its applications. In FOCS, pages 29-38. IEEE Computer Society, 2007.
[25] Balasubramaniam Srinivasan and Bruno Ribeiro. On the equivalence between positional node embeddings and structural graph representations. In ICLR. OpenReview.net, 2020.
[26] Fan-Yun Sun, Jordan Hoffmann, Vikas Verma, and Jian Tang. Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization. In ICLR. OpenReview.net, 2020.
[27] Susheel Suresh, Pan Li, Cong Hao, and Jennifer Neville. Adversarial graph augmentation to improve graph contrastive learning. In NeurIPS, pages 15920-15933, 2021.
[28] Matthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T. Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. In NeurIPS, 2020.
[29] Shantanu Thakoor, Corentin Tallec, Mohammad Gheshlaghi Azar, Mehdi Azabou, Eva L. Dyer, Rémi Munos, Petar Velickovic, and Michal Valko. Large-scale representation learning on graphs via bootstrapping. In ICLR. OpenReview.net, 2022.
[30] Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive multiview coding. In ECCV, pages 776-794. Springer, 2020.
[31] Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola. What makes for good views for contrastive learning? In NeurIPS, 2020.
[32] Leo Torres, Kevin S. Chan, and Tina Eliassi-Rad. GLEE: geometric laplacian eigenmap embedding. J. Complex Networks, 8(2), 2020.
[33] Aäron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. CoRR, abs/1807.03748, 2018.