ML$^2$-GCL: Manifold Learning Inspired Lightweight Graph Contrastive Learning
Authors: Jianqing Liang
Zhiqiang Li
Xinkai Wei
Yuan Liu
Zhiqiang Wang
Venue: ICML 2025 poster (2025)
Keywords: Manifold Learning
Graph Contrastive Learning
Primary area: deep_learning->selfsupervised_learning

Abstract: Graph contrastive learning has attracted great interest as a dominant and promising self-supervised representation learning approach in recent years. While existing works follow the basic principle of pulling positive pairs closer and pushing negative pairs far away, they still suffer from several critical problems, such as the underlying semantic disturbance brought by augmentation strategies, the failure  of GCN in capturing long-range dependence, rigidness and inefficiency of node sampling techniques. To address these issues, we propose Manifold Learning Inspired Lightweight Graph Contrastive Learning (ML$^2$-GCL), which inherits the merits of both manifold learning and GCN. ML$^2$-GCL avoids the potential risks of semantic disturbance with only one single view. It achieves global nonlinear structure recovery from locally linear fits, which can make up for the defects of GCN. The most amazing advantage is about the lightweight due to its closed-form solution of positive pairs weights and removal of pairwise distances calculation. Theoretical analysis proves the existence of the optimal closed-form solution. Extensive empirical results on various benchmarks and evaluation protocols demonstrate effectiveness and lightweight of ML$^2$-GCL. We release the code at https://github.com/a-hou/ML2-GCL.

ML2-GCL: Manifold Learning Inspired Lightweight Graph Contrastive Learning
Jianqing Liang' Zhiqiang Xinkai Wei Yuan Liu"
Zhiqiang Wang
Abstract
data to facilitate various downstream tasks (Cai et al., 2018) has shown wide applications in social network analysis (Pal et al., 2020), molecular property prediction (Liang et al., 2024) and point clouds (Du et al., 2024). Unlike traditional Euclidean data, unstructured features and complex relations in graphs pose unique challenges to representation learning. While a tremendous number of works have been developed (Kipf & Welling, 2017; Velickovic et al., 2018), the require- ment of task-dependent labels limits their applicability in practical scenarios. Inspired by the success of contrastive learning in the visual domain, graph contrastive learning (GCL) has demonstrated great potential and yielded promis- ing performance (Velickovic al., 2019; Peng et al., 2020; Hassani & Khasahmadi, 2020; Zhu et al., 2020).
Graph contrastive learning has attracted great interest as a dominant and promising self- supervised representation learning approach in recent years. While existing works follow the ba- sic principle of pulling positive pairs closer and pushing negative pairs far away, they still suffer from several critical problems, such as the under- lying semantic disturbance brought by augmen- tation strategies, the failure of GCN in capturing long-range dependence, rigidness and inefficiency of node sampling techniques. To address these issues, we propose Manifold Learning Inspired Lightweight Graph Contrastive Learning (MI.2- GCL), which inherits the merits of both manifold learning and GCN. -L avoids the poten- tial risks semantic disturbance with only one single view. It achieves global nonlinear struc- ture recovery from locally linear fits, which can make up for the defects of GCN. The most amaz- ing advantage is about the lightweight due its closed-form solution of positive pairs weights and removal of pairwise distances calculation. The- oretical analysis proves the existence of the op- timal closed-form solution. Extensive empirical results on various benchmarks and evaluation pro- tocols demonstrate effectiveness and lightweight of ML2-GCL, We release the code https: / //github... com/ Saa-hoooMMM-GGGC
GCL pulls positive pairs closer and pushes negative pairs far away without labels to learn node embeddings for down- stream tasks. Despite comparable performance to super- vised learning methods, some critical challenges of augmen- tation strategies and node sampling approaches still remain to be solved. The commonly used random augmentation strategies such as node dropping (You et al., 2020) and edge perturbation (Qiu et al., 2020; Zhang et al., 2023), may de- stroy the structural integrity and semantic consistency of graphs. Current node sampling approaches mainly rely on the similarity of structure relations and node embeddings (Lee et al., 2022; Shen al., 2023; Liang et al., 2025), probability models, especially clustering algorithms mixed models, to estimate the probability of positive pairs and negative pairs (Li et al., 2023; Dong et al., 2024; Sun et al., 2024). Such a simple and straightforward strategy may lead to high computational cost, and thus is not applica- ble to large-scale applications. Moreover, they also ignore the importance of different nodes, among which there exist significant differences in social networks. We are highly motivated to develop efficient and effective GCL method significantly differs from existing approaches.
1. Introduction
Graph representation learning, which aims to learn appro- priate -ow-dimensional representations for graph structured
'Equal contribution 'eyy Laboratory of Computational Intel- ligence and Chinese Information Processing of Ministry of Edu- cation, School of Computer and Information Technology, Shanxi University, Taiyuan 030006, Shanxi, China. Correspondence to: Jianqing Liang <liangjq @ sxu.edu.c>>.
Manifold learning, which can capture both the local struc- ture and global topology through modeling the geometric characteristics of data low dimensional manifold space, provides new perspective for graph contrastive learning. However, existing GCL methods have not fully exploited the potential of manifold learning yet, especially the lack
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

MLA-GCL: Manifold Learning Inspired Lightweight Graph Contrastive Learning
2
2
Ā
19
WF
P
WP
W
s.t.
W/ 2 0,L,J 1,2,- ...
1 122 .N
n
Loss
00
DOMO
OC m
g AX
GCN fo
20m
Z = f.
Figure 1. The overview of ML3-GCL With the original node feature X and the adjacency matrix A, we compute the positive pairs weight matrix W with a closed- form solution of manifold learning inspired node representation optimization problem. Then, we calculate the negative pairs weight matrix w and apply GCN as graph encoder and obtain the node embedding Z. Last, we compute the normalized loss function with regard the positive pairs weight matrix W negative pairs weight matrix W as well as the node emhedding then update the graph encoder.
of theoretical guidance in node sampling. Inspired by LLE (Roweis& & Saul, 2000), coherent structure of graphs leads to strong correlations between inputs, such between neigh- boring nodes, generating observations that lic on or close to a smooth low-dimensional manifold.
graph with GCN maintain semantic consistency and achieve lightweight.
We design novel contrastive loss function with the closed-form solution of anchor node reconstruction combination weights, which can better distinguish the importance of different nodes.
In this paper, we propose Manifold Learning Inspired Lightweight Graph Contrastive Learning (ML2-GCL). Fig- ure gives the overview of ML Figure 2 shows the comparison of GRACE (Zhu al., 2020), NCLA (Shen et al., 2023) and MI---CIL, Different from previous GCL. methods, ML2-GCL eliminates the need to estimate pair- wise distances between node embeddings. does not need to generate augmented views, thereby avoiding the underly- ing semantic disturbance. In addition, ML2-GCL recovers global nonlinear structure from locally linear fits, which can better distinguish the importance of neighboring nodes. Our contributions are listed as follows.
2. Related Work
2.1. Graph Contrastive Learning
Current GCL methods can be categorized into three main- stream paradigms, i.c., DGI (Velickovic et al., 2019), In- foNCE (Oord et al., 2018 8) and BGRL framework (Thakoor et al., 2022). DGI (Velickovic et al., 2019) aggregates features of all nodes in graphs obtain global feature, and then maximizes the mutual information between the global feature and the node features. Based this framework, MVGRL (Hassani & Khasahmadi, 2020) adopts graph dif- fusion and subgraph sampling strategies to advance the de- velopment of unsupervised graph contrastive learning. De- spite the outstanding performance, the global features fail to preserve node-level embedding information adequately.
To the best of our knowledge, is the first exploration to marry manifold learning with graph contrastive learning, which inherits the merits of both manifold learning and GCN.
We propose solely relying on the original
2

---

MLA--CL: Manifold Learning Inspired Lightweight Graph Contrastive Learning
Positive pairs
Negative pairs
Cg Ancher
Anchor
Anchor
G = (X, A)
91 = =,),7,)
G. = (X, A,)
(a) Input Graph
G= (X, A, W)
G2 (8,, A,)
GM (X, As) (d) NCLA
(b) ML3 GCL
(c) GRACE
Figure 2. ML3-GCL versus GRACE and NCLA. ML3-GCL learns positive and negative pairs weights with the input graphs. GRACE removes edges and masks features to generate views of graphs. NCLA generates learnable augmented views with adaptive topology by multi-head GAT.
GRACE (Zhu et al., 2020) first utilizes the InfoNCE loss (Oord et al., 2018) to maximize the mutual information be- tween positive pairs and minimize that of negative pairs under two augmented views. Subsequently, series of meth- ods have been proposed and applied to node classification
tive augmentation strategy that integrates both structural and attribute information. CI-GCL (Tan et al., 2021) proposes community community-invarian -nvariant GCL framework to preserve the com- munity structure of graphs. Similarly, HomoGCL (Li et al., 2023) employs community detection algorithms to assess the importance of edges and node features. GCS (Wei et al., 2023) leverages both structural and semantic information to achieve adaptive graph augmentation. Although these methods have achieved significant success, existing aug- mentation strategies often exhibit poor adaptability across diverse graphs, which is due to the potential risks of aug- mentation strategies to disturb the underlying semantics. To avoid the above-mentioned issues, some works (Lee et al., 2022; Mo al., 2022; Xia et al., 2022a; Liang al., 2025) apply graph encoders generate multiple views.
(You et al., 2020; 2021; Xia et al., 2022b; Shen et al., 2023; Liang et al., 2025) and graph classification (Tan al., 2021; Yin et al., 2022; Liu et al., 2023) tasks. While these meth- ods have achieved significant success, generation of mul- tiple views results in substantial computational overhead. Inspired by BYOL (Grill et al., 2020), BGRL (Thakoor et al., 2022) uses the random augmentation strategy to gen- crate two views and regards the corresponding augmented nodes as positive pairs. Despite the improvement graph representation learning and computational efficiency, ran- dom augmentation may disrupt the underlying semanties of graphs. AFGRL (Lee al., 2022) generates two views with encoders and employs the k-means algorithm for positive pair sampling. The random initialization of the k-means algorithm may influence the performance.
2.3. Node Sampling
Existing works on node sampling can be divided into two types, i.e., methods with graph structure relations and proba- bilistic models. Farlier works with graph structure relations (Zhu et al., 2020; You et al., 2020; 2021; Zhang et al., 2023) randomly perturb graphs to generate augmented views. They regard the corresponding augmented nodes as positive pairs and all the other nodes of augmented views as negative pairs. Therefore, each anchor point has only one positive pair. AFGRL (Lee al., 2022) constructs multiple positive pairs from three aspects, including k-means clustering sets, k-NN sets and neighboring nodes. NCLA (Shen et al., 2023) directly utilizes the neighbors of anchor points as positive
2.2. Graph Augmentation
Graph augmentation strategies include node dropping (You et al., 2020), edge perturbation (Qiu et al., 2020: Zhang et al., 2023), attribute masking (Zhu et al., 2021; Zhang al., 2022) and subgraph extraction (Hassani Khasah- madi, 2020). GRACE (Zhu al., 2020) uses random topol- ogy perturbation and node attribute masking to generate two augmented views. GCA (Zhu al., 2021) proposes adap-
3

---

MLA-GCL; Manifold Learning Inspired Lightweight Graph Contrastive Learning
pairs. GTCA (Liang et al., 2025) utilizes the node represen- tation with GNN and Transformer as graph encoders and topological property, and thus shows excellent performance. While these methods have made some progress, multiple views may generate large number of pairs, inevitably cause high computational complexity. HomoGCL (Li et al., 2023) regards k-means as a special case of the Gaussian Mixture Model to estimate the probability of positive pairs. NF-N2N (Dong al., 2024) uses mutual information measure the topological similarity between anchor nodes and their neigh- bors for positive pair sampling. SIGNA (Sun et al., 2024) randomly selects neighboring nodes as positive pairs. These methods may suffer from issues related to randomness and high complexity due to an excessive number of parameters.
3.2. MIL--GLL
Neighboring Sampling As Figures 1-2 present, ML GCL is based simple geometric intuitions. Suppose the anchor point and its k-hop neighboring nodes are sam- pled from the same underlying manifold. Assume there are sufficient nodes, i.e., the manifold is well-sampled, we expect each node and its neighbors to lie on or close to a locally linear patch of the manifold. We characterize the local geometry of these nodes by linear coefficients that reconstruct each anchor point from its positive pairs. Then, the optimization objective is follows
min 45
Σ
Σ
w 2 +
(wf))
3. Method
0yEP:
i=1 PyEP
In this section, we first present the preliminaries and nota- tions about GCL. Then, we propose ML2-GCL. Finally, we give the theoretical analysis and complexity analysis.
s.t. w/j 2 0, = 1,2, ..... N,.. EP
(3)
Σ
wh = 1, 1,2,.., N
1/ EP
3.1. Preliminaries and Notations
where P; is the positive pair set of vy. It consists of k-hop neighboring nodes of v;- wo is the reconstruction weight to be solved, which can also be regarded as the probability that e and V are positive pairs. À is a positive hyperparameter to balance the reconstruction error and the regularizer.
Let g EV,8) be an input graph, where V (vi,-..
uN , & C V x V are the node set and the edge set, respectively. We denote the embedding matrix and the adjacency matrix as X E NxF and E {0, 1| where æ, Rf is the node feature of oú and Aij (b).,,) E E. The goal of GCL is to learn a graph encoder fe(X, A) E RVDD and get node embeddings in low di- mensional space without label information, i.e., D & F, which can be directly applied to downstream tasks.
iff
For simplicity, Equation 3 can be converted to the problem as following vector form
Existing GCL methods typically generate positive/negative pairs with augmentation strategies. Take one of the most popular GCL methods GRACE (Zhu et al., 2020) an ex- ample, it generates 2 augmented views G. (X1, A1) and Go = (X2, A2). Then, it encodes the 2 views with the same GNN encoder to get node embeddings Z = fe(X,, (X,AA Ai) and z = fo 22,,,,0 A2). Finally, the loss function defined by the InfoNCE loss (Oord et al., 2018) is as follows
min w
Σ
100 Σ w >1lw0?10)
PyEP.
(4)
s.t. w 0, = 1,2, ..., N, N, P.
Σ
w/ = 1, 1,2,..., N
By EP
Figure shows the equivalent matrix form with wit 0, E W, The optimization problem above is a typical quadratic programming problem that is easy to solve. Refer to Sec 3.3 for detailed theoretical analysis.
N
Σ
((2, =) ((2), z))
2N
1=1
Then, for a given anchor point Us. we assign all of the nega- tive pairs the same weight
where
{(z,, x/)
1 N
n 1
U
E
(5)
exp (s(2,, +)/+))
log
exp (s(2,, =/)7))
exp (s(2,) 21/+T)
exp (,)+T,,))/T)
where N₁ is the negative pair set of u. Similarly, we set we = 0, Ps
predtive pair
inter view negative paits
intra view negalive pains
(2)
where s(::) :) is the similarity function and T is a temperature parameter.
Graph Encoders We apply GCN (Kipf & Welling, 2017) as graph encoder. GCN uses a number of graph convolution

---

MLA-GCL: Manifold Learning Inspired Lightweight Graph Contrastive Learning
layers aggregate information from neighbors and then
X; E RIP, xF is the positive pairs embedding matrix of vq and a, RR is the embedding vector of U4-
updates each layer with the equation follows
Proposition 2. The optimal closed solution of Equation can be derived with non-negativity and normalization constraints on solution of Proposition 1. Specifically, we may imput the weights into the ReLU activation function, i... wa (eLU((w)))) and normalize the weights, i.e., w.." w// Σ W
= =
(AZ" 20
(6)
where z(0 is the feature matrix at layer 1, ( = X, À the normalized adjacency matrix with self-loops, ( denotes the learnable weight matrix of layer ( and non-linear activation function, i.e., ReLU.
In fact, the essence of GCL is instance discrimination as pretext task for self- learning (Wang et al., 2022; Liu et al., 2021). However, learning semantics relies heavily on intra-class data distribution, as shown in (Wang et al., 2022). It is limited by the one-to-one positive sampling strategy, and ML2-GCL may improve it to form a new task that we refer to as personalized graph embedding task, which has less reliance on the data distribution.
Overview As lower bound of the Mutual Information, InfoNCE (Oord et al., 2018) is the most commonly used and popular loss function in GCL. In recent years, a large num- ber of related works have sprung up (Zhu et al., 2020; 2021; Xia et al., 2022b; Zhang et al., 2023; Guo et al., 2023; Yu et al., 2024). In spite of some success, the exploited augmen- tation strategy may bring potential risks of the underlying semantics disturbance. Moreover, only one positive pair for anchor point can inevitably exacerbate the sampling bias problem. To remedy these deficiencies, we construct multiple positive pairs for a given anchor point with its k- hop neighboring nodes and original node embeddings X as Figures 1-2 show. Then, we use GCN as graph encoder to get the projected node embeddings Z. In the end, we define the training objective for each anchor point U, as follows
Theorem 1. Suppose for a given anchor point v in graph, other nodes can be divided into its positive pairs or neg- ative pairs. The training objective is equivalent to graph embedding as given:
t EnEN, (11, Enjep, whys zn)
(9)
We present the whole process of our MI. -GCL. method in Algorithm 1.
f(2,))
Σ wy exp (s(2;; x;)/T)
log
Σ wly exp (s(z x, + u.
Σ ULEA
w exp (s(#1, #u)/T)
Algorithm ML.2-GCI.
Input: Graph 9 = EV,E)
(7)
Output: Node embeddings Z
where T E [0. 1] is a tunable temperature parameter to adjust the degree of attention to the hard negative samples.
1: for in I to N do
2: Construct positive pair set P. of os with its k-hop
Finally, we can define the following overall loss
neighboring nodes;
3: Construct negative pair set N₁ of u; with other nodes;
N
4: end for
C
a (2z))
(8)
N
5: Compute positive pairs weight matrix WP as in Propo-
sitions 1-2:
Compute negative pairs weight matrix n as in Equa- tion 5;
3.3. Theoretical Analysis
We propose the manifold learning inspired positive sampling strategy for GCL from the node neighbor sets. To our best knowledge, this is the first exploration to marry manifold learning with graph contrastive learning. It is superior to the similarity/distanee based approaches and probability models that previous GCL models adopt. The details of the proofs are provided in the Appendix A.
7: Initialize GNN encoder parameters (W.b)
8: while not converge do
9: Obtain GNN embeddings Z = (G))
10: Do forward pass, compute L as in Equations 7-8;
11: Do backward propagation with L:
12: end while
13: return Z for downstream tasks;
In the following, we will give the closed form solution of "" without constraints and with constraints, respectively. Proposition 1. Equation& without constraints has an opti- mal closed solution wp = (x,77' + AT) X,x, where wf E R |Pll the reconstruction weight vector of Uy
3.4. Complexity Analysis
Consider a graph with N nodes, M edges and simple encoders which compute embeddings in time and space
5

---

MLA-GCL: Manifold Learning Inspired Lightweight Graph Contrastive Learning
Table Node classification accuracy (%) on datasets. Best results are colored: first, second, third. Models with stars adopt graph augmentation strategy. Models in italic type use at least 2 views.
Model
Cora
Citeseer
Pubmed
Amazon-Phote
Amazon-Computers
Wiki-CS
GCN &Kig&& Welling, 2017)
79.6 +: 1.8
66.0 + 1.2
79.0 2.5
86.3 1.6
76.4 1.8
67.3 - 1.5
GAT (Velidooiic et 2015)
81.2 + 1.6
68.9 + 1.8
78.5 + 1.8
86.5 2.1
77.9 = 1.8
68.6 + 1.9
CGPN (Was et 2021b)
74.0 + 1.7
63.7 + 1.6
73.3 + 2.5
84.1 = 1.5
74.7 + 1.3
66.1 + 2.1
CG3 il. 20216)
80.6 + 1.6
70.9 - 1.5
78.9 2.6
89.4 1.9
77.8 = 1.7
68.0 = 1.5
DGI 2019)
82.1 + 1.3
71.6 1.2
78.3 + 2.4
83.5 1.2
78.8 = 1.1
69.1 + 1,4
GMI (Penge sl., 20000
79.4 1.2
66.9 + 2.2
76.8 2.3
86.7 1.5
76.1 = 1.2
67.8 + 1.8
MVGRI. (Iisesmi Khaciimadi
82,4+ 1.5
71.1 1.4
+
89.7 1.2
78.7 + 1.7
69.2 + 1.2
GRACE" (the 2070)
79.6
67.0 1.7
+..5
87.9 1,4
76.8 + 1.7
67.8 + 1.4
GCA"
79.0 + 1.4
65.6 2.4
81.5 + 2.5
87.0 1,9
76.9 + 1.4
67.6 + 1.3
SUGRL (Mo 2022)
81.3 81.3 + 1.2
71.0 1.8
80.5 + 1.6
90.5 1,9
78.2 = 1.2
68.7 + 1.1
AFGRL sta 2022)
78.6 1.3
2.1
76.4 2.5
89.2 1.1
77.7 = 1.1
68.0 = 1.7
NCLA (Shez at. 2023)
82.2 1.6
71.7 0.9
82.0 1.4
90.2 1.3
79.8 = 1.5
70.3 = 1.7
GTCA (Lisng 2025)
82.5 + 1.3
69.7 = 1.7
79.8 + 1.3
90.5 = 1.2
79.2 = 1.4
69.7 = 1.5
ML3-GCL
83.0 + 1.3 71.7 + 1.0
82.1 + 1.8
90.7 1.6
83.0 = 2.1
73.3 + 1.7
O(M + N). This property is satisfied by most popular GNN architectures, such as GCN (Kipf & Welling, 2017) and GAT (Velickovic et al., 2018). In each update step, MI.2-GIL perforcl encoder computation and backpropa- gates the learning signal once plus prediction step, while GRACE performs 2 encoder computations (once for each augmentation) and backpropagates the learning signal twice (once for each augmentation), plus prediction step. We costly
and testing following (Yang al., 2016), and all the other datasets following (Liu et al., 2020). For link prediction task, we follow the experimental setup of GCA (Zhu et al., 2021). For each dataset, we conduct 20 random splits of sraniing/alidationnees and report the average performance of all algorithms on the same random splits. The statisties are summarized in Appendix B.
assume the backward pass to be approximately as the forward pass. We ignore the cost of augmentation computation in this analysis. Therefore, the total time and space complexity per update step for ML2-GCL is 20encoder (M + N) + 2Crrdiitiion + CN2, compared to 4Cencoterr (M + NN + 4Cpredictionn N + 4CN2 for GRACE, where C are constants dependent on architecture of differ- ent components. Appendix D gives further details.
4.2. Node Classification
Node classification is one of the common tasks graph neural networks. The goal to predict the class of each node according to its characteristics and context information in the graph structure. In this task, the graph structure and node embeddings are jointly used as inputs, and the relations between nodes and local or global topological information are captured through graph neural networks.
4. Experiments
To evaluate node classification, we compare ML---GCL with 13 state-of-the-art methods including 2 semi-supervised GNNs, i.e., GCN (Kipf & Welling, 2017), GAT (Velickovic et al., 2018), 3), 2 semi-supervised GCL methods, i.e., CGPN (Wan et al., 2021 CG3 (Wan et al., 2021 a)), and 9 self- supervised GCL methods, i.c., DGI (Velickovic et al., 2019), GMI (Peng et al., 2020), MVGRL (Hassani & Khasahmadi, 2020), GRACE (7hu et al., 2020), GCA (Zhu et al., 2021), SUGRL (Mo et al., 2022), AFGRL (Lee et al., 2022), NCLA (Shen et al., 2023) and GTCA (Liang et al., 2025). For cho- sen dyper-parametent see Appendix C.
In this section, we conduct a series of experiments to demon- strate the superiority of MI.2-GLL. First, we briefly deseribe the datasets. Then, we evaluate the empirical performance across various graph datasets node classification and link prediction tasks. In the end, we present the ablation study, hyperparameters analysis and embeddings visualization re- sults. We implement all experiments on the platform with Py Torch 2.0.1 and PyTorch Geometric 2.6.1 on NVIDIA 3090 GPUs with 24GB memory.
Table lists the node classification accuracy of ML -CLL and baselines with a logistic regression model. The results indicate that ML2-GCL achieves state-of-the-art results with respect to previous methods, especially when there are rela- tively more classes. For example, MLL--GLL achieves 83.0% and 73.3% accuracy Amazon-Computer and Wiki-CS,
4.1. Datasets
We conduct experiments on widely-used datasets in- cluding Cora, Citeseer, Pubmed, Amazon-Phoo Amazon- Computers and Wiki-CS. For node classification task, we split Cora, Citeseer and Pubmed for the training, validation
6

---

MLL--CL: Manifold Learning Inspired Lightweight Graph Contrastive Learning
Table Link prediction results (%) on 6 datasets. results are colored: first, second, third. Models with stars adopt graph
augmentation strategy. Models in italic type use at least 2 views.
Cora
Citeseer
Pubmed
Amazon-Phont Photo
Amazon-Computers
Wiki-CS
Model
AUC
AP
AUC
AP
AUC
AP
AUC
AP
AUC
AP
AUC
AP
Spectral INg ai.,
84.6
88.5
80.5
85
84.2
87.8
83.6
84.3
86.7
87.3
81.2
80.3
DeepWalk (Pemazi et 2014)
83.1
85
80.5
83.6
84.4
84.1
83.2
85.1
87.2
87.5
82.1
81.5
GAE
2016)
91
92
89.5
89.9
96.4
96.5
89.9
89.6
93
93.2
83.2
82.1
VGAE (Kipé & Welling 2016;
91.4
92.6
90.8
92
94.4
94.7
89.3
88.8
92.6
92.8
82.5
83.4
ARGE
92.4
93.2
91.9
93
96.8
97.1
91.5
92.2
92.5
92.7
83.5
84.2
ARVGA (Pan 2018)
92.4
92.6
92.4
96.5
96.8
92.1
91.9
93.1
92.8
82.6
83.9
GRACE" (2ha M. 20200
90,9
91
92.1
92.2
97.1
90.8
89.3
91.6
91.2
89.5
88.5
GCA" (2ha
91.4
91.5
92
92.6
96.3
96.5
92.3
91.3
92.5
91.5
86.4
86.3
GDCL (Zh0n 2021)
91.7
90.9
91.9
96.5
96.3
93.1
93.6
93.1
85.2
84.6
ProGCL (XL
92.9
93.5
93.1
93.3
96.1
96.7
92.6
93.5
94.5
94.2
83.6
84.1
AUGCL (Ni et 2024)
93.3
93.2
92.5
92.8
96.3
96.5
94.2
93.9
93.7
93.9
88.9
88.5
MLL-GCL
96.9
95,9
96
95.1
98.7
98.5
95.1
94.2
96.9
96.6
89.8
88.9
Table 3. Ablation study on node classification task for positive/negative pairs weights. The metric is the Accuracy(%)).
WP
Wa
Cora
Citeseer
Pubmed
Amazon- Photo
Amazon- Computers
Wiki-CS
81.2
71.0 0.9
79.5
2.0
89.5 1.4
81.1
70.0 1.9
81.9
71.2 0.9
80.1 1.7
89.6 2.0
82.0
72.2 1.9
82.5 = 1.2
71.7 + 1.1
79.8 = 1.8
90.4 = 1.2
82.9 2.1
72.5 + 2.2
83.0 1.3
71.7 + 1.0
82.1 1.8
90.7 1.6
83.0 2.1
73.3 = 1.7
a 3.2% and 3.0% relative improvement over previous state- of-the-art respectively.
on Cora, i.e., a 3.6% relative improvement over previous state-of-the-art.
It is noteworthy that ML2-GCL achieves the tttee-ffthee art results both node classification and link prediction benchmarks with unified approach. Unlike previous unsu- pervised models, we do not devise specialized encoder for each task.
4.3. Link Prediction
Link prediction aims to predict whether there an edge between two nodes in graphs, or to predict the weights of edges. The goal of this task to capture potential relations between nodes through learning node representations. It can be widely applied in fields such social network analysis, pootein-protein interaction prediction in biological networks, and recommendation systems.
4.4. Ablation Study
In this section, we remove positive/negative pairs weights of ML2-GCL to study the impact of each component. The results are shown in Table 3. We notice that the removal of either positive pairs weights W or negative pairs weights T leads to poorer performance, which demonstrates the effectiveness of our proposed ML2-GCL. In addition, is obvious that compared with the negative pairs weights W the positive pairs weights WP is more crucial for MI.2. GCL, which is mainly due to its role on recovery of global nonlinear structure.
For link prediction, we consider our ML2-GCL with multi- ple baselines including Spectral (Ng et al., 2001), DeepWalk (Perozzi et al., 2014), GAE (Schulman al., 2016), VGAE (Kipf& & Welling. 2016), ARGE (Pan al., 2018), ARVGA (Pan et al., 2018), 8), GRACE (Zhu al., 2020), GCA (Zhu et al., 2021), GDCL (Zhao et al., 2021), ProGCL (Xia et al., 2022b) and AUGCL (Niu et al., 2024). We utilize the Area Under Curve (AUC) and Average Precision (AP) to eval- uate the performance. For chosen pereeeparsmeters see Appendix C.
4.5. Hyperparameters Analysis
The results shown in Table 2 suggest that ML--GCL sig- nificantly outperforms all the other baselines on 6 datasets under different evaluation criteria, highlighting its potential on link prediction task. For example, it achieves 96.9% AUC
Figure 3 illustrates the node classification accuracy of ML GCL with varying values of the temperature parameter r and the hidden dim on datasets. is noteworthy that the optimal temperature parameter is around 0.5 on most of the
7

---

MLA--CL; Manifold Learning Inspired Lightweight Graph Contrastive Learning
Cora
Citeseer
Pubmed
Amazon-Phood
Amazon-Computee
Wiki-CS
95
95
90
Accuracy (%)
Accuracy (%)
80
75
70
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
32
64
128
256
512
Hidden dim
Figure 3. Sensitivity analysis of the temperature parameter and the hidden dim node classification task.
(a) NCLA
(b) GTCA
(c) -CL
Figure Visualization of NCLA, GTCA and ML3-GCL embeddings on Amazon Computerss dataset with t-SNE,
datasets. general, with the increase the hidden dim, the classification performance shows an upward trend on most of the datasets such as Pubmed, Amazon-Photo and Amazon-Computeer, We notice that ML3-GCL achieves the best performance 128. This is mainly due to the fact that the original feature dimension of Wiki-CS is only 300, Increasing the hidden dim will introduce irrelevant or redundant information.
5. Conclusion
In this paper, we marry manifold learning with graph con- trastive learning and develop ML²-GCL. We achieve global nonlinear structure recovery from locally linear fits, and thus inherit the merits of both manifold learning and GCN. It allows us develop new paradigm of contrastive learn- ing to eliminate extra views generation and pairwise dis- tances calculation, thus avoiding the semantic disturbance and high computational complexity completely. In addition, we conduct positive sampling with the optimal closed-form solution of a typical quadratic programming problem. Theo- retical analysis proves existence. Extensive experiments verify both effectiveness and lightweight of ML In future, will investigate deep integration mechanisms between manifold learning and dynamic graph structures to address nonlinear evolution patterns in temporal data or dynamic networks.
Wiki-CS when the hidden dim is
4.6. Embeddings Visualization
To provide a more intuitive presentation of the node em- beddings, we utilize t-SNE (Van der Maaten & Hinton, 2008) to visualize the node embeddings of NCLA (Shen al., 2023), GTCA (Liang et al., 2025) and M on Amazon-Come Computers dataset. As Figure 4 shows, different colors represent different classes. It is obvious that ML?-. GCL can distinguish much more different classes of nodes effectively compared with NCLA and GTCA.
8

---

MLA-GCL: Manifold Learning Inspired Lightweight Graph Contrastive Learning
Acknowledgements
Li, W., Wang, C., Xiong, H., and Lai, Homogcl: Re- thinking homophily in graph contrastive learning. In Proceedings of ACM SIGKDD Conference on Knowledge Discovery and Data Mining. pp. 1341-1352, 2023.
This work supported by National Natural Science Foun- dation of China (2..1227614 U21A20473).
Impact Statement
Liang, J., Chen, M., and Liang, J. Graph external attention enhanced transformer. In Proceedings of International Conference on Machine Learning, 2024.
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Liang, J., Wei, X., Chen, M., Wang, Z., and Liang, J. nen-transformer cooperative architecture for trustwor- thy graph contrastive learning. In Proceedings of AAAI Conference on Artificial Intelligence, 2025.
References
Cai, H., Zheng, V. W., and Chang, K. C.-C. A comprehen- sive survey of graph embedding: Problems, techniques, and applications. IEEE Transactions Knowledge and Data Engineering, pp. 1616-1637, 2018.
Liu, M., Gao, H., and Ji, Towards deeper graph neural networks. Proceedings of ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 338 338-348,70 2020.
Dong, W., Yan, D., and Wang. P. Self-supervised node repre- sentation learning via node-to-neightee nere-ttt-niighbouooooood alignment. IEEE Transactions Pattern Analysis and Machine In- telligence, 2024.
Liu, X., Zhang, F., Hou, Z., Mian, L., Wang, Z., Zhang, J., and Tang, J. Self-supervised learning: Generative or contrastive. IEFE Transactions on Knowledge and Data Engineering, 35(1):857-876, 2021.
Du, Z., Liang, J., Liang, J., Yao, K., and Cao, F. Graph regulation network for point cloud segmentation. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, (1:72))9-7095555 2024.
Liu, Y., Zhao, Y., Wang, X., Geng, L., and Xiao, Z. Multi- scale subgraph contrastive learning. In Proceedings of International Joint Conference Artificial Intelligence, 2023.
pp.
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P., Buchatskaya, E., Doersch, C., Avila Pires, B., Guo, Z., Gheshlaghi Azar, M., et al. Bootstrap your own latent- new approach to self-supervised learning. Advances in Neural Information Processing Systems, pp. 21271- 21284, 2020.
Mernyei, P. and Cangea, C. Wiki-cs: A wikipedia-based benchmark for graph neural networks. In Proceedings of International Conference on Machine Learning, 2020.
Mo. Y., Peng, L., Xu, J., Shi, X., and Zhu, X. Simple unsupervised graph representation learning. In Proceed- ings of AAAI Conference on Artificial Intelligence, pp. 7797-7805,
Guo, X., Wang, Y., Wei, Z., and Wang, Y. Architecture matters: Uncovering implicit mechanisms graph con- trastive learning. In Advances in Neural Information Processing Systems, pp. 28585-28610 2023.
Ng. A., Jordan, M., and Weiss, Y. On spectral clustering: Analysis and an algorithm. Advances in Neural Informa- tion Processing Systems, 14, 2001.
Hassani, K. and Khasahmadi, A. H. Contrastive multi- view representation learning on graphs. In Proceedings of International Conference on Machine Learning, pp. 4116-4126, 2020.
Niu, C., Pang. G., and Chen, L. Affinity uncertainty-based hard negative mining in graph contrastive learning. IEEE Transactions on Neural Networks and Learning Systems, 2024.
Kipf, T. N. and Welling, M. Variational graph auto-encoders. arXiv preprint arXiv:1611.07300 2016.
Oord, A. v. d., Li, Y., and Vinyals, O. Representation learn ing with contrastive predictive coding. arXiv preprint TrXiv:1807.0374: 2018
Kipf, T. N. and Welling, M. Semi-supervised classification with graph convolutional networks. In Proceedings of International Conference on Learning Representations, 2017.
Pal, A., Eksombatchai, C., Zhou, Y., Zhao, B., Rosenberg, C., and Leskovec, J. Pinnersage: Multi-modal user em- bedding framework for recommendations at pinterest. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp.
Lee, N., Lee, J., and Park, C. Augmentation-free self- supervised learning on graphs. In Proceedings of AAAI Conference on Artificial Intelligence, pp. 7372-7380, 2022.
2020.

---

MIA-GCL; Manifold Learning Inspired Lightweight Graph Contrastive Learning
Pan, S., Hu, R., Long, G., Jiang, J., Yao, L.,, and Zhang, C. Adversarially regularized graph autoencoder for graph embedding. In Proceedings of International Joint Con- ference on Artiffcial Intelligence, pp. 1609-2615, 2018.
Thakoor, S., Tallec, C., Azar, M. G., Azabou, M., Dyer, E. L.,, Munos, R., Veličković, P., and Valko, M. Large- scale representation learning graphs via bootstrapping. In Proceedings of International Conference on Learning Representations, 2022.
Peng, Z., Huang, W., Luo, M., Zheng, Q.. Rong, Y., Xu, T., and Huang, J. Graph representation learning via graphical mutual information maximization. In Proceedings of Web Conference, pp. 259-270, 2020.
Van der Maaten, L. and Hinton, G. Visualizing data using t-sne. Journal of Machine Learning Research, 2008.
Velickovic, P., Cucurull, G., Casanova, A., Romero, A.. Lio, P., Bengio, Y., et al. Graph attention networks. In Proceedings of International Conference on Learning Representations, 2018
Pennington, J., Socher, R., and Manning, C. D. Glove: Global vectors for word representation. In Proceedings of Conference on Empirical Methods in Natural Language Processing, pp. 1532-1543, 2014.
Velickovic, P., Fedus, W., Hamilton, W. L., Liò, P., Bengio, Y., and Hjelm, R. D. Deep graph infomax. In Proceedings of International Conference on Learning Representations, 2019.
Perozzi, B., Al-Rfou, R., and Skiena, S. Deepwalk: Online learning of social representations. In Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 701-710, 2014.
Wan, S., Pan, S., Yang, J., and Gong, C. Contrastive and generative graph convolutional networks for graph-based semi-supervised learning. In Proceedings of AAAI Confer- ence on Artificial Intelligence, pp. 10049-10057, 2021 a.
Qiu, J., Chen, Q., Dong, Y.. Zhang, J., Yang. H., Ding, M., Wang, K., and Tang, J. Gee: Graph contrastive coding for graph neural network pre-training. In Proceedings of ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. pp. 1150-1160, 2020.
Wan, S., Zhan, Y., Liu, L., Yu, B., Pan, S., and Gong, C. Con- trastive graph poisson networks: Semi-supervised learn- ing with extremely limited labels. Advances in Neural Information Processing Systems, 34:6316 66276,,,222 2021b.
Roweis, S. T. and Saul, L. K. Nonlinear dimensionality reduction by locally linear embedding. Science, pp. 2323- 2326, 2000.
Wang. Y., Zhang, Q., Wang. Y., Yang, J., and Lin, Z. Chaos is a ladder: A new theoretical understanding of con- trastive learning via augmentation overlap. In Proceed- ings of International Conference on Learning Represen- tations, 2022.
Schulman, J., Moritz, P., Levine, S., Jordan, M., and Abbeel, P. figg--dmensiiona continuous control using generalized advantage estimation. In Proceedings of International Conference on Learning Representations, 2016
Wei, C., Wang, Y., Bai, B., Ni. K., Brady, D., and Fang, L. Boosting graph contrastive learning via graph contrastive saliency. In Proceedings of International Conference on Machine Learning, pp. 36839-36855 2023.
Sen, P., Namata, G., Bilgic, M., Getoor, L., Galligher, B., and Eliassi-Rad, T. Collective classification in network data. Al magazine, 29(3):93-106, 2008
Xia, J., Wu, L., Chen, J., Hu, B., and Li, S. Z. Simgrace: A simple framework for graph contrastive learning without data augmentation. In Proceedings of ACM Weh Confer- ence, pp. 1070-1079, 2022a.
Shchur, O., Mumme, M., Bojchevski, A., and Günnemann, S. Pitfalls of graph neural network evaluation. arXiv preprint arXiv: /8/ 1.08668, 2018
Shen, X., Sun, D., Pan, S., Zhou, X., and Yang, L. T. Neigh- bor contrastive learning on learnable graph augmentation. In Proceedings of AAAI Conference on Artificial Intelli- gence, pp. 9782-9791, 202.3
Xia, J., Wu, I.., Wang, G., Chen, J., and Li, S. Z. Progel: Rethinking hard negative mining in graph contrastive learning. In Proceedings of International Conference on Machine Learning, pp. 24332 22334., 2022b.
Sun, Q. Chen, C. Qiao, Z., Zheng, X., and Wang, K. Single- view graph contrastive learning with soft neighborhood awareness. In Proceedings of AAAI Conference Artifi- cial Intelligence, 2024.
Yang, Z., Cohen, W., and Salakhudinov, R. Revisiting semi- supervised learning with graph embeddings. In Proceed- ings of International Conference on Machine Learning, pp. 40-48, 2016.
Tan, S., Li, D., Jiang, R., Zhang, Y., and Okumura, M. communitt--nvvriinn graph contrastive learning. In Pro- ceedings of International Conference on Machine Learn- ing, 2021.
Yin, Y., Wang, Q., Huang, S., Xiong, H., and Zhang, X. Au- togel: Automated graph contrastive learning via learnable view generators. In Proceedings of AAAI Conference on Artificial Intelligence, pp.
2022.
10

---

MLA-GCL: Manifold Learning Inspired Lightweight Graph Contrastive Learning
You, Y., Chen, T., Sui, Y., Chen, T., Wang, Z., and Shen, Y. Graph contrastive learning with augmentations. In Pro- ceedings of Advances in Neural Information Processing Systems, pp. 5812-5823, 2020.
You, Y., Chen, T., Shen, Y., and Wang, Z. Graph contrastive learning automated. In Proceedings of International Con- ference Machine Learning, pp. 12121-12132, 2021.
Yu, Y., Wang. X., Zhang, M., Liu, N., and Shi, C. Provable training for graph contrastive learning. In Advances in Neural Information Processing Systems, 2024.
Zhang, Y., Zhu, H., Song, Z., Koniusz, P. and King, Costa: covariance-preservaa
feature augmentation for graph contrastive learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and 2022.
Data Mining, pp.
Zhang, Y., Zhu, H., Song, Z., Koniusz, P., and King, Spectral feature augmentation for graph contrastive learn- ing and beyond. In Proceedings of AAAI Conference on Artificial Intelligence, pp. 11289-11299 2023.
Zhao, H., Yang. X., Wang, Z., Yang. E., and Deng, C. Graph debiased contrastive learning with joint representation clustering. Proceedings of International Joint Confer- on Artiffcial Intelligence, pp. 3434 3440, 2021.
Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., and Wang, L. Deep graph contrastive representation learning. In Proceedings of ICML Workshop on Graph Representation Learning and Beyond, 2020.
Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., and Wang, L. Graph contrastive learning with adaptive augmentation. In Proceedings of Web Conference, pp. 2069-2080, 2021.

---

ML---CL: Manifold Learning Inspired Lightweight Graph Contrastive Learning
A. Theoretical Proofs
A.1. Proof of Proposition
Proposition 1. Equation 4 without constraints has an optimal closed solution w/ = (X;X² + "X,=
X,,, where
wo E RPPil is the reconstruction weight vector of Vi, X, E R(P is the positive pairs embedding matrix of v and ac Rf is the embedding vector of et-
Proof. First, we review Equation 4 of our proposed MLL--GLL method and rewrite the objective for each anchor point N without constraints as:
0,, i =
4(w())
|0,
Σ
(10)
eyEP:
Then, we take the derivative of 4 with respect to "
d4
2X x, + 2X? X,u, 2Xw,
(11)
dw
dw: Let d/, = O, we obtain
(X"'x + XTro = -xTx:
(12)
Considering x X; and À >> 0, we may conclude the second derivative of I, with respect to wa
d=1 d(w,)²
2x X; 221 > 0
(13)
Hence, w (X/ + X1) X x; is the extreme point of Equation 10 and also the optimal solution of Equation 4.
A.2. Proof of Proposition 2
Proposition 2. The optimal closed solution of Equation can be derived with non-negativity and normalization constraints on solution of Proposition 1. Specificallyy, may input the weights into the ReLU activation function, i.c., w ReLU(w))) and normalize the weights, i.e., w w w
Proof. As ReLU(w()) max(0, w we may infer that w = max(0, w/t) 0,i 1,2, .... N,,,N,,, E P.. addition, we derive conclusion follows
Σ
w
Σ eEP
Σ
W
bj
P
mEP::
=1, 1,2, ..., N
(14)
N
Σ EyP
+
24
fj
U EN,
j=1
where wh 0, vk Mg.
A.3. Proof of Theorem
Theorem 1. Suppose for a given anchor point v; in a graph, other nodes can be divided into its positive pairs or negative pairs. The training objective is equivalent to graph embedding given:
EVEN, w/ss(z,, 21))
8
(15)
Enjep, w 2,)
12

---

MLA-GCL: Manifold Learning Inspired Lightweight Graph Contrastive Learning
Proof. We rewrite the loss function of anchor point Us with Taylor expansion of the first order
Σ
wfy exp (s(z), 2,)/7)
Σ
with exp (z),,,)))11 =x)/T)
((zi) log PyEP
MEEN
Σ
wh exp (2, 2j)/r)
ojEP;
Σ exp (s(2) =x)/T)
logg11 + V.EN
(16)
Σ wey exp (5(z), </)/T)
MEP:
Σ with exp (s(z), TT)
My EN;
Σ
w/ exp (s(2,; 2,)/T)
PyP:
We extend the loss function of 0; to the form of the entire graph G and remove the temperature hyperparamter 7:
N
C
E(z:)
Σ
tith exp (s(z(,
(17)
" EM
1j exp (s(z,
1=1
1/ EP
Different from typical graph embedding, each positive pair or negative pair is associated with its own weight. The overall objective is calculated with the exponential ratios sum of the negative and positive similarities with anchor points.
B. Dataset Descriptions
We evaluate the performance of ML2-GCL on node-level and edge-level tasks, i.e., node classification and link prediction. We conduct experiments on 6 widely used datasets including Cora, Citeseer, Pubmed, Amazon- Photo,, Amazon-Computers and Wiki-CS, The detailed statistics are summarized in Table 4.
Cora, Citeseer and PubMed (Sen et al., 2008) are well-known citation network datasets, where nodes represent scientific papers and edges represent citation relations. Node features are hag-of-words vectors of papers, and labels represent domains of papers.
Amazon-Photo and Amazon-Come Computers (Shchur et al., 2018) are two networks of co-purchase relationships con- structed from Amazon. Nodes product and edges exist when two products frequently co-purchased. Each node has bag-of-words feature encoding product reviews and is labeled with its product category.
Wiki-CS (Memyci Cangea, 2020) is a reference network from Wikipedia references. Nodes correspond to articles about computer science and edges are hyperlinks between the articles. Articles are labeled with 10 related subfields, and their features are calculated as the average of pre-trained Glove (Pennington al., 2014) word embeddings.
Table 4. Statistics of datasets used in experiments.
Dataset
Nodes
# Edges
Features
Labels
Cora
2,708
5,429
1,433
Citeseer
3,327
4,732
3,703
Pubmed
19,717
44,338
500
Amazon-Photo
7,650
119,081
745
Amazon-Computters
13,752
245,861
767
10
Wiki-CS
11,701
216,123
300
10
13

---

MLA--CL: Manifold Learning Inspired Lightweight Graph Contrastive Learning
C. Hyperparameter Choices
In hyperparameter search, we attempt to adjust the value of & and À in MLL-GCL, as well as other deep learning hyperpa- rameters including temperature parameter т, hidden dim, learning rate, dropout and weight decay. We apply the grid search strategy choose the optimal hyperparameters. Specifically, we search & in [1, 5], {0, 0.01, 0.1, 10, 100). hidden dim from (128, 256, 500, 512), learning rate from (0.0005, 0.001, 0.005, 0.01) and weight decay in {5e-5, le-4, 5e-4, 7e-4). Tables 5-6 give the hyperparameters specifications for MLL- on node classification and link prediction tasks.
Table 5. Hyperparameters specifications for MLL-GCL on node classification task.
Cora
Citeseer
Pubmed
Amazon Photo
Amazon-Computtrs
Wiki-CS
k
1
0.01
1
100
10
0.01
10
Temperature
0.7
0.5
0.5
0.3
0.5
Hidden dim
500
256
512
512
512
128
Learning rate
0.0005
0.005
0.001
0.001
0.0005
0.01
# Epochs
40
20
80
120
240
280
Dropout
0.5
0.4
0.5
0.3
0.5
0.4
Weight decay
5e-5
7e-4
5e-4
5e-4
5e-4
1e-4
Table Hyperparameters specifications ML'-GCL on link prediction task.
Cora
Citeseer
Pubmed
Amazon-Photo
Amazon-
Wiki-CS
k
3
2
4
10
0.001
100
100
Temperature
0.4
0.5
0.3
0.3
0.3
Hidden dim
500
256
500
500
256
500
Learning rate
0.0005
0.01
0.0001
0.0001
0.01
0.001
# Epochs
200
100
200
100
200
40
Dropout
0.4
0.4
0.1
0.1
0.1
0.1
Weight decay
5e-5
5e-5
5e-5
lc-4
5e-5
5e-5
D. Memory Usage
Table 7 shows the GPU memory of ML---GLL and baselines including GRACE, GCA, AFGRL, NCLA and GTCA node classification task. The results indicate that ML2-GCL has less memory requirement compared with existing methods due to its closed-form solution of positive pairs weights and removal of pairwise distances calculation. Cross-view contrastive methods such as GRACE require additional storage for the embedding matrices of multiple augmented views, as well as the indices of positive and negative pairs, thereby increasing memory requirement.
Table GPU memory (GiBs) usage across datasets on node classification task. Best results are colored: first, second. third.
Model
Cora
Citeseer
Pubmed
Amazon-Phote Photo
Amazon-Compute
Wiki-CS
GRACE
0.47
0.55
13.17
2.11
6.23
4.46
GCA
0.58
0.69
14.32
2.34
6.61
5.01
AFGRL
0.37
0.48
12.59
2.12
5.94
4.41
NCLA
0.67
1.02
14.56
2.25
7.16
7.71
GTCA
1.64
2.64
22.03
4.09
13.14
10.89
MLL-GGL
0.33
0.45
12.15
2.01
5.74
4.22
We show part of the information from Table as a scatterplot to visualize the different scaling properties MI.--GLL and
14

---

MLA-GCL: Manifold Learning Inspired Lightweight Graph Contrastive Learning
25
NCLA
GTCA
20
ML3-GCL
Memory in GiB
2500
5000
7500
10000
12500 15000 17500 20000
Number of nodes
Figure 5. Memory usage of NCLA, GTCA and ML'-GCL across 6 standard datasets on node classification task.
NCLA, GTCA intuitively in Figure 5. Among the top 3 ranked methods on node classification task, ML consumes the least GPU memory usage. As the number of nodes increases, its memory usage increases slower compared with NCLA and GTCA.
15


Reviews:
[{"review_id": "HDajeOAAJJ", "reviewer": "Reviewer_6NFU", "summary": "As a mainstream and representative unsupervised learning method, contrastive learning has achieved great success in the field of computer vision. Inspired by such achievements, graph contrastive learning (GCL) has attracted much interests in the past few years. Despite its excellent performance, GCL suffers from underlying semantic disturbance, rigid and inefficient node sampling, etc. To address this issue, this paper develop ML2-GCL, a Manifold Learning Inspired\nLightweight Graph Contrastive Learning method. It is the first exploration to marry manifold learning with graph contrastive learning, which avoids the semantic disturbance and high computational complexity completely. Theoretical analysis and experimental results demonstrates its effectiveness and lightweight.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer’s positive feedback and valuable comments. Below, we will provide a point-by-point response to each comment.\n\nW1:  We have briefly discussed this in Introduction. Here, we will give a detailed discussion.\n\n**Similarities**\n\n**1. Consistency in Core Objectives**\n\nBoth aim to extract effective low-dimensional representations from complex data structures. Manifold learning reveals low-dimensional manifold structures embedded in high-dimensional data through dimensionality reduction, while graph contrastive learning learns discriminative embeddings through contrasting different graph structures or node relations.\n\n**2. Focus on Local Structures**\n\nManifold learning algorithms preserve local geometric properties through constructing neighborhood relations. Graph contrastive learning enhances sensitivity to local structures through contrasting node or subgraph neighborhoods.\n\n**3. Advantages in Handling Nonlinear Relations**\n\nManifold learning captures nonlinear low-dimensional structures of high-dimensional data. Graph contrastive learning excels at modeling non-Euclidean relations of graph data.\n\n**Differences**\n\n**1. Theoretical Foundations and Input Data Types**\n\nManifold learning is based on topological manifold theory, assuming high-dimensional data lies on low-dimensional manifolds. Its inputs are high-dimensional vectors, such as images and text. Graph contrastive learning is rooted in graph theory and contrastive learning frameworks. Its inputs are graph-structured data, such as nodes and edges.\n\n**2. Methodological Approaches**\n\nManifold learning reduces dimensionality via constructing local neighborhoods or global constraints, while graph contrastive learning generates positive/negative pairs and optimizes models using contrastive loss functions to distinguish similar/dissimilar structures. \n\n**3. Application Scenarios**\n\nManifold learning is primarily used for data visualization, denoising and feature extraction. Graph contrastive learning is applied to graph classification, node classification and graph generation, particularly in scenarios with missing or improvable graph structures .\n\n**4. Mathematical Tools and Optimization Goals**\n\nManifold learning relies on graph Laplacian matrices, geodesic distance calculations, and minimizes reconstruction errors or preserves local geometry. Graph contrastive learning uses contrastive loss to maximize similarity of positive pairs and minimize similarity of negative pairs, focusing on discriminability of representation spaces.\n\nW2: We will modify \"While existing works follow the basic principle that pulling positive pairs closer while pushing negative pairs far away...\" to \"While existing works follow the basic principle of pulling positive pairs closer and pushing negative pairs far away...\".\n\nW3: We will modify this in the final version.\n\nQ1: Yes, both of them are square matrices with dimension N.\n\nQ2: We observe that the optimal range of k values is generally small across all datasets, which can be primarily explained from the following two perspectives:\n\n**1. Theoretical Importance of Local Neighborhoods**\n\nIn ML²-GCL, k determines the scope of the positive pair set P_i, where the positive samples of an anchor node are derived from its k-hop neighbors. Theoretically, a smaller k makes positive pairs more localized, enhancing the preservation of local geometric structures while reducing noise interference from distant neighbors. Our optimization objective, based on manifold learning, constructs the positive pairs weight matrix W_p through locally linear embedding, making it more suitable for information propagation within small-scale neighborhoods. When k becomes excessively large, distant positive pairs may introduce excessive noise, compromising the fidelity of manifold structures and resulting in overly homogeneous representations that degrade contrastive learning performance.\n\n**2. Empirical Observations on k-Sensitivity**\n\nOur experiments reveal that larger k values lead to performance degradation and reduced model discriminability. This may occur because a larger k introduces excessive positive pairs, potentially amplifying noise and destabilizing local structures. When k increases, more nodes are included in the positive pair set. However, information from many of these nodes may already have been propagated through GCN layers, leading to information redundancy and diminished model discriminative power.\n\nBased on the above analysis, we conclude that smaller k values allow ML²-GCL to better preserve manifold-localized structures while avoiding over-smoothing issues, thereby delivering superior experimental performance.", "claims_and_evidence": "Yes, they are.", "methods_and_evaluation_criteria": "Yes, they do.", "theoretical_claims": "Yes, I did.", "experimental_designs_or_analyses": "Yes, I did.", "relation_to_broader_scientific_literature": "The application of lightweight graph contrastive learning in recommendation system has verified its universality. With lightweight representation of user product interaction graphs, it solves the problems of data sparsity and long tail recommendation.", "essential_references_not_discussed": "As far as I know, no key reference is missing.", "other_strengths_and_weaknesses": "Strengths:\n\n1. The idea is novel. This paper combines manifold learning with graph contrastive learning and proposes a lightweight design, which is in line with the current research trend of contrastive learning in reducing computational costs. \n\n2. The technique is reasonable. It designs a novel contrastive loss function with the closed-form solution of anchor node reconstruction combination weights to achieve both effectiveness and lightweight.\n\n3. Theoretical analysis proves the existence of the optimal closed-form solution. Extensive empirical studies on benchmark datasets demonstrate that the proposed method achieves state-of-the art performance in terms of effectiveness and lightweight. \n\nWeaknesses:\n1. To improve readability, authors are encouraged to discuss the similarities and differences between manifold learning and graph contrastive learning. \n\n2. Authors should carefully check and revise a few spelling and grammar mistakes. For example, \"While existing works follow the basic principle that pulling positive pairs closer while pushing negative pairs far away...\", where \" principle that pulling ... while pushing ...\" should make some adjustments.\n\n3. There are minor errors in Table 7. For the GPU memory usage on Amazon-Photo dataset, GRACE is the second least and AFGRL is the third least. The color is reversed.", "other_comments_or_suggestions": "See Weaknesses.", "questions_for_authors": "1. Are the positive pairs weight matrix Wp and negative pairs weight matrix Wn of equal size?\n\n2. From Tables 5-6, I observe that the optimal parameter k is relatively small. Could authors give more explanation?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NmVLVQ17VJ", "reviewer": "Reviewer_m6R5", "summary": "This paper explores an effective and lightweight graph contrastive learning method called ML^2-GCL, highlighting the need for a deeper understanding of graph contrastive learning methods from a manifold learning perspective. ML^2-GCL recovers global nonlinear structure from locally linear fits with closed-form solution of positive pairs weights, using which to design a novel contrastive loss function and update graph encoders. In addition, this paper proves the existence of the optimal closed-form solution and analyses the essence of ML^2-GCL. Extensive experiments verify both effectiveness and lightweight of ML^2-GCL.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the reviewer’s careful reading and valuable suggestion. In the following, we will provide a point-by-point response.\n\nWa: In future, we will investigate deep integration mechanisms between manifold learning and dynamic graph structures to address nonlinear evolution patterns in temporal data or dynamic networks. We can also extend such unsupervised contrastive paradigms to multimodal data for efficient computation in large-scale graph scenarios. The content will be supplemented upon acceptance of the paper.\n\nWb: The code will be made publicly available upon acceptance of the paper. For experimental details, please refer to Appendix C.\n\nCa: Thanks for the careful reading. We will modify this in the final version.\n\n**Cb: This may be caused by software incompatibility. You can try Adobe Acrobat 10.0 to open the PDF file.**\n\nQ: Thank you for the question. Our method, ML²-GCL, intentionally excludes the graph encoder when calculating the reconstruction weights for positive pairs. This is a deliberate design choice, primarily motivated by the following reasons:\n\n**1. Avoiding Encoder Interference to Ensure Geometric Consistency**\n\nTraditional GCL methods typically rely on embeddings from graph encoders to measure node similarity and construct positive/negative pairs. However, this approach is susceptible to instability caused by encoder initialization, training states, and parameter updates, which can lead to inconsistent positive pairs selection.\n\nIn ML²-GCL, the calculation of positive pair weights is based on locally linear embedding-based geometric optimization, directly constructing the positive pairs weight matrix W_p in the raw feature space. This avoids biases introduced by graph encoders, ensuring that W_p is solely determined by the inherent geometric structure of the data, aligning with the principles of manifold learning.\n\n**2. Enhancing Robustness of Contrastive Learning**\n\nComputing W_p in the raw feature space can serve as prior information for the encoder’s input data, providing stable supervisory signals. In contrast, computing W_p after GCN processing would risk instability, as updates to GCN parameters could disrupt the construction of positive pairs. Furthermore, positive pair selection should depend on the graph structure and node features themselves, rather than the encoder’s initial state.\n\n**3. Reducing Computational Complexity and Improving Efficiency**\n\nTraditional GCL methods require similarity calculations in the encoder’s embedding space, whereas ML²-GCL computes W_p via a closed-form solution, significantly lowering computational costs and enabling a lightweight framework. Our method also avoids exhaustive pairwise similarity computations by focusing on local geometric structures, further enhancing efficiency.\n\n**4. Ensuring Generalizability Across Graph Encoders**\n\nSince W_p is encoder-independent, ML²-GCL can flexibly integrate with various graph neural networks, such as GCN, GAT, and GraphSAGE, without altering the positive pair construction strategy. This design ensures broad applicability, allowing ML²-GCL to adapt to diverse GNN architectures seamlessly.\n\nIn summary, our design is grounded in manifold learning theory, ensuring stable positive pair construction, computational efficiency, and method generalizability. The experimental results validate the effectiveness of this approach, demonstrating its theoretical and practical soundness.", "claims_and_evidence": "Yes. This paper explores the marriage of manifold learning to graph contrastive learning for the first time, develops a new paradigm of effective and lightweight contrastive learning, and derives a series of theoretical analysis. These theoretical analysis proves the existence of the optimal closed-form solution of positive pairs reconstruction weights and reveals its connection with graph embedding, which provide strong evidence for the claim.", "methods_and_evaluation_criteria": "Yes. The proposed ML^2-GCL method is proper for the problem of GCL.", "theoretical_claims": "Yes. I checked the correctness of the proofs for theoretical claims, which proves the existence of the optimal closed-form solution and the correlation between ML^2-GCL and graph embedding.", "experimental_designs_or_analyses": "Yes. I checked the validity of the experimental designs and analyses, both of which are sound and complete.", "relation_to_broader_scientific_literature": "(a) Dynamic graph adaptation‌: Existing methods primarily focus on static graphs, necessitating exploration of temporal enhancement strategies‌.\n‌(b) Interpretability improvement‌: Lightweight model simplification may compromise semantic interpretability, requiring optimization through causal inference and related methods‌.", "essential_references_not_discussed": "No. This paper does not omit any essential references.", "other_strengths_and_weaknesses": "Strengths：\n(a) This paper proposes a novel and insightful method for GCL. By incorporating manifold learning theory, i.e., the manifold smoothness assumption, ML^2-GCL optimizes the geometric constraints of graph node embeddings and enhances the capability of low-dimensional representations to characterize complex graph structures. \n(b) By simplifying the contrastive pair generation process, ML^2-GCL reduces the computational complexity of traditional graph contrastive learning methods, especially those exploiting graph augmentation strategies.\nWeaknesses\n(a) The Conclusion Section lacks an exploration of potential future research directions.\n(b) Publicly releasing the code and experimental details may ensure the reproducibility of the results and enhance the impact of this work.", "other_comments_or_suggestions": "(a) Spelling errors: Line 31, \" lightweightness \" --> \" lightweight\".\n(b)  The mathematical expressions in the framework diagram are partially truncated, affecting the readability of key equations.", "questions_for_authors": "In general, graph encoding is the first step in GCL. However, in this paper, the computation process of positive pairs reconstruction weights does not include graph encoders, could authors explain why?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZWcTbVrvUk", "reviewer": "Reviewer_Ytzf", "summary": "Recent years have witnessed a phenomenon that graph contrastive learning faces the balance between effectiveness and efficiency. In spite of its popularity and success, several potential risks including underlying semantic disturbance brought by augmentation strategies, failure of GCN in capturing long-range dependence, rigidness and inefficiency of node sampling techniques  still remain to be solved. In this paper, authors provides a novel perspective for graph contrastive learning, called ML^2-GCL. It achieves global nonlinear structure recovery from locally linear fits, which can make up for the defects of GCN. The most amazing advantage is about the lightweight due to its closed-form solution of positive pairs weights and removal of pairwise distances calculation. A series of theoretical analysis and empirical performance completely demonstrate its superiority.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer’s positive feedback and attention. Here, we will provide a point-by-point response.\n\nW1: Thank you for your attention. In fact, the proposed novel contrastive loss can be regarded as personalized graph embedding, where positive pairs with larger weights should be more similar, while negative pairs with smaller weights should be more different.\n\nW2: In future, we will investigate deep integration mechanisms between manifold learning and dynamic graph structures to address nonlinear evolution patterns in temporal data or dynamic networks. We can also extend such unsupervised contrastive paradigms to multimodal data for efficient computation in large-scale graph scenarios. The content will be supplemented upon acceptance of the paper.\n\nC:  We will modify \"While existing works follow the basic principle that pulling positive pairs closer while pushing negative pairs far away...\" to \"While existing works follow the basic principle of pulling positive pairs closer and pushing negative pairs far away...\".\n\nQ: Personalized graph embedding is a more generalized framework which assigns each positive pair or negative pair its own weight. Traditional graph embedding can be regarded as a special form of personalized graph embedding. In Appendix A.3, we have briefly discussed the differences.", "claims_and_evidence": "Yes, the claims are supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "Yes", "experimental_designs_or_analyses": "Yes", "relation_to_broader_scientific_literature": "The cross-disciplinary innovation of lightweight graph contrastive learning with other technologies drives scientific literature development includes integration with traditional supervised learning and synergy with transfer learning. Specifically, in weakly supervised scenarios, contrastive loss compensates for the scarcity of annotated data. Cross-domain feature alignment enables knowledge transfer and model generalization.", "essential_references_not_discussed": "No, all the essential related works have been adequately discussed", "other_strengths_and_weaknesses": "1) Through introducing manifold geometric similarity metrics, such as local neighborhood preservation, the distributional consistency of positive and negative pairs in the latent space is improved. \n\n2) Removal of both graph augmentation and pairwise distances calculation can reduce the computational costs significantly, which aligns with the current demand for efficient graph models.\n\n3) Both theoretical analysis and empirical performance adequately verify the advantages in terms of effectiveness and lightweight.\n\n4) I've noticed that the proposed novel contrastive loss function uses the closed-form solution of anchor node reconstruction combination weights. Authors had better give more detailed explanations.\n\n5) In the Conclusion Section, future work is missing.", "other_comments_or_suggestions": "The authors should check some typos in Lines 16-17.", "questions_for_authors": "What are the differences between personalized graph embedding and traditional graph embedding?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cwifRHMs2k", "reviewer": "Reviewer_bpNM", "summary": "This paper proposes a lightweight graph contrastive learning (GCL) framework that integrates manifold learning theory, i.e., Manifold Learning Inspired Lightweight Graph Contrastive Learning (ML^2-GCL), aiming to optimize embedding representations through geometric structural constraints while reducing the computational complexity of traditional GCL methods. The method demonstrates innovative design, supported by rigorous theoretical analysis and extensive experimental validation.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer’s positive feedback and careful reading. Below, we will provide a point-by-point response.\n\nW1: See Q of **Reviewer m6R5**. \n\n**W2: This may be caused by software incompatibility. You can try Adobe Acrobat 10.0 to open the PDF file.**\n\nW3: We will modify this in the final version.\n\nC: We did consider using other types of graph encoders, such as Graph Transformers (GT). However, our decision to adopt GCN as the encoder was ultimately driven by the following factors:\n\n**1. Computational Efficiency and Model Lightweight**\n\nOne goal of this work is to develop a lightweight graph contrastive learning framework. Compared with Graph Transformers, GCN offers lower computational complexity and fewer parameters, making it easier to train on large-scale graph data while maintaining robust performance.\n\n**2. Compatibility with Manifold Learning**\n\nOur method is rooted in manifold learning, emphasizing the preservation of local linear relations. GCN’s hierarchical neighborhood aggregation mechanism inherently aligns with this design philosophy, enabling the learned embeddings to better retain local geometric information. In contrast, Graph Transformers primarily rely on global attention, which, while effective for capturing long-range dependencies, may dilute the fidelity of local geometric structures.\n\n**3. Experimental Results and Comparability**\n\nTo ensure fair comparisons with mainstream GCL methods, we adopted the same encoder setup. Since most baseline models use GCN, maintaining consistency ensures experimental comparability and highlights the advantages of our framework itself.\n\nEven so, we acknowledge the potential of Graph Transformers in modeling long-range dependencies. Future work could explore their integration into the ML²-GCL framework to further enhance global structural modeling capabilities.\n\nQ: The lightweight modeling in this paper primarily relies on the following core technologies and solutions, which reduce computational complexity while ensuring model effectiveness and generalization capability:\n\n**1. Manifold Learning-Driven Positive Pair Weight Calculation**\n\nWe compute the positive pairs weight matrix W_p with locally linear embedding, enabling direct learning of global nonlinear representations on the original graph structure without pairwise distance computation. \n\nThis strategy applies to other contrastive learning tasks requiring structural preservation, such as text and protein interaction networks. Additionally, it can be used for dimensionality reduction tasks, e.g., manifold-based image feature extraction.\n\n**2. Closed-Form Solution Optimization**\n\nWe solve the positive pairs weight matrix W_p through a closed-form solution, which significantly reduces computational complexity compared with gradient descent-based optimization methods, making the approach more lightweight.\nThis closed-form solution can be generalized to unsupervised dimensionality reduction, spectral clustering, and other tasks, particularly suitable for large-scale datasets such as graph neural network pretraining.\n\n**3. Single-View Modeling Avoiding Data Augmentation**\n\nTraditional graph contrastive learning relies on data augmentation to generate multiple views. This study implements contrastive learning directly on a single view, reducing storage requirements while eliminating semantic bias introduced by augmentation.\n\nThis method applies to scenarios where data augmentation is difficult to define, such as social network analysis and biological graph applications, where complex data structures pose challenges in generating high-quality augmented views.\n\nThe lightweight design in this study is not task-specific but based on rational improvements in optimization strategies, sampling methods, and modeling approaches. It can thus be generalized to other graph learning tasks, visual contrastive learning, and embedding learning in natural language processing. We believe these strategies can inspire the development of more efficient and scalable unsupervised learning methods.", "claims_and_evidence": "The claims are well-supported by both theoretical analysis and extensive experiments.", "methods_and_evaluation_criteria": "The proposed ML^2-GCL method is both effective and lightweight for the problem of graph contrastive learning.", "theoretical_claims": "I checked the correctness of the proof for Propositions 1-2 and Theorem 1.", "experimental_designs_or_analyses": "I checked the soundness of the experimental designs and analyses.", "relation_to_broader_scientific_literature": "The application of lightweight graph contrastive learning across multiple scientific domains demonstrates its versatility. For bioinformatics, this paper may integrate heterogeneous graph structures of multi-omics networks for gene-protein association prediction. For knowledge graphs, this paper can combine with knowledge-enhanced graph convolutional networks to enhance cross-entity reasoning accuracy.", "essential_references_not_discussed": "To my best knowledge, no essential related works are missing.", "other_strengths_and_weaknesses": "Strengths:\n1. This approach aligns with recent trends in geometric property modeling of graph structures, emphasizing the intrinsic low-dimensional manifold characteristics of high-dimensional data. To my best knowledge, ML^2-GCL is the first to marry manifold learning with graph contrastive learning.\n2. This paper has a clear research motivation and logical structure. It is well-written and easy to follow.\n3. Solid theoretical analysis and extensive experimental results demonstrate the effectiveness and light weight of ML^2 -GCL.\n\nWeaknesses:\n1. The first step of ML^2-GCL is the neighboring sampling, which includes anchor node reconstruction combination weights, and then the authors apply the graph encoder. Such procedure is different from general GCL. The authors should give more detailed and reasonable explanation.\n2. There are many numbers or symbols in the charts that are not displayed correctly. The author should check and modify them carefully.\n3. There are some suggestions for grammar revision and improvement:\n- Sentence structure issues: Line 16, \"basic principle that \" --> \" basic principle of\", Line 17, \"while pushing negative pairs\" --> \"and pushing negative pairs\".\n- Inconsistent parallel structures: Line 20, \"failure of\" --> \"the failure of\", Line 21, \"rigidness\" --> \"and the rigidness\"", "other_comments_or_suggestions": "Have the authors tried other types of models as graph encoders, such as Graph Transformers?", "questions_for_authors": "What are the core technologies or solutions that play a critical role in modeling lightweight in this paper? Could these technologies or solutions be applied to other tasks?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes Manifold Learning Inspired Lightweight Graph Contrastive Learning, which is well-structured and easy to understand. The reviewers generally acknowledge the paper's theoretical analysis and experimental validations. Several minor concerns have been also raised regarding grammatical issues, spelling errors and the lack of analysis on the neighborhood sampling method. I encourage the authors to address these comments in the final revision. I am confident that this paper can be accepted.
Decision: Accept (poster)