Mitigating Local Cohesion and Global Sparseness in Graph Contrastive Learning with Fuzzy Boundaries
Authors: Yuena Lin
Haichun Cai
Jun-Yi Hang
Haobo Wang
Zhen Yang
Gengyu Lyu
Venue: ICML 2025 poster (2025)
Keywords: Graph contrastive learning
Classification
Clustering
Primary area: general_machine_learning->representation_learning

Abstract: Graph contrastive learning (GCL) aims at narrowing positives while dispersing negatives, often causing a minority of samples with great similarities to gather as a small group. It results in two latent shortcomings in GCL: 1) **local cohesion** that a class cluster contains numerous independent small groups, and 2) **global sparseness** that these small groups (or isolated samples) dispersedly distribute among all clusters. These shortcomings make the learned distribution *only focus on local similarities among partial samples, which hinders the ability to capture the ideal global structural properties among real clusters, especially high intra-cluster compactness and inter-cluster separateness*. Considering this, we design a novel fuzzy boundary by extending the original cluster boundary with fuzzy set theory, which involves fuzzy boundary construction and fuzzy boundary contraction to address these shortcomings. The fuzzy boundary construction dilates the original boundaries to bridge the local groups, and the fuzzy boundary contraction forces the dispersed samples or groups within the fuzzy boundary to gather tightly, jointly mitigating local cohesion and global sparseness while forming the ideal global structural distribution. Extensive experiments demonstrate that a graph auto-encoder with the fuzzy boundary significantly outperforms current state-of-the-art GCL models in both downstream tasks and quantitative analysis.

Mitigating Local Cohesion and Global Sparseness in Graph Contrastive Learning with Fuzzy Boundaries
Yuena Lin"! "1 Haichun Cai 22
Yun-Yi Hang² Haobo Wang*
Zhen Yang Gengyu Lyu" 1
Abstract
1. Introduction
Recent years have seen the rapid development of self- supervised graph representation learning (SSGRL) (You et al., 2021), which expects to extract critical features by mining intrinsic characteristics from raw graph data, such as commonalities and discriminalities (Wang & Liu, 2022; Jiang et al., 2021). Graph contrastive learning (GCL) as one of the most well-known SSGRL paradigms, learns discrimi- nalities by dispersing negatives and mines commonalities by closing the positives (Xu et al., 2024a; Zou & Liu, 2023). Unfortunately, the learned distribution in the embedding space only reflects local similarities within each cluster, and fails to showcase global structural distribution with high intra-cluster compactness and inter-cluster separateness. As a matter of fact, the problem can be traced to two in- grained shortcomings in GCL, namely local cohesion and global sparseness.
Graph contrastive learning (GCL) aims at nar- rowing positives while dispersing negatives, often causing minority of samples with great similari- ties to gather as small group. It results in two la- tent shortcomings in GCL: 1) local cohesion that a class cluster contains numerous independent small groups, and 2) global sparseness that these small groups (or isolated samples) dispersedly distribute among all clusters. These shortcom- ings make the learned distribution only focus on local similarities among partial samples, which hinders the ability to capture the ideal global structural properties among real clusters, espe- cially high intra-cluster compactness and inter- cluster separateness. Considering this, we design a novel fuzzy boundary by extending the origi- nal cluster boundary with fuzzy set theory, which involves fuzzy boundary construction and fuzzy boundary contraction to address these shortcom- ings. The fuzzy boundary construction dilates the original boundaries to bridge the local groups, and the fuzzy boundary contraction forces the dis- persed samples or groups within the fuzzy bound- ary to gather tightly, jointly mitigating local co- hesion and global sparseness while forming the ideal global structural distribution (Zhang et al., 2024). Extensive experiments demonstrate that a graph auto-encoder with the fuzzy boundary significantly outperforms current state-of-the-art GCL models in both downstream tasks and quan- titative analysis.
To shed light the shortcomings intuitively, we present the t-SNE visualization (Van der Maaten & Hinton, 2008) from three cornerstone models in Figure 1, where DGI (Velick- ovic et al., 2019) views nodes in the raw graph as positives and those in the corrupt graph are negatives; GRACE (Zhu et al., 2020) treats corresponding nodes between two con- gruent views as positives while other nodes are negatives; BGRL (Thakoor al., 2022) selects positives in the same way GRACE but scatters the negatives in a latent fash- ion. The t-SNE of these models reveals the two common shortcomings in GCL: 1) Local cohesion results in numer- ous small groups. The sample selection strategies in GCL tend to gather a minority of samples with great similarities, which leads to numerous independent small groups in real cluster. 2) Global sparseness across cluster distributions. These small groups (or isolated samples) are discretely dis- tributed in each cluster, which only occupy part locations while leaving considerable blank space in each cluster. The crux of the shorteomings may be attributed to the fact that existing GCL models have no access to label supervision for building reliable boundaries. Unreliable boundaries, such as excessive boundaries that incur samples from other clusters, or limited boundaries that are unable to involve sufficient samples from the same cluster, are harmful to the structural distribution (Chen et al., 2023).
Equul contribution 'College of Computer Seience, Beijing University of Technology, Beijing, China 'School of Computer and Data Science, Fuzhou University. Fuzhou, China 'School of Computer Science and Engineering, Southeast University, Nan- jing. China "School of Software Technology, Zhejiang Univer- sity, Hangzhou, China. Correspondence to: Gengyu Lyu <lyu- gengyu bjut.edu.cn.>
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 20255 by the author(s).

---

Title Suppressed Due to Excessive Size
(a) DGI
(b) GRACE
(c) BGRL
(d) Ours
Figure 1. I-SNE for our model and three representative GCL models on the Citeseer dataset, i.e. DGI, GRACE, and BGRL. The boundaries for green and blue clusters are outlined. The two clusters exhibit global sparseness and local cohesion that forms numerous small groups (shown red circles). Besides, near the boundary, the samples from the two clusters are interwoven together.
Boundaries are closely related to the aforementioned short- comings but are seldom taken into consideration in GCL. Considering this, we design a novel learnable boundary called fuzzy boundary by fusing fuzzy set theory to ex- tend the original clusters for addressing the local cohesion and global sparseness. The fuzzy boundary is concerned with fuzzy representations and crisp representations, where the fuzzy representations are the learned representations applied with fuzzy set theory, or crisp representations oth- erwise. The proposed fuzzy boundary includes the fuzzy boundary construction phase and the fuzzy boundary con- traction phase. In the fuzzy boundary construction phase, transform the crisp representation space to the fuzzy rep- resentation space with fuzzy set theory, where we expand the original cluster boundaries as the new-born fuzzy bound- aries to further bridge the local groups through learning the fuzzy positives of crisp representations. When turning the fuzzy boundary contraction phase, we narrow the fuzzy boundaries and the corresponding cluster prototypes while pushing the fuzzy boundaries away from other cluster proto- types, effectively tightening the dispersed samples groups within the fuzzy boundaries. Finally, the local groups or isolated samples within each cluster become interconnected and the ideal global structural distribution with high intra- cluster compactness and inter-cluster separateness gradually emerges, forming more distinct boundaries. Our contribu- tions summarized as follows.
We implement the fuzzy boundary on a pre-trained graph auto-encoder (GAE), which makes the model performance enhanced and comparable with current otate-of-the-art SSGRL models in the downstream tasks and quantitative analysis.
2. Related Work
2.1. Fuzzy Set Theory
Fuzzy set theory generalizes classical set theory that deter- mines whether an object belongs to set, and incorporates the idea that elements have membership degrees in a set (Zadeh, 1965). Here, we introduce the critical definitions of the fuzzy set theory in the following:
Definition 2.1. universe U is defined as a set of all possi- ble terms occurring in a certain domain.
Definition 2.2. function fA: - LL R is called membership function.
Definition 2.3. A pair composed of the universe and the membership function A = (U, f is called a fuzzy set.
A fuzzy set A in the universe U is characterized by a mem- bership function fA (x) which associates each point in U with a real value in [o, 1]. The value of f.A (x) represents the membership degree vides a powerful framework for reasoning about sets with uncertainty, but the specification of membership functions depends on the domain that the objective task is located in (Zhelezniak et al., 2018 (0. Following the definitions, when L = [0, 1], A is a fuzzy set. When the membership func- tion takes on only two values and 1, namely L = f0, 1] according to whether x belongs to A, in this case, A reduces to a classical set, also called crisp set.
We first attend to two ingrained shortcomings causing the inferior global structural distribution in GCL: local cohesion and global sparseness, and disclose that these shortcomings suffer from unreliable boundaries.
A. The fuzzy set theory pro-
We propose a novel fuzzy boundary to bridge the dis- crete local groups and isolated samples, and tighten each cluster, which effectively alleviates the local co- hesion and global sparseness, and realizes ideal global structural distributions.
2

---

Title Suppressed Due to Excessive Size
Pre training Phase
Reconstruction Loss Fec
Graph Data g = (X, A)
Representation
Reconstructed Feature 8
GNN Encoder
MLP Decoder
K- K-means Clustering
Prototype Contrastive Loss Ero
Crisp Boundary
Fuzzy Boundary
Fine Tuning Phase
Push Away
Pull Together
Mapping
Prototypes
Crisp Representaions
Boundary Elements
Real Clusters
Isolated Samples
Clustering
Fuzzy Boundary Construction
Fuzzy Boundary
Local Groups
Contraction
Figure 2. Overview the proposed model. Firstly, the GNN encoder pre-trained with the reconstruction loss Crees Secondly, the output representation H is used to obtain real clusters through the K-means clustering algorithm. Then the original boundaries (in solid lines) are extended as fuzzy boundaries (in dashed lines) to incorporate isolated samples and local groups. Finally, fuzzy boundaries are contracted tighten samples within the fuzzy boundaries while separating different clusters by prototypical contrastive loss Eyro.
2.2. Self-Supervised Graph Representation Learning
3. Method
Exempt from the supervision downstream tasks, self- supervised graph representation learning (SSGRL) focuses extracting generalized and invariant features from the raw graph data. Early works were mainly based on random walk, which only took structure information into considera- tion. Subsequent works integrated feature information and turned to deep learning methods because of their tremen- dous fitting power to approximate complicated functions (L.v al., 2024). Deep SSGRL models are roughly categorized into GAEs (Hou al., 2022), graph generative adversarial networks (Wang et al., 2019), and graph contrastive mod- els. Among them, graph contrastive models stand out for the simple-yet-effeective concept that narrows positive pairs while dispersing negative pairs to maximize the lower bound of mutual information (Hu et al., 2024; Zhang et al., 2023a). Despite showing competitive performance, existing GCL models still exhibit two shortcomings, i.e., local cohesion and global sparseness, which cause inferior global structural distribution but are never addressed from the perspective of boundaries. Actually, good boundaries alleviate these short- comings and form a reliable global structural distribution, which becomes the motivation of the proposed model.
The proposed model includes a pre-training phase and fine-tuning phase, where the pre-training phase includes a GAE framework train GNN encoder, and the fine-tuning phase contains the fuzzy boundary construction process and the fuzzy boundary contraction process for fine-tuning the GNN encoder. The output representation H of the pre- trained GNN is implemented on the K-means clustering algorithm to attain reliable real clusters, which is used in the fine-tuning phase. The whole architecture shown in Figure provide details the model for convenience, we introduce the following notations.
Notations. In this paper, graph is denoted = (V,E), where y = {11, u2, .NN } is the node set with N nodes and c V x V is the edge set. The node attribute matrix and adjacency matrix are denoted as X E N and A E (0, 13 respectively. x, represents the attribute of node 04 and A = describes a relation between Y; and uj, or Aij = 0 otherwise. Our purpose is to train an L-layer graph encoder f TR N x {0, 1) NxN - N to produce low-dimensional node representations via raw attribute and structure contents, where dj the dimension of the L-th layer, D is the dimension of the raw attribute and d, < D.
3

---

Title Suppressed Due to Excessive Size
3.1. Pre-Traine Pre-Training Process
Zz
Prototype Representation
Fuzzy Representation
To obtain relatively reliable clusters for constructing the fuzzy boundaries, we adopt GAE (Thomas & Welling, 2016) for pre-training. During the pre-training process, we reconstruct the attribute matrix with O (ND) complexity instead of the adjacency matrix with O (N²) complexity, which is defined as
Boundary Element
((2 1,,2,,,, (2))) 52)22)
(1 8, 12)
(z 2z)
(1 322
H GNN(N, A). (H) Cree
X X- X
(3 3272)
where X is the reconstructed attribute matrix, and |1-1|p is F-norm. Since D < N holds in general, restoring the embeddings to the raw attributes saves much more over- head than reconstructing the adjacency matrix. To avoid the over-smoothing problems caused by numerous GNN layers (Rong et al., 2019), we employ an asymmetric GAE back- bone where GNNs and multi-layer perceptrons (MLPs) are implemented on encoding and decoding processes respec- tively as (Xiao et al., 2022; 2023). Specifically, we adopt graph convolutional network (GCN) (Kipf& & Welling, 2017; Xu et al., 2023) as the GNN encoder, and the encoding and decoding processes are given by
Z
Figure 3. The construction process of fuzzy boundary in 2- dimension space. For specific representation O it is firstly utilized to calculate the fuzzy representation C which is then mapped to be the element of the fuzzy boundary
is utilized to find the appropriate elements of the fuzzy boundaries. These elements are expected to have two essen- tial characteristics consistent with fuzzy representations. i) comantics-positiin Since the constructed fuzzy boundary is assumed as critical part of one real cluster, the elements scatted in the fuzzy boundary ought to share the common- alities with the samples within the real cluster, namely the elements of the fuzzy boundaries are the positives of the cluster samples. ii) Learnable. The fuzzy boundaries rely on the learnable membership functions, making the fuzzy boundaries adaptive to regulate the spectrum of a cluster in the training process. Among various membership functions, the Gaussian membership function is the most widely used (Zhang et al., 2023b; Li et al., 2024).
$ H 11--1) ww" + +b(). ww++b H = D 1 AD
( (t + b(().
where H and H99) are the output representation matrices from the 1-th layer encoder and the 1-th layer decoder re- spectively, W(),, W((), b( and (t) are trainable weight matrices or bias vectors, D is a diagonal matrix with diag- onal element Di = E, A,g. A = A + IN. a (.) is the non-linear activation function, and it is not implemented on the last layer of the MLP decoder.
Generally, one Gaussian membership function is in accor- dance with one specific fuzzy semantics, and in this paper, we set K membership functions to enrich the fuzzy seman- ties of the representations. For each value of one feature vector, its k-th membership degree computed by
The pre-training process unleashes the capability of the graph model to mine commonalities and discriminalities, and encourages to generate relatively reliable clusters for constructing the fuzzy boundary. For convenience, we cen- tralize all the node representations by subtracting their cor- responding prototypes C as Z, and utilize Z to construct the fuzzy boundaries, where the corresponding prototypes C are obtained by applying the K-means clustering algorithm on the output representation H of the GNN encoder. To save the costs on the clustering process, we execute the K-means clustering algorithm every 7 epochs.
{ Sh. exp {
(Zi.s m 2 20t)
. = 1,2, .d,
where Zi,j is the j-th value of the centralized feature vector z; and s is corresponding membership degree, is the dimension of z, m² and 85 are the learnable mean and stan- dard deviation of the k-th Gaussian membership function respectively. These membership degrees are then unified by Union operator to output the final one as
3.2. Fuzzy Boundary
3.2. 1. FUZZY BOUNDARY CONSTRUCTION
The proposed model transfers the fundamental concept that a crisp set can be viewed as a special type of fuzzy set into representation learning, where a crisp representation is re- garded as particular type of fuzzy representation, which
$ = Union (st.) ) } = 1,2, K.
(4)
The choice of the Union operator is explained in Section 3.2.2, and the final membership degrees from the Union
4

---

Title Suppressed Due to Excessive Size
operator define the fuzzy representations as
to the fuzzy representation defined in Eq. (5), when we use the ordinary distance function Dis (-) such as Fuclidean distance to measure the distances between one crisp repre- sentation and the corresponding fuzzy representation, we have the following conclusion as
-= S 3 Z.
(5)
where S is the membership degree matrix, Z is the central- ized representation matrix, and Z is the fuzzy representa- tion matrix that rescales the crisp representations matrix Z via the adaptive membership degree matrix S. Crisp rep- resentations and fuzzy representations are assumed to be positives, thus the coordinate difference between them is deemed acceptable semantics difference between positives. Considering this, we employ the acceptable discrepancy to expand the cluster boundary, and the process is called fuzzy boundary construction, which is shown in Figure 3. The calculated coordinates of fuzzy representations, obtained through Eq. (5), are utilized to compute the corresponding boundary elements by
Dis ((šmin); ; O 2,, Z,) 2 Dis ([5other) Ji C Zi, zi)
2 Dis ([Smar); ① Z Z,):
where {8min]: J is the membership degree vector of node vi which is filled with minimal membership degrees, and its element is (8min Ja.j = ming (st.,) Similarly, we have (Smar)i, = max ( {Suther]; denotes other simple Union operators, such as mean-pooling. This inequation means that the max-pooling is in line with searching for the representations with maximal similarities in the fuzzy representation space, and also implies that samples inside the constructed fuzzy boundary are likely to be part of this class cluster with great (membership) degrees.
- 2 OZ Z- 2
S
Z,
where and 2 are two real value matrices whose each element is I and 2 respectively, and Z contains the elements of the fuzzy boundary. It is worth noting that the fuzzy boundary won't 't deteriorate into the crisp boundary, and it effectively connects the independent groups and isolated samples to mitigate the local cohesion.
3.2.3. FUZZY BOUNDARY CONTRACTION
During the fuzzy boundary contraction phase, it is hard to achieve the inter-cluster separateness in a centralized representation space since the centralized clusters are con- centrated around the origin points, so we need to restore the coordinates of fuzzy representations as H by adding their corresponding prototypes C. Considering the restored fuzzy boundaries correspond to new-born fuzzy prototypes F, and we need to balance C and F to obtain more ideal prototypes. A naive method like the convex combination to fuse the fuzzy prototype and the crisp prototype is always suboptimal in practice, and we apply an MLP (-) to better balance the relationship of both as
Theorem 3.1. Assuming that N samples have different features, K Gaussian membership functions have different means, and K < N, fuzzy boundaries never deteriorate into crisp boundaries.
Proof: For Gaussian membership functions, the membership degree is only when the feature element equals the mean. Suppose S = which implies that the feature element must equal the mean of the Gaussian function. However, since the features of the N samples are all different and the means of the K Gaussian functions are also different, the means of the K Gaussian functions can match at most K feature elements. Therefore, it is impossible to obtain S = and the fuzzy boundaries won't 't deteriorate.
P = 9 C),
(8)
where P E &p is the fused prototype matrix, Cp is the number of clusters, Ce E RR,xd is the crisp prototype matrix obtained by applying clustering algorithms on rep- resentation H. F RC,xd is the fuzzy prototype matrix by averaging the fuzzy boundary elements for each cluster, and || is the concatenation operator. To densify the cluster distribution and differentiate different clusters, we apply one prototype loss between the prototypes and fuzzy boundaries, which defined as
3.2.2. UNION OPERATOR DETERMINATION
The fuzzy boundary elements are considered the critical part of the corresponding cluster, and the variances of the extended fuzzy clusters could reflect the compactness of the real clusters. Thus, the Union operator is crucial to adjust the cluster variances for realizing the intra-cluster compactness and inter-cluster separateness.
N log
[. P.
1
Cyro
Theorem 3.2. When Union operator adopts max-pooling, the overall variances within each cluster are minimized,
where P, is the fused prototype that node v; belongs to and is computed by Eq. (8). h, is the restored fuzzy bound- ary elements, 7 is the temperature coefficient, and higher r weighs the intra-cluster compactness more. Through con- tracting each cluster, the samples or local groups in the
The detailed proof is provided in Appendix A.1. Addition- ally, the Union operator has another advantage, namely the distances between crisp representations and fuzzy represen- tations are minimal in the representation space. According
5

---

Title Suppressed Due to Excessive Size
Table Results node classification. The best results highlighted in bold, and the second-hest results highlighted with underline.
"OOM" indicates that the graph models raise the memmoor failure.
Amazon Photo
Amazon
Dataset
Cora
Citeseer
Pubmed
DBLP
Ogbn-arxiv
Computers
DeepWalk
74.6 0.4
50.8 # 0.1
80.1 0.2
7..0
63.6 = 0,4
89.4 0.1
85.7 t 0.1
DGI
0.7
86.0
0.1
67.9 0.4
91.6 0.2
84.0 0.5
GMI
0.8
0.8
0.1
8.2
6.2
9.2
0.3
GRACE
0.4
0.5
8.17
8.12
6.4
9.5
0.4
GCA
0.3
0.3
8..0
83.1
6..2
9.2
87.5
COAST
8.2
0.3
8.2
84.5
62.1
9.5
88.3
AFGRL
0.9
7.
8.22
83.1
56.1
93.
8.3
SUGRL
0.6
7.6
8..7
8..
6.07
9..
0.2
iGCL
0.5
7.9
8..2
8.27
6.0
93.1
90.1
NCLA
8.4
0.5
8.45
8.2
58.4
93.5
GREET
85.7
0.6
8..9
83.8
0OM
9.3
87.9 0.4
HomoGCL
8,
0.4
8.2
8..4
0.5
9.2
0.5
S2GAE
0.3
0.5
8..
84.1
60.9
9.2
90.0 0.1
SGRL
8..
0.22
8..2
84.1
65.7
9.,8
9.0
PiGCL
8.
7.65
8.3
0.3
0OM
9..1
89.3 2 0.3
Bandana
0.8
0.8
8.34
83.9
6.0
9..4
89.6 0.1
Ours
0.6
74.1 0.6
0.3
8.2
0.1
0.2
90.8 0.2
cluster gather more tightly, which alleviates local cohesion and global sparseness.
and node clustering to demonstrate the superiority of our model. All experiments are implemented in PyTorch and conducted on a server with RTX 3090.
The overall loss function C composed of prototypical loss and reconstruction loss is defined
4.1. Node Classification
L = Lyoo + a.Crre:
(10)
In the node classification task, we encode the raw graph to obtain corresponding node representations and use a linear classifier to examine the quality of representations. We re- peat the experiments 10 times to obtain stable classification results, and mean accuracies with standard deviations listed in Table 1.
where a is the penalty coefficient. The overall loss function expects that the representations reserve more critical infor- mation restore the raw data, and the cluster distributions in the representation space realize intra-cluster compactness and inter-cluster separateness at the same time.
Theorem 3.3. The GAE with the fuzzy boundary achieves higher generalization than vanilla one.
From the results, we find that the proposed model is com- petitive or surpasses other advanced models across all the datasets. Particularly, our model shows 0.6%, 0.8%, and 0.7% improvements on Citeseer dataset, Ogbn-Arxin dataset, and Amazon Computers dataset, respectively. Be- sides, have the following observations:
We provide detailed proof to explain how the fuzzy boundary enhances the model generalization in Appendix A.2.
4. Experiments
(1) Compactness and separateness affect the model perfor- mance jointly. Several GCL models without dispersing negatives explicitly, such as AFGRI. and iGCL., concen- trate on aligning positives to realize compactness. However, they exhibit inferior performance when compared with other models on most datasets, which means separateness also plays significant role in representation learning.
In this section, seven public graph datasets are used for testing the model performance, which ranges from citation networks to co-purchase networks. To show the effective- ness of our model, we compare with numerous advanced self-supervised graph models proposed recently, i.e., Deep- Walk (Perozzi al., 2014). DGI (Velickovic al., 2019), GMI (Peng al., 2020), GRACE (Zhu al., 2020), GCA (Zhu et al., 2021), COAST (Zhang al., 2022), SUGRL (Mo et al., 2022), AFGRL (Leee et al., 2022), iGCL (Li et al., 2023a), NCLA (Shen et al., 2023), GREET (Liu et al., 2023), HomoGCL (Li et al., 2023b), PiGCL (He et al., 2024b), and SGRL (He et al., 2024a). We employ two common downstream tasks, namely node classification
(2) Global structural distributions with high intra-class com- pactness and inter-class separateness are casily delimited by our model. Existing GCL models realize compactness and separateness by closing positives and dispersing negatives, but these properties not well reflected by the cluster distri- bution are likely to limit the model performance. As the
6

---

Title Suppressed Due to Excessive Size
Table Results in node clustering. The best results highlighted in hold, and the second-hest results are highlighted with underline.
Datasets
Metries
iGCL
GREET
HomoGCL
SGRL
PiGCL
Ours
Acc
68.56 + 2.16
71.83 3,40
70.68 3.85
64.76 0.04
64.68 2.64
72.85 + 0.02
MaF1
62.65 = 3.31
64.77 + 5.31
66.10 5.87
63.82 0.02
61.45 # 3.55
71,50 = 0.01
Cora
NMI
51.28 = 1.34
55,41 1.16
56,44 + 1.77
48.28 # 0.04
51.87 = 1.78
54.71 = 0.01
ART
45.39 2.05
50,42 + 4.32
49.97 + 2.85
39.83 # 0.07
42.84 # 2.68
50.11 0.01
Acc
6.51
68.78
65.65 + 0.83
64.48 # 0.05
63.45 # 1.32
70.25 = 0.09
MaF1
61.61 = 3.67
63.57
61.65
61.97 0.05
61.10 1.00
65.10 = 0.05
Citeseer
NMI
39,42 = 0.41
43.62
3
39.67 2
39.71 0.69
44.99 + 0.03
ARI
40.03 0.45
44.57
39.73 0.088
43.65 0.24
39.97 0
46.68 + 0.09
Acc
67.72 3.67
#8.96
74.30 1.69
53.39 1.47
58.31
76.71 = 0.06
Amazon
MaF1
63.88
55.51 1.74
69.03 +
49.49 + 2.13
$
75.85 + 0.82
Photo
NMI
59.87
50.35
64.21 2.11
46.81 # 1.51
47.48
67.72 + 1.22
ARI
45.65 3.40
38.76
54,95 +
27.44 2.51
30.51 2.42
57.84 + 1.11
Acc
$3.49
$1.27
48.75 0.96
41.88 + 0.68
55.04 + 0.78
Amazon
MaF1
48.35 + 3.03
0.36
4.71
46.05 + 0.64
33.75 # 0.89
48.79 = 0.61
Computers
NMI
49.87 = 1.30
46.66
42.53 2.61
45.35 1.27
40.94 = 0.39
56.29 = 0.62
ART
29.75 # 1.70
+
26.91 2.59
31.11 1.27
28.01 = 0.42
36.38 0.78
classifier is a simple linear classifier, its performance heavily depends on the qualities of global structural distributions.
4.3. Quantitative Analysis
In this experiment, we propose two specific metrics, "Mean Groups" and "Total Isolated Nodes" for evaluating the model's ability to address local cohesion and global sparse- ness. Additionally, we present the "Group List" and the "List of Isolated Nodes" to help understand these two met- rics. The specific explanations and definitions of these met- ries provided in the Appendix B.3. "Group List" and "Mean Groups" reflect the local cohesion, where "Group List" details the number of independent groups in each class cluster and "Mean Groups" calculates the mean of groups from all class clusters. The value of "Mean Groups" " ap- proaches I is more expected. "List Isolated Nodes", "To- tal Isolated Nodes", and "Mean Groups" reflect global sparseness jointly, where "List of Isolated Nodes" details the number of isolated nodes for each class cluster, and "To- tal Isolated Nodes" records the total number of the isolated nodes for each dataset. The value of "Total Isolated Nodes" approaches is more expected.
In summary, the success of our model contributes to over- coming the two shortcomings, namely local cohesion and global sparseness, which form the ideal global structural distributions facilitate the downstream tasks.
4.2. Node Clustering
The clustering task is also common task to evaluate the model performance (Xu et al., 2022; 2024b), where we apply the K-means clustering algorithm to examine the en- coded node representations. We repeat the experiments 20 times to harvest reliable outcomes, and adopt the commonly- used accuracy (Acc), normalized mutual information (NMI), adjusted rand index (ARI), and macro-F1 (MaF1) metrics evaluate the clustering performance. Table displays the result comparisons between our proposed model and other advanced graph models.
Similar to node classification, the proposed model almost re- ceives the best outcomes across four datasets, and exhibits wonderful Ace and MaFl particularly. It illustrates that pro- duced node representations of our model are clustered more accurately, and implies that the intra-cluster compactness of the model is indeed effective, namely the samples in one cluster are more likely to share the same label. Besides, the proposed model behaves well in both classification and clustering tasks, which implies that the proposed model able to generate satisfactory cluster distributions in the embedding space.
For the experiment setting, we apply DBSCAN (Ester et al., 1996) on the produced node representations collect the number of independent groups and isolated nodes for each class, and the results are displayed in Table 3. "Eps" is hyperparameter of DBSCAN that limits the maximal dis- tances between the representations a detected local group. Here, we execute the DBSCAN algorithm with Eps = 0.5 and Eps 0.7. In general, when "Eps" becomes greater, DBSCAN gets more expected outcomes, namely smaller "Mean Groups" and "Total Isolated Nodes".
The results in Table 3 strongly demonstrate the existence
7

---

Title Suppressed Due to Excessive Size
Table Quantitative analysis evaluate global sparseness and local cohesion. The minimal Mean Groups (MGs) and Total Isolated
Nodes (TINs) are highlighted in bold.
Datasets
Eps
Models
Group List
MGs
List Isolated Nodes
TINs
iGCL
[14, 42, 43, 54, 53, 53]
43.2
[204, 403, 439, 421, 358, 278]
2,103
GREET
[8, 9, 12, 21, 15, 12]
12.8
[223, 540, 603, 592, 517, 440]
2,915
HomoGCL
[16, 49, 25, 49, 4944,,411 42,
37.0
[187, 259, 247, 271, 167, 188]
1,319
0.5
SGRL
[13. 23.21.8, 14.
12.2
[123, 141, 112, 149, 89)
676
PIGCL
8,9,9,98
6,8
[90, 72, 74, 89, 53, 66]
444
Ours
35.
3.8
[24, 24, 14, 11,23]
110
Citeseer
iGCL
[17, 43,
39.2
[183, 252, 258, 258, 167, 164]
1,282
GREET
28, 25]
26.0
[179, 397, 473, 437, 341, 316]
2,143
HomoGCL
[20, 34, 19, 34, 18, 34]
26.5
[157, 178, 170, 208, 100, 145]
958
0.7
SGRL
1,,
3.3
[40, 38. 35, 46, 20, 29]
208
PiGCL
5,, 1,41
3.8
[66, 65, 60, 88, 35, 56]
370
Ours
1,,
1.7
[17, 15, 18, 9, 7, 12]
78
iGCL
11.17,,,.3,,477
5.5
[17, 199, 29, 61, 30, 38, 74, 23]
471
GREET
18.3] 18, 6, 10.
9.4
[172. 661, 213, 213,383, 383, 526, 296, 1076, 253]
3,580
HomoGCL
[3. 1., 5, 6,,
10.2
[39. 251, 21, 67, 445, 18, 53, 225, 33)
1,167
0.5
SGRL
17,27, 14, 14,
1, 12, 15, 16, 12]
14.8
[58, 750, 106, 400, 258, 199, 258, 195]
2,224
PiGCL
[12, 102, 20, 35, 17, 30, 56, 3] [6. 15, 10, 4. 15, 5, 10,
Amazon
Ours
1,
1.8
68
Photo
iGCL.
1,,,4411,,,7
3.0
[8, 108, 21, 38, 18, 29, 52, 4] [59, 282, 97, 183, 232, 99, 472, 142] [17, 144, 37,
GREET
6,
5.0
1,566
HomoGCL
6, 6,,10.6.4.6.3,99.
41, 21]
0.7
SGRL
12, 4,
6,6
[22, 338, 36, 100, 49, 56, 79, 47)
727
PiGCL
1,
1.8
15, 67, 26, 10, 42, [5, 6, 2, 5,
Ours
[1, 1,
1.1
27
iGCL
12, 10, 6, 1,,,,,,,,,,,,,,,
[14, 161, 50, 329, 16, 14, 165, 301
846
GREET
[5, 1,,3,,,,
14.1
[310, 1024, 263, 155, 2157, 25, 135, 205, 977, 64)
5,315
HomoGCI.
5,,,,,,,,,,,,,,,,,,,311
10.2
[39, 251, 21, 67, 445, 18, 13, 225, 33]
1,167
0.5
SGRL
[1. 1,,
7.0
(95, 480, 142, 96, 494, 20, 50, 189, 429, 56]
2,051
PiGCL
15..2.3,
4.4
[15, 147, 22, 39, 293, 13, 6, 44, 132, 16]
727
Amazon
Ours
1,,42.111993,1,4444
3.6
[20, 89, 18, 18, 119, 6, 10, 27, 103, 18]
428
Computers
iGCL
1,, 1,,
5.7
[16. 129, 26, 29, 267, 12, 13, 131, 16)
689
GREET
[4, 10, 2, 28, 1,
[46, 237, 9, 42, 443, 12, 33, 255, 24]
1,124
HomoGCL
[3. 1,3,,
13.2
5.0
[7, 114, 11, 255, 16, 12, 108,
599
0.7
SGRL
[1,3, 1. 1,8,8, 1,
[25, 113, 18, 22, 234, 10, 7353, 123,21]
PiGCL
22 1,
2.9
[3, 75, 5, 20, 185, 11, 19, 64, 12]
398
Ours
1.6
[6. 18, 4, 21,
82
of global sparseness and local cohesion in GCL, where the groups and isolated nodes in the clusters are much higher than ideal conditions, namely "Mean Groups" is and "To- tal Isolated Nodes" is O. We also observe that the proposed model has the expected "Mean Groups" and "Total Isolated Nodes" when compared with other advanced GCL models under all the settings, which means the proposed fuzzy boundary effectively alleviates the two shortcomings.
information loss, and the result after pre-training is terrible. Even so, under the guidance of fuzzy boundaries, the distri- bution of each cluster becomes clearer, and the samples of each cluster get more compact. Since fuzzy boundaries are the outside of the real clusters, narrowing the distances be- tween prototypes and their corresponding fuzzy boundaries effectively tighten the real clusters.
5. Conclusion
4.4. Benefits from Compactness
In this paper, we first attend to two shortcomings existing GCL models, namely global sparseness and local cohe- sion, which lead to inferior global structural distribution. Such a distribution imperils the representation generaliza- tion and expressiveness, and is adverse to downstream tasks. Considering this, we design a novel fuzzy boundary that
In this experiment, project the raw data into 3- dimension hypersphere for visualization, and the representa- tions after pre-training and fine-tuning are shown in Figure 4(a) and (b) respectively. Since the raw high- high-dimension data are condensed 3-dimension space, leads abundant
8

---

Title Suppressed Due to Excessive Size
Ester, M., Kriegel, H.-P., Sander, J., Xu, X., et al. A density- based algorithm for discovering clusters in large spatial databases with noise. In International Conference on Knowledge Discovery and Data Mining, pp. 1996.
109
100
26
20
0.5
20
He, D., Shan, L., Zhao, J., Zhang, H., Wang, Z., and Zhang, W. Exploitation of latent mechanism in graph con- trastive learning; Representation scattering. In Advances in Neural Information Processing Systems, pp. 1-26, 2024a.
-423 55.55
-10
6:8 87
100
L. -10
-05
(a)
(b)
Figure Representation distributions after ve-ttaining and fine- tuning on the Cora dataset. (a) is the representation distribution after pre traiirain and is mixed, but becomes distinct with the fuzzy boundary after fine-tuning as (b).
He, D., Zhao, J., Huo, C., Huang, Y., Huang, Y., and Feng, Z. A new mechanism for eliminating implicit conflict in graph contrastive learning. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 12340-12348, 2024b.
includes fuzzy boundary construction and fuzzy boundary contraction phases. By implementing the fuzzy boundary on pre-trained GAE, the cluster distributions achieve pre- mium global structural properties, namely high intra-cluster compactness and inter-cluster separateness. The extensive experiments on two downstream tasks and the quantitative analysis comprehensively illustrate the merits of the fuzzy boundary address these shortcomings.
Hou, Z., Liu, X., Cen, Y., Dong, Y., Yang, H., Wang, C., and Tang. J. Graphmae: Self-super masked graph au- toencoders. In Proceedings of ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 594-604, 2022.
Hu, S., Lou, Z., Yan. X., and Ye, Y. A survey on information bottleneck. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(8):5325-53444 2024.
Acknowledgements
Hu, W., Fey, M., Zitnik, M., Dong, Y., Ren, H., Liu, B., Catasta, M., and Leskovec, J. Open graph benchmark: Datasets for machine learning on graphs Advances in Neural Information Processing Systems, pp. 22118- 22133, 2020.
This work was supported by the National Key Research and Development Program of China (No. 0023YFB3107100 the National Natural Science Foundation of China (No. 62306020), the Young Elite Scientist Sponsorship Program by BAST (No. BYESS2024199), Beijing Natural Science Foundation (No. L244009), the National Social Science Fund of China (No. 23VRC094), and the Major Program of the National Social Science Foundation of China (No. 22&2D147). The first author funded by the China Scholar- ship Council (CSC) from the Ministry of Education, China.
Huang, J., Shen, J., Shi, X., and Zhu, X. On which nodes does gen fail? enhancing gen from the node perspective. In International Conference on Machine Learning, pp. 1-23, 2024.
Jiang, B., Xiang, J., Wu, X., He, W., Hong, L., and Sheng, W. Robust adaptive-weighting multi-view classification. In Proceedings of ACM International Conference on In- formation & Knowledge Management, pp. 3117-112 2021.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Kipf, T. N. and Welling, M. Semi- supervised classifica- tion with graph convolutional networks. In International Conference on Learning Representations, pp. 1-14, 2017.
Lee, N., Lee, J., and Park, C. Augmentation-free free self- supervised learning on graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 7372- 7380, 2022.
References
Chen, S., Gong, C., Li, X., Yang. J., Niu, G., and Sugiyama, M. rousdtry-restrrieed metric learning. Machine Learn- ing, 12(12)442234476, 2023.
Li, A., Deng, Z., Zhang, W., Xiao, Z., Choi, K.-S,, Liu, Y., Hu, S., and Wang, S. Multi-view transfer representation learning with tsk fuzzy system for eeg epilepsy detection. IEEE Transactions on Fuzzy Systems, (1(1): (138:328-,, 2024.
Davies, D. L. and Bouldin, D. W. A cluster separation measure. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2:224-227, 1979.
9

---

Title Suppressed Due to Excessive Size
Li, H., Cao, J., Zhu, J., Luo, Q. He. S., and Wang. X. Augmentation-fiee freee graph contrastive learning of invariant- discriminative representations. IEEE Transactions Neural Networks and Learning Systems, pp. 1-11, 2023a.
Thomas, N. K. and Welling, M. Variational graph auto- encoders. In Advances in Neural Information Processing Systems Workshop on Bayesian Deep Learning, pp. 1-3, 2016.
Li, W.-Z., Wang. C.-D., Xiong, H., and Lai, J.-H. Homogel: Rethinking homophily in graph contrastive learning. In Proceedings of ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1341-1352, 2023b.
Van der Maaten, L. and Hinton, G. Visualizing data using t-sne. Journal of Machine Learning Research, 9(11): 2579-2605, 2008
Velickovic, P., Fedus, W., Hamilton, W. L., Liò, P., Bengio, Y., and Hjelm, R. D. Deep graph infomax. International Conference on Learning Representation, pp. 2019.
Liu, Y. Zheng, Y., Zhang, D., Lee, V. C., and Pan, S. Beyond smoothing: Unsupervised graph representation learning with edge heterophily discriminating. In Pro- ceedings of the AAAI Conference on Artificial Intelli- gence, pp. 4516-45244 2023.
Wang. H., Wang, J., Wang, J., Zhao, M.. Zhang, W., Zhang, F., Li, W., Xie, X., and Guo, M. Learning graph represen- tation with generative adversarial nets. IEEE Transactions on Knowledge and Data Engineering, 3398):3000-3103, 2019.
Lv, J., Liu, B., Feng. L.,, Xu, N., Xu, M., An, B., Niu, G., Geng, X., and Sugiyama, M. On the robustness of average losses for partial-label learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(5):2569- 2583, 2024.
Wang, Z. and Liu, W. Robustness verification for contrastive learning. In International Conference on Machine Learn- ing. pp. 22865 22883. PMLR, 2022.
Xiao, S., Wang, S., and Guo, W. Sgae: Stacked graph autoencoder for deep clustering. IEEE Transactions on Big Data, 91):254-266, 2022.
Mo, Y., Peng, L., Xu, J. Shi, X., and Zhu, X. Simple unsupervised graph representation learning. In Proceed- ings of the AAAI Conference on Artificial Intelligence, pp. 7797-7805, 2022.
Xiao, S., Du, S., Chen, Z., Zhang, Y., and Wang, S. Dual graph neural network for multi-view clustering. IEEE Transactions on Multimedia, 33:9203- 9215, 2023.
Natekar, P. and Sharma, M. Representation based com- plexity measures for predicting generalization in deep learning. arXiv preprint arXivv2012.02275,,2 2020. 2020.
Xu, J., Tang, H., Ren, Y., Peng, I.,, Zhu, X., and He, Multi-level feature learning for contrastive multi-view clustering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16051- 16060, 2022.
Peng, Z., Huang, W., Luo, M., Zheng, Q., Rong, Y., Xu, T., and Huang, J. Graph representation learning via graphical mutual information maximization. In Proceedings of The Web Conference, pp. 259-270, 2020.
Perozzi, B., AI-Rfou, R., and Skiena, S. Deepwalk: Online learning of social representations. In Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 701-710, 2014.
Xu, J., Chen, S., Ren, Y., Shi, X., Shen, H., Niu, G., and Zhu, X. Self-weighted contrastive learning among mul- tiple views for mitigating representation degeneration. Advances in Neural Information Processing Systems, pp. 1119-1131, 2024a.
Rong, Y., Huang, W., Xu, T., and Huang, J. Dropedge: Towards deep graph convolutional networks on node clas- sification. In International Conference on Learning Rep- resentations, pp. 2019.
Xu, J., Ren, Y., Wang, X., Feng, L., Zhang, Z., Niu, G., and Zhu, X. Investigating and mitigating the side effects of noisy views for self-supervised clustering algorithms in practical multi-view scenarios. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22957-22966 2024b.
Shen, X., Sun, D., Pan, S., Zhou, X., and Yang, L. T. Neigh- bor contrastive learning on learnable graph augmentation. In Proceedings of the AAAI Conference on Artificial In- telligence, pp. 2023.
Xu, N., Shu, J.,, Zheng, R., Geng, X., Meng, D., and Zhang, M.-L. Variational label enhancement. IEEE Transac- tions on Pattern Analysis and Machine Intelligence, 45 (5):6537-655 6551, 2023.
Thakoor, S., Tallec, C., Azar, M. G., Azabou, M., Dyer, E. L., Munos, R., Veličković, P., and Valko, M. Large- scale representation learning on graphs via bootstrapping. In International Conference on Learning Representations, pp. 1-21, 2022.
You, Y., Chen, T., Shen, Y., and Wang, Z. Graph contrastive learning automated. In International Conference on Ma- chine Learning, pp. 12121-12132. PMLR, 2021.
10

---

Title Suppressed Due to Excessive Size
Zadeh, L. A. Fuzzy sets. Information and Control, 8(3): 338-353, 1965.
Zhang, C., Lou, Z., Zhou, Q., and Hu, S. Multi-view clus- tering via triplex information maximization, IEEE Trans- actions on Image Processing, 32:4299-4313 2023a.
Zhang, C., Fang, Y., Liang, X., Wu, X., Jiang, B., et al. Efficient multi-view unsupervised feature selection with adaptive structure learning and inference. In Proceedings of International Joint Conference on Artificial Intelli- gence, pp. 5443-5452, 2024.
Zhang, W., Deng, Z., Zhang, T., Choi, K.-S., and Wang, S. Multi-view fuzzy representation learning with rules based model. IEEE Transactions on Knowledge and Data Engineering, 36(2):73-774,, 2023b.
Zhang. Y., Zhu, H., Song, Z., Koniusz, P., and King, Costa: covariance-preserving feature augmentation for graph contrastive learning. In Proceedings of ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2524 2534, 2022.
Zhelezniak, V., Savkov, A., Shen, A., Moramarco, F., Flann, Don't settle for average, go for
J., and Hammerla, the max: Fuzzy sets and max-pooled word vectors. In International Conference Learning Representations, 1-17, 2018.
Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., and Wang, L. Deep graph contrastive representation learning. International Conference Machine Learning, pp. 1-17, 2020.
Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., and Wang, L. Graph contrastive learning with adaptive augmentation. In Proceedings of the Web Conference, pp. 2069-2080, 2021.
Zou, X. and Liu, W. Generalization bounds for adversar- ial contrastive learning. Journal of Machine Learning Research, 24(114):155,22 2023.

---

Title Suppressed Due to Excessive Size
A. Theoretical Proof
A.1. Theorem 3.2
Proof: Since we use the centralized representations for constructing the fuzzy boundary, the overall variances within each cluster are formed as
min J min
12.112
IEC.
Op
min
(1 B:) 3 2212
(11)
c=1iEC.
N
min
Σ
(2 8:) 3 2: 2:
i=1
where Cp is the number of the clusters, C₂ is the index set including the node indices of the e-th cluster. For the j-th element of Si, We have following observation:
0J 2(2 - 2 O. JB;
(12)
Therefore, I is a monotonically decreasing function of 8..j. When applying max-pooling the Union operator, we have
[1 (8man) }:i @ 2:1/
12 (8max) Fio) Zi.ll
Σ [(2 (Bother Pir) Vi.jl
(13)
Σ [22 (BminJis) 2g.5)
where (8min is the membership degree vector of node i, which is filled with minimal membership degrees, and its element is {8min = ming { Similarly, we have (Smarli. = max; (s {Bother); }i denotes other simple U Unoo operators. Then we have the following conclusion as
N
(2 Smaz }:) ② z l12
[[ Bothhr H() ① 2:112
(14)
i=1
(2 (Smin h:) zz
Finally, we get the minimum of J is EN [| (Smon) J:) o Zi 2. Union {s max { 1.
when the Union operator is max-pooling as $
A.2, Theorem 3.3
We first introduce Definition A.I and Lemma A.2, which the core of the theorem proof.
Definition A.1. Complexity Measure based on the Davies-Bouldin Index. The -ovies-Bouldin Index is calculated by the ratio of intra-clusted compactness to inter-cluster separateness (Natekar & Sharma, 2020), and it stands for the consistency within a cluster and the difference among clusters. Mathematically, it is defined by
1 C₂-1 B
Qm + Qn
max mfn
(15)
Cp
Un.
U
m=0
12

---

Title Suppressed Due to Excessive Size
where
1/
Qm
pmll2
for m = ,,.,, Cps
Nm
(16)
Um,
pn ll, for m, n 1, CCp
and the mith cluster and n-th cluster, Cp is the number of clusters, Z denotes representation of the i-th sample in the m-th cluster, Pm is the prototype of the m-th cluster, Qm measures the intra-cluster compactness of cluster m, and Un. measures the inter-cluster separateness between cluster m and cluster n (Davies & Bouldin, 1979; Natekar & Sharma, 2020).
Lemma A.2. The fuzzy boundary leads to a lower bound of the complexity measure B, which is in line with a higher generalization bound of the model.
Proof: We follow the proof idea (Huang et al., 2024). For simplicity, we consider the condition with two clusters. In this case, the complexity measure B is Qu We assume the probability that a node belongs to cluster is P.. For calculating B, we firstly compute the cluster prototypes fi and μ1 as
Un.1
)[
Po = EE
E
W
X +
Σ JEN(ee)
x,
W (Po uxx
4Xx. + (1 (c) px,))
di
(17)
W (Pi px. + (1- -.))
where W the parameter matrix of the encoder, px. is the prototype of cluster in the original feature space, N(w,) stands for the neighbors of node V/- Therefore, Qo and Qı are calculated
Qu VeB VEIIZZ - Poll
V EE (X= ux. WT ( mx.) + B (1) " -xx,)
(18)
V PPEIII (X. ux. [1 + Po)
W (X1 uxx 1111.
Q V // px, || + (1 (1-
(Xo (xx
Let of = = [|[, ,xx)113] then have
Qo = V PO + (1 27.
(19)
Q V Pfot (1 P1 22.2
ag' 5र. And let Q6 Poo0 + Po) calculate
0 and get Pe कzt.ß.
o
Substitute Po into Qo. obtain
1200
Qo 2 Vog o
(20)
Similarly, we also obtain
22
Q1 >
(21)
/ Vog + of
Let ②
dnd Vat+o?
we find that
aQ
0, aQ
9
0.
(22)
dop
Viag
0o
V/ + 13²
13

---

Title Suppressed Due to Excessive Size
Therefore, Qu and Q1 increase when dy and 0j increase respectively, and the complexity measure B has the lower bound
Qa Q1 2 B
2v 20001
(23)
U0,1
+ of IP4 M1ll
Let (o:)²= [| P&x 2 and μ be the variance and prototype of real cluster when applying fuzzy boundary respectively. Since a vanilla model applies the fuzzy boundary would increase the inter-cluster separateness and reduce the intra-cluster compactness of the embedding distribution, we can get of o; and l|uo pi S 1116 rill- Then we finally
obtain the following equation
2V200011 Vož + po mil
2v 20
B
(24)
V(ag)2 + (01) A1ll
which means the fuzzy boundary could reduce the lower bound of B. Following the conclusion of (Natekar & Sharma, 2020; Huang et al., 2024), lower bound of the complexity measure corresponds to an upper bound of the generalization ability.
B. Experiments
B.1. Datasets
We conduct experiments on seven well-known public datasets, and the detailed statisties summarized in Table 4. Cora, CiteSeer, Pubmed, and DBLP are the famous citation networks. Every node in these datasets is paper, and the link between two nodes means the citation relationship. Each dimension of the node attribute vector refers to keyword of the paper. Amazon Photo and Amazon Computers are two co co-purchase graphs where the nodes are products, and the edges between the two products show they are frequently bought together. Every node feature is constructed by bag-of-words representation. Ogbn-arxiv is extracted from the Microsoft Academic Graph (Hu et al., 2020). is citation network constructed by Computer Science arXiv papers, whose nodes denote papers and edges are citation relations. Each node obtained by averaging the skip-gram word embeddings from the title and abstract.
Table 4. Statistics of different graph datasets.
Amazon
Amazon Computers
Dataset
Cora
Citeseer
Pubmed
DBLP
Ogbn-arxiv
Photo
Nodes
2,708
3,327
19,717
17,716
7,650
13,752
169,343
Edges
5,429
4,732
44,338
52,867
119,081
491,722
1,166,243
Features
1,433
3,703
500
1,639
745
767
128
Classes
7
6
4
10
40
B.2. Hyperparameter Setting
The hyperparameter setting is listed in Table 5. Pre_Epochs, Pre_Lr, and Pd are the training epochs, the learning rate, and the weight decay applied in the pre-trained processes. Dim_Hid, Dim_Emb, and Dim_Proj are hidden dimension of the GNN encoder, the output dimension of the GNN encoder, and the hidden dimension of the projection head, respectively. Act is the used activation function for all components of the proposed model. FT Lr, FT Epochs, and FT_Wd the learning rate, the training epochs, and the weight decay applied in the fine- tuning processes. T' is used for updating crisp prototypes. K is the number of the membership functions for each element of centralized feature vector is set to weigh the significance of the distances between the fuzzy boundary and the prototypes, Cp is the number of the clusters for each dataset, a is the penalty coefficient.
B.3. Details on Quantitative Analysis
The quantitative analysis measures the model ability to address the local cohesion and global sparseness, where we propose two metrics "Mean Groups" and "Total Isolated Nodes" to measure the local cohesion and global sparseness for the quantitative analysis, and present "Group List" and "List of Isolated Nodes" to assist in understanding the two metrics. Here, we provide some definitions and explanations for them:
14

---

Title Suppressed Due to Excessive Size
Table 5. Hyperparameter setting different datasets.
Amazon Photo
Amazon Computers
Dataset
Cora
Citeseer
Pubmed
DBLP
Ogbn-arxiv
Pre Epochs
900
900
1900
800
500
600
30
Pre_rr
5e-5
5c-5
5c-5
5e-5
5e-5
5e-5
3e-5
Pre Wd
le-5
le-5
5e-6
5e-6
5e-6
1.5e-4
1e-5
Dim Hid
512, 512
512, 512
1024
512
512
512
256, 256
Dim Emb
256
256
1024
256
256
256
256
Act
ELU
ELU
ELU
ELU
ELU
ELU
ELU
Dim
512
512
2048
512
512
512
512
Lr
5e-5
5e-5
5e-5
8e-5
7e-5
8e-5
le-5
FT_Wd
2c-5
6c-6
lc-5
5c-6
le-4
lc-4
le-4
FT Fp_Eposhs
700
300
4500
5000
6500
6000
30
40
20
40
40
40
40
Dropout ratio
0.5
0.5
0.5
0.5
0.5
0.5
0.5
K
5
5
5
5
5
3.5
3.2
7.0
4.5
3.2
3.5
6.0
Cp
30
30
550
250
100
150
1000
0.9
0.0
0.3
1.0
0.9
1.1
0.8
(I) "Group List" and "Mean Groups" reflect the local cohesion. "Group List" is a list containing the number of groups detected within each class cluster, with each element in the list corresponding the number of groups in a given cluster. group is set of nodes that are close each other in the embedding space, with the maximal distance between each node pair smaller than pre-defined "Eps", where "Eps" is hyperparameter of DBSCAN to limit the maximal distance between the representations in a detected local group. "Mean Groups" calculates the mean number of groups from all the classes. The calculation of "Mean Groups" is given by
Maan Groups
1 C
Σ Gi, i=1
(25)
where C' is the number of classes, G; is the number of groups for the i-th class cluster. For an ideal cluster distribution, all the samples with the same label are supposed to be gathered in one cluster, so each element in the "Group List" is expected to approach 1, the same as "Mean Groups".
(2) "List of Isolated Nodes", "Total Isolated Nodes", and "Mean Groups" reflect global sparseness jointly. "List of Isolated Nodes" details the number of isolated nodes for each class cluster, and the isolated nodes are the nodes whose distances between themselves and any other group greater than "Eps", and these nodes not assigned to any group and contribute to global sparseness. "Total Isolated Nodes" records the total number of isolated nodes for each dataset. The calculation of "Total Isolated Nodes" is given by
T Total Isolated Nodes
4,
(26)
where 7; is the number of isolated nodes for the i-th class cluster. Generally, ideal cluster distributions hardly have isolated nodes. According to the aforementioned explanations, smaller "Mean Groups" and "Total Isolated Nodes" could reflect better clustering performance in general.
B.4. Hyperparameter Analysis
We analyze two hyperparameters directly related to the loss function, namely the temperature coefficient and penalty coefficient a, whose results are exhibited in Figure 5. Not surprisingly, temperature coefficient plays significant role in the model performance as it weighs the distances between the prototypes and the fuzzy boundaries. Greater 7 makes the distances between prototypes and their corresponding fuzzy boundaries much closer than those between different prototypes.
15

---

Title Suppressed Due to Excessive Size
85.2
06.5
91.0
85.0
86.0
90.8
88.88
05.5
90.6
85.0
90.4
84.5
84.0
90.0
35
(a) Cora
(b) Computers
(c) DBLP
85.2
01.2
88..8
85.5
4 90.6
85.0
00
84.5
90.2
96.0
(d) Cora
(e) Computers
(f) DBLP
Figure 5. Results of hyperparameter sensitivity analysis, r affects the model performance significantly and a influences slightly.
In other words, relatively larger v balances the intra-cluster compactness and inter-cluster separateness. In contrast the function of influences the model performance slightly, Even so, functions to preserve the critical information much possible.
B.5. Visualization
In this experiment, we draw the t-sne of model after pre-training and fine-tuning respectively show how the fuzzy boundary forms the ideal global structural distribution in Figure 6. We find that after pre-training, the cluster distributions display global sparseness. However, after fine-tuning, the global sparseness disappears and the ideal properties of cluster distributions, namely intra-cluster compactness and inter-cluster separateness gradually appear. Besides, by comparing Figure 6(b) with (c), we find that the fuzzy boundaries indeed extend the spectrums of the real clusters. And when we realize the contraction on outer fuzzy boundaries, the inner real clusters gather more tightly. Except for improving intra-cluster compactness, the fuzzy boundaries implicitly enhance inter-cluster separateness. Concretely, the fuzzy clusters, namely the clusters that include both real samples and fuzzy boundary elements are separatable (in Figure 6(b)), and when the fuzzy clusters further distance each other, the distances between the inner real clusters saliently enlarged (in Figure 6(c)). As result, the clusters embody high intra-cluster compactness and inter-class separateness merits.
B.6. Complexity
The complexity of the proposed model mainly includes the encoding process, K-means clustering algorithm, centralization process, calculation of fuzzy boundary, calculation of the reconstruction loss, and calculation of the prototypical contrastive loss. The encoding process demands O (LFdd) complexity, where 7. is the layer number of the encoder, F. is the number of the edges, d is the dimension of the output representation. The K-means clustering algorithm demands O (C,,Nd) complexity, where Cp is the number of prototypes, is the number of samples. The centralization process, the calculation of fuzzy boundary, and the reconstruction loss all demand O (Nd) complexity. The prototypical contrastive loss demands O (d) complexity. Overall, the whole complexity of the proposed model is O (LEd + C,Nd + 3Nd + Cd.
B.7. Ablation Study
Table lists the node classification results in the ablation study, where Pre-training means the results for pre-trained GAE, Fuzzy Boundary corresponds to the model with the fuzzy boundary, Lpro the contrastive loss, and Lree the
16

---

Title Suppressed Due to Excessive Size
(a)
(b)
(c)
Figure 6. t-SNE results after pre-training and fine-tuning Citeseer dataset, where (a) includes the crisp representations after pre-training, (b) includes both crisp representations the fuzzy boundary elements after fine-tuning, and (c) solely composed of the node representations after fine-tuning. Cluster distributions of (h) and (c) show ideal intra-cluster compactness and inter-cluster separateness.
Table Results of ablation study on node classification task, which are concerned with the functions of pre-training, fuzzy boundary, prototypical contrastive loss Cpro, and reconstruction loss Lree-
Fuzzy Boundary
Amazon
Amazon
Pre-training
Epro
Lree
Cora
DBLP
Photo
Computers
84.83 # 0.75
83,40 = 0.35
93.21 = 0.20
89.72 = 0.24
85.24 0.55
84.33 0.16
93.43 0.21
90.03 = 0.15
85.73 0.62
84.67 0.17
93.60 0.21
90.54 + 0.20
V
84.86 0.58
84.35 0.14
93.53 0.19
90.46 + 0.22
V
V
85.90 0.56
84.80 0.17
93.71 0.20
90.75 + 0.20
reconstruction loss. Compared with models without fuzzy boundary, the models with fuzzy boundary behaves better, which means the fuzzy boundary significantly enhances model performance. By combing Cero with Crec the model performance is further improved.
17


Reviews:
[{"review_id": "998Bs99Sbw", "reviewer": "Reviewer_upat", "summary": "This paper addresses two integrated shortcomings in graph contrastive learning, namely local cohesion and global sparseness, which will lead to inferior global structural distributions. So this paper proposes novel fuzzy boundaries to gather discrete local groups and isolated samples, which effectively alleviate the local cohesion and global sparseness. Overall, this paper treats a detailed problem in graph contrastive learning, and the method is interesting.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1.**\n\nFollowing your kind suggestion, we provide a related work compassing graph prototype learning as follows:\n\nGraph prototypical learning has emerged as a significant direction in graph representation learning, addressing limitations in traditional contrastive and few-shot learning approaches. Existing GCL methods are limited to sub-optimal representations due to sampling bias where semantically similar graphs are incorrectly treated as negatives. To overcome this, PGCL introduces a clustering-based approach by grouping semantically similar graphs and ensures that negative samples come from different clusters [1]. Similarly, GPCL [2] enhances GCL by integrating both node-level and prototype-level contrastive objectives, maintaining consistency across different augmentations. To further capture global semantic structures, GraphLoG [3] employs K-means clustering to derive hierarchical prototypes. At the same time, X-GOAL [4] extends prototypical learning to multiplex heterogeneous graphs for aligning embeddings across different relational layers. In the domain of few-shot learning, GPN [5] leverages meta-learning to extract transferable prototypes for node classification with scarce labels. Similarly, S4GPN [6] integrates self-supervised learning with prototypical networks to improve hyper-spectral image classification. These advancements demonstrate the effectiveness of prototype-based graph learning in mining cluster-level semantic consistency which is important for realizing ideal global structural distribution.\n\n[1]Prototypical graph contrastive learning, in TNNLS 2024.\n\n[2]Graph prototypical contrastive learning, in INS 2022.\n\n[3]Self-supervised graph-level representation learning with local and global structure, in ICML 2021.\n\n[4]X-GOAL: Multiplex heterogeneous graph prototypical contrastive learning, in CIKM 2022.\n\n[5]Graph prototypical networks for few-shot learning on attributed networks, in CIKM 2022.\n\n[6]Self-supervised spectral-spatial graph prototypical network for few-shot hyperspectral image classification, in TGRS 2023.\n\n**W2.**\n\nThanks for your careful observation regarding Figure 2. To address this issue, we have revised Figure 2 by (1) adopting a different line shape to represent the local groups, and (2) replacing the samples inside the local groups with crisp representations. The updated model architecture will be provided in the refined paper to avoid misinterpretation.\n\n**Q1.**\n\nThe difference in visualization between the two figures is attributed to different data processing. For realizing the 3-dimension visualization in Figure 4, we first condense the raw data into a 3-dimension feature space with graph auto-encoder, then use the crisp and the proposed fuzzy models to train the 3-dimension features for visualization. Since the raw data is condensed into a low-dimension feature space from a high-dimension space, the data information is heavily lost, which presents terrible visualization. Even so, the proposed fuzzy model could also gather similar samples in the clusters, which leads to much clearer decision cluster boundaries when compared with the crisp model.\n\nThe visualization in Figure 6 is general t-sne. We first apply the proposed model to train the raw graph data, and then implement the t-sne algorithm on the produced representations. In this case, the models could preserve more critical information in the feature space than in the 3-dimension space, which makes Figure 6 have much better visualization than Figure 4.", "claims_and_evidence": "Yes, the claims are supported by clear and convincing evidence. The evidence such as the performance on downstream tasks and visualization illustrate that the problems are well addressed.", "methods_and_evaluation_criteria": "Yes, the proposed fuzzy boundary makes sense for the problem and addresses the problem well. It firstly expands the cluster boundary with the fuzzy set theory to gather discrete local groups and isolated samples, then contracts the boundary to tighten samples within the boundary, which effectively alleviates local cohesion and global sparseness.", "theoretical_claims": "Yes, the theoretical claims are relative to the proposed model. I checked the proofs for theoretical claims, and did not find obvious issues.", "experimental_designs_or_analyses": "Yes, the experimental designs are sound to validate the model ability. Specifically, the paper adopts node classification, node clustering, ablation study, quantitative experiments, and a series of visualizations to analyze the model.", "relation_to_broader_scientific_literature": "This paper focuses on addressing the shortcomings commonly emerging in the current research, and the results demonstrate that the proposed model could effectively address the shortcomings.", "essential_references_not_discussed": "This paper adopts prototype learning in the method. However, the paper lacks the related works of graph prototype learning such as [1].\n\n[1] X-GOAL: Multiplex heterogeneous graph prototypical contrastive learning. In CIKM.", "other_strengths_and_weaknesses": "**Strengths**\n\n++ Good writing and clear organization. The paper has good structural organization and a clear problem statement, which is easy to read and understand.\n\n++ Interesting method. This proposed model introduces the novel idea with fuzzy set theory to expand the original cluster boundary, which mitigates the local cohesion and global sparseness.\n\n++ Sufficient experiments. The experimental design is comprehensive and convincing by covering different downstream tasks, quantitative analysis, and visualization, which well demonstrates the model effectiveness.\n\n**Weaknesses**\n\n-- This paper adopts prototype learning in the method. However, the paper lacks related works on graph prototype learning, it would be better to provide a review of graph prototype learning in related works.\n\n-- The icons of local groups in Figure 2 is misleading. In my opinion, the samples in local groups are not isolated samples, but they are depicted as isolated samples in Figure 2, and the authors should correct the figure.", "other_comments_or_suggestions": "See weakness.", "questions_for_authors": "1. I have a minor question regarding the pre-training visualization. The authors mentioned that projecting raw data into a 3-dimension space can lead to information loss. However, I think projecting it into a 2-dimension space might result in more information loss and cause worse visualizations. Despite this, Figure 4 shows poor results in 3-dimension space, while Figure 6 demonstrates a more regular distribution in the 2-dimension visualization after pre-training. Could you provide a brief explanation for this phenomenon?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CHT8NXl4vq", "reviewer": "Reviewer_7eQp", "summary": "This paper presents a novel graph model by introducing fuzzy set theory to alleviate the impacts of local cohesion and global sparseness, expecting to form the ideal global structure distribution. The proposed model realizes a learned boundary to bridge the local groups and isolated samples by adjusting the cluster boundary adaptively. The experiments across several datasets validate the model performance. The quantitative analysis and visualization demonstrate the model generates better global structural distribution.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1.**\n\nThe details of the centralization are as follows:\n\nWhen centralizing the node representations, we use the K-means clustering algorithm to obtain the cluster prototype $C\\in\\mathbb{R}^{C_p\\times d}$, where $C_p$ is the number of clusters, and $d$ is the dimension of the prototype vector. For each node $v_i$ belonging to a specific cluster $k$, we assign it to its corresponding prototype $c_k$ and perform the centralization as follows:\n\n$z_i = h_i - c_k$\n\nwhere $h_i$ is the original node representation and $c_k$ is the prototype of the cluster that node $v_i$ belongs to. We also provide a matrix form by constructing an assignment matrix $M \\in \\mathbb{R}^{N\\times C_p}$, where $M_{ik}=1$ if node $v_i$ belongs to cluster $k$, and 0 otherwise. Then, the centralized representation can be expressed as:\n\n$Z=H-MC$\n\nwhere $MC$ selects the corresponding prototype for each node. This modification with the matrix form ensures the dimensions are consistent during the centralization process.\n\n**W2.**\n\nThe visualization of Figure 4(a) is affected by the choice of view angle. To ensure a fair comparison, we have updated the visualization with a new version of Figure 4(a) using a more ideal view angle and supplemented the further analysis in the refined paper. Due to inconvenience in providing images in the rebuttal phase, we only provide the analysis as follows:\n\nFor the adjusted angle in Figure 4(a), only three clusters become clearer and the embedding distribution in Figure 4(a) is inferior to Figure 4(b) since most samples in different classes are overlapped together, which further verifies that the fuzzy boundaries have the ability to refine the embedding distribution.\n\n**Q1.**\n\nEquation 9 aims to tighten samples in a cluster while pushing samples away from different clusters. Specifically, the loss encourages each node representation $h_i$ to be close to its corresponding cluster prototype $p_i$. A lower distance $\\||h_i-p_i\\||^2_2$ results in a higher probability that pushes representations toward their prototypes, thus tightening clusters.\n\nFor better understanding Equation 9, we use a softmax-based probability function that $v_i$ belongs to the corresponding prototype $p_i$ as:\n\n$P(i)=\\frac{e^{-\\tau\\||h_i-p_i\\||^2_2}}{\\sum_{j=1}^{C_p}e^{-\\tau\\||h_i-p_j\\||^2_2}}=\\frac{1}{\\sum_{j=1}^{C_p}e^{\\tau(\\||h_i-p_i\\||^2_2-\\||h_i-p_j\\||^2_2)}}$\n\nwhere $h_i$ is the representation of node $v_i$, $\\tau$ is the temperature coefficient that scales the Euclidean distances. According to the assumption that $v_i$ belongs to the corresponding prototype $p_i$, $\\||h_i-p_i\\||^2_2-\\||h_i-p_j\\||^2_2$ is negative in general since $p_j$ is not the corresponding prototype of $h_i$, so $\\||h_i-p_j\\||^2_2\\geq\\||h_i-p_i\\||^2_2$. A larger $\\tau$ will make the denominator smaller, and $P(i)$ becomes larger accordingly, which implies the cluster becomes more compact.\n\n**Q2.**\n\nAlthough fuzzy boundary contraction mainly focuses on intra-cluster compactness, the model also ensures inter-cluster separateness through the prototype contrastive loss (Equation 9). The key factor is that the prototype contrastive loss adjusts the distances between the node representation and the prototypes. The training process with Equation 9 pulls the representations and their corresponding prototypes together and pushes representations away from non-corresponding prototypes $p_j \\(j\\neq i\\)$, which **explicitly increases inter-cluster distances**.\n\nAdditionally, the fuzzy boundaries also **implicitly enhance inter-cluster separateness**, which is explained in Figure 6(b) and Figure 6(c). Concretely, the fuzzy clusters include both real samples (also forming the inner real clusters) and fuzzy boundary elements that are always outside the real samples. When the training process ends, the virtual fuzzy boundary elements are removed and the distances between the inner real clusters are further enlarged. As a result, the clusters embody higher intra-cluster compactness and inter-class separateness merits.\n\nThus, the combination of fuzzy boundary and prototype contrastive loss results in both intra-cluster compactness and inter-cluster separateness, achieving the desired global structural distribution.", "claims_and_evidence": "The Introduction clarifies the problems with visualization in Figure 1 and gives the explanations. Experiment results and quantitative analysis illustrate that the model alleviates the problems. These evidences can support the claims in the submission.", "methods_and_evaluation_criteria": "The proposed method designs a learnable boundary to involve the isolated samples belonging to the cluster with great similarities and local groups near the cluster boundary. By contracting the fuzzy boundary, the samples within the fuzzy boundary are gathered more tightly to alleviate local cohesion and global sparseness. These processes make sense for solving the problem.", "theoretical_claims": "I checked all the theoretical claims and proofs, including boundary construction and model generalization, which made the model design more reasonable.", "experimental_designs_or_analyses": "These experiments evaluate the proposed model based on downstream tasks and quantitative analysis. The experiment setting in node classification and node clustering are common, and the evaluation metrics are widely adopted. Additional quantitative analysis also shows the model ability to solve the shortcomings. The experimental designs are sound and convincing.", "relation_to_broader_scientific_literature": "This paper first attends to two ingrained shortcomings in GCL, which could cause inferior global structural distribution. Then the paper proposes a novel idea to expand cluster boundaries to bridge the discrete local groups and isolated samples. Experiment results \n demonstrate the model could alleviate the shortcomings. This paper may contribute to the broader scientific literature to some degree.", "essential_references_not_discussed": "The paper adopts a pre-training and fine-tuning learning framework in graph contrastive learning, which is similar to a previous paper on graph contrastive clustering [1], so it is better to cite it. \n\n\n[1] Dink-net: Neural clustering on large graphs. In International Conference on Machine Learning 2023.", "other_strengths_and_weaknesses": "Strengths\n(1) This paper focuses on mitigating local cohesion and global sparseness in graph contrastive learning, which corrupts global structure distributions, and the problem is meaningful.\n(2) This paper proposes fuzzy boundary construction to mitigate local cohesion, and fuzzy boundary contraction to alleviate global sparseness. The method is effective in restoring the ideal global structure distribution.\n(3) Besides promising model performance in two tasks, this paper also adopts quantitative analysis to quantify the model ability to mitigate the shortcomings when compared with other graph models.\n\nWeaknesses\n(1) When centralizing the node representations, the model utilizes node representation $H$ to subtract the corresponding prototypes $C$. However, the matrix size of $H$ is $N \\times d$ while the matrix size of $C$ is $C_p \\times d$. The authors should detail more about the centralization process.\n(2) The view angle for Figure 4(a) may be not fair. Though Figure 4(b) has better visualization than Figure 4(a), it may be attributed to the fact that Figure 4(b) is observed from a better view angle. The authors could give a more fair angle for Figure 4(a).", "other_comments_or_suggestions": "I hope the authors can tackle my weaknesses and questions.", "questions_for_authors": "(1) Equation 9 needs more detailed explanations. For example, how does it work to realize intra-cluster compactness, and why does higher temperature coefficient $\\tau$ weigh the intra-cluster compactness more?\n(2) The motivation of this paper is to generate a global structure distribution that realizes intra-cluster compactness and inter-cluster separateness. However, the fuzzy boundary contraction only focuses on realizing intra-cluster compactness intuitively. How does the model realize high inter-cluster separateness?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qLqOXufLQM", "reviewer": "Reviewer_Ytiy", "summary": "To address the ingrained shortcomings such as local cohesion and global sparseness in graph contrastive learning (GCL), this paper introduces a novel GCL model incorporating fuzzy boundaries. The proposed method dynamically extends original cluster boundaries to mitigate the two shortcomings in the embedding space. Extensive experiments on downstream tasks and quantitative analysis demonstrate that the proposed model achieves superior performance when compared with state-of-the-art methods.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1.**\n\nWe compare our model with two advanced graph generative models, S2GAE [1] and Bandana [2]. S2GAE reconstructs randomly masked edges while Bandana samples edge masks from a continuous distribution. As shown in Table 1, these graph generative models perform competitively but are limited to small datasets. On large datasets like Ogbn_arxiv, their performance declines due to graph sparsity that limits topology-based knowledge extraction. In contrast, our proposed model can connect and tighten similar samples in the representation space by introducing the fuzzy boundary, which effectively explores the commonalities regardless of dataset size.\n\nTable 1\n\n|Models|Cora|Citeseer|Pubmed|Photo|Computers|DBLP|Arxiv|\n|-|-|-|-|-|-|-|-|\n|S2GAE|85.5±0.3|73.4±0.5|86.5±0.1|93.5±0.2|90.0±0.1|84.1±0.0|60.9±0.0|\n|Bandana|85.5±0.8|73.0±0.8|86.4±0.3|93.4±0.1|89.6±0.1|83.9±0.0|66.8±0.0|\n|Ours|**85.9±0.6**|**74.1±0.6**|**87.0±0.3**|**93.7±0.2**|**90.8±0.2**|**84.8±0.2**|**69.5±0.1**|\n\n[1]S2GAE: Self-supervised graph autoencoders are generalizable learners with graph masking, in WSDM 2023.\n\n[2]Masked graph autoencoder with non-discrete bandwidths, in WWW 2024.\n\n**W2.**\n\nSection B.4 shows that the hyperparameters such as temperature coefficient and penalty coefficient are not sensitive and don’t impact the performance significantly for a given dataset. \n\nAdditionally, it is standard to adopt dataset-specific hyperparameters in graph representation learning due to variations in graph properties and sizes, which is easily found in numerous literature [1,2]. For a small dataset like Amazon Photo (a co-purchase network) and a large dataset like Ogbn-arxiv (a citation network), their differences should be addressed with subtle hyperparameter settings. However, for similar datasets, hyperparameters transfer easily, which facilitates the hyperparameter search process. For example, the Amazon Photo and Amazon Computers datasets have similar graph properties (co-purchase networks) and dataset size (this condition also applies to Cora and Citeseer datasets), so the hyperparameters for them are usually similar. It also means that it won’t spend much overhead when knowing more about the dataset properties. Thus, hyperparameter tuning is reasonable and manageable.\n\n[1]Homogcl: Rethinking Homophily in Graph Contrastive Learning, in KDD 2023.\n\n[2]A New Mechanism for Eliminating Implicit Confict in Graph Contrastive Learning, in AAAI 2024.\n\n**Q1.**\n\nWe provide concrete definitions of the terms: (1) A **Group** is a set of nodes that are close to each other in the embedding space, with the maximal distance between each node pair smaller than pre-defined “Eps”, where “Eps” is a hyperparameter of DBSCAN to limit the maximal distance between the representations in a detected local group. (2) A **Group list** is a list containing the number of groups detected within each class cluster, with each element in the list corresponding to the number of groups in a given cluster. (3) **Isolated nodes** are the nodes whose distances between themselves and any other group are greater than “Eps”, and these nodes are not assigned to any group and contribute to global sparseness.\n\n**Q2.**\n\nTo ensure rigor, we supplement Table 2 with results for models without pre-training. Comparisons show that pre-training significantly improves performance, especially on small datasets like Cora. This confirms that pre-training helps extract critical features from the data and gather samples with great similarities to form reliable clusters.\n\nTable 2\n\n|Pre-training|Cora|DBLP|Photo|Computers|\n|-|-|-|-|-|\n||84.86±0.58|84.35±0.14|93.53±0.19|90.46±0.22|\n|√|**85.90±0.56**|**84.80±0.17**|**93.71±0.20**|**90.75±0.20**|\n\n**Q3.**\n\nFuzzy boundaries **dynamically adapt to evolving cluster structures**, unlike fixed hyperparameters which fail to adjust to data distribution changes. Since clusters become tight enough during training, the pre-defined hyperparameters may be too large and prone to including samples from other clusters, which degrades the performance quickly. \n\nBesides, Fixed hyperparameters also struggle with feature dimensions of varying scales. Our fuzzy boundary with learnable membership functions flexibly adapts to different feature scales.\n\nFinally, we also provide the experiment results to compare the fuzzy boundary (represented as “Fuzzy”) with the fixed margin expansion in Table 3, which confirms that models with fixed margins require fine-tuning yet still underperform compared with our model, demonstrating the difficulty of using static hyperparameters in a dynamic training process.\n\nTable 3\n\n|$\\epsilon$|Cora|Citeseer|DBLP|Arxiv|Photo|Computers|\n|-|-|-|-|-|-|-|\n|0.05|85.6|73.5|84.3|69.4|93.6|90.5|\n|0.1|85.6|73.0|84.5|69.4|93.5|90.5|\n|0.15|85.5|72.8|84.3|69.4|93.5|90.3|\n|0.2|85.4|72.5|84.3|69.3|93.6|90.1|\n|0.25|85.3|72.6|84.2|69.3|93.2|90.2|\n|0.3|85.1|72.3|84.0|69.2|93.1|90.3|\n|Fuzzy|**85.9**|**74.1**|**84.8**|**69.5**|**93.7**|**90.8**|", "claims_and_evidence": "The evidence such as the t-sne visualization of current graph contrastive learning models could support the problem claims. The method design and the experiment results support the claims that the proposed model could mitigate the shortcomings.", "methods_and_evaluation_criteria": "This proposed method combines the graph contrastive model with the fuzzy boundary in order to extend the original cluster boundaries to fill the discrete and sparse cluster distributions. The method seems to make sense.", "theoretical_claims": "I did check all the theoretical claims and most proofs except for the proof on the model generalization. These theoretical claims and proofs are easy to understand.", "experimental_designs_or_analyses": "The experimental designs to realize comparative experiments are sound. However, the analyses or the explanations for the experiments are not well detailed, such as the definitions of group, group list, and isolated nodes, which are important for the readers to understand.", "relation_to_broader_scientific_literature": "This paper first finds two shortcomings in graph contrastive learning, but I am not sure whether the problems are significant for research on graph contrastive learning.", "essential_references_not_discussed": "Current references can help understand the paper and the comparative models are also new, but comparisons with self-supervised graph generative learning models are also essential.", "other_strengths_and_weaknesses": "**Strengths**\n\n1. Clear structure. The paper is well-written and logically organized, with a clear problem statement, detailed methodology, and thorough experimental analysis.\n\n2. Sound technique. The proposed fuzzy boundary that expands the original cluster boundary with fuzzy set theory appears technically sound. The experiments in different tasks show the comparative performance with graph contrastive models.\n\n**Weaknesses**\n\n1. Limited model diversity in baselines. The selected baseline models mainly focus on contrastive learning methods. To provide a more comprehensive evaluation, the paper could benefit from including comparisons with graph generative models.\n\n2. Troublesome hyperparameter tuning. The proposed method adopts different hyperparameters for various datasets. And sensitive hyperparameters such as the penalty coefficient and temperature coefficient may affect the model performance, which are troublesome to choose for different datasets.", "other_comments_or_suggestions": "See the above.", "questions_for_authors": "1. Quantitative analysis. Hard to understand the concepts such as \"Group\" and \"Group List\" in the quantitative analysis, and it needs to provide clearer and more concrete explanations of them.\n\n2. Ablation study. A reasonable ablation study should test the model without pre-training, which can better reveal the function of the pre-training process.\n\n3. Fuzzy boundary design. The proposed fuzzy boundary seems to be only used for expanding the original cluster boundary but works well. Do the fixed hyperparameters work well or better than the fuzzy boundary?\n\n----------------------\n\nThank you  for your careful responses. All my concerns have been well addressed and I strongly suggest you to add these new results into the final version. I am satified with your answer and decide to raise my scores.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "N4339spIFR", "reviewer": "Reviewer_6dir", "summary": "This paper introduces fuzzy boundaries to address ingrained local cohesion and global sparseness in graph contrastive learning (GCL). To address these issues, this paper proposes a novel GCL model that replaces the crisp cluster boundaries with adaptive fuzzy boundaries to adjust the cluster boundaries. Extensive experiments demonstrate that the proposed method consistently outperforms baseline models across multiple downstream tasks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1.**\n\nWe chose Max Pooling as the Comb operator due to its theoretical advantage, however, it is indeed necessary to compare it with other pooling strategies in practice. We have conducted additional experiments on other Comb operators, and the results on node classification are shown in Table 1.\n\nFrom the results, we could find that the Max Pooling could get the best results across all the datasets since Max Pooling better tightens the samples within each cluster when compared with other general operators. Besides, the samples inside the fuzzy boundary constructed with Max Pooling have the greatest degree belonging to the corresponding cluster, which effectively avoids incurring samples from other clusters.\n\nTable 1. Comparisons among different Comb operators on node classification. \n\n|Comb|Cora|Citeseer|Pubmed|DBLP|Arxiv|Photo|Computers|\n|-|-|-|-|-|-|-|-|\n|Min|85.5±0.6|73.3±0.4|86.3±0.3|83.6±0.3|69.0±0.0|92.8±0.2|89.3±0.3|\n|Mean|85.7±0.5|73.5±0.7|86.6±0.2|84.1±0.3|69.2±0.0|93.1±0.2|90.1±0.2|\n|Max|**85.9±0.6**|**74.1±0.6**|**87.0±0.3**|**84.8±0.2**|**69.5±0.1**|**93.7±0.2**|**90.8±0.2**|\n\n**Q1.**\n\nI guess you may wonder why we choose to use a coordinate difference that is multiplied by one rather than another scaling factor, and I give the following answers. Firstly, the coordinate difference between crisp and fuzzy representations is used to define an acceptable distance (semantic discrepancy) between positives, which is in line with the condition that the scaling factor is one. Secondly, the membership degree is adjusted adaptively during the training process, which can also be viewed as a scaling factor to construct the fuzzy boundaries, so we do not need to set an additional fixed scaling factor.\n\n**Q2.**\n\nWe chose DGI, GRACE, and BGRL as representative models because they are the three types most representative of current contrastive learning approaches. While these models are not the most recent graph contrastive models, they represent mainstream and fundamental GCL paradigms: (1) DGI focuses on a global-local contrastive framework. (2) GRACE adopts a node-level contrastive framework. (3) BGRL is a non-negative-sample-based GCL method. Even though these models adopt different learning paradigms, they share similar shortcomings in the representation distribution, which implies that the shortcomings originate from the inherent contrastive learning mechanism. Besides, existing advanced GCL models inherit these frameworks, so we think these issues still exist in most GCL models, which could be demonstrated in the quantitative analysis that the advanced models present unideal GMs and TINs.\n\n**Q3.**\n\nAccording to the statement in Section 4.3, it is **a general condition** that a larger “Eps” leads to fewer Mean Groups (MGs), but there also exist special cases where a larger “Eps” leads to great MGs since the “Eps” is too small to form groups. In other words, the DBSCAN algorithm is hard to detect valid groups since the minimal distance between the sparsely scattered samples is greater than “Eps”, and these samples are finally viewed as isolated samples. This can be demonstrated by the results that the “MGs” of GREET with “Eps=0.5” is higher than that with “Eps=0.7”, but the number of total isolated samples (TINs) of GREET with “Eps=0.5” is also more than that with “Eps=0.7”. Therefore, it also inspires us to evaluate the models by comprehensively considering both MGs and TINs but not only one of them. That’s also our motivation to propose the two metrics for evaluating global sparseness and local cohesion.\n\n**Q4.**\n\nWe select SOTA methods for a fair comparison. However, they fail to generate an ideal feature distribution for local cohesion and global sparseness. Including outdated methods may reduce these insights due to performance gaps caused by weaker architectures rather than inherent shortcomings. We highlight that even the SOTA models face the challenge to reinforce the necessity of our model, and a similar comparison method is adopted by [1].\n\n[1]Augmentation-free self-supervised learning on graphs, in AAAI 2022.\n\n**Q5.**\n\nNegative correlation learning in ensemble methods encourages diversity among individual learners to enhance overall performance. Similarly, the fuzzy boundary construction introduces several different Gaussian membership functions to make each cluster (individual learner) display flexibility and diversity for fitting various samples. This process prevents over-concentration within local groups and enhances the model ability to capture a more globally structured distribution. \n\nA direct analogy can be drawn to ensemble methods that introduce perturbations or penalize excessive similarity between learners, which in our model is to apply the prototypical contrastive loss for enlarging the distances between different clusters. The exploration for regularization techniques from ensemble learning could inspire additional improvements in optimizing fuzzy boundary formation.", "claims_and_evidence": "The claims made in the submission are supported by relatively convincing evidence. The paper first uses the visualization to detail the problem, and the experiment results show that the fuzzy boundary could solve the problem.", "methods_and_evaluation_criteria": "The proposed model extends the concept of crisp boundaries by introducing fuzzy boundaries, leveraging fuzzy set theory to dilate and then contract these boundaries. In this way, the fuzzy boundaries could tighten all the clusters. Technically, it makes sense for addressing local cohesion and global sparseness.", "theoretical_claims": "I have checked the proof carefully and think the theorem is almost right.", "experimental_designs_or_analyses": "The paper provides sound experimental designs to validate the model performance across multiple datasets and two downstream tasks. The analyses of the experiments are plausible.", "relation_to_broader_scientific_literature": "This paper has found two shortcomings in graph contrastive learning and proposes the fuzzy boundary to solve them. These shortcomings seem to be alleviated according to the visualization results between the proposed model and prior related research.", "essential_references_not_discussed": "The references are enough for me to follow the idea, and I have no essential references to supplement.", "other_strengths_and_weaknesses": "This paper introduces fuzzy boundaries to improve intra-cluster compactness and inter-cluster separateness in contrastive learning, and the method alleviates the shortcomings to realize satisfactory global structure distributions. Additionally, the paper validates the proposed method across multiple datasets, demonstrating its effectiveness in improving node classification and clustering tasks. Still, the paper has suffered from the following weaknesses.\n\nWeaknesses\n\nThe fuzzy boundary only adopts max-pooling as the Union operator. Even though providing the minimal variances theoretically, it is better to provide more experiments with other Union operators in practice to illustrate that max-pooling is a sensible choice.", "other_comments_or_suggestions": "Please refer to the weaknesses.", "questions_for_authors": "1.  In the construction of fuzzy boundaries using crisp and fuzzy representations, why does the author choose to use a coordinate difference that is multiplied by one? \n2. Why do the authors use the visualization of DGI, GRACE, and BGRL to illustrate the shortcomings? Since these models are not the most advanced graph contrastive models, it may cause a problem whether the shortcomings emerge in all the graph contrastive models.\n3. Some results in Quantitative Analysis are confusing. For example, the “MGs” of GREET with “EPS=0.5” and “EPS=0.7” on Citeseer shows high “EPS” will increase “MGs”. However, it is contradictory to the statement that the model gets more expected outcomes when “Eps” becomes greater in Section 4.3.\n4. Why aren't all methods compared in the Node Clustering task?\n5. The negative correlation methods in ensemble learning aim to ensure that while individual learners predict correctly, they also exhibit greater diversity, which aligns with the goals of this paper. Can the authors analyze similar ideas and methods?\n---\nThank you for your thoughtful responses. All of my concerns have been thoroughly addressed. I decided to increase my scores.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces a novel fuzzy boundary mechanism for graph contrastive learning (GCL), addressing two key shortcomings: local cohesion (discrete local groups within clusters) and global sparseness (isolated samples). The proposed method dynamically adjusts cluster boundaries using fuzzy set theory, improving intra-cluster compactness and inter-cluster separability. Experiments on node classification and clustering tasks demonstrate superior performance over state-of-the-art GCL methods.

The paper is with good theoretical grounding (fuzzy boundary construction/proofs) and comprehensive experiments (ablation, sensitivity analysis). The paper makes a contribution to GCL by addressing underexplored shortcomings with a theoretically grounded, empirically validated approach. While computational overhead and reliance on pre-training are noted limitations, all reviewers think that the rebuttal strengthened the paper’s rigor and scope.
Decision: Accept (poster)