Exploitation of a Latent Mechanism in Graph Contrastive Learning: Representation Scattering

Authors: dongxiao  he
lianze  shan
jitao  zhao
hengrui  zhang
zhen  wang
weixiong  zhang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-13 04:32:59

Abstract: Graph Contrastive Learning (GCL) has emerged as a powerful approach for generating graph representations without the need for manual annotation. Most advanced GCL methods fall into three main frameworks: node discrimination, group discrimination, and bootstrapping schemes, all of which achieve comparable performance. However, the underlying mechanisms and factors that contribute to their effectiveness are not yet fully understood. In this paper, we revisit these frameworks and reveal a common mechanism—representation scattering—that significantly enhances their performance. Our discovery highlights an essential feature of GCL and unifies these seemingly disparate methods under the concept of representation scattering. To leverage this insight, we introduce Scattering Graph Representation Learning (SGRL), a novel framework that incorporates a new representation scattering mechanism designed to enhance representation diversity through a center-away strategy. Additionally, consider the interconnected nature of graphs, we develop a topology-based constraint  mechanism that integrates graph structural properties with representation scattering to prevent excessive scattering. We extensively evaluate SGRL across various downstream tasks on benchmark datasets, demonstrating its efficacy and superiority over existing GCL methods. Our findings underscore the significance of representation scattering in GCL and provide a structured framework for harnessing this mechanism to advance graph representation learning. The code of SGRL is at https://github.com/hedongxiao-tju/SGRL.

Main paper: 
 introduction: 
 Graph Neural Networks (GNNs) have shown impressive performance across various fields, including social networks [1, 2] , bioinformatics [3, 4] , and fraud detection [5] . However, training GNNs typically requires large datasets with manually annotated labels, which can be both costly and laborintensive [6] . This limitation restricts their broader application. To address this challenge, Graph Contrastive Learning (GCL) has attracted significant attention [7, 8, 9, 10] , focusing on creating proxy tasks from the data itself to enable self-supervised training of GNN encoders [11] . Currently, most GCL research [6, 12, 13] is concentrated on enhancing one of the three main frameworks: InfoNCEbased (i.e., node discrimination) [14] , DGI-like (i.e., group discrimination) [7] , and BGRL-like (i.e., bootstrapping schemes) [10] .The three mainstream graph contrastive learning frameworks differ significantly, particularly in their approach to node-level tasks. InfoNCE-based methods prioritize node-level discrimination [8, 9, 12] , treating an anchor node as a reference while considering all other nodes as negative samples to enhance the distinctiveness of each node's representation. In contrast, DGI-like methods adopt a group discrimination paradigm [6, 7, 15] , viewing all nodes as positive samples from the same distribution and differentiating them from a noise distribution. BGRL-like methods [10, 13, 16] employ a bootstrapping scheme that eliminates the need for negative samples during training, focusing instead on aligning positive samples. Despite these differences, all three frameworks demonstrate comparable performance, leading us to conjecture that they may share a common mechanism. This idea is partially inspired by previous research in visual contrastive learning, which identified uniformity as a key factor in many contrastive learning methods [17] . However, uniformity, which focuses solely on the discriminative properties of node instances, cannot explain the shared underlying factors across these GCL frameworks, as further discussed in Appendix A.We carried out a detailed analysis of the three GCL frameworks and discovered that representation scattering is a crucial common factor in their success (in Section 3). In Section 3, we provide a formal definition of representation scattering and examine how each framework achieves it. For the DGI framework, we analyze the distributions of original and noise data, proving that after GNN message passing [18, 19] , the noise data distribution aligns with the mean distribution of the original data. This indicates that DGI's objective can be interpreted as distinguishing between the local semantics of nodes within the original graph and its mean, which correlates with representation scattering (Section 3.1). In the InfoNCE framework, existing work has demonstrated that the mechanism of negative sampling facilitates a process of achieving uniformity [17] . We extend this further by theoretically proving that the InfoNCE loss serves as an upper bound for representation scattering loss (Section 3.2). Regarding the BGRL framework, we explore the role of Batch Normalization [20] and its connection to representation scattering, showing that it acts as a specific instance of this concept. Notably, removing Batch Normalization significantly degrades BGRL's performance (Figure 2 ). Our findings confirm that representation scattering is a key mechanism present across all three mainstream GCL frameworks.However, the existing GCL frameworks have not fully leveraged the latent mechanisms inherent in their designs and neglected the interconnected nature of graphs when implementing representation scattering. This oversight results in inefficiencies and reduced robustness. Firstly, generating augmented views or computing similarities between node pairs for representation scattering [7, 8, 9] incurs significant computational and memory overhead. Secondly, manually defined negative samples may result in the generation of numerous false negative samples, introducing noise [12] that can hinder model training. Lastly, those frameworks that rely solely on positive samples [10] implement representation scattering indirectly through Batch Normalization, which lacks explicit guidance for scattering and can lead to suboptimal performance. Addressing these challenges is crucial for enhancing the effectiveness of GCL methods.To fully and effectively utilize representation scattering, we propose Scattering Graph Representation Learning (SGRL). Our approach introduces a Representation Scattering Mechanism (RSM) that embeds node representations into a designated hypersphere, positioning them away from the mean center. This method offers a direct algorithm for representation scattering compared to the existing GCL frameworks, eliminating biases introduced by manually defined negative samples. Additionally, we introduce a Topology-based Constraint Mechanism (TCM) that considers the interconnected nature of graphs. TCM aligns representations derived from structural information with scattered representations, thereby preserving topology information while facilitating scattering. Through these innovations, we aim to enhance the efficiency and robustness of GCL methods. We made the following contributions in this paper:• We discovered a common representation scattering mechanism in GCLs and showed that the three mainstream GCL frameworks implicitly utilize this mechanism, and importantly can be unified under the concept to representation scattering.• We showed that the existing methods do not fully exploit the inherent mechanism of representation scattering. We introduced the novel SGRL framework to integrate RSM and TCM, producing an adaptive scattering approach for model training.• We experimented with various downstream tasks on benchmark datasets and demonstrated the effectiveness and efficiency of SGRL. 
preliminary: 
 Graph Data. We define a graph as G = (V, E), where V = {v 1 , v 2 , . . . , v N } denotes the set of nodes, and E ⊆ V × V represents the set of edges. The node feature matrix is denoted by X ∈ R N ×D , where N is the number of nodes and D is the feature dimension. In addition, the adjacency matrix is indicated by A ∈ R N ×N , formulated such that A ij = 1 when an edge (v i , v j ) exists within the set E, or A ij = 0, otherwise. The degree matrix is denoted as D = diag(d 1 , d 2 , . . . , d N ), where each element d i = j∈V A ij . The degree-normalized adjacency matrix with self-loops is represented asA sym = D -1/2 (A + I)D -1/2 .Graph Contrastive Learning. Given a graph's attributes X and adjacency matrix A, the objective of GCL is to train an encoder f (•) in a self-supervised fashion. The learned encoder f (•) can generate representations H = f (X, A), H ∈ R N ×K , which are both topologies decoupled and dense. These representations can be applied to many downstream tasks. 
representation scattering in gcl: 
 We now examine the shared elements that contribute to the effectiveness of popular Graph Contrastive Learning (GCL) frameworks. Upon revisiting three widely used baseline GCL frameworks, we find that they all inherently utilize the mechanism of representation scattering, which plays a crucial role in their success. Here, we formally define Representation Scattering: According to Definition 1, achieving representation scattering requires identifying a scattered center c within the subspace S k , and simultaneously satisfying the Center-Away and Uniformity Constraints. We will study its relationship with the popular GCL frameworks in the following section.Definition 1. (Representation Scattering) In a d-dimensional embedding space R d comprising n vectors organized into a matrix V ∈ R n×d , consider a subspace S k (1 ≤ k ≤ d) of RDGI-like methods generate negative samples through random permutation of nodes. They employ a mutual information discriminator D(•), which maximizes the mutual information between nodes and their source graphs to train the model [6, 7, 15] . Here, we show that the objective function of DGI is a special case of representation scattering. To facilitate the proof, we present the following assumption: The following results do not strictly require Assumption 1 to be satisfied. Assumption 1 represents a common scenario and serves to simplify the proof. A discussion on the validity of Assumption 1, and proofs of the subsequent results without Assumption 1, can be found in Appendix C. Theorem 1. At the node level, minimizing the DGI loss is equivalent to maximizing the Jensen-Shannon (JS) divergence between the local semantic distribution in the original graph and its average distribution.Proof. Let p data denote the distribution of all nodes in the original graph, characterized by a mean µ and variance σ 2 . To analyze the local distribution of node v i after its embedding aggregation by GNN encoders, we define p i as the distribution of node v i and its first-order neighbors with mean µ i and variance σ 2 i . We make use of a conclusion in [6] and introduce the following lemma: Lemma 2. Minimizing the DGI loss, denoted as L DGI , equals to maximizing the Jensen-Shannon (JS) divergence between the distribution of the original graph G and the corrupted graph G, i.e., Min(L DGI ) ⇔ Max(JS(G ∥ G)).We discuss Lemma 2 in Appendix D.1. Lemma 2 establishes the relationship between the DGI loss and the distributions of the original and corrupted graphs. To investigate the distribution of representations in original and corrupted graphs, we focus on the case of a single-layer GNN [21] and have the following formulation:EQUATIONwhere W ′ = ξ(W), W ∈ R D×K with ξ being an activation function like ReLU [22] for ease of understanding, N i denotes the set of first-order neighbors of node v i , inclusive of v i itself. In Eq. 1, ∀j ∈ N i , x j ∼ p i (µ i , σ 2 i ). For node v i , subsequent to GNN message passing, we compute the mean and variance of the aggregated representation h i as follows:E[h i ] = E   j∈N (i) α ij x j • W ′   = j∈N (i) α ij E[x j ] • W ′ = j∈N (i) α ij µ i • W ′ , Var(h i ) = Var   j∈N (i) α ij x j • W ′   = j∈N (i) α ij Var(x j ) • W ′2 = σ 2 i j∈N (i) α 2 ij • W ′2 .(EQUATIONGiven that Ã is a normalized propagation matrix with j∈N (i) α ij = 1 for the aggregated representation h i in the original graph, the mean of representation h i 's distribution equals µ i • W ′ . We then analyze the distribution of the aggregated representation ĥi in the corrupted graph. Based on Assumption 1, the adjacency matrix A is unaltered and the feature vector xi is selected randomly from the feature matrix X of the original graph, following the distribution p data (µ, σ 2 ). According to Eq.2, the mean of the distribution of ĥi is µ • W ′ , and the variance isσ 2 j∈N (i) α 2 ij • W ′2 . Consequently, we have h i ∼ p ′ i (µ i • W ′ , σ 2 i • W ′2 ) and ĥi ∼ p ′ data (µ • W ′ , σ 2 • W ′2), which directly prove that maximizing the JS divergence between the original and corrupted graph distributions at the node level maximizes, in effect, the JS divergence between the local semantic distribution of node v i and the mean distribution of the nodes in the original graph.Corollary 3. Taking the mean of the original graph as the center c, and the original representation space as a subspace S k , the objective of DGI can be described as follows: within the subspace S k , DGI increases the distance between the nodes of the original graph and its center c, achieving the objective of representation scattering. Corollary 3 reveals, for the first time, that the primary goal of DGI-like methods is to position node representations away from a central point to encourage a uniform distribution of the nodes. To illustrate these theoretical insights, we performed a visualization experiment. As shown in Figure 1 (a) and (b), both a randomly initialized GNN and a trained singlelayer GNN demonstrated that their representations are distanced from the mean of the original graph nodes. Additionally, Figure 1 (b) and (c) provide clearer evidence that minimizing the L DGI is equivalent to maximizing the Jensen-Shannon (JS) divergence between the positive and negative samples, i.e., maximizing the JS divergence between the local semantic distribution in the original graph and its mean distribution. However, the row-wise shuffling mechanism may introduce potential disturbances. In a graph with n nodes, each node v retains its local semantic distribution with a probability of 1/k during the shuffling process, where k denotes the number of distinct node types, assuming class balance. Despite this, the non-discriminatory nature of the perturbation means that these unchanged nodes could still be mistakenly classified as negative samples. Consequently, genuine node representations may be incorrectly labeled as negatives, leading to bias in the learning process. The overlapping nodes depicted in Figure 1 intuitively support this perspective.We now show that the mechanism of negative sampling in the InfoNCE-based methods is equivalent to representation scattering. Theorem 4. Let h = 1/n n i=1 h i , and sim(•) be the cosine similarity function (here, h i represents the encoded representation of node v i ). For node v i , the lower bound of the InfoNCE loss L InfoNCE (h i ) exists: L InfoNCE (h i ) ≥ sim(h i , h) + ln(2n).A detailed proof is given in Appendix D.3. Theorem 4 indicates that when minimizing L InfoNCE (h i ), the similarity between node v i and mean node v, i.e., sim(h i , h) is also minimized. Taking the mean of all nodes as the scattered center c, and the hypersphere as the subspace S k , the mechanism of negative sampling is equivalent to representation scattering. However, the InfoNCE loss function is inefficient for representation scattering as it needs to compute and reduce the similarity of each negative pair. Moreover, it indiscriminately treats all negative samples, ignoring the distinctions among them, which leads to inappropriate scattering of negative samples and potential bias from false negatives. Consequently, many recent methods have incurred additional computational overhead by manually and intuitively defining positive and negative samples [12, 23] .All BGRL-like methods incorporate a key component: Batch Normalization (BN). For a feature vector x i and its corresponding batch statistics, mean µ and variance σ 2 , the BN is applied as follows: BN(x i ) = γ(x i -µ)/ √ σ 2 + ϵ + β, where γ and β are learnable parameters that scale and shift the normalized value, and ϵ is a small constant added for numerical stability. Theorem 5. The process of data normalization by batch normalization can be seen as a special case of representation scattering. A detailed proof can be found in Appendix D.2. To empirically assess the impact of batch normalization on BGRL, we conducted experiments comparing both BGRL and BGRL w/o BN across four benchmark datasets. The results, shown in Figure 2 , reveal a significant drop in accuracy for BGRL across all datasets without batch normalization, highlighting its critical role in the bootstrapping framework. While BGRL incorporates representation scattering through batch normalization, its training process lacks explicit guidance or a dedicated mechanism to efficiently manage this scattering. The absence of direct supervision during the representation scattering phase can lead to an unoptimized distribution of representation within the embedding space, resulting in suboptimal performance.In summary, our analysis of the three mainstream GCL frameworks theoretically proved that they all inherently utilize representation scattering but also fail to fully utilize this effective mechanism. This analytic result has motivated us to design a more effective representation scattering method tailored for learning on graphs. 
methodology: 
 The proceeding sections have revealed the importance of representation scattering in GCLs. Based on the findings, we have designed a novel method, namely Scattering Graph Representation Learning, short-handed as SGRL (Figure 3 ). We introduce the components of SGRL and provide discussion in the following sub-sections.To address the shortcomings in the application of representation scattering within the three mainstream graph contrastive learning frameworks, we design RSM to explicitly guide the target encoder in learning scattered representations. Following Definition 1, we introduce a subspace S k and a scattered Align Figure 3: The overview of SGRL. Consider a graph G processed using two distinct encoders (online encoder and target encoder): f θ (•) with parameters θ and f ϕ (•) with ϕ, aimed at generating node representations Honline and Htarget, respectively. For Htarget, the mean representation of all nodes is calculated to serve as the scattered center c. The parameters of f ϕ (•) are updated via RSM that encourages node representations to diverge from c. Honline is processed through TCM to incorporate topology information, resulting in H topology online . Subsequently, H topology online is embedded through a predictor q θ to predict Htarget, and the parameters in f θ (•) is updated through back-propagation while stopping the gradient of f ϕ (•). Both channels are trained simultaneously. At the end of each epoch, we employ an Exponential Moving Average (EMA) to update parameters ϕ. Finally, the representations generated by f θ (•) are employed across various downstream tasks. center c to effectively perform representation scattering. For the subspace S k , a transformation function Trans(•) is introduced to transform representations from the original space R d into S k . Specifically, we apply ℓ 2 normalization to each row vector h i in the matrix H target :EQUATIONwhere h i is representation for node v i ∈ V, generated by the target encoder,∥ hi ∥ 2 = ( k j=1 h2 ij ) 1 2, and ϵ is a small value to avoid division by zero. As defined by Eq. 3, the representations of all nodes are distributed on a hypersphere S k . This mapping prevents arbitrary scattering of representations in the space, avoiding instability and optimization difficulties during training.Next, we define the scattered center c and introduce a representation scattering loss function L scattering in S k to push node representations away from the center c, as formulated as follows:EQUATIONThrough Eq. 4, SGRL achieves uniformity of representations globally across the entire dataset, without emphasizing local uniformity. Specifically, RSM enables representations of different semantics to be globally scattered across the hypersphere while accommodating representations of the same semantics to aggregate locally.Discussion of RSM. We now provide a theoretical analysis demonstrating that the proposed representation scattering mechanism outperforms the three graph contrastive learning frameworks. RSM achieves representation scattering more effectively by encouraging distances between node representations and the scattered center, eliminating the dependence on manually designed negative samples. Most traditional methods [7, 8, 9, 15] rely on negative samples to indirectly promote representation scattering, which is inefficient and introduces biases. DGI-like methods, as discussed in Section 3.1, aim to maximize the Jensen-Shannon (JS) divergence between the distribution of the original graph and its mean distribution. Based on this, generating additional noise graph is inefficient. Moreover, some negative samples generated through random shuffling may align with the distribution of positive samples, resulting in false negative samples that hinder model training. InfoNCE-based methods consider all nodes as negative samples, except the matching ones in two augmented views. While pushing nodes away from each other ensures the discriminability of each node, it also results in significant computational overhead. Moreover, due to the potential conflict between the encoder's message-passing mechanism and the InfoNCE loss function, many negative samples can not effectively distance themselves from each other [24] . Therefore, by employing a center-away strategy, RSM effectively reduces the additional computational overhead and mitigates the biases caused by manually designed negative samples.After obtaining the scattered representations H target = f ϕ (A, X) through the target encoder, it is necessary to consider the differences in the degree of scattering of different node representations.Consider the interconnected nature of graphs, the representations of two nodes connected topologically should be closer in space S k . Specifically,∀v i , v j ∈ V , h i , h j ∈ S k , given a threshold d: ∥h i - h j ∥ 2 2 < d, if (i, j) ∈ V and ∥h i -h j ∥ 2 2 > d, if (i, j) / ∈ V.Intuitively, to this end, a simple way is to replace the individually scattered representations with the aggregated representations H topology target from first-order neighbors: H topology target = AH target . However, attempting to consider the topology information and achieve representation scattering through the same encoder may lead to conflicts.To address this issue, we propose a Topology-based Constraint Mechanism (TCM). Specifically, we separate the process of constraint from the process of scattering by letting the online encoder generate topologically aggregated representations instead of the target. The online encoder enhances its representations by summing the original representations H online with the topologically aggregated representations of its k-order neighbors Âk H online , which can be described as:EQUATIONwhere k represents the order of neighbors and Â = A + I is the adjacency matrix with selfloops. By separating scattering and constraints, SGRL can effectively achieve a balance between representation scattering and topology aggregation adaptively, rather than setting the scattering distance empirically. Next, the topology representations H topology online are fed into a predictor q θ (•) to generate the predicted representations Z online = q θ (H topology online ). Our objective is to align the predicted topology representations Z online closely to the scattered representations H target , enhancing the model's effectiveness in capturing the essential semantic details of the graph. Based on this, the alignment loss L alignment is defined as follows:EQUATIONwhere Z online and H target represent the predicted and scattered representations, respectively. During this process, the online encoder's parameters θ are updated and the target encoder's parameters ϕ stop gradient propagation. Compared to directly aligning constrained and scattered representations, this predictive objective can act as a buffer, allowing the online encoder to adaptively learn scattered representations and topology information. Furthermore, to make the target encoder consider topological semantic information into the process of representation scattering, instead of solely focusing on scattering, we employ an Exponential Moving Average at the end of each training epoch:EQUATIONwhere τ is a target decay rate and τ ∈ [0, 1]. Eq. 7 effectively mitigates the adversarial interactions between RSM and TCM, while also facilitating the integration of topology information into the representation scattering process. Moreover, RSM enhances the discriminability of representations, while TCM incorporates topology information into the representations. The interaction between these two mechanisms effectively mimics the role of data augmentation, i.e., train encoders to learn the invariance of data to perturbations. Consequently, SGRL obviates the need for explicitly designing data augmentation strategies, which leads to additional computational overhead and heavy reliance on the choice of augmentation techniques. 
experiment: 
 We evaluated SGRL on the five of the most widely used benchmark datasets, including Amazon-Photo (Photo) and Amazon-Computers (Computers) [25] , WikiCS [26] , Coauthor-CS (Co.CS) and Coauthor-Physics (Co.Physics) [27] . Detailed information of these datasets is in Appendix B.2. We compared SGRL with four types of methods: (1) Three mainstream baselines: GRACE [8] , DGI [7] , and BGRL [10] . (2)Six recently advanced algorithms: GCA [9] , ProGCL [12] , AFGRL [13] , iGCL [28] , GBT [29] , MVGRL [15] . (3)Two classic graph representation learning methods: Node2vec [30] and Deepwalk [31] . (4)The semi-supervised training baseline GCN [18] . We utilized the representations generated by the online encoder for downstream tasks. For node classification, we followed the evaluate scheme from [10] . We trained a simple linear model using only the representations from a logistic regression loss with an ℓ 2 regularization and no backpropagation of any gradient to the graph encoder network. Specifically, we trained the downstream classifier using 10% of the data and tested the classifier on the remaining 90%. We ran SGRL 20 times and report here the average and standard deviation of the F1-score. For node clustering, we adopted the evaluation method from [13] . The testing was conducted on the learned representations at each epoch, and the best performance is reported below. More details of the experiments can be found in Appendix B. Overall evaluation. Table 1 presents the averages and standard deviations of the F1-scores for all methods on the node classification task. Some statistics for the existing methods are reported from either their original papers or [13] . As shown in the table, the proposed SGRL exhibits superior performance across all five datasets tested, achieving the highest accuracy. Compared to the three baselines, BGRL, GRACE, and DGI, our new method outperforms them by 1.23%, 2.14%, and 3.26%, respectively, showing SGRL's effectiveness in more effectively exploiting representation scattering. Moreover, we observe that SGRL outperforms methods like GCA, AFGRL and iGCL, which focus on improving upon data augmentation. This demonstrates the validity of employing a graph contrastive framework based on representation scattering and topology aggregation, instead of using data augmentation. SGRL also outperforms the advanced negative sampling method, ProGCL. We attribute this to SGRL's representation scattering, which does not explicitly define negative samples, achieving better performance than manually defined negative samples. When evaluated on node clustering tasks using the scheme in [13] , it can be observed that SGRL achieves the best or second-best accuracy on most datasets. Although SGRL attempts to make node representations scattered, which seems unsuitable for clustering tasks, it still manages to deliver competitive performance. This demonstrates that TCM can preserve the topology information of the original graph in the process of representation scattering, ensuring that while Hyperparamter Analysis. In this subsection, we investigate the sensitivity of the hyperparameter k in SGRL, as shown in Eq. 5. The parameter k represents the order of neighbors aggregated, which directly influences the strength of the topology constraints. In our experiments, we adjust k in the range 0, 1, 2, • • • , 7 to evaluate the impact of different constraint strengths on SGRL. The results are shown in in Figure 5 : when k = 0, i.e., "SGRL w/o TCM", the lack of topology constraints leads to a decrease in model performance. As k increases, the constraint ability of TCM gradually strengthens, and the model performance exhibits a unimodal shape with respect to changes in k. This highlights that weak topology cannot preserve adequate topological information for adaptive representation scattering, whereas strong topology may excessively restrict the scattering of representations, which aligns with the perspectives proposed in Sections 3 and 4.Ablation Studies. To verify the effectiveness of each component of SGRL, we conducted ablation studies on five datasets. As shown in Table 3 , "SGRL w/o TCM and TCM" demonstrates significant performance improvements on four datasets with the addition of RSM alone, i.e., "SGRL w/o TCM", confirming its effectiveness. We observe that there is a slight performance decrease on the Computers dataset, which we attribute to over-scattering resulting from the absence of constraints during representation scattering. Additionally, a comparison between SGRL and "SGRL w/o TCM" reveals that SGRL achieves improved accuracy across all five datasets, particularly on the Computers dataset. This indicates that the TCM effectively constrain scattering, achieving an adaptive representation scattering, verifying our view in Section 4.2. Finally, we performed an ablation study on EMA. Given the potential adversarial relationship between RSM and TCM, we employed EMA to balance this effect. The experimental results in Table 3 highlight the necessity of incorporating EMA to mitigate the adversarial interaction between RSM and TCM. 
conclusion: 
 In this paper, we made two significant contributions to Graph Contrastive Learning (GCL), an actively researched subject with numerous applications across diverse domains. First, through a comprehensive analysis of the three popular GCL frameworks, we discovered a common latent mechanism -representation scattering -that underlies these distinct contrastive methods. This discovery highlights an essential feature of GCL and unifies these seemingly disparate frameworks under the concept of representation scattering. Despite their popularity, the existing methods have not fully leveraged this mechanism, leaving the potential of representation scattering largely untaped. However, applying representation scattering directly to GCL poses technical challenges. Our second contribution addressed this issue by introducing the Representation Scattering Mechanism (RSM) and the Topology-based Constraint Mechanism (TCM), which we integrate into a novel GCL approach named Scattering Graph Representation Learning (SGRL). In SGRL, we effectively balance the adversarial relationship between RSM and TCM using an Exponential Moving Average (EMA) strategy. Extensive experimental results on benchmark datasets validate the effectiveness of our proposed method. Future work will further explore the broader implications of representation scattering beyond GCL, discussed in Appendices F and G. 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: The paper discuss both potential positive societal impacts and negative societal impacts of the work performed in Appendix G.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: The paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: The creators or original owners of assets used in the paper are properly credited and the license and terms of use are explicitly mentioned and properly respected.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA]Justification: The paper does not release new assets.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA]Justification: The paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA]Justification: The paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.• We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: In visual contrastive learning, uniformity is recognized as a critical factor contributing to the effectiveness of contrastive methods. In [17] , uniformity is defined as the condition where feature vectors should be roughly uniformly distributed on the unit hypersphere, preserving as much information of the data as possible. This concept is based on the InfoNCE loss. Due to the similarities in implementation mechanisms between the SimCLR framework and the Deep InfoMax (DIM) framework, DIM can be considered a special case of SimCLR when the number of negative samples within a batch is limited to one. Consequently, this concept can be seen as a key factor contributing to the success of most visual contrastive learning methods.However, this concept fails to adequately unify the frameworks within graph contrastive learning, particularly for DGI framework. In DGI, all nodes of the original graph are treated as a single distribution, whereas the distribution generated from a randomly shuffling noise graph is treated as another. The encoder is trained by maximizing the Jensen-Shannon (JS) divergence between these two distributions. First, the representations in DGI do not conform to a distribution on the unit hypersphere. Moreover, considering all nodes of the original graph as one distribution more closely aligns with a process of aggregation rather than facilitating a uniform distribution of nodes throughout the space.Furthermore, while the negative sampling mechanism of InfoNCE-based methods implements uniformity, the interconnected nature of graphs can lead to conflicts during message passing within the encoder, which hinders the effective distancing of negative samples [24] . Therefore, we conducted a comprehensive theoretical analysis of three mainstream graph contrastive frameworks (Section 3) and introduced the concept of representation scattering. Specifically, the center-away constraint within representation scattering provides a more detailed elucidation of the operational mechanisms underlying the DGI framework than uniformity. Moreover, this constraint effectively mitigates the previously discussed conflicts, thereby facilitating the integration of representations and topological information and enabling adaptive representation scattering. We use Wiki-CS, Amazon-Computers (Computers), Amazon-Photo (Photo), Coauthor-CS and Coauthor-Physics to evaluate the performance of SGRL. The detailed statistics of all the used datasets are in Table 5 Wiki-CS [26] is a directed graph dataset sourced from Wikipedia. It comprises nodes that represent articles in the field of computer science, with edges corresponding to the hyperlinks interconnecting these articles. Each article is categorized into one of 10 associated subfields. The attributes of the nodes are derived as the average of the text embeddings for the respective articles, making it the sole dataset characterized by dense attributes.Amazon-Computers and Amazon-Photo [25] are networks that map out the co-purchase relationships among products on the Amazon platform. In these networks, nodes represent products, and edges are established between products that are frequently bought together. Products within each network are categorized into 10 and 8 distinct classes respectively, according to their product categories, and node features are represented using a bag-of-words model from the product reviews.Coauthor-CS and Coauthor-Physics [27] represent two academic networks based on the Microsoft Academic Graph, illustrating the co-authorship links among academics. In these networks, nodes correspond to individual authors, while edges reflect collaborative authorship between them. Authors in each network are categorized into 15 and 5 research fields respectively, with node features encapsulated as a bag-of-words representation derived from the keywords of their publications.All experiments were carried out on an NVIDIA GeForce GTX 3090 GPU, which comes equipped with 24GB of memory. For model development, we utilized PyTorch version 1.13.1 [33] , along with PyTorch Geometric version 2.3.0 [34] , which also served as the source for all the datasets used in our study. The detailed hyperparameters for node classification and clustering are shown in Table 4 BOur method employs GCN [18] as the encoder. The encoding process can be described as follows:EQUATIONwhereH (l)k represents the node embedding matrix at the l-th layer during the k-th epoch of encoding and H (l) 0 is initialized by graph feature matrix X,and A is the adjacency matrix without self-loops. In addition, the encoder structure is defined as:EQUATIONwhere Â = A + I is the adjacency matrix with self-loops, σ(•) is the PReLU activation function, D is the degree matrix of Â, and W (l) is the trainable weight matrix for the l-th layer. In our experiments, we set l = 1.In assumption 1.(a), we define the normalized propagation matrix as Ã = D -1 A.When the encoder is GCN [18] , assumption 1.(a) holds. Next, we will discuss whether assumption 1.(a) holds in Corollary 3 when the encoder is GAT [19] . The message-passing formula for GAT is similar to that of GCN. For node v i , h i = GAT(A, X) = neighbor j α ij x ij • W, but the method for obtaining the propagation matrix is different:EQUATION)where a is the weight vector used to compute the attention scores, ∥ • ∥ denotes the concatenation operation, W is the weight matrix, and LeakyReLU is the activation function. In the original graph, node v i and its first-order neighbors follow a distribution with mean µ i and variance σ 2 i , i.e., ∀j ∈ N i , x j ∼ p i (µ i , σ 2 i ). Subsequent to GAT aggregation, the representation of node v i , i.e., h i follows a local semantic distribution with a mean given by E ( α ij x ij ) = α ij µ i and a variance of σ 2 i α 2 ij . This is consistent with the conclusion in Section 3.1. In the perturbed graph, the method for calculating the aggregated distribution differs from that of GCN. For GCN, the propagation matrix α ij = 1/d i is independent of the distribution of node v i and its neighbors. In contrast, for GAT, as shown in Eq. 10, α ij reflects the correlation between node v i and its neighbor v j . Therefore, in the perturbed graph, there are two distributions present among node v i and its first-order neighbors: the local semantic distribution p i and the original data distribution p data . Let a = α ik , wherex k ∼ p i (µ i , σ 2 i ). Let b = α ik ′ , where x k ′ ∼ p data (µ, σ 2). After aggregation through GAT, the mean of the distribution that node v i follows is a • µ i + b • µ. In this case, minimizing the DGI loss is equivalent to maximizing the local semantic distribution of node v i with a mean of a • µ i + b • µ and a variance of σ 2 i α 2 ik + σ 2 α 2 ik ′ . In our problem, we aim to find a subspace S k and a scattered center c to achieve representation scattering, and this goal can be accomplished with either GCN or GAT. To this end, Corollary 3 also holds when the encoder is GAT.For assumption 1.(b), there are two ways to generate a corrupted graph in DGI [7] : one is by randomly shuffling the feature matrix X, and the other is by keeping X unchanged and shuffling the adjacency matrix A. Here, we discuss whether Theorem 1 still holds when the assumption is the latter.We have already demonstrated in Section 3.1 that, in the original graph G, the aggregated features of node v i follow a distribution with mean µ i and varianceσ 2 i j∈N (i) α 2 ij .In the corrupted graph G, randomly shuffling the adjacency matrix A is equivalent to randomly re-wiring n nodes, wheren i=1 n j=1 A ij = n i=1 n j=1Âij . The first-order neighbors of node v i still derive their features from the original graph distribution, i.e., xij ∼ p data (µ, σ 2 ), and in the corrupted graph, the new topology still satisfies (α ij ) = 1. Therefore, the aggregated representation follows a distribution with mean µ and variance σ 2 j∈N (i) α 2 ij , which is consistent with the conclusion derived from assumption (b).In our theoretical analysis, we assume class balance, which is common in many theoretical proofs [7] . When the classes are imbalanced, the conclusion is similar to what we explained in assumption 1. (a): the scattered center changes from the mean to a • µ i + b • µ, i.e., it leans towards the classes with a larger number of instances. This often occurs in long-tail distribution problems. We will further explore this aspect in our future work.Proof. Here, we rewrite the proof from work [6] for ease of understanding Lemma 2. We replace the summary vector s with an all-ones vector. By removing the weight parameters W in the discriminator D , we obtain:EQUATIONwhere sum(•) is the summation function for every dimension in vector h i . As described by Eq. 11, minimizing DGI loss can be viewed as a binary classification problem distinguishing between sum(h i ) from the original graph and sum( hi ) from the perturbed graph. Therefore, minimizing DGI loss is equivalent to maximizing the JS divergence between the original and perturbed graphs.Proof. Consider the output of a single-layer Graph Neural Network (GNN) as H ∈ R N ×K , where N represents the batch size and K denotes the dimension of representations. Each column H j of H represents all samples for a particular feature, with mean µ j and standard deviation σ j .Batch Normalization (BN) processes each feature dimension j independently by calculating the mean µ j and variance σ 2 j , and then applying the following transformation:EQUATIONwhere ϵ is a small value to prevent division by zero. The initial representations H are distributed on an arbitrary space characterized by a mean center µ = {µ 1 , . . . , µ K }. After batch normalization, the space of the transformed representations H ′ is standardized to a space with zero mean and unit variance. This transformation effectively constitutes a center-away displacement, i.e., each feature vector is shifted away from the center of the original distribution. Moreover, Batch Normalization ensures that each feature dimension exhibits a variance of unity, thereby facilitating a more uniform distribution across the transformed representations. Although the primary objective of batch normalization is not representation scattering, it can be inferred from the aforementioned analysis that BN's adjustment of data distribution indirectly facilitates a more uniform scattering of representations across space.Proof. It is well-known that the InfoNCE loss (with ℓ2 norm) is enforcing the embeddings to be uniformly distributed in a hypersphere. It is exactly representation scattering. We begin with analysing the formula of the InfoNCE [14] loss:EQUATIONwhere h i denotes the representation of node v i , obtained from one augmented view of the original graph, and h ′ i represents the representation of the corresponding node v ′ i from the other augmented view. Here, θ(h i ,h ′ i ) = Sim(h i , h ′ i )/τ, where Sim(•) denotes the cosine similarity between node representations. Specifically, Sim(h i , h ′ i ) is computed as the normalized dot product:Sim(h i , h ′ i ) = (h ⊤ i h ′ i )/(∥h i ∥∥h ′ i ∥).Eq. 13 states that minimizing the loss function requires the maximization of similarity among positive samples and the simultaneous minimization of similarity among negative samples.To simplify the proof, we define θ(h i , h ′ i ) as the cosine similarity between node pairs, omitting the hyper-parameter τ . Furthermore, we replace the logarithm with the natural logarithm ln in Equation ( 13), yielding the expression below:EQUATIONTo facilitate a clearer analysis of Equation ( 14) and given the convex nature of the exponential function, we simplify the expression using Jensen's Inequality:EQUATIONEQUATIONwhere θ j represents the cosine similarity between the anchor node v i and other nodes that form negative pairs. According to Eq. ( 15), the sum of exponentials can be transformed into the exponential of a sum, which can then cancel out with the logarithm, thereby simplifying the equation:EQUATIONIn Eq. ( 16), it is demonstrated that the InfoNCE loss associated with the anchor node v i serves as an upper bound to the sum of cosine similarities between v i and all negative samples. To elucidate the relationship between representation scattering and InfoNCE loss more clearly, the following lemma is presented: Lemma 6. Let v i be the anchor node, and let{v 1 , v 2 , v 3 , • • • , v n } be the set of nodes, excluding v i .The average of the sum of cosine similarities between v i and all other nodes in the set is equivalent to the cosine similarity between the ℓ 2 -normalized vector of v i and the ℓ 2 -normalized mean vector obtained from all other nodes.The proof is provided in Appendix D.4. Lemma 6 demonstrates that the minimization of Eq. ( 13) for the anchor node v i entails a concurrent reduction in the cosine similarity between the ℓ 2 -normalized vector of v i and the ℓ 2 -normalized centroid of all other nodes. This process can be viewed as transforming the set of nodes onto a hypersphere with a radius of 1 and distancing the anchor node from the centroid of other nodes. We consider this hypersphere as subspace S k , and the centroid as the scattered center c, which aligns with our definition of representation scattering. Thus, minimizing the InfoNCE loss effectively promotes representation scattering.Proof. Given two vectors v i and v j , their cosine similarity is defined as vivj ∥vi∥∥vj ∥ , which is the cosine of the angle between themEQUATION)For ℓ 2 -normalized vectors vi = vi ∥vi∥ , this is equivalent to their dot product: vi • vj :EQUATIONSince the dot product of vectors possesses the distributive property of multiplication:EQUATIONFinally, we compute the mean of Eq. 19 :EQUATIONAs a consequence, the mean of the sum of cosine similarity between the anchor node v i and all other nodes can be regarded as the expected similarity of the ℓ 2 -normalized v i with all other ℓ 2 normalized points.Graph Representation Learning (GRL) has attracted substantial attention in recent years due to the broader application of graph data across various real-world scenarios [35] . The primary goal of GRL is to embed graph data, which is inherently high-dimensional, sparse, and non-Euclidean, into a lowerdimensional, dense, Euclidean space to facilitate easier processing in downstream tasks. Early GRL works rely on Matrix Factorization [36] and Random Walk [37] , failing to simultaneously utilize the structural and attribute information of graphs [38] . In recent years, Graph Neural Networks (GNNs) have achieved great success, such as GCN [18] , GAT [19] , GraphSAGE [39] , which effectively extract graph attribute and structural information simultaneously through message passing and aggregation mechanisms. However, the training of GNNs requires a large amount of manually annotated labels, which is expensive and labor-intensive [6] . To overcome this limitation, Graph Contrastive Learnings (GCLs) have emerged as a promising and increasingly popular learning paradigm for handling unlabeled graph data.Currently, there are three mainstream graph contrastive frameworks, each inspired by methods from visual contrastive learning. DGI [7] , inspired by Deep InfoMax (DIM) [40] , generates a noise distribution through random shuffling and learns node representations by maximizing the Jensen-Shannon (JS) divergence between the semantic distribution of the original graph and the noise distribution. GRACE [8] , a representative method based on InfoNCE loss and inspired by SimCLR [41] , generates two correlated graph views through augmentation techniques. This framework employs the InfoNCE loss to enhance the similarity of identical nodes across the two views while minimizing the similarity among different nodes. BGRL [10] , inspired by BYOL [42] , utilizes two distinct encoders: an online encoder and a target encoder. The online encoder is trained to predict the output of the target encoder, while the target encoder is updated by Exponential Moving Average (EMA).Although the three mainstream graph contrastive learning frameworks: GRACE, DGI, and BGRL, have achieved notable success in addressing the challenges of unlabeled graph data, the underlying mechanisms driving their success have not been fully explored. Current research predominantly focuses on the implementation and optimization of these frameworks [6, 9, 12, 13] , with insufficient understanding of their theoretical foundations and intrinsic principles. In visual contrastive learning, studies have revealed that "uniformity", which refers to the uniform distribution of representations in hypersphere, is a key factor contributing to improved generalization and discriminative power of models [17] . However, due to the non-Euclidean structure of graphs and the complex dependencies among nodes, this theory cannot be directly applied to graph contrastive learning. The unique characteristics of graphs necessitate a reevaluation and adaptation of the uniformity concept to develop a corresponding theoretical framework suitable for graphs. In this paper, we reanalyze the three mainstream graph contrastive learning frameworks and propose a new mechanism: representation scattering, which is prevalent and important in these frameworks. Furthermore, considering the interconnected nature of graphs, we design a constraint mechanism to achieve an adaptive representation scattering.F A Detailed Discussion on the Limitations• Application of Heterogeneous Datasets. In this work, we focused exclusively on homogeneous datasets and achieved satisfactory results. Our experiments demonstrated the effectiveness of the proposed topology-based constraint under the assumption of homogeneity. However, this constraint may face challenges when applied to heterogeneous datasets. Heterogeneous datasets often exhibit varying structures and characteristics, which could undermine the performance of our method. Consequently, addressing these challenges and adapting our approach to handle heterogeneous datasets effectively is a crucial direction for future research. We plan to investigate and refine our method to ensure its robustness and applicability across diverse and complex datasets. • The Methods of Balancing RSM and TCM. RSM aims for the representations of nodes to be scattered from each other, while TCM prevents excessive scattering among nodes with the same semantics. These two mechanisms inherently have an adversarial relationship.We use EMA to balance this adversarial relationship, which is just one of many possible methods.As a preliminary exploration in the field of GCL, SGRL has the following impacts:• Uncovering potential success factors in GCL: SGRL explores three different mainstream graph contrastive methods and identifies their common underlying factors, providing a new perspective for future GCL research. Justification: For each experiment, the paper provide sufficient information on the computer resources needed to reproduce the experiments.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).Justification: The research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 


Decision: This paper discovers a common mechanism hidden behind main-stream GCL frameworks such as Info-NCE-based methods, DGI, and BGRL. Specifically, the authors prove that the mainstream baselines implicitly employ the representation scattering law (a combination of the center away and uniformity constraint). 

This is an important contribution to the GCL community, and reviews are consistently positive. I thus recommend accepting this paper.

Reviews: review: paper_summary: paper_summary: The authors attempt to propose a universal theory of graph contrastive learning which may benefit this field. Most existing GCLs directly inherit from other fields. While existing GCLs have achieved similar success, there are intuitive differences and even conflicts in the operations. By analyzing three representative GCL frameworks from the unique perspectives of topology and message passing, the authors of this work find a key factor behind GCLs, which is defined as representation scattering. To better achieve node representations scattering, they mine the natural linking characteristic of graphs and propose a new contrastive concept. It involves contrasting nodes with embedding centers and implementing aggregation constraints based on graph topology, which replaces the traditional inefficient augmentation and sampling-based GCL proxy tasks. The proposed method is compared to existing GCL methods across multiple downstream scenarios to demonstrate the superiority, confirming it can learn high-quality node representations in a self-supervised manner.
strength_weakness: strength_weakness: 1. The proposed SGRL is an augmentation-free framework, avoiding manual bias and reducing training overhead in augmentation. To my knowledge, there are also some augmentation-free contrastive methods available, such as [1, 2]. While data augmentation is not the main focus of this paper, it may be beneficial if the authors provide a comparison between SGRL and this type of method.

2. In SGRL, an existing component, EMA [3, 4], is given new functionality to balance the adversarial effects of two constraints (RSM and TCM) and achieve satisfactory results. I find this innovation interesting. However, it would be better if the authors provided further discussions and experiments on the chosen EMA’s hyper-parameters, since different hyper-parameters of EMA may make the two adversarial branches achieve different equilibrium.

3. The provided theory primarily discusses the impact of representation scattering mechanisms on various GCLs, encompassing multiple aspects such as data augmentation, GNN encoders, and negative contrasting. However, the authors seem to have omitted positive contrasts. While topological aggregation constraints can be seen as a more effective form of positive sample contrast, I still hope the authors can incorporate positive contrast into the proposed theory.

Some minor points:

1. p. 2 line 72 - I am not sure what is meant by "learning invariance post-disturbance."

2. p. 3 line 149 - Figure 1 is mentioned too frequently within a single section.

3. Table 1: Some names of the datasets used have prefixes (Co.CS and Co.Physics) while others do not (Computers and Photo). It would be better to maintain a consistent format.

[1] Lee N, Lee J, et al., Augmentation-free self-supervised learning on graphs, AAAI, 2022.

[2] Yu J, Yin H, et al., Are graph augmentations necessary? simple graph contrastive learning for recommendation, SIGIR, 2022.

[3] Thakoor S, Tallec C, et al., Large-scale representation learning on graphs via bootstrapping, ICLR, 2022.

[4] Grill J B, Strub F, et al., Bootstrap your own latent-a new approach to self-supervised learning, NeurIPS, 2020.
questions: questions: The analysis of DGI is based on the assumption that the perturbation is randomly shuffling the node features. While the authors have provided discussions in Theorem 1 when this assumption fails, I wonder whether the discussion on the shortcomings of DGI-like methods remains valid when this assumption fails.
limitations: limitations: NA
paper_summary: paper_summary: This paper provides an insightful perspective of representation scattering to unify various GCL frameworks, and proposes an effective framework called SGRL. Specifically, the contributions are as follows: 1) Theoretically, with a well-defined representation scattering concept, the authors provide a universal theoretical explanation for the success of existing GCL frameworks. 2) They propose SGRL, which employs a unique adversarial approach to effectively utilize this mechanism. Specifically, SGRL integrates topological aggregation with representation scattering, with two adversarial objectives smoothed by EMA. 3) The proposed SGRL achieves powerful performances in extensive experiments across multiple tasks.
strength_weakness: strength_weakness: 1. Although the results in Figure 5 clearly demonstrate how model performance varies with different strengths of topological constraints, I believe it would be beneficial for the authors to present more results from additional datasets. To my knowledge, the preprocessing method used for Wiki-CS often differs from that of the other four datasets, which may influence the choice of $k$. Therefore, I would appreciate it if the authors could provide further experiments to enrich their analysis.
2. In Definition 1, there are two constraints in representation scattering, the center-away constraint and the uniformity constraint. When considering only the goal of making representations scattered enough, the center-away constraint seems somewhat redundant, as satisfying the uniformity constraint alone can ensure the discreteness of representations. The authors should provide additional explanations regarding the role of the center-away constraint in the process of representation scattering.
3. I suggest that the authors provide more detailed explanations of some symbols, even though this is common in graph representation learning. For example, the $\alpha_{ij}$ and $d_i$ in Equation 1.
questions: questions: See the above weakness.
limitations: limitations: Yes, they have.
paper_summary: paper_summary: In this paper, the authors provide an interesting discovery: the successes with mainstream GCL paradigms essentially come from implicitly scattering representations. They point out that the bottleneck of current GCLs lies in ignoring this, and they provide detailed theoretical proofs. Furthermore, they propose a new method to fully utilize representation scattering. They propose an asymmetric framework that, through carefully designed central discrete loss, enhances the distinctiveness of representations, resulting in favorable performance improvements.
strength_weakness: strength_weakness: 1. As I mentioned in the summary, the authors have designed an asymmetric contrastive framework with two opposing types of losses, which may lead to misunderstandings of the training process. Although the description in the paper is given, including an algorithm flowchart would be better.
2. In Figure 1, the authors plot the t-SNE visualization of DGI on Co.CS . When the number of encoder layers changes, two distinctly different results are produced. I hope the authors provide a detailed explanation of this phenomenon.
3. Is the proposed central discrete loss effective in all scenarios? In extreme cases, if all node representations converge at a single point, can this loss achieve dispersion?
questions: questions: see weaknesses
limitations: limitations: see above
paper_summary: paper_summary: The authors focus on exploring an essential mechanism existing in different contrastive strategies. They first define a concept in embedding space, which contains a center $\mathbf{c}$, a subspace $\mathbb{S}$ and two constraints, called representation scattering. They then investigate the relationships between this concept and the mainstream GCL frameworks, through intuitive experiments and rigorous formula derivation. These discoveries motivate the authors to develop a new GCL framework aligned with this concept. The authors also present a model, namely SGRL, which includes a contrastive loss that directly makes representations away from the mean center. Experiments validate that SGRL has a better performance.
strength_weakness: strength_weakness: * The authors demonstrate that the representation scattering mechanism exists in several graph contrastive frameworks and argue that these methods do not fully utilize this mechanism. But the reviewer only observes discussions about the shortcomings of several baselines in the paper, lacking a deeper discussion on how these methods do not fully utilize representation scattering.
*  In the viewpoints of the reviewer, the constraint based on topological aggregation seems unnecessary. Since the encoder already aggregates information from neighbors, adding an additional loss may lack a sufficient justification.
* The SGRL employs two encoders with non-shared parameters, while the authors provide little explanation on this point. Based on the experience of the reviewer, allowing the two encoders to share parameters might perform better, as it may prevent divergences in the learned patterns between the models.
questions: questions: In this work, the theories and methods proposed by the authors are mainly based on node-level tasks. So, the reviewer wonders whether this approach can achieve the same powerful performance in graph-level tasks.
limitations: limitations: The authors have discussed the limitations of their work adequately. I have no further concerns.

Comments: comment: The authors have provided thorough and insightful responses to my previous questions, which have greatly improved the clarity of the manuscript. As I noted in my initial evaluation, the work presents a novel idea, demonstrates high readability, and is presented in a clear and accessible manner. Additionally, the experimental studies are robust and well-executed. I have no further concerns and am therefore inclined to raise my score and strongly recommend this work.
Thanks for the response. After reading the authors’ response as well as the other reviewers’ comments, my main concerns, particularly regarding the center-away constraint, have been addressed. I appreciate the interesting idea of this paper, and thus would like to increase my rating to 8.
Thanks for the authors’ response which is satisfactory. In my opinion, the proposed representation scattering is rigorously proven to be a key factor in the success of existing GCLs, which may provide valuable insights for future advancements in graph representation learning. Therefore, I consider this work to be of exceptional quality.