CLEP: Exploiting Edge Partitioning for Graph Contrastive Learning
Authors: Yilin He
Chaojie Wang
Hao Zhang
Bo Chen
Bo An
Mingyuan Zhou
Venue: Submitted to ICLR 2023 (2023)
Keywords: Graph Contrastive Learning

Abstract: Generative and contrastive are two fundamental unsupervised approaches to model graph information. The graph generative models extract intra-graph information whereas the graph contrastive learning methods focus on inter-graph information. Combining these complementary sources of information can potentially enhance the expressiveness of graph representations, which, nevertheless, is underinvestigated by existing methods. In this work, we introduce a probabilistic framework called contrastive learning with edge partitioning (CLEP) that integrates generative modeling and graph contrastive learning. CLEP models edge generation by cumulative latent node interactions over multiple mutually independent hidden communities. Inspired by the ``assembly'' behavior of communities in graph generation, CEGCL learns community-specific graph embeddings and assemble them together to represent the entire graph, which are further used to predict the graph's identity via a contrastive objective. To relate each embedding to one hidden community, we define a set of community-specific weighted edges for node feature aggregation by partitioning the observed edges according to the latent node interactions associated with the corresponding hidden community. With these unique designs, CLEP is able to model the statistical dependency among hidden communities, graph structures as well as the identity of each graph; it can also be trained end-to-end via variational inference. We evaluate CLEP on real-world benchmarks under self-supervised and semi-supervised settings and achieve promising results, which demostrate the effectiveness of our method. Various exploratory studies are also conducted to highlight the characteristics of the inferred hidden communities and the potential benefits they bring to representation learning.

Under review as a conference paper at ICLR 2023
CLEP: EXPLOITING EDGE PARTITIONING FOR GRAPH CONTRASTIVE LEARNING
Anonymous authors Paper under double-blind review
ABSTRACT
Generative and contrastive are fundamental unsupervised approaches to model graph information. The graph generative models extract intra-graph information whereas the graph contrastive learning methods focus on inter-graph information. Combining these complementary sources of information can potentially enhance the expressiveness of graph representations, which, nevertheless, underinvesti- gated by existing methods. In this work, we introduce a probabilistic framework called contrastive learning with edge partitioning (CLEP) that integrates genera- tive modeling and graph contrastive learning. CLEP models edge generation by aggregating latent node interactions over multiple overlapping hidden communities. Inspired by the assembling behavior of communities in graph generation, CLEP learns community-spesd -specfffc graph embeddings, which are assembled together to represent the entire graph and further used to predict the graph's identity via a contrastive objective. To relate each embedding to one hidden community, we define set of community-specinnn weighted edges for node feature aggregation by partitioning the observed edges according to the latent node interactions associated with the corresponding hidden community. With these unique designs, CLEP is able to model the statistical dependency among hidden communities, graph structures, as well as the identity of each graph; it can also be trained end-to-end via variational inference. We evaluate CLEP on real-world benchmarks under self-supervised and semi-supervised settings and achieve promising results, which demonstrate the effectiveness of our method. Various exploratory studies are also conducted to highlight the characteristics of the inferred hidden communities and the potential benefits they bring to representation learning.
INTRODUCTION
Generative modeling and contrastive learning are both commonly employed to learn graph represen- tations without label supervision. Both types of methods learn the embedding space by leveraging some ground-truth information from the observed graphs, but the aspects of data that each type
chooses to fit are different. Graph generative models (Kipf & Welling 2016 Mchta et al. 2019 Wang et al. 2020) prioritize intra-graph information, i.e. the information in each individual graph. The representations provided by graph generative models are usually related to the formation of its
own edges. Contrariwise, graph contrastive learning methods (You al.) 2020 2021 [022 Xie et al.)
2022) focus on capturing inter-graph information, they put graphs under comparison to highlight the inherent similarity and differences among group of graphs.
The difference in the focused graph information leads to complementary strengths and weaknesses of graph generative and contratsive learning methods. The advantage of graph generative models is their ability to recover the structural information of some latent factors, which is lost during graph generation. These latent factors, relevant to each graph in the sense of its own formation, usually preserve valuable information for various graph-analytic tasks. However, the quality of the embeddings provided by graph generative models is questionable because the encoded information limited to the" expessxonsi levels" of these latent factors, which may be insufficient to downstream tasks other than graph generation. Unlike generative models, graph contrastive learning methods cannot automatically find meaningful latent factors in the graph, but they are well recognized for producing high-quality feature representations once the raw structural information given. An integration of graph generative modeling and graph contrastive learning potentially combines the

---

Under review as a conference paper at ICLR 2023
complementary strengths of these two classes of methods, which would further benefit representation learning. However, such a direction is insufficiently explored.
In this work, introduce Contrastive Learning with Edge Partitioning (CLEP), probabilistic framework that provides a concrete solution to the integration of graph generative modeling with graph contrastive learning. To better leverage the strengths of both methods, assign different duties to the graph generative model and the contrastive learner, where the former takes the responsibility of extracting the hidden structures of the latent factors which explain graph generation, while the latter treats the inferred hidden structures as new raw inputs, learns their corresponding graph embeddings in a contrastive paradigm, and then aggregates all the embeddings in a way that is intuitively compatible with how the hidden structures interact during graph generation.
More specifically, CLEP is developed upon the graph generative models that explain the formation of edges by cumulative latent node interactions associated with some hidden communities (Yang & Leskovee 2012 2013 2014] Zhou 2015). Membership and intra-community node interactions are assumed to be independent among different hidden communities. According to the generative model, we embody the structures of these communities by explicitly modeling the average node interactions in each community and partitioning the observed edges accordingly. As indicated by Jenatton et al. (2012), node interactions that happen in different communities may follow multiple relations. is highly likely that, when nodes interact under different relations, the information exchange that comes along also focuses on different aspects. To better capture the potentially heterogeneous community- define
specific information, communities, The training of these encoders are based on aggregating their corresponding contrastive learning tasks with set of weights that measure the "importance" of each community. Finally, we gather the community-specifie graph embeddings to represent the overall information on a graph, as an analogy with the assembling behavior of communities in graph generation.
of encoders process the information that comes from different
We summarize the major contributions of this work as follows:
We propose CLEP as an integration of graph generative model and graph contrastive learning. which can effectively capture both intra- and inter-graph information.
We formulate the statistical problem of training CLEP as the maximum likelihood estimation of a latent variable model, which supports end-to-end optimization via variational inference.
We show through exploratory studies that the strength of factor-wise representation learning is capture nonrepetitive graph information from different hidden factors, which offers more flexible embedding selection & combination when facing various downstream tasks.
We train CLEP under self- supervupe and semi-supervised settings and conduct an extensive empirical evaluation of the obtained graph representations, finding that CLEP consistently outperforms existing arts on various real-world benchmarks.
2 PRELIMINARIES
Information encoding on graphs. A graph encoder maps the information on each graph to a vector representation. For a graph & with N nodes, its given information usually includes node feature matrix X E R x and an adjacency matrix A. The most effective way to aggregate these two forms of information is based on graph neural networks (GNNs) (Kipf & Welling 2017 Hamilton et al.. 2017, Veličković et 20118) Xu et al. 2019). For GNN with T layers, denoting Hg X, its propagation rule can be summarized as
H, (H A), LE [1,7),
where AGG denotes neighborhood aggregation and f()) denotes nonlinear transformation. Append- ing a readout layer a regular GNN converts it from node-level encoder to a graph-level encoder, where the readout operation is defined as h READOUT (EADOCT((E)))1)) In the sequel, we use (y, to denote node-level encoders, and use he X) to denote graph-level encoders.
Graph contrastive learning (GCL). The ground-truth information that GCL methods use train the graph encoders is the uniqueness of each graph. The distribution of the graph representations in the embedding space is expected to capture the inherent similarity and differences among the graphs. To this end, positive pairs are created, with each one consisting of two nonidentical views of the same graph. Some contrastive methods (Xie al. 2022) only pull together the representations of positive
2

---

Under review as a conference paper at ICLR 2023
Prodeders
Aagrevazzion
G
h2
1
Communtiy Mite Creah
0B
Proteder
GarmunityWee
Asgrortaiion
Figure 1: The model architecture of CLEP (left) and the computation of its objective function (right). Node-community affiliation of graph G is modeled an amortized function. This information is combined with the two augmented views G' and G" to partition these graphs in order to reveal the hidden structure of each hidden community in the augmented graphs. The graph represented by a embeddings, which are learned by individual contrastive learning
collection of community-speciinoe tasks customized for the corresponding communities. During training, these tasks are balanced by set of weights measuring the "relatvie importance" of each community.
pairs, some other methods (You et al. 2020 2021 2022) also create negative pairs and push afar the representations of these gggative-paired graphs. Unlike the graphs forming positive pairs that share the same origin, graphs in negative pair are transformed from different source graphs. The update of the embedding space is achieved by optimizing contrastive objective.
A probabilistic interpretation. Many contrastive objectives involving both positive and negative pairs, such as N N-pair (Sohn 2016). InfoNCE (Oord al., 2018) and NT-Xent (Chen et al.) 2020), are collectively related to an in-batch instance discrimination event (Wu et al., (018). Such statistical property inherited by GCL methods following these established work. Specifically, given mini- batch B = (G,, 92, ... ,IBB), let k; be the identity-preserve the probability that the in-batch graph ID of q being recognized as can be defined as
key of 9; and be a query vector,
p(i q) exp(sim(qq k;)/T)
exp(sim(q, kj)/T)
where sim(a,b) b) = a
|allibll measures the cosine similarity between vectors a and b, and r is a temperature parameter. Back to the context of GCL,, after each graph 9 being augmented into two views G' and G", representations h' and h" are obtained and mapped to f' and f" by a shared parametric function m²-)). Treating {f." )i as the keys and (f/)i_, as the queries (or the other way around) leads to similar expression to the objective functions various GCL methods (You et al. 2020: 2021). hence optimizing those objectives can be interpreted as maximizing the log-likelihood of correctly recovering the in- graph IDs of the queries.
CLEP: CONTRASTIVE LEARNING WITH EDGE PARTITIONING
3.1
LATENT NODE INTERACTION AND EDGE GENERATION
Given graph G with N nodes, suppose it contains K conceptual latent factors, interpreted as com- munities Cr, C2,... , Cx whose intra-community node interactions are recorded as (M()) M e BNXN I. the adjacency matrix of G. i.e., A [0,1] can thus be modeled under the Simoulli-Poisson link (Zhou 2015) as
Age B()
(2) we
((M(#) > 0), ( Poisson (M) u. u. E [I, u u v.
Here V denote logical OR. Equation I has the following interpretation: for any nodes v in graph G, they interact with each other for (k)) times within community Ck. which follows Poisson
3

---

Under review as a conference paper at ICLR 2023
Vqud Graph
Partfinedd Eeges
Communiit [Gc
Rede-Commant Afficatton
Compute tage Puttior Weglts
A
Look Up
e
Figure 2: The dashed block encloses the internal steps of soft edge assignment (SEA), which is edge partitioning per se. For each edge, we search for the code-community affiliation of its endpoints and compute the latent interactions of these two nodes in all K hidden communities, then use them to compute the partitioned weights. We embody the hidden structures of the communities with the partitioned graphs, and use them the raw inputs to learn community-specife graph embeddings.
distribution with rate M latent connection in Ck will be established between u and if they interact for at least once, and the edge (u, v) can be observed if the latent connection is established in least one community. Assuming that the communities are mutually independent, further derive
Auv (M, > 0), Muv ~ Poisson(M + M + +... M()),, u, [L, u + v, (2)
as an equivalence to Equation I The reformulation explains the formation of edges as result of accumulating latent node interactions over multiple hidden communities. In practice, to avoid densely parameterizing the edge generation process, each of [M k=1.K is modeled by rank- d nonnegative matrix factorization (Yang & Leskovec 2013 Zhou 2015), i.e., ( (
( (4) E R If d = 1, (k) can be interpreted as the affiliation strengths of all nodes to community if that are belonged metacommunity Cg. d by default set to one except being specified otherwise.
if > 1, (h would be the affiliation strengths of node u to the d sub-communities
3.2 DOGE--ARTIITONN-NNTE
COMMUNITY-WII INFORMATION ENCODING
In general, to analyze graph information with regard to one type of relation in a multi-relational graph. a critical step is to isolate the edges indicating such relation from the rest of edges (Schlichtkrull et al.) 2018. Vashishth et al. 2020). Likewise, when capturing the information on a specific community, we focus on identifying the pairs of nodes which are connected owing to their intense interactions this community. In CLEP, such goal is achieved by soft edge assignment (denoted by SEA), i.e.,
SEA(Aaus k, (M(1) }K=1.K) == Awv exp(M
TSEA)
A (k) ue
K [1, K].
(3)
(M
(TSEA)
Applying Equation all pairs of nodes yields weighted adjacency matrices (A K
[- with
A (k) corresponding hidden community Ck- TSEA is a positive temperature Hinton et al. 2015) that controls the concentration of the partitioned weights. Setting TSEA close to zero would drive (A ( towards binary matrices with little overlap on edges, which would highlight the structural difference among the hidden communities. Intuitively, it would encourage extracting the information on the aspects that are unique to each community. Contrariwise, assigning a large value to TSEA would suppress the expression on the differences between the communities.
With the crude relational structural information for all hidden communities embodied by (A (k) )=1,K, we can further refine it by an array of GNN-based graph encoders and obtaining community-specife graph embeddings, Le., (k) (1( (A()) (k) X), k E [1, K). We the collection (K) as the overall graph represen- tation for downstream tasks, which has comprehensive coverage the information from all hidden communities. A graphic illustration on the computation details of soft edge assignment (SEA) and community-win information encoding is given in Figure
of all the community-spent speciffc embeddings, i.e., (h, (2)
4

---

Under review as a conference paper at ICLR 2023
3.3 THE LATENT FACTOR CONTRASTIVE LEARNING MODEL
The statistical problem of fitting CLEP is defined upon mini-batches. Let us denote a generic graph batch as B {G. (X,, A=) 1.:...1.11 IBl which is uniformly sampled from the data population. Like many other GCL methods (You et al. 2020 202T). for any G E IB, we augment it into two views g' = (A', X') and g" (A", x") via perturbations T' and T" Recalling the probabílistic
interpretation of GCL as given in Section 2 the ultimate optimization problem can be defined as
max E
Σ 1=1
log Po A, X2),
where i is the in-batch ID of graph Gi which serves as the surrogate label of G, in the in-batch instance discrimination task. Unlike previous works (You et al. 2020 2021| 2022) that directly model the probability pi A X,), take the influence of latent communities into consideration and define the conditional probability pa (Z,
1K=1,K, Ai, X.) instead, i.e.
(K) JR=LKK A, X,) p (Z.
Exn
K
(R) (N) Pi Po (1l (Z) (K) 1K-,, A,,
(5)
where (1k) (i 1 A, is the contrastive loss on G, that is computed with graph em- (K) collectively measure the relative "importance" of each community to graph Gs subject to the constraint that Σ Ek=12 1//²² =.
beddings specified to community Ck. and (1) (2) . Pi
series of variables are computed in the way as suggested in Figure [ to facilitate the elab- oration on our model. h, hg AA,,X, X;) is global embedding for the original graph, Gz- The edges of the augmented graphs are soft-assigned to the communities, i.e. (K))
SFA(AL, k; (Z
(*) 2 (k)" )_=1,), A SEA(AA, k; (Z!*) .z,""" }==1,K); with the partitioned
edges, community-speciff
embeddings of Gj and G can be obtained via h
((k) = =hg (A
(K)
((k) X))
and h (((k) = (1)(( (A, ((k) X,";; they are further projected to (E((A) IK=LK and {E" 1k=1, by
group of MLPs, i.e., (((h) = m (A) (A) (A)) and p"(A) m (k) (h (()) for = 1,2, K. K.
With these variables readily available, we specify the quantities in Equation 5 as follows:
exp (sim(() (F((k) p 1/T)
Pg (i (Z 1=I, A,, X,) = EE exp (sim(1,()) . p/))))/7))
(6)
exp (v, h ((k)
(K) P
k E [1, Σ exp (vb(h; h )(
K
where (...)) measures the similarity between the graph embeddings.
3.4 THE TRAINING ALGORITHM OF CLEP
We set the prior distributions of the latent variables, namely P(Z(").
((, as
(k) itd ~ Gamma(a, 3), & E [1, E N.) E [B]]
(8)
where a, 3 are the shape and rate parameters. Although Equations 5 to compute the conditional
probability po {z J===) A,, X,), the corresponding density function does not have an analyti-
cal expression, hence the true posteriors of (z(²)) k= cannot be explicitly defined, which prohibits
direct optimization of the log-likelihood in Equation (4) via DnnectattoonMaximizaatt (Dempster
et al., 1977). Instead, we model (z(()) k=1,K by a set of variational distributions (Z )===IKK and maximizing the evidence lower bound, defined
Lelbo = Eg Σ Ez, (w(z) log Po Z; A. -
(9)
Here we use Z; to denote [ z (2) divergence term can be further expanded Ez Qa(2,) log Z..) (Q. P(Z,))
(k) to avoid clutter of notations. The negative KL
5

---

Under review as a conference paper at ICLR 2023
log p(A)). Dropping the constant log p(A,) from Equation ( yields the final objective of CLEP,
|
CCLFP EB
(10)
4 = Ez, [log Z; A, X;) + log A, Z.)] (kL((
We define (z(Z) (+) as Weibull(Z, (
(k) K ) where A (8) (k) are the matrix form of scales
and shapes. Z;" can be sampled via computing the following inverse cumulative density function:
[/d a (- log(1 - G, ) E
1/
(k) iid Uniform((. (0, 1), u e [1, [1, K].
(11)
The shapes and scales are modeled by an amortized function, i.e., [A,, K; = hy(X, A), where A; := [A A ..... | and K; := [K.") K.² K(K} With Qz(Z,) being Weibull distribution and P(Z.) being Gamma distribution, their KI. divergence, ie., DKI. (Q (Z.) P(Z,))
can be explicitly expressed as in /Zang et al. (2018):
DKL(QO(Z,) P(Z.))
Ex k=1
K
Σ
N:
(KL(Q (1)) |P(Z
(KL(QO(Z P(Z = a log A
(2)
ya
+ log (R) iiu
+ + (
1
(12)
i,u
(k)
-7- In B I(a),
where y = 0.5772 is the Euler constant. Maximizing Equation (TO) with respect to parameters (0, () would train CLEP. After that, sample ) from the well-tuned variational distribution,
computing the latent node interactions in each hidden community by M ( k E K),
then perform the operations in Section 3.2 to obtain the graph embeddings for downstream tasks.
4 OTHER RELATED WORK
Matrrixfactoriiatioo--asss graph generative models. A well-established way to build up the stochastic process that generates the adjacency matrix of an observed graph is by low-rank matrix factorization. The seminal idea can be traced back to the stochastic block model (SBM) (Holland ct al. 1983), where the latent variables infer are the assignments of nodes to some latent factors, i.e. communities. SBM models the pairwise node interactions by the dot-product of the node-community assignment matrix and transpose, which further generate the edges. SBM prohibits nodes to be affiliated with multiple communities, so the latent assignment factors associated with each node are restricted to be one-hot. Such constraint is later relaxed by various extensions of SBM (Airoldi et al.) 2008 Miller et al. 2009 Yang & Leskovec 2012, 2013; 2014 Zhou 2015 Sun et al.) (219)) that permit communities to have overlapped membership. CLEP is closely related to these methods in terms of defining the graph generation process, and the representations obtained by both CLEP and these methods are about the information on these hidden communities. The major difference is that adopt contrastive learning techniques embed community-speciff information, which brings us immense ad udvantage over these latent factor graph generative models in representation quality.
Modeling multi-relational data. Retrieving information from multi-relational data has been exten- sively studied in the literature of embedding knowledge graphs (Schlichtkrull et al.) 2018 Vashishth al. 2020). Different relation types are expressed by edge labels, which are usually provided as part of the groundtruth information. A consensus of these models is that different relations should be separately modeled, which is general carried out by factorizing the graph by edge type, then individually modeling the information on each graph factor that only contain one type of edges. In a similar way, edges in the graph that CLEP deals with are also inherently different with each other in terms of the "major contributor" of their generation, i.e.. the community that contributes the most of the interactions, which creates a special case multti-rllaional graph. We hence adopt similar high-level representation pipeline that contains both graph factorization and factor-wis, representation learning. However, the necessary information to perform graph factorization is not observable, thus we express it with set of latent variables and infer them via variational inference.
Disentangled graph learning. Isolating the structure of each community from the entire graph then modeling each isolated sub-structure with customized model relates CLEP to disentangled graph

---

Under review as a conference paper at ICLR 2023
Table Comparison of graph classification performance, results are reported format "average accuracy - standard error". "A.R." stands for average ranking.
Method
MUTAG
PTC
MR
PROTEINS
NCII
IMDB-B
IMDB-M
RDT-B
RDT-MEK
A.R.
SP
8.
58.2 2.4
75.1 +
=3.0
35.6
38.0 0.3
64.1
392
10.4
GK
81.7+2211
57.3
71.7
+ 0.6
62.3
+2.3+0.3
63.9
4.
77.
41.0
11.1
W1.
80.7 3.00
58.0
72.9
0.5
80.0
0.5
72.3
34
47.0
688
04
46.11.02
9.1
DGK
2.7
-0.1-26
73.3 0.8
80.3
0.5
0.0
44
78.0
4.2
7.3
MI.G
87.9 1: 1.6
63.3 1.5
26.1 1: 2.0
80.8
1.3
66.6
0.3
41.2
1:22:000 0.0
4.8
node2vee
72.6 10.2
3.66
$4,9 166
13.3
sub2vee
61.1
60.0 6.4
$3.0 5.6
52.8 + 1.5
55.3 1.5
36.7 0.8
71.5 + 04
36.7 0.4
12.5
graph2vec
9.
73.3 1:
73.2
71.1
50.4
75.8 1. 1.0
47.9 0.3
GAE
84.0+-0.6
74,1 + 0.5
73.3 0.6
52.1
0.2
74.8 + 0.2
37.6
9.7
VGAE
0.6
74.0 1: 0.5
73.7
52.1
12.10.0 0.2
74.8 0.2
39.1 1.6
9.3
InfoGraph
89.0 1.1
61.7 1.4
74.4 0.3
76.2 - L1
03
0.5
82.5 14
53.5 1.0
5.5
MVGRI.
89.7+ 1.1
62.5
7,2+0.7
51.2+
84.5 +.6
3.2
GraphCL
86.8
63.6 188
74.4 14.4000 0.5
77.9
7L. 04
50.7
0.4
89.5
0
5..0
4,4
JOAQ
87.7
61.1
74.6 0.4
78.4
70.8 0.3
51.0
86.4 + 1.5
56.0 0.3
4.5
LaGraph
90.2
75.2 0.4
79.9 0.5
73.7
0,9
90.4
0.8
56.4 0.4
2.2
CLEP
0.8
65.1
76.4
78.5 + 0.4
75.6
52.0
87.3 + 0.5
56.4
1.8
95
85
85
85
Aczurayy
75
75
60
70
70
85
70
60
80
2
2
communities
communilies
communilies
(a) MUTAG, accuracy-
(b) PROTEINS, accuracy-
(c) IMDB- accuracy- KK
100
85
85
Aczuncy
90
Aazurac
75
Accurscy
75
70
70
65
60
80
70
10
10
10
10
10P
10
10
10"
10
SEA lemperature
SEA temperature
(d) MUTAG, accuracy-TSEA
(c) PROTEINS, accuracy -ATAA
(f) IMDB-B B, accuracy-Tse.
Figure 3: Task performance with graph embeddings trained with alternative choices of K and TSEA. emrorbrr is set to "# standard error". The red dotted line indicates the performance of GraphCL.
learning (Ma et al.) 2019 Yang et al. 2020). In general, the effectiveness of such model design be explained by better modeling potentially heterogeneous information from different hidden factors, and increased flexibility in terms of the utilization of the information associated with each factor. To make hidden factors more interpretable, this line of work also introduces graph factorization into their frameworks. The major difference between CLEP and disentangle methods is that the graph factorization in the latter is mostly based on downstream task, whereas in CLEP, is based on both graph generation and the pretext contrastive learning task.
5 EXPERIMENTS
5.1 EMPIRICAL EVALUATION
Unsupervised learning. We first examine the quality of the representation learned by only opti- mizing Equation (10). widely adopted real-world benchmarks are selected for this evaluation task, including 4 biochemical graph datasets (MUTAG, PTC MR, PROTHINS, and NCIT) and
7

---

Under review as a conference paper at ICLR 2023
REDDIT and REDDIT-MULTD, We compare CLEP against three groups of baselines: (i) graph kernel methods, including Shortest
social network datasets BINDAY,,,,Y,
Path Kernel (SP) (Borgwardt & Kriegel 2005), Graphlet Kernel (GK) (Shervashidze al. 2009).
Weisfeiler-Lehman Sub-tree Kernel (WL.) (Shervashidze et al., 2011). Deep Graph Kernels (DGK)
Yanardag & Vishwanathan (2015), and Multi-Scale Laplacian Kernel (MI.G) (Kondor & Pan 2016);
(ii) traditional graph representation learning methods, including node2vec (Grover & Leskovec
2016), sub2vec (Adhikari et al. 2018), graph2vec (Narayanan et al. 2017), GAE and VGAE (Kipf &
Welling 2016); and (iii) recent ttaee-of-the-arts, including InfoGraph (Sun et al. 2020). MVGRL
(Hassani & Khasahmadi] 2020), GraphCL (You et al., (020), JOAO You al. (021) and LaGraph (Xie et al. 2022). We align most of the hyperparameters with You et al. (2020); for K, í.e., the number
of communities, and TSBA), L.e,, the temperature of soft edge assignment, we search (K; TSFA) ) across
the grid of {2, 4,83 {0.1, 1, 10} at the evaluation time through voss-validation. The obtained embeddings are evaluated following You et al. (2020).
We list the results under comparison in Table 1 CLEP has the-art art results on 6 out of 8 benchmarks, and has the best average ranking among all algorithms under compar- ison. On IMDB-BINARY and IMDB-MULTI, CLEP surpasses the second-best baseline by 1.4% and 0.8%, with both exceed- ing two standard errors; on MUTAG and PTC_MR, CLEP beats the second place model by 1% and 1.5%, with both exceeding one standard error, indicating significant improvement to other competitive baselines. CLEP is implemented based on GraphCL, the only difference between CLEP and GraphCL is that the latter does not systematically model the information specified to the hidden communities. When comparing CLEP with GraphCL. the advantage of learning from diverse hidden communities appears to be quite evident, i.e., CLEP outperforms GraphC1. by more than one standard deviation on 7 out of 8 benchmarks, at the evaluations on MUTAG, PROTEINS, IMDB-BINARE and IMDB-MULTI, the results obtained by CLEP are higher than those of GraphCL by more than two standard errors. These results demonstrate that capturing the information from hidden communities is beneficial for representation learning, and the way that CLEP expresses and encodes the information each hidden community is effective.
achieved state-of -ote-----fh--aae-ee
728
90.5
55
10% 30% Pacee
Figure 4: The comparison of clas- sification performance (ROC-AUC) on oghg-molhiv dataset under the set- ting of self-supervised learning.
Semi-supervised learning. Besides the common setting of unsupervised learning, we compare the developed CLEP with the strong self-supervised baseline, specifically GraphCL You et al. 2020), under the semi-supervised setting. As shown in Fig. 4 with the learned graph representations on the ogbg-molhiv dataset from Open Graph Benchmark (Hu et al., 2020), we conduct fine-tuning on the partially labeled data, where the settings of label rates are 5%, 10% and 20% respectively, and then evaluate these graph representations on the validation/test sets with binary classification measured by ROC-AUC metric. From the results, we can find that the performance of both methods tends to improve with the increase of the label rate, and our method CLEP achieves significant performance improvement over GraphCL with 2.8%, 3.5%, and 3.3% performance gains by setting the label rate as 5%, 10% and 20% respectively. These results further demonstrate that our CLEP can provide more expressive graph representations even when handling large-scale graph in semi-supervised manner, benefiting from capturing the information from hidden communitics.
5.2 ABLATION STUDY
Compared with its base model GraphCL, two hyperparameters, namely K, the number of commu- nities, and TSEA. the edge partition temperature, are unique to CLEP. The influence of adjusting K or TSEA is shown in Figure 3 general, within a reasonable range of hyperparameter variation, CLEP outperforms GraphCL. most of the time, which shows the consistency of the performance improvement we have achieved against our base model. The optimal values for appear to be around 2 to 4, and TSEA not greater than 1. When setting K 1. CLEP degenerates to GraphCL, which explains the comparable performances of CLEP and GraphCL; and due to the small size of these graphs, the number of meaningful communities is expected not to be large. As for the selection of TSEA, a large value would drive edge partition towards an even partition, leading to a result that the inputs to all community-speciff graph encoders approximately the same the original graph,
8

---

Under review as a conference paper at ICLR 2023
Table 2: Performance on the downstream task, with (w/) and without (w/o) community selection.
MUTAG
PTC.MR
PROTEINS
IMDE-BINARY
w/ community selection
91.4
68.1
75.7 0.4
75.4 + 0.4
w/o community selection
90.3
67.7 1.4
75.4 +..4+005
75.0 + 0.2
(a) PROTEINS
(b) MUTAG
(c) NCII
Figure t-SNE visualization the community-speciffe representations learned on some benchmarks.
hence in this scenario, CLEP is analogous to an aggregation of K runs of GraphCL, which cannot effectively improve the quality of the graph embeddings.
5.3 EXPLORATORY STUDIES
Shown in Figure are the t-SNE plots of the conmunity-specific specific graph embeddings obtained from MUTAG, PROTEINS, and NCI1. We color-code the embeddings learned from different communities. From Figure 5 we can observe that points sharing the same color are concentrated at different locations. We interpret the general location of a cluster of colored points the information "biases" that are related to the nature of the corresponding community category, and the variation across the points in the cluster as the uniqueness of each individual graph. So the spatial clustering of the colored points a positive sign that CLEP captures community-specinf information.
We further investigate how such property of CLEP can be leveraged benefit downstream tasks. We are interested in the situations where not all communities are needed for the task, i.e., some may be closely related to and can provide crucial information for the task, while the others may be loosely related to the task or even worse, producing more noise than information. The advantage of CLEP is that it organizes the information by communities, which enables the combine the information for different tasks. Intuitively, correctly identifying the "noisy" communities and dropping them from the overall information pool may increase the signal-to-noise ratio of the input information, thus is beneficial for the downstream tasks.
to actively select &
To illustrate the point, we conduct the following experiment on MUTAG, PTC.MR, PROTEINS and -IDBBB..ARY..... Each dataset is split into 10 folds, with one for validating, one for testing, and the remainder for training. For each benchmark, we fit two logistic classification models, one takes all community-spe specificc embeddings inputs, and another selects the embeddings by community via crossvalidatton. The experimental results, as recorded in Table show that community selection results in consistent improvement to the downstream task. Results in this experiment demonstrate flexibility of the usage of the graph representations learned by CLEP.
CONCLUSIONS
In this work, we propose a probabilistic contrastive learning framework called contrastive learning with edge partitioning (CLEP). CLEP combines the strengths of graph generative models and graph contrastie learing methods by assigning them different duties that they are adept in, i.e., the graph generative models extracts the hidden communities and thus augments the total amount information on graph, and the graph contrastie learning methods are used to convert these hidden structures to high-quality graph embeddings. The empirical evaluation on real-world datasets demonstrate the superiority of the graph representations learned by CLEP. Besides, CLEP organizes the learned graph information by communities. which creates higher flexibility in embedding selection & combining schemes when facing potentially different downstream tasks, which enhances the versatility of CLEP..

---

Under review as a conference paper at ICLR 2023
REFERENCES
Bijaya Adhikari, Yao Zhang, Naren Ramakrishnan, and B Aditya Prakash. Sub2vec: Feature learning for subgraphs. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 170-182.
Springer, 2018.
Edo M Airoldi, David Blei, Stephen Fienberg, and Eric Xing. Mixed membership stochastic blockmodels. In Advances in Neural Information Processing Systems (NeurIPS), volume 21, 2008.
Karsten M Borgwardt and Hans-Peter Kriegel. Shortest-path kernels on graphs. In Fifth IEEE international conference data mining (ICDM"OS '05), pp. 8-pp. IEEE, 2005.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning (ICML), pp. 1597-1607. PMLR, 2020.
Arthur P Dempster, Nan M Laird, and Donald B Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society: Series B (Methodological), 39(1): 1-22, 1977.
Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDI) international conference on Knowledge discovery and data mining (KDD), pp. 855-864, 2016.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems (NeurlPS), 30, 2017.
Kaveh Hassani and Amir Hosein Khasahmadi. Contrastive multi-view representation learning on graphs. International Conference on Machine Learning (ICML). pp. 4116-4126, PMLR, 2020.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge neural network. arXiv 2015.
preprint
Paul W Holland, Kathryn Blackmond I Laskey, and Samuel Leinhardt. Stochastic blockmodels: First steps. Social networks, 5(9): 1983.
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning graphs. In Advances in Neural Information Processing Systems, 2020.
Rodolphe Jenatton, Nicolas Roux, Antoine Bordes, and Guillaume Obozinski. A latent factor model for highly multi-relational data. Advances in neural information processing systems (NeurIPS), 25, 2012.
Thomas N Kipf and Max Welling, Variational graph auto-encoders. arXiv preprint arXiv:1611.07300 2016.
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations (ICLR), 2017.
Risi Kondor and Horace Pan. The multiscale laplacian graph kernel. Advances in neural information processing systems (NeurIPS), 29, 2016.
Jianxin Ma, Peng Cui, Kun Kuang, Xin Wang, and Wenwu Zhu. Disentangled graph convolutional networks. In International Conference on Machine Learning (ICML), pp. 4212-4221, 2019.
Nikhil Mehta, Lawrence Carin, and Piyush Rai. Stochastic blockmodels meet graph neural networks. In International Conference on Machine Learning (ICML), pp. 4466-4474, PMLR, 2019.
Kurt Miller, Michael Jordan, and Thomas Griffiths. Nonparametrie latent feature models for link prediction. Advances in neural information processing systems (NeurIPS), 22, 2009.
Annamalai Narayanan, Mahinthan Chandramohan, Rajasekar Venkatesan, Lihui Chen, Yang Liu, and Shantanu Jaiswal. graph2vec: Learning distributed representations of graphs. arXiv preprint arXiv: 1707000000000 2017.
10

---

Under review as a conference paper at ICLR 2023
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint 10X.v:887.00378,, 2018.
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In European semantic web conference (ESWC), pp. 593-607. Springer, 2018.
Nino Shervashidze, SVN Vishwanathan, Tobias Petri, Kurt Mehlhorn, and Karsten Borgwardt. Efficient graphlet kemels for large graph comparison. In International Conference Artificial Intelligence and Statistics (AISTATS), Proceedings of Machine Learning Research, pp. 488-498 2009.
Nino Shervashidze, Pascal Schweitzer, Erik Jan Van Leeuwen, Kurt Mehlhorn, and Karsten M Borgwardt. Weisfeiler-lemman graph kernels. Journal of Machine Learning Research, 12(9), 2011.
Kihyuk Sohn. Improved deep metric learning with multi-class n-pair loss objective. Advances in neural information processing systems (NeurIPS), 29, 2016.
Fan-Yun Sun, Meng Qu, Jordan Hoffmann, Chin-Wa Huang, and Jian Tang. vgraph: A generative model for joint community detection and node representation learning. Advances in Neural Information Processing Systems (NeurIPS), 32, 2019.
Fan-Y Yun Sun, Jordan Hoffmann, Vikas Verma, and Jian Tang. Infograph: Unsupervised and semi- supervised graph-level representation learning via mutual information maximization. In Interna- tional Conference on Learning Representations (ICLR), 2020.
Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, and Partha Talukdar. Composition-based multi- relational graph convolutional networks. In International Conference on Learning Representations (ICLR), 2020.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liō, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations (ICLR), 2018.
Chaojie Wang, Hao Zhang, Bo Chen, Dongsheng Wang, Zhengjue Wang, and Mingyuan Zhou. Deep relational topic modeling via graph poisson gamma belief network. volume 33, 2020.
Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin. Unsupervised feature learning via non- parametric instance discrimination. Proceedings of the IEÉE conference on computer vision and pattern recognition (CVPR), pp. 3733-3742, 2018.
Yaochen Xie, Zhao Xu, and Shuiwang Ji. Self-supervised representation learning via latent graph prediction. In International conference on machine learning (ICML), pp. 24460-24477. PMLR, 2022.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? International Conference on Learning Representations (ICLR), 2019.
Pinar Yanardag and SVN Vishwanathan. Deep graph kemels. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining (KDD), pp. 1365-1374, 2015.
Jaewon Yang and Jure Leskovee. Community-affilian graph model for overlapping network community detection. In 2012 IEEE 12th international conference on data mining. pp. 1170-1175 IEEE, 2012.
Jaewon Yang and Jure Leskovec. Overlapping community detection at scale: a nonnegative matrix factorization approach. In Proceedings of the sixth ACM international conference on Web search and data mining (WSDM), pp. 587-596, 2013.
Jaewon Yang and Jure Leskovec. Structure and overlaps of ground-truth communities in networks. ACM Trans. Intell. Syst. Technol. 5(2), 2014.
11

---

Under review as a conference paper at ICLR 2023
Yiding Yang, Zunlei Feng. Mingli Song. and Xinchao Wang. Factorizable graph convolutional networks. In Advances in Neural Information Processing Systems (NeurIPS), volume 33, pp. 20286-20296, 2020.
Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations. Advances in Neural Information Processing Systems (NeurIPS), 33, 2020.
Yuning You, Tianlong Chen, Yang Shen, and Zhangyang Wang. Graph contrastive learning automated. In International Conference on Machine Learning (ICML), pp. 12121-12132. PMLR, 2021.
Yuning You, Tianlong Chen, Zhangyang Wang, and Yang Shen. Bringing your own view: Graph contrastive learning without prefabricated data augmentations. In ACM International Conference Web Search and Data Mining (WSDM), 2022.
Hao Zhang, Bo Chen, Dandan Guo, and Mingyuan Zhou. WHAI: Weibull hybrid autoencoding inference for deep topic modeling. In International Conference on Learning Representations (ICLR), 2018
Mingyuan Zhou. Infinite edge partition models for overlapping community detection and link predic- tion. In International Conference Artificial Intelligence and Statistics (AISTATS), volume 38 of Proceedings of Machine Learning Research, pp. 1135-1143, 2015.
12


Reviews:
[{"review_id": "d0GBYROPDpi", "reviewer": "Reviewer_mFPj", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a probabilistic method to combine graph generation and contrastive learning. A key innovation is to factorize the graph into several subgraphs. Each subgraph represents a facet of interactions in a sub-community. The learned latent representation for each node is also factorized into sub-representations w.r.t. each community. Using this fine-grained representation, the paper unifies graph generative models and contrastive learning on the community level, demonstrating better classification performance than previous methods.", "summary_of_the_review": "This paper proposes a novel idea to combine existing approaches to achieve better classification performance via an augmented representation. The performance gain needs some experimental justification to make it more convincing.", "strength_and_weaknesses": "Strengths: \n1. This paper unifies two unsupervised learning approaches: graph generative models and graph contrastive learning in a probabilistic framework, which is novel to me.\n2. The idea to factorize the graph facilitates the representation learning framework, demonstrated by experimental results on downstream classification tasks. \n\nWeaknesses: \n1. The motivation to factorize the graph into subgraphs is unclear to me. Since assuming multiple communities will significantly enlarge the number of parameters needed to represent nodes, it is better to justify this complication is worthy.\n2. The experimental part is inadequate for me. E.g., (1) More experiments on studying community structures can better justify the proposed method. (2) Since the paper uses a Gamma prior to the latent representation, it should observe some sparsity level within the node representation. Sometimes, sparsified representations can demonstrate a clearer community representation. But a sparsified representation is probably not optimal for classification tasks. It would be better to include discussions on the choice of Gamma hyper-parameters.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "VSpFZ05ECY7", "reviewer": "Reviewer_zCcf", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a new graph contrastive paradigm named CLEP, which learns the latent community structure via variational inference for contrastive learning. It achieves better graph classification performance under both self-supervised and semi-supervised settings. The thorough ablation study also demonstrates the benefits and potential of learning the independent hidden communities.", "summary_of_the_review": "This paper is very innovative in its method and the problem it studies is also significant. The empirical results are also solid followed by a thorough ablation study. Although some parts need more explanation, this paper in general has a high quality. The main limitation of the model is its applicability, which right now only applies to the graph classification task. I vote to accept this paper, but additional experiments to demonstrate the advantage of the model on other graph mining tasks will be appreciated.", "strength_and_weaknesses": "Strengths:\n1. The method proposed in this paper is novel and well-supported, basically, community detection is also an important problem in graph mining, learning the hidden community structure for contrastive learning is an interesting problem.\n2. The empirical results are solid, consistent improvements over baselines are demonstrated and the authors perform lots of ablation studies.\n\nWeaknesses:\n1. Some parts of the paper are not clearly explained. The authors do not provide an explanation of why Gamma distribution and Weibull distribution are selected in section 3.4. In Table 3, what are task 1 and task 2? Some datasets in Table 3 are themselves binary classification ones, why they are multi-class classification?\n2. The graph classification task itself is not that significant, even considering that the number of classes in these datasets is limited. It would be better if the authors could extend the method to other graph mining tasks. \n3. The sensitivity analysis on the number of clusters K is missing.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "RFgxr6mZNfL", "reviewer": "Reviewer_9b1o", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposed a probabilistic graph learning framework named CLEP, which unifies graph generative models with contrastive learning paradigm to extract intra- and inter-graph information. CLEP consists of two major components: 1) the graph generative model, which defines a set of encoders to capture the hidden graph structures from different communities; 2) the contrastive paradigm, which is used to convert the learned hidden structures to high-quality graph embeddings. Intensive experiments have been conducted to verify the performance of the proposed CLEP.", "summary_of_the_review": "Generally speaking, the core idea of this paper is interesting. However, there are several aspects that raise non-negligible questions about the presented framework.", "strength_and_weaknesses": "Strengths: \n1. In general, the main idea and the proposed model are clearly explained. The proposed model is described in detail, and a competent researcher would be able to reproduce the model.  \n\n2. Intensive experiments have been conducted to verify the performance of the proposed CLEP. Apart from the node classification result on eight real-world benchmarks, this paper also conducted exploratory experiments to highlight the versatility of CLEP on different downstream tasks.\n\nWeaknesses:\n1. The motivation of the paper is quite unclear. At the beginning of this paper, the author argues that “An integration of graph generative models and graph contrastive learning methods potentially aggregates both intra- and inter-graph information and combines the complementary strengths of these two classes of models, which would further benefit representation learning”. But what is the definition of intra/inter graph information? And how does such information benefit graph learning? More importantly, how does the proposed framework handle such problems, i.e., is there any tailored component or mechanism proposed to extract intra and inter graph information?\n\n2. The concepts used in this paper are somehow casual and confusing. For example, the authors highlight the “intra- and inter-graph information” in the introduction section but quickly turn around to how to capture “heterogeneous community-specific information” in the whole methodology section. How to define “intra- and inter- graph information” and “heterogeneous community-specific information” in the learning framework? And what is the correlation between these concepts? The continuously introduced confusing concepts make this paper very hard to follow.\n\n3. Some important points of the paper are difficult to understand. For example, the authors summarized the proposed method with the sentence, “To better capture the potentially heterogeneous community-specific information, we define a set of encoders to process the information that comes from different communities” on page 2. Then, first, what’s the community-specific information? Second, why is such information heterogeneous? How to deal with the heterogeneity? Third, how does the proposed CLEP determine the number of communities, i.e., K, for different datasets? Is K a tunable hyperparameter? If so, is the number of communities crucial for the model’s learning capability? If not, is there any tailored mechanism used in this paper to automatically choose the suitable K for different datasets? From the whole paper, I cannot find the answers to the questions listed above.\n\n4. Some important baseline methods are missing. This paper supposes that edges in the graph are built based on K conceptual latent factors. In fact, the same setting has been intensively studied in many disentangled graph learning frameworks, such as DisenGCN(Disentangled Graph Convolutional Networks) and DGCL (Disentangled Contrastive Learning on Graphs).\n\n5. Parameter analysis is totally missing in this paper. As the key hyper-parameter in the proposed model, the positive temperature (\\tau_SEA) used in Equation (3) is not well studied.\n\n6. There are many typos. For example, the author mentioned that “Inspired by the “assembly” behavior of communities in graph generation, CEGCL learns community-specific graph embeddings and assemble them together to represent the entire graph” in the abstract. However, it seems that “CEGCL” is a totally new term that has never appeared in this paper.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "FL1FMGXDnu", "reviewer": "Reviewer_JYUf", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, author introduce a probabilistic framework called contrastive learning with edge partitioning (CLEP) that integrates generative modeling and graph contrastive learning. CLEP models edge generation by cumulative latent node interactions over multiple mutually independent hidden communities.", "summary_of_the_review": "It quite hard to follow the author's idea, It is seems not significant to introducing the generative model in the context of CL. Overall pipline is not clear.", "strength_and_weaknesses": "**Weaknesses**\n* I don't understand why the author need to use generative model in contrastive learning (CL) and the ELBO that the author propsed seems unrelated to the CL. Can the author explain that.\n\n* The pipline of the over framework is hard to follow. How to generate the embeddings for downstream tasks?\n\n* The authors claim the community information is important in their case. why is that? How do the author verify that the improvement is comes from adding community information. \n\n* To introduce the community information, this work is similar to clustering base GCL.  Related work [1,2] should be compared.\n\n[1] Prototypical Graph Contrastive Learning\n\n[2] Graph InfoClust: Leveraging cluster-level node information for unsupervised graph representation learning\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "B9-3LQM2L7", "reviewer": "Reviewer_AdF9", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes to integrate community-aware graph generation (or edge partition) and graph contrastive learning, which is claimed to encode both intra-graph and inter-graph information. Specifically, the basic contrastive loss is extended based on community-aware edge partition, incorporating contrastive learning on both original-augmented and community-community graph pairs. Extensive experiments are conducted on multiple datasets with unsupervised and semi-supervised graph classification, but the results can not demonstrate the effectiveness of the proposed CLEP.\n", "summary_of_the_review": "Although the idea is interesting and the proposed method seems promising, the experiments are not convinced for the effectiveness of model design.", "strength_and_weaknesses": "Strength\n1. The paper is written well, and the method is described clearly.\n2. Combining the idea of graph contaning multiple conceptual community-aware interactions and graph contrastive learning is novel, although these two methods are borrowed from existing works.\n3. The probabilistic perspective for the proposed CLEP is clear and robust, including the interpretation of the extended contrastive loss function as well as model optimization via varational inference.\n\nWeakness\n\nOverall, the weakness is primarily about experimental results:\n1. Compared with baselines, the improvement in unsupervised classification seems trivial, although the authors claim it's larger than one or two s.t.d. Therefore, the proposed CLEP may not be effective in terms of graph classification.\n2. For semi-supervised classification, only comparing with GraphCL is not convincing, and other baselines should be included, like LaGraph, MVGRL, etc.\n3. For the ablation study, the influence of K (#communities) should be evaluated, especially for K=1. This is necessary to demonstrate the effectiveness of the proposed community-aware contrastive learning.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject