Structure-Preserving Embedding of Multi-layer Networks
Authors: Yaoming Zhen
Shirong Xu
Junhui Wang
Venue: NeurIPS 2022 Submitted (2022)
Keywords: Community detection
latent space model
multi-layer network
network embedding
tensor decomposition

Abstract: 	This paper investigates structure-preserving embedding for multi-layer networks with community structure. We propose a novel generative tensor-based latent space model (TLSM) that allows heterogeneity among vertices. It embeds vertices into a low-dimensional latent space so that vertices within the same community are close to each other in the ambient space, and captures layer heterogeneity through a layer-effect factor matrix. With a general and flexible tensor decomposition on the expected network adjacency tensor, TLSM is dedicated to preserving the original vertex relations and layer-specific effects in the network embedding. An efficient alternative updating scheme is developed to estimate the model parameters and conduct community detection simultaneously. Theoretically, we establish the asymptotic consistencies of TLSM in terms of both multi-layer network estimation and community detection. The theoretical results are supported by extensive numerical experiments on both synthetic and real-life multi-layer networks.

Structure-Preservee Embedding of Multi-layer Networks
Anonymous Author(s)
Affiliation Address
email
Abstract
This paper investigates structure- structure-preserviin preserving embedding for multi- networks with community structure. We propose novel generative tensor-based latent space model (TLSM) that allows heterogeneity among vertices. It embeds vertices into low-dimensional latent space so that vertices within the same community are close to each other in the ambient space, and captures layer heterogeneity through a layer-effect factor matrix. With a general and flexible tensor decomposition the expected network adjacency tensor, TLSM is dedicated to preserving the original vertex relations and layer-specific effects in the network embedding. An efficient alternative updating scheme is developed estimate the model parameters and conduct community detection simultaneously. Theoretically, we establish the asymptotic consistencies of TLSM in terms of both multi-layer network estimation and community detection. The theoretical results are supported by extensive numerical experiments both synthetic and real-life multi-layer networks.
6
10
12
Introduction
15 Network has arisen as one of the most common structures to represent the relations among entities. In many complex systems, entities be multi-relational in that they may interact with each other under various circumstances. A multi-layer network, which consists of a common vertex set across all network layers representing the entities and an edge set each layer characterize particular type of relation among entities, is faithful to represent these relations. Examples of multi-layer networks biological networks of different 37 of various goods.
17
18
19
20 include social networks of multiple interaction channels
42
15
21
collaboration schemes 49 31
and world trading networks
In this paper, propose atructure-preservviie embedding framework for multi- layer networks 23 via a tensor-based latent space model. Specifically, TLSM utilizes the factorization of network adjacency tensor building block, embeds the vertices into low dimensional latent space, and 25 captures the heterogeneity among different layers through layer-effect factor matrix. Consequently, the community structure of the multi-layer network be detected from network embedding perspective, such that vertices within the same community are closer to one another in the ambient space than those in different communities. In addition, one key feature of TLSM is that introduces 29 a sparsity factor into the vanilla logit transformation of the network adjacency tensor, which allows TLSM to model sparse multi- multi-layer networks in more explicit fashion and accommodate relatively sparser lulti-layer networks as the ones considered in literature (22). More importantly, this sparsity factor can be estimated from the network adjacency tensor directly.
27
28
30
31
32
39 The main contribution of this paper is three-fold. First, the proposed TLSM is flexible and general in that it includes many popular network models as special cases. It also relaxes the layer-wise
35 positive semi-definite condition that has been frequently employed in literature 6 35 Second, a
36 joint modeling framework constructed for TLSM, consisting of the multi-layer network likelihood
Submitted to 36th Conference on Neural Information Processing Systems (NeurIPS 2022). Do not distribute.

---

37 and clustering type penalty, to estimate the multi-layer network and conduct community detection simultaneously. Its advantages are supported by extensive numerical experiments on both synthetic and real-life multi-layer networks. Third, the asymptotic consistencies of TLMM are established in terms of both multi-layer network estimation and community detection. Notably, the established theoretical results imply that the proposed methods can accommodate the sparsest multi-layer networks considered in literature.
38
39
40
41
42
43
The rest of the paper is organized as follows. The remaining of Section I discusses related works and introduces necessary notations. Section presents the proposed TLSM and its estimation scheme with an efficient algorithm. In Section 3, we establish the asymptotic consistencies of TLSM. Extensive numerical performance of TLSM on synthetic and real-life multi-layer networks as well as ablation studies on two novel components of the proposed method are carried out in Section 4. Section 5 concludes the paper. The supplementary materials contains technique proofs and necessary lemmas, additional simulation studies, detailed parameter tuning process, among others.
44
45
46
47
48
49
50
1.1
Related work
51
While there is growing number of literature focusing on community detection in single-layer network [48]] 28 13), community detection in multi-layer network is still in its infancy. One classical approach is to detect community structure in each layer separately 4 55, which fails to leverage
52
53
54 the homogeneity across different layers. Another approach is to aggregate multi-layer networks
55 into a single-layer one [2 35), which heavily relies on the assumption of homogeneous linking
58 pattern across multiple layers. Recently, (26] proposed to aggregate the biased-adjuste version of
57 the squared adjacency matrix in each layer to alleviate the information loss in aggregation. yet requires the average node degree to grow at a sub-optimal order.
58
In terms of multi-layer network generative models, [4] extended the seminal stochastic block model (SBM: (9) to the multi-layer stochastic block model (MLSBM: 34), where the probability for any two vertices to form an edge in given layer depends only their community memberships. Clearly, MLSBM heavily relies on the assumption of homogeneous vertices within communities.
60
61
62
The framework of MLSBM has also been incorporated in degree-corrected network estimation (36). spectral clustering 6 35 26), least square estimation [27) and saelihood-based approaches [45]. In
63
64
65
addition, network response regression model [46] and tensor factorization methods [ 22 have also been proposed to detect community structures in multi-layer networks.
66
To allow heterogeneous vertices, the latent space model (18] and random dot product graph model 3 have been extended to multi-layer networks[47] 32, 2). In addition, graph neural network and graph convolutional networks has been extended to multi-layer network for learning the multi-layer
68
69
70
network embedding [4] 23 39
71
1.2
Notations
Throughout the paper, use boldface calligraphic Euler scripts (A) to denote tensors, boldface capital letters (A) or Greece letters (ax, B3 to denote matrices, boldface lowercase letters (a) to denote vectors, and regular letters (a) to denote scalars. For an order three tensor A E I E A.... R Fa_ A..... ... E R xl,, and A E / are the i-th horizontal slide, j-th lateral slide and m-th frontal slide of A, respectively. Similarly, for matrix A, A.,. denotes its i-th row and A., 77 denotes its j-th column. For a vector a, diag(a) stands for the diagonal matrix whose diagonal is a.
73
74
75
We use ||- ||. |l. |lxc. and || ||r denote the 12-norm, loo -nom of vector, and the Frobenius norm
79 of matrix or tensor, respectively. For any integer n, denote [n] = {1, (1,2,..., n).
80
The 1 product between tensor A E R and a matrix U E R is a tensor Ax U
A E
J x such that its (f1, i2, it) entry is defined as (Ax,, x U) E A, .2.2g U, fn
81
82
The mode-2 or mode-3 product between AA and any matrix of appropriate dimension are defined 83 similarly. The CANDECOMP/PARAFANN (CP) decomposition of AA has the form
A = Σ
a (7) (()) o (
84 where (r) E RR,, b(r) E R and elr) E R for E [R]. and stands for the vector outer product. The CP-rank [24] of the tensor (r) o ((r) (t)) is defined to be 1, for r [R]. The minimal number

---

86 of rank-I I tensors in the CP decomposition of .A is called the CP-rank of A. Let I {0, []RxRRR if in
87
be the identity tensor such that I ,2,
i2 iz and 0 otherwise, and let A E R
88 B R R, and E R such that A. = (r), PB... Brr b(r) and C. ==== c Equation
then can be equivalently written as A I A X2 B C.
90 2 Structure-preserving embedding
91 In this paper, we consider multi-layer networks that can be represented as an undirected and un-
92 weighted M-layer graph g = (V,E), where V [n] consists of the common n vertices across representing the m-th relation network 94 among vertices. A order three adjacency tensor A = (ai.j.m E (0, 1 |xxxMM is then defined to represent G with entries di.i. if (i,j) E(m) and 0 otherwise.
93 different layers, and ع (E(m) J m=1 with ((m) c V
2.1
Tensor-based latent space model
97 To fully characterize the multi-layer network structure, we propose the following generative tensor- based latent space model (TLSM). For any i S E [ni]. and m E [M],
ind.
di.j.m Ojji.m
(ernoulli(pi...)) with
(2)
Pi.j.m
0 j. llog
and
(3)
8n pi
e I x1 X2 X3 B, E Das 3 llg
where I is the order three R-dimensional identity tensor. Basically, @ follows the standard routine in the multi-layer network literature [34 35 27
100
22 to model that a. = aj.imm are independently generated from Bernoulli distribution, for i S E [m] and m E MM. Denote P (Pi.j.m) R the network underlying probability tensor, and then A (0s.j.m) E R is the entry-wise transformation of P by (3). We call the transformation (3) as the modified logit transformation in that the constant in the standard logit transformation is replaced by sparsity factor s,. which may vanish with n and M. We further assume all entries of P are of the order s,; that is, there exists a constant S { Thus, &n essentially controls the overall network sparsity and the entries @ are ensured to locate in d4) (4) models the CP decomposition of θ by the
102
103
105
such that (1 g),, S Pi..j.m {s.. for i, [r]] and E [M].
106
107
E the interval |- log I-E log r= J. More importantly
108
R MR with CP-rank R, which can greatly reduce the number of free parameters from nunn + 1) / to (n - R.. Throughout the paper, the CP-rank It is allowed to diverge with n. In the CP decomposition of O, a is the vertex latent position matrix with each row Cis, serving as the embedding of vertex i, and /3 captures heterogeneity across different layers. Herein, 112
factor matrices a 1 and
109
110
111
we define the constraint sets for and B Qa {a RnxR; ||a.,,1l S log -ि. for E [n]} and Sg- {B E MRXR |13..-11 = 1,r E [[]]. Note that the constraint on B is necessary for model identification, and detailed discussion will be presented shortly. The constraint set Ωa x Gg is sufficient to maintain the bounded condition of θ since general Hölder inequality yields that I log & To conclude this paragraph, remake that the parameter is introduced for theoretical purpose and it is not treated as a tuning parameter. One can choose & sufficiently close to 1 in empirical studies so that the restriction
115
jm.jml I x1 a. X2 a." X3 3 S ||a.,,. [1.lll,,..11|@mmm....
118
119
120
on a will be alleviated.
121 We make several essential observations of the proposed TLSM. First and foremost, TLSM is flcxible and general. It includes the celebrated MLSBM 33
26 ically, suppose the vertices comes form disjoint communities, the standard MLSBM assumes that the underlying network probability tensor P B x1 Z X2 Z, where B K is semi- semi- symmetric core probability tensor with k2, = kk .r for k1, 12 E [K] and m E [M],
43 35 27
36 22 as special case. Specif-
123
125
and Z E {0, 1]"x is the community membership matrix with Z,, = 1 if vertex i comes from the k-th community and ( otherwise. That is, the probability of any vertex pair to form an edge in particular layer depends only on their community memberships. Equivalently, under the modified
126
128
129
(3) we have e = B x Z X2 Z, where i is the entry-wise transformation of B under (3) Taking R to be the CP-rank of B, the CP-decomposition of B then has the form
logit transformation
3

---

B = I X1 C X2 C' X3 3 for some matrix C E R and B E R R2Mx due to semi-symmetry. This leads to the CP decomposition of O has the form 4 with a ZC. It clear that MLSBM requires vertices within the same community are homogeneous and exchangeable, while TLSM allows vertices to have different embeddings even when they are in the same community.
131
132
133
134
135
Second, TLSM is identifiable when both a and 3 have full column ranks. When both a and B have full column ranks, the Kruskal's k-ranks [25 of a and satisfy ka = kig R. then O has CP-rank R. Hence, Ka + ka + Kg 2 2R + 2 as long as R 2 2. By Theorem of [40], the fixed column no-mo constraint of /g implies that the tensor factorization in (4) is unique up to column permutations of a and /3 and column sign flip of a. It is important to remark that the community structure encoded in a remains unchanged under any column permutation or sign flip.
136
137
138
139
140
141 Third, introducing a sparsity factor 8n via a modified logit transformation into the TLSM is non- trivial. We take single-layer network
example to illustrate the limitation of the standard logit transformation in handling sparse network. Suppose a vanilla logit link is used to connect the network underlying probability matrix P and its transformation O and the latent space model usually assumes that + =
143
144
qa""" axx" A sparse network requires the entries of θ diverge to negative infinite due the small magnitude of edge probability, which leads to unstable estimation of a in numerical experiments. Moreover, this may conffict with the assumption that vertices within the same community tend to be close the embedding space and their inner product likely to be positive. These difficulties can be naturally circumvented when an appropriate is chosen in (3)
145
148
147
148
149
150
2.2
Regularized likelihood
Given network adjacency tensor A and number of communities K, our goal is to estimate the multi-layer network embedding (ar, 3) and conduct community detection on the vertices. Throughout this paper, we assume the number of potential communities K is given and may diverge with n. Under the TLSM framework, with slight abuse of notation, we denote the average negative log-likelihood function of the multi-layer network is C(a, B; A) = [(:; A) with
M
((O;A) (nn, M)
Σ
Σ AS
L 1(0i..mi am.s,m)s
151 where cp(n, .M)
nn +1M is the number of potential edges, and L(f; = log (1+ (1+1-s
152 log (r- is a negative log-density of a Bernoulli random variable a. We now introduce
153
novel regularization term to detect the potential communities in g,
1 |la ZC
J(a)= min zer CERKKR n
(5)
where C encodes the vertex embedding centers and T c {0, 1 1 is the set of all possible community membership matrices; that is, for any Z r, each row of Z consists of only one 1 indicating the community membership and all others entries being O. This leads to the proposed regularized cost function,
154
155
156
157
(a, ((a,3;A) +,,(a))
(6)
158
where An is a positive tuning parameter that strikes the balance between network estimation and community detection in the cost function. It is clear that the embeddings of vertices with similar
159
180
linking pattern will be pushed towards the same center, and thus close to each other in the ambient space, leading to the desired community structure in G.
161
162
2.3
Projected gradient descent algorithm
163
We develop a scalable projected gradient descent (PGD) algorithm to optimize the penalized cost function (6). which is highly non-convex and can be solved only locally. PGD, which alternatively conducts gradient step and projection step, is one of the most popular and computationally fast
164
185
166
algorithm in tackling non-convex optimization problem 33 9..
To compute the gradients a and B, we introduce the following notations. Define T E R T(2.3)
167
188
with entries T s.jm I-s,
exp(-0,,1.m.) B. (Pt... a..j.m), and xa. E R=x whose i-th row

---

189 consists of the diagonal elements of the slice (T X2 a X3 (i... That is, X T(23) (1,r) =
TT X2 af X3 B²) Dir..r Similarly, we define X E &RRXM T(3) E RRxR and X T
RTXM such that X (1, m) (T X1 a X2 a Jr,, (1) (T X3 B²) Dan.. and 172 X (i, m) = T i.i.m. Consequently, when the vertex membership matrix Z and the community center matrix C are fixed, we can derive the gradients of Ca A) with respect a and B. as
173
X T(2.3) + + (3) *a) + 22, - ZC) and 2 M)
( 1" + (1)))) (a+)))
(n, M)
174
respectively. Herein, denotes the Hadamard product (entry-wise product) between two matrices.
175 Let (ōr, () denote the solution given by one-step gradient descent, we then project (ā, (3) onto
178
la x S2g in the following steps.
Step 1. Multiply the r-th column of ã., by ||1/ for r E [R]. Denote the resultant matrix as á'.
177
Step 2. Regularize each row of a as ar =/,, mint / log ف 111/ll
11. for i E [r]].
178
Step& 3. Normalize the columns of (3 as B., = 3../113...11 for forre E [R].
179
Next, when (ax, are given, we apply a (1 + a)pprooxinntoo K-means algorithm on & to update the vertex community membership matrix Z and community center matrix C.
180
182
The above steps will be alternatively conducted until convergence or reaching the maximum number of iterations. We further summarized the developed alternative updated scheme in Algorithm I in Appendix A of the supplementary materials
184
185
Several remarks on the algorithm are in order. First, Algorithm I can only be guaranteed to converge to a stationary point but not any local minimizer. We hence employ transformed higher order orthogonal iteration (HOOI) algorithm for warm initialization in all the numerical experiments in Section 4 and 5. Specifically, given a user-specific value т, we define O to mimic the magnitude of θ such that O i.j..m - if di.j.. m 0 and @....m T otherwise. A standard HOOI algorithm
186
187
189
is applied to θ to obtain ( and ())). We set T = 100 in all the numerical experiments. Second, the sparsity factor s, is an intrinsic quantity of the multi- layer network data, and it should be estimated from the network directly. Note that the minimal and maximal probabilities for any vertex (1 8) and Pmax {sn, respectively. Interestingly, Punin + Pmax = 8n, which does not depend on & any more. Therefore, we propose to estimate 8, as
190
192
183
pair form an edge in any layer are Pmin
194
8n
min 2
1
Σ
Σ
7
aij.
MAX iEin
1 Σ
Σ
T
d..jmm
(7)
n.
n.M
j=1
j=1
195
which is the sum of the minimal and maximal frequencies of vertex to form edges with all other vertices in all layers. Third, to optimally choose An. we extend the network cross-validation by edge sampling scheme in (30 to multi-layer networks. The detailed tuning procedure relegated to Appendix B in the supplementary materials.
196
197
199
3
Asymptotic theory
200
3.1
Consistency in estimating O*
201
Let 0= 1 a X2 a X3 ;: a E Pa, B E (2p} be the parameter space of the problem and O' =Ix X2 a² X3 Ba be the true underlying transformed network probability tensor. Denote
KT (n M) Σ IM Eisy E (L.00...m ai...)) I((0i..m. (i.j.mm be the averaged Bullback-Leible divergence of the network generation distributions parametrized by @" and θ, for any θ E Ω. The following large deviation inequality is derived to quantify the behavior of A)
M
203
205
for any θ in the neighborhood of O' defined by K (()))).
208
Proposition 1. Suppose (a") S ens and (n + M) (nn, M)e, Mog=>log(e"" log(en 11/2 S ei for some
207
208
constant C₁. Then with probability lease 1 exp
156,5 5. +25 log we have
()(n.M)en
; <
inf (
Cx(O:A) ||)24))))
Exg
5

---

Proposition 1 basically states that any estimators with sufficienlly small objective value should 210 be close enough to O* in terms of K ( We next study the asymptotic behavior of these estimators more precisely. Let (áx, 3) E fa x 2g be any estimator of (0", such that Lā 3: A) & Lx(a", ; + Cu:
209
211
(8)
and denote θ - I 1 5 X2 & X3 3. have the following theorem.
212
Theorem 1. Under the condition of Proposition cc,
I if (ā. 33 satisfies 18) then with probability at
least 1 - cxx
we have
156 28 28g
4V2VEn
θ - (1 V)8
ny M
The condition that (=)) S en in Proposition [ is mild. It implies that the true em- beddings of vertices within the same community are close one another. We remark that ( exactly equals to zero under the MLSBM discussed in Section 2.2. The condition that (n+ RRp (n, MM) log(en) vanishes with is also mild. When R O(1). can take any Consequently, to ensure O converges to O", Theorem implies the log smallest sparsity factor one can take is 8n > Cn > n mi) which means that the average degree
213
214
215
216
en such that en > Ing n Mf
217
218
219
of a vertex in any particular layer can be as small as 125 We remark that a common assumption necessary our theory. If further assume M (n), we find that the average degree of vertex in any layer under the
220
M- (n) that appears in literature, such as 27 and [22]]
proposed TLSM set up can be smaller than that in (27) by factor (M log n) 11)-1/ and in [22] by
222
223
factor (log n) showing that our theoretical result accommodates sparser multi-layer networks.
3.2
Consistency in community detection
225
We now turn to establish the consistency of community detection in multi-layer network G. Let th [n] - [K] be the true community assignment function such that 3 227 arg ming ming E ||a; - -C,.ll and then the community detection error of any esti- mated community assignment function v can be evaluated by the minimum scaled Hamming distance between d and ul* under permutations, which is defined as
a + (b))).
err( 3b 3 min TESK n2
Σ
230
where 1(.) is the indicator function and Sx is the symmetric group of degree K. Such scaled or unscaled Hamming distance has become popular metric in quantifying the performance of community detection 22).
232
239
Denote N = {i= : v = k} be the k-th true underlying community whose cardinality is rig- Let C E R RKXR be the true underlying community centers of the network embedding with Cj. = Ep oi,,, and let B I xx C* X2 C* X B The following assumptions are made to ensure that communities within the multi-layer networks are asymptotically identifiable.
234
235
236
Assumption A. Assume the difference between any two distinct horizontal slides BB satisfies that
min
B
-B² 7n-
k VKM
where To may vanish with no.
Assumption B. Assume the tuning parameter Xn satisfies that > 1, C22
239 for an absolute constant 62 that does not depend on any model parameter.
Assumption C. Denote Remin = min T& as the minimal community size. Assume
Tn Tl min /K
CE
8n.
4.22 + ca 240 where 0g (1-EIEE
(1+6) min
M. (onn(MRR)) M
and C3 is a constant that depends on & only.

---

[M] and RR.. We then rescale the column norms B to be 1 for model identifiability. Finally, we generate A according to the proposed TLSM with 8n 0.1. For the sake of fair comparisons, the embedding dimension Assumption A is the minimal community separation requirement, and similar assumption has been
242
employed 27 with a constant "n- Together with the condition \,, S c, in Proposition |
Assumption B gives feasible interval for An Assumption Callows for unbalanced communities
244
with vanishing Munin/n n if the network is not too sparse. Note that og can be further bounded by
(1 DIVE + + 8, and the first term of eg will dominate the second term if R o(M).
1v2
Theorem 2. Suppose all the assumptions in Theorem I as well as Assumptions A
B and C
are
satisfied, holds true that
E nen
err(y"" .b))
mmin K 8n
with probability at least 1 1 n2 2 exx
(n, lea
248
156 156,--
log
247
Theorem 2 assures that the community structure in a multi-layer network can be consistently recovered
(log n)
248 by the proposed TLSM. As a theoretical example, we consider sparse case with sn n min (n, (f)
where 0 Ti 1, max On + Z'C"IlF I (logn) -3/2, and both Yn. R and K
are of constant orders. With An = (log 2 Theorems and imply that en = (log 2)) 1
250
min
with
and err(""" ,0"000 + (1).
4
Numerical experiments
253
In this section, we evaluate the numerical performance of the proposed TLSM in a variety of synthetic well real-life multi-layer networks, compare it against four competitors in literature, including the mean adjacency spectral embeddings (MASE; (16), least square estimation (LSE; (2)). Tucker decomposition with HOSVD initialization (HOker-Tucker;22 22), and spectral kernel (SPECK: and conduct some ablation studies. The implementations of LSE and SPECK are available at the authors' personal websites, HOSVD-Tucker is implemented in the routine "tucker" of the Python package "tensorly", and TLSM and MASE are implemented in Python by ourselves.
255
257
259
280
4.1
Synthetic networks
The multi-layer network A= (ai.j.m E {0, 1] is generated as follows. First, we randomly select K elements uniformly from {2.5* (...=bb,,........ bR). : b, E (-1, r E [R] as community centers, which are denoted as Ck. & E [K]. Second, the latent space embedding of vertex is generated as a; = Cv. + e, with e; ~ N NN 1.5 IR), and eù E [K] are independently drawn from the multinomial distribution Multi(1; K 1x). Third, we generate , [B,,... ..r......'' with 3m. being independent standard normal random varibeles, for
282
264
265
287
269 R is set as K in all scenarios. We aim to illustrate the community detection performance of
all methods the number of vertices and number of layers increase. To this end, consider (n, M) E {200, 400, 600, 800} x (5. 10, 15, 20). The averaged hamming errors and their standard errors over 50 independent experiments of all methods reported in Table
271
273
evident that TLSM consistently outperforms its competitors, and the performances of LSE and HOSVD-Tucker are better than those of MASE and SPECK. This is expected since TLSM, LSE and HOSVD- Tuckerr work on the multi-layer network adjacency tensor directly, while MASE and SPECK are matrix aggregation methods that suffer form information loss. Furthermore, the number of vertices and number of layers increase, the community detection errors of all methods decrease rapidly. Notably, TLSM and LSE converge faster than the other methods, and attain stable performance even for relatively small n and M. Additional simulation studies for various network sparsity and unbalanced community sizes relegated Appendix C in the supplementary materials.
275
277
279
4.2
Real-life networks
282
We also apply the proposed SM method to analyze three real-life multi-layer networks, including social network in the department of Computer Science at Aarhus University (AUCS) (38), yeast
7

---

Table I: The averaged hamming errors of various methods with their standard errors in Scenario I.
The best performer in each case is bold-faced.
74
M
TLSM
LSE
MASE
HOSkDeTucckr
SPECK
5
0.1180(0.0144
0.1405(0.0118)
0.5086(0.0136)
0.4254(0.01 0.1623(0.0126)
10
0.0585(0.0044)
0.0751
0.4949(0.0131)
0.2996(0.014)) 0.0046))
200
15
0.0551(0.0067)
0.0593(0.0045) (0.05)))
0.10400.00115) 0.4910(000176)
0.2505(0.0142)
20
0.0510 00377
0.0588(0.0043)
0.49770.00661
0.1023 10)
0.1942(0.0156)
5
0.0653 (0666
0.1019(0.0087
0.3845(0.0193)
0.1220X0.0106)
0.3766(0.0195)
10
0.0608(0.0M)) 0063)
0.0636(0.0037)
0.3859(0.0160)
0.1012(0.0092)
0.2244(0.0191)
400
15
1.0511(0.00) 1))
0.0595(0.003))
0.3844(0.0221
0.0787(0.005))
0.1490(0.0123)
20
0.0536(0.0047)
0.0551
0.3985(0.0185)
0.0795(0.0063)
O. 0.1409(0.0111)
5
0.0607 (0.009)
0.000
0.3665(0.0186)
O. 0.1221(0.0108)
0.3038(0.0193)
10
0.0567 0029)
0.0688(0.0033
0.3726(0.017) 0179)
0.1003 (0.081)
O. (..65())
600
15
(0.002))))22))
0.0630(0.0030 0030)
0.3803(0.0167)
0.0918(0.006))
0.12010000076)
20
0.0548 00.008)
0.0586(0.0029)
0.3814(0.0185)
0.0883(00008))
0.1150(0.0088)
5
0..556(0.00566
0.0768(0.0055)
0.3012(0.0194)
0.2733(.017 0.1003(00010))
10
0.0560(0.006$
0.0583(0.003))
0.3004(0.0177)
0.0788(0.0065)
0.1424(0.0127)
800
1.5
0.0498 0030)
0.0539(0.0033)
0.3179(0.0195)
0.0812(0.0068)
O. 0.1146(0.0098)
20
0.0485(0.0031)
0.0516(0.0032)
0.3184(0.0211))
0.000))0.0075)
0.0979(0.0078)
Saccharomyces cerevisiae gene co-expression (YSCGC) network [44], and worldwide agriculture trading network (WAT) [TO]. Specifically, we conduct community detection on the first two networks whose vertex community memberships are available, and carry out link prediction task on the third network whose vertex community memberships are unavailable.
285
286
287
288
The AUCS dataset is publicly available h/thtttp:/mltilaaaaee
it. uu se/datasets html and it is a 61 61 x 5 multi-layer network that records pairwise relationships of types among 61 persons in AUCS, including current working relationships, repeated leisure activities, regularly eating lunch together, co-authorship of a publication, and friendship on Facebook. Since 54 persons in the dataset come from 7 research groups and the other 7 persons do not belong to any group, the dataset consists of communities corresponding to 7 research groups and an outlier community. Applying TLSM and its competitors to the dataset, the number of misclassified vertices by TLSM, LSE, MASE, VDSVVTTTkke and SPECK, are 8, 21, 19, 23, 18, respectively. Clearly, TLSM significantly outperforms its competitors by at least reducing 16.39% of community detection error.
289
291
293
295
296
297
The YSCGC dataset is publicly available at athttps://www. ncbi .nlm. mih. hoov/mmc/articlesss
PMC1 7MC156590/ and contains 205 genes of 4 functional categories, including protein metabolism and modification, carbohydrate metabolism and catabolism, nucleobase, nucleoside, nucleotide and nucleic acide metabolism, well as transportation. We regard these four functional category labels as the community memberships of the genes. Further, the gene expression responses are measured by 20 systematic perturbations with varying genetic and environmental conditions in 4 replicated hybridizations. We thus constructed a gene co-expression network AA = (at...m) E R 206xx00xx4 based on the similarities of their expressions, where each layer represents one replicated hybridization. Specifically, the similarity between genes i and in the m-th replication is measured
299
301
302
303
304
308 by Ws.j.m exp | |=="") - x (m) where x;! a!!")) E _ contains the expression levels of 20
mm)
307
perturbations in the m-th replicated hybridization for E [205] and m E [4]. The binary value ai,j. is obtained by thresholding Wi.j.m with the thresholding value being the 60% quantile of all elements in (ws.j.m : i S j E [205). m E [4]] Applying TLSM and its competitors to this dataset, the number of misclassified vertices by TL.M, LSE, MASE, HOSVD-TTcker and SPECK, are 6, 9, 12, 48, 13, respectively. TLSM again outperforms its competitors in this YSCGC dataset.
309
310
311
312
The WAT dataset is publicly available at http::/mwe wwww fao. org, and includes 364 agriculture product trading relationships among 214 countries in 2010. To process the data, we extract 130 major countries whose average degrees are greater than 9 from the 32 densest connected agriculture product trading relations, leading to a 130 130 x 32 multi-layer network. Investigating the eigen-structure of the mode-1 1 matricization of the network adjacency tensor, we identify an elbow point [20] at the
313
314
315
316
317
7th largest vigen-value, suggesting there are potential communities among the countries, and thus
we set K = 6. The corresponding rigen-value plot is attached in Appendex D of the supplementary
319
materials. We then randomly selected 80% of the entries of the adjacency tensor as the training set,
and conduct link prediction on the remaining 20% of the entries. Specifically, we employ TLSM
8

---

and the adaptations of its competitors to estimate the network expected tensor P and generate estimations for the missing entries by independent Bernoulli random variables accordingly. The averaged link prediction accuracy of TI.SM, LSE, MASE, HOSVD-Tuccer and SPECK over 50 independent replications are 79. 79.60%, 76.66%, 75. 75.96%. 77.78% and 79.08%, respectively, where the link prediction accuracy is defined as the percentile of the correctly predicted entries. Clearly, all 5 methods are comparative in terms of link prediction, while TLSM still deliver highest averaged link
322
324
325
326
prediction accuracy.
328
4.3
Ablation studies
329
In this subsection, we carry out some ablation studies on two novel components of the proposed method, namely the sparsity factor in and the conmunity-inducing regularizer J(a).. To study the
330
effectiveness 8n, we generate a 300 x 300 x multi-layer network with communities and true network sparsity 8, 0.3. The blue curve in the left panel of Figure | shows the average Hamming error of 50 independent replications given by the proposed method when employing {0.05i E [20]} in the optimization algorithm, and the red line indicates the averaged Hamming error the proposed method with š, estimated via the proposed data-adapted estimation scheme. It is clear that the Hamming enror at 8n = is much larger than that when s, is close to 0.3, showing the advantages
331
335
8n
335
337
the modified logit transformation by 8n over the standard logit transformation when the network indeed reveals sparse pattern. Moreover,
observe that the red line is even lower than the minimum Hamming error in the blue curve. This further confirms the effectiveness of the proposed data- data-adapted estimation scheme for estimating Sn.
339
A= Tusing
3.025
3.0
2.020
Lamners
5.015
2.000
05
0.2
50
100
260
400
Number ofdde
Figure 1: Ablation studies on 8n (left) and community-inducing regularizer (right).
341
To study the effectiveness the community-inducing regularizer in the proposed objective function, generate an x 5 multi-layer network with communities, for E (50, 100, 200, 400). In the right panel of Figure the black pillars indicate the network estimation error 1
343
O' given by the proposed method with \n = 0 which corresponds to the absence of J(x), while the red ones indicate the counterparts given by the proposed method with X, is selected by network cross-validation. There is clear improvement when the community-inducing regularizer is enforced in all scenarios, particularly for small n. This showcases the helpfulness of the community-inducing regularizer in detecting network community structure.
ny5
345
347
5 Conclusions
349
350
In this paper, we propose a novel tensor-based latent space model for community detection in multi-layer networks. The model embeds vertices into a low-dimensional latent space and views the community structure from an network embedding perspective, so that heterogeneous structures in different network layers can be properly integrated. The proposed model is formulated as regularization framework, which conducts multi-layer network estimation and community detection simultaneously. The advantages of the proposed method are supported by extensive numerical experiments and theoretical results. Particularly, the asymptotic consistencies of the proposed method are established in terms of both multi-layer network estimation and community detection, even for relatively sparse networks.
352
353
354
355
357

---

References
359
360
[1] Luiz GA Alves, Giuseppe Mangioni, Isabella Cingolani, Francisco Aparecido Rodrigues, Pietro Panzarasa, and Yamir Moreno. The nested structural organization of the worldwide trade multi-layer network. Scientific reports, 9(1):1-14, 2019.
362
365
[2] Jesús Arroyo, Avanti Athreya, Joshua Cape, Guodong Chen, Carey E Priebe, and Joshua Vogelstein. Inference for multiple heterogeneous networks with a common invariant subspace. Journal of Machine Learning Research, 22(142):1-49, 2021.
364
385
366
[3] Avanti Athreya, Donniell E Fishkind, Minh Tang, Carey E Priebe, Youngser Park, Joshua Vogelstein, Keith Levin, Vince Lyzinski, and Yichen Qin. Statistical inference on random dot product graphs: a survey. The Journal of Machine Learning Research, 18(1):8393-843) 2017.
368
369
[4] Matteo Barigozzi, Giorgio Fagiolo, and Giuseppe Mangioni. Identifying the community structure of the internationalltradee multi-network. Physica A: statistical mechanics and its applicationns 390(1 1-2666, 2011.
370
371
372
[5] Michele Berlingerio, Fabio Pinelli, and Francesco Calabrese. Abacus: frequent pattern mining- based community discovery in multidimensional networks. Data Mining and Knowledge Discovery, 27(3):294-322 2013.
373
374
375
[6] Sharmodeep Bhattacharyya and Shirshendu Chatterjee. Spectral clustering for multiple sparse networks: I. arXiv preprint arXiv:1805.1000 10594, 2018.
378
377
Han Chen, Garvesh Raskutti, and Ming Yuan. Non-convex projected gradient descent for generalized low-rank tensor regression. Journal of Machine Learning Research, 20:1-37, 2019.
378
379
[8] Zitai Chen, Chuan Chen, Zibin Zheng, and Yi Zhu, Tensor decomposition for multilayer net- works clustering, In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 3371-3378, 2019.
380
381
382
[9] Eric C Chi, Brian R Gaines, Will Wei Sun, Hua Zhou, and Jian Yang. Provable convex co-clustering of tensors. Journal of Machine Learning Research, 21(214):1-58, 2020.
383
384
[10] Manlio De Domenico, Vincenzo Nicosia, Alexandre Arenas, and Vito Latora. Structural reducibility of multilayer networks. Nature communications, 6(1):1-9, 2015.
385
[11] Lieven De Lathauwer, Bart De Moor, and Joos Vandewalle. On the best rank-1 1 and rank-(r 1, r 2..... m) approximation of higher-order tensors. SIAM journal on Matrix Analysis and Applications, 21(4):1324-13 1342, 2000.
387
[12] Xiaowen Dong, Pascal Frossard, Pierre Vandergheynst, and Nikolai Nefedov. Clustering with multi-layer graphs: A spectral perspective. IEEE Transactions on Signal Processing, 60(11):5820-588 -8331, 2012.
390
[13] Junxian Geng, Anirban Bhattacharya, and Debdeep Pati. Probabilistic community detection with unknown number of communities. Journal of the American Statistical Association,
393
1 114(526):833-9055 2019.
[14] Mahsa Ghorbani, Mahdieh Soleymani Baghshah, and Hamid R Rabiee. Mgen: semi-supervised classification in multi-layer graphs with graph convolutional networks. In Proceedings of the 2019 IEEE/ACM International Conference Advances in Social Networks Analysis and Mining, pages 208-211, 2019.
396
398
399
[15] Derek Greene and Pádraig Cunningham. Producing a unified graph representation from multiple social network views. In Proceedings of the 5th annual ACM web science conference, pages 1 118-121,22 2013.
401
402
[16] Qiuyi Han, Kevin Xu, and Edoardo Airoldi. Consistent estimation of dynamic and multi-layer block models. In International Conference on Machine Learning, pages 1511-1520. PMLR, 2015.
404
405
[17] Xin He, Qiong Liu, and You Yang. Mv-gnn: Multi-view graph neural network for compression artifacts reduction. IEEE Transactions Image Processing, 086829-6840 2020.
10

---

407
[18]
Peter D Hoff, Adrian E Raftery, and Mark S Handcock. Latent space approaches to social network analysis. Journal of the American Statistical Association, 97(460):1000-09933 2002.
408
409
[19] Paul W Holland, Kathryn Blackmond Laskey, and Samuel Leinhardt. Stochastic blockmodels: First steps. Social networks, 5(2):109-133) 1983.
[20]
Pengsheng Ji and Jiashun Jin. Coauthorship and citation networks for statisticians. The Annals of Applied Statistics, 1774: 2016.
412
413
[21]
Jiashun Jin. Fast community detection by score. Ann. Statist., 43(1):57-88 02 2015.
414
[22] Bing-Yi Yi Jing, Ting Li, Zhongyuan Lyu, and Dong Xia. Community detection on mixture multilayer networks via regularized tensor decomposition. The Annals of Statistics, 49(6))3181- 3205, 2021.
416
417
[23]
Muhammad Raza Khan and Joshua Blumenstock. Multi-gen: Graph convolutional networks for multi-view networks, with applications to global poverty. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 606-613, 2019.
419
420
[24] Tamara G. Kolda and Brett W. Bader. Tensor decompositions and applications. SIAM Review, 51:455-500, 2009.
422
[25] Joseph B Kruskal. Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics. Linear algebra and its applications, 18(2):95- 138. 1977.
423
424
425
[66]
Jing Lei. Tail bounds for matrix quadratic forms and bias adjusted spectral clustering in multi-layer stochastic block models. arXiv preprint arXiv:2003.08222 2020.
428
427
[27] Jing Lei, Kehui Chen, and Brian Lynch. Consistent community detection in multi-layer network data. Biometrika, 107(1):61-73, 2020.
428
429
[88] Jing Lei and Alessandro Rinaldo. Consistency of spectral clustering in stochastic block models. The Annals of Statistics, 43(1):215-23) 2015.
430
[29] Dong Li, Zhisong Pan, Guyu Hu, Graham Anderson, and Shan He. Active module identification from multilayer weighted gene co-expression networks: a continuous optimization approach. IEEE/ACM transactions on computational biology and bioinformatics, 2020.
432
[30] Tianxi Li, Elizaveta Levina, and Ji Zhu. Network cross-validation by edge sampling. Biometrika,
435
107(2):257-273)) 2020.
436
[3]] Xueming Liu, Enrico Maiorino, Arda Halu, Kimberly Glass, Rashmi B Prasad, Joseph Loscalzo, Jianxi Gao, and Amitabh Sharma. Robustness and lethality in multilayer biological molecular networks. Nature communications, 11(1):1-12, 2020.
438
439
[32] Zhongyuan Lyu, Dong Xia, and Yuan Zhang. Latent space model for higher-order networks and generalized tensor decomposition. arXiv preprint arXiv:2 16042, 2021.
[33] Zhuang Ma, Zongming Ma, and Hongsong Yuan. Universal latent space model fitting for large networks with edge covariates. Journal of Machine Learning Research, 21(4):1-67 2020.
442
443
[34] Subhadeep Paul and Yuguo Chen. Consistent community detection in multi-relational data through restricted multi-layer stochastic blockmodel. Electronic Journal of Statisties, 10(2):3807-3870, 2016.
444
445
446
[35] Subhadeep Paul and Yuguo Chen. Spectral and matrix factorization methods for consistent community detection in multi-layed networks. Ann. Statist., 48(1):230-254) 02 2020.
447
448
[36] Subhadeep Paul and Yuguo Chen. Null models and community detection in multi-layer networks. Sankhya A, pages 1-55, 2021.
449
[37] Zhuo-Ming Ren, An Zeng. and Yi-Cheng Zhang. Bridging nestedness and economic complexity in multilayer world trade networks. Humanities and Social Sciences Communications, 7(1):188, 2020.
451
11

---

453
[38]
Luca Rossi and Matteo Magnani. Towards effective visual analytics on multiplex and multilayer networks. Chaos, Solitons Fractals, 72:68-76, 2015.
454
456
[39] Uday Shankar Shanthamallu, Jayaraman Thiagarajan, Huan Song, and Andreas Spanias. Gramme: Semisupervised learning using multilayered graph attention models. IEEE transac- tions on neural networks and learning systems, 31 31110):9977-3988 2019.
456
457
[40] Nicholas D Sidiropoulos and Rasmus Bro. On the uniqueness of multilinear decomposition of n-way arrays. Journal of Chemometries: A Journal of the Chemometrics Society, 14(3):229-239, 2000.
459
460
461
[41] Wei Tang, Zhengdong Lu, and Inderjit S Dhillon. Clustering with multiple graphs. In 2009 Ninth IEEE International Conference on Data Mining, pages 1016-1021. IEEE, 2009.
462
463
[42] Edwin JCG Van Den Oord and Ronan Van Rossem. Differences in first graders' school adjustment: The role of classroom characteristics and social structure of the group. Journal of School Psychology, 40(5):371-399) 2002.
465
[43] James D Wilson, John Palowitch, Shankar Bhamidi, and Andrew B Nobel. Community extraction in multilayer networks with heterogeneous community structure. The Journal of Machine Learning Research, 18(1)55458-5506 2017.
467
489
[44] Ka Yee Yeung, Mario Medvedovic, and Roger E Bumgarner. Clustering gene-expression data with repeated measurements, Genome hiology, 4(5):1-17, 2003.
471
[45] Yubai Yuan and Annie Qu. Community detection with dependent connectivity. The Annals of Statistics, 49(4):2378-25) 2428, 2021.
475
[46] Jingfei Zhang, Will Wei Sun, and Lexin Li. Network response regression for modeling popula- tion of networks with covariates. arXiv preprint arXiv: 2018.
475
[47]
Xuefei Zhang, Songkai Xue, and Ji Zhu. flexible latent space model for multilayer networks. In International Conference on Machine Learning, pages 1128--11297 PMLR, 2020.
477
[48] Yunpeng Zhao, Elizaveta Levina, and Ji Zhu. Consistency of community detection in networks
478
under degree-correcn stochastic block models. The Annals of Statistics, 40(4):2266-2) 22222226622., 2012.
479
480
[49] Wei Zheng, Dingjie Wang, and Xiufen Zou. Control of multilayer biological networks and applied to target identification of complex diseases. BMC bioinformatics, 20(1):1-12, 2019.
481
Checklist
482
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the pa- per's contributions and scope? [Yes] See the abstract and the third paragrath of the introduction.
485
487
(b) Did you describe the limitations of your work? [Yes] The optimization algorithm can only be guaranteed to converge to stationary point.
488
(c) Did you discuss any potential negative societal impacts of your work? [No] There should be no negative socictal impacts.
490
481
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
482
483
2. If you are including theoretical results...
494
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Section 3.
495
(b) Did you include complete proofs of all theoretical results? [Yes] All technical proofs provided in Appendix E of the supplementary materials.
3. If you experiments...
12

---

498
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material or as a URL)? [Yes] The URLs for data are included in Section 4.2, and codes with instructions are included in the supplementary materials.
499
500
501
502
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section 2.3 and Appendix B in the supplementary materials. (c) Did you report error bars (e.g.. with respect the random seed after running experi- ments multiple times)? [Yes] We show the standard erros in Table I and 95% confident intervals of additional simulation studies Appendix C in the supplementary materials.
504
505
506
507
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [No]
509
4. If you are using existing assets (e.g., code, data, models) or curating/releasngg new assets...
510
(a) If your work uses existing assets, did you cite the creators? [Yes] We used publicly available datasets and cite the creators.
512
(b) Did you mention the license of the assets? [Yes] All datasets we used are publicly available.
513
(c) Did you include any assets either in the supplemental material URL? [No]
515
(d) Did you discuss whether and how consent was obtained from people whose data you' 're using/curating? [No]
516
517
(e) Did you discuss whether the data you are using/curating contains personally identifiable
518
information or offensive content? [No] All data we used do not contains personally
519
identifiable information or offensive content.
520
5. If you used crowdsourcing or conducted research with human subjects...
521
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
523
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
524
525
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
526
13


Reviews:
[{"review_id": "kzs9OHRj33P", "reviewer": "Reviewer_juke", "summary": "The paper deals with the problem of learning node  embeddings of multilayer graphs with applications mainly in community detection and link prediction. A new model is introduced, namely TLSM, that is based on flexible tensor decomposition framework. Specifically, the methodology is based on a tensor latent space model that satisfies a set of interesting properties: it allows nodes to get different embeddings even if they belong to the same community; it satisfies some key identifiability properties; and finally, it is capable of handling sparse networks though a modified logit transformation. The different claims made in the paper are supported either with theoretical arguments or empirically. ", "questions": "The different questions that I would like to ask the authors are provided in the list of weaknesses of the paper.", "limitations": "N/A", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths:\n\n- The paper addresses an important problem in graph machine learning, with many practical applications. \n- I found particularly interesting the fact that the paper comes with a consistency analysis regarding the proposed methodology.\n- The paper is also well-written, and most of the arguments made are clearly presented. I really enjoyed reading it.\n\n### Weaknesses:\n\n- Missing related work. Despite the fact that the multi-layer community detection literature is not as rich as in the case of single-layer graphs, still there are plenty of methodologies that follow different ideas. The paper mentions a few of them, but the literature could further be expanded. It would be interesting to also consider some of these models in the empirical analysis. I will just mention the article by Mercado et al. entitled “The Power Mean Laplacian for Multilayer Graph Clustering” (AISTATS ’18), and possibly some of the references within this article. \n- I enjoyed reading the part related to the consistency analysis. However, it is not clear to me how strong are these assumptions from a practical viewpoint. How will the model empirically behave if some of these assumptions will be violated?\n\nMy main concerns about the paper are related to the empirical analysis. \n- Why is the embedding dimension set to $K$? \n- The scale of the datasets used is quite small. Is there any particular reason for this choice?\n- There is no discussion about the time complexity of the model or even the empirical running time. I would suggest the authors to discuss this point.\n- Selection of baseline models. As I also mentioned above, I found that the selected models do not cover different methodological ideas on clustering multilayer graphs.\n- Lastly, despite mentioning that the paper learning embedding of multilayer graphs, most of the discussion and analysis concerns the task of community detection. There is one experiment on link prediction, but I believe this is quite limited. Did the authors think of having a more generic experimental framework that will more extensively cover the tasks of link prediction and, possibly, node classification?\n\n\n### Typos:\n\n- Line 58, *y*et\n- Line  73: Greek letters\n- Line 123: come*s*\n \n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "dDiS4dr6d4p", "reviewer": "Reviewer_6nBv", "summary": "This paper proposes a generative tensor-based latent space model, dubbed TLSM, which generates node embeddings preserving the community structure of the given multi-layer networks. Instead of directly using the log-likelihood function to encode the heterogeneous structure of the multi-layer networks, the authors also introduce a clustering type penalty to simultaneously embed the community information between nodes. Projected gradient descent (PGD) is used to optimize the model parameters, and the asymptotic consistencies of the proposed method are also analyzed.  ", "questions": "I am a bit concerned about the proposed model's time complexity. Could the authors provide more details about the model complexity?\n", "limitations": "The authors did not address the limitations and potential negative societal impact. \n", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: Firstly, the proposed model is flexible and general, with many popular network models included. Secondly, the designed regularized likelihood framework enables the proposed model to estimate the multi-layer network and conduct community detection simultaneously. Thirdly, the authors establish a theoretical analysis of the proposed model's asymptotic consistency in terms of both multi-layer network estimation and community detection.\n\nWeaknesses: Firstly, although the proposed framework is flexible and general, the idea of using tensor decomposition is not very novel, and the regularized likelihood design is only incremental. Secondly, I think the PGD algorithm does not guarantee convergence to the global minimum, and thus we may need to select the initial point carefully. Finally, in the real-life experiments, the proposed model does not perform much better than the benchmark approaches.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "6iPUuTV1yOq", "reviewer": "Reviewer_q91p", "summary": "In this paper, the authors propose TLSM, a novel tensor-based latent space model for community detection in multi-layer networks. TLSM integrates the heterogenous network structure in different layers by embedding nodes into a low-dimensional space with the aim of nodes within the same community should have closer embeddings. Further, TLSM utilizes a regularized framework consisting of the average negative log-likelihood function of the multi-layer network and the clustering regularizer, to estimate the multi-layer network and conduct community detection simultaneously. The authors also provide theoretical analysis regarding the asymptotic consistencies of TLSM of both the multi-layer network and community detection. ", "questions": "1. It seems that TLSM mainly leverages the tensor CP decomposition to integrate the heterogeneous structure of the multi-layer networks. The reviewer wonders if the integration would be better if TLSM incorporates random walks between different layers.\n\n2. The computational complexity of TLSM is not mentioned in the paper, and thus it would be great if the author could discuss further TLSM's complexity.", "limitations": "Authors did not mention the Limitations and societal impacts. The reviewer thinks the limitations of the proposed model are mainly the optimality of learned parameters and the model's performance.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. TLSM is a flexible and general framework that contains many multi-layer network generative models such as the multi-layer stochastic block model.\n\n2. TLSM estimates the multi-layer network and performs community detection simultaneously by adding a clustering penalty to the multi-layer network likelihood function.\n\n3. TLSM analyzes the asymptotic consistencies in terms of both the multi-layer network and the community detection.\n\nWeaknesses:\n\n1. TLSM applies projected gradient descent to optimize the regularized likelihood, which can only guarantee achieving a local optimum. The authors mentioned employing a transformed higher order orthogonal iteration (HOOI) algorithm for warm initialization, but it would be great if the authors could discuss it further in detail.\n\n2. TLSM outperforms baseline methods in the synthetic networks, while does not significantly outperform baselines in the real-world networks. The reviewer wonders if the authors could provide more details about the real-world experiments.\n\n3. Although TLSM is flexible and general, the techniques it used are not novel.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Reject