DIGRAC: Digraph Clustering Based on Flow Imbalance
Abstract
Node clustering is a powerful tool in the analysis of networks.
We introduce a graph neural network framework, named DIGRAC, to obtain node embeddings for directed networks in a self-supervised manner, including a novel probabilistic imbalance loss, which can be used for network clustering.
Here, we propose directed ﬂow imbalance measures, which are tightly related to directionality, to reveal clusters in the network even when there is no density difference between clusters.
In contrast to standard approaches in the literature, in this paper, directionality is not treated as a nuisance, but rather contains the main signal.
DIGRAC optimizes directed ﬂow imbalance for clustering without requiring label supervision, unlike existing graph neural network methods, and can naturally incorporate node features, unlike existing spectral methods.
Extensive experimental results on synthetic data, in the form of directed stochastic block models, and real-world data at different scales, demonstrate that our method, based on ﬂow imbalance, attains state-of-theart results on directed graph clustering when compared against 10 state-of-the-art methods from the literature, for a wide range of noise and sparsity levels, graph structures, and topologies, and even outperforms supervised methods.
Introduction
Revealing an underlying community structure of directed networks ( digraphs ) is an important problem in many applications, see for example [1] and [2], such as detecting inﬂuential social groups [3] and analyzing migration patterns [4].
While most existing methods that could be applied to directed clustering use local edge densities as main signal and directionality (i e, edge orientation) as additional signal, we argue that even in the absence of any edge density differences, directionality can play a vital role in directed clustering as it can reveal latent properties of network ﬂows.
The underlying intuition is that homogeneous clusters of nodes form meta-nodes in a meta-graph , with the meta-graph directing the ﬂow between clusters; directed core-periphery structure is such an example [5].
Loosely speaking, a meta-node is a collection of nodes, and a meta-graph is a graph on such meta-nodes, with weighted edges collecting the overall sum of edge weights between the meta-nodes.
Fig.
1(a) is an example of ﬂow imbalance between two clusters, here on an unweighted network for simplicity: while 80% of the edges ﬂow from the Transient cluster to the Sink cluster, only 20% ﬂow in the other direction.
As a real-world example, Fig. 1(b) shows the strongest ﬂow imbalances between clusters detected by our method in a network of US migration ﬂow [4]; most edges ﬂow from the red cluster (label 1) to the blue one (label 2).
Figures 1(c-d) show examples on a synthetic meta-graph.
We could also think of a social network in which a set of fake accounts
A have been
[Figure]
−−−−−−
L a b e l
[Figure]
graph.
[Figure]
of by there would be many more messages from A to B than from B to A , hinting that A is most likely comprised of fake accounts
Thus, instead of ﬁnding relatively dense groups of nodes in digraphs with a relatively small amount of ﬂow between the groups as in [6–11], our main goal is to recover clusters with strongly imbalanced ﬂow among them, in the spirit of [12, 13] where directionality is the main signal.
This task is not addressed by most methods for node clustering in digraphs, including community detection methods.
Those methods that do lay emphasis on directionality are usually spectral methods, for which incorporating features is non-trivial, or graph neural network (GNN) methods that require labeling information.
An exception is the network community detection method InfoMap [14] which uses directed random walks; however, it still relies on some edge density information within clusters, as a walk is more likely to happen when the density is higher.
[15] and [16] employ Markov chains, but we pick InfoMap as a representative for methods based on information theory/Markov chains.
Here we introduce DIGRAC, a GNN framework to obtain node embeddings for clustering digraphs (allowing weighted edges and self-loops but no multiple edges).
In a self-supervised manner, a novel probabilistic imbalance loss is proposed to act on the digraph induced by all training nodes.
The global imbalance score, one minus whom is the self-supervised loss function, is aggregated from pairwise normalized cut imbalances.
The method is end-to-end in combining embedding generation and clustering without an intermediate step.
To the best of our knowledge, this is the ﬁrst
GNN method which derives node embeddings for digraphs that directly maximizes ﬂow imbalance between pairs of clusters.
With an emphasis on the use of a direction-based ﬂow imbalance objective, experimental results on synthetic data and real-world data at different scales demonstrate that our method can achieve leading performance for a wide range of network densities and topologies.
DIGRAC’s main novelty is the ability to cluster based on direction-based ﬂow imbalance, instead of using classical criteria such as maximizing relative densities within clusters.
Compared with prior methods that focus on directionality, DIGRAC can easily consider node features and also does not require known node clustering labels.
DIGRAC complements existing approaches in various aspects: (1) Our results show that DIGRAC complements classical community detection by detecting alternative patterns in the data, such as meta-graph structures, which are otherwise not detectable by existing methods.
This aspect of detecting novel structures in directed graphs has also been emphasized in [12].
(2) DIGRAC complements existing spectral methods, through the possibility of including exogenous information, in the form of node-level features or labels, thus borrowing their strength.
(3) DIGRAC complements existing GNN methods by introducing an imbalancebased objective.
(4) DIGRAC introduces imbalance measures for evaluation when ground-truth is unavailable.
DIGRAC’s applicability extends beyond settings where the input data is a digraph: with time series data as input, the digraph construction mechanism can accommodate any procedure that encodes a pairwise directional association between the corresponding time series, such as lead-lag relationships and Granger causality [17], with applications such as in the analysis of information ﬂow in brain networks [18], biology [19], ﬁnance [20, 21] and earth sciences [22].
DIGRAC could also facilitate tasks in ranking [23] and anomaly detection [24–26], as it allows one to extrapolate from local pairwise (directed) interactions to a global structure inference, in the high-dimensional low signal-tonoise ratio regime.
Main contributions.
Our main contributions are as follows.
We propose a GNN framework for self-supervised end-to-end node clustering on (possibly attributed and weighted) digraphs explicitly taking into account the directed ﬂow imbalance.
We propose a family of probabilistic global imbalance scores to serve as the self-supervised loss function and evaluation objective, including one based on hypothesis testing for directionality signal.
To the best of our knowledge, this is the ﬁrst method directly maximizing ﬂow imbalance for node clustering in digraphs using GNNs.
We extend our method to the semi-supervised setting when label information is available.
Related Work
Directed clustering has been explored by non-GNN methods.
[27] performs directed clustering that hinges on symmetrizations of the adjacency matrix, but is not scalable as it requires large matrix multiplications.
[28] proposes a spectral co-clustering algorithm for asymmetry discovery that relies on in-degree and out-degree.
Whenever direction is the sole information, such as in a complete network with a lead-lag structure derived from time series [20], a purely degree-based method cannot detect the clusters.
While [29] produces two partitions of the node set, one based on out-degree and one based on in-degree, our partition simultaneously takes both directions into account.
The directed graph Laplacians introduced by [2] are only applicable to strongly connected digraphs, which is rarely the case in sparse networks arising in applications.
InfoMap by [14] assumes that there is a “map” underlying the network, similar to a meta-graph in DIGRAC.
InfoMap aims to minimize the expected description length of a random walk and is recommended for networks where edges encode patterns of movement among nodes.
While related to DIGRAC, InfoMap still relies on some amount of density-based signal being present within each of the modules.
[12] seeks to uncover clusters characterized by a strongly imbalanced ﬂow circulating among them, based on eigenvectors of the Hermitian matrix ( A − A T ) · i, where A is the (normalized) adjacency matrix and i the imaginary unit.
[12] is a purely spectral-based method and is not able to naturally incorporate any available node features or label information; in contrast, DIGRAC is a GNN-based method that is naturally able to account for such information.
Moreover, [12] is not driven by an optimization function, but only proposes evaluation metrics that capture the imbalance of the pairs of clusters.
In contrast, inspired by [12], in DIGRAC a family of novel imbalance loss functions is proposed, with a probabilistic interpretation, rendering DIGRAC a fully trainable end-to-end pipeline.
Furthermore, the rich class of imbalance evaluation and training objectives/losses proposed in this paper go far beyond the evaluation metrics considered in [12].
[13] uncovers higher-order structural information among clusters in digraphs, while maximizing the imbalance of the edge directions, but its deﬁnition of the ﬂow ratio restricts the underlying meta-graph to a path.
GNNs have been applied to digraph node classiﬁcation, which is similar to digraph clustering but requires known clustering labels.
[30] uses ﬁrst and second-order proximity, and constructs three
Laplacians, but the method is space and speed-inefﬁcient.
[31] simpliﬁes [30], builds a directed
Laplacian based on PageRank, and aggregates information dependent on higher-order proximity.
Building on [12, 32], [33] constructs a Hermitian matrix that encodes undirected geometric structure in the magnitude of its entries, and directional information in their phase.
[34] introduces a digraph data augmentation method called Laplacian perturbation and conducts digraph contrastive learning.
[35] proposes a spectral-based graph convolution network for digraphs, yet is restricted to strongly connected digraphs that are usually not realistic.
[36] utilizes convolution-like anisotropic ﬁlters based on local subgraph structures (motifs) for semi-supervised node classiﬁcation tasks in digraphs, but relies on pre-deﬁned structures and fails to handle complex networks.
In particular, [30, 31, 33, 34, 36] all require known labels, which are not generally available for real-world data.
[2, 12, 13, 27, 28] could not trivially incorporate node attributes or node labels.
In contrast, we propose an efﬁcient GNN-based method that maximizes a probablistic ﬂow imbalance objective, in a self-supervised manner, and which can naturally analyze attributed weighted digraphs.
To avoid potential misunderstanding, we brieﬂy mention several related works that we are aware of but do not compare against in our experiments in the main text.
While DIGRAC addresses the task of partitioning the nodes into disjoint sets, [37] locates a certain community within a network.
In particular, [37] proposes a local algorithm while this paper proposes a global one.
OSLOM by [38] is very ﬂexible but based on a density heuristic and hence a comparison to DIGRAC on networks without density signal would not be fair, to begin with.
[39] introduces directionality in the Louvain algorithm.
This algorithm optimizes a modularity-type function that compares the number of edges within communities to the expected number of edges under a speciﬁed model.
It is thus an approach that aims to ﬁnd denser-than-expected groups of vertices.
When all groups have the same density, as in our synthetic data sets, and the only structure lies in the directionality of the edges, this method simply cannot be expected to perform well.
The Leiden algorithm in [40] also builds on the Louvain method, again optimizing a modularity-type function that compares the number of edges within communities to the expected number of edges under a speciﬁed model.
It is a powerful method for that task, but cannot be fairly compared to DIGRAC which is tailored to ﬁnd imbalances.
As conﬁrmed by our experiments in Appendix (App.)
E, comparing these methods to DIGRAC is not appropriate.
We also do not compare DIGRAC against graph pooling methods [41], which are inspired by pooling in CNNs and developed to discard information that is superﬂuous for the task at hand, as a partition of the nodes which can be interpreted as clustering is only a byproduct.
Moreover, graph pooling methods are usually developed only for undirected networks.
While graph matching as in [42–44] and [45] can be viewed as a clustering method of networks, matching the graph of interest to a disconnected graph by connecting each node in the observed graph with an isolated node of the disconnected graph, this approach is not developed for directed networks.
The underlying idea of these papers is complementary to the meta-graph idea which underpins DIGRAC; in the meta-graph, the components are connected, and estimating the directionality of these connections is the main focus.
Hence this work addresses a very different task.
We emphasize that these are all excellent methods, but they address different objectives and tasks.
DIGRAC is tailored to detect an imbalance signal in directed networks, and such a signal cannot be present in an undirected network.
As it is based on imbalance, DIGRAC will not be able to detect a signal in an undirected network, thus rendering it not applicable to undirected networks.
The DIGRAC Framework
Problem deﬁnition.
Denote a (possibly weighted) digraph with node attributes as G =
( V , E , w, X ) , with V the set of nodes, E the set of directed edges or links, and w ∈ [0 , ∞ ) |E| the set of edge weights.
G may have self-loops, but no multiple edges.
The number of nodes is n = |V| , and X ∈ R n × d in is a matrix whose rows encode the nodes’ attributes.
Such a network can be represented by the attribute matrix X and the adjacency matrix A
=
( A ij ) i,j ∈V , with A ij =
if no edge exists from v i to v j ; if there is an edge e from v i to v j , we set A ij = w e , the edge weight.
Digraphs often lend themselves to interpreting weighted directed edges as ﬂows, with a meta-graph on clusters of vertices describing the overall ﬂow directions; see Fig.
A clustering is a partition of the set of nodes into K disjoint sets (clusters) V = C 0 ∪ C
[Equation]
∪ · ·
[Equation]
∪ C K − 1 (ideally, K ≥ 2 ).
Intuitively, nodes within a cluster should be similar to each other with respect to ﬂow directions, while nodes across clusters should be dissimilar.
In a self-supervised setting, only the number of clusters K is given.
In a semi-supervised setting, for each of the K clusters, a fraction set V seed ⊆
V train ⊂ V of the set
V train of all training nodes is selected to serve as the set of seed nodes, for which the cluster membership labels are known before training.
The goal of semi-supervised clustering is to assign each node v ∈ V to a cluster containing some known seed nodes, without knowledge of the underlying ﬂow meta-graph.
The corresponding self-supervised clustering task does not use seed nodes.
3.1 Self-Supervised Loss for Clustering
Our self-supervised loss function is inspired by [12], aiming to cluster the nodes by maximizing a normalized form of cut imbalance across clusters.
We ﬁrst deﬁne probabilistic versions of cuts, imbalance ﬂows, and probabilistic volumes.
For K clusters, the assignment probability matrix
P ∈ R n × K has as row
i the probability vector P ( i , :)
∈ R K with entries denoting the probabilities of each node to belong to each cluster; its k th column is denoted by P (: ,k ) .
•∀
k, l ∈ { 0 , . . .
, K − 1 } where K ≥ 2 , the probabilistic cut from cluster C k to C l is deﬁned as
[Equation]
• The imbalance ﬂow between C k and C l is deﬁned as | W ( C k , C l
[Equation]
− W
[Equation]
C l , C k
[Equation]
|
[Equation]
For interpretability and ease of comparison, we normalize the imbalance ﬂows
to obtain an imbalance score with values in
as
follows (we defer additional details to App.
B.
• The probabilistic volume for cluster C k is deﬁned as
[Equation]
Then V OL ( C k ) ≥ W ( C k , C l ) for all l = 1 , . . .
, K − 1 and
[Equation]
The imbalance term, which is used in most of our experiments, denoted CI vol_sum , is deﬁned as
[Equation]
In particular, for K = n , every node is a single cluster, and CI vol_sum ( k, l )
=1, but then the partition is not informative.
The aim is to ﬁnd a partition that maximizes the imbalance ﬂow under the constraint that the partition has at least two sets, to capture groups of nodes that could be viewed as representing clusters in the meta-graph.
The normalization by the volumes penalizes partitions that put most nodes into a single cluster.
The range [0 , 1] follows from Eq.
Other variants are discussed in App.
B.3.
To obtain a global probabilistic imbalance score , based on CI vol_sum from Eq.
(2), we average over pairwise imbalance scores of different pairs of clusters.
Since the scores discussed are symmetric and the cut difference before taking absolute value is skew-symmetric, we only need to consider the pairs in the set T =
{ ( C k , C l ) : 0 ≤ k < l ≤ K − 1 , k, l ∈ Z } .
A naive approach, which we call the “ naive " variant, considers all possible (cid:0) K 2 (cid:1) pairwise cut imbalance values.
However, due to potentially high noise levels in certain data sets, one may only be interested in pairs that are not just noise but exhibit true signals.
To this end, we introduce a “ std " variant, which only considers pairwise cut imbalance values that are 3 standard deviations away from the observed purely noisy imbalance values; the standard deviation is calculated under the null hypothesis that the between-cluster relationship has no direction preference, i.e. F k,l = F l,k
(entries of the meta-graph adjacency matrix F to be introduced later in this section), as follows.
Suppose two clusters C k and C l have only noisy links between them, with no edge in the meta-graph F , i.e. F kl = 0 .
Assume also that the underlying network is ﬁxed in terms of the number of nodes and locations of edges; the only randomness stems from the direction of the edges.
Then we can provide the following theoretical guarantee.
Proposition 1.
Suppose that C k and C l are two clusters of n k and n l nodes, respectively, with m ( k, l ) edges between them, edge weights w ij = w ji ∈ [0 , 1] and edge direction drawn independently at random with equal probability 12 for each direction.
We assume that the edge weights satisfy max e | w e | ( (cid:80) e w 2 e ) − 12 =
o ( m ( k, l )) .
Then W ( C k , C l ) − W ( C l , C k ) is approximately normally distributed with mean 0 and variance ||
w
|| 2 as m ( k, l ) → ∞ .
A consequence of Proposition 1, which is proved in App.
B.1, is that under its assumptions, approximately 99.7 % of the observations fall within 3 standard deviations from 0.
While Proposition 1 makes many assumptions and ignores reciprocal edges, the resulting threshold is still a useful guideline for restricting attention to pairwise imbalance values which are very likely to capture a true signal.
In particular, we use it as motivation for our “std" variant to pick cluster pairs from T that satisfy ( W ( C k , C l ) − W
( C l , C k )) 2 > 9 ( W ( C k , C l ) + W ( C l , C k )) .
As we are mainly concerned about the top pairs (i.e., those exhibiting the largest imbalance ﬂow), another option is the “ sort " variant, which selects the largest β pairwise cut imbalance values, where β is half of the number of nonzero entries in the off-diagonal entries of the meta-graph adjacency matrix F , if the meta-graph is known or can be approximated.
For example, for a “cycle" meta-graph with three clusters and no ambient nodes, β = 3 .
When the meta-graph is a “path" with three clusters and ambient nodes, then β = 1 .
When considering the “sort" variant, with
T ( β ) = { ( C k , C l ) ∈ T : CI vol_sum ( k, l ) is among the top β values } , where 1 ≤ β ≤ (cid:0) K 2 (cid:1) , we set
[Equation] [Equation]
, and
[Equation]
as the corresponding loss function.
Deﬁnitions of meta-graph structures are discussed in Section 4.1.
For the other variants, the corresponding scores and loss functions are deﬁned analogously.
We apply the “std" variant when we have no prior knowledge of the meta-graph structure during training, and the “sort" variant when we have information on the number of pairs to count.
When using the “std" variant for training, for the initial 50 epochs, we apply the “sort" variant with β = 3 for a reasonable starting clustering probability matrix for training, as otherwise during the initial training epochs possibly no pairs could be picked out.
During the epochs actually utilizing this “std" variant, if no pairs could be picked out, we temporarily switch to the “naive" variant for that epoch.
Regarding complexity, the objective mainly contains matrix-vector multiplications and element-wise matrix divisions, which are at most quadratic in the number of nodes, but usually faster with our sparsity-aware implementation.
3.2 Instantiation of DIGRAC
[Figure]
K , we then
To instantiate
DIGRAC, any aggregation scheme able to take directionality into account could be incorporated into our general framework, as long as it can output the node embedding matrix Z .
Here, by default, we adapt the
Signed Mixed Path
Aggregation (SIMPA) scheme from [46].
We remove the signed parts and devise a simple yet effective directed mixed path aggregation scheme, which we call Directed
Mixed Path Aggregation (DIMPA), to obtain the probability assignment matrix P by applying a linear layer followed by a unit softmax function to the embedding generated, and feed it to the loss function.
Details of DIMPA are provided in App.
A. A framework diagram is provided in Fig.
2, and an instantiation using DIMPA is visualized in Fig.
Experiments
In our synthetic experiments, when by design ground truth is available, performance is assessed by the Adjusted Rand Index (ARI) [47].
Normalized Mutual Information (NMI) results give almost the same ranking for the best-performing methods as the ARI, with an average Kendall tau value of 83.8% and standard deviation 24.9%, for pairwise ranking comparison, on the methods compared in our experiments.
We do not focus on NMI in the main text due to its shortcomings [48], see also
App. C.9.
Clustering tasks will have different ground truths, depending on the pattern they are trying to detect.
Many network clustering methods focus on detecting relatively dense clusters, and try to optimize classical network clustering measures, such as directed modularity or partition density.
Ground truth for these clustering algorithms then relates to relatively densely connected subgroups in the data.
DIGRAC is a novel method that addresses a novel task, namely that of detecting ﬂow imbalances.
To the best of our knowledge, real-world data sets with ground-truth ﬂow imbalances are not available to date, and hence we introduce normalized imbalance scores to evaluate clustering performance based on ﬂow imbalance.
As ARI and NMI require ground-truth labels, they thus cannot be applied to the available real-world data sets.
To address this shortcoming, for the real-world data sets, in Table 1, we include three performance measures which we introduce in the paper, and the appendix contains an additional 11 performance measures.
Implementation details are provided in
App. C. Codes and preprocessed data are available at https://github.com/SherylHYX/DIGRAC_
Directed_Clustering and have been included in the open-source library PyTorch Geometric
Signed Directed [49].
We compare DIGRAC against the most recent related methods from the literature for clustering digraphs.
The 10 methods are • (1) InfoMap [14], • (2) Bibliometric and • (3) Degree-discounted introduced in [27], • (4) DI_SIM [28], • (5) Herm and • (6) Herm_sym introduced in [12], • (7) MagNet [33], • (8) DGCN [30], • (9) DiGCN [31], and • (10) DiGCL [34].
The abbreviations of these methods, when reported in the numerical experiments, are InfoMap, Bi_sym, DD_sym, DISG_LR,
Herm, Herm_sym, MagNet, DGCN, DiGCN, DiGCL, respectively.
DGCN is the least efﬁcient method in terms of speed and space complexity, followed by DiGCN which involves the so-called inception blocks .
We use the same hyperparameter settings stated in these papers.
Methods (7), (8), (9), (10) are GNN methods which are trained with 80% nodes under label supervision, while all the other methods are trained without label supervision.
DIGRAC further restricts itself to be trained on the subgraph induced by only the training nodes.
All methods are designed for directed graphs, and all except Infomap require K to be known.
Runtime comparison is provided in App.
C.2, illustrating that DIGRAC is among the fastest among competing GNNs.
Implementation details for competitors are provided in App.
C.7.
4.1 Data Sets Synthetic data: Directed Stochastic Block Models A standard directed stochastic block model
(DSBM) is often used to represent a network cluster structure, see for example [1].
Its parameters are the number K of clusters and the edge probabilities; given the cluster assignment of the nodes, the edge indicators are independent.
The DSBMs used in our experiments also depend on a meta-graph adjacency matrix F = ( F k,l ) k,l =0 ,...,K − 1 and a ﬁlled version of it, ˜F =
( ˜F k,l ) k,l =0 ,...,K − 1 , and on a noise level parameter η ≤ 0 .
The meta-graph adjacency matrix F is generated from the given meta-graph structure, called M .
To include an ambient background, the ﬁlled meta-graph adjacency matrix ˜F replaces every zero in F that is not part of the imbalance structure by 0.5.
The ﬁlled meta-graph thus creates a number of ambient nodes which correspond to entries which are not part of
M and thus are not part of a meaningful cluster; this set of ambient nodes is also called the ambient cluster .
First, we provide examples of structures of F without any ambient nodes, where 1 denotes the indicator function.
• (1) “ cycle ": F k,l = (1 − η ) 1 ( l = (( k + 1) mod K ))
+ η 1 ( l = (( k − 1) mod K ))
+ 12 1 ( l = k ) .
• (2) “ path ": F k,l = (1 − η ) 1 ( l = k + 1) + η 1 ( l = k − 1) + 12 1 ( l = k ) .
• (3) “ complete ": assign diagonal entries 12 .
For each pair ( k, l ) with k < l, let F k,l be η and 1 − η
with equal probability, then assign F l,k = 1 − F k,l .
(4) “ star "
, following [50]: select the center node as ω = (cid:98) K −
[Equation]
and set F k,l = (1 − η
[Equation]
k
[Equation]
ω,
[Equation]
When ambient nodes are present, the construction involves two steps, with the ﬁrst step the same as the above, but with the following changes: For “ cycle " meta-graph structure, F k,l = (1 − η ) 1 ( l = (( k +1) mod ( K − 1)))
+ η 1 ( l = (( k − 1) mod
( K − 1)))
+ 0 .
5 1 ( l = k ) .
The second step is to assign 0 (0.5, resp.) to the last row and the last column of F ( ˜F , resp.).
Figures 1(c-d) display a “ cycle " meta-graph structure with ambient nodes (in cluster 4).
The majority of edges ﬂow in the form 0 → 1 → 2 → 3 → 0 , while few ﬂow from the opposite direction.
Fig.
1(d) illustrates the meta-graph adjacency matrix corresponding to this F .
In our experiments, we choose the number of clusters, the (approximate) ratio, ρ , between the largest and the smallest cluster size, and the number, n , of nodes.
To tackle the hardest clustering task and also focus on directionality, all pairs of nodes within a cluster and all pairs of nodes between clusters have the same edge probability, p .
Note that for M = “ cycle ", even the expected in-degree and out-degree of all nodes are identical.
Our DSBM, which we denote by DSBM ( M , 1 ( ambient ) , n, K, p, ρ, η ), is built similarly to [12] but with possibly unequal cluster sizes, with more details in App.
C.3.
For each node v i ∈ C k , and each node v j ∈ C l , independently sample an edge from node v i to node v j with probability p · ˜ F k,l .
The parameter settings in our experiments are p ∈ { 0 . 001 , 0 . 01 , 0 . 02 , 0 . 1 } ,
ρ ∈ { 1 , 1 . 5 } , K ∈ { 3 , 5 , 10 } , 1 ( ambient ) ∈ { T, F} (True and False), n ∈ { 1000 , 5000 , 30000 } , and we also vary the direction ﬂip probability η from 0 to 0.45, with a 0.05 step size.
4 η 0 .0 .0 .0 .
[Table] [Table]
I
[Table] [Table]
[Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Figure]
Real-world data We perform experiments on ﬁve real-world digraph data sets with sizes ranging from 245 to over 2 million nodes: Telegram [3], Blog [51], Migration [4], WikiTalk [52], and LeadLag [20], with details in App.
C.3.
We set the number of clusters K to be 4, 2, 10, 10, 10, respectively, and values of β to be 5, 1, 9, 10, 3, respectively.
Note that Lead-Lag comprises of 19 separate networks constructed from yearly ﬁnancial time series, rendering a total of 23 real-world networks.
4.2 Experimental results 4.2.1 Training Set-Up
As training setup, we use 10% of all nodes from each cluster as test nodes, 10% as validation nodes to select the model, and the remaining 80% as training nodes.
In each setting, unless otherwise stated, we carry out 10 experiments with different data splits.
Error bars are given by one standard error.
[Table]
on DSBM(“complete", F, n = 1000 , K = 3 , p = 0 . 1 , ρ = 1 ).
4 η 0 .0 .0 .0 .0 .1 .
A R
I
DSBM N =1000.
p =0.1.
K =5.
ρ =1.5.ambient=
True.
Fstyle=complete.
CICELICE (b) Vary loss for DiGCN on DSBM( “complete", T, n = 1000 , K = 5 , p = 0 .
1 , ρ = 1 . 5 ).
2 η 0 .0 .0 .0 .0 .0 .
A R I sort std naive (c) Vary selection methods of imbalancepairs.
3 η 0 .0 .0 .0 .
A R I 0 0.1 0.
(d) Vary seed ratio in a semi-supervised setting.
When no node attributes are given, the matrix X for DIGRAC is taken as the stacked eigenvectors corresponding to the largest K eigenvalues of the random-walk symmetrized Hermitian matrix used in the comparison method Herm_rw.
The imbalance loss function acts on the subgraph induced by the training nodes.
To further clarify the training setup, DIGRAC uses 0% of the labels in training.
As
DIGRAC is a self-supervised method, in principle, we could use all nodes for training.
However for a fair comparison with other GNN methods we use only 80% of the nodes for training.
For supervised methods our split of 80% - 10% - 10% is a standard split.
For the non-GNN methods, all nodes are used for training.
The default loss function for DIGRAC is L sortvol_sum .
[Figure]
.2.2 Results on Synthetic Data
Fig.
3 compares the numerical performance of DIGRAC with other methods on synthetic data.
For this Fig.
we generate 5 DSBM networks under each parameter setting and use 10 different data splits for each network, then average over the 50 runs.
Error bars are given by one standard error.
App.
C provides additional implementation details.
We conclude that DIGRAC compares favorably against state-of-the-art methods, on a wide range of network densities and noise levels, on different network sizes, and with different underlying meta-graph structures, with and without ambient nodes.
Being a self-supervised method, DIGRAC even attains comparable or better performance than fully-supervised GNN competitors.
4.2.3 Results on Real-World Data
For our real-world data sets, the node in- and out-degrees may not be identical across clusters.
Moreover, as these data sets do not contain node attributes, DIGRAC considers the eigenvectors corresponding to the largest K eigenvalues of the Hermitian matrix from [12] to construct an input feature matrix.
Table 1 reveals that DIGRAC provides competitive global imbalance scores in three objectives discussed and across all real-world data sets, and outperforms all other methods in 13 out of 15 instances, while attains the second-best performance for the remaining two instances.
The N/A entries for WikiTalk are caused by memory error, and the N/A entries for InfoMap on Migration and
Lead-Lag are due to its prediction of only one single cluster.
For Migration , as detailed in Fig. 1(b) and App.
D.4, DIGRAC is able to uncover nontrivial migration patterns, such as migration from
California to Arizona, as discovered by [4].
Lead-Lag results in each year are averaged over ten runs, while the mean and standard deviation values are calculated with respect to the 19 years.
The experiments indicate that edge directionality contains an important signal that DIGRAC is able to capture.
As App.
D.2 illustrates, DIGRAC is able to provide comparable or higher pairwise imbalance scores for the leading pairs.
The ﬁtted meta-graph plots in App.
D.3 reveal that DIGRAC is able to recover a directed ﬂow imbalance between clusters in all of the selected data sets.
A comprehensive numerical comparison in App.
D reveals similar conclusions.
4.3 Ablation Study
Figures 4(a-b) compare the performance of DiGCN replacing the loss function by L sortvol_sum from
Eq.
(3), indicated by “CI” (self-supervised loss only), or “LICE" (sum of supervised and selfsupervised loss), on two synthetic models.
We ﬁnd that replacing the supervised loss function with
L sortvol_sum leads to comparable results, and that adding L sortvol_sum to the loss could be beneﬁcial, indicating that the imbalance objectives are more general than only applicable to DIMPA.
Fig.
4(c) compares the test ARI performance using three variants of loss functions on the same digraph.
The current choice “ sort " performs best among these variants, indicating a beneﬁt in only considering top pairs of individual imbalance scores.
The “std" variant is comparable with the “sort" variant, but the “sort" variant performs the best with prior knowledge on the network structure.
More details on loss functions, comparison with other variants, and evaluation on additional metrics are discussed in
App. B, with similar conclusions.
As illustrated in Fig. 4(d), again on the same digraph, we also experiment on adding seeds, with the seed ratio deﬁned as the ratio of the number of seed nodes to the number of training nodes.
A supervised loss, following [46], is then applied to these seeds;
App.
C.5 contains additional details.
In conclusion, seed nodes with a supervised loss function enhance performance, and we infer that our model can further boost its performance when additional label information is available.
Conclusion, Limitations and Outlook
DIGRAC provides an end-to-end pipeline to create node embeddings and perform directed clustering, with or without available additional node features or cluster labels.
We illustrate DIGRAC on publicly available data without any personally identiﬁable information.
DIGRAC could potentially have societal impact, for example, in detecting clusters of fake accounts in social networks.
While we do not envision our work to have any negative societal impact, vigilance is of course required.
Current limitations that could be addressed by future work include detecting the number of clusters [10, 53], instead of specifying it a-priori, as this is typically not available in real-world applications.
The relatively small sizes of the networks used in the paper (the largest has 2 million nodes) also opens future direction in adapting our pipeline to extremely large networks, possibly combined with sampling methods or mini-batch [54], rendering DIGRAC applicable to large scale industrial applications.
We also intent to further explore the effect of normalization terms in our objectives, and to design more powerful objectives that could explicitly account for varying edge density.
Another future direction pertains to additional experiments in the semi-supervised setting, when there exist seed nodes with known cluster labels, or when additional information is available in the form of must-link and cannot-link constraints, popular in the constrained clustering literature [55, 56].
Further research directions will also address the performance in the sparse regime, where spectral methods are known to underperform, and various regularizations have been proven to be effective theoretically and empirically; e.g., see regularization in the sparse regime for the undirected settings [57–59].
Author
Contributions
Y.H.: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Software, Visualization, Writing – original draft; G.R.: Conceptualization, Formal analysis, Funding acquisition, Methodology, Project administration, Supervision,
Validation, Writing – review & editing; M
C.: Conceptualization, Data curation, Formal analysis,
Funding acquisition, Methodology, Project administration, Resources, Supervision, Visualization,
Writing – review & editing.
Acknowledgements
Y.H. is supported by a Clarendon scholarship.
G.R. is supported in part by EPSRC grants EP/T018445/1, EP/W037211/1 and EP/R018472/1.
M.C. acknowledges support from the EPSRC grants EP/N510129/1 and EP/W037211/1 at The Alan Turing Institute.
Appendix
A
Directed Mixed Path Aggregation
DIMPA
B Loss and Objectives
B.1 Proof of Proposition 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
B.2 Additional Details on Probabilistic Cut and Volume . . . . . . . . . . . . . . . . .
B.3 Variants of Normalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
B.4 Selection of the Loss Function
C Implementation Details
C.1 Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
C.2 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
C.3 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
C.3.
Data Splits and Preprocessing . . . . . . . . . . . . . . . . . . . . . . . .
C.3.
Synthetic Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
C.3.
Real-World Data . .
C.4 Hyperparameter Selection for DIMPA . . . . . . . . . . . . . . . . . . . . . . . .
C.5 Use of Seed Nodes in a Semi-Supervised Manner . . . . . . . . . . . . . . . . . .
C.5.
Supervised Loss . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
C.5.
Overall Objective Function . . .
C.6 Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Implementation Details
the Comparison Methods
C.8 Enlarged Synthetic Result Figures . . . . . . . . . . . . . . . . . . . . . . . . . .
C.9 NMI Results Example and Reasons against Using NMI . . . . . . . . . . . . . . .
D Additional Results on Real-World Data
D.1 Extended Result Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
D.2 Ranked Pairwise Imbalance Scores . . . . . . . . . . . . . . . . . . . . . . . . . .
D.3 Predicted Meta-Graph Flow Matrix Plots . . . . . . . . . . . . . . . . . . . . . . .
D.4 Migration Plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
D.
Coping with Outliers . .
E Discussion of Related Methods that are not Compared against in the Main Text
A Directed Mixed Path Aggregation (DIMPA)
To instantiate DIGRAC, we can employ any digraph aggregator that could generate the probability matrix P .
In this paper, we devise a simple yet effective directed mixed path aggregation scheme, to obtain the probability assignment matrix P and feed it to the loss function, as a special case of the successful SSSNET method introduced by [46].
Thus, in order to build node embeddings, we capture local network information by taking a weighted average of information from neighbors within h hops.
To this end, we row-normalize the adjacency matrix, A , to obtain A s .
Similar to the regularization discussed in [60], we add a weighted self-loop to each node and normalize by setting A s = ( ˜D s ) − 1 ˜A s , where ˜A s =
A + τ I , with ˜D s the diagonal matrix with entries ˜D s ( i, i ) =
(cid:80)
j ˜A s ( i, j ) , and τ is a small value; we take τ = 0 . 5 ; see Section C.4 for details.
The h -hop source matrix is given by ( A s ) h .
We denote the set of up-to-
h -hop source neighborhood matrices as A s,h
= { I , A s , . . .
, ( A s ) h } .
Similarly, for aggregating information when each node is viewed as a target node of a link, we carry out the same procedure for A T which is the transpose of
A .
We denote the set of up-to-
h -hop target neighborhood matrices as A t,h = { I , A t , . . .
, ( A t ) h } , where A t is the row-normalized target adjacency matrix calculated from A T .
As convention, the superscript s stands for source and the superscript t stands for target .
Next, we deﬁne two feature mapping functions for source and target embeddings, respectively.
Assume that for each node in V , a vector of features is available, and summarize these features in the input feature matrix X .
The source embedding is given by
[Equation]
where for each M , ω s M is a learnable scalar, d is the dimension of this embedding, and H s =
MLP ( s,l ) ( X ) .
Here, the hyperparameter l controls the number of layers in the multilayer perceptron (MLP) with ReLU activation; we ﬁx l = 2 throughout.
Each layer of the MLP has the same number d of hidden units.
The target embedding Z t is deﬁned similarly, with s replaced by t in
Eq.
Different parameters for the MLPs for different embeddings are possible.
After these two decoupled aggregations, we concatenate the embeddings to obtain the ﬁnal node embedding as a n × (2 d ) matrix Z =
CONCAT
( Z s , Z t ) .
The embedding vector z i for a node v
i is the i th row of Z , z i :=
[Equation]
After obtaining the embedding matrix Z , we apply a linear layer (an afﬁne transformation) to Z , so that the resulting matrix has K columns.
Next, we apply the unit softmax function to the rows and obtain the assignment probability matrix P .
Fig.
5 gives an overview of this implementation.
To avoid computationally expensive and space unfriendly matrix operations, as described in Eq. 4,
DIGRAC uses an efﬁcient sparsity-aware implementation, described in Algorithm 1, without explicitly calculating the sets of powers
A s,h and A t,h .
We omit the subscript V for ease of notation.
The
𝑿 ∈ℝ 𝑛×𝑑 𝑖𝑛
MLP
MLP 𝑯 𝒔 𝑯 𝑡
DIMPA ഥ𝑨 𝑠 ഥ𝑨 𝑡
Linear Softmax Argmax 𝒁 s 𝒁 𝒁 t CONCAT 𝑨 ∈
ℝ 𝑛×𝑛 h ∈ℤ +
Embedding
Probability Matrix
Cluster Assignment
Network embedding etc.
ARI, predicted labels etc.
Pairwise/global cut imbalance scores etc.
, we ﬁrst compute the normalized adjacency
A s and A t .
Then, we apply two separate MLPs on X , to obtain hidden representations H s and H t .
Next, we compute their decoupled embeddings using Eq.
(4), and its equivalent for target embeddings.
The concatenated decoupled embeddings are the ﬁnal embeddings.
For node clustering tasks, we add a linear layer followed by a unit softmax to obtain the probability matrix P .
Applying argmax on each row of P yields node cluster assignments.
algorithm is efﬁcient in the sense that it takes sparse matrices as input, and never explicitly computes a multiplication of two n × n matrices.
Therefore, for input feature dimension d in and hidden dimension d , if d (cid:48)
=
max( d in , d ) (cid:28) n, time and space complexity of DIMPA, and implicitly DIGRAC, is
O ( |E| d (cid:48)
h 2 + 2 nd (cid:48) K ) and O (2 |E| + 4 nd (cid:48) + nK ) , respectively [61, 62].
While it is a current shortcoming of DIGRAC that it does not scale well to very large networks, this limitation is shared by all the GNN competitors compared against in the paper, and some of the spectral methods.
DIGRAC scales well in the sense that when the underlying network is sparse, the sparsity is preserved throughout the pipeline.
In contrast, Bi_sym and DD_sym [27] construct derived dense matrices for manipulation, rendering the methods no longer scalable.
These methods resulted in N/A values in Table 1 in the main text.
For large-scale networks, DIMPA is amenable to a minibatch version using neighborhood sampling, similar to the minibatch forward propagation algorithm in [54, 63].
We are also aware of a framework [64] for scaling up graph neural networks automatically, where theoretical guarantees are provided, and ideas there will be exploited in future.
We expect that the theoretical guarantees could be adapted to our situation.
Algorithm 1:
Weighted Multi-Hop Neighbor Aggregation (DIMPA).
Input : (Sparse) row-normalized adjacency matrices A s , A t ; initial hidden representations
H s , H
; hop h ( h ≥ 2) ; lists of scalar weights Ω
[Equation]
Output : Vector representations z i for all v i ∈ V given by Z .
[Equation]
Z s ← Ω s [
[Equation]
· H
[Equation]
Ω
[Equation] [Equation]
for i ← 2 to h do ˜ X
s ← A s ˜ X s ; ˜ X
[Equation]
Z s ← Z s + Ω s [ i ] · ˜ X s ;
Z
[Equation]
←
[Equation]
Ω
[Equation]
· ˜ X t ; end
Z =
CONCAT
( Z s , Z t ) ;
B Loss and Objectives B.1 Proof of Proposition
Moreover, we clarify that we make an assumption on the limiting behavior of the weights, namely that
[Equation]
where m ( k, l ) is the number of edges.
This is a natural assumption:
In the case that all weights are equal in absolute value, this assumption is satisﬁed as then max e | w e | √ (cid:80) e w 2 e = 1 √ m ( k,l ) .
The assumption is generally satisﬁed when there is not too much variability in the weights.
If for example all but one weight pair was equal to 0, then the assumption would be violated, and also a normal approximation would not hold as there would only be two non-zero observations.
Proposition 2.
Suppose that C k and C l are two clusters of n k and n l nodes, respectively, with m ( k, l ) edges between them, with symmetric edge weights w ij = w ji ∈ [0 , 1] and with edge direction drawn independently at random with equal probability 12 for each direction.
We assume that the edge weights satisfy max e | w e | √
[Equation]
e w 2 e
=
o ( m ( k, l )) .
Then W ( C k , C l ) − W ( C l , C k ) is approximately normally distributed with mean 0 and variance ||
w
|| 2 as m ( k, l ) → ∞ .
Proof.
For each edge between the two clusters C k and C l , the edge direction is random, i.e. the edge is from C k to C l with probability 0.5, and C l to C k with probability 0.5 also.
Let E k,l denote the set of m ( k, l ) > 0 edges between
C k and C l .
For every edge e ∈ E k,l , the edge direction is encoded by a Rademacher random variable X e with X e = 1 if the edge is from C k to C l , and X e = − 1 otherwise.
Then ( X e + 1) / 2 ∼ Ber (0 . 5) is a Bernoulli(0.5) random variable with mean 2 × 0 .
5 − 1 = 0 and variance 2 2 × 0 .
5 × (1 − 0 .
= 1 .
We have the representation
[Equation]
as the sum of m ( k, l ) independent bounded random variables with ﬁnite third moments.
Moreover,
W ( C k , C l ) − W ( C l , C k ) has mean 0 and variance ||
w
|| 2 .
The assertion now follows from a version of the Central Limit Theorem, Theorem 3.4 in [65]; we repeat the relevant part here:
Theorem 1 (Extract from Theorem 3.4 in [65]) .
Let ξ 1 , . . .
, ξ n be independent random variables with zero means satisfying (cid:80) ni
=1 Var ( ξ i )
= 1 and assume that there is a δ > 0 such that | ξ i | ≤ δ for 1 ≤ i ≤ n. Let Φ denote the cumulative distribution function of the standard normal distribution.
Then
[Equation]
We apply this theorem with n replaced by m ( k, l ) , the number of edges, and take ξ e =
[Equation]
Then ξ e has mean zero and, using an enumeration of the edges, (cid:80) m ( k,l ) e =
Var ( ξ e ) = 1 .
Moreover, | ξ e | ≤ max e | w e | √ (cid:80) e w 2 e =: δ holds for all e ∈ { 1 , . . .
, m ( k, l ) } and hence the theorem applies for the limit m ( k, l ) → ∞ .
The stated result follows from using that if Z/σ has the standard normal distribution then Z has the mean zero normal distribution with variance σ 2 .
(cid:3)
B.2 Additional Details on Probabilistic Cut and Volume
Recall that the probabilistic cut from cluster C k to C l is deﬁned as
[Equation]
where P (: ,k ) , P (: ,l ) denote the k th and l th columns of the assignment probability matrix P , respectively.
The imbalance ﬂow between clusters C k and C l is deﬁned as
[Equation]
for k, l ∈ { 0 , . . .
, K − 1 } .
The loss functions proposed in the main paper can be understood in terms of a probabilistic notion of degrees, as follows.
We deﬁne the probabilistic out-degree of node v i with respect to cluster k by ˜ d (out) i,k = (cid:80) nj
=1 A i,j · P j,k = ( AP (: ,k ) ) i , where subscript i refers to the i th entry of the vector AP (: ,k ) .
Similarly, we deﬁne the probabilistic in-degree of node v i with respect to cluster k by ˜ d (in) i,k =
( A T P (: ,k ) ) i , where A T is the transpose of A .
The probabilistic degree of node v i with respect to cluster k is ˜ d
i,k = ˜
d (in) i,k + ˜ d (out) i,k = (( A T + A ) P (: ,k ) )
i =
(cid:80) nj
=
( A i,j + A j,i ) · P j,k .
For comparisons and ease of interpretation, it is advantageous to normalize the imbalance ﬂow between clusters; for this purpose, we introduce the probabilistic volume of a cluster, as follows.
The probabilistic out-volume for cluster C k is deﬁned as V OL (out) ( C k ) =
(cid:80) i,j A j,i · P j,k , and the probabilistic in-volume for cluster C k is deﬁned as V OL (in) ( C k )
( A T P (: ,k ) ) i , where A T is the transpose of A .
These volumes can be viewed as sum of probabilistic out-degrees and in-degrees, respectively; for example, V OL (in) ( C k ) =
(cid:80) ni
=1 ˜ d (in) i,k .
Then, it holds true that
[Equation]
since entries in P are probabilities, which are in [0 , 1] , and all entries of A are nonnegative.
Similarly,
V OL (in) ( C k ) ≥ W
( C l , C k ) .
The probabilistic volume for cluster C k is deﬁned as
[Equation]
Then, it holds true that V OL ( C k ) ≥ W ( C k , C l ) for all l ∈ { 0 , . .
, K − 1 } and min
[Equation]
) , V OL
[Equation]
W
[Equation]
k
[Equation]
C
[Equation]
W
[Equation]
|
[Equation]
|
[Equation]
When there exists a strong imbalance, then | W ( C k , C l ) − W ( C l , C k ) | ≈ max( W ( C k , C l ) , W ( C l , C k )) .
As an extreme case, if P j,l = 1 for all nonnegative terms in the summations in Eq.
(5), and
V OL (in) ( C k ) = 0 , then | W ( C k , C l ) − W ( C l , C k ) | =
V OL ( C k ) .
B.3 Variants of Normalization
Recall that the imbalance term involved in most of our experiments, named CI vol_sum , is deﬁned as
[Equation]
An alternative, which does not take volumes into account, is given by
[Equation]
We call this cut ﬂow imbalance CI plain as it does not penalize extremely unbalanced cluster sizes.
To achieve balanced cluster sizes and still constrain each imbalance term to be in [0 , 1] , one solution is to multiply the imbalance ﬂow value by the minimum of V OL ( C k ) and V OL ( C l ) , and then divide by max ( k (cid:48) ,l (cid:48) )
∈T (min( V OL ( C k (cid:48) ) , V OL ( C l (cid:48) ))) , where T = { ( C k , C l ) : 0 ≤ k < l ≤ K − 1 , k, l ∈ Z } .
The reason for using T is that CI plain ( k, l ) is symmetric with respect to k and l, and CI plain ( k, l ) =whenever k = l. Note that the maximum of the minimum here equals the second largest volume among clusters.
We then obtain CI vol_min as
[Equation]
Another potential choice, denoted CI vol_max , whose normalization follows from the same reasoning as CI vol_sum , is given by
[Equation]
Note that the current CI vol_sum ( k, l ) term can be reformulated as
[Equation]
with the ﬁrst term in the decomposition corresponding to the relative ratio of inter- and intra-cluster edge density.
For our synthetic data, this term is constant as we have constant edge density across the graph.
However, for certain real-world data sets, one could also maximize this ﬁrst term by increasing the inter-cluster density while decreasing the intra-cluster density, which seems to be a side effect.
However, in our experiments, we also evaluate our results with different metrics, including objectives without any normalization, and conclude that this side effect does not create any issues in our data sets.
[Table]
B.4 Selection of the Loss Function
Table 2 provides naming conventions of all the twelve pairs of variants of objectives and loss functions used in this paper.
We select the loss functions for DIGRAC based on two representative models, and compare the performance of different loss functions.
We use DIMPA (introduced in A) as an instantiation of DIGRAC’s aggregator, for which d = 32 , hidden units, h = 2 hops, and no seed nodes.
Figures 6(a) and 7 compare twelve choices of loss combinations on a DSBM with n =nodes, K = 5 blocks, ρ = 1 , p = 0 .
02 without ambient nodes, with a complete meta-graph structure.
The subscript indicates the choice of pairwise imbalance, and the superscript indicates the variant for selecting pairs.
Figures 6(b) and 8 are based on a DSBM with n = 1000 nodes, K = 5 blocks, ρ = 1 , p = 0 .
02 without ambient nodes, with a cycle meta-graph structure.
For these ﬁgures, dash lines highlight the “ sort " variant as well as the “ std " variant based on CI vol_sum , which have been introduced in the main text.
We also plot the imbalance evolution curves for the above two synthetic models when η = 0 .
05 , for all the loss variants, in Figure 9.
These ﬁgures indicate that the “ sort " variant generally provides the best test ARI performance and the best overall global imbalance scores, among which using normalizations CI vol_sum and CI vol_max perform the best.
The “ std " variant is comparable with the “ sort " variant in many instances, but is less stable in performance.
We observe, however, from Figure 9, that the “std" variants normally converge much faster.
Taking the above into account, if we have prior knowledge of the network structure, or when we could conduct some prior analysis on the value β to take, the “ sort " variant should be the variant of choice.
Further, from Figure 9, we observe that normalization in the loss function helps avoid the degenerate situation that the loss does not decrease.
Such degeneracy can occur in the “plain” variants, raising issues about the practical usefulness of these variants.
We observe that
L sortvol_min appears to behave worse than L sortvol_sum and L sortvol_max , even when using the “sort" variant to select pairwise imbalance scores.
One possible explanation is that L sortvol_min does not penalize extreme volume sizes, and that it takes minimum as well as maximum which, as functions of the data, are not as smooth as taking a summation.
Throughout our experiments in the main text, we hence use the loss function L sortvol_sum .
C Implementation Details C.1 Code
To fully reproduce our results, code and preprocessed data are available at https://github.com/
SherylHYX/DIGRAC_Directed_Clustering .
C.2 Hardware
Experiments were conducted on a compute node with 8 Nvidia RTX 8000, 48 Intel Xeon Silver 4116 CPUs and 1000 GB RAM, a compute node with 4 NVIDIA GeForce RTX 2080, 32 Intel Xeon
E5-2690 v3 CPUs and 64 GB RAM, a compute node with 2 NVIDIA Tesla K80, 16 Intel Xeon
E5-2690 CPUs and 252 GB RAM, and an Intel 2.90GHz i7-10700 processor with 8 cores andthreads.
With this setup, all experiments for spectral methods, MagNet, DiGCL, and DIGRAC can be completed within two days, including repeated experiments, to obtain averages over multiple runs.
DGCN,
DiGCN, and MagNet have much longer run time (especially DGCN, which is space-consuming, and we cannot run many experiments in parallel), with a total of three days for them to ﬁnish.
The slow speed stems from the competitor methods; some of the other GNN methods take a long time to
[Figure] [Figure]
[Figure] [Figure]
run.
Table 1 in the main text shows N/A values for Bi_sym and for DD_sym exactly for this reason.
Empirically, DIGRAC is among the fastest among all GNN methods to which it is compared.
In detail, Table 3 reports the average runtime for all GNN methods on a variety of DSBM models, and illustrates that DIGRAC indeed takes the least or second least computational time per epoch.
The results are averaged over 10 runs for the ﬁrst 200 epochs.
DiGCL is also efﬁcient in running time, but with worse performance than DIGRAC even as a supervised method, see the enlarged synthetic results in Sec.
C.8 (Figure 13).
The total number of epochs required until the validation loss does not decrease for 200 epochs (or the maximum number of 1000 epochs is reached) varies for different data sets.
[Table]
C.3 Data C.3.1 Data Splits and Preprocessing
The results comparing DIGRAC with other methods on synthetic data are averaged over 50 runs, ﬁve synthetic networks under the same setting, each with 10 different data splits.
For synthetic data, 10% of all nodes are selected as test nodes for each cluster (the actual number is the ceiling of the total number of nodes times 0.1, to avoid falling below 10% of test nodes), 10% are selected as validation nodes (for model selection and early-stopping; again, we consider the ceiling for the actual number), while the remaining roughly 80% are selected as training nodes (the actual number can never be higher than 80% due to using the ceiling for both the test and validation splits).
To further clarify the training setup, we use 0% of the labels in training.
As DIGRAC is a self-supervised method, in principle, we could use all nodes for training.
However, for a fair comparison with other GNN methods, we use only 80% of the nodes for training.
For supervised methods our split of 80% - 10% - 10% is a standard split.
For the non-GNN methods, all nodes are used for training.
For both synthetic and real-world data sets, we extract the largest weakly connected component for experiments, as our framework could be applied to different weakly connected components, if the digraph is disconnected.
Isolated nodes do not include any imbalance information.
As customary in community detection, they are often omitted in real networks.
When “ground-truth" is given, test results are averaged over 10 different data splits on one network.
When no labels are available, results are averaged over 10 different data splits.
Averaged results are reported with error bars representing one standard deviation in the ﬁgures, and plus/minus one standard deviation in the tables.
C.3.2 Synthetic Data
Our synthetic data, DSBM, which we denote by DSBM ( M , 1 ( ambient ) , n, K, p, ρ, η ), is built similarly to [12] but with possibly unequal cluster sizes: • (1) Assign cluster sizes n 0 ≤ n 1 ≤ · · · ≤ n K − 1 with size ratio ρ ≥ 1 , as follows.
If ρ = 1 then the ﬁrst K − 1 clusters have the same size (cid:98) n/K (cid:99) and the last cluster has size n − ( K − 1) (cid:98) n/K (cid:99) .
If ρ > 1 , we set ρ 0 = ρ 1 K − 1 .
Solving (cid:80) K − 1 i =
ρ i 0 n 0 = n and taking integer value gives n 0 = (cid:4) n (1 − ρ 0 ) /
(1 − ρ K 0 ) (cid:5) .
Further, set n i = (cid:98) ρ 0 n i − 1 (cid:99) , for i = 1 , · · · , K − 2 if K ≥ 3 , and n K − 1 = n − (cid:80) K − 2 i =0 n i .
Then the ratio of the size of the largest to the smallest cluster is approximately ρ K − 1= ρ.
• (2) Assign each node randomly to one of K clusters, so that each cluster has the allocated size.
• (3) For node v i , v j ∈ C k , independently sample an edge from node v i to node v j with probability p · ˜F k,k . • (4) For each pair of different clusters C k , C l with k (cid:54)
=
l , for each node v i ∈ C k , and each node v j ∈ C l , independently sample an edge from node v i to node v j with probability p · ˜F k,l .
C.3.3 Real-World Data
For real-world data sets, we choose the number K of clusters in the meta-graph and the number β of edges between clusters in the meta-graph as follows.
As they are needed as input for DIGRAC, we resort to Herm_rw [12] as an initial view of the network clustering.
When a suitable meta-graph is suggested in a previous publication, then we use that choice.
Otherwise, the number K of clusters is determined using the clustering from Herm_rw.
First, we pick a range of K , and for each K, we calculate the global imbalance scores and plot the predicted meta-graph ﬂow matrix F (cid:48) based on the clustering from Herm_rw.
Its entries are deﬁned as
[Equation]
These entries can be viewed as predicted probabilities of edge directions.
Then, we choose K from this range so that the predicted meta-graph ﬂow matrix has the highest imbalance scores and strong imbalance in the predicted meta-graph ﬂow matrix.
The choice of β , which we assume should be equal to the number of edges in the meta-graph, is as follows.
We plot the ranked pairs of CI plain values from Herm_rw and select the β which is at least as large as K − 2 , to allow the meta-graph to be connected, and which corresponds to a large drop in the plot.
Here we provide a brief description for each of the data sets; Table 4 gives the number, n , of nodes, the number, |E| , of directed edges, the number |E r | , of reciprocal edges (self-loops are counted once and for u
(cid:54) = v , a reciprocal edge u → v, v → u is counted twice) as well as their percentage among all edges, for the real-world networks, illustrating the variability in network size and density (deﬁned as |E| /
[ n ( n − 1)] ).
• Telegram [3] is a pairwise inﬂuence network between n = 245 Telegram channels with |E| = 8 , 912 directed edges.
It is found in [3] that this network reveals a core-periphery structure in the sense of [5].
A directed core-periphery structure arises when there is a densely connected group of edges – a core – and sparsely connected groups of peripheral nodes with edges leading into the core, as well as sparsely connected groups of peripheral nodes with edges coming out of the core.
Following [3] we assume K = 4 clusters, and the core-periphery structures gives β = 5 .
• Blog [51] records |E| = 19 , 024 directed edges between n = 1 , 212 political blogs from the 2004 US presidential election.
In [51] it is found that there is an underlying structure with K = 2 clusters corresponding to the Republican and Democratic parties.
Hence we choose K = 2 and β = 1 . • Migration [4] reports the number of people that migrated between pairs of counties in the US during 1995-2000.
It involves n = 3 , 075 countries and |E| = 721 , 432 directed edges after obtaining the largest weakly connected component.
We choose K = 10 and β = 9 , following [12].
Since the original digraph has extremely large entries, to cope with these outliers, we preprocess the input network by
[Equation]
which follows the preprocessing of [12].
The results for not doing this preprocessing is provided in Table 12.
• WikiTalk [52] contains all users and discussion from the inception of Wikipedia until Jan. 2008.
The n = 2 , 388 , 953 nodes in the network represent Wikipedia users and a directed edge from node v i to node v j denotes that user i edited at least once a talk page of user j. There are |E| = 5 , 018 ,edges.
We choose K = 10 clusters among candidates { 2 , 3 , 5 , 6 , 8 , 10 } , and β = 10 .
• Lead-Lag
[20] contains yearly lead-lag matrices from 269 stocks from 2001 to 2019.
We choose K = 10 clusters based on the GICS industry sectors [66], and choose β = 3 to emphasize the top three pairs of imbalance values.
The lead-lag matrices are built from time series of daily price log returns, as detailed in [20].
The lead-lag metric for entry ( i, j ) in the network encodes a measure of the extent to which stock i leads stock j , and is obtained by applying a functional that computes the signed normalized area under the curve (auc) of the standard cross-correlation function (ccf).
The resulting matrix is skew-symmetric, and entry ( i, j ) quantiﬁes the extent to which stock i leads or lags stocks j , thus leading to a directed network interpretation.
Starting from the skew-symmetric matrix, we further convert negative entries to zero, so that the resulting digraph can be directly fed
into other methods; note that this step does not throw away any information, and is pursued only to render the representation of the digraph consistent with the format expected by all methods compared, including DIGRAC.
Note that the statistics given in Table 4 are averaged over the 19 years.
[Table]
As input features, after obtaining eigenvectors from Hermitian matrices constructed as in [12], we standardize each column vector so that it has mean zero and variance one.
We use these features for all GNN methods except MagNet, since MagNet has its own way of generating random features of dimension one.
C.4 Hyperparameter Selection for DIMPA
We conduct hyperparmeter selection via a greedy search, for DIGRAC implemented with DIMPA as its aggregator.
To explain the details, consider for example the following synthetic data setting:
DSBM with 1000 nodes, 5 clusters, ρ = 1 , and p = 0 .
02 , without ambient nodes under different hyperparameter settings.
By default, we use the loss function L sortvol_sum , d = 32 hidden units, hop h = 2 , and no seed nodes.
Instead of a grid search, we tune hyperparameters according to what performs the best in the default setting of the respective GNN method.
The procedure starts with a random setting.
For the next iteration, the hyperparameters are set to the current best setting (based on the last iteration), independently.
For example, if we start with a = 1 , b = 2 , c = 3 , and we ﬁnd that under this default setting, the best a (when ﬁxing b = 2 , c = 3 ) is 2 and the best b (when ﬁxing a = 1 , c = 3 ) is 3, and the best c is 3 (when ﬁxing a = 1 , b = 2 ), then for the next iteration, we set a = 2 , b = 3 , c = 3 .
If two settings give similar results, we choose the simpler setting, for example, the smaller hop size.
When we reach a local optimum, we stop searching.
Indeed, just a few iterations (less than ﬁve) were required for us to ﬁnd the current setting, as DIGRAC tends to be robust to most hyperparameters.
Fig.
10, 11 and 12 are plots corresponding to the same setting but for three different meta-graph structures, namely the complete meta-graph structure, the cycle structure
but with ambient nodes, and the complete structure with ambient nodes, respectively.
In theory, more hidden units give better expressive power.
To reduce complexity, we use 32 hidden units throughout, which seems to have desirable performance.
We observe that for low-noise regimes, more hidden units actually hurt performance.
We can draw a similar conclusion about the hyperparameter selection.
In terms of τ, DIGRAC seems to be robust to different choices.
Therefore, we use τ = 0 .
5 throughout.
[Figure] [Figure]
[Figure]
C.5 Use of Seed Nodes in a Semi-Supervised Manner C.5.1 Supervised Loss
For seed nodes in V seed , similar to the loss function in [46], we use as a supervised loss function the sum of a cross-entropy loss and a triplet loss.
The cross-entropy loss is given by
[Equation]
[Figure]
[Equation]
where 1 is the indicator function, C k denotes the k th cluster, and ( p i )
k denotes the k th entry of probability vector ( p i ) .
With the function L : R 2 → R given by L ( x, y ) =
[ x − y ]
+ (where the subscript + indicates taking the maximum of the expression value and 0), the triplet loss is deﬁned as
[Equation]
where S ⊆ V seed × V seed × V seed is a set of node triplets: v i is an anchor seed node, and v j is a seed node from the same cluster as the anchor, while v k is from a different cluster; and CS( z i , z j ) is the cosine similarity of the embeddings of nodes v i and v j .
We choose cosine similarity so as to avoid sensitivity to the magnitude of the embeddings.
The triplet loss is designed so that, given two seed nodes from the same cluster and one seed node from a different cluster, the respective embeddings of the pairs from different clusters should be farther away than the embedding of the pair within the same cluster.
We then consider the weighted sum L CE + γ t L triplet as the supervised part of the loss function for DIGRAC, for some parameter γ t > 0 .
The parameter γ t arises as follows.
The cosine similarity between two randomly picked vectors in d dimensions is bounded by (cid:112) ln( d ) /d with high probability.
In our experiments d = 32 , and (cid:112) ln(2 d ) / (2 d ) ≈ 0 .
In contrast, for fairly uniform clustering, the cross-entropy loss grows like log n , which in our experiments ranges between 3 and 17.
Thus some balancing of the contribution is required.
Following [46], we choose γ t = 0 .
1 in our experiments.
C.5.2 Overall Objective Function
By combining Eq.
(14), Eq.
(15), and Eq.
(3)), our objective function for semi-supervised training with known seed nodes minimizes
[Equation]
(16) where γ s , γ t > 0 are weights for the supervised part of the loss and triplet loss within the supervised part, respectively.
We set γ s = 50 as we want our model to perform well on seed nodes.
The weights could be tuned depending on how important each term is perceived to be.
C.6 Training
For all synthetic data, we train DIGRAC with a maximum of 1000 epochs, and stop training when no gain in validation performance is achieved for 200 epochs (early-stopping).
For real-world data, no “ground-truth" labels are available; we use all nodes to train and stop training when the training loss does not decrease for 200 epochs, or when we reach the maximum number of epochs, 1000.
When using the “std" variant for training, for the initial 50 epochs, we apply the “sort" variant with β = 3 for a reasonable starting clustering probability matrix for training, as otherwise during the initial training epochs possibly no pairs could be picked out.
During the epochs actually utilizing this “std" variant, if no pairs could be picked out, we temporarily switch to the “naive" variant to count all the pairs for that epoch.
For the two-layer MLP, we do not have a bias term for each layer, and we use Rectiﬁed Linear Unit (ReLU) followed by a dropout layer with 0.5 dropout probability between the two layers, following [46].
We use Adam [67] as the optimizer and (cid:96) 2 regularization with weight decay 5 · 10 − 4 to avoid overﬁtting.
We use as learning rate 0.01 throughout.
C.7 Implementation Details for the Comparison Methods
In our experiments, we compare DIGRAC against ﬁve spectral methods, InfoMap, and four GNNbased supervised methods on synthetic data, and spectral methods and InfoMap on real data.
The reason we are not able to compare DIGRAC with the other GNNs (namely, DGCN, DiGCN, MagNet, and DiGCL) on these data sets is due to the fact that these data sets do not have labels, which are required by the other GNN methods.
We use the same hyperparameter settings stated in these papers.
Data splits for all models are the same; the comparison GNNs are trained with 80% nodes under label supervision.
For MagNet, we use q = 0 .
25 for the phase matrix as in [33], because it is mentioned that q =
.lays the most emphasis on directionality, which is our main focus in this paper.
Code for MagNet is from https://github.com/matthew-hirn/magnet .
For DiGCN, we use the code from https: //github.com/flyingtango/DiGCN/blob/main/code/digcn_ib.py with option “adj_type" equals “ib".
As a recommended option in [31], we use three layers for DiGCN.
All other settings are the same as in the original paper [31].
Code for DiGCL is from https://github.com/ flyingtango/DiGCL , where we adopt the settings for Cora_ML for hyperparameters.
C.8 Enlarged Synthetic Result Figures
Figure 13 enlarges the results in the main text on synthetic data, with the same conclusions to be drawn.
C.9 NMI Results Example and Reasons against Using NMI
As NMI is an often used measure for assessing similarities between partitions, Fig.
14 provides NMI results on some synthetic models mentioned in the main text.
The results are qualitatively similar to the ARI results in Fig.
We do not use NMI in the main text to evaluate results as NMI is known to suffer from ﬁnite size effects [48, 68].
In particular NMI prefers a larger number of partitions.
Moreover it has been observed that the NMI between two independent partitions can be much larger than zero.
This feature makes NMI more difﬁcult to interpret than for example ARI.
D Additional Results on Real-World Data D.1 Extended Result Tables
Tables 5, 6, 7 and 8 provide a detailed comparison of DIGRAC with spectral methods and InfoMap.
Since no labeling information is available and all of the other competing GNN methods require labels, we do not compare DIGRAC with them on these real data sets.
In Tables 5, 6, 7 and
[Table]
we report 12 combinations of global imbalance scores by data set.
The naming convention of these imbalance scores is provided in Table 2.
To assess how balanced our recovered clusters are in terms of sizes, we also report the size ratio, which is deﬁned as the size of the largest predicted cluster to the smallest one
[Table]
and the standard deviation of sizes, size std, in order to show how varied the sizes of predicted clusters are.
For a relatively balanced clustering, we expect the latter two terms to be small
[Table] [Table]
[Table]
Tables 5, 6, 7, 8, 9, 10 and 11 reveal that DIGRAC provides competitive global imbalance scores in all of the 12 objectives introduced
[Table]
and across all the real data sets, usually outperforming all the other methods.
Among the tables, Table 11 provides results in terms of the distance to the best yearly performance, averaged across the 19 years; DIGRAC usually outperforms all the other methods across all the years.
Note that Bi_sym and DD_sym are not able to generate results for WikiTalk , as
[Table] [Table]
large n × n matrix multiplication with its transpose causes memory issue, when n = 2 , 388 , 953 .
Small values of the size ratio and size standard deviation suggest that the normalization in the loss function penalizes tiny clusters
[Table]
and that DIGRAC tends to predict balanced cluster sizes.
[Table] [Table]
[Figure] [Figure]
[Table]
C I v o l s u m r a n k e d p a
i r s
InfoMap Bi sym
DD sym DISG LR
Herm Herm rw
DIGRAC 0 .0 .0 .
C I v o l m i n r a n k e d p a
i r s     6 rank 0 .0 .1 .1 .2 .2 .3 .
C I v o l m a x r a n k e d p a
i r s     6 rank 0 .0 .0 .0 .0 .1 .
C I p
l a i n r a n k e d p a i r s
D.2 Ranked Pairwise Imbalance Scores
We also plot the ranked pairwise imbalance scores for all data sets except Blog , which has only one possible pairwise imbalance score.
For Lead-Lag , we only plot the year 2015 as an example; the plots for the other years are similar.
Figures 15, 16, 17 and 18 illustrate that DIGRAC is able to provide comparable or higher pairwise imbalance scores for the leading pairs, especially on CI vol_min pairs.
We also observe that except for CI plain , DIGRAC has a less rapid drop in pairwise imbalance scores after the ﬁrst leading pair compared to Herm and Herm_rw, which can have a few pairs with higher imbalance scores than DIGRAC.
[Figure]
i n
[Figure]
e
[Figure]
p a i r s
[Figure]
[Figure] [Figure]
D.3 Predicted Meta-Graph Flow Matrix Plots
For each data set, we plot the predicted meta-graph ﬂow matrix F (cid:48) deﬁned in Eq.
From Fig.
19, we conclude that DIGRAC is able to recover a directed ﬂow imbalance between clusters in all of the selected data sets.
Fig.
19a shows a clear cut imbalance between two clusters, possibly corresponding to the Republican and Democratic parties.
Fig.
19b plots imbalance ﬂows in the real data set Telegram , where cluster 3 is a core-transient cluster, cluster 0 is a core-sink cluster, cluster 2 is a periphery-upstream cluster, while cluster 1 is a periphery-downstream cluster [3, 5].
For
WikiTalk , illustrated in Fig.
19d, the lower-triangular part entries are typically source nodes for edges, while the upper-triangular part are target nodes.
For Lead-Lag , taking the year 2015 as an example,
DIGRAC is also able to recover high imbalance in the data.
We also note that DIGRAC would not necessarily predict the same number of clusters as assumed, so that we do not need to specify the exact number of clusters before training DIGRAC; specifying the maximum number of possible clusters sufﬁces.
0 .0 .0 .0 .(a)
Blog        0 .0 .
.(b)
Telegram          0 .0 .(c
Migration          0 .0 .0 .0 .0 .1 .(d)
WikiTalk          0 .0 .0 .0 .0 .1 .(e)
Lead-Lag
D.4 Migration Plots
We compare DIGRAC to ﬁve spectral methods for recovering clusters for the US migration data set, and plot the recovered clusters on a map, in Fig.
The visualization in Fig.
(a-c) shows that clusters align particularly well with the political and administrative boundaries of the US states, as previously observed in [69].
This outcome is not deemed too insightful, as it trivially reveals the fact there there is signiﬁcant intra-state and inter-state migration, and does not uncover any of the information on latent migration patterns between faraway states, and more generally, between regions which are not necessarily geographically cohesive.
DIGRAC outcomes, however, reveal nontrivial migration patterns, for example migration from New
York to Florida, and from California to Arizona, which is consistent with the patterns discovered by [4].
Fig.
21 details on the top pair migration patterns uncovered by DIGRAC.
D.5 Coping with Outliers
As mentioned in Section C.3, the preprocessing step to use ratio of migration instead of absolute migration numbers is a way to cope with outliers (here, extremely large entries in the original digraph) in Migration .
To validate the effectiveness of this approach to cope with outliers, Table 12 provides imbalance results for Migration when we do not transform the nonzero entries into ratios.
Comparing with Table 7, we witness an overall decrease in the performance.
In this case InfoMap no longer predicts a single huge cluster.
However, its predicted number of clusters is about 44, which is too large.
This also implies that InfoMap is very sensitive to the magnitude of digraph entries, while
DIGRAC is not.
Indeed, InfoMap gives 43 (too many) clusters for Blog , 19 (too many) for Telegram , 1 (too small) for Migration , and 17498 (far too many) for WikiTalk .
We compare DIGRAC to ﬁve spectral methods as well as InfoMap for recovering clusters for the US migration data set without the preprocessing step discussed earlier, and plot the recovered clusters on a map in Fig.
Note that all methods, except DIGRAC, recover either clusters which are trivially small in size or contain one very large dominant cluster (as in (a), (b), (e) and to some extent, also
[Figure]
(f)).
The DISG_LR clustering and InfoMap clustering provide clear geographic boundaries, but were not able to recover the imbalance among clusters.
Other spectral methods generally have a dominant cluster containing most of the nodes, whereas DIGRAC has more balanced cluster sizes.
When employing methods that symmetrize the adjacency matrix (as in (a) and (b)), the migration ﬂows between counties in different states will be lost in the process.
Furthermore, the visualization in
Fig.
(c) shows that clusters align particularly well with the political and administrative boundaries of the US states, as previously observed in [69].
The same is for Fig.
(d).
This outcome is not deemed very insightful, as it trivially reveals the fact that there is signiﬁcant intra-state and inter-state migration, and does not uncover any of the information on latent migration patterns between far-away states, and more generally, between regions which are not necessarily geographically cohesive.
Fig.
21 further plots the top three pairs of clusters based on four different imbalance scores given by
DIGRAC.
As shown in the ﬁgure, DIGRAC uncovers the migration trend from coastal to interior, across states.
This trend of the directed ﬂow agrees with that discussed in [4], with many people migrating from New York and California to the interior states.
−−−−−−
L a b e l (
[Table]
{+−−−−−− 
+}
L a b e l
[Table]
{+−−−−−− 
+}
L a b e l
[Table]
−−−−−−
L a b e l
[Table]
{+−−−−−− 
+}
L a b e l
[Table]
{+−−−−−− 
+}
L a b e l
[Table]
−−−−−−
L a b e l
[Table]
−−−−−−
L a b e l
[Table]
−−−−−−
L a b e l
[Table]
−−−−−−
L a b e l (j) CI plain : top pair −−−−−−
[-−−−−−− 
-] L a b e l (k) CI plain : 2nd pair −−−−−−
[-−−−−−− 
-]
[-L a b e l (j) CI plain : top pair −−−−−− 
-]
[-L a b e l (k) CI plain : 2nd pair −−−−−− 
-]
L a b e l (l) CI plain : 3rd pair
E Discussion of Related Methods that are not Compared against in the Main Text
To further emphasize the importance of directionality, our synthetic data sets have no difference in density between clusters; their sole signal is in the directionality of the edges.
If all edge directions were to be removed, then no algorithm should be available to detect the clusters.
To further support our claim why some methods mentioned in Section 2 in the main text are not appropriate for comparison, we have applied the default setting versions of the Louvain method [39], the Leiden algorithm [40] and OSLOM [38], to our synthetic data sets, and ﬁnd that they do not detect the structure in the data, with ARI and NMI values very close to zero, and very low imbalance values.
In particular, Louvain and Leiden tend to give a larger number of clusters than the ground truth which is designed to have small cluster sizes.
OSLOM outputs clusters with extreme sizes, either a huge cluster containing (almost always) all the nodes, or every node forming a cluster by itself.
To further demonstrate that comparing DIGRAC with density-based methods is unfair, We report the test ARI results for Infomap,
Louvain and Leiden in Figure 23.We can see that Infomap, Louvain and Leiden normally produces nero-zero ARI values, which are much worse than the results from DIGRAC given in Figure 3.
On the real-world data sets, these methods often give numbers of clusters that do not match our expectations.
( Blog has two underlying parties, Telegram has a four-cluster core-periphery structure).
Louvain clusters nodes from Blog into 8-13 clusters (too many), Telegram into 4-5 clusters
[Table]
(acceptable), Migration into 5-7 clusters (acceptable), WikiTalk into 150-219 clusters (too many)
[Table]
and
Lead-Lag into 10-55 clusters (acceptable or a bit too many).
Leiden gives 12 (too many) clusters for Blog , 4-5 for Telegram , 5-6 for Migration , 170-248 (too many) for WikiTalk , and 10-55 clusters (acceptable or a bit too many) for Lead-Lag .
OSLOM gives 6 clusters for Blog (too many), 16 for
Telegram (too many), and 46 for Migration (too many).
It could not generate results for WikiTalk after running for 12 hours, and hence we omit its discussion here.
On Lead-Lag , OSLOM places every node in a single cluster for most of the years, and clusters the rest of the years into either a huge single cluster or two clusters.
None of the methods outperform DIGRAC on our chosen performance measures from Table 1 , except on the Lead-Lag data set (See Tables 14, 15, 16 and 17 for the other results).
With regards to the 12 imbalance measures from Appendix Table 6, leaving out OSLOM as before, Louvain and
Leiden perform poorly on all of the real data sets, except on Lead-Lag .
Indeed, for Lead-Lag , the number of clusters we use for DIGRAC is ten according to the GICS sector memberships.
However, if we use the sector memberships as labels, the imbalance values are poor, which implies that ten may not be a desirable choice of the number of clusters.
Further, DIGRAC usually clusters the nodes into smaller number of clusters, while Louvain and Leiden usually cluster the nodes into a larger number of clusters (usually around 30, and sometimes above 50 clusters).
Finally, we provide more examples/explanations on why these density-based methods or even other methods that are based on random-walk should fail.
We would mainly like to point out a family of illustrative examples demonstrating the subtle nuance concerning edge density.
Consider a meta graph with K = 3 nodes (clusters) A,B,C with directed edges AB, BC, CA, hence a directed cycle (our "cycle" DSBM models).
Each pair of nodes ( v i , v j ) in the graph of size n is connected by an edge independently with probability p (which can even be equal to 1, in the case of a complete graph), hence the graph has the same density throughout.
Now suppose we consider a pair of nodes ( v i , v j ) such that v i belongs to cluster A, and v j to cluster B. Since this edge is part of the metagraph, with probability 1-eta, it is directed from v i to v j , and with probability eta, v j sends an edge to v i (here, eta is the noise level parameter).
Similar arguments can be made when v i (resp v j ) belongs to cluster B (resp C); and when v i (resp v j ) belongs to cluster C (resp A).
See Figurefor an illustration.
We also see that when the network is complete (see Figure 23 (g) and Table 9),
InfoMap [14] fails empirically as it produces a single huge cluster.
As a method based on random walks, this failure might occur as the chain could hardly be trapped inside a cluster as in the usual setting.
In such synthetic DSBM models with a "cycle" meta-graph structure, it can be shown that all nodes have the same in-degree and out-degree in expectation.
Therefore, any density-based methods or modularity-based methods should fail.
As the simplest possible example, one could just consider
K = 3 clusters as above, without any noise (thus η = 0 ).
InfoMap [14] tries to minimize the
[Figure]
Altogether, this demonstrates the robustness of DIGRAC to outliers in the data, which is not a characteristic of other state-of-the-art methods such as Herm and Herm_rw.
description length, but as no description length difference occurs in the ground-truth clustering structure for such "cycle" DSBMs, if we consider a brute-force optimization of the map equation.
Indeed, for any method that is based on a random walk, the probability of the random walker going from one cluster to another is the same as staying within the cluster.
Therefore, we could hardly optimize anything if we base our clustering structure on a random walker’s visit frequencies/path lengths.
Similarly, the Markov clustering algorithm
[70] is based on the intuition that higher-length paths would be relatively more likely to stay within clusters – an assumption that is not warranted when there is no density difference.
[15] and [16] are two interesting Markov aggregation algorithms
[Figure]
i)
[Figure]
“cycle"
[Figure]
F
[Figure]
n =
[Figure]
based on information theory and automatic control ideas that might be able to cover the above example and may inspire some further comparison, but we omit comparison to them for now as we already have more than ten comparison methods and that InfoMap shares similar ideas to these two papers.
As another example, as shown in [19], using belief propagation, in our model community structure should not be detectible (the right-hand side of (20) in [19] is zero for our "cycle" DSBMs).
Therefore, at least methods that rely on belief propagation will fail on our benchmark models.
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table]
0.22 ± 0.
0.26 ± 0.
0.28 ± 0.
O sortvol_max 0.01 ± 0.
0.16 ± 0.
0.09 ± 0.
0.00 ± 0.
0.19 ± 0.
O sortplain 1.00 ± 0.
0.87 ± 0.
0.99 ± 0.
0.98 ± 0.
1.00 ± 0.
O stdvol_sum 0.00 ± 0.
0.17 ± 0.
0.06 ± 0.
0.01 ± 0.
0.14 ± 0.
O stdvol_min 0.01 ± 0.
0.09 ± 0. 0.09 ± 0.
0.27 ± 0.
0.18 ± 0.
O stdvol_max 0.00 ± 0.
0.15 ± 0.
0.04 ± 0.
0.00 ± 0.
0.11 ± 0.
O stdplain 0.42 ± 0.
0.72 ± 0.
0.70 ± 0.
0.98 ± 0.
0.84 ± 0.
O naivevol_sum 0.00 ± 0.
0.10 ± 0.
0.04 ± 0.
0.00 ± 0.
0.12 ± 0.
O naivevol_min 0.01 ± 0.
0.06 ± 0.
0.07 ± 0.
0.26 ± 0.
0.15 ± 0.
O naivevol_max 0.00 ± 0.
0.09 ± 0.
0.03 ± 0.
0.00 ± 0.
0.09 ± 0.
O naiveplain 0.43 ± 0.
0.64 ± 0.
0.61 ± 0.
0.98 ± 0.
0.76 ± 0.
𝐶𝐶 𝐵𝐵 𝐴𝐴