Title: The Map Equation Goes Neural: Mapping Network Flows with Graph Neural Networks
Venue: NeurIPS 2024
Accepted: yes
Keywords: community detection

graph clustering

random walk

map equation
Abstract: Community detection is an essential tool for unsupervised data exploration and revealing the organisational structure of networked systems. With a long history in network science, community detection typically relies on objective functions, optimised with custom-tailored search algorithms, but often without leveraging recent advances in deep learning. Recently, first works have started incorporating such objectives into loss functions for deep graph clustering and pooling. We consider the map equation, a popular information-theoretic objective function for unsupervised community detection, and express it in differentiable tensor form for optimisation through gradient descent. Our formulation turns the map equation compatible with any neural network architecture, enables end-to-end learning, incorporates node features, and chooses the optimal number of clusters automatically, all without requiring explicit regularisation. Applied to unsupervised graph clustering tasks, we achieve competitive performance against state-of-the-art deep graph clustering baselines in synthetic and real-world datasets.

The Map Equation Goes Neural: Mapping Network Flows with Graph Neural Networks

Abstract

Community detection is an essential tool for unsupervised data exploration and revealing the organisational structure of networked systems. With a long history in network science, community detection typically relies on objective functions, optimised with custom-tailored search algorithms, but often without leveraging recent advances in deep learning. Recently, first works have started incorporating such objectives into loss functions for deep graph clustering and pooling. We consider the map equation, a popular information-theoretic objective function for unsupervised community detection, and express it in differentiable tensor form for optimisation through gradient descent. Our formulation turns the map equation compatible with any neural network architecture, enables end-to-end learning, incorporates node features, and chooses the optimal number of clusters automatically, all without requiring explicit regularisation. Applied to unsupervised graph clustering tasks, we achieve competitive performance against state-of-the-art deep graph clustering baselines in synthetic and real-world datasets.

1 Introduction

Many real-world networked systems are organised in communities: groups of nodes that are more similar to each other than to the rest. Communities provide insights into network structure at the mesoscale, revealing sub-systems by analysing link patterns. Motivated by different research questions, several characterisations of what constitutes 'good' communities have been proposed [1, 2], however, neither of them is fundamentally more correct than any other. Moreover, no single community-detection method outperforms all others on any given network [3], motivating the ongoing efforts of research on community detection. Typically, community-detection approaches formulate an objective function that calculates a quality score for a given partition of the network's nodes into communities. Finding the best partition is an NP-hard search problem and often involves custom heuristic algorithms that attempt to minimise their objective function [4, 5, 6].

Graph neural networks (GNNs) have enabled applying deep learning to graph-structured data by utilising the input graph as the neural network's computational graph [7, 8, 9]. Typical tasks for GNNs include node labelling, graph labelling, and link prediction, all of which involve learning meaningful representations jointly from the graph's topology, the nodes' features, and, possibly, the edges' features. Graph labelling relies on coarse-graining the graph through identifying groups of 'similar' nodes and aggregating their links and features, also referred to as pooling [10, 11, 12], which is related to graph clustering [13], however, these two tasks have different goals.

While GNNs excel at incorporating node and edge features with graph topology, including this information is also possible but more challenging with traditional network science approaches, typically requiring modelling or adjusting objective functions and their optimisation algorithms. On the other hand, objective functions for community detection provide precise interpretations as to why one partition is considered better than another while deep-learning-based approaches are black boxes. Model selection in deep learning is often done through regularisation techniques or cross-validation; in contrast, objective functions that are based on the minimum description length (MDL) principle naturally implement Occam's razor, preventing overfitting and enabling principled model selection without requiring extra regularisation or cross-validation [14, 15].

Here, we combine the benefits of traditional community-detection approaches and deep learning and consider the map equation, an information-theoretic objective function for community detection [16]. By adapting the map equation for soft cluster assignments and implementing it in differentiable tensor form, we enable end-to-end optimisation of the map equation as a loss function with gradient descent and GNNs. In analogy to the map equation's stochastic optimisation algorithm Infomap [6], we call our approach Neuromap and evaluate it against Infomap and several recent GNN-based graph clustering methods. Applied to synthetic and real-world networks, Neuromap demonstrates competitive performance against recent deep graph clustering baselines.

Our key contributions can be summarised as follows:

We adapt the map equation as a differentiable loss function for end-to-end deep graph clustering and propose Neuromap, a deep-learning-based alternative to the popular Infomap algorithm for unsupervised community detection with the map equation. Neuromap is compatible with any neural network architecture, detects overlapping communities, leverages node features for improved performance on real-world networks, and, by following the minimum description length principle, does not require explicit regularisation.

We extensively evaluate Neuromap on hundreds of synthetic and ten real datasets against recent baselines paired with various neural network architectures. Neuromap outperforms the baseline on the synthetic networks in most settings and is amongst the best performers in seven out of ten real datasets.

By choosing a higher maximum number of clusters than previous works, we show empirically that recent baselines tend to overfit and report considerably more than the ground-truth number of communities. Moreover, we find that choosing a small maximum number of communities is often detrimental to graph clustering performance.

2 Related work

Community detection. Communities , also called clusters or modules , are groups of nodes that are more 'similar' to each other than to the rest, often understood as having more links inside than between groups [1, 2]. However, this rather general characterisation leaves precise details of what constitutes a community open. Modularity compares the observed link densities inside communities against a randomised version of the network [17]. The stochastic block model and its variants assume a latent block structure where the probability that two nodes are connected depends only on their block memberships [18, 19]. The map equation identifies communities as regions where a random walker tends to stay for a relatively long time [16, 20]. Traditional clustering approaches, such as k-means, group nodes based on their proximity in space, however, here we consider identifying communities from the link patterns in networked systems. For a detailed overview of community detection in complex networks, we refer to [1, 2].

Minimum description length principle. The minimum description length principle (MDL) is a model-selection approach that formalises Occam's razor and frames learning as a compression

problem [14, 15]. MDL states that the best model for data D is the one that compresses the data the most. In traditional MDL, the data's two-part description length L ( D ) = min M L ( M ) + L ( D | M ) is the smallest achievable length over all models M , where L ( M ) is the model's description length, and L ( D | M ) is the data's description length, given the model. MDL has been adopted for a wide range of applications, including regularising neural networks' weights [21], investigating deep neural networks' data-compression capabilities [22], analysing the characteristics of datasets [23], and community detection [16, 24].

Deep graph clustering and pooling. Graph clustering has long been a research focus in machine learning [13, 25]. Spectral approaches cluster, for example, the eigenspace of a graph's Laplacian matrix or identify communities through graph cuts [26, 27]. Methods based on neural embeddings involve learning node representations with, for example, DeepWalk [28] or node2vec [29], followed by applying standard clustering approaches such as k-means, assuming that similar nodes are embedded at similar locations [30, 31]. Other approaches include graph autoencoders [32, 33], contrastive learning [34], and self-expressiveness [35]. Recently, minimum cuts [11, 12], modularity [36, 37], and the Bernoulli-Poisson model [38] have been integrated with GNNs as loss functions for graph pooling and clustering. Such GNN-based approaches can incorporate graph structure as well as node and edge features in end-to-end optimisation of the clustering objective. Inspired by pooling in convolutional neural networks, graph pooling coarse-grains links, node features, and edge features to summarise graphs, enabling GNNs with improved performance on node and graph classification tasks [10, 12]. Consequently, graph pooling has become a research focus for GNNs, emphasising the importance of graph clustering as a primary objective [10, 12, 36]. For recent surveys of deep graph clustering, we refer to [39, 40].

3 Background: the map equation

The map equation [16, 20] is an information-theoretic objective function for unsupervised community detection that follows the MDL principle [14], and has demonstrated high performance in synthetic and real networks from across domains [41, 42, 43]. The map equation formulates community detection as a compression problem and uses random walks as a proxy to model dynamic processes on networks, also called flow . The goal is to describe the random walk as efficiently as possible by minimising its expected per-step description length - also called codelength - by partitioning the network into groups of nodes, called modules , where the random walker tends to stay for a relatively long time. In practice, however, the map equation does not simulate random walks; instead, the codelength is calculated analytically.

101

11011

1000

010

1010

1111

0000

1100

001

11010

011

1001

0001

1110

1011

010

110

00

111

10

01

11

100

10

00

010

110

111

011

0

011

11

00

10

Let G = ( V, E ) be a graph with nodes V , links E , and let w uv ∈ R + 0 denote the non-negative link weight on the link from node u to v . When all nodes are assigned to the same module, the codelength is defined by the Shannon entropy H over the nodes' visit rates [44], H ( P ) = -∑ u ∈ V p u log 2 p u , where p u is node u 's visit rate and P = { p u | u ∈ V } is the set of node visit rates. In undirected graphs, we compute visit rates directly as p u = s u / ∑ v ∈ V s v , where s u = ∑ v ∈ V w uv is node u 's strength. In directed graphs, we compute the visit rates numerically with smart teleportation [45] and a power iteration. When we partition the nodes into modules, the codelength becomes a weighted average of the modules' entropies and the entropy at the so-called index level for switching between modules. Figure 1 illustrates the coding principle behind the map equation using Huffman codes [46]; note, however, that these codewords are only for illustration and we only care about their expected length in the limit to evaluate the map equation.

Minimising the map equation means balancing between small modules to achieve low module-level entropies and large modules for low index-level entropy. This trade-off between module- and indexlevel entropies prevents trivial solutions where all nodes are assigned to the same module or each node is assigned to a singleton module [14]. The map equation calculates the codelength for a partition M ,

Here, q = ∑ m q m is the random walker's module entry rate, q m = ∑ u/ ∈ m ∑ v ∈ m p u t uv is module m 's entry rate, and Q = { q m /q | m ∈ M } is the set of normalised module entry rates; p m = m exit + ∑ u ∈ m p u is the rate at which the random walker moves in module m , including the module exit rate m exit = ∑ u ∈ m ∑ v/ ∈ m p u t uv , and P m = { m exit /p m } ∪ { p u /p m | u ∈ m } is the set of normalised node visit and exit rates for module m . The random walker's transition probability from node u to v is t uv = w uv / ∑ v ∈ V w uv . We can rewrite the map equation as (see Appendix A)

The map equation framework has been extended for overlapping communities through state-space expansions with higher-order network models [47, 48], avoiding over-partitioning in sparse networks using a Bayesian regularisation approach [49], and to deal with sparse constrained structures [50]. Moreover, the map equation framework can incorporate node features through an extension [51] or by preprocessing data [52]. Detecting communities relies on Infomap [6], a greedy stochastic search algorithm that optimises the map equation. However, each of the above extensions requires preprocessing the input data, adjusting the loss function, or adapting the search algorithm. In contrast, adapting the map equation as a loss function for optimisation with gradient descent does not require any custom algorithm, thus enabling flexible experimentation with variations, scalability to GPU clusters, and incorporating it into other loss functions.

4 The map equation goes neural

Weset out to detect communities by optimising the map equation with GNNs through gradient descent, which essentially means learning coarse-graining node representations in the form of communities (Figure 2). While the standard map equation considers hard clusters where each node is assigned to exactly one module, we introduce a soft cluster assignment matrix S n × s to make the map equation differentiable and enable overlapping clusters. We optimise S = softmax (GNN θ ( A , X )) indirectly by optimising the GNN's parameters θ , that is, its weights, with respect to the codelength L . Here, A n × n is the graph's adjacency matrix, X n × d is the node features matrix, n = | V | is the number of nodes, s is the maximum allowed number of clusters, and d is the node feature dimension.

Without loss of generality, we assume directed networks. We denote the graph's total weight as w tot = ∑ i ∈ V ∑ j ∈ V w ij . Let T n × n be the random walker's transition matrix and d in be the vector of weighted node in-degrees with

To compute the vector p of node visit rates, we use smart teleportation [45] and the power iteration method: With probability α , the random walker teleports to a random node, chosen proportionally to

1

2

3

4

9

8

7

5

6

Node features Xn×d

G

GNN

Map Equation Loss

gradient calculation using backpropagation

Soft Clusters Sn×s

softmax

1

2

3

4

5

6

7

8

9

1

2

3

4

6

5

7

8

9

1

2

3

4

5

6

7

8

9

0

1

1

1

0

0

0

0

0

1

0

1

1

0

0

0

0

0

1

1

0

1

1

0

0

0

0

1

1

1

0

1

0

0

0

0

0

0

1

1

0

1

1

0

0

0

0

0

0

1

0

1

1

1

0

0

0

0

1

1

0

1

1

0

0

0

0

0

1

1

0

1

0

0

0

0

0

1

1

1

0

1

2

3

4

9

8

7

5

6

=

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

1

1

1

1

.5

1

1

1

1

0

0

0

0

.5

+

the nodes' in-degrees, or follows a link with probability 1 -α . This approach leads to the iterative update rule p ( t +1) ← α w tot d in +(1 -α ) p ( t ) T , and we set p (0) = d in . The graph's flow matrix F n × n encodes the flow between each pair of nodes, where F = α w tot A +(1 -α ) diag ( p ) T . We obtain the flow C s × s between clusters from S and F as C = S ⊤ FS . Following Equation (2), we define

and assemble the map equation

where 1 k is the k -dimensional vector of ones, and logarithms are applied component-wise. The third term is constant since it only depends on the node visit rates and can be omitted during optimisation.

The map equation naturally incorporates Occam's razor by following the MDL principle for balancing between model complexity and fit [14, 15], choosing the optimal number of communities automatically, but at most s . In contrast, recent GNN-based clustering approaches require explicit regularisation to avoid over-partitioning [10, 36, 37, 38], and our results show that they often return the maximum allowed number of communities instead of determining the number of communities in a data-driven fashion (see Section 5). In principle, any neural network architecture, such as a multi-layer perceptron (MLP) or GNN, can be used to learn the soft cluster assignment matrix S . Since the map equation involves logarithms, we add a small constant ϵ to each value in the output S before the backpropagation step to ensure differentiability. We refer to the combination of using map equation loss (Equation (3)) together with a (graph) neural network to learn (overlapping) communities as Neuromap .

Complexity and limitations. The most expensive calculation is the pooling operation C = S ⊤ FS which depends on the network's density. When s ≪ n and the number of edges is m = O ( n ) , the complexity of Neuromap is linear in n . When the network is dense, m = O ( n 2 ) , or the maximum number of clusters approaches the number of nodes s ≈ n , we approach quadratic complexity. Therefore, we recommend keeping s ≪ n for scalability.

We assume connected networks, otherwise, clustering should be run on the individual components. The node features X aid the GNN in learning patterns, however, they do not contribute to the loss. When no node features are available, Neuromap can use, for example, the adjacency matrix as node features; designing expressive low-dimensional node features remains an active research area [53].

5 Experimental evaluation

We evaluate Neuromap on synthetic and real-world networks with different neural network architectures: a 2-layer graph convolutional network (GCN) [9], a 2-layer graph isomorphism network (GIN) [54], and a 2-layer SAGE network [55]. To investigate whether GNNs are required for clustering, we also include a fully connected linear layer and a 2-layer MLP. We include a learnable temperature

parameter for the softmax operation, which we found speeds up convergence. In all cases, we use the models provided by PyTorch Geometric with SELU activation [56]. Because the specifics between architectures differ, such as message-passing details and aggregation functions, they may be interpreted as using different search algorithms which return different communities. We use the Adam optimiser [57], apply batch normalisation, and for comparability between different methods, set the learning rate for the linear layer to 10 -1 , for MLP to 10 -2 , and for GCN, GIN, and SAGE to 10 -3 . We train all models for up to 10 , 000 epochs with a patience of 100 epochs and dropout probability 0 . 5 . Because the datasets contain hard clusters, we convert the resulting communities to hard clusters, assigning each node to that cluster where it has its strongest membership. As baselines, we use Infomap [6] and five recent approaches for unsupervised graph clustering with GNNs: DMoN [36], NOCD [38], DiffPool [10], MinCut [11], and Ortho [36]. We base our implementation 3 on PyTorch [58] and PyTorch Geometric [59] and ran our experiments on a workstation with an Intel i9-11900K @ 3.50GHz CPU, 32 GB of RAM, and a GeForce RTX 3090 with 24 GB of memory.

5.1 Synthetic networks with planted communties

We generate directed and undirected Lancichinetti-Fortunato-Radicchi (LFR) benchmark networks with planted ground-truth communities [60] with n = 1000 nodes, average node degree k ∈ { ln n, 2 ln n } , maximum node degree k max = 2 √ n , both rounded to the nearest integer, and mixing parameter µ between 0 . 1 and 0 . 8 with a step size of 0 . 1 . We set the power-law exponents for the node degree distribution to τ 1 = 2 , and for the community size distribution to τ 2 = 1 . For each combination of parameters, we generate 10 LFR networks using the implementation provided by the authors, 4 resulting in a total of 320 networks. For each parameter combination, there are 10 LFR networks; for each of these LFR networks, we run each model 10 times, measuring its performance as the average adjusted mutual information (AMI) [61] against the ground truth, and plot the average of those AMI values over the 10 networks per parameter combination. To verify that the number of communities is inferred from the data, we set the maximum number of communities to s = n . Since LFR networks do not have node features, we use the adjacency matrix as node features.

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

AMI

undirected, k = 7

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

undirected, k = 14

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

directed, k = 7

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

directed, k = 14

0

.

2

0

.

4

0

.

6

0

.

8

0

100

200

|

M

|

0

.

2

0

.

4

0

.

6

0

.

8

0

100

200

0

.

2

0

.

4

0

.

6

0

.

8

0

100

200

0

.

2

0

.

4

0

.

6

0

.

8

0

100

200

0

.

2

0

.

4

0

.

6

0

.

8

µ

6

8

10

L

0

.

2

0

.

4

0

.

6

0

.

8

µ

6

8

10

0

.

2

0

.

4

0

.

6

0

.

8

µ

6

8

10

0

.

2

0

.

4

0

.

6

0

.

8

µ

6

8

10

true

Infomap

Linear

MLP

GCN

GIN

SAGE

We find that the detected communities' quality depends on the choice of neural network architecture (Figure 3). Neuromap achieves the best AMI scores with SAGE. GCN, MLP, and Infomap perform slightly worse, however, with some variation depending on the networks' properties. The dense linear layer and GIN show weaker performance but still identify relevant communities. In the sparser directed networks, Infomap performs slightly better than SAGE when the mixing µ is low. However, the AMI values do not tell the whole story: Infomap and MLP tend to report considerably more communities than are present in the ground truth whereas the dense linear layer and GIN tend to report much fewer communities than the ground truth, especially for higher mixing values. GCN reports more communities than are present in the ground truth in the sparser undirected networks but fewer in the directed networks. SAGE detects close to the true number of communities in all cases. Infomap achieves the lowest codelength across all networks. GCN, MLP, and SAGE achieve close to Infomap's codelength, whereas the dense linear layer and GIN have slightly higher codelength.

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

AMI

undirected, k = 7

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

undirected, k = 14

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

directed, k = 7

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

directed, k = 14

0

.

2

0

.

4

0

.

6

0

.

8

µ

10

0

10

1

10

2

10

3

|

M

|

0

.

2

0

.

4

0

.

6

0

.

8

µ

10

0

10

1

10

2

10

3

0

.

2

0

.

4

0

.

6

0

.

8

µ

10

0

10

1

10

2

10

3

0

.

2

0

.

4

0

.

6

0

.

8

µ

10

0

10

1

10

2

10

3

true

Neuromap

DMoN

NOCD

DiffPool

MinCut

We compare Neuromap against recent deep-learning-based community detection methods on the same networks by swapping out the loss function while keeping everything else the same, with the exception of using weight decay for NOCD as per the original paper [38]. For DiffPool, Mincut, Ortho, and DMoN, we use the implementation from PyTorch Geometric, for NOCD, we use the implementation provided by the authors. 5 Figure 4 shows the results for SAGE; in Appendix B, we also include results for the remaining architectures. Neuromap outperforms the baselines across all architectures, except for NOCD which performs better than Neuromap with GIN and with GCN on directed networks. Different from previous works, we have not limited the maximum number of communities, which allows us to analyse the methods' overfitting behaviour: While Neuromap reports close to the ground-truth number of communities, the remaining methods often overfit the networks' structure and report considerably more communities (note the logarithmic scale). MinCut fails to identify meaningful communities on directed networks for mixing values µ > 0 . 3 . NOCD performs best with the GCN architecture, which was also used in the original paper [38]. Neuromap performs best with SAGE in our experiments.

5.2 Real-world networks with node features

We benchmark Neuromap on ten real-world datasets (Table 1) from PyTorch Geometric [59], PyTorch Geometric Signed Directed [62], and Open Graph Benchmark [63], and compare it against the same baselines as before. In contrast to previous works that choose a fixed number of hidden dimensions and set the maximum number of communities to a constant [36] or the 'ground-truth' number of communities [38], we reflect the networks' sizes in our choices: We set the number of hidden dimensions to 4 √ n and the maximum number of communities to s = √ n . Our choices are based on empirical observations showing that the number of communities typically scales as O ( √ n ) [64].

However, a few words of caution are in order: while nodes' true communities determine the link patterns in synthetic networks, it is generally infeasible to obtain ground truth communities for real networks. Often, metadata labels are used as a drop-in, and the inferred communities' quality depends on how well the metadata, which is potentially noisy, aligns with the unknown ground truth [3]. Moreover, determining the number of communities in a network is hard and setting s = √ n should be seen as a simplification rather than an attempt to guess the exact number.

For each method and architecture, we run 25 trials and show the average achieved AMI in Figure 5; we include a similar plot for the number of detected communities as well as the average AMI and the detected number of communities, both with standard deviations, in tabulated form in Appendix C. When several of the best-performing methods achieve similar AMI, we use an independent twosample t-test to determine whether one of them can be considered to perform better than the other (see Appendix C). In cases where their performances do not differ significantly, we mark both as best.

Neuromap and NOCD are amongst the best performers in seven cases and DiffPool in two. The GCN architecture performs best in seven cases, MLP and SAGE in four cases, and GIN in two cases. A

CiteSeer

Pubmed

PC

Photo

0

20

40

60

80

AMI x 100

Neuromap

DMoN

NOCD

DiffPool

MinCut

Infomap

CS

Physics

ogb-arxiv

0

20

40

60

80

AMI x 100

Linear

MLP

GCN

GIN

SAGE

Infomap

possible explanation for why the simpler linear layer and MLP architectures perform well in several cases could be that the map equation loss function captures global information in the random walker's flow patterns, making GNNs superfluous in some cases. All methods tend to detect more communities than are present in the 'ground truth', however, this tendency is most pronounced in DMoN, Ortho, and MinCut, which may have gone unnoticed in previous evaluations where the maximum number of communities was set to a much lower, constant value [36], thus artificially preventing overfitting. Infomap is the only baseline that does not utilise node features; instead, it relies solely on topological information, which may explain the large number of detected communities. Comparing Neuromap's performance against Infomap's performance suggests that incorporating node features substantially improves the detected communities' quality in most cases (see Appendix C for significance tests) while drastically reducing the number of detected communities.

We repeat the same experiments with 512 hidden features and s = | Y | , that is, the 'true' number of communities, following [38] (results in Appendix D). Limiting the number of allowed communities often leads to better performance for DMoN, MinCut, and Ortho, however, with a few exceptions, it diminishes the performance of Neuromap, NOCD, and DiffPool across all datasets and neural network architectures. Appendix E tabulates the differences in average AMI score between setting the hidden features to 4 √ n and s = √ n versus using 512 hidden features and s = | Y | . In the case of Neuromap, imposing a lower bound on the number of communities interferes with the MDL principle, limiting what models for the data may be explored.

5.3 Synthetic networks with overlapping communities

Weapply Neuromap and the baselines to a small synthetic network with overlapping communities [65]. We set the maximum number of communities to s ∈ { 2 , 3 } , run each combination of loss function and neural network architecture for 10 trials, and keep the solutions with the lowest loss. Figure 6 shows the results obtained with GCN, Appendix F shows results for the remaining architectures.

True

s

= 2

s

= 3

Neuromap

DMoN

NOCD

DiffPool

MinCut

We find that Neuromap, DMoN, NOCD, and MinCut identify the correct communities for s = 2 . DiffPool does not detect overlapping communities and Ortho assigns each node to two communities. For s = 3 , only Neuromap identifies the correct communities. DiffPool returns the same communities as for s = 2 . All remaining methods return three communities. These results provide further evidence that the baselines suffer from overfitting when they are not provided with the correct number of communities, which, in general, is unknown. Neuromap identifies meaningful communities while inferring the number of communities in a data-driven fashion by following the MDL principle. However, we leave a more rigorous study of overlapping communities for future work.

6 Conclusion

Network science and deep learning on graphs tackle community detection from different perspectives. Community detection in network science typically relies on custom heuristic optimisation algorithms to optimise objective functions but often does not leverage recent deep learning advances. Recently, deep graph learning methods have started to incorporate methods from network science for deep graph clustering. We contribute to this young field by adapting the map equation, a popular unsupervised information-theoretic community-detection approach, as a differentiable loss function for end-to-end optimisation with (graph) neural networks through gradient descent, and use PyTorch to implement our approach, which we call Neuromap.

We evaluated Neuromap on various synthetic and real-world datasets, using different neural network architectures to detect communities. Our results show that Neuromap achieves competitive performance and detects close to the ground-truth number of communities across datasets while the baselines tend to overfit and report considerably more communities. Across all tested methods, the achieved performance depends on the used neural network architecture. However, on several realworld benchmarks, Neuromap outperforms several of the the baselines even with simpler, non-GNN, neural network architectures. We hypothesise that this may be because the map equation builds on capturing flow patterns, which contain global information.

While we have considered first-order networks with two-level community structures, complex realworld networks often involve higher-order dependencies and can have multi-level communities [47, 66], prompting a generalisation of our approach. Furthermore, incorporating our method for graph pooling as well as uncovering the precise connection between the utilised neural network architecture and the achieved community-detection performance requires further empirical and theoretical studies.

Acknowledgments and Disclosure of Funding

We thank Martin Rosvall, Jelena Smiljani´ c, and Daniel Edler for fruitful discussions and helpful feedback. We are indebted to Lisi Qarkaxhija for helping with PyTorch implementations. Christopher Blöcker and Ingo Scholtes acknowledge funding from the Swiss National Science Foundation, grant 176938, and the German Federal Ministry of Education and Research, grant 100582863 (TissueNet).

References

Santo Fortunato. Community detection in graphs. Physics Reports , 486(3):75-174, 2010.

Santo Fortunato and Mark EJ Newman. 20 years of network community detection. Nature Physics , 18(8):848-850, 2022.

Leto Peel, Daniel B. Larremore, and Aaron Clauset. The ground truth about metadata and community detection in networks. Science Advances , 3(5):e1602548, 2017.

Vincent D Blondel, Jean-Loup Guillaume, Renaud Lambiotte, and Etienne Lefebvre. Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment , 2008(10):P10008, oct 2008.

V. A. Traag, L. Waltman, and N. J. van Eck. From louvain to leiden: guaranteeing wellconnected communities. Scientific Reports , 9(1):5233, Mar 2019.

Daniel Edler, Ludvig Bohlin, and Martin Rosvall. Mapping higher-order network flows in memory and multilayer networks with infomap. Algorithms , 10(4), 2017.

Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE Transactions on Neural Networks , 20(1):61-80, 2009.

Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning , volume 70 of Proceedings of Machine Learning Research , pages 1263-1272. PMLR, 06-11 Aug 2017.

Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations , 2017.

Zhitao Ying, Jiaxuan You, Christopher Morris, Xiang Ren, Will Hamilton, and Jure Leskovec. Hierarchical graph representation learning with differentiable pooling. Advances in neural information processing systems , 31, 2018.

Filippo Maria Bianchi, Daniele Grattarola, and Cesare Alippi. Spectral clustering with graph neural networks for graph pooling. In International conference on machine learning , pages 874-883. PMLR, 2020.

Filippo Maria Bianchi and Veronica Lachi. The expressive power of pooling in graph neural networks. In Advances in Neural Information Processing Systems , volume 36, pages 7160371618. Curran Associates, Inc., 2023.

Satu Elisa Schaeffer. Graph clustering. Computer science review , 1(1):27-64, 2007.

Jorma Rissanen. Modeling by shortest data description. Automatica , 14(5):465-471, 1978.

Peter D Grünwald, In Jae Myung, and Mark A Pitt. Advances in minimum description length: Theory and applications . MIT press, 2005.

Martin Rosvall and Carl T. Bergstrom. Maps of random walks on complex networks reveal community structure. Proceedings of the National Academy of Sciences , 105(4):1118-1123, 2008.

M. E. J. Newman. Modularity and community structure in networks. Proceedings of the National Academy of Sciences , 103(23):8577-8582, 2006.

Paul W. Holland, Kathryn Blackmond Laskey, and Samuel Leinhardt. Stochastic blockmodels: First steps. Social Networks , 5(2):109-137, 1983.

Tiago P Peixoto. Efficient monte carlo and greedy heuristic for the inference of stochastic block models. Physical Review E , 89(1):012804, 2014.

M. Rosvall, D. Axelsson, and C. T. Bergstrom. The map equation. The European Physical Journal Special Topics , 178(1):13-23, Nov 2009.

Geoffrey E Hinton and Drew Van Camp. Keeping the neural networks simple by minimizing the description length of the weights. In Proceedings of the sixth annual conference on Computational learning theory , pages 5-13, 1993.

Léonard Blier and Yann Ollivier. The description length of deep learning models. In Advances in Neural Information Processing Systems , volume 31. Curran Associates, Inc., 2018.

Ethan Perez, Douwe Kiela, and Kyunghyun Cho. Rissanen data analysis: Examining dataset characteristics via description length. In Proceedings of the 38th International Conference on Machine Learning , volume 139 of Proceedings of Machine Learning Research , pages 8500-8513. PMLR, 18-24 Jul 2021.

Tiago P. Peixoto. Hierarchical block structures and high-resolution model selection in large networks. Phys. Rev. X , 4:011047, Mar 2014.

Kai Yu, Shipeng Yu, and Volker Tresp. Soft clustering on graphs. Advances in neural information processing systems , 18, 2005.

Jianbo Shi and Jitendra Malik. Normalized cuts and image segmentation. IEEE Transactions on pattern analysis and machine intelligence , 22(8):888-905, 2000.

Ulrike von Luxburg. A tutorial on spectral clustering. Statistics and Computing , 17(4):395-416, Dec 2007.

Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , KDD '14, page 701-710, New York, NY, USA, 2014. Association for Computing Machinery.

Aditya Grover and Jure Leskovec. Node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , KDD '16, page 855-864, New York, NY, USA, 2016. Association for Computing Machinery.

Aditya Tandon, Aiiad Albeshri, Vijey Thayananthan, Wadee Alhalabi, Filippo Radicchi, and Santo Fortunato. Community detection in networks using graph embeddings. Physical Review E , 103(2):022316, 2021.

Sadamori Kojaku, Filippo Radicchi, Yong-Yeol Ahn, and Santo Fortunato. Network community detection via neural embeddings. arXiv preprint arXiv:2306.13400 , 2023.

Chun Wang, Shirui Pan, Guodong Long, Xingquan Zhu, and Jing Jiang. Mgae: Marginalized graph autoencoder for graph clustering. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management , pages 889-898, 2017.

Nairouz Mrabah, Mohamed Bouguessa, Mohamed Fawzi Touati, and Riadh Ksantini. Rethinking graph auto-encoder models for attributed graph clustering. IEEE Transactions on Knowledge and Data Engineering , 2022.

Maedeh Ahmadi, Mehran Safayani, and Abdolreza Mirzaei. Deep graph clustering via mutual information maximization and mixture model. Knowledge and Information Systems , Apr 2024.

Sambaran Bandyopadhyay and Vishal Peter. Unsupervised constrained community detection via self-expressive graph neural network. In Uncertainty in Artificial Intelligence , pages 1078-1088. PMLR, 2021.

Anton Tsitsulin, John Palowitch, Bryan Perozzi, and Emmanuel Müller. Graph clustering with graph neural networks. Journal of Machine Learning Research , 24(127):1-21, 2023.

Tsuyoshi Murata and Naveed Afzal. Modularity optimization as a training criterion for graph neural networks. In Complex Networks IX , pages 123-135, Cham, 2018. Springer International Publishing.

Oleksandr Shchur and Stephan Günnemann. Overlapping community detection with graph neural networks. Deep Learning on Graphs Workshop, KDD , 2019.

Liu Yue, Xia Jun, Zhou Sihang, Wang Siwei, Guo Xifeng, Yang Xihong, Liang Ke, Tu Wenxuan, Liu Xin Wang, et al. A survey of deep graph clustering: Taxonomy, challenge, and application. arXiv preprint arXiv:2211.12875 , 2022.

Su Xing, Xue Shan, Liu Fanzhen, Wu Jia, Yang Jian, Zhou Chuan, Hu Wenbin, Paris Cecile, Nepal Surya, Jin Di, et al. A comprehensive survey on community detection with deep learning. IEEE Trans. Neural Netw. Learn. Syst , 2022.

Andrea Lancichinetti and Santo Fortunato. Community detection algorithms: A comparative analysis. Phys. Rev. E , 80:056117, 2009.

Rodrigo Aldecoa and Ignacio Marín. Exploring the limits of community detection strategies in complex networks. Sci. Rep. , 3:2216, 2013.

Lovro Šubelj, Nees Jan van Eck, and Ludo Waltman. Clustering scientific publications based on citation relations: A systematic comparison of different methods. PLOS ONE , 11(4):1-23, 04 2016.

C. E. Shannon. A mathematical theory of communication. Bell Syst. Tech. J. , 27:379-423, 1948.

R. Lambiotte and M. Rosvall. Ranking and clustering of nodes in networks with smart teleportation. Phys. Rev. E , 85:056107, May 2012.

David A. Huffman. A method for the construction of minimum-redundancy codes. Proceedings of the IRE , 40(9):1098-1101, 1952.

Martin Rosvall, Alcides V. Esquivel, Andrea Lancichinetti, Jevin D. West, and Renaud Lambiotte. Memory in network flows and its effects on spreading dynamics and community detection. Nature Communications , 5(1):4630, Aug 2014.

Anton Holmgren, Christopher Blöcker, and Martin Rosvall. Mapping biased higher-order walks reveals overlapping communities, 2023.

Jelena Smiljani´ c, Christopher Blöcker, Daniel Edler, and Martin Rosvall. Mapping flows on weighted and directed networks with incomplete observations. Journal of Complex Networks , 9(6):cnab044, 12 2021.

Daniel Edler, Jelena Smiljani´ c, Anton Holmgren, Alexandre Antonelli, and Martin Rosvall. Variable markov dynamics as a multi-focal lens to map multi-scale complex networks, 2022.

Scott Emmons and Peter J. Mucha. Map equation with metadata: Varying the role of attributes in community detection. Phys. Rev. E , 100:022301, Aug 2019.

Aleix Bassolas, Anton Holmgren, Antoine Marot, Martin Rosvall, and Vincenzo Nicosia. Mapping nonlocal relationships between metadata and network structure with metadata-dependent encoding of random walks. Science Advances , 8(43):eabn7558, 2022.

Derek Lim, Joshua David Robinson, Lingxiao Zhao, Tess Smidt, Suvrit Sra, Haggai Maron, and Stefanie Jegelka. Sign and basis invariant networks for spectral graph representation learning. In The Eleventh International Conference on Learning Representations , 2023.

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In International Conference on Learning Representations , 2019.

Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems , volume 30. Curran Associates, Inc., 2017.

Günter Klambauer, Thomas Unterthiner, Andreas Mayr, and Sepp Hochreiter. Self-normalizing neural networks. Advances in neural information processing systems , 30, 2017.

Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017.

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32 , pages 8024-8035. Curran Associates, Inc., 2019.

Matthias Fey and Jan Eric Lenssen. Fast graph representation learning with pytorch geometric. arXiv preprint arXiv:1903.02428 , 2019.

Andrea Lancichinetti, Santo Fortunato, and Filippo Radicchi. Benchmark graphs for testing community detection algorithms. Phys. Rev. E , 78:046110, Oct 2008.

Simone Romano, James Bailey, Vinh Nguyen, and Karin Verspoor. Standardized mutual information for clustering comparisons: One step further in adjustment for chance. In Proceedings of the 31st International Conference on Machine Learning , volume 32 of Proceedings of Machine Learning Research , pages 1143-1151, Bejing, China, 22-24 Jun 2014. PMLR.

Yixuan He, Xitong Zhang, Junjie Huang, Benedek Rozemberczki, Mihai Cucuringu, and Gesine Reinert. Pytorch geometric signed directed: A software package on graph neural networks for signed and directed graphs. In Learning on Graphs Conference , pages 12-1. PMLR, 2024.

Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems , 33:22118-22133, 2020.

Amir Ghasemian, Homa Hosseinmardi, and Aaron Clauset. Evaluating overfit and underfit in models of network community structure. IEEE Transactions on Knowledge and Data Engineering , 32(9):1722-1735, 2020.

Alcides Viamontes Esquivel and Martin Rosvall. Compression of flow can reveal overlappingmodule organization in networks. Phys. Rev. X , 1:021025, Dec 2011.

M. Rosvall and C. T. Bergstrom. Multilevel Compression of Random Walks on Networks Reveals Hierarchical Organization in Large Integrated Systems. PLoS One , 6:e18209, 2011.

David F. Gleich. Pagerank beyond the web. SIAM Review , 57(3):321-363, 2015.

A Map equation details

Let G = ( V, E ) be a graph with nodes V , links E , and let w uv ∈ R + 0 denote the non-negative weight on the link from node u to v . Further, let p u be node u 's ergodic visit rate, which can be calculated in closed form as p u = s u / ∑ v ∈ V s v in undirected graphs, where s u = ∑ v ∈ V w uv is node u 's strength. In directed graphs, p u can be calculated numerically with smart teleportation and a power iteration by solving the recursive set of equations p v = α s out v ∑ u ∈ V s out u +(1 -α ) ∑ u ∈ V p u t uv , where s out u = ∑ v ∈ V w uv is node u 's out-strength, t uv = w uv / ∑ v ∈ V w uv is the random walker's transition probability from node u to node v , and α is a teleportation parameter, typically set to α = 0 . 15 [45]. Smart teleportation is similar to PageRank [67] but, instead of uniformly teleporting to nodes, the random walker teleports to links proportional to their weight.

For a given partition M of the nodes into modules, the map equation [16] calculates the average per-step description length - also called codelength - for describing the position of a random walker on the graph:

Here, q = ∑ m q m is the random walker's module entry rate, q m = ∑ u/ ∈ m ∑ v ∈ m p u t uv is module m 's entry rate, and Q = { q m /q | m ∈ M } is the set of normalised module entry rates; p m = m exit + ∑ u ∈ m p u is the rate at which the random walker moves in module m , including the exit rate m exit = ∑ u ∈ m ∑ v/ ∈ m p u t uv , and P m = { m exit /p m } ∪ { p u /p m | u ∈ m } is the set of normalised node visit rates and exit rates for module m .

A.1 Rewriting the map equation

The map equation can be rewritten by expanding the entropy terms and cancelling common factors

Applying logarithm rules gives

and further simplification yields

In undirected networks where q m = m exit, this can be further simplified [20] to

The last term in Equation (10), that is, the nodes' contribution to the codelength, is constant because it does not depend on the module structure, and can be omitted during optimisation.

A.2 Node flow with soft cluster assignments

While the nodes' codelength contribution is constant with hard clusters (Equation (10)), expressing the map equation with soft cluster assignments allows modelling the flow of nodes that are assigned to more than one module in at least two different ways which we illustrate in Figure 7.

The first option is to reflect the nodes' partial cluster assignment for assigning codewords. Consider node u with visit rate p u and partial assignments of 1 2 to clusters m 1 whose usage rate is p m 1 and 1 2

L = 3.1  bits

L = 3.3  bits

L = 3.12  bits

L = 3.13  bits

.332

.332

.332

.332

.332

.332

.332

.332

.464

.1

.281

.1

.232

.281

.281

.281

.281

.361

.232

.232

.232

.232

.1

.258

.1

.258

.258

.258

.258

.258

.258

.258

.258

.258

.158

.158

.1

.258

.1

.258

.258

.258

.258

.258

.258

.258

.258

.258

.258

.258

(a)

(b)

(c)

(d)

to cluster m 2 whose usage rate is p m 2 . Then, half of u 's flow, that is p u 2 , falls into each of the two clusters and u 's contribution to the overall codelength is -p u 2 log 2 p u 2 p m 1 -p u 2 log 2 p u p 2 m 2 . In general, let node u 's assignment to cluster m i be s ui ∈ [0 , 1] with ∑ i s ui = 1 , and let m i 's usage rate be p m i . Then, node u 's contribution to the overall codelength is -∑ i s ui p u log 2 s ui p u p m i . Essentially, this approach splits each node with assignments to more than one cluster into its corresponding parts, resulting in potentially many small pieces with low visit rates.

The second option is to treat the nodes as indivisible when computing their codeword lengths but reflecting their soft assignments in the contribution to the overall codelength. Using the same example as above, node u 's contribution to the overall codelength is -p u 2 log 2 p u p m 1 -p u 2 log 2 p u p m 2 , which is lower because the values inside the logarithms are bigger. In general, u 's contribution to the codelength is -∑ i s ui p u log 2 p u p m i , where the difference to the previous option is that s ui does not appear inside the logarithm.

B Results on synthetic networks with different (G)NN architectures

Here, we report further results on the synthetic LFR networks with planted communities for Neuromap, DMoN, NOCD, DiffPool, MinCut, and Ortho using the following architectures: a dense linear layer, a 2-layer MLP, a 2-layer GCN, and a 2-layer GIN. The setup is as described in the main text.

B.1 Linear-layer-based results

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

AMI

undirected, k = 7

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

undirected, k = 14

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

directed, k = 7

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

directed, k = 14

0

.

2

0

.

4

0

.

6

0

.

8

µ

10

0

10

1

10

2

10

3

|

M

|

0

.

2

0

.

4

0

.

6

0

.

8

µ

10

0

10

1

10

2

10

3

0

.

2

0

.

4

0

.

6

0

.

8

µ

10

0

10

1

10

2

10

3

0

.

2

0

.

4

0

.

6

0

.

8

µ

10

0

10

1

10

2

10

3

true

Neuromap

DMoN

NOCD

DiffPool

MinCut

B.2 MLP-based results

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

AMI

undirected, k = 7

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

undirected, k = 14

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

directed, k = 7

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

directed, k = 14

0

.

2

0

.

4

0

.

6

0

.

8

µ

10

0

10

1

10

2

10

3

|

M

|

0

.

2

0

.

4

0

.

6

0

.

8

µ

10

0

10

1

10

2

10

3

0

.

2

0

.

4

0

.

6

0

.

8

µ

10

0

10

1

10

2

10

3

0

.

2

0

.

4

0

.

6

0

.

8

µ

10

0

10

1

10

2

10

3

true

Neuromap

DMoN

NOCD

DiffPool

MinCut

B.3 GCN-based results

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

AMI

undirected, k = 7

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

undirected, k = 14

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

directed, k = 7

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

directed, k = 14

0

.

2

0

.

4

0

.

6

0

.

8

µ

10

0

10

1

10

2

10

3

|

M

|

0

.

2

0

.

4

0

.

6

0

.

8

µ

10

0

10

1

10

2

10

3

0

.

2

0

.

4

0

.

6

0

.

8

µ

10

0

10

1

10

2

10

3

0

.

2

0

.

4

0

.

6

0

.

8

µ

10

0

10

1

10

2

10

3

true

Neuromap

DMoN

NOCD

DiffPool

MinCut

B.4 GIN-based results

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

AMI

undirected, k = 7

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

undirected, k = 14

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

directed, k = 7

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

5

1

.

0

directed, k = 14

0

.

2

0

.

4

0

.

6

0

.

8

µ

10

0

10

1

10

2

10

3

|

M

|

0

.

2

0

.

4

0

.

6

0

.

8

µ

10

0

10

1

10

2

10

3

0

.

2

0

.

4

0

.

6

0

.

8

µ

10

0

10

1

10

2

10

3

0

.

2

0

.

4

0

.

6

0

.

8

µ

10

0

10

1

10

2

10

3

true

Neuromap

DMoN

NOCD

DiffPool

MinCut

C Results on Real-World Networks

Here, we report further results on real-world networks for setting the number of hidden dimensions to 4 √ n and the maximum number of communities to s = √ n . Specifically, we visualise the average detected number of communities per loss function and neural network architecture for each dataset. We tabulate the average AMI and average number of detected communities, including standard deviations. Two-sample t-tests show when Neuromap or a baseline performs significantly better than the other.

C.1 Number of detected communities on real-world networks

CiteSeer

Pubmed

PC

Photo

10

0

10

1

10

2

|

M

|

Neuromap

DMoN

NOCD

DiffPool

MinCut

CS

Physics

ogb-arxiv

10

0

10

1

10

2

|

M

|

Linear

MLP

GCN

GIN

SAGE

C.2 Tabulated AMI results on real-world networks

Here, we report the detailed AMI performance for Neuromap, DMoN, NOCD, DiffPool, MinCut, Ortho, and Infomap on real-world networks for setting the number of hidden dimensions to 4 √ n and the maximum number of communities to s = √ n . When several of the best-performing methods achieve similar AMI, we use an independent two-sample t-test to determine whether one of them can be considered to perform better than the other. In cases where their performances do not differ significantly, we mark both as best.

C.3 Significance of best Neuromap results vs. best baseline

C.4 Significance of Neuromap results vs. Infomap

C.5 Tabulated number of detected communities on real-world networks

Here, we report the average number of detected communities for Neuromap, DMoN, NOCD, DiffPool, MinCut, Ortho, and Infomap on real-world networks for setting the number of hidden dimensions to 4 √ n and the maximum number of communities to s = √ n .

D Results on real-world networks with fixed number of hidden channels

Weprovide further results on real-world networks with a different configuration of the neural networks: we set the number of hidden channels to 512 and set the maximum number of communities to the 'ground-truth' number of communities, that is s = | Y | .

D.1 Average Achieved AMI

CiteSeer

Pubmed

PC

Photo

0

20

40

60

80

AMI x 100

Neuromap

DMoN

NOCD

DiffPool

MinCut

Infomap

CS

Physics

ogb-arxiv

0

20

40

60

80

AMI x 100

Linear

MLP

GCN

GIN

SAGE

Infomap

D.2 Average Number of Detected Communities

CiteSeer

Pubmed

PC

Photo

0

.

0

2

.

5

5

.

0

7

.

5

10

.

0

|

M

|

Neuromap

DMoN

NOCD

DiffPool

MinCut

CS

Physics

ogb-arxiv

0

10

20

30

40

|

M

|

Linear

MLP

GCN

GIN

SAGE

D.3 Tabulated AMI results on real-world networks

Here, we report the detailed AMI performance for Neuromap, DMoN, NOCD, DiffPool, MinCut, Ortho, and Infomap on real-world networks for setting the number of hidden dimensions to 512 and the maximum number of communities to s = | Y | . When several of the best-performing methods achieve similar AMI, we use an independent two-sample t-test to determine whether one of them can be considered to perform better than the other. In cases where their performances do not differ significantly, we mark both as best.

D.4 Significance of best Neuromap results vs. best baseline

D.5 Significance of Neuromap results vs. Infomap

D.6 Tabulated number of detected communities on real-world networks

Here, we report the average number of detected communities for Neuromap, DMoN, NOCD, DiffPool, MinCut, Ortho, and Infomap on real-world networks for setting the number of hidden dimensions to 512 and the maximum number of communities to s = | Y | .

E Performance impact of hidden layers and max. number of communities

Here, we report the performance difference in terms of achieved average AMI score between using (i) 4 √ n hidden layers and a maximum of s = √ n communities, and (ii) 512 hidden layers and a maximum of s = | Y | communities, that is, the 'ground-truth' number of communities. The tabulated values show much better or worse the methods perform when using setup (ii) as compared to setup (i). That is, a positive value, shown in green, means that setup (ii) gives a better result while a negative value, shown in red, means that setup (i) gives a better result.

We find that, for Neuromap, NOCD, and DiffPool, setup (i) generally works better. In contrast, DMoN, MinCut, and Ortho often perform better with setup (ii), indicating that they require knowing the correct number of communities to perform well.

F Further results on a synthetic network with overlapping communities

Here we show the detected communities on a synthetic network with overlapping communities with a dense linear layer, a 2-layer MLP, a 2-layer GIN, and a 2-layer SAGE architecture. We use | V | hidden channels and set the maximum number of communities to s ∈ { 2 , 3 } .

F.1 Linear-layer-based results

True

s

= 2

s

= 3

Neuromap

DMoN

NOCD

DiffPool

MinCut

F.2 MLP-based results

True

s

= 2

s

= 3

Neuromap

DMoN

NOCD

DiffPool

MinCut

F.3 GIN-based results

True

s

= 2

s

= 3

Neuromap

DMoN

NOCD

DiffPool

MinCut

F.4 SAGE-based results

True

s

= 2

s

= 3

Neuromap

DMoN

NOCD

DiffPool

MinCut

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Abstract and introduction introduce the research gap and summarise the developed method and experimentally achieved results on a high level.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitation of our work in section 4.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: We focus on adapting the well-established map equation framework for community detection with (graph) neural networks and show its utility empirically, however, our work does not include theoretical results or proofs.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Our main contribution is adapting the map equation as a differentiable loss function for optimisation with (graph) neural networks. We present the mathematical formulation of our adaptation in section 4. In section 5, we describe our experimental setup, including hyperparameter settings for our experiments.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We made our code available through a public GitHub repository and the synthetic networks through a Zenodo repository. The used real-world networks are publicly available.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We detail our experimental settings, including hyperparameters, in section 5.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We report the average achieved performance with standard deviations.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We describe the utilised computational hardware in section 5.

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We have reviewed the code of ethics and find that we conform with it.

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [No]

Justification: We consider the task of detecting communities in complex networks. While any algorithm can potentially be abused for malicious purposes, we believe that, in our case, this would be a rather contrived scenario.

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [No]

Justification: We find it not applicable to provide safeguards in the context of detecting communities in complex networks to understand their organisational structure.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We cite the original works of the models we use as baselines as well as the sources of the datasets we use.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We do not introduce new assets.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Our work involves neither crowdsourcing nor research with human subjects.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Our work involves neither crowdsourcing nor research with human subjects.

Peer reviews:
comments: SUMMARY:
The paper titled "The Map Equation Goes Neural: Mapping Network Flows with Graph Neural Networks" presents a novel approach for community detection in networked systems by combining the traditional map equation with modern graph neural networks (GNNs). The authors propose Neuromap, a method that adapts the map equation into a differentiable form for optimization through gradient descent, enabling the integration of node features and end-to-end learning. The paper claims competitive performance against state-of-the-art graph clustering baselines on both synthetic and real-world datasets, emphasizing automatic cluster number determination without the need for explicit regularization.

STRENGTHS:
- Novel Integration: The paper successfully integrates the traditional map equation with GNNs, offering a new approach for community detection that leverages the strengths of both methodologies.

- Comprehensive Evaluation: Extensive experiments on both synthetic and real-world datasets demonstrate the competitive performance of Neuromap compared to existing methods.

WEAKNESSES:
- Complexity Concerns: The complexity analysis indicates potential scalability issues for dense networks or when the number of clusters approaches the number of nodes.

- Presentation: for example, it is unclear that what MDL principle is, and how it relates to this work in detail.

QUESTIONS:
The improvement in overfitting being one of the major advantages of the method, is that possible the author can dive a little deeper into the underlying driver (other than only referencing to how MDL addresses the concern of Occam's razor)?

LIMITATIONS:
There is limitation regarding complexity/scalability.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-10

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes an information-theoretic centered approach for clustering nodes of a given graph. The main intuition at the core of the manuscript is to rewrite the map equation in a differentiable form and to train a Graph Neural Network (GNN) with the intent of minimizing its value (thus optimizing the clustering of the provided nodes). As GNNs are naturally suited to process features available on the nodes of the provided domain, this allows to incorporate such information in the optimization of the considered equation and hopefully achieve better assignments. Experiments on both synthetic and real datasets show good performance of the proposed method that generally matches or outperforms prior art.\n\nSTRENGTHS:\nThe paper proposes an interesting approach for inferring clusters of nodes from a given graph. The manuscript is generally well written, and rather easy to follow (although some details could be better explained in the main text of the paper, see weaknesses). The approach is generally efficient (time complexity is linear in the number of edges for sparse graphs), and experiments on a variety of datasets show good performance w.r.t. previously presented approaches (with the proposed method achieving comparable or superior performance to prior art).\n\nWEAKNESSES:\nOverall, I don’t have particular criticisms for the paper. The main items I would like the authors to address are:\n\n1) I believe equation 1 could be better explained in the paper. In particular, $q_m$ and $m_{exit}$ are not defined in the paper, and their meaning is outlined only at a high level. Having a more formal definition of such values would improve the clarity of the manuscript (I would personally also clarify that the entropy function re-normalizes the probabilities in such equation, as I believe $Q$ and $P_m$ do not sum up to 1 in equation 1)\n\n2) While the performance of the proposed solutions are indeed good compared to previous approaches, if we inspect table 2 in the supplemental material, we can see that the results produced by the proposed Neuromap and and NOCD are quite close on CORA, PC, Wiki CS and ogb-arxiv, once we take into account the standard deviation of the models. As a result of this, stating that Neuromap achieves best results on these datasets is potentially misleading. I would suggest the authors to report the result of a statistical significance test to highlight when the proposed model actually achieves (with high chance) superior performance w.r.t. the best baseline.\n\nQUESTIONS:\nNone.\n\nLIMITATIONS:\nGenerally yes, the authors for instance discussed the complexity of the model and how this can become quadratic for dense graphs (or when large number of clusters are used), and how discovering why different GNNs lead to different performance remains an open question in the manuscript.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors formulate the well-known MAP equation for community detection as an unsupervised objective for graph clustering with GNNs. The implement this \"soft\" neural MAP equation in various GNN architectures, showing reasonable performance on both synthetic and real-world graph clustering tasks.\n\nSTRENGTHS:\nThere are many strengths of this paper:\n\n1) Presentation: the paper is quite well-written and the figures are very clear. I did not find any serious (or even minor) confusions in prose, notation, or figure details. The organization of the paper makes sense and helps the reader understand the results.\n\n2) Novelty: in my opinion, the paper is best compared to Tsitsulin et al. 2023 which introduced the modularity-based DMoN pooling method for neural graph clustering. The paper improves upon DMoN in a few key ways:\n * By adopting the InfoMap objective instead of the Modularity objective, DMoN's need for the collapse regularization is removed, which likely greatly helps with convergence and learing a good community model. It also simplifies the model in general.\n * Empirically, using the InfoMap objective in graph clustering seems to achieve or surpass the performance of all competitors (including DMoN), on synthetic LFR graphs, real graphs, and a toy graph with overlapping communities.\n * The authors do a better job than Tsitsulin et al. at noting the independence of the objective and the neural graph encoder architecture. The encoder architectures are explicitly varied in this work on both synthetic and real graphs, and across all methods on which the encoder architecture can vary.\n\n3) Significance: the clear exposition of the method, empirical dominance, and good commentary about the relationships between network science and neural graph clustering make this a solid contribution to the field.\n\nWEAKNESSES:\nThe main weakness of this paper is that the proposed modeling approach has limited novelty compared with that from Tsitsulin et al. 2023:\n\n1) Both papers propose to use a neural graph encoder to obtain a soft clustering embedding matrix $\\mathbf{S}$.\n2) Both papers train the encoder by adopting an objective from network science (here, Infomap; previously, Modularity) and pass $\\mathbf{S}$ in place of the hard-clustering (one-hot) community assigment matrix.\n\nThe primary difference between the approaches is that the proposed objective does not need the regularization component introduced in Tsitsulin et al. 2023 to deal with the collapse condition, i.e. all nodes are assigned to one cluster. However, the reason for this is simply the choice of the InfoMap objective -- as the authors explain, this objective has inherent ability to balance cluster size with clustering complexity (a result that has been well-known for >10years).\n\nOverall, the authors combine pre-existing work to achieve a better neural graph clusterer which nonetheless closely resembles existing graph clusterers that have adopted objectives from network science. This weakeness does not necessarily block acceptance, but it should be well-noted that the novelty is limited.\n\nThere are very few other weaknesses -- the paper is quite well-written, and the experiments convincingly show that the InfoMap objective is probably better overall than previous objectives.\n\nQUESTIONS:\nQ1: I'm curious, do you in fact agree that your approach is best compared to DMoN?\n\nQ2: Is my characterization about the novelty fair? Are there other architectural/practical differences that I missed or that you wish to highlight?\n\nLIMITATIONS:\nYes, adequate", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a deep learning approach for graph clustering called NeuroMap, which can be seen as a neural version of InfoMap. The idea is to minimize the a relaxation of the InfoMap objective using gradient descent. The proposed approach can be combined with different neural network architectures (MLP, GNN, etc.), works with directed graphs, and can identify overlapping communities. Results using synthetic (LFR) and real datasets (up to 170K nodes) show that NeuroMap often outperforms the baselines in terms of Adjusted Mutual Information.\n\nSTRENGTHS:\n1. The paper is easy to follow.\n\n2. The experiments are based on multiple datasets (real and synthetic).\n\n3. The proposed method does not require specifying the number of clusters.\n\nWEAKNESSES:\n1. There are multiple papers proposing relaxations of clustering objectives optimized via deep learning.\n\n2. It is not clear why InfoMap is the ideal community detection algorithm to be made neural given the variety of existing solutions and results showing that there isn't an algorithm that is superior across datasets.\n\n3. The paper lacks any theoretical or empirical analysis trying to explain why relaxation with gradient descent is more effective than the combinatorial heuristics used to optimize the Map objective.\n\nQUESTIONS:\n1. What are the specific challenges in developing a relaxation of the map objective compared with other clustering objectives?\n\n2. Why InfoMap should be chosen as a classical community detection algorithm to be made neural?\n\n3. Why is the combination of relaxation and gradient descent more effective than the classical heuristics applied to optimize the map objective?\n\nLIMITATIONS:\nI believe that there are limitations beyond the scalability and need of features (see my comments above) but these are indeed important limitations recognized by the authors.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}