Paper: The Map Equation goes Neural (ICLR-2024)

Reviewer comment (Novelty):
The contribution itself is marginal. The authors seem to simply replace the objective of Tsitsulin et al. 2023 with the MAP equation.

Author response:
We did not use the approach of Tsitsulin et al. 2023. The contribution and approach of Tsitsulin et al. 2023 was a neural clustering objective. Our work was inspired by DiffPool and, similar to Tsitsulin et al. (2023), we also propose an objective function for community detection with GNNs. As the results in Fig. 2 illustrate, the objectives of DMoN and Neurmap cannot simply be exchanged: using DMoN to cluster small synthetic networks without informative node features returns singleton clusters.

Our work contributes to narrowing the gap between the network science and deep learning communities. As we explain, optimising the map equation with GNNs naturally addresses several issues of Infomap:

_"[...] our approach naturally addresses over-partitioning in sparse networks, detecting overlapping communities, and avoiding splitting constrained structures."_

Reviewer comment (Theory):
The authors claim that the MAP equation avoids over-partitioning, but do not provide any theoretical justification.

Author response:
Avoiding collapsing to a trivial solution where all nodes are assigned to the same cluster or each node is assigned to its own singleton cluster is achieved by following the minimum description length principle, an information-theoretic model-selection approach that has long been used to balance between model complexity and fit [1,2]. The map equation avoids over- and underfitting by following the minimum description length principles as extensively theoretically explored and discussed in previous work related to the map equation [3,4].

[1] Rissanen, Jorma. "Modeling by shortest data description." Automatica 14.5 (1978): 465-471. \
[2] Grünwald, Peter D., In Jae Myung, and Mark A. Pitt, eds. Advances in minimum description length: Theory and applications. MIT press, 2005. \
[3] Rosvall, Martin, Daniel Axelsson, and Carl T. Bergstrom. "The map equation." The European Physical Journal Special Topics 178.1 (2009): 13-23. \
[4] Rosvall, Martin, and Carl T. Bergstrom. "Maps of random walks on complex networks reveal community structure." Proceedings of the national academy of sciences 105.4 (2008): 1118-1123.

Reviewer comment (Novelty):
The authors claim the ability to detect overlapping communities as a contribution of their work, but this is also true of any "soft clustering" neural method including Tsitsulin et al. 2023.

Author response:
Our work enables detecting overlapping clusters from first-order data with Neuromap whereas the standard map equation requires a higher-order model as we explain in section 4.2.:

_"Through soft cluster assignment, we naturally discover overlapping communities (Figure 2a-b). In contrast, the standard map equation with hard cluster assignments requires higher-order data (Rosvall et al., 2014) or flow modelling (Holmgren et al., 2023) and a memory network representation to detect overlapping communities"_

We have clustered the networks in Fig. 2 with DMoN and added the results to Fig. 2. Because the nodes in these networks have no features, we use the identity matrix as node features. We find that DMoN returns each node in its own singleton community, suggesting that DMoN focuses on node features rather than topological patterns.

Reviewer comment (Theory):
re W2, on page 4, the authors claim "the map equation naturally incorporates Occam's razor: minimising the map equation requires a trade-off between choosing small modules for low module-level codelength and choosing a small number of modules for low index-level codelength". This is a strong claim but no theoretical justification was given. It is not clear nor obvious how the Occam's razor concept can be rigorously formulated in (or satisfied by) a neural clustering objective. As was done in Tsitsulin et al. 2023, the authors should formally argue how their objective avoids the collapse condition (all nodes in singleton clusters or in the unity cluster).

Author response:
Avoiding collapsing to a trivial solution where all nodes are assigned to the same cluster or each node is assigned to its own singleton cluster is achieved by following the minimum description length principle, an information-theoretic model-selection approach that has long been used to balance between model complexity and fit [1,2]. The map equation avoids over- and underfitting by following the minimum description length principles as extensively theoretically explored and discussed in previous work related to the map equation [3,4].

[1] Rissanen, Jorma. "Modeling by shortest data description." Automatica 14.5 (1978): 465-471. \
[2] Grünwald, Peter D., In Jae Myung, and Mark A. Pitt, eds. Advances in minimum description length: Theory and applications. MIT press, 2005. \
[3] Rosvall, Martin, Daniel Axelsson, and Carl T. Bergstrom. "The map equation." The European Physical Journal Special Topics 178.1 (2009): 13-23. \
[4] Rosvall, Martin, and Carl T. Bergstrom. "Maps of random walks on complex networks reveal community structure." Proceedings of the national academy of sciences 105.4 (2008): 1118-1123.

Reviewer comment (Writing):
The presentation of this paper is quite poor. The notations in the Map Equation Loss section, which is the most significant part of this paper, are totally confusing.

Author response:
Thank you for pointing out that the notation may be a hindrance for understanding our work. We have revised the mathematical notation, adopting Einstein notation for greater clarity of the rank of tensors, including the differences between matrices and vectors.

Reviewer comment (Writing):
The description of Neuromap is too compressed. The details of GNNs with the map equation loss are missing.

Author response:
We have revised our notation for clarity. For details about the GNNs, we refer the reviewer to section 4.1. and 4.3. where the GNN architectures are specified.

Reviewer comment (Evaluation):
The experimental results are not convincing. The results of Neuromap in Figure 1, Tables 2 and 3 are hard to say competitive. It seems that the original Infomap algorithm performs better on many benchmarks.

Author response:
Infomap achieves lower codelengths and higher NMI values especially for high mixing values. However, examining the number of detected communities shows that Infomap is overfitting and detects considerably more communities than are present in the ground truth. For low mixing values, Figure 1 shows that Neuromap achieves similar codelengths and NMI values as Infomap, even slightly outperforming Infomap. Importantly, achieving a low codelength alone does not guarantee that the identified modules are meaningful because the map equation tends to detect spurious modules in sparse network regions [1].

As the revised version of table 2 and 7 shows, we observe a similar behaviour in real networks where Infomap achieves lower codelengths at the expense of a considerably higher number of communities than Neuromap. Again, a low codelength alone does not guarantee meaningful communities because random patterns in sparse network regions appear as though they are communities -- Neuromap avoids this overfitting.

[1] Smiljanić et al., Mapping flows on weighted and directed networks with incomplete observations

Reviewer comment (Reproducibility):
How do you identify the overlapping communities in your algorithm?

Author response:
We detect communities by optimising the soft cluster assignment matrix $\mathbf{S}$ using the map equation loss function. Each respective row of $\mathbf{S}$ specifies the fraction of communitiy membership for each respective node. Optimising $\mathbf{S}$ with respect to map equation loss determines whether nodes belong to one or several communities.

Reviewer comment (Evaluation):
What is the efficiency of Neuromap in the experiments?

Author response:
We discuss the complexity in the Complexity paragraph under section 3. Neuromap's time-efficiency depends on the chosen (graph) neural network architecture and the number of chosen maximum number of clusters. As we highlight in the results section, the detected communities' quality also depends on the chosen neural network architecture, which allows users to trade off between time-efficiency and community quality.

Reviewer comment (Novelty):
Paragraph 3 of “Introduction”: I don’t think the community detection using GNNs is “under explored”. There are a few works for this task such as [1], [2], [3], [4] and those discussed in the first paragraph of “Related work”.

Author response:
Thank you for pointing out further related works. We have changed the wording in our manuscript.

Reviewer comment (Writing):
Paragraph 1 and 2 of “Background”: I’m still confused about the goal of the map function. For example, what is the “per-step description length”? What is “Huffman code”? I would suggest maybe the author could introduce this in more detail in Appendix.

Author response:
We have revised the manuscript and explain the technical details about the map equation in the appendix as the reviewer suggests. We have also added a figure where we use a coding example to explain how the map equation works.

Reviewer comment (Reproducibility):
In “The map equation goes neural”, I’m still confused about how the model learns S. The paper claims that S is learned via MLP or GNN, but S is a soft cluster assignment matrix. How could we learn a matrix using MLP or GNN? Is it an output from MLP or GNN? If so, what is the input?

Author response:
The input we provide to the MLP or GNN is the node feature matrix. In cases when no node features are available, we use the identity matrix, that is, a one-hot encoding of node identities. The MLP or GNN transforms the node features and outputs a matrix which we interpret as soft cluster assignments. The specific network architecture determines how the graph's adjacecny matrix is used during learning. _Any_ neural network, MLP, GNN, or others can be used to learn a soft cluster assignment matrix. We then use the map equation loss to calculate the quality of this soft cluster assignment matrix and use back propagation to minimise the loss. This way, the GNN clusters the network using the map equation.

Reviewer comment (Novelty):
What is the advantage of the proposed model over traditional ones such as KNN and DeepWalk? The paper discusses the existing approaches in paragraph 2 of “Introduction”, but does not mention the motivation of the proposed one. To me the complexity of KNN is O(nd), where d is the feature dimension, whereas the proposed method has the complexity of O(n^2), which is worse than KNN.

Author response:
The objectives of the map equation, KNN and DeepWalk are quite different.

KNN is an approach that, given d-dimensional data points, can be used to construct a graph where nodes are connected to their K nearest neighbours. However, KNN does not produce a clustering of the nodes.

DeepWalk simulates a large number of short random walks to learn the "context" of a node to perform dimensionality reduction. To detect clusters, however, DeepWalk needs to be combined with another approach such as k-means which does not automatically determine the optimum number of clusters.

The map equation is an objective function for unsupervised community detection. The standard formulation of the map equation seeks to identify groups of nodes that are more densely connected to each other than to the rest. Using the map equation as a loss function for (graph) neural networks enables combining topological features, that is link patterns, with node features for community detection. Importantly, the map equation does not have any hyperparameters. Specifically, it does not require choosing the number of clusters; it is derived from the data using the minimum description length principle.

Reviewer comment (Evaluation):
The results in Table 2 show that DmoN has superior performance than the proposed method in many settings. Why? The paper should discuss this. Also, the proposed method performs badly in “arXiv” dataset, which is also not discussed.

Author response:
We have added additional experimental results where Neuromap performs more competitively on the `ogbn-arxiv` dataset. We have also highlighted the best results in Table 2 more clearly. The experiments run by Tsitsulin et al. involve a peculiar choice of the maximum number of clusters -- 16 -- despite the number of ground-truth clusters in `ogbn-arxiv` being 40. As valuably suggested, we ran DMoN on our overlapping community examples, which evidenced DMoN neither easily finds an optimum number of clusters, nor finds reasonable overlapping communities. When run with a larger number of maximum clusters, 100 and 1000, Neuromap performs competitively on the `ogbn-arxiv` dataset among others, which highlights its ability to automatically infer a meaningful number of clusters and avoid over-partitioning, a significant advantage over existing methods.

Reviewer comment (Presentation):
I would suggest the authors put the caption of the table on the top to make the presentation more formal.

Author response:
We have updated the manuscript and placed the table captions above the tables.