Paper: Mixture Stochastic Block Model for Multi-Group Community Detection in Multiplex Graphs (ICLR-2024)

Reviewer comment (Novelty):
The motivation needs to be strengthened. It is unclear why research on multi-group community detection in multiplex graphs is important. Although some studies on multiplex graphs exist, the scarcity of research on multi-group community detection may be due to the difficulty of the topic or its limited occurrence in the real world. Please clarify the reasons behind the lack of research or provide examples from the real world to support the importance of this study.

Author response:
We appreciate your valuable review feedback. All your comments have been thoroughly considered, and we have implemented the necessary revisions to the paper in accordance with your suggestions. To clarify the motivation behind our model, consider a scenario involving multi-modal data, such as that used in a recommendation system. Users' similarity may be assessed based on information from multiple sources that are not directly comparable. Therefore, we represent this data as a multiplex graph. Now, imagine the task of making recommendations for a user based on the communities to which they belong. One might consider constructing clusters independently for each layer or constructing clusters that take into account all layers simultaneously. The former approach is time-consuming and may not leverage information from other layers effectively, while the latter may yield a consensus division that is not optimal for all layers. Our proposed method strikes a balance between these two models. We first identify a group of layers, and then we cluster each group. This approach allows us to obtain more refined and accurate recommendations for items.

Q1:This is indeed a novel and crucial question, prompted by its significance in our industrial application. The scarcity of research in this area may be attributed, in part, to the absence of open-source datasets that effectively capture the essence of this question. Our modelization approach is versatile, applicable to various multimodal or multiview datasets, aligning with current trends in machine learning clustering fields. We believe that this modelization offers a means to achieve a finer understanding of datasets, enabling the creation of models tailored to the inherent nature of similarities within the data

Reviewer comment (Novelty):
The paper lacks novelty. While the authors mention Stanley et al. (2015) as existing literature on multi-group community in multiplex graphs, the paper only highlights the difference between this work and Stanley et al. (2015) in terms of the learning of layer-to-group assignments, with the former using the EM algorithm instead of k-means.

Author response:
In Stanley (2015), the utilization of K-means on the parameters introduces a potential limitation as K-means assumes well-behaved spherical topological spaces, which might lead to erroneous clusters. In contrast, our model presents a coherent and well-structured pipeline that relies solely on probability distributions. Our optimization process is centered around likelihood optimization. While our assumptions align with those in the stochastic block community, we distinguish ourselves through our logical approach, reliance on probability distributions

Q2: While our modelization may share similarities with Stanley et al. (2015), we distinguish ourselves through a robust mathematical formulation that ensures asymptotic convergence to the best division. Notably, our contribution extends to offering a novel initialization method, a departure from random initialization, which significantly enhances the speed of convergence. Additionally, we contribute theoretical proofs establishing the identifiability and consistency of our model, providing a solid foundation for its reliability. Moreover, our model, with the proposed optimization techniques, easily extends to accommodate other modelizations that consider degree correction and various weighted distributions. This flexibility makes our approach versatile and applicable to a broader range of scenarios, showcasing its adaptability and potential for addressing diverse modeling challenges.

Reviewer comment (Theory):
It is unclear what the challenges are in replacing k-means with the EM algorithm in this context. Are there any differences in the modeling assumptions between this paper and Stanley et al. (2015)?

Author response:
Q3: Indeed, the K-means algorithm has found widespread use in various fields. However, it comes with inherent limitations, most notably its native assumption that clusters have a spherical shape. This assumption does not always align with the actual structure of the data, presenting a significant drawback. In contrast, the EM (Expectation-Maximization) algorithm does not impose any assumptions about the shape of clusters. This flexibility allows the EM algorithm to model complex distributions, making it more adaptable to the diverse structures encountered in real-world data. The EM algorithm generally converges to a good local optimum, and its versatility extends to incorporating any available prior knowledge, adding a layer of flexibility and customization to its application.

Reviewer comment (Experiments):
The experimental section is weak because the paper does not compare the proposed model, which incorporates layer-to-group assignments, with other models that handle multiplex graphs but do not consider layer-to-group assignments [1,2,3,4]. This comparison is necessary to highlight the importance of layer-to-group assignments.

Author response:
Q4: We add Generalized Louvain and Graph Fusion Spectral Clusteting models algorithms to compare with, such that those algorithm are not assuming the existence of multigroup clustering as you proposed.

Reviewer comment (Experiments):
The datasets used in the experiments are relatively small, while real-world data often exhibit large-scale characteristics. What is the time complexity of the proposed algorithm? Can it handle large-scale data?

Author response:
As best of our knowledge, there is a lack of open-source labeled datasets suitable for this application. We aim to publish a new dataset from our field in the future. We augment our experimentation, particularly focusing on demonstrating the model's performance with large graphs. Thanks to our initilalization model that helps to get fast convergence of MSBM.

Q5: The EM and VEM algorithms exhibit sensitivity to initialization. Thank to our spectral initialization that serves as a valuable aid, providing the model with a helpful initialization of clusters that accelerates the convergence process. We conducted new experiments in which we variate the number of layers and the number of vertices. Thank you for bringing this to our attention.

Reviewer comment (Writing):
The authors defined a large number of parameters with superscripts and subscripts for expression accuracy, but the basic architecture of the model is not very clear to the reviewer.

Author response:
Q1: As mentioned earlier, our multiplex graph comprises $|L|$ layers, and our objective is to identify $K$ groups where each group is composed of similar layers. Within each group k, our aim is to cluster its vertices into $Q^k$ clusters. The superscript $k$ is utilized to explicitly convey that each group may have its distinct number of clusters or blocks. Therefore, we apply joint clustering of layers, and clustering of vertices within each group of layers.

Reviewer comment (Writing):
The reviewer’s understanding is that the underlying layers are proposed for the simulation of multiplexity of a graph, where each layer is partitioned into \(K\) groups, and each group is further partitioned into blocks. The reviewer finds this a bit weird and suspects they may have a misunderstanding.

Author response:
To better comprehend our model, consider a multilayer graph where nodes are shared across layers, and each layer possesses its distinct set of edges. This multiplex graph serves as a representation of multiview data, such that information may come from different perspectives of similarity. Our goal is to identify groups of layers that share similar characteristics, a task we refer to as group clustering. Subsequently, we aim to cluster the nodes within each group. Here, a group denotes a collection of layers or graphs, and a block signifies a set of nodes. Therefore, we work with two types of assignment variables: layer-to-group for identifying the group/cluster to which each layer belongs and vertex-to-block for assigning each node within a group to a specific block/cluster.

Reviewer comment (Writing):
The probability of the presence of edges depends on the vertex-to-block assignments and the probability matrices for inter and intra blocks, similar to the traditional SBM. However, the reviewer does not understand how a layer is generated from a group, given that a layer has several groups.

Author response:
Here, a group denotes a collection of layers or graphs, and a block signifies a set of nodes. Therefore, we work with two types of assignment variables: layer-to-group for identifying the group/cluster to which each layer belongs and vertex-to-block for assigning each node within a group to a specific block/cluster. The variable $y_{lk}$ signifies that layer $l$ belongs to group $k$, and $\beta$ represents a vector of probabilities indicating the likelihood of a layer being generated from group $k$.

Reviewer comment (Writing):
The reviewer finds it unclear how to generate a layer from a single group and how to understand vector \(\beta\) and \(y_{lk}\).

Author response:
The variable $y_{lk}$ signifies that layer $l$ belongs to group $k$, and $\beta$ represents a vector of probabilities indicating the likelihood of a layer being generated from group $k$.

Reviewer comment (Writing):
A simple example would be much helpful in understanding, but there is none provided in the paper.

Author response:
Q2: Thank you for your suggestion. It's a relevant idea to include an example that explains the model. We Add now a figure that illustrates the application which will likely enhance the clarity and understanding of your work.

Reviewer comment (Writing):
In Section 3.1, the reviewer asks what is meant by "let's consider a partition of layer \(L\) into \(K\) groups and assume that group \(k\) comprises \(Q^k\) blocks", and inquires about the relationship between group and block in the authors' definition, as well as the purpose of them.

Author response:
Here, a group denotes a collection of layers or graphs, and a block signifies a set of nodes. Therefore, we work with two types of assignment variables: layer-to-group for identifying the group/cluster to which each layer belongs and vertex-to-block for assigning each node within a group to a specific block/cluster. Q1: As mentioned earlier, our multiplex graph comprises $|L|$ layers, and our objective is to identify $K$ groups where each group is composed of similar layers. Within each group k, our aim is to cluster its vertices into $Q^k$ clusters. The superscript $k$ is utilized to explicitly convey that each group may have its distinct number of clusters or blocks. Therefore, we apply joint clustering of layers, and clustering of vertices within each group of layers.

Reviewer comment (Writing):
The reviewer asks whether there is a simple example to help understand the notations in Section 3.1.

Author response:
Q2: Thank you for your suggestion. It's a relevant idea to include an example that explains the model. We Add now a figure that illustrates the application which will likely enhance the clarity and understanding of your work.

Reviewer comment (Experiments):
None of the presented state-of-the-art methods from the related work sections are compared against. Only two naive approaches, K-means and the native SBM, are used as competing methods -- even though their drawbacks were made abundantly clear in the paper. Here, some validation on how well the communities in recovered multiplex SBM describes/generate the observed data in comparison to other community detection methods would be useful instead of only considering the two individual steps of the presented approach independently. Such a comparison might also contain synthetic SBMs with only shared groups as assumed by many existing techniques as a baseline and their failure to capture non-shared groups (highlighting the improvement of the new MSBM method).

Author response:
We are grateful for your insightful review feedback. All your comments have been thoroughly considered, and we have incorporated the necessary revisions into the paper according to your suggestions. We have structured the related work section to provide an overview of existing approaches in multiplex community detection. Our intention is to highlight that none of them adequately addresses the specific question we explore, thereby introducing a novel modeling approach, as detailed in the paper. The utilization of the K-means algorithm serves the purpose of identifying layers that share common clusters. In contrast, the application of the Stochastic Block Model (SBM), which assumes one blocks devision for all layers, demonstrates the model's limitation in defining distinct block distributions for each layer. We compare also the MSBM with Generalized Louvain and Graph Fusion Spectral Clustering, which are method for multiplex clustering. We use the NMI and we add the Adjusted Mutual Information (AMI) to quantify the average performance across all layers and provide a clearer assessment of the model's effectiveness

Reviewer comment (Presentation):
The description of task 5.1 is confusing as it is initially referred to as the vertex-to-block assignment task, but within the same paragraph it is described how the NMI is used to evaluate the layer-to-group association. It is assumed that the method taking all layers into account is compared to 30 independently fitted SBMs, which does not seem like a fair comparison; however, this might be a misunderstanding due to the confusing structure of 5.1 and 5.2 and missing details regarding the actual tasks that are performed.

Author response:
In Section 5.1, we explore the impact of varying block sizes within each group and examine whether the Multiplex Stochastic Block Model (MSBM) can effectively identify the optimal clusters. Given the joint computation of group-layer and vertex-block assignments, evaluating the performance of both aspects is crucial. To elaborate, layers with different block sizes possess distinct structures, and our model aims to accurately define these differences. As mentioned earlier, our method is compared to many multiplex clustering algorithms.  We have revised the text to enhance reader-friendliness. In Section 5.2, unlike Section 5.1 where we considered varying sizes of SBM, we now explore different distributions of SBM within each group. Rather than altering all layer distributions, we opt for simplicity by fixing the intra-distribution and modifying the inter-distribution. To improve clarity, we will revise this section for enhanced readability. Furthermore, we asses the performance of the model when the number of layer and the number of node are variating, showcasing the model's performance and runtime scalability.

Reviewer comment (Experiments):
For task 5.2 multiple multiplex SBMs with different parametrizations should be considered to sample synthetic data for the graph clustering, such as varying block or group sizes and varying blocks across layers. It is also not clear which different graph clusters the task 5.2 tries to recover, as only a single parametrization of an SBM is given. It is guessed that the clustering is not done on the level of multiplex graphs, but on the level of their individual layers as graphs, leading to a very small dataset of just 30 graphs with 3 very distinct inter-block probabilities that might be considered trivial to recover in monolayer graphs. It is not clear how recovering these graphs is an improvement to the state of the art. To highlight potential benefits various other graph clustering methods should be considered. This might also be a misunderstanding because initially there was an expectation of clustering the actual multiplex graphs based on their MSBMs when hearing “Graph clustering”.

Author response:
In Section 5.1, we explore the impact of varying block sizes within each group and examine whether the Multiplex Stochastic Block Model (MSBM) can effectively identify the optimal clusters. Given the joint computation of group-layer and vertex-block assignments, evaluating the performance of both aspects is crucial. To elaborate, layers with different block sizes possess distinct structures, and our model aims to accurately define these differences. As mentioned earlier, our method is compared to many multiplex clustering algorithms.  We have revised the text to enhance reader-friendliness. In Section 5.2, unlike Section 5.1 where we considered varying sizes of SBM, we now explore different distributions of SBM within each group. Rather than altering all layer distributions, we opt for simplicity by fixing the intra-distribution and modifying the inter-distribution. To improve clarity, we will revise this section for enhanced readability. Furthermore, we asses the performance of the model when the number of layer and the number of node are variating, showcasing the model's performance and runtime scalability.

Reviewer comment (Experiments):
The impact of larger graphs on computational complexity is not explored, as currently only small graphs of size 100 are considered. This is especially crucial as other competing approaches like MLSBMs suffer from exponentially large parameter space, and such drawbacks should be highlighted on large graphs. Runtime or a similar measure should be included in the results. Additionally, the impact of varying levels of noise on recovering the clusters would also be an interesting addition to the synthetic experiments.

Author response:
Furthermore, we asses the performance of the model when the number of layer and the number of node are variating, showcasing the model's performance and runtime scalability.

Reviewer comment (Evaluation):
Regarding the real-world application, it seems that task 1 was to differentiate between real datapoints and randomly generated data points, which does not seem to be a suitable benchmark. Task 2 appears to differentiate the article-article graphs from two different datasets; however, the meaning of the two rows for each dataset in Table 4 is unclear, as a single group-to-layer or vertex-to-block classification result would be expected. Clarification is needed on how the mutual information is used in this task. Additionally, the reason why the adjusted mutual information (adjusted to random chance) was not used should be explained.

Author response:
In real-world applications, the primary objective of Task 1 is to illustrate the method's capability to discern random distributions within actual datasets. We underscore its significance in scenarios where random data may be present in real applications. Concerning Task 2, we introduce new line for model performance on dataset.

Reviewer comment (Evaluation):
Despite the well-motivated need for methods such as this, the promising underlying idea, and sound theoretical foundation, it is not clear that the presented methods actually achieve their goals or improve the state-of-the-art. The experimental section does not quantify the quality of the recovered (Mixture) Multiplex SBM, as it only shows that it performs better in the two individual steps compared to very naïve approaches.

Author response:
We are grateful for your insightful review feedback. All your comments have been thoroughly considered, and we have incorporated the necessary revisions into the paper according to your suggestions. We have structured the related work section to provide an overview of existing approaches in multiplex community detection. Our intention is to highlight that none of them adequately addresses the specific question we explore, thereby introducing a novel modeling approach, as detailed in the paper. The utilization of the K-means algorithm serves the purpose of identifying layers that share common clusters. In contrast, the application of the Stochastic Block Model (SBM), which assumes one blocks devision for all layers, demonstrates the model's limitation in defining distinct block distributions for each layer. We compare also the MSBM with Generalized Louvain and Graph Fusion Spectral Clustering, which are method for multiplex clustering. We use the NMI and we add the Adjusted Mutual Information (AMI) to quantify the average performance across all layers and provide a clearer assessment of the model's effectiveness

Reviewer comment (Experiments):
The actual performed task in the experiment section should be described more clearly, including a clear definition of what is referred to as graph clustering in the context of multiplex graphs.

Author response:
Q1:In our case, the graph clustering is used to mean grouping the similar layers of multiplex graphs. We make a new more clear version of the experiment set for this point.

Reviewer comment (Experiments):
State-of-the-art methods should be included, and the recovered multiplex SBMs should be directly compared rather than only comparing the results of individual steps of the MSBM.

Author response:
Q2: We now compare our MSBM algorithm to single SBM, Generalized Louvain and GFSC, all this comparison are shown in the experiment section.

Reviewer comment (Experiments):
A larger synthetic dataset should be considered with respect to the number of graphs, nodes, blocks, and different parameter sets to make the analysis seem less exemplary.

Author response:
Q3: We set experiments on block size and block distribution. We add a new experiments on number of layers and the vertices size.

Reviewer comment (Evaluation):
More metrics than just NMI should be included, as well as a runtime comparison.

Author response:
Q4:  We now have added Adjusted Mutual Information AMI for clustering comparison.

Reviewer comment (Reproducibility):
No statement about reproducibility can be made as no code was provided.

Author response:
Q5: Maybe you did not pay attention, but we provide the code that we used in our experimentation.

Reviewer comment (Novelty):
It is not completely clear why similar results could not be achieved through consensus-based approaches that relax consensus across all layers to only consensus across groups of layers exhibiting similar community structures, and allowing disagreement between layers of different groups, akin to the approach in this paper.

Author response:
Q7: Our application could potentially be modeled using a consensus-based technique, incorporating the constraints you mentioned. However, to the best of our knowledge, there is currently no existing model capable of achieving this. Despite this, the consensus model faces a significant drawback: once the graphs are subdivided, the graph is constrained to reflect only the information division, potentially inadequately capturing the underlying graph's data structure. Additionally, our model operates as a generative model, enabling the creation of data with inherent structural information.

Reviewer comment (Experiments):
Most of the experiments are synthetic, and the only real world data is sort of manually constructed. It would be better if this model is application driven by more and larger real datasets.

Author response:
To the best of our knowledge, no existing real dataset has been constructed with labels for both groups and blocks.

Reviewer comment (Evaluation):
The variational EM algorithm only guarantees to converge to a local minimizer, and the optimization objective is seen to be highly non-convex. How different are the minimizers that the algorithm converge to at different runs? And how would you interpret these different minimizers?

Author response:
Q1: As you rightly pointed out, the function is highly non-convex, and the convergence of the algorithm relies on the initialization of assignment variables. A well-informed initialization, guided by a good understanding of the groups and blocks, can lead to faster convergence. Conversely, random initialization may require more iterations to achieve optimal clustering. In any scenario, the multilayer structure proves advantageous in the assignment process by providing more samples from the same distribution. According to the law of large numbers, this abundance of samples contributes to faster convergence compared to a single-layer block model. Any discrepancies in cluster assignment between the iteration may stem from an initialization that is distant from the local minimum of the optimal solution. Thanks to our initialization model that helps to get faster and high quality convergence. It could also arise from the approximation of the posterior distribution using the variational method. We encourage users of this model to thoroughly verify its underlying hypotheses and assess whether the data aligns with these assumptions. Successful validation of these hypotheses would enhance the model's utility.

Reviewer comment (Experiments):
In the case of graph related modeling, it often appears that a simple spectral initialization can already give a pretty decent result. How much improvement is actually achieved by using variational EM after the spectral initialization? And without using spectral initialization, is the variational EM still capable of numerically finding a good local minimizer given sufficiently many iterations? If yes, how much iterations is saved by using spectral initialization?

Author response:
Q2: In this paper, the spectral model serves as an initialization method due to its utility and speed. In all the experiments conducted, it provides a valuable indication of the assignment variables, although not a perfect partitions with less performance after VEM iteration. Despite being a generative model capable of clustering and generating data from the same distribution, the stochastic block model, in contrast to the spectral model, may yield to better result. The difference generaly depends on the type of data and the initialization of each model, notably the spectral model. Even when the spectral model provides an optimal assignment, our model possesses the ability to generate data, which is a capability lacking in the spectral model. We also explore scenarios where the initialization is randomly generated, and the variational model consistently converges to a satisfactory local minimum, although requiring more iterations, as observed in our experiments. While there are no theoretical guarantees regarding time savings from spectral initialization, our empirical results demonstrate that it is more than 10/20 times faster. It's important to note that the actual time savings may vary depending on the structure of the data.