Original: New Frontiers in Graph Autoencoders: Joint Community Detection and Link Prediction
Revised: New Frontiers in Graph Autoencoders: Joint Community Detection and Link Prediction

Original: Abstract
Revised: Abstract

Original: Graph autoencoders (GAE) and variational graph autoencoders (VGAE) emerged as powerful methods for link prediction (LP).
Revised: Graph autoencoders (GAE) and variational graph autoencoders (VGAE) emerged as powerful methods for link prediction (LP).

Original: Their performances are less impressive on community detection (CD), where they are often outperformed by simpler alternatives such as the Louvain method.
Revised: Their performances are less impressive on community detection (CD), where they are often outperformed by simpler alternatives such as the Louvain method.

Original: It is still unclear to what extent one can improve CD with GAE and VGAE, especially in the absence of node features.
Revised: It is still unclear to what extent one can improve CD with GAE and VGAE, especially in the absence of node features.

Original: It is moreover uncertain whether one could do so while simultaneously preserving good performances on LP in a multi-task setting.
Revised: It is moreover uncertain whether one could do so while simultaneously preserving good performances on LP in a multi-task setting.

Original: In this workshop paper, summarizing results from our journal publication, we show that jointly addressing these two tasks with high accuracy is possible.
Revised: In this workshop paper, summarizing results from our journal publication [44], we show that jointly addressing these two tasks with high accuracy is possible.

Original: For this purpose, we introduce a communitypreserving message passing scheme, doping our GAE and VGAE encoders by considering both the initial graph and Louvain-based prior communities when computing embedding spaces.
Revised: For this purpose, we introduce a communitypreserving message passing scheme, doping our GAE and VGAE encoders by considering both the initial graph and Louvain-based prior communities when computing embedding spaces.

Original: Inspired by modularity-based clustering, we further propose novel training and optimization strategies specifically designed for joint
Revised: Inspired by modularity-based clustering, we further propose novel training and optimization strategies specifically designed for joint

Original: LP and CD.
Revised: LP and CD.

Original: We demonstrate the empirical effectiveness of our approach, referred to as Modularity-Aware GAE and VGAE, on various real-world graphs.
Revised: We demonstrate the empirical effectiveness of our approach, referred to as Modularity-Aware GAE and VGAE, on various real-world graphs.

Original:  Introduction
Revised:  Introduction

Original: Extracting relevant information from nodes of a graph is essential to tackle a wide range of machine learning problems [9, 13, 14, 56].
Revised: Extracting relevant information from nodes of a graph is essential to tackle a wide range of machine learning problems [9, 13, 14, 57].

Original: This includes link prediction (LP) [26, 29], which consists in inferring the presence of new or unobserved edges between node pairs and community detection (CD) [4, 9], which consists in clustering nodes into similar groups, according to a chosen similarity metric.
Revised: This includes link prediction (LP) [26, 29], which consists in inferring the presence of new or unobserved edges between node pairs, and community detection (CD) [4, 9], which consists in clustering nodes into similar groups, according to a chosen similarity metric.

Original: To address such problems significant efforts have recently been devoted to the development of node embedding methods [13, 14, 23].
Revised: To address such problems, significant efforts have recently been devoted to the development of node embedding methods [13, 14, 23].

Original: These methods aim to learn vectorial representations of nodes in an embedding space where node positions should reflect and summarize the initial graph structure.
Revised: These methods aim to learn vectorial representations of nodes in an embedding space where node positions should reflect and summarize the initial graph structure.

Original: They assess the probability of a new edge between two nodes, or their likelihood of belonging to the same community, by evaluating the proximity of these nodes in the embedding space [7, 24, 52].
Revised: They assess the probability of a new edge between two nodes, or their likelihood of belonging to the same community, by evaluating the proximity of these nodes in the embedding space [7, 24, 53].

Original: In particular, graph autoencoders (GAE) and variational graph autoencoders (VGAE) [24, 48, 52, 53] recently emerged as two powerful families of node embedding methods.
Revised: In particular, graph autoencoders (GAE) and variational graph autoencoders (VGAE) [24, 49, 53, 54] recently emerged as two powerful families of node embedding methods.

Original: Both methods rely on an encoding-decoding strategy that consists in encoding nodes into an embedding space from which decoding , i.e., reconstructing the original graph, should ideally be possible.
Revised: Both methods rely on an encoding-decoding strategy that consists in encoding nodes into an embedding space from which decoding , i.e., reconstructing the original graph, should ideally be possible.

Original: Originally mainly designed for LP (at least in their modern formulation leveraging graph neural networks (GNN) [24]), the effectiveness of GAE and VGAE models and their extensions on this task has been experimentally confirmed [12, 16, 18, 36, 43, 44, 49].
Revised: Originally mainly designed for LP (at least in their modern formulation leveraging graph neural networks (GNN) [24]), the effectiveness of GAE and VGAE models and their extensions on this task has been experimentally confirmed [12, 16, 18, 36, 43, 45, 50].

Original: On the other hand, several studies [7, 8, 40, 41] have pointed out their limitations on CD.
Revised: On the other hand, several studies [7, 8, 40, 41] have pointed out their limitations on CD.

Original: These studies emphasized that GAEs and VGAEs are often outperformed by simpler CD alternatives, such as the popular Louvain method [4].
Revised: These studies emphasized that GAEs and VGAEs are often outperformed by simpler CD alternatives, such as the popular Louvain method [4].

Original: The question of how to improve
Revised: The question of how to improve

Original: CD with GAEs and VGAEs remains incompletely addressed, especially in the absence of node features.
Revised: CD with GAEs and VGAEs remains incompletely addressed, especially in the absence of node features.

Original: Moreover, it is still unclear to which extent one can improve CD with these models without simultaneously deteriorating LP, and jointly address these two problems.
Revised: Moreover, it is still unclear to which extent one can improve CD with these models without simultaneously deteriorating LP, and jointly address these two problems.

Original: These questions are highly relevant in practice, as learning node embedding spaces suitable for multi-task settings leads to consistent inference between tasks and saves costs in real-world applications.
Revised: These questions are highly relevant in practice, as learning node embedding spaces suitable for multi-task settings leads to consistent inference between tasks and saves costs in real-world applications.

Original: This paper 1 presents several contributions pushing the frontiers of GAEs and VGAEs, and showing that jointly addressing CD and LP with high accuracy is possible with these models.
Revised: This paper 2 presents several contributions pushing the frontiers of GAEs and VGAEs, and showing that jointly addressing CD and LP with high accuracy is possible with these models.

Original: After reviewing key concepts in Section 2, we explain why GAEs and VGAEs underperform on CD in Section 3.
Revised: After reviewing key concepts in Section 2, we explain why GAEs and VGAEs underperform on CD in Section 3.

Original: We simultaneously introduce Modularity-Aware GAE and VGAE , our solution leveraging modularitybased clustering concepts [4, 5, 45] to improve CD while preserving the ability to identify missing edges in LP.
Revised: We simultaneously introduce Modularity-Aware GAE and VGAE , our solution leveraging modularitybased clustering concepts [4, 5, 46] to improve CD while preserving the ability to identify missing edges in LP.

Original: We report an in-depth evaluation2 of our method in Section 4, and conclude in Section 5.
Revised: We report an in-depth evaluation of our method in Section 4, and conclude in Section 5.

Original:  Preliminaries
Revised:  Preliminaries

Original: We consider an undirected graph G =
Revised: We consider an undirected graph G =

Original: ( V , E ) with n nodes and m edges.
Revised: ( V , E ) with n nodes and m edges.

Original: We denote its n × n adjacency matrix by A.
Revised: We denote its n × n adjacency matrix by A.

Original: Each node i ∈ V is equipped with a feature vector x i ∈ R f .
Revised: Each node i ∈ V is equipped with a feature vector x i ∈ R f .

Original: We denote the n × f matrix having x i vectors as rows by X. For a featureless graph, we set X
Revised: We denote the n × f matrix having x i vectors as rows by X. For a featureless graph, we set X

Original: = I n , the identity matrix.
Revised: = I n , the identity matrix.

Original: GAE and VGAE.
Revised: GAE and VGAE.

Original: The term GAE refers to a family of unsupervised two-component models learning node embedding spaces in the absence of node labels [23, 24, 48, 53].
Revised: The term GAE refers to a family of unsupervised two-component models learning node embedding spaces in the absence of node labels [23, 24, 49, 54].

Original: The first component is the encoder , a parameterized function processing A and X , and mapping each node i ∈ V to an embedding vector z i ∈ R d , with d ≪ n .
Revised: The first component is the encoder , a parameterized function processing A and X , and mapping each node i ∈ V to an embedding vector z i ∈ R d , with d ≪ n .

Original: In practice, a GNN [13, 25, 56] often acts as the encoder, i.e., Z = GNN ( A, X ) , with Z the n × d matrix having z i vectors as rows.
Revised: In practice, a GNN [13, 25, 57] often acts as the encoder, i.e., Z = GNN ( A, X ) , with Z the n × d matrix having z i vectors as rows.

Original: The second component is the decoder , estimating an adjacency matrix ˆ A from embedding vectors: ˆ A =
Revised: The second component is the decoder , estimating an adjacency matrix ˆ A from embedding vectors: ˆ A =

Original: Decoder ( Z ) .
Revised: Decoder ( Z ) .

Original: Decoders can be neural networks, or simpler functions, e.g., based on inner products between z i vectors [24, 28, 37, 53].
Revised: Decoders can be neural networks, or simpler functions, e.g., based on inner products between z i vectors [24, 28, 37, 54].

Original: When training a GAE, one wishes to learn z i vectors from which reconstructing G should be possible.
Revised: When training a GAE, one wishes to learn z i vectors from which reconstructing G should be possible.

Original: Intuitively, this would indicate that the embedding space preserves some important information about G .
Revised: Intuitively, this would indicate that the embedding space preserves some important information about G .

Original: For this purpose, model weights are trained via gradient descent minimization [10] of a reconstruction loss , usually a cross entropy [24], evaluating the similarity between ˆ A and A.
Revised: For this purpose, model weights are trained via gradient descent minimization [10] of a reconstruction loss , usually a cross entropy [24], evaluating the similarity between ˆ A and A.

Original: Introduced as probabilistic extensions of GAEs, VGAE models associate z i vectors with distributions.
Revised: Introduced as probabilistic extensions of GAEs, VGAE models associate z i vectors with distributions.

Original: Notably, in the seminal VGAE from Kipf and Welling [24], each vector z i ∼ N ( µ i , Σ i ) .
Revised: Notably, in the seminal VGAE from Kipf and Welling [24], each vector z i ∼ N ( µ i , Σ i ) .

Original: Their model incorporates two GNN encoders processing both A and X : one of them learns mean vectors µ i ∈ R d , and the other learns variance matrices Σ
Revised: Their model incorporates two GNN encoders processing both A and X : one of them learns mean vectors µ i ∈ R d , and the other learns variance matrices Σ

Original: i ∈ R d × d , for all i ∈ V .
Revised: i ∈ R d × d , for all i ∈ V .

Original: Moreover, instead of a reconstruction loss, they optimize the variational evidence lower bound (ELBO) of the model’s likelihood [22], using gradient ascent.
Revised: Moreover, instead of a reconstruction loss, they optimize the variational evidence lower bound (ELBO) of the model’s likelihood [22], using gradient ascent.

Original: Besides constituting promising generative models [20, 30, 46], variants of VGAEs also turned out to be effective alternatives to GAEs in some LP and CD tasks [8, 16, 24, 40, 42, 43].
Revised: Besides constituting promising generative models [20, 30, 47], variants of VGAEs also turned out to be effective alternatives to GAEs in some LP and CD tasks [8, 16, 24, 40, 42, 43].

Original: Evaluation.
Revised: Evaluation.

Original: Over the past years, LP 3 has become the most prominent way to evaluate the quality of embedding vectors learned from a GAE or VGAE [12, 15, 16, 18, 19, 36, 43, 44, 49].
Revised: Over the past years, LP 3 has become the most prominent way to evaluate the quality of embedding vectors learned from a GAE or VGAE [12, 15, 16, 18, 19, 36, 43, 45, 50].

Original: Previous work widely confirmed the effectiveness of GAEs and VGAEs on this task.
Revised: Previous work widely confirmed the effectiveness of GAEs and VGAEs on this task.

Original: Their performances are less impressive on CD 3 , another important graph problem with numerous applications [6, 17, 34, 47, 50].
Revised: Their performances are less impressive on CD 3 , another important graph problem with numerous applications [6, 17, 34, 48, 51].

Original: In the presence of node embedding representations z i , CD boils down to the common problem of clustering n vectors, e.g., via a k -means [33] in the embedding space.
Revised: In the presence of node embedding representations z i , CD boils down to the common problem of clustering n vectors, e.g., via a k -means [33] in the embedding space.

Original: Nonetheless, concurring work [7, 8, 40, 41] recently pointed out the limitation of this approach for GAEs and VGAEs, and its lower performance w.r.t.
Revised: Nonetheless, concurring work [7, 8, 40, 41] recently pointed out the limitation of this approach for GAEs and VGAEs, and its lower performance w.r.t.

Original: simpler CD alternatives, such as the popular Louvain method learning communities by iteratively maximizing the density-based modularity value in the graph [4].
Revised: simpler CD alternatives, such as the popular Louvain method learning communities by iteratively maximizing the density-based modularity value in the graph [4].

Original: While recent studies aimed to address the underwhelming performance of GAEs and VGAEs on CD, they still suffer from limitations that motivate our work.
Revised: While recent studies aimed to address the underwhelming performance of GAEs and VGAEs on CD, they still suffer from limitations that motivate our work.

Original: Firstly, several studies [7, 8, 27] considered clustering-oriented probabilistic priors for VGAEs (such as Gaussian mixtures in VGAECD [7] and
Revised: Firstly, several studies [7, 8, 27] considered [Figure] C o mm un i t y - P r e s e r v

Original: VGAECD-OPT [8]), that cannot be transposed to the deterministic GAE setting.
Revised: i ng E n c ode r [Figure] r [Figure]

Revised: Reconstructed

Revised: [Figure]

Revised: M odu l a r i t y - I n s p

Revised: i r ed

Revised: D e c ode r

Revised: [Figure]

Revised: Text of Section clustering-oriented probabilistic priors for VGAEs (such as Gaussian mixtures in VGAECD [7] and

Revised: VGAECD-OPT [8]), that cannot be transposed to the deterministic GAE setting.

Original: Secondly, a closer look at these models reveals that their empirical gains mostly stem from the addition of node features.
Revised: Secondly, a closer look at these models reveals that their empirical gains mostly stem from the addition of node features.

Original: They offer little advantage when features are absent (see Table 4).
Revised: They offer little advantage when features are absent (see Table 4).

Original: Other studies did not consider featureless graphs at all [18, 19, 36, 37, 44].
Revised: Other studies did not consider featureless graphs at all [18, 19, 36, 37, 45].

Original: This motivates the need to investigate GAE/VGAE-based
Revised: This motivates the need to investigate GAE/VGAE-based

Original: CD on featureless graphs.
Revised: CD on featureless graphs.

Original: Thirdly, previous studies did not try to preserve good performances on
Revised: Thirdly, previous studies did not try to preserve good performances on

Original: LP [7, 8, 27, 52].
Revised: LP [7, 8, 27, 53].

Original: It is still uncertain whether one can jointly address LP and CD with accuracy in multi-task settings, which, as argued in the Introduction, is highly relevant in practice.
Revised: It is still uncertain whether one can jointly address LP and CD with accuracy in multi-task settings, which, as argued in the Introduction, is highly relevant in practice.

Original: In conclusion, the question of improving CD with GAEs and VGAEs remains incompletely addressed.[Figure] C o mm un i t y - P r e s e r v
Revised: In conclusion, the question of improving CD with GAEs and VGAEs remains incompletely addressed.

Original: i ng E n c ode

Original: [Figure]

Original:  Modularity-Aware GAE and VGAE for Joint LP and CD
Revised:  Modularity-Aware GAE and VGAE for Joint LP and CD

Original: To address these limitations, we introduce our Modularity-Aware GAE/VGAE, illustrated in Figure 1.
Revised: To address these limitations, we introduce our Modularity-Aware GAE/VGAE, illustrated in Figure 1.

Original: Community-Based Encoders.
Revised: Community-Based Encoders.

Original: Firstly, we argue that most GAEs/VGAEs leverage encoders that do not specifically aim to capture graph communities.
Revised: Firstly, we argue that most GAEs/VGAEs leverage encoders that do not specifically aim to capture graph communities.

Original: This includes graph convolutional networks (GCN) [25], which remain the most popular encoders in practice [12, 16, 18, 19, 36, 44], and encoders identifying clusters from features rather than the graph [8, 52].
Revised: This includes graph convolutional networks (GCN) [25], which remain the most popular encoders in practice [12, 16, 18, 19, 36, 45], and encoders identifying clusters from features rather than the graph [8, 53].

Original: Modularity-Aware GAE and VGAE overcome this issue by incorporating a community-based encoder .
Revised: Modularity-Aware GAE and VGAE overcome this issue by incorporating a community-based encoder .

Original: Specifically, we first obtain a partition of the node set using the Louvain method [4] and store it in an n × n membership matrix A c , defined as ( A c ) ij = 1 if nodes i ̸
Revised: Specifically, we first obtain a partition of the node set using the Louvain method [4] and store it in an n × n membership matrix A c , defined as ( A c ) ij = 1 if nodes i ̸

Original: = j are in the same community, and 0 otherwise.
Revised: = j are in the same community, and 0 otherwise.

Original: Then, when learning embedding vectors, we leverage this partition as a prior signal , from which the encoder should benefit, but also have the ability to deviate.
Revised: Then, when learning embedding vectors, we leverage this partition as a prior signal , from which the encoder should benefit, but also have the ability to deviate.

Original: Formally, we replace the Z =
Revised: Formally, we replace the Z =

Original: GNN ( A, X ) component 4 by: Z = GNN ( A + λA s , X ) , where λ ∈ R + and s ∈ N + are hyperparameters, and where A s is a s -regular sparsified 5 version of A c .
Revised: GNN ( A, X ) component 4 by: Z = GNN ( A + λA s , X ) , where λ ∈ R + and s ∈ N + are hyperparameters, and where A s is a s -regular sparsified 5 version of A c .

Original: This change alters the GNN message passing scheme .
Revised: This change alters the GNN message passing scheme .

Original: Nodes will now aggregate information from their neighbors and some nodes of their prior community ( λ balances the importance of these two information sources).
Revised: Nodes will now aggregate information from their neighbors and some nodes of their prior community ( λ balances the importance of these two information sources).

Original: Therefore, nodes from the same prior community will tend to have more similar embedding vectors than with a standard GAE or VGAE.
Revised: Therefore, nodes from the same prior community will tend to have more similar embedding vectors than with a standard GAE or VGAE.

Original: Besides its simplicity and good performance on CD [40], our justification for using Louvain as a prior is threefold.
Revised: Besides its simplicity and good performance on CD [40], our justification for using Louvain as a prior is threefold.

Original: Firstly, it automatically selects the relevant number of prior communities to consider.
Revised: Firstly, it automatically selects the relevant number of prior communities to consider.

Original: Secondly, it runs in O ( n log n ) time [4] and, therefore, scales to graphs with millions of nodes.
Revised: Secondly, it runs in O ( n log n ) time [4] and, therefore, scales to graphs with millions of nodes.

Original: Thirdly, it optimizes a modularity criterion that complements the encoding-decoding paradigm.
Revised: Thirdly, it optimizes a modularity criterion that complements the encoding-decoding paradigm.

Original: We will show in Section 4 that learning representations from complementary criteria is beneficial.
Revised: We will show in Section 4 that learning representations from complementary criteria is beneficial.

Original: Nonetheless, our framework remains valid for any alternative method providing prior communities.
Revised: Nonetheless, our framework remains valid for any alternative method providing prior communities.

Original: Modularity-Inspired Losses.
Revised: Modularity-Inspired Losses.

Original: Previous models were also trained in a fashion that, by design, favors LP over CD.
Revised: Previous models were also trained in a fashion that, by design, favors LP over CD.

Original: The cross entropy and ELBO losses involve the reconstruction of node pairs from the embedding space [24].
Revised: The cross entropy and ELBO losses involve the reconstruction of node pairs from the embedding space [24].

Original: However, a good reconstruction of local pairwise connections does not necessarily imply a good reconstruction of the global community structure [31, 54].
Revised: However, a good reconstruction of local pairwise connections does not necessarily imply a good reconstruction of the global community structure [31, 55].

Original: Consequently, in Modularity-Aware GAE (respectively, VGAE), we minimize (resp., maximize), using gradient descent (resp., gradient ascent), an alternative function that subtracts (resp., adds) the following global regularizer to the cross entropy (resp., ELBO) term:
Revised: Consequently, in Modularity-Aware GAE (respectively, VGAE), we minimize (resp., maximize), using gradient descent (resp., gradient ascent), an alternative function that subtracts (resp., adds) the following global regularizer to the cross entropy (resp., ELBO) term:

Original: [Equation]
Revised: [Equation]

Original:  with d i the degree [13] of node i ∈ V and two hyperparameters β ∈ R + and γ ∈ R + .
Revised: , with d i the degree [13] of node i ∈ V and two hyperparameters β ∈ R + and γ ∈ R + .

Original: A soft and differentiable version of the modularity [35] (independent of any ground truth community), this regularizer aims to push closer vectors
Revised: A soft and differentiable version of the modularity [35] (independent of any ground truth community), this regularizer aims to push closer vectors

Original: z i of densely connected parts of the graph, and, therefore, to permit a k -means-based detection of communities with higher density.
Revised: z i of densely connected parts of the graph, and, therefore, to permit a k -means-based detection of communities with higher density.

Original: Several studies out of the
Revised: Several studies out of the

Original: GAE/VGAE scope emphasized the effectiveness of comparable approaches for learning communitypreserving representations [32, 54, 55].
Revised: GAE/VGAE scope emphasized the effectiveness of comparable approaches for learning communitypreserving representations [32, 55, 56].

Original: On the other hand, the remaining presence of the local cross entropy (resp., ELBO) in our optimized loss aims to preserve good performances on LP.
Revised: On the other hand, the remaining presence of the local cross entropy (resp., ELBO) in our optimized loss aims to preserve good performances on LP.

Original: β balances the relative importance of the global regularizer.
Revised: β balances the relative importance of the global regularizer.

Original: γ regulates the magnitude of ∥
Revised: γ regulates the magnitude of ∥

Original: z i − z j ∥ 22 in the exponential term, which tends to 1 when z i and z j get closer, and to 0 when they move apart.
Revised: z i − z j ∥ 22 in the exponential term, which tends to 1 when z i and z j get closer, and to 0 when they move apart.

Original: Hyperparameter Selection.
Revised: Hyperparameter Selection.

Original: GAEs and VGAEs include several important hyperparameters such as dropout and learning rates [24] (our models also introduce λ , s , β , and γ ).
Revised: GAEs and VGAEs include several important hyperparameters such as dropout and learning rates [24] (our models also introduce λ , s , β , and γ ).

Original: In previous studies, their selection procedure was sometimes solely based on LP validation sets [40, 41].
Revised: In previous studies, their selection procedure was sometimes solely based on LP validation sets [40, 41].

Original: However, optimal values for CD might differ from those for LP, partly explaining the low performance on CD.
Revised: However, optimal values for CD might differ from those for LP, partly explaining the low performance on CD.

Original: In this paper, we consider an alternative hyperparameter selection procedure.
Revised: In this paper, we consider an alternative hyperparameter selection procedure.

Original: As detailed in Appendix A, the hyperparameters selected for our models are chosen by maximizing the average of: (1) an Area under the ROC Curve (AUC) score computed on an LP validation set, and (2) the modularity score computed from the communities extracted from final vectors
Revised: As detailed in Appendix A, the hyperparameters selected for our models are chosen by maximizing the average of: (1) an Area under the ROC Curve (AUC) score computed on an LP validation set, and (2) the modularity score computed from the communities extracted from final vectors

Original: z i , via a k -means.
Revised: z i , via a k -means.

Original: We expect this dual criterion to identify hyperparameters that will be jointly relevant for LP and CD in a multi-task setting.
Revised: We expect this dual criterion to identify hyperparameters that will be jointly relevant for LP and CD in a multi-task setting.

Original:  Experimental Evaluation
Revised:  Experimental Evaluation

Revised: We now report results from an in-depth experimental evaluation of our method.

Revised: Our code is available on GitHub: https://github.com/GuillaumeSalhaGalvan/modularity_aware_gae .

Original: Setting.
Revised: Setting.

Original: For evaluation, we consider a “pure” CD problem, as well as a multi-task LP/CD problem, on seven graphs of various origins and sizes (from 1124 to 2.5 million nodes).
Revised: For evaluation, we consider a “pure” CD problem, as well as a multi-task LP/CD problem, on seven graphs of various origins and sizes (from 1124 to 2.5 million nodes).

Original: For both problems and all graphs, we compare our approach to 12 baselines, including the Louvain method [4], standard
Revised: For both problems and all graphs, we compare our approach to 12 baselines, including the Louvain method [4], standard

Original: GAE/VGAE models [24] with varying encoders, and existing extensions of GAEs/VGAEs for CD.
Revised: GAE/VGAE models [24] with varying encoders, and existing extensions of GAEs/VGAEs for CD.

Original: For brevity, we report technical details on tasks, datasets, models, and hyperparameters in Appendix A.
Revised: For brevity, we report technical details on tasks, datasets, models, and hyperparameters in Appendix A.

Original: Results on CD.
Revised: Results on CD.

Original: CD results from Table 3 confirm the discussed limitations of standard GAE/VGAE, which Louvain outperforms on 5 of 7 featureless graphs (e.g., +7.45 Adjusted Mutual Information (AMI) points for Louvain on Pubmed).
Revised: CD results from Table 3 confirm the discussed limitations of standard GAE/VGAE, which Louvain outperforms on 5 of 7 featureless graphs (e.g., +7.45 Adjusted Mutual Information (AMI) points for Louvain on Pubmed).

Original: On the contrary, our Modularity-Aware GAE/VGAE almost always surpass the Louvain method, and the use of a standard GAE/VGAE (e.g., with a top 21.64%
Revised: On the contrary, our Modularity-Aware GAE/VGAE almost always surpass the Louvain method, and the use of a standard GAE/VGAE (e.g., with a top 21.64%

Original: AMI on the largest Album graph).
Revised: AMI on the largest Album graph).

Original: Interestingly, combining Louvain and a GAE/VGAE into our
Revised: Interestingly, combining Louvain and a GAE/VGAE into our

Original: Modularity-Aware models is beneficial even when the GAE/VGAE initially outperforms Louvain (e.g., for Cora-Large).
Revised: Modularity-Aware models is beneficial even when the GAE/VGAE initially outperforms Louvain (e.g., for Cora-Large).

Original: This confirms that modularity-based clustering à la Louvain complements the encoding-decoding paradigm, and that leveraging complementary criteria is empirically beneficial.
Revised: This confirms that modularity-based clustering à la Louvain complements the encoding-decoding paradigm, and that leveraging complementary criteria is empirically beneficial.

Original: We also compare favorably to other baselines in most experiments (e.g., +2.11 AMI points w.r.t.
Revised: We also compare favorably to other baselines in most experiments (e.g., +2.11 AMI points w.r.t.

Original: VGAECD-OPT [8] on Cora with features), with or without the addition of node features.
Revised: VGAECD-OPT [8] on Cora with features), with or without the addition of node features.

Original: Figureprovides a visualization of node embedding representations learned by our models.
Revised: Figureprovides a visualization of node embedding representations learned by our models.

Original: Results on Multi-Task CD/LP.
Revised: Results on Multi-Task CD/LP.

Original: We now assess whether improving CD implies deteriorating the effectiveness on LP.
Revised: We now assess whether improving CD implies deteriorating the effectiveness on LP.

Original: The last columns of Table 3 confirm the ability of Modularity-Aware GAE/VGAE to preserve good performances on LP (we achieve comparable scores w.r.t standard GAE/VGAE on all graphs).
Revised: The last columns of Table 3 confirm the ability of Modularity-Aware GAE/VGAE to preserve good performances on LP (we achieve comparable scores w.r.t standard GAE/VGAE on all graphs).

Original: While performances on CD decrease slightly w.r.t.
Revised: While performances on CD decrease slightly w.r.t.

Original: pure CD (an expected result, as some edges are masked during training for the purpose of LP), we continue to outperform baselines in most experiments.
Revised: pure CD (an expected result, as some edges are masked during training for the purpose of LP), we continue to outperform baselines in most experiments.

Original: This demonstrates the effectiveness of our approach at jointly addressing CD and LP.
Revised: This demonstrates the effectiveness of our approach at jointly addressing CD and LP.

Original: Discussion on Model Components.
Revised: Discussion on Model Components.

Original: For most models, using a linear encoder [42] gives competitive
Revised: For most models, using a linear encoder [42] gives competitive

Original: LP/CD results w.r.t.
Revised: LP/CD results w.r.t.

Original: a 2-layer GCN [24].
Revised: a 2-layer GCN [24].

Original: Also, VGAE models often outperform their GAE counterparts, even though scores are relatively close.
Revised: Also, VGAE models often outperform their GAE counterparts, even though scores are relatively close.

Original: Our proposed hyperparameter selection procedure had a noticeable impact on the choices of λ , β , γ , and s , as well as on the required number of training iterations, which we illustrate in Figure 3.
Revised: Our proposed hyperparameter selection procedure had a noticeable impact on the choices of λ , β , γ , and s , as well as on the required number of training iterations, which we illustrate in Figure 3.

Original: In such cases, optimal values for joint LP and CD differ from those for LP only.
Revised: In such cases, optimal values for joint LP and CD differ from those for LP only.

Original: Lastly, one might wonder whether our performance gains mainly come from our novel encoder or our regularized loss.
Revised: Lastly, one might wonder whether our performance gains mainly come from our novel encoder or our regularized loss.

Original: Figure 4 reports an ablation study , consisting in training variant versions of Modularity-Aware VGAEs with one component only (i.e., the novel encoder but not the regularized loss, or vice versa).
Revised: Figure 4 reports an ablation study , consisting in training variant versions of Modularity-Aware VGAEs with one component only (i.e., the novel encoder but not the regularized loss, or vice versa).

Original: We show that incorporating any of these two individual contributions into the VGAE improves CD, and that their simultaneous use leads to the best results.
Revised: We show that incorporating any of these two individual contributions into the VGAE improves CD, and that their simultaneous use leads to the best results.

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we introduced a well-performing approach for joint CD and LP with GAEs and VGAEs.
Revised: In this paper, we introduced a well-performing approach for joint CD and LP with GAEs and VGAEs.

Original: We demonstrated its effectiveness through in-depth experimental validation.
Revised: We demonstrated its effectiveness through in-depth experimental validation.

Original: Our work paves the way for various future research, including replacing Louvain with other prior methods, using our regularizer in conjunction with other reconstruction losses (e.g., ELBO variants computed from
Revised: Our work paves the way for various future research, including replacing Louvain with other prior methods, using our regularizer in conjunction with other reconstruction losses (e.g., ELBO variants computed from

Original: Gaussian mixtures [7, 8]), and extending our approach to dynamic graphs.
Revised: Gaussian mixtures [7, 8]), and extending our approach to dynamic graphs.

Original: The journal version 1 of this work  includes several additional extensions as well as results, omitted here for brevity.
Revised: The journal version 2 of this work [44] includes several additional extensions as well as results, omitted here for brevity.

Original: This includes further comparisons to non-GAE/VGAE methods, a spectral analysis of our message passing operator, and discussions on how this research helps a music streaming service  address real-world multi-task LP and CD problems for music recommendation purposes.
Revised: This includes further comparisons to non-GAE/VGAE methods, a spectral analysis of our message passing operator, and discussions on how this research helps the music streaming service Deezer address real-world multi-task LP and CD problems for music recommendation purposes.

Original: Appendix
Revised: Appendix

Original: This appendix provides details on our experimental setting in Appendix A, and complementary tables and figures from our experiments in Appendix B.
Revised: This appendix provides details on our experimental setting in Appendix A, and complementary tables and figures from our experiments in Appendix B.

Original: A Experimental Setting
Revised: A Experimental Setting

Original: Datasets.
Revised: Datasets.

Original: We consider seven graphs of various origins, characteristics, and sizes.
Revised: We consider seven graphs of various origins, characteristics, and sizes.

Original: Firstly, we study the Cora ( n = 2708 , m = 5429 ), Citeseer ( n = 3327 , m = 4732 ) and Pubmed ( n = 19717 , m = 44338 ) citation networks [25], with and without node features that correspond to bag-of-words vectors of dimensions f = 1433 , 3703 , and 500 , respectively.
Revised: Firstly, we study the Cora ( n = 2708 , m = 5429 ), Citeseer ( n = 3327 , m = 4732 ) and Pubmed ( n = 19717 , m = 44338 ) citation networks [25], with and without node features that correspond to bag-of-words vectors of dimensions f = 1433 , 3703 , and 500 , respectively.

Original: In these datasets, nodes are clustered in 6, 7, and 3 topic classes, respectively, acting as the communities to be detected.
Revised: In these datasets, nodes are clustered in 6, 7, and 3 topic classes, respectively, acting as the communities to be detected.

Original: These graphs are commonly used to evaluate GAEs and VGAEs.
Revised: These graphs are commonly used to evaluate GAEs and VGAEs.

Original: We, therefore, see value in studying them as well, especially in their featureless version where previous GAE and VGAE extensions fall short on CD.
Revised: We, therefore, see value in studying them as well, especially in their featureless version where previous GAE and VGAE extensions fall short on CD.

Original: In addition, we consider a larger version of Cora, referred to as Cora-Large ( n = 23166 , m = 91500 ) [42].
Revised: In addition, we consider a larger version of Cora, referred to as Cora-Large ( n = 23166 , m = 91500 ) [42].

Original: Nodes are documents clustered in 70 topic-related communities.
Revised: Nodes are documents clustered in 70 topic-related communities.

Original: Additionally, we consider the Blogs web graph ( n = 1224 , m = 19025 )
Revised: Additionally, we consider the Blogs web graph ( n = 1224 , m = 19025 )

Original: Nodes correspond to webpages of political blogs connected through hyperlinks, and clustered in two communities corresponding to politically left-leaning or right-leaning blogs.
Revised: Nodes correspond to webpages of political blogs connected through hyperlinks, and clustered in two communities corresponding to politically left-leaning or right-leaning blogs.

Original: Thirdly, we examine the SBM graph ( n = 100000 , m = 1498844 ), generated from a stochastic block model , i.e., a generative model for communitybased random graphs [2].
Revised: Thirdly, we examine the SBM graph ( n = 100000 , m = 1498844 ), generated from a stochastic block model , i.e., a generative model for communitybased random graphs [2].

Original: Nodes are clustered in 100 ground truth communities of 1000 nodes each.
Revised: Nodes are clustered in 100 ground truth communities of 1000 nodes each.

Original: Nodes from the same community are connected with probability p = 2 × 10 − 2 , while nodes from different communities are connected with probability q = 2 × 10 − 4 < p .
Revised: Nodes from the same community are connected with probability p = 2 × 10 − 2 , while nodes from different communities are connected with probability q = 2 × 10 − 4 < p .

Original: Albeit being synthetic, this graph includes actual node communities by design, and is, therefore, relevant to evaluate CD methods.
Revised: Albeit being synthetic, this graph includes actual node communities by design, and is, therefore, relevant to evaluate CD methods.

Original: Lastly, we consider Album ( n = 2503985 , m = 25039155 ) a private graph provided by a music streaming service.
Revised: Lastly, we consider Album ( n = 2503985 , m = 25039155 ) a private graph provided by the music streaming service Deezer.

Original: Nodes are music albums available on this service, connected through an undirected edge when they are regularly co-listened to by users.
Revised: Nodes are music albums available on this service, connected through an undirected edge when they are regularly co-listened to by users.

Original: The service is jointly interested in (1) predicting new connections 
Revised: The service is jointly interested in (1) predicting new connections in the graph

Original: in
Revised: corresponding to new albums pairs

Original: the graph, corresponding to new albums pairs that users would enjoy listening to together; and (2) learning groups of similar albums, with the aim of providing usage -based recommendations (i.e., if users listen to several albums from a community, other unlistened albums from this same community could be recommended to them).
Revised: that users would enjoy listening to together; and (2) learning groups of similar albums, with the aim of providing usage-based recommendations (i.e., if users listen to several albums from a community, other unlistened albums from this same community could be recommended to them).

Original: In such an application, learning album representations that would jointly enable effective LP and CD would therefore be desirable.
Revised: In such an application, learning album representations that would jointly enable effective LP and CD would therefore be desirable.

Original: For evaluation, communities will be compared to a ground truth clustering of albums in 20 groups defined by their main music genre , allowing us to assess the musical homogeneity of node communities.
Revised: For evaluation, communities will be compared to a ground truth clustering of albums in 20 groups defined by their main music genre , allowing us to assess the musical homogeneity of node communities.

Original: Tasks.
Revised: Tasks.

Original: For each of these graphs, we assess the performance of our models on two downstream tasks.
Revised: For each of these graphs, we assess the performance of our models on two downstream tasks.

Original: • Task 1: We first examine a “pure” CD task, consisting in the extraction of a partition of the node set V which ideally agrees with the ground truth communities of each graph.
Revised: • Task 1: We first examine a “pure” CD task, consisting in the extraction of a partition of the node set V which ideally agrees with the ground truth communities of each graph.

Original: Communities will be retrieved by running a k -means (with k -means++ initialization [3]) in the final embedding space of each model to cluster the vectors
Revised: Communities will be retrieved by running a k -means (with k -means++ initialization [3]) in the final embedding space of each model to cluster the vectors

Original: z i , with k matching the known number of communities; except for some baseline methods that explicitly incorporate another strategy to partition nodes.
Revised: z i , with k matching the known number of communities; except for some baseline methods that explicitly incorporate another strategy to partition nodes.

Original: We compare the obtained partitions to the ground truth using the Adjusted Mutual Information (AMI) and Adjusted Rand Index (ARI) scores 6 .
Revised: We compare the obtained partitions to the ground truth using the Adjusted Mutual Information (AMI) and Adjusted Rand Index (ARI) scores 6 .

Original: • Task 2: We also study a joint LP and CD task.
Revised: • Task 2: We also study a joint LP and CD task.

Original: In such a multi-task setting, we learn all node embedding spaces from incomplete versions of the seven graphs, where 15% of edges were randomly masked.
Revised: In such a multi-task setting, we learn all node embedding spaces from incomplete versions of the seven graphs, where 15% of edges were randomly masked.

Original: We create a validation and a test set from these masked edges (from 5% and 10% of edges, respectively) and the same number of randomly picked unconnected node pairs acting as “non-edge” negative pairs.
Revised: We create a validation and a test set from these masked edges (from 5% and 10% of edges, respectively) and the same number of randomly picked unconnected node pairs acting as “non-edge” negative pairs.

Original: Then, using decoder predictions ˆ A ij computed from vectors z i and z j , we evaluate each model’s ability to distinguish edges from nonedges, i.e., LP, from the embedding space, using the Area Under the ROC Curve (AUC) and Average Precision (AP) scores 6 .
Revised: Then, using decoder predictions ˆ A ij computed from vectors z i and z j , we evaluate each model’s ability to distinguish edges from nonedges, i.e., LP, from the embedding space, using the Area Under the ROC Curve (AUC) and Average Precision (AP) scores 6 .

Original: Jointly, we evaluate the CD performance obtained from such incomplete graphs, using the same methodology and scores as in Task 1.
Revised: Jointly, we evaluate the CD performance obtained from such incomplete graphs, using the same methodology and scores as in Task 1.

Original: [Table]
Revised: [Table]

Original: In the case of Task 2, we expect AMI and ARI scores to decrease w.r.t.
Revised: In the case of Task [Table] we expect AMI and ARI scores to decrease w.r [Table]

Original:  Task 1, as models will only observe incomplete versions of the graphs when learning embedding spaces.
Revised: t. Task 1, as models will only observe incomplete versions of the graphs when learning embedding spaces.

Original: With Task 2, we aim to assess whether improving CD inevitably leads to deteriorating performances on LP.
Revised: With Task 2, we aim to assess whether improving CD inevitably leads to deteriorating performances on LP.

Original: Models: Details on the Hyperparameter Selection Procedure.
Revised: Models: Details on the Hyperparameter Selection Procedure.

Original: For these two tasks and seven graphs, we compare the performances of our proposed Modularity-Aware GAE and VGAE to standard
Revised: For these two tasks and seven graphs, we compare the performances of our proposed Modularity-Aware GAE and VGAE to standard

Original: GAE and VGAE and to several other baselines.
Revised: GAE and VGAE and to several other baselines.

Original: All models described below will verify d = 16 (the journal version of this work also discusses results obtained with d ∈ { 32 , 64 } , which lead to similar conclusions as d = 16 ).
Revised: All models described below will verify d = 16 (the journal version of this work also discusses results obtained with d ∈ { 32 , 64 } , which lead to similar conclusions as d = 16 ).

Original: We choose other hyperparameters using the selection procedure mentioned in Section 3, and further described in the next paragraph.
Revised: We choose other hyperparameters using the selection procedure mentioned in Section 3, and further described in the next paragraph.

Original: Foremost, as CD is an unsupervised task, we cannot rely on train/validation/test splits as for the supervised LP classification task 7 .
Revised: Foremost, as CD is an unsupervised task, we cannot rely on train/validation/test splits as for the supervised LP classification task 7 .

Original: Consistently with our other contributions, we rather rely on the modularity [35], an unsupervised density-based criterion computed independently of ground truth communities.
Revised: Consistently with our other contributions, we rather rely on the modularity [35], an unsupervised density-based criterion computed independently of ground truth communities.

Original: Precisely, we select hyperparameters that maximize the average of:
Revised: Precisely, we select hyperparameters that maximize the average of:

Original: • the AUC obtained for LP on the validation set of Task 2; • the modularity: Q = 12 m (cid:80) ni,j =

Revised: • the AUC obtained for LP on the validation set of Task 2; • the modularity: Q = 12 m (cid:80) ni,j =

Original: [ A ij − d i d j 2 m ] δ ( i, j ) , computed from the communities extracted by running a k -means on the final vectors
Revised: [ A ij − d i d j 2 m ] δ ( i, j ) , computed from the communities extracted by running a k -means on the final vectors

Original: z i , learned from the train graph of Task 2.
Revised: z i , learned from the train graph of Task 2.

Original: In this equation, δ ( i, j ) = 1 if nodes i and j belong to the same community and 0 otherwise.
Revised: In this equation, δ ( i, j ) = 1 if nodes i and j belong to the same community and 0 otherwise.

Original: We expect this dual criterion to identify hyperparameters jointly relevant to LP and CD.
Revised: We expect this dual criterion to identify hyperparameters jointly relevant to LP and CD.

Original: Models: Modularity-Aware GAE and VGAE.
Revised: Models: Modularity-Aware GAE and VGAE.

Original: We trained two versions of our Modularity-Aware
Revised: We trained two versions of our Modularity-Aware

Original: GAE and VGAE: one with the linear encoder proposed by Salha et al.
Revised: GAE and VGAE: one with the linear encoder proposed by Salha et al.

Original: [42], and one with the 2-layer
Revised: [42], and one with the 2-layer

Original: GCN encoder used by Kipf and Welling [24].
Revised: GCN encoder used by Kipf and Welling [24].

Original: The latter encoder includes a 32-dimensional hidden layer.
Revised: The latter encoder includes a 32-dimensional hidden layer.

Original: As most GAE/VGAE models, we use a simple inner product decoder: ˆ A ij = σ ( z Ti z j ) .
Revised: As most GAE/VGAE models, we use a simple inner product decoder: ˆ A ij = σ ( z Ti z j ) .

Original: During training, we used the Adam optimizer [21], without dropout (but we tested models with dropout values in { 0 , 0 . 1 , 0 . 2 } in our grid search optimization).
Revised: During training, we used the Adam optimizer [21], without dropout (but we tested models with dropout values in { 0 , 0 . 1 , 0 . 2 } in our grid search optimization).

Original: For each graph, we considered learning rates from the grid { 0 . 001 , 0 .
Revised: For each graph, we considered learning rates from the grid { 0 . 001 , 0 .

Original: 05 , 0 . 1 , 0 . 2 } , number of training iterations in { 100 , 200 , 300 , ..., 800 } , with λ ∈ { 0 , 0 . 01 , 0 . 
Revised: 05 , 0 . 1 , 0 . 2 } , number of training iterations in { 100 , 200 , 300 , ..., 800 } , with λ ∈ { 0 , 0 . 01 , 0 . 3 , ..., 1 . 0 } , β ∈ { 0 , 0 . 01 , 0 . 05 , 0 . 1 , 0 . 5 , 1 . 0 , 1 . 5 , 2 . 0 } , γ ∈ { 0 . 1 , 0 . 5 , 1 . 0 , 2 , 5 , 10 } and s ∈ { 1 , 2 , 5 , 10 } .

Original: The best hyperparameters for each graph are reported in Table 1.
Revised: The best hyperparameters for each graph are reported in Table 1.

Original: We adopted the same optimal hyperparameters for GAE and VGAE variants.
Revised: We adopted the same optimal hyperparameters for GAE and VGAE variants.

Original: Lastly, as the exact loss computation was computationally infeasible for our two largest graphs, SBM and Album, their corresponding models were trained by using the FastGAE method [40], approximating losses by reconstructing degree-based sampled subgraphs of n = 10000 nodes (a different one at each training iteration).
Revised: Lastly, as the exact loss computation was computationally infeasible for our two largest graphs, SBM and Album, their corresponding models were trained by using the FastGAE method [40], approximating losses by reconstructing degree-based sampled subgraphs of n = 10000 nodes (a different one at each training iteration).

Original: We used Tensorflow [1], training our models (as well as GAE/VGAE baselines described below) on an NVIDIA GTX 1080 GPU, and running other operations on a double Intel Xeon Gold 6134 CPU 8 .
Revised: We used Tensorflow [1], training our models (as well as GAE/VGAE baselines described below) on an NVIDIA GTX 1080 GPU, and running other operations on a double Intel Xeon Gold 6134 CPU 8 .

Original: Models: Standard GAE and VGAE.
Revised: Models: Standard GAE and VGAE.

Original: We examine two variants of the standard GAE and VGAE: one with 2-layer GCN encoders with a 32-dimensional hidden layer (which is equal to the GAE and VGAE from Kipf and Welling [24]) and one with a linear encoder (which is equal to the linear GAE and VGAE from Salha et al.
Revised: We examine two variants of the standard GAE and VGAE: one with 2-layer GCN encoders with a 32-dimensional hidden layer (which is equal to the GAE and VGAE from Kipf and Welling [24]) and one with a linear encoder (which is equal to the linear GAE and VGAE from Salha et al.

Original: We note that these models are particular cases of our Modularity-Aware GAE/VGAE with GCN or linear encoder and with λ = [-
-] and β = 0 .
Revised: We note that these models are particular cases of our ModularityAware GAE/VGAE with GCN or linear encoder and with λ = [-
-]0 and β = 0 .

Original: As for our Modularity-Aware models, LP is performed from inner product decoding, and CD via a k -means on vectors z i .
Revised: As for our ModularityAware models, LP is performed from inner product decoding, and CD via a k -means on vectors z i .

Original: We selected similar learning rates and numbers of iterations to the values reported in Table 1.
Revised: We selected similar learning rates and numbers of iterations to the values reported in Table 1.

Original: Models: Other Baselines.
Revised: Models: Other Baselines.

Original: We also report experiments on VGAECD [7], a VGAE for CD model that replaces Gaussian priors by learnable Gaussian mixtures .
Revised: We also report experiments on VGAECD [7], a VGAE for CD model that replaces Gaussian priors by learnable Gaussian mixtures .

Original: Such a change permits recovering communities from node embedding spaces without relying on an additional k -means step.
Revised: Such a change permits recovering communities from node embedding spaces without relying on an additional k -means step.

Original: We also tested VGAECD-OPT, an improved version of VGAECD by the same authors [8].
Revised: We also tested VGAECD-OPT, an improved version of VGAECD by the same authors [8].

Original: Specifically,
Revised: Specifically,

Original: VGAECD-OPT replaces GCN encoders with linear models.
Revised: VGAECD-OPT replaces GCN encoders with linear models.

Original: It also adopts a different optimization procedure based on neural expectation-maximization, which guarantees that communities do not collapse during training and experimentally leads to better performances [8].
Revised: It also adopts a different optimization procedure based on neural expectation-maximization, which guarantees that communities do not collapse during training and experimentally leads to better performances [8].

Original: We set similar hyperparameters to the above other GAE/VGAE-based models.
Revised: We set similar hyperparameters to the above other GAE/VGAE-based models.

Original: In all models, the number of Gaussian mixtures matches the ground truth number of communities in each graph.
Revised: In all models, the number of Gaussian mixtures matches the ground truth number of communities in each graph.

Original: Besides, we also report experiments on the Dirichlet Graph Variational Autoencoder (DGVAE) [27], another extension of VGAE which uses Dirichlet distributions as priors on latent vectors, acting as indicators of community membership.
Revised: Besides, we also report experiments on the Dirichlet Graph Variational Autoencoder (DGVAE) [27], another extension of VGAE which uses Dirichlet distributions as priors on latent vectors, acting as indicators of community membership.

Original: We set similar learning rates and layer dimensions to the above
Revised: We set similar learning rates and layer dimensions to the above

Original: GAE/VGAE-based models.
Revised: GAE/VGAE-based models.

Original: In the case of DGVAE, we use 2-layer GCN encoders for consistency with other models in our experiments.
Revised: In the case of DGVAE, we use 2-layer GCN encoders for consistency with other models in our experiments.

Original: We nonetheless acknowledge that the authors also proposed another encoder, denoted Heatts in their paper (but unavailable in their public code at the time of writing) that could replace GCNs both in DGVAE and in Modularity-Aware GAE and VGAE.
Revised: We nonetheless acknowledge that the authors also proposed another encoder, denoted Heatts in their paper (but unavailable in their public code at the time of writing) that could replace GCNs both in DGVAE and in Modularity-Aware GAE and VGAE.

Original: We also examine the Adversartially-Regularized (Variational) Graph Autoencoder (ARGA and
Revised: We also examine the Adversartially-Regularized (Variational) Graph Autoencoder (ARGA and

Original: ARVGA) models [36], that incorporate an adversarial regularization scheme to GAE and VGAE, with similar hyperparameters as previous models.
Revised: ARVGA) models [36], that incorporate an adversarial regularization scheme to GAE and VGAE, with similar hyperparameters as previous models.

Original: ARGA and ARVGA emerged as some of the most cited GAE/VGAE extensions and, while they were not specifically introduced for CD, Pan et al.
Revised: ARGA and ARVGA emerged as some of the most cited GAE/VGAE extensions and, while they were not specifically introduced for CD, Pan et al.

Original: [36] reported empirical gains on this task w.r.t.
Revised: [36] reported empirical gains on this task w.r.t.

Original: standard GAE/VGAE, on graphs with node features.
Revised: standard GAE/VGAE, on graphs with node features.

Original: For completeness, we add three baselines not utilizing the autoencoder paradigm.
Revised: For completeness, we add three baselines not utilizing the autoencoder paradigm.

Original: We report results obtained from the popular node embedding methods node2vec [11] and DeepWalk [39], training models from 10 random walks with length 80 per node, a window size of 5 and on a single epoch.
Revised: We report results obtained from the popular node embedding methods node2vec [11] and DeepWalk [39], training models from 10 random walks with length 80 per node, a window size of 5 and on a single epoch.

Original: For node2vec, we further set p = q = 1 .
Revised: For node2vec, we further set p = q = 1 .

Original: We use a similar strategy as GAEs/VGAEs ( k -means/inner products) for CD and LP from embedding spaces.
Revised: We use a similar strategy as GAEs/VGAEs ( k -means/inner products) for CD and LP from embedding spaces.

Original: Lastly, we also compare to the Louvain method [4] for CD.
Revised: Lastly, we also compare to the Louvain method [4] for CD.

Original: We see value in comparing to a direct use of Louvain, as this method is directly leveraged in our Modularity-Aware GAE/VGAE as a pre-processing step for the computation of A c and A s .
Revised: We see value in comparing to a direct use of Louvain, as this method is directly leveraged in our Modularity-Aware GAE/VGAE as a pre-processing step for the computation of A c and A s .

Original: B Figures and Tables
Revised: B Figures and Tables

Original: We now provide complementary tables and figures from our experiments.
Revised: We now provide complementary tables and figures from our experiments.

Original: Table 2 details complete results for the Cora dataset and Table 3 reports more summarized results for several more graphs.
Revised: Table 2 details complete results for the Cora dataset and Table 3 reports more summarized results for several more graphs.

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Table]
Revised: [Table]