Original: F AST TOPOLOGICAL CLUSTERING WITH W ASSERSTEIN DISTANCE
Revised: F AST T OPOLOGICAL C LUSTERING WITH W ASSERSTEIN D ISTANCE

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: The topological patterns exhibited by many real-world networks motivate  development of topology-based methods for assessing the similarity of networks.
Revised: The topological patterns exhibited by many real-world networks motivate the development of topology-based methods for assessing the similarity of networks.

Original: However, extracting topological structure is difﬁcult, especially for large and dense networks whose node degrees range over multiple orders of magnitude.
Revised: However, extracting topological structure is difﬁcult, especially for large and dense networks whose node degrees range over multiple orders of magnitude.

Original: In this paper, we propose a novel and computationally practical topological clustering method that clusters complex networks with intricate topology using principled theory from persistent homology and optimal transport.
Revised: In this paper, we propose a novel and computationally practical topological clustering method that clusters complex networks with intricate topology using principled theory from persistent homology and optimal transport.

Original: Such networks are aggregated into clusters through a centroid-based clustering strategy based on both their topological and geometric structure, preserving correspondence between nodes in different networks.
Revised: Such networks are aggregated into clusters through a centroid-based clustering strategy based on both their topological and geometric structure, preserving correspondence between nodes in different networks.

Original: The notions of topological proximity and centroid are characterized using a novel and efﬁcient approach to computation of the Wasserstein distance and barycenter for persistence barcodes associated with connected components and cycles.
Revised: The notions of topological proximity and centroid are characterized using a novel and efﬁcient approach to computation of the Wasserstein distance and barycenter for persistence barcodes associated with connected components and cycles.

Original: The proposed method is demonstrated to be effective using both simulated networks and measured functional brain networks.
Revised: The proposed method is demonstrated to be effective using both simulated networks and measured functional brain networks.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Network models are extremely useful representations for complex data.
Revised: Network models are extremely useful representations for complex data.

Original: Signiﬁcant attention has been given to cluster analysis within a single network, such as detecting community structure (Newman, 2006;
Revised: Signiﬁcant attention has been given to cluster analysis within a single network, such as detecting community structure (Newman, 2006;

Original: Rohe et al., 2011; Yin et al., 2017).
Revised: Rohe et al., 2011; Yin et al., 2017).

Original: Less attention has been given to clustering of collections of network representations.
Revised: Less attention has been given to clustering of collections of network representations.

Original: Clustering approaches typically group similar networks based on comparisons of edge weights (Xu & Wunsch, 2005), not topology.
Revised: Clustering approaches typically group similar networks based on comparisons of edge weights (Xu & Wunsch, 2005), not topology.

Original: Assessing similarity of networks based on topological structure offers the potential for new insight, given the inherent topological patterns exhibited by most real-world networks.
Revised: Assessing similarity of networks based on topological structure offers the potential for new insight, given the inherent topological patterns exhibited by most real-world networks.

Original: However, extracting meaningful network topology is a very difﬁcult task, especially for large and dense networks whose node degrees range over multiple orders of magnitude (Barrat et al., 2004; Bullmore & Sporns, 2009; Honey et al., 2007).
Revised: However, extracting meaningful network topology is a very difﬁcult task, especially for large and dense networks whose node degrees range over multiple orders of magnitude (Barrat et al., 2004; Bullmore & Sporns, 2009; Honey et al., 2007).

Original: Persistent homology (Barannikov, 1994; Edelsbrunner et al., 2000; Wasserman, 2018) has recently emerged as a powerful tool for understanding, characterizing and quantifying complex networks (Carrière et al., 2020; Chung et al., 2019).
Revised: Persistent homology (Barannikov, 1994; Edelsbrunner et al., 2000) has recently emerged as a powerful tool for understanding, characterizing and quantifying complex networks (Songdechakraiwut et al., 2021).

Original: Persistent homology represents a network using topological features such as connected components and cycles.
Revised: Persistent homology represents a network using topological features such as connected components and cycles.

Original: Many networks naturally divide into modules or connected components (Bullmore & Sporns, 2009; Honey et al., 2007).
Revised: Many networks naturally divide into modules or connected components (Bullmore & Sporns, 2009; Honey et al., 2007).

Original: Similarly, cycles are ubiquitous and are often used to describe information propagation, robustness and feedback mechanisms (Keizer et al., 1995; Kwon & Cho, 2007;Ozbudak et al., 2005; Venkatesh et al., 2004;
Revised: Similarly, cycles are ubiquitous and are often used to describe information propagation, robustness and feedback mechanisms (Kwon & Cho, 2007;

Original: Weiner et al., 2002).
Revised: Lind et al., 2005).

Original: Effective use of such topological descriptors requires a notion of proximity that quantiﬁes the similarity between persistence barcodes, a convenient representation for connected components and cycles (Ghrist, 2008).
Revised: Effective use of such topological descriptors requires a notion of proximity that quantiﬁes the similarity between persistence barcodes, a convenient representation for connected components and cycles (Ghrist, 2008).

Original: Wasserstein distance, which measures the minimal effort to modify one persistence barcode to another (Rabin et al., 2011), is an excellent choice due to its appealing geometric properties (Staerman et al., 2021) and its effectiveness shown in many machine learning applications (Kolouri et al., 2017; Mi et al., 2018; Solomon et al., 2015).
Revised: Wasserstein distance, which measures the minimal effort to modify one persistence barcode to another (Rabin et al., 2011), is an excellent choice due to its appealing geometric properties (Staerman et al., 2021) and its effectiveness shown in many machine learning applications (Kolouri et al., 2017; Mi et al., 2018; Solomon et al., 2015).

Original: Importantly,
Revised: Importantly,

Original: Wasserstein distance can be used to interpolate networks while preserving topological structure (Songdechakraiwut et al., 2021), and the mean under the Wasserstein distance, known as Wasserstein barycenter (Agueh & Carlier, 2011), can be viewed as the topological centroid of a set of networks.
Revised: Wasserstein distance can be used to interpolate networks while preserving topological structure (Songdechakraiwut et al., 2021), and the mean under the Wasserstein distance, known as Wasserstein barycenter (Agueh & Carlier, 2011), can be viewed as the topological centroid of a set of networks.

Original: The high cost of computing persistence barcodes, Wasserstein distance and the Wasserstein barycenter limit their applications to small scale problems, see, e.g., (Clough et al., 2020;
Revised: The high cost of computing persistence barcodes, Wasserstein distance and the Wasserstein barycenter limit their applications to small scale problems, see, e.g., (Clough et al., 2020;

Original: Hu et al., 2019; Kolouri et al., 2017; Mi et al., 2018).
Revised: Hu et al., 2019; Kolouri et al., 2017; Mi et al., 2018).

Original: Although approximation algorithms have been developed (Cuturi, 2013;
Revised: Although approximation algorithms have been developed (Cuturi, 2013;

Original: Cuturi & Doucet, 2014; Lacombe et al., 2018; Li et al., 2020; Solomon et al., 2015; Vidal et al., 2019;
Revised: Cuturi & Doucet, 2014; Lacombe et al., 2018; Li et al., 2020; Solomon et al., 2015; Vidal et al., 2019;

Original: Xie et al., 2020; Ye et al., 2017), it is unclear whether these approximations are effective for clustering complex networks as they inevitably limit sensitivity to subtle topological features.
Revised: Xie et al., 2020; Ye et al., 2017), it is unclear whether these approximations are effective for clustering complex networks as they inevitably limit sensitivity to subtle topological features.

Original: Indeed, more and more studies, see, e.g., (Robins & Turner, 2016; Xia & Wei, 2014) have demonstrated that such subtle topological patterns are important for the characterization of complex networks, suggesting these approximation algorithms are undesirable.
Revised: Indeed, more and more studies, see, e.g., (Robins & Turner, 2016; Xia & Wei, 2014) have demonstrated that such subtle topological patterns are important for the characterization of complex networks, suggesting these approximation algorithms are undesirable.

Original: Recently, it was shown that the Wasserstein distance and barycenter for network graphs have closedform solutions that can be computed exactly and efﬁciently (Songdechakraiwut et al., 2021) because the persistence barcodes are inherently one dimensional.
Revised: Recently, it was shown that the Wasserstein distance and barycenter for network graphs have closedform solutions that can be computed exactly and efﬁciently (Songdechakraiwut et al., 2021) because the persistence barcodes are inherently one dimensional.

Original: Motivated by this result, we present a novel and computationally practical topological clustering method that clusters complex networks of the same size with intricate topological characteristics.
Revised: Motivated by this result, we present a novel and computationally practical topological clustering method that clusters complex networks of the same size with intricate topological characteristics.

Original: Topological information alone is effective at clustering networks when there is no correspondence between nodes in different networks.
Revised: Topological information alone is effective at clustering networks when there is no correspondence between nodes in different networks.

Original: However, when networks have meaningful node correspondence, we performthe cluster analysis using combined topological and geometric information to preservethe node correspondence.
Revised: However, when networks have meaningful node correspondence, we perform cluster analysis using combined topological and geometric information to preserve node correspondence.

Original: Statistical validation based on ground truth information is used to demonstrate the effectiveness of our method when discriminating subtle topological features in simulated networks.
Revised: Statistical validation based on ground truth information is used to demonstrate the effectiveness of our method when discriminating subtle topological features in simulated networks.

Original: The method is further illustrated by clustering measured functional brain networks associated with different levels of arousal during administration of general anesthesia.
Revised: The method is further illustrated by clustering measured functional brain networks associated with different levels of arousal during administration of general anesthesia.

Original: Our proposed method outperforms other clustering approaches in both the simulated and measured data.
Revised: Our proposed method outperforms other clustering approaches in both the simulated and measured data.

Original: The paper is organized as follows.
Revised: The paper is organized as follows.

Original: Background on our one-dimensional representation of persistence barcodes is given in section 2, while section 3 presents our topological clustering method.
Revised: Background on our one-dimensional representation of persistence barcodes is given in section 2, while section 3 presents our topological clustering method.

Original: In sections 4 and 5, we compare the performance of our method to several baseline algorithms using simulated and measured networks, and conclude the paper with a brief discussion of the potential impact of this work.
Revised: In sections 4 and 5, we compare the performance of our method to several baseline algorithms using simulated and measured networks, and conclude the paper with a brief discussion of the potential impact of this work.

Original:  O NE DIMENSIONAL PERSISTENCE BARCODES 2. G RAPH FILTRATION
Revised:  O NE -D IMENSIONAL P ERSISTENCE B ARCODES 2. G RAPH FILTRATION

Original: Consider a network represented as a weighted graph G =
Revised: Consider a network represented as a weighted graph G =

Original: ( V, w ) comprising a set of nodes V with symmetric adjacency matrix w = ( w ij ) , with edge weight w ij representing the relationship between node i and node j .
Revised: ( V, w ) comprising a set of nodes V with symmetric adjacency matrix w = ( w ij ) , with edge weight w ij representing the relationship between node i and node j .

Original: The number of nodes is denoted as | V | .
Revised: The number of nodes is denoted as | V | .

Original: The binary graph G (cid:15) =
Revised: The binary graph G (cid:15) =

Original: ( V, w (cid:15) ) of G is deﬁned as a graph consisting of the node set V and binary edge weights w (cid:15),ij = 1{+
+} if w ij > (cid:15) and w ij = 0{+
+} otherwise.
Revised: ( V, w (cid:15) ) of G is deﬁned as a graph consisting of the node set V and binary edge weights w (cid:15),ij = {+
+} if w ij > (cid:15) and w (cid:15),ij = {+
+} otherwise.

Original: We view the binary network G (cid:15) as a 1-skeleton (Munkres, 2018), a simplicial complex comprising only nodes  (0-dimensional topological features) and edges (1-dimensional topological features).
Revised: We view the binary network G (cid:15) as a 1-skeleton (Munkres, 2018), a simplicial complex comprising only nodes and edges. In the 1-skeleton, there are two types of topological features: connected components (0-dimensional topological features) and cycles (1-dimensional topological features).

Original: There are no topological features of higher dimensions in the 1-skeleton, in contrast to well-known Ripscomplexes (Ghrist, 2008).
Revised: There are no topological features of higher dimensions in the 1-skeleton, in contrast to well-known Rips (Ghrist, 2008) and clique complexes (Otter et al., 2017).

Original: In the 1-skeleton, there are two types of topological features: connected components and cycles.

Original: The number of connected components and the number of cycles in the binary network are referred to as the 0-th Betti number β 0 ( G (cid:15) ) and the 1-st Betti number β 1 ( G (cid:15) ) , respectively.
Revised: The number of connected components and the number of cycles in the binary network are referred to as the 0-th Betti number β 0 ( G (cid:15) ) and the 1-st Betti number β 1 ( G (cid:15) ) , respectively.

Original: A graph ﬁltration of G is deﬁned as a collection of nested binary networks (Lee et al., 2012):
Revised: A graph ﬁltration of G is deﬁned as a collection of nested binary networks (Lee et al., 2012):

Original: [Equation]
Revised: [Equation]

Original: where (cid:15) 0 ≤ (cid:15) 1 ≤ · · · ≤ (cid:15) k are ﬁltration values.
Revised: where (cid:15) 0 ≤ (cid:15) 1 ≤ · · · ≤ (cid:15) k are ﬁltration values.

Original: As (cid:15) increases, more and more edges are removed from the network G since we threshold the edge weights at higher connectivity.
Revised: As (cid:15) increases, more and more edges are removed from the network G since we threshold the edge weights at higher connectivity.

Original: For instance, G −∞ has each pair of nodes connected by an edge and thus is a complete graph consisting of a single connected component, while G ∞ has no edges and represents the node set.
Revised: For instance, G −∞ has each pair of nodes connected by an edge and thus is a complete graph consisting of a single connected component, while G ∞ has no edges and represents the node set.

Original: Figure 1 illustrates the graph ﬁltration of a four-node network and the corresponding Betti numbers.
Revised: Figure 1 illustrates the graph ﬁltration of a four-node network and the corresponding Betti numbers.

Original: Note that otherﬁltrations for analyzing graphs have been proposed, includinguse of descriptor functions such as heat kernels (Carrière etal., 2020) and task-speciﬁc learning(Hofer et al., 2020).
Revised: Note that other graphs have been proposed, including of descriptor functions such as heat kernels Carrière et and task-speciﬁc learning et al

Original: 2. B IRTH - DEATH DECOMPOSITION
Revised: 2. B IRTH - DEATH DECOMPOSITION

Original: Persistent homology keeps track of birth and death of connected components and cycles over ﬁltration values (cid:15) to determine their persistence , that is, the lifetime from their birth to death over (cid:15) .
Revised: Persistent homology keeps track of birth and death of connected components and cycles over ﬁltration values (cid:15) to deduce their persistence , i.e., the lifetime from their birth to death over (cid:15) .

Original: The persistence is represented as a persistence barcode PB ( G ) comprising intervals [ b i , d i ] representing the life-time of a connected component or a cycle that appears at the ﬁltration value b i and vanishes at d i .
Revised: The persistence is represented as a persistence barcode PB ( G ) comprising intervals [ b i , d i ] representing the lifetime of a connected component or a cycle that appears at the ﬁltration value b i and vanishes at d i .

Original: In the edge-weight threshold graph ﬁltration deﬁned in Section 2.1, connected components are born and cycles die as the ﬁltration value increases (Chung et al., 2019).
Revised: In the edge-weight threshold graph ﬁltration deﬁned in Section 2.1, connected components are born and cycles die as the ﬁltration value increases (Songdechakraiwut et al., 2021).

Original: Speciﬁcally, β 0 is monotonically increasing from β 0 ( G −∞ ) = 1 to β 0 ( G ∞ )
Revised: Speciﬁcally, β 0 is monotonically increasing from β 0 ( G −∞ ) = 1 to β 0 ( G ∞ )

Original: = | V | .
Revised: = | V | .

Original: There are β 0 ( G ∞ ) − β 0 ( G −∞ ) =
Revised: There are β 0 ( G ∞ ) − β 0 ( G −∞ ) =

Original: | V | −connected components that are born over the ﬁltration.
Revised: | V | − 1 connected components that are born over the ﬁltration.

Original: Connected components will never die once they are born, implying that every connected component has  death value at ∞ .
Revised: Connected components will never die once they are born, implying that every connected component has a death value at ∞ .

Original: Thus, we can represent their persistence as a collection of ﬁnite
Revised: Thus, we can represent their persistence as a collection of ﬁnite

Original: birth values B ( G ) =
Revised: birth values B ( G ) =

Original: { b i } | V |− 1 i =1 .
Revised: { b i } | V |− 1 i =1 .

Original: On the other hand, G −∞ is a complete graph containing all possible cycles; thus, all cycles have birth values at −∞ .
Revised: On the other hand, G −∞ is a complete graph containing all possible cycles; thus, all cycles have birth values at −∞ .

Original: Again, we can represent the persistence of the cycles as a collection of ﬁnite death values D ( G ) =
Revised: Again, we can represent the persistence of the cycles as a collection of ﬁnite death values D ( G ) =

Original: { d i } .
Revised: { d i } .

Original: How many cycles are there?
Revised: How many cycles are there?

Original: Since the deletion of an edge w ij must result in either the birth of a connected component or the death of a cycle, every edge weight must be in either B ( G ) or D ( G ) .
Revised: Since the deletion of an edge w ij must result in either the birth of a connected component or the death of a cycle, every edge weight must be in either B ( G ) or D ( G ) .

Original: Thus, the edge weight set W = { w ij | i > j } decomposes into the collection of birth values B ( G ) and the collection of death values D ( G ) .
Revised: Thus, the edge weight set W = { w ij | i > j } decomposes into the collection of birth values B ( G ) and the collection of death values D ( G ) .

Original: Since G −∞ is a complete graph with | V | ( | V |− 1)edge weights and | V | − 1 of these weights are associated with the birth of connected components, the number of cycles in G −∞ is thus equal to | V | ( | V |− 1)− ( | V |− 1)
Revised: Since G −∞ is a complete graph with | V | ( | V |− 1)edge weights and | V | − 1 of these weights are associated with the birth of connected components, the number of cycles in G −∞ is thus equal to | V | ( | V |− 1)− ( | V | [Equation]

Original:  = 1+ | V | ( | V |− 3).
Revised: 1) = 1+ | V | ( | [Equation]

Original: In the example of Figure 1, we have B ( G ) =
Revised: In the example of Figure 1, we have B ( G ) =

Original: { e 3 , e 5 , e 6 } and D ( G ) =
Revised: { e 3 , e 5 , e 6 } and D ( G ) =

Original: { e 1 , e 2 , e 4 } .
Revised: { e 1 , e 2 , e 4 } .

Original: Other graph ﬁltrations (Carrière et al., 2020; Hofer et al., 2020) do not necessarily share this monotonicity property and consequently one-dimensional barcode representations are not applicable.
Revised: Other graph ﬁltrations (Carrière et al., 2020; Hofer et al., 2020) do not necessarily share this monotonicity property and consequently one-dimensional barcode representations are not applicable.

Original: Finding the birth values in B ( G ) is equivalent to ﬁnding edge weights comprising the maximum spanning tree of G and can be done using well-known methods such as Prim’s and Kruskal’s algorithms (Lee et al., 2012).
Revised: Finding the birth values in B ( G ) is equivalent to ﬁnding edge weights comprising the maximum spanning tree of G and can be done using well-known methods such as Prim’s and Kruskal’s algorithms (Lee et al., 2012).

Original: Once B ( G ) is known, D ( G ) is simply given as the remaining edge weights.
Revised: Once B ( G ) is known, D ( G ) is simply given as the remaining edge weights.

Original: Finding B ( G ) and D ( G ) requires only O ( n log n ) operations, where n is the number of edges in the network, and thus is extremely computationally efﬁcient.
Revised: Finding B ( G ) and D ( G ) requires only O ( n log n ) operations, where n is the number of edges in the network, and thus is extremely computationally efﬁcient.

Original:  C LUSTERING M ETHOD 3. T OPOLOGICAL DISTANCE SIMPLIFICATION
Revised:  C LUSTERING M ETHOD 3. T OPOLOGICAL DISTANCE SIMPLIFICATION

Original: Use of edge-weight threshold ﬁltration and limiting consideration to connected components and cycles as topological features results in signiﬁcant simpliﬁcation of the 2-Wasserstein distance (Rabin et al., 2011) between barcode descriptors (Cohen-Steiner et al., 2010) of networks as follows.
Revised: Use of edge-weight threshold ﬁltration and limiting consideration to connected components and cycles as topological features results in signiﬁcant simpliﬁcation of the 2-Wasserstein distance (Rabin et al., 2011) between barcode descriptors (Cohen-Steiner et al., 2010) of networks as follows.

Original: Let
Revised: Let

Original: G and H be two given networks that have the same number of nodes.
Revised: G and H be two given networks that have the same number of nodes.

Original: The topological distance d top ( G, H ) is deﬁned as the optimal matching cost:
Revised: The topological distance d top ( G, H ) is deﬁned as the optimal matching cost:

Original: [Equation]
Revised: [Equation]

Original: where the optimization is over all possible bijections τ from barcode PB ( G ) to barcode PB ( H ) .
Revised: where the optimization is over all possible bijections τ from barcode PB ( G ) to barcode PB ( H ) .

Original: Intuitively, we can think of each interval [ b i , d i ] as a point ( b i , d i ) in  2-dimensional plane and that the topological distance measures the minimal amount of work to move points in PB ( G ) to PB ( H ) .
Revised: Intuitively, we can think of each interval [ b i , d i ] as a point ( b i , d i ) in a 2-dimensional plane. The topological distance measures the minimal amount of work to move points in PB ( G ) to PB ( H ) .

Original: Note this alternative representation of points in the plane is equivalent to the persistence barcode and called the persistence diagram (Edelsbrunner & Harer, 2008).
Revised: Note this alternative representation of points in the plane is equivalent to the persistence barcode and called the persistence diagram (Edelsbrunner & Harer, 2008).

Original: Moving a connected component point ( b i , ∞ ) to a cycle point ( −∞ , d j ) or vice versa takes an inﬁnitely large amount of work.
Revised: Moving a connected component point ( b i , ∞ ) to a cycle point ( −∞ , d j ) or vice versa takes an inﬁnitely large amount of work.

Original: Thus, we only need to optimize over bijections that match the same type of topological features.
Revised: Thus, we only need to optimize over bijections that match the same type of topological features.

Original: Subsequently, we can equivalently rewrite d top in terms of B ( G ) , D ( G ) , B ( H ) and D ( H ) as
Revised: Subsequently, we can equivalently rewrite d top in terms of B ( G ) , D ( G ) , B ( H ) and D ( H ) as

Original: [Equation]
Revised: [Equation]

Original: where τ 0 is a bijection from B ( G ) to B ( H ) and τ 1 is a bijection from D ( G ) to D ( H ) .
Revised: where τ 0 is a bijection from B ( G ) to B ( H ) and τ 1 is a bijection from D ( G ) to D ( H ) .

Original: The ﬁrst term matches connected components to connected components and the second term matches cycles to cycles.
Revised: The ﬁrst term matches connected components to connected components and the second term matches cycles to cycles.

Original: Matching each type of topological feature separately is commonly done in medical imaging and machine learning studies (Clough et al., 2020;
Revised: Matching each type of topological feature separately is commonly done in medical imaging and machine learning studies (Clough et al., 2020;

Original: Hu et al., 2019).
Revised: Hu et al., 2019).

Original: The topological distance d top has a closed-form solution that allows for efﬁcient computation as follows (Songdechakraiwut et al., 2021).
Revised: The topological distance d top has a closed-form solution that allows for efﬁcient computation as follows (Songdechakraiwut et al., 2021).

Original: [Equation]
Revised: [Equation]

Original: where τ ∗ 0 maps the l -th smallest birth value in B ( G ) to the l -th smallest birth value in B ( H ) and τ ∗maps the l -th smallest death value in D ( G ) to the l -th smallest death value in D ( H ) for all l .
Revised: where τ ∗ 0 maps the l -th smallest birth value in B ( G ) to the l -th smallest birth value in B ( H ) and τ ∗maps the l -th smallest death value in D ( G ) to the l -th smallest death value in D ( H ) for all l .

Original: A proof is  in the supplementary material. As a result, the optimal matching cost can be computed quickly and efﬁciently by sorting birth and death values, and matching them in order.
Revised: A proof is provided in Appendix A. As a result, the optimal matching cost can be computed quickly and efﬁciently by sorting birth and death values, and matching them in order.

Original: The computational cost of evaluating d top is O ( n log n ) , where n is the number of edges in networks.
Revised: The computational cost of evaluating d top is O ( n log n ) , where n is the number of edges in networks.

Original: 3. T OPOLOGICAL CLUSTERING
Revised: 3. T OPOLOGICAL CLUSTERING

Original: Let G =
Revised: Let G =

Original: ( V, w ) and H =
Revised: ( V, w ) and H =

Original: ( V, u ) be two networks.
Revised: ( V, u ) be two networks.

Original: We deﬁne the network dissimilarity d 2 net between
Revised: We deﬁne the network dissimilarity d 2 net between

Original: G and H as a weighted sum of the squared geometric distance and the squared topological distance:
Revised: G and H as a weighted sum of the squared geometric distance and the squared topological distance:

Original: [Equation]
Revised: [Equation]

Original: where λ ∈ [0 , 1] controls the relative weight between the geometric and topological terms.
Revised: where λ ∈ [0 , 1] controls the relative weight between the geometric and topological terms.

Original: The geometric distance measures the node-by-node dissimilarity in the networks that is not captured by topology alone and is helpful when node identity is meaningful, such as in neuroscience applications.
Revised: The geometric distance measures the node-by-node dissimilarity in the networks that is not captured by topology alone and is helpful when node identity is meaningful, such as in neuroscience applications.

Original: Given observed networks with identical node sets, the goal is to partition the networks into k clusters
Revised: Given observed networks with identical node sets, the goal is to partition the networks into k clusters

Original: C =
Revised: C =

Original: { C h } kh
Revised: { C h } kh

Original: =
Revised: =

Original: 1 with corresponding cluster centroids or representatives M = { M h } kh =1 such that the sum of the network dissimilarities d 2 net from the networks to their representatives is minimized, i.e.,[Equation]
Revised: 1 with corresponding cluster centroids or representatives M = { M h } kh =1 such that the sum of the network dissimilarities d 2 net from the networks to their representatives is minimized, i.e.,

Original: [Equation]
Revised: [Equation]

Original: The topological clustering formulation given in (5) suggests a natural iterative relocation algorithm using coordinate descent (Banerjee et al., 2005).
Revised: The topological clustering formulation given in (5) suggests a natural iterative relocation algorithm using coordinate descent (Banerjee et al., 2005).

Original: In particular, the algorithm alternates between two steps: an assignment step and a re-estimation step.
Revised: In particular, the algorithm alternates between two steps: an assignment step and a re-estimation step.

Original: In the assignment step, L is minimized with respect to C while holding M ﬁxed.
Revised: In the assignment step, L is minimized with respect to C while holding M ﬁxed.

Original: Minimization of L is achieved simply by assigning each observed network to the cluster whose representative M h is the nearest in terms of the criterion d 2 net .
Revised: Minimization of L is achieved simply by assigning each observed network to the cluster whose representative M h is the nearest in terms of the criterion d 2 net .

Original: In the re-estimation step, the algorithm minimizes L with respect to M while holding C ﬁxed.
Revised: In the re-estimation step, the algorithm minimizes L with respect to M while holding C ﬁxed.

Original: In this case, L is minimized by re-estimating the representatives for each individual cluster:
Revised: In this case, L is minimized by re-estimating the representatives for each individual cluster:

Original: [Equation]
Revised: [Equation]

Original: We will consider solving the objective function given in (6) for λ = 0 , λ = 1 and λ ∈ (0 , 1) .
Revised: We will consider solving the objective function given in (6) for λ = 0 , λ = 1 and λ ∈ (0 , 1) .

Original: λ = 0 describes conventional edge clustering (MacQueen, 1967) since the topological term is excluded.
Revised: λ = 0 describes conventional edge clustering (MacQueen, 1967) since the topological term is excluded.

Original: λ = 1 describes clustering based on pure topology and ignores correspondence of edge weights.
Revised: λ = 1 describes clustering based on pure topology and ignores correspondence of edge weights.

Original: This case is potentially useful for clustering networks whose node sets are not identical or of different size.
Revised: This case is potentially useful for clustering networks whose node sets are not identical or of different size.

Original: Each representative M h of cluster C h minimizes the sum of the squared topological distances, i.e.,min M h
Revised: Each representative M h of cluster C h minimizes the sum of the squared topological distances, i.e.,

Original: [Equation]
Revised: [Equation]

Original: [Equation] Thus, we only need to optimize over the topology of the network, i.e., B ( M h ) and D ( M h ) , instead of the original network M h itself.
Revised: Thus, we only need to optimize over the topology of the network, i.e., B ( M h ) and D ( M h ) , instead of the original network M h itself.

Original: The topology solving (7) is the topological centroid of networks in cluster C h .
Revised: The topology solving (7) is the topological centroid of networks in cluster C h .

Original: Interestingly, the topological centroid has  closed-form solution and can be calculated analytically as follows.
Revised: Interestingly, the topological centroid has a closed-form solution and can be calculated analytically as follows.

Original: Lemma 1.
Revised: Lemma 1.

Original: Let G 1 , ..., G n be n networks each with m nodes.
Revised: Let G 1 , ..., G n be n networks each with m nodes.

Original: Let B ( G i ) :
Revised: Let B ( G i ) :

Original: b i, 1 ≤ · · · ≤ b i,m − 1 and
Revised: b i, 1 ≤ · · · ≤ b i,m − 1 and

Original: D ( G i ) :
Revised: D ( G i ) :

Original: d i, 1 ≤ · · · ≤ d i, 1+ m ( m − 3) / 2 be the topology of G i .
Revised: d i, 1 ≤ · · · ≤ d i, 1+ m ( m − 3) / 2 be the topology of G i .

Original: It follows that the l -th smallest birth value of the topological centroid of the n networks is given by the mean of all the l -th smallest birth values of such networks, i.e., (cid:80) ni
Revised: It follows that the l -th smallest birth value of the topological centroid of the n networks is given by the mean of all the l -th smallest birth values of such networks, i.e., (cid:80) ni

Original: =1 b i,l /n .
Revised: =1 b i,l /n .

Original: Similarly, the l -th smallest death value of the topological centroid is given by (cid:80) ni =1 d i,l /n .
Revised: Similarly, the l -th smallest death value of the topological centroid is given by (cid:80) n i =1 d i,l /n .

Original: Since Eq.
Revised: Since Eq.

Original: (7) is quadratic, Lemma 1 can be proved by setting its derivative equal to zero.
Revised: (7) is quadratic, Lemma 1 can be proved by setting its derivative equal to zero.

Original: The complete proof is given in the supplementary material.
Revised: The complete proof is given in Appendix A.

Original: The results in (Songdechakraiwut & Chung, 2020) may be used to ﬁnd the centroid of different size networks.
Revised: The results in (Songdechakraiwut & Chung, 2020; Songdechakraiwut et al., 2022) may be used to ﬁnd the centroid of different size networks.

Original: For the most general case considering both correspondence of edge weights and topology, i.e., when λ ∈ (0 , 1) , we can optimize the objective function given in (6) by gradient descent.
Revised: For the most general case considering both correspondence of edge weights and topology, i.e., when λ ∈ (0 , 1) , we can optimize the objective function given in (6) by gradient descent.

Original: Let H =
Revised: Let H =

Original: ( V, u ) be a cluster representative being estimated given C h .
Revised: ( V, u ) be a cluster representative being estimated given C h .

Original: The gradient of the squared topological distance ∇ H d 2 top ( G, H ) with respect to edge weights u = ( u ij ) is given as a gradient matrix whose ij -th entry is
Revised: The gradient of the squared topological distance ∇ H d 2 top ( H, G ) with respect to edge weights u = ( u ij ) is given as a gradient matrix whose ij -th entry is

Original: [Equation]
Revised: [Equation]

Original: This follows because the edge weight set decomposes into the collection of births and the collection of deaths.
Revised: This follows because the edge weight set decomposes into the collection of births and the collection of deaths.

Original: Intuitively, by slightly adjusting the edge weight u ij , we have the slight adjustment of either a birth value in B ( H ) or a death value in D ( H ) , which slightly changes the topology of the network H .
Revised: Intuitively, by slightly adjusting the edge weight u ij , we have the slight adjustment of either a birth value in B ( H ) or a death value in D ( H ) , which slightly changes the topology of the network H .

Original: The gradient computation consists of computing persistence barcodes and ﬁnding the optimal matching using the closed-form solution given in (3), requiring O ( n log n ) operations, where n is the number of edges in networks.
Revised: The gradient computation consists of computing persistence barcodes and ﬁnding the optimal matching using the closed-form solution given in (3), requiring O ( n log n ) operations, where n is the number of edges in networks.

Original: Evaluating the gradient for (6) requires computing the gradients of all the observed networks.
Revised: Evaluating the gradient for (6) requires computing the gradients of all the observed networks.

Original: This can be computationally demanding when the size of a dataset is large.
Revised: This can be computationally demanding when the size of a dataset is large.

Original: However, an equivalent minimization problem that allows faster computation is possible using the following result:
Revised: However, an equivalent minimization problem that allows faster computation is possible using the following result:

Original: Lemma 2.
Revised: Lemma 2.

Original: [Equation]
Revised: [Equation]

Original: where w h = ( w h,ij ) are edge weights in the sample mean network M h
Revised: where w h = ( w h,ij ) are edge weights in the sample mean network M h

Original: = ( V, w h ) of cluster C h , and (cid:99) M top,h is the topological centroid of networks in cluster C h .
Revised: = ( V, w h ) of cluster C h , and (cid:99) M top,h is the topological centroid of networks in cluster C h .

Original: Thus, instead of computing the gradient for every network in the set, it is sufﬁcient to compute the gradient at the cluster sample means M h and topological centroids (cid:99)
Revised: Thus, instead of computing the gradient for every network in the set, it is sufﬁcient to compute the gradient at the cluster sample means M h and topological centroids (cid:99)

Original: M top,h .
Revised: M top,h .

Original: Hence, one only needs to perform topological interpolation between the sample mean network M h and the topological centroid (cid:99)
Revised: Hence, one only needs to perform topological interpolation between the sample mean network M h and the topological centroid (cid:99)

Original: M top,h of each cluster.
Revised: M top,h of each cluster.

Original: That is, the optimal representative is the one whose geometric location is close to M h and topology is similar to (cid:99)
Revised: That is, the optimal representative is the one whose geometric location is close to M h and topology is similar to (cid:99)

Original: M top,h .
Revised: M top,h .

Original: At each current iteration, we take a step in the direction of negative gradient with respect to an updated H  from the previous iteration.
Revised: At each current iteration, we take a step in the direction of negative gradient with respect to an updated H h from the previous iteration.

Original: Note that the proposed method constrains the search of cluster centroids to a space of meaningful networks through topological interpolation.
Revised: Note that the proposed method constrains the search of cluster centroids to a space of meaningful networks through topological interpolation.

Original: In contrast, the sample mean, such as would be used in a naive application of k -means, does not necessarily represent a meaningful network.
Revised: In contrast, the sample mean, such as would be used in a naive application of k -means, does not necessarily represent a meaningful network.

Original: Furthermore, we have the following theorem:
Revised: Furthermore, we have the following theorem:

Original: Theorem 1.
Revised: Theorem 1.

Original: The topological clustering algorithm monotonically decreases L in (5) and terminates in a ﬁnite number of steps at a locally optimal partition.
Revised: The topological clustering algorithm monotonically decreases L in (5) and terminates in a ﬁnite number of steps at a locally optimal partition.

Original: The proofs for Lemma 2 and Theorem 1 are provided in the supplementary material.
Revised: The proofs for Lemma 2 and Theorem 1 are provided in Appendix A.

Original:  V ALIDATION USING SIMULATED NETWORKS
Revised:  V ALIDATION U SING S IMULATED N ETWORKS

Original: Simulated networks of different topological structure are used to evaluate the clustering performance of the proposed approach relative to that of seven other methods.
Revised: Simulated networks of different topological structure are used to evaluate the clustering performance of the proposed approach relative to that of seven other methods.

Original: We use the term topology to denote the proposed approach with λ > 0 .
Revised: We use the term topology to denote the proposed approach with λ > 0 .

Original: Clustering using the special case of λ = 0 is termed the edge-weight method.
Revised: Clustering using the special case of λ = 0 is termed the edge-weight method.

Original: Recallthat the graph ﬁltration used decomposes the edge weights into two groups of birth and death values corresponding to connected components and cycles (section 2.2).
Revised: Recall the graph ﬁltration deﬁned in section 2.2 decomposes the edge weights into two groups of birth and death values corresponding to connected components and cycles.

Original: In order to assess the impact of separating edge weights into birth and death sets, we evaluate an ad hoc method called sort that applies k -means to feature vectors obtained by simply sorting edge weights.
Revised: In order to assess the impact of separating edge weights into birth and death sets, we evaluate an ad hoc method called sort that applies k -means to feature vectors obtained by simply sorting edge weights.

Original: In addition, we evaluate several clustering algorithms that utilize network representations.
Revised: In addition, we evaluate several clustering algorithms that utilize network representations.

Original: Two persistent homology clustering methods based on the well-known Rips ﬁltration (Ghrist, 2008) are evaluated.
Revised: Two persistent homology clustering methods based on conventional two-dimensional barcodes (Otter et al., 2017) are evaluated.

Original: The Rips ﬁltration results in standard two-dimensional barcodes representing connected components and cycles.

Original: Consequently, two separate Wasserstein distances are needed to match connected components to connected components and cycles to cycles.

Original: The Wasserstein method performs clustering using k -medoids based onthe sum of the two Wasserstein distances for connected components and cycles.
Revised: The Wasserstein method performs clustering using k -medoids based on Wasserstein distance between two-dimensional barcodes for connected components and cycles.

Original: The computational complexity of the Wasserstein method is managed by utilizing an approximation algorithm
Revised: The computational complexity of the Wasserstein method is managed by utilizing an approximation algorithm

Original: (Lacombe et al., 2018) to compute  Wasserstein distance.
Revised: (Lacombe et al., 2018) to compute the Wasserstein distance.

Original: A Wasserstein barycenter clustering algorithm called Bregman Alternating Direction Method of Multipliers (B-ADMM)
Revised: A Wasserstein barycenter clustering algorithm called Bregman Alternating Direction Method of Multipliers (B-ADMM)

Original: (Ye et al., 2017) is used to cluster two-dimensional barcodes for cycles.
Revised: (Ye et al., 2017) is used to cluster two-dimensional barcodes for cycles.

Original: In order to make B-ADMM computationally tractable, each barcode is limited to no more than themost persistent cycles.
Revised: In order to make B-ADMM computationally tractable, each barcode is limited to no more than the 50 most persistent cycles.

Original: The Net Stats clustering approach uses k -means to cluster three-dimensional feature vectors composed of the following network summary statistics: (1) average weighted degree over all nodes (Rubinov &
Revised: The Net Stats clustering approach uses k -means to cluster three-dimensional feature vectors composed of the following network summary statistics: (1) average weighted degree over all nodes (Rubinov &

Original: Sporns, 2010), (2) average clustering coefﬁcient over all nodes (Rubinov & Sporns, 2010) and (3) modularity (Newman, 2006; Reichardt & Bornholdt, 2006).
Revised: Sporns, 2010), (2) average clustering coefﬁcient over all nodes (Rubinov & Sporns, 2010) and (3) modularity (Newman, 2006; Reichardt & Bornholdt, 2006).

Original: Lastly, we evaluate two  clustering approaches based on graph kernels with kernel k -means (Dhillon et al., 2004). One is the edge histogram kernel (Sugiyama & Borgwardt, 2015) method.
Revised: Lastly, we evaluate two graph kernel based clustering approaches using kernel k -means (Dhillon et al., 2004): the GraphHopper kernel (Feragen et al., 2013) and the propagation kernel (Neumann et al., 2016).

Original: The other is the propagation kernel method of (Neumann et al., 2016) with each node assigned the continuous attribute of the sum ofedge weights incident to the node.
Revised: Supplementary description and implementation details of the candidate methods are provided in Appendix B.

Original: Initial clusters for all methods are selected at random.
Revised: Initial clusters for all methods are selected at random.

Original: Modular network structure Random modular networks
Revised: Modular network structure Random modular networks

Original: X i are simulated with | V | nodes and m modules such that the nodes are evenly distributed among modules.
Revised: X i are simulated with | V | nodes and m modules such that the nodes are evenly distributed among modules.

Original: Figure 2 displays modular networks with | V | = 60 nodes and m = 5 modules such that | V | /m
Revised: Figure 2 displays modular networks with | V | = 60 nodes and m = 5 modules such that | V | /m

Original: = 12 nodes are in each module.
Revised: = 12 nodes are in each module.

Original: Edges connecting two nodes within the same module are assigned a random weight following a normal distribution
Revised: Edges connecting two nodes within the same module are assigned a random weight following a normal distribution

Original: N ( µ, σ 2 ) with probability r or otherwise Gaussian noise N (0 , σ 2 ) with probability 1 − r .
Revised: N ( µ, σ 2 ) with probability r or otherwise Gaussian noise N (0 , σ 2 ) with probability 1 − r .

Original: On the other hand, edges connecting nodes in different modules have probability 1 − r of being N ( µ, σ 2 ) and probability r of being N (0 , σ 2 ) .
Revised: On the other hand, edges connecting nodes in different modules have probability 1 − r of being N ( µ, σ 2 ) and probability r of being N (0 , σ 2 ) .

Original: The modular structure becomes more pronounced as the within-module connection probability r increases.
Revised: The modular structure becomes more pronounced as the within-module connection probability r increases.

Original: Any negative edge weights are set to zero.
Revised: Any negative edge weights are set to zero.

Original: This procedure yields random networks
Revised: This procedure yields random networks

Original: X i that exhibit topological connectedness.
Revised: X i that exhibit topological connectedness.

Original: We use µ = 1 and σ = 0 .
Revised: We use µ = 1 and σ = 0 .

Original: 5 universally throughout the study.
Revised: 5 universally throughout the study.

Original: Simulation Three groups of modular networks L 1 = {X i } 20 i =1 , L 2 = {X i } 

Revised: Simulation Three groups of modular networks L 1 = {X i } 20 i =1 , L 2 = {X i }

Original: i =21 and L 3 = {X i } 60 i = corresponding to m = 2 , 3 and 5 modules, respectively, are simulated.
Revised: i =21 and L 3 = {X i } 60 i = corresponding to m = 2 , 3 and 5 modules, respectively, are simulated.

Original: This results in 60 networks in the dataset, each of which has a group label L 1 , L 2 or L 3 .
Revised: This results in 60 networks in the dataset, each of which has a group label L 1 , L 2 or L 3 .

Original: We consider r = 0 .
Revised: We consider r = 0 .

Original: 7 and 0 .
Revised: 7 and 0 .

Original: 6 to vary the strength of the modular structure, as illustrated in Figure 2.
Revised: 6 to vary the strength of the modular structure, as illustrated in Figure 2.

Original: The dataset is partitioned into three clusters C 1 , C 2 and C 3 using the candidate algorithms.
Revised: The dataset is partitioned into three clusters C 1 , C 2 and C 3 using the candidate algorithms.

Original: Clustering performance is then evaluated by ﬁrst assigning each cluster to the group label that is most frequent in that cluster and then calculating the accuracy statistic s as the fraction of correctly labeled networks, i.e., s = 160 (cid:80) 

Revised: Clustering performance is then evaluated by ﬁrst assigning each cluster to the group label that is most frequent in that cluster and then calculating the accuracy statistic s as the fraction of correctly labeled networks, i.e., s = 160 (cid:80)

Original: i =1 max j {| C i ∩ L j |} , where | C i ∩ L j | denotes the number of common networks in both C i and L j .
Revised: i =1 max j {| C i ∩ L j |} , where | C i ∩ L j | denotes the number of common networks in both C i and L j .

Original: Note this evaluation of clustering performance is called purity , which not only is transparent and interpretable but also works well in this simulation study where the number and size of clusters are small and balanced, respectively (Manning et al., 2008).
Revised: Note this evaluation of clustering performance is called purity , which not only is transparent and interpretable but also works well in this simulation study where the number and size of clusters are small and balanced, respectively (Manning et al., 2008).

Original: Since the distribution of the accuracy s is unknown, a permutation test is used to determine the empirical distribution under the null hypothesis that sample networks and their group labels are independent (Ojala & Garriga, 2010).
Revised: Since the distribution of the accuracy s is unknown, a permutation test is used to determine the empirical distribution under the null hypothesis that sample networks and their group labels are independent (Ojala & Garriga, 2010).

Original: The empirical distribution is calculated by repeatedly shufﬂing the group labels and then re-computing the corresponding accuracy for one million random permutations.
Revised: The empirical distribution is calculated by repeatedly shufﬂing the group labels and then re-computing the corresponding accuracy for one million random permutations.

Original: By comparing the observed accuracy to this empirical distribution, we can determine the statistical signiﬁcance of the clustering performance.
Revised: By comparing the observed accuracy to this empirical distribution, we can determine the statistical signiﬁcance of the clustering performance.

Original: The p -value is calculated as the fraction of permutations that give accuracy higher than the observed accuracy s .
Revised: The p -value is calculated as the fraction of permutations that give accuracy higher than the observed accuracy s .

Original: The average p -value and average accuracy across 100 different initial assignments is reported.
Revised: The average p -value and average accuracy across 100 different initial assignments are reported.

Original: Results Figure 3 indicates that all methods considered perform relatively well with pronounced modularity ( r = 0 . 9 ).
Revised: Results Figure 3 indicates that all methods considered perform relatively well with pronounced modularity ( r = 0 . 9 ).

Original: As the modularity strength diminishes with decreasing r , clustering performance also decreases.
Revised: As the modularity strength diminishes with decreasing r , clustering performance also decreases.

Original: The proposed topological clustering algorithm ( λ = 1 ) performs relatively well for the more nuanced modularity associated with r = 0 .
Revised: The proposed topological clustering algorithm ( λ = 1 ) performs relatively well for the more nuanced modularity associated with r = 0 .

Original: 7 and 0 . 6 .
Revised: 7 and 0 . 6 .

Original: Since the dataset is purposefully generated to exhibit dependency between the sample networks and their group labels, the p -value indicates the degree of statistical signiﬁcance to which structure is differentiated (Ojala & Garriga, 2010).
Revised: Since the dataset is purposefully generated to exhibit dependency between the sample networks and their group labels, the p -value indicates the degree of statistical signiﬁcance to which structure is differentiated (Ojala & Garriga, 2010).

Original: Small p -values indicate the algorithm is able to differentiate network structure.
Revised: Small p -values indicate the algorithm is able to differentiate network structure.

Original: The proposed method has very low p -values indicating that its improved accuracy over the baseline methods is signiﬁcant.
Revised: The proposed method has very low p -values indicating that its improved accuracy over the baseline methods is signiﬁcant.

Original: The Wasserstein and B-ADMM methods require very high computation costs for the Rips ﬁltration (Otter et al., 2017), and the approximations of Wasserstein distance (Lacombe et al., 2018) and barycenter (Ye et al., 2017).
Revised: The Wasserstein and B-ADMM methods require very high computation costs for the conventional two-dimensional barcodes (Otter et al., 2017), and the approximations of Wasserstein distance (Lacombe et al., 2018) and barycenter (Ye et al., 2017).

Original: The ad hoc sort method performs nearly identically to the topology approach, showing that the separation into births of connected components and deaths of cycles has no apparent impact for these simulated networks with densely connected structure.
Revised: The ad hoc sort method performs nearly identically to the topology approach, showing that the separation into births of connected components and deaths of cycles has no apparent impact for these simulated networks with densely connected structure.

Original: However, we hypothesize the separation will impact performance when networks are sparse.
Revised: However, we hypothesize the separation will impact performance when networks are sparse.

Original: The complete experimental results including an additional set of module sizes m = 2 , 5 and 10 are provided in the supplementary material.
Revised: Supplementary experimental results including an additional set of module sizes m = 2 , 5 and 10 are provided in Appendix B.

Original: Figure 4 displays topological clustering performance as a function of λ for the strongest ( r = 0 . 9 ) and weakest ( r = 0 . 6 ) degree of modularity.
Revised: Figure 4 displays topological clustering performance as a function of λ for the strongest ( r = 0 . 9 ) and weakest ( r = 0 . 6 ) degree of modularity.

Original: The performance is not very sensitive to the value of λ .
Revised: The performance is not very sensitive to the value of λ .

Original: When the within-module connection probability is small ( r = 0 . 6 ), increasing the weight of the topological distance by increasing λ results in the best performance.
Revised: When the within-module connection probability is small ( r = 0 . 6 ), increasing the weight of the topological distance by increasing λ results in the best performance.

Original: We hypothesize that in this case the relatively strong random nature of the edge weights introduces noise into the geometric term that hinders performance as λ decreases.
Revised: We hypothesize that in this case the relatively strong random nature of the edge weights introduces noise into the geometric term that hinders performance as λ decreases.

Original: Conversely, the topological distance appears less sensitive to this particular edge weight noise, resulting in the best performance when λ = 1 .
Revised: Conversely, the topological distance appears less sensitive to this particular edge weight noise, resulting in the best performance when λ = 1 .

Original:  A PPLICATION TO FUNCTIONAL BRAIN NETWORKS
Revised:  A PPLICATION TO F UNCTIONAL B RAIN N ETWORKS

Original: Dataset We evaluate our method using an extended brain network dataset from the anesthesia study reported by Banks et al.
Revised: Dataset We evaluate our method using an extended brain network dataset from the anesthesia study reported by Banks et al.

Original: (2020) (see the supplementary material).
Revised: (2020) (see Appendix C).

Original: The measured brain networks are based on alpha band (8-12 Hz) weighted phase lag index (Vinck et al., 2011) applied to 10-second segments of resting state intracranial electroencephalography recordings from eleven neurosurgical patients administered increasing doses of the general anesthetic propofol just prior to surgery.
Revised: The measured brain networks are based on alpha band (8-12 Hz) weighted phase lag index (Vinck et al., 2011) applied to 10-second segments of resting state intracranial electroencephalography recordings from eleven neurosurgical patients administered increasing doses of the general anesthetic propofol just prior to surgery.

Original: The network size varies from 89 to 199 nodes while the number of networks (10-second segments) per subject varies from 71 to 119.
Revised: The network size varies from 89 to 199 nodes while the number of networks (10-second segments) per subject varies from 71 to 119.

Original: Each segment is labeled as one of the three arousal states: pre-drug wake (WA), sedated/responsive (S), or unresponsive (U); these labels are used as ground truth in the cluster analysis.
Revised: Each segment is labeled as one of the three arousal states: pre-drug wake (W), sedated/responsive (S), or unresponsive (U); these labels are used as ground truth in the cluster analysis.

Original: Figure 5 illustrates sample mean networks and Betti plots describing topology for a representative subject.
Revised: Figure 5 illustrates sample mean networks and Betti plots describing topology for a representative subject.

Original: Performance evaluation We apply the adjusted Rand index (ARI) (Hubert & Arabie, 1985) to compare clustering performance against ground truth.
Revised: Performance evaluation We apply the adjusted Rand index (ARI) (Hubert & Arabie, 1985) to compare clustering performance against ground truth.

Original: Lower scores indicate less similarity while higher scores show higher similarity between estimated and ground-truth clusters.
Revised: Lower scores indicate less similarity while higher scores show higher similarity between estimated and ground-truth clusters.

Original: Perfect agreement is scored as 1.0.
Revised: Perfect agreement is scored as 1.0.

Original: For each subject, we calculate these performance metrics by running the algorithm for 20 different initial conditions, resulting in 20 scores which are then averaged.
Revised: For each subject, we calculate these performance metrics by running the algorithm for 100 different initial conditions, resulting in 100 scores which are then averaged.

Original: We also average across subjects ( 11 × 20 scores) to obtain a ﬁnal overall score, which describes the overall output clusters across trials and subjects.
Revised: We also average across subjects ( 11 × 100 scores) to obtain a ﬁnal overall score, which describes the overall output clusters across trials and subjects.

Original: We calculated average confusion matrices for each method by assigning each cluster to the state that is most frequent in that cluster.
Revised: We calculate average confusion matrices for each method by assigning each cluster to the state that is most frequent in that cluster.

Original: Cluster analysis All baselines used in the simulation study are evaluated on the brain network dataset.
Revised: Cluster analysis All baselines used in the simulation study are evaluated on the brain network dataset.

Original: In addition, we consider clustering using k -medoids and three previously proposed network distance measures for brain network analyses: Gromov-Hausdorff (GH) (Lee et al., 2012); Kolmogorov-Smirnov (KS) (Chung et al., 2019); and the spectral norm (Banks et al., 2020).
Revised: In addition, we consider clustering using k -medoids and three previously proposed network distance measures for brain network analyses: Gromov-Hausdorff (GH) (Lee et al., 2012); Kolmogorov-Smirnov (KS) (Chung et al., 2019); and the spectral norm (Banks et al., 2020).

Original:  Initial clusters are selected at random for all methods.
Revised: Supplementary description of the three methods is provided in Appendix B. Initial clusters are selected at random for all methods.

Original: Figure 6 reports the ARI of the topological approach for multiple predeﬁned λ ’s including λ = 0 and λ = 1 .
Revised: Figure 6 reports the ARI of the topological approach for multiple predeﬁned λ ’s including λ = 0 and λ = 1 .

Original: The relative performance of topological clustering is reported in Figuresand 8 assuming λ = 0 .
Revised: The relative performance of topological clustering is reported in Figuresand 8 assuming λ = 0 .

Original: 5 , which results in equal weighting of topological and geometric distances.
Revised: 5 , which results in equal weighting of topological and geometric distances.

Original: Results Figure 6 indicates that the performance on the brain network data set varies signiﬁcantly across subjects, but in general varies stably with λ .
Revised: Results Figure 6 indicates that the performance on the brain network data set varies signiﬁcantly across subjects, but in general varies stably with λ .

Original: The best performance occurs with λ between 0.
Revised: The best performance occurs with λ between 0.

Original: and 0.7, suggesting that a combination of geometric and topological distances gives the best result and that performance is not sensitive to the exact value chosen for λ .
Revised: and 0.7, suggesting that a combination of geometric and topological distances gives the best result and that performance is not sensitive to the exact value chosen for λ .

Original: Note that different regions of the brain are functionally differentiated, so it is plausible that topological changes due to changing arousal level vary with location, giving geometric distance a role in the clustering of brain networks.
Revised: Note that different regions of the brain are functionally differentiated, so it is plausible that topological changes due to changing arousal level vary with location, giving geometric distance a role in the clustering of brain networks.

Original: Prior expectations of this sort can be used to determine whether to cluster based only on topology ( λ = 1 ) or a combination of geometry and topology.
Revised: Prior expectations of this sort can be used to determine whether to cluster based only on topology ( λ = 1 ) or a combination of geometry and topology.

Original: Figure 7 compares the ARI performance metric for individual subjects and the combination of all the subjects.
Revised: Figure 7 compares the ARI performance metric for individual subjects and the combination of all the subjects.

Original: All methods demonstrate signiﬁcant variability in individual subject performance.
Revised: All methods demonstrate signiﬁcant variability in individual subject performance.

Original: This is expected due to the variations in effective signal to noise ratio, the number of nodes, and the number of networks in each underlying state.
Revised: This is expected due to the variations in effective signal to noise ratio, the number of nodes, and the number of networks in each underlying state.

Original: Consequently the combined performance across subjects has high variability.
Revised: Consequently the combined performance across subjects has high variability.

Original: The topological method with λ = 0 .
Revised: The topological method with λ = 0 .

Original: 5 performs relatively well across all subjects.
Revised: 5 performs relatively well across all subjects.

Original: Figure 8 illustrates that  proposed approach is particularly effective at separating wake (WA) and unresponsive (U) states.
Revised: Figure 8 illustrates that the proposed approach is particularly effective at separating wake (W) and unresponsive (U) states.

Original: Transitioning from the wake state to the unresponsive state results in dramatic changes in brain connectivity (Banks et al., 2020).
Revised: Transitioning from the wake state to the unresponsive state results in dramatic changes in brain connectivity (Banks et al., 2020).

Original: The majority of errors from the proposed topological clustering approach are associated with the natural overlap between wake and sedated states (S).
Revised: The majority of errors from the proposed topological clustering approach are associated with the natural overlap between wake and sedated states (S).

Original: The sedated brain, in which subjects have been administered propofol but are still conscious, is expected to be more like the wake brain than the unresponsive brain.
Revised: The sedated brain, in which subjects have been administered propofol but are still conscious, is expected to be more like the wake brain than the unresponsive brain.

Original: Thus, errors are expected to be more likely in differentiating sedated and wake states than sedated and unresponsive states.
Revised: Thus, errors are expected to be more likely in differentiating sedated and wake states than sedated and unresponsive states.

Original: This suggests that the types of errors observed in the proposed method are consistent with prior biological expectations.
Revised: This suggests that the types of errors observed in the proposed method are consistent with prior biological expectations.

Original: Impact The demonstrated effectiveness and computational elegance of our approach to clustering networks based on topological similarity will have a high impact on the analysis of large and complex network representations.
Revised: The demonstrated effectiveness and computational elegance of our approach to clustering networks based on topological similarity will have a high impact on the analysis of large and complex network representations.

Original: In the study of brain networks, algorithms that can demonstrate correlates of behavioral states are of considerable interest.
Revised: In the study of brain networks, algorithms that can demonstrate correlates of behavioral states are of considerable interest.

Original: The derived biomarkers of changes in arousal state presented here demonstrate potential for addressing the important clinical problem of passively assessing arousal state in clinical settings, e.g., monitoring depth of anesthesia and in establishing diagnosis and prognosis for patients with traumatic brain injury and other disorders of consciousness. 
Revised: The derived biomarkers of changes in arousal state presented here demonstrate potential for addressing the important clinical problem of passively assessing arousal state in clinical settings, e.g., monitoring depth of anesthesia and in establishing diagnosis and prognosis for patients with traumatic brain injury and other disorders of consciousness. More broadly, the algorithm presented here will contribute to elucidating the neural basis of consciousness, one of the most important open problems in biomedical science. A CKNOWLEDGMENTS

Original: More broadly, the algorithm presented here will contribute to elucidating the neural basis of consciousness, one of  the most important open problems in biomedical science.
Revised: This work is supported in part by the National Institute of General Medical Sciences under award R01 GM109086, and the Lynn H. Matthias Professorship from the University of Wisconsin. A PPENDIX A P ROOFS A. C LOSED - FORM SOLUTION OF TOPOLOGICAL DISTANCE The topological distance d top has the closed-form solution as follows

Revised: [Equation]

Revised: where τ ∗ 0 maps the l -th smallest birth value in B ( G ) to the l -th smallest birth value in B ( H ) and τ ∗maps the l -th smallest death value in D ( G ) to the l -th smallest death value in D ( H ) for all l .

Revised: Proof.

Revised: Let G and H be networks with m nodes.

Revised: The topological distance is originally deﬁned as

Revised: [Equation]

Revised: We rewrite the ﬁrst term as follows

Revised: [Equation]

Revised: where C 1 is a constant.

Revised: Note that (cid:80) b ∈ B ( G ) τ 20 ( b ) is equivalent to (cid:80) b ∈ B ( H ) b 2 , which is a constant over all bijections τ 0 .

Revised: Similarly for the second term, we have

Revised: [Equation]

Revised: Let B ( G ) : x 1 ≤ · · · ≤ x m − 1 and B ( H ) : y 1 ≤ · · · ≤ y m − 1 be the collections of birth values for G and H respectively.

Revised: By the rearrangement inequality (Mitrinovic, 1970), it is known that x 1 y 1 + · · · + x m − 1 y m − 1 ≥ x π (1) y 1 + · · · + x π ( m − 1) y

Revised: ( m − 1) for every permutation x π (1) , ..., x π ( m − 1) of x 1 , ..., x m − 1 .

Revised: Thus, optimal bijection τ ∗ 0 maps the l -th smallest birth value in B ( G ) to the l -th smallest birth value in B ( H ) .

Revised: Similarly for the second term, we have τ ∗ 1 that maps the l -th smallest death value in D ( G ) to the l -th smallest death value in

Revised: D ( H ) .

Revised: A. C LOSED - FORM SOLUTION OF TOPOLOGICAL CENTROID

Revised: Lemma 1.

Revised: Let G 1 , ..., G n be n networks each with m nodes.

Revised: Let B ( G i ) :

Revised: b i, 1 ≤ · · · ≤ b i,m − 1 and

Revised: D ( G i ) :

Revised: d i, 1 ≤ · · · ≤ d i, 1+ m ( m − 3) / 2 be the topology of G i .

Revised: It follows that the l -th smallest birth value of the topological centroid of the n networks is given by the mean of all the l -th smallest birth values of such networks, i.e., (cid:80) ni

Revised: =1 b i,l /n .

Revised: Similarly, the l -th smallest death value of the topological centroid is given by (cid:80) ni

Revised: =1 d i,l /n .

Revised: Proof.

Revised: Let M be a cluster representative.

Revised: Recall that the topological centroid minimizes the sum of the squared topological distances, i.e.,

Revised: [Equation]

Revised: Let B ( M ) : γ 1 ≤ · · · ≤ γ m − 1 and D ( M ) : δ 1 ≤ · · · ≤ δ 1+ m ( m − 3) / 2 be the topology of M .

Revised: The ﬁrst term can be expanded as

Revised: [Equation]

Revised: which is quadratic and thus can be solved by setting its derivative equal to zero.

Revised: The solution is given by (cid:98) γ l = (cid:80) ni

Revised: =1 b i,l /n .

Revised: Similarly, the second term can be expanded as

Revised: [Equation]

Revised: which is again quadratic.

Revised: By setting its derivative equal to zero

Revised: [Equation]

Revised: we ﬁnd the minimum at (cid:98) δ l = (cid:80) ni

Revised: =1 d i,l /n .

Revised: A. T OPOLOGICAL INTERPOLATION

Revised: Lemma 2.

Revised: [Equation]

Revised: where w h = ( w h,ij ) are edge weights in the sample mean network M h

Revised: = ( V, w h ) of cluster C h , and (cid:99) M top,h is the topological centroid of networks in cluster C h .

Revised: Proof.

Revised: Let G =

Revised: ( V, w ) ∈ C and Θ =

Revised: ( V, θ ) .

Revised: Denote the cardinality of C by m .

Revised: We will show that

Revised: [Equation]

Revised: We ﬁrst rewrite the geometric term as follows.

Revised: [Equation]

Revised: where C 1 , C 2 are constants.

Revised: Similarly for the topological term, we have (cid:88)

Revised: [Equation]

Revised: C

Revised: [Equation]

Revised: Thus,

Revised: [Equation]

Revised: A. C ONVERGENCE TO A LOCALLY OPTIMAL PARTITION

Revised: Theorem 1.

Revised: The topological clustering algorithm monotonically decreases L and terminates in a ﬁnite number of steps at a locally optimal partition

Revised: [Equation]

Revised: Proof.

Revised: Let C ( t ) =

Revised: { C ( t ) h } kh =1 be a partition of observed networks with corresponding representatives

Revised: M ( t ) =

Revised: { M ( t ) h } kh =1 after the t -th iteration.

Revised: Then,

Revised: [Equation]

Revised: C ( t +1) , M ( t +1) ) , where (a) follows from the assignment step and (b) follows from applying Lemma 1 (when λ = 1 ) and

Revised: Lemma 2 (when λ ∈ (0 , 1) and step size in gradient descent is sufﬁciently small) to the re-estimation step.

Revised: Note that λ = 0 describes conventional edge clustering (MacQueen, 1967).

Revised: This result shows that the algorithm monotonically decreases L .

Revised: Since the number of distinct partitions is ﬁnite and L is monotonically decreasing, eventually L will not be decreased either by (a) the assignment step or (b) re-estimation step and the algorithm terminates.

Revised: B E XPERIMENTAL D ETAILS B. S UPPLEMENTARY DESCRIPTION FOR CANDIDATE METHODS M ETHODS BASED ON TWO - DIMENSIONAL BARCODES

Revised: The Wasserstein method clusters conventional two-dimensional barcodes representing connected components and cycles.

Revised: To this end, two separate Wasserstein distances are needed to match connected components to connected components and cycles to cycles.

Revised: The Wasserstein method performs clustering using k -medoids based on the sum of the two Wasserstein distances for connected components and cycles.

Revised: Bregman Alternating Direction Method of Multipliers (B-ADMM)

Revised: (Ye et al., 2017) is a Wasserstein barycenter based algorithm for clustering discrete distributions.

Revised: We follow the standard approach (Turner et al., 2014) to representing underlying discrete distributions on two-dimensional barcodes in the form of Dirac masses.

Revised: The high computational complexity of B-ADMM is managed by limiting the clustering to no more than the 50 most persistent cycles.

Revised: The two methods require computation of two-dimensional persistence barcodes from networks.

Revised: We compute a two-dimensional persistence barcode using the approach of Otter et al.

Revised: (2017) in which edge weights are inverted via the function f ( w )

Revised: = 1 / (1 + w ) .

Revised: Then a point cloud is obtained from the shortest path distance between nodes.

Revised: Finally, we compute Rips ﬁltration (Ghrist, 2008) and its corresponding persistence barcode from the point cloud.

Revised: M ETHODS BASED ON GRAPH KERNELS

Revised: The two graph kernel methods based on GraphHopper and propagation kernels require node continuous attributes.

Revised: We follow the experimental protocol of Borgwardt et al.

Revised: (2020) in which a node attribute is set to the sum of edge weights incident to the node.

Revised: M ETHODS BASED ON NETWORK DISTANCE MEASURES FOR BRAIN NETWORK ANALYSES

Revised: Gromov-Hausdorff (GH) distance (Lee et al., 2012) compares dendrogram shape differences in brain networks.

Revised: GH distance requires brain networks that exhibit metric spaces.

Revised: To this end, we follow the approach of Otter et al.

Revised: (2017) for converting a brain network to a metric space.

Revised: The metric space is obtained by ﬁrst inverting edge weights via the function f ( w ) =

Revised: 1 / (1 + w ) and then using the shortest path distance between nodes as a metric.

Revised: Kolmogorov-Smirnov (KS) distance (Chung et al., 2019) compares topological differences in brain networks using their Betti numbers.

Revised: Two separate KS distances are needed to compare 0-th Betti numbers to 0-th Betti numbers and 1-st Betti numbers to 1-st Betti numbers.

Revised: The sum of the two KS distances is used to compare brain networks.

Revised: We follow the approach of Banks et al.

Revised: (2020) to comparing brain networks using the operator or spectral norm of the difference between network adjacency matrices.

Revised: B. I MPLEMENTATION OF CANDIDATE METHODS

Revised: For all the baseline methods, we used existing implementation codes from authors’ publications and publicly available repository websites.

Revised: We used parameters recommended in the public code without any modiﬁcation.

Revised: Code for Wasserstein distance approximation (Lacombe et al., 2018) is available at https://gudhi.inria.fr/ .

Revised: Code for Bregman Alternating Direction Method of

Revised: Multipliers (B-ADMM) method (Ye et al., 2017) is available at https://github.com/bobye/

Revised: WBC_Matlab .

Revised: Code for the three network summary statistics used in the Net Stats method is available at https://sites.google.com/site/bctnet/ .

Revised: Code for GraphHopper and propagation graph kernels is available at https://github.com/ysig/GraKeL .

Revised: B. S UPPLEMENTARY RESULTS FOR SIMULATED AND MEASURED DATASETS

Revised: For simulated networks, clustering performance comparison is reported in Supplementary Table 1.

Revised: For measured brain networks, confusion matrices of all the candidate methods for subject R376 (see

Revised: Supplementary Table 2) are reported in Supplementary Figure 1.

Revised: [Table]

Revised: [Table]

Revised: C B RAIN N ETWORK D ATASET

Revised: Brain network data were obtained from eleven neurosurgical patients (5 female, ages 19 - 59 years old, median age 38 years old; Supplementary Table 2).

Revised: The patients had been diagnosed with medically refractory epilepsy and were undergoing chronic invasive intracranial electroencephalography (iEEG) monitoring to identify potentially resectable seizure foci.

Revised: All human subjects experiments were carried out in accordance with The Code of Ethics of the World Medical Association (Declaration of

Revised: Helsinki) for experiments involving humans.

Revised: The research protocols were approved by the University of Iowa Institutional Review Board and the National Institutes of Health.

Revised: Written informed consent was obtained from all subjects.

Revised: Research participation did not interfere with acquisition of clinically required data.

Revised: Subjects could rescind consent at any time without interrupting their clinical evaluation.

Revised: Recordings were made using subdural and depth electrodes (Ad-Tech Medical, Oak Creek, WI) placed solely on the basis of clinical requirements, as determined by the team of epileptologists and neurosurgeons.

Revised: The brain network dataset was based on recordings made in the operating room prior to electrode removal, before and during induction of general anesthesia with propofol.

Revised: Details of the experimental procedure, recording, and electrophysiological data analysis are available in (Banks et al., 2020).

Revised: D T OY I LLUSTRATION OF T OPOLOGICAL C LUSTERING

Revised: Supplementary Figure 2 illustrates a cluster analysis of the proposed topological approach using only topological centroids ( λ = 1 ) on a toy dataset generated using random modular networks described in Section 4 in the main body of the paper