Fast topological clustering with Wasserstein distance

1 INTRODUCTION . Network models are extremely useful representations for complex data . Significant attention has been given to cluster analysis within a single network , such as detecting community structure ( Newman , 2006 ; Rohe et al. , 2011 ; Yin et al. , 2017 ) . Less attention has been given to clustering of collections of network representations . Clustering approaches typically group similar networks based on comparisons of edge weights ( Xu & Wunsch , 2005 ) , not topology . Assessing similarity of networks based on topological structure offers the potential for new insight , given the inherent topological patterns exhibited by most real-world networks . However , extracting meaningful network topology is a very difficult task , especially for large and dense networks whose node degrees range over multiple orders of magnitude ( Barrat et al. , 2004 ; Bullmore & Sporns , 2009 ; Honey et al. , 2007 ) . Persistent homology ( Barannikov , 1994 ; Edelsbrunner et al. , 2000 ; Wasserman , 2018 ) has recently emerged as a powerful tool for understanding , characterizing and quantifying complex networks ( Carrière et al. , 2020 ; Chung et al. , 2019 ) . Persistent homology represents a network using topological features such as connected components and cycles . Many networks naturally divide into modules or connected components ( Bullmore & Sporns , 2009 ; Honey et al. , 2007 ) . Similarly , cycles are ubiquitous and are often used to describe information propagation , robustness and feedback mechanisms ( Keizer et al. , 1995 ; Kwon & Cho , 2007 ; Ozbudak et al. , 2005 ; Venkatesh et al. , 2004 ; Weiner et al. , 2002 ) . Effective use of such topological descriptors requires a notion of proximity that quantifies the similarity between persistence barcodes , a convenient representation for connected components and cycles ( Ghrist , 2008 ) . Wasserstein distance , which measures the minimal effort to modify one persistence barcode to another ( Rabin et al. , 2011 ) , is an excellent choice due to its appealing geometric properties ( Staerman et al. , 2021 ) and its effectiveness shown in many machine learning applications ( Kolouri et al. , 2017 ; Mi et al. , 2018 ; Solomon et al. , 2015 ) . Importantly , Wasserstein distance can be used to interpolate networks while preserving topological structure ( Songdechakraiwut et al. , 2021 ) , and the mean under the Wasserstein distance , known as Wasserstein barycenter ( Agueh & Carlier , 2011 ) , can be viewed as the topological centroid of a set of networks . The high cost of computing persistence barcodes , Wasserstein distance and the Wasserstein barycenter limit their applications to small scale problems , see , e.g. , ( Clough et al. , 2020 ; Hu et al. , 2019 ; Kolouri et al. , 2017 ; Mi et al. , 2018 ) . Although approximation algorithms have been developed ( Cuturi , 2013 ; Cuturi & Doucet , 2014 ; Lacombe et al. , 2018 ; Li et al. , 2020 ; Solomon et al. , 2015 ; Vidal et al. , 2019 ; Xie et al. , 2020 ; Ye et al. , 2017 ) , it is unclear whether these approximations are effective for clustering complex networks as they inevitably limit sensitivity to subtle topological features . Indeed , more and more studies , see , e.g. , ( Robins & Turner , 2016 ; Xia & Wei , 2014 ) have demonstrated that such subtle topological patterns are important for the characterization of complex networks , suggesting these approximation algorithms are undesirable . Recently , it was shown that the Wasserstein distance and barycenter for network graphs have closedform solutions that can be computed exactly and efficiently ( Songdechakraiwut et al. , 2021 ) because the persistence barcodes are inherently one dimensional . Motivated by this result , we present a novel and computationally practical topological clustering method that clusters complex networks of the same size with intricate topological characteristics . Topological information alone is effective at clustering networks when there is no correspondence between nodes in different networks . However , when networks have meaningful node correspondence , we perform the cluster analysis using combined topological and geometric information to preserve the node correspondence . Statistical validation based on ground truth information is used to demonstrate the effectiveness of our method when discriminating subtle topological features in simulated networks . The method is further illustrated by clustering measured functional brain networks associated with different levels of arousal during administration of general anesthesia . Our proposed method outperforms other clustering approaches in both the simulated and measured data . The paper is organized as follows . Background on our one-dimensional representation of persistence barcodes is given in section 2 , while section 3 presents our topological clustering method . In sections 4 and 5 , we compare the performance of our method to several baseline algorithms using simulated and measured networks , and conclude the paper with a brief discussion of the potential impact of this work . 2 ONE DIMENSIONAL PERSISTENCE BARCODES . 2.1 GRAPH FILTRATION . Consider a network represented as a weighted graph G = ( V , w ) comprising a set of nodes V with symmetric adjacency matrix w = ( wij ) , with edge weight wij representing the relationship between node i and node j . The number of nodes is denoted as |V | . The binary graph G = ( V , w ) of G is defined as a graph consisting of the node set V and binary edge weights w , ij = 1 if wij > and wij = 0 otherwise . We view the binary network G as a 1-skeleton ( Munkres , 2018 ) , a simplicial complex comprising only nodes ( 0-dimensional topological features ) and edges ( 1-dimensional topological features ) . There are no topological features of higher dimensions in the 1-skeleton , in contrast to well-known Rips complexes ( Ghrist , 2008 ) . In the 1-skeleton , there are two types of topological features : connected components and cycles . The number of connected components and the number of cycles in the binary network are referred to as the 0-th Betti number β0 ( G ) and the 1-st Betti number β1 ( G ) , respectively . A graph filtration of G is defined as a collection of nested binary networks ( Lee et al. , 2012 ) : G 0 ⊇ G 1 ⊇ · · · ⊇ G k , where 0 ≤ 1 ≤ · · · ≤ k are filtration values . As increases , more and more edges are removed from the network G since we threshold the edge weights at higher connectivity . For instance , G−∞ has each pair of nodes connected by an edge and thus is a complete graph consisting of a single connected component , while G∞ has no edges and represents the node set . Figure 1 illustrates the graph filtration of a four-node network and the corresponding Betti numbers . Note that other filtrations for analyzing graphs have been proposed , including use of descriptor functions such as heat kernels ( Carrière et al. , 2020 ) and task-specific learning ( Hofer et al. , 2020 ) . 2.2 BIRTH-DEATH DECOMPOSITION . Persistent homology keeps track of birth and death of connected components and cycles over filtration values to determine their persistence , that is , the lifetime from their birth to death over . The persistence is represented as a persistence barcode PB ( G ) comprising intervals [ bi , di ] representing the life-time of a connected component or a cycle that appears at the filtration value bi and vanishes at di . In the edge-weight threshold graph filtration defined in Section 2.1 , connected components are born and cycles die as the filtration value increases ( Chung et al. , 2019 ) . Specifically , β0 is monotonically increasing from β0 ( G−∞ ) = 1 to β0 ( G∞ ) = |V | . There are β0 ( G∞ ) − β0 ( G−∞ ) = |V | − 1 connected components that are born over the filtration . Connected components will never die once they are born , implying that every connected component has death value at∞ . Thus , we can represent their persistence as a collection of finite birth values B ( G ) = { bi } |V |−1i=1 . On the other hand , G−∞ is a complete graph containing all possible cycles ; thus , all cycles have birth values at −∞ . Again , we can represent the persistence of the cycles as a collection of finite death values D ( G ) = { di } . How many cycles are there ? Since the deletion of an edge wij must result in either the birth of a connected component or the death of a cycle , every edge weight must be in either B ( G ) or D ( G ) . Thus , the edge weight set W = { wij |i > j } decomposes into the collection of birth values B ( G ) and the collection of death values D ( G ) . Since G−∞ is a complete graph with |V | ( |V |−1 ) 2 edge weights and |V | − 1 of these weights are associated with the birth of connected components , the number of cycles inG−∞ is thus equal to |V | ( |V |−1 ) 2 − ( |V |−1 ) = 1+ |V | ( |V |−3 ) 2 . In the example of Figure 1 , we have B ( G ) = { e3 , e5 , e6 } and D ( G ) = { e1 , e2 , e4 } . Other graph filtrations ( Carrière et al. , 2020 ; Hofer et al. , 2020 ) do not necessarily share this monotonicity property and consequently one-dimensional barcode representations are not applicable . Finding the birth values in B ( G ) is equivalent to finding edge weights comprising the maximum spanning tree of G and can be done using well-known methods such as Prim ’ s and Kruskal ’ s algorithms ( Lee et al. , 2012 ) . Once B ( G ) is known , D ( G ) is simply given as the remaining edge weights . Finding B ( G ) and D ( G ) requires only O ( n log n ) operations , where n is the number of edges in the network , and thus is extremely computationally efficient . 3 CLUSTERING METHOD . 3.1 TOPOLOGICAL DISTANCE SIMPLIFICATION . Use of edge-weight threshold filtration and limiting consideration to connected components and cycles as topological features results in significant simplification of the 2-Wasserstein distance ( Rabin et al. , 2011 ) between barcode descriptors ( Cohen-Steiner et al. , 2010 ) of networks as follows . Let G and H be two given networks that have the same number of nodes . The topological distance dtop ( G , H ) is defined as the optimal matching cost : ( min τ ∑ p∈PB ( G ) ||p− τ ( p ) ||2 ) 1 2 = ( min τ ∑ p= [ bp , dp ] ∈PB ( G ) [ bp − bτ ( p ) ] 2 + [ dp − dτ ( p ) ] 2 ) 12 , ( 1 ) where the optimization is over all possible bijections τ from barcode PB ( G ) to barcode PB ( H ) . Intuitively , we can think of each interval [ bi , di ] as a point ( bi , di ) in 2-dimensional plane and that the topological distance measures the minimal amount of work to move points in PB ( G ) to PB ( H ) . Note this alternative representation of points in the plane is equivalent to the persistence barcode and called the persistence diagram ( Edelsbrunner & Harer , 2008 ) . Moving a connected component point ( bi , ∞ ) to a cycle point ( −∞ , dj ) or vice versa takes an infinitely large amount of work . Thus , we only need to optimize over bijections that match the same type of topological features . Subsequently , we can equivalently rewrite dtop in terms of B ( G ) , D ( G ) , B ( H ) and D ( H ) as dtop ( G , H ) = ( min τ0 ∑ b∈B ( G ) [ b− τ0 ( b ) ] 2 +min τ1 ∑ d∈D ( G ) [ d− τ1 ( d ) ] 2 ) 12 , ( 2 ) where τ0 is a bijection from B ( G ) to B ( H ) and τ1 is a bijection from D ( G ) to D ( H ) . The first term matches connected components to connected components and the second term matches cycles to cycles . Matching each type of topological feature separately is commonly done in medical imaging and machine learning studies ( Clough et al. , 2020 ; Hu et al. , 2019 ) . The topological distance dtop has a closed-form solution that allows for efficient computation as follows ( Songdechakraiwut et al. , 2021 ) . dtop ( G , H ) = ( ∑ b∈B ( G ) [ b− τ∗0 ( b ) ] 2 + ∑ d∈D ( G ) [ d− τ∗1 ( d ) ] 2 ) 12 , ( 3 ) where τ∗0 maps the l-th smallest birth value in B ( G ) to the l-th smallest birth value in B ( H ) and τ ∗ 1 maps the l-th smallest death value in D ( G ) to the l-th smallest death value in D ( H ) for all l. A proof is in the supplementary material . As a result , the optimal matching cost can be computed quickly and efficiently by sorting birth and death values , and matching them in order . The computational cost of evaluating dtop is O ( n log n ) , where n is the number of edges in networks .

Summary: Given a weighted graph, authors construct a filtration on top of it that is associated with a persistence diagram. This persistence diagram summarizes the topology of the weighted graph. Authors propose then to cluster different graphs using either the distance between the persistence diagrams (that they call the topological distance) or an interpolation between the L2 distance between the weights and the topological distance.  The relevance of this algorithm is then showcased on synthetic examples and real-world datasets.