Fuzzy c-Means Clustering for Persistence Diagrams

1 INTRODUCTION . Persistence diagrams , a concise representation of the topology of a point cloud with strong theoretical guarantees , have emerged as a new tool in the field of data analysis ( Edelsbrunner & Harer , 2010 ) . Persistence diagrams have been successfully used to analyse problems ranging from financial crashes ( Gidea & Katz , 2018 ) to protein binding ( Kovacev-Nikolic et al. , 2014 ) , but the non-Hilbertian nature of the space of persistence diagrams means it is difficult to directly use persistence diagrams for machine learning . In order to better integrate diagrams into machine learning workflows , efforts have been made to map them into a more manageable form ; primarily through embeddings into finite feature vectors , functional summaries , or by defining a positive-definite kernel on diagram space . In all cases , this explicitly or implicitly embeds diagrams into a Hilbert space which deforms the metric structure , potentially losing important information . With the exception of Topological Autoencoders , techniques to integrate these persistence-based summaries as topological regularisers and loss functions currently require prior knowledge about the correct topology of the dataset , which is clearly not feasible in most scenarios . Against this background , we give an algorithm to perform Fuzzy c-Means ( FCM ) clustering ( Bezdek , 1980 ) directly on collections of persistence diagrams , giving an important unsupervised learning algorithm and enabling learning from persistence diagrams without deforming the metric structure . We perform the convergence analysis for our algorithm , giving the same guarantees as traditional FCM clustering : that every convergent subsequence of iterates tends to a local minimum or saddle point . We demonstrate the value of our fuzzy clustering algorithm by using it to cluster datasets that benefit from both the topological and fuzzy nature of our algorithm . We apply our technique in two settings : lattice structures in materials science and the decision boundaries of CNNs . A key property for machine learning in materials science has been identified as “ invariance to the basis symmetries of physics [ ... ] rotation , reflection , translation ” ( Schmidt et al. , 2019 ) . Geometric clustering algorithms do not have this invariance , but persistence diagrams do , making them ideally suited for this application ; we can cluster transformed lattice structure datasets where geometric equivalents fail . In addition to this , our probabilistic membership values allow us to rank the top-k most likely lattices assigned to a cluster . This is particularly important in materials science , as further investigation requires expensive laboratory time and expertise . Our second application is inspired by Ramamurthy et al . ( 2019 ) , who show that models perform better on tasks if they have topologically similar decision boundaries . We use our algorithm to cluster models and tasks by the persistence diagrams of their decision boundaries . Not only is our algorithm able to successfully cluster models to the correct task , based just on the topology of its decision boundary , but we show that higher membership values imply better performance on unseen tasks . 1.1 RELATED WORK . Means of persistence diagrams . Our work relies on the existence of statistics in the space of persistence diagrams . Mileyko et al . ( 2011 ) first showed that means and expectations are well-defined in the space of persistence diagrams . Specifically , they showed that the Fréchet mean , an extension of means onto metric spaces , is well-defined under weak assumptions on the space of persistence diagrams . Turner et al . ( 2012 ) then developed an algorithm to compute the Fréchet mean . We adapt the algorithm by Turner et al . to the weighted case , but the combinatoric nature of their algorithm makes it computationally intense . Lacombe et al . ( 2018 ) framed the computation of means and barycentres in the space of persistence diagram as an optimal transport problem , allowing them to use the Sinkhorn algorithm ( Cuturi & Doucet , 2014 ) for fast computation of approximate solutions . The vectorisation of the diagram required by the algorithm by Lacombe et al . makes it unsuitable for integration into our work , as we remain in the space of persistence diagrams . Techniques to speed up the matching problem fundamental to our computation have also been proposed by Vidal et al . ( 2020 ) and Kerber et al . ( 2017 ) . Learning with persistence-based summaries . Integrating diagrams into machine learning workflows remained challenging even with well-defined means , as the space is non-Hilbertian ( Turner & Spreemann , 2019 ) . As such , efforts have been made to map diagrams into a Hilbert space ; primarily either by embedding into finite feature vectors ( Kališnik , 2018 ; Fabio & Ferri , 2015 ; Chepushtanova et al. , 2015 ) or functional summaries ( Bubenik , 2015 ; Rieck et al. , 2019 ) , or by defining a positivedefinite kernel on diagram space ( Reininghaus et al. , 2015 ; Carrière et al. , 2017 ; Le & Yamada , 2018 ) . These vectorisations have been integrated into deep learning either by learning parameters for the embedding ( Hofer et al. , 2017 ; Carrière et al. , 2020 ; Kim et al. , 2020 ; Zhao & Wang , 2019 ; Zieliński et al. , 2019 ) , or as part of a topological loss or regulariser ( Chen et al. , 2018 ; Gabrielsson et al. , 2020 ; Clough et al. , 2020 ; Moor et al. , 2019 ) . However , the embeddings used in these techniques deform the metric structure of persistence diagram space ( Bubenik & Wagner , 2019 ; Wagner , 2019 ; Carrière & Bauer , 2019 ) , potentially leading to the loss of important information . Furthermore , these techniques generally require prior knowledge of a ‘ correct ’ target topology which can not plausibly be known in most scenarios . In comparison , our algorithm acts in the space of persistence diagrams so it does not deform the structure of diagram space via embeddings , and is entirely unsupervised , requiring no prior knowledge about the topology . Hard clustering . Maroulas et al . ( 2017 ) gave an algorithm for hard clustering persistence diagrams based on the algorithm by Turner et al . Lacombe et al . ( 2018 ) gave an alternate implementation of hard clustering based on their algorithm for barycentre computation , providing a computational speed-up over previous the work by Maroulas et al . The primary advantages of our work over previous work on hard clustering are as follows . ( i ) The probabilistic membership values allow us to rank datasets in the cluster , enabling top-k candidate selection in settings where verifying correctness is expensive . The value provided by this fuzzy information is demonstrated in the experiments . ( ii ) The fuzzy membership values provide information about proximity to all clusters , whereas hard labelling loses most of that information . In our experiments we demonstrate that this additional information can be utilised in practice . ( iii ) The weighted cost function makes the convergence analysis ( which we provide ) entirely nontrivial in comparison to the non-fuzzy case . We consider this convergence analysis a primary contribution of our paper . ( iv ) Fuzzy membership values have been shown to be more robust to noise than discrete labels ( Klawonn , 2004 ) . ( v ) Unlike hard clustering , fuzzy clustering is analytically differentiable , allowing integration of the fuzzy clustering step into deep learning methods ( Wilder et al. , 2019 ) . Geometric equivalents . The most similar unsupervised learning technique to our algorithm is Wasserstein Barycentre Clustering ( WBC ) . It clusters datasets of point clouds by the Wasserstein distance between the point clouds , rather than the Wasserstein distance between their persistence diagrams . We compare our algorithm experimentally to WBC using ADMM ( Ye & Li , 2014 ) , Bregman ADMM ( Ye et al. , 2017 ) , Subgradient Descent ( Cuturi & Doucet , 2014 ) , Iterative Bregman Projection ( Benamou et al. , 2015 ) , and full linear programming ( Li & Wang , 2008 ) . Each of these algorithms computes or approximates the Wasserstein barycentre in different ways . Theoretically , fuzzy discrete distribution clustering ( d. A. T. de Carvalho et al. , 2015 ) is similar to our algorithm , but the addition of the diagonal in the persistence diagram makes our work distinct . 1.2 OUR CONTRIBUTIONS . 1 . Our main contribution is an algorithm for Fuzzy c-Means clustering of persistence diagrams , along with the convergence analysis . Given a collection of persistence diagrams D1 , . . . , Dn , we alternatively calculate cluster centres M1 , . . . , Mc and membership values rjk 2 [ 0 , 1 ] which denote the degree to which diagram Dj is associated with cluster Mk . We prove Theorem 1 , showing that every convergent subsequence of these alternative update steps tends to a local minimum or saddle point of the cost function . This is the same convergence guarantee provided by traditional FCM clustering ( Bezdek et al. , 1987 ) , but requires additional work as the space of persistence diagrams with the Wasserstein distance has far weaker theoretical properties than Euclidean space . 2 . Updating the cluster centres requires computing the weighted Fréchet mean . We extend the algorithm given by Turner et al . ( 2012 ) to the weighted case , justifying our addition of weights by extending their proof to show that the updated algorithm converges . 3 . We implement our algorithm in Python , available in the supplementary materials . It works with persistence diagrams from commonly used open-source libraries for Topological Data Analysis ( TDA ) ,1 so is available for easy integration into current workflows , offering a powerful unsupervised learning algorithm to data science practitioners using TDA . 4 . We demonstrate the application of our algorithm to settings where ( i ) the properties of persistence diagrams makes clustering them the natural choice over geometric equivalents and ( ii ) the probabilistic membership values can be used to rank candidates for top-k selection . Our algorithm classifies transformed lattice structures from materials science where geometric equivalents fail , whilst giving probabilistic rankings to help prioritise expensive further investigation . We also cluster the persistence diagrams of decision boundaries and labelled datasets , showing that our fuzzy clustering captures information about model performance on unseen tasks . 2 TOPOLOGICAL PRELIMINARIES . Topological Data Analysis emerged from the study of algebraic topology , providing a toolkit to fully describe the topology of a dataset . We offer a quick summary below ; for more comprehensive details see Edelsbrunner & Harer ( 2010 ) . A set of points in Rd are indicative of the shape of the distribution they are sampled from . By connecting points that are pairwise within ✏ > 0 distance of each other , we can create an approximation of the distribution called the Vietoris-Rips complex ( Vietoris , 1927 ) . Specifically , we add the convex hull of any collection of points that are pairwise at most ✏ apart to the ✏-Vietoris-Rips complex . However , choosing an ✏ remains problematic ; too low a value and key points can remain disconnected , too high a value and the points become fully connected . To overcome this we use persistence : we consider the approximation over all values of ✏ simultaneously , and study how the topology of that approximation evolves as ✏ grows large . We call the collection of complexes for all ✏ a filtration . For each ✏ , we compute the p-homology group . This tells us the topology of the ✏-Vietoris-Rips complex : the 0-homology counts the number of connected components , the 1-homology counts the number of holes , the 2-homology counts the number of voids , etc . ( Edelsbrunner et al. , 2000 ) . 1Dionysus and Ripser ( Bauer , 2019 ) . The p-persistent homology ( p-PH ) group is created by summing the p-homology groups over all ✏ . This results in a p-PH group that summarises information about the topology of the dataset at all granularities . If a topological feature , such as a connected component or hole , persists throughout a large range of granularities , then it ’ s more likely to be a feature of the distribution . If it only persists for a short amount of time , then it ’ s more likely to be noise ( Cohen-Steiner et al. , 2007 ) . We can stably map a p-PH group into a multiset in the extended plane called a persistence diagram ( Chazal et al. , 2012 ) . Each topological feature has a birth and death : a feature is born when it enters the complex , and dies when the complex grows enough to destroy it . For example , in Figure 1 ( a ) , a feature is born at ✏ = 1 when four lines form a hole . This feature dies at ✏ = 1.42 when the hole is filled in . This is shown in the persistence diagram in Figure 1 ( b ) as a point at ( 1 , 1.42 ) in 1-PH . By computing the birth/death points for each topological feature in the filtration , we get a complete picture of the topology of the point cloud at all granularities ( Zomorodian & Carlsson , 2005 ) . The persistence diagram is the collection of birth/death points , along with the diagonal = { ( a , a ) : a 2 R } with infinite multiplicity , added in order to make the space of persistence diagrams complete ( Mileyko et al. , 2011 ) .

Summary: The authors propose a novel algorithm for the fuzzy clustering of persistence diagrams. To determine cluster centroids, the Wasserstein-2 distance is used to minimize the weighted Fr\’echet mean between a potential cluster center and all PDs considered for clustering. The authors proof convergence of the clustering algorithm and conduct experiments on 1) synthetic data, 2) lattice structures, and 3) decision boundaries of neural networks. In the last experiment is was shown that models whose PDs cluster close to the PDs of a given task lead to higher classification performance than random classifiers, demonstrating the merit of PDs as a useful tool for model selection.