The Holy Trinity of Topological Machine Learning: Gudhi, Scikit-Learn and Tensorflow | by Mathieu Carrière | Towards Data Science

Hi everyone! Today, I would like to highlight the power of Topological Data Analysis in machine learning and demonstrate how to use it practically using a combination of three Python libraries: Gudhi, Scikit-Learn and Tensorflow.
First things first, let’s talk about Topological Data Analysis (TDA). It’s a relatively small field of data science (especially when compared to machine learning and deep learning), which is nevertheless actively growing and gathering attention from data scientists. Many start-ups and companies are actually integrating these techniques inside their toolboxes (we’re talking about IBM, Fujitsu, Ayasdi...), due to the recent successes it’s had in various applications, such as biology, time series, finance, scientific visualization, computer graphics... I might write a post about the general usefulness and best practices of TDA in the future, so stay tuned ;-)
The goal of TDA is to compute and encode the topology of your data, which means recording the various connected components, loops, cavities, and higher-dimensional structures within your datasets. This can be extremely useful, mostly because this type of information cannot be computed by other descriptors, so TDA really stores a unique set of data features that you cannot find everywhere else. Turns out that these features have very often proved useful for enhancing machine learning predictions, so if you haven’t seen or heard about these features before, I’ll get you up to speed ;-)
I’ve already written a couple of articles on this topic, and you can find many other posts about TDA on Medium, so I won’t spend time on the mathematical definitions, but rather illustrate how you can apply TDA on your datasets by breaking down a classic example of the TDA literature.
This dataset was introduced in a seminal TDA article. It consists of a set of point clouds obtained by generating orbits of the following dynamical system:
This means that we’ll randomly pick an initial point in the unit square, and generate a sequence of points with the equations above. This will give a single point cloud. Now we can repeat the operation as much as we like and end up with a bunch of point clouds. An interesting property of these clouds is that depending on the value of the r parameter that you use to generate the sequence of points, the point clouds end up having very different and interesting structures. For instance, if r = 3.5, the resulting point cloud seems to cover the whole unit square, whereas if r = 4.1, some areas in the unit square are empty: in other words, you have holes in your point clouds. That’s good news for us: TDA can directly compute the presence of these structures.
The way TDA tracks these holes is actually quite simple. Imagine balls of a given radius R centered on each point of your point cloud. If R = 0, the union of these balls is the point cloud itself. If R = infinity, the union of balls is the full unit square. But if R is carefully chosen, the union of balls might contain a lot of topological structures, like holes.
Hence, in order to avoid having to choose manually this “good” value for R, TDA will compute this union of balls for all possible values of R between 0 and infinity, and record the radii for which each hole appeared and disappeared, and use these values as 2D coordinates for some points. The output of TDA is then another point cloud, in which each point represents a hole: it is called a Rips persistence diagram. Assuming the point cloud is stored in a numpy array X of shape (n x 2), the diagram can be computed in two lines with Gudhi with the following piece of code:
import gudhirips = gudhi.RipsComplex(points=X).create_simplex_tree()dgm = rips.persistence()
The task we’ll solve is then to predict the value of r, given the point cloud.
Persistence diagrams are neat, right? The issue with them though is that persistence diagrams computed from different point clouds might have a different number of points (since the point clouds might have a different number of holes). So if you want to predict r from the persistence diagrams with Scikit-Learn, there’s, unfortunately, no direct way since these libraries expect structured vectors as input. That’s why there’s been a large amount of work about turning persistence diagrams into Euclidean vectors of fixed length, or even developing kernels for persistence diagrams. That’s great, but which one should you use then?
Worry no more! Gudhi is here for you again. With its representations module, not only can you compute all of these vectorizations and kernels, but you can even fully use Scikit-Learn in order to cross-validate and/or select the best ones. It’s as simple as this:
import gudhi.representations as tdafrom sklearn.pipeline import Pipelinefrom sklearn.svm import SVCfrom sklearn.ensemble import RandomForestClassifier as RFfrom sklearn.neighbors import KNeighborsClassifier as kNNfrom sklearn.model_selection import GridSearchCVpipe = Pipeline([("TDA",       tda.PersistenceImage()),                 ("Estimator", SVC())])param =    [{"TDA": [tda.SlicedWassersteinKernel()],              "TDA__bandwidth": [0.1, 1.0],             "TDA__num_directions": [20],             "Estimator": [SVC(kernel="precomputed")]},                        {"TDA": [tda.PersistenceWeightedGaussianKernel()],              "TDA__bandwidth": [0.1, 0.01],             "TDA__weight": [lambda x: np.arctan(x[1]-x[0])],              "Estimator": [SVC(kernel="precomputed")]},                        {"TDA": [tda.PersistenceImage()],              "TDA__resolution": [ [5,5], [6,6] ],             "TDA__bandwidth": [0.01, 0.1, 1.0, 10.0],             "Estimator": [SVC()]},                        {"TDA": [tda.Landscape()],              "TDA__resolution": [100],             "Estimator": [RF()]},                       {"TDA": [tda.BottleneckDistance()],              "TDA__epsilon": [0.1],              "Estimator: [kNN(metric="precomputed")]}           ]model = GridSearchCV(pipe, param, cv=3)model = model.fit(diagrams, labels)
In the previous code, I am basically trying kernel SVM with the Sliced Wasserstein kernel and the Persistence Weighted Gaussian kernel, C-SVM with the Persistence Images, Random Forests with the Persistence Landscapes, and a simple k-NN with the so-called bottleneck distance between persistence diagrams. Many other possibilities are available in Gudhi, so you should definitely check it out! You can also take a look at this Gudhi tutorial if you want a little more details.
I’m pretty sure you’ve become a TDA addict by now. If you are still a non-believer, I have something else for you, which was inspired by this paper. Imagine you want to solve an even harder problem now: I want you to give me a point cloud whose persistence diagram has as many points as possible. In other words, you have to generate a point cloud with a lot of holes.
I can see the sweat on your forehead. But I am a merciful kind of person, and let you know that Gudhi can do this in the blink of an eye. Think about it: when you generate a persistence diagram, the coordinates of the different points in this diagram do not depend on the full original point cloud, right? For a given point p in this diagram, the coordinates of p only depend on the positions of the points that form the hole (that corresponds to p) in the original point cloud in a simple way: these coordinates are simply the radii for which the union of balls makes this hole appear or disappear — or, equivalently, the maximal pairwise distance within these points. Turns out that Gudhi can spit out these dependencies with its persistence_pairs() function. The gradient is then simply defined as the derivatives of the Euclidean distance function (see this paper for a formal definition).
Let’s thus write a couple of functions, one that computes the Rips persistence diagram from a point cloud, and the one that computes the derivatives of the persistence diagram points. I simplified the code a little bit for readability, the actual code can be found here.
def Rips(DX, mel, dim):    rc = gd.RipsComplex(distance_matrix=DX, max_edge_length=mel)    st = rc.create_simplex_tree(max_dimension=dim+1)    dgm = st.persistence()    indices = st.persistence_pairs()    return indicesclass RipsModel(tf.keras.Model):    def __init__(self, X, mel=12, dim=1, card=50):        super(RipsModel, self).__init__()        self.X = X        self.mel = mel        self.dim = dim        self.card = card            def call(self):        m, d, c = self.mel, self.dim, self.card                # Compute distance matrix        DX = tfa.losses.metric_learning.pairwise_distance(self.X)        DXX = tf.reshape(DX, [1, DX.shape[0], DX.shape[1]])                # Turn numpy function into tensorflow function        RipsTF = lambda DX: tf.numpy_function(Rips, [DX, m, d, c], [tf.int32 for _ in range(4*c)])                # Compute vertices associated to positive and negative simplices         # Don't compute gradient for this operation        ids = tf.nest.map_structure(tf.stop_gradient, tf.map_fn(RipsTF,DXX,dtype=[tf.int32 for _ in range(4*c)]))                # Get persistence diagram by simply picking the corresponding entries in the distance matrix        dgm = tf.reshape(tf.gather_nd(DX, tf.reshape(ids, [2*c,2])), [c,2])        return dgm
Now, let’s define a loss which is the opposite of the distances of the persistence diagram points to the diagonal. This will force the diagram to have many points, with ordinates much bigger than abscissae, and equivalently a point cloud with a lot of holes of large sizes.
model = RipsModel()optimizer = tf.keras.optimizers.Adam(learning_rate=1e-2)for epoch in range(100):        with tf.GradientTape() as tape:                dgm = model.call()                loss = -tf.math.reduce_sum(tf.square(.5*(dgm[:,1]-dgm[:,0])))        gradients = tape.gradient(loss, model.trainable_variables)        optimizer.apply_gradients(zip(gradients, model.trainable_variables))
Now let’s optimize! Here are the results at epochs 0, 20 and 90:
So many holes, so beautiful... We’re living the dream. If you want to go ahead and play with other losses, check this Gudhi tutorial, which also contains another example on image optimization.
This post was only a glimpse of the many possibilities offered by combining Gudhi, Scikit-Learn and Tensorflow. I hope I convinced you that integrating TDA in your pipelines has now become as simple as it could possibly be. Even though many applications of TDA have already been presented in the literature, there is definitely much more that still needs to be discovered!