C OVER F ILTRATION AND S TABLE P ATHS IN M APPER
Anonymous authors
Paper under double-blind review
A BSTRACT
The contributions of this paper are two-fold.
We deﬁne a new ﬁltration called the cover ﬁltration built from a single cover based on a generalized Jaccard distance.
We provide stability results for the cover ﬁltration, and show its equivalence to the ˇCech ﬁltration under certain settings.
We then develop a language and theory for stable paths within this ﬁltration, inspired by ideas of persistent homology.
This framework can be used to develop new learning representations in applications where an obvious metric may not be deﬁned but a cover is readily available.
We demonstrate the utility of our framework as applied to recommendation systems and explainable machine learning.
We demonstrate a new perspective for modeling recommendation system data sets that does not require deﬁning a bespoke metric.
As an application, we ﬁnd that stable paths identiﬁed by our framework in a movies data set represent sequences of movies constituting a gentle transition and ordering from one genre to another.
For explainable machine learning, we present stable paths between subpopulations in the mapper of a model as explanations.
Our framework provides an alternative way to build a ﬁltration from a single mapper, and explore stable paths in it.
For illustration, we build a mapper from a supervised machine learning model trained on the FashionMNIST data set.
Stable paths in the cover ﬁltration provide improved explanations of relationships between subpopulations of images.
I NTRODUCTION AND M OTIVATION
The need to rigorously seed a solution with a notion of stability in topological data analysis (TDA) has been addressed primarily using topological persistence (Carlsson, 2009; Ghrist, 2008).
Persistence arises when we work with a sequence of objects built on a data set, a ﬁltration , rather than with a single object.
One line of focus of this work has been on estimating the homology of the data set.
This typically manifests itself as examining the persistent homology represented as a diagram or barcode, with interpretations of zeroth and ﬁrst homology as capturing signiﬁcant clusters and holes, respectively (Adams & Carlsson, 2009; Edelsbrunner et al., 2002; Edelsbrunner & Harer, 2009; Zomorodian, 2005).
In practice it is not always clear how to interpret higher dimensional homology (even holes might not make obvious sense in certain cases).
A growing focus is to use persistence diagrams as a form of feature engineering to help compare different data sets rather than interpret individual homology groups (Adams et al., 2017; Chazal et al., 2009; Turner et al., 2014).
The implicit assumption in most such TDA applications is that the data is endowed with a natural metric, e.g., points exist in a high-dimensional space or pairwise distances are available.
In certain applications, it is also not clear how one could assign a meaningful metric.
For example, memberships of people in groups of interest is captured simply as sets specifying who belongs in each group.
An instance of such data is that of recommendation systems, e.g., as used in Netﬂix to recommend movies to the customer.
Graph based recommendation systems have been an area of recent research.
Usually these systems are modeled as a bipartite graph with one set of nodes representing recommendees and the other representing recommendations.
In practice, these systems are augmented in bespoke ways to accommodate whichever type of data is available.
It is highly desirable to analyze the structure directly using the membership information.
Another distinct TDA approach for structure discovery and visualization of high-dimensional data is based on a construction called Mapper (Singh et al., 2007).
Deﬁned as a dual construction to a cover of the data (see Figure 1), Mapper has found increasing use in diverse applications in the past several years (Lum et al., 2013).
Attention has recently focused on interpreting parts of the 1 -skeleton of the Mapper, which is a simplicial complex, as signiﬁcant features of the data.
Paths, ﬂares, and cycles have been investigated in this context (Li et al., 2015; Nicolau et al., 2011; Torres et al., 2016).
The framework of persistence has been applied to this construction to deﬁne a multi-scale
Mapper, which permits one to derive results on stability of such features (Dey et al., 2016).
At the same time, the associated computational framework remains unwieldy and still most applications base their interpretations on a single Mapper object.
We illustrate the standard Mapper construction in
Figure 1.
We start with overlapping intervals covering values of a parameter, e.g., height of the points sampled from a circle.
We then cluster the data points falling in each interval, and represent each cluster by a vertex.
If two clusters share data points, we add an edge connecting the corresponding vertices.
If three clusters share data points, we add the triangle, and so on.
The Mapper could present a highly sparse representation of the data set that still captures its structure—the large number of points sampled from the circle is represented by just four vertices and four edges here.
More generally, we consider higher dimensional intervals covering a subspace of R d .
But in recommendation systems, the cover is just a collection of abstract sets providing membership info (rather than intervals over the range of function values).
Could we deﬁne a topological construction on such abstract covers that still reveals the topology of the dataset?
We could study paths in this construction, but as the topological constructions are noisy, we would want to deﬁne a notion of stability for such paths.
With this goal in mind, could we deﬁne a ﬁltration from the abstract cover?
But unlike in the setting of, e.g., multiscale Mapper (Dey et al., 2016), we do not have a sequence of covers (called a tower of covers)—we want to work with a single cover.
How do we deﬁne a ﬁltration on a single abstract cover?
Could we prove stability results for such a ﬁltration?
Finally, could we demonstrate the usefulness of our construction on real data?
1. O UR C ONTRIBUTIONS
We introduce a new type of ﬁltration deﬁned on a single abstract cover.
Termed cover ﬁltration , our construction uses Steinhaus distances between elements of the cover.
We generalize the Steinhaus distance between two elements to those of multiple elements in the cover, and deﬁne a ﬁltration on a single cover using the generalized Steinhaus distance as the ﬁltration index.
Working with a bottleneck distance on covers, we show a stability result on the cover ﬁltration—the cover ﬁltrations of two covers are α/m interleaved, where α is a bound on the bottleneck distance between the covers and m is the cardinality of the smallest element in either cover (see Theorem 3.5).
We conjecture that in
Euclidean space, the cover ﬁltration is isomorphic to the standard ˇCech ﬁltration built on the data set.
We prove the conjecture holds in dimension 1 and independently that the VietorisRips ﬁltration completely determines the cover ﬁltration in arbitrary dimensions (see Section 4).
This ﬁltration is quite general, and enables the computation of persistent homology for data sets without requiring strong assumptions.
With real life applications in mind, we study paths in the 1 -skeleton of our construction.
Paths provide intu- itive explanations of the relationships between the objects that the terminal vertices represent.
Our perspective of path analysis is that shortest may not be more descriptive—see Figure 2 for an illustration.
Instead, we deﬁne a notion of stability of paths in the cover ﬁltration.
Under this notion, a stable path is analogous to a highly persistent feature as identiﬁed by persistent homology.
We demonstrate the utility of stable paths in cover ﬁltrations on two real life applications: a problem in movie recommendation system and Mapper.
We ﬁrst show how recommendation systems can be modeled using the cover ﬁltration, and then show how stable paths within this ﬁltration suggest a sequence of movies that represent a “smooth” transition from one genre to another (Appendix C.1).
We then deﬁne an extension of the traditional Mapper (Singh et al., 2007) termed the Steinhaus
Mapper Filtration , and show how stable paths within this ﬁltration can provide valuable explanations of populations in the Mapper, focusing on the case of explainable machine learning (Appendix C.2).
1. R ELATED W ORK
Cavanna & Sheehy (2017) developed theory for a cover ﬁltration , built from a cover of a ﬁltered simplicial complex.
But we work from more general covers of arbitrary spaces.
We are inspired by similar goals as those of Dey et al.
and Carri ` ere et al.
(2018), who addressed the question of stability in the Mapper construction.
Our goal is to provide some consistency, and thus interpretability, to the Mapper.
We incorporate ideas of persistence in a different manner into our construction using a single cover, which considerably reduces the effort to generate results.
While stability of persistence diagrams is a well studied topic (Edelsbrunner & Harer, 2009), we may not get stability to simplices and cycles that generate the persistence homology classes (Bendich et al., 2019).
In contrast, our deﬁnition of path stability (Section 5) aligns exactly with our result on stability for cover ﬁltrations (Section 3).
We believe this is a unique feature of our method and its stability result—paths are automatically stable with respect to perturbations of input data.
The multi-scale Mapper deﬁned by Dey et al.
(2016) builds a ﬁltration on the Mapper by varying the parameters of a cover.
This construction yields nice stability properties, but is unwieldy in practice and difﬁcult to interpret.
Carri ` ere et al. develop ways based on extended persistence to automatically select a single cover that best captures the topology of the data (Carri ` ere et al., 2018), producing one ﬁnal Mapper that is easy to interpret
Methods have been developed to track populations within the Mapper by identifying interesting paths (Kalyanaraman et al., 2019) and interesting ﬂares (Kamruzzaman et al., 2018).
Interesting paths maximize an interestingness score, and are manifested in the Mapper as long paths that track particular populations that show trending behavior.
Flares capture subpopulations that diverge, i.e., show branching behavior.
In our context, we are interested in shorter paths, under the assumption that they provide the most succinct explanations for relationships between subpopulations.
Our work is similar to that of Parthasarathy et al.
(2017) in that they use the Jaccard Index of an observed graph to estimate the geodesic distance of the underlying graph.
We take an approach more akin to persistence and make fewer assumptions about underlying data.
Hence we are unable to make rigorous estimates of distances and instead provide many possible representative paths.
S-paths deﬁned by Purvine et al.
(2018) are similar to stable paths when we realize that covers can be modeled as hypergraphs, and vice versa.
Stable paths incorporate the size of each cover elements (or hyperedges), normalizing the weights by relative size.
This perspective allows us to compare different parts of the resulting structure which may have wildly difference sizes of covers.
In this context, a large overlap of small elements is considered more meaningful than a proportionally small intersection of large elements.
C OVER F ILTRATIONS
We introduce the notions of distance on covers required to construct our ﬁltrations and then provide the general deﬁnition of the cover ﬁltration.
We begin with the deﬁnition of Steinhaus distance, a generalization of the standard Jaccard distance between two sets, and further generalize it to an arbitrary collection of subsets of a cover.
These generalizations take a measure µ and assumes that all sets are taken modulo differences by sets of measure 0 .
We will take µ as arbitrary unless otherwise stated.
Deﬁnition 2.1 (Steinhaus Distance (Marczewski & Steinhaus, 1958)) .
Given a measure µ , the Steinhaus distance between two sets A, B is
[Equation]
This distance is also bounded on [0 , 1] , i.e., two sets have Steinhaus distance 0 when they are equal (as sets differing by measure 0 are identiﬁed) and distance 1 when they do not intersect.
We extend this distance from an operator on a pair of elements to an operator on a set of elements.
Deﬁnition 2.2 (Generalized Steinhaus distance) .
Deﬁne the generalized Steinhaus distance of a collection of sets { U i } as
[Equation]
Much of our results that consider paths used the standard version of Jaccard distance for ﬁnite sets.
But we require the Steinhaus distance for general measures to present equivalence results to ˇCech and VR ﬁltrations (Section 4).
Hence we present all results for the generalized Steinhaus distance.
We make use of this generalized distance to associate birth times to simplices in a nerve.
Given a cover, we deﬁne the cover ﬁltration as the ﬁltration induced from sublevel sets of the generalized
Steinhaus distance function.
In other words, consider a cover of the space and the nerve of this cover.
For each simplex in the nerve, we assign as birth time the value of its Steinhaus distance.
This ﬁltration captures information about similarity of cover elements, and the overall structure of the cover.
Deﬁnition 2.3 (Nerve) .
A nerve of a cover U
= { U i } i ∈ C is an abstract simplicial complex deﬁned such that each subset { U j } j ∈ J ⊆ U , i.e., with J ⊆ C , deﬁnes a simplex if (cid:84) j ∈ J
{ U j } (cid:54) = ∅ .
In this construction, each cover element U i ∈
U deﬁnes a vertex.
Deﬁnition 2.4 (Steinhaus Nerve) .
The Steinhaus nerve of a cover U , denoted Nrv St ( U ) , is deﬁned as the nerve of U with each simplex assigned their generalized Steinhaus distance as weight:
[Equation]
Note that w σ < 1 by deﬁnition for every simplex σ ∈ U .
We will use Nrv St when the cover U is evident from context.
This can be thought of as a weighted nerve, but the weighting scheme satisﬁes the conditions of a ﬁltration (see Page 14 for the proof).
Theorem 2.5.
The Steinhaus nerve of a cover U is a ﬁltered simplicial complex.
Following this result, we refer to the construction as the Steinhaus ﬁltration .
The only cover ﬁltrations we use in this paper is Steinhaus ﬁltrations, so we will use the two terms interchangeably.
We could study an adaptation of cover ﬁltration to an analog of the VR complex by building a weighted clique rank ﬁltration from the 1 -skeleton of the cover ﬁltration (Petri et al., 2013).
This adaptation drastically reduces the number of intersection and union checks required for the construction.
The weight rank clique ﬁltration is a way of generating a ﬂag ﬁltration from a weighted graph (Petri et al., 2013).
We can apply this technique to build a VR analog of the cover ﬁltration.
S TABILITY
We consider notions of stability in the cover ﬁltration with respect to changes in the cover.
We ﬁrst modify the standard edit distance to deﬁne a bottleneck distance on the space of covers of a ﬁnite set that have the same cardinality.
Under this setting, we show that the cover ﬁltration is interleaved with respect to this distance.
Deﬁnition 3.1 (Bottleneck metric on covers) .
Let U and V be two ﬁnite covers of ﬁnite set X with same cardinality, and let M ( U , V ) be the set of all possible matchings between them.
Let (cid:52) denote the symmetric difference.
Then the bottleneck distance d B ( · , · ) between two covers is deﬁned as
[Equation]
We can verify that d B is indeed a metric (see Page 14 for the proof).
Proposition 3.2.
Let U , V , and W be ﬁnite covers of a ﬁnite set X with equal cardinalities, and let d B be the bottleneck distance between any pair of these covers (Deﬁnition 3.1).
Then d B is a metric.
We now present two somewhat technical lemmas, which we subsequently employ in the proof of the main stability result.
See Page 14 for proof of Corollary 3.4.
Lemma 3.3.
Let a, b, c, d, e, f be real numbers with a < b , c + d = e + f , | b | > | d | , and | b | > | f | .
Then we have that ( a − c ) /
( b + d ) > ( a − e ) /
( b + f ) when c < e .
In words, for a weight α = c + d that we can distribute between decreasing the numerator of a proper fraction and increasing its denominator, the greatest decrease comes from decreasing the numerator by the entire weight.
Proof.
We will show that, with the given conditions, a − c b + d − a − e b + f > 0 .
[Equation]
, since f = c + d − e, d = e + f − c.
[Equation]
e
[Equation]
c
[Equation] [Equation]
= −
[Equation]
e
[Equation]
d
[Equation]
f )
[Equation]
( b + d )( b + f ) , since e − c = d − f.
=
[Equation]
Since c < e , d > f , so ed > cf and e − c > 0 .
Also, since a/b is proper, a < b , so − a + b > 0 .
Thus ( − a + b )( e − c )+( ed − cf ) > 0 .
Since b + d, b + f > 0 , we get that ( − a + b )( e − c ) + ed − cf ( b + d )( b + f ) > 0 , which in turn shows that a − c b + d > a − e b + f , as desired.
Corollary 3.4.
Similar to Lemma 3.3, the greatest increase possible in such a scenario comes from assigning the negative of the total weight to the numerator.
We now present a theorem that provides basic stability guarantees for the constructed ﬁltration, assuming that each element is not too small.
Theorem 3.5.
Suppose that U
= { U i } and V = { V j } are two covers of X with |U| =
|V| such that d B ( U , V ) ≤ α .
Given m = min { min U ∈U µ ( U ) , min V ∈V µ ( V ) } , Nrv St ( U ) and Nrv St ( V ) are α/m interleaved ﬁltrations.
Proof.
Given two covers U
= { U i } and V = { V j } , we use the notation that U i and V i , for generic indices i and j , are paired in a matching that minimizes the bottleneck distance between the two covers.
We assume that the bottleneck distance is α , a positive integer.
We consider the following question: what is the largest change in generalized Steinhaus distance possible between a collection U I and V J , where index sets I and J are paired elementwise in a matching.
To answer this question, we keep U I ﬁxed and consider how large a difference in generalized Steinhaus distance we can achieve by taking the symmetric difference with a set S i with measure up to α for each U i .
That is, we want to maximize change in µ ( ∩ U I ) /µ
( ∪ U I ) .
To get the maximum increase in µ ( ∩ U I )
/µ
( ∪ U I ) , we must increase the numerator and/or decrease the denominator.
Likewise, we must decrease the numerator and/or increase the denominator to get maximum decrease.
First, we note that S i can be partitioned into two sets: s i,
=
U i ∩ S i , and s i,
2 = S i \ s i, 1 , those not in U i ∩ S i .
µ ( s i, 1 + s i, 2 ) =
µ ( S i ) ≤ α .
Replacing U i with U
i \ s i, 1 cannot increase the size of the intersection or union, but it can decrease the size of the intersection or union by up to µ ( s i, 1 ) .
Likewise, replacing U i with U i ∪ s i, 2 cannot decrease the size of the intersection or union, but could increase the size of the intersection or union by up to µ ( s i, 2 ) .
The greatest possible change would occur if it were possible to select an S i for each U i in the cover such that replacing each U i in turn with U i (cid:52) S i increases or decreases the numerator with µ ( s i, 1 ) or µ ( s i, 2 ) , respectively, and does the opposite to the size of the denominator by µ ( s i, 2 ) or µ ( s i, 1 ) , again respectively.
Since µ ( s i, 1 + s i, 2 ) =
µ ( S i ) ≤
α , it follows that each change has a weight of at most α which it could throw into increasing or decreasing the size of the intersection and doing the opposite to the size of the union.
Lemma 3.3 and Corollary 3.4 imply that the maximum possible change in those situations will be achieved when all weight is directed toward increasing or decreasing the size of the intersection, since µ ( ∩ U I ) /µ
( ∪ U I ) must be between 0 and 1.
As we want to bound the possible change in the
Steinhaus distance, it will sufﬁce to use the observation that
[Equation]
to obtain bounds on the change of Steinhaus distance between covers with maximum bottleneck distance of α and with S as the cover cardinality.
Then we have
[Equation]
Sm ≤ d St ( U I ) , where m is the size of the smallest set, since
[Equation]
⇒
[Equation]
Similarly, since 1 −
[Equation]
µ
[Equation]
∪ U
[Equation]
) , we get d St ( V I )
+ α m ≥ d St
( U I ) .
Hence d St ( V I ) − α m ≤ d St ( U I ) ≤
d St ( V I ) + α m , giving that U and V are α/m interleaved.
Remark 3.6 .
Consider the case when |U| (cid:54) = |V| .
Assume without loss of generality that |U| > |V| .
Then there is a vertex v ∈ Nrv St ( U ) that is not present in Nrv St ( V ) .
Hence Nrv St ( U ) and
Nrv St ( V ) cannot be interleaved in the current setting.
We need to ﬁrst generalize matchings and the bottleneck distance to allow covers with unequal cardinalities.
E QUIVALENCE
To situate the cover ﬁltration, we wish to show that it is isomorphic to the ˇCech and VR ﬁltrations under certain conditions.
We conjecture that the ˇCech ﬁltration on a ﬁnite set of points, i.e., the nerve of balls with radius r around each point and over a sequence of r , and the Steinhaus Nerve constructed from the terminal cover of the ˇCech ﬁltration are isomorphic.
More precisely, the insertion order of simplices is equivalent between the two cases, and there exists a continuous bijection between insertion times of the Steinhaus Nerve and insertion times of the ˇCech ﬁltration.
We prove this result for n = 1 , i.e., when X is drawn from the real line.
We also provide experimental evidence for the 1-skeleton equivalence, and prove one direction of this equivalence (that the VR ﬁltration completely determines the cover ﬁltration).
Let ˇ C r ( X ) be the cover of X by balls of radius r centered on points in X .
The ˇCech complex is nerve of this cover.
The ˇCech ﬁltration is the sequence of simplicial complexes for all values of r .
Conjecture 4.1 ( ˇCech equivalence) .
Given a ﬁnite data set X ⊂ R n and some radius R > diam ( X ) the ˇCech ﬁltration constructed from X is isomorphic to the the cover ﬁltration on X constructed from ˇ C R ( X ) , given the Lebesgue (i.e. volume) measure.
A comprehensive proof for arbitrary order of intersections and arbitrary dimension is incomplete.
We provide the proof for the case of n -skeleton in 1 -dimension and provide a mapping from the ˇCech ﬁltration to cover ﬁltration for the 1 -skeleton in arbitrary dimension.
Proof.
We will deﬁne ˇ C ( { v i } ) as the birth radius of the simplex deﬁned by the set { v i } .
This can be computed as ˇ C ( { v i } )
= ( max i ( v i )
− min i ( v i )) / 2 since in a 1 -dimensional space the associated simplex is born precisely when the balls around the two outermost points intersect.
Let { V i } be the set of balls of radius R centered on the set { v i } .
Recall that we are using the Steinhaus distance for Lebesgue measure, so µ computes volume here.
Then the generalized Steinhaus distance for those balls is given by d St ( { V i } )
= 1 − min ( v i + R ) − max ( v
i − R ) max ( v i + R ) − min ( v i − R ) , since the mutual intersection of all
[Equation]
Solving for ˇ C ( { v i } ) , we get ˇ C ( { v i } ) =
Rd St ( { V i } ) 2 − d St
{ V i } ) , establishing a bijection between birth times.
Now since R is the radius of X and v i ⊆ X , 0 ≤ ˇ C ( { v i } ) ≤ R .
Also, R − x R + x decreases monotonically over the range x ∈ [0 , R ] .
Thus 1 − R − x R + x increases monotonically on x ∈ [0 , R ] .
Thus if we order the subsets of of X by increasing birth radius ( s 1 , s 2 , · · · , s n ) , ( d St ( s 1 ) , d St ( s 2 ) , · · · , d St ( s n )) is also in increasing order.
Thus the two ﬁltrations are isomorphic.
We now address the case of 1 -skeleton of the ˇ Cech and cover ﬁltrations in arbitrary dimension.
It is clear that if the 1 -skeletons are isomorphic, then the cover ﬁltration is isomorphic to the VR ﬁltration.
We prove one direction of this isomorphism (see Page 15 for the proof).
Lemma 4.2.
The VR ﬁltration completely determines the cover ﬁltration in arbitrary dimensions.
Hence one can derive the cover ﬁltration from the Vietoris-Rips ﬁltration.
In Appendix B, we present some experimental evidence suggesting that the 1-skeleton of the Steinhaus ﬁltration and the Vietoris-Rips ﬁltration are isomorphic.
S TABLE P ATHS
We develop a theory of stable paths in the 1 -skeleton of a cover ﬁltration.
We provide an algorithm to ﬁnd a maximally stable path from one vertex to another, with Steinhaus distances as edge weights.
Note that a maximally stable path might not be a shortest path in terms of number of edges.
Conversely, a shortest path might not be highly stable.
Since the two objectives are at odds with each other, we provide an algorithm to identify a family of shortest paths as we vary the stability level, similar in a loose sense to computing persistent homology.
We were studying shortest paths in a Mapper constructed on a machine learning model as ways to illustrate the relations between the data as identiﬁed by the model.
In this context, shortest paths found could have low Steinhaus distance, and thus could be considered noise.
This motivated our desire to ﬁnd stable paths, as they would intuitively be most representative of the data set and stable with respect to changing parameters in Mapper or changing data.
Deﬁnition 5.1 ( ρ -Stable Path) .
Given a Steinhaus distance ρ , a path P is deﬁned to be ρ -stable if max { d St ( e ) | e ∈ P } ≤ ρ .
In other words, the largest edge weight (Steinhaus distance) along the path is at most ρ .
Note that a ρ 1 -stable path is also ρ 2 -stable for any ρ 2 ≥ ρ 1 .
Also, a ρ 1 -stable path P 1 is more stable than a ρ 2 -stable path P 2 when ρ 1 < ρ 2 .
Hence we have a higher conﬁdence that edges in P 1 do exist (are not due to noise) than those in P 2 .
We now deﬁne maximally stable paths between a pair of vertices.
Deﬁnition 5.2 (Maximally Stable Path) .
Given a pair of vertices s and t , a maximally stable s - t path is a ρ -stable path between s and t for the smallest value of ρ .
If there are multiple s - t paths at the same minimum ρ value, a shortest path among them is deﬁned as a maximally stable path.
Input : 1 - s k e l e t
o
n G of cover f i l t r a t i o n and v e r t
i c e s s, t s e t LIST =
[ ∅ , ∅ ] / / s t
o r e s [ P, ρ ] p a i r s while s, t are connected in G compute s h
o r t e s t path P between s and
t f i n d ρ = max { d St ( e ) | e ∈ P } i f LIST has no p a
i r [ P (cid:48) , ρ (cid:48) ] : | P | = | P (cid:48) | add [ P, ρ ] to LIST e l s e i f ρ < ρ (cid:48)
f o r [ P (cid:48) , ρ (cid:48) ]
∈ LIST : | P | = | P (cid:48) | r e p l a c e [ P (cid:48) , ρ (cid:48) ] with [ P, ρ ] in LIST remove a l l edges e from G with d St ( e ) ≥ ρ
Return : LIST
The problem of ﬁnding the maximally stable s - t path can be solved as a minimax path problem on an undirected graph, which can solved efﬁciently using, e.g., range minimum queries (Demaine et al., 2009).
We are then left with two paths between vertices s and t , the shortest and the maximally stable.
The shortest path is not necessarily stable and the stable path is not necessarily short.
As these two notions, stable and short, are at odds with each other, we are interested in computing the entire
Pareto frontier between the short and stable path.
We present an algorithm to identify the Pareto frontier in Figure 3, and a visualization of the output from this algorithm in Figure 4.
We repeatedly compute the shortest path, while essentially sweeping over the Steinhaus
Distance.
This process results in a Pareto frontier balancing the shortest paths with the stability of those paths.
The blue points in Figure 4 are on the Pareto frontier, while the orange points are the pairs [ P (cid:48) , ρ (cid:48) ] that get replaced from the LIST in the course of the algorithm.
We then visualize the paths on the Pareto frontier in Figure 5.
Continuing our analogy to persistence, the path corresponding to a point on the Pareto frontier which sees a steep rise to the left is considered highly persistent, e.g., the path with length 21 on the frontier.
A PPLICATIONS
We apply the cover ﬁltration and stable paths to two applications, recommendation systems and
Mapper.
We ﬁrst show how recommendation systems can be modeled using the cover ﬁltration and then show how stable paths within this ﬁltration can answer the question what movies should I show my friend ﬁrst, to wean them into my favorite (but potentially weird) movie?
We then deﬁne an extension of the traditional Mapper (Singh et al., 2007; Carri`ere & Oudot, 2017) called the Steinhaus
Mapper Filtration, and show how stable paths within this ﬁltration can provide valuable explanations of populations in the Mapper.
As a direct illustration, we focus on the case of explainable machine learning, where the Mapper is constructed with a supervised machine learning model as the ﬁlter function, and address the question what can we learn about the model?
Movie Recommendations: We illustrate how our cover ﬁltration framework could be used to ﬁnd a sequence of movies to gently transition from Mulan to Moulin Rouge .
It would be abrupt to switch directly from one to the other movie, since they belong to completely different genres.
We compute stable paths that identify such feasible gentle sequences.
Two sequences corresponding to a shortest path and the most stable path are presented in Table 1.
See Appendix C.1 for details. [Table] Mulan (1998) 2. Dumbo (1941) 3.
Sound of Music, The (1965) 4.
Moulin Rouge (2001) 1.
Mulan (1998) 2.
Robin Hood (1973) 3.
Dumbo (1941) 4.
Sound of Music, The (1965) 5.
Gone with the Wind (1939) 6.
Psycho (1960) 7.
High Fidelity (2000) 8.
Moulin Rouge (2001)
Explanations in a Machine Learning Model: With a goal to develop a method of model induction for inspecting a machine learning model, we build a Mapper (Singh et al., 2007) from the predicted probability space of a logistic regression model built from the Fashion-MNIST data set (Xiao et al., 2017).
The goal is to understand the model structure by characterizing the relationship between the feature space and the prediction space.
We extend the constructed Mapper to be a
Steinhaus Mapper Filtration and analyze the stable paths in that object.
See Appendix C.2 for details.
We look at three regions of shoes (sneaker, ankle boot, and sandals) that are understandably confusing to the machine learning model.
We try to elucidate where these confusions arise by studying the stable paths connecting representative nodes for each type of shoe.
Figure 11 shows representatives from each vertex in the path for the shortest path and the stable path.
In both paths, the vertices start predominately containing sneakers and sneaker-like sandals.
They then transition to containing a larger proportion of ankle boots, with all three classes showing higher cut tops or high heels.
Along each path we can see the relationships between nodes change.
In the most stable path on the right, we a slow transition from sneaker space to ankle boot space, with some amount of sandals spread throughout.
Through the path, shoes from each of the three classes become taller.
Near the middle of the path, the images from sneakers and ankle boots are nearly indistinguishable.
And earlier in the path, we see how some white strips in the sneakers and boots might easily be confused with negative space in the sandals.
C ONCLUSION
In this work, we established the cover ﬁltration, a new kind of ﬁltration that enables application of
TDA to previously inaccessible types of data.
We then developed a theory of stable paths in the cover ﬁltration, and provide algorithms for computing the Pareto frontier between short and stable paths.
As proof of their utility to real world applications, we show how these two ideas can be applied to the analysis of recommendation systems and of Mapper in the context of explainable machine learning.
The results in this paper suggest many new questions.
The application of recommendation systems leaves us curious if the cover ﬁltration along with new results such as the one on predicting links in graphs using persistent homology (Bhatia et al., 2018) could provide methods for answering the main question in recommendation system research: what item to recommend to the user next?
Proving Conjecture 4.1 is a highly desirable goal.
The applications of cover ﬁltration and stable paths are not limited to ones we highlighted.
Another possibility not explored is for sensor networks.
Sensor coverage areas are often not uniform balls, and the cover ﬁltration is aptly suited for developing a ﬁltration.
In the context of communication networks, stable paths could be interpreted as reliable routes.
Yet another direct application could be in ﬁnding driving directions that take not only short, but also “easy” routes.
While paths and connected components are most amenable to practical interpretations, could other structures in the cover ﬁltration also suggest insights?
What would holes and loops in the cover ﬁltration for recommendation systems mean?
Our application for explanations in machine learning models raises the following question.
What are the implications of understanding a path as in Figure 11?
How could the predictive model be updated to account for such implications?
We showed that the cover ﬁltration is stable to small changes within the cover.
Does this result imply that the persistence diagram of the Steinhaus Mapper ﬁltration is stable with respect to changes in the data, cover parameters, or ﬁlter functions?
Appendix A M ISSING P ROOFS
We restate theorems and lemmas stated without proofs in the main paper, and present their proofs.
Theorem 2.5.
The Steinhaus nerve of a cover U is a ﬁltered simplicial complex.
Proof.
This proof makes use of standard set theory results.
Let U be an arbitrary cover of some set
X and let Nrv St be its Steinhaus nerve.
We consider Nrv St as a ﬁltration by assigning as the birth time of simplex σ ∈ Nrv St its weight w σ .
To show this is indeed a ﬁltration, we focus on a single simplex σ and a face τ (cid:22) σ to show that the face always appears in the ﬁltration before the simplex.
Suppose σ is generated from cover elements { U i }
i ∈ I over some index
set I .
Let a face τ (cid:22) σ be generated by cover elements indexed by a subset J ⊂ I .
The birth time of τ is
[Equation]
and the birth time of σ is d St ( { U i }
i
[Equation]
= 1 − µ (
[Equation]
i
[Equation]
I U i ) µ ( ∪
[Equation]
I U i ) .
Clearly, with { U i } i ∈ J ⊂ { U i } i ∈
I , we have that µ ( ∩ i ∈
J U i ) ≥ µ ( ∩ i ∈
I
U i ) and µ (
∪ i ∈
J U i ) ≤ µ ( ∪ i ∈
I U i ) .
It follows then that d St ( τ ) ≤
d St ( σ ) .
With K α denoting the subcomplex that includes all simplices in Nrv St with birth time at most α ∈ [0 , 1) , for any α, β ∈ [0 , 1) with α < β , we have K α ⊆ K β .
Hence Nrv St ( U ) is a monotonic ﬁltration.
Proposition 3.2.
Let U , V , and W be ﬁnite covers of a ﬁnite set X with equal cardinalities, and let d B denote the bottleneck distance between any pair of these covers as speciﬁed in Deﬁnition 3.1.
Then d B is a metric.
Proof.
We make the following observations.
d B ( U , U ) = 0 , as matching each set in the cover to itself gives a distance of 0 and the smallest cardinality of a symmetric distance is 0 , hence this matching gives the minimum possible symmetric difference.
Likewise, if d B ( U , V ) = 0 , there is a matching where the symmetric difference between each matched pair has cardinality 0 .
Hence the sets are equal for each pair in the matching, and hence U
= V . 2.
d B ( U , V ) ≥ 0 , since 0 is the greatest lower bound for measures.
d B ( U , V
[Equation]
Let d B ( U , V ) =
α and d B ( V , W ) =
β .
Then U i and its matched set V j differ on a set Dwith measure at most α , and V j and its matched set W k differ on a set D 2 with measure at most β .
Then U i and W k differ on some subset of D 1 ∪ D 2 , so their difference has measure at most µ ( D 1 ∪ D 2 ) ≤ µ ( D 1 ) + µ ( D 2 ) ≤ α + β .
Since this result holds for all i , there is a matching between U and W with a maximum symmetric difference measure of α + β .
Hence d B ( U , W ) ≤ α + β =
d B ( U , V ) + d B ( V , W ) .
Hence d B is a metric with respect to covers of equal cardinality.
Corollary 3.4.
Similar to Lemma 3.3, the greatest increase possible in such a scenario comes from assigning the negative of the total weight to the numerator.
Proof.
Let a, b, c, d, e, f be numbers as speciﬁed.
Then − c > − e , and − c − d = − e − f , and | b | > | d | , | b | > | f | , so a, b, − c, − d, − e, − f fulﬁll the hypotheses of Lemma 3.3.
Then we get that a + c b − d < a + e b − f .
Lemma 4.2.
The VR ﬁltration completely determines the cover ﬁltration in arbitrary dimensions.
Proof.
The intersection of two hyperspheres was derived by Li (2011).
The volume of intersection of two hyperspheres of equal radius R in R n with centers distance d apart is deﬁned as
[Equation]
where Γ is the gamma function and I is the regularized incomplete beta function:
[Equation]
We can reduce this equation to
[Equation]
The volume of an n -sphere of radius R is
[Equation]
and so the volume of union of two n -spheres is
[Equation]
We then compute the Steinhaus distance with Lebesgue measure of two spheres in R n and radius R with Euclidean distance d apart as
[Equation]
This equation provides a mapping from the birth time of the edge in the ˇ Cech ﬁltration to the birth time of the edge in the cover ﬁltration.
Once an n and R are chosen, the equation readily reduces, producing the birth times of a simplex in the cover ﬁltration.
B E QUIVALENCE
We detail experimental results suggesting that the 1-skeleton of the Steinhaus ﬁltration and the 1skeleton the ˇCech ﬁltration are isomorphic (i.e., the Vietoris-Rips ﬁltration).
To estimate the area of intersection of 1-spheres, we use Monte Carlo integration with uniform sampling.
The ﬁrst plot in Figure 6 shows the 50 landmark points along with 20,000 points uniformly sampled around the landmarks.
The middle plot shows the persistence diagrams of dimension 0 and 1 for the VietorisRips ﬁltration on the landmarks.
Finally, we show an approximated Steinhaus ﬁltration on the landmarks, using the balls with radii 0.5 as the covers.
We approximate the Steinhaus ﬁltration similar to how the Vietoris-Rips approximates ˇCech ﬁltration, i.e., by only computing the 1-skeleton of the nerve, and including any higher order simplices for which all faces are already contained in the ﬁltration, taking the maximum birth time of all faces.
We note that the two persistence diagrams have only minor differences (only in dimension 1).
C A PPLICATIONS C. R ECOMMENDATION S YSTEMS
In this application, we apply the cover ﬁltration to a recommendation system data set and employ the stable paths analysis to compute sequences of movies that ease viewers from one title to another title.
Our viewpoint on recommendation systems is similar to work of graph-based recommendation systems.
This is an active area of research and we believe our new perspective of interpreting such systems as covers and ﬁltrations will yield useful tools for advancing the ﬁeld.
The general approach of graph-based recommendation systems is to model the data as a bipartite graph, with one set of nodes representing the recommendation items and the other set representing the recommendees.
We can interpret a bipartite graph as a cover, either with elements being the recommendees covering the items, or elements being the items covering the recommendees.
For an example that we will see more of shortly, suppose you have only ever seen the movie Mulan and your partner wants to show you Moulin Rouge .
It would be jarring to just watch the movie, so your partner might gently build up to Moulin Rouge by showing you movies similar to both Mulan and Moulin Rouge .
We compute stable paths that identify such a feasible gentle sequence.
We use the MovieLens-20m data set (Harper & Konstan, 2016).
This data set is comprised ofmillion ratings by 138,493 users of 27,278 movies.
Often, these types of data sets are interpreted as bipartite graphs.
Once we realize that a bipartite graph can be equivalently represented as a covering of one node set with the other, we can apply the cover ﬁltration to build a ﬁltration.
In our case, we interpret each movie as a cover element of the users who have rated the movie.
To ensure the entire computation runs efﬁciently on a laptop, and to avoid noise, we remove all movies with less thanratings and then sample 4000 movies at random from the remaining movies.
shows the computed
Pareto frontier of stable paths for the case of Mulan and Moulin
Rouge .
In Table 1, we show two stable paths that might be chosen.
The stable path with length 4 is found after a large drop in instability.
As the length and stability must be traded off, we think this would be a decent path to choose if you want to optimize both.
The second path shown is the most stable.
For readers who have seen the movies in this path, the relationship between each edge is clear, even if one might consider the path a bit on the longer side.
[Table]
Mulan (1998) 2.
Dumbo (1941) 3.
Sound of Music, The (1965) 4.
Moulin Rouge (2001)
[Table]
Mulan (1998)
[Table]
Robin Hood (1973)
[Table]
Dumbo (1941) 4.
Sound of Music, The (1965)
[Table]
Gone with the Wind (1939)
[Table]
Psycho (1960)
[Table]
High Fidelity (2000)
[Table]
Moulin Rouge (2001)
C. S TEINHAUS M APPER F ILTRATION
As supervised learning has become more powerful, the need for explanations is also grown.
We develop a method of model induction for inspecting a machine learning model.
The goal is to develop an understanding of the model structure by characterizing the relationship between the feature space and the prediction space.
The gleaned understanding can help non-experts make sense of algorithmic decisions and is essential when models are too complex to fully understand in a white-box fashion.
The Mapper (Singh et al., 2007) is aptly suited for visualizing this functional structure.
This application is based on the work of using paths in the Mapper to provide explanations for supervised machine learning models (Saul & Arendt, 2018).
We build a Mapper from the predicted probability space of a logistic regression model.
We then extend the constructed Mapper to be a
Steinhaus Mapper Filtration and proceed to analyze the stable paths in that object.
Given topological spaces X, Y , a function f : X → Y , and a cover of Y , we deﬁne Mapper to be the nerve of the reﬁned pullback cover of f ( Y ) .
A reﬁned cover is one such that each cover element is split into its path-connected components.
Deﬁnition C.1 (Steinhaus Mapper ﬁltration) .
Given data X , a function f : X → Y , and a cover
U of Y , we deﬁne the Steinhaus Mapper as the Steinhaus nerve (Deﬁnition 2.4) of the reﬁned pullback cover of f ( U ) :
[Equation]
By incorporating information about the amount of overlap between the cover produced by the Mapper, our analysis is robust to noise and largely insensitive to the chosen parameters of the Mapper construction.
Figure 8 shows the Steinhaus Mapper ﬁltration constructed from a logistic regression model built from the Fashion-MNIST data set (Xiao et al., 2017).
This data set consists of 70,000 images of clothing items from 10 classes.
Each image is 28 pixels.
It is widely regarded as a more difﬁcult drop-in replacement for the ubiquitous MNIST handwritten digits data set
The dimensionality of the data set is ﬁrst reduced to 100 dimensions using Principal Components Analysis, and then a logistic regression classiﬁer with l 1 regularization is trained on the reduced data using 5-fold cross validation on a training set of 60,000 images.
The model is evaluated at 93% accuracy on the remaining 10,000 images.
We then extract the 10-dimensional predicted probability space and use UMAP (McInnes & Healy, 2018; McInnes et al., 2018) to reduce the space to 2 dimensions.
This 2-dimensional space is taken as the ﬁlter function of the Mapper, using a cover consisting of 40 bins along each dimension with 50% overlap between each bin.
We explored ranges of values for the number of bins (10–50) and the overlap percentage (20–60%), and observed variabilities in the constructed Mappers.
The ﬁnal choice of parameters (40 bins, 50% overlap) were chosen as the
Mappers showed least variability over subsets of values centered at these parameter values.
DBSCAN is used as the clustering algorithm in the reﬁnement step (Ester et al., 1996).
KeplerMapper is used for constructing the Mapper (van Veen & Saul, 2017).
Finally, the cover is extracted and the
Steinhaus Mapper ﬁltration is constructed.
To illustrate the power of the path explanations, we start with two vertices selected from the sneaker and ankle boot regions of the resulting graph.
The three regions of shoes (sneaker, ankle boot, and sandals) are understandably confusing to the machine learning model, and we are interested in where these confusions arise.
Figure 9 shows the paths associated with the Pareto frontier (Figure 10).
In Figure 10, we show the Pareto frontier between the two chosen vertices.
This frontier shows a large decrease in instability value (thus increase in stability) when moving to a path length of 12.
As noted in Section 5, paths found after a large increase in stability correspond to highly stable paths, i.e., the path remains the shortest path while sweeping the instability value over a comparatively large range.
Figure 11 shows representatives from each vertex in the path for the shortest path and the stable path with length 12.
Each row corresponds to one vertex and the columns show a representative from each class represented in the vertex.
Each image shows the multiplicity of that type of shoe in the vertex.
In both paths, the vertices start predominately containing sneakers and sneaker-like sandals.
They then transition to containing a larger proportion of ankle boots, with all three classes showing higher cut tops or high heels.
Along each path we can see the relationships between nodes change.
In the most stable path on the right, we a slow transition from sneaker space to ankle boot space, with some amount of sandals spread throughout.
Through the path, shoes from each of the three classes become taller.
Near the middle of the path, the images from sneakers and ankle boots are nearly indistinguishable.
And earlier in the path, we see how some white strips in the sneakers and boots might easily be confused with negative space in the sandals.
These two paths provide a holistic representation of how the trained logistic model interprets the data.
By exploring these paths, we gain valuable insight into why a model is making a decision.
This can help either reinforce our trust in the model or reject the prediction.
In either outcome, these explanations can strengthen the results of the predictions by including humans in the loop.
Even though the case of predicting clothing types is a low stakes application, this framework is readily applicable to much more important data sets.
D N OTE ON C OMPLEXITY OF C OVER F ILTRATIONS
The complexity of constructing the cover ﬁltration is by and large inherited directly from the computational complexity of the nerve.
Given a cover U , the nerve could have at most 2 |U| − 1 simplices and dimension at most |U| − 1 Otter et al.
These bounds are equivalent to the corresponding worst case bounds for VR and ˇ Cech complexes.
The work involved for each simplex in constructing Nrv St includes computing the volume of intersection and volume of union of the elements in the simplex.
The complexity of union and intersection operations is largely dependent on the type of data being used.
Let C Unn ( V ) and C Int ( V ) be the costs of computing the union and intersection, respectively, of a set of cover elements V ⊆ U .
In the worst case, we have to do C Unn ( U ) + C Int ( U ) operations per simplex, leading to an overall worst case computational complexity of ( C Unn ( U ) + C Int ( U ))
(2 |U| − 1) .
For instance, if we assume that a hashing-based dictionary could be produced for each set in U , both C Unn ( V ) and C Int ( V ) will be at most linear in |V| Bille et al.