SlowGuess/NeurIPS_2023_OCR huggingface dataset:
David Loiseaux
DataShape
Centre Inria d'Université Côte d'Azur
Biot, France
Mathieu Carrière
Andrew J. Blumberg
Irving Institute for Cancer Dynamics
New-York, NY, USA
Topological data analysis (TDA) is an area of data science that focuses on using invariants from algebraic topology to provide multiscale shape descriptors for geometric data sets, such as graphs and point clouds. One of the most important such descriptors is persistent homology, which encodes the change in shape as a filtration parameter changes; a typical parameter is the feature scale. For many data sets, it is useful to simultaneously vary multiple filtration parameters, for example feature scale and density. While the theoretical properties of single parameter persistent homology are well understood, less is known about the multiparameter case. In particular, a central question is the problem of representing multiparameter persistent homology by elements of a vector space for integration with standard machine learning algorithms. Existing approaches to this problem either ignore most of the multiparameter information to reduce to the one-parameter case or are heuristic and potentially unstable in the face of noise. In this article, we introduce a new general representation framework that leverages recent results on decompositions of multiparameter persistent homology. This framework is rich in information, fast to compute, and encompasses previous approaches. Moreover, we establish theoretical stability guarantees under this framework as well as efficient algorithms for practical computation, making this framework an applicable and versatile tool for analyzing geometric data. We validate our stability results and algorithms with numerical experiments that demonstrate statistical convergence, prediction accuracy, and fast running times on several real data sets.
Topological Data Analysis (TDA) [8] is a methodology for analyzing data sets using multiscale shape descriptors coming from algebraic topology. There has been intense interest in the field in the last decade, since topological features promise to allow practitioners to compute and encode information that classical approaches do not capture. Moreover, TDA rests on solid theoretical grounds, with guarantees accompanying many of its methods and descriptors. TDA has proved useful in a wide variety of application areas, including computer graphics [13, 33], computational biology [34], and material science [6, 35], among many others.
The main tool of TDA is persistent homology. In its most standard form, one is given a finite metric space  $X$  (e.g., a finite set of points and their pairwise distances) and a continuous function
$f: X \to \mathbb{R}$ . This function usually represents a parameter of interest (such as, e.g., scale or density for point clouds, marker genes for single-cell data, etc), and the goal of persistent homology is to characterize the topological variations of this function on the data at all possible scales. Of course, the idea of considering multiscale representations of geometric data is not new [14, 32, 41]; the contribution of persistent homology is to obtain a novel and theoretically tractable multiscale shape descriptor. More formally, persistent homology is achieved by computing the so-called persistence barcode of  $f$ , which is obtained by looking at all sublevel sets of the form  $\{f^{-1}((-\infty, \alpha])\}_{\alpha \in \mathbb{R}}$ , also called filtration induced by  $f$ , and by computing a decomposition of this filtration, that is, by recording the appearances and disappearances of topological features (connected components, loops, enclosed spheres, etc) in these sets. When such a feature appears (resp. disappears), e.g., in a sublevel set  $f^{-1}((-\infty, \alpha_b])$ , we call the corresponding threshold  $\alpha_b$  (resp.  $\alpha_d$ ) the birth time (resp. death time) of the topological feature, and we summarize this information in a set of intervals, or bars, called the persistence barcode  $D(f) := \{(\alpha_b, \alpha_d)\}_{\alpha \in A} \subset \mathbb{R} \times \mathbb{R} \cup \{\infty\}$ . Moreover, the bar length  $\alpha_d - \alpha_b$  often serves as a proxy for the statistical significance of the corresponding feature.
However, an inherent limitation of the formulation of persistent homology is that it can handle only a single filtration parameter  $f$ . However, in practice it is common that one has to deal with multiple parameters. This translates into multiple filtration functions: a standard example is when one aims at obtaining meaningful topological representation of a noisy point cloud. In this case, both feature scale and density functions are necessary (see Appendix A). An extension of persistent homology to several filtration functions is called multiparameter persistent homology [3, 9], and studies the topological variations of a continuous multiparameter function  $f: X \to \mathbb{R}^n$  with  $n \in \mathbb{N}^*$ . This setting is notoriously difficult to analyze theoretically as there is no result ensuring the existence of an analogue of persistence barcodes, i.e., a decomposition into subsets of  $\mathbb{R}^n$ , each representing the lifespan of a topological feature.
Still, it remains possible to define weaker topological invariants in this setting. The most common one is the so-called rank invariant (as well as its variations, such as the generalized rank invariant [24], and its decompositions, such as the signed barcodes [4]), which describes how the topological features associated to any pair of sublevel sets  $\{x\in X:f(x)\leq \alpha \}$  and  $\{x\in X:f(x)\leq \beta \}$  such that  $\alpha \leq \beta$  (w.r.t. the partial order in  $\mathbb{R}^n$ ), are connected. The rank invariant is a construction in abstract algebra, and so the task of finding appropriate representations of this invariant, i.e., embeddings into Hilbert spaces, is critical. Hence, a number of such representations have been defined, which first approximate the rank invariant by computing persistence barcodes from several linear combinations of filtrations, a procedure often referenced as the fibered barcode (see Appendix E), and then aggregate known single-parameter representations for them [17, 18, 39]. Adequate representations of the generalized rank invariant have also been investigated recently for  $n = 2$  [42].
However, the rank invariant, and its associated representations, are known to be much less informative than decompositions (when they exist): many functions have different decompositions yet the same rank invariants. Therefore, the aforementioned representations can encode only limited multiparameter topological information. Instead, in this work, we focus on candidate decompositions of the function, in order to create descriptors that are strictly more powerful than the rank invariant. Indeed, while there is no general decomposition theorem, there is recent work that constructs candidate decompositions in terms of simple pieces [1, 7, 29] that always exist but do not necessarily suffice to reconstruct all of the multiparameter information. Nonetheless, they are strictly more informative than the rank invariant under mild conditions, are stable, and approximate the true decomposition when it exists<sup>1</sup>. For instance, in Figure 2, we present a bifiltration of a noisy point cloud with scale and density (left), and a corresponding candidate decomposition comprised of subsets of  $\mathbb{R}^2$ , each representing a topological feature (middle). One can see that there is a large green subset in the decomposition that represents the circle formed by the points that are not outliers (also highlighted in green in the bifiltration).
<sup>1</sup>Although multiparameter persistent homology can always be decomposed as a sum of indecomposable pieces ([3, Theorem 4.2], [20]), these decompositions are prohibitively difficult to interpret and work with.
Figure 1: Common pipelines for the use of multiparameter persistent homology in data science—our work provides new contributions to the arrow highlighted in red.
Unfortunately, while more informative, candidate decompositions suffer from the same problem than the rank invariant; they also need appropriate representations in order to be processed by standard data science methods. In this work, we bridge this gap by providing new representations designed for candidate decompositions. See Figure 1 for a summarizing figure.
Figure 2: (left) Bi-filtration of a noisy point cloud induced by both feature scale (using unions of balls with increasing radii) and sublevel sets of codensity. The cycle highlighted in the green zone can be detected as a large subset in the corresponding candidate decomposition computed by the MMA method [29] (middle), and in our representation of it (right).
Contributions. Our contributions in this work are listed below:
- We provide a general framework that parametrizes representations of multiparameter persistent homology decompositions (Definition 1) and which encompasses previous approaches in the literature. These representations take the form of a parametrized family of continuous functions on  $\mathbb{R}^n$  that can be binned into images for visualization and data science.
- We identify parameters in this framework that result in representations that have stability guarantees while still encoding more information than the rank invariant (see Theorem 1).
- We illustrate the performance of our framework with numerical experiments: (1) We demonstrate the practical consequences of the stability theorem by measuring the statistical convergence of our representations. (2) We achieve the best performance with the lowest runtime on several classification tasks on public data sets (see Sections 4.1 and 4.2).
Related work. Closely related to our method is the recent contribution [10], which also proposes a representation for decompositions. However, their approach, while being efficient in practice, is a heuristic with no corresponding mathematical guarantees. In particular, it is known to be unstable: similar decompositions can lead to very different representations, as shown in Appendix B. Our approach can be understood as a subsequent generalization of the work of [10], with new mathematical guarantees that allow to derive, e.g., statistical rates of convergence.
Outline. Our work is organized as follows. In Section 2, we recall the basics of multiparameter persistent homology. Next, in Section 3 we present our general framework and state our associated stability result. Finally, we showcase the numerical performances of our representations in Section 4, and we conclude in Section 5.
In this section, we briefly recall the basics of single and multiparameter persistent homology, and refer the reader to Appendix C, Appendix D, and [31, 34] for a more complete treatment.
Persistent homology. The basic brick of persistent homology is a filtered topological space  $X$ , by which we mean a topological space  $X$  together with a function  $f\colon X\to \mathbb{R}$  (for instance, in Figure 5,  $X = \mathbb{R}^2$  and  $f = f_{P}$ ). Then, given  $\alpha >0$ , we call  $F(\alpha)\coloneqq f^{-1}((-\infty ,\alpha])\subseteq X$  the sublevel set of  $f$  at level  $\alpha$ . Given levels  $\alpha_{1}\leq \dots \leq \alpha_{N}$ , the corresponding sublevel sets are nested w.r.t. inclusion, i.e., one has  $F(\alpha_{1})\subseteq F(\alpha_{2})\subseteq \ldots \subseteq F(\alpha_{i})\subseteq \ldots \subseteq F(\alpha_{N})$ . This system is an example of filtration of  $X$ , where a filtration is generally defined as a sequence of nested subspaces  $X_{1}\subseteq \ldots \subseteq X_{i}\subseteq \ldots \subseteq X$ . Then, the core idea of persistent homology is to apply the kth homology functor  $H_{k}$  on each  $F(\alpha_{i})$ . We do not define the homology functor explicitly here, but simply recall that each  $H_{k}(F(\alpha_{i}))$  is a vector space, whose basis elements represent the kth dimensional topological features of  $F(\alpha_{i})$  (connected components for  $k = 0$ , loops for  $k = 1$ , spheres for  $k = 2$ , etc). Moreover, the inclusions  $F(\alpha_{i})\subseteq F(\alpha_{i + 1})$  translate into linear maps  $H_{k}(F(\alpha_{i}))\to H_{k}(F(\alpha_{i + 1}))$ , which connect the features of  $F(\alpha_{i})$  and  $F(\alpha_{i + 1})$  together. This allows to keep track of the topological features in the filtration, and record their levels, often called times, of appearance and disappearance. More formally, such a sequence of vector spaces connected with linear maps  $\mathbb{M} = H_*(F(\alpha_1))\to \dots \to H_*(F(\alpha_N))$  is called a persistence module, and the standard decomposition theorem [15, Theorem 2.8] states that this module can always be decomposed as  $\mathbb{M} = \oplus_{i = 1}^{m}\mathbb{I}[\alpha_{b_i},\alpha_{d_i}]$ , where  $\mathbb{I}[\alpha_{b_i},\alpha_{d_i}]$  stands for a module of dimension 1 (i.e., that represents a single topological feature) between  $\alpha_{b_i}$  and  $\alpha_{d_i}$ , and dimension 0 (i.e., that represents no feature) elsewhere. It is thus convenient to summarize such a module with its persistence barcode  $D(\mathbb{M}) = \{[\alpha_{b_i},\alpha_{d_i}]\}_{1\leq i\leq m}$ . Note that in practice, one is only given a sampling of the topological space  $X$ , which is usually unknown. In that case, persistence barcodes are computed using combinatorial models of  $X$  computed from the data, called simplicial complexes. See Appendix C.
Multiparameter persistent homology. The persistence modules defined above extend straightforwardly when there are multiple filtration functions. An  $n$ -filtration, or multifiltration, induced by a function  $f: X \to \mathbb{R}^n$ , is the family of sublevel sets  $F = \{F(\alpha)\}_{\alpha \in \mathbb{R}^n}$ , where  $F(\alpha) := \{x \in X: f(x) \leq \alpha\}$  and  $\leq$  denotes the partial order of  $\mathbb{R}^n$ . Again, applying the homology functor  $H_k$  on the multifiltration  $F$  induces a multiparameter persistence module  $\mathbb{M}$ . However, contrary to the single-parameter case, the algebraic structure of such a module is very intricate, and there is no general decomposition into modules of dimension at most 1, and thus no analogue of the persistence barcode. Instead, the rank invariant has been introduced as a weaker invariant: it is defined, for a module  $\mathbb{M}$ , as the function  $\mathrm{RI}:(\alpha,\beta) \mapsto \mathrm{rank}(\mathbb{M}(\alpha) \to \mathbb{M}(\beta))$  for any  $\alpha \leq \beta$ , but is also known to miss a lot of structural properties of  $\mathbb{M}$ . To remedy this, several methods have been developed to compute candidate decompositions for  $\mathbb{M}$  [1, 7, 29], where a candidate decomposition is a module  $\tilde{\mathbb{M}}$  that can be decomposed as  $\tilde{\mathbb{M}} \simeq \oplus_{i=1}^{m} M_i$ , where each  $M_i$  is an interval module, i.e., its dimension is at most 1, and its support  $\operatorname{supp}(M_i) := \{\alpha \in \mathbb{R}^n : \dim(M_i(\alpha)) = 1\}$  is an interval of  $\mathbb{R}^n$  (see Appendix D). In particular, when  $\mathbb{M}$  does decompose into intervals, candidate decompositions must agree with the true decomposition. One also often asks candidate decompositions to preserve the rank invariant.
Distances. Finally, multiparameter persistence modules can be compared with two standard distances: the interleaving and bottleneck (or  $\ell^{\infty}$ ) distances. Their explicit definitions are technical and not necessary for our main exposition, so we refer the reader to, e.g., [3, Sections 6.1, 6.4] and Appendix D for more details. The stability theorem [27, Theorem 5.3] states that multiparameter persistence modules are stable:  $d_{\mathrm{I}}(\mathbb{M},\mathbb{M}^{\prime})\leq \| f - f^{\prime}\|_{\infty}$ , where  $f$  and  $f^{\prime}$  are continuous multiparameter functions associated to  $\mathbb{M}$  and  $\mathbb{M}^{\prime}$  respectively.
3 T-CDR: a template for representations of candidate decompositions
Even though candidate decompositions of multiparameter persistence modules are known to encode useful data information, their algebraic definitions make them not suitable for subsequent data science and machine learning purposes. Hence, in this section, we introduce the Template Candidate
Decomposition Representation (T-CDR): a general framework and template system for representations of candidate decompositions, i.e., maps defined on the space of candidate decompositions and taking values in an (implicit or explicit) Hilbert space.
3.1 T-CDR definition
Notations. In this article, by a slight abuse of notation, we will make no difference in the notations between an interval module and its support, and we will denote the restriction of an interval support  $M$  to a given line  $\ell$  as  $M|_{\ell}$ .
Definition 1. Let  $\mathbb{M} = \oplus_{i=1}^{m} M_i$  be a candidate decomposition, and let  $\mathcal{M}$  be the space of interval modules. The Template Candidate Decomposition Representation (T-CDR) of  $\mathbb{M}$  is:
$$
V _ {\mathrm {o p}, w, \phi} (\mathbb {M}) = \mathrm {o p} \left(\left\{w \left(M _ {i}\right) \cdot \phi \left(M _ {i}\right) \right\} _ {i = 1} ^ {m}\right), \tag {1}
$$
where op is a permutation invariant operation (sum, max, min, mean, etc),  $w: \mathcal{M} \to \mathbb{R}$  is a weight function, and  $\phi: \mathcal{M} \to \mathcal{H}$  sends any interval module to a vector in a Hilbert space  $\mathcal{H}$ .
The general definition of T-CDR is inspired from a similar framework that was introduced for single-parameter persistence with the automatic representation method PersLay [11].
Relation to previous work. Interestingly, whenever applied on candidate decompositions that preserve the rank invariant, specific choices of op,  $w$  and  $\phi$  reproduce previous representations:
- Using  $w: M_i \mapsto 1, \phi: M_i \mapsto \left\{ \begin{array}{ll} \mathbb{R}^n & \to \mathbb{R} \\ x & \mapsto \Lambda(x, M_i|_{\ell_x}) \end{array} \right.$  and  $\mathrm{op} = k$ th maximum, where  $l_x$  is the diagonal line crossing  $x$ , and  $\Lambda(\cdot, \ell)$  denotes the tent function associated to any segment  $\ell \subset \mathbb{R}^n$ , induces the  $k$ th multiparameter persistence landscape (MPL) [39].
- Using  $w: M_i \mapsto 1$ ,  $\phi: M_i \mapsto \left\{ \begin{array}{ll} \mathbb{R}^n \times \mathbb{R}^n & \to \mathbb{R}^d \\ p, q & \mapsto w'(M_i \cap [p, q]) \cdot \phi'(M_i \cap [p, q]) \end{array} \right.$  and  $\mathrm{op} = \mathrm{op}'$ , where  $\mathrm{op}', w'$  and  $\phi'$  are the parameters of any persistence diagram representation from Perslay, induces the multiparameter persistence kernel (MPK) [17].
- Using  $w: M_i \mapsto \mathrm{vol}(M_i)$ ,  $\phi: M_i \mapsto \left\{ \begin{array}{ll} \mathbb{R}^n & \to \mathbb{R} \\ x & \mapsto \exp(-\min_{\ell \in L} d(x, M_i|_\ell)^2 / \sigma^2) \end{array} \right.$  and  $\mathrm{op} = \sum$ , where  $L$  is a set of (pre-defined) diagonal lines, induces the multiparameter persistence image (MPI) [10].
Recall that the first two approaches are built from fibered barcodes and rank invariants, and that it is easy to find persistence modules that are different yet share the same rank invariant (see [38, Figure 3]). On the other hand, the third approach uses more information about the candidate decomposition, but is known to be unstable (see Appendix B). Hence, in the next section, we focus on specific choices for the T-CDR parameters that induce stable yet informative representations.
3.2 Metric properties
In this section, we study specific parameters for T-CDR (see Definition 1) that induce representations with associated robustness properties. We call this subset of representations Stable Candidate Decomposition Representations (S-CDR), and define them below.
Definition 2. The S-CDR parameters are:
1. the weight function  $w: M \mapsto \sup \{ \varepsilon > 0 : \exists y \in \mathbb{R}^n \text{ s.t. } \ell_{y,\varepsilon} \subset \operatorname{supp}(M) \}$ , where  $\ell_{y,\varepsilon}$  is the segment between  $y - \varepsilon \cdot [1, \ldots, 1]$  and  $y + \varepsilon \cdot [1, \ldots, 1]$ ,
2. the individual interval representations  $\phi_{\delta}(M):\mathbb{R}^n\to \mathbb{R}$
(a)  $\phi_{\delta}(M)(x) = \frac{1}{\delta} w(\operatorname {supp}(M)\cap R_{x,\delta}),$  (b)  $\phi_{\delta}(M)(x) = \frac{1}{(2\delta)^n}\mathrm{vol}(\operatorname {supp}(M)\cap R_{x,\delta}),$
(c)  $\phi_{\delta}(M)(x) = \frac{1}{(2\delta)^n}\sup_{x',\pmb{\delta}'}\{\mathrm{vol}(R_{x'},\pmb{\delta}') : R_{x',\pmb{\delta}'} \subseteq \mathrm{supp}(M) \cap R_{x,\pmb{\delta}}\}$ ,
where  $R_{x,\delta}$  is the hypersquare  $\{y\in \mathbb{R}^n:x - \delta \leq y\leq x + \delta \} \subseteq \mathbb{R}^n$ $\pmb {\delta}\coloneqq \pmb {\delta}\cdot [1,\dots ,1]\in$ $\mathbb{R}^n$  for any  $\delta >0$  , and vol denotes the volume of a set in  $\mathbb{R}^n$
3. the permutation invariant operators  $\mathrm{op} = \sum$  and  $\mathrm{op} = \sup$ .
Intuitively, the S-CDR weight function is the length of the largest diagonal segment one can fit inside  $\operatorname{supp}(M)$ , and the S-CDR interval representations (a), (b) and (c) are the largest normalized diagonal length, volume, and hypersquare volume that one can fit inside  $\operatorname{supp}(M) \cap R_{x,\delta}$ , respectively. These S-CDR interval representations allow for some trade-off between computational cost and the amount of information that is kept: (a) and (c) are very easy to compute, but (b) encodes more information about interval shapes. See Figure 2 (right) for visualizations.
Equipped with these S-CDR parameters, we can now define the two following S-CDRs, that can be applied on any candidate decomposition  $\mathbb{M} = \oplus_{i=1}^{m} M_i$ :
$$
V _ {p, \delta} (\mathbb {M}) := \sum_ {i = 1} ^ {m} \frac {w \left(M _ {i}\right) ^ {p}}{\sum_ {j = 1} ^ {m} w \left(M _ {j}\right) ^ {p}} \phi_ {\delta} \left(M _ {i}\right), \quad (2) \quad V _ {\infty , \delta} (\mathbb {M}) := \sup  _ {1 \leq i \leq m} \phi_ {\delta} \left(M _ {i}\right). \tag {3}
$$
Stability. The main motivation for introducing S-CDR parameters is that the corresponding S-CDRs are stable in the interleaving and bottleneck distances, as stated in the following theorem.
Theorem 1. Let  $\mathbb{M} = \oplus_{i=1}^{m} M_i$  and  $\mathbb{M}' = \oplus_{j=1}^{m'} M_j'$  be two candidate decompositions. Assume that we have  $\frac{1}{m} \sum_{i} w(M_i)$ ,  $\frac{1}{m'} \sum_{j} w(M_j') \geq C$ , for some  $C > 0$ . Then for any  $\delta > 0$ , one has
$$
\left\| V _ {0, \delta} (\mathbb {M}) - V _ {0, \delta} \left(\mathbb {M} ^ {\prime}\right) \right\| _ {\infty} \leq 2 \left(d _ {\mathrm {B}} \left(\mathbb {M}, \mathbb {M} ^ {\prime}\right) \wedge \delta\right) / \delta , \tag {4}
$$
$$
\left\| V _ {1, \delta} (\mathbb {M}) - V _ {1, \delta} \left(\mathbb {M} ^ {\prime}\right) \right\| _ {\infty} \leq \left[ 4 + \frac {2}{C} \right] \left(d _ {\mathrm {B}} \left(\mathbb {M}, \mathbb {M} ^ {\prime}\right) \wedge \delta\right) / \delta , \tag {5}
$$
$$
\left\| V _ {\infty , \delta} (\mathbb {M}) - V _ {\infty , \delta} \left(\mathbb {M} ^ {\prime}\right) \right\| _ {\infty} \leq \left(d _ {\mathrm {I}} \left(\mathbb {M}, \mathbb {M} ^ {\prime}\right) \wedge \delta\right) / \delta , \tag {6}
$$
where  $\wedge$  stands for minimum.
A proof of Theorem 1 can be found in Appendix F.
These results are the main theoretical contribution in this work, as the only other decomposition-based representation in the literature [10] has no such guarantees. The other representations [17, 18, 39, 42] enjoy similar guarantees than ours, but are computed from the rank invariant and do not exploit the information contained in decompositions. Theorem 1 shows that S-CDRs bring the best of both worlds: these representations are richer than the rank invariant and stable at the same time. We also provide an additional stability result with a similar, yet more complicated representation in Appendix G, whose upper bound does not involve taking minimum.