SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Naoki Nishikawa
The University of Tokyo
nishikawa-naoki259@g.ecc.u-tokyo.ac.jp
Yuichi Ike
ike@imi.kyushu-u.ac.jp
Kenji Yamanishi
yamanishi@mist.i.u-tokyo.ac.jp
Machine learning for point clouds has been attracting much attention, with many applications in various fields, such as shape recognition and material science. For enhancing the accuracy of such machine learning methods, it is often effective to incorporate global topological features, which are typically extracted by persistent homology. In the calculation of persistent homology for a point cloud, we choose a filtration for the point cloud, an increasing sequence of spaces. Since the performance of machine learning methods combined with persistent homology is highly affected by the choice of a filtration, we need to tune it depending on data and tasks. In this paper, we propose a framework that learns a filtration adaptively with the use of neural networks. In order to make the resulting persistent homology isometry-invariant, we develop a neural network architecture with such invariance. Additionally, we show a theoretical result on a finite-dimensional approximation of filtration functions, which justifies the proposed network architecture. Experimental results demonstrated the efficacy of our framework in several classification tasks.
Analysis of point clouds (finite point sets) has been increasing its importance, with many applications in various fields such as shape recognition, material science, and pharmacology. Despite their importance, point cloud data were difficult to deal with by machine learning, in particular, neural networks. However, thanks to the recent development, there have appeared several neural network architectures for point cloud data, such as DeepSet (Zaheer et al., 2017), PointNet (Qi et al., 2017a), PointNet++ (Qi et al., 2017b), and PointMLP (Ma et al., 2022). These architectures have shown high accuracy in tasks such as shape classification and segmentation.
In point cloud analysis, topological global information, such as connectivity, the existence of holes and cavities, is known to be beneficial for many tasks (see, for example, (Hiraoka et al., 2016; Kovacev-Nikolic et al., 2016)). One way to incorporate topological information into machine learning is to use persistent homology, which is a central tool in topological data analysis and attracting much attention recently. By defining an increasing sequence of spaces, called a filtration, from a point cloud  $X \subset \mathbb{R}^d$ , we can compute its persistent homology, which reflects the topology of  $X$ . Topological features obtained through persistent homology are then combined with neural network methods and enhance the accuracy of many types of tasks, such as point cloud segmentation (Liu et al., 2022), surface classification (Zeppelinzauer et al., 2016), and fingerprint classification (Giansiracusa et al., 2019).
Figure 1: Procedure to utilize our framework to a classification task and its comparison to when we use Rips filtration. For a given set of points, define a value called "weight" for each point in (a), using a neural network method. In (b), a ball centered at each point is gradually expanded. The balls centered at points with larger weights expand later than the balls centered at points with smaller weights. In (c), we examine the topology of the sum of the balls as they grow larger, and aggregate the information into a diagram called a persistence diagram. In (d), the persistence diagram is converted to a vector, which can be used as input of machine learning models. When we use standard Rips filtration, the balls expand uniformly from all of the points, so that we cannot correctly capture the large global hole. On the other hand, when we use our method, the suitable filtration to solve a classification task is chosen, and the weights on the outliers get larger. As a result, we can capture the large global hole in the point cloud. Note that the resulting weights assigned to each point are not pre-specified, but are fully learned from the data.
The standard pipeline to use persistent homology in combination with machine learning is split into the following:
(i) define a filtration for data and compute the persistent homology;
(ii) vectorize the persistent homology;
(iii) input it into a machine learning model.
In (i), one needs to construct appropriate filtration depending on a given dataset and a task. A standard way to define a filtration is to consider the union of balls with centers at points in  $X \subset \mathbb{R}^d$  with a uniform radius, i.e., take the union  $S_t = \bigcup_{x \in X} B(x; t)$ ,  $B(x; t) := \{y \in \mathbb{R}^d \mid \| y - x \| \leq t\}$  and consider the increasing sequence  $(S_t)_t$ . Regarding (ii), one also has to choose a vectorization method depending on the dataset and the task. For example, one can use typical vectorization methods such as persistence landscape and persistence image. As for (iii), we can use any machine learning models such as linear models and neural networks, and the models can be tuned with the loss function depending on a specific task.
Recently, several studies have proposed ideas to learn not only (iii) but also (i) and (ii). Regarding (ii), Hofer et al. (2017) and Carrière et al. (2020), for example, have proposed methods to learn vectorization of persistence homology based on data and tasks. Furthermore, for (i), Hofer et al. (2020) have proposed a learnable filtration architecture for graphs, which gives a filtration adaptive to a given dataset. However, this method heavily depends on properties specific to graph data. To establish a method to learn filtration for point clouds, we need to develop a learnable filtration architecture for a point cloud incorporating the pairwise distance information. In this paper, we tackle this challenging problem to adaptively choose a filtration for point clouds.
In this paper, we propose a novel framework to obtain topological features of point clouds based on persistent homology. To this end, we employ a weighted filtration, whose idea is to take the union of balls with different radii depending on points. Choosing a weight function  $w \colon X \to \mathbb{R}$ , we can define
a filtration  $(S_{t})_{t}$  by
$$
S _ {t} = \bigcup_ {x \in X} B (x; t - w (x)).
$$
This type of filtration often extracts more informative topological features compared to non-weighted filtrations. Indeed, a weighted filtration plays an important role in various practical applications (Anai et al., 2020; Nakamura et al., 2015; Hiraoka et al., 2016; Obayashi et al., 2022). Then, we introduce a neural network architecture to learn a map that associates a weight function  $f(X, \cdot)$  with a point cloud  $X \subset \mathbb{R}^d$ .
The desirable property for network  $f$  is isometry-invariance, i.e.,  $f(TX, Tx) = f(X, x)$  for any isometric transformation  $T$  of  $\mathbb{R}^d$ , so that the associated weighted filtration is isometry-invariant. To address this issue, we implement a function  $f$  with a network architecture based on distance matrices. Then, the resulting weight function satisfies the isometry-invariance rigorously. Additionally, we show a theoretical result about the approximation ability for a wide class of weight functions, which supports the validity of our architecture.
Our main contributions are summarized as follows:
1. We propose a novel framework to obtain adaptive topological features for point clouds based on persistent homology. To this end, we introduce an isometry-invariant network architecture for a weight function and propose a way to learn a weighted filtration.
2. We theoretically show that any continuous weight function can be approximated by the composite of two continuous functions which factor through a finite-dimensional space. This theoretical result motivates our architecture.
3. We conducted experiments on public datasets and show that the topological features obtained by our method improved accuracy in classification tasks.
Figure 1 presents one typical use of our framework for a classification task. Our method first calculates the weight of each point using a neural network model. The colored point clouds in the second row depict the assigned weights for each point. In the bottom one, our architecture assigns large weights on the outliers in the point cloud, which implies that our method can choose the filtration adaptively to data and tasks. Although the DTM filtration is also effective in the example shown in Figure 1, our method can learn a filtration appropriately for a given task and data in a supervised way. Therefore, our method would be effective for data and tasks with difficulties other than outliers. In §5 we will show that our method improves the classification accuracy compared to the Rips or DTM filtration in several experiments.
A neural network architecture for finite point sets was first proposed by Zaheer et al. (2017), as DeepSets. Later, there appeared several geometric neural networks for point clouds, such as PointNet (Qi et al. 2017a), PointNet++ (Qi et al. 2017b), and PointMLP (Ma et al. 2022). These studies addressed the rotation invariance by approximating rotation matrices by a neural network. (Xu et al., 2021) used Gram matrices to implement an isometry-invariant neural network for point clouds. We instead use distance matrices for our isometry-invariant network architecture since the pairwise distance information is important in persistent homology.
Regarding vectorization of persistent homology, in the past, it has usually been chosen in unsupervised situations, as seen in Bubenik and Kim (2007); Chung et al. (2009); Bubenik et al. (2015); Adams et al. (2017); Kusano et al. (2017). However, recent studies such as Hofer et al. (2017) and Carrière et al. (2020) have proposed end-to-end vectorization methods using supervised learning. In particular, Carrière et al. (2020) used the DeepSets architecture Zaheer et al. (2017), whose input is a finite (multi)set, not a vector, for vectorization of persistent homology to propose PersLay. Although our method also depends on DeepSets, it should be distinguished from PersLay; our method uses it for learning a weight for computing persistent homology while PersLay learns a vectorization method of persistence homology.
For graph data, Hofer et al. (2020) proposed the idea of filtration learning in supervised way, i.e., to learn the process in (i), and this idea was followed by Horn et al. (2021) and Zhang et al. (2022). They
introduced a parametrized filtration with the use of graph neural networks and learned it according to a task. The persistent homology obtained by the learned filtration could capture the global structure of the graph, and in fact, they achieved high performance on the graph classification problem. However, filtrations for point clouds cannot be defined in the same way, as they are not equipped with any adjacency structure initially, but equipped with pairwise distances. Moreover, the filtration should be invariant with respect to isometric transformations of point clouds, and we need a different network architecture to that for graphs. While many previous studies, such as Bendich et al. (2007); Cang and Wei (2017); Cang et al. (2018); Meng et al. (2020), have tried to design special filtration in an unsupervised way based on data properties, we aim to learn a filtration in a data-driven and supervised way. To the best of our knowledge, this study is the first attempt to learn a filtration for persistent homology on point cloud data in a supervised way.
Another approach to obtaining topological features is to approximate the entire process from (i) to (iii) for computing vectorization of persistent homology by a neural network (Zhou et al., 2022) and de Surrel et al. (2022). Note that similar methods are proposed for image data (Som et al., 2020) and graph data (Yan et al., 2022). Since these networks only approximate topological features by classical methods, they cannot extract information more than features by classical methods or extract topological information adaptively to data.
2.1 Network Architectures for Point Clouds
There have been proposed several neural network architectures for dealing with point clouds for their input. Here, we briefly explain DeepSets (Zaheer et al., 2017), which we will use to implement our isometry-invariant structure. A more recent architecture, PointNet (Qi et al., 2017a) and PointMLP (Ma et al., 2022), will also be used for the comparison to our method in the experimental section.
The DeepSets architecture takes a finite (multi)set  $X = \{x_{1},\ldots ,x_{N}\} \subset \mathbb{R}^{d}$  of possibly varying size as input. It consists of the composition of two fully-connected neural networks  $\phi_1\colon \mathbb{R}^d\to \mathbb{R}^{d'}$  and  $\phi_2\colon \mathbb{R}^{d''}\to \mathbb{R}^{d''}$  with a permutation invariant operator op such as max, min and summation:
$$
\operatorname {D e e p S e t s} (X) := \phi_ {2} (\mathbf {o p} (\{\phi_ {1} (x _ {i}) \} _ {i = 1} ^ {N}))
$$
For each  $x_{i} \in X$ , the map  $\phi_{1}$  gives a representation  $\phi_{1}(x_{i})$ . These pointwise representations are aggregated via the permutation invariant operator op. Finally, the map  $\phi_{2}$  is applied to provide the network output. The output of DeepSets is permutation invariant thanks to the operator op, from which we can regard the input of the network as a set. The parameters characterizing  $\phi_{1}$  and  $\phi_{2}$  are tuned through the training depending on the objective function.
2.2 Persistent Homology
Here, we briefly explain persistence diagrams, weight filtrations for point clouds, and vectorization methods of persistence diagrams.
Persistence Diagrams Let  $S: \mathbb{R}^d \to \mathbb{R}$  be a function on  $\mathbb{R}^d$ . For  $t \in \mathbb{R}$ , the  $t$ -sublevel set of  $S$  is defined as  $S_t := \{p \in \mathbb{R}^d \mid S(p) \leq t\}$ . Increasing  $t$  from  $-\infty$  to  $\infty$  gives an increasing sequence of sublevel sets of  $\mathbb{R}^d$ , called a filtration. Persistent homology keeps track of the value of  $t$  when topological features, such as connected components, loops, and cavities, appear or vanish in this sequence. For each topological feature  $\alpha$ , one can find the value  $b_{\alpha} < d_{\alpha}$  such that the feature  $\alpha$  exists in  $S_t$  for  $b_{\alpha} \leq t < d_{\alpha}$ . The value  $b_{\alpha}$  (resp.  $d_{\alpha}$ ) is called the birth (resp. death) time of the topological feature  $\alpha$ . The collection  $(b_{\alpha}, d_{\alpha})$  for topological features  $\alpha$  is called the persistence diagram (PD) of  $f$ , which is a multiset of the half-plane  $\{(b, d) \mid d > b\}$ . The information of connected components, loops, and cavities are stored in the 0th, 1st, and 2nd persistence diagrams, respectively.
Given a point cloud  $X \subset \mathbb{R}^d$ , i.e., a finite point set, one can take  $S$  to be the distance to point cloud, defined by
$$
S (z) = \min  _ {x \in X} \| z - x \|.
$$
Then, the sublevel set  $S_{t}$  is equal to the union of balls with radius  $t$  centered at points of  $X$ :  $\bigcup_{x \in X} B(x; t)$ , where  $B(x; t) := \{y \in \mathbb{R}^{d} \mid \| x - y \| \leq t\}$ . In this way, one can extract the topological features of a point cloud  $X$  with the persistence diagram. The persistent homology of this filtration can be captured by the filtration called Čech or Vietoris-Rips filtration (Rips filtration for short). In this paper, we use Rips filtration for computational efficiency. See Appendix A for details. The Rips filtration can be computed only by the pairwise distance information.
Weighted Filtrations for Point Clouds While the Rips filtration considers a ball with the same radius for each point, one can consider the setting where the radius value depends on each point (see, for example, Edelsbrunner (1992)). For instance, Hiraoka et al. (2016); Nakamura et al. (2015); Obayashi et al. (2022), which use persistent homology to find the relationship between the topological nature of atomic arrangements in silica glass and whether the atomic arrangement is liquid or solid, use such a filtration. Given a point cloud  $X \subset \mathbb{R}^d$ , one can define the radius value  $r_x(t)$  at time  $t$  for  $x \in X$  as follows. We first choose a function  $w \colon X \to \mathbb{R}$ , which is called weight, and define
$$
r _ {x} (t) := \left\{ \begin{array}{l l} - \infty & \text {i f} t <   w (x), \\ t - w (x) & \text {o t h e r w i s e .} \end{array} \right.
$$
The associated weighted Rips filtration is denoted by  $R[X, w]$ . One choice as a weight function is the distance-to-measure (DTM) function. The resulting filtration is called a DTM-filtration, for which several theoretical results are shown in Anai et al. (2020). One of those results shows that a persistent homology calculated by the DTM-filtrations is robust to outliers in point clouds, which does not hold for the Rips filtrations.
Vectorization Methods of Persistence Diagrams Persistence diagrams are difficult to handle by machine learning since they are a multiset on a half-plane. For this reason, several vectorization methods of persistence diagrams have been proposed, such as persistence landscape (Bubenik et al., 2015) and persistence image (Adams et al., 2017). Carriere et al. (2020) proposed the method called PersLay, which vectorizes persistence diagrams in a data-driven way using the structure based on DeepSets. In this paper, we utilize this method as a vectorization method. Note that PersLay can be replaced with any vectorization method in our architecture.
Given a function transformation map  $\phi_c$  with a learnable parameter  $c$  and permutation invariant operator  $\mathbf{op}$ , we can vectorize a persistence diagram  $D$  by PersLay with
$$
\operatorname {P e r s L a y} (D) := \mathbf {o p} \left(\left\{\phi_ {c} (q) \right\} _ {q \in D}\right).
$$
If we choose appropriate parametrized function  $\phi_c$ , we can make PersLay similar to classical vectorization methods such as persistence landscape and persistence image. In this paper, we utilized the following map as  $\phi_c$ , which makes PersLay similar to persistence image:
$$
\phi_ {c} (q) = \left[ \exp \left(- \frac {\| q _ {1} - c _ {1} \| ^ {2}}{2}\right), \ldots , \exp \left(- \frac {\| q _ {M} - c _ {M} \| ^ {2}}{2}\right) \right] ^ {\top}.
$$
3 Proposed Framework
In this section, we describe our framework to obtain topological representations of given point clouds. Our idea is to use persistence homology for extracting topological information and model a weight function  $w$  in §2.2 by a neural network, which can be learned adaptively to data. Moreover, we propose a way to combine the topological features with deep neural networks.
3.1 Filtration Learning for Point Clouds
First, we introduce a structure to learn a filtration on point clouds end-to-end from data. For that purpose, we learn a weight function  $w$  for a weighted filtration, by modeling it by a neural network. We assume that the weight function depends on an input point cloud  $X = \{x_{1},\ldots ,x_{N}\} \subset \mathbb{R}^{d}$ , i.e.,  $w$  is of the form  $f(X,\cdot)\colon \mathbb{R}^d\to \mathbb{R}$ . Designing such a function by a neural network has the following three difficulties:
1. The weight function  $f(X, \cdot)$  should be determined by the whole point cloud set  $X$  and does not depend on the order of the points in  $X$ .
Figure 2: The architecture of the network. This figure is for the case of getting the weight of point  $x_{1}$  in the point cloud  $X$ . Note that  $d(X, x_{i})$  is equal to (the transpose of) the  $i$ -th row of  $D(X)$ , i.e.,  $(d(x_{i}, x_{j}))_{j=1}^{N}$ .
2. The resulting function should be isometry-invariant, i.e., for any isometric transformation  $T$ , any point cloud  $X$ , and  $x \in X$ , it holds that  $f(TX, Tx) = f(X, x)$ .
3. The output of the  $f(X, x)$  should have both of global information  $X$  and pointwise information of  $x$ .
To address the problem 1, we use the DeepSets architecture (§2.1). To satisfy invariance for isometric transformations as in problem 2, we utilize the distance matrix  $D(X) = (d(x_i, x_j))_{i,j=1}^N \in \mathbb{R}^{N \times N}$  and the relative distances  $d(X, x) = (\|x - x_j\|)_{j=1}^N$  instead of coordinates of points, since  $D(TX) = D(X)$  and  $d(TX, Tx) = d(X, x)$  hold for any isometric transformation  $T$ . Finally, to deal with problem 3, we regard  $f$  as a function of the feature of  $D(X)$  and the feature of  $d(X, x)$ . Below, we will explain our implementation in detail.
First, we implement a network to obtain a pointwise feature vector of a point  $x \in \mathbb{R}^d$  by
$$
g _ {1} (x) := \phi^ {(2)} \left(\mathbf {o p} \left(\left\{\phi^ {(1)} \left(\| x - x _ {j} \|\right) \right\} _ {j = 1} ^ {N}\right)\right), \tag {1}
$$
where  $\phi^{(1)}$  and  $\phi^{(2)}$  are fully-connected neural networks. This structure (1) can be regarded as the DeepSets architecture whose input is a set of one-dimensional vectors, hence it is permutation invariant.
Next, we use the DeepSets architecture to obtain a feature vector for the entire point cloud  $X$  as follows. We first implement pointwise feature vectors with the same architecture as  $g_{1}$ , i.e.,
$$
g _ {2} (x _ {i}) := \phi^ {(4)} (\mathbf {o p} (\{\phi^ {(3)} (d (x _ {i}, x _ {j})) \} _ {j = 1} ^ {N})),
$$
where  $\phi^{(3)}$  and  $\phi^{(4)}$  are fully-connected neural networks. Then, we get a feature vector  $h(X)$  for the entire point cloud  $X$  by applying the DeepSets architecture again:
$$
h (X) = \phi^ {(5)} (\mathbf {o p} \left(\left\{g _ {2} \left(x _ {i}\right) \right\} _ {i = 1} ^ {N}\right)), \tag {2}
$$
where  $\phi^{(5)}$  is a fully-connected neural network. Here we omit the first inner network for the DeepSets architecture since it can be combined with  $\phi^{(4)}$  in  $g_{2}$ .
Finally, we implement  $f$  as a neural network whose input is the concatenated vector of  $h(X)$  and  $g_{1}(x)$ . Specifically, we concatenate  $h(X)$  with the feature vector  $g_{1}(x)$  for  $x \in \mathbb{R}$  to get a single vector, and then input it into a fully-connected neural network  $\phi^{(6)}$ :
$$
f _ {\theta} (X, x) = \phi^ {(6)} ([ h (X), g _ {1} (x) ] ^ {\top}),
$$
where  $\theta := (\theta_k)_{k=1}^6$  is the set of parameters that appear in network  $\phi^{(k)}$  ( $k = 1, \ldots, 6$ ). The overall neural network architecture is shown in Figure 2.
Remark 3.1. Since our implementation of the weight function depends on distance matrices, not on the coordinates of each point, we can apply our method to a dataset given in distance matrix format. In fact, we will apply our framework to such a dataset in Section 5.
Remark 3.2. If we want to consider features other than the position information assigned to each point in the calculation of the filtration weights, as in the silica glass studies, we vectorize those features and concatenate them together.
By using the weight function  $f_{\theta}(X,\cdot)$ , we can calculate the persistent homology of filtration  $R[X,f_{\theta}(X,\cdot)]$ . Then, we can obtain the vectorization using PersLay parametrized by  $c$ . Then, we can input this vector into a machine learning model  $F_{\theta'}$  with parameter  $\theta'$ . Given the dataset  $\{(X_k,y_k)\}_{k=1}^M$ , the objective function to be minimized can be written as
$$
\mathcal {L} (\theta , \theta^ {\prime}, c) := \frac {1}{M} \sum_ {k = 1} ^ {M} \ell \left(F _ {\theta^ {\prime}} \left(\operatorname {P e r s L a y} \left(R [ X _ {k}, f _ {\theta} (X _ {k}, \cdot) ]\right)\right), y _ {k}\right),
$$
where  $\ell$  is an appropriate loss function depending on tasks. Typically, if the task is regression,  $\ell$  is the mean square loss, and if the task is classification,  $\ell$  is the cross entropy loss. The optimization problem
$$
\min  _ {\theta , \theta^ {\prime}, c} \mathcal {L} (\theta , \theta^ {\prime}, c)
$$
is solved by stochastic gradient descent. Note that the differentiability of the objective function and the convergence of the optimization algorithm is guaranteed by results in Carrière et al. (2021).
3.2 Combining Topological Features with a DNN-based method