Simplicial Complex Networks
Authors: Mohammad Firouzi
Sadra Boreiri
Hamed Firouzi
Venue:  (2020)
Keywords: topological data analysis
supervised learning
simplicial approximation

Abstract: Universal approximation property of neural networks is one of the motivations to use these models in various real-world problems. However, this property is not the only characteristic that makes neural networks unique as there is a wide range of other approaches with similar property. Another characteristic which makes these models interesting is that they can be trained with the backpropagation algorithm which allows an efficient gradient computation and gives these universal approximators the ability to efficiently learn complex manifolds from a large amount of data in different domains. Despite their abundant use in practice, neural networks are still not well understood and a broad range of ongoing research is to study the interpretability of neural networks. On the other hand, topological data analysis (TDA) relies on strong theoretical framework of (algebraic) topology along with other mathematical tools for analyzing possibly complex datasets. In this work, we leverage a universal approximation theorem originating from algebraic topology to build a connection between TDA and common neural network training framework. We introduce the notion of automatic subdivisioning and devise a particular type of neural networks for regression tasks: Simplicial Complex Networks (SCNs). SCN's architecture is defined with a set of bias functions along with a particular policy during the forward pass which alternates the common architecture search framework in neural networks. We believe the view of SCNs can be used as a step towards building interpretable deep learning models. Finally, we verify its performance on a set of regression problems.

Under review as a conference paper at ICLR 2020
SIMPLICIAL COMPLEX NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
Universal approximation property of neural networks is one of the motivations to use these models in various real-world problems. However, this property is not the only characteristic that makes neural networks unique as there is a wide range of other approaches with similar property. Another characteristic which makes these models interesting is that they can be trained with the backpropagation algorithm which allows an efficient gradient computation and gives these universal approximators the ability to efficiently learn complex manifolds from a large amount of data in different domains. Despite their abundant use in practice, neural networks are still not well understood and broad range of ongoing research to study the interpretability of neural networks. On the other hand, topological data analysis (TDA) relies on strong theoretical framework of (algebraic) topology along with other mathematical tools for analyzing possibly complex datasets. In this work, we leverage a universal approximation theorem originating from algebraic topology build connection between TDA and common neural network training framework. We introduce the notion of automatic subdivisioning and devise a particular type of neural networks for regression tasks: Simplicial Complex Networks (SCNs). SCN's architecture is defined with a set of bias functions along with particular policy during the forward pass which alternates the common architecture search framework in neural networks. We believe the view of SCNs can be used as a step towards building interpretable deep learning models. Finally, verify its performance set of regression problems.
INTRODUCTION
is well-known that under mild assumptions on the activation function, a neural network with one hidden layer and finite number of neurons can approximate continuous functions. This characteristic of neural networks is generally referred to as the universal approximation property. There are various theoretical universal approximators. For example, a result of the Kione-Weierstrrn theorem Stone (1948); Cotter (990)) is that multivariate polynomials are dense in the space of continuous real valued functions defined over hypercube. Another example is that the reproducing kernel Hilbert space (RKHS) associated with kernel functions with particular properties can be dense in the same space of functions. Kernel functions with this property are called universal kernels Micchelli et al. (2006). A subsequent result of this theory is that the set of functions generated by a Gaussian process regression with an appropriate kernel can approximate any continuous function over a hypercube with arbitrary precision. Although multivariate polynomials and Gaussian processes also have this approximation property, each has practical limitations that cause neural networks to be used more often in practice compared to these approaches. For instance, polynomial interpolations may result a model that overfits to the data and suffers from a poor generalization, and Gaussian processes often become computationally intractable for large number of training data Bernardo et al.)
Neural networks, with an efficient structure for gradient computation using backpropagation, can be trained using gradient based optimization for large datasets in tractable time. Moreover, in contrast to existing polynomial interpolations, neural networks generalize well in practice. Theoretical and empirical understanding of the generalization power of neural networks is ongoing research Novak et al. (2018); Neyshabur et al. (017))
Topological Data Analysis (TDA), geometric approach for data analysis, is growing field which provides statistical and algorithmic methods to analyze the topological structures data often referred to point clouds. TDA methods mainly relied deterministic methods until recently where

---

Under review as a conference paper at ICLR 2020
Nell
h1
1v2
Figure 1: General view of an SCN: Left: without applying any transformation to its input, an SCN locates the position of input in the imput space through set of nested simplexes. Right: architecture of an SCN: v2 are given set of visible vertices of primary simplex in the input space that the sample falls inside. network of hidden vectors h2 are then used to parameterize a sequence of nested simplexes for locating the input. Each h; is convex combination of subset of its preceding vectors. parallel, another network is used to generate SCN's output utilizing the output of SCN at all vs and hg. combined with group of bias values b,.
statistical approaches were proposed for this purpose Carriere et al. ); Chazal & Michel (207))
In general, TDA methods assume point cloud in metric space with inducing distance (c.g. Euclidean, Hausdorff, or Wasserstein distance) between samples and build a topological structure upon point clouds. The topological structure is then used to extract geometric information from data Chazal & Michel (2017). These models are not trained with gradient based approaches and they are generally limited to predetermined algorithms whose application to high dimensional spaces may be challenging Chazal (2016).
In this work, by leveraging geometrical perspective of TDA, we provide class of restricted neural networks that preserve the universal approximation property and can be trained using forward pass and the backpropagation algorithm. Motivated by the approximation theorem used to develop our method, Simplicial Complex Network (SCN) is chosen to refer these models. SCNs do not require an activation function and architecture search in the way that conventional neural networks do. Their hidden units are conceptually well defined, in contrast to feed-forward neural networks for which the role hidden unit is yet an ongoing problem. SCNs are discussed more details in later sections.
Our contribution can be summarized in building novel class of neural networks which we believe can be used in the future for developing deep models that are interpretable, and robust to perturbations. The rest of this paper is organized as follows: Section 2 specified for the explanation of SCNs and their training procedure. In section 3 related works are explained. Sections to experiments, limitations, and conclusion.
5 and are specified
2 SIMPLICIAL COMPLEX NETWORKS
We first describe some necessary notation in the section 2.1 In section 2.2 we discuss the barycentric subdivision and the simplicial approximation theorem. In section 2.3 modify the barycentric subdivision order to develop an approach that allows us to learn a subdivision of the input space into small simplexes. We then introduce general framework for defining SCN for regression tasks that simultaneously learns a subdivision of the input space and a piece-wise wise linear mapping where the linear pieces defined each simplex of the subdivision.
2.1 NOTATION
We consider dataset D - (xx(6, y())) of N i.i.d. input/output pairs, where for each 1 < í N, z()) E Rd and y())) R Let fo denotes mapping from the input to the output space in which represents its parameters. In a regression task, we wish to minimize (0) = ( (+)),, where is a given loss function.
We indicate a dimensional simplex (d-simplex) with vertices Vp .... Vs with the notation (Wo, v,,..., Ma]. For simplicity of the presentation, we further assume that each (1)) lies inside
2

---

Under review as a conference paper at ICLR 2020
(a)
Figure 2: (a) Second barycentric subdivision of a 2-simplex. (b) First barycentric subdivision of 3-simplex. (c), (d) Nested set of simplexes to get a sample simplex (shown black) from: (c) the second barycentric subdivision of 2-simplex, (d) the first barycentric subdivision of 3-simplex.
a d-simplex = [Vo. ui,..., Val, where vo is the origin, and Uy... are the standard basis vectors. Similarly, we assume each ( also lies in the standard probability k-simplex.
2.2 SIMPLICIAL APPROXIMATION
Simplicial approximation theorem allows the approximation of continuous functions using simplicial mappings. Before stating the theorem, we borrow a few definitions from the algebraic topology literature.
Definition 1. (simplicial complex) A simplicial complex K a set of simplexes such that: 1) Every face of a simplex of K is in K. 2) The intersection of any simplexes in K is face of each of them. (A simplicial complex can be informally defined as a set of simplexes glued together through their faces.)
Definition 2. (simplicial mapping) A mapping between two simplicial complexes is called simplicial mapping if the images of the vertices are vertices.
We also use the definition of standard subdivisioning method which used break a d-simplex (or any other simplicial complex) into arbitrary small simplexes with the same dimension.
Definition 3. (barycentric subdivision) Barycentric subdivision (BCS) of a d-simplex K consists of
(d + 1)! d-simplexes. Each d-simplex [vo, U1, va] out of these (d + 1)! simplexes associated with permutation po: Pi...., Pd of the vertices of K such that denotes the barycenter (centroid) of Po. Pi..... P; where S i n.
Figure aa). (b) visualize examples of BCS for 2-simplex, and 3-simplex. Note that i-th BCS is the result of applying BCS to each simplex in the (i - 1)-th BCS. Using these definitions, simplicial approximation theorem be stated as follows,
Theorem 1. (Simplicial Approximation Theorem) Let X and Y be two simplicial complexes and f: X + Y be a continuous function. Then an arbilrary there exist sufficiently large k and I and a simplicial mapping g: (ik (I) approximating I such that supaex (z) g(x))l (k) and ()) represent the k-th and 1-th barycentric subdivision of X and Y, respectively.
In the appendix A provide short topological proof for this theorem. Although the theorem provides a general approximation framework, its approximation through using BCS of the input and output spaces. Each time BCS is applied, the number of simplexes is multiplied by (d + 1)! and simplexes in higher order subdivisions become flatter and flatter Diaconis Miclo (201T). This fact limits the use of this subdivision algorithm in practice. Moreover, BCS subdivides input output space completely independent of the data. In the next section, we modify the BCS to a data-driven approach which allows to learn suitable subdivision given data.
Apart from BCS, in TDA, building simplicial complexes from data often based on deterministic approaches. For instance, Vietoris-Rips complex Carlsson all, (2006); Attali et al. (2013) is the set of all simplexes which their vertices are data points that their pair distances are less than a threshold. Čech complex Attali Attali et al. (2013): Kerber & Sharathkumar (2013), is the set all those simplexes that completely lie in closed ball with specific radius. While these non-parametric data driven
3

---

Under review as a conference paper at ICLR 2020
methods can extract rich topological structures of the data using mathematical tools such persistent homology, they are not often used as standard features in machine learning algorithms immediately Chazal & Michel (017);; Attali et al. (2013). In the next section, modify BCS to parametric framework that allows automation of the subdivisioning process and can be used directly during the training of an SCN.
2.3 AUTOMATIC SUBDIVISION
In the Algorithm| 1 we have shown the process of generating random simplex from the set of all simplexes BCS of d-simplex. The algorithm gives an uncommon view for identification of a simplex in the BCS which is through a set of nested simplexes. We modify this view to a data driven approach in a way that BCS is a special case of the modified version.
In the Algorithm 1 for random permutation P, No 3 D.. S Na form a nested set of simplexes. This fact is visualized in Figure 2)) (d). Nu indicates the simplex in the BCS that corresponds to P. Knowing the nested sequence uniquely determines simplex in the BCS. Also, we note that each N; can be obtained by replacing one of the vertices of N with a new vertex inside or on the boundary (closure) of it. Thereby, since the new vertex is in the closure of N it can be represented as a convex combination of the vertices of - The weights of these convex combinations can be computed in straightforward way as shown the Algorithm ! to the BCS.
Note that these weights are specified
To build data driven approach initially we eliminate two restrictions of the Algorithm | First, we allow the number of repeats to be any arbitrary integer rather fixing it to d. This number is later referred as the subdivision depth. Second, at each repeat, we let the weights to freely have any arbitrary values as long as they can be used as convex combination weights. This removes the restriction of having deterministic weights specified for BSC. This modification allows us learn the weights through optimization process.
natural approach to make the subdivision process data-driven to, instead of randomly sampling a simplex as in the Algorithm|] [ sample simplex with the probability proportion to the likelihood that it contains data and update the subdivision parameters accordingly. However, the number of all possible simplexes grows exponentially these probabilities intractable. Alternatively, we sample from data and identify the simplex in the subdivision that contains the sample and use that simplex in stochastic optimization framework to update the subdivision towards a desired one. But how can we identify the simplex that contains given sample? Our parameterization of the subdivisioning process using the nested sequence of simplexes along with the following lemma help to locate the input using a forward pass,
increases, thereby making the computation of all
Lemma 1. Let x denote sample inside d-simplex a =[oo,.., va). Assume x - way; is represented as a convex combination of the vertices of Let h= h w/y; be another point in the simplex written in the same format. Indicating the simplex with vertices of except vs replaced by h with o lies in the simplex o where arg min; !i w!-
Proof. x can be represented with the following convex combination which results the lemma,
x
Wi w
w
v.) +
W w
h
Assume a sample x ====== ΣE i.v; represented as convex combination of vertices of given d-simplex [Do, U2,,.., Dz). Similar to the steps of Algorithm and based on our view for identification of the target simplex through a nested set of simplexes, at the first step, a point is added inside the simplex using a convex combination of vertices be used to locate the in the and accordingly replace one of its vertices with the new vertex. This process is repeated with the new d-simplex up to times to extract the simplex in the subdivision that contains æ. Having the final simplex and given cost function, parameters of the subdivision can be updated accordingly using the backpropagation and gradient descent framework. The procedure is formally shown in the Algorithm
with weights wT. Then lemma Maan
4

---

Under review as a conference paper at ICLR 2020
We refer the general procedure described in Algorithm to automatic subdivision. Note that the barycentric subdivision can be a particular outcome of the automatic subdivisioning.
Algorithm Generating simplex in barycentric subdivision of d-simplex
Algorithm One gradient step in automatic sub- divisioning of a d-simplex using one data sample (Vo, 02 ... val, sample a Σi= Way Us θ {w] Wi J. Loss L initialize: No o. =
d-simplex a (vo, 12 .., Val, permu- tation P = (po. Pi...., Pd)
input:
input: d-simplex
initialize: Nú = 0, = 1d, 00
repeat
Compute u = Ei wy N, WEy wa
repeat
Set k = arg min,
compute u >
[d+
Set N N, with k-th vertex replaced with u
Set up; =
Set Nj+1 Nj with py-th vertex replaced
Set wn as convex combination weights represented with vertices in Nj+1
by u
= ++ +1
j=j+1
until
until j =1 - θ a (QL(((
return: Na
Project each w; E Q the standard d-simplex
2.4 SIMPLICIAL COMPLEX NETWORKS
As theorem 2 states, any continuous function from simplicial complex to another can be approxi- mated with simplicial mapping from BCSs of the input space to BCSs of the output space. In the last section explained how we can subdivision the input space through the automatic subdivision process where BCS was particular outcome. In this section, we develop SCNs to parameterize class of simplicial mappings with trainable values vertices of the input space subdivision and it is defined linearly on the d-simplexxs of the subdivision using the evaluations at its vertices, resulting a piece-wise linear simplicial mapping. Parameters of this simplicial mapping are then optimized for function approximation or a regression tasks. We leverage a same technique used in the previous section parameterize the SCN output.
Recalling our initial assumption that inputs lie in a d-simplex o = [wo: fal (Uu representing the origin and other v; are the standard basis vectors), given input be represented with the following convex combination,
x =
+
In the previous section, we showed that how the position of x can be found in through the subdivisioning process. SCN's output at calculated using the values of its mapping at the vertices of a and added vertices during locating æ. Simplicial mapping at the m-th added vertex is defined recursively using its preceding vertices. Assuming that the m-th added vertex represented as hm = Ei= U. where u, are vertices of the preceding simplex the nested sequence of simplexes used to locate and Wm, are its corresponding convex combination weights, the value of the simplicial mapping at hm is defined as,
f(hm)
Σ Wm.if (u;) + bm (hm)
In other words, f(hm) is defined using convex combination of í' f(u;) with the same weights, added with a bias which is a function of hm- A geometrical view of equation for 2-simplex is shown in the Appendix C Using straightforward to show that as long as we consider each b,, is an arbitrary function of the hm, the simplicial mapping holds the universal approximation property. In our experiments, however,
proof of simplicial approximation theorem in the appendix A it is
5

---

Under review as a conference paper at ICLR 2020
used primitive model for biases and defined them as constant parameters. We will empirically show that even this simple model for biases can result complicated simplicial mappings and accurate approximations.
All in all, an SCN is defined using the mentioned process (Figure visualizes general architecture). Parameters that should be learned are bias function parameters, and the weights that used to subdivision the input space. All these parameters can be updated using the backpropagation algorithm. Another point that must be noted here is that, shown in Figure inputs to h2, ... hu are not specified. Even though we described how to obtain the value of these inputs using lemma IT the order that the vertices of the preceding simplex are combined is specified. We refer to the policy on the ordering of the vertices used to fed to the next convex combination as the SCN's network policy.
Specifying the depth, network policy, and the bias functions fully determines the architecture of an taken with respect
SCN. To train an SCN, the derivative of the loss function over to the weights and the bias function parameters, and these parameters are updated using gradient descent approach such as stochastic gradient descent, or Adam Kingma & Ba (0014). General training process for SCN is shown in the Algorithm 3 Note that after updating weights of the network, for each hidden node, a projection of weights is used such that their summation is equal to 1. This projection can be avoided through of logits and the Softmax function in parameterizing the weights.
Algorithm 3 training procedure for general SCN
S = [Mo...., Ual. 1 = depth, 6, Ow (bias function, and weight params), (network policy), (xr = -iio Wzz S4 y) (input/output pair), (learning rate)
// forward pass
for m {l....,I O do Permute using
hm
= Ei= Sm,s..;
((hm) = -E (S,) 6m (S; 0g)
Extract j and update wy using S, and lemma S,= hmm
end for
Wz; (xx) - E;= I2, (S:)
11 backward pass and parameter updates
0, 0, a V C(f(x) (x), y)
Ow Ow a ow [(f(x)))
for m E f1..... l] do
project wm on the standard d- d-simplex end for
3 RELATED WORK
In TDA, Persistent homology methods are commonly used to extract features that are robust to perturbations of the input Otter al. (2017); Adams al. (2017). A range of works use these features in a feed-forward architecture. For instance, in Liu et al. (2016), persistence landscape, topological summary of data, is used develop persistent layers. These layers are used in a neural network architecture along with convolutional layers and the resulting architecture is applied to the problem of music tagging Law et al. (2009), and multi-label classification Firouzi al. (2017). A similar approach is applied in Umeda (201)) for the time series classification Xi et al. (2006). Cang Weil (017)) introduces TopologyNet, an architecture that uses a persistent homology method for 3D biomolecular structures to extract features suitable for convolutional layers. Hofer et al. (2017) proposes a trainable layer to learn suitable representation from persistence diagram of the data. This layer which extracts topological signatures from the data is used along with common neural network architecture and their approach achieved the state-of oftte-of-thh-rn the-art a specific task on classification of social network graphs the time.

---

Under review as a conference paper at ICLR 2020
(a)
(b)
Figure 3: Approximation of summation of two dimensional Gaussian functions with an SCN. (a) target function, (b) approximated function using SCN with depth of 4, (c) the learned subdivision on the input space.
Although aforementioned methods try improve the performance of neural networks through using topological scenarios, the TDA geometrical perspective vanishes as they are aligned with commonly used architectures. In addition, specific persistent homology method is applied for a determined task. SCNs are single-model architectures that can be trained with the common forward and backward passes. In addition, no specific persistent homology used extract the topological geometry of the inputs, which enables its generalization to other domains.
4 EXPERIMENTS
We perform regression tasks in order to evaluate the performance and the complexity of functions that SCN can learn. Mean squared error used the loss function in all the experiments. In some cases, an SCN with a few number of parameters can estimate a function better than a neural network with fully connected layers even with around 10 times more parameters. In the appendix B with straightforward derivation, we also show that how does an SCN without a hidden unit reformulates the linear regression problem.
4.1 APPROXIMATING SUM OF GAUSSIAN FUNCTIONS
As primary experiment, approximate sum of three Gaussian functions with an SCN with depth of 4 and constant parameters as the biases. The resulting simplicial mapping and the learned subdivision is shown in Figure
4.2 TWO MORE TOY EXPERIMENTS
We compare the performance of using an SCN in the problem of learning particular structures to the results obtained by a neural network with fully-connected layers and ReLU activations. This is more general experiment done in Anil et al. (2018) to learn the absolute value function. The experiment was used Anil et al. (2018) show the limitations of specific activation functions for neural networks in learning some of the basic functions. Here, we show that SCN can learn more complicated structures even with constant parameters as bias functions. Models are trained using Adam Kingma& & Ba (2014) with learning rate 0.001, and same mini-bated size. Figure freuree/resssssetts the comparison. More details about the experiments, and a binary classification experiment on the MNIST dataset can be found in the appendix F
4.3 MEMORY ANALYSIS
Several works have proposed approaches to improve the memory efficiency of training deep and very deep neural networks Behrmann al. (2018a); Srivastava al.] (2015): Chen al. (2016); Gomez et al. allaa112210 (2017). Particularly, reversible architecture Chang et al. (2018) Behrmann et al. (20 8a); Gomez al. (201)) may result an ((1) memory cost of activation in terms of the number of layers. Although due to non-existence of activation function in SCNs the term reversibility is not applicable
Moore accurate approximations and complicated subdivisions using deeper SCNs is provided in the ANONY.
MOUS link.
7

---

Under review as a conference paper at ICLR 2020
Full Conracted
tc JAX
(8)
(b)
(d)
Figure Comparison of an SCN with a fully-connected neural network architecture in approximation of specifie functions: (a) and (c) indicate the target and fitted functions, (b) and (d) indicate the comparison terms of log loss.
here, to backpropagate through the SCN's architecture it is required to store only out of d + vectors and their function evaluations, which were used to extract the last hidden node and evaluate its function value, latest convex weights for the input in the forward pass, and array of integers with a length of indicating the order that nodes were removed during the forward pass. Storing not
these values enough for the gradient computation of the network parameters. Interestingly, needed to store the weights of the network and function evaluations at the previous nodes as they all can be extracted with these information (proof in the Appendix H). This approach trades the cost of memory with an additional computational cost and can be helpful in training very deep SCNs.
5 LIMITATIONS
In some conditions, the bias function for a hidden node may outputs high norm vectors. In these conditions, output may not be close to a smooth curve. Accordingly, SCN's output not behave well in practice and potentially overfits to data. case of using a conventional neural networks, to increase the stability of the output, ctt al. (2017), Gouk et al. (2018); Behrmann et al. (2018b). Enforcing Lipschitz constraints in an SCN can be done through their bias functions. The simplest approach is perhaps to clip the values of the bias functions to a predetermined interval.
approach is to enforce Lipschitz constraints Anni et al. (2018): Arjovssy
In cases, input weights to hidden node might converge to degenerate values with the value one in one position and zero elsewhere. In these cases, the corresponding layer does not change the learned subdivision and it may be assumed as weight dristing. In situations that we have deep SCN architecture with higher than the true required capacity, these layers with a bias value close to zero may be helpful automatically adjust the capacity of the model and prevent SCN from overfitting.
redundant layer. We refer this phenomena
Throughout this work we assumed that inputs lie in a given d-simplex. The assumption was used just for the sake of presentation. It can be assumed that samples lie in any given simplicial complex, not specific d-simmlex. The vertices of any simplex in the simplicial complex that sample falls in can be used as the primary nodes in the network.
Choosing an appropriate network policy for SCNs may be assumed as a challenging task different domains. In our experiments, we observed that even simple policies result proper function approxi- mation. In fact, in one dimensional case, a direct use of the Stone- Weierstrass theorem can be used to prove that SCNs are universal approximators even with random policies and fixed weights. In our experiments, the sensitivity of SCNs to the choice their bias function observed be larger than the network policy.
6 CONCLUSION AND FUTURE WORK
In this work, we have used techniques from topological data analysis to build a class of neural network architectures with the universal approximation property which can be trained using the common neural network training framework. Topological data analysis methods are based on the geometrical structure of the data and have strong theoretical analysis. SCNs made using the geometrical view
8

---

Under review as a conference paper at ICLR 2020
of TDA and we believe that they can be used step towards building interpretable deep learning models.
Most of the experiments in the paper synthetic. More practical applications of the paper is considered as an immediate continual work. Moreover, throughout this work, bias functions of the simplest kinds (constant parameters) were used. We mentioned earlier that bias function may be an arbitrary function of its input to keep the universal approximation property of SCNs. A natural idea use common neural network architectures the bias function. In this case, backpropagation can be continued to the bias function parameters as well. This is also considered as another continuation of this work.
ACKNOWLEDGMENTS
Anonymous
REFERENCES
Henry Adams, Tegan Emerson, Michael Kirby, Rachel Neville, Chris Peterson, Patrick Shipman, Sofya Chepushtanova, Eric Hanson, Francis Motta, and Lori Ziegelmeier. Persistence images: A stable vector representation of persistent homology. The Journal of Machine Learning Research,
18(1):218-252, 2017.
Cem Anil, James Lucas, and Roger Grosse. Sorting out lipschitz function approximation, arXiv preprint 1rX1v:1811.005381, 2018.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. 1rX1v:1701.07875. 2017.
Wasserstein gan.
arXiv preprint
Dominique Attali, André Lieutier, and David Salinas. Vietoris-rips complexes also provide topo- logically correct reconstructions of sampled shapes. Computational Geometry, 46(4):448-465. 2013.
Jens Behrmann, Sören Dittmer, Pascal Fernsel, and Peter Maass. Invariance and inverse stability under relu. 2018a.
Jens Behrmann, David Duvenaud, and Jörn-Henrik Jacobsen. Invertible residual networks. arXiv preprint 1Xiv:1811.0009. 2018b.
J Bernardo, Berger, et al. Regression and classification using gaussian process priors.
Zixuan Cang and Guo-Wei Wei. Topologynet: Topology based deep convolutional and multi-task neural networks for biomolecular property predictions. PLOS Computational Biology, 13(7): e1005690, 2017.
Erik Carlsson, Gunnar Carlsson, and Vin De Silva. An algebraic topological method for feature identification. International Journal of Computational Geometry & Appliccations 16(04))291-314 2006.
Mathieu Carriere, Marco Cuturi, and Steve Oudot. Sliced wasserstein kernel for persistence diagrams. arXiv preprint arXiv:1706.00 033503338, 2017.
Bo Chang. Lili Meng, Eldad Haber, Lars Ruthotto, David Begert, and Elliot Holtham. Reversible architectures for arbitrarily deep residual neural networks. In Phirry-Second AAAI Conference on Artificcal Intelligence, 2018.
Frédéric Chazal. High- High-ddmensional topological data analysis. 2016.
Frédéric Chazal and Bertrand Michel. An introduction to topological data analysis: fundamental and practical aspects for data scientists. arXiv preprint 1rXiv:1710.00: 2017.
Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. Training deep nets with sublinear memory cost. arXiv preprint arXiv: 1604 2016.

---

Under review as a conference paper at ICLR 2020
Neil E Cotter. The stone- weieestsisssssss theorem and its application to neural networks. IEEE Transac- tions on Neural Networks, 1(4):290-29)) 2992, 1990.
Persi Diaconis and Laurent Miclo. On barycentric subdivision. Combinatorics, Probability and Computing, 20(2):213-237, 2011.
Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non-linear independent components
estimation. arXiv preprint vrXivv/410.88 16, 2014.
Mohammad Firouzi, Mahmood Karimian, and Mahdich Soleymani. Nmff based label space factor- ization for multi-label classification. In Machine Learning and Applications (ICMLA), 2017 16th IEEE International Conference on, pp. 297-303. IEEE, 2017.
Aidan N Gomez, Mengye Ren, Raquel Urtasun, and Roger B Grosse. The reversible residual network: Backpropagation without storing activations, In Advances in Neural Information Processing Systems, pp. 2214-2224, 2017.
Henry Gouk, Eibe Frank, Bernhard Pfahringer, and Michael Cree. Regularisation of neural networks by enforcing lipschitz continuity. arXiv preprint 3.X1::8804004366 2018.
Christoph Hofer, Roland Kwitt, Marc Niethammer, and Andreas Uhl. Deep learning with topological signatures, In Advances in Neural Information Processing Systems, pp. 1634-1644, 2017.
Michael Kerber and Sharathkumar. Approximate čech complex in low and high dimensions. In International Symposium on Algorithms and Computation, pp. 666-676. Springer, 2013.
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Edith Law, Kris West, Michael Mandel, Mert Bay, and J Stephen Downie. Evaluation of algorithms using games: The case of music tagging. In ISMIR, pp. 387-392, 2009.
Jen- Yu Liu, Shyh-Kang Jeng, and Yi-Hsuan Yang. Applying topologiaal persistence in convolutional neural network for music audio signals. arXiv preprint rrXiv:1608.0737, 2016.
Charles A Micchelli, Yuesheng Xu, and Haizhang Zhang. Universal kernels. Journal of Machine Learning Research, 7(Dec):2651-266 2006.
Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nati Srebro. Exploring general- ization deep learning. In Advances in Neural Information Processing Systems, pp. 5947-5956, 2017.
Roman Novak, Yasaman Bahri, Daniel A Abolafia, Jeffrey Pennington, and Jascha Sohl- Dickstein. Sensitivity and generalization in neural networks: an empirical study. arXiv preprint arXiv:1802.08760, 2018.
Nina Otter, Mason A Porter, Ulrike Tillmann, Peter Grindrod, and Heather Harrington. A roadmap for the computation of persistent homology. EPJ Data Science, 6(1):17, 2017.
Rupesh K Srivastava, Klaus Greff, and Jürgen Schmidhuber. Training very deep networks. In Advances in neural information processing systems, pp. 2377-238. 2015.
Marshall H Stone. The generalized weierstrass approximation theorem. Mathematics Magazine, 21 (5):237-254, 1948.
Yuhei Umeda. Time series classification via topological data analysis. Information and Media Technologies, 12:228-239, 2017.
Xiaopeng Xi, Eamonn Keogh, Christian Shelton, Li Wei, and Chotirat Ann Ratanamahatana, Fast time series classification using numerosity reduction. In Proceedings of the 23rd international conference on Machine learning, pp. 1033-1040. ACM, 2006.
10

---

Under review as a conference paper at ICLR 2020
A PROOF OF SIMPLICIAL APPROXIMATION THEOREM
We provide a short topological proof of the approximation theorem we used in main text. Some mathematical precision is dropped throughout the proof.
Theorem 2. (Simplicial Approximation Theorem) Let X and Y be two simplicial complexes. Let f: | |Y] be confinuous function. Then for an arbitrary there exist sufficiently large k and I
and simplicial mapping g [x + [ approximating f such that supnex ( g(x))l <
Proof: In order to prove this theorem we use the following definition:
Definition 4. (Star) Let K be a simplicial complex and v E K a vertex of K. The star of represented by SI(v) is defined as the union of interior of all simplexes of K that contain v as vertex. Note that SI(v) is an open set. Figure pictures an example for star of a vertex.
Figure 5: Star of vertex of simplicial complex (gray region)
Choose / in order that the diameter of simplexes of ( is less than e. Let W denotes the Y vertices. Since f is a continuous function, pre-image of an open set in an open set. Therefore, for each E W, f- (St((()) is an open set. {f (St((S(ww)) (w)) wee is an open covering for [X]] Let 8 represents the Lebesgue number of this covering. Choose k such that the diameter of simplexes of ((k be less than 5. Denote the vertices of X with V. can be shown that for each E V. diameter of St(v)) is less than ó (proof is shown in figure 6). So, for each V, there exist a E W such that St(v) C (((((w)) We define the simplicial map g such that g(v) = w. Also, note that f(St(v)) St(w).
Now prove that g approximates f as desired. Let E X (k) be simplex in A. Let U1 U2 ... Up denote the vertices of this simplex. For any x E a and a vertex U i E 21, P). x is in the St(u,). So. f(x) is in the ( Using the last note in the previous paragraph, we have I E ni St(g(u)) This fact means that f(r) lies in the simplex that its vertices are [(v1), J (V2), .... J(Np).
We now extend the definition of for all non-vertex elements of X. Let x E |X] be a non-vertex element represented convex combination of number of vertices of V as El.7, We define g(x) as,
g(z) - 9( = Eng((e)
Straightforward steps can be used to prove that is a continuous simplicial mapping. Using the facts in the last two paragraphs, we conclude that for each x E |X] and simplex X containing æ, both f(x), and g(z), lie in the simplex with vertices {g(u)) Due to the initial choice of I, we know that diameter this simplex is less than e, which means supaex [x) - =(x)]| <e.
B REFORMULATION OF LINEAR REGRESSION PROBLEM USING AN SCN WITHOUT HIDDEN NODES
In case that an SCN has no hidden node (no subdivisioning process), it can be viewed as linear regression reformulation. A real valued linear function A + R from d-dimensional simplex A = [eo,... Vzl. can be specified by the values of f at each v; These values are represented by f(v:).
Assume a data matrix e R Nxd of N samples within A4, and their corresponding output in a vector y. We formulate the linear regression problem with training weight to that minimizes ||x yl
11

---

Under review as a conference paper at ICLR 2020
B
Figure 6: Let A, B be two simplexes containing v. Using triangle inequality for distance function d. for each E A. E B we have: d(a, b) & d(a, + d(v, b) < + < This fact shows that the diameter of St(v) is less than or equal to 8.
We represent the coefficients of representing samples in X convex combination of vo. Vd in matrix C (xx+(1+1) with rank of most d, where i-th indicates the corresponding coefficients for i-th sample. Then the linear regression problem can be reformulated as.
||Cf - -yyll
where f is (d + 1) dimensional vector representing the function value v; its i-th element. With a straightforward computation, One can verifies that the optimal w or f can be computed from the optimal value of the other one.
C GEOMETRIC VISUALIZATION OF EQUATION
fwo
Figure A geometrical view of how does equation 2 evaluaaee the simplicial mapping at m-th added point using its preceding vertices A same convex combination of u, used to generate hm is applied to the corresponding /(,,). This value added with bias term determines f(hm).
D PROOF OF LEMMA
can be represented with the following convex combination which results the lemma,
Wj W
vs)
+
Wy 2n
h
E RETRIEVING PREVIOUS LAYERS PARAMETERS
In contrast Dinh et al. (2014); Gomez et (011 7)). no separation of input into blocks is needed for SCNS to extract outputs of the previous layers. Retrieving the values for previous in SCN is a result the fact that knowing all weights, all vertices except one, the resulting vector of convex combination is enough to extract the missing vertex. Formally, let æ = Σie W;;2; represents
convex combination E Wi 1, and VA < E d: wa * 0). Assuming ty is unknown, it can be
12

---

Under review as a conference paper at ICLR 2020
computed simply as,
Vj
(x W 17
Σ wwws)
Assume an SCN with a depth of Given the SCN's last layer weights wy, the bias functions bi,... bis and its network policy P, the following algorithm shows that knowing the last hidden nodes, their function values, and an array indicating the order of vertices that was removed during the forward pass to locate æ, is enough to extract the previous function values, hidden nodes, and also the weights. For simplicity of the algorithm, uo...., U is used as indicators for vp. ..., h1 hy with the same order.
Algorithm 4 Retrieving preceding layers and parameters for the backward pass of SCN
given d u44d-1,--- U1+1, o f( f()+1)) and convex weights w, of representing the input as the convex combination of dd4d d14-d-1, ................................ Ut array [a., a|-d] a] of integers storing the indices u, were removed during the forward pass with the exact order.
j =
repeat
Mt-j (4[-)) (x Ed w (luta-du44a
Extract Wi-j as the convex combination weights of representing ui+d-j using
ui-ddj-1;.. U1-j Extract Wi - (4-a-s-1) (40 S sis d) (element of wi-j that is assigned to u--d-j-i) using the
network policy P. and as-d-j-
btemp bi (1-) .... h(+d-j)
[(b1-))
11 (1140-3) (ʃ (w) btemp" - >E wi-
(hie--ss)) ((h--s-
Update
Wr to the convex combination
weights
x
represented
using
u-+d-j- j-11j-11- 1..... u-j; u1-j-1
j+
until d-1
F EXPERIMENTAL DETAILS
We provide the details of the experiments in the main text as well as results a binary classification experiment MNIST as primary proof of concept for practical usages of SCNs.
In all the three toy experiments, the neural network consisted two fully connected layers with 300 and hidden neurons respectively. ReLU activations depth of 4. Accordingly, the network had bias functions which all were constant parameters. A learning rate of 0.001, mini-batches of size 100 were used. The network policy was random. Thereby, preceding nodes were assigned randomly to the next convex combination weights (recall the fact that one dimensional case, SCNs are universal approximators even with random policy and fix convex combination weights. Thereby the optimization can be done through their biases only). In the sum of Gaussians experiment, inputs lied in a 2-simplex with vertices (0, 0), (0. 1). and (1, 0) as the way that SCNs are presented throughout the paper. Similarly, in the one dimensional case inputs lied in the interval (1 (1-simplex) [0, 1).
used for both layers. The SCN model had
To provide observation that SCNs can be used in practice, we did a primary experiment on the MNIST data set. The aim was the binary classification of zeros and ones. We lessened the dimensionality of the data 20 using PCA. We compared SCN simple logistic regression, and neural network with the same architecture used our synthetic experiments. Logistic regression can be seen SCN without any hidden nodes followed by Sigmoid. We used single unit SCN where its bias function was again a single constant parameter. Models were trained using the binary cross entropy loss and same learning rate and mini-batch size were used in training the models. The
13

---

Under review as a conference paper at ICLR 2020
performance is shown in the Figure and the test accuracy and the number of parameters shown in the Table [ Although the perfonmance of the SCN with a single hidden unit not as good as the neural network with fully connected layers, SCN could improve the accuracy of logistic regression by around 7% with adding a single hidden node to its architecture. Scaling up the SCN architecture to higher dimensional spaces is considered continuation of the paper.
80-
accuracy
SCN (1 hidden un tl
2-Layer Fully-Connected Logistic Regression
epoch
Figure 8: Accuracy of the models on the binary classification of zeros and ones in MNIST.
Table 1: Binary classification on MNIST
Model
# of parameters
accuracy
2-layer Fully Connected NN
6300
99.61 0.16
SCN (1 hidden unit)
41
89.00 + 1.01
Logistic Regression
21
82.11 2.85
14


Reviews:
[{"review_id": "S1lAcZpKFH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "1. Summary of the paper\n\nThis paper introduces *simplicial complex networks*, a new class of\nneural networks based on the idea of the subdivision of a simplicial\ncomplex. Using the simplicial approximation, a classical theorem in\nalgebraic topology, the paper demonstrates that the network is capable\nof learning a suitable function approximation that in turn can be used\nfor regression tasks.\n\n2. Summary of the review\n\nThis paper brings an interesting perspective to the table, viz. the use\nof concepts from algebraic topology to develop new neural network\narchitectures with improved approximation capabilities. I am\nappreciative and excited of any work in this direction, yet I deem the\ncurrent version of this paper not acceptable for publication because of\nthe following issues:\n\n- The paper suffers from a lack of clarity that makes assessing the\n  results and the proposed architecture harder than it needs to be.\n  In particular, numerous aspects such as scalability or the\n  generalization capabilities are glossed over. Even though I am\n  familiar with the underlying concepts and techniques, I found the\n  exposition not always easy to follow.\n\n- The experiments in the paper appear to be highly preliminary and need\n  more refinement. I understand that it is tough to develop a new\n  technique and 'compete'  against so many already existing ones, but\n  nevertheless I would suggest to at least extend one of the experiments\n  to a larger-scale comparison. Personally, I found the function\n  approximation experiment to be very interesting, so to me, this would\n  be a natural starting point for a more in-depth comparison.\n\nAgain, I want to stress that I think this paper has the potential to\nbecome a strong contribution but for this to happen, it requires an\nadditional revision cycle. I will outline some suggestions below.\n\n3. Clarity & technical correctness\n\nIt is my understanding that the paper is technically correct on all\nlevels. At times, imprecise language is used that might mislead or\nconfuse readers.\n\nMy overall recommendation would be to start the paper with a brief\nintuition of _why_ BCS is required and how to make use of the\napproximation theorem. After this, I would propose a high-level overview\nof the new architecture and its individual components. Only _then_ would\nI suggest describing each of these components in more detail.\n\nHere are some detailed suggestions:\n\n- The flow of the abstract could be improved; it is not clear why TDA\n  methods are suitable here.\n\n- Topological data analysis is not directly a 'geometric approach' for\n  data analysis. It incorporates specific attributes of geometry by\n  means of filtrations, for example, to 'escape' the coarseness of\n  unweighted simplicial complexes\n\n- TDA is also not _just_ restricted to point clouds\n\n- I find the difference between 'deterministic' and 'statistical'\n  methods incorrect in the introduction; the kernel-based approaches\n  that were developed to compare persistence diagrams, for example, are\n  _not_ the opposite of 'deterministic'. I would rather make\n  a distinction between 'distance-based methods' that are hard to\n  compute, and 'kernel-based methods' with improved scalability.\n\n- When analysing point clouds, TDA does not use 'an inducing' distance;\n  it can make use of the induced distance of a graph, for example, but\n  typically, the distances are pre-defined and evaluated between all\n  pairs of points.\n\n- I find the statement that hidden units are conceptually well-defined\n  only in SCNs to be highly problematic; I fully agree that\n  interpretability of architectures is relevant, but another expression\n  should be used here---FCN architectures (or in fact, all other\n  architectures in deep learning) are perfectly well-defined on the\n  conceptual level. Maybe the paper should be more clear about what it\n  expects the 'role' of a hidden unit to be; in fact, this is also not\n  discussed satisfactorily at present.\n\n- Introduce the concept of a *simplex* briefly before discussing\n  simplicial complexes.\n\n- Figure 1 cannot be understood in the context in which it is placed\n  currently; either move it back or create another figure that is more\n  high-level and explains the concepts\n\n- The requirement that all data points are within the standard\n  probability simplex seems vacuous to me; this can always be achieved\n  by scaling. Am I misunderstanding this? I would rewrite this and make\n  it clear that this is _not_ a limiting assumption.\n\n- Definition 2 is lacking formality; please add a brief description of\n  the function itself\n\n- Definition 3 is could use some more intuition; I understand that it\n  is technically correct, but a brief illustration of the iterative\n  nature of subdivision would be helpful.\n\n- After introducing the simplicial approximation theorem, please give\n  a brief connection on its relevance to the problem at hand.\n\n- The brief introduction of the Vietoris--Rips complex is relatively\n  hard to understand; please rephrase it. It is the complex in which\n  a set of points is a simplex if and only if the pairwise distances\n  of the points are less than or equal to the pre-defined threshold.\n\n- Section 2.3 is highly confusing at first because it requires readers\n  to understand Algorithm 1; please rephrase the introduction to this\n  section to make it clear what the _purpose_ is. To my understanding,\n  you want to use the BCS in order to obtain a situation as described\n  by the approximation theorem---but at this point, the utility of the\n  subdivision is not clear yet.\n\n- What does it mean that weights are 'specified to the BCS'?\n\n- When discussing the 'data-driven' approach, please add a motivation\n  for this. I understand that such a subdivision is required in order\n  to be computationally feasible, but this needs to be motivated.\n\n- Lemma 1 appears to be a direct consequence of the definition of\n  simplicial subdivision. I would suggest moving its proof to the\n  appendix.\n\n- Please give an improved explanation and intuition of the parameters of\n  the subdivision and why they require updating. It took me a while to\n  understand the relevance of the weights and what is actually updated\n  in the algorithm.\n\n- When talking about SCNs in Section 2.4, the salient points (learning\n  a subdivision with trainable vertex function values etc.) need to be\n  mentioned well before this section. When reading the paper for the\n  first time, I started understanding the method better while reading\n  the section, so I think that this intuition should become as early as\n  possible.\n\n- Please add intuition why _finding_ the position of a point is\n  required; I understand that this is necessary in order to obtain\n  weights and neighbours, from which to obtain a mapping, but at first,\n  this 'location problem' struck me as artificial.\n\n- When talking about the 'network policy', more details are needed. How\n  relevant is this choice for the network?\n\n- When discussing related work, please look at the follow-up\n  publication from Hofer et al., which discuss more architectures and\n  use cases:\n\n      Hofer et al., Connectivity-Optimized Representation Learning via Persistent Homology\n      ICML 2019\n\n- 'Topological geometry' on p. 7 is an unfamiliar term to me. What is\n  meant by this?\n\n- What about the computational complexity? This is somewhat glossed over\n  in the description of the BCS algorithm. Is this currently\n  a bottleneck for the deploying the architecture?\n\n- How easy would it be to extend this to _classification_ problems? The\n  title suggests that the architecture is highly generic, yet the paper\n  only discusses regression tasks. At the very least, this should be\n  briefly discussed.\n\n4. Experiments\n\n- The experiments are lacking a brief explanation: what are they\n  supposed to show or analyse? I find the function approximation\n  example to be very interesting, but somewhat preliminary insofar\n  they only compare *two* functions. Why not generate sequences or\n  families of functions and make this into a large-scale comparison?\n\n- I found the discussion of storage to be somewhat surprising; is this\n  a property that the paper wants to stress for SCNs? If so, more\n  experiments in that direction are required.\n\n- The observations on bias function choice are not reflected in the\n  experiments that are presented in the main text. I understand that it\n  works for the toy examples, but a more in-depth comparison is required\n  for this sort of experiment to be compelling.\n\n- Is it possible to _tune_ the number of subdivisions or provide an\n  upper bound? It would be interesting to see how such a bound might\n  change the results---this is just a suggestion, but I feel that it\n  might be interesting to discuss how much performance or approximation\n  qualities are gained by _not_ restricting the BCS algorithm.\n\n5. Minor style comments\n\nThe presentation quality of the paper could be improved. Here are some\nsuggestions.\n\n- Please use `\\citep` when citing works in the text; currently, citations are intermingled with sentences and make it harder to read the paper.\n- 'Universal approximation property' --> 'The universal approximation property'\n- 'with similar property' --> 'with similar properties'\n- 'alternates the common architecture' --> 'changes the common architecture'\n- the citation 'Bernardo et al.' should be fixed; it is missing information\n- I suggest removing the sentences 'The rest of this paper is organized as follows'...\n- 'Let f denotes a mapping' --> 'Let f denote a mapping'\n- 'As theorem 2' --> 'As theorem 1'\n- 'we can subdivision' --> 'we can subdivide'\n- 'a same' --> 'the same'\n- 'that how the position' --> 'how the position'\n- 'used to fed' --> 'used to feed'\n- 'the appendix F' --> 'appendix F'\n- 'may outputs' --> 'may output'\n- 'this phenomena' --> 'this phenomenon'\n- 'bias function observed to be' -->' bias function was observed to be'\n' 'can be continued' --> 'can be extended'", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "BkgPbIw2tB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a new neural network architecture called simplicial complex networks. The authors draw inspiration from topological data analysis in order to motivate this new architecture. The SCN architecture is motivated through a parametric sub-divisioning of the input space into a complex of d-dimensional simplices. The authors give a universal approximation theorem for a model of this type, and give a simple algorithm to perform the sub-divisioning. The resulting network output is a convex combination of vertices after l nested sub-divisions. The authors present a number of toy experiments to validate their claims.\n\nAlthough I believe the authors present an intriguing idea, I ultimately tend for a rejection of this paper. I have three major reasons for this: \n(1) Quality of writing and clarity is lacking: Citations are all done in-line, symbols are being used without proper explanation, and many sentences are just purely broken and impossible to understand. \n(2) The authors do not properly compare their approach to the most common architectures in use: Fully-connected nets and CNNs. What are the shortcomings and benefits of SCNs as compared to these architectures? \n(3) Usefulness of the approach. I believe this ties in with point (2). The authors make no explicit mention of the computational complexity of their approach, or how the number of parameters depends on the number of input dimensions. Since the simplicial complex is defined on the input space the number of parameters scales with the number of input dimensions, and becomes restrictively high. This explains why the author use only low-dimensional toy examples.", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The aper introduces simplicial complex networks, a new class of
neural networks based on the idea of the subdivision of a simplicial
complex. The paper is interesting and brings ideas of algebraic topology to inform the design of new neural network architectures. 

Reviewer 1 was positive about the ideas of this paper, but had several concerns about clarity, scalablity and the sense that the paper might still be in an early phase. Reviewer 2 had similar concerns about clarity, comparisons, and usefulness. Although there were no responses form the author, the discussion explored the paper further, but continued to think the idea is still in its early phase.

The paper is not currently ready for acceptance, and we hope the authors will find useful feedback for their ongoing reasearch. 
Decision: Reject