Simplicial Representation Learning with Neural $k$-Forms
Authors: Kelly Maggs
Celia Hacker
Bastian Rieck
Venue: ICLR 2024 poster (2024)
Keywords: geometric deep learning
differential forms
representation learning
graph learning
geometry
topology
Primary area: learning on graphs and other geometries & topologies
TL;DR: We learn differential $k$-forms on embedded graphs, leveraging a connection to singular cochains to obtain efficient, interpretable representations.

Abstract: Geometric deep learning extends deep learning to incorporate information about the geometry and topology data, especially in complex domains like graphs. Despite the popularity of message passing in this field, it has limitations such as the need for graph rewiring, ambiguity in interpreting data, and over-smoothing. In this paper, we take a different approach, focusing on leveraging geometric information from simplicial complexes embedded in $\mathbb{R}^n$ using node coordinates. We use differential $k$-forms in $\mathbb{R}^n$ to create representations of simplices, offering interpretability and geometric consistency without message passing. This approach also enables us to apply differential geometry tools and achieve universal approximation. Our method is efficient, versatile, and applicable to various input complexes, including graphs, simplicial complexes, and cell complexes. It outperforms existing message passing neural networks in harnessing information from geometrical graphs with node features serving as coordinates.

Published as a conference paper at ICLR 2024
SIMPLICIAL REPRESENTATION LEARNING WITH NEURAL K-FORMS
Kelly Maggs', Celia Hacker?, Bastian Rieck 3,4
1 'ccoe Polytechnique Fédérale de Lausanne (EPFL.
Max Planck Institute for Mathematics in the Sciences 3 "AIDOS Lab, Institute of AI for Health, Helmholtz Munich *Technical University of Munich (TUM)
ABSTRACT
Geometric deep learning extends deep learning to incorporate information about the geometry and topology data, especially in complex domains like graphs. Despite the popularity of message passing in this field, it has limitations such as the need for graph rewiring, ambiguity in interpreting data, and over-smoothing. In this paper, we take different approach, focusing on leveraging geometric information from simplicial complexes embedded in IR" using node coordinates. We use differential k-forms in IR" to create representations of simplices, offering interpretability and geometric consistency without message passing. This approach also enables us apply differential geometry tools and achieve universal approximation. Our method is efficient, versatile, and applicable to various input complexes, including graphs, simplicial complexes, and cell complexes. It outperforms existing message passing neural networks in harnessing information from geometrical graphs with node features serving as coordinates.
INTRODUCTION
Geometric deep learning (Bronstein et al., 2017) expanded the scope of deep learning methods to include information about the geometry-and, to a lesser extent, topology of data, thus enabling their use in more complicated and richer domains like graphs. While the recent years have seen the development of a plethora of methods, the predominant paradigm of the field remains message passing (Veličković, 2023), which was recently extended handle higher-order domains, including simplicial complexes (Ebli et al., 2020), cell complexes (Hajij et al., 2020), and hypergraphs (Heydari Livi, 2022). However, despite its utility, the message passing paradigm suffers from inherent limitations like swor-shoo,,,ng, over-squashing, and inability capture long-range dependencies. These limitations often require strategies like graph rewiring, which change the underlying graph structure (Gasteiger et al., 2019; Topping al., 2022) and thus affect generalisation performance.
Our paper pursues a completely different path and strives to leverage additional geometric information from data set to obtain robust and interpretable representations of the input data. Specifically, we consider input data in the form of simplicial complexes embedded in R" via node coordinates. This type of complex can be built from any graph with node features, with node features acting as the coordinates, for example. Our key insight is the use of differential k-forms in R". A k-form in I can be integrated over any k-simplex embedded in E= to produce real number. Thus an f-tuple of k-formm produces an é-dimensional representation of the simplex independently of any message passing. From this perspective, k-forms play the role of globally consistent feature maps over the space of embedded k-simplices, possessing the geometric semantics and interpretability of integration. This enables us to use tools from differential geometry to prove a version of universal approximation, as well as number of other theoretical results. Moreover, the structure of differential forms in IR" makes learning algorithms (computationally) feasible. In particular, a multi-layer perceptron with the right dimensions induces k-form on IR" that be integrated. This implies that learnable, mnitely-parametriseddd differential forms can be woven into existing machine learning libraries and applied to common tasks in geometric deep learning. We find that method is better capable

---

Published as a conference paper at ICLR 2024
of harnessing information from geometrical graphs than existing message passing neural networks. Next to being efficient, our method is also generally applicable to a wide variety of imput complexes, including graphs, simplicial complexes, and cell complexes.'
In a nutshell: We consider DATA in the form of simplicial chains on embedded simplicial complexes, defining learnable differential k-forms FEATURE MAPS, and introduce the concept of an integration matrix, which serves as an overall REPRESENTATION of the data.
Organisation of Paper. We present the relevant background for simplicial complexes and differen- tial forms in Section 2. In Section 3, we introduce neural k-forms and prove universal approximation statement. We also show how neural k-forms induce a so-called integration matrix, and use the prop- erties of integration to prove a number of propositions. In Section we present the basic architecture and algorithms. Finally, in Section 5, we perform several intuitive experiments and benchmark our method on standard geometrical deep learning data sets.
Related Work. Several methods focus on generalising graph neural networks (GNNs) to higher- dimensional domains, proposing simplicial neural networks (Bodnar et al., 2021b; Bunch et al., 2020; Ebli et al., 2020; Giusti et al., 2022; Goh et al., 2022; Keros et al., 2022; Roddenberry et al., 2021), methods that can leverage higher-order topological features of data (Hajij et al., 2023; Hensel et al., 2021; Horn et al., 2022), optimisation algorithms for learning representations of simplicial com- plexes (Hacker, 2020). All of these methods operate on simplicial complexes via diffusion processes or message passing algorithms. Some works also extend message passing or aggregation schemes to cell complexes (Bodnar et al., 2021a; Hajij et al., 2020; 2023) or cellular sheaves (Barbero et al., 2022; Hansen & Gebhart, 2019). However, these existing methods exhibit limitations arising from the use of message passing or aggregation along a combinatorial structure. Message passing often results in over-smoothing (a regression to the mean for all node features, making them indistinguishable) or over-squashing (an inability to transport long-range node information throughout the graph), necessitating additional interventions (Gasteiger et al., 2019; Nguyen et al., 2023; Topping et al., 2022). Hence, there is need for methods that go beyond message passing. Our work provides novel perspective based on the integration of learnable k-forms on combinatorial domains like graphs or simplicial complexes embedded in R", L.c. assume the existence of (vertex) coordinates.
2 BACKGROUND
This section introduces the required background of simplicial complexes and differential forms. We restrict our focus simplices and differential forms in RR" given this is the only setting we will use in practice to make the theory more accessible. For additional background references, we recommend Nanda (2021) for computational topology and Lee (2003) or Tao (2009) for differential forms. More details are also provided in Appendix A.
Abstract Simplicial Complexes. An abstract simplicial complex S is generalisation of graph on set of vertices S In a graph, we have pairwise connections between nodes given by edges, or 1-dimensional simplices (denoted by S²). In simplicial complexes, there are higher-dimensional counterparts of these connections, called the k-dimensional simplices, or just k-simplices. The k-simplices are connections between + vertices of the set S with 2-simplex forming a triangle, a 3-simplex forming a tetrahedron, and so on. We denote k-simplex by [vo,... ... Uk], where is E 8", writing S to refer to the set of all k-simplices. If a simplex a S, then so are all of its faces, i.e. all simplices formed by subsets of the vertices of o.
Affine Embeddings. Data in geometric deep learning most often comes as a geometric object- a graph simplicial compleex--mmbieedd with node features in R". Formally, node features correspond to a node embedding of a simplicial complex S, which is a map o: 5 - R". The standard geometric k-simplex is the convex hull A& [0, f1, ... tx] C R where t, is the endpoint of the i-th basis vector. For each k-simplex a = [Do..... 1x]] the map & induces an affine embedding dz: A + R whose image is the convex hull (ó(vo), .. (vx))]
For example, by taking barycentric subdivision, integration of forms over cell complexes is recoverable by integration over simplicial complexes.
2

---

Published as a conference paper at ICLR 2024
Chains and Cochains over R. Cx(S; R) are the vector space
For oriented simplicial complex S, the simplicial k-chains
[ Σ Noo 10 oo, E AER
C& (s;
(I)
of formal linear combinations of k-simplices in S. The simplicial k-cochains (S; over IR are the dual space Hom(C& Hom(C&(S)):R) R) of linear functionals over the simplicial k-chains.
Differential Forms. The tangent space (Ik") p is the space of all vectors originating point p E IR, and its elements are called tangent vectors. In our case, this is space is canonically isomorphic the underlying space R= A differential form is a function that assigns notion of volume to tuples of tangent vectors at each point in R Given a tuple of k standard basis vectors ( Css)) in R" indexed by (f1, i2, ......,,,,, ig) and vector Tp (RR", the vector of is the projection of onto the I-subspace spanned by (e,, ,,....., eig. The associated monomial k-form
d U2, 2k)) E! (U1, U2,.... 2k) == det [uf ui...,,,,
(2)
represents the standard volume spanned by & tangent vectors o (RR) in the L-subspace of the langent space.
Scaling Functions. General differential forms are built from locally linear combinations of re- scaled monomial k-forms. Formally, a general differential k-form E (RR" can be written as
"p(U1 U2 -2, mk) Σ ar (r()ddd (11, (,,
where E I= and E TRR))) and I ranges over the (*) subspaces spanned by sets of k basis vectors in R", The scaling functions ay: R" represent re-scaling of the standard volume in the T-subspace for each point p E IR"". Intuitively, the scaling functions in a differential k-form specify the size and orientation of subspace for each point.
Integration. Differential fororms can be integrated over embedded k-simplices o: A - R". Let De(() be the Jacobian matrix of Φ at E A For affinely-embedded simplices, De((1) is given by
DØs.j = (b(tt) o (0
(4)
The integral of w E (R*) over the image of & can be expressed as
[ |W ar(o(t))
(5)
where the integral is interpreted as a standard Riemann integral over the (compact) subset A C This represents the signed volume of the image of e with respect the differential form w.. In practice, this integral is approximated with finite Riemann sum. We provide more background on integration of forms Appendix A. The integral in Eq. (5) is well-defined & is C embedding function and a, ø is integrable.
RR
3 NEURAL A-FORMS AND INTEGRATION MATRICES
Broadly speaking, representation learning is the process turning data into vectors in Rd followed by the use of standard tools of machine learning to classify/predict attributes based on these vector representations. In graph learning and simplicial learning more generally, data comes in the form of simplicial complexes embedded in R.,. One simple vectorization is take the standard volume of a k-simplex embedded in R.. The central idea of our paper is to create vector representations of k-simplices as volumes relative to a tuple of differential k-forrm. Indeed, tuple of k-forms wi, W2, ... E Ω2 (R?)) determines a representation function
( W1 wT.....
[
(Φ:
ER
(6)
"The choice of orientation of cach simplex corresponds choice of sign for each basis vector.

---

Published as a conference paper at ICLR 2024
that vectorises any k-simplex embedded in R" by calculating its volume via integration of forms. In this paradigm, each k-form takes the role of a feature map on the space of embedded k-simplices.
Neural k-forms. The key to making integral-based representations learnable is model the scaling functions in Eq. (3) as a multi-layer perceptron (MLP). For a set of k basis vectors 1, let Ty: (e) + I. be the projection onto the I-subspace and define y == Ty sh: R" RR
Definition Let wh R ( be an MLP. The neural k-forrm w (R") associated to e the k-form w En dT
In words, the components of the MLP correspond to the scaling functions of the neural k-form. The goal of a neural k-form is thus to learn the size and orientation of k-dimensional subspaces at each point in R" according some downstream learning task. In case the activation function is a sigmoid function or tanh, Definition I produces smooth k-forms, whereas for ReLU activation function, one obtains piecewise linear k-forms.
Remark 2. A neural O-formm over R with l features is an MLP db from IR to RV The O-simplices p E IR are points in R", and integration of a 0-form corresponds the evaluation (p)). In this way, neural fororms are a direct extension of MLPs to higher-dimensional simplices in R".
Universal Approximation. We would like know which k-forms on R are possible to ap- proximate with neural k-forms. The following proposition translates the well-known Universal Approximation Theorem (Cybenko, 1989; Hornik et al., 1989) for neural networks into the language of neural k-forms. Here, the norm ll (lo.(R.)) on k-forms is induced by the standard Riemannian structure on RR, explained in Appendix C.
Theorem 3. Let a E C(R, I.) be a non-polynomial activation function. For every n E N and compactly supported k-form 7 E (RR" and >> O there exists neural k-forrm w with one hidden layer such that ||W - (R.) <
Cochain Matrices. Most simplicial neural networks follow a similar procedure. The key step is that the k-simplices of each simplicial complex S are replaced by a matrix Xs(3, 7) containing selection of l simplicial cochains (71.... .e) as columns. This can be thought of as a matrix containing evaluations b (3.) with respect to some basis (Bi..... 3.) for the sim- plicial k-chains (s; R). However, there is no canonical initialisation when one starts with only the data set of embedded finite simplicial complexes." We address this issue by introduc- ing the integration matrix induced by a neural k-form. This produces the same data type, but has the advantage that the feature cochains cor- respond to integration of the same form defined over the ambient space.
(Beses) ws
+
0
Figure An integration matrix with data in di- mension 1, where embedded oriented 1-simplices correspond to paths and 1-forms are canonically identified with vector fields (see Appendix A for details). Integration of 1-form against a path cor- responds to path integration against the respective vector field. Thus, integration of the paired paths and 1-forms in the left matrix recovers real values with the signs given in the right matrix.
Integration Matrices. For embedded finite simplicial complex, f-tuple (W1, W2 we) of k--frrms induces a matrix suitable to simplicial learning algorithms in a natural way via integration. Let Φ: S Ra be affine embedding of simplicial complex and (31, B2 Bm) C (S; RR) be a set of specified k-chains.
Definition 4. The integration matrix with respect and (B( [Se, wyles'
The integral of w over a simplicial chain B = Noo E (S; with respect to the embedding Φ: S Rd corresponds to the integral
Σ> w.
Indeed, if one takes random initialisation, the feature cochains do not have shared interpretable meaning across different complexes.
4

---

Published as a conference paper at ICLR 2024
Remark 5 (Interpretability). The key point is that the integration matrices of two different simplicial complexes Φ: S - IR" and : S' + RR" embedded in I" have shared interpretation. That is, the j-th column of both of their integration matrices corresponds to the volume of k-simplices against the same feature k-form wij- E (2((TR))
tuple of neural networks vy I (e)) with associated neural k-forms w induces learnable matrix representation
(8, (3,
"
of the given simplicial chain data 3 with embedding o: S R.. This intermediate representation is finitely parametrised by & and can thus be updated via backpropagation. The matrix representation of simplex-ane and by extension a simplicial chain-dependss on whether the neural k-form decides that it is embedded in large or small subspace, and with what orientation.
Basic Properties. There are number of useful basic properties about integration matrices that follow from the well-known properties of integration. the next proposition, we conceptualise
B (81 ... Bm)² E M (C)(S:R) and (wi,..., we) E M ( (R"))
chain-valued column and k-form valued row vectors, respectively. Real-valued matrices both vectors by scalar multiplication and linearity. Proposition (Multi-linearity). Let Φ) S R" be an embedded simplicial complex. For any matrices Mm xm (R) and R E M (IR), we have
(X,(B,
(10)
staple requirement of geometric deep learning architectures is that they should be permutation and orientation equivariant. our setting, these properties are direct corollary of Proposition 6.
Corollary 7 (Equivariance). Let 3 = (B1,..., Bm)) be a basis for the k-chains C& IR) of an embedded oriented simplicial complex Φ: S IR".
1. (Permutation) (2(P3.w) w) = PX (B, w) for all permutation matrices P E M x (R).
2. (Orientation) Xx(QB, w) QX (3,w(, for all signature matrices Q E _m (R).
4 ARCHITECTURE
Embedded Chain Data. The input data D (SS,, Oa, Pa)) to our learning pipeline consists of a set of triples (Sa; &a, Ba) of embedded chain data, where
Sa is a simplicial complex
da: S R is affine embedding and
3n E ① m. Ca(S; IR) is tuple of m. data k-chains on S.,.
If no chains provided, complex as the input chains. The canonical example a dataset consisting of graphs {(Ga Pa. 8a)) with node features do, S% RR" and B, corresponding to the standard edge chains with arbitrary orientations.
can take the standard basis of oriented k-simplices of each simplicial
Approximating Integration Matrices. The main departure from standard geometric deep learning architectures is the transformation from embedded k-chain data to integration matrices. The high- level structure of this process is presented in Algorithm 1. Given a tuple of neural k-forms, each
represented by an MLP dly: IR" + (t)) integrals of embedded k-simplices oe: A + R are calculated by a finite approximation (Appendix B) of the integral formula
| wi -
E/x, Via( ( (Dos (1)) dt VolApprox( wi)
(11)
*A signature matrix is diagonal matrix with +1 entries.
5

---

Published as a conference paper at ICLR 2024
FEATURES: Neural k-form t): R" + (2(2)x
Xp(Bww) [[p. wil
Integration matrix
Readout
Task-specific network
DATA: Embedded simplicial complex
Figure 2: A schematic of our proposed neural k-form learning architecture.
Algorithm 1 Generate Integration Matrix
Inputs: Embedded chain data (S, d. 8) with k-simplices a E S%; k-chains 3, Σ A (R) i I...., m; MLPS bb:: R - R (i) = =1, I.....f.
X = 0 E M (Ik)
b Initialise X as the 0-matrix with x elements D Iterate over forms >> Iterate over chains
for S < l do
for < i m do
Xij < Vo
Return: Xo(Bww")
Return Integration Matrix
appearing in Eq. (5). Integrals of chains 3; are caleulated linear combinations. The entries of the integration matrix (3(3, w thus depend in a differentiable manner on the component functions y of the underlying MLP.
Readout Layers. Once an embedded simplicial complex is transformed into an integration matrix, is then fed into readout layer (as in the case for standard simplicial or graph neural networks). The output of readout layer is a single representation of the entire complex, and should not depend on the number of simplices one wishes to compare representations among different complexes. Common read-out layers include summing column entries, and Li L2 norms of the columns. Note that only the latter two are invariant under change orientation.
Neural k-form Backpropagation. For fixed dataset of embedded k-chains, neural k-forms the learnalle feature functions. Algorithm 2 and Figure 2 illustrate the basic pipeline for performing backpropagation of neural k-forms over embedded chain data and loss function. Neural k-forms and user-determined classifier are randomly initialised using any standard MLP initialisation. Each embedded chain data (Sa, Φa. (a)) is transformed into an integration matrix by Algorithm 1, before being read out, classified, and evaluated against a loss function.
Implementation. Our methods can be realised using any deep learning framework that permits training MLP. We created proof-of-concept implementation using PyTorch wwTTorch-Geomerre (Fey Lenssen, 2019) and PyTorch- -warrch-Lighhtiing (Falcon & The PyTorch Lightning team, 2019) and make it publicly available under https: peatts://qitaan com. aidos lab -
EXPERIMENTS AND EXAMPLES
This section presents examples and use cases of neural k-forms classification tasks, highlighting their interpretability and computational efficiency.

---

Published as a conference paper at ICLR 2024
Algorithm 2 Neural f-form Backpropagation
Data: {(Sa: Pas Ba) Oa S. - RR", Br + ma Cx(S; ma E NN
Initialise MI.Ps why: R + RR);; S j < E; classifier ry: Re + R.C.
for (Sa, Oar 3a) E D do
Xs. (B,, + IntegrationMatrixxx.... Bar
X + Readout(Xa, w))
Vector Representation A> Prediction
X + (X))
(X)
Backward(L,V,, 177)
> Update k-forms and Classifier
(a) Learned 1-forms corresponding to paths in each class.
(b) Path representations.
Figure 3: Synthetic Path Classification via Learnable 1-forms. The 1-form (a vector field in this case)
adjusts itself to the data, resulting in distinct path representations.
5.1 SYNTHETIC PATH CLASSIFICATION
Our first experiment classifying paths in R², The goal of the experiment is pedagogical; illustrates how to interpret the learned 1-forms rather than benchmark the method. A piecewise linear path in I a simplicial complex determined by an ordered sequence of node embeddings. The I-simplices are linear maps o; I + R" from the i-th embedded node and to the (i+ 1)-st embedded node, where the full path corresponds to the 1-chain E; dg The integral of this chain against a 1-form corresponds to the path integral. Figure 3a shows three classes of piecewise linear paths in IR22 that we will classify using method. The idea is that will learn three 1-forms, which correspond to each class. To initialise the three 1-forms in IR2, we randomly initialise an MLP Φ IR2 + [ A forward pass consists of two stages: integration of the 1-simplices in the path against each of the three forms to produce an integration matrix, followed by taking column sum and applying sof tmax to produce a distribution over which we perform Cros entt ropyLoss........ The i-th column sum corresponds to path integral of the path against the i-th form: the prediction of path is thus determined by which feature 1-form produces the highest path integral. Backpropagation against this loss function thus attempts to modify the i-th 1-form so that it produces more positive path integral against the paths in class and more negative otherwise.
Figure 3a also shows the feature 1-forms as vector fields over their corresponding classes of paths. Note that the vector fields are randomly initialised and updated while the paths are the fixed data points. The learned 1-forms of each class resemble vector fields that roughly reproduce the paths in their class as integral flow lines, and are locally orthogonal or negatively correlated to paths in the other classes. This is direct result of the objective function, which attempts maximise the path integral within each class and minimise it for others. Figure 3b depicts the paths as points coloured by class, where the coordinates correspond to the path integrals against the three learned 1-forms. We observe a clear separation between the classes, indicating that the representations trivialise the downstream classification task. We also compare the model with standard MLP in Appendix E.
TThe orientation is indicated by the arrow the end of each path.
7

---

Published as a conference paper at ICLR 2024
Table 1: Results (mean accuracy and standard deviation of a 5-fold cross- valiiatoit) on small graph benchmark datasets that exhibit "geometrical" node features. Parameter numbers are approximate because the number of classes differ.
Params.
BZR
COX2
DHFR
Letter- low
Letter-med
Letter-high
EGNN (Satorras et al., 2021)
1M
79.51 + 1.87
78.16
0.46
64.02 2.68
93.20 0.68
65.91 1.60
68.98 1.80
GAT (Veličković, 2022)
5K
81.48
+ 2.90
80.73
a 2.45
73.02 + 2.54
90.04 2.23
63.69 4. 5.997
43.73 a: 4.13
GCN (Kipf & Welling, 2017)
5K
79.75
0.68
79.88 1.65
70.12 5.43
1.57 81.38
62.00
43.06 1.67
GIN (Xu et al., 2019)
9K
79.26
1,03
78.38
#.79 0.79
68.52 + 7.38
85.00
67.07
2.47
50.93 3,47
N&F (ours)
4K
78.77 0.55
80.30 a 2.42
64.42 a 2.09
93.42 a 1.94
67.69 + 1.28
62.93 a 4.13
5.2 SYNTHETIC SURFACE CLASSIFICATION
This example demonstrates how our framework is of interest to deal with higher-dimensional data, i.e. simplicial complexes of dimension k 2 2. Conceptually, the difficulty is going from 1-dimensional objects to k-dimensional objects with any k 2 2. We restrict ourselves to k 2 since higher dimensions are similar to this case. The data we consider here are triangulated surfaces embedded in R3, the underlying combinatorial complex is always the same- triangulation of square-bu but the embeddings are different. For a given surface, the embedding of each 2-simplex is given by linear interpolation between the coordinates of its three vertices in IR We consider two classes of surfaces, in the first class the embeddings of the nodes are obtained by sampling along sinusoidal surface in the z-direction, while the second class is given by a surface following a sinusoidal shape in the g-direction, in each case with added translation and noise. We use learnable 2- 2-forms W1. W2 on R.3 represented by an MI.P tb: IR3 - R3x22 In a forward pass of the model, we integrate the 2-forms given by the MLP over the 2-simplices of surface. Each point in R33 has a value in R2322 given by the MLP evaluated at that point. The process of integrating over the 2-simplices corresponds to integrating the point-wise evaluation of the MLP over the regions of g defined by each embedded 2- 2-simplex. This process gives the integration matrix of the forms W1, W2 over the 2-simplices of the complex. Learned representations the surfaces
The next step is the readout layer, which sums the entries in each column of the integration matrix, corresponding to summing the value of each cochain over the simplicial complex, giving the total surface integral. As there two 2-forms this yields a vector representation in IR? each entry corresponding to 2-form. Finally, these representations are then passed through a so ftmax for classification and CrossEnt ropy Less is used as a loss function. The MLP is then updated by backpropa- gation. Figure 4 plots the representations in IR2 learned through the model described above. Each point represents a surface in the data set and the colour is given by the class of the corre- sponding surface, showing the clear separation learned by the neural 2-forms. Further details can be found in Appendix E.2.
Figure 4: Synthetic surface classifi- cation via learnable 2-forms.
5.3 REAL-WORLD GRAPHI
In this experiment we attempt to use our model to leverage the geometry of non-equivariant node features for graph classification on a set of benchmark datasets. The basic architecture of the model follows that described in the Architecture and Parameters section. Graphs are represented as 1-chain consisting of all their edges. We randomly initialise a set of l feature 1-forms. produce and read-out the integration matrices before feeding through a simple MLP classifier and performing backpropagation. We use an L2-column readout layer so the network is invariant under edge orientations. Please refer to Appendix E.3 for specific architectural details. We use mfte-oofthhe-ar graph neural networks (GNNs) benchmark (Dwivedi et al., 2023), experimenting with different
following a recently-descries numbers of layers. As an additional comparison partner, we also use a recent equivariant GNN architecture that is specifically geared towards analysing data with an underlying geometry (Satorras et al., 2021). Table depicts the results on smaller graph datasets in the TU dataset (Morris et al., 2020). Here the node features that carry both equivariant information (corresponding to 3D coordinates of molecules, for instance) and non-equivariant information (one-hot atomic type, weight,
8

---

Published as a conference paper at ICLR 2024
Table 2: Results (mean AUROC and standard deviation of 5 runs) on benchmark datasets from the 'MoleculeNet' database (Wu et al., 2018). While the GNNs also train for smaller numbers of parameters, we observed significant drops in predictive performance. We thus report only the best results for GNNs, using the most common models described in the literature.
Params.
BACE
BBBP
HIV
GAT (Veličković et al., 2018)
135K
69.52 + 17.52
76.51 = 3.36
56.38 4.41
GCN (Kipf & Welling, 2017)
133K
66.79 + 1,56
73.77 = 3.30
68.70
1.67
GIN (Xu et al., 2019)
282K
42.91 + 18.56
61.66 = 19.47
55.28 + 17.49
N&F (ours)
9K
83.50 + 0.55
86.41 = 3.64
76.70 = 2.17
etc.). For the non-equivariant models (ours, GCN, GAT) the position features omitted. Overall, our method exhibits competitive performance, in particular given the fact that it does not make use of any message passing and has smaller parameter footprint. In Table 2 we compare our model on the larger datasets in the 'MoleculetNet' benchmark (Wu et al., 2018). Note that the datasets we have chosen have provided "positional" node features, the given node features (i.e. atomic weights, valence, etc.) are not equivariant and cannot be compared with EGNN. As Table 2 shows, our model based on neural k-forms outperforms state-of-the-art graph neural networks in terms of AUROC, using fraction of the number of parameters (this also holds for accuracy and average precision, which are, however, not typically reported for these data sets).
DISCUSSION
Summary. We developed neural k-forms, a method for learning representations of simplicial cochains solve tasks embedded graphs and simplicial complexes. Deviating from the predom- inant paradigms in geometric deep learning, our method adopts a fundamentally different novel perspective based on integration of forms in the ambient feature space. We have shown the feasibility of the method through comprehensive experimental suite, demonstrating its effectiveness using only a very small number of parameters. Notably, our method does not utilise any kind of message passing, and we hypothesise that it is possible that this implies that issues like over-smoothing may affect our method less than graph neural networks.
Limitations. A conceptual limitation is that we require (at least) the existence of node or vertex coordinates, i.e. our method only operates on embedded complexes. The computational feasibility of higher order k-forms in large embedding spaces is another possible limitation. Indeed, the number of monomial k-forms in R" is (i), and similar issues arise for numerical integration over higher- dimensional simplices. Further, we have only benchmarked our method on graph classification tasks. It remains to be seen whether the method performs as well on graph regression tasks, as well as for benchmark learning tasks on higher-dimensional simplicial complexes." 7 Finally, our model currently cannot deal with node features that are equivariant with respect to the embedding space.
Outlook. We aim to study these issues, as well as the behaviour of our methods in the context of long-range dependencies, in a follow-up work. In addition, since our neural k-form formulation is equivalent to an MLP, the learning process may benefit from the plethora of existing methods and tricks that are applied to optimise MLPs in practice. We argue that our experiments point towards the utility of using a geometric interpretation of our representations as integrals over k-forms may provide valuable insights to practitioners. Lastly, we provide a small example of a Convolutional 1-form network in Appendix D that may lead to better incorporation of equivariant node embeddings. We provide this auxiliary example as part of a broader future work program of rebuilding common ML architectures on top of neural k-forms rather than message passing schemes.
We note that comprehensive, agreed-upon framework for benchmarking simplicial neural networks has yet be established.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGEMENTS
B.R. is supported by the Bavarian state government with funds from the Hightech Agenda Bavaria. K.M. received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Cun the Leibniz Supercomputing Centre for funding this project by providing computing time on its Linux cluster. The authors also acknowledge Darrick Lee, Kathryn Hess, Julius von Rohrscheidr, Katharina Limbeck, and Alexandros Keros for providing useful feedback on early manuscript. They also wish to extend their thanks to the anonymous reviewers and the area chair, who believed in the merits of our work.
grant agreement No 859860. The authors gratefully acknowledge
REFERENCES
Federico Barbero, Cristian Bodnar, Haitz Sáez Ocáriz Borde, Michael Bronstein, Petar Veličković, and Pietro Liò. Sheaf Neural Networks with Connection Laplacians, 2022. URL https:
//arxiv. org/ abbb/206.... 08702.
Cristian Bodnar, Fabrizio Frasca, Nina Otter, Yu Guang Wang, Pietro Liò, Guido Montufar, and Michael M. Bronstein. Weisfeiler and Lehman go cellular: CW networks, In Advances in Neural Information Processing Systemx, 2021a.
Cristian Bodnar, Fabrizio Frasca, Yuguang Wang, Nina Otter, Guido F Montufar, Pietro Lió, and Michael Bronstein. Weisfeiler and Lehman go topological: Message passing simplicial networks. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 1026-1037. PMLR, 2021b.
Michael M. Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geometric deep learning: Going beyond Euclidean data. IEEE Signal Processing Magazine, 14(4))18-42 2017.
Eric Bunch, Qian You, Glenn Fung, and Vikas Singh. Simplicial 2-complex convolutional neural nets, 2020. URL
G. Cybenko. Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals, and Systems, 2(4):303-314 1989.
Vijay Prakash Dwivedi, Chaitanya K. Joshi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Benchmarking Graph Neural Networks. Journal of Machine Learning Research, 24(43):1-48, 2023.
Stefania Ebli, Michaël Defferrard, and Gard Spreemann. Simplicial Neural Networks. In Topological Data Analysis and Beyond Workshop NeurIPS, 2020. URL https: org abs/ 2010. 03633.
William Falcon and The PyTorch Lightning team. PyTorch Lightning, March 2019. URL https: //github. cww http:/Lgghtnnnngg AI/lightning
Matthias Fey and Jan Eric Lenssen. Fast Graph Representation Learning with PyTorch Geometric, May 2019. URL https: / / thub.
Johannes Gasteiger, Stefan WeiBenberger, and Stephan Günnemann. Diffusion improves graph learning. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32, Curran Associates, Inc., 2019.
L. Giusti, C. Battiloro, P. Di Lorenzo, S. Sardellitti, and S. Barbarossa. Simplicial Attention Neural Networks, 2022. URL https: aaaii.iii...///ii wow.ivoorrggaaae
Christopher Wei Jin Goh, Cristian Bodnar, and Pietro Liò. Simplicial Attention Networks, 2022. URL https: //axxi/rxvvv .orllvoorggga/e.. 22244 09455.
10

---

Published as a conference paper at ICLR 2024
Celia Hacker. k-simplex2vec: simplicial extension of node2vec. In Topological Data Analysis and Beyond Workshop at NeurIPS, 2020. URL
orq/ abs 2000.00000000 05636.
Mustafa Hajij, Kyle Istvan, and Ghada Zamzmi. Cell Complex Neural Networks. 2020. URL
wttporgg//////oooooo 00743.
Mustafa Hajij, Ghada Zamzmi, Theodore Papamarkou, Nina Miolane, Aldo Guzmán-Sáenz, Karthikeyan Natesan Ramamurthy, Tolga Birdal, Tamal K. Dey, Soham Mukherjee, Shreyas N. Samaga, Neal Livesay, Robin Walters, Paul Rosen, and Michael T. Schaub. Topological Deep Learn- ing: Going Beyond Graph Data, 2023. URL https: / aarxiv org/ abs /206 00606.
Jiaqi Han, Yu Rong, Tingyang Xu, and Wenbing Huang. Geometrically equivariant graph neural networks: A survey, 2022.
Jakob Hansen and Thomas Gebhart. Sheaf Neural Networks. 2019. URthttss://axxi...om abs/2012 06333.
Felix Hensel, Michael Moor, and Bastian Rieck. A survey of Topological Machine Learning Methods. Frontiers in Artificial Intelligence, 4, 2021.
Sajjad Heydari and Lorenzo Livi. Message Passing Neural Networks for Hypergraphs. In Elias Pimenidis, Plamen Angelov, Chrisina Jayne, Antonios Papaleonidas, and Mehmet Aydin (eds.), Artificial Neural Networks and Machine Learning ICANN 2022, pp. 583-592, Cham, Switzerland, 2022. Springer.
Max Horn, Edward De Brouwer, Michael Moor, Yves Moreau, Bastian Rieck, and Karsten Borgwardt. Topological Graph Neural Networks. International Conference on Learning Representations, 2022. URL https: opwnreview. net forum? wttpr//mmdddxxxxeeeeeeeeeeeeeeee
Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks are universal approximators. Neural Networks, 2(5):359-366, 1989.
Jürgen Jost. Riemannian Geometry and Geometric Analysis. Springer, Heidelberg, Germany, 2017. ISBN 978-3-319-61859-1
Alexandros D Keros. Vidit Nanda, and Kartic Subr. Dist2Cycle: A simplicial neural network for homology localization. Proceedings of the AAAI Conference on Artificial Intelligncee 36(7): 7133-7142, 2022.
Thomas N. Kipf and Max Welling. Semi-Supervised Classification with Graph Convolutional Networks. In International Conference Learning Representations, 2017. URL https:
//openreview..../ .nt www.oooiiidduuuoo.......o
John M. Lee. Introduction to Smooth Manifolds. Springer, New York, NY, USA, 2003.
Christopher Morris, Nils M. Kriege, Franka Bause, Kristian Kersting, Petra Mutzel, and Marion Neumann. Tudataset: A collection of benchmark datasets for learning with graphs. In ICML 2020 Workshop Graph Representation Learning and Beyond (GRL+ 2020), 2020. URL https: http://wwwgaaaaaaaaaaiae
io.
Vidit Nanda. Lecture notes in computational algebraic topology, February 2021.
Khang Nguyen, Nong Minh Hieu, Vinh Due Nguyen, Nhat Ho, Stanley Osher, and Tan Minh Nguyen. Revisiting smoothin and over-squashin using Ollivier-Ricci curvature. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 25956-25979, PMLR, 2023.
Allan Pinkus. Approximation theory of the MLP model in neural networks. Acta Numerica, 8: 143-195, 1999.
Mitchell Roddenberry, Nicholas Glaze, and Santiago Segarra. Principled simplicial neural networks for trajectory prediction. 2021. URL ht http:///axxv..///. 2rtg//s//1/22222222 02.0 0058.

---

Published as a conference paper at ICLR 2024
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E(n) equivariant graph neural networks. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 9323-9332. P, 2021.
Terence Tao. 111.6 Differential Forms and Integration. In Timothy Gowers, June Barrow-Green, and Imre Leader (eds.). The Princeton Companion Mathematics, pp. 175-180. Princeton University Press. Princeton, 2009.
M.E. Taylor. Measure Theory and Integration. American Mathematical Society, 2006.
Jake Topping. Francesco Di Giovanni, Benjamin Paul Chamberlain, Xiaowen Dong, and Michael M. Bronstein. Understanding over-squashing and bottlenecks on graphs via curvature. In International Conference Learning Representations, 2022. URL https: httss id=7UmjRGaz A.
Petar Veličković. Message passing all the way up. 2022. URL Ittps//////a/ 2202. 11097.
Petar Veličković. Everything is connected: Graph neural networks. Current Opinion in Structural Biology, 79:102538 2023.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations, 2018. URL https:tpp////c / /ttpss///////eeeeeeeeee ew. net/ forumfiruiioom d=rJXMp kCC.
Zhenqin Wu, Bharath Ramsundar, Evan N. Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh Pappu, Karl Leswing, and Vijay Pande. MoleculeNet: A benchmark for molecular machine learning. Chemical Science, 9:513-530, 2018.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How Powerful are Graph Neural Networks? In International Conference Learning Representations, 2019. URL https: // //operreeiww.. net forum ddeeiidddrrreaee 6155km.
12

---

Published as a conference paper at ICLR 2024
A DIFFERENTIAL FORMS IN R" BACKGROUND
In this section, we provide a basic background in the theory of differential forms. The more complete references see Lee (2003) for basic differential geometry and Jost (2017) for Riemannian geometry of k-forms. Moreover, see Tao (2009) for a short but highly illuminating article on the intuition behind differential forms.
Tangent and Cotangent Bundles on RR In IR"", the tangent space Tp p(R)) at point p E IR=" the vector space space spanned by the partial derivatives 0/ (p) The cotangent space p is the linear dual T* (IR") of the tangent space; i.e. the space of linear maps Home (T, (IR"), IR),, Note that both spaces are isomorphic to R The tangent bundle is the space (R)) = Lip (RR) consisting of gluing all tangent space together, where the topology is induced by the projection map T: (R*)) - RR". Likewise, the cotangent bundle is the space ("RR") = U, (R").
The space of vector fields x(R") over Fn is the space of sections of the tangent bundle; that idge. Vector fields decompose into the form Σ, (p))) (0000/(()))))))))/) (p), where U, : R + R. The space of 1-forms Q (BE") over R consists of the sections Rn - T (IR") of the cotangent bundle.
is. maps V R + T. (RR") such that π
Riemannian Structure on R The space Ih is a Riemannian manifold. That is, has a non- degenerate bilinear form
(---)p (R") @ (RR)) + I. The inner product is defined on by linear extension of the formula
(12)
(0/0z.(p), 0/0r, (p)), (z,, zzy
(13)
where the second inner product is the standard inner product on R". Over IR", the spaces of vector fields and 1-forms are isomorphic via the sharp isomorphism
# x(R") - Ω2 (TR")
(14)
Σ 020000 + Σ vade;
(15)
where dæ; is the 1-form which is locally (p) (-,0/0z : TR(R") R.
Exterior Products. Let V be a real vector space. Recall that an alternating k-covector on V is map
(16)
that is alternating with respect permutations. That is, that alv1, .2,.....k)) = (-1))9m(r)a((s) (r(2). (+(K)).
(17)
The k-th exterior product A (VV" over V is the space of alternating k-covectors.
k-Forms. The k-th exterior product of the cotangent bundle (TE* is the tensor bundle defined by locally taking the k-th exterior product A (RT)) of each cotangent space. A disferential k-form E (R") over R" is a smooth section of the k-th exterior product of the cotangent bundle A M. The space of forms Ω* (RR" of any dimension has an algebra structure given by the wedge product. The wedge product is moulti-lineat and satisfies anti-commutattvity
dx dx, dxx; da;
(18)
as well permutation equivariance
dr,, A... A dr;, (-1)sgn(t)dazn dai A... dois
(19)
for permutations T. As described the body of the paper, k-forms have a canonical monomial decomposition given by
w Σ ordrı-
(20)
where dx, dr.; A... ... A dr,, for multi-index I= (61,..., is) and scaling maps oz R
R.
13

---

Published as a conference paper at ICLR 2024
Types of k-forms. Differential k-forms are k-forms where the scaling maps as smooth; this is equivalent to the condition w that is a smooth section k-th exterior product of the cotangent bundle. When working with both neural k-forms and over non-compact spaces like RR, we often need to define other types forms.
A f-form w E 22 is compactly supported whenever each of the aj are compactly supported.
2. A k-form w E (25)"(RR)) is La whenever each az is square integrable.
3. A k-form Sp., (1R)) is piecewise linear if there exists triangulation of R² such that each a is piece-wise smooth over the triangulation.
Inner Products on k-forms. The choice of Riemannian metric on R extends an inner product on the k-th exterior product of the cotangent space by
-)p A";; (R") - R
(w) A Wk; !/ .... mp)p del (urs, (1),,
where w², E -2 (IR".. This induces an inner product over the compactly supported k-forms (M) by integration
(wi A... Awk. A mk)
(wi A Wk: m A A mh) dV (R")
(21)
PER"
against the volume form (Rl( on R".
Orthonormal Coframes. Equip R with the usual Riemannian structure, the compactly supported k-forms 22((RR) (RR) with the inner product -) described above and induced norm
w (w. w)n
for (RR). With this structure, the monomial k-forms form orthornomal coframe in the sense that
[1 if = I' else
(dn', dx )p
(22)
for each E R
Integration of k-forrm. Following (Taylor, 2006), we define integration of monomial k-forms = E, d then extend via linearity arbitary k-forms. Let ò A I be smooth map and coordinatize A with (11, 12..2,,,44 (4) above.
The pullback map of & is a map 6" ((R")) + 12(4*) taking forms on Th"h to forms on A In coordinates, the pullback d'w E (A*) of w along & defined by the formula
@"w= Σ arl dz,, (x;, A... A (Ø dx:,))
(23)
where
Oo. 8" dx; Σ
dt,
(24)
Øt
and Φ; is the an-cooponeet of Φb Note that by the monomial decomposition in Eq. (3), the pullback
can be written as
o'w fdl, dlz A... die
(25)
for some smooth function f: A R.
We define the integral So to be the standard Riemann integral
[ [w= / fdl ...d2...... dlk.
(26)
14

---

Published as a conference paper at ICLR 2024
The function f can be computed explicitly by unwinding 23 using the algebraic relations 18 and 19. Namely, for the monomial k-form w aydr, we have
Φ-w = ar(()
Σ
Seis dlp) J11
A A...A
deis dl3) 01i
(27)
0o.,
aoin (K))
ar(ō)
Σ
(T
dli A A... dlk
(28)
(1)
a)(()) (D)) A... dt
(29)
The above calculation in conjunction with 26 recovers the formula for the integral
J w -1=/// (D)))=
(30)
of general k-form given in Remark 8. In the special case that is an affine map, then the Jacobian expressed as
Don = (o'(t)) of (tt)
Properties of Integration. Throughout the paper, we refer to the linearity and orientation equivari- ance of integration of forms over simplices. These are consequence of the following theorem.
Proposition 9 ((Lee, 2003), 16.21). Suppose M an n-manifold with corners and w, (M) are smooth n-forms. Then
1. For a, b E R we have
I aw + bm = a MM
S
M
2. Let -M denote M with the opposite orientation. Then
[ 1M
w [ w.
B INTEGRATION OF k-FORMS IN PRACTICE
We give further practical details how to approximate integrals in the case of 2-forms. For more detailed and accessible introductions we recommend the following references (Tao, 2009; Taylor, 2006).
Explicit computations for 2-forms. In Section we use integration of 2-forms 2-dimensional simplicial complexes in order to classify surfaces. We will derive an explicit method to approximate the integral of 2-forms IR" over an embedded 2-simplex based on the theory above.
Recall that a 2-form w E (R") can be written in coordinates as
Σ DSiKjSn
anjdæ; A dzj,
(31)
where oijy RR" R are smooth maps. We consider a map & 2 - R." giving the embedding of the standard 2 simplex in IR.. Using the expression from Equation 26:
(129) w
/ 0" d'(w)
will show how integrate the emhedded simplex of 2).
15

---

Published as a conference paper at ICLR 2024
Using local coordinates to express the embedding ((t,,t)) 12) (I1,..., write out explicitly the pullback a 2-form w E 52 (R") via the map 0:
dr,
dri dt2)
Ozg di +
Ozi dl2)
6'( Σ ogiejsn
ds drz)= = Σ
(ais o0)
di +
0l2
Oy
012
uSisisn
dzg dl, ^
dxj dl1) +
Uzi
azj dll1)
Σ USiSJSn
(ais o o0)
dl2 ^
011
012
01
011
dr, dr,
Dz, Ozg
Σ OsicJnn
(ais (o))o0)
dtq A dtz
(32)
0t dt2
0tz
dt
In the computational context that we consider we want to integrate a 2-form on the 2-simplices of complex embedded in IR., We some notation here, each 2-simplex X is the image of an affine map ((t) Φt + b: A - I where + = DΦ [ 0i2loseen is (n x 2ammtr..x. Since @ is an affine map the term A in 32 becomes
= (Do) = det $
(33)
with & denoting the (2 x 2) submatrix of & corresponding to the rows and j.
Putting all this together we obtain:
('(w)= =
Σ
ai.( oldh, A dl2.
(34)
oSi<jsn
So the integration becomes
w
/ Aa
('(w)))
Σ OSi<isn
(n) det o de, A dl2.
(35)
A
Computationally we cannot perform exact integration of 2-forms in practise we approximate the integral on A above by a finite sum, as is done with Riemann sums in the classical case. In general this is how the Vol Approx function defined. To do so the first step to subdivide the domain of integration 2 into collection S of smaller simplices with vertices denoted by Then the integral approximated by summing the average value of the map g on each simplex of the subdivision:
f gdtz A dtg
E vol(s).
(36)
8ES2 3
An explicit method of this sort can be easily generalized to higher dimensional simplices in order to compute integrals of k-forms k-dimensional simplicial complexes.
C PROOFS
Universal Approximation I.
We start with the proof of Proposition 3. First, recall the well-known following universal approximation theorem for neural networks. We cite here a version appearing in (Pinkus, 1999).
Theorem 10 (Universal Approximation Theorem, Thm 3.1 (Pinkus, 1999)). Let a C(RR RR) be a non-polynomial activation function. For every n, l E N, compact subset C R", function C(K,R, IR) and > 0 there exists:
integer E N;
matrix W, E RR
16

---

Published as a conference paper at ICLR 2024
bias vector b I/ and
matrix W2 E [ixj
such that
|lf - g1100
where g the single hidden layer MLP
g(a) (20 b).
Theorem 3. Let CRR, be a non-polynomial activation function. For every n. N and compactly supported k-form vj E ( and >> O there exists a neural k-form w with one hidden layer such that
| nl12 <c
Proof.
Denote the scaling functions of y by
r Σ mıdzr-
Since is compactly supported, each fy is compactly supported over some domain D. By 10, for any > there exists a one layer MLP v R" - k²()) such that
| millo < </Voo(DD)
Using the orthogonality form Eq. (22), we have that
||w 1 (w) ") w 1)p Σ (w)(() - mr(p)) - -mm)) E&²
/VVooD(D).
((:))
Integrating the above, we attain
| n112 < / (w0 n, -V
< <<\/Vol(D) / dVol(R")
=
proving the result.
Equivariance. Integration is linear in both k-forms (Lee (2003), 16.21) and embedded simplicial chains 7 in the sense that it defines bilinear pairing
C+(S; 60 R
(37)
for some embedded simplicial complex @ S S R This property directly implies the kind of multi-linearity described in Proposition 6. We present proof here for completeness. Proposition (Multi-linearityy.
& S + Ra be an embedded simplicial complex. For any matrices ' xm (RR) and R E M ExE (R) we have
(LB, (B)
(38)
Proof. On the left have
Xo(LB,w),s=X x Σ LIABR....... Lm Bkkss))))) is
(39)
f Ex 1i, Wj
(40)
-Eui, Liii f W/
Jan
(41)
=[LXo(3,w))..
(42)
17

---

Published as a conference paper at ICLR 2024
Similarly, the right we have
(o(3,wR),, =Xo(3,
E Rb Rk .....
E
Rs, Rkew
(43)
Σ Re.jeak
(44)
Riad
Wk
(45)
[x(3, W)
(46)
D ADDITIONAL EXPERIMENTS
D.1 CONVOLUTIONAL 1-FORM NETWORKS
Convolution and Equivariance. Beyond orientation and permuation equivariance, the node em- beddings themselves may be equivariant with respect to group action on the feature space. The class of geometric graph neural networks Han al. (2022) were designed specifically deal with such equivariances. The lesson from image processing is that translation equivariance can be mitigated via convolution. In this paradigm processing embedded (oriented) graphs with learnable, 'template' 1-forms is directly analogous processing images with learnable convolutional filters. To make this connection precise, we will perform a small example to classify oriented graphs.
Synthetic Data. In Figure 5, there are two classes of cycle and star graphs. The cycle graphs have clockwise orientation and the star graph are oriented so edges point inwards. Each data point is a simplicial complex representing the disjoint union of either three cycle or star graphs which are randomly recentered around the unit square. The chains on each complex are initialised using the standard oriented I-simplices as basis.
Architecture. We initialise a neural network : R2 R2x2 with single 64-dimensional hidden layer and ReLU activation which corresponds to two feature 1-forms. To perform a convolutional pass, we first discretize the unit square to produce set translations. each translation, we restrict the embedded graph to the subgraph within neighbourhood and weight it by the local node density approximated by a standard kernel density estimator. This is equivalent to translating the 1-form by the corresponding (negative) vector in IR2 and integrating over a small neighbourhood (whose area is a hyperparameter). Integration produces an integration matrix at each point with two columns, and taking the column sum represents the oriented integral of the two convolutional filters against the local neighbourhood of the oriented graph. cepossEnttopyyoo..o is calculated by summing the integrals over all translations and applying softmax.
Interpreting the Results. The integrals are shown in Figure 5 as colouring of each point in the grid of translations. We plot the learned 'template" 1-forms next to an example of their respective classes. By the construction of the objective function, the algorithm is trying to maximise the sum of the integrals across all translations within the class. As the synthetic paths example, the learned filters successfully capture the interpretable, locally relevant structure; the edges in each class resemble flow-lines of the learned vector field in different local neighbourhoods around the relevant translations.
D.2 VISUALISING SIMPLICIAL LAPLACIAN 1-EIGENVECTORS
In this example, we show how we can use neural 1-form to visualise the 1-eigenvectors of the simplicial Laplacian a Rips complex in 2R² (Figure 6). The idea is that we start with a collection of eigenvectors of simplicial 1-cochains, and learn 1-forms which integrate to them. We see the results
See Appendix for definitions.
18

---

Published as a conference paper at ICLR 2024
Learable 1-form
Cony Filter
Learnable Conv Filttrr
Figure 5: convolutional 1-form network with the learned convolutional filters forms.
in Figure 6b, where the three eigenvectors belong to the harmonic, gradient-like and curl components of the simplicial Hodge decomposition respectively.
First we generate point cloud in 2 noisey approximation of circle, then take the Rips complex for fixed parameter E. We then calculate the first 9 eigenvectors of the 1-simplicial Laplacian, sorted by increasing eigenvalue, and store them as columns matrix Y with respect to the standard basis B. We initialise 9 neural 1-forms
= (WI, W2, wg) (Rp2(R3
on R using ReLU activation function. Our loss function is then
((w) = (B, - Y||
where the norm is the standard matrix norm, and (z(,ww) is the evaluation matrix yielded from integrating over B. When L is small, the 1-forms over I2² will correspond to the selected simplicial eigenvectors when integrated over the complex.
Figure 6 shows the results-an one notes that the different 1 1-forms appear to coalesce around geometric features in the underlying point cloud. Additionally, is important to observe that the Laplacian 1 1-eigenvectors be categorized into three distinct classes depending on their membership within the components of the Hodge decomposition:
1. The first class consists of eigenvectors that belong the image of the adjoint of the differential operator d'.
2. The second class comprises eigenvectors that reside within the kernel of the Laplacian operator A
3. Lastly, the third class includes eigenvectors that part of the image of the differential operator d
These classes correspond, in the context of simplicial structures, analogues of rotational, harmonic, and gradient-lite like vector fields found in Riemannian manifolds. By referring to Figure 6b, one can visually identify the class to which each eigenvector belongs based on whether or not there exists rotational structure. We have color-coded these as green, red, and blue, respectively.
E EXPERIMENTAL SETUP AND IMPLEMENTATION DETAILS
E. SYNTHETIC PATH CLASSIFICATION
Synthetic Path Classification. We initialise neural 1-forms W1, W2, Wa E 2/ (R?) with ReLU activation for each of the three classes. Each path p is represented where the orientation induced by the direction of the path. Letting 3 be the standard basis, the three 1-forms generate evaluation matrix X (B,w) w) whose entries the integration of the 1-form Jc, Wy against each 1-simplex in the paths.
oriented simplicial complex,
The readout function sums the entries each column, which by the linearity integration, represents the path integral along p against each wi. In short, each path p is represented vector in R using
19

---

Published as a conference paper at ICLR 2024
(a) Čech Cover
(b) Learned 1-forms
Figure 6: Approximating the 1-eigenvectors of the simplicial Laplacian with learnable 1-forms using the Cech Cover of point cloud.
Compeeece
Coopornet
Compureet
235
1.0
1.
Figure 7: Learned O-forms for Synthetic Path Classifcation.
the map
p - S Wl:
/ W2, w3) E RR3.
(47)
We then use cross-entropy loss between this vector and the class vector. In this design, each 1-form corresponds class, and a path should ideally have high path integral against this form if and only if belongs to the class.
Comparison with Neural Networks. our synthetic experiment, we test whether edge data is necessary by examining whether we can attain comparable results using only the embedded vertices of the path.
In our context, the right framework analyse the vertices is set of 0-forms on R in other words, scalar functions over R². We use the same setup as before, where we initialise three functions f2, fa (R?) corresponding to the three classes. Integration of the vertices in the path against each of f, simply corresponds to evaluating f; at the vertex. Summing the columns of the evaluation matrix then sums up the value of fa all points in path. In this sense, each functions much like an approximated 'density" for the vertices of each class albeit with negative values.
In Figure 7, show example paths from each class against the learned scalar function representing that class. In this example, the vertices paths in each class have a similar density. One sees that the algorithm learns something reasonable, picking out minor fluctuations density, but struggles overall to separate out the classes. With the same number of parameters and training time, the algorithm is only able achieve training accuracy of 37%.
E.2 SYNTHETIC SURFACE CLASSIFICATION
We consider two classes of synthetic surfaces obtained by embedding a triangulated square in R3, The embedding of the vertices of the triangulated square given by functions of the type ó1 y) = sin(x) + e(x, 3) for the first class and Φ2(2, y) = sin (y) + E(x, y) for the second class, where e is random noise. We initialise
neural 2-forms W1, w2 22 (RR) with ReLU activation for each of the classes. Letting 3 be the standard basis, the two 2-forms generate an evaluation matrix (x(3,w) w) whose entries are the integration of the 2-form fe, wj against each 2-simplex in the surfaces. The readout function sums the entries in each column, which represents the integral of each
20

---

Published as a conference paper at ICLR 2024
Cass 1
Closs
1.0
-10
0.0
(a) Integral of wi representative of the first Integral of w/ on a representative of the first class.
class.
Figure 8: Representative surfaces of each classes, colored by the integral of the learned 2-form w1 over the surface. The integral of w1 over elements of the first class yields positive values whereas the integral of the same form over elements of the second class yields negative values. For the form w2 the opposite is true.
wi over the entire surface, thus yielding a representation of the surfaces in R2 in the following way
s s W1. W2,) E R²
(48)
Finally, we use the cross entropy loss function to classify the surfaces into the two classes. In this design, each 2-form corresponds class, and surface should ideally have high integral against this form if and only if it belongs to the class.
Figure 8 shows the integral of wi on surfaces taken from each of the two classes. This integral is positive for elements of the first class and negative for elements of the second class. Similarly the values for the integral of W2 is negative on surfaces of the first class and positive on elements of the second class.
E.3 GRAPH BENCHMARK DATASETS
For the small graph benchmark datasets (AIDS, BZR, COX2, DHFR, Letter-low, Letter-med, Letter- high), we use a learning rate of le 3, batch size of 16. hidden dimension of 16, and h discretisation steps for all k-forms. For our comparison partners, i.e. for the graph neural networks, we use hidden layers to permit message passing, followed by additive pooling. As result, all models have roughly equivalent parameter budgets, with our model having access to the smallest number of parameters.
Architectures. Our model architectures for our comparison partners follow the implementations described in the respective papers (Kipf Welling, 2017; Veličković et al., 2018; Xu et al., 2019). package and the classes GAT, GCN, and GIN,
We make use of the wwworrch-gemeeeee respectively. Our own model consists a learnable vector field and a classifier network. Letting H refer to the hidden dimension and Dm, D to the input/output dimension, respectively,
realise the vector field as an MLP of the form Linear [Di., H] - ReLU Linear [H, H/2] ReLU Linear [1/2,, Dout 1. The classifier network consists of another MLP, making use the number of steps h for the discretisation of our cochains. has an architecture of Linear (h, H
ReLU Linear H, H/2] ReLU Linear [H /2, c]. where e refers the number of classes.
Training. We train all models in the same framework, allocating at most 100 epochs for the training. We also add early stopping based on the validation loss with patience of 40 epochs. Moreover, use a learning rate scheduler to reduce the learning rate upon a plateau.
21


Reviews:
[{"review_id": "TycAdhiTZM", "reviewer": "Reviewer_VjaY", "summary": "In this paper, the authors introduced a new method, neural k-form,  of utilizing integration of differential forms over embedded simplicial complexes for robust and interpretable data representations. The authors provided necessary background information. Then they introduced the basic idea of neural k-form, and showed that neural k-form can approximate any k-form with compact support. The authors discussed the concept of integration matrices and their basic properties and presented numerical examples of classification problems utilizing the neural k-form method.", "questions": "On page 3, why is the parallelepiped spanned by tangent vectors called \"infinitesimal parallelepiped\"? The intuition and the formal definition are not very well connected. \n\nOn page 4, second paragraph, should $b\\in R^{1\\times m}$ instead of $b\\in R^{1\\times n}$?\n\nCan you provide an argument for your stamens on the dimension of singular k-chains?\n\nOn Page 15, right below equation (38), should it be $\\alpha_{i,j}$ instead of $f_{i,j}$?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The authors tap into the rich area of topology and geometry, established a potentially very powerful connection.", "weaknesses": "Overall, I feel that while the method proposed in the paper has potential it has not been fully developed and analyzed. \n\nMany statements, for example, \"Thus an l-tuple of k-forms produces an l-dimensional representation of the simplex independently of any message passing\" in the introduction and \"In practice, this will facilitate the training process and permit interpreting the learned cochains\" in remark 5, are not backed up by detailed analysis. \n\nThe neural k-form learning architecture is presented very briefly in the caption of a figure, it is very difficult for others to figure out how to implement the method. \n\nIt is understandable audience would want to find out the performance of the method proposed for high dimensional problems. Unfortunately, the experiments are limited to 1-form and 2-form, and the differences in performance are not very significant except in the last experiments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pNCYh2c4lT", "reviewer": "Reviewer_pNcs", "summary": "This paper proposes a geometric deep learning method termed neural k-forms. The proposed method leverages geometric information of data and creates representations of simplices using differential k-forms. The authors show the universal approximation property of the neural k-forms in approximating k-forms by applying differential geometry tools. Through several stylized tasks, the authors demonstrate the effectiveness and interpretability of the approach. The proposed neural k-form also outperforms message passing based state-of-the-art graph neural networks in several real-world graph classification benchmark datasets.", "questions": "1. How does the neural k-form compare to other simplicial neural networks in terms of performance and computation?\n2. Is there a way to incorporate convolution in neural k-forms?\n\nTypo: in eq. (6), $d_{x_{i_k}}$ should be $dx_{i_k}$.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. The paper provides an interesting view of MLPs as neural 0-forms and extend it to neural k-forms to harness the geometric information in data via embedded simplicial complexes.\n2. The integration of neural forms is introduced to construct finitely-parametrizable, learnable singular cochains such that the neural k-form is usable in practice. Moreover, the integration matrix induced by a neural k-form produces feature cochains that facilitate training process and permit interpretation.\n3. Albeit rather technical and math-heavy, the presentation is self-contained and consistent, and the authors provide strong connections between mathematical machinery and model intuition.\n4. The proposed method generates interpretable features in synthetic tasks and outperforms standard GNNs by a large margin in several real-world graphs.", "weaknesses": "1. The neural k-form relies heavily on the geometric information of the node features and makes use of the simplicial complex structure of data but to a lesser extent the graph information. Learning on graphs with weak or no geometric information can pose a great challenge to the proposed method.\n2. The computation feasibility for higher order k-forms and the numerical integration may hamper their application in practice.\n3. The approach triumphs in graph classification thanks to the simplicial representations but may suffer in node prediction when smoothing is a blessing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gg1YRZbhgH", "reviewer": "Reviewer_E4oE", "summary": "This paper studies the neural representation learning over simplicial complexes. Specifically, They propose to embed the simplician complex into a integration matrix. The integration matrix is computed by the integral of neural k-forms, which is a linear combination of k-forms weighted by MLP outputs. They argue that many previous works rely on an non-canonical initialisation of feature cochains to compute the representation, while their method bypass this problem as integration matrix does not involve the cochains.", "questions": "- I think the whole pipeline is not very clear to me: when producing the integration matrix using neural k-forms, which are the learnable parameters and which are the hyper-parameters?", "limitations": "", "rating": 5, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The background introduction is comprehensive and technical part seems sound", "weaknesses": "- The baseline GAT, GCN and GIN in experiments are too weak: these are the most basic message passing GNNs. Comparisons with other neural networks on simplicial complexes could make the evaluation more solid.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dZ5e5Zgxyy", "reviewer": "Reviewer_gTeC", "summary": "This paper proposed a novel method for representation learning of geometric simplicial complexes, in which a differential form parameterized by a multilayer perceptron is integrated over the simplices of an embedded complex, and these integrals are then read out into a representation of the complex that is not dependent on the size or dimension of the complex. This method is considered in a variety of tasks for processing simplicial complexes and graphs with geometric information.", "questions": "1. My primary suggestions relate to the writing of the experiments section. There is not sufficient detail in Section 5 to understand each of the experiments. I think it would be better to shorten some of the background material in order to have space for better descriptions of your experiments. In particular:\n\na. Path classification: are the \"paths\" that you classify themselves simplicial complexes? I.e., is the idea to represent each path as a 1-d simplicial complex embedded in $\\mathbb{R}^2$, and then integrate against the simplices of the path? I see this to be the case in the appendix, but these details should really be in the main body of the paper for the purpose of readability.\n\nb. Visualizing simplicial Laplacian 1-eigenvectors: the description of this is far too short to understand what is going on here, and the utility of this is questionable. Comparing to the eigenvectors of the simplicial Laplacian is reasonable for the purpose of identifying (co)homological features of the complex, but wouldn't it be better to use neural k-forms to get a proxy for the intrinsic eigenfunctions of some manifold embedded in $R^n$ from which a complex is formed? This experiment needs a lot more motivation.\n\nc. Synthetic surface classification: again, having a description of the dataset in the main body of the paper is needed for readability here. It is fine to leave some details in the appendix, but the body of the paper should be sufficient to get a good understanding of what is going on.\n\n2. I have some questions about invariants that can be incorporated into these networks. As I commented in the weaknesses section, the function learned by neural k-forms is highly dependent on particular geometric embeddings of the simplicial complexes in Euclidean space, as opposed to the intrinsic geometry of the complex. For instance, if I apply a translation to the embedding of the complex, I am likely to get a completely different result when integrating the neural k-form against it. Do you have any comments or insights on handling issues like this? This relates also to your experiment on classifying molecular graphs: the graphs can be geometrically embedded in $\\mathbb{R}^3$, but is there a canonical rotation that these embeddings should have? Two identical molecules might be embedded in different ways that yield different results by the network, which is not a property that a message-passing network suffers from. Perhaps a group symmetry should be incorporated for this issue. See the following reference, for instance:\n\nHan, J., Rong, Y., Xu, T., & Huang, W. (2022). Geometrically equivariant graph neural networks: A survey. arXiv preprint arXiv:2202.07230.\n\nI think comparison between neural k-forms and geometric GNNs should be made, in order to yield a fairer picture of where your method stands for graph classification tasks.\n\n3. Can you please comment on whether or not simplicial complexes with non-geometric features could be incorporated into the method you propose? For instance, if the geometric complex not only has an embedding, but also a particular (co)chain supported on it supplied, how would one incorporate that information into a learning pipeline?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The approach proposed by the authors is novel, yet simple, and rooted in a geometric view of learning for simplicial complex data.\n2. The proposed method is able to apply one learned function (k-form) to tasks over many geometric simplicial complexes, rather than only learning a function for a given complex.", "weaknesses": "1. This paper suffers from vagueness in some parts, particularly in the experiments section. This is my reason for rating the presentation as \"fair.\" Otherwise, the writing of the paper is good.\n2. Compared to message-passing schemes, the learned k-forms appear to be highly dependent on the particular geometric embedding of the complex, instead of the intrinsic topology/geometry.\n3. The type of tasks considered by the authors are not common ones in the literature, so I am not sure if there is a regime in which the proposed methods can be compared to existing simplicial neural networks, which are largely based on combinatorial, rather than geometric, information. However, there are message-passing graph neural networks that are designed to handle geometric information as well, which the authors did not compare to.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5yO2Hr2AJW", "reviewer": "Reviewer_rtvN", "summary": "This paper proposes using differential $k$-forms in $\\mathbb{R}^n$ to create representations of geometric simplices. Specifically, *neural* $k$-forms are parameterized by MLPs that take geometric information. By then integrating the $k$-forms using the simplicial complex, the authors obtain the *integration matrix*, which can be read out for further processing. The method allows processing simplicial data without the need for (simplicial) message passing. Solid theoretical results are supported with several motivating experiments, opening new directions for future research.", "questions": "* Could you discuss the unique linear map $\\phi|_\\sigma: \\Delta^k \\to \\mathbb{R}^n$? It is not quite clear how this function for $k < n$ is well-defined?\n* At equation 9, I was a bit lost how we define $\\phi$. Before eq. 4 we define $\\phi: S \\to \\mathbb{R}^n$, but here $\\phi$ seems to take $t \\in \\Delta^k$ which seems more in line with $\\phi|_\\sigma$. Did we suppress some arguments here, or are we assuming something about the elements of $S$?\n* What are the inputs and outputs to your neural networks?  Are the positions of the node locations input as $(\\mathbb{R}^d)^n$ when you have $n$ points, or do you concatenate them?\n* Could you elaborate a bit more on how you go from $\\mathbb{R}^{\\binom{n}{k} \\times \\ell}$, together with the simplicial complex structure, to the integration matrix $X_\\phi(\\beta, \\omega)$? Specifically, in eq. 13, the sum is over all multi-indices, correct? Do we need to know $\\lambda_a$ somehow? \n* Could you discuss the advantages of your method over (simplicial) message passing? Why does it yield better empirical results? Should this method replace message passing altogether, or are there limitations?\n\n\n\n**Final Remarks**. Admittedly I'm lacking some background knowledge to objectively assess the paper. Despite that, I'm really excited about this project. However, I think if the authors put a bit more effort into conceptualizing some core ideas, the paper would improve considerably.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "* The presented ideas are original, relevant and opening up new directions of research.\n* The theory, though not trivially graspable for the typical machine learner, is presented clearly.\n* The theory is motivated with convincing experiments.", "weaknesses": "* While the content of the paper is naturally foreign for most machine learning researchers, the authors could put a bit more effort into making the content slightly more accessible. I would encourage them to include as much examples as possible, especially since the full 9 pages were not completely used yet. The authors can gain even more space by e.g. leaving out the introduction to MLPs, which should be familiar for the majority of readers.\n* Experimental details are highly suppressed, it is not quite clear what the inputs, outputs, and learning objective of the experiments are.\n\nMinor:\n* Above eq. 1, you introduce the notation of a convex hull using $v_0, v_1, \\dots, v_k \\in \\mathbb{R}^n$, but before and after you use $v$ and $v_i$ for vertices in $V$. This caused some confusion for me while reading this part. Similarly on page 3 when introduction $k$-vectors.\n* Experiments are mostly limited to the synthetic setting.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)