E(n) Equivariant Topological Neural Networks
Authors: Claudio Battiloro
Ege Karaismailoglu
Mauricio Tec
George Dasoulas
Michelle Audirac
Francesca Dominici
Venue: ICLR 2025 Poster (2025)
Keywords: Topological Deep Learning
Equivariance
Equivariant Neural Networks
Geometric Deep Learning
Geospatial data
Air Pollution Prediction
Molecular Property Prediction
Primary area: learning on graphs and other geometries & topologies
TL;DR: We introduce E(n) Equivariant Topological Neural Networks, i.e.  a wide  and provably expressive class of E(n) Equivariant Message Passing Networks over combinatorial complexes

Abstract: Graph neural networks excel at modeling pairwise interactions, but they cannot flexibly accommodate higher-order interactions and features. Topological deep learning (TDL) has emerged recently as a promising tool for addressing this issue. TDL enables the principled modeling of arbitrary multi-way, hierarchical higher-order interactions by operating on combinatorial topological spaces, such as simplicial or cell complexes, instead of graphs. However, little is known about how to leverage geometric features such as positions and velocities for TDL. This paper introduces E(n)-Equivariant Topological Neural Networks (ETNNs), which are E(n)-equivariant message-passing networks operating on combinatorial complexes, formal objects unifying graphs, hypergraphs, simplicial, path, and cell complexes. ETNNs incorporate geometric node features while respecting rotation, reflection, and translation equivariance. Moreover, being TDL models, ETNNs are natively ready for settings with heterogeneous interactions.  We provide a theoretical analysis to show the improved expressiveness of ETNNs over architectures for geometric graphs. We also show how E(n)-equivariant variants of TDL models can be directly derived from our framework. The broad applicability of ETNNs is demonstrated through two tasks of vastly different scales: i) molecular property prediction on the QM9 benchmark and ii) land-use regression for hyper-local estimation of air pollution with multi-resolution irregular geospatial data.  The results indicate that ETNNs are an effective tool for learning from diverse types of richly structured data, as they match or surpass SotA equivariant TDL models with a significantly smaller computational burden, thus highlighting the benefits of a principled geometric inductive bias. Our implementation of ETNNs can be found at https://github.com/NSAPH-Projects/topological-equivariant-networks.

Published as a conference paper at ICLR 2025
E(n) EQUIVARIANT TOPOLOGICAL NEURAL NETWORKS
Claudio Battiloro .1.,, Ege Karaismailoglu" 1122, Mauricio Tec".,, George Dasoulas* Michelle Audirac', Francesca Dominici Harvard University, 'ETH Zurich
ABSTRACT
Graph neural networks excel at modeling pairwise interactions, but they cannot flexibly accommodate higher-order interactions and features. Topological deep learning (TDL) has emerged recently
promising tool for addressing this issue. TDL enables the principled modeling of arbitrary multi-way, hierarchical higher-order interactions by operating on combinatorial topological spaces, such simplicial or cell complexes, instead of graphs. However, little is known about how leverage geometric features such as positions and velocities for TDL. This paper introduces -Enuivarrrr Topological Neural Networks (ETNNs), which are Eln>eequivariant pressage-passingg networks operating on combinatorial complexes, formal objects unifying graphs, hypergraphs, simplicial, path, and cell complexes. ETNNS incorporate geometric node features while respecting rotation, reflection, and translation cquivariance. Moreover, being TDL models, ETNNs are natively ready for settings with heterogeneous interactions. We provide theoretical analysis to show the improved expressiveness of ETNNS over architectures for geometric graphs. We also show how equivvaair variants of TDL models can be directly derived from our framework. The broad applicability of ETNNS is demonstrated through two tasks of vastly different scales: i) molecular property prediction on the QM9 benchmark and ii) land-use regression for hyper-local estimation of air pollution with multi-resolution irregular geospatial data. The results indicate that ETNNS are an effective tool for learning from diverse types of richly structured data, as they match or surpass SotA equivariant TDL models with significantly smaller computational burden, thus highlighting the benefits of principled geometric inductive bias. Our implementation of ETNNS can be found here.
INTRODUCTION
Graph Neural Networks (GNNs) are employed across various fields, including computational chem- istry (Gilmer et al., 201 7a), physics simulations (Shlomi et al., 2020), and social networks (Xia et al., 2021), to highlight a few. Their effectiveness stems from merging neural network adaptability with insights about data relationships through the graph topology. Research on GNNs covers a broad range of categories, mainly divided into spectral (Bruna et al., 2014) and non-spectral (Gori et al., 2005). In both cases, GNNs aim to learn representations for node (and/or) edge attributes through local aggregation driven by the graph topology, essentially defining message passing networks (MPN) (Gilmer et al., 2017c). Utilizing this capability, GNNs have delivered significant achievements in tasks like node and graph classification (Kipf & Welling, 2017), link prediction (Zhang & Chen, 2018), and specific challenges such as protein folding (Jumper et al., 2021).
However, GNNs usually struggle to model bigher-order and multi-way interactions that the intrin- sically pairwise structure of graphs cannot capture. Consequently, the field of Topological Deep Learning (TDL) (Barbarossa & Sardellitti, 2020; Hajij et al., 2022b; Papamarkou et al., 2024) started gain significant interest it seeks to address these limitations. Although the general scope of TDL is designing models for data defined over a variety of topological spaces, the main focus usually on combinatorial topological spaces (CTS), i.e., topological spaces that can be described combinatorially, such as cell complexes (Grady & Polimeni, 2010; Bodnar, 2023; Battiloro, 2024). Leveraging this
'qqual contribution. Author ordering determined by random number generator. 'Corresponding author. (cbatti ooro, ekara ama i 1 lu, mauri r i ogtec, maudí rac) @haph harvar rd. edu, georgi os_ dascul as@hms. ha rvard. .ddu

---

Published as a conference paper at ICLR 2025
Geometric Graph
Combinatorial Complex
E(n) Equivariant TNN
Lb
Non-Geametric Ceormetric
Higher
Equhorlant Message Passing Induaced by the Comples
Updated Non- Cepmetricccc and
Caometric Invartants
Figure I: Overview of the Equivariant Topological Neural Networks framework. An input geometric graph/point cloud with (possibly) non-geometric features provided. A combinatorial complex, whose elements are called cells, is constructed from the input geometric graph/point cloud to encode higher-order hierarchical interactions, based on topological or commain-speciffe considerations. If available, higher-order cell features can be injected. Geometric invariants for the cells (e.g. pairwise distances, Hausdorff distances, volumes, ...) are computed. Finally, En) Equivariant Topological Neural Networks update both geometric and non- -onneeemetric features to improve a downstream task while respecting rotation, reflection, and translation equivariance.
combinatorial structure, it is possible design more sophisticated adjacency schemes among cells, i.e. single nodes or groups of nodes, than the usual adjacency in graphs. Message passing networks over CTS have been proven to more expressive (Bodnar et al., 2021b; Battiloro al., 2023e; Bodnar et al., 2021a; Truong & Chin, 2024; Jogl et al., 2023) (using the Weisfeiler-Lehman test criterion (Xu et al., 2019)), more suitable to handle long-range interactions (Giusti et al., 2023b;a), and more effective to handle heterophilic data settings (Bodnar al., 2022: Hansen & Gebhart, 2020) than classical GNNs. Cell complexes are powerful objects to model certain classes of higher-order interactions along with hierarchical structure among them, while general combinatorial objects like hypergraphs are useful to model arbitrary higher-order interactions, but without any hierarchy. Hajij al. (2022b) introduced the notion of Combinatorial Complex (CC) provide formal, flexible, yet simple structure that pseudo-gent generrallsss simplicial/path/cell complexes and hypergraphs, thus enabling the design of arbitrary higher-ordee interactions along with a hierarchical structure. They also introduced message passing networks over combinatorial complexes.
Little known about how to integrate and leverage geometric information of the data in such TDL models. In this context, nodes are embedded in manifold or, in general, in metric space, and have geometric features, such as positions or velocities. In the case of geometric graphs, common practice is designing message passing networks able to respect some suitable symmetry, i.e., being equivariant w.r.t. the action of some symmetry group. Here we focus on E(n) -qquivraaaa neural networks. i.c., networks being equivariant w.r.t. rotations, reflections, and translations. The E(n)- Equivariant Graph Neural Network (EGNN) from Satorras et al. (2021) is one of the simplest yet powerful equivariant models for graph-structured data. Eijkelboom et al. (2023) generalized the approach from Satorras et al. (2021) to simplicial complexes (a particular instance of cell complexes), introducing Equivariaa Message Passing Simplicial Networks (EMPSNs), able to handle graph or simplicial-structured data. However, as we will show in the paper, both graphs and simplicial complexes are combinatorial objects often not flexible enough model arbitrary higher-order interactions, and a general framework for designing equivariant neural networks over CTS is missing, Appendix B presents a review of related works.
Contribution. We generalize the approaches from Satorras et al. (2021) and Eijkelboom et al. (2023) to design Equivaanna Topological Neural Networks (ETNNs), i.e., E(n) (Qu)-aquia Message Passing Networks over Combinatorial Complexes. CCs represent of graphs, simplicial complexes, path complexes, cell complexes, and hypergraphs, therefore ETNNS represent legitimate framework for designing arbitrary -Equivvraaai Message Passing Networks over combinatorial topological spaces or, more in general, over combinatorial objects used to model interactions among entities, such hierarchical graphs. An ETNN layer is made of a message passing round over the CC comprising scalar invariants of the geometric features and a learnable update of the geometric features added up in the current directions. As such, ETNNS scalarization
2

---

Published as a conference paper at ICLR 2025
method (Han et al., 2022; 2024). We study the improved expressiveness of ETNNs by evaluating their ability to distinguish d-distinct geometric graphs (Joshi et al., 2023), showing that they are at least expressive as other well-known scalarization- based En)) equivariant graph neural networks. Furthermore, we empirically show that ETNNs are also able to distinguish most of the counterexample structures from (Pozdnyakov et al., 2020; Joshi et al., 2023) that are indistinguishable using k-body scalarization. Generalizing (Hajij et al., 2022b) to the geometric setting, we show how several [(n)- equivariant variants of TDL models can be directly derived from our framework. As an additional contribution, we test ETNNS in applications where domain knowledge induces combinatorial complex structures that are more natural and parsimonious than the graphs or higher-order structures that are usually employed to model the same phenomena. Specifically, we show the efficacy of ETNNS on i) molecular properties prediction on the QM9 benchmark, and on ii) land-use regression for hyper-local estimation of air pollution with multi-resolution irregular geospatial data, representing also a completely new challenging benchmark for TDL models. Crucially, thanks to the ficxible and principled inductive bias induced by the underlying combinatorial complexes, ETNNs match or surpass the results of EMPSNs with less than half of the memory usage and almost half of the runtime per epoch. The obtained results jointly validate ETNNs, our CC-based approach, and, in general, the injection of geometric information in TDL models.
2 COMBINATORIAL COMPLEXES AND EQUIVARIANCE
We review the fundamentals of combinatorial complexes (CCs), fairly general objects providing an effective and flexible way represent higher-order multi-way interactions. Such structures generalize graphs, simplicial complexes, path complexes, cell complexes, and hypergraphs. We first define our domain of interest -CCs-, the relations among its elements -neighborhood functions-, the data defined on it -topological signals-, and a class of deep networks operating on it -CC message passing networks (CCMPNs)-(Hal) (Hajij et al., 2022b). We finally introduce the formal notion of equivariance, showing some properties of CCMPNs.
Combinatorial Complex. A combinatorial complex (CC) a triple (8, X' rk) consisting of a set S, subset X' of (0)), and a function rk X + Zz0 with the following properties:
1. for all & E S (+) E X: the function rk order-preserving, i.e., xr y E X' satisfy x S y, then rk(x)) S rk(y).
The elements of S and X' are nodes and cells, respectively, and rk(-) is the rank function. X' simplifies notation for (S, X. rk). The rank function flexibly organizes the cells hierarchically. Therefore, CCs can model both hierarchical (as in simplicial and cell complexes) and set-type (as in hypergraphs) relations. For each singleton cell {s} in a CC, we set rk((s)) O, aligning CCs with simplicial and cell complexes. The rank of a cell x E X is h == rk(x). and we call a k-cell. The dimension dim(X)) of a CC is the maximal rank among its cells.
Neighbhorhood functions. Combinatorial Complexes can be equipped with notion of neighborhood among cells. In particular, we can define "neighborhood" function N : X N - P(X)) on X' as function that assigns to each cell x in collection of "neighbor cells" N(x) C X. Usually, two main types of neighborhood functions are considered (Hajij et al., 2022b): adjacencies, in which all of the elements of N Nz()) have the same rank as x, and incidences, in which its elements are from a higher or lower rank. A generally applicable choice for these functions is follows. First, up/down incidences Ni,t and N1 are defined by the containment criterion
N1,(()) = {y E (yk(() = rxx) + 1, Cyb, Nr.(()) = (v x(ck(y) = rk(x) - 1, C c..
Therefore, a & + 1-cell y is a neighbor of a k-cell a w.r.t to No, if x is contained in y: analogously, ak 1-cell is neighbor of k-cell w.r.t to Nr,1 if is contained in æ. The incidences induce up/down adjacencies NA, and NA.1 by the common neighbor criterion
(x)(()) &y e M)rk(y) = rk(z), 32 E X rk(2) = rk(x) - 1, C y, and C x), (x. Ev e ()r(()) = rk(z), Bz E X rk(z) = rk(z) +1,y C c and x C 2).
(2)
Therefore, a k-cell is neighbor of a k-cell a w.r.t to NA.t if they are both contained in a k + 1-cell 2; analogously, a k-cell y is a neighhor of k-cell w.r.t. NA,1 if they both contain a k 1-cell
3

---

Published as a conference paper at ICLR 2025
While this choice of neighborhood functions applies to any CC, other neighborhood functions more natural in specific applications. This scenario will be illustrated in Section 4.
Topological signals. A topologiaal signal (or feature) over A' is mapping :x'' + R from the set of cells X' to real numbers. Therefore, the feature vectors h E E and h, E _R" cells x and collection of F CC signals, i.e., h [fi (x). ..... fr(r) and h, (v), fr()).
Message passing networks over CCs. Let X' and CN be a CC and collection i... a set, of neighborhood functions on it. The 1-th layer of CC Message Passing Network (CCMPN) updates the embedding h2 of cell x as
h = 3
h NECN (EN(w)
(N (h' h2))
where h2 = h are the initial features, + is an intra-neighborrood permutation invariant aggre- gator, X is an neggh-neighborhood (possibly) permutation invariant aggregator, and the rank- and sethbborhooddeeeeddee
message functions (x(()(()) and the update function 3 are learnable func- tions. In other words, the embedding of cell updated in learnable fashion through aggregated messages with its neighboring cells over a set of neighborhoods, generalizing what has been done with Graph MPN (Gilmer et al., 2017c). Interestingly. the message functions can be customized to incorporate additional information, such as the orientation of the cells (Barbarossa & Sardellitti, 2020; Sardellitti et al., 2021) (if available), or the embeddings of "common" cells (if present) like x U or x ny. Finally, please notice that CCMPN in (3) are inherently able to model heterogeneous settings, as two cells can be neighbors in multiple neighborhoods, but messages among them will be dependent the neighhorhood itself. Thus, this property allows types through different (overlapping) neighborhoods. In Appendix C, we show how to model graphs, simplicial complexes, cell complexes, and hypergraphs through the CC framework, deriving the corresponding message passing architectures from the CCMPN in (3).
model different relation
Equivariances of CCMPNs. Symmetries a group with an action a G x y + y on a set V Given a function e y - 3/, it is said to be cquivariant w.r.t. the action of G if, for all x E and g E G, holds
framed as groups (Bronstein al., 2021). Let G be
e(a(g, x)) alg, e(x)).
(4)
If (((a(g,))) = e(xx), e is said to be invariant w.r.t. the action of G. As Graph MPNs (Gilmer et al., 2017c), CCMPNs are permutation equivariant too (Hajij et al., 2022b), i.e. they are equivariant w.r.t. the action of the symmetric group Sym(X) on X' In other words, CCMPNs are equivariant to the relabeling of cells the complex.
E(N) EQUIVARIANT TOPOLOGICAL NEURAL NETWORKS
Let us now introduce the setting in which nodes (0-cells) are embedded in some Euclidean space, i.e., they come with both -enmeeemttrcc and geometric features, such as positions or velocities. In the following, we assume have only positions for the sake of exposition, but velocities can be directly injected into our model as explained in Appendix D. We denote the non-geometric features with h2 Rf for cell x E X' and the position with x, E IR" for cell = E S. Since many problems exhibit n-dimensional rotation, reflection, and translation symmetries, it is desirable to design models that are equivariant w.r.t. the action of En)) (Satorras et al., 2021; Eijkelboom et al., 2023; Han al., 2022). An element of the F(nn) group is tuple (O, b) consisting of an orthogonal matrix O E Raxx (the rotation/reflection) and a vector b E R. (the translation). The action a of Fi(n) on a vector E IR" is given by
(5) As has been done for graphs (Satorras et al., 2021) and simplicial complexes (Eijkelboom et al., 2023), represent equivariance by scalarization (Han al., 2024). Therefore, geometric features undergo an initial transformation into invariant scalars. Subsequently. they are processed in a learnable way (e.g., MLPs) before being combined along the original directions to achieve equivariance.
a((O, b), x) = Ox +
E(n) Equivariant Topological Neural Networks. Let X and C. be a CC and collection of neighborhood functions on it. The 1-th layer of F(n) Equivariant Topological Neural Network
4

---

Published as a conference paper at ICLR 2025
(ETNN) updates the embeddings h/ of every cell x E X and the position x of every node 2 S as
4+1 B h
*N.nk())) (h by Inv (fx) {x (xe)))
for all x E & (6)
NECN (EN((z)
m
x- x C Σ
E
(x- - x!) (m for all 2 ES,
(7)
NECN (ES:(())N((2)
where h2 = hg, x² = x. x., and & is a scalar learnable function. Inv takes as input the positions of the nodes forming the involved cells, and it is invariant function w.r.. the action of E(n), ie.. [nv( + be (0x, + zey) = Tnv (f ((x² >eEn (x (=w)) We refer to a specific Inv as a geometric invariant. The update in (7) again naturally allows for neighhorhood heterogeneities: two nodes can be neighbors via more than a single neighborhood function, carrying different messages. This is common in many applications, illustrated in Section 4.
The following theorem demonstrates the equivariance property of ETNNS.
"nex" Theorem 1 An ETNN layer in (6)-()), synthetically denoted as (ha**) JOEx, (x(*" Jaes ETNN ((b')eex {x. bew), is E(n) equivariant, that is
(h''' (0x + bbJee ETNN ((h')eex (Ox;, + b)sez)
(8)
for all (O, E E(n).
Proof. See Appendix E.
When only (6) is applied but the update step in (7) is not performed, then ETNNS are E(n) -invaiiiariaat. At this point, it is important discuss geometric invariants. As has been shown in (Satorras et al., 2021: Eijkelboom et al., 2023), geometric invariants should make use of the underlying topological structure. For instance, the pairwise distance connected nodes has been used in EGNN (Satorras et al., 2021), while the simplex volume has been used in EMPSN (Fijkelhoom et al., 2023). However, both EGNN and EMPSN operate on rigid domains, such as simplicial complexes, while CCs can have much more arbitrary set of relations. In the following, that can be used in ETNNs, without assuming any specific structure for the CC. We consider two cells x and along their corresponding node positions (x2) ) and {x=)aeg.
present series of geometric invariants
Permutation invariant functions of pairwise distances. Any function of the form
(l|x= xx,l)
IE
a geometric invariant, with + being a permutation invariant aggregator, and r a (possibly) learnable function. A basic instance is + Σ and Id, i.e., the sum of pairwise distances.
Distances of permutation invariant functions. Any function of the form
x=
X
(10)
KEW
a geometric invariant, with + being a linear permutation invariant aggregator, and t a (possibly) learnalle function. A basic instance is + Σ and Id, i.c., the distance between centroids.
Hausdorff distance. The Hausdorff distance, defined
max
max min KEy
|x, -xell max min ||x; DEy SEE
xx1|
(11)
SEB
a geometric invariant. The Hausdorff distance measures the mutual proximity two sets whose elements live in metric space. It does so by computing the maximal distance between any point of one set to the other set. Moreover, the two max min terms inside the outer max, usually referred to directed Hausdorff distances, can be used geometric invariants themselves. This geometric
5

---

Published as a conference paper at ICLR 2025
invariant has been used in various machine learning applications, especially in the area of computer vision (Aydin et al., 2021; van Kreveld et al., 2022).
Volume of the convex hull. Any function of the volumes of the convex hulls of (x )ee and sey is a geometric invariant. Computing the volume of a convex hull is quite an expensive procedure, thus it is not always suitable to be performed at each training step. However, it can still be used if the cells have small cardinality, or if ETNNS are used in an invariant fashion, i.e., without the update in (7), because the volume needs be computed only once before training. Finally, the volume of convex hulls reduces to the volume of simplices in case the considered CC is a (geometric) simplicial complex. generalizing what has been done in (Eijkelboom et al., 2023).
Remark 1. It is noteworthy that if higher rank cells, not only nodes, come with attached geometric features, they can be seamlessly integrated into ETNNs. In that case, there would be geometric invariants of node geometric features, geometric invariants of higher-rank geometric features, and, possibly, geometric invariants of both nodes and higher rank cells. The equivariant update in (7) would be straightforwardly modified to include all the cells with geometric features in the sum.
Expressiveness of ETNN. The expressive power of neural networks over CTS is proportional to their ability distinguish non-isomorphic CTSs. As the renowned Weisfeiler-Leman (WL) test for GNNs (Xu et al., 2019), several generalized WL tests have been developed for CTSs (e.g. simplicial (Bodnar et al., 2021b), cell (Bodnar et al.. 2021a), or path (Truong Chin, 2024) complexes) and have been shown to be upper bounds on the expressiveness of the corresponding TDL models. These tools also demonstrated that TDL. models are usually more expressive than vanilla GNNs when it comes to distinguishing isomorphic graphs. For geometric graphs, the Geometric WL (GWL) test (Joshi et al., 2023) has been recently introduced as an upper bound on the expressiveness of invariant/cquiivriiatt geometric GNNs. In Appendix F, we introduce the notion of Geometric Augmented Hasse Graph and we combine with the GWL to show that ETNNS are, in general, more expressive than scalarization- based graph methods. We also perform experiments to validate our claim, including testing ETNN on the counterexample structures from (Pozdnyakov et al., 2020; Joshi et al., 2023). Next, we present the statements of our results. Proofs in Appendix F.
Proposition 2. (Informal) An ETNN is at least as powerful as an EGNN (Satorras et al., 2021) in distinguishing k-hop distinct graphs. In most of the cases, an ETNN is strictly more powerful than an EGNN. The quantitative gap between ETNNS and EGNNs depends on how the graphs are lifted into a CC, ie., how the set of cells X' and the collection of neighborhood functions C.N' are defined.
Proposition 3. There exist a pair of CCs whose nodes come with geometric features, and collection of neighborhoods such that the CCs are undistinguishable by ETNN.
Proofs. See Appendix F.
Computational complexity of ETNN. In Appendix G, we present an analysis of the computational complexity and runtime of ETNNS. In particular, we show that, if the connectivity of the CC is sufficiently sparse, which is usually the case, then the overall complexity is linear in the number of cells. In the worst case, all the cells all connected among them, resulting in quadratic cost in the number of cells. In the same Appendix G, we also empirically validate these results by comparing the runtimes of ETNN with EGNN (Satorras et al., 2021) and EMPSN (Fijkelboom et al., 2023).
4 REAL- WORLD COMBINATORIAL COMPLEXES
Designing equivariant models for CCs tailored for specific applications is an exciting direction since CCs can accommodate hierarchical higher-order features that the literature's 's most used CTSs cannot readily incorporate, including graphs, simplicial complexes, path complexes, cell complexes, and hypergraphs This section presents two vastly different applications. The first application considers molecular data. While this data has been investigated under the lens of CTS, our approach allows us to naturally integrate their higher-order structures such as rings and functional groups. Next, we present a novel TDL framework for geospatial data. We show that the interactions between geographical objects in geospatial data such as polylines and polygons can be modeled via CCs. In the next section, we will use instances of these CCs in experiments evaluating the ETNN architecture
Molecular combinatorial complexes. We introduce the notion of Molecular Combinatorial Com- plexes, showing that they can jointly retain all the properties of graphs, cell complexes, and hyper-
Appeendi C provides an overview of the aforementioned CTSs.

---

Published as a conference paper at ICLR 2025
Molecular CCs
Geospatial CCs
Rank
Rank O
Donds
Fanctional
Groups
Rank
Points
Roaos
Rankk
Rank Census Tracts
Figure 2: Real-World Combinatorial Complexes. (Left): A molecular CC where O-cells are atoms, I-cells are bonds, and 2-cell represent rings and functional groups. (Right): geospatial CC where O-cellls are grid points, 1-cells are road polylines, and 2-cells are census tract polygons. For visibility, the geospatial CC only shows one road (rank 1) and its incident lower and higher rank cells.
graphs, summarized in Appendix H, thus overcoming their mutual limitations. In a molecular CC, the set of nodes S is the set of atoms, while the set of cells X' and the rank function rk such that: atoms are O-cells, bonds and functional groups made of two atoms are 1-cells, and rings and functional groups made of more than two atoms are 2-cells (see Figure 2).
Geospatial combinatorial complexes. Geospatial CCs (GCCs) are CCs in which all cells are subsets of geospatial domain. For example, cells of GCC consist of polygons, polylines (sequences of connected lines), and points?, all which are elementary geometric objects used in geospatial information systems (Ayers, 1992). Fig. illustrates this scenario, where O-cells points where air quality monitoring units are located (e.g., in a city), l-cells are traffic roads as polylines, and 2-cells are census tracts represented as polygons, geographic areas with homogeneous demographic characteristics. Formally, GCCs are characterized by spatial representation and spatially-determet -etermined neighborhood system. Let T C TR. represent a geospatial domain, spatial representation is embedding X + P(T) mapping each cell x to subset s(x) C T. Although any neighborhoood system can be used with GCCs (such as up/down incidences), it is more natural to construct the neighborhood system of GCC from the spatial representation s(x), namely:
Na(x) (v X
rk(y) rk(x), s(x) (g) # 6)
Nr,(1)= = [u E x rk(y) = rk(x) - 1, 1, st)) # 0} Nr,((x) - (u
rk(y) rk(x) (,
/ 0).
(12)
For instance, (x)x) can represent adjacencies between geographic spaces that share a portion or a boundary, while the up and down incidences (z.) and (z.)(z) can capture the notion of road crossing a county, or two geographic areas of different rank overlapping. Evidently, more than one neighborhood function can be used or combined with the standard choices in (1) -(2) according to the application.
not always necessary to use points and polylines. One can also consider geographic divisions (polygons) with multiple resolution levels, each mapping to different rank. For example, in the United States, census tracts are smaller than zip codes, which A census tract intersect multiple zip codes, and zip codes
smaller than counties, and can belong to multiple counties, partitioning space irregularly. As an example, Fig. portrays a road polyline traversing multiple census tract polygons. Unlike classical multi-resolution spatial modeling (Kolaczyk& & Huang, 2001; Hengl et al., 2021), we don't 't require regular structures such as grids or nested partitions. More importantly, we allow each resolution have unique features. This point illustrates a crucial difference and advantage over graph modeling, which requires all data to be aggregage the same rank (typically the O-cells), see Appendix H. This work is the first to explore combinatorial topological modeling of multi-resolution irregular geospatial data, which are the backbone of geographic information systems. The dataset and hyper-local pollution
Represented as singleton sets.
7

---

Published as a conference paper at ICLR 2025
downscaling task that we will present in the next section will not only propose novel technique for learning from irregular multi-resolution geospatial data, but can be used as a benchmark for TDL.
5 EXPERIMENTS
Molecular property prediction. We evaluate our model on molecular property prediction using the QM9 dataset (Ramakrishnan et al., 2014), a comprehensive collection of quantum chemical calcula- tions for small organic molecules. includes geometric, energetic, electronic, and thermodynamic properties for 134k stable molecules with up to nine heavy atoms (C, O, N. F). See Appendix 1.1.1 for a summary.
We consider several different variants of ETNNs (see Appendix J). but the most general molecular combinatorial complexes (CCs) that we design is the one as in Fig. 2. It employs three ranks to capture different levels of molecular interactions: atoms as O-cells, ii) bonds, i.e. pairs of atoms, as 1 I-cells, and iii) rings and functional groups of size greater than two. i.c., at least triplets of atoms, as 2-cells. Similar methodologies for global connections (e.g. EGNN(Satorras al., 2021) and VN-EGNN (Sestak et al., 2024)), we generalize the notion of virtual node by incorporating a single cell. that we refer as virtual cell, containing the whole set of O-cells (the atoms), and having maximum rank, e.g. in the most general case it would have rank 3. As neighborhood functions, we employed the up/down adjacencies and incidences from (1)-(2), and "max" adjacency defined as
Aax (z) = {y E X(rk(()) = rk(z), 3a E X' rk(z)= max rk(2), C 2, and y C 2} (13)
=EX
The virtual cell together with the max adjacency in (13) acts as hub across all cells of the same rank, connecting each pair, following a common practice (Satorras et al., 2021: Eijkelboom et al., 2023). However, the max adjacency is an object of independent interest, even when the virtual cell not in the complex. In that case, would allow cells of lower ranks to communicate based on the highest order cells they are part of, e.g. atoms belonging to the same ring or functional group. In this setting, it is clear how ETNN naturally handles heterogeneity, e.g., the same pair of bonds could be connected because part of the same functional group (up adjacency) and the virtual cell (max adjacency), but the two messages will be different across the neighborhoods. Similarly, ETNN allows us to equip each rank with unique features without any ad-hoc aggregations at the atom or bond levels. We consider four distinct types of non- gere-eemmeticc feature vectors: atom features, bond features, ring features, and functional group features. We provide full list of the non-geometric feature vectors we used in Appendix I. 1.1.2. As geometric features, we use the atoms' positions. The complete list of the model parameters is in Appendix 1.3.3. A description of all the configurations and a detailed reproducibility discussion is found in Appendix J. In all the experiments, following (Satorras et al., 2021; Eijkelboom et al., 2023), we use the invariant version of ETNN.
In Table 1, we present the results for the first 6 molecular properties, from which it is clear that ETNN significantly outperforms EGNN (Satorras et al., 2021). Details and full results for all the properties are described in Appendix J. ETNN generally improves the prediction performance of EGNN, with notable improvements on a and Jr, achieving state-of-the-art performance for ja despite being a general framework not tailored for chemical tasks. To ensure further fairness in the comparison, the table shows the results of a 1.5M EGNN obtained through our framework and codebase (EGNN- MeNN-graph-W- w) as explained in Appendix J. Comparing this with the original EGNN, we observe performance improvements. Finally, the low prediction performance of ETNN-w/o VC (ETNN without the virtual cell) highlights the significant impact of the virtual cell. The reproduced EMPSN* (Eijkelboom et al., 2023) is obtained by running ETNN on a combinatorial complex being the same exact Vietoris-Rips simplicial complex of the original paper, similarly using the up/down adjacencies and incidences plus the max adjacency, and with architectural hyperparameters as similar as possible to the original architecture. See Appendix J for a detailed discussion. As the reader can see, ETNN generally matches or outperform the reproduced EMPSNs. However, ETNN employs almost two orders of magnitude less cells. partieular, the original VR complex of dimension 2 used in EMPSN has 555.25 cells per molecule on average (18 nodes, 157.9 edges, 379.36 triangles) and it is also expensive to compute in the preprocessing step, while our molecular CC (in the worst case) has 39.8 cells per molecule on average (18 nodes, 19 edges. 1.8 rings, and 1.3 functional groups) and it requires slightly more than a lookup table to be computed. As a consequence, ETNNS have half of the memory usage and almost half of the runtime per epoch w.r.t. EMPSNs, as we describe in Appendix G. This fact ultimately showcases the effectiveness of the principled inductive bias given by the molecular CC.
8

---

Published as a conference paper at ICLR 2025
Table I: Mean Absolute Error for the molecular property prediction benchmark in QM9 dataset. ETNN-* corresponds to the configurations, among the ones we tested, that produced the best results for each property. ETTNN-iinle- corresponds to the configuration, among the ones we tested, that produced the average best results across properties. ETTNN-gaph- (Satorras et al., 2021) and EMPSN (Eijkelboom al., 2023) derived as instances of ETNNS through our codebase. All the models except ETNN-w/o VC use the virtual cell. We report the improvement of ETNNN* over the original EGNN. TopNet' is the best performing variant of TopNets (Verma et al., 2024). The other baselines from (Satorras et al., 2021). Details and full results in Appendix J.
and EMPSN* are EGNN
Task Units (4)
Ae
HOOO meV
ELUMO meV
C₁
hohr
meV
D
cal/mol
NMP
.092
69
43
38
.030
.040
Schnet
.235
63
34
.033
.033
Cormorant
.085
61
34
38
.038
.026
LINet
.088
68
46
35
.043
.031
LieCony
.084
49
30
25
.032
.038
DimeNet++
.044
33
25
20
.030
.023
TFN
.223
58
40
38
.064
.01
SE(3)-Tr.
.12
53
35
33
.051
.054
Equiformer (SotA)
.046
30
15
14
.011
.023
TopNet'
.083
47
37
24
.035
.032
EGNN
.071
48
29
25
.029
.031
ETNN-
.062
45
26
22
.022
.030
ETNN- MaIN-gaaph-WW W
.067
46
27
25
.030
.036
ETNN-w/o VC
.161
73
49
48
.306
.051
EMPSN*
.061
42
25
.030
.028
ETNN-sinee ETNN-single-
.069
46
26
23
.025
.033
Improvement over EGNN
-13%
-6%
-10%
-12%
-26%
-3%
Hyperlocal air pollution downscaling. Next, introduce novel benchmark for TDL based the geospatial framework introduced in Section 4. We first provide a data an overview of the data and tasks for completeness.
The task consists of predicting PM2.5 air pollution high resolution of 0.0002 = 22m. The prediction targets of PM2.s measurements are obtained from a publicly available dataset (Wang et al., 2023) consist- ing of measurements by mobile air sensors installed on cars, correspond- ing to the last quarter of 2021 in the Bronx, NY, USA. The targets are ag- gregated the desired spatial resolu- tion. The resulting Geospatial CC con- sists of 3,946 point measurement units (0-cells), 550 roads (1-cells), and 151 census tracts (2-cell), Fig. provides an overview of the features for each rank, including demographics, land- use information, daily traffic rates and traffic composition per road, and distance to schools and other points of interest. The most important predictor for the downscaling task is the coarse-level annual PM 2.5 from the previous year (Di et al., 2019), available at a coarser km resolution which is projected to the roughly equivalent census - tract area level. In total, The full list of features and their data sources is in Appendix 1.2.2. All the pre-processing code is included in our code repository.
Table Results for the air pollution downscaling task. AEV is the average I82 (explained variance) over 30 seeds, DEV
the difference in R and MSE the Mean Squared Error.
Baseline AEV(R")
Std. Err (I²")
MSE
Linear
0.51%
0.95%
1.106
MLP
2.35%
1.61%
1.022
GNN
2.44%
0.99%
0.987
EGNN
1.43%
1.2%
1.041
ETNN
9.34%
2.05%
0.935
Baseline model comparison
Baseline
DEV (R²)
Std. Err (H²)
MSE
no virtual node no position update:
-1,0%%
2.32%
0.957
-1.37%
2.52%
0.956
invariant ETNN no geometric features:
-1.08%
2.05%
0.946
CCMPN
(b) Ablation study results
include 5 levet-level features, 2 road-level features, and 17 tract-level features.

---

Published as a conference paper at ICLR 2025
2-cell festures (cemaus tracts): PM25 intimattss, damographits, land use (eig.. Gaily traffic and traffic compositoon, etc. resiential, cominercial, parke), tlc.
(ronds); unnual averagei
(points): distance
menrest nchool and Internention, etc.
Overview of features used in the air pollution downscaling benchmark. The task is to predict the local 2M2.5 measured the O-cells, which have an approximate resolution of 22 x 22 meters. As such, this is a node regression task.
Figure 3:
We create geospatial CC using the neighborhood functions described by Eq. (12). For the geometric features, we use the coordinates of the O-cells with Mercator projection. This projection has the key property that Euclidean distance in coordinate space approximates geodesic distance in meters. As such, E (m))eequ [[n)--qqivariance is implicitly essential to the purpose of the Mercator projection. The exact coordinate values are irrelevant: What most important for our application is usefulness in expressing geodesic distances. By using these geometric features in GCCs, we allow the model to use the coordinates but only modulo Euclidean transformations, which is the original intention Mercator projection. In other words, the model can still capture location-specife effects in node-level tasks without using the coordinates directly, which would lead to poor inductive biases.
Since only CCs can support multi-level features, create point-level features for other baselines (MLP, GNN, and EGNN) by concatenating the features of each point with the features of the I 1-cell and 2-cell that the point belongs to. For the base ETNN, we again employ virtual cell defined in the previous paragraph. Since the task is at the node level, the base implementation uses the equívariant update from 1. We will perform ablation experiments over these defaults. Appendix I.2.3 presents in detail the hyper-parameter and architecture confi configurations used for the experiments
The experiment results shown in Table 2a. The results averaged over 30 seeds. Standard errors are computed with the formula o/ /n, which is recommended when having at least sample size 30. The table indicates that ETNN outperforms EGNN, GNN, and MIP, emphasizing the benefits of multi-level modeling combined with E(n. equivariance. We additionally conduct ablations studies on ETNN, presented in Table (b). The results show that, as expected, not including geometric features decreases performance. Perhaps surprisingly, the ablation of ETNN using only the geospatial CC structure without geometric features (invariants) performed slightly better than using the features but not using the position update in (7), i.e., using the invariant version of ETNN. Nonetheless, the largest decrease came from not using the virtual cell.
CONCLUSION
We introduced E(n) Equivariant Topological Neural Networks (ETNNs), a framework for designing scalarization-baat based equivariant message passing networks combinatorial complexes. Combinato- rial complexes flexibly represent arbitrary hierarchical higher-order interactions, thus ETNNS are legitimate proxy for designing equivariant models wide class of combinatorial topological spaces, such as simplicial or cell complexes, or, more in general, over combinatorial objects used to model interactions among entities, such as hierarchical graphs. We showed that ETNNs are least as expressive as existing scalarization-but based equivariant architectures for geometric graphs. Experi- ments on the QM9 molecular benchmark demonstrated ETNNs' effectiveness for richly structured molecular data. Moreover, we introduced novel geographic regression task, highlighting ETNNS ability and showing for the first time how combinatorial complexes can be used to model irregular multi-resolution geospatial data. We discuss the limitations and future directions in Appendix A. ETNNs represent, overall, a unifying framework showcasing the benefits of a principled synergy between combinatorial and geometric equivariances over beyond- gradhhhphhh domains. We believe that our methodological contribution, together with the sasy-to-use use and solid codebase we provide, can mark step in boosting and standardizing research for calaaizattion-aase equivariant TDI..
10

---

Published as a conference paper at ICLR 2025
REPRODUCIBILITY STATEMENT
We include all the details about our experimental setting, e.g. the choice of hyperparam- and the specifications of our machine, in Appendix I. The code, data splits, and virtual environment needed to replicate the experiments and easily use the ETNN frame- work are provided at the following repository: https: /tts:///gthbb. Com/NSAPH-Prooeee topo cal-call-uaaa requivar's -aariintteettte works. An in-depth discussion about the reproducibility of the molecular property prediction task is in Appendix
BIBLIOGRAPHY
Mathieu Alain, So Takao, Brooks Paige, and Mare Peter Deisenroth. Gaussian processes on cellular complexes. arXiv preprint arXiv:2311.01199 1988 2023.
Brandon Anderson, Truong-Sov Hy, and Risi Kondor. Cormorant: covariant molecular neural networks. Curran Associates Inc., Red Hook, NY, USA, 2019.
Orhun Utku Aydin, Abdel Aziz Taha, Adam Hilbert, Ahmed A. Khalil, Ivana Galinovic, Jochen B. Fiebach, Dietmar Frey, and Vince Istvan Madai. On the usage of average hausdorff distance for segmentation performance assessment: hidden error when used for ranking. European Radiology Experimental, 5(1), January 2021. ISSN 2509-9280. doi: 10. 0111/////177--220000000000
URL
http:// doi. org 10. 1188118881 01118///4177--000
00200-2.
Daniel Ayers.
Data formats for gis: A survey of popular systems, 1992.
URL
content..
Paolo Bajardi, Matteo Delfino, André Panisson, Giovanni Petri, and Michele Tizzoni. Unveiling patterns of international communities global city using mobile phone data. EPJ Data Science, (I), April 2015. doi: 10.1140/ppjss/s 13888001-000811101111 URL https: http://i..oro..11////1// epjas bggds/s13360 616888 015 0041-5.
S. Barbarossa and S. Sardellitti. Topological signal processing over simplicial complexes. IEEE Transactions on Signal Processing, 2020.
Federico Barbero, Cristian Bodnar, Haitz Sáez Ocáriz Borde, Michael Bronstein, Petar Veličković, and Pietro Liò. Sheaf neural networks with connection laplacians, 2022.
llyes Batatia, David P Kovacs, Gregor Simm, Christoph Ortner, and Gábor Csányi. Mace: Higher order equivariant message passing neural networks for fast and accurate force fields. Advances in Neural Information Processing Systems, 35:1 1423-11436, 2022.
C. Battiloro, Z. Wang, H. Riess, P. Di Lorenzo, and A. Ribeiro. Tangent bundle filters and neural networks: From manifolds to cellular sheaves and back. In ICASSP 2023 -023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5, 2023a.
Claudio Battiloro. Signal Processing and Learning Topological Spaces. PhD thesis, Sapienza Unviersity of Rome, 2024.
Claudio Battiloro, Stefania Sardellitti, Sergio Barbarossa, and Paolo Di Lorenzo. Topological signal processing over weighted simplicial complexes. In ICASSP 2023 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5, 2023b.
Claudio Battiloro, Lucia Testa, Lorenzo Giusti, Stefania Sardellitti, Paolo Di Lorenzo, and Sergio Barbarossa. Generalized simplicial attention neural networks. arXiv preprint vrXiv:2309.02/53 2023c.
Claudio Battiloro, Indro Spinelli, Lev Telyatnikov, Michael M. Bronstein, Simone Scardapane, and Paolo Di Lorenzo. From latent graph to latent topology inference: Differentiable cell complex module. The Twelfth International Conference on Learning Representations, 2024a. URL https: httts:://ppereet http:///ppeeeeeeeeeeeeeeee net /tt.://mmomooommeeaaaa

---

Published as a conference paper at ICLR 2025
Claudio Battiloro, Zhiyang Wang, Hans Riess, Paolo Di Lorenzo, and Alejandro Ribeiro. Tangent bundle convolutional learning: from manifolds to cellular sheaves and back. IEEE Transactions on Signal Processing, 2024b.
Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess E Smidt, and Boris Kozinsky. E e3u-vuirraii graph neural networks for data-efficient and accurate interatomic potentials. Nature communications, 13(1):2453, 2022.
C. Bodnar, F. Frasca, N. Otter, Y. Wang. P. Liò, G. F. Montufar, and M. Bronstein. Weisfeiler and lehman go cellular: Cw networks. In Advances in Neural Information Processing Systems, 2021a la
C. Bodnar, F. Frasca, Wang, N. Otter, G.F. Montufar, P. Liò, and M. Bronstein. Weisfeiler and lehman go topological: Message passing simplicial networks. In International Conference Machine Learning, 2021
Cristian Bodnar. Topological Deep Learning: Graphs, Complexes, Sheaves. PhD thesis, Cambridge University, 2023.
Cristian Bodnar, Francesco Di Giovanni, Benjamin Paul Chamberlain, Pietro Lio, and Michael M. Bronstein. Neural sheaf diffusion: A topological perspective on heterophily and oversmoothing in GNNs. In Advances in Neural Information Processing Systems, 2022.
Geoff Boeing. Osmnx: A python package work with openstreettmap data. https://ggttheei com/ hoom/gboengg//see osmnx, 2017. Version 1.1.1, Accessed: 2024-05-22.
Johannes Brandstetter, Rob Hesselink, Elise van der Pol, Erik J Bekkers, and Max Welling, Ge- ometric and physical quantities improve (3) equivariant message passing. arXiv preprini arXiv221 2021.
Johann Brehmer, Pim De Haan, Sönke Behrends, and Taco S Cohen. Geometric algebra transformer. Advances in Neural Information Processing Systems, 36, 2024.
M.M. Bronstein, J. Bruna, T. Cohen, and P. Veličković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104. 13478, 2021.
J. Bruna, W. Zaremba, A. Szlam, and Y. LeCun. Spectral networks and locally connected networks on graphs. In International Conference Learning Representations, Banff, Canada, 2014.
Junwu Chen and Philippe Schwaller. Molecular hypergraph neural networks. arXiv preprint 1331:333.. 2023.
Taco Cohen and Max Welling. Group equivariant convolutional networks. In International contrmaaeennloooeeeeneccc conference machine learning, pp. 2990-2992 PMLR, 2016.
Taco S Cohen, Mario Geiger, and Maurice Weiler. A general theory of equivariant enns on homoge- neous spaces. Advances in neural information processing systems, 32, 2019.
Lorenz Demey and Hans Smessaert. The relationship between aristotelian and hasse diagrams. In International Conference Theory and Application of Diagrams, pp. 213-227. Springer, 2014.
Qian Di, Heresh Amini, Liuhua Shi, Itai Kloog, Rachel Silvern, James Kelly, M Benjamin Sabath, Christine Choirat, Petros Koutrakis, Alexei Lyapustin, et al. An ensemble-base model pm2. concentration across the contiguous united states with high spatiotemporal resolution. Environment international, 130: 10499, 2019.
Francesco Di Giovanni, Lorenzo Giusti, Federico Barbero, Giulia Luise, Pictro Lio, and Michael M Bronstein. On over-squashing in message passing neural networks: The impact of width, depth, and topology. In International Conference on Machine Learning, pp. 7866-7888 PMLR, 2023.
Tobias Dichl, Thomas Brunner, Marco Tiezzi Le, and Alois Knoll. Towards graph pooling by edge contraction. In ICML 2019 Workshop Learning and Reasoning with Graph-Structtred Data, 2019.
S. Ebli, M. Defferrard, and G. Spreemann. Simplicial neural networks. In Advances in Neural Information Processing Systems Workshop on Topological Data Analysis and Beyond, 2020.
12

---

Published as a conference paper at ICLR 2025
Floor Eijkelboom, Rob Hesselink, and Erik J Bekkers. E (n) equivariant message passing simplicial networks. In International Conference on Machine Learning, pp.
PMLR, 2023.
Yam Eitan, Yoav Gelberg, Guy Bar-Shalom, Fabrizio Frasca, Michael Bronstein, and Haggai Maron. Topological blind spots: Understanding and extending topologiaal deep learning through the lens of expressivity. arXiv preprint arXiv:2408.05488 2024.
Michelle Feng and Mason A. Porter. Persistent homology of geospatial data: A case study with voting. SIAM Review, 63(1):67-99, 2021. doi: 10.1137/19M1241159, URL https: //tpss://dd... /doi.
org/10. 2tt//0..1337111 191124111222.1...
Yifan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, and Yue Gao. Hypergraph neural networks. In Proceedings of the AAAI conference artificial intelligence, volume 33, pp. 3558-3565, 2019.
Mare Finzi, Samuel Stanton, Pavel Izmailov, and Andrew Gordon Wilson. Generalizing convolutional neural networks for equivariance to lie groups on arbitrary continuous data. In International Conference on Machine Learning, pp. 3165-3176. PMLR, 2020.
Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling. Se (3)ttrassformmes:: 3d roto- translation equivariant attention networks. Advances neural information processing systems, 33: 1970-1981, 2020.
Johannes Gasteiger, Janek GroB, and Stephan Günnemann. Directional message passing for molecular graphs. In International Conference Learning Representations, 2020. URL https:// openreview aeriiiiwwee//eee WwworrriddBeeliiiii Blebbbd
J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl. Neural message passing for quantum chemistry. International Conference on Machine learning. 2017a
Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning Volume 70, ICML' 17, pp. 1263-1272. JMLR.org, 2017b.
Justin Gilmer, Samuel Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In Proceedings the 34th International Conference on Machine Learning Volume 70, ICML'17, pp. 1263-1272. JMLR.org, 2017c
C. Giusti, R. Ghrist, and D. S. Bassett. Two's company, three (or more) is simplex. Journal of computational neuroscience, 2016.
L. Giusti, C. Battiloro, P. Di Lorenzo, Sardellitti, and S. Barbarossa. Simplicial attention neural networks, 2022.
Lorenzo Giusti, Claudio Battiloro, Lucia Testa, Paolo Di Lorenzo, Stefania Sardellitti, and Sergio Barbarossa. Cell attention networks. In 2023 International Joint Conference on Neural Networks (IJCN), pp. 1-8. IEEE, 2023a.
Lorenzo Giusti, Teodora Reu, Francesco Ceccarelli, Cristian Bodnar, and Pietro Liò. Cin++: Enhanc- ing topological message passing. arXiv preprint arXiv:2306.00 035611, 2023b.
C. Wei Jin Goh, C. Bodnar, and P. Lio. Simplicial attention networks. In International Conference on Learning Representations Workshop on Geometrical and Topological Representation Learning, 2022.
M. Gori, G. Monfardini, and F. Scarselli. A new model for learning in graph domains. In IEEE International Joint Conference Neural Networks, 2005.
L. J. Grady and J. R. Polimeni. Discrete calculus: Applied analysis on graphs for computational science. Springer, 2010.
Phillip Griffiths and Wilffrid Schmid. Recent developments in hodge theory: a discussion of techniques and results. 1975.
13

---

Published as a conference paper at ICLR 2025
Sravanthi Gurugubelli and Sundeep Prabhakar Chepuri. SaNN: Simple yet powerful simplicial-awaan neural networks. In The Twelfth International Conference on Learning Representations, 2024.
URL http:///eaaaeeeeee.....reeaaaaad Eorumm21d-eUUgSS1q@33..
net Eorum?
M. Hajij, G. Zamzmi, T. Papamarkou, N. Miolane, A. Sermánn-áeezz and K. N. Ramamurthy. Higher-order attention networks, 2022a.
Mustafa Hajij. Kyle Istvan, and Ghada Zamzmi. Cell complex neural networks. In Advances in Neural Information Processing Systems Workshop on TDA & Beyond, 2020.
Mustafa Hajij, Ghada Zamzmi, Theodore Papamarkou, Nina Miolane, Aldo Guzmán-Sáene Karthikeyan Natesan Ramamurthy, Tolga Birdal, Tamal K Dey, Soham Mukherjee, Shreyas N Sam- aga, et al. Topological deep learning: Going beyond graph data. arXiv preprint arXiv:2206.00 006666 2022b.
Mustafa Hajij, Mathilde Papillon, Florian Frantzen, Jens Agerberg, Ibrahem AlJabea, Ruben Ballester, Claudio Battiloro, Guillermo Bernárdez, Tolga Birdal, Aiden Brent, et al. Topox: a suite of python packages for machine learning on topological domains. arXiv preprint arXiv:2402.024411 2024.
Jiaqi Han, Yu Rong, Tingyang Xu, and Wenbing Huang. Geometrically equivariant graph neural networks: A survey, arXiv preprint arXiv:2202.072.00 2022.
Jiaqi Han, Jiacheng Cen, Liming Wu, Zongzhao Li, Xiangzhe Kong, Rui Jiao, Ziyang Yu, Tingyang Xu, Fandi Wu, Zihe Wang, et al. A survey of geometric graph neural networks: Data structures, models and applications. arXiv preprint arXiv:2403. 00485, 2024.
J. Hansen and R. Ghrist. Toward spectral theory of cellular sheaves. Journal of Applied and Computational Topology, 2019.
Jakob Hansen and Thomas Gebhart. Sheaf neural networks, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016.
Tomislav Hengl, Matthew A. E. Miller, Josip Križan, Keith D. Shepherd, Andrew Sila, Milan Kilibarda, Ognjen Antonijević, Luka Glušica, Achim Dobermann, Stephan M. Haefele, Steve P. McGrath, Gifty E. Acquah, Jamie Collinson, Leandro Parente, Mohammadreza Sheykhmousa, Kazuki Saito. Jean-Martial Johnson, Jordan Chamberlin, Francis T. Silatsa, Martin Yemefack, John Wendt, Robert A. MacMillan, Ichsani Wheeler, and Jonathan Crouch. African soil properties and nutrients mapped at 30 m spatial resolution using two-scale ensemble machine learning. Scientific Reports, 11(1), March 2021. doi: 10.1038/<41598-0211 org/10. 1038 54155//115555 021 - 85 85639-y.
URL https://doi.
Sajjad Heydari and Lorenzo Livi. Message passing neural networks for hypergraphs. In International Conference on Artificial Neural Networks, pp. 583-592. Springer, 2022.
Abigail Hickok, Deanna Needell, and Mason A. Porter. Analysis spatial and spatiotempo- ral anomalies using persistent homology: Case studies with covid-19 data. SIAM Journal on Mathematics of Data Science, 16-)1111 2022. doi: 10.1137211M133003. URL bttps://doi..or
. 1137 1
A. N. Hirani, K. Kalyanaraman, and S. Watts. Least squares ranking on graphs, Hodge Laplacians, time optimality, and iterative methods. arXiv preprini arXiv: 2010.
Peter J. Huber. Robust estimation of a location parameter. Annals of Mathematical Statisties, 35(1): 73-101, 1964. doi: 10.1214/aomss 177703732.
Michael J Hutchinson, Charline Le Lan, Sheheryar Zaidi, Emilien Dupont, Yee Whye Teh, and Hyunjik Kim. Lietransformer: Equivariant self-attention for lie groups. In International Conference Machine Learning, pp. 4533-4543. PMLR, 2021.
14

---

Published as a conference paper at ICLR 2025
Zhixuan Jia, Yushun Fan, Chunyu Wei, and Ruyu Yan. A temporal hypergraph based method for service recommendation in the mobile internet of things-enabled service platform. Advanced Engineering Informatics, 57:102038, 2023.
Dejun Jiang, Zhenxing Wu, Chang- Yu Hsieh, Guangyong Chen, Ben Liao, Zhe Wang, Chao Shen, Dongsheng Cao, Jian Wu, and Tingjun Hou. Could graph neural networks learn better molecular representation for drug discovery? comparison study of descriptor- and graph-based models. Journal of Cheminformattie, 13(1), February 2021. ISSN 1758-2946. doi: 10. I 186/ s13321-0200004779 URL http: /xx doi. org 10. 11111//11311---11 0200--3300100000000009909 00470-0009---00
Fabian Jogl, Maximilian Thiessen, and Thomas Gärtner. Expressivity-presevvng GNN simulation. In Thirty Conference on Neural Information Processing Systems, 2023. URL https:
wwt.eiidtyttttnneeneeeeoennnnnne 9Wd.
Chaitanya K Joshi, Cristian Bodnar, Simon V Mathis, Taco Cohen, and Pietro Lio. On the expressive power of geometric graph neural networks. In International Conference on Machine Learning, pp- 15330-15355. PMLR, 2023.
Jumper, R. Evans, A. Pritzel, T. Green, M. Figurnov, O. Ronneberger, K. Tunyasuvunakool, R. Bates, A. Žídek, A. Potapenko, Highly accurate protein structure prediction with alphafold. Nature, 2021.
L. Kanari, P. Dlotko, M. Scolamiero, R. Levi, J. Shillcock, K. Hess, and H. Markram. A topological representation of branching neuronal morphologies. Neuroinfor. 16(1):3-13, 2018.
N. Kipf and M. Welling. Semi-Supervised Classification with Graph Convolutional Networks. In International Conference on Learning Representations, 2017.
Jonas Köhler, Leon Klein, and Frank Noé. Equivariant flows: sampling configurations for multi-body systems with symmetric energies. arXiv preprint arXiv:1910.0075. 2019.
Eric D. Kolaczyk and Haiying Huang. Multiscale statistical models for hierarchical spatial aggre- gation. Geographical Analysis, 33(2):95-118 2001. doi:
2001.tb00439.x URL https: : / /tt///onnnnnaaaeeeeeee wilewwewrrrr..weeeeyy........ wtt.c//coooo///ooo http///i/abbsooooo/ cctp://ob///ab//.111////. 1538-46322 2001 tb00439.... x.
Veljko Kovač, Erik J Bekkers, Pietro Liò, and Floor Eijkelboom. E (n) equivariant message passing cellular networks. arXiv preprint S4Niv:2406.034444 2024.
Junhyun Lee, Inyeong Lee, and Jaewoo Kang. Self-attention graph pooling. In Proceedings of the 36th International Conference on Machine Learning, pp. 3743, 2019.
Longlong Li, LIU Xiang, Guanghui Wang, Yu Guang Wang, and Kelin Xia. Path complex neural network for molecular property prediction. In ICML 2024 Workshop on Geometry- grounded Representation Learning and Generative Modeling, 2024a.
Geometry-grounded
Ming Li, Emmanuel Perrier, and Chengbin Xu. Deep hierarchical graph convolution for election prediction from geospatial census data. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence, pp.
2019.
Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for semi- suni-super learning. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, pp. 3538-3545, 2018
Xuan Li, Zhanke Zhou, Jiangchao Yao, Yu Rong, Lu Zhang, and Bo Han. Neural atoms: Propagating long-range interaction in molecular graphs through efficient communication channel. In The Twelfth International Conference on Learning Representations, 2024b. URL https: //tp://eeeepeeeeee///mmm net/ Eorum? / ddC.ffseeee......mmm
L. Lim. Hodge Laplacians on graphs. Siam Review, 2020.
Cong Liu, David Ruhe, Floor Eijkelboom, and Patrick Forré Clifford group equivariant simplicial message passing networks. The Tweifth International Conference on Learning Representations, 2024. URL https::////. wpwp/eeeeeveeeeeeo/e ew. net. / fo forummi 2= 9UBNOH.
15

---

Published as a conference paper at ICLR 2025
Derek Lo and Briton Park. Modeling the spread of the zika virus using topological data analysis. PLOS ONE, 13(2)e0112120, February 2018. doi: 10.1371/joumall wwo.ooolll.o.....o... pwwppp.92............. URL https: //doi. . org/10. 1371 wttp:///ouoaaa..oooo. pone. 0192120.
Hehuan Ma, Yatao Bian, Yu Rong, Wenbing Huang, Tingyang Xu, Weiyang Xie, Geyan Ye, and Junzhou Huang. Multi-view graph neural networks for molecular property prediction, 2020.
Hiren Madhu, Sravanthi Gurugubelli, and Sundeep Prabhakar Chepuri. Unsupervised parameter- free simplicial representation learning with scattering transforms. Forty-first International Conference on Machine Learning.
Kelly Maggs, Celia Hacker, and Bastian Rieck. Simplicial representation learning with neural SkS-forms, In The Twelfth International Conference on Learning Representations, 2024. URL https: http::://ppeereeeee wpwpreveeeeeeweeeeeeeeeeeeeeeeem nnt. forum? jww
D. Mulder and G. Bianconi. Network geometry and complexity. Journal of Statistical Physics, 173 (3):783-805, 2018.
New York City Department of City Planning. Primary land use tax lot output (pluto) data. https::/wwwl nyc. http///iteplaaningggg../e bolaaning/dataamaappps dor--plutoommapplutooooo page, 2024. Accessed: 2024-05-22.
New York City Department of Transportationn. Average daily traffic (aadt) data, https: //data. cityofnewyork, us us/Transportation wtwp:ttttiiioot/aaeeeeeeee Dai -aapgee--DDly----ffiiieee Traffraiic--AD--
fp86 2024. Accessed: 2024-05-22.
Marco Pacini, Xiaowen Dong, Bruno Lepri, and Gabriele Santin. A characterization theorem for equiv- ariant networks with point-wise activations. In The Twelfth International Conference on Learning Representations, 2024. URL https: http://pppneeeeeeeeeeeeee wwt.:iewee.eeeeooeeeeeeeeeeoee
21d=79FVDdfooooo wtteee//forrmmdidd999ddodeo
forum
Theodore Papamarkou, Tolga Birdal, Michael Bronstein, Gunnar Carlsson, Justin Curry, Yue Gao, Mustafa Hajij, Roland Kwitt, Pietro Liò, Paolo Di Lorenzo, et al. Position paper: Challenges and opportunities in topological deep learning. arXiv preprint arXiv:2402.088: 088771,,.88,, 2024.
Mathilde Papillon, Sophia Sanborn, Mustafa Hajjj, and Nina Miolane. Architectures of topological deep learning: A survey on topological neural networks, 2023.
Junyoung Park, Fangying Chen, and Jinkyoo Park. A molecular hypergraph convolutional network with functional group information. 2021.
A. Patania, G. Petri, and F. Vaccarino. The shape of collaborations. EPJ Data Sci., 6(1):18, 2017.
Sergey N Pozdnyakov, Michael J Willatt, Albert P Bartók, Christoph Ortner, Gábor Csányi, and Michele Ceriotti. Incompleteness of atomic structure representations. Physical Review Letters, 126(16):16600,, 2020.
Raghunathan Ramakrishnan, Pavlo O. Dral, Matthias Rupp, and O. Anatole von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific Data, August 2014. ISSN 2052-4463. doi: 10.1038/sdata.2014 2014.22. URL http: 1031 10a.//adta....... 2014.22.
Emily Ribando-Gros, Rui Wang, Jiahui Chen, Yiying Tong, and Guo- Wei Wei. Graph and hodge laplacians: Similarity and difference. arXiv preprint arXiv:2204. 12218, 2022.
T. M. Roddenberry, N. Glaze, and S. Segarra. Principled simplicial neural networks for trajectory prediction. In International Conference on Machine Learning, 2021.
T. Mitchell Roddenberry, Michael T. Schaub, and Mustafa Hajij.
Signal processing on cell complexes. In ICASSP 2022 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1852-8856, 2022. doi: 10. N0L1199/CAASSP44922222222.7
Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. The truly deep graph convolutional networks for node classification. arXiv preprint 11Xiv:1907.10903 2019. URL http: : / /arxiv. org/abs 907. 19033.
16

---

Published as a conference paper at ICLR 2025
David Ruhe, Johannes Brandstetter, and Patrick Forré. Clifford group equivariant neural networks. Advances in Neural Information Processing Systems, 36, 2024.
S. Sardellitti, S. Barbarossa,, and L. Testa. Topological signal processing over cell complexes. In Asilomar Conference on Signals, Systems, and Computers, 2021.
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. (n) equivariant graph neural networks.
In International conference on machine learning, pp. 9323-9332, PMLR, 2021.
M. T. Schaub, Y. Zhu, J.B. Seby, T. M. Roddenberry, and S. Segarra. Signal processing on higher-order networks: Livin' 'n the edge... and beyond. Signal Processing, 2021.
Kristof Schütt, Pieter-Jan Kindermans, Huziel Enoc Sauceda Felix, Stefan Chmicla, Alexandre Tkatchenko, and Klaus-Rober Müller. Schnet: continuous- convolutional neural network for modeling quantum interactions. Advances in neural information processing systems, 30, 2017.
Florian Sestak, Lisa Schneckenreiter, Johannes Brandstetter, Sepp Hochreiter, Andreas Mayr, and Günter Klambauer. Vn-egnn: E -3)-iuurrr graph neural networks with virtual nodes enhance protein binding site identification. arXiv preprint arXiv:2404.07194, 2024.
Jonathan Shlomi, Peter Battaglia, and Roch Vlimant. Graph neural networks in particle physics.
Machine Learning: Science and Technology, 2(2):021001, dec 2020. doi: 10. 10.1088/2632-2155 abbf9a. URL https:ttp/// /x doi .aaliorggg org/ 10. 088 / 2322-3322222 2113322-1533 2153 10/5////bfff 9a.
Bernadette Stolz, Heather Harrington, and Mason Alexander Porter. The topological 'shape' of brexit. SSRN Electronic Journal, 2016. doi: 10.21339sssm.2243662 URL https://eee ///oi. org/olgg///./ 2139/ 2/39//srn... 28n.288440 2843662.
Bohan Tang, Siheng Chen, and Xiaowen Dong. Hypergraph-mllp learning on hypergraphs without message passing. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 13476-13480. IEEE, 2024.
Lev Telyatnikov, Guillermo Bernardez, Marco Montagna, Pavlo Vasylenko, Ghada Zamzmi, Mustafa Hajij, Michael T Schaub, Nina Miolane, Simone Scardapane, and Theodore Papamarkou. Topobenchmarkx: A framework for benchmarking topological deep learning. arXiv preprint arXiv:2406. 06644, 2024.
Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-and translation ranslation-equivarian neural networks for 3d point clouds. arXiv preprini arXiv:/802.088 1802.08219, 2018
Quang Truong and Peter Chin. Weisfeiler and lehman go paths: Learning topological features via path complexes. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pp. 15382-15391, 2024.
Mare van Kreveld, Tillmann Miltzow, Tim Ophelders, Willem Sonke, and Jordi L. Vermeulen. Between shapes, using the hausdorff distance. Computational Geometry, 100:101817, 2022. ISSN 0925-7721. doi: cttp:///eeenneeerarr..ee//////// sciencedirect
2011.0000000002111111.1..111 URL https: www.
Yogesh Verma, Amauri Souza, and Vikas Garg. Topological neural networks go persistent, equivariant, and continuous. arXiv preprint 20Xv:2406.031644, 2024.
An Wang, Simone Mora, Yuki Machida, Priyanka deSouza, Sanjana Paul, Oluwatobi Oyinlola, Fábio Duarte, and Carlo Ratti. Hyperlocal environmental data with mobile platform in urban environments. Scientific data, 10(1):524, 2023.
Jing Wei, Jun Wang, Zhanqing Li, Shobha Kondragunta, Susan Anenberg, Yi Wang, Huanxin Zhang. David Diner, Jenny Hand, Alexei Lyapustin, et al. Long-term mortality burden trends attributed to black carbon and pm2- from wildfire emissions across the continental usa from 2000 to 2020: a deep learning modelling study. The Lancet Planetary Health, (c)))963--975 2023.
17

---

Maurice Weiler, Patrick Forré, Erik Verlinde, and Max Welling. Equivariant and Coordinate Independent Convolutional Networks. 2023. URL https: httptt////mmuuuueceeeeee wtt.///muaeeeeeeeeeeeeeee gitlab. io/ enn ht//cnnnbbooo cow..ooo//ouuvrrraaatteeeeeeeeee htwp:///wwaaattattoooooaaaaaaaaa
pdf.
B. Weisfeiler and A. Leman. The reduction of graph to canonical form and the algebra which appears therein. NTI, Series, 1968.
Ning Wu, Xiaowen Wu, Jingyuan Wang, and Daojing Pan. Learning effective road network rep- resentation with hierarchical graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, pp. 6 14, 2020.
Wenwen Xia, Yuchen Li, Jun Wu, and Shenghong Li. Deepis: Susceptibility estimation on social networks. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, WSDM '21, pp. 761-769, New York, NY, USA, 2021. Association for Computing Machinery.
K. Xu, W. Hu, J. Leskovee, and S. Jegelka. How powerful are graph neural networks? In International Conference Learning Representations, 2019.
Maosheng Yang and Elvin Isufi. Convolutional learning on simplicial complexes, 2023.
Ruochen Yang, Frederic Sala, and Paul Bogdan. Efficient representation learning for higher-order data with simplicial complexes. In Bastian Rieck and Razvan Pascanu (eds.), Proceedings of the First Learning on Graphs Conference, volume 198 of Proceedings of Machine Learning Research, pp. 13:1-13:21. PMLR, 09-12 Dec 2022. URL https: http:::://proccednngg. mll www.ppreessillle yang22a html.
Rex Ying, Jiaxuan You, Christopher Morris, Xiang Ren, William Hamilton, and Jure Leskovec. Hierarchical graph representation learning with differentiable pooling. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 4805-48151 2018.
Bowen Yu, Haonan Yin, and Zhanxing Zhu. St-unet: A spatio-temporal u-network for graph- 2019. URL http://arxiv
structured time series modeling. arXiv preprint orq/abs/ 19019 05631.
Chuxu Zhang, Dongjin Song, Chao Huang, Ananthram Swami, and Nitesh V Chawla. Heterogeneous graph neural network. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 793-803, 2019.
M. Zhang and Y. Chen. Link prediction based on graph neural networks. Advances in Neural Information Processing Systems, 2018
Wei Zhang, Hongzhi Liu, Yu Liu, Jianping Zhou, and Hui Xiong. Semi-supervised hierarchical recurrent graph neural network for city-wide parking availability prediction. In Proceedings of the 34th AAAI Conference on Artificial Intelligence pp. 1186-1193, 2020.
Zuobai Zhang, Minghao Xu, Arian Rokkum Jamasb, Vijil Chenthamarakshan, Aurelie Lozano, Payel Das, and Jian Tang. Protein representation learning by geometric structure pretraining. In The Eleventh International Conference on Learning Representations, 2023. URL https:
//openrevi New nwt.et///eeeeeeeemommmmm... forum? d=to3
Appendix Contents
Limitations Future Directions
19
Related Works
20
C Combinatorial Topological Spaces Combinatorial Complexes
20
C.I Graphs
21
C.2 Simplicial Complexes
21
C3 Cell Complexes
22
C44 Hypergraphs..
23
18

---

Published as a conference paper at ICLR 2025
D ETNNs with Velocity Type Inputs
24
E Proof of Theorem
24
Expressiveness of ETNNS
25
Complexity Analysis
31
H Additional References on Real- World Combinatorial Topological Spaces
33
H.1
Molecular Data
33
H.2
Geospatial Data
33
I Experimental Setup
34
LI
Molecular Property Prediction
11.1
Dataset Statistics
34
1.1.2
Cell Features
I.1.3
Parameter Configuration
35
1.2
Hyperlocal Air Pollution Downscaling
36
1.2.1
Dataset Statistics
36
1.2.2
Cell Features
36
I,2.3
Parameter Configuration
37
I.3
Geometrie Features
J Ablation Study and Reproducibility for Molecular Property Prediction
38
Architecture Description
40
A LIMITATIONS & FUTURE DIRECTIONS
To our knowledge, ETNN is the first unifying framework for scalarization-based E(n) equivariant networks over combinatorial topological spaces or, more in general, over combinatorial objects used to model interactions among entities. In this work, we introduced ETNNS as a significantly more general. expressive, and flexible framework compared prior geometric/tooological deep learning models operating geometric graphs (Satorras et al., 2021) simplicial complexes (Fijkelboom al., 2023). The promising results open several avenues.
Methodological. First, while we introduced several useful invariants like pairwise distances, cen- troids, Hausdorff distances, and convex hull volumes, there are likely many other meaningful geometric quantities that could be leveraged, especially for specific application domains. Therefore, developing a more comprehensive set of geometric invariants, as well as learning methods to discover them from data automatically, is an important future direction. In this sense, generalizing the approach from (Liu et al., 2024) integrating Clifford group-equivariant layers with message passing layers. i.e., going beyond scalarization, possible solution, although requiring to solve possible scalability issues. Second, extending our framework to make it scale invariant could be useful in geospatial tasks. Third, at this stage ETNNS cannot directly handle time-varyee varying scenarios, thus aim to go beyond the static settings and develop dynamic or temporal variants of ETNNs. This could enable new applications in areas like spatio-temporal forecasting, and physical simulations. Fourth, and finally, ETNNs assume that (potentially) each rank should satisfy the same (n)) equivariance, but another interesting direction is studying the setting in which each rank is required satisfy different symmetries, pointing towards a principled use of products of groups.
Computational. While our experimental validation of ETNNS covered both supervised inductive and semisupervised transductive tasks, future works could tailor ETNNs to fit challenging specific applications scale. this sense, unsupervised or self-supervised training methods for ETNNs could enable the learning of meaningful representations in data-driven manner without requiring labeled data for every task. This could allow ETNNS to better leverage large, unlabeled geometric datasets. Moreover, though computationally tractable, ETNNs may benefit from further improvements to effectively scale on very large datasets. The main bottlenecks are MP operations and geometric invariants computation. A possible solution could be neighbor samplers (Zhang et al., 2019),
19

---

Published as a conference paper at ICLR 2025
developing Enn) equivariant message passing- fasseng-free ETNNS, building works like (Maggs et al., 2024).
B RELATED WORKS
Topological Deep Learning. TDL is informed by foundational work in Topological Signal Processing (TSP) (Barbarossa & Sardellitti, 2020; Schaub et al., 2021; Roddenberry et al., 2022; Sardellitti et al., 2021), highlighting the value of analyzing multi-way relationships in data. The works in (Bodnar et al., 2;a) extended the Weisfeiler-Lehmatr graph isomrrphism test to simplicial and regular cell complexes (Bodnar et al., 2021a), respectively, showing that message passing over these spaces more expressive than message passing over graphs. Convolutional (Ebli et al., 2020; Yang et al., 2022; Hajij et al., 2020; Yang & Isufi, 2023; Roddenberry et al., 2021; Hajij al., 2022a) and attentional architectures (Battiloro et al., 2023c; Giusti et al., 2022: Goh et al., 2022; Giusti et al., 2023a) over simplicial and cell complexes have been introduced as well. Additionally, the application of message passing or diffusion on cellular sheaves (Hansen & Ghrist, 2019) over graphs (Hansen & Gebhart, 2020; Bodnar et al., 2022: Battiloro et al., 2023a; 2024b; Barbero al., 2022) has proven effective in heterophilic scenarios. Models without message passing have been introduced for simplicial complexes (Madhu et al.: Gurugubelli & Chepuri, 2024; Maggs et al., 2024) and hypergraphs (Tang et al., 2024). An architecture for inferring latent regular cell complex to improve a downstream task has been introduced in (Battiloro et al., 2024a). Gaussian Processes over cell complexes have been introduced in (Alain et al., 2023). Message passing networks over path complexes have been introduced in (Truong& & Chin, 2024; Li et al., 2024a). An extensive review of TDL is available in (Papillon et al., 2023). A comprehensive framework for TDL was outlined in (Hajij et al., 2022b), proposing combinatorial complexes and message- message-passing networks (CCMPNs) over them. A general topological framework for expressivity analysis of CCMPNs has been proposed in (Eitan et al., 2024). Finally, software for neural networks on combinatorial topological spaces has been presented in (Hajij et al., 2024; Telyatnikov et al., 2024).
Equivariant Neural Networks The effectiveness of Convolutional Neural Networks (He et al., 2016), and their first group extension to the SO(2) group (Cohen & Welling, 2016) initially demonstrated the advantages of equivariant models, and paved the way for a series of works exploring how to design models respecting some symmetries (Weiler et al., 2023). Loosely speaking, some of these works employ natively equivariant function spaces (Thomas et al., 2018; Fuchs et al., 2020), or port the spatial space into high-dimensional spaces (Cohen et al., 2019; Finzi et al., 2020; Hutchinson et al., 2021; Batatia et al., 2022). An alternative strategy is designing neural networks (Köhler et al., 2019; Schütt et al.. 2017: Brandstetter et al., 2021; Batzner et al.. 2022) that perform equivariant operations directly within the original space, including the (VN-)EGNN from (Satorras et al., 2021; Sestak et al., 2024). A characterization theorem for equivariant networks has been presented in (Pacini et al., 2024). Recent works showed how achieve equivariance by leveraging neural network architectures operating on the Clifford algebra (Ruhe et al., 2024; Brehmer et al., 2024). Each of these approaches has limitations and advantages, and the effectiveness of a specific design choice is related to the considered application and computational constraints (Han et al., 2022).
Our paper builds on both classes of works. To the best of our knowledge, the most relevant works merging TDL models and E'(n) equivariant neural networks are (Eijkelboom et al., 2023; Liu al., 2024; Verma al., 2024). In partieular, (Eijkelboom al., 2023) generalizes the approach from (Satorras et al., 2021) and introduces E(n) Equivariant Simplicial Message passing Networks (EMPSNs). The framework in (Verma et al., 2024) injects information about persistent homology (PH) on top of (equivariant) simplicial message passing networks, thus interestingly combining TDL and Topological Data Analysis. Finally, (Liu et al., 2024) adopts the approach from (Ruhe et al., 2024) by combining Clifford group equivariant layers with simplicial message passing networks. However, no other combinatorial topological spaces are considered, and, overall, a general framework for designing equívariant neural networks over combinatorial topological spaces is missing. Here address this important gap by introducing our ETNNS.
C COMBINATORIAL TOPOLOGICAL SPACES AS COMBINATORIAL COMPLEXES
In this appendix, we revisit a subclass of mathematical objects (not always formal topological spaces) that can be described combinatorially, message passing networks operating them, and how they
20

---

Published as a conference paper at ICLR 2025
Figure 4: A simplicial complex of order 2, comprising of nodes (cells of rank 0), edges (cells of rank 1), and triangles (cells of rank 2).
can be cast in the ETNN frameworks. As we did for CCs, we will often overload the notation X' in the following subsections, incurring some notation abuses for the sake of exposition and consistency.
C.I GRAPHS
The first combinatorial objects we describe are obviously graphs. They are remarkably useful in several fields, across several tasks, thanks their ability to encode prior data knowledge, i.e., their networked structure.
Graphs. graph X' is a pair (S,E), where the set of nodes, and is the set of edges (i.e., pairs of nodes).
Neighbhorhood Functions Graph. Given a graph X, for all x E S, the up adjacency Na.t (i.e., the usual node adjacency) is defined as
(x) = {u E S E & x C = and y C =).
(14)
Graphs as Combinatorial Complexes. Graphs can be easily cast as combinatorial complexes. Let us denote the set of singletons = {{ a graph is a combinatorial complex (S,X, rk), where S is the set of nodes, the set of cells X' = is the set of singletons, and the rank function rk assigns rank 0 to each singleton. Please notice that there is no need to include edges as cells in the complex, differently from, e.g., simplicial complexes, as detailed in the next subsection, because CCMPN as in (3) with CN = (Wa,1) recovers the standard Graph Message Passing Network from (Gilmer et al., 2017b). An ETNN as in (6) with the same collection of neighborhoods C. recovers the En) Equivariant Graph Neural Network (EGNN) from (Satorras et al., 2021).
C.2 SIMPLICIAL COMPLEXES
Graphs capable of explicitly managing only pairwise interactions among nodes. However, in numerous applications (Giusti et al., 2016; Kanari et al., 2018; Patania et al., 2017; Sardellitti et al., 2021), including the ones presented in this work, higher-order multiway interactions need to be taken into account. Simplicial complexes are one of the most intuitive objects to start incorporating them.
Simplicial Complexes. Given finite set of nodes S, k-simplex is subset of S with cardinality k ++1. 1. A face of x is a 1-simplex being a subset of it, thus a k-simplex has k + 1 faces. A coface of is (k -i-sippeex being superset of (Barbarossa & Sardellitti, 2020; Lim, 2020). A simplicial complex (SC) X' of order K is a collection of k-simplices x. k = 0,..., K such that, if a simplex æ belongs to X. then all its subsets y x also belong to the complex (inclusivity property). In most of the cases, the focus is on complexes of order up to two, thus having a set of nodes, a set of edges, and set of triangles. To give a simple example, Fig. we sketch simplicial complex of order 2.
Neighbhorhood Functions in a Simplicial Complex. Given a SC X', the coboundary and boundary neighborhood functions NC and Nn. respectively, are defined, as
Nx(x) = fy E X' fy is coface of z), NB(x) = {u E X is a face of x}.
(15)
The up and down adjacencies NA. and NA.1, respectively, are defined,
(x) (y Xlyy shares coface with z), (x. {y E Xly shares face with z). (16)
21

---

Published as a conference paper at ICLR 2025
Simplicial Complexes as Combinatorial Complexes. It is easy to frame SCs in the CC framework. In particular, a simplicial complex X' is a combinatorial complex (S, 2. rk), where S is the set of nodes, the set of cells X' is the set of simplices, and the rank function rk assigns to each cell its cardinality minus one. A CCMPN as in (3) with C. = (Nc.NB.M V.,,N. 1, NA.1) recovers the Message Passing Simplicial Network from (Bodnar et al., 2021b). An ETNN in (6) with the same collection of neighborhoods C.V' recovers the En) Equivariant Message Passing Simplicial Network (EMPSN) from (Bijkelboom et al., 2023).
C.3 CELL COMPLEXES
Simplicial complexes are capable of managing relationships of varying degrees, yet the inclusion property often binds them. This property mandates that if a set is part of the space, then all its respective subsets must also be included in that space. In numerous applications, this restriction is not only unnecessary but also lacks justified basis. Consequently, cell complexes are frequently employed instead (Mulder & Bianconi, 2018; Hirani et al., 2010). These structures are versatile and support hierarchical organization.
Regular Cell Complexes. A regular cell complex (or regular CW complex) a topological space X together with a partition [Ka)=EHs of subspaces X2 of X' called cells, where Hx is the indexing set of XX such that
1. For each æ E X, every sufficient small neighborhood of x intersects finitely many Xzi
2. For all z.y we have that X₂ n Xy y * Ø iff X₂ C X where Xy is the closure of the cell;
3. Every Xr is homeomorphic to g& for some k;
4. For every x E Hx there is a homeomorphism g of closed ball in RR to Xx such that the restriction of g to the interior of the ball is a homeomorphism onto Xz.
From 2. Hx has a poset structure, given by S. x iff X, C Xx, and we say that bounds x. This known as the face poset of X. From 4, all of the topological information about X' is encoded in the poset structure of Hxx Then, regular cell complex can be identified with its face poset. From now on we will indicate the cell X with its corresponding face poset element z. The dimension or order dim(a) of cell a is k and we call it a cell. The dimension or order of a regular cell complex is the largest dimension of any of its cells. In the following, we will refer to regular cell complexes simply cell complexes or CW complexes. Cell complexes can be described via an incidence relation (boundary relation) with a reflexive and transitive closure that is consistent with the partial order introduced in the above definition. In particular, we have the boundary relation y < iff dim(y) (y) I dim (z) and there is no cell 2 such that y S 2 S x. In other words, the boundary of a cell x of dimension k is the set of all cells of dimension less than & bounding x. The k-skeleton of regular cell complex X' is the subcomplex of X' consisting of cells of dimension at most k. Usually, CW complexes X' are constructed through hierarchical gluing procedure starting from set of nodes S (Hansen & Ghrist, 2019; Bodnar et al., 2021a). Therefore, we can interpret the O-skeleton of a cell complex X' as the set S of O-cells (nodes) and the 1-skeleton as the set of nodes together with the set of 1-cells (edges), thus a graph. For this reason, given a graph G. i.e., realization of an order-1 1 cell complex, it is possible to lift to an order-2 cell complex X' whose 1-skeleton is isomorphic to G. A way to do it is by attaching order 2 cells to the/a set/subset of simple/induced cycles of G. Cells of order greater than 2 can be attached as well using the same gluing procedure mentioned above (Sardellitti et al., 2021: Grady & Polimeni, 2010), but there is usually little interest in them. In Fig. 5, we sketch an order 2 CW complex.
Remark 2. It is easy to notice that a CW complex considering cycles of length up to is equivalent to a simplicial complex of order 2 having the same set S of O-simplices (nodes), the same set of I-simplices (edges), and the same set of 2-simplices (triangles). In general, simplicial complexes are special cases of CW complexes for which any k-cell is composed of exactly k nodes.
The reader may have noticed that we gave no explicit characterization of a simplicial complex topological space, although it is pretty easy to particularize cell to simplicial complexes. This is because our definition of SC in Sec. C.2 is, to be completely rigorous, the definition of an abstract simplicial complex, that itself is not topological space. However, when we refer to simplicial complex as a topological space, we are implicitly talking about the geometric realization of an abstract simplicial complex. Therefore, we are implicitly stating that the SC has topology
Remark 3.
22

---

Published as a conference paper at ICLR 2025
Figure 5: A CW complex of order 2, comprising of nodes (cells of rank 0), edges (cells of rank 1), and cycles (cells of rank 2).
coherent with its simplices. which are "topologized" as homeomorphisms of the standard simplices living in Euclidean space. Please refer to (Barbarossa & Sardellitti, 2020) for detailed but accessible description. We will not incur any ambiguity by referring to abstract simplicial complexes as just simplicial complexes.
Neighhhorhood Functions in Cell Complex. Given CW complex X' the coboundary and boundary neighborhood functions Ne and NB, respectively, are defined, as
Nc(x) = ty X < x), NB(2) = {v E X| < y).
(17)
The up and down adjacencies NA.t and MA.1, respectively, are defined,
(x)((x)= yy E x13z such that 2, 2}, (T) {y E 22 such such that z - x, < y)
(18)
Cell Complexes as Combinatorial Complexes. It is again straightforward to frame CW complexes in the CC framework. In particular, cell complex combinatorial complex (S, X,, rk). where S is the set of nodes, X' is the set of cells, and the rank function rk assigns to each cell its dimension dim. A CCMPN as in (3) with CA (WB, (WB.NAa NN.tt) on a CW complex recovers the Molecular Message Passing Network from (Bodnar et al., 2021a). ETNNs in (6) with the same collection of neighborhoods C.N give rise to E(n) Equivariant Molecular Message Passing Networks, not present in literature?.
Remark 4. Simplicial Complexes and Cell Complexes rich topological objects. Neighborhoods in a simplicial or a cell complex are strongly rooted in arguments from algebraic topology. In particular, Hodge theory (Griffiths & Schmid, 1975) used derive formal notions of boundary, coboundary, and Hodge Laplacians (Barbarossa & Sardellitti, 2020). More detailed descriptions can be found in (Grady & Polimeni, 2010; Barbarossa & Sardellitti, 2020; Battiloro al., 2023b; Ribando Gros et al., 2022). In this work, we are interested in the definition of cells (in the CC sense) and neighborhood functions that these objects (SCs and CWCs) induce. For this reason, reader familiar with algebraic topology could find our definitions and treatment not totally comprehensive. However, we aim to provide unifying framework in the context of (equivariant) TDL, thus our presentation sufficiently consistent and strikes good tradcoff between generality, rigor, and applicability.
C.4 HYPERGRAPHS
Cell complexes have their formal hierarchical and gyolloy-ggrounder characterization as their main strength. However, their structure is still constrained by the homeomorphism requirements, preventing them from modeling arbitrary higher-order hierarchical interactions. On the other hand, hypergraphs combinatorial objects that have total flexibility, but without any hierarchical and (in the general case) topological structure.
Hypergraphs. A hypergraph (HG) on nonempty set of nodes S is pair (S, where X is subset of the power set PSS) [0] of S. Elements of are called hyperedges. In Fig. 6, we sketch hypergraph with nodes and hyperedges.
This approach was recently explored in (Kovač et al., 2024), appeared online only after released the first preprint of ETNN.
23

---

Published as a conference paper at ICLR 2025
Figure 6: hypergraph, comprising of nodes (cells of rank 0), hyperedges (cells of rank 1).
Neighbhorhood Functions in Hypergraph. Let again denote the set of singletons $ {s)]eek Given a HG X, for all x E SUE, the up and down incidences Nr.1 and Nr. respectively, are defined as
(x)(2)= (u E SUElE c.), N1,((=) S
(19)
Therefore, if node, the down and up incidences evaluated at x are the empty set and the hyperedges containing respectively. If x is a hyperedge, then the down and up neighborhood functions evaluated at æ are the nodes contained in x and the empty set, respectively.
Hypergraphs as Combinatorial Complexes. Even in this case, framing HGs in the CC framework straightforward. particular, a hypergraph is combinatorial complex (S, X, rk), where S the set of nodes, the set of cells X' = S U &, and the rank function rk assigns rank O to the nodes and rank 1 to the hyperedges. A CCMPN in (3) with CN (Ni.t: recovers the Hypergraph Message Passing Networks from (Feng et al., 2019) (in the convolutional realization) and (Heydari Livi, 2022). ETNNs as in (6) with the same collection of neighborhoods C.N' give rise E(n) Equivariant Hypergraph Message Passing Networks, not present in literature.
D ETNNS WITH VELOCITY TYPE INPUTS
As EGNNs, ETNNs can handle settings in which nodes come with velocity type geometric features. In this case, we have non-geometric feature hg E TRF,, and positions x, E IR" and velocities v, for cell S. We simply need to modify the position update in (7) to include the velocity update. Let us denote the velocity of the z-th node at layer with v At the (-th layer, an ETNN updates the embeddings hi of every cell x X' in (6), and the position x/. and velocity v of every node z E S as
v < v. +
Σ
Σ
(x xx) (m,)),
x x +
(20)
NECN 1EC:)ES::)))NN
E PROOF OF THEOREM 1
The proof of Theorem is straightforward generalization of the proof provided for Equivariant Graph Neural Networks (EGNN) in (Satorras et al., 2021). I). In particular, we can directly state that the update of the features in (6) is E(n) invariant because the positions {x: are used only input to the geometric invariant(s) Inv and the features h, contain no geometric information, i.c.
h
h!
veN.ak(e) (h',, Inv ((Ox! + b) sem (Ox² b) b)
(21)
WECA (E)(()
for all E X' and (0, b) F(n). Therefore, we just need to show that the update of the positions in (7) is (n) equivariant, i.c., we want to prove that
Ox 1+1 Ox++1 +b= Ox + C Σ E
((0x) + +b) (Ox² b)) (m,))))
(22)
WECN (E)
24

---

Published as a conference paper at ICLR 2025
for all 2 E S and (0, E(n). By direct computation, as showed in (Satorras al., 2021),
can write
Ox +
Σ
Σ
((Ox) + +b) - -Ox; + b)) (m
IECNNIE::1)) (eS:(t))E
= Ox +b+0C
Σ
E
(x--x() x1) (m
(ES))E))))))
= 0
x x
E
(x xx) & (m
+b= Ox +b=0x + b.
(23)
NECN (t)))
Therefore, the proof is concluded the updates in (6)-(7) are E(n) invariant and equivariant, respectively.
The proof is slightly different if velocities va are available. In this case, we want prove that the updates in (20) are (n)) equivariant. Therefore, for the velocity update, we want to show that
Ov |+1 (h²) Ov + +C
Σ E ((Ox), +b - (Ox) b)) E (m,)), (24)
(ES:)))E)))))
for all 2 S and (O,b) E(n). Again by direct computation, we can write
(hh) 0v + Σ
E
((0x) +b) (ox² + b)) (m
NECN (ES)())E(())
OC (h/) v2 + OC Σ
E
-x-xx)) & (m?))
VECNIES:(1)ENN
=0
(bi)) v (()
E NECN (ES:(()EN(ss)
Σ
E
(x- x+) (mm))))
= Ov!t!.
(25)
It is trivial to see that the position update in (20) is Enn) equivariant. Therefore, the proof is concluded as the updates (6)- are F((n) invariant and equivariant, respectively.
F EXPRESSIVENESS OF ETNNS
The ability of Graph Neural Networks (GNNs) to differentiate between non-isomorphic graphs is typically employed as an expressivity metric. In particular, it is well known that vanilla GNNs are at maximum as powerful as the Weisfeiler-Leman (WL.) test (Weisfeiler & Leman, 1968) in discriminating isomorphic graphs. As such, the WL test sets limit on the expressiveness of GNNs. To broaden the use of this method include geometric graphs, the Geometric Weisfeiler-Leman test (GWL) has been introduced (Joshi al., 2023). This test evaluates whether two (undirected) graphs are geometrically isomorphic, as defined next.
Geometrically Isomorphic (roshs.(Joshi et al., 2023; Sestak et al., 2024) Two geometric graphs = E,) and G2 (82,E) with [S]] = [S2], non-geometric and geometric i-th node features he and x (j E 2). are called geometrically isomorphic if there exists edge-preeerving C;
bijection b I + Iz between their corresponding node indices set I, and Iz, such that their positions are equivalent up to the E(n) group action, i.c.:
G2 Faca)
Go Face)
h Ox +b
(26)
for all (O, E(n).
k-h Distinct Graphs.(Joshi al., 2023; Sestak al., 2024) Two geometric graphs G1 and G2 said to be k-hop distinct if for all graph isomorphisms b, there is some node E Iy, b(i) e In such that the corresponding k-hop neighborhood subgraphs are distinct (not geometrically isomorphic). Otherwise, if they are identical up to E(n) actions for all i E In we say T1 and G2 are said to be k-hop identical.
To study Combinatorial Complexes (CCs) and the expressiveness Equivariant CCMPNs (ETNNs), introduce the notion of Geometric Augmented Hasse Graph representation of CCs, generalizing
25

---

Published as a conference paper at ICLR 2025
BR
(A
(b)
Figure 7: (a) A combinatorial complex with nodes of the underlying graph as cells of rank 0, and with three arbitrary cells rank 1. (b) The corresponding geometric augmented Hasse graph N1
CN = t: N1,1, with NA.1 as in (14), and + is the mean. This is also an example of skeleton-preseriing lift.
(Hajij et al., 2022b) and the notion of Hasse diagram for finite partially ordered sets (Demey Smessaert, 2014) to the geometric setting.
Geometric Augmented Hasse Graph Combinatorial Complex, Let XS, be a CC where each node E S comes with positions Xg, and CN collection of neighborhood functions. The Geometric Augmented Hasse graph Gx of (S, X, rk) a (possibly) directed geometric graph Gx = (X, with cells as nodes, edges given by
=(,,))| X, EN :
(27)
and positions of cell being a linear permutation invariant function Dzea X₂ of its corresponding node positions (x. h>ea, S. Therefore, the geometric augmented Hasse graph of CC is graph representation of obtained by considering the cells as nodes, by inserting edges among them if the cells are neighbors in the CC, and by assigning positions to higher-order cells as functions of the nodes (of the CC) they contain. Fig. 7 shows an example
CC and the corresponding geometric augmented Hasse graph. Given CC C, standard graph cressage-passingg network that runs over the augmented Hasse graph Gx induced by a collection neighborhoods CN is equivalent to a specific CCMPN derived particular case of (3) and running X' using C. (Hajij et al., 2022b). The following also holds.
Proposition 1 Let (S, X, rk) be a CC where each node E S comes with positions X C. collection of neighborhood functions, and Gx the corresponding geometric augmented Hasse graph. Assuming that:
1. a single message function for all neighhorhoods and ranks, i.e,, viN (Wrk(()) = 1/ in (6);
2. the inter- and intra-neighhorhood aggregation functions the same, i.e., + in (6);
only employed geometric invariants are sum of distances of linear permutation invariant functions in (10), e.g. sum mean.
Then ETNNs over (S, X' rk) are equivalent to EGNNs (Satorras et al., 2021) over Gx with the following modified update for positions of node æ (of the Hasse graph):
X ENECN EIES:OEENL) (xt, -x+) (ma,), i S, x 1+1 OPEr x/41 E
(28)
Proof. The proof is trivial and obtained by simple direct substitution in (6)-(7).
Intuitively, the modified update is required because higher-order cells do not come with positions, thus they cannot be updated as the positions of the nodes S.
Lifting of Graph into CC. Given graph, many ways exist to lift it into combinatorial complex. As shown in Appendix C.I, denoting the set of singletons S {{8)) graph g(S, E) is
26

---

Published as a conference paper at ICLR 2025
combinatorial complex (S, XX, rk), where S is the set of nodes, the set of cells X' = is the set of singletons, the rank function rk assigns rank 0 to each singleton, and the up adjacency from (14) is employed. However, we can enrich the representation by including the edges & as cells, i.e., X =USUE, and assign rank 1 to them. In this setting, it is possible to make nodes communicate with edges and vice-versa, rather than just making nodes communicate through edges. In general, a natural question is how to define higher-order cells, i.e., cells of rank greater than O. We have already shown in Section 4 that domain knowledge can be injected by properly modeling higher-order cells. However, we are now interested in understanding which conditions should higher-order cells respect to result in an increased expressive power of ETNN. To perform a fair analysis, we consider only lifts (Bodnar et al.. 2021a), i.c. lifts in which the original connectivity among the nodes in S is preserved. In other words, we don't directly rewire the underlying graph G. As shown in Fig. 7, a straightforward example of skeleton-presee functions without necessarily adding the edges as cells. Another example is choosing (1)-(2) as neighborhood functions including the edges as cells.
dacleton-preservve
lifts is choosing (1)-(14) as neighborhood
Expressivity of ETNN. Due to Proposition 1, given two geometric graphs 9n and G2 and two combinatorial complexes Kg, and X2₂ obtained by respectively lifting them, we can study how expressive ETNNs are by analyzing how the GWL performs on the geometric augmented Hasse graphs Gxo, and Gxg, of Kg, and Xgz. respectively, w.r.t. how it would perform directly on the underlying graphs G1 änd G2- Since the GWL operates on undirected graphs, we need assume that the produced Hasse graph is undirected. The two examples of skeleton- preserving lifts clearly result in undirected Hasse graphs. We would obtain a directed Hasse graph if, e.g., only one of the up/down incidences in (1) is used as neighborhood function. Similar to the standard graph WL test, the GWL method updates node colors iteratively based on the features of nodes in the local neighborhood. Additionally, GWL maintains E(n) -qquivvrarr hash values that capture each node's 's local geometry. Therefore, any k-hop distinct, (I: 1)-hop identical geometric graphs can be distinguished by iterations of GWL, i.e., when the updated hash values that capture each node's local geometry differ across the two graphs for the first time, or by & layers of an EGNN, i.e., when the geometric invariants differ across the two graphs for the first time (Joshi et al., 2023).
Proposition 2. Assume to have two k-hop distinct geometric graphs 91 = (St, and G2 = (S2, E2) where the underlying graphs are isomorphic in the standard sense, two combinatorial complexes Xg. and X2₂ and a collection C.N' of neighborhood functions obtained via Akkeleton-preserviii lift and leading to undirected geometric Hasse graphs Gxe, and Gxon respectively. An ETNN operating over Xg, and Xg, respecting Assumptions 1-3 can distinguish 91 and G2 in M layers, where M is the number of layers required to have at least one cell in Xg, whose receptive field is the whole set of nodes in /9/
Proof. To prove the result, we study the behavior of the GWL on the geometric augmented Hasse graphs 9xo, and Ixe, of Xg, and Aç., respectively. For 1 1-hop distinct graphs, one iteration of GWL suffices to distinguish them even without any lifting into a CC and, thus, the proposition holds.
Now, let us assume that Gi and G2 are k-hop distinct and (k- hh-hop identical for any k >> 1, E N. As direct consequence of Proposition 1, M GWL iterations are required for (at least one) node of the Hasse graphs Gxo, /Gx;, to have the entire set of nodes S, in its receptive field. Denote that node with x; E G1 and Tb(1) E G2, where is any isomorphism between Gj and Go. Due to the k-hop distinctness of the graphs, the hash values corresponding to node 2; E G1 and Za(3) G2 will differ in the M-th iteration, thus M iterations of GWL can distinguish the graphs. Therefore. Combining the fact that EGNNs are as powerful as the GWL in distinguishing k-hop distinct graphs (Sestak et al., 2024; Joshi et al., 2023) with the result from Proposition 1, the proof is completed.
Corollary 1. An ETNN in Proposition 2 is strictly more powerful than an EGNN (Satorras et al., 2021) in distinguishing k-hop distinct graphs Gi and G2 if the employed skeleton-preserving lifting is such that the number of ETNN layers required have least one cell in Xg, whose receptive field is the whole set of nodes in G1/G, is smaller than the number of EGNN layers required to have at least one node in /9 whose receptive field is the whole set of nodes in /9/
Proof. Direct consequence of Proposition 2.
27

---

Published as a conference paper at ICLR 2025
nodes
nodes
Figure 8: k-chain geometric graphs, i.e., i aligned nodes and two end nodes with opposite orientations.
Remark 5 The reader may have noticed that, in proving Proposition 2, we used the result from Proposition ignoring the modified position update rule in (28). i... implicitly assuming that also the positions of higher-order cells, when seen as nodes of the corresponding augmented Hasse graph, are updated as in (7), following the standard EGNN. This not relevant, as the modified update rule in (28) is actually more powerful than the standard rule in (7), because the positions of the higher-order cell æ at layer/ / is updated using the already updated node positions Dz x!+1 and not the x For this reason, Proposition 2 is somehow pessimistic scenario for the practical use of ETNNs, and there could be corner cases in which ETNNS can distinguish k-hop distinct geometric graphs with a lower number of layers than prescribed. Finally, in practice, we observed it is not important to stick to the homogeneous setting (Assumption 1), to a unique aggregation function (Assumption 2), or to a certain geometric invariant (Assumption 3). These assumptions are useful to simplify the tractation and have a perfect match with the geometric augmented Hasse graph formalism.
Proposition 2 provides formal hints about how different lifts of graphs into combinatorial complexes can impact the expressive power of ETNNs. Our result can be seen as a proper generaliza- tion of Proposition 2 in (Sestak et al., 2024) about the expressivity of VN-EGNNs. Indeed, adding a virtual node connected to all the other nodes can be interpreted as a combinatorial complex which the cells are the set of singletons (the nodes) plus cell containing all the nodes (the virtual node). A (simplified) variant of a VG-EGNN is then obtained by using the adjacency induced by the edges of the original graph and the up/down incidences neighborhood functions. In this case, as prescribed from Proposition 2, the receptive field of the virtual node is the whole graph by definition, making VN-EGNNs able to distinguish k-hhpp distinct graphs in one layer. Our results also go in the direction of (Jogl et al., 2023), in which the expressivity of standard graph message passing networks on transformed/litted graphs is systematically studied. This work provides first, preliminary intuition on how the framework from (Jogl et al., 2023) could be extended to the geometric graph setting. Moreover, is worth noting that our expressivity analysis could be enriched through the same GWL framework from (Joshi et al., 2023), eventually leading to appologica/higghe-ootoe of it. An interesting direction is studying the expressivity of ETNNS when the lifts result in directed geometric augmented Hasse graphs. Finally, our result could shed light on which lifts should be preferred. the current literature (Bodnar et al., 2021a; Hajij et al., 2022b; Battiloro et al., 2023c; Eijkelboom et al., 2023) except for (Battiloro et al., 2024a), the lifts are always decided based on simple criteria (e.g.. if the domain is a simplicial complex, all the triangles of the underlying graphs considered 2-simplices). However, lifts could strike desirable tradeoffs between expressivity and learning performance (e.g. if the domain is again a simplicial complex, only some triangles could be chosen without expressivity drop).
Discussion.
generalization
Experimental Validation. To validate our theoretical claims, we follow the approach from (Joshi et al., 2023; Sestak et al., 2024). In particular, we employ pairs of k-chain geometric graphs, where each graph pair consists of k nodes linearly aligned and two endpoints with different orientations, as shown in Fig. 8. These graphs are clearly (I + 1) -hop distinct, and thus should be distinguishable
by (1$ 2024). To show the correctness of our result, we consider & 4, and we lift each pair of graphs into a CC in different ways, listed below. In all the cases, we use the mean to assign positions to cells in the Hasse graph (Assumption 3 of Proposition 1). Without loss of generality, we use NA,1 as in (14) (the usual node adjacency) and the up/down incidences Ni, in (I) as neighborhood functions, i.., we keep the connectivity among nodes as in the original pair of graphs without including edges as cells and we allow higher-order cells to communicate only with nodes but not among them (thus resulting in a hypergraph- like structure). The readout classifier takes as input the geometric invariants computed on the final updated positions.
2 ++1 EGNN layers or the same number of GWL iterations (Joshi et al., 2023; Sestak et al.,
Lift la: we consider a set of cells composed of the nodes of the original graphs plus one 1-cell containing all the nodes, as shown in Fig. 9a (a). In Figure 9b, show the corresponding
28

---

Published as a conference paper at ICLR 2025
Table 3: Classification Accuracy on 4-chain geometric graphs.
Number of Layers
k 4
Hidden Dim.
4
GWL
50%
50%
100%
100%
32
50%
50%
56.5%
50%
EGNN from (Sestak al., 2024)
64
50%
50%
100%
99%
128
50%
50%
96.5%
98.5%
32
100%
100%
100%
100%
ETNN Lift la
64
100%
100%
100%
100%
128
100%
100%
100%
100%
32
100%
100%
100%
100%
ETNN Lift 2a
64
90%
100%
100%
100%
128
90%
100%
100%
100%
32
50%
85%
100%
100%
ETNN Lift 3a
64
50%
80%
95%
85%
128
50%
80%
95%
95%
geometric Hasse graphs. All the nodes will have the whole graph as their receptive field before the first iteration, thus we expect the GWL on the Hasse graph, hence ETNN, distinguish the graphs using one layer. This case would be solved even by just feeding the initial geometric invariants to the readout classifier (using "0" layers).
Lift 2a: we consider set of cells composed of the nodes of the original graphs plus two 1-cells of the same cardinality being overlapping paths of length 4, as shown in Fig. 10a. In Figure 10b, we show the corresponding geometric Hasse graphs. It is clear that nodes {pi)-(p2) will have the whole graph their receptive field in one iteration, thus expect the GWL on the Hasse graph, hence ETNN, to distinguish the graph using one layer.
Lift 3a: we consider set of cells composed of the nodes of the original graphs plus two 1-cells of different cardinality and not overlapping, as shown Fig. 11a. In Figure 11b, we show the corresponding geometric Hasse graphs. It is clear that nodes {pi). }-p2) {o }.(p2) will have the whole graph as their receptive field in two iterations, thus we expect the GWL on the Hasse graph, hence ETNN, to distinguish the graph using two layers.
Per each lift, we trained ETNNs with an increasing number of layers to classify the pairs of graphs. Due to possible oversquashing effects (Joshi al., 2023), we also trained ETNNS with different hidden dimensions and averaged over 10 different seeds. In Table 3, we report the results. We also report the results for EGNN (taken from (Sestak et al., 2024)) and the GWL on the original pair of see, empirical results confirm the validity
graphs without any lifting into CC. As the reader of our analysis, and the clear advantage, in terms of expressivity, of using CCs and ETNNS over standard EGNNs. Further empirical advantage is reported, pointing at higher-order interactions practical solution to oversquashing (Giusti et al., 2023b; Di Giovanni et al., 2023).
Remark 6.
The lifts we used in the experiments above are handcrafted lifts, employed to validate Proposition
easy and consistent way. However, our result obviously holds all the cases, even with more ranks and more sophisticated collections of neighborhood functions (e.g. induced by Axcleton-preserrve
lift to simplicial, cell, or path complexes (Bodnar et al., 2021a; Giusti et al., 2023a; Battiloro et al., 2023c; Truong & Chin, 2024; Bodnar et al., 202 Moreover, it is worth it to notice that, with the experiments above, we investigated the role the choice of the cells given the collection of neighborhood functions. However, it is interesting to study even the opposite case. in which the cells are fixed and the neighborhood functions are chosen. For example, Lift 2a, adding the lower adjacency from (2) would have led solving the task with just one layer because the cells
29

---

Published as a conference paper at ICLR 2025
Table Classification Accuracy on the counterxamples graphs.
Body Order
2-body
3-body
4-body
EGNN from (Joshi et al., 2023)
50%
50%
50%
Inv.
100%
100%
50%
ETNN Lift 1b
Equiv.
100%
100%
50%
Inv.
100%
50%
50%
ETNN Lift 2b
Equiv.
100%
100%
50%
Inv.
100%
100%
50%
ETNN Lift 3b
Equiv.
100%
100%
50%
Inv.
100%
100%
50%
ETNN Lift 4b
Equiv.
100%
100%
50%
cal02 and d1/d2 would have had the whole graph as their receptive field before the first iteration. An opposite example serves as proof of Proposition of the main body.
Proposition 3. There exists pair of CCs whose nodes come with geometric features and collection of neighborhoods such that the CCs undistinguishable by ETNN.
Proof. We take again pair of k-chain graphs 91 and Gi. and we lift them in two CCs using Lift 3a employing only the up adjacency from (2) as neighborhood function. In this case, ETNN not able to distinguish the CCs because the resulting geometric augmented Hasse graph is disconnected graph, since p1/p2 and 0y/02 are been connected.
Counterexamples Structures. To provide an even more in-depth analysis of the expressivity of ETNNs beyond the distinguishability of k-hop distinct graphs, we also tested them on the coun- terexamples graphs from (Pozdnyakov et al., 2020; Joshi al., 2023), which test a layer's ability to create distinguishing fingerprints for local neighborhoods and stress exactly the importance of higher body order of scalarization. particular, this task evaluates single-layer geometric architectures in distinguishing counterexample graphs that cannot be differentiated using k-body scalarization. we refer the reader to (Pozdnyakov al., 2020; Joshi al., 2023) for further details. We tested different lifts:
Lift bb: we consider nodes and edges O-cells and 1-cells.
Lift 2b: consider set of cells composed of the nodes of the original graphs plus l-cell containing all the nodes (equivalent to Lift la).
Lift 3b: we consider nodes, edges, and a cell containing all the nodes as O-cells, 1-cells, and 2-cell, respectively.
Lift 4b: we consider nodes, edges, and paths of length O-cells, 1-cells, and 2-cells (this is equivalent path complex).
For all the lifts, we use the up/down incidences /N, and adjacencies A/N. from (1)-(2), respectively, as neighborhood functions, and we test both the invariant and equivariant versions of ETNN. The results are presented in Table 4. As the reader can see, ETNN is able to distinguish all the counterexamples but the 4-body (chiral).
Remark 7, Finally, please notice that we also indirectly showed that lifting (in a sreleton-preserving simplicial complex
way) graph CC and using ETNN can be more effective than lifting (SC) and using EMPSN (Eijkelboom et al., 2023). In particular, on both the numerical expressivity tasks, EMPSN with any skeleton preserving lifting would be expressive as an EGNN (and, thus, less expressive than an ETNN), as there are no triangles nor cliques.
30

---

Published as a conference paper at ICLR 2025
D1
la
T-2
(a)
11
j1
91
J1
js
92
P2
00
Tv2
(b)
Figure 9: (a) Lift la. (b) The corresponding geometric augmented Hasse graphs.
c: St.P..
(t..Js:00.PP)
dj
dg = {ps, og. 1z}
J:
g2
Pz
O₂
(1))
(i))
L
(Pt)
(0L)
(Js)
(12)
(oz)
(h:
(t/}
Q2
da
(b)
Figure 10: (a) Lift 2a. (b) The corresponding geometric augmented Hasse graphs
G
COMPLEXITY ANALYSIS
Given the flexibility in the definition of cells and neighborhood functions, the complexity of an ETNN layer can vary. To provide reference complexity, let us consider CC with |.Y cells, and the up/down incidences and adjacencies from (1)-(2 neighborhood functions. A cell E X has then [w)+ + (x)) incident cells and there (=)) [Wa,(()) (W).r()))+WWi.oo))) up/down adjacencies between them. Therefore, each node E has (x) (Wig(e))) adjacent nodes. Then, ETNN layer in (6)-() has computational complexity given by
31

---

Published as a conference paper at ICLR 2025
(1..).
d1 = (o),As)
(t..
for
9h
P1
Ps
hi
f2
hv
(PL)
(0L)
{PS
{Os
d:
(dz)
(11)
(is]
(hg)
(b)
Figure 11: (a) Lift 3a. (b) The corresponding geometric augmented Hasse graphs
(WNit(t))+ Nr,(x)) + -
+
(W
Σ
2
BEA
EES
h update from (6)
update from (7)
Σ ([Ni,(()) (NVrr)))) +
D Wan(())
FEX
SES
= -
Σ
[War(=)]+[aa.dl
(29)
TEX
If we consider the connectivity of the CC to be sufficiently sparse, which is usually the case, then
Avg. runtime (seconds/epoch)
EGNN-grat -raph- W ETNN
60.34
193.16
*MPSN*
251.3
Table 5: Average runtime per epoch for different ETNN configurations
Avg. memory consumption (GB)
EGNN- CGNN-graph-We W ETNN
13.63
12.05
#MPSN*
27.7
Table 6: Average memory usage for different ETNN configurations
the neighborhoods' cardinalities can be absorbed in the bound. Therefore, the overall complexity will be linear the number the cells (Ix1). Clearly, in the worst case, each cell could be connected to all the other cells, resulting in quadratic cost in the number of cells. In practice, with our implementation, both the model's forward and backward pass scale linearly with the number of neighborhood functions, as they are processed in loop. In Table 5, we compare the runtimes of EGNN MeaNN-grah-VV W. *MPSN* (i.e., EMPSN reproduced with our codebase as an instance of ETNN), and an instance of ETNN using the complete Molecular CC (configs. 35 of Tables 11-12). We decided to use this configuration of ETNN because is one of the most computationally intensive. However, better learning performance can be obtained with even (much) lighter configurations, shown in
32

---

Published as a conference paper at ICLR 2025
Tables 11-12. In this case, ETNN uses exactly times as many neighborhoods as in EGNN- graph--. as the runtimes confirm. A different implementation could parallelize the processing. making the runtime constant w.r.t the number of neighborhoods. Most importantly Table 5 provides the reader with quantitative metric of the significantly reduced computational burden of ETNN w.r.t. EMPSN (Fijkelhoom et al., 2023), as described in the main body. To further highlight this gain, in Table 6 we show the memory usage of the same architectures, proving that ETNN needs less than half memory w.I.t. EMPSN.
H ADDITIONAL REFERENCES ON REAL- WORLD COMBINATORIAL TOPOLOGICAL SPACES
H.1 MOLECULAR DATA
Molecular graphs. One of the most renowned representations of molecules is graphs. In this case, atoms are the nodes, and bonds among them are the edges(Gilmer et al., 2017b; Gasteiger et al., 2020; Jiang et al., 2021). Message passing networks on molecular graphs have been proven to be effective on several tasks. However, graphs cannot explicitly model higher-order, multiway interactions among the atoms. For instance, graphs cannot explicitly consider the information coming from the rings, i.c., the induced cycles of the molecular graphs. To represent different atomic relations, efforts focused on learning representations for small molecules and larger biomolecules (i.e., proteins) via multi view or relational graphs(Ma et al., 2020; Zhang et al., 2023; Sestak et al., 2024; Li et al., 2024b).
Molecular cell complexes. To overcome the above limitation, cell complexes (CW complexes), being a generalization of simplicial complexes (Eijkelboom et al., 2023; Bodnar et al., 202 1b), have been employed (Bodnar et al., 2021a). Molecular CW complexes can be seen "hierarchical augmented graphs" in which atoms are O-cells (nodes), bonds are 1-cells (edges), and rings are 2-cells (induced cycles). Message passing networks on molecular CW complexes have shown superior performance w.r.t. models on molecular graphs (Giusti et al., 2023h; Bodnar et al., 2021a). However, CW complexes cannot model higher-order interactions among the atoms and bonds that are not described by a cycle in the underlying molecular graph. For instance, molecular CW complexes cannot explicitly consider the information coming from the functional groups of the molecule, i.e.. specific motifs (not necessarily cycles) of the underlying molecular graphs.
Molecular hypergraphs. Hypergraphs (HGs) can model arbitrary higher-order interactions due to the flexibility of hyperedges. HGs and message passing networks on HGs have been employed for modeling and learning from molecules (Chen & Schwaller. 2023; Park al., 2021). In molecular HGs, atoms are nodes, and bonds, rings, functional groups, etc, are hyperedges. However, HGs cannot explicitly consider any hierarchy in the hyperedges, thus putting pairwise and higher-order interactions of any kind on the same level.
H.2 GEOSPATIAL DATA
Spatial graphs and CTS The single resolution version of the spatial setting in Section has been traditionally handled using proximity graphs, i.c., geometric graphs whose connectivity is dictated by some notion of spatial closeness. In the multiresolution case, the most common approach is to aggregate all the data information is clearly lost in this process. Some GNN methods for expanding the receptive field of GNNs in spatial data have been proposed. For instance, through edge removal during training to address over-smoothing and over-fitting (Rong et al., 2019), and the introduction of skip connections (He et al., 2016; Li al., 2018). Yu al. (Yu et al., 2019) introduced ST-UNec a heuristic graph pooling method for coarsened graph construction. Note that these methods do not admit multi-level data, i.e., features available in each level of a hierarchy. Hierarchical GNNs (HGNNs) (Diehl et al., 2019; Lee et al., 2019; Ying et al., 2018) create multi-level graphs balancing the expansion of receptive fields and preservation of local features. HGNNs can admit multi-level features. A highly relevant application is due to Li et al. (Li et al., 2019), who utilized geography- based HGNN for modeling geographic information systems. Importantly, this work does not consider the use of geometric (coordinate information) directly, and therefore, it does not explore equivariant architectures. Other related work includes Wu et al. (Wu et al., 2020), who proposed HRNR for learning road segment representations. Zhang et al. (Zhang al., 2020) combined HGNNs with sepi-supervised learning
common denominator of spatial resolution. However, substantial
33

---

Published as a conference paper at ICLR 2025
for citywide parking availability prediction. is worth noticing that the constructions of such HGNNs are highly reliant on the specifics of the application, while Geospatial CCs offer a more generally applicable formalism. Motivated by topological approaches, some works used hypergraphs (Jia et al., 2023), whose limitation have been explained in the previous subsection. Some Topological Data Analysis (TDA) work leveraged simplicial complexes (Bajardi al., 2015; Lo & Park, 2018; Stolz et al., 2016; Feng & Porter, 2021; Hickok et al., 2022). However, TDL should not be confused with TDA, which data-driven approach describing the mathematical properties and "shape" of topological space. Instead, TDL focuses modeling relations and data distributions defined over such topological spaces (Barbarossa & Sardellitti, 2020).
I EXPERIMENTAL SETUP
1.1 MOLECULAR PROPERTY PREDICTION
In this section, present all dataset information, data featurization, and model parameter configura- tion for the molecular property prediction task.
1.1.1 DATASET STATISTICS
QM9 dataset consists of over 130, 000 molecules with 12 property prediction targets. See Table for detailed list of the 12 molecular properties, well statisties over the number of molecules, atoms, bonds, rings, and functional groups.
Table 7: Statistics of the QM9 Dataset
Property
Description
Units
& Ac
Electronic spatial extent
bohr
Gap between Hомо and ÉLUMO Energy of the highest occupied molecular orbital Energy of the lowest unoccupied molecular orbital
meV
meV
ELUMO
p
Dipole moment
C
Heat capacity at 298. 15 K
cal/mol K
G
Free energy 298. 15 K
mcV
Enthalpy at 298. 15 K
mcV
U
Internal energy at 298. 15 K
mcV
Ug
Internal energy at K Zero-point vibrational energy
mcV
ZPVE
mcV
Number of molecules Average number atoms
133,885
nodes
Average number of bonds
19
Average number of rings
1.6
Average number of functional groups
1.2
I.1.2
CELL FEATURES
For the molecular prediction task, we considered different peculiar feature vectors for each rank, that can be easily integrated thanks to the ETNN structure.
1. O-cells features (atom features):
Atom Types: QM9 molecular samples consist of 5 atoms: hydrogen (H), carbon (C), nitrogen
(N), oxygen (O), and fluorine (F). We used one-hot encoding for the atom types.
Functions over atom types: Following Cormorant's 's caahhoooloyyAAndersse et al., 2019), computed the atomic fraction (current atomic number divided by the maximum atomic number), multiplied the atomic number with the atomic fraction, and computed the squared quantities.
2. 1-cells features (bond features):
34

---

Published as a conference paper at ICLR 2025
Bond Types: We can meet 4 different bond types in QM9 molecules: single, double, triple, aromatic. We used one-hot encoding for the bond types.
Conjugatiio:: A binary feature telling whether the bond is part of conjugated system, affecting electronic and optical properties.
Ring Membership: binary feature telling whether the bond is part of ring. affecting electronic and structural characteristics of the molecule.
3. 2-cells features (ring and functional group features):
Ring features
Ring Size: Number of atoms that form the ring.
Aromaticity: A binary feature telling whether the ring is aromatic.
Has Heteroatom: A binary feature telling whether the ring contains heteroatom, which is an atom other than carbon, such as nitrogen, oxygen, or sulfur. Heteroatoms can introduce different electronic effects and reactivity patterns within the ring structure.
Saturation: A binary feature telling whether the ring is saturated, meaning all the bonds between carbon atoms single bonds.
Functional Group features
Conjugation: A binary feature telling whether the functional group is part of conjugated system, which involves alternating double and single bonds.
Acidity: The ability of the functional group to donate protons (1++ ions) in solution.
Possible values: ["neutral", "high", "basic", "weakly acidic"].
Hydrophobicity: Tendency of the functional group to repel water. Possible values: [by- drophilic, "moderate", "hydrophobic"]
Electropiilicit: Tendency the functional group to accept electrons during a chemical reac- tion. High electrophilicity means the functional group readily accepts electrons, influencing its reactivity with nucleophiles. 3 Possible values: ["low", "moderate", "high"].
Nucleophilicity: Tendency of the functional group to donate electrons during chemical reaction. High nucleophilicity means the functional group readily donates electrons, affecting its reactivity with electrophiles. 3 possible values: ["low", "moderate", "high"].
Polarity: Distribution of electron density within the functional group. High polarity can significantly affect solubility, reactivity, and interactions with other molecules. 3 possible values: ["low", "moderate", "high"J.
I.1.3 PARAMETER CONFIGURATION
Next, we present the ETNN's hyperparameters we used for the molecular property prediction tasks. Table 8 shows the list of the hyperparameters of the optimizer, as well of our model. Specifically:
Following EGNN's experimentation setup, we use le-3 as the initial learning rate for the properties Δe, 6HомO, éLUMO, and 5e-4 for the rest of the properties.
We calibrate the number of hidden units (i.e., layer width) among {70, 104, 128, 182) for each configuration, that maintain roughly the same size of parameter space (ss 1.5M). The motivation for this choice is to provide the fairest comparison possible. see Appendix for further motivation and details.
All the reported configurations were run for 1000 epochs, in agreement with the original EGNN(Satorras al., 2021).
We used the same train/validation/tes splits as introduced in EGNN(Satorras et al., 2021).
See Appendix for further motivation and details.
For all the configurations we normalize the geometric invariants via batch normalization. That includes the normalization of pairwise distances. Hausdorff distances, and volumes. Moreover, we perform gradient clipping to maximum norm of gradients equal to 1.0.
35

---

Published as a conference paper at ICLR 2025
Table 8: Hyperparameters for ETNN Model in the Molecular Task
Hyperparameter
Value
Optimizer
Adam
Initial Learning Rate
[5 10-4, 10 10-31
Learning Rate Scheduler
Cosine Annealing
Weight Decay
le-5
Batch Size
96
Epochs
[100, 200, 350, 1000]
Model
Number of Message Passing Layers
[4. 7, 10)
Hidden Units per Layer
[70, 104, 128, 182
Activation Function
SiLU
Invariant Normalization
[True, False] [True, False]
Gradient Clipping
1.2
HYPERLOCAL AIR POLLUTION DOWNSCALING
1.2.1 DATASET STATISTICS
We create geospatial CC using the neighborhood functions described by Eq. (12). The hyperlocal air measurements points (target) are taken from (Wang et al., 2023). The targets are aggregated at the desired resolution of (1.0022 ~ 22m. After removing roads and tracts that do not contain any measurements, the resulting dataset contains 3,946 point measurements, 550 roads, and census tracts, corresponding 0-, 1- and 2-cells, respectively.
1.2.2 CELL FEATURES
The learning targets are the fine-resolution PM 2.5 from (Wei et al., 2023) as described in the main text. As such, this is semisupervised node regression task. We now describe the features of each rank. For the O-cells, we generate point-level features using the osmnx Python package (Boeing, 2017), calculating metrics such as density and distance to points of interest within a buffer radius. We remove roads and tracts that do not contain any measurements. We consider only points that correspond to weekdays between 9 and pm. For I-cells, we use the 2021 annual average daily traffic information (New York City Department of Transportation, 2024). For 2-cells generate land-use features from the Primary Land Use Tax Lot Output (PLUTO)(NNe) York City Department of City Planning, 2024) and previous masse--rsoolutin estimates of PM 2.5 the census-tract level are taken from Wei et al. (Wei et al., 2023). Values are monthly averages over the last quarter of 2020, projected to the census tract level using bilinear interpolation from their native km resolution. Below is a list of the employed features. Table 9 summarizes the data sources.
1. O-cells features (points; OpenStreetMap (OSMnx) and Wang et al.):
Intersection density: Using the osmnx Python package, we calculate the density of intersec- tions within 100m buffer radius (Boeing, 2017).
Nearest school distance: Calculated using osmrx determine the distance the nearest
school (Boeing. 2017).
Restaurant density: Using osmnx, we calculate the density of restaurants within 300m
radius (Bocing, 2017).
Road density: Derived from osmnx by calculating the density of roads within a buffer radius (Boeing, 2017).
Time of day: We include the median time of measurements by Wang al. (Wang et al., 2023)
for each of the points in a 0.0002 x 0.0002 area.
1-cells features (road/polylines; AADT, NYC, Dept. of Transportation):
AADT 2021: Annual Average Daily Traffic data for 2021 (New York City Department of Transportation, 2024).
PCT 202/: Percentage of trucks traffic (New York City Department of Transportation, 2024).
36

---

Published as a conference paper at ICLR 2025
Table Data sources of the Geospatial Air Pollution Downscaling Benchmark
Source
Citation
LICENSE
Wang al.
(Wang et al., 2023)
CC 4.0
PLUTO, NYC Dept. of Planning
(New York City Department of City Planning, 2024)
Open Data
AADT, NYC Dept. of Transportation
(New York City Department Transportation, 2024)
Open Data
OpenStreeiMap (OSMnx)
(Bocing. 2017)
Open Data
Wei al.
(Wei al., 2023)
CC
3. 2-cells features (census tracts):
PLUTO, NYC Dept. of Planning (New York City Department of City Planning, 2024)
Assessment total
Building area
Commercial area (% area)
Factory area (% area)
Mixed residential/commeriaal (% tax lots)
Multi-family elevator (% tax lots)
One/two family (% tax lots)
Open space/outdoor recreation (% tax lots)
Parking/vacant (% tax lots)
Public Aacilicacclitteen fucilities/msstiutiion (% tux lots)
Transportatin//uility (% tax lots)
Lot area (sq. km)
Lot depth (m)
Office area (% area)
Residential area (% area)
Retail area (% area)
Other sources
Coarse PM2.5 predictor (Di et al., 2019): Coarse-resolution monthly estimates of mean PM2.5 by census tract.
1.2.3 PARAMETER CONHIGURATION
The training and evaluation scheme is similar to the molecular case. For training, use the Huber loss which provides additional robustness to outliers (Huber, 1964). The evaluation metric is obtained similarly the molecular task. Since the data consists of a single graph, for semisupervised tasks on spatial data and generate training, validation, and test masks. To do so while being mindful of spatial leaks and correlations, training and split the remaining 30% equally in test and validation tracts. Since points in the dataset only belong single census tract, we can use these splits to partition points in their corresponding sets, ensuring that neighboring points belong, on average, to the same split. While all the nodes (training/validattee /alitto//iaiii//esssss are given as inputs the model, the loss function for backpropagation is only computed from the error in the training nodes, as in any standard semisupervised setting (Kipf Welling, 2017).
standard procedure
randomly select 70% of census tracts for
The evaluation metric is the root-mean-squaree error (RMSE) in the test split from the best model of each baseline. The base model is determined by the lowest RMSE in the validation RMSE at any point during training from sweep over the hyperparameters described in Table 10. The test split only reports the final metric to ensure validity. The EGNN baseline used the same architecture but no lower/upper adjacencies. It also has higher-order features concatenated with point-level features. The multi-layer perception and linear baselines are evaluated similarly. The MLP baseline is implemented with 5 layers to match the number of hidden layers in the ETNN and graph-based baselines.
37

---

Published as a conference paper at ICLR 2025
Table 10: Hyperparameters for ETNN Model in the Air Pollution Downscaling Task
Hyperparameter
Value
Optimizer
Adam
Initial Learning Rate
[10-3, 10 10
Learning Rate Scheduler
Cosine Annealing
Weight Decay
le-4
Batch Size
Epochs
500
Model
Number Message Passing Layers
Hidden Units per Layer
[4, 32]
Activation Function
SiLU
Invariant Normalization
False
Gradient Clipping
True
Dropout
[0.025, 0.25]
1.3 GEOMETRIC FEATURES
We note that for both the molecular property prediction, and the hyperlocal air pollution downscaling task we using coordinates as geometric features. Specifically, we employ O-cell geometric features, which are inherited in the higher ranks. Regarding the molecular property prediction, we are using the 3D atom coordinates, while for the geospatial task, we are using the latitude and longitude variables.
Computational Resources All experiments that produced the reported configurations in both tasks were conducted using Nvidia A 100-SXMM440GGB GPUs. Each system has a capacity of 80GB RAM, 40GB GPU memory, and 6 CPUs per unit.
J ABLATION STUDY AND REPRODUCIBILITY FOR MOLECULAR PROPERTY PREDICTION
In this section, we present the whole set of experiments we carried out to obtain our results in Table 1, i.e., all the different configurations of ETNN we tested. Moreover, we want to further specify some details that are useful to fairly evaluate our results and ensure reproducibility.
Reproducibility. When we started to set up the experiments, we found a poor reproducibility level for the molecular prediction task on QM9 the literature. In particular, practically every model of the ones listed in Table 1 uses different data splits. The two closest models to ETNN are EGNN (Satorras et al., 2021) and EMPSN (Eijkelboom et al., 2023), which are also particular cases of our framework. We were not able reproduce the results of EMPSN (and this is also the reason did not include it Table 1), while we were able to reproduce the EGNN results using the original codebase. For all of the reasons above, we decided to employ the same data splits as EGNN(Satorras et al., 2021). To guarantee a fair comparison with EMPSN, however, in this appendix we also report the results of EMPSN from the original paper, EMPSN reproduced in TopNets (Verma et al., 2024), and EMPSN reproduced with our codebase (as described in the main body). Again, and as explained in Appendix C.1, EGNN can be directly derived from ETNN. Before exploring different ETNN configurations, we aligned our training and architecture parameters with EGNN. We ensured that our ETNN code, configured as an EGNN, could reproduce all the considered properties of the QM9 dataset reported in the original EGNN paper. Moreover, we trained models for 1000 epochs in EGNN and EMPSN. Finally, to ensure further consistency, we also trained a configuration of ETNN, dubbed MENN-graph-Ve W, that is again EGNN derived from codebase but with higher number of hidden units, chosen such that the number of parameters of ETNN- ETNN-graph- W matches the average number of parameters of the other ETNN configurations (1.5M, while the original EGNN has 748k).
Results. In Tables 11- 12, we present the full list of the ETNN configurations we tested, representing also an exhaustive ablation study all the main components of our architectures. We also add the
38

---

Published as a conference paper at ICLR 2025
results of TopNet, presented in the contemporary work (Verma et al., 2024). We grouped the relevant model hyperparameters of ETNN into the following five groups:
Definition of the Molecular CC. We study the performance of ETNN when bonds, rings, and functional groups, are included or not as cells in the complex. In Tables 11 1- 12, this is described in the first argument, Lifters. The values of the argument indicate what are the considered cells, e-g. A+B+F+R means that atoms, bonds, functional groups, and rings are used as cells. The rank is assigned in increasing order as described in Section
Adjacencies. We study the performance of ETNN when different adjacencies are employed in the collection of neighborhood functions C.N. In Tables 11-12, this is described in the second particular, we test the usage of NA.maa as in (13) in conjunction with
argument, Adjacencies. the up/down adjacencies NA.1 and NA.1 from (2). In Tables 11- 12, Adjacencies max means that only the max adjacency from (13) is used, while Adjacencies = max.f.J means that both the max adjacency from (13) and the up/down adjacencies from (2) are used (effectively resulting in heterogeneous setting as described in the body of the paper).
Incidences. We study the performance of ETNN when different incidences are employed in the collection of neighborhood functions C. In Tables 11- 12, this is described in the third argument, Incidences. Testing only the he up/down incidences Ni.1 and Ni,1 from (2) would always result in configurations in which cells of rank & communicate only with cells of ranks k 1 and k + 1. For this reason, we also test the k-up/down incidences, defined as:
Ni.tu (1) = tu E Xirk(y)
+ k r v), NT (I) [v E X(r(()) = k. cz..
(30)
In this way, cells can communicate in a more flexible way across ranks. In Tables 11- 12, Incidences all means that the k-up/down incidences from (30) are used for all the possible i... each rank communicates with all the other ranks; Incidences = 1 means that the up/down incidences from (I) (equivalently, & = 1 in (30)) are used; Incidences 0 means that no incidences are used, i.e., there are no inter-rank message exchanges and the embeddings are aggregated only in the readout layer.
Higher-order Features. We study the performance of ETNN when higher-order features (not only atom features) are employed. In Tables 11- 12, this is described in the fourth binary argument, HO Features. If HO Features 1, higher-order order features (bonds, ring and/or functional groups based on the complex) are fed to the model.
Virtual Cell. We study the performance of ETNN when the virtual cell is employed. In Tables 11- 12, this is described in the fifth binary argument, Virtual Cell. If Virtual Cell = 1,, the virtual cell is included in the complex.
Remark 8. In all the configurations, we don't allow the virtual cell to exchange messages or being included in the readout layer. The reason is that we mostly use it in the experiments obtain fully connected computational graph each rank when it is used together with the max adjacency from (13). However, it remains an interesting object, as it is i) a straightforward way to include whole molecule-level features in ETNN by assigning them as features to the virtual cell, and ii) useful to improve expressiveness and generality of ETNN, as shown in Appendix F. Therefore, when Adjacencies = max and Virtual Cell=1, the resulting configuration is equivalent not including the virtual cell in the complex and using the neighborhood function an) = (v E Xrrk(y) = rk(z))-
To provide the reader with some usage examples of ETNN employing the above hyperparameter grouping (that we follow in our codebase), EGNN in the original paper (Satorras et al., 2021) can be obtained setting Lifters = A, Adjacencies max, Incidences 0, HO Features = 0, Virtual Cell 1; an instance of EMPSN (Eijkelboom al., 2023) can be obtained setting Lifters A+B+R, Adjacencies = f. J. Incidences 1, HO Features = 1, Virtual Cell = 0, and considering only rings of length up to three; an equivariant version of (Bodnar et al., 2021a) can be obtained setting Lifters- A+B+R, Adjacencies = 1, J, Incidences 1, HO Features = 1, Virtual Cell = 0, and considering rings of arbitrary length.
As the reader can notice from Tables 11- 12, several higher-order ETNN configurations reach near-SotA results and significant improvements over the reference EGNNs baselines, i.e., the original EGNN (Satorras et al., 2021) in the upper part of the tables, and its implementations ETNN -graph and ETNN- ETNN-graph- W obtained through our codebase. ETNN-graph is the same exact architecture as the original EGNN but trained with gradient clipping and geometric invariants normalization.
39

---

Published as a conference paper at ICLR 2025
Table 11: Mean Absolute Error for the first six molecular property predictions in QM9.
SHOMO mcV
FLUMO meV
Units
bohr!
meV
cal/mol K
NMP
092
69
43
.030
.040
Schnet
.355
63
41
.033
.033
Cormorant
085
34
.038
J026
LINet
088
68
46
.043
.031
LicCoav
.84
49
30
.032
.038
DimeNot++*
044
33
25
20
.030
.023
TEN
223
40
.101
SB
142
53
35
33
.051
.044
EMPSN(Fijkellooom ol., 2023)
066
37
25
.023
.024
EMPSN*
.01
42
29
.030
J028
EMPSN(Verma et al., 2024)
066
32
.027
TopNet (VC, discrette) Verma al., 2024)
083
47
37
.035
.032
TopNet (VC, continuous) Verma al., 2024)
.075
49
36
.030
.035
TopNet (RePHINE, discrete)) Verma al., 2024)
002
57
33
.029
J028
TopNet (RePHINE, continuous) ((rruu et al., 2024)
.070
30
35
.032
.030
ETNN (Configurationss
[ Lifters Adjacencies Incidences | Features Virtual Cell
[A+B max [01111]
.044
30
.03
.032
2: [A+B+F max 1011111
.078
32
.029
.032
3: [A+B+R max 11111
079
34
.09
.033
4: [A+B+I +R max 101111]
.07
54
31
.032
.033
5: JA+B Ima 010111
.62
27
.033
[A+B Imax all
.07
47
26
7: |A+B+F max all 1111]
072
47
27
.024
.032
8: [A+B+R] max all 11
071
49
26
.023
.032
9: IA+B+F+R max lall
066
26
.033
10: [A+B-F-RI max, T. 1all |alll110
.157
72
45
46
.31
.053
101
45
12: [A max 10101110000 (ETNN-graph)
.07
45
28
.029
.035
[A.
28
14: [X+B+F-R max. 1 110]
.13
66
44
42
.885
.D46
[A+
(triangles only, EMPSN-like)
27
16: [X+B+F-RR max. Iall 1 (no geometric Teatures, CCMPN like)
.191
85
62
59
.339
.078
JA+B-E-R
(2o geometric CCMPN-like)
18: [X+B max 1111111 [[11]]
069
47
26
22
.024
.03
[A+B+F
20: [A+B+R] max 11111110000
.069
46
26
24
.033
.033
JA+B-F-RR
23: [A+B-F-RI max 11111 [0]
.199
68
41
43
281
/046
25: [A+B-F-R max 11110 []
geometric features, CCMPN-like)
.193
60
.39
.075
26: [A+B max, 101111111 10]1
.187
85
61
58
342
.074
27: [A+B=F]
101111 100110000
I.179
438
282
.772
.678
28: [X+BER max, T.U 10111
(equivariant CWN like)
1.143
352
262
262
.713
.628
29: [A+B max, 111101011
1,038
332
237
.721
.548
30: [A+B max, 1]
.84
267
209
229
.657
.496
31: [A+B max, 111111
1,224
410
251
.122
.888
32: [A+B+F] max. ",.1110
.141
74
50
312
.D49
33: [A+BER
T.l 16.11111
.143
75
48
337
.088
34: [A+B+F-RI max, 1011
.129
44
297
.046
35: [A+B+F-RI max, 1.1.1 [010]
.134
46
302
J046
36: [A+B+R] max, T.L. 111101100 (triangles only. heterogencous EMPSN-like
.132
45
309
.045
37: [A+B+R] max, Tel 1111 (no geometric features, CWN-like)
.155
75
49
49
324
.049
38: [A+B+F4R] max, 1 10100000 (no geometric features, CCMPN-like)
.184
62
.077
39:
[A+B+F-RI max, 1001 (no geometric features, CCMPN-like)
.19
92
62
60
.36
.076
Best ETNN Configuration
EINN.
062
45
26
.022
.030
Improvement HONN""
-3%%
-8%%
10%
-1%%
28%
As mentioned at the beginning of this section, we made sure that ETNN MoNN-graph without gradient clipping and normalization almost exactly reproduces the results of the original EGNN. We report EGNN-graph because it is sometimes able to reach better results than the original EGNN on some targets. Moreover, some configurations that test are also instances of other architectures like EMPSN (Eijkelboom et al., 2023) or CWN (Bodnar et al., 2021a), further highlighting the versatility of our framework. Notably, ETNN consistent outperforms TopNet( Verma al., 2024) too. Overall, these results validate our TDL. approach and show that the flexibility of CCs is necessary.
K ARCHITECTURE DESCRIPTION
In this section, we describe the implementation of E(n) Equivariant Topological Neural Networks (ETNNs). The model architecture is kept as similar to EGNN possible so not introduce
40

---

Published as a conference paper at ICLR 2025
Table 12: Mean Absolute Error for the last five molecular property predictions in QM9.
Task
H
U
Ug mcV
ZPVE
Units
meV
mcV
meV
meV
NMP
19
17
20
20
1.50
Schnel
14
14
19
14
1.70
Cormorant
22
2.03
LINet
14
LicConv
19
19
2.28
DimeNet++
6
1.21
TFN
SE(3)-Tr:
BGNN
12
12
12
11
1.55
EMPSN(Fijkelloom et al., 2023)
9
7
10
1.37
EMPSN*
11
10
10
1.46
EMPSN (Verma 2024)
1.44
TopNet (VC, desccrtt)) Verma et al.. 2024)
TopNet (VC, continuous) Verma 2024)
TopNet (RePIIINE, discrete) Verma et al., 2024)
1.38
TopNet (RePHINE, (Vernua al., 2024)
1.37
ETNN (Configurations)
[ Lifters Adjacencies Incidences IHO Features Virtual Cell
1: [A+B max 10110000
15
13
14
13
1.75
2: [A+B+F max 101110
14
12
12
12
1.733
3: [A+B+R max 101111]
15
14
14
1,763
[A+B+F max [01111]
15
15
1.749
[A+B 1010111
12
12
1,589
[A+B
111
12
11
L.648
7: [A+B+F max alll
12
12
12
1,66
8: [A+B+R I max all 1111]
11
13
11
13
1 1.668
9: [A+B+F+R max all
12
12
13
11
I. 1.742
11: [A+B max 1
26
27
27
27
2.275
12: [A max 010110 (ETNN-graph)
14
16
1,658
[A 0101 (ETNN-graph-W) [A+B+F+R max all
1.624
24
2.278
15: [A+B+R max all
(triangles only, EMPSN-like)
12
12
I1
12
1.703
16: [A+B+F4R max all 1 (no geometric features, CCMPN-like
32
34
35
32
3.268
17: [A+B+F+R| max all
[no geometric features, CCMPN-like)
32
34
34
34
3.202
18: [A+B max
12
11
13
1.65
19: [A+B+F max
14
12
13
20: [A+B+R max
12
11
1.714
[A+B+F4R
11
1.694
23: [A+B+F+R
24
24
24
2.183
25: [A+B+F4R max
[111
(no geometric features, CCMPN-liks
32
35
35
34
3.361
26: [A+B+F+R I max
[1]
32
35
34
34
3.183
27: [A+B max. T.l
427
432
434
431
17.279
28: [A+B+R max, 1--10
(equivariant CWN-like)
371
376
373
16.298
29: [A+B+R max. 1-110 [100111]
302
302
302
13,019
30: [A+B max. 1-1101011]
238
237
236
12.175
31: [A+B max.
1.1101011]
477
491
485
484
25.588
32: [A+B max. 1-1111011]
29
30
29
30
2.369
33: [A+B+F max, 1.1111011]
28
29
28
30
2.227
34: [A+B+R max, 111111011]
23
24
25
23
2.165
35: [A+B+F4R max, 111111111
24
2.178
36: [A+B+R max. 111110110
(triangles only, heterogencous EMPSN-like 37: [A+B+R max, 111111010] geometric features, CWN-like)
25
2,215
27
2,222
38: [A+B+R
1.1111110 geometric features, CCMPN-like)
34
34
34
3.154
39:
[A+B+F4R max. 11:110 0J (no geometric features. CCMPN-
32
33
33
32
3.148
Best ETNN Configuration
ETNN-*
11
12
11
11
1.589
Improvement over FGNN
-8
8%
0%
88 %
0%
2.5%
alternative explanations for differences in model performance. The main components of ETNNs as follows:
Model Initialization:
41

---

Published as a conference paper at ICLR 2025
Normalizes invariant features dynamically using batch normalization with no learnable parameters. Each rank is normalized separately.
Invariant Normalizer:
Feature Embedding: Embeds input features into hidden representation. Input features of each rank are embedded with separate featurizer.
Equivariant Message Passing (EMP) Layers: Repeatedly updates the hidden representations of each cell using messages from neighboring cells. For details, see below.
Pre-Pool Layer: Updates the hidden representation of each cell using distinct MLPs for each rank.
Post-Pool Layer: MLP applied after the global pooling operation to produce the final output.
Forward Pass:
Input Handling:
Retrieves pre-computed non-geometric cell features, adjacency relationships, and positional information.
2. Initial Feature Computation:
Computes node and membership features for each rank. Cells with higher rank may either use their own non-geometric features, the mean feature vector of the nodes the cell is composed of, membership vector denoting the lifter that generated the cell, or any combination of these.
Embeds the chosen geo-eeemt features into common hidden representation.
3. Geometric Invariant Computation:
Computes E(n) invariant geometric features using the absolute positions of nodes and the specified compute_ii e_iieeriiirrareeee antts function. ants
Optionally normalizes the computed invariant features.
4. Message Passing:
Applies EMP layers to update cells' hidden represntations via message passing. For details on message passing, see below.
5. Readout and Pooling:
Applies pre-pooling MLPs to the hidden representations of cells. Each rank is processed by a different MI.P
If the task graph-level prediction, hidden representations of cells of same rank are added together to arrive at rank representations. These representations concatenated and a post-pooling MLP is applied to generate the final output.
Equivariant Message Passing (EMP) Layers: The computation the EMP layer takes place in the following steps:
Computation of individual messages:
For each cell, a message is computed for each its parents. This message is computed by an udjacency-speciff MLP that takes as input the hidden representations of the sending and receiving cells, and the E(n) invariant geometric features derived from the absolute positions of the nodes they are composed of. A separate MLP is defined for each rank pair.
2. Adjacency-wise aggregation of computed messages:
For any given cell, the set of messages it received from its neighbors grouped according the rank the sender. The messages within each group are aggregated using weighted sum of the messages, where the weights are the nonnegative, scalar outputs of an dejjacecc--speciffe MLP that takes as input a message.
3. Cell representation update:
Finally, an update is computed by feeding the concatenation of the adjacency aggregated messages to a rank-specific update MLP. This update added to the cell's hidden representation.
42

---

Published as a conference paper at ICLR 2025
4. (Optional) position update:
Optionally, node positions are updated as weighted sum of differences between the position of the node and its neighboring nodes. The weight is a learnable function of the message that was passed to the node from its neighbor.
43


Reviews:
[{"review_id": "MWQqN6Ngv6", "reviewer": "Reviewer_NQcw", "summary": "This paper introduces an equivariant model within the framework of topological deep learning. The architecture generalizaes the equivariant graph neural network architecture from Santorras at al. from the setting of graphs to message passing over combinatorial complexes. Notably, this architecture allows for message passing with cells that have heterogeneous node feature over differening ranks. \n\nThe authors first introduce the relevant theory of combinatorial complexes, topological deep learning and important noitions of equivariance.  In section 3, they introduce their architecture using the formalism from the previous section. The authors then discuss important theoretical aspects of their model; proving equivariance, comparing expressiveness with traditional equivariant, scalarization-based techniques and discussing computational complexity of their design. \n\nThe authors discuss in section 4 two complementary examples of how to attain combinatorial complexes: firstly in molecular data and secondly in geospatial data. In section 5, the authors use said combinatorial complexes to benchmark their method against common architectures on the QM9 dataset. Secondly, the authors introduce a novel benchmark for predicting airpollution from annoted geospatial data.", "questions": "Experiments\n\nCould you provide more details on how functional groups were defined or identified in the QM7 dataset? Were these pre-annotated in the data or determined through some other process?\n\nIt also seems like many of the best results are given in hyper parameter configurations without the use of higher order cells. Can the authors comment on this? Does this detract from the main thrust of the paper, which is the inclusion of higher order cells?\n\nNovelty: TDL vs GDL\n\nTurning my comments into the weaknesses section in a specific question: what do we gain from introducing the notion of a combinatorial complex? As above, (1) the procedures for turning graphs into combinatorial complexes in this paper are not objects commonly studied in topology — i.e. do not correspond to a simplicial or cellular complex — so do not have access to additional mathematical theory and (2) we could seemingly equally well-formulate this paper as an EGNN over a specific procedure for turning the data into a graph﻿ as per Prop. 1. Why not just say this is a specific example of a EGNN using the construction in Prop 1?\n\nI am very open to hear the author’s thoughts/be corrected on this!\n\nVirtual cells\n\nThe virtual cell seems like to dramatically increase performance. However, my understanding is that this totally change the connectivity of the underlying data, meaning that\neverything is now connected (as per the comment on 389). ﻿What does this say about the actual importance of the graph structure? \n\nIt seems to suggest the ’topology’ of the underlying molecular graph is basically ignored once the virtual cell is included. Why not just define the architecture as a family of fully connected EGNN over the nodes, edges and potentially the faces, given that the hyperparameters including the virtual cell almost always are superior?\n\nOn a related note: \"it is clear how ETNN naturally handles heterogeneity, e.g., the same pair of bonds could be connected because part of the same functional group (up adjacency) and the virtual cell (max adjacency), but the two messages will be different across the neighborhood”. Unless I’ve missed something, how are we certain that the two types of propagation add to performance, rather than just overcomplicating the situtation?\n\nMiscellaneous\n\nLine 354: In the defintiion of geospatial CC, the mapping function s : X \\to T takes cells to points in T, but the geographic space s(X) is now a subset of T rather than an element. Should s be mapping to the powerset of T, rather than T, if indeed it takes each cell to a subspace of T? \n\nIn general, it would better to have a more concrete definition of what is meant by a ‘geospatial combinatorial complex’. I’m also unfamilar with the term polyline. \n\nLine 375: \"This work is the first to explore combinatorial topological modeling of multi-resolution irregular geospatial data..” The claim that this is the first work to explore combinatorial topological modeling of multi-resolution irregular geospatial data seems overstated given existing literature in Topological Data Analysis applied to geospatial data. Could you clarify how your approach differs from or advances beyond these existing works?\" https://arxiv.org/pdf/2104.00720, https://www.researchgate.net/publication/366891451_Topological_data_analysis_for_geographical_information_science_using_persistent_homology, https://pubmed.ncbi.nlm.nih.gov/26353267/\n\nTable 2: It would be helpful to have the variance included over the multiple runs\n\nLine 242: \"geometric invariants should make use of the underlying topological structure.” The pairwise distance is more of a geometric structure than a topological one.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "Presentation, clarity and experimental transparency\n\nI found the paper to be written clearly, and the mathematical statements and proofs were precise, well formulated and easy to follow. Further, the authors included a lot of helpful background about topological deep learning, and explained concepts clearly. \n\nOne of the biggest strengths of this paper is the much appreciated transparency around testing in the appendix. This allowed a confident and clear understanding of what the authors actually did to obtain their results. I really appreciated the detailled description of cell features in section I.1.2 and I.2.2. The detailled ablation studies on the were also super helpful to understand and evaluate overall performance.\n\nNovel geospatial task\n\nI really like that the authors introduced a novel geometric prediction task into the literature. In reading the literature for this review, it struck me that many of the benchmarks for TDL were somewhat old and outdated. It seemed appropriate that the task featured integration of data over different dimensional regions (points, lines, cells) in a way that showcased the central feature of the paper — reconciling data with features on subspaces of differing dimension i.e. the ‘heterogeneous interactions’ promised in the abstract. \n\nComputational\n\nI think one of the main strengths of the paper is the computational benefits, and I would personally focus on this more in the introduction. The tailored lifting of molecular graphs to the higher order CCs dramatically decreases the number of higher order cell, which is a problem that plagues many architectures in learning on simplicial/cellular complexes. The section on computational complexity also demonstrates this in a robust mathematical setting.", "weaknesses": "TDL vs GDL: novelty as a conceptual framework\n\nFor me personally I find it hard to understand the framing of this as a part of an entirely new conceptual field of topological deep learning beyond GNNs, and question the genuine novelty of papers like this. This is a concern I have of the field of TDL more generally, but I hope that the authors may be able to help clarify given their excellent communication skills demonstrated in the paper. \n\nUnless I’m mistaken, the basic content of proposition (1) is that an ETNN can be reformulated as an EGNN. This means that the main novelty is the clever choice of ‘lifting' the data into a certain graph and some delination of the learning based on ‘rank’. Indeed, even the proof of theorem (1) is basically a straightforward adaption of the corresponding result for EGNNs. I think the need to reformulate everything as an ETNN then show it’s equivalent to some specific EGNN needs more justification.\n\nI still think that the experiments, results and set-up of the paper are interesting, but I personally get the sense that the ’topology’ part — and hence the novelty of these kinds of architectures — is overplayed a little.  One gets the impression from reading the introduction that there is some topological thing deep down somewhere that is making the difference in performance, whereas my feeling is that the inclusion of domain-specific data — functional groups, rings, etc. — along with the design of the graph is doing most of the work. \n\nOn a similar theme, I don’t personally see a strong connection with the ‘lifted’ combinatorial complexes used in this paper and topology in the classical sense. It’s true that cell complexes and simplicial complexes also have the structure of a combinatorial complex (as in Appendix C), but these classical objects have additional connections to topology — i.e. they are stratifications of a genuine topological space, posses a homology theory, etc. \n\nQM7 Benchmarking comments\n\nI find the approach to testing for this benchmark slightly unfair on other methods. Searching through so many iterations of hyperparameters, it seems inevitable that eventually some parameter set will outperform current methods at least once just on the balance of probabilities rather than model strength. ﻿Surely a fairer test would be run the competing methods along a similarly vast set of hyperparameters? At minimum, this experiment should be run multiple times, with the variance on results included, to test the robustness of these results.\n\nCould the authors elaborate on why they chose to focus only on comparing the ETNN only with EGNN in the bottom line of Table 1? Upon first look, I assumed that ETNN outperformed SotA, but then on closer look saw that Equiformer outperformed ETNN in (almost)( every category. I would recommend highlighting the best performing method in bold in each column so that it’s more immediately clear that the while ETNN is improving on EGNN, it is still behind the SotA methods like DimeNet++ and Equiformer. My personal opinion is that the bottom line should be removed — the fact that it outperforms EGNNs is not surprising considering the fact that EGNNs are somehow a sub-architecture given the results of the expressiveness appendix.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dzp2j9EsR9", "reviewer": "Reviewer_h23Z", "summary": "The authors propose an equivariant Topological Deep Learning framework that deals with geometric node features. The frameworks can be generalized to many topological domains including simplicial, cell, combinatorial, and path complexes. The authors provide theoretical analysis regarding his design choice as well as expressiveness of the proposed method. Lastly, the authors support his arguments via real-world datasets QM9 and his proposed benchmark - air pollution downscaling benchmark.", "questions": "1. If I understand correctly, your argument on “heterogeneous interactions” focuses on different relationships between cells. Meanwhile, this property was actually mentioned in prior literature ([3] to name a few). I don’t think it is fair to claim that ETNNs are set up for this characteristic, but more like TDL in general already possesses this property.\n2. The paper mentioned that Combinatorial Complex subsumes Path Complex. I believe there are two distinct lines of work regarding complexes arising from paths. One is path-based Combinatorial Complex [3], and another work is a simplified path complex [2] based on path complex [1]. [2] can’t be derived directly from [3] because [2] reserves the sequential information of paths. From my understanding, your work focuses more on path-based combinatorial complexes.\n3. A relevant work [4] is not discussed in the paper.\n4. It seems to me that $\\mathcal{N}_{A, \\text{max}}$ isn’t supported by any experiments. It would be better if the authors provide a set of experiments to support this design choice.\n5. A minor subjective feedback on writing. I think it is better to have different notations for “containment” and “is a subset of” operation for clarity (Equation 6 and 7).\n6. Regarding section I.1.2, it seems that 2-cell features do not encode any geometric features (velocity for example) but only invariant features. I think it is a missing piece to convince the audience that your framework can work with geometric features. Also, suppose that even when we have geometric features at node levels, it is unclear how to lift these features into higher-order spaces.\n\n[1] Grigor’yan, A.A., Lin, Y., Muranov, Y.V. et al. Path Complexes and their Homologies. J Math Sci 248, 564–599 (2020).\n\n[2] Truong, Q., & Chin, P. (2024). Weisfeiler and Lehman Go Paths: Learning Topological Features via Path Complexes. Proceedings of the AAAI Conference on Artificial Intelligence, 38(14).\n\n[3] Hajij, M. et al. Topological Deep Learning: Going Beyond Graph Data (2023).\n\n[4] Li, L. et al. Path Complex Neural Network for Molecular Property Prediction. ICML 2024 Workshop GRaM (2024).", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "The authors add an important piece of work for the Topological Deep Learning (TDL) community as there is not much literature on Equivariant TDL. The work is well-formulated with clear motivations. The theoretical contributions are well-written. The paper is also self-contained and easy to follow, given the substantial explanations from related prior literature. The ablation studies are well-conducted via many synthetic graphs and additional information (hyperparameters, data statistics, etc.) are provided. The novel benchmark based on geospatial information is novel.", "weaknesses": "Novelty is the key disadvantage of the paper. It seems that the work just extends prior works on graphs to TDL. Even though the theoretical insights are important, yet they are mostly an extension from graphs. Another important weakness is scalability and practicability  of the problem. There are only two real-world datasets evaluated, and in both cases, graph sizes are small. Furthermore, the performance isn’t convincing given there are only minor improvements over the graph counterparts. The performance could be due to extra parameters for higher-order filters. Perhaps an experiment on model comparison with constraints on parameter budgets and an ablation study on higher-order features masked out are needed to prove your arguments. Lastly, even when the framework makes sense, it is unclear how we can obtain geometric features for higher-order cells. Please refer to question 6 for my concern.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MdO055yRe2", "reviewer": "Reviewer_wwvW", "summary": "This paper extends message passing supported on geometric graphs [3] and geometric simplicial complexes [2] to the more general setting of geometric combinatorial complexes. Theoretically, similar to how [1] establishes that higher-order message passing is more expressive than standard graph message passing, this paper demonstrates that the same holds true in the geometric setting. The paper also highlights the effectiveness of the proposed architecture through two novel real-world applications: (1) property prediction over geometric graphs representing molecules, where higher-order cells consist of rings and active groups, and (2)  a regression task over geospatial combinatorial complexes. The proposed architecture outperforms previous geometric graph architectures on these tasks.\n\n\n[1] Cristian Bodnar, Fabrizio Frasca, Nina Otter, Yuguang Wang, Pietro Lio, Guido F Montufar, and Michael Bronstein. Weisfeiler and lehman go cellular: Cw networks. Advances in neural information processing systems, 34:2625–2640, 2021.\n\n[2]  Floor Eijkelboom, Rob Hesselink, and Erik J Bekkers. E (n) equivariant message passing simplicial networks. In International Conference on Machine Learning, pages 9071–9081. PMLR, 2023.\n\n[3] Vıctor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E (n) equivariant graph neural networks. In International conference on machine learning, pages 9323–9332. PMLR, 2021.", "questions": "see weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper offers a simple and straightforward way to adapt higher order message passing to respect $O(d)$ symmetries.\n\n2. The experimental section effectively demonstrates the architecture's performance and introduces two new, interesting real-world TDL benchmarks, addressing a need highlighted in a recent position paper [1]. \n\n\n\n[1] Theodore Papamarkou, Tolga Birdal, Michael M Bronstein, Gunnar E Carlsson, Justin Curry, Yue Gao, Mustafa Hajij, Roland Kwitt, Pietro Lio, Paolo Di Lorenzo, et al. Position: Topological deep learning is the new frontier for relational learning. In Forty-first International Conference on Machine Learning, 2024.", "weaknesses": "1. The novelty of the proposed architecture and the theoretical section is somewhat limited. The architecture closely resembles those presented in [4] and [2]. Additionally, the theoretical contributions feel somewhat straightforward, offering limited new insights.\n\n2. Building on the previous comment about the theoretical section, the paper lacks an analysis of how the choice of invariant function (see Equation (6)) affects the architecture's expressivity. It would be valuable to examine whether simpler, computationally efficient invariant functions could result in architectures that are  as expressive as architectures which use more complex alternatives, thereby guiding the choice of which invariant function to use in practice. Additionally, it would have been insightful to analyze whether there exist any natural geometric invariant functions that  geometric graph models are unable to compute while the proposed model succeeds in doing so.\n\n3. The end of Proposition 2 states \"In most of the cases, an ETNN is strictly more powerful than an EGNN\". I tried finding the proof to this in appendix F and had a hard time. I think the authors refer to proposition 2 in appendix F but I'm not sure. A clearer framing of this result in the appendix, and perhaps an illustrative example or plot in the main body, would improve readability and support this claim.\n\n4. The paper refers to the architecture proposed in [2] for geometric simplicial complexes. A comparison of the expressive power of ETNN using different lifts compared to this architecture would be interesting.\n\n5. The paper [3] benchmarks higher-order message passing on several geometric benchmarks, using data augmentation to address $O(d)$ symmetries. An empirical comparison to this approach would provide valuable insight.\n\n[1] Cristian Bodnar, Fabrizio Frasca, Nina Otter, Yuguang Wang, Pietro Lio, Guido F Montufar, and Michael Bronstein. Weisfeiler and lehman go cellular: Cw networks. Advances in neural information processing systems, 34:2625–2640, 2021.\n\n[2]  Floor Eijkelboom, Rob Hesselink, and Erik J Bekkers. E (n) equivariant message passing simplicial networks. In International Conference on Machine Learning, pages 9071–9081. PMLR, 2023.\n\n[3] Mustafa Hajij, Ghada Zamzmi, Theodore Papamarkou, Nina Miolane, Aldo Guzm´an-S´aenz, Karthikeyan Natesan Ramamurthy, Tolga Birdal, Tamal K Dey, Soham Mukherjee, Shreyas N Samaga, et al. Topological deep learning: Going beyond graph data. arXiv preprint arXiv:2206.00606, 2022.\n\n[4] Vıctor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E (n) equivariant graph neural networks. In International conference on machine learning, pages 9323–9332. PMLR, 2021.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "E940uXyZNn", "reviewer": "Reviewer_J6CW", "summary": "**Method:** The paper introduces a framework, termed ETNN, for processing combinatorial complexes with geometric cell features in an $\\mathrm{E}(n)$-equivariant way (explicitly introduces a method for $0$-cell geometric features and explains how to generalize to geometric features of cells of arbitrary rank). ETNNs are equivariant to the action of both $\\mathrm{Sym}(\\mathcal{X})$ (cell renaming) and $\\mathrm{E}(n)$ isometries acting on the geometric features. The architecture builds on higher-order message-passing over CCs and has both invariant and equivariant versions. ETNNs generalizes previous work on $\\mathrm{E}(n)$-equivariant graph neural networks, and $\\mathrm{E}(n)$-equivariant cellular complex networks.\n\n**Theory:** The expressivity of the proposed methods is evaluated based on its ability to distinguish $k$-hop distinct geometric graphs. The paper proves that in *most cases* ETNN is strictly more expressive than baseline geometric graph methods (i.e. message-passing architectures on geometric graphs that do not operate on higher-order cells).\n\n**Experiments:** \n- **QM9:** The first application considered in the paper is QM9 molecular property prediction, where molecules are represented as combinatorial complexes with geometric features. ETNN variants demonstrate clear performance increase over standard geometric graph methods.\n- **Air Pollution Benchmark:** The second application introduces a new benchmark for air pollution prediction. CCs are constructed from point measurements (0-cells), road measurements (1-cells), and census data (2-cells). \n\n**Notes.**\n- Typo in equation (13); should be \"$x \\subset z$, and $y \\subset z$\".", "questions": "- Can the authors clarify the motivation for using \"k-hop distinct\" graphs as the primary measure of expressivity? Is it possible extend the expressivity results to CCs? E.g. is it possible to define a notion of \"k-hop distinctness\" for geometric combinatorial complexes and analyze ETNN's ability to distinguish between such complexes?\n- How does ETNN's expressivity compare to that of simplicial complex networks in distinguishing non-isomorphic geometric graphs/complexes?\n- In proposition 2, can you specify the conditions under which ETNNs are provably more expressive than EGNNs? \n- Could you include standard deviations for the air pollution benchmark and QM9 results to assess statistical significance?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- **Framework and Theoretical Contributions:**\n  - The ETNN architecture introduced in the paper is a novel framework for $\\mathrm{E}(n)$-equivariant processing of a wide class of topological data objects.\n  - The framework elegantly unifies and generalizes existing architectures -- e.g. EGNN [1] and EMPSN [2].\n  - The paper theoretically proves expressivity improvements over $\\mathrm{E}(n)$-equivariant graph methods.\n\n- **Empirical Results:**\n  - ETNN variants achieve clear performance gains over standard $\\mathrm{E}(n)$-equivariant graph methods for QM9 molecular property prediction tasks; ETNNs also improves upon EMPSNs while using less memory and having a faster runtime.\n\n- **Applications and Dataset Contributions:**\n  - The paper introduces a principled approach to modeling irregular multi-resolution geospatial data using combinatorial complexes.\n  - The *air pollution downscaling* benchmark introduced in the paper is a new dataset for benchmarking TDL architectures. Additionally, the construction and analysis of geospatial combinatorial complexes is in itself an interesting contribution of the paper.\n\n[1] Satorras et al. \"E(n) equivariant graph neural networks\", ICML 2021.\n\n[2] Eijkelboom et al. \"E(n) equivariant message passing simplicial networks\", ICML 2023.", "weaknesses": "- **Novelty:** While the proposed method generalizes previous work to general combinatorial complexes, the architectural changes are incremental.\n  \n- **Theoretical analysis:**\n  - The expressivity analysis focuses solely on distinguishing geometric *graphs*, despite defining expressive power in terms of separating more general non-isomorphic *CTSs*.\n  - Proposition 2's statement is imprecise: the claim of improved expressivity \"in most cases\" lacks formal qualification, and the relationship between the expressivity gap and choice of lifting method needs clearer formulation and concrete characterization. A clearer restatement would be helpful.\n  - No formal comparison to expressivity of other TDL methods (e.g. equivariant simplicial networks).\n\n- **Empirical evaluation:**\n   - Results lack statistical significance analysis (no standard deviations reported).\n  - On the air pollution benchmark, improvements over the MLP baseline are modest (~1.5% RMSE reduction) and their statistical significance cannot be assessed due to unreported standard deviations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)