MANTRA: The Manifold Triangulations Assemblage
Authors: Rubén Ballester
Ernst Röell
Daniel Bin Schmid
Mathieu Alain
Sergio Escalera
Carles Casacuberta
Bastian Rieck
Venue: ICLR 2025 Poster (2025)
Keywords: simplicial complex
topological deep learning
high-order
high-order dataset
simplicial complex learning
Primary area: datasets and benchmarks
TL;DR: We introduce MANTRA, the first large-scale, diverse, and intrinsically high-order dataset for testing graph and topological models.

Abstract: The rising interest in leveraging higher-order interactions present in complex systems has
led to a surge in more expressive models exploiting higher-order structures in the data,
especially in topological deep learning (TDL), which designs neural networks on higher-order domains such as simplicial complexes. However, progress in this field is hindered
by the scarcity of datasets for benchmarking these architectures. To address this gap, we
introduce MANTRA, the first large-scale, diverse, and intrinsically higher-order dataset for
benchmarking higher-order models, comprising over 43,000 and 250,000 triangulations
of surfaces and three-dimensional manifolds, respectively. With MANTRA, we assess
several graph- and simplicial complex-based models on three topological classification
tasks. We demonstrate that while simplicial complex-based neural networks generally
outperform their graph-based counterparts in capturing simple topological invariants, they
also struggle, suggesting a rethink of TDL. Thus, MANTRA serves as a benchmark for
assessing and advancing topological methods, paving the way towards more effective
higher-order models.

Published as conference paper ICLR 2025
MANTRA: THE MANIFOLD TRIANGULATIONS ASSEMBLAGE
Rubén Ballester" 111.2 Ernst Röell 223, Daniel Bin Schmid 2,2,,4 Carles Casacuberta'
Mathieu Alain" *
Sergio Escalera
Bastian Ricck 2.3,4
"These authors contributed equally to this work 'Departament de Matemàttquus i Informàtica, Universitat de Barcelona, Spain
"AIDOS Lab, University of Fribourg. Switzerland
Innstitute of Al for Health, Helmholtz Munich, Germany
"Technical University of Munich, Germany
*Centre for Artificial Intelligence, University College London, UK
"Computer Vision Center, Spain
ABSTRACT
The rising interest in leveraging higher-order interactions present in complex systems has led to a surge in more expressive models exploiting higher-order structures in the data, especially in topological deep learning (TDL), which designs neural networks order domains such as simplicial complexes. However, progress in this field is hindered by the scarcity of datasets for benchmarking these architectures. To address this gap, we introduce MANTRA, the first large-scale, diverse, and intrinsically higher-order dataset for benchmarking higher-order models, comprising over 43,000 and 250,000 triangulations of surfaces and three-dimensional manifolds, respectively. With MANTRA, we assess several graph- and simplicial complex-based models on three topological classification tasks. We demonstrate that while simplicial complex -bsed neural networks generally outperform their graph-based counterparts in capturing simple topological invariants, they also struggle, suggesting a rethink of TDL. Thus, MANTRA serves as benchmark for assessing and advancing topological methods, paving the way towards more effective higher-order models.
higher-
INTRODUCTION
Success in machine learning commonly measured by model's ability solve tasks on benchmark datasets. While researchers typically devote a large amount of time to build their models, less time is devoted to data and its curation. As a consequence, graph learning is facing some issues terms of reproducibility and wrong assumptions, which serve as obstructions to progress. An example of this was recently observed while analyzing long-range features: additional hyperparameter tuning resolves performance differences between pessaag-passsng (MP) graph neural networks on one side and graph transformers on the other (Tönshoff et al., 2023). similar vein, earlier work pointed out the relevance of better taxonomies for existing datasets (Liu al., 2022), well the need for strong basclines, highlighting the fact that structural information is not exploited equally by all models (Errica et al., 2020). Recently, new analyses even showed that for some benchmark datasets, well their associated tasks, graph information may be detrimental for the overall predictive performance (Bechler-Speite et al., 2024; Coupette et al., 2025), raising concerns about the future of graph learning as research field (Bechler-Speicher et al., 2025).

---

Published as a conference paper at ICLR 2025
These troubling trends concerning data are accompanied by increased interest in leveraging higher-order structures in data, with new models, usually called topological models, extending graph-learning concepts to simplicial complexes, i.e., generalizations of graphs that incorporate higher-order relations, going beyond the dyadic relations captured by graphs (Alain et al., 2024a; Bodnar et al., 2021c; Maggs et al., 2024; Ramamurthy et al., 2023; Röell & Rieck, 2024; Yang et al., 2024). Some topological models already incorporate state-of- the-art mechanisms for learning such as message- message-passing (Gilmer et al., 2017) or transformer layers (Ballester et al., 2024), but adapted to higher-order domains, sometimes outperforming their original counterparts in graph datasets. However, as pointed out in recent position paper (Papamarkou et al., 2024), there is a dire need for "higher-order datasets," i.e., datasets that contain non-trivial higher-order structures. The scarcity of such datasets impedes the development of reliable benchmarks for assessing (i) the utility of higher-order structures present in data, and (ii) the performance of new models that leverage them, thus potentially croding trust in topological models among the broader deep learning community.
Some of the aurrlatly-aviilable higher-order datasets belong to the realm of networks, complex systems, and the life sciences. Benson et al. (2018) present a rich collection of such datasets, comprising nineteen complex networks enhanced with higher-order information. Similar works have also employed higher-order structures in data; for instance. Tadié et al. (2019) use clique complexes on top on graphs coming from brain imaging data. Similarly, Giusti et al. (2016) propose modeling neural data with simplicial complexes by constructing clique, concurrence (and its dual), and independence complexes on the data. However, most of these datasets are either annotated or derived from simpler data like graphs or time series. In the case of annotated data, it is unclear whether current non-higher-orddr (graph) neural networks or algorithms can extract the information contained in the higher-order structures using only annotations vertices and edges. Similarly, for datasets obtained from simpler data, it is also uncertain whether non-higher-orde algorithms recover the higher-order structural information by reconstructing the processes used to generate these relationships explicitly. To the best of our knowledge, the only publicly-available purely higher-order dataset is the "Torus" dataset proposed in Eitan et al. (2025), which consists of a small number of unions of torus triangulations. Due to the nature of the dataset, the only varying topological property among the samples is the number of connected components of each union, making it hard assess the true capacity of the models to learn and exploit higher-order structures. The lack of higher-order datasets is also remarked upon in a recent benchmarking paper for topological models (Telyatmikov et al., 2024), which restricts itself existing graph datasets that were subjected to variety of topological liftings, i.e., methods for endowing graph datasets with higher-order structures (Bernárdez et al., 2024; Jonsson, 2007). remains unelear whether graph neural network architectures can also learn and take advantage of the information provided by the topological liftings, as they are solely based on the graph structure.
Contributions. To address these issues, we present MANTRA, the manifold triangulations assemblage, which constitutes the first instance of a large, diverse, and intrinsically higher-order dataset, consisting of triangulations of combinatorial 2-manifolds and 3-manifolds. Along with the data, we provide a list of tasks, as well as a preliminary assessment of the performance of existing methods, both graph-based and segher-order-basede We focus on a subset of tasks concerned with the classification of simplicial complexes according to topological labels, where we can interpret the success of a model by its effectiveness in extracting higher-order topological information. However, these tasks are by no means exhaustive, and we believe that the generality offered by MANTRA encourages the emergence of even more demanding tasks. Some of these tasks, such as the prediction or approximation of the Betti numbers from topological data, have been previously studied in learning (Paul & Chalup. 2019) and non-learning (Apers et al., 2023) contexts. A noteworthy aspect of MANTRA is the conspieuous absence of any intrinsic vertex or edge features such coordinates or signals. We believe that this absence renders tasks more topological, as models can only rely on topology, instead of non-topological information contained in features. Morcover, as manifold triangulatioons are directly related to the topological structure of the underlying manifold, we study to which extent higher-order models are invariant to transformations of a triangulation that preserve the topological structure of the associated manifold.
2

---

Published as a conference paper ICLR 2025
Geometric realizations of some manifold triangulations included in MANTRA. The precise coordinates of vertices in Euclidean space are not geometrically significant; what matters is the topology of the resulting polyhedra. Hence, MANTRA is a purefy combinatorial dataset.
Figure 1:
2 DATASET SPECIFICATION
MANTRA contains 43,138 and 250,359 simplicial complexes corresponding to triangulations of closed connected two- and three-dimensional manifolds, respectively, with varying number of vertices, originally curated by Frank H. Lutz and compiled in Lutz (2017). Manifolds have many applications: The configuration space of robotic arm can be seen as a manifold (e.g.. a torus or hyperbolic space, see Jaquier et al., 2022); 3D shapes in geometry processing are triangulated manifolds (Crane & Wardetzky, 2017); physical fields in climate forecasting naturally live on a sphere (Bonev et al., 2023), and the manifold hypothesis argues that nigh-dimensional data often lies in or close lower-dimensional manifolds (Fefferman et al., 2016). Throughout the text, we use the term surface to refer to two-dimensional manifold. A triangulation of a manifold M is a pair consisting of simplicial complex K and homeomorphism between M and the geometric realization of K. For brevity, we use the term triangulation to refer exclusively to the simplicial complex K. See Appendix A.3 for precise definitions and further information.
Table I: Number of triangu- lations by manifold dimen- sion 12-M: 2-manifolds; 3-M: 3-manifolds) and number of vertices [V] in a triangulation.
Triangulations of surfaces and 3-manifolds encode higher-order topological information that cannot be inferred solely from their underlying graphs. In- deed, there exist non-homeomorphin surfaces with identical graph structures. Specifically, for n > 7, the complete graph with n vertices triangulates both connected sum of tori and connected sum of projective planes, which are non- homcomorphic (Lawrencenko & Negami, 1999). Figure I contains exam- ples of geometric realizations of MANTRA triangulations. Table 1 contains the distribution of triangulations in terms of their number of vertices. Each triangulation contains a of labels based on its dimension. Common labels are the number of vertices of the triangulation, the first three Betti numbers Bu. B1, B2, and torsion in homology with integer coefficients. Appendix A.2 provides definitions for these concepts. For triangulations of Klein bottle K, real projective plane IRP2. a 2-dimensional sphere S or torus T the homeomorphism type is included explicitly as surface label. We additionally specify the top Betti number 83 and the homeomorphism type, which can be 3-sphere 53, product S2 x S of 2-sphere and circle, a Möbius-like 5 5a-bundle along S denoted by S² X s'. An exploration of the distributions of labels is made in Appendix A.5.
VI
2-M
3 M
4
1
5
1
6
3
2
7
9
43
39
1,297
10
42,426
249,015
Total
43,138
250,359
We make the dataset and benchmark code available via two repositories:
i
3

---

Published as a conference paper ICLR 2025
These repositories contain (i) the raw and processed datasets, and (ii) the code reproduce all our results. Detailed hyperparameter settings can be found in Appendix D. Step-by-step instructions on how to set up and execute the benchmark experiments are attached in the README file of the repository. Docker images and workflow, together with package dependencies are included to ensure unique environment across different machine configurations. Finally, random seeds were used to split the datasets each run.
The dataset is available in two formats, namely raw version and version for PyTorch Geomet- ric. The raw version currently consists of a pair of compressed files 2_mani folds json, gz and 3 manifolds json. gz, containing JSON list with the triangulations of the corresponding dimension. Each object of the JSON list consists of a set of the following fields, depending on the dimension of the associated triangulation:
FIELD
TYPE
DESCRIPTION
id
str
This attribute refers to the original ID of the triangulation as used by Lutz (2017) when compiling the triangulations. This facilitates comparisons to the original dataset if necessary.
triangulation
list
A doubly-nested list of the facets the triangulation.
n_vertices
int
The number of vertices in the triangulation.
name
str
Homeomorphism type of the triangulation. Possible values are ' Kiein bottle', 'R2 2', S "2",, "I"21 for surfaces, where indicates that the explicit homeomorphism type is not available. For 3-dimensional manifolds, possible values are 'S"2 twist s^1', "S"2 x 1' 'S"
betti_numbers 1ist
A list of Betti numbers of the triangulation, computed using Z. i.e., integer coefficients.
torsion
1ist
A list of the torsion subgroups of the triangulation. Possible values are 2 2", where an empty string indicates that no torsion present in that dimension.
genus orientable
int
For surfaces, contains the genus of the triangulation. For surfaces, specifies if the triangulation is orientable or not.
bool
The PyTorch Geometric (Fey & Lenssen, 2019, PyG) version available as a Python package that be installed using the command pip install 11 mant.raddat implemented as a PyG Data object, containing the same attributes as JSON objects in the raw version. The main difference with the data in the raw version is that numerical values are stored as PyTorch tensors. Both formats, raw and processed, are versioned using the Semantic Versioning 2.0.0 convention (Preston- Werneer) and are also available via Zenodo, thus ensuring reproducibility and clear tracking of the dataset evolution.
Each example of the dataset is
Dataset limitations. In the present version of MANTRA, triangulations are restricted to two- and three- dimensional complexes up to 10 vertices. This may pose a limitation concerning the transferability of our findings to datasets with significantly higher number of vertices per sample, such as fine-gran fine-grained mesh datasets. While extending the dataset beyond 10 vertices is theoretically possible, it poses substantial storage and computational challenges due to the exponential growth possible triangulations for example, there are over 11 million surfaces for triangulations of 11 vertices andtices-anddl the unavailability of complete enumerations of triangulations for more than 13 vertices, which may potentially lead to incomplete datasets and skewed label distributions. Note that this does not preclude the addition of triangulations with other properties, for instance certain minimality properties (like certex-transitive triangulations). Additionally, focusing solely on two- and lire-diimnnional manifolds excludes nigher-dimension triangulations and data, which remain active areas of research. Nevertheless, we believe that MANTRA provides a valuable benchmark for testing
/nttps://dci.orgg//
14103581
4

---

Published as a conference paper ICLR 2025
higher-order models on the most common types higher-order structured data, that is, graphs, surfaces, and volumes. Finally, we want to highlight the fact that MANTRA does not (yet) encompass the full spectrum of properties present in real-world data- for example, the large simplicial complex sizes found in complex networks the geometric information contained in some datasets like the existence of vertex coordinates meshes. Therefore, although we consider MANTRA a valuable dataset for testing the capabilities of higher-order models, it should be studied in conjunction with other conceptually diverse datasets to better study the capabilities models.
3 EXPERIMENTS
TL;DR:
We assess twelve state-of-the-art simplicial complex- and graph-based architectures, on various topological prediction tasks such Betti number homeomorphism type classification, and orientability detection. Our experiments confirm that simplicial complex-based neural networks almost always achieve better results than graph-based ones in extracting the topological invariants mentioned above. However, we also find that the performance of the assessed models may be suboptimal, given that they use the moniker "topological models." In particular, model performances deteriorate when applying barycentric subdivisions to the original test datasets, suggesting that the tested models are unable to learn topologically invariant functions.
discover that all
Sections 3.1 and 3.2 presents the comprehensive experimental design for MANTRA, outlining the key scientific questions addressed. Section 3.3 provides a detailed analysis of the experimental results.
3.1 MAIN EXPERIMENTS
In this section, we demonstrate MANTRA's effectiveness comprehensive benchmark for higher-order models. Leveraging the extensive set of labels and triangulations available, our experiments are designed to address the following critical research questions:
Q1 To what extent are higher-order models needed to perform inference tasks on higher-order domains like simplicial complexes? Are graph-based models enough successfully capture the full set of combinatorial properties present in the data?
Q2 Do current neural networks, both graph- and simplicial complex-based, capture topological properties in data? Are they able to predict basic topological invariants such as Betti numbers of simplicial complexes?
Q3 How invariant are state-of- the-hhhr art models to transformations that preserve topological properties of data?
The difference between Q1 and Q2, Q3 is subtle. Combinatorial information is related to the structure of the data, input values, in our case, simplicial complexes, while topological information related properties that are invariant under topological transformations of the data. For example. in prediction tasks involving molecules, we expect combinatorial information than topological features, since the structure of molecule is crucial in predicting properties of the molecule. both types of information are intertwined: to properly compute topological properties data, we need to consider its the input as explained in Appendices A.2 and A.3. To answer the above questions, we benchmarked twelve models: five graph-based (Fey & Lenssen, 2019), using only zero- and one-dimeed mie-dnnnniionl simplices of complexes, and seven simplicial complex-bart based models, four from the TopoModelX library (Hajij et al., 2024) and three extra cellular models, using the full set of simplicial complexes in five different tasks per manifold dimension:
T1 Predicting the Betti numbers 3, for triangulated surfaces and 3-dimensional manifolds. T2 Predicting the homeomorphism type of triangulated surfaces.
T3 Predicting orientability of triangulated surfaces.
5

---

Published as a conference paper at ICLR 2025
These tasks are not challenging when viewed in the context of algebraic topology, since explicit algorithms for all of them are known. However, we are interested in analyzing to what extent neural network can provide (approximate) solutions here, with the goal of ultimately addressing more challenging tasks in (computational) algebraic topology. To address the high proportion of surfaces without explicitly assigned homeomorphism type, we duplicated the experiments on both the full set of surfaces and the subset of surfaces with known type. Throughout the paper, we denote by 2- 2-M". 2- -Mrr and 3- 3- the full set of surfaces, the set of surfaces with known homcomorphism type, and the full set of 3- B-manifolds, respectively.
Models. The graph-based models benchmarked are the Multi-Layer Perceptron (MLP), the Graph Con- volutional Network (Kipf & Welling, 2017, GCN), the Graph Attention Network (Veličković et al., 2018, GAT), the Graph Transformer (Shi al., 2021, UniMP), and the Topology Adaptive Graph Convolutional Network (Du et al., 2017, TAG). The simplicial complex-based benchmarked models are the Simplicial Attention Network (Giusti et al., 2022, SAN), three convolution- basood simplicial neural networks previously benchmarked in Telyatnikov et al. (2024) and introduced in Yang et al. (2022, SCCN), Yang & Isufi (2023, SCCNN), and Wu et al. (2024, SCN), respectively, the cellular message passing from (Bodnar et al., 2021b, CellMP), the cellular transformer (Ballester et al., 2024, CT), and the Differentiable Euler Characteristic Trans- form (Röell & Rieck, 2024, DECT). Note that except for the MLP model, the graph and cellular transformers, and the DECT, all models implement some variant of a (higher-order message- passing paradigm (Hajij et al., 2023; Papillon et al., 2024). More information about the models can be found in Appendix C.
Features. All twelve models assume that simplicial complexes are equipped with feature vectors on top of a subset of the simplices. The feature vectors for graph-baser models and DECT are either: (1) scalars randomly generated, (2) degrees of each vertex, (3) degree one-hot encodings of each vertex. For the rest of simplicial complex-based models, the feature vectors are either: (1) eight-dimensional vectors generated randomly, (2) number of upper-adjacent neighbors aupper-connectviigy index) of each simplex of dimensions lower than the dimension of the simplicial complex and number of lower-adjacent neighbors (lower- connecccnnettivity index) for simplices of the same dimension as the simplicial complex. By defimition, two simplices are upper-adjacent, and both are upper-adjacent neighbors of the other, if they share coface of one dimension higher. Similarly, two simplices are lower-adjacent if they share a face of one dimension lower.
Training details. In total, our experiments span 240 training results across various tasks, feature generation, and models. To ensure fairness, all configurations use the same learning rate of 0,01 and the same number of epochs of 6; we observe that graph- based models already overfit after a single epoch, though. Hyperparameters for graph-based models were mostly extracted from the default examples from Py PyToorch Geometric, while hyperparameters for simplicial complex-based models were set to values similar to the ones from the TopoBenchmarkX paper (Telyatnikov et al., 2024), specially for the already benchmarked SCCN, SCCNN, and SCN models. Hyperparameter details can be found in Appendix D. To mitigate the effects of training randomness, we re-ran each experiment five times and considered both the best and the mean (together with standard deviation) performance obtained across these runs for each model and initialisation of features. Due the high imbalance in the datasets for most labels, we performed stratified train/validation/tes splits for each task individually, with 60/20/20 percentage of the data for each split, respectively. Splits were generated using the same random seed for each run, ensuring that the same splits are used across all configurations. All models were trained using the Adam optimizer.
Loss and metric functions. Each task (T1, T2, T3) was treated as a classification task during testing. We report the area under the ROC curve (AUROC) (Bradley, 1997) as performance metric. which is standard for imbalanced classification problems, on all tasks except for predicting 8o. where we report accuracy due the fact that we only have the label 1, all our triangulations correspond to connected manifolds. For both the homeomorphism type and orientability tasks, we train the models using the standard cross-entropy loss for classification problems. We also experimented with weighting the cross-entropy loss penalize
6

---

Published as a conference paper ICLR 2025
mispredictions in under-represented classes more heavily, but we did not obtain improvements. To avoid increasing the computational complexity of our experiments, we chose not to implement more involved methods for handling the class imbalances and leave this issue for future work. For Betti number prediction, we approached training
multivariate regression task, since Betti numbers can theoretically be arbitrarily large. Our loss function in this case was the mean squared error, and the Betti number prediction was obtained by rounding the model outputs to the nearest integer.
3.2 BARYCENTRIC SUBDIVISION EXPERIMENTS
The previous experiments try to answer Q1 and Q2: if performances are good for simplicial complex-based graph-based ones, then we can conclude that higher-order models are needed to perform inference tasks on domains with higher-order and topological information. By contrast, if performances are good for graph-based models then we can conclude that graph models are enough to capture the full set of combinatorial and topoloical features present in MANTRA's dataset, questioning the need for higher-order models. However, Q3 is more subtle. Although it is closely related to Q2, Q3 emphasizes the invariance of the models to transformations that preserve the topological properties of the input data, desirable property for TDL models known as remeshing symmetry (Papamarkou et al., 2024). For example, if a model is well-trained with dataset containing only triangulations up to a certain number of vertices, we can expect the model to perform correct predictions in new examples that also have at most the maximum number of vertices seen in the training dataset. However, what happens we try to predict from refinement of a manifold triangulation? For instance, barycentric subdivisions increase the (combinatorial) distances between the original vertices in a triangulation, and this can be harmful for networks relying on the MP algorithm, since distances determine how many layers are needed to propagate information from one vertex to another. In fact, Horn et al. (2022) showed that MP-based graph neural networks with a small number of layers struggled to obtain good performances on synthetic datasets where the number of cycles and connected components played a crucial role.
To answer Q3, performed additional evaluation of the models trained on surface tasks with known homeomorphism type for the experiments described in Section 3.1. Particularly, for each task, we evaluated the performance of the trained models dataset obtained by performing one barycentric subdivision on each triangulation in the original test dataset, and then we compared the performances of the models on both datasets, original and subdivided. Throughout the text, we denote the subdivided test dataset 2 MMI. We did not analyze barycentric subdivisions of 3-dimensional manifolds due to computational constraints. Also, for these experiments, we leave out the DECT model from the analysis, since the DECT invariant with respect to barycentric subdivision by construction if used appropiately.
3.3 ANALYSIS
Our analysis reports aggregated results and focuses primarily on the comparison between graph-based models (9) and simplicial complex based models (T). Comprehensive results are available in Appendix E. Table 2 presents the mean and standard deviation of the maximum performance achieved by each combination of feature vector initialization and model type across the 5 runs of each task for both graph-based (g) and simplicial complex- based (T) model families, including performances the test triangulations for each experiment run in the set of surfaces with known homeomorphism type, as described in Section 3.2. Notably. our experiments suggest that higher-order MP-based and transformer models are not invariant relative to topological transformations and therefore cannot be considered topological in the strictest sense of the term: higher-order models predicting better than random in any task suffer from a performance degradation when testing on the subdivided examples, as shown by the full set of results in sections E. to E.2 and tables 2 and 11.
the barycentric subdivisions of
7

---

Published as a conference paper ICLR 2025
Table 2: Predictive performance of graph- and simplicial complex-basee models on surface and m-nniflldd tasks. Results for the full set of surfaces (2-M²), for the set of surfaces with known homeomorphism type (2-M% h). ). and for the full set of three-manifolds (8) are reported. Additionally, performance metrics for the barycentric subdivision of the test set on the models trained on 2 D-M i.e., 2 M are included; see Section 3.2 for details. For each family of models, ς (graph-based) and 7 (simplicial complex-based we report the mean and standard deviation of the maximum performance achieved across five runs by each combination of feature vector initialization and model contained in the family. The tasks reported are prediction of Bp. $1. 32, S3. prediction of the homeomorphism type, and prediction of orientability. For all tasks except for prediction of Bu, we report the AUROC metric. For So. we report accuracy. Best average result among both families for each task in bold. Note that the reported averages and standard deviations are not calculated from individual model performances across different random seeds. Instead, for each model, we selected its best performance achieved across all seeds for each experiment. Then, we aggregated these best performances within each category- graph-based and simplicial complex-based models compute the averages and standard deviations reported in the table.
Accurucy
AUROC
DATASET
MODEL FAMILY
3
31
A2
8g
HOMBO,
TYPE
ORIENTABILITY
G
1,00 0.00
0.50 0.00
0.50 &: 0.00
0.47 + 0.01
0.50 0.00
2-
T
0.73 0.39
0.68 0.16
0.59 + 0.10
0.69 # 0.18
0.56 0.07
2- 2-M9
G
1,00 = 0.00
0.21 + 0.00
0.50 + 0.00
0.49 + 0.01
0.50 = 0.00
T
0.57 = 0.44
0,25 0.03
0.52 = 0.02
0.66 =
0.52 = 0.02
2-M
G
0.47 # 0.51
0.22
0,50 +: 0,00 0.51 + 0.01
0.49 +:2 0,04
0,50 = 0,00
T
0.21 = 0.38
0.25
0.60 + 0.10
0.51 # 0.01
G
1.00 0.00
0.23 0.00
0.12 a: 0.00
0.14 0.00
0.14 0.00
3-
T
0.78 0.41
0,25 = 0.04
0,13 0.03
0.16 0.03
0.15 0.02
Weaknesses in the MP-based models not recent phenomenon, highlighted by oversmoothing (Li et al., 2018) and oversquashing (Alon & Yahav, 2021; Topping et al., 2022), and the MP paradigm has required numerous fixes since its existence (inleuding, but not limited to, virtual nodes, feature augmentation, and graph lifting). More recently, Eitan et al. (2025) argued that, in many cases, higher-order MP-based models cannot distinguish combinatorial objects based on simple topological properties, and has devised another MP variant to compensate for this.
Graph-based (g) vs. simplicial complex-based (T) models. Table 2 together with the full results of Appendix E show that simplicial complex-based models consistently obtain better equivalent performances predicting non-trivial topological properties of triangulated manifolds, meaning 31, 32, 83. orientability, and homeomorphism type. We note that graph models always correctly detect the connectivity of triangulations in 2- MM, 2 2M and 3-dimensional manifolds, thus predicting So exactly, while topological models consistently fail to predict connectivity, except for the CT. DECT, and SCCN architectures in our experiments. The fact that some higher-order message passing networks cannot accurately predict connectivity was also found, and theoretically proved, in Eitan al. (2025, Proposition 4.3). Moreover, although simplicial complex-based models obtain better results overall, these are far from being highly accurate, with averages below 70 for all tasks, a high performance variance across the models some tasks, and tasks for which simplicial complex-based models obtain performance similar to random- -uulssingg strategy. Nonetheless, the best performances obtained by specific simplicial complex complex-based models, as described in the full results of Appendix E, are promising, achieving excellent AUROC results in some tasks, such as homeomorphism type prediction, where the CT and SCCN models obtained average AUROCs of 91 and 83 and 85 and 80 for the full and known homeomorphism type surface datasets, respectively, and Betti number prediction for the full
8

---

Published as a conference paper at ICLR 2025
set of surfaces, where the CT and SCCN models obtained an average AUROC of 93 when predicting the first Betti number. Overall, the results suggest that higher-order models are indeed necessary to capture topological and higher-order characteristics of data, although several current models are not yet able to do so effectively, partially answering questions Q1 and Q2. Such results were expected, given that one-dimensional structures are insufficient, in principle, to fully characterize the topology of two- or three-dimensional triangulated manifolds, as stated at the beginning of Section 2. However, it is plausible that graph-based networks can accurately classify approximately 50% of homeomorphism types of surfaces, since the underlying graph of triangulation determines the Euler characteristic, which in turn defines the homeomorphism type up to orientability (see Appendix A.4).
Orientability. Predicting orientability turns out to be the most difficult task for graph- and simplicial complex-based based models, obtaining performances equivalent random guessing in most cases for surfaces and performances worse than random for 3-manifolds. Moreover, we do not find significative differences between the performances of predicting the Betti number 32 and orientability for surfaces binary problem. This is consistent with the similar results obtained for predicting B2 and orientability type as a binary classification problem for surface datasets.
Barycentric subdivisions. Table 2 shows that the performance of all models decreases when subdividing the triangulations of the test dataset if the models were performing better than random guessing, indicating that the models are not learning the invariance of topological properties with respect subdivisions transfor- mations that leave topological properties invariant. This is crucial property that any model dealing with topological domains should have, as real data is often highly variable in terms of combinatorial information and representation, but not in terms of their topology. This phenomenon is particularly evident in mesh datasets, where combinatorial structure varies with resolution. Meshes typically comprise more triangles, yet all or nearly all input data represent triangulations of connected closed surfaces, regardless of resolution. In fact, Papamarkou et al. (2024) posit the capacity of TDL. models to capture this invariance, denoted remeshing symmefry, one of the reasons for using topological deep learning models. Our preliminary experimental results challenge this claim, opening the door to new line of research based on the invariance of input transformations that leave topological properties of the input data unaltered. This is important from the perspective of model nomenclature, as many current topological models can be viewed as graph-based approaches that handle heterogeneous node types, with simplices of different dimensions effectively treated as distinct node categories. We believe that if TDL is to emerge as a distinct research area, it must advance the field by introducing higher-order mechanisms that cannol be replicated through straightforward adap- tations of existing graph models, effectively moving beyond (unaugmented) message-passing approaches. Also, we believe that some tools like (persistent) homology or MAPPER (Singh et al., 2007), which are presently not used in our experiments, could potentially be used to address or at least alleviate this issue: The expressivity of persistent homology in the context of graph learning has already been studied, and was proven to provide complementary information to traditional Immonen et al., 2023; Rieck et al., 2019), whereas MAPPER was shown to be an effective graph-pooling strategy (Bodnar et al., 2021a). Beyond these two frameworks, we believe that other techniques, drawing upon geometrical ceommetricul-topological concepts, can address this challenge.
pesssage-passingg approaches (Ballester & Rieck, 2024;
Experimental limitations. Although our results challenge the efficiency of state-of-the-an art higher-order models to predict topological properties of data and open the door to exciting new research avenues, they must be interpreted with care. For example, we mostly tested message-passing networks in our experiments, leaving aside interesting proposals such as higher-order state-space models (Montagna al., 2024), combinatorial complex networks (Bodnar et al., 2021b; Hajij et al., 2023), topological Gaussian processes (Alain et al., 2024b) equívariant higher-order neural networks (Battiloro al., 2025). Due computational limitations, training procedures were limited to 6 epochs, model hyperparameters were not necessarily selected optimally, and barycentric subdivisions experiments were limited to one subdivision. A significant computational
9

---

Published as a conference paper ICLR 2025
bottleneck arose from the implementations of simplicial complex-based models, which processed data noticeably slower than their graph counterparts as observed in Table 7, highlighting the need for more efficient implementations of TDL methods as well as conceptual improvements. Despite these limitations, we believe that each of our initially-stated questions should be investigated individually, requiring broader set of experiments and ablations to fully answered.
4 CONCLUSION
We proposed MANTRA, a higher-order dataset of manifold triangulations that is (i) diverse, containing triangulations of surfaces and tiree-dimensiona manifolds with different topological invariants and homeo- morphism types, (ii) large, with over 43,000 triangulations of surfaces and 250,000 triangulations of three- dimensional manifolds, and (iii) naturally higher-order, the triangulations are directly related to the topological structure of the underlying manifold. Using MANTRA, we observed that existing models, both graph-based and grgher-order-baseee struggle to learn topological properties of triangulations, such as the orientability of two-dimensional manifolds, which was the hardest topological property to predict for surface triangulations, suggesting that new approaches are needed leverage higher-order structure associated with the topological information in the dataset. However, we also observed that current higher-order models outperform graph-based models in our benchmarks, substantiating the promises of this new trend of higher-order machine-learning models. Regarding invariance, we observed that barycentric subdivision deeply affects the performance of the models, suggesting that current state-of-the the- models fail to be invariant to transformations that preserve the topological structure of data, opening an interesting research direction for future work. In the case of MP-based models, this could be potentially related to sensitivity of message-passing to the distances between simplices in simplicial complexes. Another interesting research direction for barycentric subdivisions is their application as inputs to graph neural networks. The induced graph of barycentric subdivision represents each simplex of the original complex vertex, with edges encoding face relationships on the original complex. This structure provides an effective representation of simplicial complexes for graph-based neural architectures, potentially facilitating the processing of higher- order topological information. We hope that MANTRA will serve as a benchmark for the development of new models leveraging higher-order and topological structures in data, and reference for the development of new higher-order datasets.
ACKNOWLEDGMENTS
The authors are indebted to Corinna Coupette, the anonymous reviewers, and the area chair for their helpful comments and for believing in our work. RB, SE, and CC were supported by the Ministry of Science and Innovation of Spain through projects 136436NB-I00. RB was additionally supported by the Ministry of Universities of Spain through the FPU contract FPU2 1/00968. RB and CC were also supported by the Departament de Recerca Universitats de Generalitat de Catalunya (2021 SGR 00697), and SE was additionally supported by ICREA under the ICREA Acadèmia programme. MA was supported by Mathematical Sciences Doctoral Training Partnership held by Prof. Helen Wilson, funded by the Engincering and Physical Sciences Research Council (EPSRC), under Project Reference EP/W523835/1. This work has received funding from the Swiss State Secretariat for Education, Research, and Innovation (SERI).
100119--05993BBUUS I00, PID2020- 1000-11770 119971GB C22, and PID2022-
The authors wish to dedicate this work to the memory of Frank H. Lutz (1968-2023) who started a collection of triangulations one of his many research endeavors.
10

---

Published as a conference paper ICLR 2025
REFERENCES
Elements of Algebraic Topology. Addison- Wesley Publishing Company, 1984. (Cited page 16.)
Mathieu Alain, So Takao, Brooks Paige, and Mare Peter Deisenroth. Gaussian Processes on Cellular Complexes. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning (ICML), volume 235 of Proceedings of Machine Learning Research, pp. 879-905. PMLR, 2024a. (Cited on page 2.)
Mathieu Alain, So Takao, Bastian Rieck, Xiaowen Dong, and Emmanuel Noutahi. Graph Classification Gaussian Processes via Hodgelet Spectral Features. In NeurIPS Workshop Workshop on Bayesian Decision- making and Uncertainty (BDU), 2024b. URL page 9.)
http:///bs/24400..0 2410. 10546. 6. (Cited on
Uri Alon and Eran Yahav. On the Bottleneck of Graph Neural Networks and its Practical Implications. In International Conference on Learning Representations (ICLR), 2021. URL https::/ (Cited on page 8.)
net/
Simon Apers, Sander Gribling, Sayantan Sen, and Dániel Szabó. A (Simple) Classical Algorithm for Estimating Betti Numbers. Quantum, 7:1202, 2023. doi: 10.22331/q-2000 2023- 12-06- 1202. (Cited on page 2.)
Rubén Ballester and Bastian Rieck. On the Expressivity of Persistent Homology in Graph Learning. In Proceedings of the Third Learning on Graphs Conference (LoG), 2024. In press. (Cited on page 9.)
Rubén Ballester, Pablo Germández-Gareit Mathilde Papillon, Claudio Battiloro, Nina Miolane, Tolga Birdal, Carles Casacuberta, Sergio Escalera, and Mustafa Hajij. Attending to Topological Spaces: The Cellular Transformer, 2024. URL https: /ttwavavvvv///.///////e..//aawva org/ abs /2005 1 14094. (Cited on pages 6, and 24.)
Claudio Battiloro, Ege Karaismailoğlu, Mauricio Tec, George Dasoulas, Michelle Audirac, and Francesca Dominici. E(n) Equivariant Topological Neural Networks. In International Conference on Learning Repre- sentations (ICLR), 2025. URL https: http:////pppeeeeeeeeeeoee net. www.ooonunuunuulleoeo.e page 9.)
(Cited on
Maya Bechler- Speichr, Ido Amos, Ran -aaad-Baccrrch, and Amir Globerson. Graph Neural Networks Use Graphs When They Shouldn't. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference Machine Learning (ICML), volume 235 of Proceedings of Machine Learning Research, pp.
3284-3304 PMLR, 2024. (Cited on page 1.)
Maya Bechler-Speicher, Ben Finkelshtein, Fabrizio Frasca, Luis Müller, Jan Tönshoff, Antoine Siraudin, al. Position: Graph learning will lose relevance due to poor benchmarks, 2025. URL https: // //arxiv.. org/ abs httr/bbs/220000 2502 14546. (Cited on page 1.)
Austin R. Benson, Rediet Abebe, Michael T. Schaub, Ali Jadbabaie, and Jon Kleinberg. Simplicial Closure and Higher-Order Link Prediction. Proceedings of the National Academy of Sciences, 11544) 2018. doi: 10. 10.3/a///p waw.aas..8808838 1800683115. (Cited page 2.)
Guillermo Bernárdez, Lev Telyatnikov, Marco Montagna, Federica Baccini, Mathilde Papillon, Miquel Ferriol-Galmés, et al. ICML Topological Deep Learning Challenge 2024: Beyond the Graph Domain, 2024. URL https: / http//axvvvco/// wwt. wtwp//////////eaee///e////eaaaee abs /2409, 05211. (Cited on page 2.)
Cristian Bodnar, Cätälina Cangea, and Pietro Liò. Deep Graph Mapper: Seeing Graphs Through the Neural Lens. Frontiers in Big Data, 4, 2021a. doi: 10.3389/fdataa.2211..8 6880535. (Cited on page 9.)

---

Published as a conference paper ICLR 2025
Cristian Bodnar, Fabrizio Frasca, Nina Otter, Yuguang Wang, Pictro Liò, Guido Montufar, and Michael Bronstein. Weisfeiler and Lehman Go Cellular: CW Networks. In M. Ranzato, A. Beygelzimer, Y. Dauphin, S. Liang, and Wortman Vaughan (eds.), Advances in Neural Information Processing Systems (NeurlPS), volume 34, pp. 2625-2640. Curran Associates, Inc., 10. (Cited on pages 6 and 9.)
Cristian Bodnar, Fabrizio Frasca, Yuguang Wang, Nina Otter, Guido Montufar, Pictro Liò, and Michael Bronstein. Weisfeiler and Lehman Go Topological: Message Passing Simplicial Networks. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning (ICML), volume 139 of Proceedings of Machine Learning Research, pp. 1026-1037. PMLR, 2021c. (Cited on page 2.)
Boris Bonev, Thorsten Kurth, Christian Hundt, Jaideep Pathak, Maximilian Baust, Karthik Kashinath, and Anima Anandkumar. Spherical Fourier Neural Operators: Learning Stable Dynamics on the Sphere. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning (ICML), volume 202 of Proceedings of Machine Learning Research, pp. 2806-2823, PMLR, 2023. (Cited on page 3.)
Andrew P. Bradley. The Use of the Area Under the ROC Curve in the Evaluation of Machine Learning Algorithms. Puttern Recognition,300 30(7):1145-1159) 1997. doi: 10. 100000/00001333030000000000 on page 6.)
(Cited
Corinna Coupette, Jeremy Wayland, Emily Simons, and Bastian Rieck. No Metric to Rule Them All: Toward Principled Evaluations of Graph-Learning Datasets, 2025. URL http:::s//axxvv.orrge/e 2502.. 02379. (Cited page 1.)
Keenan Crane and Max Wardetzky. Glimpse into Discrete Differential Geometry. Notices of the AMS, 2017. doi: 10. 1..10090nnt15788 (Cited on page 3.)
Jian Du, Shanghang Zhang, Guanhang Wu, José M. F. Moura, and Soummya Kar. Topology Adaptive Graph Convolutional Networks, 2017. URL https: / /wrxiv..vvvvva///// wrggabr/////ooooooo.oo.../...ese /1700.11.... 10370 (Cited on page 6.)
Yam Eitan, Yoav Gelberg, Guy Bar-Shalom, Fabrizio Frasca, Michael Bronstein, and Haggai Maron. Topological Blind Spots: Understanding and Extending Topological Deep Learning Through the Lens of Expressivity. In International Conference on Learning Representations (ICLR), 2025. URL (Cited on pages 2 and 8.)
https: opppreveeneeeeeeeeeeeeeeeee .net/
Federico Errica, Marco Podda, Davide Bacciu, and Alessio Micheli. A Fair Comparison of Graph Neural Networks for Graph Classification. In International Conference on Learning Representations (ICLR), 2020.
URL https: / opwnreveewreeeeeeeeeeeeeee net/ forum? didaumnddddgggeee 6NFPB. (Cited on page 1.)
William Falcon and The PyTorch Lightniig team. PyTorch Lightning, March 2019. URL https:// github. wwt.com///aooo wttp:/ccmm/iiiiiie Liwm//ighhttinggggg AI lightningttnnnnnnn (Cited on page 26.)
Charles Fefferman, Sanjoy Mitter, and Hariharan Narayanan. Testing the Manifold Hypothesis. Journal of the American Mathematical Society, 29(4)9983-1049 2016. (Cited on page 3.)
Matthias Fey and Jan F. Lenssen. Fast Graph Representation Learning with PyTorh Geometric. In ICLR Workshop Representation Learning on Graphs and Manifolds, 2019. (Cited pages 4 and 5.)
Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural Message Passing for Quantum Chemistry. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning (ICML), volume 70 of Proceedings of Machine Learning Research, pp. 1263-1272. PMLR, 2017. (Cited on page 2.)
12

---

Published as a conference paper ICLR 2025
Chad Giusti, Robert Ghrist, and Danielle S. Bassett. Two's Company, Three (Or More) is a Simplex. Journal of Computational Neuroscience, 41(1):1-14, 2016. doi: 10.1007/110827-016-00000 0-000606-6666.... (Cited on page 2.)
L. Giusti, C. Battiloro, P. Di Lorenzo, S. Sardellitti, and S. Barbarossa. Simplicial Attention Neural Networks, 2022. URL
07485.. (Cited on pages 6 and 23.)
Mustafa Hajij. Ghada Zamzmi, Theodore Papamarkou, Nina Miolane, Aldo Guzmán-Sáenn Karthikeyan Nate- san Ramamurthy, et al. Topological Deep Learning: Going Beyond Graph Data, 2023. URL https: http:////axvvvvoooo abs// 2206. 00606. (Cited on pages 6 and 9.)
Mustafa Hajij, Mathilde Papillon, Florian Frantzen, Jens Agerberg, Ibrahem AlJabea, Rubén Ballester, et al. TopoX: A Suite of Python Packages for Machine Learning on Topological Domains. Journal of Machine Learning Research, 25(374): 2024. (Cited on page 5.)
Allen Hatcher. Algebraic Topology. Cambridge University Press, Cambridge, UK, 2002. (Cited on page 17.)
Max Horn, Edward De Brouwer, Michael Moor, Yves Moreau, Bastian Rieck, and Karsten Borgwardt. Topological Graph Neural Networks. International Conference on Learning Representations (ICLR), 2022. URL https: wwwpereeeewweeeeeoorr....... wttprr/mnxidxxxxxeeeeeeeee
(Cited on pages 7 and 24.)
Johanna Immonen, Amauri Souza, and Vikas Garg. Going Beyond Persistent Homology Using Persistent Homology. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems (NeurIPS), volume 36, pp. 63150-63173. Curran Associates, Inc., 2023. (Cited on page 9.)
Noémie Jaquier, Viacheslav Borovitskiy, Andrei Smolensky, Alexander Terenin, Tamim Asfour, and Leonel Rozo. Geometry-awan Bayesian Optimization Roboties using Riemannian Matérn Kernels. In Aleksandra Faust, David Hsu, and Gerhard Neumann (eds.), Proceedings of the 5th Conference Robot Learning, volume 164 of Proceedings of Machine Learning Research, pp. 794 805. PMLR, 2022. (Cited on page 3.)
Jakob Jonsson. Simplicial Complexes of Graphs. Springer, Heidelberg, Germany, 2007. (Cited on page 2.)
Thomas N. Kipf and Max Welling, Sume-SSperyvised Classification with Graph Convolutional Networks. http::/wwweeeeeeeeeeeeeeeeeeeeee http://ww..eeooooommddd//ss////. (Cited pages 6 and 23.)
2017. URL https:/
Serge Lawrencenko and Seiya Negami. Constructing the Graphs That Triangulate Both the Torus and the Klein Bottle. Journal of Combinatorial Theory, Series B, 17:211-218 1999. (Cited on pages 3 and 17.)
Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence and 30th Innovative Applications of Artificial Intelligence Conference and 8th AAAI Symposium on Educational Advances in Artificial Intelligence, 2018. (Cited on page 8.)
Renming Liu, Semih Cantürk, Frederik Wenkel, Dylan Sandfelder, Devin Kreuzer, Anna Little, al. Taxonomy of benchmarks in graph representation learning. Bastian Rieck and Razvan Pascanu (eds.), Proceedings the First Learning Graphs Conference (LoG), number 198 in Proceedings of Machine Learning Research, pp. 6:25, PMLR, 2022. (Cited on page 1.)
Frank H. Lutz. Enumeration and Random Realization of Triangulated Surfaces, pp. 235-253. Birkhäuser, Basel, Switzerland, 2008, doi: 10. (0.1007/978-3. (1643-8621-4-- (Cited on page 22.)
Frank H. Lutz. The Manifold Page, 2017. URL https: : /www3.ww....m//..oo/..... . ma ththhh uu con-berling de/ I EM/ Nachrufe/Pe httpa/////eaanntttttttteeelaaeae Accessed: September 19, 2024. (Cited pages 3, 4, and 20.)
13

---

Published as a conference paper ICLR 2025
Kelly Maggs, Celia Hacker, and Bastian Rieck. Simplicial Representation Learning with Neural k-forms. In International Conference on Learning Representations (ICLR), 2024. URL https: /ew.r//eeeee...ee..e..mem.. net/ Forum?id=D) forun (Cited on page 2.)
Edwin E. Moise. Affine Structures in 3-Manifolds: V. The Triangulation Theorem and Hauptvermutung. Annals of Mathematics, 56(1):96-114, 1952. (Cited on page 17.)
Marco Montagna, Simone Scardapane, and Lev Telyatnikov. Topological Deep Learning with State-Space Models: A Mamba Approach for Simplicial Complexes, 2024. URL https: http/aiiiivv..oo..//..a... org/ /bs/ 2409. 12033. (Cited on pages and 24.)
John W. Morgan and Gang Tian. Ricci Flow and the Poincaré Conjecture, volume 3 of Clay Mathematics Monographs. American Mathematical Society and Clay Mathematics Institute, 2007. (Cited on page 17.)
Luis Müller, Mikhail Galkin, Christopher Morris, and Ladislav Rampäšek. Attending Graph Transformers. Transactions on Machine Learning Research, 2024. URL HhbeHBBrf2. (Cited on page 24.)
net wwt./nnen////iooooooooooooooo Forum?id-
Vidit Nanda. Computational Algebraic Topology Lecture Notes, 2022. URL https:///ppol math.. pd.. (Cited on page 16.)
ox.ac. /
Theodore Papamarkou, Tolga Birdal, Michael Bronstein, Gunnar Carlsson, Justin Curry, Yue Gao, Mustafa Hajij, Roland Kwitt, Pietro Liò, Paolo Di Lorenzo, Vasileios Maroulas, Nina Miolane, Farzana Nas- rin, Karthikeyan Natesan Ramamurthy, Bastian Rieck, Simone Scardapane, Michael T. Schaub, Petar Veličković, Bei Wang, Yusu Wang, Guo-Wei Wei, and Ghada Zamzmi. Position: Topological Deep Learning is the New Frontier for Relational Learning. Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learming (ICML), volume 235 of Proceedings of Machine Learning Research, pp. 39529-39555. PMLR, 2024. (Cited on pages 2, 7, and 9.)
Mathilde Papillon, Sophia Sanborn, Mustafa Hajij, and Nina Miolane. Architectures of Topological Deep Learning: A Survey of Message-Passingg Topological Neural Networks, 2024. URL https: https://aaxiv.. org/ abs 2304 10031. (Cited on pages 6 and 23.)
Rahul Paul and Stephan Chalup. Estimating Betti Numbers Using Deep Learning. In International Joint Conference Neural Networks (IJCNN), pp. 1-7, 2019. doi: 10.1109//CCNN.2211 (Cited on page 2.)
Tom Preston- Werner. Semantic Versioning 2.0.0. http::/ org. Accessed: September 21, 2024. (Cited on page 4.)
Tibor Radó. Über den Begriff der Riemannschen Fläche. Acta Litt. Sei. Szeged, 2:101-121, 1925. (Cited on page 17.)
Karthikeyan
Natesan Ramamurthy, Aldo Guzmán-Säee -áenz, and Mustafa Hajij. TOPO-MLP: A Simplicial Network without Message Passing. In IEEE International Conference on Acoustics, Speech and Signal (Cited on page 2.)
Processing (ICASSP), pp. 1-5, 2023. doi: 10.1 10.1109//CASSP99333....
Bastian Rieck, Christian Bock, and Karsten Borgwardt. Persistent Weisfeiler-Lchmar Procedure for Graph Classification. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning (ICML), volume 97 of Proceedings of Machine Learning Research, pp. 5448 5458. PMLR, 2019. (Cited on page 9.)
14

---

Published as a conference paper ICLR 2025
Ernst Röell and Bastian Rieck. Differentiable Euler Characteristic Transforms for Shape Classification. In International Conference on Learning Representations (ICLR), 2024. URL https: opw..eve.........eeeee net/ Eorum? do
(Cited on pages and 6.)
Yunsheng Shi, Zhengjie Huang, Shikun Feng, Hui Zhong, Wenjin Wang, and Yu Sun. Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification. In Zhi-Hua Zhou (ed.), Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI), pp. 1548- 1554, 2021. doi: 10.24963/ijcai.22222244 (Cited on pages 6 and 23.)
Gurjeet Singh, Facundo Mémoli, and Gunnar Carlsson. Topological Methods for the Analysis of High Dimensional Data Sets and 3D Object Recognition. In M. Botsch, R. Pajarola, B. Chen, and M. Zwicker (eds.), Eurographics Symposium on Ppint-Based Graphics, 2007. doi: 10.2112SPRG//SKK6770)9--000 (Cited on page 9.)
Bosiljka Tadic, Miroslav Andjelković, and Roderick Melnik. Functional Geometry of Human Connectomes. Scientific Reports, 9(1):12060, Aug 2019. doi: 10. 10388//41998001944555--88 (Cited on page 2.)
Lev Telyatnikov, Guillermo Bernardez, Marco Montagna, Pavlo Vasylenko, Ghada Zamzmi, Mustafa Hajij, Michael T. Schaub, Nina Miolane, Simone Scardapane, and Theodore Papamarkou. TopoBenchmarkX: A orrggioorrrr..oo org /bs/
Framework for Benchmarking Topological Deep Learning, 2024. URL
2406. 06642. (Cited on pages 2 and 6.)
Jan Tönshoff, Martin Ritzert, Eran Rosenbluth, and Martin Grohe Where Did the Gap Go? Reassessing the Long-Range Graph Benchmark, 2023. URL www.hhttt////tttte oow.iiii.o.ooooooooooooooo abs/ 2309. 00367. (Cited on page 1.)
Jake Topping, Francesco Di Giovanni, Benjamin Paul Chamberlain, Xiaowen Dong, and Michael M. Bron- stein. Understanding Over-Squashing and Bottlenecks on Graphs via Curvature. In International Con- ference on Learning Representations (ICLR), 2022. URL https: http:s/////peeeeeew...eeeo htwp:r/wwww...et/oou/iiii...o 7UmjRGZp-B A. (Cited on page 8.)
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is AI You Need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems (NeurIPS), volume 30. Curran Associates, Inc., 2017. (Cited on page 24.)
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pictro Liò, and Yoshua Bengio. Graph Attention Networks. In International Conference on Learning Representations, 2018. URL https: http::///peeeeeeieeeeemee net/ / forum? didd (Cited on pages and 23.)
Hanrui Wu, Andy Yip, Jinyi Long, Jia Zhang, and Michael K. Ng. Simplicial Complex Neural Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(1):561-575, 2024. doi: 10.1109/ TPAMM.2003.33226224 (Cited on pages 6 and 23.)
Maosheng Yang and Elvin Isufi.
Convolutional Learning on Simplicial Complexes, 2023. URL https: (Cited on pages 6 and 23.)
//arxiv. org/ /ass///////////
Maosheng Yang, Viacheslav Borovitskiy, and Elvin Isufi. lolgge-Comoosttoonall Edge Gaussian Processes. In International Conference on Artificial Intelligence and Statistics (AISTATS), 2024. (Cited on page 2.)
Ruochen Yang. Frederic Sala, and Paul Bogdan. Efficient Representation Learning for Higher-Order Data With Simplicial Complexes. In Bastian Rieck and Razvan Pascanu (cds.), Proceedings of the First Learning on Graphs Conference, volume 198 of Proceedings of Machine Learning Research, pp. 13:1-13:21. PMLR, 09-12 Dec 2022. URL https: httr///proocc ppwpppeooddeeeee perroceeiing..... ngs. mlr. press / v1 98/ 20//ang22a html. (Cited on pages 6 and 23.)
15

---

Published as a conference paper ICLR 2025
A MATHEMATICAL BACKGROUND
A.1 SIMPLICIAL COMPLEXES
A simplicial complex K is a family of non-empty finite sets such that, if a E K and 7 C a, then T E K. Each a E K is called simplex of K, and is called d-dimensional face or d-face of K if its cardinality is d + 1. The O-faces of K are called vertices and the 1-faces are called edges. We denote by Kd the set of d-faces of K, and define the dimension of K as the largest d for which Kd is non-empty. A simplicial complex of dimension is called a graph.
A geometric realization of simplicial complex K is the union of a collection of affine simplices A, in Euclidean space I" for some n 2 1, one for each simplex a E K, where a is mapped bijectively to the vertices of A and two affine simplices A, and A, share a face corresponding to n whenever this intersection non-empty. Any two geometric realizations of a simplicial complex K are homeomorphic through face-present fuce-preserving map.
The barycentric subdivision of simplicial complex K is the simplicial complex Sd( obtained by setting its d-dimensional faces be sequences of strict inclusions dg ai C... aa of simplices of K. It then follows that and Sd(K) have homeomorphic geometric realizations (Nanda, 2022, Proposition 1.13).
A.2 SIMPLICIAL HOMOLOGY AND BETTI NUMBERS
Simplicial homology of a simplicial complex K equipped with an order on its set of vertices is defined as follows (Mun, 1984, $ 34). Let R be any commutative ring with unit (including the ring of integers Z or any field). The chain complex of K with coefficients in R is sequence of R-modules (Cn (K))ez whose elements are formal sums of n-simplices of K with coefficients in R, i.e.,
Cn {EOEKn aod a. E R).
linked by boundary homomorphismss on: (K) (n) for all Z. given by
On (EOER- 0, = EOEK On))
= E.
(wr)).
if Uo...., Un are the ordered vertices of a. The main property of the boundary homonorphisms that On 0x+1 = 0 for all n. implying that 1m(0.+1) C Ker(a,) for all n. This yields homology R-modules, defined as quotients H (K) = (er(0,) /mmm)mmmmm))))))))m))))))))m)))) for all n.
If is finite simplicial complex and R 2, then (K) is finitely generated abelian group and therefore it decomposes as a direct sum
H K TM + Zy, + Zg.
where 3n is the n-th Betti number of K, while 91..... 9t are prime powers. The sum Zg, O ...... + Zgg is the torsion subgroup of H, (K). Examples of Betti numbers are provided in Figure 2. The n-th Betti number of simplicial complex counts the number of linearly independent n-dimensional cavities in a geometric realization of K. In low dimensions, fo is equal to the number of connected components, and B1 counts the number of linearly independent loops that are not boundaries of any 2-dimensional region.
A.3 TRIANGULATED MANIFOLDS
An n- u--imensional manifold is a second-countable Hausdorff topological space M such that every point of M is contained in some open set, called chart, equipped with homeomorphism into open subset of Euclidean space IR" (Mun, 1984, $ 36). This definition does not include manifolds with boundary, which are not considered in this article. A manifold is called closed if its underlying topological space is compact.
16

---

Published as a conference paper ICLR 2025
% 81 82
% 8
8g
31
31
8
32
32
Figure 2: From left to right, four simplicial complexes K,, K2, K3. and K4 with their respective Betti numbers 3o. 31, and 3g. The n-th Betti number indicates the number of r-ddimensional holes in geometric realization of simplicial complex. Here K₁ is a solid tetrahedron with S = 1, Br = 0, and 33 = O. since K₁ has only one connected component, no unfilled cycles, and no empty cavity enclosed by 2-faces; Kg is hollow tetrahedron with 8o 1, 81 O, and 82 = 1 (the difference with K1 is that the triangles of K₂ enclose a cavity); K3 the underlying graph, with By = 1, 81 = 3, and 82 = 0, since there is no cavity and there are three linearly independent cycles; K4 consists of four vertices and has 3o 4, 31 0. and 32 0, since there are four connected components and no cycles nor cavities.
A collection of charts covering manifold M is an atlas of M. A manifold M is called orientable if it admits atlas with compatible orientations in its charts. For closed n-dimensional manifold M, orientability is determined by its n-th Betti number Sn. which is nonzero if and only if M is orientable.
A triangulation of a manifold M is simplicial complex whose geometric realization is homcomorphic to M. Radó (1925) proved that every surface admits a triangulation (which can be chosen to be finite if the surface is compact), and that any two such triangulations admit common refinement. Moise (1952) proved that the same facts are true for 3-dimensional manifolds. For dimensions greater than 3, however, there are examples of manifolds that cannot be triangulated.
A.4 CLASSIFICATION
Closed connected surfaces can be classified, up homeomorphism, as given by the following list: (i) the two-dimensional sphere S²; (ii) connected sum of tori T2; (iii) connected sum of projective planes IRP2.. The genus of surface M is defined zero if M = S2 and equal to g if M is connected sum of g tori or g projective planes. Thus the homeomorphism type of M is determined by its orientability and genus.
The Euler characteristic of a finite triangulation of a manifold M is the alternating sum of the numbers of simplices of each dimension. It does not depend on the choice of triangulation, and it is equal to the alternating sum of the Betti numbers of M (Hatcher, 2002). The Euler characteristic of a closed connected surface M of genus g is equal to 2 2g if M is orientable and 2 g if M is not orientable.
The underlying graph of finite triangulation of closed surface M determines the Euler characteristic V - e - t. This is due to the fact that, in any triangulation of M. cach edge bounds precisely two triangles, so 3t = 2e. Therefore, the underlying graph of a triangulation of closed surface M determines the homeomorphism type of M up orientability. As shown in Lawrencenko & Negami (1999), the torus and the Klein bottle admit triangulations with the same underlying graph.
For manifolds of dimension greater than 2, classification up to homeomorphism is so far unfeasible. In dimension 3, the geometrization theorem (Morgan & Tian, 2007) describes all possible geometries of prime components of closed 3-manifolds. The Euler characteristic does not carry any information about the homeomorphism type in dimension 3, since if M is any ald-dimenssioal closed manifold then X(M) = 0 by Poincaré duality (Hatcher, 2002, 3.37). However, the underlying graph of finite triangulation of closed 3-manifold determines the number t of triangles and the number f of 3-faces, since 4f = 2t and + f 0.
17

---

Published as conference paper ICLR 2025
A.5 DISTRIBUTION LABELS
Tables 3, 4, 5, and 6 contain statistical information about the distribution of labels in the dataset.
Table 3: Distribution of Betti numbers 3; for triangulations of manifolds. Percentages are rounded to the nearest integer and are computed for each pair of manifold dimension or 3) and Betti number. Columns represent values of Betti numbers and contain the number of manifolds with each value.
M
2
3
2-M
43,138
90
(100%)
3-M
250,359
(100%)
2-M
1,670
4,655
14,146
13,694
7,917
1,022
34
B1
(4%)
(11%)
(33%)
(32%)
(18%)
(2%)
(0%)
3-M
249,225
1,134
0
0
0
(100%)
(0%)
(0%)
(0%)
(0%)
(0%)
(0%)
2- MM
39,718
3,420
82
(92%)
(8%)
3 M
249,841
518
(100%)
(0%)
M-M
33 3-M
616
249,743
(0%) (100%)
Table 4: Distribution of torsion subgroups for triangulations of manifolds. Percentages are rounded to the nearest integer and are computed for each pair of manifold dimension and homological degree.
Ho
H1
H2
H3
M
0
Z2
0
Z,2
0
0
M
43,138
39,718
3,420
43,138
(100%)
(92%)
(8%)
(0%)
(100%)
3-M
250,359
250,359
616
249,743
250,359
(100%)
(0%)
(100%)
(0%)
(100%)
(100%)
Table 5: Distribution of genus for triangulations of surfaces. Percentages are rounded to the nearest integer.
M
5
7
2-M
306
3,593
5,520
11,937
13,694
7,052
1,022
14
(1%)
(8%)
(13%)
(28%)
(32%)
(16%)
(2%)
(0%)
18

---

Published as conference paper ICLR 2025
Table 6: Distribution of homeomorphism types for triangulations of manifolds. Percentages are rounded to the nearest integer and are computed for each manifold dimension. Surfaces classified as "Other" do not have any explicit homeomorphism type assigned.
M
S2
R.22
T
S3
S2 S
S2XS
Other
M
306
1,364
2,229
4,655
34,584
(1%)
(3%)
(5%)
(11%)
(80%)
3-M
249,225 (100%)
518
616
0
(0%)
(0%)
(0%)
19

---

Published as conference paper ICLR 2025
B DATASET DETAILS
This section provides additional details about the dataset and the design choices involved in its creation.
B.1 DATA FORMAT
This section is mostly information-orieente and provides brief overview of the data format, followed by a short example.
As complement to Section 2 in the main text, we provide extended description of dataset attributes. Each dataset consists list of triangulations, with each triangulation having the following attributes:
id (required, str): This attribute refers the original ID of the triangulation following Lutz (2017).
This facilitates comparisons to the original dataset if necessary and simplifies future contributions by other authors.
triangulat ion (required, 1ist of 1ist of int): A doubly-nested list of the top-level sim- plices of the triangulation.
n_vert ices (required, int): The number of vertices in the triangulation. This is not the number of simplices.
name (required, str): canonical name of the triangulation, such for the two-dimensional sphere. If no canonical name exists, we store an empty string.
(required, 1ist of int): A list of the Betti numbers of the triangulation, computed using Z coefficients. This implies that torsion coefficients stored in another attribute.
corsiion_ccefficcienn (required, list of str): list of the torsion coefficients of the triangulation. An empty string indicates that no torsion coefficients are available in that dimension. Otherwise, the original spelling of torsion coefficients retained, so valid entry might be 22_2".
genus (optional, int): For 2-manifolds, contains the genus of the triangulation.
orientable (optional, bool): Specifies whether the triangulation orientable not.
We picked JSON underlying data format, since facilitates exchanging information, extending the dataset, and can be easily processed in all major programming languages. By only ever storing the top-level be easily compressed. Moreover, the textual format
simplices of simplicial complex, the dataset facilitates tracking changes over different versions of the dataset. The subsequent listing depicts a simple example of the data format; below for additional design choices.
20

---

Published as conference paper ICLR 2025
"id": mma_i_old_2___11 "triangulation": [1,2,3],
[1,
[1,3,4], [2,3,4]
1,
10
"dimensien"; 2, "n_vertices"; 4,
11
12
"betti_numbers
13
1,
15
1
16
1,
17
"torsioo__cefiiciennst''
19
""
20
."
21
},
22
"name": "S"2", "genus": 0,
24
"orientable": true
25
26
27
"id": fold__old22_5_0 *triangulation": [1,2,31, [1,2,4],
29
30
[1,
32
[1,4,5),
3,,3,41,
34
[3,4,5]
35
},
36
"dimension": 2, "n_vertices"
"betti_numbers 1,
39
40
0,
1
},
43
"torsiioncceeficcienss
44
45
."
1,
48
"name": "g^2", "genus"; 0,
49
"orientable"; true
52
21

---

Published as conference paper ICLR 2025
B.2 DESIGN CHOICES
This section is understanding oriented oorientt and provides additional justifications for our data format.
The datasets converted from their original (mixed) lexicographical format (Lutz, 2008). A triangulation in lexicographical format could look like this:
manifold_lon liftood_llx____e___ 1_______1_[____ 2, 31, [1, 2, 41, [1, 3, 4], 13,5, 61, 54 611
[2, 4, 51, [3, 4, 61,
triangulation mixed lexicographical format could look like this:
manifolde 2_ 2________11__11______11_11221121 (1, 31, 2, 41, (1, [1, 6), 5, 61, [2, 3, 41, [3, 5], [4, 5, 611
This format is hard to parse and prror-prone. Moreover, any additional information about the triangulations, including information about homology groups or orientability, for instance, requires additional files. We thus decided to format that permits to keep everything in place. including any additional attributes for a specific triangulation. desirable data format needs to satisfy the following properties:
1. It should be easy parse and modify, ideally in number of programming languages.
2. It should be human cumaa--readable and ff in order to permit simplified comparisons.
3. It should scale reasonably well to larger triangulations.
After some considerations, decided to opt for qz c-moppessee JSON files. JSON is well-specified and supported in virtually all major programming languages out of the box. While the compressed file is not numan-readabll
its own, the uncompressed version can easily be used for additional data analysis tasks. This also greatly simplifies maintenance operations on the dataset. While it can be argued that there are formats that scale even better, they are not well-applicable our use case since each triangulation typically consists of different numbers of top-level simplices. This rules out column-based formats like Parquet.
We are open to revisiting this decision in the future. Our current API can be adjusted to accommodate other data formats. End users are not interacting with the raw data.
As for the storage the data as such, we decided to keep only the top-level simplices (as is done in the original format) since this substantially saves disk space. The drawback is that the client has to supply the remainder of the triangulation. Given that the triangulations in our dataset are not too large, we deem this be an acceptable compromise. Moreover, data structures such as simplex trees can be used further improve scalability if necessary. Finally, our data format includes, whenever possible and available, additional information about triangulation, including the Betti numbers and name, i.e., canonical description, of the topological space described by the triangulation. We opted to minimize any inconvenience that would arise from having to perform additional parsing operations.
Overall, this data format remains extensible- permitting additional information about triangulation attributes like coordinates-while still benefiting from easy accessibility. We make our code and data publicly available and use Zenodo for long-term archival with DOIs. The most recent version of our dataset is accessible via:
https: https://doi.org//// wttp:///o2222/eeooooooooo http://222oooo..113155.515 14103582
Old versions are archived and can be accessed using our data loader. We hope that this system, while not perfect, may serve as suitable starting point for other benchmark datasets.
22

---

Published as a conference paper ICLR 2025
C
MODEL DETAILS
We provide brief description of the models used in the experiments.
Message passing neural networks. Most of the models used in the literature for graphs and higher-order structures such as simplicial or cell complexes are based on the peessage-passing paradigm. For graph and simplicial complexes, these models pass messages between neighboring nodes or simplices in the graph or complex, updating their features based on the features of their neighbors. Let K be a simplicial complex or a graph seen as a simplicial complex with simplicial features given by a family maps {F, [F))== where F;: K, R A message- paseeegasnnggggg layer updates the features of a simplex o using the following steps (Papillon et al., 2024):
1. Selection of neighborhoods: Given a simplex we first start by defining sets of neighboring simplices (N)( where the neighborhoods are defined depending on the context. For example, adjacent or incident simplices are two types of neighborhoods that can be defined in an arbitrary simplicial complex. Usually, neighborhoods are defined in the same way for the same dimension of simplices, and each set of neighboring simplices contain simplices of the same dimension.
2. Message computation: For each set of neighboring simplices (xx); we compute messages {my-soi be from the features of the simplices in (z( and the features of a, this is
- My(()) (Faim - (r), Faim (o),
where O are the learnable parameters of the layer.
3. Iutra-aggregation: The messages are aggregated to obtain a single message for each neighborhood (x),,, this is
(x)
(1m-
where (z) is an permutation invariant aggregation function, for example, sum, mean, or any other function that aggregates the messages.
4. Inter-aggregattin: The aggregated messages for the neighborhoods are then aggregated together to obtain a single message for the simplex o, this is
Agg (fm) h:),
where Agg, is permutation invariant aggregation function again.
5. Update: The message my used update the features of the simplex o, this is
F Faim L (a) = Update(Faim.<())) m.; O).
For graphs, GCN (Kipf & Welling, 2017), GAT (Veličković et al., 2018), and UniMP (Shi al., 2021) are examples of message- passing networks. In the case of GCN and GAT, adjacency with self-loops is used as neighborhood sets for nodes, whereas UniMP uses concatenated adjacencies up to order k. meaning that we consider as neighbors of a vertex all the other vertices of the graph at a distance of at most & from the vertex. In the case GAT, the fundamental difference lie in the message computation, where the message from a concept of attention, which is computed using the features of and
simplex 7 to simplex a depends and a learnable parameter +
In the of simplicial complexes, SAN (Giusti al., 2022) and SCN (Wu al., 2024) (upper and lower) higher-order Laplacians to define neighborhoods, SCCN (Yang et al., 2022) uses (co)adjacency and incidence structures, and SCCNN (Yang & Isufi, 2023) uses all together.
23

---

Published as a conference paper ICLR 2025
Non-message passing neural networks Although the pesssae-passinng paradigm is predominant in the lit- erature, there are other state-of- of-the-art models that do not follow this paradigm, such as transformers (Ballester al., 2024), space topological models (Montagna et al., 2024), or TDA-based networks (Horn et al., 2022). In our case, we only select graph and cellular transformers and multi-layer perceptrons (MLP) for comparison. Graph and cellular transformers are based on the original transformer's decoder architec- ture (Vaswani et al., 2017). Original transformer architectures are permutation-invariaat networks that use positional encoding to break the symmetry the input data by means of localizing the position of each element in the input sequence. In the case of graph and cellular transformers, which do not always have linear structure as text, positional encodings encode the position of the different simplices in the simplicial complex using the combinatorial structure of the complex. Famous positional encodings for graphs are built using eigenvectors of the graph Laplacian and random walks (Müller et al., 2024). For simplicial transformers, preliminary positional encodings are based also eigenvectors of combinatorial Laplacians, random walks, and graph positional encodings for barycentric subdivisions of the simplicial complexes.
24

---

Published as conference paper ICLR 2025
D HYPERPARAMETER DETAILS
More information on the meaning of specific hyperparameters can be found in the PyTorch geometric, TopoModelX, CelIMP, CT and DECT original implementations.
GRAPH MODELS DECT
TOPOLOGICAL MODELS
GAT
SAN
Hidden neurons
64
Hidden channels
64
Hidden layers
4
Hidden layers
I
Readout
Mean
n-filters
2
Dropout last linear layer
0.5
Order harmonic
Activation last layer
Identity
Epsilon harmonic
le-1
Readout
Sum of sums per dimension
GCN
SCCN
Hidden neurons
64
Hidden channels
64
Hidden layers
Hidden layers
2
Readout
Mean
Maximum rank
Dropout last linear layer
0.5
Aggregation activation function
Sigmoid Sum of sums per dimension
Activation layer
Identity
Readout
MLP
SCCNN
Hidden neurons
64
Hidden channels
64
Hidden layers
4
Hidden layers
2
Readout
Mean
Order convolutions
Dropout last linear layer
0.0
Order simplicial complexes
Activation layer
Identity
Readout
Sum of sums per dimension
TAG
SCN
Hidden channels
64
Hidden channels per dimension
Same as input
Hidden layers
Hidden layers
2
Readout
Mean
Readout
Sum sums per dimension
Dropout last linear layer
0.5
Activation layer
Identity
UNIMP
CELLMP
Hidden channels
64
Hidden channels
64
Hidden layers
4
Hidden layers
10
Readout
Mean
Dropout
0.5
Dropout last linear layer
0.5
Hidden dimension mul-
Activation layer
Identity
tiplier for final lincar layer
2
Readout
Sum of sums per dimension
DECT
CT
Hidden channels
64
Hidden channels
64
Hidden layers
Positional encoding type
Hodge Laplacian Eigenvectors
Number of 0
32
Positional encoding lengths
Bump steps
32
Hidden layers
2
1.1
Number of heads
8
Normalized
True
Dropout
0
Hidden layers final MLP
2
Attention tensor diagram
Adjacent dimensions
Mask type
Sum
Readout
Average of dimension zero features
25

---

Published as conference paper ICLR 2025
E ADDITIONAL EXPERIMENTAL DETAILS
Table 7 reports the mean and standard deviation of training iterations processed per second for each model and dataset. Sections E.1 E.3 report the full set of experimental results.
Feature vector initialization analysis We observe different behaviours for the two families of models, graph-based and simplicial complex-based. For the graph models, random initialization works slightly better or equal than the degree features. On the other hand, for the simplicial complex models, upper- and lower-connectivity index initializations consistently outperform their random counterparts on average. Degrees and upper-connectivity indices for vertices coincide for both families of models. suggesting that higher-order connectivity indices contain more useful information than their dimension zero counterpart predict topological properties, supporting the need for models that leverage higher-order information of the input. Having signal contained in features can make sense if the task in question requires additional information. For example, molecules are more than just combinatorial or topological objects: the types of atoms and the nature of bonds are important for predicting their properties. However, in purely topological tasks, such as predicting topological invariants, the need to enforce topological information into features raises the question: do MP-based models correctly capture topological properties in the first place? Still, standard deviations the aggregated data for simplicial complex- based models large, and better ablation needed to fully understand the differences in initializations and the expressivity of higher-order indices in the context of topological prediction tasks.
Table 7: Mean and standard deviation of training iterations processed per second (t), as measured by PyTorch Lightning (Falcon & The PyTorch Lightning team, 2019), across all experiments for each model and dataset. The measurements are subject to variations caused by external server usage fluctuations.
MODEL (ClASS)
2-M"
2-
3-M²
MLP (g)
9.72 + 4.54
12.39
13.22
13.29 8.36
GAT (9)
9.41 + 4.02
11.33 10.60
11.01 + 6.52
UniMP (9)
9.31 + 3.79
11.71
12.42 6.80
TAG (g)
9.41 3.53
11.46 10.66
9.76 6.55
GCN (G)
9.45 3.79
12.10
12.72 7.59
SAN (7)
0.65 0.97
1.21 2.93
0.53 0.31
SCN (T)
0,83 2.63
1.72 6.99
0,83 0.64
SCCN (7)
0.85 3.06
1.89
7.90
0.80 0.53
SCCNN (T)
0.73 1.93
1.67
5.79
0.79 0.53
CelIMP (7)
2.31 2.38
2.32
2.43
0.25 0.19
CT (T)
1.06 2.19
1.16
2.77
0.59 0.28
DECT (7)
6.59 + 3.63
3.65 6.61
12.78 8.03
26

---

Published as conference paper ICLR 2025
E.1 BETTI NUMBER PREDICTION
Full results for the Betti number prediction task on all datasets with mean and standard deviation reported over 5 runs. In this table, we report AUROC as performance metric. Transforms are abbreviated as DT (Degree Transform), DTO (Degree Transform Onehot) and RNF (Random Node Features).
Table 8:
AUROC
F
f2
Ag
DATASET
(CLASS)
GAT (9)
0.50 0.000
0.50 a 0.00
1
0.00
0.50
0.00
0.00
GCN
0.00
0.50 0.00
0.50 0.00
0.00
0.50 0.00
0.00
0.50
0.00
0.00
TAG (5)
0.00
0.50 0.00
0.30
0.00
0.50
0.00
UniMP (g)
0.00
0.00
0.50 0.00
0.00
0.00
0.00
2- M-M
(
0.07
CT (7)
0.93 0.01
0.66 0.02
0.55
0.53 0.01
DECT
0.50 0.50
0.50
0.50 0.00
SAN (T)
0.55 0.05
0.06
0.22
0.53 0.01
SCCN (T)
0,93 0.04
0.04
0.55 0.00
0.53 0.01
SCONN
0.50+00:01
0.50
0.50
0.52 0.014
SCN (T)
0.56 1: 0.13
0.51 0.03
0,48
GAT (G)
0,23 0.00
0.23 0.00
0.23 0.00
0,12 0.00
0.12 0.00
0.12 0.00
0.04
0.14 0.00
0.14
0.12
ML.(G)
0.23 0.00
0.23 0.00
0.23+
0.00
0.12 0.00
0.12 +.000
0.14
0.14
0.14
TAG (G)
0,23 0.00
0.23 0.00
0,23 0.00
0.12
0.12 0.00
0.12 0.00
0.14
0.14
0.14
UniMP
3-
CellMP (
0.20
0.23
0.00
0,12
0.14
0.14 0.00
CT
0.23 0.00
0,23++0.000 0.00
0.00
0.12 =1.2000.000
0.14
0.14 0.00
SAN (T)
0.09
0.24
0.05
0.12
0.19 0.04
0.15 0.01
SCON (T)
0.23 0.00
0,23++0.00 0.00
0.12+
0.00
0.14 0.00
0,14 0.00
SCONN
SCN (T)
0.20 4 0.04
0.23 0.00
0.15 0.04
0.12 0.00
0.11 2 0.07
0.14
GAT (g)
0.21 1 0.00
0.21 1 0.00
0.20
0,50 0.00
0.50 1 0.00
0,50
(LP(G)
0:21 0.00
0.21 0.00
0.21 0.00
0.50 0.00
0.50 0.00
0.50
TAG (g)
0.21 10,00
0.21 1.11:0000
1.000
0,50
0.50 0.00
0.50
2-M
CellMP (
0.01
0.29
0.52 0.64
0.51 0.02
(T)
0.00
0.21
0.52 0.03
1.50::0:00
DECT
0.25 0.00
SAN (T)
0.22
0.48 0.04
0.50 1.00
(T)
0.29 0.01
0.23 0.01
0.01
0,50
(
0.05
SCN (T)
0,22 0.00
0,21 0.00
0.49 1 0.02
0.50 0.00
GAT (
0.22 0.00
0.21 0.00
0.21 0.00
0.50
0.50
0.50
0.22
0.21
0.50
0.00
0.50
MLP (G)
0.21 1.001
0.21 0.00
0.21
0.50 0.00
0.50 :: 0.00
0.50
TAG (G)
0.000
0.22
0.22
0.00
UnIMP (g)
1.000
0.22 0.00
0.22
0.50 0.00
0.50 0.00
0.50
2.M
CellMP (T)
0.23+
0.26
0.49
0.49 0.01
0.02
0.00
(A)
0.24 0.01
0.22
0.50 0.00
0.19 0.01
SCCN (T)
0.27 0.01 0.01
0.22
0.52
0.51 0.00
SOCNN
0.21
0.22+0002
0.50
0.50
SCN (T)
0.21 1 0.00
0.21 0.00
0.49 0.02
0.51 1 0.01
27

---

Published as a conference paper ICLR 2025
Table 9: Full results for the Betti number prediction task on all datasets with mean and standard deviation reported over 5 runs. In this table, report accuracy as performance metric. Transforms abbreviated DT (Degree Transform), DTO (Degree Transform Onehot) and RNF (Random Node Features).
Accwary
Fs
Fg
Rs
GAT (5)
1,00 00,00
1.00 20.00
1 = 0.60
0.31 0.00
0.31 0.000
=
0.92.100 0..00
0,92 10.00
[192 00,00
(CN
1,00 0:.0
1.00 20.00
1.00 x 8.60
0.31 $..00
0.31 8.000
0.31 20.0
0.92 3 0..00
1.92 10,00
(U92 x 20,00
MLP (()
1 -0.00
1,100 9.60
0.31 11100
0131 4:000
11:1 =0:00
11:0
+1111
+0000
TAG
1.00
0::0
8:60
0.32
000
+.00
(2)
3.MM
C20MP
CT:77)
1,00
0.50
1 0.00
0.93 0..0
0.57 060
0:93 0.00
0,92 00,00
(FCT(()
1,00 0,00
1,00 0.00
1 8.00
0.12 0100
0,52 0.00
0.53 0.00
0.92
(1:00
0.92 A (1:00
(U92 x 0.00
SAN (7)
2005
0.57= 0013
11.10
0.11
0.52 01.14
0173 0006
SOCN
0.00
8.006
8.99 00.00
0.005
0.00
1..04.4444
SOUNN
=001
0.31 0.25
a.055:0000
GAT (9)
1,00
1.09
=
1
1.00 0.90
1.00 = 0.00
1 9:00
1.00 0.09
1,00 x
1
1.00
2.00
0.00
(CN
1,00 0.00
1,000
0.00
1.00 88.0
1.00 1L80
1.00 0,00
1
1,00 x (000
1,00 11.00
1,00 0.000
1.00 EL6
1.80.113 11,00
1.001:0.00
MLP
1.01 0.000
0:00
0.00
10.00
1:00
0..0
10.10
1:000
1.00
8:00
UnMP
1.00
1:00
1.00
1.00
3-A0
CalMP
(T:7)
1.00
1,01
1
0.00
1 1U10
0.00
1.00
1.00 0.00
1.00 (U0)
1.00
1,00
1.00
1.40
2.00
0010
(6CT(7)
1,00
1.00 0.00
1.00 8:000
1
1.00 0.00
1.00
1.00 0000
1,00 0.,00
1:000
1.00
1,00
1.00 20000
SAN (F)
0.00
0-112
0:49-11 -1111
00:11
0.22
00.10
0.01 (1:0
0000
SOON
0.0000000
1.00 8.00
00000
0:00
1.00
1
1.00 0000
(T)
0.06
0.95 0.06
0.19
0.0
0.00
0
GAT (C)
1,01 0,00
1
0.00
1 0.00
0.56 1100
0.54 0
11 0::0
0.70 (1:00
1.70 11:00
a..0
OCN (7)
2.00
1.00
0:00
1.00 0.00
0.54 0.00
0.54 9.00
0
0
0.10
0.00
MLP
0.00
0:00
8:00
00.0
000
0.00
00,00
UuMP
1.00
1.00
1,08 0.00
0.00
0.50
0.00
2-
C2IMP (2)
CT.77
1,0
0.09
1 9:00
0.000
0.234
0
0:50
0.04 0.331
0.64 +
0.99 0.001
x
(7CT
1,00 0.00
3 0.00
0.56 0100
0.54 9.00
U151 0:00
0 010
a 11,10
00,90
(T)
0006
0.26
0.26 0.05
0:05
SCCNN
0.00
0.06
0.02
2
OC
0.07
0.30
0.03
GAT (
0.00
0.64 o 0.50
1.00 8:00
2.20 0.00
0.43
0.16
0.54
8.70 0.00
00,00
0.700
OCN (G)
0:00
0:400
1 9.00
0.30 1100
07
0.11
I154 9:00
0:70 (L0)
11.10
00.00
1.00::::0
8:00
8.000
0.70
0.00
0.00
2..00:0.00
ToMP
0100000 0.00
-0.01
000-00.01
0.30 0.90
0130 0.00
11:0--000 0.01
0.70 (0,0
IL70 (UIB
(L7) 00,00
3.M4
CHIMP (7)
0.00
0.00
0.08 0..5
0.02
0.09 0.21
0.30220.00
(T.7)
1,01 0,00
1 8.000
8.54 (180
RL54 0.00
0,22 100 11
(L70 0.,00
(T)
0000
0.00 =0.01
0000
0.56
+0.31
+1.12
0.10+011
SCCNN
0.12
0000
0.01
0.99
1113
0.28
28

---

Published as conference paper ICLR 2025
E.2 ORIENTABILITY PREDICTION
Table 10: Full results for the orientability prediction task on all datasets with AUROC and accuracy reported. Mean and standard deviation are taken over runs. Transforms are abbreviated as (Degree Transform), DTO (Degree Transform Onehot) and RNF (Random Node Features).
AUROC
Accuracy
DATASET MODEL (CLASS)
DT
DTO
RNF
DT
DTO
RNF
GAT (G)
0.00
0.50 = 0,00
0.00
0.92
0.92 0.00
0.92
GCN (9)
0.00
0.50 0,00
0.50
0.00
0.92 0.00
0.00
MLP (G)
0.00
0.50 0.00
0.50
0.00
0.92 0.00
0.00
TAG
0.00
0.50 0.00
0.50
0.00
0.92 0.00
0.00
UniMP (G)
0.00
0.50 0,00
0.50
0.92
0.92 0.00
0.92
CellMP (7)
0.05
0.00
0.24
0.93 0.00
2- 2-Me
CT (7)
0.00
0.50
0.00
=
DECT
0.00
0.50
0.00
0.00
0.92 0.00
0.90
SAN (T)
0.03
0.01
0.92
0.92
SCCN (T)
0.55 0.01
0.54
0,93 0.00
0,93 0.00
SCCNN (T)
0.55
0.53
0.87 0.11
0.91 1 0.01
SCN (T)
0.53 0.04
0.00
0.91
0.92 0.01
GAT
0.14 0.00
0.14 0.00
0.00
1.00 0.00
1.00 0.00
1.00 0.00
GCN (G)
0.14 0.00
0.14 0.00
0.14
1.00 0.00
1.00 0.00
1.00 0.00
ML. (G)
0.14 + 0.00
0.14 0.00
0.14 0.00
1.00 + 0.00
1.00 + 0.00
1.00 = 0.00
TAG (C)
0.14 0.00
0.14 0.00
0.14 0.00
1.00 = 0.00
1.00 # 0.00
1.00 = 0.00
UniMP (G)
0.14
0.04
1.00
1.00 0.00
CelIMP (T)
0,18 0.04
0.00
1.00 0.00
1.00 0.00
3- Mo
CT (T)
0.14
0.14
1.00 0.00
1.00 0.00
DECT (T)
0.14
0.14 0.00
0.14
1.00 0.00
1.00 + 0.00
1.00 0.00
SAN
SCCN
0.14 0.00
0.00
1.00 0.00
1.00 0.00
SCCNN (T)
0.14 0.00
0.14
1.00 0.00
1.00 0.00
SCN (T)
0.14 + 0.00
0.00
1.00 = 0.00
1.00 + 0.00
GAT (0)
0.50 0.00
0,50 = 0,00
0.00
0.70
0.70 + 0.00
0.70 0.00
GCN (g)
0.50 0.00
0.50
0.70
0.70 = 0.00
0.70 0.00
MLP (G)
0.00
0.50 0.00
0.50
0.00
0.70
0.70 0.00
TAG (9)
0.00
0.50 0.00
0.50
0.00
0.70 0.00
0.70 0.00
UniMP (g)
0.50
0.50 0.00
0.50
0.70
0.70 0.00
0.70 = 0.00
CelIMP (T)
0.01
0.50
0.01
0.70 0.01
M.M
CT
0.50 0.02
0.50
0.02
0.70 0.00
DECT(T)
0.50 = 0.00
0.50
0.70
0.70 0.00
0.70 0.00
SAN (T)
0.50 0.02
0.51
0.59
0.60 = 0.03
SCCN
0.50 0.01
0.50
0.73
0.65 0.02
SCCNN
0.50
0.23
0.59 0.10
SCN (T)
0.51 0.02
0.001
0.55 0.23
0.61 0.01
GAT (G)
0.50 0.00
0.50 0.00
0.00
0.70 = 0.00
0.70 + 0.00
0.70 0.00
GCN (G)
0.50 1 0.00
0.50 - 0.00
0.50
0.70 0.00
0.70 + 0.00
0.70 = 0.00
MIP (G)
0.50 0.00
0.50 = 0.00
0.50
0.70 0.00
0.70 0.00
0.70 0.00
(C)
0.50 0.00
0.50 = 0.50
0.14
2-M
UniMP (G)
0.50 1.50 0.01
0.50
0.00
0.50
0.15
0.70 0.00
0.70 0.00
CelIMP (T)
0.50 0.01
0.00
0.18
0.70 = 0.00
CT
0.50 0.00
0.50
0.70 0.00
0.70 + 0.00
SAN (T)
0.50 0.00
0.50
0.54
0.49
SCCN (T)
0.50 2 0.00
0.51
0.70 0.00
0.69 0.02
SCCNN (T)
0.50 + 0.00
0.50
0.46 0.22
0.55 + 0.17
SCN (T)
0.50 0.00
0.50 0.00
0.54 0.22
0.68 0.04
29

---

Published as conference paper ICLR 2025
E.3 HOMEOMORPHISM PREDICTION
Table 11: Full results for the homeomorphism type prediction task on the full set of surfaces. Performances are reported with mean and standard deviation over five runs with different seeds.
AUROC
Accuracy
DATASET MODEL (CLASS)
DT
DTO
RNF
DT
DTO
RNF
GAT (9)
0.46
0,46 = 0.00
0.47 = 0.01
0.000
0,80 + 0.00
0.80 0.00
GCN (G)
0.46
0,46 0.00
0.01
0.80
0.80 + 0.00
0.80
MLP (G)
0.46
0.46 + 0.00
0.46
0.80
0.80 + 0.00
0.80 0.00
TAG (9)
0.46
0.46 0.00
0.46
0.80
0.80 = 0.00
0.80
UniMP (g)
0.46
0.46 + 0.00
0.46
0.80
0.80 + 0.00
0.80
CelIMP (T)
0.15
0.01
0.30
0.94
2- M-Mo
CT (T)
0.00
0.69
0.94
0
DECT (T)
0.00
0.45 = 0.00
0.45
0.80
0.80 0.00
0,80
SAN (T)
0.14
0.67
0.35
0.72
SCCN
0.05
0.06
0.78
0.37
SCCNN (T)
0.10
0.02
0.00
0.26
SCN (T)
0.12
0.04
0.40
0.09
(G)
0.00
0.00
0.54 0.00
0.54
GCN (G)
0.00
0.48
0.02
0.54 0.00
0.50
MLP (9)
0.00
0.00
0.08
0.00
0.54 0.00
0.54
TAG (G)
0.49
0,49
0.00
0.01
0.00
0.54 0.00
0.54
UniMP (G)
0.49
0.00
0.09
0.50
0.54 0.00
0.54
CelIMP (T)
0.63
0.00
0.19
0.73
2-M
CT (T)
0.81
0.00
0.74
0.50
DECT (T)
0.50
0.50 + 0.00
0.48 0.10
0,00
0.54 0.00
0.54
SAN (T)
0.49
0.54
0.00
SCCN (T)
0.80
0.05
0.00
0.53
SCCNN (T)
0.59
0.52
0.51 0.12
0.01
SCN
0.13
0.49 0.06
0.10
0.43
GAT (G)
0.41 # 0.03
0.53 + 0.02
0.50
0.00
0.54 + 0.00
0.54
GCN (G)
0.42
0.51 + 0.04
0.50
0.54
0.54
0.00
MLP (C)
0.49 0.04
0.06
0.50
0.54 0.54
0.54
TAG (G)
0.02
0.50 a: 0.03
0.43
0.59
0.33 0.15
0.04
UniMP (G)
0.03
0.42 0.03
0.41
0.15
0.54 0.00
0.54
2-M
CellMP (T)
0.06
0.02
0.47
0.55
CT
0.1
0.09
0.52
0.54
SAN (T)
0.02
0.53
0.48
0.54
SCCN (T)
0.04
0.53
0.04
0.54
SCCNN (T)
0.01
0.01
0.54
0.00
SCN (T)
0.50
0.49
0.35
0.53
30


Reviews:
[{"review_id": "UyHLoHukrV", "reviewer": "Reviewer_r6xH", "summary": "This paper introduces MANTRA, a large-scale, diverse, and high-order dataset of triangulations for benchmarking models in the topological deep learning space. The authors also assess graph and simplicial complex-based models on tasks like betti number estimation, homeomorphism type, and orientability detection. The introduction section is written well with enough definitions/context about various topological properties. The dataset specification includes all properties that are being computed for each object. Furthermore, nine different neural nets are evaluated on the classification task.\n\nOverall, this paper could be a good contribution to topological deep learning, especially for developing advanced TDL methods and benchmarking them. However, the limiting factor is the evaluations (just MP networks) as they do not give a full picture of the impact of the dataset.", "questions": "1. I'm not sure if you mention this in the paper but how are the initial labels and triangulations generated for each surface and three-dimensional manifold?\n\n2. For a given surface, how much computational time is required for computing the fields mentioned Section 2?\n\n3. Would MANTRA be useful for developing models for topological reconstruction (to recover incomplete surfaces or perform noise correction)? If so, how?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Extensive evaluation of currently available graph and simplicial complex-based models on MANTRA.\n\n2. Provides a foundation for developing and benchmarking advanced TDL methods.\n\n3. Insight into the ability of higher-order models to be invariant to triangulation transformations.", "weaknesses": "1. Evaluations are mostly focused on MP networks. It would help get better overall picture of the impact the dataset has by evaluating on other architectures like equivariant high-order neural nets and topological transformers.\n\n2. The variance across results is surprising and I am not sure if running for more epochs/more hyperparameter tuning will address this.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RxHMiyZAx1", "reviewer": "Reviewer_fkvZ", "summary": "The paper introduces MANTRA, a new dataset designed for benchmarking topological deep learning (TDL) models. MANTRA consists of triangulations of two- and three-dimensional manifolds, labeled with various topological invariants such as Betti numbers, Homology torsion, homomorphism type (four options for 2D manifold and five options for 3D manifolds), and more. Using this dataset, the authors benchmark standard graph methods and TDL models on three types of tasks: Betti number prediction (2D + 3D), homeomorphism type prediction (2D), and orientability prediction (2D). Their results suggest that while TDL models are better at predicting higher order topological properties, they struggle with invariant to topology preserving transformations. The latter is demonstrated through additional experiments where barycentric subdivision -- a form of remeshing that preserves topological properties -- is performed on the test set.", "questions": "- **Dataset Generation**\n  - How were the manifolds and triangulations generated?\n  - What considerations did you make, if any, in order to encourage topological diversity in the dataset?\n\n- **Experimental Results**\n  - For the standard deviation results in Table 2, do they represent variation within a single model or across the entire class of models?\n\n- **Computational Issues**\n  - The authors cite computational constraints for limiting some aspect of the experiments, can the authors elaborate on the nature of these limitations given that all triangulations have 10 or fewer vertices?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- MANTRA is a large dataset specifically designed for topological property prediction, filling an important a gap in the TDL literature. Existing datasets are limited in scope, size, topological diversity and often rely on artificial topological lifting of graph data.\n- The paper presents systematic analysis of graph methods vs. topological methods on fundamental topological property prediction tasks, establishing important baselines for future research in TDL.\n- TDL models' ability to capture topological properties (**Q2** in the paper) is a fundamental question in TDL that remained under-explored to this point. Empirical results presented in the paper suggest that TDL models are better, compared to graph based methods, at learning topological properties from data.\n- The question of invariance to topology preserving transformations (**Q3**) is also under-explored (albeit less clearly motivated). This paper demonstrate TDL models' sensitivity to remeshing by barycentric subdivisions -- a simple topology preserving transformation.\n- The paper is clearly organized, with well defined questions, answered in a methodical manner. \n- The authors ensure reproducibility by providing well-documented code repos, versioned datasets, and multiple data format options (raw and PyTorch Geometric), making it easy to build upon this work.", "weaknesses": "- **Topological diversity**\n  - Other than the genus, first Betti number, and homeomorphism type for 2D manifolds, the distribution of topological invariants is limited and unbalanced.\n    - For 3D manifolds:\n       - Most Betti numbers, homeomorphism types, and torsion subgroups have only one or two possible values.\n       - When two values do exist, one value represents over 99% of the cases.\n  - All triangulations are limited to 10 or fewer vertices.\n\n- **Limited model coverage**\n  - The only TDL baselines included in the evaluation are simplicial complex models, excluding:\n      - Cellular complex methods (e.g. [1, 2]).\n      - Combinatorial complex methods (e.g. [3]).\n      - Hypergraph methods (e.g. [4, 5]).\n  - For both TDL and graph based methods, the evaluation only includes message-passing based architectures, excluding non-message passing graph neural networks (e.g. [6, 7, 8, 9]) and attention based TDL models (e.g. [3, 10]).\n  - It would be informative to report the best performing model for each class in the main text.\n\n- **Training methodology**\n  - Models were trained for only 6 epochs, hyperparameters were not optimized. The can possibly lead to underestimation of the fully optimized performance of some of the architectures, limiting the scope of the conclusions that can be drawn from the experiments\n\n---\n\n[1] Bodnar et al. \"Weisfeiler and Lehman Go Cellular: CW Networks\", NeurIPS 2021.\n\n[2] Hajij et al. \"Cell complex neural networks\", Topological Data Analysis and Beyond Workshop NeurIPS 2020.\n\n[3] Hajij et al. \"Topological Deep Learning: Going Beyond Graph Data\", arXiv:2206.00606\n\n[4] Feng et al. \"Hypergraph neural networks\", AAAI 2019.\n\n[5] Kim et al. \"Equivariant hypergraph neural networks.\" ECCV 2022.\n\n[6] Maron et al. \"Provably powerful graph networks.\" NeurIPS 2019.\n\n[7] Abboud et al. \"The surprising power of graph neural networks with random node initialization\", arXiv:2010.01179.\n\n[8] Bouritsas et al. \"Improving graph neural network expressivity via subgraph isomorphism counting\", IEEE PAMI 2022.\n\n[9] Zhang et al. \"A complete expressiveness hierarchy for subgraph gnns via subgraph weisfeiler-lehman tests\", ICML 2023.\n\n[10] Battiloro et al. \"Generalized simplicial attention neural networks\", arXiv:2309.02138.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Fu72hm6H0x", "reviewer": "Reviewer_1p1z", "summary": "The paper aims to benchmark the performance of graph-based and simplicial complex-based models on a newly-introduced 2-/3- manifold triangulation dataset. The paper focused on topological feature prediction tasks, including predicting Betti numbers, homeomorphism type and orientability.\nOverall, the research question is interesting but the paper requires more work to be considered for publication. Please see my detailed comments and questions below.", "questions": "1. about claims in the paper: 1) (058-060) To the best of our knowledge, the only publicly-available high-order dataset is the “Torus” dataset proposed in Eitan et al.(2024), which consists of a small number of unions of tori triangulations. 2) (069-071) The authors claimed this is the __first__ instance of a large, diverse, and intrinsically high-order dataset, comprising triangulations of combinatorial 2- and 3-manifolds. \n\n  - Authors refer to datasets that contain nontrivial higher-order structures as higher-order datasets. It's worthwhile to point out that such structure and the datasets have been long- and well-studied by network science community, as well as numerical modeling, and there has been numerous efforts in building and modeling higher-order topological structures in various networks. However, there are way more such datasets as far as I'm aware of. \n    - In computation biology and social networks, such datasets with higher-order structures appear very often. \n    - in numerical modeling, computational geometry, 2D and 3D manifold triangulations are almost everywhere and they easily grow to the order of millions. \n    - In topological deep learning community, such datasets have been applied by different papers as well, e.g., 1) SaNN: Simple Yet Powerful Simplicial-aware Neural Networks; 2) Unsupervised Parameter-free Simplicial Representation Learning with Scattering Transforms. \n\n  - As a reviewer, I really shouldn't provide a more complete list of such literatures. I believe this paper could benefit from a more thorough literature review. Note that _there is no such dataset_ does not work as a strong motivation. Moreover, this work is hardly the __first__  dataset involved with high-order topological structures.\n\n2. about the experiment tasks: as a researcher in TDA, I would like to see how graph- and simplicial complex-based models compare to the TDA methods for the investigated three tasks in this paper. Again, this is necessarily needed for being a datasets and benchmarks paper.\n\n3. I do not agree with authors *define* the lower/upper degree of a simplex as the number of simplices which share a face/coface with it. This number is merely the number of neighboring simplices, but not the degree for simplices. The _degree_ of a simplex is the number of faces/cofaces it has. Please refer to _Spectra of combinatorial Laplace operators on simplicial  complexes_ and _RANDOM WALKS ON SIMPLICIAL COMPLEXES AND THE NORMALIZED HODGE 1-LAPLACIAN_ for such definitions. \n\n4. Confusing experimental setup: \n   1. Randomly generated scalars are used as feature vectors for graph-based models and randomly generated eight-dim vectors are used as feature vectors for simplicial complex-based models? \n      - Questions: What is the motivation of using randomly generated values as input features? Any useful information do they provide to the models or tasks? \n\n   2. In predicting 0-betti numbers, you only have label 1. When seeing results on predicting $\\beta_0$, I have the following questions:\n      -  If graph-based models have 100% accuracy, how come simplicial-complex based models can be poor not nearly 50%? My question comes from the fact that: simplicial complex based models are __generalizations__ of graph-based models. If GAT/GCN has an accuracy of 100%, then SAN/SCCN, which are really extensions to higher-order simplices, should be able to achieve the same accuracy _when using the same output_ (Note that here I assume authors considered using the same type of output for predictions, e.g., when using node representations of GCN for predicting $\\beta_0$, one should also use node representation of SCCN for predicting $\\beta_0$). \n      -  On the other hand, most of the models implemented in this paper are performing representation learning, there is a need for a classifier or a regressor to predict the labels. Here it matters how a classifier/regressor, which takes the learned representation and computes the needed output, is defined. Yet, I couldn't find any details on this, but only __readout__ layers of different models, which are either mean, or sum of all embeddings. I wonder if authors simply used the readout to predict topological features. It doesn't make much sense to apply a direct readout to predict different topological features. \n   3. what is the dataset $2-\\mathcal{M}_H^1$? I tried to check the appendix but couldn't find it. If I guessed correctly, it is a refined version of $2-\\mathcal{M}_H^0$. But how is this refinement performed? \n   4. I appreciate the authors'effort in investigating the topological invariance of the models. This however is not well-analyzed in the paper, yet, the authors made the claim that high-order MP-based models are not invarinat relative to topological transformations ... in line 284. \n      1. First of all, how is topological transformation defined? Is it only refinement of the triangulation? \n      2. A proper topological predictor should be considered, yet this is unclear from the paper. \n      3. I do not really see message-passing models are implemented. By that, I mean Bodnar et al., 2021a. \n  \n\nother minor suggestions:\n- line 150: It seems incorrect to say something was degraded. I'm not an English major, but I think it should be \"someone was degraded\" or \"something degraded\". Please ignore it if I'm wrong. \n- Please figure out a way to present Tables 9-22 without rotating them 90 degrees. Though I tried reading them, they are still not easy for readers.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "The research question is interesting. It is novel to investigate the performance of graph and high-order network models on predicting topological features.", "weaknesses": "- There are various works on estimating Betti numbers, but the authors did not mention them. They should provide as solid baselines. For example:\n  - Estimating Betti Numbers Using Deep Learning\n  - A (simple) classical algorithm for estimating Betti numbers\n- The experiments are rather unclear, for example, details on how the architectures are not provided. As a paper in datasets and benchmarks, this is necessary and as a reviewer, I'm interested in knowing how the architectures are designed and if they are reasonable. \n  - Some experimental setup choices are not well-justified. For example, why randomly generated scalars are used as input features for graph-based models and randomly generated eight-dim vectors are used as feature vectors for simplicial complex-based models?\n- Authors made a few claims which are not well-supported by either the experiments or literature. \n\nNote: Please see my detailed questions below related to these points.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2jnO0fOfNx", "reviewer": "Reviewer_ChKW", "summary": "This work proposes a synthetic benchmark dataset for topological deep learning. Contrary to graphs representing pairwise interactions, this dataset focuses on higher-order interactions. It is generated by considering the simplicial complexes of 2 and 3-dimensional manifolds, resulting in structures with up to 10 vertices. Samples are labeled with topological descriptors, such as Betti numbers, torsion, orientability, genus, or homeomorphisms.\n\nAfter introducing the dataset, several experiments are carried out to compare the performance of five graph and four topological learning models on topological prediction tasks. The main findings are that topological models perform better than graph models on these tasks, however, topological models are far from having a strong performance on these tasks.", "questions": "- What is your view on the scope of the dataset in terms of the vertices and dimensionality? I would suggest adding a short discussion on this in the limitations. \n- I am not sure if I agree with the statement in line 204: \"To ensure fairness, all configurations use the same learning rate of 0.01\". Different models may work best with different learning rates, which likely requires cumbersome hyperparameter tuning. The authors acknowledge this as a limitation, but I would reconsider the phrasing of this sentence -- is this really a \"fair\" comparison?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The paper is very well presented in terms of structure, rigor, and backing statements with relevant references or evidence.\n- As far as I can tell, good conventions are followed to make the dataset easily available and usable.\n- The experimental protocol is overall solid and the drawn conclusions are relevant and adequate.\n- Limitations are discussed.", "weaknesses": "I have two minor remarks:\n- The scope of the dataset is limited in dimension (order of interactions) and number of vertices, as well as being purely synthetic. While these are not bad aspects in their own right and this certainly is justified given that TDL is in its early days as it allows to study basic properties of models (e.g., remeshing invariance), the benchmark could potentially quickly lose its relevance.\n- As the authors acknowledge, several non-message passing TDL architectures were not included in the experiments. As such, one of the most intriguing interpretations of the results is somewhat diminished (\"suggesting that new approaches are needed to leverage higher-order structure associated with the topological information in the dataset\").", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)