TopoSRL: Topology preserving self-supervised Simplicial Representation Learning
Authors: Hiren Madhu
Sundeep Prabhakar Chepuri
Venue: NeurIPS 2023 poster (2023)
Keywords: Simplicial representation learning
Self-supervised learning
Message passing simplicial networks

Abstract: In this paper, we introduce $\texttt{TopoSRL}$, a novel self-supervised learning (SSL) method for simplicial complexes to effectively capture higher-order interactions and preserve topology in the learned representations. $\texttt{TopoSRL}$ addresses the limitations of existing graph-based SSL methods that typically concentrate on pairwise relationships, neglecting long-range dependencies crucial to capture topological information. We propose a new simplicial augmentation technique that generates two views of the simplicial complex that enriches the representations while being efficient. Next, we propose a new simplicial contrastive loss function that contrasts the generated simplices to preserve local and global information present in the simplicial complexes. Extensive experimental results demonstrate the superior performance of $\texttt{TopoSRL}$ compared to state-of-the-art graph SSL techniques and supervised simplicial neural models across various datasets corroborating the efficacy of $\texttt{TopoSRL}$ in processing simplicial complex data in a self-supervised setting.

TopoSRL: Topology Preserving Self-Supervised Simplicial Representation Learning
Hiren Madhu and Sundeep Prabhakar Chepuri
Indian Insitute of Science
hirenmadhu, spchepuri@lisc.ac.. ac.in in
Abstract
In this paper, introduce TopoSRL, novel self-supervised learning (SSL.) method for simplicial complexes to effectively capture higher-order interactions and pre- serve topology in the learned representations. TopoSRL addresses the limitations of existing graph-based SSL methods that typically concentrate on pairwise re- lationships, neglecting long-range dependencies erucial to capturing topological information. We propose a new simplicial augmentation technique that generates two views of the simplicial complex that enriches the representations while being efficient. Next, we propose new simplicial contrastive loss function that contrasts the generated simplices preserve local and global information present the simplicial complexes. Extensive experimental results demonstrate the superior performance of TopoSRL compared to state-of-the-ae graph SSL. techniques and su- pervised simplicial neural models across various datasets corroborating the efficacy TopoSRL in processing simplicial complex data in self-supervised setting.
1 Introduction
Simplicial complexes are mathematical structures that explicitly capture higher-order relationships between entities (as nodes) using simplicies of different orders (as edges, triangles, and so on). There has been a growing interest in developing simplicial representation learning models, such as simplicial neural networks (SNN) [T simplicial complexes are generally more expressive than graphs, which only capture pairwise relations. SNNs incorporate topological information available in simplicial complexes while learning representations of simplicies of different orders, which are useful for various downstream tasks such as node, graph, higher-order link, and trajectory prediction tasks 6 However, significant challenge in learning representations for simplicial complexes using existing SNN models the need for task-specific labels required for training. Acquiring meaningful labels for real-world, high-dimensional, and complex data is difficult due intricate structures, multiple valid labeling schemes, or privacy and ethical concerns.
Self-supervised learning (SSL) schemes learn expressive and powerful representations without requiring labeled data. Specifically, the main goal in SSL is to model an encoder, which is learned using objective function and unlabelled training data. This paper proposes an SSL method for simplicial complex data that preserves topological and geometric information while learning representations. Although no existing studies focus SSL for simplicial complex data, closely related field of SSL for graph data has been extensively studied. SSL for simplicial complex data important generalization as every simplicial complex inherently includes underlying graph, making SSL on graphs a specialized subset of SSL. on simplicial complexes.
The general idea behind SSL graphs is to augment a graph create two views of the available graph and then maximize the mutual information (MI) between the augmented graphs. So, the research focus thus far has been on designing augmentation techniques and objective functions that maximize MI. However, existing methods of SSL on graphs require complex augmentation (7]. negative
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

sampling algorithms [8] or require components to empirically avoid degenerative solutions (9 12 This leads to a more complex neural model, hindering their direct extension for SSL on simplicial complexes. For instance, deep graph infomax (DGI) [9] maximizes MI between node and its sub- graph by learning two MLPs: one for the subgraph readout and another as a discriminator function classifying if a node exists in the given subgraph or not. Extending DGI to simplicial complexes would necessitate training A + 1 discriminators for a simplicial complex of order (see Section for the definition of the order of simplicial complex), drastically increasing the training complexity. Similarly, graph contrastive representation learning (GRACE) [8] and graph contrastive representation learning with adaptive augmentation (GCA) 7 need the selection of effective negative samples and require additional storage by selecting all the other nodes in the graph as negative samples. Directly extending these methods to simplicial complexes would lead to prohibitive computational complexity going up to ()2%) for a simplicial complex with N nodes. To mitigate the need for negative samples or additional networks, recent approaches like bootstrapped graph representation learning (BGRL) (T SelfGNN (12), and canonical correlation analysis inspired self-supervised learning on graphs (CCA-SSG) (33 attempt learn representations by contrasting corresponding node pairs in the two augmented graphs. Self-supervised masked graph autoencoders (GraphMAE) (14) is an alternative approach that uses reconstruction loss rather than the commonly used contrastive loss in the graph-based SSL methods. However, these methods focus only on contrasting or reconstructing local information and do not account for global long-diste dintdiittnce information available in the network.
Motivated by the aforementioned limitations of SSL methods on graphs, we introduce TopoSRL, a self-supervisee learning pipeline for simplicial complex data that preserves topology information in the representation space. Preserving topology information is crucial because it allows the learned representations to capture higher-order interactions and relationships unique simplicial complexes. TopoSRL comprises an intuitive technique to generate stochastically augmented views of simplicial complex. We introduce new contrastive loss function preserve topology information of simplicial complexes in the geometric space to learn more expressive representations. In sum, our major contributions in TopoSRL are as follows:
Simplicial augmentation: We introduce simple and effective augmentation technique for simplicial complexes that captures the inherent relationships between simplices. In particular, the augmentation method stochastically removes closed simplicies and adds open simplicies guided by the topological structure of a simplicial complex, where a closed simplex is simplex that is part of the simplicial complex, while an open simplex is simplex that itself is not a part of the simplicial complex, but all of its subsets are. This procedure is computationally efficient and leads to superior performance than randomly adding simplicies without accounting for the topological structure of a simplicial complex.
Simplicial contrastive loss:
We propose a loss function for SSL on simplicial complexes that contrasts pairs of corresponding simplicies as well as considers the relational distance between pairs of simplicies and their augmented counterparts, where the relational distance refers to measure of dissimilarity between pairs of simplices. We also provide theoretical evidence that the proposed loss function implicitly maximizes MI between a simplex and its neighborhood within the same augmented simplex and the other augmented simplex, helping the model capture the inherent structures and patterns in the data, thereby improving the model's performance and adaptability to various tasks without the requirement of training additional components such as MI estimators.
We conduct experiments demonstrating our proposed method on downstream tasks such as node classification, simplicial closure, graph classification, and trajectory prediction. We also highlight the effectiveness of TopoSRL in learning expressive representations for simplicial complexes with the proposed augmentation technique opposed to the random augmentation technique. Experiments show that without any complex architectures or expensive augmentation techniques, our method outperforms existing state-of-the-ard graph representation learning methods while being competitive with supervised simplicial representation learning methods.
2 Related works
This section briefly discusses a few popular SNN models, one of which can be used in TopoSRL. The first step of TopoSRL's 's learning phase is simplicial augmentation. Hence, we also discuss existing

---

augmentation techniques used in graph SSL and how they can not be extended (due to computationally or empirical limitations) for simplicial complexes. Then, we move on to discussing SSL on graphs, a specialized version of SSL. on simplicial complexes.
Simplicial representation learning. SNN models learn representations of simplices of different orders (e.g., nodes, edges, triangles, and so on) in simplicial complex [2]3], and are based on an extension of graph convolutions to convolutions over simplicial complexes. Simplicial attention networks (SAN) [T] extends graph attention networks (GAT) [15] for simplicial complexes. Message passing simplicial networks (MPSN) (5) proposes a framework to design more expressive SNN models related to the so-called simplicial Weisfeiler-Lehmar isomorphism test. TopoSRL is free from the choice of a specific SNN encoder, and any one of the SSN models can be used as an encoder.
Graph augmentation. Most of the graph SSL, methods described in Section I. namely, GRACE, GCA, and BGRL., an augmentation technique that includes two steps: (i) add and remove edges at random to generate two graphs from an input graph, and (ii) randomly mask dimensions of initial node features. GCA (7) proposes an adaptive data augmentation technique that models the edge-removal probability differently for each edge in the graph based on the importance of the edge. However, this method requires more expensive augmentations to attain peak performance (M]. Applying an identical augmentation technique to simplicial complexes necessitates modeling removal probabilities for each simplex. This increases the augmentation process's 's complexity and mandates the development of innovative metrics for measuring simplex importance. Other than this, most methods use uniform probability add and remove edges. Although adding and removing simplicies uniformly at random is a naive and simple extension, we empirically show that TopoSRL. performs considerably better than random augmentation.
contrast, TopoSRL uses simpler augmentation technique that only requires the addition of open simplicies, which are inherently rich in information as discussed later in Section 4.1 and is very easy to implement.
SSL on graphs. Contrastive learning methods for images have recently been adapted for graphs. This includes DGI (9. which contrasts node-local patches with global graph representations was inspired by Deep InfoMax [16]. GMI (17) maximizes a concept of graphical mutual information inspired by MINE (18) enabling a more granular contrastive loss than DGI. GRACE (8) and its derivatives. such as GCA 77. which on more complex data adaptive enhancements, have adapted the SimCLR [19] algorithm for graphs. GraphCI. (20) also adapts SimCLR to learn graph-level representations with a contrastive objective. Multi-view graph representation learning (MVGRL) [10] extends contrastive multi-view coding to graphs. All these methods suffer from considerable computational complexities because they rely heavily negative samples. BGRL (1I) and SelAGNN (12) extend Bootstrap you own latent YOL) [2I], which uses different online and target encoders, wherein the target encoder is updated an exponential moving average of the online encoder while the online encoder is updated by optimizing loss function. The difference between BGRL and SelfGNN is that BGRL uses different online and target encoders, but SelfGNN utilizes the same encoder as online and target. Lastly, CCA-SSG uses CCA-base loss. BGRL, SelfGNN, and CCA-SSG are methods free from negative samples and incur lower computational complexity than previous methods. Inspired by these graph SSL methods, TopoSRL aims at a negative sampling- free approach focusing on preserving topology and implicitly maximizing MI.
3 Background
A simplicial complex X' is a collection of a finite number of simplicies. A simplex of order 1: (or, k-simplex) is (ks + 1)-cardinality subset of the vertex set V so that if simplex a% E X' then all the non-empty subsets of σx also belong to the simplicial complex X. The order, K, of a simplicial complex X is the order of the maximally ordered simplex in the simplicial complex.
Suppose oke Tk and Pk: are some k-simplices in a simplicial complex. A k-simplex o has the fol- lowing neighbors in the simplicial complex, namely, nower-adjacent neighbors [(ax) = (Tk PA-1 C ak A Pk-1 C }. apper-adjacent neighbors U(Tx)) = [TeloR C Pk+1 A T C Pk+1). boundary B(Ox) = 1 C ), and co-boundary C(ax) = f 10 C 1.
An open k-simplex is defined In other words, if all the boundaries of simplex are present the simplicial complex, but the simplex itself is not part of the simplicial complex, it is considered open simplex.
k-simplex ds such that all its boundaries B(Ox)) C X, but 0% $ X.
3

---

Drirgnt
Phaa Beir
Rval
(rill gell
Stagpe
Sarzkad ontratioo
Brail
Dereect
Belz
3md
Stepiiaal segamtatii
Figure (a) The TopoSRL pipeline. Here, W weight matrix that emphasizes the relations between simplices in the two augmented simplicial complexes. (b) Simplicial augmentation example.
A generic SNN model has the following aggregation steps
" B (ax) ACCREGATE((1 (N's M VT. E B((x))
1+1
m ++1 (ak) ACGREGATE((c (Noy, n MEE VT&t1 (((x)))
++1 (σx) (C Ni VTe E f(ox)))
J.
mij (ox) AGGREGATE((1) (1o2 1 VTL u((o))))
and combining step
h h++
COMBINE(N', mg 1 (ox)) m (ax), (ax), my m (ax)),
where t is the layer index of the SNN model, on is the transformation function with adjacency of type (We use one layer MLP), h², is the representation of the simplex 0 at layer with ho being the initial embedding. We write the final representations of all the simplices in the simplicial complex X' obtained from such an SNN model with layers compactly as
Z (, H).
where / denotes the set of learnable parameters and H is the initial feature matrix of all the simplices X. Although the above SNN model based on MPSN, different SNN variants can be derived from the above model by choosing different AGGREGATE and COMBINE operators and which neighborhoods to aggregate.
4 The proposed TopoSRL pipeline
The main aim of the proposed approach is to learn an SNN encoder function Ee(-) to compute expressive representations for simplices in a simplicial complex X' self-supervised manner. Towards this end, we first compute two augmented views of simplicial complex X, namely, 2 and (2) (as described later Section 4.1b. Then learn the representations of the two simplicial complexes through the encoder E-(-) as Z (x((1)) H)) and (2)
(x)( H2))), where
(t) = (z Z z the set embedding matrices and z(4) E N xD being the D- dimensional embedding matrix of all the k-simplicies in the simplicial complex g00 and Nx(()) being the total number of f--simplicies in (i), We contrast these representations and learn E-(-) by optimizing the simplicial contrastive loss (developed in Section 4.2) An illustration of the TopoSRL pipeline shown in Figure I a).
4.1 Simplicial augmentation
We introduce novel stochastic augmentation technique for simplicial complexes to generate two simplicial complexes (1) and (2) from X' by adding open simplices and removing closed simplices at random via Bernoulli sampling. To begin with, we compute all the open simplices in X. Next,
Identifying open simplices of order & + 1 in a simplicial complex X costs ((N2), where Ne is the total number of k- simplices. We only compute once outside the training process.

---

Algorithm Simplicial complex augmentation
1: procedure SMMPLICIAJAAIGGEENTTOOON/
2: Kopen < OpenSimplices((
Compute open k-simplices in X' with k 2
3: for i = and 2 do
4:
X² <- X'
5:
for each dı X' with A > do
6:
Draw r ~ Bernoulli(p)
7:
if r = then
D remove closed simplex
8:
x² + (x ox
9:
for each o E Kopen do
10:
Draw a ~ Bernoulli(p) if
11:
then
b add open simplex
12:
x( (x10 Uo}
return x1(1) and x(2)
draw Bernoulli random variable for each open simplex determine whether include the open simplex in X)(0) for i = 2. Similarly, draw another Bernoulli random variable for each k-simplex (k >> 1) in X to determine whether to retain the closed simplex in X for = 1, 2. This procedure summarized as Algorithm
The proposed augmentation method preserves the inherent structure and relationships within the simplicial complex as illustrated in Fig For example, consider the walmart- -tristtttipppa simplicial complex datase wherein a set of items purchased together by a customer in one trip to Walmart a simplex. Now, if customer purchases (peanut butter and j jam), {bread and jam}, and {peanut butter and bread} on three separate occasions, it is highly probable that the customer will buy (peanut butter, jam, bread} together on subsequent trip. Adding the simplex [peanut butter, jam, bread] as in view 2 of Figure [bb will represent an instance of the simplicial complex the data were sampled at a later point in time, and hence still retains and provides better information to contrast compared to adding random simplices.
In contrast, in a random sampling method, where add simplices at random, we need include all the subsets of the simplex that is to be added, which costs ((2²) for each k-order simplex for E (2... K}. As opposed to this, in the proposed simplicial augmentation technique, since we are adding an open simplex, all the subsets of this simplex are present in the simplicial complex, making the additional cost only (1)) and thereby making it computationally more efficient.
4.2 Simplicial contrastive loss
The proposed method aims to learn simplex representations by capturing topological information in higher-order simplices and their distant neighbors. In contrast, most of the existing self- eeffssupervied graph representation learning approaches primarily focus on contrasting corresponding nodes in augmented graphs and do not preserve topological information the objective function does not focus on the relational distance between pairs of two nodes that are not neighbors.
Cost matrices. We define the following cost matrices to compute the TopoSRL loss function:
the intra-view cost matrix, denoted by C((1) E R xN(") for i 1,2, measures the distance between representations of two simplices the simplicial complex x and the inter-view cost matrix, denoted by ((1.2) R
(( x measures the distance between the k-simplex in the two views. The intra-view cost matrix is used to minimize the relational distance between pairs of simplices from two augmented simplicial complexes. Specifically, the (p. entry of C(O) is defined as [c(. Jp.7 || ) where [z("), Ip is the representation of the k-simplex p in (6) On the other hand, the inter-views cost matrix is used to minimize the distance between the representation of simplex and an aggregate representation of a sub-simplicial complex surrounding
httpps:/www.co. htwp:///wwwcccccclllll...cc cornell...c....oooo. htw.a//lletttaaaabtttaaaa//aaaaa
5

---

this simplex in the other augmented simplicial complex. The (p, entry of ((1.2) is defined as [C P:9 = | 1 - 1 2 1,122
(
Weight matrix. We also construct weight matrix Wa E RNE for each k-simplex to capture the relation of a simplex ( E (1)) and oz (2) E x2(2) which will be used to calculate the aggregate representation of sub-simplicial complexes by assigning different importance weights that depend
how many hops 0% is away from 0% in the simplicial complex x²(1). We assign the entries of W follows. If of of
assign higher value Y/o to [Wx], Wolggg oo If o
((o UU(((l))
i.e., if of is in one-hop neighhorhood of ok in x2))), then we assign it a lower value 7)1 Similarly, if (2) o% is in the two-hop neighborhood of o (1) in x²(1), we assign a value of 7/2, such that to > 7/1 > 722 and then we apply row-wise softmax to We normalize it. By construction, We is row-stochastic matrix.
Loss function. The proposed simplicial contrastive loss is convex combination of two terms, namely, the sub-simplicial complex loss Laub and the relative simplicial complex loss Lrel- The term Laub measures the cumulative (over all the simplicies) weighted distance between the representation of simple of x2)) and k-simplices of x²(2) with appropriate weights in the matrix WA. and is given by
K N N ΣΣΣ j=1
Cyub
0.2) k
Je
Walks
(I)
k=
The term Liel calculates the relational distance between pairs of simplices and is given by
N (x(
K
Liel = Fee= Σ Σ
(1c(% [ (Wllss-
(2)
km 2.
For instance, consider two pairs of simplices (f and (af such that of e x! of of E xx(²), [Wilot 0, and [W WWelotoo > 0, that is, (o2.of and (ok,oz) are distant neighbors (not necessarily one-hop). Hence, minimizing frel minimizes the squared difference of the distance between the representations of (of of) and (of of ), which leads similar pairs of simplices from two distinct simplicial complexes maintaining an equal distance. In other words, if of is m-hop away from of and of is m-hop away from of then minimizing Cret will reduce the difference between the distance of the pairs and preserve the m-hop information present in X' Finally, the overall loss is
L aL + (1 Crel,
where [0, 1] is tunable parameter. The overall loss preserves the topological properties as adjacent simplices will be embedded more closely in the Euclidean space because of the Cuub term, capturing the local information. The difference in distance of contrasting simplex pairs will be minimized by minimizing L,el. capturing the global information.
We end this section with two interesting theoretical results about the simplicial contrastive loss function and an overview of the TopoSRL pipeline in Algorithm 2 In the next proposition, we show that Laub preserves the local information in simplicial complex.
Proposition 1. Minimizing Luub is equivalent jointly minimizing a lower bound the distance between the representation of a simplex o x(1) and the aggregate representation of o x( that is adjacent to of and the distance between the representation of of (2) and the aggregate of representations of of. E (2) that is adjacent to of E (1).
Most graph SLL methods maximize MI between augmented graphs using MI estimators 9.. In the next theorem, we show that optimizing the proposed loss function is also related to optimizing MI between representations of simplicies one augmented simplicial complex and representations of
Prooffs available in the supplementary material.

---

Algorithm 2 TopoSRL
1: procedure TOPOSRL(X, P, H)
Di k. is the maximum order of the simplices of interest
2: Initialize 0
- 4: while e < #epochs do
3:
5:
x(1), (((1))(12) & SimpiciilAAugmenaationn// p) 0.....A K.
6:
Calculate Wt for k
7:
z t (2((1) H and ( + (o(A H 2)). Calculate C( cl2) ((1) for 0, ,K.
8:
9:
Minimize L = abuu + orel
C c+1
+ (, H)
12: return
its neighbors in the other augmented simplicial complex and representations of simplices and their neighborhoods within the same augmented simplicial complex conditioned on the input data.
Suppose X represents the random variable corresponding to the input data, X₂ represents the k- simplices in X, and ( represents an augmented view of X, sharing the same sample space as X, and the input simplicial complex data X' X. TopoSRL is designed to learn representations, denoted as Z for the input data and ( for its augmentation. We also denote (AA) and T [(A, B) as the entropy of the random variable A and the MI between the random variables A and B, respectively.
Theorem 1. Minimizing the expected loss Laub (expectation with respect to the random variable X) is equivalent to maximizing the MI between _ and X i.e.
minimize 8
Luub, = maximize T(Z Xx);
for ls 0.1...., K, where Laub, is the kih summand ①. I(Z X2) the MI between the representations of the augmented k-simplices z and the k-simplicies in the originial data Xt.
The above theorem shows that in expectation, minimizing the term Caub is equivalent to mini- mizing the variance between representations in the augmented simplicial complexes, which leads to minimization of the conditional entropy ((Z))) [Xx), implying the maximization of MI as I(Z (0) .x)= = H(z (z [xx). The proof also shows that minimizing the simplicial con- trastive loss implicitly maximizes MI between simplicies from one augmented simplicial complex and representations of neighbors in the other augmented simplicial complex conditioned on the input and representations of simplices and their neighborhoods within the same augmented simplicial complex. This result along the lines of earlier methods like DGI 99) and InfoGraph (22). However, TopoSRL does not require additional components in DGI or InfoGraph for MI maximization.
5 Experiments
We follow the standard setting of self-supervised learning methods. Firstly, we train the encoder with the proposed simplicial contrastive loss. Next, we freeze the encoders' model parameters, extract representations for all the simplicies, and train classifier for the following two downstream tasks. The code is available / com/ Hoom/Hireesnn hawarrer/aauuuooSR.......
Downstream tasks. We focus on node classification, simplicial closure, Irajectory prediction and graph classification tasks. Node classification focuses predicting labels for O-simplicies (aka nodes) in given simplicial complex. We perform node classification on the following publicly available dataset. namely, Serracctppriar-eeeee school, contact hhigh-ihghhhhhe school, We use classification accuracy as the performance metric. In simplicial closure, the aim is to predict the closure of open simplicies in a time series of simplicial complex data. We perform simplicial
conate-bill
'Dataset details are presented in the supplementary material, Datasets are available at uttttps://www.cc cornell. wwwpneelledddeaeeeeeeeeeeaeeeeae aatb//tab///a///////b
7

---

Method
Type
high-school
primary-school
senate-bills
GCN
S
0.440.04
0.30=1..04
0.67=00.6
GraphSage
S
0.27=0.05
0.37=0.05
0.54=0.03
GIN
S
0.18=0.04
0.16#0.02
0.53=0.04
GAT
S
0.34++0.05
0.19+-0.06
0.5+0.04
CCA-SSG
SSL
0.68+0.116
0.14+10.07
0.62+00.04
GCA
SSL
0.18+0.08
0.12+00.05
0.5+0.00
BGRL
SSL
0.11+0.01
0.09+00.01
0.5+0.0
GraphMAE
SSL
0.78
0.2:50.02
0.57
SAN
S
0.86+ 0.04
0.29+ 0.06
0.53 + 0.09
SCNN
S
0.81 = 0.01
0.67
0.615 + 0.05
MPSN
S
0.89 + 0.01
0.79 - 0.06
0.75 = 0.05
TopoSRL
SSL
0.92 + 0.05
0.61 + 0.05
0.724 0.06
Table I: Node classification accuracies on simplicial complex datasets; S stands for supervised setting, and SSL stands for supervueed setting. (Best accuracy bold and second best accuracy is underline)
closure on email-EN email-Enr contact-hipe saraactthigg-schhol datasets. We first split the data across time on these temporal datasets and then train the encoder on the first 80% of the data. The last 20% is used for inference. Since the class distribution is very skewed, we use F1 F1-macro to evaluate the performance. Trajectory prediction focuses on predicting the next node trajectory, given sequence of nodes. We perform experiments on two datasets, namely, Ocean and the Synthetic dataset generated using the method described in (6). We use accuracy the metric for comparison. In graph classification, the focus is on learning representations for the whole graph and classifying them. Clique lifting is used to convert graph to simplicial complex and then use an SNN model to extract representations for the whole graph. We evaluate TopoSRL's performance in graph classification task on PROTEINS, NCI1, REDDIT-E, REDDIT-M and IMDB-B B datasets from the TUDatasets 23 repository.
Baselines. We compare TopoSRL against several supervised techniques employing architectures such as SAN (1), SCNN (3), and MPSN [5). This comparison builds confidence in the expressive capabilities TopoSRL and solidifies its usefulness in learning simplex representations settings with less-la or unlabeled data. Additionally, we conduct experiments involving various graph neural network models, including graph convolutional network (GCN) (24). GAT GIN (25), and GraphSage(26). Furthermore, we also compare our method with the current state-of-the-art graph SSI. methods (3)) BGRL. (TT), GraphMAE (14) and GCA 7 This comparison also clarifies that in the SSL setting. the proposed simplicial SSL method (i.c., TopoSRL) performs better than graph SSL methods, as demonstrated by the cxisting supervised simplicial representation learning methods. For TopoSRL, use 3-layer MPSN the encoder network E-). Details about the experimental setup, hyperparameters, and results with other SNN encoder models are available in the supplementary material. We follow the standard practice where all the results averaged over ten different seeds, and one run is performed for each seed.
Results and discussion. Our results demonstrate that TopoSRL consistently outperforms state-of- the-art graph SSL. methods. Table [ we see that TopoSRL surpasses supervised methods on graphs and state-of-the-art graph SSL techniques while being competitive with supervised approaches for the node classification task. TopoSRL has a superior performance compared to MPSN on the contact -hicctthhhh--ie saigg-schooll dataset while being competitive with MPSN the other two datasets. As can be seen in experiments reported the supplementary material, contact cirraccctpimmryya.m -piiary-sschoo.... school, being a denser dataset, does not benefit significantly from minimizing both the terms, but minimizing only Lrel (i.e., a = 1) improves performance.
Table 2 showcases experimental results on simplicial closure datasets. On this task, we see that the less parameterized SCNN model outperforms both TopoSRL and MPSN on the email-Ena and sestact-primary-seee
datasets, while TopoSRL performs better than MPSN and is competitive overall. These results can attributed to the skewed nature of simplicial closure datasets, where models with large parameters like MPSN and SAN are prone to overfitting labels. Although ToLoSRL

---

Method
Type
email-Earoo
contact-primastt contact-priimary-schoo contact-high-schen
SAN
S
0,57:0.12
0.3910.03
0.3:0.07
SCNN
S
0.61 0.61-00.08
0.49+00.9
0,41+0.11
MPSN
S
0.43-10.07
0.43:10.05
0.4710020
TopoSRL SSL
0.59:10.05
0.46:10.01
0.43:10.0
Table Simplicial closure performance using Fl scores; stands for supervised setting and SSL stands for
self-supervised setting.
Method
Type
Ocean
Synthetic
Projection
S
27.15:1..0
52.0+F0.0
SCoNe
30.0++0.6
55.4
SCNN
S
28.5+0.66
50.5+1.0
TopoSRL
SSL
42.0=3..0
50.0:1.0
Table 3: Trajectory prediction performance using accuracy scores; stands for supervised setting, and SSL stands for self-supervised setting.
focuses preserving topology rather than concentrating specific downstream task, it has improved performance compared to these models.
Table 3 presents the results trajectory prediction. As we can the results indicate that TopoSRL outperforms SCNN and ScoNe when applied to the Ocean dataset. When tested on synthetic dataset, TopoSRL demonstrates performance par with supervised SCNN. These results further highlight the expressive representation capabilities TopoSRL on oriented simplicial complexes and its use cases in practical applications. As supervised graph baselines and simplicial basclines. Further, TopoSRL outperforms or performs on par with graph SSL baselines, showing the advantages of TopoSRL standard graph dataset with clique lifting compared to graph SSL methods.
Table 4 TopoSRL performs par with
Method
Type
PROTEINS
NCI1
REDDIT-B
REDDIT-K
IMDB-B
GCN
0.58
0.53
0.71
0.49
0.69
GraphSage
S
0.61
0.48
0.69
0.5
0.68
GIN
S
0.61
0.702
0.73
0.57
0.71
GAT
S
0.57
0.45
0.53
0.5
0.6
CCA-SSG
SSL
0.64
0.67
0.72
0.58
0.675
BGRL
SSL
0.62
0.63
0.72
0.61
0.66
SAN
S
0.64
0.702
0.71
0.59
0.72
SCNN
S
0.61
0.370
0.69
0.57
0.67
MPSN
S
0.63
0.702
0.71
0.59
0.67
TopoSRL
SSL
0.75
0.700
0.72
0.606
0.695
Table 4: Graph classification accuracies on TUDatasets; S stands for supervised setting, and SSL
stands for self-supervised setting.
To further validate the efficacy of our augmentation technique, we conduct an ablation study, compar- ing TopoSRL with and without the proposed augmentation strategy. The results indicate significant performance improvement when incorporating the proposed simplicial augmentation, underlining its crucial role in capturing long-range dependencies and higher-order interactions. Table 5 shows that our augmentation technique results in considerable performance gains over random augmentation methods. We observe that only adding open simplices and not removing closed simplices is better augmentation technique than random augmentation, but its performance deteriorates compared to the proposed augmentation technique. This occurs because the contrastive objective function is more ef- fective when there is more information to contrast. Removing closed simplices allows the contrastive loss and encoder contrast more information, resulting in higher performance instead only adding open simplices. Additionally, we have observed that incorporating higher-order information from simplices with more than three vertices leads to reduced performance. This reduction due to the limited presence of higher-order simplices in these datasets, which causes overparameterization of

---

Dataset
Order
R
00
0
contact-high-schoo
0.62=0005
0.81=00044
0.92=0005
contact-high-schon contact-prinar--sttool
0.6::0.05
0.78+0.03
0.85=0.06
0.36=0.06
0.51
0.61=0.05
contact-primary-schood
0.49
0.44
0.52+
senate-bills
0.52+00007
0.66++0.05
0.02
Table 5: Performace node classification using random augmentation; R stands for random
augmentation, O0 stands for adding only open simplices augmentation, and O stands for augmentation with open simplicies using Algorithm
the model and slight decline in performance. More experimental results comparing different SNN encoders and different values of alpha are reported in the supplementary material.
To test the ability of TopoSRL to learn expressive representations, we perform the following experi- ment: 1) Use the pre-trained TopoSRL encoders to extract representations and 2) Use only partially labeled (e.g., 20% train and 80% test, 40% train and 60% test, etc.) data to train logistic regression school dataset. MPSN and GCN are trained with cross-entropy loss the train set. Since the weights for the TopoSRL encoder trained without labels have been saved, no new encoders were trained to produce the results. As we can see in Table with increase in the size of the train set, the performance increases across all the methods. Furthermore, TopoSRL has a significantly improved performance of about 5% over supervised MPSN in the 20-80 and 40-60 split. This provides empirical evidence about the expressive capabilities of TopoSRL and its efficacy with less-labeled data. Hence, TopoSRL would be preferable over standard supervised models in the less- data setting.
classifier for node classification task in the contact-high hontact-highsschool
Method
Type
20-80
40-60
60-40
80-20
GCN
S
0.314
0.34
0.36:10.04
0.4:40.04
CCA-SSG
SSL
0.39110.04
0.45
0.53
0.68
BGRL
SSL
0.48:10.00
0.49+10.00
0,48:10.00
0.51
SCNN
S
0.71
0.74
0.81
0.86110.04
MPSN
S
0.74
0,80+
0.86
0.89100011
TopoSRL
SSL
0.79
0.84
0.86+0.02
0.92+00.05
Table 6: Node classification accuracies on contact-high migh-scchoo;; S stands for supervised setting, and SSL stands for self supervised setting. 20-80 refers to 20% data for training. 80 percent for test.
Conclusions
We have introduced novel framework for SSL on simplicial complexes by leveraging topological properties underlying a simplicial complex. In particular, we have proposed two key components: stochastic simplicial augmentation method and a simplicial contrastive loss function, which col- lectively provide a mechanism for learning representations that retain local and global topological information in the simplicial complex. The proposed simplicial augmentation method offers the advantage of generating topologically consistent views of the original simplicial complex, allowing the model to learn rich set of features efficiently. The contrastive loss function comprises two primary terms: the sub-simplicial complex loss and the relative simplicial complex loss. The former focuses on minimizing the distance between the representation a simplex and its adjacent simplices in the augmented complex, thereby preserving local information. The latter aims to maintain relative spatial relationships present in the original simplicial complex, thereby capturing global information. We have also theoretically proved that the proposed loss function is related to the MI objective function, which is commonly used SSL on graphs. Our model outperforms state-of-the-e graphs SSL methods on variety of datasets. We believe this work lays a solid foundation for further exploration of SSL methods for simplicial complexes, e.g., via alternative augmentation methods, contrastive loss functions, scalable models, and new applications, potentially opening up new avenues for topological data analysis.
10

---

References
[1] Christopher Wei Jin Goh, Cristian Bodnar, and Pietro Liò. Simplicial Attention Networks. 2022.
DO1: 48550/ 2204. 09455 URL: https:://araai org/ aba/2204 09455
[2] Stefania Ebli, Michaël Defferrard, and Gard Spreemann. "Simplicial neural networks". In: arXiv preprint arXiv:2010.0366 (2020).
[3] Maosheng Yang, Elvin Isufi, and Geert Leus. "Simplicial convolutional neural networks".
In: ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE. 2022, pp. 8847-8851.
[4] Eric Bunch et al. "Simplicial 2-complex convolutional neural nets". In: arXiv preprint arXiv:2012.660/4 (2020).
[5] Cristian Bodnar et al. "Weisfeiler and lehman go topological: Message passing simplicial networks". In: International Conference on Machine Learning. PMLR. 2021, pp. 1026-1037.
[6] T. Mitchell Roddenberry. Nicholas Glaze, and Santiago Segarra. "Principled Simplicial Neural Networks for Trajectory Prediction". In: Proceedings of the 38th International Conference on Machine Learning. Ed. by Marina Meila and Tong Zhang. Vol. 139. Proceedings of Machine Learning Research. PMLR, 2021, pp. 9020-9029. URL: https: proceedings mlr
Bates/v139/roddeneerr///
html
[7] Yanqiao Zhu et al. "Graph contrastive learning with adaptive augmentation". In: Proceedings of the Web Conference 2021. 2021, pp. 2069 2069-2080.
[8] Yanqiao Zhu al. "Deep Graph Contrastive Representation Learning". In: CoRR
abs/2006.0413/ (2020). arXiv: 2006. .4131 URL: htl::htttss//arxiv org/abs/ 2/1//00000011111 [9] Petar Velickovic et al. "Deep graph infomax." In: ICLR (Poster) 2.3 (2019), p.
[10] Kaveh Hassani and Amir Hosein Khasahmadi. "Contrastive multi-view representation learning graphs". In: International conference on machine learning. PMLR. 2020, pp. 41 4116-4126
[11] Shantanu Thakoor et al. "Large-scale representation learning graphs via bootstrapping". In: arXiv preprint 1xX:::2102.063 (2021).
[12] Zekarias T. Kefato and Sarunas Girdzijauskas. "Self-supervised Graph Neural Networks
without explicit negative sampling". In: CoRR abs/2103. 14958 (2021). arXiv: 2103. 14958
URL: wttph:ttt/stttt /tttps://ariivv http://aaxxvvvo.oabbb/ at///bbb//103 14958
[13] Hengrui Zhang et al. "From canonical correlation analysis to self-supervised graph neural networks". In: Thirty-Fifth Conference on Neural Information Processing Systems. 2021.
[14] Zhenyu Hou et al. "Graphmae: Self-supervised masked graph autoencoders". In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2022,
pp. 594-604.
[15] Petar Veličković et al. Graph Attention Networks. 2018. arXiv: 1710 10903 [stat.M, ML]
[16] R Devon Hjelm et al. Learning deep representations by mutual information estimation and maximization. 2019. arxiv: 1808. 06670 [stat. ML]]
[17] Zhen Peng et al. "Graph Representation Learning via Graphical Mutual Information Maxi- mization". In: Proceedings of The Web Conference 2020. WWW '20. Taipei, Taiwan, 2020, URL: hittps://do
259-270. ISBN: 978 450370233, DOI: 10. 33883333362233331122
org/10. 10/ 1145/3366423 3380112
[18) Mohamed Ishmael Belghazi al. "Mutual Information Neural Estimation". In: Proceedings of the 35th International Conference on Machine Learning. Ed. by Jennifer Dy and Andreas Krause. Vol. 80. Proceedings of Machine Learning Research. PMLR, 2018, pp. 531-540. URL: attps://proceeding
mlr. press v80/bel wel.ghazi18....
[19] Ting Chen al. "A Simple Framework for Contrastive Learning of Visual Representations" In: Proceedings of the 37th International Conference on Machine Learning. Ed. by Hal Daumé III and Aarti Singh. Vol. 119. Proceedings of Machine Learning Research. PMLR, 2020, pp. 1597-1607. URL:
mlr [20] Yuning You et al. "Graph contrastive learning with augmentations". In: Advances in neural information processing systems 33 (2020), pp. 5812-5823.
[21] nan-BBstien Grill et al. "Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning" In: CoRR abs/2006.07733 (2020). arXiv: 2006. 07733
URL: https: //rxxvv
org/ faabb//2006.7777

---

[22] Fan- Yun Sun et al. "Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization". In: arXiv preprint arXiv:1908. 01000 (2019).
[23] Christopher Morris et al. "TUDataset: A collection of benchmark datasets for learning with graphs". In: /CML 2020 Workshop Graph Representation Learning and Beyond (GRL+ 2020). 2020. aiv: 2007 08663 URL: www. www.ggrphl..om earning.
[24] Thomas N. Kipf and Max Welling. "Semi-Suye Semi--upervised Classification with Graph Convolutional
Networks". In: International Conference on Learning Representations. 2017. URL: https
//openreview wwwwwwwwwww. weee//oorumidddSJ44aaa
[25] Keyulu Xu et al. "How powerful are graph neural networks?" In: arXiv preprint arXiv:1810.0032. (2018).
[26] William L. Hamilton, Rex Ying, and Jure Leskovec. "Inductive Representation Learning Large Graphs", In: CoRR abs/1 70s/1706600216 (2017). arXiv: 1706 02216 URL: http
//arxiv.oo warxiv.org/abeee /rgg/bb//700...o
12


Reviews:
[{"review_id": "E9LAD1G38n", "reviewer": "Reviewer_RqW1", "summary": "The work proposes a Self-Supervised Learning (SSL) approach for simplicial complexes, extending graph SSL to higher-order structures. While recent works exist that extend and adapt standard graph representation learning techniques to simplicial complexes, none consider SSL. In particular, the authors propose to employ a contrastive learning procedure that involves two steps: first, for each complex, two artificial simplicial complexes are created with a data augmentation technique that randomly removes closed simplices and randomly adds open ones to the complex. Then, the complexes are contrasted with an ad hoc contrastive loss that has two goals: the first is to minimize the distance of each k-simplex across the two augmented views, and the second is to minimize the distance of a $k$-simplex and the aggregated representation of a sub-simplicial complex surrounding it in the other view. The per-simplex distances are simply given by the squared euclidean distance. The authors then prove that the proposed contrastive loss implicitly maximizes the mutual information between a simplex and its neighborhood within the same augmented simplex and the mutual information between a simplex and its neighbors in the other augmented complex. The approach is tested over node classification and simplicial closure for a set of social network datasets, and shows competitive performance wrt a set of supervised baselines both acting on graphs and simplicial complexes. The work is also shown to outperform an existing graph SSL baseline.  ", "questions": "- Table 2: to be called an ablation study, shouldn’t it also consider the case in which the closed simplices are not removed while the open simplices are still added?\n- Are the results averaged over different seeds? how many runs were performed?\n", "limitations": "N/A", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "- The paper is well written and easy to understand, providing the reader the necessary background and explaining the contribution in simple terms.\n- The work is the first to study the problem of self-supervised learning on simplicial complexes, noting that existing graph SSL methods may not be directly adapted. Given the recent raise of interest in the field, this work may well be the first of many.\n- The approach makes intuitive sense, is simple and has theoretical justification. Being model agnostic, the approach can (and has been) used on any simplicial complex encoder.\n- The experimental evidence seems to indicate that, on the considered tasks, the approach is surprisingly competitive with supervised baselines.", "weaknesses": "- Part of the reason why there are no works on self-supervised learning on simplicial complexes is that data distributions that can be naturally designed as simplicial complexes are rare, and the work indeed only considers social network datasets that are only seen in works of the same niche. For this reason, it would be interesting to see whether the approach still shows benefits on a standard graph dataset converted to complex with lifting technique (e.g. using clique complexes). Indeed, main works in the field, e.g. [1], usually also compare on established graph datasets such as TUDatasets. \n- The method only presents a single graph SSL baseline, assessing it to be the state of the art. In general, it would still be desirable to have several baselines for two reasons: i) the considered SOTA model could prove to be better only in the datasets, tasks or evaluation setting considered in the original paper, and ii) the comparison with less-performing baselines would still be valuable to the experimenter as well as the reader.\n- It is not immediately clear how the supervised graph-learning baselines were chosen, and why no chosen network is at least 1-WL expressive. It would be good to at least report the results for GIN [2].\n- The method and insights may not be of immediate interest to the broader NeurIPS community as they are specifically tailored to simplicial complexes, and would probably be of greater interest in a graph-specific venue. While the venue also accepts domain-specific works, these usually have more general insight that can be appreciated by practitioners outside the niche. \n- Much of the utility of self-supervised learning comes when it can be used for pretraining an encoder on a large dataset that can be later fine-tuned for a downstream task. It would be great to see if this is the case for the proposed approach, as otherwise it's not immediately clear why it would be advisable wrt to standard supervised models.\n\nOverall, I am more than happy to raise my score if my concerns are addressed.\n\n**References**: \n\n[1] Bodnar, C., Frasca, F., Wang, Y., Otter, N., Montufar, G. F., Lio, P., & Bronstein, M. (2021, July). Weisfeiler and lehman go topological: Message passing simplicial networks. In International Conference on Machine Learning (pp. 1026-1037). PMLR.\n\n[2] Xu, K., Hu, W., Leskovec, J., & Jegelka, S. (2018). How powerful are graph neural networks?. arXiv preprint arXiv:1810.00826.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable feedback. We address your concerns and questions as follows:\n### Graph classification on TUDatasets using clique lifting\nWe have incorporated your suggestion and included Table G2 in the global response PDF, presenting the performance comparison between the proposed method $\\texttt{TopoSRL}$ with the graph and simplicial baselines on graph classification datasets. \n\nAs we can see in Table G2, $\\texttt{TopoSRL}$ performs on par with supervised graph baselines and simplicial baselines. Further, $\\texttt{TopoSRL}$ outperforms or performs on par with graph SSL baselines, showing the advantages of $\\texttt{TopoSRL}$ on a standard graph dataset with clique lifting compared to graph SSL methods.\n\nFor more details, please refer to **[GC]** in the global response.\n\n### Graph baselines\nTo compare with more graph SSL baselines, we have added BGRL and GCA as graph SSL baselines with GIN as graph supervised baseline. The results are available in Table G1 in the global response PDF. \n\nThe graph baselines do not perform as well as simplicial baselines. Whereas, $\\texttt{TopoSRL}$ outperforms all the graph baselines on the node classification task and is competitive with supervised simplicial neural network baselines.\n\nPlease refer to **[GBS]** in the global response for more details.\n\n### Transfer learning/Fine-tuning\nIn graph SSL, generally, the encoder is trained on a graph that it is supposed to infer on, unlike in images where a large network can be fine-tuned to a domain-specific dataset later. However, to show the expressiveness of the representation of our method, we perform and present an experiment with partially labeled data. The results for the same are available in Table G4 in the global response PDF. \n\n$\\texttt{TopoSRL}$ significantly improved performance by about 5% over supervised MPSN in the 20-80 and 40-60 split. This provides empirical evidence about the expressive capabilities of $\\texttt{TopoSRL}$ and its efficacy with less-labeled data. Hence, $\\texttt{TopoSRL}$ would be preferable over standard supervised models in the less-labeled data setting. \n\nPlease refer to **[PLD]** in the global response for more details about the experiment setup.\n\n### Open-only augmentation\nAdding open simplices is a better augmentation technique than random augmentation, but its performance deteriorated from the proposed augmentation technique. This occurs because the contrastive objective function is more effective when more information to contrast is present. Removing closed simplices allows the contrastive loss and encoder to contrast more information, resulting in higher performance instead of only adding open simplices.\n\nPlease find our answer in **[OOA]** in global response as a response to this question.\n\n### Seeds and runs\nWe follow the standard practice where all the results are averaged over ten different seeds, and one run is performed for each seed.\n\n### Interests to the NeurIPS community and impact\nWe understand that the NeurIPS community has a broader spectrum of interest and that there are many graph representation learning works that were reported to NeurIPS. We believe that our new results on SSL and the importance of topology preservation have broader relevance. The proposed method has implications beyond simplicial complexes and can inspire more developments in representation learning for other complex data structures, such as hypergraphs or multi-relational data. As such, developing representation learning networks for higher-order networks is a timely topic gaining significant attention. \n\nWe hope you appreciate our responses and that all your concerns are addressed."}, {"review_id": "y9fQRtxHmB", "reviewer": "Reviewer_T7H9", "summary": "This work designs an SSL pipeline coined as TopoSRL for simplicial complex data, and proposes a new contrastive loss function to preserve topology information to learn more expressive representations.", "questions": "In lines 131-132,  the statements about lower-adjacent neighbors and upper-adjacent neighbors are not clear enough.  Can authors give some examples like Figure 1?\nIn lines 137-138, What does the \\phi mean? And the statements like \"∀τk+1 ∈ C(σk)\" need to move outside.", "limitations": "N/A.", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "1. The motivation is meaningful. \n2. The experiments are fairly comprehensive. ", "weaknesses": "1. In Table 1 and  Table  3, the baselines are all supervised settings, i.e., lack of comparison with other self-supervised algorithms.\n2. The visualization results in Figure 2 do not adequately reflect the differences in clustering ability among the algorithms. Can authors give some numerical evaluations such as AMI and ARI?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable feedback. We address your concerns and questions as follows:\n### Lack of SSL baselines\nTo compare with more graph SSL baselines, we have added BGRL and GCA as graph SSL baselines and GIN [1] as graph supervised baseline. The results are available in Table G1 in the global response PDF. The graph baselines do not perform as well as simplicial baselines. Whereas, $\\texttt{TopoSRL}$ outperforms all the graph baselines on the node classification task and is competitive with supervised simplicial neural network baselines. \n\nPlease refer to **[GBS]** in global response for more details.\n### AMI and ARI\nWe report the AMI and ARI metrics for contact-high-school dataset presented in Figure 2 from the manuscript in the following table:\n|      |  AMI   |  ARI   |\n|------|--------|--------|\n| $\\texttt{TopoSRL}$ | <u>0.7608</u> | <u>0.7094</u> |\n| CCA-SSG | 0.6003 | 0.3965 |\n| MPSN   | **0.7811** | **0.7498** |\n| GCN    | 0.1634 | 0.1049 |\n\nAs we can see by these metrics and as mentioned in the manuscript, MPSN achieved better clustering compared to $\\texttt{TopoSRL}$. However, MPSN also loses some expressiveness over $\\texttt{TopoSRL}$ as all the nodes in a cluster are in a confined space and lose information. In contrast, $\\texttt{TopoSRL}$ manages to cluster most classes but the nodes in a cluster are not too close like in MPSN. For more details, please refer to lines 302-310 in the manuscript.\n### Upper and lower adjacent neighbors\nThank you for the question. In Figure 1b in the manuscript, in the original view, the 2-simplices (or filled triangles) indicated by {Sanitizer, Wipes, Shampoo} and {Sanitizer, Wipes, Detergent} are lower adjacent neighbors as they share a common lower order 1-simplex (or edge) {Sanitizer, Wipes}. At the same time, 1-simplices {Sanitizer, Wipes} and {Sanitizer, Detergent} are upper adjacent as they are both part of a higher order 2-simplex {Sanitizer, Wipes, Shampoo}. Additionally, 2-simplices {Sanitizer, Wipes, Shampoo} and {Sanitizer, Wipes, Detergent} are also upper adjacent neighbors as they are both part of 3-simplex {Sanitizer, Wipes, Shampoo, Detergent}.\n### What is $\\phi$?\n$\\phi$ is a message transformation function. In our setting, we use a one-layer MLP with no bias. \n### Correcting notation\nThank you for pointing this out. The RHS of eqautions in line 138 should be $AGGREGATE(\\phi_C(h^t_{\\sigma_k},h^t_{\\tau_{k+1}}),\\forall \\tau_{k+1}\\in C(\\sigma_k))$. This will be corrected in the revised paper.\n\nWe hope you appreciate our responses and that all your concerns are addressed.\n\n[1] Xu, Keyulu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. \"How powerful are graph neural networks?.\" arXiv preprint arXiv:1810.00826 (2018)."}, {"review_id": "xuVjRqSw6T", "reviewer": "Reviewer_VjqH", "summary": "The paper introduces a new algorithm for self-supervised learning for simplicial complex-based networks.\nThe paper proposes a new simplicial contrastive loss function that contrasts the generated simplices to preserve local and global information present in the simplicial complexes. \nExperiments show the utility of the proposed method. In particular, it outperforms another SSL method for simplicial complexes. Also, its quality is on par with supervised counterparts.", "questions": "1. Can you please explain the Fig.2, right. What is the structure of the dataset? What is the prediction target?\n2. The inter-view cost matrix is used to minimize the distance between the representation of a simplex and an aggregate representation of a sub-simplicial complex surrounding this simplex in the other augmented simplicial complex.\nWhy this value should be minimized?\n3. Why you haven't compared with other SSL baselines in Table 3?", "limitations": "Authors adequately addressed the limitations.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper belongs to a new field: topological deep learning, which is a generalization of graph neural networks to more complex structures (simplicial complexes, cell complexes, hypergraphs, etc). There are not so many papers dedicated to this field. Literature review is adequate.\nExperiments show that the proposed SSL method outperforms another (SOTA) SSL method. Confidence intervals are provided. Theoretical results regarding the proposed losses are provided. The quality is almost on par with supervised methods. \nhe language of the manuscript is clear.", "weaknesses": "Learning with simplical complexes is a quite niche research area. So, the impact of the paper is limited.\nAlso, main ideas like using augmented complexes is similar to such augmentation for graphs [1] and the contrastive loss is also quite popular.\nPossible applications of simplicial networks must be discussed in more details.\n\n[1] Kaveh Hassani and Amir Hosein Khasahmadi. Contrastive multi-view representation learning\n429 on graphs. In ICML, pages 4116–4126, 2020.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable feedback. We address your concerns and questions as follows:\n\n### On similarity with graph augmentation and contrastive learning\nWe acknowledge that the concept of augmentation for graphs has gained popularity in graph SSL methods. However, as highlighted in the Related Works section and supported by empirical evidence in Table 2 in the main manuscript, merely extending graph augmentation methods yields unsatisfactory results on considered tasks. Furthermore, as presented in Table S2 in supplementary material, both the terms involved in the proposed contrastive loss are essential. Although we drew inspiration from the general graph SSL pipeline, adapting it to simplicial complex data posed non-trivial challenges such as requiring complex augmentation techniques, computational cost from negative sampling algorithms based, or requiring components to avoid degenerative solutions empirically. A detailed discussion of these challenges is available in the Introduction section from lines 34-53 in the manuscript.\n\n### Possible applications\nWe appreciate your feedback. There are quite a few applications [4] of simplicial complexes, image processing [1], molecular structure analysis [2], trajectory prediction [3], etc.\n\nIn response to Reviewer Lthj, we assessed $\\texttt{TopoSRL}$ on the trajectory prediction task and presented the findings in Table G3a within the global response PDF. The results indicate that $\\texttt{TopoSRL}$ outperforms SCNN and ScoNe [3] when applied to the Ocean Drifters dataset. When tested on synthetic data, $\\texttt{TopoSRL}$ demonstrates performance on par with a supervised SCNN. These results further highlight the expressive representation capabilities of $\\texttt{TopoSRL}$ on oriented simplicial complexes and its use cases in practical applications.\n\nPlease refer to **[TP]** in the global response for more details.\n\n### More explanation on Figure 2\nFigure 2 compares T-SNE plots for four different methods: $\\texttt{TopoSRL}$, CCA-SSG, MPSN and GCN. The dataset used is contact-high-school simplicial complex, and the node classes are the prediction target. Each point in the plot corresponds to the representations of a node from the contact-high-school dataset. To restate what is mentioned in the manuscript, MPSN can cluster most nodes in a confined space and create clear class boundaries except for a few. Meanwhile, GCN and CCA-SSG methods cannot capture higher-order information and show similar artifacts. $\\texttt{TopoSRL}$ manages to cluster some classes, but nodes in clusters are far from the center as in MPSN, which preserves more information. For example, the two clusters on the bottom and one from the right (corresponding to classes 1,2,3) are students from the same year but in different divisions. This information is preserved with $\\texttt{TopoSRL}$ as we can see three different clusters with some separation. However, there are a few overlaps as well, aligning with the nature of real-world data as students from the same year are often good friends (either due to extracurricular activities or study groups).\n\n### Importance of minimizing distance between simplex and sub-simplial complex\nProposition 2 demonstrates that minimizing the proposed loss function simultaneously maximizes the mutual information between simplices and their corresponding representations. On the other hand, Proposition 1 reveals that minimizing the proposed loss function leads to minimizing the distance between a simplex's representation and the aggregate representation of a sub-simplicial complex surrounding that simplex in the other augmented simplicial complex. This process also maximizes mutual information (MI), aligning with the principles seen in earlier methods like DGI, GCA, and CCA-SSG. However, unlike DGI and GCA methods, $\\texttt{TopoSRL}$ achieves MI maximization without the need for additional components such as inductive readout functions and discriminator functions or the need for negative samples. As a result, $\\texttt{TopoSRL}$ maximizes MI drastically while being computationally efficient.\n\n### Lack of graph baselines in simplicial closure task\nSimplicial closure is a task inherently specific to simplicial complexes and is not well-suited for a graph encoder. Due to this, the results presented in the manuscript do not include a direct comparison between simplicial closure and graph-based baselines. Instead, our focus has been on evaluating the performance of simplicial closure using appropriate methods tailored for simplicial complex tasks.\n\n### Impact\nThank you for your response. As such, representation learning for graphs and higher-order generalizations such as hypergraphs, simplicial, and cell complexes is a timely topic receiving much attention in the community. The method is developed for simplicial complexes but has implications beyond simplicial complexes. It can inspire more developments in representation learning for other complex data structures, such as hypergraphs or multi-relational data. Since annotating simplices of different orders or labeling a simplicial complex is difficult, developing and understanding SSL methods for simplicial complexes is natural.\n\n[1] Asao, Y., Nagase, J., Sakamoto, R. and Takagi, S., 2021. Image recognition via Vietoris-Rips complex. arXiv preprint arXiv:2109.02231.\n\n[2] Gong, W., Wee, J., Wu, M.C., Sun, X., Li, C. and Xia, K., 2022. Persistent spectral simplicial complex-based machine learning for chromosomal structural analysis in cellular differentiation. Briefings in Bioinformatics, 23(4), p.bbac168.\n\n[3] Roddenberry, T.M., Glaze, N. and Segarra, S., 2021, July. Principled simplicial neural networks for trajectory prediction. In International Conference on Machine Learning (pp. 9020-9029). PMLR.\n\n[4] Hajij, Mustafa, Ghada Zamzmi, Theodore Papamarkou, Nina Miolane, Aldo Guzmán-Sáenz, and Karthikeyan Natesan Ramamurthy. \"Higher-order attention networks.\" arXiv preprint arXiv:2206.00606 (2022)."}, {"review_id": "PiSfE9GoX8", "reviewer": "Reviewer_3cmn", "summary": "This paper proposed a novel self-supervised learning (SSL) method for simplicial complexes. Advanced results are achieved by introducing appropriate augumentation methods and contrastive loss, taking into account the topological properties of the simplicial complex. It also compares the proposed method with the supervised and SSL methods, including the conventional method for graphs, and shows the effectiveness of the proposed method.\n", "questions": "Please give me your opinion on what you stated in Weakness.", "limitations": "The authors do not explicitly address Limitation. The conditions of applicability should be clarified.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper proposes a novel SSL framework for simplicial complexes. The key to SSL, data augmentation and contrastive loss, are analogous to general methods, but they are non-trivial because they require consideration of data-specific characteristics, so it is commendable that a specific framework has been developed.", "weaknesses": "The concern is what effect the proposed method is intended to have. The experiments section includes a comparison of graph analysis methods and simplical complexes analysis methods and a comparison of supervised and SSL. Some of the references [1-6] include those where the effect is formally certified, but the target seems to be those that are treated as simplical complexes(e.g. polygons with holes). In other words, it is an object that cannot be processed as a graph.\nTherefore, the comparison of the graph method with the simplical complexes method reiterates the superiority of the simplical complexes method, but since the data sets used are not common in graph analysis and are few in number, there is a strong suspicion that the superior data sets have been selected. On the other hand, if the subject is to propose SSL methods for simplical complexes, the main focus is on comparisons within simplical complexes. However, since supervised and SSL cannot be simply compared, it should be clarified how the effect should be recognized.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's insightful comments.\n\n### On comparison with graph analysis methods\nWhile it is true that simplicial complexes cannot be directly processed as graphs, it is important to note that every simplicial complex inherently includes an underlying graph that captures the 0-simplices and 1-simplices. The comparison with graph analysis methods was motivated by the node classification task, an area where graph neural networks have shown significant advancements. By comparing results in node classification between $\\texttt{TopoSRL}$ and graph analysis methods, we aim to highlight the potential value of $\\texttt{TopoSRL}$, even in tasks traditionally handled by graph neural networks. The comparison with graph analysis methods further reinforces that including higher-order information improves performance, even in tasks at which graph neural networks are proficient. \n\n### Regarding Dataset selection\nOur choices were not biased toward datasets with superior performance. Instead, we use the standard simplicial complex datasets representing real-world applications and enabling meaningful comparisons. Furthermore, to address reviewer RqW1's comment, we extended our experiments to include graph classification tasks on the TUDatasets. The results demonstrate that $\\texttt{TopoSRL}$ can perform on par with supervised graph baselines and simplicial baselines. Other than this, $\\texttt{TopoSRL}$ also outperforms or performs on par with graph SSL baselines, showing benefits on a standard graph dataset compared to graph SSL methods. For more details, please refer to **[GC]** in the global response.\n\n### Comparison between supervised and SSL method\nThe reason for comparison with supervised methods is to provide insights into how well the SSL method works compared to the supervised method. This comparison builds confidence in the expressive capabilities of $\\texttt{TopoSRL}$ and solidifies its usefulness in learning simplex representations in settings with less-labeled or unlabeled data. \n\nWe hope you appreciate our responses and that all your concerns are addressed.\n"}, {"review_id": "ivHXHa2UAy", "reviewer": "Reviewer_Lthj", "summary": "The paper introduces a new self-supervised learning (SSL) method called TopoSRL for simplicial complexes. It claims to overcome the limitations of existing graph-based SSL methods by capturing higher-order interactions and preserving topology in learned representations. The authors propose a simplicial augmentation technique to generate enriched representations. They also propose a simplicial contrastive loss function to preserve local and global information in simplicial complexes. The experimental results allegedly demonstrate that TopoSRL outperforms state-of-the-art graph SSL techniques and supervised simplicial neural models on various datasets.", "questions": "- It would be valuable to explore the proper downstream task that can showcase the true benefits of the proposed method.\n- Additional information on the number of 3-simplices and higher-dimensional simplices in the datasets would help assess the effectiveness of the proposed method.\n- It would be beneficial to understand how the method addresses the computational burden arising from higher-order simplices.", "limitations": "- The datasets used in the paper lack higher-order simplices, leading to overparameterization.\n- The effectiveness of the proposed method may not be fully showcased without datasets containing a sufficient number of higher-order simplices.\n- Addressing computational issues associated with higher-order simplices is important.\n- If datasets with an adequate number of higher-order simplices are not readily available, it may be challenging for the proposed method to demonstrate its effectiveness in real-world data without access to extremely large datasets.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 3, "strengths": "- The proposed method offers an interesting and intuitive approach.\n- It addresses the capture of higher-order interactions and preserves topology in learned representations.\n- The simplicial augmentation technique enriches representations.\n- The proposed simplicial contrastive loss function preserves local and global information in simplicial complexes.", "weaknesses": "- The paper could improve its presentation and writing to enhance accessibility for general readers.\n- The organization of the method section can be refined to provide clearer explanations and reduce overflow of mathematical expressions.\n- Important definitions are placed in footnotes, which is not ideal.\n- The effectiveness of the proposed method in practice is not adequately demonstrated.\n- The choice of datasets used may not fully reflect the performance potential of the proposed method.\n\nTo enhance accessibility for general readers, improvements can be made in the presentation and writing style of the paper. Specifically, organizing the method section more effectively, providing clearer explanations, and reducing the overflow of mathematical expressions would greatly improve understanding. Placing important definitions within the main text instead of footnotes would also aid comprehension.\n\nWhile the proposed method holds promise, the paper would benefit from further demonstration of its effectiveness in practical applications. The choice of datasets used in the experiments may not fully reflect the performance potential of the proposed method. Exploring a wider range of datasets, including those with a higher number of 3-simplices and higher-dimensional simplices, would help evaluate the method more comprehensively.\n\nIn terms of demonstrating the method's effectiveness, the results of the experiments are unclear. On one hand, the proposed method appears to improve the node-classification task, but it seems that 0-simplicies can already be readily incorporated into graphs. On the other hand, the authors mention that the method does not display outstanding performance in the experiment on simplicial closure datasets because it is not designed for a specific downstream task. This raises the question of what would be the proper downstream task that can truly showcase the benefits of the proposed method.\n\nTherefore, it would be valuable to investigate the proper downstream task that can best showcase the advantages of the proposed method. Additionally, understanding how the method addresses the computational burden associated with higher-order simplices would provide valuable insights into its scalability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable feedback. We address your concerns and questions as follows:\n\n### Improving readability\nTo improve the readability, we will place the important definitions in the main text, reduce the overflow of equations, and add an image to explain intuitively the working of the loss function.\n\n### Choice of datasets and inclusion of higher order simplices\nWe have chosen the standard simplicial complex datasets available used in the literature. It is common practice in the simplicial complex literature to not consider simplices of order four or above [1-5]. In fact, our results in Table 2 in the main manuscript align with this. Including higher-order information increases computational complexity without significant or no gains in performance at all. As such, it is a hyperparameter to restrict the simplicial complex to a certain order. Due to these reasons, we evaluate our method on datasets with 3-simplices and do not consider orders higher than that.\n\n### Clarity on experiments and results\n$\\texttt{TopoSRL}$ aims to learn task-agnostic representations without the requirements of any labels. \n\nWe would like to clarify the confusion created by lines 284-290 in the manuscript. While $\\texttt{TopoSRL}$ does not perform the best in the simplicial closure task, it has competitive results to a supervised MPSN. The reasoning is that the datasets for simplicial closure are skewed; therefore, the learned transformation functions $\\phi$ are not expressive enough to predict simplicial closure. On the other, since $\\texttt{TopoSRL}$ learns task-agnostic representations, it makes $\\texttt{TopoSRL}$ more expressive than a supervised MPSN. This claim can be further verified by analyzing the confidence intervals of the best-performing methods in simplicial closure tasks. \n\nFor example, on the email-Enron dataset, while SCNN performs the best in expectation, the confidence interval ranges from 0.53 to 0.69. While for $\\texttt{TopoSRL}$, it ranges from 0.54 to 0.64. Similarly, the best-performing method on contact-high-school has a confidence interval of 0.27 to 0.67. While for $\\texttt{TopoSRL}$, it is 0.43 to 0.43. This shows that even though $\\texttt{TopoSRL}$ has sub-optimal performance for simplicial closure tasks, it learns expressive representations and is more reliable than its supervised counterparts.\n\n### Effectiveness of $\\texttt{TopoSRL}$ and Proper downstream tasks\nSince $\\texttt{TopoSRL}$ is intended to be an SSL representation learning method for simplicial complexes, the proper downstream tasks are various tasks that exist in the simplicial complex literature. We have shown the capabilities of $\\texttt{TopoSRL}$ on two such tasks, node classification and simplicial closure. \n\nFurthermore, we have conducted experiments for two more downstream applications, namely, 1) graph classification and 2) trajectory prediction. We have also presented the performance of partially labeled data when inferred with $\\texttt{TopoSRL}$ and other baselines. These results are available in the global response and global response PDF. Please refer to **[GC], [TP],** and **[PLD]** in the global response where we highlight the effectiveness of $\\texttt{TopoSRL}$.\n\n### Computational Cost\nAs mentioned in the manuscript, the cost of finding open simplices of order k is $\\mathcal{O}(N_k^2)$. This is required to be done once outside of the training loop. Using the proposed augmentation technique, the complexity of adding random open simplices is $\\mathcal{O}(N^o_k)$ where $N^o_k$ is the number of open simplices of order k. The complexity of one forward pass for one layer of an MPSN is $\\mathcal{O}(D(N_{k-1} + N_{k} + N_{k+1}))$ for an arbitrary order k where D is the diameter of the underlying graph. Hence, calculating one forward pass costs $\\mathcal{O}(2D(\\sum_k N_k))$. Complexity to calculate $\\mathcal{L}$ is $\\mathcal{O}(DN_k^2+\\delta^2)$ for any arbitrary order k, and $\\delta$ is the number of sampled pairs for $\\mathcal{L}_{rel}$ calculation. So the total complexity of the system is of $\\mathcal{O}(LD(\\sum_k N_k^2) + \\sum_k N^{o}_k + K\\delta^2)$, where $L$ is the number of layers in the encoder. So the complexity of $\\texttt{TopoSRL}$ scales polynomially with the number of simplices in the $k^{th}$ order and the order of the simplicial complex $K$.\n\n[1] Christopher Wei Jin Goh, Cristian Bodnar, and Pietro Liò. Simplicial Attention Networks. 2022.DOI: 10.48550/ARXIV.2204.09455. URL: https://arxiv.org/abs/2204.09455.\n\n[2] Stefania Ebli, Michaël Defferrard, and Gard Spreemann. “Simplicial neural networks”. In:arXiv preprint arXiv:2010.03633 (2020).\n\n[3] Eric Bunch et al. “Simplicial 2-complex convolutional neural nets”. In: arXiv preprint arXiv:2012.06010 (2020).\n\n[4] Cristian Bodnar et al. “Weisfeiler and lehman go topological: Message passing simplicial\nnetworks”. In: International Conference on Machine Learning. PMLR. 2021, pp. 1026–1037.\n\n[5] Roddenberry, T.M., Glaze, N. and Segarra, S., 2021, July. Principled simplicial neural networks for trajectory prediction. In International Conference on Machine Learning (pp. 9020-9029). PMLR.\n"}]

Author rebuttal:
We thank all the reviewers for their insightful comments and thoughtful questions. As suggested by reviewers, we have conducted the following experiments and presented them in the global response PDF. The new results are in red color.

### **Graph Classification on TU Datasets [GC]**
- As suggested by reviewer RqW1, we have included new experiments to provide insights on how well $\texttt{TopoSRL}$ works on standard graph datasets with clique lifting. To this end, we have conducted experiments on graph classification task on Proteins, NCI1, Reddit-B, Reddit-M, and IMDB-B datasets from the TUDatasets repository [2]. To extract graph embeddings from simplex embeddings, we perform average pooling on the embeddings of each order $k$ as $z^i_k$, where $i$ denotes $i^{th}$ graph and the graph embeddings are the concatenation of all the orders as follows: $z^i = [z^i_0|| z^i_1|| …|| z^i_K]$ where $K$ is the maximal order of the simplex in the graph and $||$ is the concatenation operator. Due to the time limitation, we have only included results for one seed.
- As we can see in Table G2, $\texttt{TopoSRL}$ performs on par with supervised graph baselines and simplicial baselines. Further, $\texttt{TopoSRL}$ outperforms or performs on par with graph SSL baselines, showing the advantages of $\texttt{TopoSRL}$ on a standard graph dataset with clique lifting compared to graph SSL methods. Nonetheless, we can include results from more seeds in the revised paper.


### **More Graph SSL baselines [GBS]**
- As suggested by most of the reviewers, we have added three more graph baselines for comparison. In particular, our analysis has added GCA and BGRL as graph SSL baselines and GIN as supervised graph baselines. All the graph SSL baselines were trained with GIN as an encoder for both graph and node classification. Due to time constraints, we have only added the two graph SSL baselines. However, simplicial baselines still outperform the newly added graph baselines, and the results will also hold for other graph baselines.
- The results for node classification with new baselines are available in Table G1 in the global response. As we can see, the graph baselines do not perform as well as simplicial baselines. Whereas $\texttt{TopoSRL}$ outperforms all the graph baselines on the node classification task and is competitive with supervised simplicial neural network baselines. 

### **Partially labeled data [PLD]**
- To test the ability of $\texttt{TopoSRL}$ to learn expressive representations, we perform the following experiment, 1) Use the pre-trained $\texttt{TopoSRL}$ encoders to extract representations and 2) Use only a partial labeled (e.g., 20% train and 80% test, 40% train and 60% test, etc.) data to train a logistic regression classifier for node classification task in the contact-high-school dataset. MPSN and GCN are trained with cross-entropy loss on the train set.
- Since the weights for the $\texttt{TopoSRL}$ encoder trained without labels have been saved, no new encoders were trained to produce the results. The results for the same are presented in Table G4 in the global responses PDF.
- As we increase the size of the train set, the performance increases across all the methods. Furthermore, $\texttt{TopoSRL}$ has a significantly improved performance of about 5% over supervised MPSN in the 20-80 and 40-60 split. This provides empirical evidence about the expressive capabilities of $\texttt{TopoSRL}$ and its efficacy with less-labeled data. Hence, $\texttt{TopoSRL}$ would be preferable over standard supervised models in the less-labeled data setting.

### **Open-only simplex argumentation [OOA]**
- As suggested by reviewer RqW1, we now considered a case of augmentation where only open simplices are added while close simplices are not removed. We have conducted experiments with this method and presented the results in Table G3b of the global response PDF.
- Only adding open simplices is a better augmentation technique than random augmentation, but its performance deteriorates compared to the proposed augmentation technique. This occurs because the contrastive objective function is more effective when there is more information to contrast. Removing closed simplices allows the contrastive loss and encoder to contrast more information, resulting in higher performance instead of only adding open simplices.

### **Trajectory Prediction (additional task) [TP]**
- As suggested by reviewer LthJ, we perform experiments on the trajectory prediction task to demonstrate the effectiveness of $\texttt{TopoSRL}$ in downstream applications. 
- The task of trajectory prediction (TP) is described formally as follows: A trajectory over a simplicial complex $\mathcal{X}$ with node set $\mathcal{V}$ is a sequence of nodes $[i_0, i_1, \dots, I_{p-1}]$ such that $ I_{j-1}$ and $i_j$ are adjacent. Now, TP aims to predict what node $i_p$ will be. For more details on trajectory predictions, please refer to [1].
- The results for the task are presented in Table G3a in the global response, where we compare our method with supervised ScoNe [1] and SCNN on ocean drifters and synthetically generated data using the approach proposed in [1].
- The results indicate that $\texttt{TopoSRL}$ outperforms SCNN and ScoNe [1] when applied to the Ocean Drifters dataset. When tested on synthetic data, $\texttt{TopoSRL}$ demonstrates performance on par with a supervised SCNN. These results further highlight the expressive representation capabilities of $\texttt{TopoSRL}$ on oriented simplicial complexes and its use cases in practical applications.

[1] Roddenberry, T.M., Glaze, N. and Segarra, S., 2021, July. Principled simplicial neural networks for trajectory prediction. In International Conference on Machine Learning (pp. 9020-9029). PMLR.

[2] Morris, C., Kriege, N.M., Bause, F., Kersting, K., Mutzel, P. and Neumann, M., 2020. Tudataset: A collection of benchmark datasets for learning with graphs. arXiv preprint arXiv:2007.08663.

Meta-review: The paper presents a self-supervised method for simplicial complexes, demonstrating competitive performance over prior art over a range of tasks. The paper received borderline-positive reviews initially, and then underwent further discussion in the post-rebuttal phase. Key strengths were consistently identified by the reviewers in the quality of the exposition, the novelty of the problem (which could potentially become seminal in the graph learning commmunity), and the quality of the experimental results. The main weaknesses were given in terms of limited comparisons, to which the authors responded with an extensive rebuttal, including several additional experiments and comparisons as requested by the reviewers. As a consequence, two of the initial ratings were raised to Weak Accept. We believe the paper carries a good potential impact, and meets the quality bar for presentation at NeurIPS. In the light of the main strengths as recognized by the reviewers and the additional input in the authors' responses, we recommend acceptance with the additional recommendation to revise the paper accordingly for the camera ready version.
Decision: Accept (poster)