Title: Hodge-Aware Learning on Simplicial Complexes
Venue: NeurIPS 2023
Accepted: no
Keywords: hodge decomposition

simplicial complexes

spectral simplicial theory

simplicial neural network

stability
Abstract:   Neural networks on simplicial complexes (SCs) can learn from data residing on simplices such as nodes, edges, triangles, etc. 
  However, existing works often overlook the Hodge theory that decomposes simplicial data into three orthogonal characteristic subspaces, such as the identifiable gradient, curl and harmonic components of edge flows.
  In this paper, we aim to incorporate this data inductive bias into learning on SCs. 
  Particularly, we present a general convolutional architecture 
  which respects the three key principles of uncoupling the lower and upper simplicial adjacencies, accounting for the inter-simplicial couplings, and performing higher-order convolutions. 
  To understand these principles, we first use Dirichlet energy minimizations on SCs to interpret their effects on mitigating the simplicial oversmoothing. 
  Then, through the lens of spectral simplicial theory,
  we show the three principles promote the Hodge-aware learning of this architecture, in the sense that the three Hodge subspaces are invariant under its learnable functions and the learning in two nontrivial subspaces are independent and expressive.
  To further investigate the learning ability of this architecture, we also study it is stable against small perturbations on simplicial connections.
  Finally, we experimentally validate the three principles by comparing with methods that either violate or do not respect them.
  Overall, this paper bridges learning on SCs with the Hodge decomposition, highlighting its importance for rational and effective learning from simplicial data.

Abstract

Neural networks on simplicial complexes (SCs) can learn from data residing on simplices such as nodes, edges, triangles, etc. However, existing works often overlook the Hodge theory that decomposes simplicial data into three orthogonal characteristic subspaces, such as the identifiable gradient, curl and harmonic components of edge flows. In this paper, we aim to incorporate this data inductive bias into learning on SCs. Particularly, we present a general convolutional architecture which respects the three key principles of uncoupling the lower and upper simplicial adjacencies, accounting for the inter-simplicial couplings, and performing higher-order convolutions. To understand these principles, we first use Dirichlet energy minimizations on SCs to interpret their effects on mitigating the simplicial oversmoothing. Then, through the lens of spectral simplicial theory, we show the three principles promote the Hodge-aware learning of this architecture, in the sense that the three Hodge subspaces are invariant under its learnable functions and the learning in two nontrivial subspaces are independent and expressive. To further investigate the learning ability of this architecture, we also study it is stable against small perturbations on simplicial connections. Finally, we experimentally validate the three principles by comparing with methods that either violate or do not respect them. Overall, this paper bridges learning on SCs with the Hodge decomposition, highlighting its importance for rational and effective learning from simplicial data.

1 Introduction

It is not uncommon to have polyadic interactions in such as friendship networks [1], collaboration networks [2], gene regulatory networks [3], etc [4-6]. To remedy the pitfall that graphs are limited to model pairwise interactions between data entites on nodes, simplicial complexes (SCs) have become popular among others [7]. A SC can be informally viewed as an extension of a graph, which is the simplest SC, by including, not limited to, some triangles over the edge set. SCs like graphs have algebraic representations - the Hodge Laplacians, an extension of graph Laplacians [8, 9]. Moreover, besides node-wise data, SCs can support data on general simplices, e,g., flow-type data, e.g., water flows [10], traffic flows [11], information flows [12], etc., naturally arise as data on edges, and data related to three parties, e.g., triadic collaborations [2], can be defined on triangles in a SC.

Thus, existing works have built NNs on SCs to learn from such simplicial data, to name a few, 30 [13-19]. In analogous to graph neural networks (GNNs) learning from node data relying on the 31 adjacency between nodes, the idea behind these works is to rely on the relations between simplices. 32 Such relations can be twofold: first, two simplices can be lower and upper adjacent to each other, 33 such as an edge can be (lower) adjacent to another edge via a common node, and can also be (upper) 34 adjacent to others by sharing a common triangle; and second, there exist the inter-simplicial couplings 35 (or simplicial incidences) such that a node can induce data on its incident edge and a triangle can 36 cause data on its three edges, or the other way around. Along with this idea, [15, 16, 19] proposed 37 convolutional-type NNs by applying the simplicial adjacencies, [14, 20] included also inter-simplicial 38 couplings, and [17, 21] generalized the graph message-passing [22] to SCs based on both relations. 39

However, these works often solely focus on the simplicial structures, overlooking the Hodge decom40 position [23], which gives three orthogonal subspaces that uniquely characterize the simplicial data. 41 An edge flow can be decomposed into three distinct parts: a curl-free part induced by some node 42 data, a divergence-free (div-free) part that follows flow conservation (in-flows equal to out-flows at 43 nodes), and a harmonic part that is both div- and curl-free. Meanwhile, real-world simplicial data 44 often presents properties to be in certain subspaces but not others, or its components carry physical 45 usefulness, e.g., statistical ranking, exchange market [24], traffic networks [11], brain networks [12], 46 game theory [25], etc. Thus, intuitively, as an example, a Hodge-biased learner should not, at least not 47 primarily, learn in the div-free space if the edge flow is curl-free. If the learning function preserves the 48 subspaces and operates independently in three subspaces, the learning space is substantially shrunk. 49 This in fact provides an important inductive bias allowing for rational and effective learning on SCs. 50

51

52

53

54

55

56

57

Motivated by this, in this paper, we present the general convolutional learning on SCs, SCCNN, which respects three key principles of uncoupling the lower and upper simplicial adjacencies, accounting for the inter-simplicial couplings, and performing higher-order convolutions. Unlike existing convolutional methods [14-16], which either lack theoretical insights or only discuss their architectural differences in the simplicial domain, we focus on providing a theoretical analysis of these three principles from both the perspectives of simplicial and simplicial data, specifically Hodge theory. This offers deeper and unique insights when compared to the more closely related works [19, 17].

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

Main contributions. In Section 3.2, we first use Dirichlet energy minimizations on SCs to understand how uncoupling the lower and upper adjacencies in Hodge Laplacians and the inter-simplicial couplings can mitigate the oversmoothing inherited from generalizing GCN to SCs. Under the help of spectral simplicial theory [26-28], in Section 4, we characterize the spectral behavior of SCCNN and its expressive power. We show SCCNN performs independent and expressive learning in the three subspaces of the Hodge decomposition, which are invariant under its learning operators. This Hodge-awareness (or Hodge-aided bias) allows for effective and rational learning on SCs compared to MLP or simplicial message-passing [17]. In Section 5, we also prove it is stable against small perturbations on the strengths of simplicial connections, and show how three principles can affect the stability. Lastly, we validate our findings on different simplicial tasks, including recovering foreign currency exchange (forex) rates, predicting triadic and tetradic collaborations, and trajectories.

2 Background

Simplicial complex and simplicial adjacencies. A k -simplex s k is a subset of V = { 1 , . . . , n 0 } with cardinality k + 1 . A face of s k is a subset with cardinality k . A coface of s k is a ( k + 1) -simplex that has s k as a face. Nodes, edges and (filled) triangles are geometric realizations of 0-, 1- and 2-simplices. A SC S of order K is a collection of k -simplices, k = 0 , . . . , K , with the inclusion property: s k -1 ∈ S if s k -1 ⊂ s k for s k ∈ S . A graph is a SC of order one and by taking into account some triangles, we obtain a SC of order two. We collect all k -simplices of S in set S k = { s k i } i =1 ,...,n k with n k = |S k | , i.e., S = ∪ K k =0 S k . For s k , We say a k -simplex is lower (upper) adjacent to s k if they share a common face (coface). For computations, an orientation of a simplex is chosen as an ordering of its vertices (a node has a trivial orientation). Here we consider the lexicographical ordering s k = [1 , . . . , k +1] , e.g., a triangle s 2 = { i, j, k } is oriented as [ i, j, k ] .

Algebraic representation. Incidence matrix B k describes the relations between ( k -1) - (i.e., faces) and k -simplices, e.g., B 1 is the node-to-edge incidence matrix and B 2 edge-to-triangle. We have B k B k +1 = 0 by definition [9]. The k -Hodge Laplacian is L k = B /latticetop k B k + B k +1 B /latticetop k +1 with the lower Laplacian L k, d = B /latticetop k B k and the upper Laplacian L k, u = B k +1 B /latticetop k +1 . We have a set of L k , k = 1 , . . . , K -1 in a SC of order K with the graph Laplacian L 0 = B 1 B /latticetop 1 , and L K = B /latticetop K B K . Note that L k, d and L k, u encode the lower and upper adjacencies of k -simplices. For example, for k = 1 , they encode the edge-to-edge adjacencies through nodes and triangles, respectively.

Simplicial data. A k -simplicial data (or k -signal) x k ∈ R n k is defined by an alternating map f k which assigns a real value to a simplex, and restricts that if the orientation of a simplex is anti-aligned with the reference orientation, then the sign of the signal value will be changed [9].

Incidence matrices as derivative operators on SCs. We can measure how a k -signal x k varies w.r.t. 90 the faces and cofaces of k -simplices by applying B k x k and B /latticetop k +1 x k [29]. For a node signal x 0 , B /latticetop 1 x 0 91 computes its gradient as the difference between adjacent nodes. Thus, a constant x 0 has zero gradient. 92 For an edge flow x 1 , [ B 1 x 1 ] j = ∑ i<j [ x 1 ] [ i,j ] -∑ j<k [ x 1 ] [ j,k ] computes its divergence , which is the 93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

difference between the in-flow and the out-flow at node j , and [ B /latticetop 2 x 1 ] t = [ x 1 ] [ i,j ] -[ x 1 ] [ i,k ] +[ x 1 ] [ j,k ] computes the curl of x 1 , which is the net-flow circulation in triangle t = [ i, j, k ] .

Theorem 1 (Hodge decomposition [23, 9]) . The k -simplicial data space admits an orthogonal direct sum decomposition R n k = im( B /latticetop k ) ⊕ ker( L k ) ⊕ im( B k +1 ) . Moreover, we have ker( B /latticetop k +1 ) = im( B /latticetop k ) ⊕ ker( L k ) and ker( B k ) = ker( L k ) ⊕ im( B k +1 ) .

In the node space, this is trivial as R n 0 = ker( L 0 ) ⊕ im( B 1 ) where the kernel of L 0 contains constant node data and the image of B 1 contains nonconstant data. In the edge case, three subspaces carry more tangible meaning: the gradient space im( B /latticetop 1 ) collects edge flows as the gradient of some node signal, which are curl-free ; the curl space im( B 2 ) consists of flows cycling around triangles, which are div-free ; and flows in the harmonic space ker( L 1 ) are both div- and curl-free. In this paper, we inherit the names of three edge subspaces to general k -simplices. This theorem states that any x k can be uniquely expressed as x k = x k, G + x k, H + x k, C with gradient part x k, G = B /latticetop k x k -1 , curl part x k, C = B k +1 x k +1 , for some x k ± 1 , and harmonic part following L k x k, H = 0 .

3 Simplicial Complex CNNs

We start by introducing the general convolutional architecture on SCs, followed by its properties, then we discuss its components from an energy minimizations perspective. We refer to Appendix B for some illustrations. In a SC, a SCCNN at layer l computes the k -output x l k with x l -1 k -1 , x l -1 k and x l -1 k +1 as inputs, i.e., a map SCCNN l k : { x l -1 k -1 , x l -1 k , x l -1 k +1 } → x l k , for all k . It admits a detailed form

1) Previous output x l -1 k is passed to a simplicial convolution filter (SCF [30]) H l k of orders T d , T u , which performs a linear combination of the data from up to T d -hop lower-adjacent and T u -hop upper-adjacent simplices, weighted by two sets of learnable weights { w l k, d ,t } , { w l k, u ,t } .

2) x l -1 k, d = B /latticetop k x l -1 k -1 and x l -1 k, u = B k +1 x l -1 k +1 are the lower and upper projections from ( k ± 1) -simplices via incidence relations, respectively. Then, x l -1 k, d is passed to a lower SCF H l k, d := ∑ T d t =0 w ′ l k, d ,t L t k, d , and the upper projection x l -1 k, u is passed to an upper SCF H l k, u := ∑ T u t =0 w ′ l k, u ,t L t k, u . Lastly, the sum of the three SCF outputs is passed to an elementwise nonlinearity σ ( · ) .

This architecture subsumes the methods in [14-16, 19, 18, 20]. Particularly, we emphasize on the key three principles. 1) Uncouple the lower and upper Laplacians: this leads to an independent treatment of the lower and upper adjacencies, achieved by using two sets of learnable weights; 2) Account for the inter-simplicial couplings: x k, d and x k, u carry the nontrivial information contained in the faces and cofaces (by Theorem 1); and 3) Perform higher-order convolutions: considering T d , T u ≥ 1 in SCFs which leads to a multi-hop receptive field on SCs. In short, SCCNN propagates information across SCs based on two simplicial adjacencies and two incidences in a multi-hop fashion.

3.1 Properties

Simplicial locality. SCFs admit an intra-simplicial locality: H k x k is localized in T d -hop lower and T u -hop upper k -simplicial neighborhoods [30]. A SCCNN preserves such locality as σ ( · ) does not alter the information locality. It also admits the inter-simplicial locality between k - and ( k ± 1) -simplices, which extends to simplices of orders k ± 2 if L ≥ 2 because B k σ ( B k +1 ) = 0 [31]. Moreover, the two localities are coupled in a multi-hop way through SCFs such that a node not only interacts with its incident edges and the triangles including it, but also those further hops away.

/negationslash

Complexity. A SCCNN layer has the parameter complexity of order O ( T d + T u ) and the computational complexity O ( k ( n k + n k +1 ) + n k m k ( T d + T u )) , linear to the simplex dimensions, where m k is the maximum of the number of neighbors for k -simplices.

Equivariance. SCCNNs are permutation-equivairant, which allows us to list simplices in any order, and orinetation-equivariant if σ ( · ) is odd, which gives us the freedom to choose reference orientations. In Appendix B.3, we provide formal discussions on such equivariances and why permutations form a symmetry group of a SC and orientation changes are symmetries of data space but not SCs.

3.2 A perspective of SCCNN from Dirichlet energy minimization on SCs

Definition 2. The Dirichlet energy of x k is D ( x k ) = D d ( x k )+ D u ( x k ) := ‖ B k x k ‖ 2 2 + ‖ B /latticetop k +1 x k ‖ 2 2 .

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

For node signals, D ( x 0 ) = ‖ B /latticetop 1 x 0 ‖ 2 2 = ∑ i ∑ j ‖ x 0 ,i -x 0 ,j ‖ 2 is a /lscript 2 -norm of the gradient of x 0 . For edge flows, D ( x 1 ) is the sum of the total divergence and curl, which measure the flow variations w.r.t. nodes and triangles, respectively. In general, D ( x k ) measures the lower and upper k -simplicial signal variations w.r.t. the faces ( D d ( x k ) ) and cofaces ( D u ( x k ) ). A k -signal x k with D ( x k ) = 0 follows L k x k = 0 , called harmonic , e.g., a constant node signal and a div- and curl-free edge flow.

Simplicial shifting as Hodge Laplacian smoothing. [14, 20] considered H k as a weighted variant of I -L k , generalizing the GCN layer [32]. This simplicial shifting step is necessarily a Hodge Laplacian smoothing [31]. Given an initial x 0 k , we consider the Dirichlet energy minimization:

with step size η > 0 . The simplicial shifting x l +1 k = w 0 ( I -L k ) x l k is a gradient descent with η = γ = 1 and weighted by w 0 , then followed by nonlinearity. A minimizer of Eq. (2) with γ = 1 is in the harmonic space ker( L k ) . Thus, an NN composed of simplicial shifting layers may lead to an output with exponentially decreasing Dirichlet energy as it deepens, i.e., simplicial oversmoothing .

Proposition 3. If w 2 0 ‖ I -L k ‖ 2 2 < 1 , D ( x l +1 k ) in a simplicial shifting exponentially converges to 0.

/negationslash

This generalizes the oversmoothing of GCN and its variants [33-35]. However, when uncoupling the lower and upper parts of L k in this shifting, associated with γ = 1 , the decrease of D ( x k ) can slow down or cease because the objective instead looks for a solution primarily in either ker( B k ) (for γ /lessmuch 1 ) or ker( B /latticetop k +1 ) (for γ /greatermuch 1 ), not necessarily in ker( L k ) , as we show in Section 6.

Inter-simplicial couplings as sources. Given some nontrivial x k -1 and x k +1 , we consider

which has a gradient descent x l +1 k, gd = ( I -η L k ) x l k + η ( x k, d + x k, u ) . It resembles the whole layer in [14, 20], x l +1 k = w 0 ( I -L k ) x l k + w 1 x k, d + w 2 x k, u with some weights, followed by nonlinearity. We have D ( x l +1 k ) ≤ w 2 0 ‖ I -L k ‖ 2 2 D ( x l k ) + w 2 1 λ max ( L k, d ) ‖ x k, d ‖ 2 2 + w 2 2 λ max ( L k, u ) ‖ x k, u ‖ 2 2 , by triangle inequality. The projections here act as energy sources, and also the objective looks for an x k in the images of B k +1 and B /latticetop k , instead of ker( L k ) when x k -1 and x k +1 are not trivial. Thus, inter-simplicial couplings can potentially mitigate the oversmoothing as well.

Here we show simply generalzing GCN will inherit its oversmoothing to SCs. However, both the separation of the lower and upper Laplacians and inter-simplicial couplings could potentially mitigate this oversmoothing. We here considered a Dirichlet energy minimization perspective. They can also be explained by means of diffusion process on SCs [36]. We refer to Appendix B.4 for this.

4 From convolutional to Hodge-aware

In this section, we show how SCCNN, guided by the three principles, performs the Hodge-aware learning , allowing for rational and effective learning on SCs while remaining expressive. To ease the exposition, we first provide a more fine-grained spectral view on how SCCNN learns from simplicial data of different variations in the three subspaces based on the simplicial spectral theory [27, 26, 30]. Then, we characterize its expressive power and discuss its Hodge-awareness.

Definition 4 ([27]) . The simplicial Fourier transform (SFT) of x k is ˜ x k = U /latticetop k x k where the Fourier basis U k can be found as the eigenbasis of L k and the eigenvalues are simplicial frequencies .

Proposition 5 ([26]) . The SFT basis can be found as U k = [ U k, H U k, G U k, C ] where 1) the zero eigenspace U k, H of L k spans ker( L k ) , and an eigenvalue λ k, H = 0 is a harmonic frequency ; 2) the nonzero eigenspace U k, G of L k, d spans im( B /latticetop k ) , and an eigenvalue λ k, G is a gradient frequency , measuring the lower variation D d ( u k, G ) ; 3) the nonzero eigenspace U k, C of L k, u spans im( B k +1 ) , and an eigenvalue λ k, C is a curl frequency , measuring the upper variation D u ( u k, C ) .

Thus, the SFT of x k can be found as ˜ x k = [˜ x /latticetop k, H , ˜ x /latticetop k, G , ˜ x /latticetop k, C ] /latticetop , where each component is the intensity of x k at a simplicial frequency. Consider y k = H k, d x k, d + H k x k + H k, u x k, u in a SCCNN layer. Multiplying on both sides by U k , we then have the SFT ˜ y as

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

G

C

G,

G,

G,

G

C,

C,

C,

C

G,

=

G,

G,

G,

G

G,

G

G

G

(a)

0

2

4

6

0

2

4

6

0

1

2

0

2

4

6

(b)

0

10

-

3

10

-

2

dist edge

dist tri

0

10

-

3

10

-

2

10

-

1

10

0

/epsilon1

0

0

10

-

3

10

-

2

dist edge

dist tri

(c)

is the frequency response of H k as ˜ h k = diag( U /latticetop k H k U k ) [30], and ˜ h k, d and ˜ h k, u , the responses of H k, d and H k, u , can be expressed accordingly. This spectral relation (4) shows how the learning of SCCNN is performed at frequencies in different subspaces. Specifically, the gradient SFT ˜ x k, G is learned by a gradient response ˜ h k, G , which is independent of the curl response ˜ h k, C learning the curl SFT ˜ x k, C , and they only coincide at the trivial harmonic frequency, as shown in Fig. 1a. Likewise, the lower and upper projections are independently learned by ˜ h k, d and ˜ h k, u , respectively.

The nonlinearity induces the information spillage that one type of spectra could be spread over other types. That is, σ (˜ y k, G ) could contain information in harmonic or curl subspaces, as illustrated in Fig. 1b. This is to increase the expressive power of SCCNN, which can be characterized as follows. Theorem 6. A SCCNN layer with inputs x k, d , x k , x k, u is at most expressive as an MLP layer σ ( G ′ k, d x k, d + G k x k + G ′ k, u x k, u ) with G k = G k, d + G k, u where G k, d and G k, u are analytical matrix functions of L k, d and L k, u , respectively, and G ′ k, d and G ′ k, u likewise. Moreover, this expressivity can be achieved when setting T d = T ′ d = n k, G and T u = T ′ u = n k, C in Eq. (1) with n k, G the number of distinct gradient frequencies and n k, C the number of distinct curl frequencies.

The proof follows from Cayley-Hamilton theorem [37]. This expressive power can be better understood spectrally. The gradient SFT of x k can be learned most expressively by an analytical function g k, G ( λ ) , the eigenvalue of G k, d at a gradient frequency. And the curl SFT of x k can be learned most expressively by another analytical function g k, C ( λ ) , the eigenvalue of G k, u at a curl frequency. These two functions only need to coincide at harmonic frequency λ = 0 . The SFTs of lower and upper projections can be learned most expressively by two independent functions as well. Given this expressive power and Eq. (4), we show SCCNN performs the Hodge-aware learning as follows.

Theorem 7. ASCCNNis Hodge-aware in the sense that 1) three Hodge subspaces are invariant under the learnable SCF H k , i.e., H k x ∈ im( B /latticetop k ) if x ∈ im( B /latticetop k ) , and likewise for im( B k +1 ) , ker( L k ) ; 2) the gradient and curl spaces are invariant under the learnable lower SCF H k, d and upper SCF H k, u , respectively; 3) the learning in the gradient and curl spaces are independent and expressive .

This theorem essentially shows SCCNN performs expressive learning independently in the gradient and curl subspaces from three inputs while preserving the three subspaces to be invariant w.r.t its learning functions. This allows for the rational and effective learning on SCs. On one hand, the invariance of subspaces under the learnable SCFs substantially shrinks the learning space and makes SCCNN effective, meanwhile, its expressive power is guaranteed by the independent expressive learners, together with the nonlinearity. Instead, the non-Hodge-aware learning, e.g., MLP or simplicial message-passing using MLP to aggregate and update [17], has a much larger learning space which requires more training data for accurate learning, as well as larger computational complexity.

On the other hand, simplicial data often presents (implicit or explicit) properties that Hodge subspaces can capture. For example, water flows, traffic flows, electric currents [29, 11] follow flow conservation (div-free, in ker( B 1 ) ), or curl-free forex rates, as we show in Section 6, or the gradient component of pairwise comparison data gives consistent global ranking but others are unwanted [24]. SCCNN is able to capture these characteristics effectively, generating rational outputs due to the invariance of subspaces and independent learning in gradient and curl spaces. We illustrate a trivial example below.

Example 8 . Suppose learning to remove non-div-free noise from some input for flow conservation. 225 SCCNN can correctly do so because when a not-well-learned SCF, preserving the noise and useful 226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

parts primarily in their own spaces, causes large loss, e.g., mse, the Hodge-awareness restricts it to suppress in the gradient space and preserve in others. This however can be difficult non-Hodge-aware learners, e.g., MLP or MPSN [17], especially when the amount of data is limited, because the non-div-free noise can be disguised as useful by their unbiased transformation into other spaces, and the useful parts could be transformed into noise space, generating irrational non-div-free output though the overall mse can be small. Thus, simplicial data characteristics can be easily ignored by non-Hodge-aware learners when the invariance condition is not satisfied.

Comparison to others. Wehere discuss some other existing learning methods on SCs to emphasize on the Hodge-awareness. [15] considered H k = ∑ i w i L i k to perform convolutions without uncoupling the lower and upper parts of L k , which makes it strictly less expressive and non-Hodge-aware, because it cannot perform different learning at frequencies in both gradient and curl spaces, though deeper layers and higher orders can compensate its expressive at other frequencies. [16] applied H k with T d = T u = 1 , which has a limited linear learning response. SCCNN returns the methods in [19, 38] when there is no inter-simplicial coupling needed. [14, 20] took the form of simplicial shifting by generalizing the GCN without uncoupling the two adjacencies, which is not-Hodge-aware. Spectrally, this gives a limited lower-pass linear spectral response, shown in Fig. 1a.

5 How robust are SCCNNs to domain perturbations?

In practice, a SCCNN is often built on a weighted SC to capture the strengths of simplicial adjacencies and incidences, with a same form as Eq. (1), except for that the Hodge Laplacians and the projection matrices are weighted, denoted as general operators R k, d , R k, u . These matrices are often defined following [29, 39, 40], e.g., [14, 20] considered a particular random walk formulation [41], or can be learned from data, e.g., via an attention method [42, 38]. Since SCCNN relies on the Hodge Laplacians and projection matrices, in this section, we address the question, when these operators are perturbed, how accurate and robust are the outputs of a SCCNN? This models the domain perturbations on the strengths of adjacent and incident relations such as a large weight is applied when two edges are weakly or not adjacent, or data on a node projects on an edge not incident to it. By quantifying this stability, we can explain the robust learning ability of SCCNN. We consider a relative perturbation model, also used to study the stability of CNNs [43-45] and GNNs [46-49].

Denote the perturbed lower and upper Laplacians as ̂ L k, d and ̂ L k, u by perturbations E k, d and E k, u , and the lower and upper projections as ̂ R k, d and ̂ R k, u by perturbations J k, d and J k, u , respectively. Definition 9 (Relative perturbation) . Consider some perturbation matrix E of an appropriate dimension. For a symmetric matrix A , its (relative) perturbed version is ̂ A ( E ) = A + EA + AE . For a rectangular matrix B , its (relative) perturbed version is B ( E ) = B + EB .

̂ This relative perturbation model, in contrast to an absolute one [47], quantifies perturbations w.r.t. the local simplicial topology in the sense that weaker connections in a SC are deviated by perturbations proportionally less than stronger connections. We further consider the integral Lipschitz property, extended from [47], to measure the variability of spectral response functions of H k .

Definition 10. A SCF H k is integral Lipschitz with constants c k, d , c k, u ≥ 0 if the derivatives of response functions ˜ h k, G ( λ ) and ˜ h k, C ( λ ) follow that | λ ˜ h ′ k, G ( λ ) | ≤ c k, d and | λ ˜ h ′ k, C ( λ ) | ≤ c k, u .

This property provides a stability-selectivity tradeoff of SCFs independently in gradient and curl frequencies. A spectral response can have both good selectivity and stability in small frequencies (a large | ˜ h ′ k, · | for λ → 0 ), while in large frequencies, it tends to be flat for better stability at the cost of selectivity (a small variability for large λ ), as shown in Fig. 1a. As of the polynomial nature of responses, all SCFs of a SCCNN are integral Lipschitz. We also denote the integral Lipschitz constant for the lower SCFs H k, d by c k, d and for the upper SCFs H k, u by c k, u . Given the following reasonable assumptions, we are ready to characterize the stability bound of a SCCNN.

Assumption 11. a) The perturbations are small such that ‖ E k, d ‖ 2 ≤ /epsilon1 k, d , ‖ J k, d ‖ 2 ≤ ε k, d , ‖ E k, u ‖ 2 ≤ /epsilon1 k, u and ‖ J k, u ‖ 2 ≤ ε k, u . b) The SCFs H k of a SCCNN have a normalized bounded frequency response (for simplicity, though unnecessary), likewise for H k, d and H k, u . c) The lower and upper projections are finite ‖ R k, d ‖ 2 ≤ r k, d and ‖ R k, u ‖ 2 ≤ r k, u . d) The nonlinearity σ ( · ) is c σ -Lipschitz (e.g., relu , tanh , sigmoid ). e) The initial input x 0 k , for all k , is finite, ‖ x 0 k ‖ 2 ≤ [ β ] k , .

Theorem 12. Let x L k be the k -simplicial output of an L -layer SCCNN on a weighted SC. Let ˆ x L k be the output of the same SCCNN but on a relatively perturbed SC. Under Assumption 11, the Euclidean

distance between the two outputs is finite and upper-bounded ‖ ˆ x L k -x L k ‖ 2 ≤ [ d ] k where 280

281

282

283

284

285

for K = 2 , which are tridiagonal, and ̂ Z is defined as Z but with off-diagonal entries ˆ r k, d = r k, d (1+ ε k, d ) and ˆ r k, u = r k, u (1+ ε k, u ) . Diagonal entries of T are t k = c k, d ∆ k, d /epsilon1 k, d + c k, u ∆ k, u /epsilon1 k, u , and off-diagonals are t k, d = r k, d ε k, d + c k, d ∆ k, d /epsilon1 k, d r k, d and t k, u = r k, u ε k, u + c k, u ∆ k, u /epsilon1 k, u r k, u , where ∆ k, d captures the eigenvector misalignment between L k, d and perturbation E k, d with a factor √ n k , and likewise for ∆ k, u .

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

This result bounds the outputs of a SCCNN on all simplicial levels, showing they are stable to small perturbations on the strengths of simplicial adjacencies and incidences. Specifically, we make two observations from the seemingly complicated expression. 1) The stability bound depends on i) the degree of perturbations including their magnitude /epsilon1 and ε , and eigenspace misalignment ∆ , ii) the integral Lipschitz properties of SCFs, and iii) the degree of projections r . 2) The stability of k -output depends on factors of not only k -simplices, but also simplices of adjacent orders due to inter-simplicial couplings. When L = 1 , node output bound d 0 depends on factors in the node space, as well as the edge space factored by the projection degree, and vice versa for edge output. As the layer deepens, this mutual dependence expands further. When L = 2 , factors in the triangle space also affect the stability of node output d 0 , and vice versa for triangle output, as observed in Fig. 1c.

More importantly, this stability provides practical implications for learning on SCs. While accounting for inter-simplicial couplings may be beneficial, it does not help with the stability of SCCNNs when the number of layers increases due to the mutual dependence between different outputs. Thus, to maintain the expressive power, higher-order SCFs can be used in exchange for shallow layers. This does not harm the stability because, first, the components of high-frequency can be spread over the low frequency due to the nonlinearity where the spectral responses are more selective without losing the stability; and second, higher-order SCFs are easier to be learned with smaller integral Lipschitz constants than lower-order ones, thus, better stability. The latter can be easily seen by comparing one-order and two-order cases. We also experimentally show this in Fig. 4.

6 Experiments

Synthetic. We first illustrate the evolution of Dirichlet energies of outputs on nodes, edges and triangles of a SC of order two by numbers of simplicial shifting layers with σ = tanh . The inputs on them are randomly sampled from U ([ -5 , 5]) . Fig. 2 shows simply generalizing GCN on SCs could lead to oversmoothing on simplices of all orders. However, uncoupling the lower and upper parts of L 1 by setting, e.g., γ = 2 could mitigate the oversmoothing on edges. Lastly, the inter-simplicial coupling could almost prevent the oversmoothing.

1

10

100

Layer

10

-

23

10

-

19

10

-

15

10

-

11

10

-

7

10

-

3

10

1

D

(

x

k

)

shift,node

with proj.,node

shift,edge

with proj.,edge

uncouple lower/upper,edge

shift,tri.

with proj.,tri.

Foreign currency exchange. In forex problems, for any currencies i, j, k , the arbitray-free condition of a fair market reads as r i/j r j/k = r i/k with the exchange rate r i/j between i and j . That is, the exchange path i → j → k provides no profit or loss over a direct exchange i → k . By modeling the forex as a SC of order two and the exchange rates as edge flows [ x 1 ] [ i,j ] = log( r i/j ) , this condition translates as x 1 is curl-free, i.e., [ x 1 ] [ i,j ] +[ x 1 ] [ j,k ] -[ x 1 ] [ i,k ] = 0 in any triangle [ i, j, k ] [24]. Here we consider a real-world forex market from [50] at three timestamps, which contains certain degree of arbitrage. We artificially added some random noise and 'curl noise' (only in the curl space) to this market, in which we aim to recover the forex rates. We also randomly masked 50 % of the rates, where we aim to interpolate the market such that it is arbitrage-free. Three settings create three types of learning needs. To evaluate the performance, we measure both normalized mse and total arbitrage (total curl), both equally important for the goal of creating a fair market by small price fluctuations .

From Table 1, we make the following observations. 1) MPSN [17] fails at this task: although it can reduce nmse, it outputs unfair rates with large arbitrage, which is against the forex principle, because it is not Hodge-aware, unable to capture the arbitrage-free property with small amount of data. 2) SNN [15] fails too: as discussed in Section 4, it restricts the gradient and curl spaces to be always learned in the same fashion, unable to meet the need of disjoint learning of this task in two

Methods

Curl Noise

Interpolation

Input

.

119

±

.

004

|

29

.

19

±

.

874

.

552

±

.

027

|

122

.

4

±

5

.

90

.

717

±

.

030

|

106

.

4

±

.

902

Baseline

.

036

±

.

005

|

2

.

29

±

.

079

.

050

±

.

002

|

11

.

12

±

.

537

.

534

±

.

043

|

9

.

67

±

.

082

SNN [15]

.

110

±

.

005

|

23

.

24

±

1

.

03

.

446

±

.

017

|

86

.

95

±

2

.

20

.

702

±

.

033

|

104

.

74

±

1

.

04

PSNN [16]

.

008

±

.

001

|

.

984

±

.

170

.

000

±

.

000

|

.

000

±

.

000

.

009

±

.

001

|

1

.

13

±

.

329

MPSN [17]

.

039

±

.

004

|

7

.

74

±

0

.

88

.

076

±

.

012

|

14

.

92

±

2

.

49

.

117

±

.

063

|

23

.

15

±

11

.

7

SCCNN, id

.

027

±

.

005

|

.

000

±

.

000

.

000

±

.

000

|

.

000

±

.

000

.

265

±

.

036

|

.

000

±

.

000

SCCNN, tanh

.

002

±

.

000

|

.

325

±

.

082

.

000

±

.

000

|

.

003

±

.

003

.

003

±

.

002

|

.

279

±

.

151

Methods

2-simplex

3-simplex

Mean [2]

62.8

±

2.7

63.6

±

1.6

MLP

68.5

±

1.6

69.0

±

2.2

GNN [51]

93.9

±

1.0

96.6

±

0.5

SNN [15]

92.0

±

1.8

95.1

±

1.2

PSNN [16]

95.6

±

1.3

98.1

±

0.5

SCNN [19]

96.5

±

1.5

98.3

±

0.4

Bunch [14]

98.3

±

0.5

98.5

±

0.5

MPSN [17]

98.1

±

0.5

99.2

±

0.3

SCCNN

98.7

±

0.5

99.4

±

0.3

Missing

2-Simplex

Param.

-

98.7

±

0.5

L

= 2

Edge-to-Node

93.9

±

1.0

L

= 5

98.7

±

0.4

L

= 4

Edge-to-Edge

98.5

±

1.0

L

= 3

98.8

±

0.3

L

= 4

Node input

98.2

±

0.5

T

= 4

Edge input

98.1

±

0.4

T

= 3

0

10

-

3

10

-

2

10

-

1

10

0

/epsilon1

0

10

-

3

10

-

2

10

-

1

10

0

10

1

10

2

‖

x

L

k

-

ˆ

x

L

k

‖

Node

Edge

Tri.

Node thm.

Edge thm.

Tri. thm.

0

.

4

0

.

6

accuracy

T

= 1

T

= 3

T

= 5

0

.

00

0

.

25

0

.

50

0

.

75

1

.

00

0

1

2

‖

x

1

-

ˆ

x

1

‖

/

‖

x

1

‖

T

= 1

T

= 3

T

= 5

331

332

333

334

335

336

337

spaces. 3) PSNN [16] can reconstruct relatively fair forex rates with small nmse. In the curl noise case, the reconstruction is perfect, while in the other two cases, the nmse and arbitrage are three times larger than SCCNN due to its limited linear learning responses. 4) SCCNN performs the best in both reducing the total error and the total arbitrage. We also notice that with σ = id , the arbitrage-free rule is fully learned by SCCNN. However, it has relatively larger errors in the random and interpolation cases due to its limited linear expressive power. With σ = tanh , SCCNN can tackle these more challenging cases, finding a good compromise between overall error and data characteristics.

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

Simplex Prediction. We then test SCCNN on simplex prediction task which is an extension of link prediction in graphs [52]. Our approach is to first learn the features of lower-order simplices and then use an MLP to identify if a simplex is closed or open. We built a SC as [15] on a coauthorship dataset [53] where nodes are authors and collaborations of k -authors are ( k -1) -simplices. The input simplicial data is the number of citations, e.g., x 1 and x 2 are those of dyadic and triadic collaborations, which does not present explicit properties like forex rates. Thus, 2-simplex (3-simplex) prediction amounts to predict triadic (tetradic) collaborations. From the AUC results in Table 2, we make three observations. 1) SCCNN, MPSN and Bunch [14] methods outperform the rest due to the inter-simplicial couplings. 2) Uncoupling the lower and upper parts in L k imrpoves the feature learning (SCNN [19] better than SNN). 3) Higher-order convolution further improves the prediction (SCNN better than PSNN, SCCNN better than Bunch). Note that MPSN has three times more parameters than SCCNN under the settings of the best results.

Ablation study. Table 3 reports the results of SCCNN when certain simplicial relation is missing, which helps understand their roles. When not considering the edge-to-node incidence, it (when using node features) is equivalent to GNN with poor performance. When removing other adjacencies or incidences, the best performance remains but with an increase of model complexity, more layers required. This, however, is not preferred, because the stability decreases as the model deepens and becomes influenced by factors in other simplicial space, as shown in Fig. 1c. We also considered the case with limited input, e.g., when the input on nodes or on edges is missing. The best performance of SCCNN only slightly drops with an increase of convolution order, compared to before T = 2 .

How tight is the stability bound? We consider the perturbations which relatively shift the eigenvalues of Hodge Laplacians and the singular values of projection matrices by /epsilon1 . We compare the bound in Eq. (5) with experimental distance on each simplex level. Fig. 3 shows the bound becomes tighter as perturbation increases.

Trajectory prediction. We lastly test on predicting trajectories in a synthetic SC and of ocean drifters from [41], introduced by [16].

From Table 4 we first observe SCCNN and Bunch with inter-simplicial couplings do not perform 365 better than those without. This is because zero inputs are applied on nodes and triangles [16], which 366 makes inter-couplings inconsequential. Secondly, using higher-order convolutions improves the 367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

average accuracy in both datasets (SCNN better than PSNN on average, SCCNN better than Bunch). Note that the prediction here aims to find a candidate from the neighborhood of end node, which depends on the node degree. Since the average node degree of the synthetic SC is 5 . 24 and that in ocean drifter data is 4 . 81 , a random guess has around 20% accuracy. The high standard derivations could come from the limited ocean drifter dataset.

Convolution orders on stability. We also show that NNs with higher-order SCFs have more potential to learn better integral Lipschitz properties, thus, better stability. We consider SCNNs [19] with orders T d = T u = 1 , 3 , 5 and train them with a regularizer to reduce the integral Lipschitz constants. As shown in Fig. 4, the higher-order case has a smaller distance (better stability) between the outputs without and with perturbations, with consistent better accuracy, comapred to the lower-order case.

7 Related Work, Discussion and Conclusion

Related work mainly concerns learning methods on SCs. [13] first used L 1 , d to build NNs on edges in a graph setting without the upper edge adjacency. [15] then generalized convolutional GNNs [32, 51] to simplices by using the Hodge Laplacian. [16, 19] instead uncoupled the lower and upper Laplacians to perform one- and multi-order convolutions, to which [42, 38, 54] added attention schemes. [55] considered a varaint of [16] to identity topological holes and [18] combined shifting on nodes and edges for link prediction. Above works learned within a simplicial level and did not consider the incidence relations (inter-simplicial couplings) in SCs, which was included by [14, 20]. These works considered convolutional-type methods, which can be subsumed by SCCNN. Meanwhile, [17, 21] generalized the message passing on graphs [22] to SCs, relying on both adjacencies and incidences. Most of these works focused on extending GNNs to SCs by varying the information propagation on SCs without many theoretical insights into their components. Among them, [16] discussed the equivariance of PSNN to permutation and orientation, which SCCNN admits as well. [17] studied the messgae-passing on SCs in terms of WL test of SCs built by completing cliques in a graph. The more closely related work [19] gave only a spectral formulation based on SCFs.

Discussion. In our opinion, the advantage of using SCs is not only about them being able to model higher-order network structure, but also support simplicial data, which can be both human-generated data like coauthorship, and physical data like flow-typed data. This is why we approcahed the analysis from the perspectives of both simplicial structures and the simplicial data, i.e., the Hodge theory and spectral simplicial theory [23, 9, 26-28, 30, 56]. We provided deeper insights into why three principles are needed and how they can guide the effective and rational learning from simplicial data. As what we practically found, in experiments where data exhibits properties characterized by the Hodge decomposition, SCCNN performs well due to the Hodge-awareness while non-Hodge-aware learners can fail at giving rational results. In cases where data does not possess such properties, SCCNN has better or comparable performance than the ones which violate or do not respect the three principles. This also shows the advantages of SCCNN, especially when data has certain properties.

Concurrently, there are works on more general cell complexes, e.g., [57-61], where 2-cells inlcude not only triangles, but also general polygon faces. We focus on SCs because a regular CW complex can be subdivided into a SC [62, 29] to which the analysis in this paper applies, or we can generalize our analysis by allowing B 2 to include 2-cells. This is however informal and does not exploit the power of cell complexes, which lies on cellular sheaves, as studied in [63, 64].

Limitation. A major limitation of our method is that it cannot learn differently from features at the frequencies of the same type and the same value. For instance, harmonic features are learned in a same fashion because they all have zero frequency. This is however common in convolutional type learning methods on both graphs and SCs. Also, our stability analysis concerns the perturbations on the connection strengths and did not consider the case where simplices join or disappear. Both of them can be interesting future directions, together with more physical-based data applications.

Conclusion. We proposed three principles for convolutional learning on SCs, summarized in a general architecture, SCCNN. Our analysis showed this architecture, guided by the three principles, demonstrates an awareness of the Hodge decomposition and performs rational, effective and expressive learning from simplicial data. Furthermore, our study reveals that SCCNN exhibits stability and robustness against perturbations in the strengths of simplicial connections. Experimental results validate the benefits of respecting the three principles and the Hodge-awareness. Overall, our work establishes a solid fundation for learning on SCs, highlighting the importance of the Hodge theory.

422

423

424

425

426

427

428

429

430

431

432

433

434

435

436

437

References

Mark EJ Newman, Duncan J Watts, and Steven H Strogatz. Random graph models of social networks. Proceedings of the national academy of sciences , 99(suppl_1):2566-2572, 2002.

Austin R Benson, Rediet Abebe, Michael T Schaub, Ali Jadbabaie, and Jon Kleinberg. Simplicial closure and higher-order link prediction. Proceedings of the National Academy of Sciences , 115(48): E11221-E11230, 2018.

Hosein Masoomy, Behrouz Askari, Samin Tajik, Abbas K Rizi, and G Reza Jafari. Topological analysis of interaction patterns in cancer-specific gene regulatory network: persistent homology approach. Scientific Reports , 11(1):1-11, 2021.

Federico Battiston, Giulia Cencetti, Iacopo Iacopini, Vito Latora, Maxime Lucas, Alice Patania, JeanGabriel Young, and Giovanni Petri. Networks beyond pairwise interactions: structure and dynamics. Physics Reports , 874:1-92, 2020.

Austin R Benson, David F Gleich, and Desmond J Higham. Higher-order network analysis takes off, fueled by classical ideas and new data. arXiv preprint arXiv:2103.05031 , 2021.

Leo Torres, Ann S Blevins, Danielle Bassett, and Tina Eliassi-Rad. The why, how, and when of representations for complex systems. SIAM Review , 63(3):435-485, 2021.

Christian Bick, Elizabeth Gross, Heather A Harrington, and Michael T Schaub. What are higher-order 438 networks? arXiv preprint arXiv:2104.11329 , 2021. 439

James R Munkres. Elements of algebraic topology . CRC press, 2018. 440

Lek-Heng Lim. Hodge laplacians on graphs. SIAM Review , 62(3):685-715, 2020. 441

Rohan Money, Joshin Krishnan, Baltasar Beferull-Lozano, and Elvin Isufi. Online edge flow imputation 442 on networks. IEEE Signal Processing Letters , 2022. 443

Junteng Jia, Michael T Schaub, Santiago Segarra, and Austin R Benson. Graph-based semi-supervised & 444 active learning for edge flows. In Proceedings of the 25th ACM SIGKDD International Conference on 445 Knowledge Discovery & Data Mining , pages 761-771, 2019. 446

D Vijay Anand, Soumya Das, and Moo K Chung. Hodge-decomposition of brain networks. arXiv preprint 447 arXiv:2211.10542 , 2022. 448

T Mitchell Roddenberry and Santiago Segarra. Hodgenet: Graph neural networks for edge data. In 2019 449 53rd Asilomar Conference on Signals, Systems, and Computers , pages 220-224. IEEE, 2019. 450

Eric Bunch, Qian You, Glenn Fung, and Vikas Singh. Simplicial 2-complex convolutional neural networks. 451 In TDA & Beyond , 2020. URL https://openreview.net/forum?id=TLbnsKrt6J-. 452

Stefania Ebli, Michaël Defferrard, and Gard Spreemann. Simplicial neural networks. In NeurIPS 2020 453 Workshop on Topological Data Analysis and Beyond , 2020. 454

T Mitchell Roddenberry, Nicholas Glaze, and Santiago Segarra. Principled simplicial neural networks for 455 trajectory prediction. In International Conference on Machine Learning , pages 9020-9029. PMLR, 2021. 456

457

458

459

Cristian Bodnar, Fabrizio Frasca, Yuguang Wang, Nina Otter, Guido F Montufar, Pietro Lio, and Michael Bronstein. Weisfeiler and lehman go topological: Message passing simplicial networks. In International Conference on Machine Learning , pages 1026-1037. PMLR, 2021.

460

461

462

463

464

465

466

467

468

Yuzhou Chen, Yulia R. Gel, and H. Vincent Poor. Bscnets: Block simplicial complex neural networks. Proceedings of the AAAI Conference on Artificial Intelligence , 36(6):6333-6341, 2022. doi: 10.1609/aaai. v36i6.20583. URL https://ojs.aaai.org/index.php/AAAI/article/view/20583 .

Maosheng Yang, Elvin Isufi, and Geert Leus. Simplicial convolutional neural networks. In ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 8847-8851, 2022. doi: 10.1109/ICASSP43922.2022.9746017.

Ruochen Yang, Frederic Sala, and Paul Bogdan. Efficient representation learning for higher-order data with simplicial complexes. In The First Learning on Graphs Conference , 2022. URL https://openreview. net/forum?id=nGqJY4DODN .

Mustafa Hajij, Ghada Zamzmi, Theodore Papamarkou, Vasileios Maroulas, and Xuanting Cai. Simplicial 469 complex representation learning. arXiv preprint arXiv:2103.04046 , 2021. 470

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? 471 arXiv preprint arXiv:1810.00826 , 2018. 472

William Vallance Douglas Hodge. The theory and applications of harmonic integrals . CUP Archive, 1989. 473

Xiaoye Jiang, Lek-Heng Lim, Yuan Yao, and Yinyu Ye. Statistical ranking and combinatorial hodge theory. 474 Mathematical Programming , 127(1):203-244, 2011. 475

476

477

Ozan Candogan, Ishai Menache, Asuman Ozdaglar, and Pablo A Parrilo. Flows and decompositions of games: Harmonic and potential games. Mathematics of Operations Research , 36(3):474-503, 2011.

478

479

480

Maosheng Yang, Elvin Isufi, Michael T. Schaub, and Geert Leus. Finite Impulse Response Filters for Simplicial Complexes. In 2021 29th European Signal Processing Conference (EUSIPCO) , pages 2005-2009, August 2021. doi: 10.23919/EUSIPCO54536.2021.9616185. ISSN: 2076-1465.

Sergio Barbarossa and Stefania Sardellitti. Topological signal processing over simplicial complexes. IEEE 481 Transactions on Signal Processing , 68:2992-3007, 2020. 482

John Steenbergen. Towards a spectral theory for simplicial complexes . PhD thesis, Duke University, 2013. 483

Leo J Grady and Jonathan R Polimeni. Discrete calculus: Applied analysis on graphs for computational 484 science , volume 3. Springer, 2010. 485

Maosheng Yang, Elvin Isufi, Michael T. Schaub, and Geert Leus. Simplicial convolutional filters. IEEE 486 Transactions on Signal Processing , 70:4633-4648, 2022. doi: 10.1109/TSP.2022.3207045. 487

Michael T Schaub, Yu Zhu, Jean-Baptiste Seby, T Mitchell Roddenberry, and Santiago Segarra. Signal 488 processing on higher-order networks: Livin'on the edge... and beyond. Signal Processing , 187:108149, 489 2021. 490

Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In 491 International Conference on Learning Representations (ICLR) , 2017. 492

Chen Cai and Yusu Wang. A note on over-smoothing for graph neural networks. arXiv preprint 493 arXiv:2006.13318 , 2020. 494

T Konstantin Rusch, Michael M Bronstein, and Siddhartha Mishra. A survey on oversmoothing in graph 495 neural networks. arXiv preprint arXiv:2303.10993 , 2023. 496

497

498

Hoang Nt and Takanori Maehara. Revisiting graph neural networks: All we have is low-pass filters. arXiv preprint arXiv:1905.09550 , 2019.

Cameron Ziegler, Per Sebastian Skardal, Haimonti Dutta, and Dane Taylor. Balanced hodge laplacians 499 optimize consensus dynamics over simplicial complexes. Chaos: An Interdisciplinary Journal of Nonlinear 500 Science , 32(2):023128, 2022. 501

Roger A Horn and Charles R Johnson. Matrix analysis . Cambridge university press, 2012. 502

Lorenzo Giusti, Claudio Battiloro, Paolo Di Lorenzo, Stefania Sardellitti, and Sergio Barbarossa. Simplicial 503 attention networks. arXiv preprint arXiv:2203.07485 , 2022. 504

Danijela Horak and Jürgen Jost. Spectra of combinatorial laplace operators on simplicial complexes. 505 Advances in Mathematics , 244:303-336, 2013. 506

Nicola Guglielmi, Anton Savostianov, and Francesco Tudisco. Quantifying the structural stability of 507 simplicial homology. arXiv preprint arXiv:2301.03627 , 2023. 508

Michael T Schaub, Austin R Benson, Paul Horn, Gabor Lippner, and Ali Jadbabaie. Random walks on 509 simplicial complexes and the normalized hodge 1-laplacian. SIAM Review , 62(2):353-391, 2020. 510

511

512

Christopher Wei Jin Goh, Cristian Bodnar, and Pietro Lio. Simplicial attention networks. In ICLR 2022 Workshop on Geometrical and Topological Representation Learning , 2022.

513

514

Joan Bruna and Stéphane Mallat. Invariant scattering convolution networks. IEEE transactions on pattern analysis and machine intelligence , 35(8):1872-1886, 2013.

Qiang Qiu, Xiuyuan Cheng, Guillermo Sapiro, et al. Dcfnet: Deep neural network with decomposed 515 convolutional filters. In International Conference on Machine Learning , pages 4198-4207. PMLR, 2018. 516

Alberto Bietti and Julien Mairal. Group invariance, stability to deformations, and complexity of deep 517 convolutional representations. arXiv preprint arXiv:1706.03078 , 2017. 518

Fernando Gama, Alejandro Ribeiro, and Joan Bruna. Stability of graph scattering transforms. Advances in 519 Neural Information Processing Systems , 32, 2019. 520

521

522

Fernando Gama, Joan Bruna, and Alejandro Ribeiro. Stability properties of graph neural networks. IEEE Transactions on Signal Processing , 68:5680-5695, 2020.

523

524

525

526

527

528

529

530

531

532

533

534

535

536

537

538

539

540

541

542

543

544

545

546

547

548

549

550

551

552

553

554

555

556

557

558

559

560

561

562

563

564

565

Henry Kenlay, Dorina Thano, and Xiaowen Dong. On the stability of graph convolutional neural networks under edge rewiring. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 8513-8517. IEEE, 2021.

Alejandro Parada-Mayorga, Zhiyang Wang, Fernando Gama, and Alejandro Ribeiro. Stability of aggregation graph neural networks. arXiv preprint arXiv:2207.03678 , 2022.

Oanda Corporation. Foreign exchange data. https://www.oanda.com/ , 2018/10/05.

Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 29.

Curran Associates, Inc., 2016. URL https://proceedings.neurips.cc/paper/2016/file/ 04df4d434d481c5bb723be1b6df1ee65-Paper.pdf .

Muhan Zhang and Yixin Chen. Link prediction based on graph neural networks. Advances in neural information processing systems , 31, 2018.

Waleed Ammar, Dirk Groeneveld, Chandra Bhagavatula, Iz Beltagy, Miles Crawford, Doug Downey, Jason Dunkelberger, Ahmed Elgohary, Sergey Feldman, Vu Ha, et al. Construction of the literature graph in semantic scholar. arXiv preprint arXiv:1805.02262 , 2018.

See Hian Lee, Feng Ji, and Wee Peng Tay. Sgat: Simplicial graph attention network. arXiv preprint arXiv:2207.11761 , 2022.

Alexandros D Keros, Vidit Nanda, and Kartic Subr. Dist2cycle: A simplicial neural network for homology localization. Proceedings of the AAAI Conference on Artificial Intelligence , 36(7):7133-7142, 2022. doi: 10.1609/aaai.v36i7.20673. URL https://ojs.aaai.org/index.php/AAAI/article/view/20673 .

Kiya W Govek, Venkata S Yamajala, and Pablo G Camara. Spectral simplicial theory for feature selection and applications to genomics. arXiv preprint arXiv:1811.03377 , 2018.

Mustafa Hajij, Kyle Istvan, and Ghada Zamzmi. Cell complex neural networks. In NeurIPS 2020 Workshop on Topological Data Analysis and Beyond , 2020.

Mustafa Hajij, Ghada Zamzmi, Theodore Papamarkou, Nina Miolane, Aldo Guzmán-Sáenz, and Karthikeyan Natesan Ramamurthy. Higher-order attention networks. arXiv preprint arXiv:2206.00606 , 2022.

Stefania Sardellitti, Sergio Barbarossa, and Lucia Testa. Topological signal processing over cell complexes. In 2021 55th Asilomar Conference on Signals, Systems, and Computers , pages 1558-1562. IEEE, 2021.

T Mitchell Roddenberry, Michael T Schaub, and Mustafa Hajij. Signal processing on cell complexes. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 8852-8856. IEEE, 2022.

Cristian Bodnar, Fabrizio Frasca, Nina Otter, Yu Guang Wang, Pietro Liò, Guido F Montufar, and Michael Bronstein. Weisfeiler and lehman go cellular: Cw networks. Advances in Neural Information Processing Systems , 34, 2021.

Albert T Lundell, Stephen Weingram, Albert T Lundell, and Stephen Weingram. Regular and semisimplicial cw complexes. The Topology of CW Complexes , pages 77-115, 1969.

Jakob Hansen and Robert Ghrist. Toward a spectral theory of cellular sheaves. Journal of Applied and Computational Topology , 3(4):315-358, 2019.

Cristian Bodnar, Francesco Di Giovanni, Benjamin Chamberlain, Pietro Liò, and Michael Bronstein. Neural sheaf diffusion: A topological perspective on heterophily and oversmoothing in gnns. Advances in Neural Information Processing Systems , 35:18527-18541, 2022.

Elvin Isufi and Maosheng Yang. Convolutional filtering in simplicial complexes. In ICASSP 2022 - 2022 566 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 5578-5582, 567 2022. doi: 10.1109/ICASSP43922.2022.9746349. 568

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors first use Dirichlet energy minimizations on simplicial complexes (SCs) to interpret their effects on mitigating the simplicial oversmoothing. Then, through the lens of spectral simplicial theory, they show the three principles promote the Hodge-aware learning of this architecture, in the sense that the three Hodge subspaces are invariant under its learnable functions and the learning in two nontrivial subspaces are independent and expressive. Moreover,  the authors prove it is stable against small perturbations on the strengths of simplicial connections, and show how three principles can affect the stability. Lastly, we validate our findings on different simplicial tasks, including recovering foreign currency exchange (forex) rates, predicting triadic and tetradic collaborations, and trajectories.\n\nSTRENGTHS:\n1. The paper is well-written. The introduction and background give a nice overview and motivation for the problem.\n2. The problem of simplicial complexes learning is really interesting and in many aspects understudied.\n3. In the simplex prediction, the proposed SCCNN achieves promising empirical performance. \n4. The proposed SCCNN is supported by a theoretical analysis.\n\nWEAKNESSES:\n1. Limited applications and examples. It would be interesting to see more applications of the proposed SCCNN over widely used datasets (e.g., citation networks - Cora, CiteSeer, PubMed) for node and graph classification tasks. Moreover, although SCCNN achieves promising performance compared with SC-based model, can the authors compare it with other the state-of-the-art graph neural network (GNN)-based models?\n2. Can the authors provide the running time of SCCNN and compare it with the state-of-the-art baselines?\n3. How to select the dimension of $k$-simplex in the SCCNN model?\n\nQUESTIONS:\nSee comments and questions in Weaknesses.\n\nLIMITATIONS:\nIn general, I think this is a good paper with tackling a well motivated task. It would be helpful that this paper explores more datasets and compares with more advanced graph neural network-based models.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors identified the limitations of non-Hodge aware learners on simplicial complex (SC) data and proposed a convolutional structure that 1) decomposes the upper and lower k-Laplacian and 2) takes the inter-simplicial couplings into account. The paper has presented a justification for the performance by analyzing the Dirichlet energy and oversmoothing. Additionally, they provided theoretical perturbations bound to study the robustness of the proposed convolution layer. The claims are supported by experiments on synthetic and real SC datasets.\n\nSTRENGTHS:\n1. [Originality] Incorporate the well-known Hodge theorem into the learning task on simplicial complex. The Hodge theorem provides a good intuition and explanation for the learning of the simplicial signal on SC. \n1. [Quality] Empirical examples on the synthetic datasets on Dirichlet energy and stability bound to support the theoretical claims. \n1. [Clarity] Great overview of the simplicial complex and Hodge decomposition. The authors also provided a motivation/justification for why the proposed layer works with Dirichlet energy minimization. \n1. [Significance] Being able to learn the simplicial signal in different Hodge subspaces is an important task.\n\nWEAKNESSES:\n1. If this framework needs to be applied to graph only having edges (i.e., SC of order 1), one usually can apply something like clique-complex (or any other methods to fill in the Simplicios) from that graph. In this case, $n_k$ is generally large (worst case $n_k = \\mathcal O(n^k)$), resulting in a huge $L_k$ matrix. How practical is it to use the proposed method under this scenario?\n1. Can you provide a definition/discussion or citation of Hodge-aware? It is not clear to me where it is defined throughout the manuscript. I can get some high-level ideas by reading Theorem 7, but I think it would be nice if you could explicitly call it out at the beginning (e.g., in introduction or background).\n1. The discussion for preventing “over-smoothing” in Section 3 is great, it provides some high-level motivations of the choices you made. However, I am not sure if that it can support the claims.  Specifically, to really prevent “over-smoothing” of the Dirichlet energy, shouldn’t we bound the $D(x_k^{\\ell+1})$ in other way around, i.e., with a lower bound rather than an upper bound? If we can show that $D(x_k^{\\ell+1})$ can be lower bounded, the claim can be more convincing. \n1. Consider adding some high-level intuition on what harmonic flow is using the edge space example (e.g., flow cycling around global topoplogical structures); this will give readers having no background in Hodge decomposition a better understanding of what a “harmonic flow” is.\n1. [Typo] L186 there is typo/grammatical issue, do you mean “$\\tilde{h}_k = \\text{diag}(...)$ is the frequency response of $\\mathbb H_k$”?\n\nQUESTIONS:\n1. Related to Weakness #2, why the Hodge Laplacian smoothing [31] not Hodge-aware? I think it can also learn from the different subspaces of the Hodge Laplacian, just not independently. Maybe adding some definition/citation as per #2 will clarify it a bit. \n1. [Minor language usage suggestion] Consider rewrite L24-L25 to improve clarity; for instance, you might rewrite it as something like “A SC can be informally viewed as an extension of a graph. For example, one of the simplest SC (SC_2) can be constructed from a graph by inducing some triangles over the edge set.”.\n1. [Minor language usage suggestion] There is an extra e.g., in L27\n1. [Minor language usage suggestion] Consider breaking L27-29 into multiple sentences to improve clarity.\n1. [Minor notation issue] I would consider changing the notation of the $\\mathbf B$ matrix in L259 to reduce confusion.\n\nLIMITATIONS:\nThe paper discusses some of its limitations and requirements/assumptions. No significant social impact is identified from this work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to propose a convolutional architecture which incorporates the Hodge theory. Specifically, the proposed architecture incorporates the following three properties: uncoupling the lower and upper simplicial adjacencies, accounting for the inter-simplicial couplings, and performing higher-order convolutions.\n\nSTRENGTHS:\nThe paper claims a new architecture that incorporates the following three properties: uncoupling the lower and upper simplicial adjacencies, accounting for the inter-simplicial couplings, and performing higher-order convolutions, but the clear differences with respect to existing simplicial neural models are unclear.\n\nWEAKNESSES:\n- Errors: The paper is very poorly written. The typographic, punctuation, and grammatical errors throughout the paper make it hard to follow. Lines 21, 25, 31, 32, and 37 (in the first page of the paper) are some examples of lines that containing such errors. Many abbreviations are used before defining them. For example, NN (line 30), SCCNN (line 51), MLP (line 65), SCF (line 264) etc., were never defined. Many variables, like \\mathcal{V} in line 70 and most of the variables in eq. (4), and terms, like alternating map in line 87, were never introduced.\n- Contributions: The contributions of the paper are unclear. In line 104, for example, the authors say, “we inherit the names of three edge subspaces to general k-simplices”, while the Hodge theory is already in place for simplices of all dimensions. The main contribution of the paper, which is supposed to be an architecture that incorporates the Hodge theory, is also not clearly presented.\n\nQUESTIONS:\n- It is not clear if Section 3 is about the proposed model or an explanation of the existing models.  The expression in eq. (1) is a general expression for the existing SNN models, which is also what the authors say in line 119. Given this, it is not clear what the contribution of the paper is.  Even the three properties in lines 119-125 that the paper claims to be present in the proposed model, I believe, are possessed also by MPSN. In the section named “From convolutional to Hodge-aware”, where the authors compare the proposed model to the existing models, the comparison with MPSN is missing. Furthermore, the motivation for defining Dirichlet energy the way it is defined in Definition 2 is not clear. Should it not be || (B_k + B_{k+1}^T)x_k ||^2 if it was a direct extension from graphs? If it was defined as in Definition 2 in some work earlier, the reference to the work should be provided. \n\n- The authors claim that the good performance of the proposed model is due to the three properties (lines 345-348) that it incorporates. Since MPSN also incorporates the three properties, should it not perform as well as the proposed model?\n\nLIMITATIONS:\nOverall, the paper is very hard to follow. The motivation and contributions of the work are not clear. The errors in the paper make it more difficult to follow", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel architecture designed to operate on simplicial data, drawing its foundation from the Hodge decomposition. This decomposition ensures that features associated with a simplicial complex of order $k$ can be represented by three distinct quantities: a curl-free quantity, a divergence-free quantity, and a harmonic quantity.\nThe authors develop SCCNN, a new architecture for simplicial data that abides by these decomposition principles. The authors demonstrate the relevance of these principles by examining the Dirichlet energy, serving as a measure of oversmoothing within simplicial networks. The authors show that this new architecture reduces oversmoothness.\nTheoretical guarantees for the stability of the proposed method when subjected to small perturbations are proven. The proposed approach is benchmarked against two exploratory tasks: a forex test and simplex prediction. The findings indicate superior performance of the SCCNN over previous architectures in these tasks.\n\nSTRENGTHS:\n- This paper delivers a substantial theoretical advancement to the field of neural networks operating on simplicial complexes. It introduces compelling theorems within a well-structured framework. Theorem 6. on the expressiveness of the SCCNN model is a result of potentially good interest for this subfield.\n-  The authors have thoroughly compared their approach to existing methods, including standard graph neural networks (GNNs) and other simplicial neural networks. The results on toy examples are compelling.\n- The paper does a good job of introducing the key concepts in a clear and precise way. I appreciated the background on Hodge decomposition. \n- Due to the simplicity of the overall principle behind SCCNN,  it has the potential for broad application within the field and formalizes new fundamental design principles for future architectures.\n\nWEAKNESSES:\n- The paper will benefit from enhanced clarity. Currently, it contains abundant results, which, while potentially insightful, obscure the core message of the research. Streamlining these results and focusing on the most salient points would aid in transmitting the core of the research, which is the SCCNN architecture, more effectively. The discussion around stability, while interesting, feels out of place. There are no clear motivations for studying it so thoroughly in the main text. On the other hand, the critical Theorems, such as Theorems 6. and 7., will benefit from a lengthier exposition. In particular, giving intuition behind the proofs of Theorem 6 would be appreciated. While I appreciate it is a theoretical paper, the results go in every possible direction with no clear target. In such a short paper, one should focus on a few key ideas and move as much as possible to appendices.\n\n- The forex example is somewhat tailored to align with the proposed method. While it proves the paper's point and should be kept, it feels too synthetic. I am happy to be contradicted by the authors on that.\n\nQUESTIONS:\n- Could the authors provide more insight on this sentence in the limitations: SCNN \"cannot learn differently from features at the\nfrequencies of the same type and the same value\". I find the sentence very confusing. For example GNNs, can learn different node features over different channels, while they are all harmonic features. \n- I would like to understand why stability deserves such lengthy exposition in this paper. Currently, it feels out of place. What makes the stability of higher importance for SCCNN than for other architectures? It would deserve more experiments to show why it is essential in networks on simplicial complexes.\n\nLIMITATIONS:\nLimitations are discussed. Some points need to be clarified. See my question above.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-26"}

comments: SUMMARY:
The authors propose a general convolutional architecture with principles of uncoupling the lower and upper simplicial adjacencies, which accounting for the inter-simplicial couplings, and performing higher-order convolutions for learning of higher-order structure and simplicial signal.

STRENGTHS:
- The authors show that the proposed SCCNN structure demonstrates awareness of the Hodge decomposition and performs efficient learning on simplicial data
- Effect on mitigating simplicial complex oversmoothing is explained with Dirichlet energy minimization
- Comprehensive experimental results 
- Stability against robustness is also studied in this work

WEAKNESSES:
See questions

QUESTIONS:
- How is the orientation of simplex determined in the experiments?
- In the definition of simplicial data / k-signal, is it defined for one simplex or a k-chains? What are their dimensions? Is it similar to a feature vector or it is changing over time? It would be helpful if the authors show an example of it the k signal and related terms, as one of the main claim is that "advantage of using SCs is not only about them being able to model higher-order network structure, but also support simplicial data"
- There are some typos in the paper, for example, line 395 "approcahed".
- For the simplex prediction problem, performance and baseline comparison with higher-order simplices (order=4,5,6...) could be also provided.

LIMITATIONS:
n/a
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-27