Original: N EURAL S HEAF D IFFUSION : A T OPOLOGICAL P ERSPECTIVE ON H ETEROPHILY AND O VERSMOOTHING IN GNN S
Revised: N EURAL S HEAF D IFFUSION : A T OPOLOGICAL P ERSPECTIVE ON H ETEROPHILY AND O VERSMOOTHING IN GNN S

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Cellular sheaves equip graphs with “geometrical” structure by assigning vector spaces and linear maps to nodes and edges.
Revised: Cellular sheaves equip graphs with “geometrical” structure by assigning vector spaces and linear maps to nodes and edges.

Original: Graph Neural Networks (GNNs) implicitly assume a graph with a trivial underlying sheaf.
Revised: Graph Neural Networks (GNNs) implicitly assume a graph with a trivial underlying sheaf.

Original: This choice is reflected in the structure of the graph Laplacian operator, the properties of the associated diffusion equation, and the characteristics of the convolutional models that discretise this equation.
Revised: This choice is reflected in the structure of the graph Laplacian operator, the properties of the associated diffusion equation, and the characteristics of the convolutional models that discretise this equation.

Original: In this paper, we use cellular sheaf theory to show that the underlying geometry of the graph is deeply linked with the performance of GNNs in heterophilic settings and their oversmoothing behaviour.
Revised: In this paper, we use cellular sheaf theory to show that the underlying geometry of the graph is deeply linked with the performance of GNNs in heterophilic settings and their oversmoothing behaviour.

Original: By considering a hierarchy of increasingly general sheaves, we study how the ability of the sheaf diffusion process to achieve linear separation of the classes in the infinite time limit expands.
Revised: By considering a hierarchy of increasingly general sheaves, we study how the ability of the sheaf diffusion process to achieve linear separation of the classes in the infinite time limit expands.

Original: At the same time, we prove that when the sheaf is non-trivial, discretised parametric diffusion processes have greater control than GNNs over their asymptotic behaviour.
Revised: At the same time, we prove that when the sheaf is non-trivial, discretised parametric diffusion processes have greater control than GNNs over their asymptotic behaviour.

Original: On the practical side, we study how sheaves can be learned from data.
Revised: On the practical side, we study how sheaves can be learned from data.

Original: The resulting sheaf diffusion models have many desirable properties that address the limitations of classical graph diffusion equations (and corresponding
Revised: The resulting sheaf diffusion models have many desirable properties that address the limitations of classical graph diffusion equations (and corresponding

Original: GNN models) and obtain state-of-the-art results in heterophilic settings.
Revised: GNN models) and obtain state-of-the-art results in heterophilic settings.

Original: Overall, our work provides new connections between GNNs and algebraic topology and would be of interest to both fields.
Revised: Overall, our work provides new connections between GNNs and algebraic topology and would be of interest to both fields.

Revised: from data via a parametric function Φ .

Revised:  I NTRODUCTION

Revised: Graph Neural Networks (GNNs) Sperduti (1994); Goller & Kuchler (1996); Gori et al. (2005);

Original: back.
Revised: Scarselli et al. ; Bruna et al. (2014); Defferrard et al. ; Kipf & Welling (2017); Gilmer et al.

Revised: (2017) have recently become very popular in the ML community as a model of choice to deal with relational and interaction data due to their multiple successful applications in domains ranging from social science and particle physics to structural biology and drug design.

Revised: We focus on two main problems often observed in GNNs: their poor performance in heterophilic graphs Zhu et al.

Revised: (2020) and their oversmoothing behaviour (Oono & Suzuki, 2019; Cai & Wang, 2020).

Revised: The former arises from the fact that GNNs are usually built on the strong assumption of homophily , i.e., that nodes tend to connect to other similar nodes.

Original: B ACKGROUND
Revised: The latter refers to a phenomenon of deeper GNNs producing features that are too smooth to be useful. In this work, we show that these two fundamental problems are linked by a common cause: the underlying “geometry” of the graph (used here in a very loose sense). When this geometry is trivial, as is typically the case, the two phenomena described above emerge. We make these statements precise through the lens of cellular sheaf theory (Curry, 2014), a subfield of algebraic topology (Hatcher, 2000).

Revised: Intuitively, a cellular sheaf associates a vector space to each node and edge of a graph, and a linear map between these spaces for each incident node-edge pair (Figure 1).

Revised: In Section 3 we analyse how by considering a hierarchy of increasingly general sheaves, starting from a trivial one, a diffusion equation based on the sheaf Laplacian (Hansen & Ghrist, 2019) can solve increasingly more complicated node-classification tasks in the infinite time limit.

Revised: In this regime, we show that oversmoothing can be avoided by equipping the graph with the right sheaf structure for a task.

Revised: In practice we apply our theory for designing simple and practical GNN models (Section 4) by learning sheaves from data.

Revised: The resulting sheaf models obtain state-of-the-art results in heterophilic graphs and show strong performance in homophilic ones.

Revised: Overall, our paper provides the first successful instantiation of Sheaf Neural Networks (Hansen & Gebhart, 2020) in a non-synthetic setting as well as the first theoretical motivation for such models in graph machine learning applications.

Original: Note.
Revised: Note.

Original: This represents a condensed version of a full paper on sheaves, heterophily and oversmoothing in GNNs.
Revised: This represents a condensed version of a full paper on sheaves, heterophily and oversmoothing in GNNs.

Original: Therefore, the main content covers only some of our essential theoretical and experimental results.
Revised: Therefore, the main content covers only some of our essential theoretical and experimental results.

Original: For additional introductory material, theory and experiments, please consult the Appendix.
Revised: For additional introductory material, theory and experiments, please consult the Appendix.

Revised:  B ACKGROUND

Original: Definition 1.
Revised: Definition 1.

Original: A cellular sheaf ( G, F ) (Curry, 2014) on an undirected graph G =
Revised: A cellular sheaf ( G, F ) (Curry, 2014) on an undirected graph G =

Original: ( V, E ) consists of: a vector space F ( v ) for each v ∈ V , a vector space F ( e ) for each e ∈ E , and a linear map
Revised: ( V, E ) consists of: a vector space F ( v ) for each v ∈ V , a vector space F ( e ) for each e ∈ E , and a linear map

Original: F v ⊴ e : F ( v ) → F ( e ) for each incident v ⊴ e node-edge pair.
Revised: F v ⊴ e : F ( v ) → F ( e ) for each incident v ⊴ e node-edge pair.

Original: The vector spaces are called stalks , while the linear maps are referred to as restriction maps (Figure 1).
Revised: The vector spaces are called stalks , while the linear maps are referred to as restriction maps (Figure 1).

Original: The space formed by all the spaces associated to the nodes of the graph is called the space of 0cochains and is denoted by C 0 ( G, F ) :
Revised: The space formed by all the spaces associated to the nodes of the graph is called the space of 0cochains and is denoted by C 0 ( G, F ) :

Original: = (cid:76) v ∈ V F ( v ) .
Revised: = (cid:76) v ∈ V F ( v ) .

Original: Similarly, C 1 ( G, F ) :
Revised: Similarly, C 1 ( G, F ) :

Original: = (cid:76) e ∈ E F ( e ) – the space of 1- cochains – contains the data associated with all the edges of the graph.
Revised: = (cid:76) e ∈ E F ( e ) – the space of 1- cochains – contains the data associated with all the edges of the graph.

Original: For a 0-cochain x ∈ C 0 ( G ; F ) , we use x v to refer to the vector in F ( v ) of node v and similarly for 1-cochains.
Revised: For a 0-cochain x ∈ C 0 ( G ; F ) , we use x v to refer to the vector in F ( v ) of node v and similarly for 1-cochains.

Original: From an opinion dynamics perspective (Hansen & Ghrist, 2021), x v can be thought of as the private opinion of node v , while F v ⊴ e expressed how that opinion manifests publicly in a discourse space formed by F ( e ) .
Revised: From an opinion dynamics perspective (Hansen & Ghrist, 2021), x v can be thought of as the private opinion of node v , while F v ⊴ e expressed how that opinion manifests publicly in a discourse space formed by F ( e ) .

Original: It is natural to define a linear co-boundary map δ between
Revised: It is natural to define a linear co-boundary map δ between

Original: C 0 ( G, F ) and C 1 ( G, F ) , which measures the disagreement between all nodes.
Revised: C 0 ( G, F ) and C 1 ( G, F ) , which measures the disagreement between all nodes.

Original: Definition 2.
Revised: Definition 2.

Original: For some arbitrary choice of orientation for each edge e = u → v ∈ E , δ : C 0 ( G, F ) → C 1 ( G, F ) , δ ( x ) e := F v ⊴ e x v − F u ⊴ e x u .
Revised: For some arbitrary choice of orientation for each edge e = u → v ∈ E , δ : C 0 ( G, F ) → C 1 ( G, F ) , δ ( x ) e := F v ⊴ e x v − F u ⊴ e x u .

Original: Given a cellular sheaf ( G, F ) , using the co-boundary operator δ , one can define a Sheaf Laplacian operator (Hansen & Ghrist, 2019) measuring the average disagreement between all the nodes.
Revised: Given a cellular sheaf ( G, F ) , using the co-boundary operator δ , one can define a Sheaf Laplacian operator (Hansen & Ghrist, 2019) measuring the average disagreement between all the nodes.

Original: Definition 3.
Revised: Definition 3.

Original: The sheaf Laplacian of a sheaf ( G, F ) is a map L F : C 0 ( G, F )
Revised: The sheaf Laplacian of a sheaf ( G, F ) is a map L F : C 0 ( G, F )

Original: → C 0 ( G, F ) given by L F :=
Revised: → C 0 ( G, F ) given by L F :=

Original: δ ⊤ δ , L F ( x ) v := (cid:80) v,u ⊴ e F ⊤ v ⊴ e
Revised: δ ⊤ δ , L F ( x ) v := (cid:80) v,u ⊴ e F ⊤ v ⊴ e

Original: ( F v ⊴ e x v − F u ⊴ e x u )
Revised: ( F v ⊴ e x v − F u ⊴ e x u )

Original: The sheaf Laplacian is a positive semi-definite block matrix.
Revised: The sheaf Laplacian is a positive semi-definite block matrix.

Original: The diagonal blocks are L F vv = (cid:80) v ⊴ e F ⊤ v ⊴ e F v ⊴ e , while the non-diagonal blocks L F vu =
Revised: The diagonal blocks are L F vv = (cid:80) v ⊴ e F ⊤ v ⊴ e F v ⊴ e , while the non-diagonal blocks L F vu =

Original: −F ⊤ v ⊴ e F u ⊴ e .
Revised: −F ⊤ v ⊴ e F u ⊴ e .

Original: A normalised version of this Laplacian can also be defined.
Revised: A normalised version of this Laplacian can also be defined.

Original: Let D be the block-diagonal of L F .
Revised: Let D be the block-diagonal of L F .

Original: Then the normalised-sheaf
Revised: Then the normalised-sheaf

Original: Laplacian ∆ F :=
Revised: Laplacian ∆ F :=

Original: D − 1 / 2 L F D − 1 / 2 .
Revised: D − 1 / 2 L F D − 1 / 2 .

Original: For simplicity, we assume from now on that all the stalks have dimension d .
Revised: For simplicity, we assume from now on that all the stalks have dimension d .

Original: In that case, the sheaf
Revised: In that case, the sheaf

Original: Laplacian is a nd × nd real matrix, where n is the number of nodes of G .
Revised: Laplacian is a nd × nd real matrix, where n is the number of nodes of G .

Original: When the vector spaces are set to R and the linear maps to id , the underlying sheaf is trivial, d = 1 and one recovers the well-known n × n (normalised) graph Laplacian matrix.
Revised: When the vector spaces are set to R and the linear maps to id , the underlying sheaf is trivial, d = 1 and one recovers the well-known n × n (normalised) graph Laplacian matrix.

Original: The harmonic space of L F is characterised by the global sections of the sheaf.
Revised: The harmonic space of L F is characterised by the global sections of the sheaf.

Original: Definition 4.
Revised: Definition 4.

Original: The global sections of a sheaf H 0 ( G ; F ) :
Revised: The global sections of a sheaf H 0 ( G ; F ) :

Original: = { x ∈ C 0 ( G ; F ) | F v ⊴ e x v = F u ⊴ e x u }
Revised: = { x [Equation] [Equation] [Equation]

Revised: F

Revised: [Equation]

Revised: =

Revised: [Equation]

Original:  This set corresponds to the signals that agree with the restriction maps globally (i.e. along all edges of the graph).
Revised: u ⊴ [Equation] x u } This set corresponds to the signals that agree with the restriction maps globally (i.e. along all edges of the graph).

Original: The central theorem of discrete Hodge theory says that the space formed by all these 0 -cochains is isomorphic to the kernel of the sheaf Laplacian.
Revised: The central theorem of discrete Hodge theory says that the space formed by all these 0 -cochains is isomorphic to the kernel of the sheaf Laplacian.

Original: The sheaves ( G, F ) with orthogonal restriction maps (i.e. F v ⊴ e ∈ O ( d ) the Lie group of d × d orthogonal matrices), will play an important role in our analysis.
Revised: The sheaves ( G, F ) with orthogonal restriction maps (i.e. F v ⊴ e ∈ O ( d ) the Lie group of d × d orthogonal matrices), will play an important role in our analysis.

Original: Such sheaves are called discrete
Revised: Such sheaves are called discrete

Original: O ( d ) -bundles since they can be seen as a discrete equivalent of vector bundles from differential geometry (Tu, 2011).
Revised: O ( d ) -bundles since they can be seen as a discrete equivalent of vector bundles from differential geometry (Tu, 2011).

Original: Intuitively, these are objects describing how vector spaces are attached to the points of a manifold.
Revised: Intuitively, these are objects describing how vector spaces are attached to the points of a manifold.

Original: In our case, the role of the manifold is played by the graph.
Revised: In our case, the role of the manifold is played by the graph.

Original: The sheaf
Revised: The sheaf

Original: Laplacian of a discrete O ( d ) -bundle is equivalent to a connection Laplacian (Singer & Wu, 2012), describing how the elements of a vector space are transported via rotations in another neighbouring space.
Revised: Laplacian of a discrete O ( d ) -bundle is equivalent to a connection Laplacian (Singer & Wu, 2012), describing how the elements of a vector space are transported via rotations in another neighbouring space.

Original: This is analogous to how tangent vectors are transported across a manifold (see Figure 2).
Revised: This is analogous to how tangent vectors are transported across a manifold (see Figure 2).

Original:  T HE S EPARATION P OWER OF S HEAF D IFFUSION
Revised:  T HE S EPARATION P OWER OF S HEAF D IFFUSION

Original: Preliminaries Let G =
Revised: Preliminaries Let G =

Original: ( V, E ) be a graph and consider that all nodes have features that are ddim vectors x v ∈ F ( v ) .
Revised: ( V, E ) be a graph and consider that all nodes have features that are ddim vectors x v ∈ F ( v ) .

Original: The features of all nodes are represented as a single vector x ∈ C 

Revised: The features of all nodes are represented as a single vector x ∈ C

Original: ( G ; F ) stacking all the individual d -dim vectors.
Revised: ( G ; F ) stacking all the individual d -dim vectors.

Original: Additionally, if we allow for f feature channels, everything can be represented as a matrix X ∈ R ( nd ) × f , whose columns are vectors in C 0 ( G ; F ) .
Revised: Additionally, if we allow for f feature channels, everything can be represented as a matrix X ∈ R ( nd ) × f , whose columns are vectors in C 0 ( G ; F ) .

Original: Finally, we are interested in the sheaf diffusion process governed by the PDE:
Revised: Finally, we are interested in the sheaf diffusion process governed by the PDE:

Original: [Equation]
Revised: [Equation]

Original: We now analyse the ability of certain classes of sheaf Laplacian operators to linearly separate the features in the limit of the diffusion processes they induce.
Revised: We now analyse the ability of certain classes of sheaf Laplacian operators to linearly separate the features in the limit of the diffusion processes they induce.

Original: We consider this a proxy metric for the capacity of certain diffusion processes to avoid oversmoothing.
Revised: We consider this a proxy metric for the capacity of certain diffusion processes to avoid oversmoothing.

Original: Definition 5.
Revised: Definition 5.

Original: A hypothesis class of sheaves with d -dimensional stalks H d has linear separation power over a set of graphs G if for any labelled graph G =
Revised: A hypothesis class of sheaves with d -dimensional stalks H d has linear separation power over a set of graphs G if for any labelled graph G =

Original: ( V, E ) ∈ G , there is a sheaf ( F , G ) ∈
Revised: ( V, E ) ∈ G , there is a sheaf ( F , G ) ∈

Original: H d that can linearly separate the classes of G in the time limit of Equation 1 for a dense subset
Revised: H d that can linearly separate the classes of G in the time limit of Equation 1 for a dense subset

Original: X F ⊂ R nd × f of initial conditions.
Revised: X F ⊂ R nd × f of initial conditions.

Original: The restriction to a set of initial conditions that is dense in the ambient space is necessary because, in the limit, diffusion behaves like a projection in the harmonic space and there will always be degenerate initial conditions that will yield a zero projection.
Revised: The restriction to a set of initial conditions that is dense in the ambient space is necessary because, in the limit, diffusion behaves like a projection in the harmonic space and there will always be degenerate initial conditions that will yield a zero projection.

Original: Definition 6.
Revised: Definition 6.

Original: Consider the class of sheaves with symmetric and invertible transport maps and ddimensional stalks:
Revised: Consider the class of sheaves with symmetric and invertible transport maps and ddimensional stalks:

Original: [Equation]
Revised: [Equation]

Original: We note that for d = 1 , the sheaf Laplacians induced by this class of sheaves coincides with the set of the well-known weighted graph Laplacians with strictly positive weights, which also includes the usual graph Laplacian (see proof in Appendix I).
Revised: We note that for d = 1 , the sheaf Laplacians induced by this class of sheaves coincides with the set of the well-known weighted graph Laplacians with strictly positive weights, which also includes the usual graph Laplacian (see proof in Appendix H).

Original: Therefore, this hypothesis class is of particular interest since it includes those graph Laplacians typically used by graph convolutional models such as GCN (Kipf & Welling, 2017) and ChebNet (Defferrard et al., 2016).
Revised: Therefore, this hypothesis class is of particular interest since it includes those graph Laplacians typically used by graph convolutional models such as GCN (Kipf & Welling, 2017) and ChebNet (Defferrard et al., 2016).

Original: We first show that this class of sheaf Laplacians can linearly separate the classes in binary classification settings under certain homophily assumptions:
Revised: We first show that this class of sheaf Laplacians can linearly separate the classes in binary classification settings under certain homophily assumptions:

Original: Proposition 7.
Revised: Proposition 7.

Original: Let G be the set of connected graphs G =
Revised: Let G be the set of connected graphs G =

Original: ( V, E ) with two classes A, B ⊂ V such that for each v ∈ A , there exists u ∈ A and an edge ( v, u ) ∈ E .
Revised: ( V, E ) with two classes A, B ⊂ V such that for each v ∈ A , there exists u ∈ A and an edge ( v, u ) ∈ E .

Original: Then H 1sym has linear separation power over G .
Revised: Then H 1sym has linear separation power over G .

Original: In contrast, under certain heterophilic conditions, this hypothesis class is not powerful enough to linearly separate the two classes no matter what the initial conditions are:
Revised: In contrast, under certain heterophilic conditions, this hypothesis class is not powerful enough to linearly separate the two classes no matter what the initial conditions are:

Original: Proposition 8.
Revised: Proposition 8.

Original: Let G be the set of connected bipartite graphs G =
Revised: Let G be the set of connected bipartite graphs G =

Original: ( A, B, E ) , with partitions A, B forming two classes and | A | = | B | .
Revised: ( A, B, E ) , with partitions A, B forming two classes and | A | = | B | .

Original: Then H 1sym cannot linearly separate any graph in G for any initial conditions
Revised: Then H 1sym cannot linearly separate any graph in G for any initial conditions

Original: X (0) ∈ R n × f .
Revised: X (0) ∈ R n × f .

Original: We now consider a larger hypothesis class that also includes non-symmetric relations.
Revised: We now consider a larger hypothesis class that also includes non-symmetric relations.

Original: Definition 9.
Revised: Definition 9.

Original: H d := { ( F , G ) | det( F v ⊴ e ) ̸ = 0 }
Revised: H d := { ( F , G ) | det( F v ⊴ e ) ̸ = 0 }

Original: Proposition 10.
Revised: Proposition 10.

Original: Let G contain all the connected graphs with two classes.
Revised: Let G contain all the connected graphs with two classes.

Original: Then, H 1 has linear separation power over G .
Revised: Then, H 1 has linear separation power over G .

Original: These results show that heterophilic settings require an asymmetric transport of the features between neighbouring nodes belonging to different classes.
Revised: These results show that heterophilic settings require an asymmetric transport of the features between neighbouring nodes belonging to different classes.

Original: This provides a sheaf-theoretic explanation for why a recent body of work (Yan et al., 2021; Chien et al., 2021; Bo et al., 2021) has found negativelyweighted edges to help in heterophilic settings.
Revised: This provides a sheaf-theoretic explanation for why a recent body of work (Yan et al., 2021; Chien et al., 2021; Bo et al., 2021) has found negativelyweighted edges to help in heterophilic settings.

Original: From this perspective, negatively-signed edges constrain the product F v ⊴ e F u ⊴ e of an implicit underlying sheaf to be negative and hence F v ⊴ e ̸ =
Revised: From this perspective, negatively-signed edges constrain the product F v ⊴ e F u ⊴ e of an implicit underlying sheaf to be negative and hence F v ⊴ e ̸ =

Original: F u ⊴ e .
Revised: F u ⊴ e .

Original: So far we have only studied the effects of changing the type of sheaves in dimension one.
Revised: So far we have only studied the effects of changing the type of sheaves in dimension one.

Original: We now consider the effects of adjusting the dimension of the stalks and begin by stating a fundamental limitation of (sheaf) diffusion when d = 1 .
Revised: We now consider the effects of adjusting the dimension of the stalks and begin by stating a fundamental limitation of (sheaf) diffusion when d = 1 .

Original: Proposition 11.
Revised: Proposition 11.

Original: Let G be a connected graph with C ≥ 3 classes.
Revised: Let G be a connected graph with C ≥ 3 classes.

Original: Then H 1 cannot linearly separate any X ∈ R n × f .
Revised: Then H 1 cannot linearly separate any X ∈ R n × f .

Original: This is essentially a consequence of dim (cid:0) ker(∆ F ) (cid:1) being at most one in this case.
Revised: This is essentially a consequence of dim (cid:0) ker(∆ F ) (cid:1) being at most one in this case.

Original: From a GNN perspective, this means that in the infinite depth setting, sufficient stalk width (i.e., dimension) is needed in order to solve tasks involving more than two classes.
Revised: From a GNN perspective, this means that in the infinite depth setting, sufficient stalk width (i.e., dimension) is needed in order to solve tasks involving more than two classes.

Original: Definition 12.
Revised: Definition 12.

Original: Consider the class of sheaves with diagonal invertible maps and d -dimensional stalks 
Revised: Consider the class of sheaves with diagonal invertible maps and d -dimensional stalks H

Original: [Equation]
Revised: [Equation]

Revised: invertible diagonal

Revised: [Equation]

Original: Proposition 13.
Revised: Proposition 13.

Original: Let G be the set of connected graphs with nodes belonging to C ≥ 3 classes.
Revised: Let G be the set of connected graphs with nodes belonging to C ≥ 3 classes.

Original: Then for d ≥ C , H d diag has linear separation power over G .
Revised: Then for d ≥ C , H d diag has linear separation power over G .

Original: This proposition illustrates the benefits of using higher-dimensional stalks, while maintaining a simple and computationally convenient class of diagonal restriction maps.
Revised: This proposition illustrates the benefits of using higher-dimensional stalks, while maintaining a simple and computationally convenient class of diagonal restriction maps.

Original: By using more complex restriction maps, we can show that lower-dimensional stalks can be used to achieve linear separation in the presence of even more classes.
Revised: By using more complex restriction maps, we can show that lower-dimensional stalks can be used to achieve linear separation in the presence of even more classes.

Original: Definition 14.
Revised: Definition 14.

Original: The class of discrete O ( d ) -bundles H d orth :=
Revised: The class of discrete O ( d ) -bundles H d orth :=

Original: { ( F , G ) | F v ⊴ e ∈ O ( d ) }
Revised: { ( F , G ) | F v ⊴ e ∈ O ( d ) }

Original: We show that for stalks of dimension d ∈ { 2 , 4 } , one can classify at least 
Revised: We show that for stalks of dimension d ∈ { 2 , 4 } , one can classify at least C = 2 d classes. Theorem 15. Let G be the class of connected graphs with C ≤ 2 d classes. Then, for all d ∈ { 2 , 4 } ,

Revised: H d orth has linear separation power over G .

Original: [Table]
Revised: This shows that orthogonal maps are able to make more efficient use of the space available to them than diagonal restriction maps.  M ODEL & R ESULTS The results in the previous section show that solving any node classification can be reduced to finding the right sheaf over the graph. Therefore, we aim to learn the sheaf from data.

Revised: To that end, we consider the following diffusion-type equation:

Revised: [Equation]

Revised: Note that this model contains the sheaf diffusion equation as a particular case and, therefore it inherits all of its positive theoretical properties.

Revised: Crucially, the sheaf ( G, F ( t )) that evolves over time as a parametric function of the data ( G, F ( t ))

Revised: =

Revised: g ( G, X ( t ); θ ) .

Revised: This allows the model to learn a sheaf from the latest available node representations.

Revised: We also consider a discrete version of this equation, using a new set of weights at each layer t .

Revised: [Equation]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Original:  [Figure]
Revised: [Table] [Figure] [Table] [Figure]

Original: [Table] 
Revised: [Table] [Figure] [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure] [Table] [Figure] [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure] [Table] [Figure] [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure] [Table] [Figure] [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure] [Table] [Figure] [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure] [Table] [Figure] [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure] [Table] [Figure] [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure] [Table] [Figure] [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure] [Table] [Figure] [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure] [Table] [Figure] [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure] [Table] [Figure] [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure] [Table] [Figure] [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure] [Table] [Figure] [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure] [Table] [Figure] [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure] [Table] [Figure] [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: Theorem 15. Let G be the class of connected graphs with C ≤ 2 d classes. Then, for all d ∈ { 2 , 4 } , H d orth has linear separation power over G .
Revised: [Figure]

Original: This shows that orthogonal maps are able to make more efficient use of the space available to them than diagonal restriction maps.  M ODEL & R ESULTS We consider the following diffusion-type equation, which contains the sheaf diffusion equation as a particular case.
Revised: [Table]

Original: [Equation]
Revised: [Figure]

Original: Crucially, the sheaf Laplacian ∆ F ( t ) is that of a sheaf ( G, F ( t )) that evolves over time . More specifically, the evolution of the sheaf structure is described by a learnable function of the data ( G, F ( t )) = g ( G, X ( t ); θ ) . We also consider a discrete version of this equation, using a new set of weights at each layer t .
Revised: [Table]

Original: [Equation] [Equation] For both,models we use an initialMLP to compute X (0) from theraw features and a final linear layer toperform the node classification.
Revised: both, we use an initial to compute X ( the to

Original: Real-world experiments We test our models on real-world datasets proposed by Rozemberczki et al.
Revised: Real-world experiments We test our models on real-world datasets proposed by Rozemberczki et al.

Original: ; Pei et al.
Revised: ; Pei et al.

Original: (2020) to evaluate heterophilic learning.
Revised: (2020) to evaluate heterophilic learning.

Original: These datasets have an edge homophily coefficient h ranging from h = 0 .
Revised: These datasets have an edge homophily coefficient h ranging from h = 0 .

Original: 11 (very heterophilic) to h = 0 .
Revised: 11 (very heterophilic) to h = 0 .

Original: 81 (very homophilic) and therefore offer a view of how a model performs in both regimes.
Revised: 81 (very homophilic) and therefore offer a view of how a model performs in both regimes.

Original: We evaluate our models on thefixed splits provided by Pei et al.
Revised: We evaluate our models on thefixed splits provided by Pei et al.

Original: and report the mean accuracy and standard deviation.
Revised: and report the mean accuracy and standard deviation.

Original: Each split contains 48% / 32% / 20% of nodes per class for training, validation and testing, respectively.
Revised: Each split contains 48% / 32% / 20% of nodes per class for training, validation and testing, respectively.

Original: Results From the results in Table 1 we see that the discrete version of our models are first in 5 /benchmarks with high heterophily ( h < 0 . 3 ) and second-ranked in the other 1 / 6 .
Revised: Results From the results in Table 1 we see that the discrete version of our models are first in 5 /benchmarks with high heterophily ( h < 0 . 3 ) and second-ranked in the other 1 / 6 .

Original: At the same time, the models exhibit strong performance on the homophilic graphs (Cora, Pubmed, Citeseer) being within approximately 1% of the top-performing model.
Revised: At the same time, the models exhibit strong performance on the homophilic graphs (Cora, Pubmed, Citeseer) being within approximately 1% of the top-performing model.

Original: The second part of Table 1 includes continuous models, which our model outperforms on 7 / 9 benchmarks with particularly large improvements on Chameleon and Squirrel.
Revised: The second part of Table 1 includes continuous models, which our model outperforms on 7 / 9 benchmarks with particularly large improvements on Chameleon and Squirrel.

Original: Among the discrete diffusion models, the O ( d ) -vector bundle diffusion model performs best overall confirming the intuition that it can better avoid overfitting, while also transforming the vectors in sufficiently complex ways.
Revised: Among the discrete diffusion models, the O ( d ) -vector bundle diffusion model performs best overall confirming the intuition that it can better avoid overfitting, while also transforming the vectors in sufficiently complex ways.

Original: We also remark the strong performance of the model learning diagonals maps.
Revised: We also remark the strong performance of the model learning diagonals maps.

Original:  A I NTRODUCTION
Revised:  R ELATED W ORK AND C ONCLUSION

Original: Graph Neural Networks (GNNs) Sperduti (1994); Goller & Kuchler (1996); Gori et al. (2005); Scarselli et al ; Bruna et al.
Revised: Sheaf Neural Networks Sheaf Convolutional Networks (SCN) were originally proposed by

Original: (2014); Defferrard et al. ; Kipf & Welling (2017); Gilmer et al.
Revised: Hansen & Gebhart (2020) using layers of the form X t +1 =

Original: (2017) have recently become very popular in the ML community as a model of choice to deal with relational and interaction data due to their multiple successful applications in domains ranging from social science and particle physics to structural biology and drug design. We focus on two main problems often observed in GNNs: their poor performance in heterophilic graphs Zhu et al. (2020) and their oversmoothing behaviour Nt & Maehara (2019); Oono & Suzuki (2020). The former arises from the fact that GNNs are usually built on the strong assumption of homophily , i e., that nodes tend to connect to other similar nodes.
Revised: σ (( I − d − 1max L F )( I ⊗ W t 1 )

Original: The latter refers to a phenomenon of deeper GNNs producing features that are too smooth to be useful. In this work, we show that these two fundamental problems are linked by a common cause: the underlying “geometry” of the graph (used here in a very loose sense). When this geometry is trivial, as is typically the case, the two phenomena described above emerge.
Revised: X t W t 2 ) .

Original: We make these statements precise through the lens of cellular sheaf theory (Curry, 2014), a subfield of algebraic topology (Hatcher, 2000). Intuitively, a cellular sheaf associates a vector space to each node and edge of a graph, and a linear map between these spaces for each incident node-edge pair (Figure 1). In Section 2 we analyse how by considering a hierarchy ofincreasingly general sheaves, starting from a trivial one, a diffusion equation based on the sheaf Laplacian (Hansen & Ghrist, 2019) can solve increasingly more complicated node-classification tasks in the infinite time limit.
Revised: This model was only evaluated in a toy experimental setting where L F was hand-crafted with knowledge of the data-generating process.

Original: In this regime, we show that oversmoothing can be avoided by equipping the graph with the right sheaf structure for a task.
Revised: This made the model inapplicable to real-world graphs, which do not have a natural sheaf structure.

Original: In Section C, we study the behaviour of a non-linear, parametric, and discrete version of this process. This results in  aSheaf Convolutional Network (Hansen & Gebhart, 2020) that generalises GCN of Kipf & Welling (2017). We prove that this discrete diffusion process does not necessarily converge to the harmonic space of the sheaf Laplacian, even when the norm of the weights is controlled from above, in contrast to GCNs (Cai & Wang, 2020; Oono & Suzuki, 2019).
Revised: Compared to SCN, the model in Equation 3 is residual and uses a sheaf Laplacian learned from data at each layer, which makes it applicable to any graph dataset.

Original: In other words, the weights can steer the diffusion process to a larger extent, which is important when the underlying sheaf is only approximately correct. All these results are based on the properties of the harmonic space of the sheaf Laplacian. We begin by studying these properties from a spectral perspective in Section
Revised: Supported by the theory, our model also takes advantage of the generality of sheaves, making use of higher-dimensional stalks and restriction maps of various types.

Original: B.
Revised: In contrast, SCN only considered scalar restriction maps in practice.

Original: We provide a new Cheegertype inequality for the spectral gap of the sheaf Laplacian and note that these results might be of independent interest for spectral sheaf theory. Finally, in Section D, we apply our theoryfor designing simple and practical GNN models. We describe how to construct such models by learning sheaves from data.
Revised: Finally, our paper provides the first theoretical justification for sheaves in graph ML by revealing their deep connections with heterophily and oversmoothing. Conclusion In this work, we used cellular sheaf theory to provide a novel topological perspective on heterophily and oversmoothing in GNNs.

Original: The resulting sheaf models obtain state-of-the-art results in heterophilic graphs and show strong performance in homophilic ones.
Revised: We showed that the underlying sheaf structure of the graph is intimately connected with both of these important factors affecting the performance of GNNs.

Original: To summarise our contributions: We (1) introduce anovel Graph ML framework based on cellular sheaves; (2) Prove new results about the spectral properties of the sheaf Laplacian that improve our understanding ofits harmonic space; (3) Study the linear separation power of sheaf diffusion equations in node classification tasks and demonstrate it can avoid oversmoothing; (4)
Revised: To mitigate this, we proposed a new paradigm for graph representation learning where models do not only evolve the features at each layer but also the underlying geometry of the graph. A CKNOWLEDGEMENTS We are grateful to Iulia Duta, Dobrik Georgiev and Jacob Deasy for valuable comments.

Original: Prove that a nonlinear sheaf-based GCN-like operator can steer the diffusion process, unlike in the classical case; (5) Show how to learn sheaves from data and construct sheaf diffusion-based GNN models that obtain state-of-the-art results on heterophilic datasets.
Revised: CB would also like to thank the Twitter Cortex team for making the research internship a fantastic experience. This research was supported in part by ERC Consolidator grant No. 724228 (LEMAN)

Original: B H ARMONIC S PACE OF S HEAF L APLACIANS
Revised: A H ARMONIC S PACE OF S HEAF L APLACIANS

Original: In this section we study general properties of the harmonic space ker(∆ F ) of the normalised sheaf
Revised: In this section we study general properties of the harmonic space ker(∆ F ) of the normalised sheaf

Original: Laplacian.
Revised: Laplacian.

Original: As with the normalised graph Laplacian, the normalised sheaf Laplacian is preferred for most practical purposes because its spectrum is bounded.
Revised: As with the normalised graph Laplacian the normalised sheaf Laplacian is preferred for most practical purposes because its spectrum is bounded.

Original: Since a major role in the analysis below is played by discrete vector bundles, we focus on this case.
Revised: Since a major role in the analysis below is played by discrete vector bundles, we focus on this case.

Original: We note though, that our results below generalise to the general linear group F v ⊴ e ∈ GL ( d ) , the Lie group of d × d invertible matrices, provided we can also control the norm of the restriction maps from below.
Revised: We note though that our results below generalise to the general linear group F v ⊴ e ∈ GL ( d ) , the Lie group of d × d invertible matrices, provided we can also control the norm of the restriction maps from below.

Original: Given a discrete O ( d ) -bundle, the block diagonal of L F has a block structure since L F vv = d v I d , where d v is the degree of node v .
Revised: Given a discrete O ( d ) -bundle, the block diagonal of L F has a block structure since L F vv = d v I d , where d v is the degree of node v .

Original: Accordingly, if a signal ˜ x ∈ ker( L F ) , then the signal x : v (cid:55)→ √ d v x v ∈ ker(∆ F ) and similarly for the inverse transformation.
Revised: Accordingly, if a signal ˜ x ∈ ker( L F ) , then the signal x : v (cid:55)→ √ d v x v ∈ ker(∆ F ) and similarly for the inverse transformation.

Original: In general the harmonic space may be trivial: this happens when the constraints F v ⊴ e x v = F u ⊴ e x u are not compatible with each other.
Revised: In general the harmonic space may be trivial: this happens when the constraints F v ⊴ e x v = F u ⊴ e x u are not compatible with each other.

Original: Since the harmonic space will be related to the linear separation power of sheaf diffusion, we investigate below when this space is non-empty and derive to what extent the graph structure affects that.
Revised: Since the harmonic space will be related to the linear separation power of sheaf diffusion, we investigate below when this space is non-empty and derive to what extent the graph structure affects that.

Original: Key to our analysis is studying transport operators induced by the restriction maps of the sheaf.
Revised: Key to our analysis is studying transport operators induced by the restriction maps of the sheaf.

Original: Given nodes v, u ∈ V and a path γ v → u = ( v, v 1 , . . .
Revised: Given nodes v, u ∈ V and a path γ v → u = ( v, v 1 , . . .

Original: , v ℓ , u ) from v to u , we consider a notion of transport from the stalk F ( v ) to the stalk F ( u ) via map composition:
Revised: , v ℓ , u ) from v to u , we consider a notion of transport from the stalk F ( v ) to the stalk F ( u ) via map composition:

Original: [Equation]
Revised: [Equation]

Original: In general then, transport maps act on node stalks and are constructed by composing single restriction maps (and their transpose) along edges.
Revised: In general then, transport maps act on node stalks and are constructed by composing single restriction maps (and their transpose) along edges.

Original: For general sheaf structures, the graph transport is path dependent , meaning that how the vectors are transported across two nodes depends on the path between them (see Figure 2).
Revised: For general sheaf structures, the graph transport is path dependent , meaning that how the vectors are transported across two nodes depends on the path between them (see Figure 2).

Original: In fact, we show that this property characterises the spectral gap of a sheaf Laplacian (we note again that, differently from the classical case, the kernel of ∆ F may be trivial, so by ‘spectral gap’ we refer here to the value of the smallest eigenvalue of ∆ F ).
Revised: In fact, we show that this property characterises the spectral gap of a sheaf Laplacian (we note again that, differently from the classical case, the kernel of ∆ F may be trivial, so by ‘spectral gap’ we refer here to the value of the smallest eigenvalue of ∆ F ).

Original: Proposition 16.
Revised: Proposition 16.

Original: If F is a discrete O ( d ) bundle over a connected graph and r := max [Equation] [Equation] P 
Revised: If F is a discrete O ( d ) bundle over a connected graph and r := max γ v → u ,γ ′ v → u || P γv → u − P γ ′ v → u || , then we have λ F 0 ≤ r 2 2 .

Original: [Equation] u || , then we have λ F 0 ≤ r 2 2 . A consequence of the previous result is that there is always a non-trivial harmonic space (i.e. λ F 0 = 0 ) if the transport maps generated by an orthogonal sheaf are path-independent .
Revised: A consequence of the previous result is that there is always a non-trivial harmonic space (i.e. λ F 0 = 0 ) if the transport maps generated by an orthogonal sheaf are path-independent .

Original: Next, we address the opposite direction.
Revised: Next, we address the opposite direction.

Original: Proposition 17.
Revised: Proposition 17.

Original: If F is a discrete O ( d ) bundle over a connected graph and x ∈ H 

Revised: If F is a discrete O ( d ) bundle over a connected graph and x ∈ H

Original: ( G, F ) , then for any cycle γ based at v ∈ V we have x v ∈ ker( P γv → v − I ) .
Revised: ( G, F ) , then for any cycle γ based at v ∈ V we have x v ∈ ker( P γv → v − I ) .

Original: The previous proposition highlights the interplay between the graph and the sheaf structure.
Revised: The previous proposition highlights the interplay between the graph and the sheaf structure.

Original: In fact, a simple consequence of this result is that for any cycle-free subset S ⊂ V , we have that any connection Laplacian restricted to S always admits a non-trivial harmonic space.
Revised: In fact, a simple consequence of this result is that for any cycle-free subset S ⊂ V , we have that any connection Laplacian restricted to S always admits a non-trivial harmonic space.

Original: A natural question connected to the previous result is whether a Cheeger-like inequality holds in the other direction.
Revised: A natural question connected to the previous result is whether a Cheeger-like inequality holds in the other direction.

Original: This turns out to be the case.
Revised: This turns out to be the case.

Original: Proposition 18.
Revised: Proposition 18.

Original: Let F be a discrete O ( d ) bundle over a connected graph G with n nodes and let || ( P γv → v − I )
Revised: Let F be a discrete O ( d ) bundle over a connected graph G with n nodes and let || ( P γv → v − I )

Original: x || ≥
Revised: x || ≥

Original: ϵ ||
Revised: ϵ ||

Original: x
Revised: x

Original: || for all cycles γ v → v .
Revised: || for all cycles γ v → v .

Original: Then λ F 0 ≥ ϵ 2 (2diam( G ) n d max ) − 1 .
Revised: Then λ F 0 ≥ ϵ 2 (2diam( G ) n d max ) − 1 .

Original: While the bound above is of little use in practice, it shows how the spectral gap of a sheaf Laplacian is indeed related to the deviation of the transport maps from being path-independent.
Revised: While the bound above is of little use in practice, it shows how the spectral gap of a sheaf Laplacian is indeed related to the deviation of the transport maps from being path-independent.

Original: We note that the Cheeger-like inequality presented here is not unique and other types of bounds on λ F 0 have been derived by Bandeira et al.
Revised: We note that the Cheeger-like inequality presented here is not unique and other types of bounds on λ F 0 have been derived by Bandeira et al.

Original: We conclude this section by further analysing the dimensionality of the harmonic space of discrete
Revised: We conclude this section by further analysing the dimensionality of the harmonic space of discrete

Original: O ( d ) -bundles.
Revised: O ( d ) -bundles.

Original: Lemma 19.
Revised: Lemma 19.

Original: Let F be a discrete O ( d ) bundle over a connected graph G .
Revised: Let F be a discrete O ( d ) bundle over a connected graph G .

Original: Then dim( H 0 ) ≤ d and dim( H 0 ) =
Revised: Then dim( H 0 ) ≤ d and dim( H 0 ) =

Original: d iff the transport is path-independent.
Revised: d iff the transport is path-independent.

Original: C A SYMPTOTICS OF S HEAF C ONVOLUTIONS
Revised: B A SYMPTOTICS OF S HEAF C ONVOLUTIONS

Original: In Section 2 we analysed the ability of sheaf diffusion to linearly separate the node-classes in the limit.
Revised: In Section 3 we analysed the ability of sheaf diffusion to linearly separate the node-classes in the limit.

Original: However, when considering a discrete, parametric and non-linear version of this process, it is important to know how much the weights can steer it.
Revised: However, when considering a discrete, parametric and non-linear version of this process, it is important to know how much the weights can steer it.

Original: This is particularly relevant if the underlying sheaf is only approximately correct for the task to be solved.
Revised: This is particularly relevant if the underlying sheaf is only approximately correct for the task to be solved.

Original: The continous diffusion process from Equation 1 has the following Euler discretisation with unit step-size:
Revised: The continous diffusion process from Equation 1 has the following Euler discretisation with unit step-size:

Original: [Equation]
Revised: [Equation]

Original: Assuming X ∈ R nd × f 1 , we can equip the right side with weight matrices W 1 ∈ R d × d , W 2 ∈ R f 1 × fand a non-linearity σ to arrive at the following model originally proposed by Hansen &
Revised: Assuming X ∈ R nd × f 1 , we can equip the right side with weight matrices W 1 ∈ R d × d , W 2 ∈ R f 1 × fand a non-linearity σ to arrive at the following model originally proposed by Hansen &

Original: Gebhart (2020):
Revised: Gebhart (2020):

Original: [Equation]
Revised: [Equation]

Original: where f 1 , f 2 are the number of input and output feature channels, and ⊗ denotes the Kronecker product.
Revised: where f 1 , f 2 are the number of input and output feature channels, and ⊗ denotes the Kronecker product.

Original: Here, W 1 multiplies from the left the vector feature of all the nodes in all channels (i.e.
Revised: Here, W 1 multiplies from the left the vector feature of all the nodes in all channels (i.e.

Original: W 1 x iv for all v and channels i ), while W 2 multiplies the features from the right and can adjust the number of feature channels, just like in GCNs.
Revised: W 1 x iv for all v and channels i ), while W 2 multiplies the features from the right and can adjust the number of feature channels, just like in GCNs.

Original: It is natural to call this model a Sheaf Convolutional Network (SCN) since when ∆ F is the usual normalised graph Laplacian, W 1 becomes a scalar and one recovers GCN of Kipf & Welling (2017).
Revised: It is natural to call this model a Sheaf Convolutional Network (SCN) since when ∆ F is the usual normalised graph Laplacian, W 1 becomes a scalar and one recovers GCN of Kipf & Welling (2017).

Original: To study this prototypical nonlinear and parametric discrete diffusion model in the limit of an infinite number of layers, we adapt the proof technique of Cai & Wang (2020) to track the sheaf Dirichlet energy as the signal is passed through multiple SCN layers.
Revised: To study this prototypical nonlinear and parametric discrete diffusion model in the limit of an infinite number of layers, we adapt the proof technique of Cai & Wang (2020) to track the sheaf Dirichlet energy as the signal is passed through multiple SCN layers.

Original: Definition 20.
Revised: Definition 20.

Original: The sheaf Dirichlet energy E F ( x ) of a signal x ∈ C 0 ( G, F ) is defined as:
Revised: The sheaf Dirichlet energy E F ( x ) of a signal x ∈ C 0 ( G, F ) is defined as:

Original: [Equation]
Revised: [Equation]

Original: Similarly, for multiple channels the energy is trace( X ⊤ ∆ F X ) .
Revised: Similarly, for multiple channels the energy is trace( X ⊤ ∆ F X ) .

Original: It is easy to see that x ∈ ker(∆ F ) ⇔ E F ( x ) = 0 .
Revised: It is easy to see that x ∈ ker(∆ F ) ⇔ E F ( x ) = 0 .

Original: Therefore, we can use the energy of a signal as a metric to measure its distance from the harmonic space.
Revised: Therefore, we can use the energy of a signal as a metric to measure its distance from the harmonic space.

Original: We begin by studying the sheaves for which the features end up asymptotically in ker(∆ F ) .
Revised: We begin by studying the sheaves for which the features end up asymptotically in ker(∆ F ) .

Original: The first such example is for O ( d ) -bundles with symmetric relations.
Revised: The first such example is for O ( d ) -bundles with symmetric relations.

Original: Let λ ∗ = max (cid:0) ( λ min − 1) 2 , ( λ max − 1) 2 (cid:1) , where λ min , λ max are the smallest and largest non-zero eigenvalues of ∆ F .
Revised: Let λ ∗ = max (cid:0) ( λ min − 1) 2 , ( λ max − 1) 2 (cid:1) , where λ min , λ max are the smallest and largest non-zero eigenvalues of ∆ F .

Original: Theorem 21.
Revised: Theorem 21.

Original: Let ( F , G ) be an O ( d ) -bundle in H d orth , sym and assume σ = ReLU or LeakyReLU.
Revised: Let ( F , G ) be an O ( d ) -bundle in H d orth , sym and assume σ = ReLU or LeakyReLU.

Original: Then E F ( Y ) ≤ λ ∗ ∥
Revised: Then E F ( Y ) ≤ λ ∗ ∥

Original: W 1 ∥ 22 ∥ W ⊤ 2 ∥ 22 E F
Revised: W 1 ∥ 22 ∥ W ⊤ 2 ∥ 22 E F

Original: ( X ) .
Revised: ( X ) .

Original: In particular, this means that if λ ∗ ∥ W 1 ∥
Revised: In particular, this means that if λ ∗ ∥ W 1 ∥

Original: 22 ∥ W ⊤ 2 ∥ 22 < 1 , the signal converges exponentially fast to ker(∆ F ) .
Revised: 22 ∥ W ⊤ 2 ∥ 22 < 1 , the signal converges exponentially fast to ker(∆ F ) .

Original: In some sense, this is not surprising because when we have symmetric relations along all edges (i.e. F v ≤ e = F u ≤ e ) and hence the conditions in Theorem 21 are satisfied, then the harmonic space ker(∆ F ) contain the same information as the kernel of the classical normalised Laplacian ∆ 0 .
Revised: In some sense, this is not surprising because when we have symmetric relations along all edges (i.e. F v ≤ e = F u ≤ e ) and hence the conditions in Theorem 21 are satisfied, then the harmonic space ker(∆ F ) contain the same information as the kernel of the classical normalised Laplacian ∆ 0 .

Original: Proposition 22.
Revised: Proposition 22.

Original: If F is an O ( d ) -bundle in H d orth , sym , then x ∈ ker∆ F if and only if x k ∈ ker ∆for all 1 ≤ k ≤ d .
Revised: If F is an O ( d ) -bundle in H d orth , sym , then x ∈ ker∆ F if and only if x k ∈ ker ∆for all 1 ≤ k ≤ d .

Original: Importantly, the symmetry of the relations along all edges is a necessary condition in Theorem 21.
Revised: Importantly, the symmetry of the relations along all edges is a necessary condition in Theorem 21.

Original: As soon as we have an asymmetric transport map, we can find an arbitrarily small linear transformation W that increases the energy.
Revised: As soon as we have an asymmetric transport map, we can find an arbitrarily small linear transformation W that increases the energy.

Original: Proposition 23.
Revised: Proposition 23.

Original: For any connected graph G and ε > 0 , there exist a sheaf ( G, F ) / ∈ H d sym , W with ∥ W ∥ 2 < ε and feature vector x such that E F
Revised: For any connected graph G and ε > 0 , there exist a sheaf ( G, F ) / ∈ H d sym , W with ∥ W ∥ 2 < ε and feature vector x such that E F

Original: (( I ⊗ W ) x ) > E F ( x ) .
Revised: (( I ⊗ W ) x ) > E F ( x ) .

Original: Beyond O ( d ) -bundles, we also have the following result for general sheaves with stalks of dimension d = 1 which generalises that of Cai & Wang (2020);
Revised: Beyond O ( d ) -bundles, we also have the following result for general sheaves with stalks of dimension d = 1 which generalises that of Cai & Wang (2020);

Original: Oono & Suzuki (2019) for GCNs:
Revised: Oono & Suzuki (2019) for GCNs:

Original: Theorem 24.
Revised: Theorem 24.

Original: Let ( F , G ) be a sheaf in H 1+ and assume σ is ReLU or LeakyReLU.
Revised: Let ( F , G ) be a sheaf in H 1+ and assume σ is ReLU or LeakyReLU.

Original: Then E F ( Y ) ≤ λ ∗ ∥
Revised: Then E F ( Y ) ≤ λ ∗ ∥

Original: W 1 ∥ 22 ∥ W ⊤ 2 ∥ 22 E F
Revised: W 1 ∥ 22 ∥ W ⊤ 2 ∥ 22 E F

Original: ( X ) .
Revised: ( X ) .

Original: As before, having positively-signed relations is a necessary condition in the non-linear case to ensure oversmoothing happens.
Revised: As before, having positively-signed relations is a necessary condition in the non-linear case to ensure oversmoothing happens.

Original: However, in this case, the proof also holds for negatively-signed relations when using a linear model (i.e. when σ is the identity).
Revised: However, in this case, the proof also holds for negatively-signed relations when using a linear model (i.e. when σ is the identity).

Original: Due to this result, we note that Propositons 8 and 11 also generalise to GCNs.
Revised: Due to this result, we note that Propositons 8 and 11 also generalise to GCNs.

Original: Any GCN using a weighted graph Laplacian that oversmooths as in Theorem 24 cannot linearly separate more than two classes.
Revised: Any GCN using a weighted graph Laplacian that oversmooths as in Theorem 24 cannot linearly separate more than two classes.

Original: Furthermore, such a GCN cannot separate the classes of a bipartitie graph with equally-sized partitions.
Revised: Furthermore, such a GCN cannot separate the classes of a bipartitie graph with equally-sized partitions.

Original: We have two main takeaways: (1) Discrete sheaf diffusion models can avoid (in general) the kernel of the diffusion operator even in the deep linear case.
Revised: We have two main takeaways: (1) Discrete sheaf diffusion models can avoid (in general) the kernel of the diffusion operator even in the deep linear case.

Original: The asymmetry of the transport maps play (again) an important role in that.
Revised: The asymmetry of the transport maps play (again) an important role in that.

Original: D S HEAF L EARNING
Revised: C S HEAF L EARNING

Original: The advantage of learning a sheaf is that one does not require any sort of embedding of the nodes in an ambient space (as e.g. done in Chamberlain et al.
Revised: The advantage of learning a sheaf is that one does not require any sort of embedding of the nodes in an ambient space (as e.g. done in Chamberlain et al.

Original: (2021a)).
Revised: (2021a)).

Original: Instead, everything can be learned locally .
Revised: Instead, everything can be learned locally .

Original: Each d × d matrix F v ⊴ e is learned via a parametric function Φ : R d × 2 → R d × d:
Revised: Each d × d matrix F v ⊴ e is learned via a parametric function Φ : R d × 2 → [Equation] d × d

Original: [Equation]
Revised: [Equation]

Original: For simplicity, the equation above uses a single feature channel, but in practice, all channels are supplied as input.
Revised: For simplicity, the equation above uses a single feature channel, but in practice, all channels are supplied as input.

Original: This function retains the inductive bias of locality specific to GNNs since it only utilises the features of the nodes forming the edge.
Revised: This function retains the inductive bias of locality specific to GNNs since it only utilises the features of the nodes forming the edge.

Original: At the same time, it is important that this function is non-symmetric in order to be able learn asymmetric transport maps along each edge.
Revised: At the same time, it is important that this function is non-symmetric in order to be able learn asymmetric transport maps along each edge.

Original: In what follows, we distinguish between several types of functions Φ depending on the type of matrix they learn.
Revised: In what follows, we distinguish between several types of functions Φ depending on the type of matrix they learn.

Original: Diagonal The main advantage of this parametrization is that fewer parameters need to be learned per edge and the sheaf Laplacian ends up being a matrix with diagonal blocks, which also results in fewer operations in sparse matrix multiplications.
Revised: Diagonal The main advantage of this parametrization is that fewer parameters need to be learned per edge and the sheaf Laplacian ends up being a matrix with diagonal blocks, which also results in fewer operations in sparse matrix multiplications.

Original: The main disadvantage is that the d dimensions of the stalks do not interact.
Revised: The main disadvantage is that the d dimensions of the stalks do not interact.

Original: Orthogonal
Revised: Orthogonal

Original: In this case, the model effectively learns a discrete vector bundle.
Revised: In this case, the model effectively learns a discrete vector bundle.

Original: Orthogonal matrices provide several advantages: (1) they are able to mix the various dimension of the stalks, (2) the orthogonality constraint prevents overfitting while reducing the number of parameters, (3) they have better understood theoretical properties, and (4) the resulting Laplacians are easier to normalise numerically since the diagonal entries correspond to the degrees of the nodes.
Revised: Orthogonal matrices provide several advantages: (1) they are able to mix the various dimension of the stalks, (2) the orthogonality constraint prevents overfitting while reducing the number of parameters, (3) they have better understood theoretical properties, and (4) the resulting Laplacians are easier to normalise numerically since the diagonal entries correspond to the degrees of the nodes.

Original: In our model, we build orthogonal matrices from a composition of Householder reflections (Mhammedi et al., 2017).
Revised: In our model, we build orthogonal matrices from a composition of Householder reflections (Mhammedi et al., 2017).

Original: General Finally, we consider the most general option of learning arbitrary matrices.
Revised: General Finally, we consider the most general option of learning arbitrary matrices.

Original: The maximal flexibility provided by these maps can be useful, but it also comes with the danger of overfitting.
Revised: The maximal flexibility provided by these maps can be useful, but it also comes with the danger of overfitting.

Original: At the same time, the sheaf Laplacian is more challenging to normalise numerically since one has to compute D − 1 / 2 for a positive semi-definite matrix D .
Revised: At the same time, the sheaf Laplacian is more challenging to normalise numerically since one has to compute D − 1 / 2 for a positive semi-definite matrix D .

Original: To perform this at scale, one has to rely on
Revised: To perform this at scale, one has to rely on

Original: SVD, whose gradients can be infinite if D has repeated eigenvalues.
Revised: SVD, whose gradients can be infinite if D has repeated eigenvalues.

Original: Therefore, this model is more challenging to train.
Revised: Therefore, this model is more challenging to train.

Original: Computational Complexity Treating the number of channels and layers as constant, a typical message passing GNN scales with O ( n + m ) , where n is the number of nodes and m is the number of edges.
Revised: Computational Complexity Treating the number of channels and layers as constant, a typical message passing GNN scales with O ( n + m ) , where n is the number of nodes and m is the number of edges.

Original: A sheaf diffusion model with diagonal maps has complexity O
Revised: A sheaf diffusion model with diagonal maps has complexity O

Original: ( d ( n + m ))
Revised: ( d ( n + m ))

Original: because the matrix multiplication required to compute the transport maps reduces to a scalar multiplication.
Revised: because the matrix multiplication required to compute the transport maps reduces to a scalar multiplication.

Original: When learning orthogonal or general d × d matrices, the complexity becomes O ( d 3 ( n + m )) because matrix multiplication is O ( d 3 ) .
Revised: When learning orthogonal or general d × d matrices, the complexity becomes O ( d 3 ( n + m )) because matrix multiplication is O ( d 3 ) .

Original: We note that additional costs for the latter methods such as that of
Revised: We note that additional costs for the latter methods such as that of

Original: SVD for computing D − 1 / 2 for general matrices and that of parametrising the orthogonal group can be done in O ( d 3 ) , so the overall complexity is not affected.
Revised: SVD for computing D − 1 / 2 for general matrices and that of parametrising the orthogonal group can be done in O ( d 3 ) , so the overall complexity is not affected.

Original: In practice, we use 1 ≤ d ≤ 5 and all the operations above benefit from batched GPU computations in PyTorch (Paszke et al., 2019) which effectively results in a constant overhead.
Revised: In practice, we use 1 ≤ d ≤ 5 and all the operations above benefit from batched GPU computations in PyTorch (Paszke et al., 2019) which effectively results in a constant overhead.

Original: For learning orthogonal matrices, we rely on the library Torch Householder (Obukhov, 2021) which provides support for fast transformations with very large batch sizes.
Revised: For learning orthogonal matrices, we rely on the library Torch Householder (Obukhov, 2021) which provides support for fast transformations with very large batch sizes.

Original: E A DDITIONAL E XPERIMENTS E. S YNTHETIC EXPERIMENTS
Revised: D A DDITIONAL E XPERIMENTS D. S YNTHETIC EXPERIMENTS

Original: Opinion polarisation (1D)
Revised: Opinion polarisation (1D)

Original: We first consider a simple synthetic setup given by a connected bipartite graph, where the two partitions form two equally sized classes.
Revised: We first consider a simple synthetic setup given by a connected bipartite graph, where the two partitions form two equally sized classes.

Original: We sample the features from two overlapping isotropic Gaussian distributions in order to make the classes linearly non-separable at initialisation time.
Revised: We sample the features from two overlapping isotropic Gaussian distributions in order to make the classes linearly non-separable at initialisation time.

Original: From Proposition 8 we know that diffusion models using symmetric restriction maps cannot separate the classes in the limit, while a diffusion process using asymmetric maps should be able to.
Revised: From Proposition 8 we know that diffusion models using symmetric restriction maps cannot separate the classes in the limit, while a diffusion process using asymmetric maps should be able to.

Original: Therefore, we consider two simple versions of the model from Equation 2, where we set W 1 = I d ,
Revised: Therefore, we consider two simple versions of the model from Equation 2, where we set W 1 = I d ,

Original: W 

Revised: W

Original: =
Revised: =

Original: I f and σ = id .
Revised: I f and σ = id .

Original: In the first, the maps F v ⊴ e =( v,u ) are learned by a simple layer of the form σ ( w ⊤ [ x v || x u ]) , where σ = tanh .
Revised: In the first, the maps F v ⊴ e =( v,u ) are learned by a simple layer of the form σ ( w ⊤ [ x v || x u ]) , where σ = tanh .

Original: For the second model, we use a similar layer but constraint
Revised: For the second model, we use a similar layer but constraint

Original: F v ⊴ e = F u ⊴ e > 0 , which results in a weighted graph Laplacian.
Revised: F v ⊴ e = F u ⊴ e > 0 , which results in a weighted graph Laplacian.

Original: Figure 3 presents the results of this experiment across five seeds.
Revised: Figure 3 presents the results of this experiment across five seeds.

Original: As expected, for diffusion time zero (i.e. no diffusion)
Revised: As expected, for diffusion time zero (i.e. no diffusion)

Original: , we see that a linear classifier cannot separate the classes.
Revised: , we see that a linear classifier cannot separate the classes.

Original: Also as expected, the diffusion process using symmetric maps cannot perfectly fit the data.
Revised: Also as expected, the diffusion process using symmetric maps cannot perfectly fit the data.

Original: In contrast, with the more general sheaf diffusion, as more diffusion is performed and the signal approaches the harmonic space, the model gets better and the features become linearly separable.
Revised: In contrast, with the more general sheaf diffusion, as more diffusion is performed and the signal approaches the harmonic space, the model gets better and the features become linearly separable.

Original: In the second subfigure we take a closer look at the sheaf that the model learns in the time limit by plotting a histogram of all the transport (scalar) maps F ⊤ v ⊴ e F u ⊴ e .
Revised: In the second subfigure we take a closer look at the sheaf that the model learns in the time limit by plotting a histogram of all the transport (scalar) maps F ⊤ v ⊴ e F u ⊴ e .

Original: In accordance with Propositions 10 the model learns a negative transport map for all edges.
Revised: In accordance with Propositions 10 the model learns a negative transport map for all edges.

Original: Two dimensional synthetic experiment In the main text we focused on a synthetic example involving sheaves with one-dimensional stalks.
Revised: Two dimensional synthetic experiment In the main text we focused on a synthetic example involving sheaves with one-dimensional stalks.

Original: We now consider a graph with three classes and two-dimensional features, with edge homophily level 0 .
Revised: We now consider a graph with three classes and two-dimensional features, with edge homophily level 0 .

Original: We use 80% of the nodes for training  and 20% for testing.
Revised: We use 80% of the nodes for training [Figure] [Table] and 20% for testing.

Original: First, we know that a discrete vector bundle with two-dimensional stalks that can solve the task in the limit exists from Theorem 15, while based on Proposition 11 no sheaf with one-dimensional stalks can perfectly solve the tasks.
Revised: First, we know that a discrete vector bundle with two-dimensional stalks that can solve the task in the limit exists from Theorem 15, while based on Proposition 11 no sheaf with one-dimensional stalks can perfectly solve the tasks.

Original: Therefore, similarly to the synthetic experiment in the main text, we compare two similar models learning the sheaf from data: one using 1 D stalks and another using 2 D stalks.
Revised: Therefore, similarly to the synthetic experiment in the main text, we compare two similar models learning the sheaf from data: one using 1 D stalks and another using 2 D stalks.

Original: As we see from
Revised: As we see from

Original: Figure 4, the discrete vector bundle model has better training and test-time performance than the one-dimensional counterpart.
Revised: Figure 4, the discrete vector bundle model has better training and test-time performance than the one-dimensional counterpart.

Original: Nonetheless, none of the two models manages to match the perfect[Figure] [Table] performance of the ideal sheaf on this more challenging dataset.
Revised: Nonetheless, none of the two models manages to match the perfect performance of the ideal sheaf on this more challenging dataset.

Original: From the final subfigure we also see that the model learns to rotate more across the heterophilic edges in order to push away the nodes belonging to other classes.
Revised: From the final subfigure we also see that the model learns to rotate more across the heterophilic edges in order to push away the nodes belonging to other classes.

Original: The prevalent angle of this rotation is 2 radians, which is just under 120 ◦ = 360 ◦ /C , where C = 3 is the number of classes.
Revised: The prevalent angle of this rotation is 2 radians, which is just under 120 ◦ = 360 ◦ /C , where C = 3 is the number of classes.

Original: Thus the model learns to position the three classes at approximately equal arc-lengths from each other for maximum linear separability.
Revised: Thus the model learns to position the three classes at approximately equal arc-lengths from each other for maximum linear separability.

Original: Positional encoding ablation Based on the Remark from the previous section, we proceed to analyse the impact of increasing the expressive power of the model by making the nodes more distinguishable.
Revised: Positional encoding ablation Based on the Remark from the previous section, we proceed to analyse the impact of increasing the expressive power of the model by making the nodes more distinguishable.

Original: For that, we equip our datasets with additional features consisting of graph Laplacian positional encodings as originally done in Dwivedi et al.
Revised: For that, we equip our datasets with additional features consisting of graph Laplacian positional encodings as originally done in Dwivedi et al.

Original: In Table 2 we see that positional encodings do indeed improve the performance of the continuous models compared to the numbers reported in the main table.
Revised: In Table 2 we see that positional encodings do indeed improve the performance of the continuous models compared to the numbers reported in the main table.

Original: Therefore, we conclude that the interaction between the problem of sheaf learning and that of the expressivity of graph neural networks represents a promising avenue of future research.
Revised: Therefore, we conclude that the interaction between the problem of sheaf learning and that of the expressivity of graph neural networks represents a promising avenue of future research.

Original: Visualising diffusion To develop a better intuition of the limiting behaviour of sheaf diffusion for node classification tasks we plot the diffusion process using an oracle discrete vector bundle for two graph with C = 3 (Figure 6) and C = 4 (Figure 5) classes.
Revised: Visualising diffusion To develop a better intuition of the limiting behaviour of sheaf diffusion for node classification tasks we plot the diffusion process using an oracle discrete vector bundle for two graph with C = 3 (Figure 6) and C = 4 (Figure 5) classes.

Original: The diffusion processes converges in the limit to a configuration where the classes are rotated at 2 πC form each other, just like in the cartoon diagrams of Figure 8.
Revised: The diffusion processes converges in the limit to a configuration where the classes are rotated at 2 πC form each other, just like in the cartoon diagrams of Figure 8.

Original: Note that in all cases, the classes are linearly separable in the limit.
Revised: Note that in all cases, the classes are linearly separable in the limit.

Original: We note that this approach generalises to any number of classes, but beyond C = 4 it is not guaranteed that they will be linearly separable in 2 D .
Revised: We note that this approach generalises to any number of classes, but beyond C = 4 it is not guaranteed that they will be linearly separable in 2 D .

Original: However, they are still well-separated.
Revised: However, they are still well-separated.

Original: We include an example with C = 10 classes in Figure 7.
Revised: We include an example with C = 10 classes in Figure 7.

Original: E. A DDITIONAL DETAILS ON REAL - WORLD EXPERIMENTS
Revised: D. A DDITIONAL DETAILS ON REAL - WORLD EXPERIMENTS

Original: Baselines for real-world experiments In Table 1 we evaluate against an ample set of GNN models that can be placed in four categories: (1) classical: GCN (Kipf & Welling, 2017), GAT (Veli ˇ ckovi ´ c et al., 2018), GraphSAGE (Hamilton et al., 2017); (2) models specifically designed for heterophilic settings: GGCN (Yan et al., 2021), Geom-GCN (Pei et al., 2020), H2GCN (Zhu et al., 2020),
Revised: Baselines for real-world experiments In Table 1 we evaluate against an ample set of GNN models that can be placed in four categories: (1) classical: GCN (Kipf & Welling, 2017), GAT (Veliˇckovi´c et al., 2018), GraphSAGE (Hamilton et al., 2017); (2) models specifically designed for heterophilic settings: GGCN (Yan et al., 2021), Geom-GCN (Pei et al., 2020), H2GCN (Zhu et al., 2020),

Original: GPRGNN (Chien et al., 2021), FAGCN (Bo et al., 2021), MixHop (Abu-El-Haija et al., 2019); (3) models addressing oversmoothing: GCNII (Chen et al., 2020), PairNorm (Zhao & Akoglu, 2020); (4) continuous models: CGNNs (Xhonneux et al., 2020), GRAND (Chamberlain et al., 2021b), and
Revised: GPRGNN (Chien et al., 2021), FAGCN (Bo et al., 2021), MixHop (Abu-El-Haija et al., 2019); (3) models addressing oversmoothing: GCNII (Chen et al., 2020), PairNorm (Zhao & Akoglu, 2020); (4) continuous models: CGNNs (Xhonneux et al., 2020), GRAND (Chamberlain et al., 2021b), and

Original: BLEND (Chamberlain et al., 2021a).
Revised: BLEND (Chamberlain et al., 2021a).

Original: For (4) we fine-tune and evaluate the models ourselves.
Revised: For (4) we fine-tune and evaluate the models ourselves.

Original: The other results are taken from Yan et al.
Revised: The other results are taken from Yan et al.

Original: (2021), except for FAGCN and MixHop, which come from
Revised: (2021), except for FAGCN and MixHop, which come from

Original: Lingam et al.
Revised: Lingam et al.

Original: (2021) and Zhu et al.
Revised: (2021) and Zhu et al.

Original: (2020), respectively.
Revised: (2020), respectively.

Original: All of these were evaluated on the same set of splits as ours.
Revised: All of these were evaluated on the same set of splits as ours.

Original: F A DDITIONAL MODEL DETAILS AND HYPERPARAMETERS
Revised: E A DDITIONAL MODEL DETAILS AND HYPERPARAMETERS

Original: Hybrid transport maps Consider the transport maps −F ⊤ v ⊴ e F u ⊴ e appearing in the off the diagonal entires of the sheaf Laplacian L F .
Revised: Hybrid transport maps Consider the transport maps −F ⊤ v ⊴ e F u ⊴ e appearing in the off the diagonal entires of the sheaf Laplacian L F .

Original: When learning a sheaf Laplacian there exists the risk that the features are not sufficiently good in the early layers (or in general) and, therefore, it might be useful to consider a hybrid transport map of the form −F ⊤ v ⊴ e F u ⊴ e (cid:76) F , where (cid:76) is the direct sum of two matrices and F represents a fixed (non-learnable map).
Revised: When learning a sheaf Laplacian there exists the risk that the features are not sufficiently good in the early layers (or in general) and, therefore, it might be useful to consider a hybrid transport map of the form −F ⊤ v ⊴ e F u ⊴ e (cid:76) F , where (cid:76) is the direct sum of two matrices and F represents a fixed (non-learnable map).

Original: In particular we consider maps of the form −F ⊤ v ⊴ e F u ⊴ e (cid:76) I 1 (cid:76) − I 1 which essentially appends a diagonal matrix with 1 and − 1 on the diagonal to the learned matrix.
Revised: In particular we consider maps of the form −F ⊤ v ⊴ e F u ⊴ e (cid:76) I 1 (cid:76) − I 1 which essentially appends a diagonal matrix with 1 and − 1 on the diagonal to the learned matrix.

Original: From a signal processing perspective, these correspond to a low-pass and a high-pass filter that could produce generally useful features.
Revised: From a signal processing perspective, these correspond to a low-pass and a high-pass filter that could produce generally useful features.

Original: We treat the addition of these fixed parts as an additional hyper-parameter.
Revised: We treat the addition of these fixed parts as an additional hyper-parameter.

Original: Adjusting the activation magnitudes We note that in practice we find it useful to learn an additional parameter ε ∈ [ − 1 , 1] d (i.e. a vector of size d ) in the discrete version of the models:
Revised: Adjusting the activation magnitudes We note that in practice we find it useful to learn an additional parameter ε ∈ [ − [Equation] [Equation] [Equation]

Revised: (i.e. a vector of size d ) in the discrete version of the models:

Original: [Equation]
Revised: [Equation]

Original: This allows the model to adjust the relative magnitude of the features in each stalk dimension.
Revised: This allows the model to adjust the relative magnitude of the features in each stalk dimension.

Original: This is used across all of our experiments in the discrete models.
Revised: This is used across all of our experiments in the discrete models.

Original: Augmented normalised sheaf Laplacian Similarly to GCN which normalises the Laplacian by the augmented degrees (i.e. ( D + I n ) − 1 / 2 , where D is the usual diagonal matrix of node degrees), we similarly use ( D + I nd ) − 1 / 2 for normalisation to obtain greater numerical stability.
Revised: Augmented normalised sheaf Laplacian Similarly to GCN which normalises the Laplacian by the augmented degrees (i.e. ( D + I n ) − 1 / 2 , where D is the usual diagonal matrix of node degrees), we similarly use ( D + I nd ) − 1 / 2 for normalisation to obtain greater numerical stability.

Original: This is particularly helpful when learning general sheaves as it increases the numerical stability of SVD.
Revised: This is particularly helpful when learning general sheaves as it increases the numerical stability of SVD.

Original: Learning sheaves and the k -WL test According to our theoretical results, one should learn a pair of non-equal restriction maps along the edges between two classes.
Revised: Learning sheaves and the k -WL test According to our theoretical results, one should learn a pair of non-equal restriction maps along the edges between two classes.

Original: Suppose this is done via a local functions F v ⊴ e :=( v,u ) = ϕ
Revised: Suppose this is done via a local functions F v ⊴ e :=( v,u ) = ϕ

Original: ( x v , x u ) .
Revised: ( x v , x u ) .

Original: Then, ϕ ( x v , x u ) ̸
Revised: Then, ϕ ( x v , x u ) ̸

Original: = ϕ
Revised: = ϕ

Original: ( x u , x v ) only if x u ̸ = x v .
Revised: ( x u , x v ) only if x u ̸ = x v .

Original: This leads to the following remark.
Revised: This leads to the following remark.

Original: Remark.
Revised: Remark.

Original: Let G be a graph with an initial colouring x such that ∃ ( v, u ) ∈ E and v and u are not k -WL distinguishable.
Revised: Let G be a graph with an initial colouring x such that ∃ ( v, u ) ∈ E and v and u are not k -WL distinguishable.

Original: Then no model upper bounded by k -WL can learn an asymmetric relation along the edge ( v, u ) .
Revised: Then no model upper bounded by k -WL can learn an asymmetric relation along the edge ( v, u ) .

Original: This observation motivates our decision to evolve the geometry at each layer of the model since at initialisation time many nodes might not be distinguishable.
Revised: This observation motivates our decision to evolve the geometry at each layer of the model since at initialisation time many nodes might not be distinguishable.

Original: At the same time, it suggests that provably expressive architectures might be able to learn better sheaves.
Revised: At the same time, it suggests that provably expressive architectures might be able to learn better sheaves.

Original: Hyperparameters for discrete models For the discrete models we searched in the following hyperparameters:
Revised: Hyperparameters for discrete models For the discrete models we searched in the following hyperparameters:

Original: • Hidden channels: [8 , 16 , 32] for WebKB datasets and [8 , 16 , 32 , 64] for all the other datasets.
Revised: • Hidden channels: [8 , 16 , 32] for WebKB datasets and [8 , 16 , 32 , 64] for all the other datasets.

Original: • d : [1 , 2 , 3 , 4 , 5] • Layers: [2 , 3 , 4 , 5 , 6 , 7 , 8] • Learning rate: 0 .
Revised: • d : [1 , 2 , 3 , 4 , 5] • Layers: [2 , 3 , 4 , 5 , 6 , 7 , 8] • Learning rate: 0 .

Original: 02 for the WebKB datasets and 0 .
Revised: 02 for the WebKB datasets and 0 .

Original: 01 for all the other datasets.
Revised: 01 for all the other datasets.

Original: • Weight decay for the regular model parameters: searched in a log-uniform range over [ − 4 .
Revised: • Weight decay for the regular model parameters: searched in a log-uniform range over [ − 4 .

Original: 5 , 11 . 0] • Weight decay for the parameters learning the sheaf: searched in a log-uniform range over [ − 4 .
Revised: 5 , 11 . 0] • Weight decay for the parameters learning the sheaf: searched in a log-uniform range over [ − 4 .

Original: 5 , 11 . 0] • Dropout on the inputs: searched uniformly over [0 , 0 . 9] .
Revised: 5 , 11 . 0] • Dropout on the inputs: searched uniformly over [0 , 0 . 9] .

Original: • Dropout for the other layers: searched uniformly over [0 , 0 . 9] .
Revised: • Dropout for the other layers: searched uniformly over [0 , 0 . 9] .

Original: • Early stopping patience: 100 epochs for the Wiki datasets and 200 for the others.
Revised: • Early stopping patience: 100 epochs for the Wiki datasets and 200 for the others.

Original: • Maximum training epochs: 1000 epochs for the Wiki datasets and 500 for the others.
Revised: • Maximum training epochs: 1000 epochs for the Wiki datasets and 500 for the others.

Original: Hyperparameters for continous models For the continous models we searched over the following hyperparameters:
Revised: Hyperparameters for continous models For the continous models we searched over the following hyperparameters:

Original: • Hidden channels: [8 , 16 , 32 , 64] • d : [1 , 2 , 3 , 4 , 5] • Learning rate: searched in a log-uniform range over [0 . 01 , 0 . 1] • Weight decay : searched in a log-uniform range over [ − 6 . 9 , 13 . 8] • Dropout: searched uniformly in [0 , 0 . 95] .
Revised: • Hidden channels: [8 , 16 , 32 , 64] • d : [1 , 2 , 3 , 4 , 5] • Learning rate: searched in a log-uniform range over [0 . 01 , 0 . 1] • Weight decay : searched in a log-uniform range over [ − 6 . 9 , 13 . 8] • Dropout: searched uniformly in [0 , 0 . 95] .

Original: • ODE Integration method: euler.
Revised: • ODE Integration method: euler.

Original: • Maximum integration time: searched uniformly over [1 . 0 , 9 . 0] .
Revised: • Maximum integration time: searched uniformly over [1 . 0 , 9 . 0] .

Original: • Training epochs: 50.
Revised: • Training epochs: 50.

Original: We train all models using the Adam optimiser (Kingma & Ba, 2015).
Revised: We train all models using the Adam optimiser (Kingma & Ba, 2015).

Original: All models use ELU activations.
Revised: All models use ELU activations.

Original: G R ELATED W ORK , D ISCUSSION AND C ONCLUSION
Revised: F L IMITATIONS

Original: Sheaf Neural Networks Sheaf Neural Networks with a fixed sheaf Laplacian were originally introduced by Hansen & Gebhart (2020) in a toy experimental setting. Since then, they have remained completely unexplored. We hope that this paper will fill this lacuna.

Original: In contrast to their work, we provide an ample theoretical analysis justifying the use of sheaves in Graph ML.

Original: On the practical side, we study for the first time how sheaves can be learned from data using neural networks.

Original: Finally, this is the first successful application of sheaf-based neural networks in a real-world setting.

Original: Limitations One of the main limitations of our theoretical analysis is that it does not discuss the learnability and generalisation properties of sheaves, only their existence.
Revised: Limitations One of the main limitations of our theoretical analysis is that it does not discuss the learnability and generalisation properties of sheaves, only their existence.

Original: Nonetheless, this setting was sufficient to produce many valuable insights about heterophily and oversmoothing and a basic understanding of what various types of sheaves can do.
Revised: Nonetheless, this setting was sufficient to produce many valuable insights about heterophily and oversmoothing and a basic understanding of what various types of sheaves can do.

Original: Much more theoretical work remains to be done in this direction, and we expect to see further cross-fertilization between ML and algebraic topology research in the future.
Revised: Much more theoretical work remains to be done in this direction, and we expect to see further cross-fertilization between ML and algebraic topology research in the future.

Original: Conclusion In this work, we used cellular sheaf theory to provide a novel topological perspective on heterophily and oversmoothing in GNNs.

Original: We showed that the underlying sheaf structure of the graph is intimately connected with both of these important factors affecting the performance of GNNs.

Original: To mitigate this, we proposed a new paradigm for graph representation learning where models do not only evolve the features at each layer but also the underlying geometry of the graph.

Original: In practice, we demonstrated that this framework can achieve state-of-the-art results in heterophilic settings.

Original: H H ARMONIC S PACE P ROOFS
Revised: G H ARMONIC S PACE P ROOFS

Original: Proof of Proposition 16 .
Revised: Proof of Proposition 16 .

Original: We first note that on a discrete O ( d ) bundle the degree operator D v =
Revised: We first note that on a discrete O ( d ) bundle the degree operator D v =

Original: d v I since by orthogonality F ⊤ v ⊴ e F v ⊴ e
Revised: d v I since by orthogonality F ⊤ v ⊴ e F v ⊴ e

Original: = I .
Revised: = I .

Original: We can use the Rayleigh quotient to characterize λ F 0 as
Revised: We can use the Rayleigh quotient to characterize λ F 0 as

Original: [Equation]
Revised: [Equation]

Original: || x ||
Revised: = min

Original: [Equation]
Revised: [Equation]

Revised: ∈ R nd ⟨ x , ∆ F x ⟩ || x ||.

Original: Fix v ∈ V and choose a minimal path γ v → u for all u ∈ V .
Revised: Fix v ∈ V and choose a minimal path γ v → u for all u ∈ V .

Original: For an arbitrary non-zero z v , consider the signal z u =
Revised: For an arbitrary non-zero z v , consider the signal z u =

Original: P γ v → u z v
Revised: P γv → u z v

Original: and we set ˜
Revised: and we set ˜

Original: z u √ d u = z u .
Revised: z u √ d u = z u .

Original: [Equation]
Revised: [Equation]

Original: where we have again used that the maps are orthogonal.
Revised: where we have again used that the maps are orthogonal.

Original: Since ( F ⊤ u ⊴ e F w ⊴ e ) P γv → w = P γ ′ v → u we find that the right hand side can be bound from above by r 2 || ˜
Revised: Since ( F ⊤ u ⊴ e F w ⊴ e ) P γv → w = P γ ′ v → u we find that the right hand side can be bound from above by r 2 || ˜

Original: z
Revised: z

Original: v || 2 .
Revised: v || 2 .

Original: Therefore, by using Definition 20 we finally obtain
Revised: Therefore, by using Definition 20 we finally obtain

Original: [Equation]
Revised: [Equation]

Original: Since the transport maps are all orthogonal we get
Revised: Since the transport maps are all orthogonal we get

Original: [Equation]
Revised: [Equation]

Original: We conclude that
Revised: We conclude that

Original: [Equation]
Revised: [Equation]

Original: Proof of Proposition 17 .
Revised: Proof of Proposition 17 .

Original: Assume that x ∈ H 

Revised: Assume that x ∈ H

Original: ( G, F ) and consider v ∈ V and any cycle based at v denoted by γ v → v = ( v 0 =
Revised: ( G, F ) and consider v ∈ V and any cycle based at v denoted by γ v → v = ( v 0 =

Original: v, v 1 , . .
Revised: v, v 1 , . .

Original: . , v L = v ) .
Revised: . , v L = v ) .

Original: We have that
Revised: We have that

Original: [Equation]
Revised: [Equation]

Original: By composing all the maps we find:
Revised: By composing all the maps we find:

Original: [Equation]
Revised: [Equation]

Original: which completes the proof.
Revised: which completes the proof.

Original: Proof of Proposition 18 .
Revised: Proof of Proposition 18 .

Original: If ϵ = 0 there is nothing to prove.
Revised: If ϵ = 0 there is nothing to prove.

Original: Assume that ϵ > 0 .
Revised: Assume that ϵ > 0 .

Original: By Proposition 17 we derive that the harmonic space is trivial and hence λ F 0 > 0 .
Revised: By Proposition 17 we derive that the harmonic space is trivial and hence λ F 0 > 0 .

Original: Consider a unit eigenvector x ∈ ker(∆ F − λ F 0 I ) and let v ∈ V such that || x v || ≥
Revised: Consider a unit eigenvector x ∈ ker(∆ F − λ F 0 I ) and let v ∈ V such that || x v || ≥

Original: || x
Revised: || x

Original: u || for u ̸ = v .
Revised: u || for u ̸ = v .

Original: There exists a cycle γ based at v such that P γv x v ̸ = x v for otherwise we could extend x v ̸ = 0 to any other node independently of the path choice and hence find a non-trivial harmonic signal. 
Revised: There exists a cycle γ based at v such that P γv x v ̸ = x v for otherwise we could extend x v ̸ = 0 to any other node independently of the path choice and hence find a non-trivial harmonic signal. In particular

Original: In particular, we can assume this cycle to be non-degenerate, otherwise if there existed a non-trivial degenerate loop contained in γ that does not fix x we could consider this loop instead of γ for our argument.
Revised: [Equation] we can assume this cycle to be non-degenerate, otherwise if there existed a non-trivial degenerate loop contained in γ that does not fix x we could consider this loop instead of γ for our argument.

Original: Let us write this path as ( v 0 = v, v 1 , . .
Revised: Let us write this path as ( v 0 = v, v 1 , . .

Original: . , v L = v ) and consider the rescaled signal ˜ x v √ d v = x v .
Revised: . , v L = v ) and consider the rescaled signal ˜ x v √ d v = x v .

Original: By assumption we have
Revised: By assumption we have

Original: [Equation]
Revised: [Equation]

Original: By iterating the approach above we find:
Revised: By iterating the approach above we find:

Original: [Equation]
Revised: [Equation]

Original: From Definition 20 we derive that the last term can be bounded from above by (cid:112) 2 LE F ( x )
Revised: From Definition 20 we derive that the last term can be bounded from above by (cid:112) 2 LE F ( x )

Original: = (cid:112) 2 L ⟨ x , ∆ F x ⟩ .
Revised: = (cid:112) 2 L ⟨ x , ∆ F x ⟩ .

Original: Therefore, we conclude:
Revised: Therefore, we conclude:

Original: [Equation]
Revised: [Equation]

Original: By construction we get || x v || ≥ 1 / √ n [Equation] meaning that
Revised: By construction we get || x v || ≥ 1 / √ n , meaning that

Original: [Equation]
Revised: [Equation]

Original: Proof of Lemma 19 .
Revised: Proof of Lemma 19 .

Original: We first note that the argument below extends to weighted O ( d ) -bundles as well.
Revised: We first note that the argument below extends to weighted O ( d ) -bundles as well.

Original: Let x ∈ H 

Revised: Let x ∈ H

Original: ( G, F ) .
Revised: ( G, F ) .

Original: According to Proposition 17, given v, u ∈ V , we see that x u =
Revised: According to Proposition 17, given v, u ∈ V , we see that x u =

Original: P γ v → u x v for any path γ v → u .
Revised: P γv → u x v for any path γ v → u .

Original: It means that the harmonic space is uniquely determined by the choice of x v ∈ F ( v ) .
Revised: It means that the harmonic space is uniquely determined by the choice of x v ∈ F ( v ) .

Original: Explicitly, given any cycle γ based at v , we know that x v ∈ ker( P γv → v − I ) .
Revised: Explicitly, given any cycle γ based at v , we know that x v ∈ ker( P γv → v − I ) .

Original: If the transport is everywhere path-independent, then the kernel coincides with the whole stalk F ( v ) and hence we can extend any basis { x v i } ∈ F ( v ) ∼ = R d to a basis in H 0 ( G, F ) via the transport maps, i.e. dim( H 0 ( G, F ))
Revised: If the transport is everywhere path-independent, then the kernel coincides with the whole stalk F ( v ) and hence we can extend any basis { x v i } ∈ F ( v ) ∼ = R d to a basis in H 0 ( G, F ) via the transport maps, i.e. dim( H 0 ( G, F ))

Original: =
Revised: =

Original: d .
Revised: d .

Original: If instead there exists a transport map over a cycle γ v → v with non-trivial fixed points, then ker( P γv → v − I ) <
Revised: If instead there exists a transport map over a cycle γ v → v with non-trivial fixed points, then ker( P γv → v − I ) <

Original: F ( v ) ∼ = R d
Revised: F ( v ) ∼ = R d

Original: and hence dim( H 0 ( G, F )) <
Revised: and hence dim( H 0 ( G, F )) <

Original: d .
Revised: d .

Original: I P ROOFS FOR THE P OWER OF S HEAF D IFFUSION
Revised: H P ROOFS FOR THE P OWER OF S HEAF D IFFUSION

Original: Definition 25.
Revised: Definition 25.

Original: Let G =
Revised: Let G =

Original: ( V, W ) be a weighted graph, where W is a matrix with w vu =
Revised: ( V, W ) be a weighted graph, where W is a matrix with w vu =

Original: w uv ≥for all v ̸ = u ∈ V , w vv = 0 for all v ∈ V , and ( v, u ) is an edge if and only if w vu > 0 .
Revised: w uv ≥for all v ̸ = u ∈ V , w vv = 0 for all v ∈ V , and ( v, u ) is an edge if and only if w vu > 0 .

Original: The graph Laplacian of a weighted graph is L = D − W , where D is the diagonal matrix of weighted degrees (i.e. d v = (cid:80) u w vu ).
Revised: The graph Laplacian of a weighted graph is L = D − W , where D is the diagonal matrix of weighted degrees (i.e. d v = (cid:80) u w vu ).

Original: Its normalised version is (cid:101)
Revised: Its normalised version is (cid:101)

Original: L
Revised: L

Original: = D − 1 / 2 LD − 1 / 2 .
Revised: = D − 1 / 2 LD − 1 / 2 .

Original: Proposition 26.
Revised: Proposition 26.

Original: Let G be a graph.
Revised: Let G be a graph.

Original: The set { ∆ F | ( G, F ) ∈ H 1sym } is isomorphic to the set of all possible weighted graph Laplacians over G .
Revised: The set { ∆ F | ( G, F ) ∈ H 1 sym } is isomorphic to the set of all possible weighted graph Laplacians over G .

Original: Proof.
Revised: Proof.

Original: We prove only one direction.
Revised: We prove only one direction.

Original: Let W be a choice of valid weight matrix for the graph G .
Revised: Let W be a choice of valid weight matrix for the graph G .

Original: We can construct a sheaf
Revised: We can construct a sheaf

Original: ( G, F ) ∈
Revised: ( G, F ) ∈

Original: H 1sym such that for all edges v, u ⊴ e we have that F v ⊴ e = F u ⊴ e
Revised: H 1sym such that for all edges v, u ⊴ e we have that F v ⊴ e = F u ⊴ e

Original: = ±√ w vu .
Revised: = ±√ w vu .

Original: Then, L vu = − w vu and L vv = (cid:80) e ∥F v ⊴ e ∥ 2 = (cid:80) u w vu .
Revised: Then, L vu = − w vu and L vv = (cid:80) e ∥F v ⊴ e ∥ 2 = (cid:80) u w vu .

Original: The equality for the normalised version of the Laplacians follows directly.
Revised: The equality for the normalised version of the Laplacians follows directly.

Original: We state the following Lemma without proof based on Theorem 3.1 in Hansen & Ghrist (2021).
Revised: We state the following Lemma without proof based on Theorem 3.1 in Hansen & Ghrist (2021).

Original: Lemma 27.
Revised: Lemma 27.

Original: Solutions X ( t ) to the diffusion in Equation 1 converge as t → ∞ to the orthogonal projection of X (0) onto ker(∆ F ) .
Revised: Solutions X ( t ) to the diffusion in Equation 1 converge as t → ∞ to the orthogonal projection of X (0) onto ker(∆ F ) .

Original: Due to this Lemma, the proofs below rely entirely on the structure of ker(∆ F ) that one obtains for certain ( G, F ) .
Revised: Due to this Lemma, the proofs below rely entirely on the structure of ker(∆ F ) that one obtains for certain ( G, F ) .

Original: Proof of Proposition 7 .
Revised: Proof of Proposition 7 .

Original: Let G =
Revised: Let G =

Original: ( V, E ) be a graph with two classes A, B ⊂ V such that for each v ∈ A , there exists u ∈ A and an edge ( v, u ) ∈ E .
Revised: ( V, E ) be a graph with two classes A, B ⊂ V such that for each v ∈ A , there exists u ∈ A and an edge ( v, u ) ∈ E .

Original: Additionally, let x (0) be any channel of the feature matrix X (

Revised: Additionally, let x (0) be any channel of the feature matrix X (

Original: ) ∈ R n × f .
Revised: ) ∈ R n × f .

Original: We can construct a sheaf ( F , G ) ∈
Revised: We can construct a sheaf ( F , G ) ∈

Original: H 1sym as follows.
Revised: H 1sym as follows.

Original: For all nodes v ∈ V and edges e ∈ E ,
Revised: For all nodes v ∈ V and edges e ∈ E ,

Original: F ( v ) ∼ = F ( e ) ∼ = R .
Revised: F ( v ) ∼ = F ( e ) ∼ = R .

Original: For all v, u ∈ A and edge ( u, v ) ∈ E , set F v ⊴ e = F u ⊴ e = √ α > 0 .
Revised: For all v, u ∈ A and edge ( u, v ) ∈ E , set F v ⊴ e = F u ⊴ e = √ α > 0 .

Original: Otherwise, set F v ⊴ e = 1 .
Revised: Otherwise, set F v ⊴ e = 1 .

Original: Denote by h v the number of neighbours of node v in the same class as v .
Revised: Denote by h v the number of neighbours of node v in the same class as v .

Original: Note that based on the assumptions, h v > 1 if v ∈ A .
Revised: Note that based on the assumptions, h v > 1 if v ∈ A .

Original: Then the only harmonic eigenvector of ∆ F is:
Revised: Then the only harmonic eigenvector of ∆ F is:

Original: [Equation]
Revised: [Equation]

Original: Denote its unit-normalised version (cid:101) a := a ∥ a ∥ .
Revised: Denote its unit-normalised version (cid:101) a := a ∥ a ∥ .

Original: In the limit of the diffusion process, the features converge to h = ⟨ x (0) , (cid:101) a ⟩ (cid:101) a by Lemma 27.
Revised: In the limit of the diffusion process, the features converge to h = ⟨ x (0) , (cid:101) a ⟩ (cid:101) a by Lemma 27.

Original: Assuming, x (0) / ∈ ker(∆ F ) ⊥ , which is nowhere dense in R n and, without loss of generality, that ⟨ x (0) , (cid:101) a ⟩ > 0 , for sufficiently large α , (cid:101) a v ≥ (cid:101) a u for all v ∈ A, u ∈ B .
Revised: Assuming, x (0) / ∈ ker(∆ F ) ⊥ , which is nowhere dense in R n and, without loss of generality, that ⟨ x (0) , (cid:101) a ⟩ > 0 , for sufficiently large α , (cid:101) a v ≥ (cid:101) a u for all v ∈ A, u ∈ B .

Original: Proof of Proposition 8 .
Revised: Proof of Proposition 8 .

Original: Let G =
Revised: Let G =

Original: ( A, B, E ) be a bipartite graph with | A | = | B | and let x (0) ∈
Revised: ( A, B, E ) be a bipartite graph with | A | = | B | and let x (0) ∈

Original: R n be any channel of the feature matrix X (

Revised: R n be any channel of the feature matrix X (

Original: ) ∈ R n × f .
Revised: ) ∈ R n × f .

Original: Consider an arbitrary sheaf
Revised: Consider an arbitrary sheaf

Original: ( G, F ) ∈ H 1sym .
Revised: ( G, F ) ∈ H 1sym .

Original: Since the graph is connected, the only harmonic eigenvector of ∆ F is y ∈ R n with y v = (cid:113)(cid:80) v ⊴ e ∥F v ⊴ e ∥ 2 (i.e. the square root of the weighted degree).
Revised: Since the graph is connected, the only harmonic eigenvector of ∆ F is y ∈ R n with y v = (cid:113)(cid:80) v ⊴ e ∥F v ⊴ e ∥ 2 (i.e. the square root of the weighted degree).

Original: Based on Lemma 27, the diffusion process converges in the limit (up to a scaling) to ⟨ x , y ⟩ y .
Revised: Based on Lemma 27, the diffusion process converges in the limit (up to a scaling) to ⟨ x , y ⟩ y .

Original: For the features to be linearly separable we require that ⟨ x , y ⟩ ̸ = 0 and, without loss of generality, for all v ∈ A, u ∈ B that y v < y u ⇔ (cid:80)[Equation]
Revised: For the features to be linearly separable we require that ⟨ x , y ⟩ ̸ = 0 and, without loss of generality, for all v ∈ A, u ∈ B that y v < y u ⇔ (cid:80)

Original: ⊴

Original: [Equation]
Revised: [Equation]

Original: ∥F

Original: [Equation]

Original: ⊴ e ∥

Original: [Equation]

Original: < (cid:80)

Original: [Equation]

Original: ∥F u ⊴

Original: [Equation]

Original: ∥ 2 .

Original: Suppose for the sake of contradiction there exists a sheaf in H 1 sym with such a harmonic eigenvector.
Revised: Suppose for the sake of contradiction there exists a sheaf in H 1sym with such a harmonic eigenvector.

Original: Then, because | A | = | B | :
Revised: Then, because | A | = | B | :

Original: [Equation]
Revised: [Equation]

Original: However, because ( F , G ) ∈ H 1sym , we have F v ⊴ e = F u ⊴ e and the sum above is zero.
Revised: However, because ( F , G ) ∈ H 1sym , we have F v ⊴ e = F u ⊴ e and the sum above is zero.

Original: Proof of Proposition 10 .
Revised: Proof of Proposition 10 .

Original: Let G =
Revised: Let G =

Original: ( V, E ) be a connected graph with two classes A, B ⊂ V .
Revised: ( V, E ) be a connected graph with two classes A, B ⊂ V .

Original: Additionally, let x (0) be any channel of the feature matrix X (

Revised: Additionally, let x (0) be any channel of the feature matrix X (

Original: ) ∈ R n × f .
Revised: ) ∈ R n × f .

Original: We can construct a sheaf ( F , G ) ∈ H 1 ( G ) as follows.
Revised: We can construct a sheaf ( F , G ) ∈ H 1 ( G ) as follows.

Original: For all nodes v and edges e , F ( v ) ∼ = F ( e ) ∼ = R .
Revised: For all nodes v and edges e , F ( v ) ∼ = F ( e ) ∼ = R .

Original: For all v ∈ A , set F v ⊴ e = α .
Revised: For all v ∈ A , set F v ⊴ e = α .

Original: For all u ∈ B , set F u ⊴ e = β .
Revised: For all u ∈ B , set F u ⊴ e = β .

Original: Additionally, let α < 0 < β .
Revised: Additionally, let α < 0 < β .

Original: Since the graph is connected, by Lemma 19, the only harmonic eigenvector of ∆ F is:
Revised: Since the graph is connected, by Lemma 19, the only harmonic eigenvector of ∆ F is:

Original: [Equation]
Revised: [Equation]

Original: Assume x (0) / ∈ ker(∆ F ) ⊥ , which is nowhere dense in R n and, without loss of generality, that ⟨ x (0) , y ⟩ > 0 .
Revised: Assume x (0) / ∈ ker(∆ F ) ⊥ , which is nowhere dense in R n and, without loss of generality, that ⟨ x (0) , y ⟩ > 0 .

Original: Then, y v > 0 > y u for all v ∈ A, u ∈ B .
Revised: Then, y v > 0 > y u for all v ∈ A, u ∈ B .

Original: Definition 28.
Revised: Definition 28.

Original: The class of sheaves over G with non-zero maps, one-dimensional stalks, and similarly signed restriction maps:
Revised: The class of sheaves over G with non-zero maps, one-dimensional stalks, and similarly signed restriction maps:

Original: [Equation]
Revised: [Equation]

Original: Proposition 29.
Revised: Proposition 29.

Original: Let G be the connected graph with two nodes belonging to two different classes.
Revised: Let G be the connected graph with two nodes belonging to two different classes.

Original: Then H 1+ cannot linearly separate the two nodes for any initial conditions X ∈ R 2 × f .
Revised: Then H 1+ cannot linearly separate the two nodes for any initial conditions X ∈ R 2 × f .

Original: Proof of Proposition 29 .
Revised: Proof of Proposition 29 .

Original: Let G be the connected graph with two nodes V = { v, u } .
Revised: Let G be the connected graph with two nodes V = { v, u } .

Original: Then any sheaf ( F , G ) ∈ H 1+ ( G ) has restriction maps of the form F v ⊴ e
Revised: Then any sheaf ( F , G ) ∈ H 1+ ( G ) has restriction maps of the form F v ⊴ e

Original: = α, F u ⊴ e = β and (without loss of generality)
Revised: = α, F u ⊴ e = β and (without loss of generality)

Original: α, β > 0 .
Revised: α, β > 0 .

Original: As before, the only (unormalised) harmonic eigenvector for a sheaf of this form is y =
Revised: As before, the only (unormalised) harmonic eigenvector for a sheaf of this form is y =

Original: ( | α | β, α | β | )
Revised: ( | α | β, α | β | )

Original: = ( αβ, αβ ) .
Revised: = ( αβ, αβ ) .

Original: Since this is a constant vector, the two nodes are not separable in the diffusion limit.
Revised: Since this is a constant vector, the two nodes are not separable in the diffusion limit.

Original: We state the following result without proof (see Exercise 4.1 in Bishop (2006)).
Revised: We state the following result without proof (see Exercise 4.1 in Bishop (2006)).

Original: Lemma 30.
Revised: Lemma 30.

Original: Let A and B be two sets of points in R n .
Revised: Let A and B be two sets of points in R n .

Original: If their convex hulls intersect, the two sets of points cannot be linearly separable.
Revised: If their convex hulls intersect, the two sets of points cannot be linearly separable.

Original: Proof of Proposition 11 .
Revised: Proof of Proposition 11 .

Original: If the sheaf has a trivial global section, then all features converge to zero in the diffusion limit.
Revised: If the sheaf has a trivial global section, then all features converge to zero in the diffusion limit.

Original: Suppose H 0 ( G, F ) is non-trivial.
Revised: Suppose H 0 ( G, F ) is non-trivial.

Original: Since G is connected and all the restriction maps are invertible, by Lemma 19, dim( H 0 )
Revised: Since G is connected and all the restriction maps are invertible, by Lemma 19, dim( H 0 )

Original: = 1 .
Revised: = 1 .

Original: In that case, let h be the unit-normalised harmonic eigenvector of ∆ F .
Revised: In that case, let h be the unit-normalised harmonic eigenvector of ∆ F .

Original: By Lemma 27, for any node v , its scalar feature in channel k ≤ f is given by x kv ( ∞ ) = ⟨ x k (0) , h ⟩ h v .
Revised: By Lemma 27, for any node v , its scalar feature in channel k ≤ f is given by x kv ( ∞ ) = ⟨ x k (0) , h ⟩ h v .

Original: Note that we can always find three nodes v, u, w belonging to three different classes such that h v ≤ h u ≤ h v .
Revised: Note that we can always find three nodes v, u, w belonging to three different classes such that h v ≤ h u ≤ h v .

Original: Then, there exists a convex combination h u
Revised: Then, there exists a convex combination h u

Original: = α
Revised: = α

Original: h v + (1 − α ) h w , with α ∈ [0 , 1] .
Revised: h v + (1 − α ) h w , with α ∈ [0 , 1] .

Original: Therefore:
Revised: Therefore:

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Since this is true for all channels k < f , it follows that x u ( ∞ ) =
Revised: Since this is true for all channels k < f , it follows that x u ( ∞ ) =

Original: α x v ( ∞ ) + (1 − α ) x w ( ∞ ) .
Revised: α x v ( ∞ ) + (1 − α ) x w ( ∞ ) .

Original: Because x u ( ∞ ) is in the convex hull of the points belonging to other classes, by Lemma 30, the class of v is not linearly separable from the other classes.
Revised: Because x u ( ∞ ) is in the convex hull of the points belonging to other classes, by Lemma 30, the class of v is not linearly separable from the other classes.

Original: Proof of Proposition 13 .
Revised: Proof of Proposition 13 .

Original: Let G =
Revised: Let G =

Original: ( V, E ) be a connected graph with C classes and ( F , G ) , an arbitrary sheaf in H C ( G ) .
Revised: ( V, E ) be a connected graph with C classes and ( F , G ) , an arbitrary sheaf in H C ( G ) .

Original: Because F has diagonal restriction maps there is no interaction during diffusion between the different dimensions of the stalks.
Revised: Because F has diagonal restriction maps there is no interaction during diffusion between the different dimensions of the stalks.

Original: Therefore, the diffusion process can be written as d independent diffusion processes, where the i -th process uses a sheaf F i with all stalks isomorphic to R and F iv ⊴ e = F v ⊴ e ( i, i ) for all v ∈ V and incident edges e .
Revised: Therefore, the diffusion process can be written as d independent diffusion processes, where the i -th process uses a sheaf F i with all stalks isomorphic to R and F iv ⊴ e = F v ⊴ e ( i, i ) for all v ∈ V and incident edges e .

Original: Therefore, we can construct d sheaves F i ∈ H 1 ( G ) with i < d as in Proposition 10, where (in one vs all fashion) the two classes are given by the nodes in class i and the nodes belonging to the other classes.
Revised: Therefore, we can construct d sheaves F i ∈ H 1 ( G ) with i < d as in Proposition 10, where (in one vs all fashion) the two classes are given by the nodes in class i and the nodes belonging to the other classes.

Original: It remains to restrict that the projection of x (0) on any of the harmonic eigenvectors of ∆ F in the standard basis is non-zero.
Revised: It remains to restrict that the projection of x (0) on any of the harmonic eigenvectors of ∆ F in the standard basis is non-zero.

Original: Formally, we require x i (0) / ∈ ker(∆ F i ) ⊥ for all positive integers i ≤ d .
Revised: Formally, we require x i (0) / ∈ ker(∆ F i ) ⊥ for all positive integers i ≤ d .

Original: Since ker(∆ F i ) ⊥ is nowhere dense in R n , x (0) belongs to the direct sum of dense subspaces, which is dense.[Figure]
Revised: Since ker(∆ F i ) ⊥ is nowhere dense in R n , x (0) belongs to the direct sum of dense subspaces, which is dense.

Original: Lemma 31.
Revised: Lemma 31.

Original: Let G =
Revised: Let G =

Original: ( V, E ) be a graph and ( F , G ) a (weighted) orthogonal vector bundle over G with path-independent parallel transport and edge weights α e .
Revised: ( V, E ) be a graph and ( F , G ) a (weighted) orthogonal vector bundle over G with path-independent parallel transport and edge weights α e .

Original: Consider an arbitrary node v ∗ ∈ V and denote by e i the i -th standard basis vector of R d .
Revised: Consider an arbitrary node v ∗ ∈ V and denote by e i the i -th standard basis vector of R d .

Original: Then { h 1 , . .
Revised: Then { h 1 , . .

Original: , h d } form an orthogonal eigenbasis for the harmonic space of ∆ F , where:
Revised: , h d } form an orthogonal eigenbasis for the harmonic space of ∆ F , where:

Original: [Equation]
Revised: [Equation]

Revised: [Figure]

Original: Proof.
Revised: Proof.

Original: First
Revised: First

Original: [Equation]
Revised: [Equation]

Original: we show that h iv is harmonic.
Revised: we show that h iv is harmonic.

Original: [Equation]
Revised: [Equation]

Original: By path independence
Revised: By path independence

Original: [Equation]
Revised: [Equation]

Original:  definition 
Revised: By definition of

Revised: [Equation]

Revised: Orthogonality of F v

Original: [Equation]
Revised: [Equation]

Original: For orthogonality, notice that for any i, j ≤ d and v ∈ V , it holds that:
Revised: For orthogonality, notice that for any i, j ≤ d and v ∈ V [Equation] it holds that:

Original: [Equation]
Revised: [Equation]

Original: Lemma 32.
Revised: Lemma 32.

Original: Let R 1 , R 2 be two 2D rotation matrices and e 1 , e 2 the two standard basis vectors of R 2 .
Revised: Let R 1 , R 2 be two 2D rotation matrices and e 1 , e 2 the two standard basis vectors of R 2 .

Original: Then ⟨ R 1 e 1 , R 2 e 2 ⟩ = −⟨ R 1 e 2 , R 2 e 1 ⟩ .
Revised: Then ⟨ R [Equation]

Original: Proof.
Revised: Proof.

Original: The angle between e 1 and e 2 is π 2 .
Revised: The angle between e 1 and e 2 is π 2 .

Original: Letting ϕ, θ be the positive rotation angles of the two matrices, the first inner product is equal to cos( π/ 2+( ϕ − θ )) while the second is cos( π/ 2 − ( ϕ − θ )) .
Revised: Letting ϕ, θ be the positive rotation angles of the two matrices, the first inner product is equal to cos( π/ 2+( ϕ − θ )) while the second is cos( π/ 2 − ( ϕ − θ )) .

Original: The result follows from applying the trigonometric identity cos( π/ 2 + x ) = − sin x .
Revised: The result follows from applying the trigonometric identity cos( π/ 2 + x ) = − sin x .

Original: We first prove Theorem 15 in dimension two in the following lemma and then we will look at the general case.
Revised: We first prove Theorem 15 in dimension two in the following lemma and then we will look at the general case.

Original: Lemma 33.
Revised: Lemma 33.

Original: Let G be the class of connected graphs with C ≤ 4 classes.
Revised: Let G be the class of connected graphs with C ≤ 4 classes.

Original: Then, H 2orth ( G ) has linear separation power over G .
Revised: Then, H 2orth ( G ) has linear separation power over G .

Original: Proof.
Revised: Proof.

Original: Idea: We can use rotation matrices to align the harmonic features of the classes with the axis of coordinates as in Figure 8a.
Revised: Idea: We can use rotation matrices to align the harmonic features of the classes with the axis of coordinates as in Figure 8a.

Original: Then, for each side of each axis, we can find a separating hyperplane separating each class from all the others.
Revised: Then, for each side of each axis, we can find a separating hyperplane separating each class from all the others.

Original: Let G be a connected graph with C ≤ 4 classes.
Revised: Let G be a connected graph with C ≤ 4 classes.

Original: Denote by P the following set of rotation matrices together with their signed-flipped counterparts:
Revised: Denote by P the following set of rotation matrices together with their signed-flipped counterparts:

Original: [Equation]
Revised: [Equation]

Original: and by C = { 1 , . . . , C } the set of all class labels.
Revised: and by C = { 1 , . . . , C } the set of all class labels.

Original: Then, fix a node v ∗ ∈ V and construct an injective map g : C → P assigning each class label one of the signed basis vectors such that g ( c ( v ∗ ))
Revised: Then, fix a node v ∗ ∈ V and construct an injective map g : C → P assigning each class label one of the signed basis vectors such that g ( c ( v ∗ ))

Original: =
Revised: =

Original: R 1 , where c ( v ∗ ) denotes the class of node v ∗ .
Revised: R 1 , where c ( v ∗ ) denotes the class of node v ∗ .

Original: Then, we can construct an sheaf ( G, F ) ∈ H 2orth ( G ) in terms of certain parallel transport maps along each edge, that will depend on P .
Revised: Then, we can construct an sheaf ( G, F ) ∈ H 2orth ( G ) in terms of certain parallel transport maps along each edge, that will depend on P .

Original: For all nodes v and edges e , F ( v ) ∼ = F ( e ) ∼ = R 2 .
Revised: For all nodes v and edges e , F ( v ) ∼ = F ( e ) ∼ = R 2 .

Original: For each u ∈ V , we set P v ∗ → u = g ( c ( u )) .
Revised: For each u ∈ V , we set P v ∗ → u = g ( c ( u )) .

Original: Then for all v, u ∈ V , set P v → u = P v ∗ → v P − 1 u → v ∗ .
Revised: Then for all v, u ∈ V , set P v → u = P v ∗ → v P − 1 u → v ∗ .

Original: It is easy to see that the resulting parallel transport is path-independent because it depends purely on the classes of the endpoints of the path.
Revised: It is easy to see that the resulting parallel transport is path-independent because it depends purely on the classes of the endpoints of the path.

Original: Based on Lemma 31, the i -th eigenvector of ∆ F is h i ∈
Revised: Based on Lemma 31, the i -th eigenvector of ∆ F is h i ∈

Original: R 2 × n with h iu =
Revised: R 2 × n with h iu =

Original: P v ∗ →
Revised: P v ∗ →

Original: u e i √ d u .
Revised: u e i √ d u .

Original: Now we will show that the projection of x (0) in this subspace will have a configuration as in Figure 8a up to a rotation.
Revised: Now we will show that the projection of x (0) in this subspace will have a configuration as in Figure 8a up to a rotation.

Original: Let u, w be two nodes belonging to two different classes.
Revised: Let u, w be two nodes belonging to two different classes.

Original: Denote by α i = ⟨ x (0) , h i ⟩ .
Revised: Denote by α i = ⟨ x (0) , h i ⟩ .

Original: Then the inner product between the features of nodes u, w in the limit of the diffusion process is:
Revised: Then the inner product between the features of nodes u, w in the limit of the diffusion process is:

Original: [Equation]
Revised: [Equation]

Original: It can be checked that by substituting the transport maps P v ∗ → u , P v ∗ → w with any R a , R b from P such that R a ̸ = ± R b , the inner product above is zero.
Revised: It can be checked that by substituting the transport maps P v ∗ → u , P v ∗ → w with any R a , R b from P such that R a ̸ = ± R b , the inner product above is zero.

Original: Similarly, substituting any R a = − R b , the inner product is −√ d u d w (cid:80) k
Revised: Similarly, substituting any R a = − R b , the inner product is −√ d u d w (cid:80) k

Original: α 2 k
Revised: α 2 k

Original: = −√ d u d w ∥ x (0) ∥ 2 , which is equal to the product of the norms of the two vectors.
Revised: = −√ d u d w ∥ x (0) ∥ 2 , which is equal to the product of the norms of the two vectors.

Original: Therefore, the diffused features of different classes are positioned at π 2 , π, 3 πfrom each other, as in Figure 8a.
Revised: Therefore, the diffused features of different classes are positioned at π 2 , π, 3 πfrom each other, as in Figure 8a.

Original: Proof of Theorem 15 .
Revised: Proof of Theorem 15 .

Original: To generalise the proof, we need to find a set P of size d
Revised: To generalise the proof, we need to find a set P of size d

Original: containing rotation matrices that make the projected features of different classes be pairwise orthogonal for any projection coefficients α .
Revised: containing rotation matrices that make the projected features of different classes be pairwise orthogonal for any projection coefficients α .

Original: For that, each term in Equation 19 must be zero for any coefficients α .
Revised: For that, each term in Equation 19 must be zero for any coefficients α .

Original: Therefore, P = { P 0 , . . .
Revised: Therefore, P = { P 0 , . . .

Original: , P d − 1 } must satisfy the following requirements:
Revised: , P d − 1 } must satisfy the following requirements

Original: P 

Revised: P

Original: = I ∈ P , since transport for neighbours in the same class must be the identity.
Revised: = I ∈ P , since transport for neighbours in the same class must be the identity.

Original: Therefore, P 0 P k = P k P 0 =
Revised: Therefore, P 0 P k = P k P 0 =

Original: P k for all k . 2.
Revised: P k for all k . 2.

Original: Since ⟨ P 0 e i , P k e i ⟩ = 0 for all i and k ̸ = 0 , it follows that the diagonal elements of P k are zero.
Revised: Since ⟨ P 0 e i , P k e i ⟩ = 0 for all i and k ̸ = 0 , it follows that the diagonal elements of P k are zero.

Original: From ⟨ P 0 e i , P k e j ⟩ = −⟨ P 0 e j , P k e i ⟩ for all i ̸
Revised: From ⟨ P 0 e i , P k e j ⟩ = −⟨ P 0 e j , P k e i ⟩ for all i ̸

Original: = j, k ̸ = 0 and point (2) it follows that P − 1 k
Revised: = j, k ̸ = 0 and point (2) it follows that P − 1 k

Original: = P ⊤ k = − P k .
Revised: = P ⊤ k = − P k .

Original: Therefore, P k P k = − I for all k ̸ = 0 .
Revised: Therefore, P k P k = − I for all k ̸ = 0 .

Original: We have ⟨ P k e i , P l e i ⟩ = 0 for all i and k ̸ = l .
Revised: We have ⟨ P k e i , P l e i ⟩ = 0 for all i and k ̸ = l .

Original: Together with (3), it follows that the diagonal elements of P k P l are zero.
Revised: Together with (3), it follows that the diagonal elements of P k P l are zero.

Original: We have ⟨ P k e i ,
Revised: We have ⟨ P k e i ,

Original: P l e j ⟩ = −⟨ P k e j , P l e i ⟩ for all i ̸
Revised: P l e j ⟩ = −⟨ P k e j , P l e i ⟩ for all i ̸

Original: = j , and k ̸ = l , with k, l ̸ = 0 .
Revised: = j , and k ̸ = l , with k, l ̸ = 0 .

Original: Together with point (4) it follows that ( P k P l ) ⊤ = − P k P l .
Revised: Together with point (4) it follows that ( P k P l ) ⊤ = − P k P l .

Original: Similarly, from point (3) we have that ( P k P l ) ⊤ =
Revised: Similarly, from point (3) we have that ( P k P l ) ⊤ =

Original: P ⊤ l P ⊤ k
Revised: P ⊤ l P ⊤ k

Original: = ( − P l )( − P k ) =
Revised: = ( − P l )( − P k ) =

Original: P l P k .
Revised: P l P k .

Original: Therefore, the two matrices are anticommutative: P k P l = − P l P k .
Revised: Therefore, the two matrices are anticommutative: P k P l = − P l P k .

Original: We remark that points (1), (3), (5) coincide with the defining algebraic properties of the algebra of complex numbers, quaternions, octonions, sedenions and their generalisations based on the CayleyDickson construction (Schafer, 2017).
Revised: We remark that points (1), (3), (5) coincide with the defining algebraic properties of the algebra of complex numbers, quaternions, octonions, sedenions and their generalisations based on the CayleyDickson construction (Schafer, 2017).

Original: Therefore, the matrices in P must be a representation of one of these algebras.
Revised: Therefore, the matrices in P must be a representation of one of these algebras.

Original: Firstly, such algebras exist only for d that are powers of two.
Revised: Firstly, such algebras exist only for d that are powers of two.

Original: Secondly, matrix representations for these algebras exist only in dimension two and four.
Revised: Secondly, matrix representations for these algebras exist only in dimension two and four.

Original: This is because the algebra of octonions and their generalisations, unlike matrix multiplication, is non-associative.
Revised: This is because the algebra of octonions and their generalisations, unlike matrix multiplication, is non-associative.

Original: As a sanity check, note that the matrices R 1 , R 2 from Lemma 33 are a classic representation of the unit complex numbers.
Revised: As a sanity check, note that the matrices R 1 , R 2 from Lemma 33 are a classic representation of the unit complex numbers.

Original: We conclude this section by giving out the matrices for d = 4 , which are the real matrix representations of the four unit quaternions:
Revised: We conclude this section by giving out the matrices for d = 4 , which are the real matrix representations of the four unit quaternions:

Original: [Equation] 0  0 0  ,
Revised: R 1 =   1 0 00 1 00 0 10 0 0  ,

Original: R 2 =   0 − 1       0 −    
Revised: R 2 =   0 − 1       0 −    

Original:  ,
Revised:  ,

Original: R 3 =    − 1        0 −  
Revised: R 3 =    − 1        0 −  

Original:  ,
Revised:  ,

Original: R 4 =   0 −0 0 − 0  1  
Revised: R 4 =   0 −0 0 − 0  1  

Original:  .
Revised:  .

Original: It can be checked that these matrices respect the properties outlined above.
Revised: It can be checked that these matrices respect the properties outlined above.

Original: Thus, in d = 4 , we can select the transport maps from the set {± R 1 , ± R 2 , ± R 3 , ± R 4 } containing eight matrices, which also form a group.
Revised: Thus, in d = 4 , we can select the transport maps from the set {± R 1 , ± R 2 , ± R 3 , ± R 4 } containing eight matrices, which also form a group.

Original: Therefore, following the same procedure as in Lemma 33, we can linearly separate up to eight classes.
Revised: Therefore, following the same procedure as in Lemma 33, we can linearly separate up to eight classes.

Original: Proposition 34.
Revised: Proposition 34.

Original: Let G be the class of connected regular graphs with a finite number of classes.
Revised: Let G be the class of connected regular graphs with a finite number of classes.

Original: Then, H 2orth ( G ) has linear separation power over G .
Revised: Then, H 2orth ( G ) has linear separation power over G .

Original: Proof of Propositon 34 .
Revised: Proof of Propositon 34 .

Original: Idea: Since the graph is regular, the harmonic features of the nodes will be uniformly scaled and thus positioned on a circle.
Revised: Idea: Since the graph is regular, the harmonic features of the nodes will be uniformly scaled and thus positioned on a circle.

Original: The aim is to place different classes at different locations on the circle, which would make the classes linearly separable as shown in Figure 8b.
Revised: The aim is to place different classes at different locations on the circle, which would make the classes linearly separable as shown in Figure 8b.

Original: Let G be a regular graph with C classes and define θ = 2 πC .
Revised: Let G be a regular graph with C classes and define θ = 2 πC .

Original: Denote by R i the 2D rotation matrix:
Revised: Denote by R i the 2D rotation matrix:

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: Then let P =
Revised: Then let P =

Original: { R i | 0 ≤ i ≤ C − 1 , i ∈ N } the set of rotation matrices with an angle multiple of θ .
Revised: { R i | 0 ≤ i ≤ C − 1 , i ∈ N } the set of rotation matrices with an angle multiple of θ .

Original: Then we can define a bijection g : C → P and a sheaf ( G, F ) ∈
Revised: Then we can define a bijection g : C → P and a sheaf ( G, F ) ∈

Original: H 2orth ( G ) as in the proof above.
Revised: H 2orth ( G ) as in the proof above.

Original: Checking the inner-products from Equation 19 between the harmonic features of the nodes, we can verify that the angle between any two classes is different from zero.
Revised: Checking the inner-products from Equation 19 between the harmonic features of the nodes, we can verify that the angle between any two classes is different from zero.

Original: By Lemma 32, the cross terms of the inner product vanish:
Revised: By Lemma 32, the cross terms of the inner product vanish:

Original: [Equation]
Revised: [Equation]

Original: Thus, the angle between classes i, j is ( i − j ) θ .
Revised: Thus, the angle between classes i, j is ( i − j ) θ .

Original: J P ROOFS FOR OVERSMOOTHING
Revised: I P ROOFS FOR OVERSMOOTHING

Original: Proof of Proposition 22 .
Revised: Proof of Proposition 22 .

Original: Let x ∈ H 

Revised: Let x ∈ H

Original: ( G, F ) .
Revised: ( G, F ) .

Original: Then we have
Revised: Then we have

Original: [Equation]
Revised: [Equation]

Original: The last term vanishes if and only if x k ∈ ker ∆ 0 for each 1 ≤ k ≤ d .
Revised: The last term vanishes if and only if x k ∈ ker ∆ 0 for each 1 ≤ k ≤ d .

Original: Proposition 35.
Revised: Proposition 35.

Original: Let F be an O ( d ) -bundle over G and ε > 0 .
Revised: Let F be an O ( d ) -bundle over G and ε > 0 .

Original: Assume that F v ⊴ e = F u ⊴ e for each ( u, v ) ̸
Revised: Assume that F v ⊴ e = F u ⊴ e for each ( u, v ) ̸

Original: = ( u 0 , v 0 ) and that F ⊤ v 0 ⊴ e F u 0 ⊴ e − I :=
Revised: = ( u 0 , v 0 ) and that F ⊤ v 0 ⊴ e F u 0 ⊴ e − I :=

Original: B ̸ = 0 with dim ( ker B ) > 0 .
Revised: B ̸ = 0 with dim ( ker B ) > 0 .

Original: Then there exist a linear map W ∈ R d × d with || W || 2 = ε and x ∈ H 

Revised: Then there exist a linear map W ∈ R d × d with || W || 2 = ε and x ∈ H

Original: ( G, F ) such that E F
Revised: ( G, F ) such that E F

Original: (( I ⊗ W ) x ) > 0 .
Revised: (( I ⊗ W ) x ) > 0 .

Original: Proof.
Revised: Proof.

Original: We sketch the proof.
Revised: We sketch the proof.

Original: Let g ∈ ker
Revised: Let g ∈ ker

Original: ( B ) .
Revised: ( B ) .

Original: Define then x ∈ C 

Revised: Define then x ∈ C

Original: ( G, F ) by
Revised: ( G, F ) by

Original: [Equation]
Revised: [Equation]

Original: Then x ∈ H 0 ( G, F ) .
Revised: Then x ∈ H 0 ( G, F ) .

Original: If we now take W = ε P ker B ⊥
Revised: If we now take W = ε P ker B ⊥

Original: the rescaled orthogonal projection in the orthogonal complement of the kernel of B we verify the given claim.
Revised: the rescaled orthogonal projection in the orthogonal complement of the kernel of B we verify the given claim.

Original: We provide below a proof for the equality in Definition 20.
Revised: We provide below a proof for the equality in Definition 20.

Original: Proposition 36.
Revised: Proposition 36.

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: We prove the result for the normalised sheaf Laplacian, and other versions can be obtained as particular cases
Revised: We prove the result for the normalised sheaf Laplacian, and other versions can be obtained as particular cases.

Original: [Equation]
Revised: [Equation]

Original: Note that D v is symmetric for any node v
Revised: Note that D v is symmetric for any node v

Original: and so is any D − 1 / 2 v .
Revised: and so is any D − 1 / 2 v .

Original: Therefore, the two vectors in the parenthesis are the transpose of each other and the result is their inner product.
Revised: Therefore, the two vectors in the parenthesis are the transpose of each other and the result is their inner product.

Original: Thus, we have:
Revised: Thus, we have:

Original: [Equation]
Revised: [Equation]

Original: The result follows identically for other types of Laplacian.
Revised: The result follows identically for other types of Laplacian.

Original: For the augmented normalized Laplacian, one should simply replace D with ˜ D
Revised: For the augmented normalized Laplacian, one should simply replace D with ˜ D

Original: = D
Revised: = D

Original: + I and for the non-normalised Laplacian, one should simply remove D from the equation.
Revised: + I and for the non-normalised Laplacian, one should simply remove D from the equation.

Original: Proof of Theorem 21 .
Revised: Proof of Theorem 21 .

Original: We first prove a couple of Lemmas before proving the Theorem.
Revised: We first prove a couple of Lemmas before proving the Theorem.

Original: The structure of the proof follows that of Cai & Wang (2020), which in turn generalises that of Oono &
Revised: The structure of the proof follows that of Cai & Wang (2020), which in turn generalises that of Oono &

Original: Suzuki (2019).
Revised: Suzuki (2019).

Original: The latter proof technique is not directly applicable to our setting because it makes some strong assumptions about the harmonic space of the Laplacian (i.e. that the eigenvectors of the harmonic space have positive entries).
Revised: The latter proof technique is not directly applicable to our setting because it makes some strong assumptions about the harmonic space of the Laplacian (i.e. that the eigenvectors of the harmonic space have positive entries).

Original: λ [Equation] [Equation] (cid:1) , where λ min , λ max are the smallest and largest non-zero eigenvalues of ∆ F .
Revised: λ ∗ = max (cid:0) ( λ min − 1) 2 , ( λ max − 1) 2 (cid:1) , where λ min , λ max are the smallest and largest non-zero eigenvalues of ∆ F .

Original: Lemma 37.
Revised: Lemma 37.

Original: For P = I − ∆ F , E ( Px ) <
Revised: For P = I − ∆ F , E ( Px ) <

Original: λ ∗ E F ( x ) .
Revised: λ ∗ E F ( x ) .

Original: Proof.
Revised: Proof.

Original: We can write x = (cid:80) i c
Revised: We can write x = (cid:80) i c

Original: i
Revised: i

Original: h
Revised: h

Original: i as a sum of the eigenvectors { h i } of ∆ F .
Revised: i as a sum of the eigenvectors { h i } of ∆ F .

Original: Then x ⊤ ∆ F x = (cid:80) i c
Revised: Then x ⊤ ∆ F x = (cid:80) i c

Original: i λ
Revised: i λ

Original: i h
Revised: i h

Original: i , where { λ i } are the eigenvalues of ∆ F .
Revised: i , where { λ i } are the eigenvalues of ∆ F .

Original: Using this for E F ( Ph ) :
Revised: Using this for E F ( Ph ) :

Original: [Equation]
Revised: [Equation]

Original: The inequality follows from the fact that the eigenvectors of the normalied sheaf Laplacian are in the range [0 , 2] (Hansen & Ghrist, 2019, Proposition 5.5).
Revised: The inequality follows from the fact that the eigenvectors of the normalied sheaf Laplacian are in the range [0 , 2] (Hansen & Ghrist, 2019, Proposition 5.5).

Original: We note that the original proof of Cai &
Revised: We note that the original proof of Cai &

Original: Wang (2020) bounds the expression by (1 − λ min ) 2 instead of λ ∗ , which appears to be an error.
Revised: Wang (2020) bounds the expression by (1 − λ min ) 2 instead of λ ∗ , which appears to be an error.

Original: Lemma 38.
Revised: Lemma 38.

Original: E F ( XW ) ≤ ∥ W ⊤ ∥ [Equation]
Revised: E F ( XW ) ≤ ∥ W ⊤ ∥ 22 E F ( X )

Original: Proof.
Revised: Proof.

Original: Following the proof of Cai & Wang (2020) we have:
Revised: Following the proof of Cai & Wang (2020) we have:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: trace cyclic property
Revised: trace cyclic property

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: see Lemma 3.1 in Lee (2008)
Revised: see Lemma 3.1 in Lee (2008)

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Lemma 39.
Revised: Lemma 39.

Original: For conditions as in Theorem 21, E F (cid:0) ( I n ⊗ W ) x (cid:1) ≤ ∥ W ∥ 2 2 E ( f ) .
Revised: For conditions as in Theorem 21, E F (cid:0) ( I n ⊗ W ) x (cid:1) ≤ ∥ W ∥ 22 E ( f ) .

Original: Proof.
Revised: Proof.

Original: First, we note that for orthogonal matrices, D v =
Revised: First, we note that for orthogonal matrices, D v =

Original: I (cid:80) v ⊴ e α 2 e
Revised: I (cid:80) v ⊴ e α 2 e

Original: =
Revised: =

Original: I d v (Hansen & Ghrist, 2019, Lemma 4.4)
Revised: I d v (Hansen & Ghrist, 2019, Lemma 4.4)

Revised: E F (cid:0) ( I ⊗ W )

Original: [Equation]
Revised: [Equation]

Revised: (cid:1)

Revised: = 1 2 (cid:88) v

Revised: [Equation]

Revised: ∥F

Revised: [Equation]

Original: property of the operator norm
Revised: property of the operator norm

Original: [Equation]
Revised: [Equation]

Original: The proof can also be extended easily extended to vector bundles over weighted graphs (i.e. allowing weighted edges as in Ghrist & Riess (2020)).
Revised: The proof can also be extended easily extended to vector bundles over weighted graphs (i.e. allowing weighted edges as in Ghrist & Riess (2020)).

Original: For the non-normalised Laplacian, the assumption that
Revised: For the non-normalised Laplacian, the assumption that

Original: F e is orthogonal can be relaxed to being non-singular and then the upper bound will also depend on the maximum conditioning number over all F e .
Revised: F e is orthogonal can be relaxed to being non-singular and then the upper bound will also depend on the maximum conditioning number over all F e .

Original: Lemma 40.
Revised: Lemma 40.

Original: For conditions as in Theorem 21, E F (cid:0) σ ( x ) (cid:1) ≤ E ( x ) .
Revised: For conditions as in Theorem 21, E F (cid:0) σ ( x ) (cid:1) ≤ E ( x ) .

Original: Proof.
Revised: Proof.

Original: E (cid:0)

Original: [Equation]

Original: orthogonality

Original: [Equation]
Revised: [Equation]

Original: Lipschitz continuity of ReLU
Revised: Lipschitz continuity of ReLU

Original: [Equation]
Revised: [Equation]

Original: Combining these three lemmas for an entire diffusion layer proves the Theorem.
Revised: Combining these three lemmas for an entire diffusion layer proves the Theorem.

Original: Proof of Theorem 24 .
Revised: Proof of Theorem 24 .

Original: If d = 1 , then Lemma 39 becomes superfluous as W 1 becomes a scalar that can be absorbed into the right-weights.
Revised: If d = 1 , then Lemma 39 becomes superfluous as W 1 becomes a scalar that can be absorbed into the right-weights.

Original: It remains to verify that a version of Lemma 40 holds in this case.
Revised: It remains to verify that a version of Lemma 40 holds in this case.

Original: Lemma 41.
Revised: Lemma 41.

Original: For conditions as in Theorem 24, E F (cid:0) σ ( x ) (cid:1) ≤ E ( x ) .
Revised: For conditions as in Theorem 24, E F (cid:0) σ ( x ) (cid:1) ≤ E ( x ) .

Original: Proof.
Revised: Proof.

Original: [Equation]
Revised: [Equation]

Original: Lipschitz continuity of ReLU
Revised: Lipschitz continuity of ReLU

Original: [Equation]
Revised: [Equation]

Original: We note that if F v ⊴ e F w ⊴ e < 0 (i.e. the reation is signed), then it is very easy to find counterexamples where ReLU does not work anymore.
Revised: We note that if F v ⊴ e F w ⊴ e < 0 (i.e. the reation is signed), then it is very easy to find counterexamples where ReLU does not work anymore.

Original: However, the result still holds in the deep linear case.
Revised: However, the result still holds in the deep linear case.