Copresheaf Topological Neural Networks: A Generalized Deep Learning Framework
Authors: Mustafa Hajij
Lennart Bastian
Sarah Osentoski
Hardik Kabaria
John L. Davenport
Sheik Dawood
Balaji Cherukuri
Joseph G. Kocheemoolayil
Nastaran Shahmansouri
Adrian Lew
Theodore Papamarkou
Tolga Birdal
Venue: NeurIPS 2025 poster ()
Keywords: topological deep learning
graph networks
sheaf
transformers
Primary area: deep_learning

Abstract: We introduce copresheaf topological neural networks (CTNNs), a powerful unifying framework that encapsulates a wide spectrum of deep learning architectures, designed to operate on structured data, including images, point clouds, graphs, meshes, and topological manifolds. While deep learning has profoundly impacted domains ranging from digital assistants to autonomous systems, the principled design of neural architectures tailored to specific tasks and data types remains one of the field's most persistent open challenges. CTNNs address this gap by formulating model design in the language of copresheaves, a concept from algebraic topology that generalizes most practical deep learning models in use today. This abstract yet constructive formulation yields a rich design space from which theoretically sound and practically effective solutions can be derived to tackle core challenges in representation learning, such as long-range dependencies, oversmoothing, heterophily, and non-Euclidean domains. Our empirical results on structured data benchmarks demonstrate that CTNNs consistently outperform conventional baselines, particularly in tasks requiring hierarchical or localized sensitivity. These results establish CTNNs as a principled multi-scale foundation for the next generation of deep learning architectures.

Copresheaf Topological Neural Networks: A Generalized Deep Learning Framework
Mustafa Hajij
Lennart Bastian
Sarah Osentoski!
Hardik Kabaria'
John L. Davenport'
Sheik Dawood'
Balaji Cherukuri
Joseph G. Kocheemoolayil'
Nastaran Shahmansouri
Adrian Lew"
Theodore Papamarkou
Tolga Birdal
Vinci4D University of San Francisco Tecchnical University of Munich 'PolyShape
*Stanford University
*Imperial College London 7MCML
Abstract
We introduce copresheaf topological neural networks (CTNNs), powerful unify- ing framework that encapsulates a wide spectrum of deep learning architectures, designed to operate on structured data, including images, point clouds, graphs, meshes, and topological manifolds. While deep learning has profoundly impacted domains ranging from digital assistants autonomous systems, the principled de- sign of neural architectures tailored to specific tasks and data types remains one of the field's most persistent open challenges. CTNNs address this gap by formulating model design in the language of copresheaves, concept from algebraic topology that generalizes most practical deep learning models in use today. This abstract yet constructive formulation yields a rich design space from which theoretically sound and practically effective solutions can be derived to tackle core challenges in repre- sentation learning, such as long-range dependencies, oversmoothing, heterophily, and non- Non-Euclidean domains. Our empirical results on structured data benchmarks demonstrate that CTNNs consistently outperform conventional baselines, particu- larly in tasks requiring hicrarchical or localized sensitivity. These results establish CTNNs principled multi-scale foundation for the next generation of deep learning architectures.
Introduction
Deep learning has excelled by exploiting structural biases, such as convolutions for im- ages Krizhevsky al. 2012), transformers for sequences [Vaswaní al.) 2017]. and message passing for graphs [Gilmer et al., 2017]. However, the design of architectures that generalize across domains with complex, irregular, or multiscale structure remains notorious challenge (Bronstein] et al. 2017 Hajij et al. [2023b). Real-world data, which span physical systems, biomedical signals, and scientific simulations, rarely adhere to the regularity assumptions embedded conventional architectures. These data are inherently heterogeneous, directional, and hierarchical, often involving relations beyond pairwise connections or symmetric neighborhoods.
Convolutional neural networks (CNNs), designed for uniform grids, do not fully capture local irregularities; graph neural networks (GNNs) often rely on homophily and tend oversmooth feature representations as depth increases; and transformers, while excellent at capturing long-range dependencies, assume homogeneous embedding spaces, incur quadratic complexity, and lack built- in notions of anisotropy or variable local structures. These shortcomings highlight the need for framework that can natively encode diverse local behaviors, respect directional couplings, and
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

propagate information across scales without sacrificing local variations or imposing unwarranted homogeneity.
To address this foundational gap, we propose copresheaf topological neural networks (CTNNs), a unifying framework for deep learning based copresheaves, categorical structure that equips each local region of domain with its own feature space, along with learnable maps specifying how information flows between regions. Unlike traditional models that assume global latent space and isotoopic propagation, our framework respects local variability in representation and directional flow of information, enabling architectures that are multiscale, anisotropic and expressive.
By constructing CTNNs on combinatorial com- plexes (CCs) (Hajij al. 2023b a which gener- alize graphs, simplicial complexes, and cell com- plexes, we enable principled message-passing mechanism over general topological domains formulated within the theory of copresheaves. This unified perspective subsumes many deep learning paradigms, including GNNs, attention mechanisms, sheaf neural networks [Hansen and
hy
Pz
Py-
Zz
h
P
Ghrist
2019b Bodnar al.) 2022]. J. and topo- logical neural networks [Papillon et al., [023]
Figure 1: A copresheaf topological neural net- work (CTNN) operates combinatorial com- plexes (CCs), which generalize Euclidean grids, graphs, meshes, and hypergraphs. A CTNN is characterized by set of locally indexed copresheaf maps Px.-x,, defined between cells x, and x, in the CC. and directed from 2; to Iz. The figure illus- trates how CTNN updates local representation h2 of a cell x using neighborhood representations hy and h,, which are sent to via the learnable local copresheaf maps Pa- Py-rz and Pa-+z-
Hajij et al. 2023b [Bodnar al.) 2021a Ebli
et 2020 Giusti et 2023] within single formalism. Our approach further departs from the traditional assumption of single shared latent space by modeling task-specific, direc- tional latent spaces that bridge diverse deep learning frameworks. Furthermore, CTNNs flex- ibly handle both Euclidean and non-Euclidean data, supporting expressive architectures such copresheaf GNNs, transformers, and CNNs, which learn structure-awar directional transport maps. CTNNs offer a promising framework for developing next-generation, topologically informed, structure-awar machine learning models. See Figure [ for an illustration.
2 Related Work
Our work here is related sheaf neural networks (SNNs), which extend traditional GNNs by employing the mathematical framework of cellular sheaves to capture higher-order or heterogeneous relationships. Early work by Hansen and Ghrist (2019a b) introduced methods to learn sheaf Laplacians from smooth signals and developed a spectral theory that connects sheaf topology with graph structure. Building on these ideas, Hansen and Gebhart [2020) proposed the first SNN architecture, demonstrating that incorporating edge-specific linear maps can improve performance on tasks involving asymmetric or heterogeneous relations.
Recent advances have focused on mitigating common issues in GNNs, such as oversmoothing and heterophily. For instance, [Bodnar al. [2022) introduced neural sheaf diffusion processes that address these challenges by embedding topological constraints into the learning process. Similarly, Barbero al. 2022a b] developed connection Laplacian methods and attention-basel mechanisms that further enhance the expressiveness and efficiency of SNNs. The versatility of the sheaf framework has also been demonstrated through its extension hypergraphs and heterogeneous graphs (Duta et al.) 2023 Braithwaite et al.) 2024], which enables modeling of higher- order interactions. Moreover, novel approaches incorporating joint diffusion processes [Hernandez Caralt et al., 2024] and Bayesian formulations [Gillespie et al. 2024 have improved the robustness and uncertainty quantification of SNNs. Finally, the application of SNNs in recommender systems [Purificato et al., (2023) exemplifies their practical utility in woll-oordd domains. Together, these contributions demonstrate the potential of SNNs to enrich graph-based learning by integrating topological and geometric information directly into neural architectures. Our proposed CTNNs generalize these architectures while avoiding restrictive co-boundary maps or specifci Laplacian operators. Appendix T provides a more thorough literature review of related work.

---

3
Preliminaries
This section presents preliminary concepts needed for developing our theoretical framework. revisits CCs and neighborhood structures, reviews sheaves and copresheaves on directed graphs, and compares cellular sheaves with copresheaves in graph- modeling.
3.1
Combinatorial Complexes and Neighborhood Structures
To ensure generality, we base our framework on CCs (Haji et al. 2023b a which unify set-type and hierarchical relations which data defined. CC-neighborhoood functions then formalize local interactions forming a foundation for defining sheaves and higher-order message passing schemes for CTNNS.
2023b A CC is triple (S, X, rk), where Definition (Combinatorial complex (Hajij et al. is finite non-empty set of vertices, X C P(S) (0). with P(S) denoting the power set of S, and rk X' + Zz0 is rank function such that if {s} X. rk((s}) = 0 for all E S, and b = rk(x) rk(y) for all x, 1.yyE. E.
When context permits, we write CC (S, rk) simply as X. Each E X' has rank rk(x), and dim = max rk(x). We refer to elements of X' by cells. The k-cells =² of X are defined be the cells z with rk(x) = k. We use the notation X = f2 E X rk(x) = k] rk ({K)). See 2 for an example.
20
X
x²
Definition 2 (Neighborhood function). A neighborhood function on CC (S, S, rk) is map V : X' + P(X)), which assigns each cell x in X' collection of neighbor cells (x) C X, referred to as the neighborhood of x. In our context, two neighborhood functions are commonly used, namely the adjacency neighborhood Nadj( - [
Figure 2: A combinatorial
complex of dimension 2.
2, y }} and the incidence neighborhood Nino (x) {y X
x cC..
In practice, neighborhood functions are stored via matrices called neighborhood matrices.
Definition 3 (Neighborhood matrix). Let A be a neighborhood function on a CC X. Let y [uh, 3n C X' 2 = (.i,..., Zm C X be two collections of cells such that N(,;)) S 2 for all 1 S j S n. An element of the neighborhood marrix G {0, 1]mxn defined as
[G] = {1
if (N))))
otherwise.
The copresheaf structure that develop on CCs depends the neighborhood function. To introduce it, we first review sheaves and copresheaves on graphs, and then extend these notions to CCs.
3.2 Sheaves and Copresheaves on Directed Graphs
The copresheaf formalism assigns each vertex its unique feature space F(x)), respecting the potentially heterogeneous nature the data, and each directed edge a transformation Pa-t that tells how data move between those spaces, F(x) + F(y). This separation between where data reside and how they travel provides a foundation for learning beyond the single-latent-space assumption of standard deep learning architectures. Concretely, copresheaves are defined as follow!!]
Definition 4 (Copresheaf on directed graphs). A copresheaf (F, p, G) on directed graph G = (V, E) given by
a real vector space F(x) for every vertex & E V: linear map Pa-ty F(x) Fly) for every directed edge x k-> E.
Think of a copresheaf as a system for sending messages across a network, where each node has its own language (stalk), and edges translate messages (linear maps) to match the recipient's language. More specifically, on directed graph G = (V, E), each vertex x E V carries task-specific latent space
While we avoid overly complicated jargon, the appendix links our constructs to those of category theory for rigorous exposition.

---

F(x), and every edge 2 z E E applies a learnable, edge-indexed linear map Pr-Py F(z) + F(y) that re-embeds z's features into y's coordinate frame, thus realizing directional, embedding-level message passing throughout the network.
While much of the recent literature has focused on sheaf learning [Ayzenberg et al., 2025 J. our approach is based on a copresheaf perspective. This setup departs from the traditional deep learning core assumption of a single, shared latent space, enabling the modeling of heterogeneous, task- specific latent spaces and directional relations. The significance of this approach lies in its ability to generalize and connect different deep learning paradigms. Copresheaf- -type architectures extend beyond SNNs [Hansen and Gebhart 22020] Bodnar et al. [2022] Barbero et al. 2022b] Duta et al., 2023 Battiloro et al., 2024b) and TNN architectures [Hajij et al. [023b]. typically designed for non- non-Euclidean data, by also accommodating Euclidean data effectively. This versatility allows them to unify applications across diverse data domains and architectural frameworks, providing a unified
structure that uses directional information flow and adapts to task-specific requirements.
In graph- modeling, cellular sheaves provide formal framework to ensure data consistency in undirected graphs by encoding symmetric local-to-global relations via an incidence structure between vertices and edges. These structural points, formalized in the following definition, are encoded through restriction maps, ensuring data consistency between vertices and incident edges.
Definition (Cellular sheaf). Let G (V, E) be undirected graph. Let indicate that vertex E V is incident to edge e E E. A cellular sheaf on G' consists of:
a vector space F(x) to each vertex
V;
a vector space F(e) to each edge E E;
a linear restriction map Fzge: :(x)) + F(e) for each incidence 5 e.
For any cell c (vertex or edge), the vector space F(e) is typically called the stalk at c. The data on nodes x and y, denoted by h, E F(x) and hy E Fly). "agree" on the edge if their images under the restriction maps coincide:
F(a) Forr, F(e) Frar F(y)-
(1) A global section of sheaf on a graph G assigns data h, E F(v) to each vertex v and h. E F(e) to each edge e, such that for every edge between nodes x and y holds that Fa (ha) Fyee (hy). This consistency condition ensures data consistency across local connections in a network. Global sections represent equilibrium states, where this local agreement holds across the entire graph, enabling unified data representations for complex system. Most sheaf- based architectures have focused on diffusion -ype models [Hansen and Gebhart 2020 Bodnar et al.) 2022 Barbero et al. 2022b Duta et al. 2023 Battiloro et al.) 2024b), where the sheaf Laplacian A minimizes the Dirichlet energy, ensuring global consistency. Precisely, let (G; = D Dev F(v) and
(G;FF) = DDeF F(e) denote the spaces of vertex-valued and edge-valued cochains of the sheaf F. respectively. Then for some arbitrary choice of orientation for each edge, define the coboundary map
8: -
(2) (ōh). (oh).= Fase (hx) - Fyde (by) for (Z. E,
which measures local disagreement with respect to the edge e. The sheaf Laplacian AF = 8T aggregates all the restriction maps {Frse) } into single symmetric, positive semidefinite operator. Its associated quadratic form, h" Ay h, has a trace that defines the sheaf Dirichlet energy.
Unlike sheaves, which ensure data consistency across overlaps, copresheaves model directional data flow, making them well-suited for processes such as information propagation, causality, and hierarchi- cal dependencies. Copresheaves assign vector spaces F(z) only to vertices and define learnable linear maps P>- F(x) + Fly) along directed edges, without imposing sheaf consistency constraints. This vertex-centric anisotropic framework naturally integrates with message-passing architectures such as GNNs and TNNs, allowing parameterized maps to adapt during training. See Appendix [3.3 for further discussion, and Appendix for the definition and properties of the copresheaf Laplacian.
4 Copresheaf Topological Neural Networks
We are now ready to introduce copresheaf topological neural networks (CTNNs), higher-order message- passing mechanism that generalizes the modeling of relational structures within TNNs, as

---

illustrated in Figure 1 We begin by making use of copresheaves induced by neighborhood functions on CCs. providing structured way to model general, vycal-to-globay relations.
Let A' be a CC and W X - ((X) be neighborhood function. We define the effective support of V as the set Vx == (x E X' (xx) # 0). This set identifies cells that receive input from neighbors. The neighborhood function N induces directed graph GN = (Vvr EN), where the vertex set
VV= = U U N(x),
ZĒR
and the edge set is Fx == non-empty neighborhoods (targets) and their neighbors (sources). This graph determines how data propagates across the complex, with each edge encoding directional relation from neighbor y to target
(v E X EX,YE The vertex set VN includes both cells with
copresheaf), Let X' be a CC, .' neighborhood function, and GN (Vv, Ex) the induced directed graph with Vy = Xv U UREx (xx). An N -eeepeddnnt copresheaf assigns vector space F(x) to each Vx, and linear map Py-oz F(y) F(x) for each edge y - x E Ex-
Definition 6 Neeighbohhodd-dependee
See, Appendix B. 4 for concrete examples. When clear from context, we simplify the notation from JEN to F and Py Py- to Py-*æ.
Copresheaf neighborhood matrices. Having introduced neighhorhood matrices binary encodings of local interactions, we now generalize this notion to define copresheaf neighborhood matrices (CNMs). Instead of binary entries, CNMs consist of copresheaf maps between data assigned to cells a CC, allowing richer encoding of local dependencies. Subsequently, we define specialized versions, such copresheaf adjacency and incidence matrices, that capture specific topological relations, facilitating structured message-passing in our CTNNs.
In particular, define the k-cochain space be the direct sum C²XXFFN) DDeEx denote by Hom(FM(i) (N) (j)) the space of linear maps from the data at the i-th stalk to that the j-th stalk. Here, the maps encode how data is transferred or transformed between neighboring cells. We next define the CNM.
FN and
Definition 7 (Copresheaf neighborhood matrices). For a CC X equipped with a neighborhood function N, let (X, p, Gx) Gx) be a copresheaf' on X. Also let y ... yn S X and Z = (z1...,, } S. X' be two collections of cells such that A N(y;) S 2 for all 1 S j S n. The copresheaf neighborhood matrix of N with respect to y and Z is the m x n matrix G.V
2 E NW,)). P2.tv Hom(((z)) 0,
[G:1
otherwise.
The neighborhood function N determines the directed relationships between the cells, and the CNM encodes the interactions between cells induced by these relationships. collecting the maps of a cell from its neighbors. Next, analogous to Definition 2 we define copresheaf adjacency and copresheaf incidence matrices specialized forms of the general CNM.
Definition 8 (Copresheaf adjacencyficidence matrices). For fixed r, k. define N( (x) = {yr
X" 32 E xr+k, x I z, I 2} and Adtr (x) = tu E x x 3 y). The copresheaf adjacency matrix (CAM) Ar, |1a""xxa"" and copresheaf incidence matrix (CIM) B E 1xx"""
Pr. Py--a, E Hom((Fy))
if E EN adj (1z)) otherwise.
aAr =
(4)
Pri-ry E Hoom(F(z)
if 2 E Addri))) inc (y1), otherwise.
Br. =
(5)
These specialized matrices capture distinct topological relations. CAM encodes relations which cells share an upper cell, while CIM encodes relations in which one cell is incident to another.
Copresheaf-basse message passing. We now generalize traditional graph message-passing to heterogeneous and higher-order interactions involving cells of varying ranks multi-way relations
5

---

going beyond pairwise connections, by explicitly incorporating copresheaf structures defined over CCs. This leads to copresheaf- appresheafbbasdd based passagg-passing... flexible and expressive tool for capturing complex, multi-scale relations in structured data.
Definition 9 (Copresheaf message- paatgggg neural network). Let G (V, be a directed graph and (F, P. G) copresheaf. For each layer and vertex E V, let h E F(z). A copresheaf message-passinng neural network (CMPNN) is neural network whose meassage passing is defined by
h ((+1) = ( 1
a (y-t))EE
where Py- F(y) - F(x) is the linear map associated with edge y - I. a is learnable message function, + a cotinuttationnnvvaiian aggregator, and / a learnable update function.
Definition 9 establishes a unifying framework that generalizes graph-based message passing neu- ral networks (MPNNs) [Gilmer al., 2017] by incorporating learnable, anisotropic linear maps associated with directed edges. This formulation subsumes many standard GNNs, such as graph convolutional networks [Kipi and Welling 2017], graph attention networks (Veličković et al. 2018). and their variants [ by viewing message passing as copresheaf maps on directed graphs. Consequently. all architectures derived from these foundational GNN models fit within our copresheaf pressage-passing paradigm. Moreover, SNNs. which operate on cellular sheaves over undirected graphs, can be adapted into this framework by reinterpreting their mede-meddated transport operator as direct tortex-to-vertex morphisms on bidirected graph. This adaptation is detailed in the following theorem.
Theorem 1. (SNNs are CMPNNs) Let G = (V, E) be an undirected graph equipped with cellular sheaf assigning vector spaces to vertices and edges, and linear maps Fz for each vertex æ E e. Then for each edge = {a, y E E, the SNN message passing from to given by the composition
"ee o Fygee FFy) + F(x), can be realized as a single morphism Py-ax Fly) + F(x) in
copresheaf on the bidirected graph G" = E'), E' = {(I, v), (V. {2, E E} by setting
Py-+= = -FT zge o Fyge-
Theorem D establishes that the message-passingg scheme employed by SNNs can be interpreted as a special case of CMPNNs when restricted to bidirected graphs. This perspective generalizes most existing SNN architectures found in the literature, including those in Hansen and Gebhart (2020), Bodnar et al. [2022]. , Barbero et al. [2022b]. The map F
xde Fyae arises from composing the restriction map yae:: Fly) + F(e) with its adjoint F F(e) + F(x), thereby capturing both vertex-to-edge and edge-to-vertex transformations defined by the cellular sheaf. As consequence of this connection between SNNs and CMPNNs, diffusion-style updates (commonly used theaf-based models, such as those based the sheaf Laplacian) can be succinetly expressed within the CMPNN framework. This result is formally stated in Proposition| I with a full proof provided in Appendix E Proposition 1 (Neural-sheat diffusion [Bodnar al., 2022 as copresheaf peessag--passingg Let G = E) be an undirected graph endowed with a cellular sheaf F. Given vertex features H hhleev with h. E F(v), and learnable linear maps W1, W2, define the diffusion update
H = (A= @ (), @ HW)
(6)
where Ay = [LF has blocks Lyvee = Eoge Foger Ly = -FT F for u o 2, u S e, e. Then, H + can be expressed in the copresheaf pesssag--passing form of Definition 9
The next definition formalizes the notion of general multi Definition 10 (Copresheaf based higher-order message passing). Let X' be CC, and 97 = (N&+) a collection of neighborhood functions. For each k; let ( G...) be a copresheaf in which the maps N J.V, (y) + JN (x) define the transformations associated to the copresheaf. Given features h.! the next layer features are defined as
multi-way way propagation.
h (f-1) =
h
ON ( (820)))))))))
1=1 (xN)(()
where ON,, the message function, @ permutation-invariann aggregator over neighbors y E (x),.. @ combines information from different neighborhoods, and B is the update function.

---

Table 1: A unified view across domains and architectures CNNs, GNNs, Transformers, SNNs, and TNNs as instances of Copresheaf Topological Neural Networks (CTNNs) defined by neigh-
borhood graphs Gy and directional transports Py-ra-
Classical model
CTNN form
Domain /
Py / refs
CNN Li al. 2021
CopreshcaiConn
Grid; adjacency (CAM)
Transsation-cooniiseen transport (shared local
filters). App.[G
MPNN Gilmer et al.
2017
CMPNN
Graph; adjacency
Shared / edge-Indexed
linear maps; GCN/GAT special cases. Def. To
Euclidean / cellular
Copresheaf Transformer
Tokens grid/seq; full/masked adj
In-head value transport;
Tr. Vaswani et al. 2017
p == = dot-product attention. Sec. 5
Barshey et al. [025
App.
SNNnBod
2022
CMPNN on bidirected graph
Graph; incidence (CIM)
Puto F uode
Hansen and Gebharl
22000
(vertex-vertex) Prop.
Thm. App.
E B.3
TNN (hyper-
graplisimppicca/clllaaaCCC]CC Higher-order CMPNN
CC: multi-V via CAM/CIM
Rank-aware maps across
overlaps: multi-way @
jij al. [0023b]
Def.
Abbrev: CC combinatorial complex; C = copresheaf adjacencyfinccdenne; Adj/Inc= adjacencyfincidence.
Definition 10 lays the foundational framework that umifies a broad class of topological deep learning architectures, bridging higher-order message passing methods, transformers and SNNs. This synthesis not only consolidates existing approaches but also opens avenues for novel architectures based on
topological and categorical abstractions. Notably, the formulation in Proposition TO encompasses
simplicial message passing [Ebli et al. 2020 Bunch et al. 2020 Bodnar et al.) 2021b). cellular
message passing (Hajij et al., [2200] Bodnar et al., 2021a] stable message passing via Hodge
theory [Hayhoe et al.] 2022], and recurrent simplicial architectures for sequence prediction [Mitchel]
et al. 2024). It also subsumes more recent developments that harness multiple signals and higher-order
operators such as the Dirac operator [Calmon et al. 2022 Hajij et al.) 2023a] and TNNs [Hajji et al., 20235). These diverse models are unified under the copresheaf- neighborhood aggregation, feature transport, and signal interaction within a coherent framework. See Appendix E for derivations showing how several of these architectures emerge as special cases of this general formulation. See also Table for summary.
formulation by interpreting
Remark (Graph vs CC copresheaf models). Unlike graph-based models, which propagate in- formation edge by edge, copresheaf models on CC aggregate messages across all overlapping neighborhoods at once. Overlapping neighborhoods have common cells, potentially at different ranks, allowing simultaneous aggregation of multi-way interactions. Applying each neighborhood function Ng turn we compute its map-driven messages and then merge them into a single update.
5 Architectures Derived from the Copresheaf Fra Framework
Having established the abstract copresheaf- copresheaf-based framework on CC X, now present several concrete instantiations. Copresheaf transformers (CTs) extend the standard attention mechanism by dynamically learning linear maps Py-rx F(y) + F(x) encoding directional, anisotropic relationships between tokens, i.e., cells, in X. Integrating these maps into attention enables CT's to capture rich, structured interactions. Copresheaf graph neural networks (CGNNs) generalize message- passing GNNs by incorporating copresheaf linear maps to model relational structures. Copresheaf convolutional networks define convolution-like operations on CCs, modeled as a Euclidean grid, using these linear maps. We present the CT construction next and leave the exact formulations of copresheaf networks, CGNNs, and copresheaf convolution layers the appendices
and G
Copresheaf transformers. Having established the abstract framework of CTNNS on CC X, we now introduce a concrete instantiation: the copresheaf transformer (CT) layer. This layer extends the standard attention mechanism by dynamically learning linear maps Py- RR + Rd that encode both the combinatorial structure and directional, anisotropic relationships within the complex. By integrating these maps into the attention computation, the CT layer captures rich, structured
7

---

Table Mean squared error (mean standard deviation) of classical vs copresheaf architectures for learning various physics simulations.
Heat
Advection (Transformer)
Unsteady stokes (Conv-transformeer)
Network
(Transformer)
Classical
2.64 10-4 + 3.50 x 10-5
3.52 x 10-1 7.70
10-5
1.75 10-2 + 1.32 x 10
Copresheaf
9.00 x 10 7.00 x 10 1.20
10 1.20 x 10
1.48 10 + 1.48 10
interactions across X. At layer f, each cell X' is associated with feature (2) Rdd, where dz denotes the feature dimension of cell x.
Definition 11 (Copresheaf self-attention)) For a fixed rank & and neighborhood N₂ (c.g., adjacency between k-cells), let Wg. W R2xd W. E Rdxd denote learnable projection matrices, where p is the dimension of the query and key spaces, and is the feature dimension (assumed uniform across cells for simplicity). For each k-cell X², copresheaf self-attention defines the message (9) aggregation and feature update as h B(bb mm),, where m. = EyeNs(s) anyPy-oz(t, and
exp((uz; kig) VT)
Ozy Σ EyEN(( exp( (4us kv))
where Ou W Wi ke Wahe (6) and Up Wohhe Here, the softmax normalizes over all neighbors y' E (x(x)) (z) and Py-+2 F(y) + F(x) is the learned copresheaf map. The update function 3 is chosen to be neural network.
Similarly, we define copresheaf cross-attention among s and f rank cells in X, as well as a general algorithm for a corpresheaf transformer layer (Appendix FF.
6 Experimental Evaluation
We conduct experiments on synthetic and real data in numerous settings to support the generality of our framework. These include learning physical dynamics, graph classification in homophillic and heterophilic cases and classifying higher-order complexes.
6.1 Evaluations on Physics Datasets
To verify the validity of our networks in toy setups of different phenomena, we generate a series of synthetic datasets. These include:
Heat. We generate 600 realisations by solving the heat equation u; = Vuge on [0, 1) with 0.1 to horizon 7' = 0.1; each un is 10-mode sine series and uy is its Gaussian-kerree convolution, sampled on
100 grid points.
2. Advection. Similar to heat, we generate 600 realisations of u4 + cuu 0 with = 1; the solution a pure phase shift of tgg sampled on N 130 points. Each pair is normalized the interval [0. 1] and the dataset is split into 500:100 train/test samples.
3. Unsteady stokes. Let u1 y, 1) = (u( y, (), v(ar, ()) E TR²2 denote the incompressible velocity field, p(z, t) E IR the kinematic pressure, and v > O the kinematic viscosity. Throughout, O is the time derivative, V (0., 22)) the spatial gradient, and A Òrz + Oyp the Laplacian operator. The periodic unsteady Stokes system reads 0,u vAu + Vp = O, where V = 0. We synthesize 200 samples by drawing 8-mode Fourier Preaa--unction v/, setting un vー vb and evolving to T = 0.1 with v = 0.1 via the analytic heat kernel. Each pair is sampled on a 16 16 grid, with each channel normalized to the interval [0, 1). and the dataset is split into 160:40 train/test samples.
Model and training. For heat and advection, we use two transformer layers (positional encoding, four heads, stalk dimension equal to 16), followed by a mean pooling and linear head yielding 64D token embeddings. We train our networks using AdamW W wiht a learning rate of 10 33 cosine LR scheduling, and batch size of two. We use 50 epochs for the heat dataset and 80 for the advection dataset, and report the results over three seeds. For the unsteady Stokes data, we test compact cennoluiion-transfomon
U-Net consisting of a convolutional encoder with two input and 32 output channels, followed by two transformer layers (four heads, hidden dimension equal to 32, and stalk dimension 8), and a convolutional decoder mapping back to two output channels. We train it for 300
8

---

epochs using AdamW with a batch size of four. The classical baselines use dot- dot-product attention, whereas the copresheaf variants employ learned auter-product maps.
Results. As shown in Table [ in the heat and advection tests, copresheaf attention significantly outperforms classical dot-product attention, reducing the test MSE by over 50% and achieving more stable results across seeds. For unsteady Stokes, the copresheaf attention lowers the test MSE by 15% and reduces variance by an order of magnitude, confirming that pair-specific linear transports capture viscous diffusion of vorticity more faithfully than standard self-attention under identical compute budgets.
6.2 Graph Classification
We evaluate whether incorporating copresheaf structure into GNNs improves performance on graph classification tasks.
Data. MUTAG dataset, a nitroaromatic compound classification benchmark consisting of 188 molecular graphs, where nodes represent atoms and edges represent chemical bonds. Each node is associated with a 7-dimensional feature vector encoding atom type, and each graph is labeled as mutagenic or non- mutamenge (two classes). The dataset is split into 80% train and 20% test samples.
Baselines, backhone and training. We compare standard GNN models (GCN, GraphSAGE, GIN) against their copresheaf-counterpee below. All models are two-layer networks with a hidden dimension of 32 for GCN and GraphSAGE, and 16 for GIN, followed by global mean pooling and a linear classifier to predict graph labels. The standard models (GCN [Kipf and Welling 2017). GraphS (Hamilton et al. 2017). GIN (Xu et al., 2019]) use conventional GNN convolutions: GCN with symmetric normalization, GraphSAGE with mean aggregation, and GIN with sum aggregation. The copresheaf-enhancee with learned per-edge copresheaf maps, introducing local consistency constraints via transformations. All models are trained using Adam with a learning rate of 0.01, and batch size of 16. GCN and GIN models are trained for 100 epochs, while GraphSAGE for 50. The negative log- ligliiklihood loss minimized, and performance is evaluated via test accuracy. For GCN and GraphSAGE, we use five runs, while GIN uses ten runs.
(CopresheafGCN, CopresheafSage, CopresheafGIN) derived
models augment these
Enhancing GNNs via copresheaves. The copresheaf structure enhances each GNN by learning a transport map Pij I + Aij for each edge (i,j), where Aij is learned transformation and the identity matrix. In what follows D represents the dimension of the input feature in where we apply the copresheaf maps Pij. Denote by [h; hy) to the concatenation of the two node fea- ture vectors h, and h, along their feature dimension. The process for each model is as follows:
Table Mean test accuracy (#: std).
CopresheafGCN. For node features h;, hy, compute Ai tanh [Lincar( [h;: h,])), take its diagonal to get a diagonal x D matrix with D 7. Form Pig = I+ -ij- Aggregate neighbor features as h/ EjeN()) dd.d, Piyhi where d,, dj are the degrees of nodes and j, respectively. Combine with the self- feature: h" = -1 +1+ h/.
Model
Accuracy
GCN
0.674 1 0.014
CopresheafGCN
0.721 + 0.035
GraphSAGE
0.689 = 0.022
CopresheafSage. Compute Di tanh [b,, h D), diagonal matrix, and form Pij = I A.,. Aggregate via mean:
CopresheafSage
0.732 = 0.029
GIN CopresheafGIN
0.700 + 0.039
0.724 + 0.021
h mean h;), Combine: h, (1+ h/. The map Púj enhances local feature alignment.
CopresheafGIN. Compute A, = tanh(Linear (h,, h,))), a full D x D matrix, and form Psy = I + Aij. Aggregate: hÇ (peNro Combine:
h" = h1))) h/.
Results. On MUTAG, copresheff-enhanced GNNs consistently outperform their standard versions across GCN, GraphSAGE, and GIN. CopresheafS AGE achieves the highest average accuracy (0.732) and the largest relative gain over the GraphSAGF baseline (0.689), Learned per-edge transport maps better capture complex structure and enforce local consistency, improving classification. These results demonstrate the promise of copresheaf structures for molecular graph classification.

---

6.3 Combinatorial Complex Classification
Finally, we assess whether incorporating copresheaf structure into transformer baased attention mech- anisms improves performance on classifying higher-order, general data structures, such as CCs. Specifically, we compare classical transformer model against two CT variants (CT-FF and CT- Sharedl .oc) on a synthetic dataset of CCs.
Data. Our synthetic dataset comprises 200 training and 50 test CCs derived from Erdös-Rényí graphs, each with 10 nodes and base edge probability of 0.5. Triangles are added to form higher-order structures with probability q = 0.1 for class (low density) or q = 0.5 for class 1 (high density). Each node has 2D feature vectors, consisting of its degree and the number of triangles it participates in, with added Gaussian noise 0.10..
Backbone and training. All models transformer-basee classifiers with single block, using two attention heads, an embedding dimension of 8, and a head dimension of 4. The model embeds 2D node features, applies attention, performs global average pooling, and uses linear classifier to predict one of two classes (low or high triangle density). The classical model employs standard multi-head attention. The CT-FC and CT-SharedLoc models augment attention with learned per-edge transport maps to enforce local consistency. Models are trained for four epochs using Adam with a learning rate of 10-3 and batch size of 8, minimizing cross-entropy loss. Performance evaluated via test accuracy. Experiments are over four runs with different random seeds to ensure robustness.
Copresheaf attention. Similar to GNNs, the copresheaf structure enhances attention by learning a transport map Pij = I + Aij. The value vector v, is transformed as Pi before attention- weighted aggregation. The process for each model is as follows:
CT-FC. For node features h;, hy. compute A;j tanh(Linnar((P) h,])), full d x d matrix (d = 4). Form Pij = I + Dij. Apply Pij value vectors in attention: V, = PijY.. The map Pij enables rich feature transformations across nodes.
CT-SharedLoc. Compute a shared A, = tanh(MLPP(h,, h,l)), full x d matrix, and local scalar aij = (MLP([h, hyl)). Form Pty = I + any Dig. Apply Ptj to value vectors: V. = PiyVy. The map Pij balances shared transformations with local modulation.
Results. Copresheaf transformer models outperform the standard transformer on the CC classification task, with CT-SharedLoc achieving the highest average accuracy (0.970) and competitive stability (std 0.010). The learned per-edge transport maps Pij enhance the model's ability to capture higher- order structural patterns, such as triangle density, by aligning node features effectively. CT-SharedLoc' 's combination of shared transport maps and local modulation yields the best performance, showcasing the value of copresheaf structures in transformer-bast models for CC classification.
Table 4: Mean + std test accu- racy for CC classification.
Model
Accuracy
Classic
0.940 + 0.014
CT-FC
0.955 1 0.009
CT-SharedLoc
0.970 1 0.010
7 Discussion and Conclusions
We proposed CTNNs, a unified deep learning framework on (un)structured data. By develping models on copresheaves over CCs, CTNNS generalize GNNs, SNNs, and TNNs through directional, heterogeneous message passing. Besides theoretical advances, CTNNs offer empirical benefits across diverse tasks, laying the principles for multiscale and anisotropic representation learning.
Limitations and future work. CTNNs incur additional overhead from per-edge transformations and have so far been evaluated in modest-scale settings. We plan to explore well-engineered scalable parameterizations, extend CTNNs to large-scale and dynamic domains, and further connect categorical structure with robustness and inductive bias in deep learning.
Acknowledgments and Disclosure of Funding
Professor Adrian Lew's contributions to this publication were as a paid consultant and were not part of his Stanford University duties or responsibilities. T. Birdal acknowledges support from the Engineering and Physical Sciences Research Council [grant EP/XO1 1364/1]. Birdal was supported by UKRI Future Leaders Fellowship [grant number MR/Y01881 18018818/111 18/1] well Royal Society
10

---

Research Grant RG/RI/241402. The authors thank Hans Riess for pointing out the relationship between the CTNNs and the quiver Laplacian.
References
Samson Abramsky. Notes presheaf representations of strategies and cohomological refinements of k-consistency and k-equivalence. arXiv preprint arXiv:2206. 12166, 2022.
Ibrahem Al-Jabea and Thomas John Baird. arXiv: 01100, 2018.
Cohomology gkm-sheaves. arXiv preprint
Yash Atri, Karan Rungta, Lili Mou, Kai-Wei Chang, and Nanyun Joshi. Promoting topic coherence and inter-document consorts in multi-document summarization via simplicial complex and sheaf graph. In Proceedings of the 2023 Conference Empirical Methods in Natural Language Processing, pages 2154-2166, 2023.
Anton Ayzenberg. Thomas Gebhart, German Magai, and Grigory Solomadin. Sheaf theory: from deep geometry to deep learning. arXiv preprint a5X2v:2502.15476 2025.
Song Bai, Feihu Zhang, and Philip H. S. Torr. Hypergraph convolution and hypergraph attention. Pattern Recognition, 110:107637, 2021.
Rubén Ballester, Pablo Hernández- Mathilde Papillon, Claudio Battiloro, Nina Miolane, Tolga Birdal, Carles Casacuberta, Sergio Escalera, and Mustafa Hajij. Attending to topological spaces: The cellular transformer. arXiv preprint arXiv:2405. 14.40.14494 14094, 2024.
Jacob Bamberger, Federico Barbero, Xiaowen Dong, and Michael M. Bronstein. Bundle neural networks for message diffusion graphs. In ICML 2024 Workshop on Geometry-grounded Representation Learning and Generative Modeling, 2024.
Federico Barbero, Cristian Bodnar, Haitz S. Ocáriz Borde, Michael M. Bronstein, Petar Veličković, and Pietro Liò. Sheaf neural networks with connection laplacians. In Topology, Algebra, and Geometry in Machine Learning (TAG-ML) Workshop at ICML, pages 28-36, 2022a
Federico Barbero, Cristian Bodnar, Haitz S. de Ocáriz Borde, and Pietro Liò. Sheaf attention networks. In NewrlPS 2022 Workshop Symmetry and Geometry in Neural Representations, 2022b.
Melih Barsbey, Rubén Ballester, Andac Demir, Carles Casacuberta, Pablo Jernández-Gareía David Pujol-Perich, Sarper Yurtseven, Sergio Escalera, Claudio Battiloro, Mustafa Hajij, et al. Higher- order molecular learning: The cellular transformer. In ICLR 2025 Workshop Generative and Experimental Perspectives for Biomolecular Design, 2025.
Claudio Battiloro, Lucia Testa, Lorenzo Giusti, Stefania Sardellitti, Paolo Di Lorenzo, and Sergio Barbarossa. Generalized simplicial attention neural networks. IEEE Transactions on Signal and Information Processing over Networks, 2024a.
Claudio Battiloro, Zhiyang Wang, Hans Riess, Paolo Di Lorenzo, and Alejandro Ribeiro. Tangent bundle convolutional learning: from manifolds to cellular sheaves and back. IEEE Transactions Signal Processing, 12:18921 2024b.
Federico Battiston, Giulia Cencetti, Iacopo lacopini, Vito Latora, Maxime Lucas, Alice Patania, Jean-Gabriel Young, and Giovanni Petri. Networks beyond pairwise interactions: structure and dynamics. Physics Reports, 874:1-92, 2020.
Christian Bick, Elizabeth Gross, Heather A Harrington, and Michael T Schaub. What are higher-order networks? SIAM review, 65(3)=686-731, 2023.
Cristian Bodnar, Fabrizio Frasca, Nina Otter, Yuguang Wang, Pietro Lio, Guido F Montufar, and Michael Bronstein. Weisfeiler and Lehman go cellular: CW networks. Advances in Neural Information Processing Systems, 34:2625-2640, 2021a.
Cristian Bodnar, Fabrizio Frasca, Yuguang Wang, Nina Otter, Guido F Montufar, Pietro Lio, and Michael Bronstein. Weisfeiler and Lehman go topological: message passing simplicial networks. pages 1026-1037, 2021b.
11

---

Cristian Bodnar, Fabrizio Frasca, Yuguang Wang, Nina Otter, Pietro Liò, Guido Montúfar, and Michael Bronstein. Neural sheaf diffusion: A topoloical perspective on heterophily and over- smoothing in gnns. Advances in Neural Information Processing Systems, 35:18527-18541, 2022.
Luke Braithwaite, Iulia Duta, and Pietro Liò. Heterogeneous sheaf neural networks. arXiv preprint arXiv:2409.08036, 2024.
Glen E. Bredon. Theory, volume 170 of Graduate Texts in Mathematics. Springer, 1997.
Michael M. Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geometric deep learning: going beyond Euclidean data. IEEE Signal Processing Magazine, 34(4):18 42, 2017.
Eric Bunch, Qian You, Glenn Fung, and Vikas Singh. Simplicial 2-complex convolutional neural nets. NeurlPS Workshop on Topological Data Analysis and Beyond, 2020.
Lucille Calmon, Michael T. Schaub, and Ginestra Bianconi. Higher-order signal processing with the Dirac operator. In 56th Asilomar Conference on Signals, Systems, and Computers, pages 925-929. IEEE, 2022.
Adam Ó Conghaile. Cohomology in constraint satisfaction and structure isomorphism. arXiv preprint arXiv:2206.15253, 2022.
Justin Michael Curry. Sheaves, Cosheaves and Applications. PhD thesis, University of Pennsylvania, 2014.
Julia Duta, Giulia Cassarà, Fabrizio Silvestri, and Pietro Liò. Sheaf hypergraph networks. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 2023.
Stefania Ebli, Michaël Defferrard, and Gard Spreemann. Simplicial neural networks. NeurlPS Workshop on Topological Data Analysis and Beyond, 2020.
Yifan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, and Yue Gao. Hypergraph neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):355--336 2019.
Michael Fourman, Christopher Mulvey, and Dana Scott, editors. Applications of Sheaves: Proceed- ings of the Research Symposium on Applications of Sheaf Theory to Logic, Algebra and Analysis, volume 753 of Lecture Notes in Mathematics. Springer, 1977.
Thomas Gebhart, Jakob Hansen, and Paul Schrater. Knowledge sheaves: A sheaf-theoretic framework for knowledge graph embedding. In International Conference on Artificial Intelligence and Statistics, pages 9094-9116. PMLR, 2023.
Robert Ghrist and Yasuaki Hiraoka. Applications of sheaf cohomology and exact sequences to network coding. In NOLTA, 2011.
Patrick Gillespie, Vasileios Maroulas, and Ioannis D. Schizas. Bayesian sheaf neural networks. arXiv preprint arXiv:2410.09599. 2024.
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In Inf. Conf. Mach. Learn., pages 1263-1272. PMLR, 2017.
Lorenzo Giusti, Claudio Battiloro, Lucia Testa, Paolo Di Lorenzo, Stefania Sardellitti, and Sergio Barbarossa. Cell attention networks. In 2023 International Joint Conference on Neural Networks (IJNN), pages 1-8. IEEE, 2023.
Joseph A. Goguen. Sheaf semantics for concurrent interacting objects. Mathematical Structures in Computer Science, 2(2): 191, 1992.
Christopher Wei Jin Goh, Cristian Bodnar, and Pietro Lio. Simplicial attention networks. In ICLR 2022 Workshop on Geometrical and Topological Representation Learning. 2022.
Mustafa Hajij, Kyle Istvan, and Ghada Zamzmi. Cell complex neural networks. NeurIPS 2020 Workshop TDA and Beyond, 2020.
12

---

Mustafa Hajij, Karthikeyan Natesan Ramamurthy, Aldo Saenz, and Ghada Zamzmi. High skip net- works: a higher order generalization of skip connections. In ICLR 2022 Workshop on Geometrical and Topological Representation Learning, 2022.
Mustafa Hajij, Ghada Zamzmi, Theodore Papamarkou, Aldo Guzmán- Tolga Birdal, and Michael T Schaub. Combinatorial complexes: bridging the gap between cell complexes and hypergraphs. arXiv preprint arXiv:23/2.09504 2023a.
Mustafa Hajij, Ghada Zamzmi, Theodore Papamarkou, Nina Miolane, Aldo Guzmán-Sáenz, Karthikeyan Natesan Ramamurthy, Tolga Birdal, Tamal K. Dey, Soham Mukherjee, Shreyas N. Samaga, Neal Livesay, Robin Walters, Paul Rosen, and Michael T. Schaub. Topological deep learning: going beyond graph data, arXiv, 2023b.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Adv. Neural Inform. Process. Syst., 30, 2017.
Jakob Hansen and Thomas Gebhart. Sheaf neural networks. In Workshop on Topological Data Analysis and Beyond, 2020.
Jakob Hansen and Robert Ghrist. Learning sheaf laplacians from smooth signals. In ICASSP 2019- 2019 JEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages IEEE, 2019a.
Jakob Hansen and Robert Ghrist. Toward spectral theory of cellular sheaves. Journal of Applied and Computational Topology, 3(4):315-358, 2019b.
Mikhail Hayhoe, Hans Riess, Victor M Preciado, and Alejandro Ribeiro. Stable and transferable hyper-graph neural networks. arXiv preprint arXiv:2211.06513 2022.
Ferran Hernandez Caralt, Guillermo Bernárdez Gil, Iulia Duta, Pietro Liò, and Eduard Alarcón Cot. Joint diffusion processes as an inductive bias in sheaf neural networks. In Proceedings of the Geometry-grounde Representation Learning and Generative Modeling Workshop (GRaM), vol- ume 251 of Proceedings of Machine Learning Research, pages 249-263. PMLR, 29 Jul 2024.
Yiming Huang and Tolga Birdal. Hog-diff: Higher-order guided diffusion for graph generation. arXiv preprint arXiv:2502..m 04308, 2025.
Jianwen Jiang, Yuxuan Wei, Yifan Feng, Jingxuan Cao, and Yue Gao. Dynamic hypergraph neural networks. In IJCAL, pages 2635-2641, 2019.
Eun-Sol Kim, Woo Young Kang, Kyoung-Woo Woon On, Yu-Jung Heo, and Byoung- Tak Zhang. Hyper- graph attention networks for multimodal learning. In IEEE Conf. Comput. Vis. Pattern Recog.. pages 14581-14590, 2020.
Thomas Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey F. Hinton. ImageNet classification with deep convolu- tional neural networks. In Advances in Neural Information Processing Systems, pages 1097-1105, 2012.
Henry Kvinge, Brett Jefferson, Cliff Joslyn, and Emilie Purvine. Sheaves as a framework for understanding and interpreting model fit. In Proceedings the IEEE/CVF International Conference on Computer Vision, pages 4222-4230, 2021.
Zewen Li, Fan Liu, Wenjie Yang, Shouheng Peng, and Jun Zhou. A survey of convolutional neural networks: analysis. applications, and prospects. IEEE transactions neural networks and learning systems, 33(12)::699-7000 -0199, 2021.
Wenfei Liang, Yanan Zhao, Rui She, Yiming Li, and Wee Peng Tay. Fedsheafhn: Personalized federated learning graph-structured data. CoRR, 165/24 2024.
Edward C. Mitchell, Brittany Story, David Boothe, Piotr J. Franaszczuk, and Vasileios Maroulas. A topological deep learning framework for neural spike decoding. Biophysical Journal, 2024. doi: http://bo..org/10.00110.....pppp 202...0.....
13

---

Bao Nguyen, Lorenzo Sani, Xinchi Qiu, Pietro Liö, and Nicholas D. Lane. Sheaf hypemeeworrss for personalized federated learning. 2024. 2rX2v:2405.20882.
Theodore Papamarkou, Tolga Birdal, Michael Bronstein, Gunnar E Carlsson, Justin Curry, Yue Gao, Mustafa Hajij, Roland Kwitt, Pietro Lio, Paolo Di Lorenzo, et al. Position: Topological deep learning is the new frontier for relational learning. In International Conference on Machine Learning, pages 39529-39555 PMLR, 2024.
M. Papillon, M. Hajij, A. Myers, F. Frantzen, G. Zamzmi, H. Jenne, J. Mathe, J. Hoppe, M. Schaub, T. Papamarkou, A. Guzmán-Sáenz, B. Rieck, N. Livesay, T. Dey, A. Rabinowitz, A. Brent, A. Salatiello, A. Nikitin, A. Zia, C. Battiloro, D. Gavrilev, G. Bökman, G. Magai, G. Bazhenov, G. Bernardez, Spinelli, J. Agerberg, K. Nadimpalli, L. Telyatninkov, L. Scofano, L. Testa,
M. Lecha, M. Yang, M. Hassanin, O. H. Gardaa, O. Zaghen, P. Hausner, Snopoff, P. Melnyk, R. Ballester, S. Barikbin, S. Escalera. S. Fiorellino, H. Kvinge. J. Meissner, K. N. Ramamurthy, Sanborn, T. Emerson, M. Scholkemper, Rosen, R. Walters, S. N. Samaga, S. Mukherjee,
T. Doster, T. Birdal, V. Grande, A. Khamis, S. Scardapane, S. Singh, T. Malygina, Y. Yue, and N. Miolane. ICML 2023 topological deep learning challenge: design and results. In Proceedings of 2nd Annual Workshop on Topology. Algebra, and Geometry in Machine Learning (TAG-ML), volume 221, pages 8.8. PMLR, 2023.
Robert Peach, Matteo Vinao-Carl, Nir Grossman, Michael David, Emma Mallas, David Sharp, Paresh A Malhotra, Pierre Vandergheynst, and Adam Gosztolai. Implicit gaussian process repre- sentation vector fields over arbitrary latent manifolds. In The Twelfth International Conference Learning Representations.
Antonio Purificato, Giulia Cassarà, Federico Siciliano, Pietro Liò, and Fabrizio Silvestri. Sheaf4rec: Sheaf neural networks for graph-based recommender systems. ACM Transactions on Recommender Systems, 2023.
Michael Robinson. Topological signal processing, volume 81. Springer, 2014.
Amit Singer and H-T Wu. Vector diffusion maps and the connection laplacian. Communications on pure and applied mathemutics, 6566)11067-1144, 2012.
Yellamraju V. Srinivas. A sheaf-theoretic approach to pattern matching and related problems. Theoretical Computer Science, 112(1):53-97, 1993.
Otto Sumray, Heather A Harrington, and Vidit Nanda. Quiver laplacians and feature selection. arXiv preprint arXiv:2404.0699 2024.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Petar Veličković. Message passing the way up. ICLR 2022 Workshop on Geometrical and Topological Representation Learning, 2022.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations, 2018.
Amne Wolf and Anthea Monod. Topological community detection: sheaf-theoretic approach. In International Conference on Complex Networks and Their Applications, pages 29-42. Springer, 2023.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In International Conference on Learning Representations, 2019.
14

---

Copresheaf Topological Neural Networks: A Generalized Deep Learning Framework
-Supplementary Material-
Table of Contents
Notation
16
Sheaves and Copresheaves on Graphs: A Category Theoretical Look
16
Copresheaf Laplacian, Energy, and CTNN Transport-diffusion
21
D Expressive Power of CTNNS
22
Sheaf Neural Networks Are Copresheaf' Message-Passing Neural Networks
23
A General Copresheaf- -Base Transformer Layer
25
Copresheaf Learning on Euclidean Data
26
Experiments
27
Extended Related Work on Topological and Sheaf Neural Networks
37
15

---

A
Notation
We provide reference summary of the notation and acronyms used throughout the main text. Table details key mathematical symbols, while Table 6 lists abbreviations and their expansions.
Table 5: Summary of key notation used throughout the paper.
Notation
Description
Underlying vertex set a combinatorial complex (Def. 1)
Set nonemply cells S P(S) (Def. 1)
P(S)
Power set S (Def. 1)
kk: Zzo
Rank function on cells. mapping to non-negative integers (Def. 1)
Z20
Non-negative integers
x
{ar E X' : rk((x) = kj. the k-cells (Def. 1)
dim
Dimension of the complex, maX rk(x) (Sec. 2.1)
N: N' P(2)
Neighborhood function, mapping cells to sets of neighbor cells (Def. 2)
Nag
Adjacency neighborhood function (Def. 2)
Nic
Incidence neighborhood function (Def. 2)
XN
Effective support of NV, far E X (ax) * 0} (Sec. 3)
GE (0, 1)mxX
Binary neighborhood matrix (Def. 3)
GV
Copresheaf neighborhood matrix, entries Pı,-æ, or (Def. 7) Copresheaf morphism Fly) - F(x) for edge - (Def. 4) Copresheal udjacency matrix between r-cells (Def. 8)
Py-t
Ar
B.
Copresheuf incidence matrix between ranks r and & (Def. 8)
CF(
k-cochain space, Dexex F(x) (Sec. 3)
(lo)(F())FFU))
Space of linear maps from F(f) F()) (Def. 7)
h
Feature vector at cell x: in layer (Prop. 1)
Learnable message function (Prup. 1)
8
Learnable update function (Prop. 1)
Permutation cormutaatooninvariiin invariant aggregator (Prop. I, Prop. 3) Directed or undirected graph (Sec. 2.2)
== EV.
GN (Nv, EN)
Directed graph induced by N. edges - x if E ((x) (Def. 6)
Flc), Fle)
Stalks: vector spaces at vertex or edge (Def. 4, Def. 5)
Jage: F(u) 7 F(e)
Restriction map in cellular sheaf for S (Def. 5)
8
Coboundary map, measures local disagreement in cellular sheaf (Sec. 2.2)
Ar
Sheaf' I aplacian, A, = 8 (Sec. 2.2)
V. 2
Collections of cells, used in neighborhood matrices (Def. 3, Def. 7)
Wq. Wh, W.
Learnable projection matrices for queries, keys, values in copresheaf self-attention (Prop. 4)
4z kz. U/2
Query. key, and value vectors for cell x copresheaf self-attention (Prop.4) Attention coefficient for cells and y in copresheaf self-attention (Prop. 4)
0gg
B Sheaves and Copresheaves on Graphs: A Category Theoretical Look
This appendix provides tahegory-theoretic exposition of sheaves and copresheaves, emphasizing their definitions within the language of category theory. Additionally, we illustrate the construction of copresheaf neighborhood matrices through explicit combinatorial examples, instantiating the concepts developed in the main text.
B.1 Copresheaves
Before diving into the technical definition, it helpful think of a copresheaf way of assigning data that flows along the structure of graph, like signals along neurons, resources in a network. In categorical terms, this structure formalizes the idea of consistently associating elements some category C.
Definition (Copresheaf on a directed graph). Let G EV,E) be directed graph, and let category. A copresheaf G is a functor F G - C, where the graph G is regarded as category whose objects are the vertices V. and whose morphisms are the directed edges (#r -> y) F. When C Vectg, this structure corresponds to quiver representation.
16

---

Table List of acronyms used throughout the paper.
Acronym
Expansion
CC
Combinatorial
Complex
CTNN
Copresheaf Topological Neural Network
CMPNN
Copresheaf Message-Passingg Neural Network
SNN
Sheaf Neural Network
GNN
Graph Neural Network
CNN
Convolutional Neural Network
CT
Copresheaf Transformer
CGNN
Copresheaf Graph Neural Network
GCN
Graph Convolutional Network
GraphSAGE
Graph Sample and Aggregate
GIN
Graph Isoorpphssm Network
CopresheafGGNN
Copresheaf Graph Convolutional Network
CopreshcafSage
Copresheaf Graph Sample and Aggregate
CopresheafGINN
Copresheaf Graph Isomorphism Network
NSD
Neural Sheaf Diffusion
SAN
Sheaf Attention Network
MLP
Multi-Layer Perceptron
GAT
Graph Attention Network
CAM
Copresheaf Adjacency Matrix
CIM
Copresheaf Incidence Matrix
CNM
Copresheaf Neighborhood Matrix
B.2 Cellular Sheaves
Definition 13 (Cellular sheaf on an undirected graph). Let EV be an undirected graph. Define the incidence poset (P, s), where P = U E, and the order relation is given by x S whenever vertex æ E V is incident edge e E. A cellular sheaf on G with values category is a functor F P - C, which assigns:
each vertex E V, object F(x) EC:
to each edge e E E, an object F(e) EC:
each incidence relation x S e, a morphism Fage: : F(x) F(e), called restriction map, such that the functoriality condition is satisfied on composable chains in the poset.
B.3 Comparison Between Copresheaves and Cellular Sheaves
Copresheaves provide versatile and powerful framework for machine learning applications across diverse domains, particularly excelling in scenarios where directional data flow and hierarchical dependencies are paramount. Unlike cellular sheaves, which are defined over undirected graphs and enforce consistency through restriction maps Fase F(x) + F(e), copresheaves operate on Fly) along edges. This enables
directed graphs, assigning learnable linear maps Pa-ty anisotropic information propagation, making them ideal for tasks such as physical simulations, where data flows asymmetrically, as in fluid dynamics or heat transfer, or natural language processing, where sequential word dependencies dominate. More importantly. the vertex-centris design, assigning vector spaces F(x) solely to vertices, aligns seamlessly with message-passing architectures like Graph Neural Networks (GNNs) and Topological Neural Networks (TNNs), allowing these maps to be parameterized and optimized during training. Empirical evidence from our experiments highlights that Nopresheaf-basee models outperform traditional architectures in capturing complex dynamics, demonstrating their superior ability to model spatially varying patterns and long-range dependencies in general applications. See Table 7 for summary of the comparison between copresheaves and cellular sheaves.
Furthermore, copresheaves enhance machine learning models with a principled approach to regu- larization and expressiveness, broadening their suitability across heterogeneous domains. Standard neural network regularizers, such f2 decay dropout, be readily applied to copresheaf maps,
17

---

with optional structural losses like path connisssstt ensuring morphism compositionality. This adaptability stands in stark contrast the rigid cohomological constraints of cellular sheaves. which enforce local-to-glote agreement through terms like |Fage (h,) hell? limiting their flexibility in domains with asymmetric relationships. Copresheaves, by learning edge-wise maps, offer greater expressiveness for tasks involving non-Euclidean or multi-scale data, as evidenced by the superior performance of CopresheafConv layers on grid-based tasks. This makes them particularly effective for applications such as image segmentation, 3D mesh processing, token-relation learning, where traditional methods like Convolutional Neural Networks (CNNs) struggle with directional or hierar- chical structures. Our experiments further corroborate that improve accuracy and detail recovery across diverse tasks, positioning them as a more suitable and generalizable tool for machine learning applications spanning Euclidean and non-Euch -on-Euclidean domains alike.
models consistently
Table 7: Comparison between copresheaves and cellular sheaves.
Aspect
Copresheaf
Cellular sheaf [Hansen and Ghrist, 2019b
Graph type Assigned to vertices
Directed graph G = EV,
Undirected graph G = (V. E) Vector spuce F(a) for each V
Vector spuce F(a) for each E V
Assigned edges
Linear map Px-yy F(xr) F(y) for each di-
Vector space F(e) for each edge E
rected edge x - 6
Associated maps Map direction Interpretation
Pushforward: moves data forward along edges F(x) + F(y) (source to target)
Restriction: pulls data back from vertices to edges F(x) + F(e) (vertex to incident edge)
Nodes have local features; edges transform and transmit
Edges represent shared contexts; vertex features are restricted
Goal / Objective
Leam and compose edge-wise feature space maps Stundard NN regularizers (l, decay. spectrul- norm, dropout, norm); optional structure lnsses (path-consistencyy holonomy)
Enforce coherence across by gluing local data
Typical Regularization
Agreement between restricted vertex features and
the edge-stalk, e.g. I|Sss. (h,) h.ll
Use learning
Emnhedinng- level message passing, directional in- fluence, anisotropic information flow
Compatibility across shared structures, enforcing local consistency. cohomological constraints
B.4 Copresheaf Neighborhood Matrix Example
Example 1. Setup. Let the combinatorial complex X' = (S, have symbols S = {a, b, d} and cells
x= = {{a), (c),(c).)d))
X [(a, (b,c), (c,a), (d,b), (c
X² = (d.,b,
Ranks satisfy r2(2)) = O, rk(x)t) 1, rk(x²) = 2. Geometrically, this is the union of two triangles (a, b, c) and (d, c) sharing the edge (b. c).
e3, es, es] with Let E [e1, e2, 63,
c==(a,b):
(b=
cs=(c,a,,
c==(d,b), c=f(c.d.
Edge-via-face neighborhood. Define No : x + P(X')) by
(e(e) { x' Aʃ x² with e C J, e' C, &
Thus, two edges are neighbors iff they both bound the same 2-cell. Concretely,
(eie eah, (a(e2) (ei, e4, est. (ea) = (e1, Na(es) = (c2,05). Na(es) = (12
The effective support is Xx- X (edges only).
Induced directed graph. The induced directed graph GN₂, = (VN, Ex) has vertices Vyv = x and directed edges
Ex = (e' + -elee e' (c)
Messages flow from an edge e' to an adjacent edge whenever both bound common triangle.
Copresheaf on the edge-adjacency poset. Assign to every edge a feature space F(e) = TR. For each directed adjacency e' e, attach linear map Pe RRa - R²a. To keep notation compact,
18

---

we write diagonal maps as (aag(a,3) 3) and Id, for the 2 x 2 identity:
Pea diag(1, 11,,0,000 0.8),
Pes-re diag(1, 0.6);
Per = diag(1, 0.8),
Pea-ez diag(1 [, ..8))
Pei-te Id2, Pes-te2 = diag(110.6))
Pei+te (agg(1,06))
Peg- diag(1, 0.8);
Pez-tes = Id2: Pes-tes diag(1,0.7);
Pez-tes (.,g(,,,)), Pei-zes (1.77(1.
(Any consistent choice works; the point is a map per directed adjacency e' + e.)
Copresheaf Neighborhood Matrix (CNM). For the ordering [C1, 62 c3, Cs, Cs, (s], the CNM
GNa E (R²2x2) 6x6 has block entries
[GNN]es
Pey O2x2,
e E (N((e,,
otherwise.
Displayed explicity as:
O2x2
Pez-be1
Pey-bes
02x2
02x2
Pei-te2
02x2
Pas Pe2-te2
Par-te2
Pes-te2
GNa
Pei-tes
Peg-h03
02x2
02x2
02x2
02x2
Pog-hi4
02x2
02x2
Peg-tea
02x2
Peg
02x2
Pea-te
02x2
See Figure g
O2w2
Pry-te
Oaxz
02x2
Pel-tez
02x2
Pei-eex
Pey-tes
01xa
Oax2
02x2
0zx2
Pog-tes
02w2
02w2
65
0xx2
Pey-tes
Ox
Pea
02
(a)
(b)
(c)
Figure 3: (a) combinatorial complex X (S with S fa, c, d. edges
X' {aa b), {b, ct. ac (d. b), fc, d)), and faces X² = (a b, (d, b)) (b) Induced adjeeedjy digraph: nodes represent the edges of X' and the edges represent the face adjacencies. (c) Copresheaf neighborhood matrix G.V2.
This CNM performs edge-to-edge directional message passing along face-adjacency: each edge e; aggregates transformed features from its face-adja adjeeenjaeent neighbors e, via the maps Pe, The shared edge naturally becomes high-degree conduit between the two triangles.
Example We define copresheaf neighborhood matrix (CNM) for a combinatorial complex with an incidence neighborhood, guiding the reader through the setup, neighborhood, graph, copresheaf, and matrix.
Setup. Consider combinatorial complex X (.r with S ba, cells X {a), (b), (c), {a, b). (b, c}}, and ranks rk(fa)) = rk((b)) = rk((c)) O, rk (a, b)) =
rk((b,c)) 1. Thus, X (6), {c}}. X = ((a, b), (b, c).. See Figure
Incidence neighborhood. The incidence neighborhood Wine : X + P(X) is:
Nimo = {v X 2 v}. For 0-cells: (fa)) b), ((b)) = {fa, b}. (b,c)], Nune (c]) (mn(ff)) Nine(a, b}) Nine (bb, ct) = The effective support is Xx = ({a), (b), {c}}.
c)). For 1-cells:
Induced graph. We induce directed graph GN (Vv, Ex), with:
Viv Xy U U
Min {{a), (6), (c), (a. 6), (b,c)) X.
ZEN
Ex- = {y + + Xv. E (x))
= [[aabb] + (a), (a. + (b), (b. + (b, + (c))
19

---

((b)))+(())
((b)))+
{a
(b,c)
p(a,b)-((a)
0
{a.b}
(b,c)
G N-vine
((a,b)-(b)
P(b,c)++(b)
0
(())
P1) P(ho)-at))
(a)
(b)
(c)
(S, with S = fa,b,c), cells X' {a), (a), (c), fa, b), (b,,)) The figure also indicates the induced directed graph GNL.. (Vr, Ex) from the incidence neighborhood structure on the combinatorial complex X. Each arrow 2 represents directed edge from 1-cell to a O-cell where y C 2, and associated with linear map Ps-v as part of the copresheaf. (b) The induced directed graph GNi,.. (Vx, EE)) from the incidence neighborhood structure Nine- (c) The copresheaf neighborhood matrix (CNM) GAii,, where rows are indexed by O-cells {a), {b), {e} and columns by 1-cells {a, b), {6, c}. The matrix entries are linear maps P2-ty when iic (v), and 0 otherwise. This matrix supports directional feature propagation from I-cells to O-cells.
Figure 4: (a) A combinatorial complex X'
Copresheaf. The "needddedd copresheaf assigns F(x) I23 to each Vv, and linear maps Py-tz R22 + R.2 for y + x E Ex:
0 P(a.))+(b) = -(b,c)-
[ 0
1
0
((a.)-((a)
0 0.5
0 1
P(),o)-(()
0
0.75
Neighborhood matrix. The CNM G. for y =x,0 2 x2 is:
[GM $100
P2,
if E (B)),
otherwise.
For y [b) (ct), 2 {fa, b). (b,
((a,0)+(()) Plab)-)16) 0
GV
((b,c)-(6) (())c)-(e)
This matrix facilitates message passing from 1-cells to 0-cells, e.g., bond-to-atom feature propagation.
More generally, F illustrative example of the general setup of copresheaf higher-order message passing on a CC with multiple neighborhood functions.
(
Na
(x)
N2(x)
No EN
(z)
Na
Figure 5: Illustration of copresheaf higher-order message passing. Left-hand side: Shows a central cell x (as circle) with arrows pointing to boxes labeled the collection of neighborhood functions D7 = (Ni ) Right-hand side: Depicts the message- passing process for the same cell Each neighborhood Nx. (xr) produces aggregated message
(N((),, representing
using + (E) aN,, (Py-z (f))). )). These messages are then combined using the inter-neighborhood function 3, shown as a box, with an arrow updating s.
20

---

C Copresheaf Laplacian, Energy, and CTNN Transport-diffusion
In this section we introduce linear transport-discrepancy operator Bp that maps node fields edge discrepancies, define the quadratic energy E, and the associated Laplacian Lp B, W Bp. derive its block form and kernel, and show that a CTNN residual layer implements an explicit gradient step that monotonically decreases Ep for suitable step sizes. This energy perspective provides an interpretable and theoretical foundation for the CTNN architecture, framing it as a diffusion process that minimizes specific energy function. Our copresheaf I. Laplacian also coincides with the quiver Laplacian [Sumray et al. 2024] when viewing the transport maps as arrow representations, connecting our framework established spectral methods while enabling directional, anisotropic information flow distinct from the symmetric diffusion in theaf-based models Bodnar et al. [2022].
Spaces and operator. Let (F, p. G) be copresheaf defined on directed graph G (V, E). To each node E V attach a dinen--imensional real inner- anner-product space F(r). For every edge y - z E E fix linear transport Py-ra Fly) -> F(x). Define the node and edge product spaces
Hy
F(x), HE
F(x),
ÆEV
(N-z)EE
equipped with canonical (blockwise) inner products (h, h)v = Exev (h lan and (5,E)
((y= Ey-z)-
Definition 14 (Transport operator). The linear operator
Bp: Hy - HE
defined componentwise by
(B.h)( h Py-ra hy F(x),
V x)
(8)
Let BJ He Hy denote the Euclidean adjoint. direct calculation yields
E (B. = Σ EN-o t リー
V Hg.
(9)
Definition 15 (Energy and copresheaf Laplacian). Let wy-sr > O be edge weights and define the diagonal operator W HE - HE by (WE) Wy-ro Ey-ra- The weighted transport energy and the copresheaf Laplacian are
(b) |B,bll
Σ (g-x)EE
w wy-r h2 Po- = W Bp
(10)
Remark 2 (Relation to quiver Laplacian). Interpreting the transports Py arrow maps of a quiver representation, copresheaf Laplacian Lp Equation coionciies with the standard (weighted)
quiver Laplacian BT W of that representation. See Sumray al. 2024 for more about the quiver Laplacian.
It is sometimes useful to have the copresheaf Laplacian in its exact nodewise form. Namely, for every outgoing edge x >> z with map Pa->2 and weight Wz-22, include the reverse edge = + x with P=-tz == PI- and Wz-z = Wx-+2- With this convention one has the nodewise form of the copresheaf Laplacian:
(L,h)2
Σ VEN(2)
Wy-re (h+ Py-rohy).
(11)
Theorem 2. With wy > 0, T, = BT WBp Hv - Hv is symmetric positive semidefinite and
h L,h |BB 2 0. Its block action at node is
(L,h)2 = Σ Wyro (be Py-zzhy) + Σ
Wh-sz Po-z(Po-ssh - h.).
wーvx
x-22
Moreover,
ker Lp = ker B, = {he Hv hg = Py- for all - x E E}.
Proof. P.s.d. and the quadratic identity follow from L, = BJ W B, with W > 0. The block formula follows by expanding BJ (W (BBh)) via 10 Finally, | i 0 + B₁ 0.
21

---

C.1 CTNN/CMPNN residual as copresheaf diffusion.
choose the edge message ab2, Py-rhy) h2 Py-tzby sum aggregation, and residual update B(h,, m) = he mym. Then CTNN/CMPNN layer updates
From the copresheaf message passing equation in Definition 9
h (++)_ = h 17
Σ vEN(m)
h Py Pa-ra b.a))))) = (1-n h
(14)
We have the following theorem.
Theorem 3. For E in (13),
h(+t)) = h - ( E ( (1!)))
(E, = 2Lp
Hence (14) is an explicit gradient step on Ep. IfO < < || 12- then Ep(h(+t)) I (h with strict inequality whenever ((() & ker Lp. Moreover, 1 - n is non-expansive in || ll and strictly contractive on ker L h()) converges to the orthogonal projection of h(") onto ker Lp.
Proof. Since (h) (B,h) T (BPh), the chain rule gives ( 2B W WBBh 2L,h, so (14) is gradient descent on 2 F Let I, = UAU with = diag(A (A; 2 0) and write h()) = Uc(().
Then o ((+1) (1 - -)
Σ. Ac If S n S for i, then (1 n2 s X giving monotone decay, strict if some X > 0 has c (2) and convergence follow from |1 - m7>al < 1 (and for A > 0).
(6) and (h()))
+ 0. Non-expansiveness
Remark 3. Py-+z may be (i) direct per-edge linear maps, (ii) factored as Py-tx = F F. to reduce
parameters, or (iii) constrained (e.g. softly orthogonal via |lp /11:)) to regularize the spectrum of Lp. A learnable (possibly per-layer/head) stepsize 1), normalized by running estimate
of ||اا enforces the energy-decay guarantee in Theorem 3 When Py-+2 I, (13) reduces the
(vector-valued) graph Laplacian; orthogonal p recovers a connection Laplacian.
D Expressive Power of CTNNS
D.1 Universal Approximation of N-Dependent Copresheaves
Here, we demonstrate that multilayer perceptrons (MLPs) can approximate arbitrary copresheaf morphisms induced by neighborhood functions. This result ensures that the proposed sheaf-based model is sufficiently expressive to capture complex data interactions.
--dependent copresheaves). Let X' be a finite combina- torial complex and N neighborhood function on X. Suppose GA - Vecta is an VVdependent copresheaf with stalks J=N (x) = RRd and morphisms pz)- Let a feature map
Proposition 2 (Universal approximation
h: X Rdd,
x + h
be given so that the 2d-dimensional vectors (hy, h₂) are pairwise distinct for every directed edge y + x. Define
A = (hy, ha) y + x) C R2d, g: A ddd g(,,, h2) pyy
Then for any 0 there exists multilayer perceptron 4: Rdd + gdxd with sufficiently many hidden units such that (hy, h.) - - الليو < for all - x.
Proof. Since A is finite and its elements are distinct, the assignment A + Rdxd is well-defined. Enumerate A {a, (ir), choose disjoint open neighborhoods U, og and pick smooth "bump" functions
:p: Rd2 [0, 1], f(a) Eglas)
pe(as)) 1, supp({::) U..
Then the sum
sel
22

---

is a smooth map f: I22d + Raxd satisfying flA = 9.
Since A is finite, choose a compact set _ 2 containing A. ensuring the applicability of the Universal Approximation Theorem, By that theorem, for any 0 there is an MLP d such that
sup aEk
p(pp - f(a)] <e
In particular, for each E A we have
(aa))
g(a:) (hy, h2z pzizzl <E,
for every - v. This completes the proof.
E Sheaf Neural Networks Are Copresheaf Message-Passing Neural Networks
The computational use of a cellular sheaf on a graph rests on the incidence poset. Let G = (V, E) and consider the poset VU E with space to each cell and linear structure map to each incidence. In our convention, the "restriction" along æ 3 implemented
whenever e. cellular sheaf F assigns vector
vertex-to-edge lift Fzde F(x) - F(e). Equipping F(e) with an inner product yields the adjoint Fage F(e) + F(x), a canonical edge-to-vertex back-projection. Message passing between adjacent vertices then arises by composing incidence maps: for e {2, y). the message from to is
FIge o F(y) F(x).
This composition induces a direction of information flow on an undirected graph while remaining faithful the sheaf poset structure. Diffusion-style updates and sheaf-Laplacian operators recovered by aggregating such edge-mediated messages over (zx). See Figure 6 for an illustration.
F(e)
F(e)
F(x)
Fade
F Fyee
Fly)
F(x)
Face
J 4c
F(y)
F(x)
F Jec
J ydc
Fly)
F(e)
FI o Fysle
(a)
(b)
(c)
Figure 6: Sheaf-induced message passing on an edge e = (z,y). (a) Local spaces
F(), with vertex-to-edge lifts Faste and F Fyde along the incidences (b) Incidence-poset view: arrows encode the sheaf's linear maps attached to x I e and y 3 e. (c) Edge-mediated message passing: the inner product F(e) yields adjoints F. The message from
3
y to x the composition FIge Fyyge i.e., direct vertex-to-vertee -e-veex morphism Py-to F(y) - F(x). compatible with the bidirected expansion G'. This realizes SNN message passing as CMPNN mor- phisms assembled from sheaf structure maps.
In this appendix, we prove Theorem T and Proposition which demonstrate that existing sheaf neural networks (SNNs), including sheaf diffusion networks, are special cases of the copresheaf message- passengg neural network (CMPNN) framework (Definition)) (9). Moreover, we summarize how other sheaf-based neural architectures align with our unifying message-passing framework (see Table B).
Proof of Theorem 7 We construct the bidirected graph G' and define the copresheaf G on it, then operations.
demonstrate the equivalence of the message-
First, construct the bidirected graph G' = (V, E') from G = (V, F) by replacing each undirected edge {ar,y] E E. with two directed edges (ar, y). (u. .t) E F'. This ensures that G' retains the connectivity G while introducing explicit directionality.
Next, define the copresheaf G G + Vecto as follows:
For each vertex V. assign (xx) F(z), where F(x) is the vector space associated with by the cellular sheaf F.
For each directed edge (y. x) E', corresponding to the undirected edge (z, v} E, define the linear morphism Py-* (g)) + ((z) by Py-* = =T Fyee, where Fyce Fly) F(e)
23

---

and Frde: F(x) + F(e) are the restriction maps of F, and F F(ee - F(x) is the adjoint with respect an inner product on F(e).
Now, consider the SNN message-passing mechanism along the edge e = (I, For feature vector hy E FV) at vertex y, the message transmitted to vertex is given by Fzde o FFyae (by))
In the copresheaf G on G', the morphism associated with the directed edge (y. .)) is Py-+o Fase F gee. Applying this morphism. the cesssae--assiing operation in the CMPNN yields Py-tz (hy) F Ide Fyee (hy), which is identical to the SNN message.
To ensure that G is well-defined copresheaf, observe that it assigns vector spaces vertices and linear maps to directed edges a functorial manner. Specifically, for each directed edge (y. x) E E', the map Py->x is composition of linear maps and thus linear. The identity and composition properties are satisfied implicitly through the consistency the sheaf restriction maps.
Therefore, the SNN message passing, which operates via intermediate edge spaces in F, is equiv- alently represented as direct vertex-to-vertex message passing in the copresheaf G on G'. This completes the proof.
O
Proof of Proposition Compute:
(T, @ Wi)H b ba
(Ar & (T, @ W, HW, = Σ
LFZY Wyh W2
VVE
h h
Σ
W2 Lr Wihy,
VEN(=)U(z}
since LF for y $ Ux))
Interpret G as a directed graph with edges y + x for y E (xx) and x + x. Define:
Message function: a(he, Py-rahy) W2 LF Wihg,
Morphisms: Py-+z implicitly encoded via LF,,0
Aggregator: + =
Update: (hg, m) h2 m.
Thus:
h = hr Σ a(he Ea(tes
8-2
matching Definition
g
Table provides a summary of sheaf neural networks realized in terms of Definition
We finally prove the following theorem to show the relationship more precisely between CTNNs, MPNNs and SNNs.
Theorem 4 (CTNNs strictly subsume SNNs and contain MPNNs), Let FCTNN. FSNN, and FMPNN denote the function classes realized by Copresheaf Topological Neural Networks (CTNNs), Sheaf Neural Networks (SNNs), and Dassage-Passing Neural Networks (MPNNs), respectively. Then
FSNN C. FCTNN
and FMPNN C FCTNN.
Proof. FSNN C FeTNN. First from Theorem containment, fix {u, v) and let F(u) F(v) R22 Define a CTNN G' by
I we know FsNx C FeTNN. To prove the strict
Pe-ta = Iz. Pu-+o =0 0.
No SNN can realize this, since SNN transports necessarily reciprocate across an undirected edge:
Potra F F Fuar
Pa-s F Fuse Po-tu
Thus Pa-so would force Pv-va 0, contradicting Pa-va = 1z. Therefore FSNN C FCTNN is strict.
24

---

Table 8: Unified message passing formulations of various sheaf neural networks using copresheaf topological neural network (CTNN) notation given Proposition T0 may be linear, data-driven, or attentional depending the model.
The restriction maps Py-+
Method (Paper)
Message Pasiigg Equation
Notable Features
Restriction Pyme
Linear restiction maps po assigned Ps in Fla lised linear per edge: enables high dimensomal. direction aware message passing via sheaf stucture.
Sheuf Neural Network (SNN)
map, - (2.9)
Hansen & Gebtart (2020)
Difteion leamed sheaf Laplacian. restiction maps oyne leumalke. re flecting edge medizedd intrrctions.
Pyree m Fee learned cid map, e (x.3)
Nenral Sheaf Diflision
(NSD) Bodnar et al. (2102)
Attomtinnal shoof: stteriion wrights day modilt:: the natricted siighte loceinn miligaics uonmmnsttingg in GAT-styk sctups.
leaered lincar map. mendtnivaal In capeare fosture spuane relainnssiigs
Atteation Network Rutbero (3002)
bg
Σ Cop hyl
Edge mape Ouy are orthenormal, de-
Ouy" onhenormal matris,
Connection Laplacian SNN Barhero (2002)
Oua
from fisture spoor aligmment: no- duase lamuhik prramciens and reflects goonditic prics.
learno aligs feature spaces across olges
Type-sware morphions: Poice depend node edge types, en- abling structured heterogeseity acooss graph.
Pessel type ascare
Heteropmenus Sheaf Neural Network (HetShraf) Rnoihwati at (2034)
cut map, pazametetized mode types
122
Noslincar Laplacias-Hie dytunics with adaptive, Seatuse restiction echunces expeessi to-
leanee Peesei parametetized
Aduptive Sheaaf Diffusion Zaghen et al. (2024)
bi to
Σ (aooooe
A Oue h.) 1. -b9)
(2) FMPNN FOTNN. Let an MPNN layer on a directed graph have the form
h(+1) o0 (N()) DeEN ( (h() A/2 emm))
for a intpuuttttioonivviiiai aggregator D.
The results follows immediately from Definition !
F A General Copresheaf-Based Transformer Layer
The main idea to introduce copresheaf structure to the transformer is the following. For every ordered pair y + (within an attention head) we define a parametrized copresheaf map
Rd
Rd
Py+x
Uy
Py-ex By.
which transports the value vector from the stalk at to the stalk at x. Given attention weights the head message is my = EyeN(e) Gzy Py-r By.
agy = (((t
In Definition II we introduced the notion of copresheaf self-attention. A natural extension of this definition is the copresheaf cross-attention.
Definition 16 (Copresheaf cross-attention). For source rank k and target rank ki with neighborhood
No define learnable projection matrices W E Rpxd, We E Rpxd, W Rd. xd. where de and d, are the feature dimensions of source and target cells, respectively. We then propose the copresheaf cross-attention as the aggregation and update h(++1) ((h)), mz) where m. ΣyeNs azypy-ra (Uy) with Py-rz Fly) F(x) being learned map and
expl kg) vP) Ozy Eye expl (9 Ky /pD)'
(12)
where k, 1/ Uyy = W and for each target cell æ E 2 W
Figure illustrates the cross attention in the copresheaf transformer.
we provide the pseudocode for our generic copresheaf-basse based transformer layer. This algorithm outlines the layer-wise update rule combining self-attention within cells equal rank and cross-attention between different ranks, using learned copresheaf morphisms to transfer features between stalks. generalizes standard transformer mechanisms by introducing neighborhood- dependent transformations.
In Algorithm I
25

---

yy
Py-+
Oaygn
y
Vyz
Py.+r
Oayya
y2
U ya
U y/3
Py-+v
Oerys
ma
Vy
Pıc-+
Ouryy
y3
U y4
Py,+
drys
(ax)
Vys
(a)
(b)
(c)
Figure 7: Copresheaf cross-attention. (a) A target cell (yellow) in a combinatorial complex with its neighborhood (x)) (red). Sources may be at different rank than æ (c.g., faces + vertex). (b) Cross-attention schematic: each neighbor y (x)) contributes value by that is first transported into the target's s local feature space via learned map Py-+x F(y) - F(r); attention weights dry are computed from lu and hy. (c) Implementation view: for every y. apply Py-* l'gs scale by azy- and sum to form the message m. E yEN((s) Gay Py-va (1y), which updates h Transporting values with p enables directional, cross-rank, and anisotropic information flow beyond standard attention.
General copresheaf -aadd transformer layer
1: procedure COPLESHAATTTAAEEOOLLLLLELEE
(1 e F(()))exx)
for x X* do
Self-attention on k-cells
as +- Wih ks + Weh v2 f Woh
4: my +
5:
for (z) do
6:
aay softmax (1) //)/ vP)
Dzy t Py-er (vy)
8:
mh c- mz + U2y
9:
end for h
10:
3(h22 (1)
,m²)
end for
for x x do
Cross-attention from to liç
13:
Qa Wa- (8)
14:
Ma
for E NN++e(1)) do
16:
ky + W By W W0
17:
Gny +- softmaxy. (1 kn)/ vP)
18:
Vzy - Pg-r (u,)
19:
me my UayDay
20:
end for
21:
ho (E++) + 3(122 (1)
mz)
22:
end for
23:
return (A(e+1) JEEN
24:
end procedure
Algorithm 1: Copresheaf transformer layer integrating standard attention with learned presheaf morphisms.
G Copresheaf Learning on Euclidean Data
The CopresheafConv layer leverages copresheaf structures to process data on D-dimensional grid X' C zD, offering distinct advantages over traditional convolutional neural networks (CNNs). By defining a copresheaf on combinatorial complex (CC) constructed from the grid, where cells represent grid points (0-cells) and their pairwise connections (I (1-cells), the layer employs learnable morphisms Py- Fly) + F(x) that dynamically adapt to directional relationships between points. Unlike static convolutional filters, these morphisms capture anisotropic, directionally dependent interactions, preserving topological nuances of the grid's geometry. In contrast, regular convolutional kernels enforce translation invariance, limiting their ability to model spatially varying directional
26

---

patterns. The copresheaf is defined over an adjacency neighborhood function (z) {y E X {z, y) E X'), J. restricting computation to local, grid-adjacent neighbors, thus ensuring efficiency comparable to CNNs. The morphisms, potentially nonlinear, are conditioned on input features h hg and grid positions, enabling the layer to model complex, multi-scale dependencies, This makes CopresheafConv ideal for tasks like image segmentation, 3D mesh processing, geometric deep learning, where local and hierarchical relationships are critical. Empirical results demonstrate superior performance in capturing physical dynamics, showcasing the ability of CopreshealCony to handle spatially varying patterns. Algorithm shows the pscudocode for the CopresheafConv Conv used our experiments.
CopresheafConv on D-dimensional grid
1: procedure CoPRESHEAFCONVCC ZZ: (12 E F(x)) = R(()x
2: for x E X do
3: My 0 E R Con
for y E (x) do
5:
Py-ナ CopreshcalMorphism(g...
map conditioned 1 ng (and thus
(x,
my <- my + Po-ro (h.g) end for
7:
8:
(C+1) t m2
9:
end for return ( JzEX
end procedure
12: procedure COPEESHEAFMORPPISMtyyy..
b Return the learned copresheaf morphism Py-+x
potentially nonlinear,
b conditioned on both source and target features (hz (2)
13: return Py->
14: end procedure
Algorithm 2: CopresheafConv D-dimensional grid.
H Experiments
H.1 Mechanistic Notes for Physics Experiments
Scope. The empirical results for advection and unsteady Stokes appear the main text (Sec. 6.1 Tab. This appendix augments those experiments with architectural rationale and ablations-informed design choices, without introducing any new datasets, training budgets, or evaluation metrics.
2).
Advection (pure transport). The advection equation
8,u + Vu O, u(x, uo(x ct)
is rigid translation. Classical self-attention aggregates values in single global latent space, so ++y interactions effectively symmetric and only weakly directional (positional encodings help but cannot enforce upwind behavior). The copresheaf transformer (CT) replaces value mixing
m
classical
Σ VEN(2)
azy By
by
m
axy Py-tx
BEN(2)
with a learnable edge map Py- F(y) F()) before aggregation. This yields: (i) Directionality
(Py-z # Pa--y) for upwind-like asymmetry, (ii) Phase-faithful shifts (identity-near, head-wise maps accumulate small signed translations coherently), (iii) Path compositionality: products along y chains bias the model toward consistent transports.
Unsteady Stokes (incompressible viscous flow). For
d.u vAu + Vp = O. v-n=
accuracy depends on encoding (i) anisotropic diffusion and (ii) rotational structure (vorticity), un- der divergence-fiee -freeneeeeeeeeeeeeoooooe constraint. Standard attention lacks built-in diffusion geometry frame
27

---

aligmment. CT's edge maps Py-+z act as local linear operators that: (1) Align with diffusion ten- sors: SPD/orthonormal variants (Table 18] mimic smoothing/rotation along principal directions. (ii) Support structured coupling: With cross-rank paths, CT allows vertex-edge/value transports akin to discrete parallel transport and ceesuure-ellcityy interactions, without imposing global sheaf consistency.
Take-away. Across both tasks, CT's gains come from a geometriic of attention into
(weights) x (directional transport) with Py- Fly) F(x).
not merely extra parameters. This aligns with the CTNN principle that heterogeneous stalks + edge-specific morphisms provide natural inductive biases for transport-dominated and anisotropic dynamics.
H.2 Synthetic Control Tasks
Six canonical univariate time-series patterns (normal, cyclic, increasing trend, decreasing trend, upward shift, downward shift) are procedurally generated. We obtain 600 sequences of length 60 (100 per class), normalised to the interval [-1, with an 80:20 split for training and test.
Models and set-up. A lightweight vanilla Transformer (32-d model, heads, 2 layers) is compared with identically sized Copresheaf Transformer, where multi-head attention is replaced by gated outer-products censor-attntiin layer with orthogonality (A = 0.01) and sparsity (A =A= 10-4) regularisers. Both models share sinusoidal-with- linear-decay positional encodings, use Adam (10-3 learning rate), batch 32, train for 15 epochs, and each experiment is repeated with three random seeds.
Table 9: Synthetic control: mean over runs.
Model
Max acc. (%)
Standard Transformer
98.61 + 0.40
Copresheaf Transformer
99.44 +: 0.39
Results, As seen in Table the Copresheaf Transformer yields a consistent improvement of +0.8-1.0 percentage points (pp) over the vanilla Transformer while remaining lightweight and training in comparable wall-clock time (under one minute per run on a single GPU), highlighting the benefit of richer token-pair transformations for recognition tasks.
H.2.1 Structure Recognition Datasets
In this experiment we consider two synthetic image datasets containing oriented ellipses or hierarchi- cal triangles.
Dataset (oriented ellipses). Each 32 RGB image contains single black ellipse on white back- ground. The horizontal and vertical semi-axes b are drawn uniformly from (4, ...., 12) pixels, and the ellipse is rotated by a random angle in [0, 180%). The task is to predict the coarse orientation bin (4 bins of 45" each). We synthesise 6,000 images, keep 5,000:1,000 for train/validation, and rescale pixels to 1].
Dataset (hierarchical triangles). Each 32 x2x33 RGB image contains six coloured circles-red, green, blue, yellow, cyan, magenta-placed on two nested equilateral triangles (inner radius 8 px, outer 12 px). Colours randomly permuted. A hand-crafted hierarchy of linear maps tnneertriangle, outer-triangle, cross-level) is applied to the circles' one-hot colour vectors; the image is labelled 1 when the resulting scalar exceeds fixed threshold, else We generate 6,000 images and keep 5,000:1,008 for train/validation.
Models and set-up. Both tasks use the same compact Transo-rransrrmmer backbone: 32- dim patch embeddings (patch size 8), 4 heads, layers, learnable positional embeddings, AdamW (3 x 10 learning rate). The baseline is a Regular VIT: its counterpart an identically sized Copresheaf ViT in which multi-head attention is replaced by an outer-product copresheaf mechanism (stalk-dim 8). Oriented Ellipses is trained with batch 128; Hierarchical Tri-
Table 10: Validation accuracy on both synthetic vision tasks (sta over three seeds).
Dataset
Regular ViT
Copresheaf VIT
Oriented Ellipses
$1.13 + 4.12
96.23 + 0.33 96.87 + 0.26
Hierarchical Triang.
95.47 + 1.31
angles with batch 64. All runs use 30 epochs and three independent seeds.
28

---

Results. Across both synthetic vision tasks the Copresheaf ViT consistently surpasses the Regular ViT: a dramatic +12.1 pp gain on Oriented Ellipses and a subtler yet statistically tighter +1.4 pp on Hierarchical Triangles, while also cutting variance by an order of magnitude in the latter case (see Table TOP. These outcomes underscore that replacing standard attention with saarrehhaa-ggiddee outer-product maps yields robust improvements for both low-level geometric orientation recognition and higher-level nested-structure reasoning, all without a significant increase of model size or training budget.
H.2.2 Classifying Hierarchical Polygons
Similar to the previous section, we now synthesize hierarchy of nested regular polygons. In particular, each x22 RGB image contains variable number n E {6, 8, 10} of coloured circles arranged on two nested regular polygons (inner radius 8 px, outer 12 px). The first n/22 circles form the inner polygon, the remainder the outer; colours are drawn from a palette of a distinct hues and randomised per sample. A hierarchy of hand-crafted linear maps is applied to one-hot colour vectors: pairwise maps on the inner polygon (Finner), on the outer polygon (Goute), and a cross-level map H. The image is labelled 1 when the resulting scalar exceeds threshold, else O. For each synthesise 6,000 images, keep 5,000:1,000 for train/validation, and normalise pixels to [-1,
Models and training. We reuse the compact VIT backbone (32-dim patch embeddings, patch size 8, heads, 2 layers, learnable positional embeddings). The Regular ViT is compared with an identically sized Copresheaf VIT, which replaces multi-head attention with rank-restricted copresheaf outer-product maps (stalk-dim= = 8). Both networks are trained for 10 epochs with Adam (3 x 10-1 learning rate), batch 64; each configuration is run three times.
Results. Figure 8 shows that the Copresheaf VIT consistently exceeds the Regular VIT at n= 0.72 vs 0.66) and regains a clear lead n=10 ( 0.63 vs 0.57) despite both models dipping at n=8. The Copresheaf curve displays narrower uncertainty bands at the hardest setting, indicating greater run-to-run stability. Overall, copresheaf- -uidee attention scales more gracefully with combinatorial complexity, capturing cross-level dependencies that standard self self-attention struggles to model.
Performance vs. Task Difficulty
Validation Accuracy
Copresheaf Transformer Regular Transformer
Q,7
0.6
H.2.3
Airfoil Self-Noise Regression
7
10
Number Circtes
The UCI airfoil dataset (1 503 rows) maps five continuous de- scriptors feeeuenereeuc...ee angle of attack, chord length, free-stream velocity, Reynolds number-to the sound- preessurruuee level (dB). Inputs and target are min-max scaled to [0. 1J; we keep only 400:100 train/test samples for low-data setting.
Figure 8: Validation accuracy
(mean + s.d., 3 runs) as task difficulty increases.
Models. Both regressors share minimalist backbone consisting sequentially of the following: 64-d token embedding 2-layer, 4-head transformer, mean pooling, scalar head. The copresheaf variant swaps dot-product attention for learned outer-produce maps Piy that depend each token pair, whereas the Regular baseline keeps standard self-attention. Training uses Adam (10-1 learning rate), 1000 epochs, batch 32.
Results. On the small 100-sample test set the copresheaf regressor lowers MSE by 7.2% relative to the regular transformer and maintains sub- 10-4 run-to-run variance (see Table 11). con- firming that pair-specific linear transports help model heterogencous feature interactions even in data-scarce regimes.
Table 11: Test MSE (mean over two runs).
Model
MSE
Regular Transformer
0.0223 + 0.0001
Copresheaf Transf. 0.0208 + 0.0002
H.3 Pixelwise Regression Tasks: Evaluating CopresheafConn2DD Layers
We evaluate neural network models incorporating Capreshea/Conv2D layers, custom convolutional layers with wise trainable linear morphisms, against standard convolutional models across four synthetic pixelwise regression tasks: PDE regression (Bratu and convection-diffusia equations), image denoising, distance transform regression, and edge enhancement. In all tasks, Copresheaf-basee
29

---

models consistently achieve lower Mean Squared Error (MSE) and Root Mean Squared Error (RMSE) compared to standard convolutional models, suggesting improved modeling of spatial structures and relationships.
Task Definitions.
PDE regression.
Bratu equation. A nonlinear pouction-diffusiee PDE:
-uu = g0, v)ee, e", " on 0,
where g(z, y) is a source intensity.
CConvction-difiusion equation. A transport PDE:
Au cu u.u + cy ogu u - v), ulan= 0,
with diffusion y/ and velocities Ca. Cy.
Image denoising. Recovering clean structured images (sinusoidal patterns with Gaussian bump, normalized to [0,1]) from Gaussian noise (a 0.3).
Distance transform regression. Predicting the normalized Euclidean distance transform of binary segmentation (thresholded at 0.5) of structured images.
Edge enhancement. Predicting edge maps from structured images using a sitprreecccoo--nnistrrroo Gaussians (DoG) transformation.
Model and training setup. For PDE regression and distance transform tasks, we use U-Net variants: CopresheafUNer (with CopresheafConv2D layers) and ConvUNet (with standard Conv2d layers), both with a four-level backbone 164~112-- 1--62-665555 and edge enhancement, channels). All models are trained on 64 x 64 inputs using the Adam optimizer and MSE loss, with task-specific settings (learning rates 10-3 10-4 batch sizes 8 or 16, 80-300 epochs). Results are averaged over 3 random seeds.
channels). For image denoising use four-layer convolutional networks: CopresheafNes and ComNer
(1~8->16-)
Table 12: Mean (+ std over 3 seeds) of MSE and RMSE across all tasks.
Task
Model
MSE
RMSE
Bratu Equation
Uopt
0.0001 = 0.00020
0.0108 0.0003
ConvUNct
0.0003 - 0.00020
0.0183 + 0.0007
Convection-Difluss
Copreshea/UNet
0.0004 + 0.00010
0.0205 1 0.0010
ConvUNet
0.0006 0.00020
0.0232 1 0.0012
Image Denoising
CopresheafNet
0.0010 + 0.00010
0.0310 0.0010
ConvNet
0.0011 = 0.00020
0.0336 = 0.0015
Distance Transform
CopresheafUNet
0.0001 + 0.00002
0.0105 = 0.0003
ConvUNet
0.0002 0.00003
0.0156 = 0.0005
Edge Fnhancement
CopresheafNet
0.0008 0.00010
0.0283 0.0010
ConvNet
0.0009 + 0.00020
0.0300 + 0.0015
Take-away. Across all tasks, replacing standard convolutional layers with CopresheafConv2D layers results in lower MSE and RMSE (see Table[ This consistent improvement suggests that patch-wise linear maps enhance the models' ability to capture complex spatial patterns. These findings highlight the potential of Copresheaf based architectures for pixelwise regression problems. Subsequently, we address two challenges related token classification: real/fake token sequence detection and segment-wise token classification. Finally, we conduct preliminary study on shape classification using pegresheaf-augmenee
attention and graph classification molecular benchmark, MUTAG. These are followed by applications in graph connectivity classification and text classification TREC coarse label benchmark.
H.3.1 Learning Booken-Relations with Copresheaf Attention
We study five problems that differ only in the (non)linear operator unknown applied to the first half of a random token sequence (or to a second related sequence). The classifier must decide whether the tail is just noisy copy (label 0) or transformed version of the head (label 1).
Orthogonal block. Fight 16-d "head" tokens are either copied (+-0.05 noise) or rotated by sample-specific orthogonal matrix before adding the same noise.
30

---

irputgitt.t
Sarget
Cor-uher Prediction
25
108
1.5
64
04
43
1.0
08
00
06
0.4
13
Irput Str sruucured image
Target Transform
1.0
108
26
No Irput
Clean Target
Coprsheattuatteeee Predictton
ConwNet Prediction
C4
3.4
244
GereshorUUettrriiiiion
60
52
32
Figure 9: Model outputs across tasks. A: Bratu equation: input target u, Copresheaf vs. ConvUNet predictions. B: Convection-diifusion input g, target u, CopresheafUNet ConvUNG predictions. C: Distance transform: input image, target transform, CopresheafUNet vs. ConvUNet predictions. D: Image denoising: noisy input, clean target, CopresheafNet ConvNet predictions. E: Edge enhancement: input image, target edge map, CopresheafNet vs. ConvNet predictions. Copresheaf-based models show subtle improvements in detail recovery.
Per-token scaling. As above, but the tail is a; x noise with U(0.4, 1.6].
Rotated copy (embedded 2-D). Six 2-D points mapped 16 by fixed linear embed, duplicated to 12-token sequence; the tail is either noisy copy or the points after random planar rotation.
Query and context linearity. Two parallel sequences (50 x 16 "query", 50 x 24 "context"), Class O: context a global affine transform of the query with partly correlated semantics. Class 1: context comes from a quadratic warp and weak semantic correlation.
Affine vs. quadratic token relations. Two parallel sequences (length-6, query dim 16, context dim 24) are considered. For class 0, the context is linear spatial transformation (rotation and translation) of the query plus correlated semantic noise. For class 1, the context is generated via spatial quadratic (nonlinear) transformation with weaker semantic correlation.
31

---

Table 13: Mean accuracy (# std, 3 seeds).
Task
Classic
Copresheaf
Orthogonal block
0.732 + 0.009
0.928 + 0.007
Per-token scaling
0.521 = 0.005
0.707 = 0.004
Rotated copy (2-D)
0.739 = 0.010
0.896 + 0.033
Query to context Affine vs. Quadratic Relations
0.608 = 0.046
0.992 + 0.012
0.588 + 0.047
0.900 + 0.027
Data. Tasks 1-2 use 16 tokens, task uses 12. task 4 uses two length-50 sequences, task 5 uses two length-6 sequences. For each of three seeds we draw 4,096:1,024 train/test sequences (task 4-5: 320:80).
Backbone and training. A tiny Transformer encoder (4 heads, token dim 16, stalk-dim 4) mean-pool + 2-way classifier. Classic uses vanilla dot-product attention; copresheaf augments it with learned token-pair copresheaf maps (we chose General Copresheaf for the first two tasks, and Non-linear MLP for tasks 3-5). We train for 8 12 10 10 10 epochs respectively with Adam (10 learning rate), batch 64.
Take-away. Across in-sequence, clement-wise, embedded geoometric and cross- woss-sequence set- incluinc varying degrees of spatial and semantic correlation injecting copresheaf transports into self-attention consistently lifts accuracy significantly (up to +38 pp. seen in Table 13). This highlights a general principle: tasks whose signals reside in relations between tokens rather than in absolute token content strongly benefit from explicitly modeling these relations through learnable pprresheaf-iduucae attention.
poneeation-ijeecting
Limited attention capacity. We study the impact of attention capacity on relational reasoning. by varying the number of heads in small transformer and test- ing its ability classify the query to context dataset provided in Section H33
Accuracy is Number of heads (ncl Copresheaf
hepua "aratsmee Postoral Encoding Logretteat nedtreer heade)
We evaluate three setups: baseline transformer, the same model augmented with positional encoding (PE), and a appresheaf-augmenteee transformer with 2 heads. Figure 10 shows accuracy as function of attention capacity.
While positional encoding improves baseline accuracy slightly, the
attention with just two heads out- performs all classic models, even those with eight times more heads. This highlights the value of inductive relational structure over brute-force capacity scaling.
30
32
34
Manber Vf Prodda
Figure 10: Accuracy as function of number of attention heads. Even with low capacity, the copresheaf- augmented model perfect general- ization of the Query to Context task.
H.3.2 Segment-wise Token Classification
We test whether copresheaf attention improves token-level clas- sification in sequence partitioned into contiguous segments with distinct patterns. The classifier must assign a segment label (0, 1, or 2) to each token based on its local context.
Data. Each input is a sequence of 100 tokens, where each token a 16-dimensional feature vector. The sequence is divided into three contiguous segments, each following different pattern: (i) cosine oscillation, (ii) a linearly increasing ramp, or (iii) an exponentially decreasing signal, with additive noise. The task is to predict the correct segment label for each token. We generate 300 training sequences and evaluate on three random seeds.
Backbone and training. We use 2-layer encoder with 4 heads, token dim 16, and stalk-dim 4. A linear classifier maps each token to one of 3 segment labels. Classic is standard attention; copresheaf augments each attention head with learned per-token transport maps. We train for 10 epochs using Adam (10-3 learning rate), with batch size 32.
Take-away. Injecting copresheaf structure into attention substantially improves token-wise
32

---

segmentation accuracy, especially with expressive MLP kernels (see TableI44 This demonstrates that local consistency constraints enforced by per-token transport maps help resolve semantic boundaries even in noisy, positionally ambiguous settings.
Table 14: Mcan segmentation accuracy (#:
std, 3 seeds).
Model
Accuracy
Classic
0,705 + 0.010
Copresheaf-Pe
0.833 = 0.015
H.3.3 Topological Shape Classification
Copresheaf-MI.
0.831 = 0.007
Copresheaf-SPP
0.743 0.017
We evaluate copresheaf-augmmente attention on a synthetic 3D) point cloud classification task. Each input is set of 128 points in R sampled from one of four classes: cube, sphere, torus, and twisted torus. Rotations are applied to remove alignment bias.
Data. The dataset consists of 480:160 train/test samples, balanced across the four classes. Each point cloud is processed as a sequence of 128 points with 3D coordinates.
Backbone and training. Both models use a 4-layer point transformer with heads and head dimension 32. The Classic model uses standard self-attention. The Copresheaf model augments attention with diagonal copresheaf morphisms. We train each model for 50 epochs using AdamW and cosine learning rate decay across 3 random seeds.
Take-away. Copresheaf-augmentt attention improves accuracy on 3D shape classification by enhancing sensitivity to latent geometric structure Table 15: Mean accuracy (+ std, 3 seeds). (see Table [5).
Model
Accuracy
H.4 TREC Text Classification Task
Classic
0.708 + 0.031
We evaluate two transformer-based models on the TREC coarse-label question classification task, which involves categorizing questions into 6 classes (e.g.. abbreviation, entity, description). The models are:
Copresheaf
0.746 1 0.034
Classic, standard transformer with multi-head self-attention; and Copresheaf-FC, which incorpo- rates a GeneralSheafLeamee to model stalk transformations.
Task definition. The TREC dataset consists of questions labeled with one of 6 coarse categories. Inputs are tokenized questions truncated to 16 tokens, mapped to a vocabulary of size [V], and embedded into an 8-dimensional space. The task is to predict the correct class label for each question.
Model and training setup. Both models use a single transformer block with 2 attention heads, an embedding dimension of 8, and stalk dimension of 4 for the sppresheaf-basee model. The architecture includes an embedding layer, a transformer block with attention and feed-forward components, adap- tive average pooling, and a linear classifier. Compared to state-of-the-ar (SOTA) models, which often employ multi- ple transformer layers and high-dimensional embeddings for maximal performance, our networks are intentionally small, using a single block and low embedding dimension to priori- tize computational efficiency and controlled experimentation. We train on the TREC training set (5452 samples) over 30
Table 16: Mean (+ std over 4 seeds) of test accuracy for the TREC classifi- cation task.
Model
Test Accuracy
Classic Copresheaf-FC
0.7320 + 0.0080 0.7500 + 0.0150
epochs with a batch size of 32, using the Adam optimizer with a learning rate of 10-3 and cross- entropy loss. The test set (500 samples) is used for evaluation. Each experiment is repeated over 4 random seeds. As seen in Table 16 the copresheaf models outperform their SOTA counterparts.
H.S Mixed Dirichlet-Robin Reaction-Anisotroppi Diffusion on Cellular Complexes
Problem.. Let Ω C R be a Lipschitz domain with a hole (nonconvex), and let ön = M2p U URR with Mdp O 2u
Ø. We consider the steady caneiioonnnnstroopicc diffusion equation
V K (ar Vu(x)) + A(x) u(x) + u())(()) = g(x) in R
(13)
with mixed boundary conditions
u(z) wz(x)
Msp.
(14)
n(x)-K(x) Vu(x) + r(x) u()) = a(x up( on 09R.
33

---

Here K(x) E R2x2 is a symmetric positive definite anisotropy tensor (we store its local components as (Kar, Kay: Kyy). X, o 0 are reaction weights, g is source, a, are Robin coefficients, ug boundary signal, and n the outward unit normal. All spatial fields are discretized and provided as vertex/edge attributes on the mesh.
rk) be 2D simplicial complex (trian- gulation) of 52), H with O-, 1-, and 2-cells X0, X21 X². We employ the neighborhood functions (Def. 2)
Discretization simplicial complexes.. Let K = (S, Xk
(e) = x lecee
A10000 (0) = {w E 2 3t x² 0 C w Ct}.
inc
and the induced directed graphs Gy. Edgewise anisotropy enters through per-edge tensors (Kzz, Kzy, Kyy) and yields a stiffness-like coupling consistent with equation 13 Vertex features earry (uB:9: 9, X, G, a, r); edge features carry (Kaz, Kay, Kny). Masks Mhp and ÖRR enforce the boundary conditions in the loss.
Higher-order copresheaf structure.. We equip Gy with an --dependent copresheaf (F,P,GN) over multiple neighborhoods:
stalks:
F(u)= 12 F(e) = RR F(t) d (DE e (Ex²),
morphisms:
P00 F(w) + F(u) for E ( (v), pCol) FF()) for e EEA (v),
p F(t) for T E / CT
Thus information flows across ranks (edges/triangles vertices) in addition to within rank (vertex + vertex). This realizes a higher-order CMPNN and may be viewed as principled generalization of topological neural networks (TNNs). where transport maps are learned as copresheaf morphisms instead of being tied to fixed co(boundary) operators.
Copresheaf Cellular Transformer layer.. For features ( E F(x) cell x in layer L, we copresheaf self- and cross-attention:
h (1+1) = 3 h
Σ wEA-10.0)
Oyw Purto (Uw) +
Σ
aue Po (v)))((V))
+
Σ Gut
(0-22) Pi-+o (vr)
(w)
e (v)
13v
2-+0
1-0
where Ge W Woht h so (1ge, k /D). and B is residual MLP with normalization. We instantiate p with sheaf-style transport maps SheafFC Py-+z Id + tanh(W kyl) (xero-initialized), optionally constrained to SPD via p Id + QQT All morphisms act per head and per rank; weights shared across complexes but conditioned on (lz, h,y), enabling directionality and anisotropy. Observe that when the copresheaf structure is the identity, we retain the Cellular Transfomer introduced in |Basbby et al. [2025).
Ww
U2
W W. (6)
the coefficients auy
Dataset and training. We synthesize simplicial complexes by jittered grids with warped holes to emulate nontrivial 02n. Per-sample scalar fields (ug, g, X, ar, a, are sampled on vertices; (Kzz. Keg. Kyy) on edges. Ground-truth u is obtained by a finite-element solve of equation 13 equation [4] We train to regress î at vertices with MSE, enforcing Dirichlet via clamped targets and Robin boundary- weighted residuals. Optimization uses Adam W (10- learning rate), cosine schedule, grad clip 0.8, layer norm, and vertex/edge feature normalization computed per split.
Baselines and size control. We compare the Cellular Transformer on vertices (same depth/width, no copresheaf maps) to our Copresheaf Cellular Transformer on K. using identical token dimensions and heads; differences arise solely from the learnable morphisms and cross-rank attention paths.
Results. Quantitative test MSE averaged across n seeds is summarized below (lower is better).
Error maps highlight that copresheaf cross-rank transport (1 (1-0 and 2-+) attenuates bias along anisotropy directions of K and reduces leakage across 08p Despite identical model size, higher- order morphisms recover boundary layers and interior ridges more faithfully. See Figure [1 for sample of the results.
34

---

Table 17: Mixed Dirichlet Rolient---oinnn-e reaction aniioorrppio diffusion on simplicial complexes. Test MSE
(mean + std over n=4 seeds). Lower is better.
Model
Test MSE +
Runs (n)
Cellular Transformer Barsbey et al. Copresheaf Cellular Transtormer
2025
0.3277 + 0.0408
4
0.3172 + 0.0365
Twke-away. Endowing the simplicial complex with a higher-order copresheaf, learning Py- across adjacency and incidence neighborhoods and across ranks, yields CMPNN that generalizes TNNS while remaining faithful to the PDE structure in equation [3] Observe that such a network is generalization the cellular transformer introduced in Barsbey transport improves anisotropic coupling and mixed-
[2025). The learned, directional mixed-boundar boundary handling without increasing parameter
count, offering a principled and practical route to physics-awar anysics-aware attention on combinatorial domains.
Target (GT)
Cellular Transformer
Copresheaf Cellular
1.0
05
-55
-15
Target (GT)
Cellular Transformer
Copresheaf Cellular
-43
-1.
15
65
08
-00
Figure 11: Mixed Dirichlet-Robin reaction reaction-anisotrppe anisotropic diffusion on triangulated complexes. For each example (rows), the left block shows 3 tile of normalized inputs: Dirichlet values up, source reaction A, spatial reaction field a, Robin coefficients a and r, and per-vertex edge-averaged conductivity features (K 22: K Kyy K ). The remaining panels show the ground-truth solution u, the Cellular Transformer prediction, and the Copresheaf Cellular Transformer prediction, respectively, Rows illustrate diverse geometries and boundary layouts. Copresheaf transport produces slightly crisper fields and cleaner boundary behavior.
H.6 Catalogue of Copresheaf Maps
We also provide the table of copresheaf maps that we used throughout our experiments in Table 18
Computational Complexity
This section analyzes the computational complexity of the Copresheaf Dessage-Passing Neural Network (CMPNN) and the Copresheaf" Transformer (CT), defined in the framework of Copresheaf
35

---

Table 18: Catalogue of copresheaf maps Py+* used our training our copresheaf transformer model. All maps act stalk-wise and are evaluated independently for each attention head; a is the logistic function. Map family
Copresheaf map Py-r (per head)
Learnable params
General Copresheaf
Pr tanb( W
9 Riy
W R2dxd?
Pre-Linear Map
Por+ = kyy
none
Diagonal MLP Map
Pe++ = diag, a (MLP|2e, kyl))
2-layer MLP
Graph Attention Map
Py-r ar <MLLP(q.. (q.. kyy]) La
2-1ayer MLP
Vision Spatial Map
Py-+x (MMLP(P= Pb))
2-layer MLP
(p2, Py E [0, 1]² pixel coords)
Outer-Product Map
Pa W 2o (Wak;))
Wg: W₂ E Rdxd
Non-linear MLP Map
Py-r reshapd MLP gr. kyl)
2-layer MLP (2d
2d df))
Gaussian RBF Map
Po++ = e 11 -ky11 Ia
a (scalar)
Dynamic Map
Py-rr reshape y...))
Wj mdddd
Bilinear Map
Po-tx = (b" (gx, ky)) Is
b E Rdxd
SheafFC Map
Py-eo la + tanh W
qu
W 2dxx (zero init)
SheafMLP Map
Pr++2 + tanh (MLP Qu kiyl)
2-layer MLP (last layer zero init)
SheafSPD Map
Py-tz = I + QQ" Q W
9u ky
W E R2dxd (no bias)
Topological Neural Networks (CTNNs). We consider a directed graph GN = (VN, Ex), induced by combinatorial complex X' with neighborhood function N,, where [Vv] n, |EN] = m. and the average degree is m/n. Each vertex VN is assigned feature space F(z) Rd and each edge y + E Ex has a linear map Py-tz Rd + Rd, computed at runtime using map family from Table 16 (e.g.. General Copresheaf, Low-rank). The complexity of computing a single morphism Py-vz is denoted C(p).
Unless otherwise stated, we assume sparse graphs (i.e., m O(n)). We derive the following propositions for per-layer complexities. followed by a comparison with standard architectures.
Proposition (Copresheaf Message-Passing Complexity).. Consider directed graph Gy (VN. Ex) induced by combinatorial complex X with neighborhood function N,, where [V> = |Eyl = m, dim F(x) = d, and let @ be any permutation-invarte
aggregator. Let each y - x EN have a linear map Py-vz E Rdxd computed at runtime via a map family with complexity C(p). The per-layer computational complexity of the CMPNN, as defined in Definition 9 is
TCMPNN (nC(pp + m df + n
Proof. The CMPNN message-passing operation involves computing morphisms, applying them to features, aggregating messages, and updating vertex states:
1. Morphism Computation: For each edge y - E EN, compute Py-to E RRdxd based on
source and target features (h ((())) E R2d, using a map family from Table 18 This costs O(m C(p).
2. Morphism Application: Apply Pø-tz to the feature vector h E Rd. This matrix vector multiplication requires (dd)) operations per edge, totaling O(m
3. Aggregation: For each vertex æ E VN. aggregate messages from neighbors E Na()) via
+ DueN(w) P (2) Each addition involves IRd vector, costing O(d) per neighbor. With [xx) neighbors, this step is (=)((=)) -d). Summing over all vertices: O <(Eevy (((x). d) O(m d).
36

---

4. Update: Assign Na (e+1) = B(-) with negligible cost. If an optional single-layer MLP with d hidden units is applied (common in GNNs), the cost per vertex is (dd)), yielding On df).
Total: C(p) O(n + (m df)
(p)) + m m + md + nd²). Since md? dominates and m = (n). this simplifies to
Proposition (Copresheaf Transformer Complexity).. Consider directed graph GN (VN, Ex) induced by a combinatorial complex X with neighborhood function N, where |V = n, EN = m, and the average degree is c = m/n. Each vertex x E Vy has a feature space F(z) R and each of H attention heads computes a morphism P--v E IR x(d/H) via a map family (Table 16) with complexity C(p). The per-layer computational complexity of the Copresheaf Transformer, as defined in Algorithm 1 with sparse cross-attention based on N is
Ter = O(H m (p) + m m& + n + ndH..
Proof. The Copresheaf Transformer integrates self-attention within cells of equal rank and sparse cross-attention between neighbors, using learned copresheaf morphisms. Each head attends on features (queries/keys) of dimension p = ddd/H d/H:
Morphism Construction: For each edge y x EN and each head, compute Py-bar E ((d/1))(4/11) based on (hy ()) The complexity per morphism is C(p). totaling O(H C(p)).
2. Self- and Cross-Attention: Compute query (Q). key (K), and value (V) matrices per head (d/H dimensions), costing Onn d' H)) across all heads. For sparse cross- cross-attention, each vertex attends to [M(x) neighbors, with attention scores QK and aggregation costing ((I((z)) -d/ H) per vertex per head. Summing over vertices and heads: O(H Ex [=)))
d/H)= = dm.
3. Morphism Application: Apply Pa to value vectors (d/ H per head) on each edge, costing (dd / H) per head, (dd) across H heads. For m edges this contributes Om df-).
4. Output and Feed-Forward: Combine head outputs and apply per-token forwafo network (FFN) with hidden units, costing On
Total: O(H m C(p) + m d + + Hn)H.
Comparison to Standard Architectures. Compared to standard GNNs (e.g., GCN) and dense Transformers, CMPNN and CT incur higher costs due to morphism computation (the C(p) terms) and morphism application (m d²). On sparse graphs with small H, CMPNN's complexity is (n ((p) m d² + d²), and CT's is O(H m C(p) mdd d + n + n H...H. These costs enable superior expressiveness, as shown empirically.
Model
Per-layer complexity (sparse graph)
GCN
O dd² Ol C(p) md + nd) Oin² d + nd)
CMPNN
Transformer (Dense)
Copresheaf Transformer (CT) O(H m C'p) + m d + n + n H)
Table 19: Per-layer computational complexity on sparse graphs. If is diagonal (or rank-r), replace d² by (or r d) in the morphism-application terms.
I Extended Related Work on Topological and Sheaf Neural Networks
Foundations of sheaf theory. Sheaf theory offers a unifying categorical framework across algebraic
geometry, topology. and algebra [Bredon 1997]. Early computer-science applications exploited its logical structure Fourman et al.) 1977 Goguen| 1992]. and Srinivas generalized pattern matching via
sheaves on Grothendieck topologies [Srinivas] 1993] later extended to NP-hard problems [Conghaile]
2022] Abramsky 2022]. Cellular sheaves, formalized [Curry] 2014], underpin discrete topological
data analysis and signal processing Ghrrist and Hiraokal 2011
Robinson 2014]. Hansen & Ghrist
37

---

introduced the sheaf Laplacian [Hassn and Ghrist 2019b]. learnable by convex optimization Hansen and Ghrist 2019a]. Connection sheaves model discrete vector bundles [Singer and Wu 2012 and support manifold learning and Gaussian processes (Peach et a..]. GKM-sheaves further connect equivariant cohomology and sheaf cohomology, enriching this framework with applications to torus actions on CW complexes Al-Jabea and Baird 2018).
Higher-order representations in deep learning. The growing interest in higher-order network
models [aaaamakoou et al. 2024 Battiston et al. 2020 Bick et al. 2023 has catalyzed geometric
and topological deep learning. Techniques include simplicial, hypergraph, and cellular message-
passing schemes [Gilmer al.) 2017 Ebli et al. 2020 Hayhoe al.
2022 Hajij al. [0200 Bunch
ct al. 2020]. skip connections [Haji et 2022), and convolutional operators [Jiang et al. 2019
Feng et al. 2019]. Recent years also witnessed leap in higher-order diffusion models for graph
generation Huang and Birdal 2025 as well as higher-order (cellular) transformers [Ballester et al.) 2024].
Sheaf neural networks. In recent years, sheaf-based generalizations of graph neural networks (GNNs) have demonstrated notable improvements on tasks involving heterogeneous or non-Euclidean data. Hansen and Gebhart [Hansen and Gebhart 2020) first introduced sheaf neural networks (SNNs), which generalize graph neural networks (GNNs) by replacing neighborhood aggregation with learnable linear "restriction maps", thereby customizing information flow between nodes. By allowing each edge to carry its own linear transformation, SNNs capture relationships in heterophilic graphs more effectively than degree-normaling
convolutions. Building on this idea, Bodnar et al. [Bodnar et al.) 2022] proposed Neural Sheaf Diffusion (NSD), which jointly learns the underlying sheaf structure and the diffusion dynamics. NSD layers adaptively infer the sheaf Laplacian from data, mitigating the oversmoothing problem common in deep GNNs and achieving superior performance on range of heterophilic benchmark datasets. Barbero et al. [Barbero et al. [002bb] then combined NSD's s principled diffusion with attention mechanisms to formulate Sheaf Attention Networks (SANs). SANs modulate self-attention weights by the learned sheaf maps, preserving while respecting local sheaf geometry.
dependencies
Alternative formulations include Bundle Neural Networks (BNNs) by Bamberger et al. [Bamberger et al. 2024), which reinterpret propagation as parallel transport in a flat vector bundle rather than discrete message passing. Duta et al. (Duta et al., 2023) extended sheaf methods to hypergraphs, defining linear and nonlinear hypergraph Laplacians that capture higher-order interactions among groups of nodes. On manifold-structuee by Battiloro et al. [Battiloro et al. 2024b) treat features as elements of tangent spaces and propagate them along estimated geodesics, bridging continuous and discrete models.
data, Tangent Bundle Neural Networks (TBNNs) proposed
Attention mechanisms higher-order structures. Attention mechanisms have been generalized to
hypergraphs (Kim al. 2020 Bai al. 2021] and simplicial complexes [Goh al. 2022 Battiloro et al. 2024a among else via Hodge or Dirac operators. On combinatorial complexes, feature-
attention facilitates hierarchical information propagation [Giusti et al.) 2023 Hajij et al. 2023b
Applications and extensions. These sheaf-theoretic architectures have found diverse applications, from multi-document summarization (Atri et al., 2023 and recommendation systems (Purificato et al. 2023) to community detection via sheaf cohomology [Wolf and Monod [2023] and person- alized federated learning with Sheaf HyperNetworks (Nguyen et al. [024|] Liang et al. 2024]. In representation learning, many knowledge- graph embedding techniques have been reinterpreted as sheaf global-section problems [Gebhart et al., 2023 Kvinge et al. 2021]. Collectively, these ad- vances highlight the expressive power and flexibility of sheaf-based models in handling complex, heterogeneous, and higher-order order data domains.
38

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Paper about Copresheaf Neural Networks and the paper presents the frame- work as well as set of experiments.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We briefly discuss them at the end of conclusion and provide an extended version our appendix.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiitiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
39

---

Answer: [Yes]
Justification:
Our main theorem and propositions are proven in the appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We provide detailed experimental settings in the appendix. We will also release our code and data publicly upon publication.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
40

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes)
Justification:
We will also release our code and data publicly upon publication.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ ptaip..cc/ubblic/guu.eee CoodeSumeeieeeeee ssi iessonPolll mollicyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: For each experiment, we separately mention the hyperparameters, optimizer, data splits and the curation of data.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Paper report error bars over experiments and describes the number runs these error bars over.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
41

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative етrоr rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Compute explicitly discussed; however, timings are not presented as part of the argument about the proposed approach and are not needed for a to replicate the results. Compute resources are modest (single GPU runs); details are not essential reproducing results.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: The paper focuses on a generalized framework for neural networks, and does not address areas or applications of ethical concern as defined by the Ethics Guidelines.
Data sets the paper do not contain any personally identifiable information/ data about real people.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our work primarily theoretical and is not expected to have immediate socictal impacts.
42

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Paper presents a theoretical framework and does not release models or present results on data scraped from the internet etc.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We cite datasets used in the paper.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
43

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thood cww datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We will also release our code and data publicly upon publication. The repository has license. We do not have assets of others within data/code base.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: No human subject research done
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
44

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [No]
Justification: LLMs were only used for wording help.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standare components.
Please refer to our LLM policy for what should or should not be described.
45


Reviews:
[{"review_id": "o39Doj3rOt", "reviewer": "Reviewer_hjdt", "summary": "This paper proposes a novel graph neural network framework—Copresheaf Topological Neural Networks (CTNNs). Based on the concept of copresheaves from algebraic topology, this framework effectively captures directionality and local heterogeneity in graph structures, enabling more flexible and expressive information propagation. CTNNs offer a unified and powerful approach to graph modeling, with strong potential for future applications in large-scale and dynamic graphs. The authors present their ideas with clarity, supported by rigorous mathematical reasoning and a well-structured modeling approach.", "questions": "1. In the Introduction, the authors repeatedly emphasize that CTNNs generalize traditional CNNs/GNNs/Transformers, but the explanation of how this unification is achieved is vague. For example, it merely states “extends many deep learning paradigms” without clarifying the specific mechanism.\n2. In the Preliminaries section, the authors do not explain why combinatorial complexes are chosen over simplicial complexes or hypergraphs. Using CCs instead of more common structures may raise questions from readers about the motivation and advantages behind such generalization.\n3. In Section 5.1 “Evaluations on Physics Dataset,” although the results indicate that copresheaf outperforms classical methods, there is a lack of deeper analysis on why it performs better—particularly regarding the physical implications of copresheaf in the advection and Stokes problems.\n4. The authors could briefly discuss the complexity of the proposed architecture.\n5. While some visualizations are included in the supplementary material, incorporating visualizations on actual task datasets in the main paper could help readers better understand the model's behavior.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The paper demonstrates a high level of originality.\n2. The writing is very clear and well-structured.\n3. The authors provide detailed explanations of the mathematical reasoning involved.\n4. The research presented has significant implications for the advancement of the artificial intelligence field.\n\nWeaknesses:\n1. In the Introduction, the authors repeatedly emphasize that CTNNs generalize traditional CNNs/GNNs/Transformers, but the explanation of how this unification is achieved is vague. For example, it merely states “extends many deep learning paradigms” without clarifying the specific mechanism.\n2. In the Related Work section, although it is mentioned that “our work is closely related to SNNs,” the fundamental differences between CTNNs and SNNs are not explicitly explained.\n3. In the Preliminaries section, the authors do not explain why combinatorial complexes are chosen over simplicial complexes or hypergraphs. Using CCs instead of more common structures may raise questions from readers about the motivation and advantages behind such generalization.\n4. In Section 5.1 “Evaluations on Physics Dataset,” although the results indicate that copresheaf outperforms classical methods, there is a lack of deeper analysis on why it performs better—particularly regarding the physical implications of copresheaf in the advection and Stokes problems.\n5. The authors could briefly discuss the complexity of the proposed architecture.\n6. While some visualizations are included in the supplementary material, incorporating visualizations on actual task datasets in the main paper could help readers better understand the model's behavior.", "was_revised": true, "final_justification": "We thank the authors for providing detailed and comprehensive feedback on the concerns we raised. The paper demonstrates a solid theoretical foundation and rigorous justification, and extensive experiments have verified the application advantages of CTNN. We hope that the proposed CTNN technology can find broader applications in the field of computer science in the future. In view of its promising application prospects, we have decided to appropriately increase the score.", "rebuttal": "We thank the reviewer for their effort in reviewing our work and finding the paper ``demonstrates a high level of originality``, and that our research ``has significant implications for the advancement of artificial intelligence``.\nIn the following, we address their comments point by point.\n\n> *W1. Clarification of the Framework, how it \"extends many deep learning paradigms\".*\n\n**Author Response.**\nWe thank the reviewer for pointing out that our unification claims could benefit from clearer articulation. While our manuscript presents a general framework that subsumes a wide variety of deep learning architectures, we acknowledge that the explanation was previously distributed across the main text and appendix, and could be better centralized.\n\nTo address this, we have taken the following concrete steps in the revised version of the paper:\n\n**1. Centralized mapping table (added at the end of Section 3)**\n\nIn the revised manuscript, we will include a new table titled *\"Mapping Classical Architectures to CTNN Instances\"*, placed at the end of Section 3. This table maps each classical model (CNN, GNN, Transformer, SNN) to its CTNN counterpart. It specifies the domain and neighborhood function, describes the form of the copresheaf morphism (ρ\\_{y → x}), and provides direct references to the relevant sections in the main text and appendix. This consolidated format makes the unification claim more transparent and easier to verify.\n\nExample excerpt from the new table:\n\n| Classical Model | CTNN Form                   | Domain & Neighborhood  | ρ\\_{y → x} Morphism          | Reference            |\n| --------------- | --------------------------- | ---------------------- | ---------------------------- | -------------------- |\n| CNN             | CopresheafConv              | Grid, adjacency        | Learnable directional map    | Sec. 4.2, Appendix F |\n| GNN             | CMPNN                       | Graph, adjacency       | Shared / edge-indexed map    | Prop. 1, Sec. 3      |\n| Transformer     | Copresheaf Transformer (CT) | Grid or token sequence | ρ\\_{y → x} in attention head | Sec. 4.1, Appendix E |\n| SNN             | CMPNN on bidirected graph   | Graph, incidence       | Composition of sheaf maps    | Thm. 1, Appendix D   |\n\n**2. Explicit examples of generalization**\n\nWe now provide clear examples in the revised text showing how CTNNs recover standard models:\n\n* **CNNs** are recovered by using CopresheafConv layers over regular grids, with translation-invariant directional maps (Sec. 4.2, App. F).\n* **GNNs** emerge as a special case of CTNNs using graph adjacency neighborhoods and shared identity morphisms (Prop. 1, Sec. 3).\n* **Transformers** are instantiated by interpreting attention weights through learnable ρ\\_{y → x} maps, which reduce to dot-product attention when ρ = Identity (Sec. 4.1, App. E).\n* **Sheaf Neural Networks (SNNs)** are modeled using morphism compositions in bidirected graphs. We demonstrate this equivalence formally in Theorem 1 and provide supporting details in Appendix D.\n\n**3. Clarified language in the Introduction**\n\nWe have revised Section 1 (lines 46–62) to clarify the unification message. Consider the previous sentence:\n\n> “This unified perspective subsumes and extends many deep learning paradigms...”\n\nWe now justify this sentence as follows after the introduction of the main message passing on CCs : \n\n “This unified perspective subsumes classical architectures: CNNs (via copresheaf convolutions, Appendix F), GNNs (via anisotropic message passing, Proposition 1), Transformers (via directional self-attention, Appendix E), and Sheaf Neural Networks (via morphism composition, Theorem 1 and Appendix D).” \n\nThis new formulation explicitly states which models are recovered and directs readers to the supporting material.\n\nWe believe these changes substantially clarify the unification claim and thank the reviewer again for highlighting this point.\n\n---\n\n> *W2. Clearly distinguishing CTNNs vs SNNs (specifically related works section).*\n\n**Author Response.**\nThis is addressed in the appendix B.3 Comparison between copresheaves and cellular sheaves. We also included Table 7 that shows how all existing SNNs in the literature  can be realized in terms of our unifying notation. This further shows the relationship between SNNs and CTNNs. We now clarify their distinction also with regard to the related work.\n\n---\n\n> *W3. Why are Combinatorial Complexes chosen over simplicial complexes or hypergraphs? What are the motivations and advantages?*\n\n**Author Response.** Combinatorial complexes generalize simplicial complexes and hypergraphs and offer several advantages: while hypergraphs allow for set-based relations among nodes, these relations lack any inherent hierarchy. In contrast, simplicial complexes endow relationships with a natural notion of hierarchy through simplex dimensions, capturing higher-order structure in a more principled way. However, simplicial complexes impose a strong constraint on relationships: if a set of nodes $\\sigma$ forms a simplex, then all subsets of $\\sigma$ must also be present in the complex. This closure property, while mathematically elegant, may be overly restrictive or undesirable in certain contexts where such subset relationships do not naturally occur. CCs can model both of these two features, subsuming both complexes. We aim for mathematical generality, and we need a template complex that allows us to apply to Euclidean and non-Euclidean downstream applications without having to worry about the underlying domain. Indeed, many of our experiments in the paper, especially in the appendix, involve images, CCs, and graphs. Furthermore, in the context of a copresheaf specifically, CCs offer an advantage. For instance, when the copresheaf maps are directed towards higher-order cells, they offer a learnable way to \"pool\" the CC structure to a rougher structure. \n\n---\n\n> *W4. Deeper analysis for the \"Evaluations on Physics Dataset\" for advection and Stokes problems.*\n\n**Author Response.**\nWe appreciate the reviewer’s suggestion and have expanded Section 5.1 to provide a deeper analysis of the performance gains observed in the advection and unsteady Stokes experiments. In particular, we now clarify the physical and architectural reasons why copresheaf models outperform their classical counterparts:\n\n*Advection task:* The advection equation represents pure translation — a signal is transported without deformation. Classical transformer models use dot-product attention in a shared latent space, which treats all token interactions as symmetric and context-independent. This limits their ability to model directed transport. In contrast, the copresheaf transformer uses learnable edge-specific transport maps (denoted ρ\\_{y→x}) that adaptively translate features from source to target. These maps act similarly to numerical advection schemes that preserve directional information across time steps. As shown in Table 1, this leads to over 60% reduction in test MSE and significantly lower variance across seeds. Appendix G.2.1 further explains how these learned linear maps serve as inductive biases that improve generalization for transport-dominated dynamics.\n\n*Unsteady Stokes task:* This problem involves incompressible viscous flow, where accurately modeling diffusion and preserving rotational structure is essential. Standard attention mechanisms have no way to encode divergence-free or anisotropic interactions. The copresheaf transformer, by contrast, learns directional feature maps that better reflect the structure of the underlying PDEs. As detailed in Appendix G.2.2 and G.4, the edge maps effectively capture spatial anisotropy and allow the network to model local vector field transformations — leading to more faithful representations of vorticity and pressure dynamics. This results in improved prediction accuracy and a tenfold reduction in variance compared to classical attention.\n\nWe have updated Section 5.1 to emphasize these points and referenced relevant experiments and design choices from the appendix. Together, these results illustrate how copresheaf attention captures essential physical structure in a way that classical attention cannot, reinforcing the advantages of our framework for learning on structured scientific data.\n\n---\n\n> *W5. Missing discussion of the complexity of the proposed framework.*\n\n**Author Response.** We have added a detailed section on the complexity of various architectures in our paper. We kindly ask the reviewer to refer to our response to Reviewer uisE for details.  \n\n---\n\n> *W6. Visualizations from the main paper on actual tasks could be incorporated into the main paper.*\n\n**Author Response.**\nWe appreciate the reviewer’s suggestion and fully agree that visualizations can significantly aid in illustrating model behavior, especially in the context of structured physical tasks. Due to space constraints and our focus on clearly presenting the core framework and empirical results, we omitted such visualizations from the original submission. We have now incorporated representative figures, such as learned transport maps, flow-aligned feature propagation in the advection task, or spatial attention patterns in the Stokes setup, to visually demonstrate how copresheaf architectures capture directional and topological structure.\n\nWe believe these additions will provide valuable insight and further strengthen the narrative, and we kindly ask the reviewer to reconsider their final score."}, {"review_id": "KSq2GJ2WIJ", "reviewer": "Reviewer_EnAf", "summary": "Based on combinatorial complexes methods, the paper aims to develop a unified network architecture to handle different types of data: including images, point clouds, graphs, meshes, and topological manifolds.", "questions": "(1) The methods compared by the author in Table 2 are too old and are not compared with recent SOTA methods. In addition, the scale and diversity of datasets used in the authors' experiments are insufficient: for graph classification, only the MUTAG dataset (188 graphs) was used, lacking large-scale benchmarks such as OGB or molecular datasets like QM9.\n\n(2) CTNN introduces per-edge copresheaf mappings, which may significantly increase the computational overhead. It is recommended that the authors analyze the computational resources required by the proposed method as well as its convergence behavior.\n\n(3) Although copresheaf maps are theoretically interpretable, the paper lacks discussion on their visualization or semantic interpretation. It is recommended that the authors include some visualization experiments to help readers better understand the proposed method.\n\n(4) How does the proposed method compared to the capsule networks, which is also a message passing based design.\n\n(5) The paper needs to provide more test results on different types of data, such as images, meshes, and point clouds.", "limitations": "Yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 1, "originality": 3, "strengths_and_weaknesses": "Strengths\n\nThe paper extends traditional GNN based on copresheaves, a concept from algebraic topology. It uses  a higher-order\nmessage-passing mechanism to model relational structures. The overall idea is very interesting. \n\nWeaknesses\n\nAlthough the idea is very interesting, the experimental results are somehow disappointing. It did not show extensive evaluation results on a wide range of images, point clouds, graphs, meshes, and topological manifolds, as claimed in the beginning of the paper.", "was_revised": true, "final_justification": "I agree that this paper has novelty and theoretical contributions by introducing a new network structure based on Copresheaf formulations. It can be considered as another message passing scheme. My major concerns about the paper are (1)  for each test, the comparison is very limited. (2) It does not test on hardcore CV tasks, such as image classification and object detection.", "rebuttal": "We sincerely thank the reviewer for the thoughtful and constructive feedback, and that they found our work `very interesting`. Below we address each of the points in turn:\n\n> *W1/Q1. Scope of baselines and datasets*\n\n**Author Response:** We appreciate the reviewer’s concern regarding the scope of benchmarks and baselines. The main goal of this work is to *demonstrate the expressive and unifying power of Copresheaf Topological Neural Networks (CTNNs)* by showing how they recover—and meaningfully extend—canonical deep learning architectures: CNNs, GNNs, Transformers, and Sheaf Neural Networks (SNNs). To this end, our experimental design deliberately focused on domains where the structural inductive biases introduced by copresheaf morphisms could be most clearly evaluated.\n\nThe datasets used—including PDE dynamics on grids, synthetic graphs, and MUTAG—were chosen to test these models across spatial, graph-based, and topological settings. While some are smaller-scale, they were sufficient to evaluate generalization, expressivity, and the benefits of directional feature propagation. For example:\n\n* In **Appendix F**, we introduce **CopresheafConv**, a direct generalization of CNNs applied to regular 2D grids. Here, we replace static, translation-invariant convolutional filters with learned, input- and position-conditioned morphisms ρ\\_{y→x}, enabling anisotropic and spatially-varying processing. Unlike CNNs, which impose uniform filters across the grid, CopresheafConv allows local directional adaptation, making it well-suited for data with varying geometry or dynamics.\n\nThis setup allowed us to show not only performance benefits but also how CTNNs naturally recover these architectures as special cases—substantiating the theoretical claims made in Section 3.\n\nWe acknowledge that large-scale datasets such as QM9 or OGB are important for benchmarking at scale and plan to include them in future work. Our current experiments across numerous diverse settings aim to establish CTNNs as a general-purpose, topologically grounded modeling framework.\n\n---\n\n> *Q2. Computational cost and convergence*\n\n**Author Response:** We understand the reviewer’s concern about computational overhead due to per-edge morphisms. \n\nWe kindly refer the reviewer to our discussion of computational complexity in the response to Reviewer uisE, which contains additional technical details. Due to space constraints, we did not include that material in the current response. \n\nIn summary:\n- **CMPNN complexity**: O(n·k·d² + n·d²) vs. O(n·k·d + n·d²) for standard GNNs\n- **CT complexity**: O(H·n·k·d² + n·d²) vs. O(n²·d + n·d²) for dense Transformers\n- The additional factor of d comes from computing morphisms ρ_{y→x}, with complexity C(ρ) ranging from O(d) for diagonal maps to O(d²) for full matrices\n\nIn addition to the response above, we plan to incorporate runtime experiments in the main paper. Empirically, we observe that using low-dimensional copresheaf feature spaces is sufficient for strong performance across tasks, contributing to practical efficiency. Finally, we would like to mention that we observed that CTNNs converge at rates comparable to baseline models. In some tasks—such as directional advection—they converge faster due to better alignment with the task’s inductive structure.\n\n---\n\n> *Q3. Visualization and semantic interpretation of copresheaf maps*\n\n**Author Response:** We appreciate the reviewer’s suggestion and fully agree that visualizations can significantly aid in illustrating model behavior, especially in the context of structured physical tasks. Due to space constraints and our focus on clearly presenting the core framework and empirical results, we omitted such visualizations from the original submission.\n\n---\n\n> *Q4. Comparison to capsule networks*\n\n**Author Response:** Capsule networks offer a specific, dynamic routing mechanism to pass information, usually in the form of some clustering from primary to latent capsules. This involves an alternative message passing implementation, which can be constructed on a directed graph. We acknowledge that the various routing algorithms can be explained as a special case in our copresheaf framework. We leave a detailed taxonomy of CapsNets for future work.\n\n---\n\n> *Q5. Broader modality coverage: images, meshes, point clouds*\n\n**Author Response:** We acknowledge the reviewer’s concern that our current experiments do not include image classification benchmarks, mesh data, or point clouds. This is a valid point.\n\nThat said, **Appendix F demonstrates that CopresheafConv naturally generalizes standard convolutions over Euclidean grids**, with learnable anisotropic morphisms replacing static kernels. This makes the method well-suited for tasks like image segmentation and grid-based simulations. We also emphasize that morphisms are conditioned not only on features but also on spatial context, making them more expressive than translation-invariant filters. We also have experiments on graphs, physical simulations  (pixel-wise regression), GNNs, text tasks, and a few ablation studies (Appendix G in general has many experiments along these directions). \n\nNote that, most of the networks we discuss, expressed as a special case in our framework, extensively evaluate on point clouds, meshes and graphs. One prominent example is combinatorial complex neural networks (Hajij et al. 2023). In this work, while we demonstrated the applicability of CTNN on non-Euclidean domains (graphs, CCs), we focused our experiments on Euclidean ones because we believe that such focus is missing from topological deep learning literature in general. \n\nWe thank the reviewer for bringing these up and now discuss all of these in our revision."}, {"review_id": "9PkSHw8Hki", "reviewer": "Reviewer_uisE", "summary": "This paper introduces Copresheaf Topological Neural Networks, a unified deep learning framework built on the formalism of copresheaves over combinatorial complexes. CTNNs generalize and extend many existing architectures, such as GNNs, Transformers and sheaf neural networks. The authors provide a solid theoretical foundation, unifying message-passing paradigms via copresheaf-induced maps and validate their framework empirically across physics simulations, graph classification and complex combinatorial structures. Results consistently demonstrate performance gains and improved stability over classical baselines.", "questions": "* What are the key tradeoffs between CTNNs and traditional Sheaf Neural Networks in terms of expressivity?\n* How do CTNNs scale with larger graphs or real-world data (e.g., social networks, protein interactions)?\n* Are there theoretical guarantees that CTNNs are strictly more expressive than GNNs or SNNs?", "limitations": "n/a", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n* The paper is well-written\n* Easy to understand from anyone with a background in GNNs\n* Nice generalized framework.\n* Theoretical soundness grounded in algebraic topology and category theory.\n* Empirical validation across diverse domains.\n* The paper is detailed and explains its main idea in depth. It contains both theoretical and empirical results, something that is necessary for such a type of paper. Two different use cases/scenarios have been tested to empirically evaluate the effectiveness of the proposed solution. I understand that it’s difficult to obtain real-world data for the physics dataset. Towards overcoming this roadblock, the authors created three synthetic datasets, which I personally believe suffice to benchmark and test the robustness of the proposed approach. For the second scenario, the MUTAG dataset was employed, which is one of the most-used datasets for evaluation. The results on this dataset demonstrate that the proposed solution improves the performance of the given model. Another synthetic dataset was used for the graph classification problem as well, in which the proposed solution demonstrates convincing evidence. All in all, I believe that both the theoretical and empirical evidence of the experiments justify the validity of the approach. \n* On the theoretical justifications of the paper.  From a theoretical point of view, I believe CTNNs can easily generalize and be combined with multiple GNN and transformer-based graph architectures. The definitions of the paper are not coupled with any specific GNN architecture, which is the strongest evidence that the approach can easily be generalized. For instance, definitions 1 and 4 are well-formulated in a way that different types of graphs $G$ and $F$ can be employed.\n* Theoretical gains that CTNNs offer over GNN/Transformer architectures and theoretical costs incurred to obtain them. I believe that CTNNs offer theoretical gains by unifying diverse architectures (GNNs, transformers, sheaf networks) under a single categorical framework that enables heterogeneous, directional and higher-order message passing across structured domains. These benefits come from replacing global latent spaces with local feature spaces and learnable transport maps.\n\n\nWeaknesses:\n* The paper is overwhelmed by mathematical notation and it's easy to get lost in it.\n\nTo sum up, this paper proposes a new approach which is theoretically and empirically grounded and fits the style of NeurIPS.", "was_revised": true, "final_justification": "I believe the authors answer to all of my questions and to those of my colleagues reviewers. The paper is technically sound and I believe this paper is worth accepting.", "rebuttal": "We thank the reviewer for their thoughtful feedback. We are delighted they find our work ``theoretically sound``, ``well written`` and ``easy to understand for anyone with a background in GNNs`` with ``empirical validation across diverse domains``. \n\n> *W1. Complexity of mathematical notation.*\n\n**Author Response:**\nWe tried to make the paper self-contained. Yet, our framework relies on numerous constructs from algebraic topology and sheaf theory. We now include several additional introductory resources to better guide the reader through theory and notation, such as [A. Ayzenberg, 2025] for sheaf theory and [M. Besta, 2024] for delineating neural architectures on higher-order structures, including CCs [M. Hajij, 2023].\n\n---\n\n> *Q1.  Are there tradeoffs between CTNNs and traditional Sheaf Neural Networks in terms of expressivity?*\n\nIn Appendix D, we demonstrate that CTNNs subsume sheaf message passing neural networks (SNN). Moreover, CTNNs are strictly more expressive, as we demonstrate in the following result.\n\n**Author Response:**\n**Proposition (Strict Expressivity Superset)**: Let $F_{\\\\text{CTNN}}$, $F_{\\\\text{SNN}}$, and $F_{\\\\text{GNN}}$ denote the function classes realized by Copresheaf Topological Neural Networks (CTNNs), Sheaf Neural Networks (SNNs), and Graph Neural Networks (GNNs), respectively. Then:\n\n$F_{\\\\text{GNN}} \\\\subset F_{\\\\text{SNN}} \\\\subset F_{\\\\text{CTNN}}$, with $F_{\\\\text{CTNN}} \\\\setminus F_{\\\\text{SNN}} \\\\neq \\\\emptyset$.\n\n**Proof**: Any GNN can be viewed as an SNN with all restriction maps set to the identity, so $F_{\\\\mathrm{GNN}} \\\\subseteq F_{\\\\mathrm{SNN}}$, and strictness follows since SNNs can model heterophilic interactions that GNNs cannot. By Theorem 1, every SNN message‐passing update is realized by a CTNN on the bidirected graph, giving $F_{\\\\mathrm{SNN}} \\\\subseteq F_{\\\\mathrm{CTNN}}$. Finally, CTNNs impose no sheaf‐consistency constraints and by Proposition 5 can approximate arbitrary edge‐specific linear maps via MLPs, so there exist functions in $F_{\\\\mathrm{CTNN}}$ not realizable by any SNN. Hence the strict hierarchy holds.\n\nWe will include this in Appendix D, where we distinguish between our copresheaf CTNN framework and existing sheaf message passing networks.\n\n---\n\n> *Q2/3. How do CTNNs scale with larger graphs or real-world data?*\n\n**Author Response:**\nThe reviewer is right to point out computational cost, an important aspect we have now included in the main manuscript, with detailed discussion in the appendix. \nExpressivity indeed comes at a higher computational cost, as we quantify in the following propositions.\n\n**Computational Complexity**\n\nThis section analyzes the computational complexity of the Copresheaf Message-Passing Neural Network (CMPNN) and the Copresheaf Transformer (CT), as defined in the framework of Copresheaf Topological Neural Networks (CTNNs). We consider a directed graph $G_N = (V_N, E_N)$, induced by a combinatorial complex $\\\\mathcal{X}$ with neighborhood function $\\\\mathcal{N}$, where $|V_N| = n$, $|E_N| = m$, and the average degree is $k = m/n$. Each vertex $x \\\\in V_N$ is assigned a feature space $\\\\mathcal{F}(x) = \\\\mathbb{R}^d$, and each edge $y \\\\to x \\\\in E_N$ has a linear map $\\\\rho_{y \\\\to x}: \\\\mathbb{R}^d \\\\to \\\\mathbb{R}^d$, computed at runtime using map families from Table 16 (e.g., General Copresheaf, Low-rank). The complexity of computing a single morphism $\\\\rho_{y \\\\to x}$ is denoted $\\\\mathcal{C}(\\\\rho)$, which varies by map family. We assume a sparse graph ($k \\\\ll n$) and uniform feature dimensions. The following propositions formalize the per-layer complexities, followed by a comparison with standard Graph Neural Networks (GNNs) and Transformers.\n\n**Proposition (CMPNN Complexity)**: Consider a directed graph $G_N = (V_N, E_N)$ induced by a combinatorial complex $\\\\mathcal{X}$ with neighborhood function $\\\\mathcal{N}$, where $|V_N| = n$, $|E_N| = m$, and the average degree is $k = m/n$. Each vertex $x \\\\in V_N$ has a feature space $\\\\mathcal{F}(x) = \\\\mathbb{R}^d$, and each edge $y \\\\to x \\\\in E_N$ has a linear map $\\\\rho_{y \\\\to x}: \\\\mathbb{R}^d \\\\to \\\\mathbb{R}^d$, computed at runtime via a map family from Table 16 with complexity $\\\\mathcal{C}(\\\\rho)$. The per-layer computational complexity of the CMPNN, as defined in Algorithm 2 (CopresheafConv), is: $O(m \\\\cdot (\\\\mathcal{C}(\\\\rho) + d^2) + n \\\\cdot d^2) = O(n \\\\cdot k \\\\cdot (\\\\mathcal{C}(\\\\rho) + d^2) + n \\\\cdot d^2)$.\n\n**Proof**: The CMPNN message-passing operation (Algorithm 2, CopresheafConv) involves computing morphisms, applying them to features, aggregating messages, and updating vertex features. We analyze each step:\n\n1. **Morphism Computation**: For each edge $y \\\\to x \\\\in E_N$, compute $\\\\rho_{y \\\\to x} \\\\in \\\\mathbb{R}^{d \\\\times d}$ based on source and target features $(h_y^{(t)}, h_x^{(t)}) \\\\in \\\\mathbb{R}^{2d}$, using a map family from Table 16. The complexity per morphism is $\\\\mathcal{C}(\\\\rho)$, yielding: $O(m \\\\cdot \\\\mathcal{C}(\\\\rho))$.\n\n2. **Morphism Application**: Apply $\\\\rho_{y \\\\to x}$ to the feature vector $h_y^{(t)} \\\\in \\\\mathbb{R}^d$. This matrix-vector multiplication requires $O(d^2)$ operations per edge, totaling: $O(m \\\\cdot d^2)$.\n\n3. **Aggregation**: For each vertex $x \\\\in X \\\\subseteq V_N$, aggregate messages from neighbors $y \\\\in \\\\mathcal{N}(x)$ via $m_x = \\\\sum_{y \\\\in \\\\mathcal{N}(x)} \\\\rho_{y \\\\to x}(h_y^{(t)})$. Each addition involves a $\\\\mathbb{R}^d$ vector, costing $O(d)$ per neighbor. With $|\\\\mathcal{N}(x)|$ neighbors, the cost per vertex is $O(|\\\\mathcal{N}(x)| \\\\cdot d)$. Summing over all vertices: $O\\\\left( \\\\sum_{x \\\\in X} |\\\\mathcal{N}(x)| \\\\cdot d \\\\right) = O(m \\\\cdot d)$.\n\n4. **Update**: Assign $h_x^{(t+1)} = m_x$, with negligible cost. If an optional single-layer MLP with $d$ hidden units is applied (common in GNNs), the cost per vertex is $O(d^2)$, yielding: $O(n \\\\cdot d^2)$.\n\nThe total complexity is: $O(m \\\\cdot \\\\mathcal{C}(\\\\rho) + m \\\\cdot d^2 + m \\\\cdot d + n \\\\cdot d^2)$. Since $m = k \\\\cdot n$ and $d^2$ dominates $d$, this simplifies to: $O(m \\\\cdot (\\\\mathcal{C}(\\\\rho) + d^2) + n \\\\cdot d^2) = O(n \\\\cdot k \\\\cdot (\\\\mathcal{C}(\\\\rho) + d^2) + n \\\\cdot d^2)$.\n\n**Proposition (Copresheaf Transformer Complexity)**: Consider a directed graph $G_N = (V_N, E_N)$ induced by a combinatorial complex $\\\\mathcal{X}$ with neighborhood function $\\\\mathcal{N}$, where $|V_N| = n$, $|E_N| = m$, and the average degree is $k = m/n$. Each vertex $x \\\\in V_N$ has a feature space $\\\\mathcal{F}(x) = \\\\mathbb{R}^d$, and each of $H$ attention heads computes a morphism $\\\\rho_{y \\\\to x}: \\\\mathbb{R}^{d/H} \\\\to \\\\mathbb{R}^{d/H}$ via a map family from Table 16 with complexity $\\\\mathcal{C}(\\\\rho)$. The per-layer computational complexity of the Copresheaf Transformer, as defined in Algorithm 1 (SheafTransformerLayer), with sparse cross-attention based on $\\\\mathcal{N}$, is: $O(H \\\\cdot m \\\\cdot (\\\\mathcal{C}(\\\\rho) + d^2 / H) + n \\\\cdot d^2) = O(H \\\\cdot n \\\\cdot k \\\\cdot (\\\\mathcal{C}(\\\\rho) + d^2 / H) + n \\\\cdot d^2)$.\n\n**Proof**: The Copresheaf Transformer layer (Algorithm 1, SheafTransformerLayer) integrates self-attention within cells of equal rank and sparse cross-attention between neighbors, using learned copresheaf morphisms. Each of $H$ attention heads operates on features of dimension $d/H$. The complexity is:\n\n1. **Morphism Computation**: For each edge $y \\\\to x \\\\in E_N$ and each head, compute $\\\\rho_{y \\\\to x} \\\\in \\\\mathbb{R}^{(d/H) \\\\times (d/H)}$ based on $(h_y^{(t)}, h_x^{(t)})$. The complexity per morphism is $\\\\mathcal{C}(\\\\rho)$, totaling: $O(H \\\\cdot m \\\\cdot \\\\mathcal{C}(\\\\rho))$.\n\n2. **Self- and Cross-Attention**: Compute query ($Q$), key ($K$), and value ($V$) matrices per head ($d/H$ dimensions), costing $O(n \\\\cdot d^2 / H)$. For sparse cross-attention, each vertex attends to $|\\\\mathcal{N}(x)|$ neighbors, with attention scores $QK^\\\\top$ and application to $V$ costing $O(|\\\\mathcal{N}(x)| \\\\cdot d/H)$ per vertex per head. Summing over vertices and heads: $O\\\\left( H \\\\cdot \\\\sum_{x \\\\in X} |\\\\mathcal{N}(x)| \\\\cdot d/H \\\\right) = O(m \\\\cdot d)$. Self-attention within cells (assuming $O(n)$ tokens) costs $O(n \\\\cdot d^2 / H)$ per head, or $O(n \\\\cdot d^2)$ for $H$ heads, but sparse cross-attention dominates in sparse graphs.\n\n3. **Morphism Application**: Apply $\\\\rho_{y \\\\to x}$ to value vectors ($d/H$ dimensions) per edge and head, costing $O((d/H)^2) = O(d^2 / H^2)$. For $H$ heads and $m$ edges: $O(H \\\\cdot m \\\\cdot d^2 / H^2) = O(m \\\\cdot d^2 / H)$.\n\n4. **Output and Feed-Forward**: Combine head outputs and apply a per-token feed-forward network (FFN) with $d$ hidden units, costing: $O(n \\\\cdot d^2)$.\n\nThe total complexity is: $O(H \\\\cdot m \\\\cdot \\\\mathcal{C}(\\\\rho) + m \\\\cdot d + m \\\\cdot d^2 / H + n \\\\cdot d^2)$. Since $d^2 / H$ dominates $d$, and $m = k \\\\cdot n$, this simplifies to: $O(H \\\\cdot m \\\\cdot (\\\\mathcal{C}(\\\\rho) + d^2 / H) + n \\\\cdot d^2) = O(H \\\\cdot n \\\\cdot k \\\\cdot (\\\\mathcal{C}(\\\\rho) + d^2 / H) + n \\\\cdot d^2)$.\n\n**Comparison with Standard Architectures**\n\nCompared to standard GNNs (e.g., GCN, $O(n \\\\cdot k \\\\cdot d + n \\\\cdot d^2)$) and dense Transformers ($O(n^2 \\\\cdot d + n \\\\cdot d^2)$), CMPNN and CT incur higher costs due to morphism computation ($O(m \\\\cdot \\\\mathcal{C}(\\\\rho))$ and $O(H \\\\cdot m \\\\cdot \\\\mathcal{C}(\\\\rho))$). Assuming $\\\\mathcal{C}(\\\\rho) = O(d^2)$, CMPNN's complexity is $O(n \\\\cdot k \\\\cdot d^2 + n \\\\cdot d^2)$, and CT's is $O(H \\\\cdot n \\\\cdot k \\\\cdot d^2 + n \\\\cdot d^2)$. These costs enable superior expressiveness, as shown in Table 11 and Figure 6 (e.g., 7.2% MSE reduction for CT). The table below summarizes the per-layer complexities for a sparse graph.\n\n| Model | Complexity |\n|-------|------------|\n| GCN | $O(n \\\\cdot k \\\\cdot d + n \\\\cdot d^2)$ |\n| CMPNN | $O(n \\\\cdot k \\\\cdot d^2 + n \\\\cdot d^2)$ |\n| Transformer (Dense) | $O(n^2 \\\\cdot d + n \\\\cdot d^2)$ |\n| Copresheaf Transformer (CT) | $O(H \\\\cdot n \\\\cdot k \\\\cdot d^2 + n \\\\cdot d^2)$ |"}, {"review_id": "H5m4dzU4eN", "reviewer": "Reviewer_bhEu", "summary": "The paper Copresheaf Topological Neural Networks: A Generalized Deep Learning Framewor introduces a novel category-theoretic framework for neural networks using copresheaves over topological spaces, leading to the architecture of Copresheaf Topological Neural Networks (CTNNs). The author's empirical results on structured data benchmarks demonstrate that CTNNs consistently outperform conventional baselines, particularly in tasks requiring hierarchical or localized sensitivity.", "questions": "Can the copresheaf structure be generalized to sheaves, and what additional expressivity would that provide?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\n1. The approach is deeply rooted in category theory and topology, offering a powerful abstraction beyond traditional graph-based models.\n2. CTNNs unify several neural architectures (CNNs, GNNs, transformers) under a single topological formalism.\n3. The use of copresheaves and colimits provides a rigorous language to describe local-global behavior in networks.\n\nWeaknesses:\n\n1. This is a really minor one but, there is no proofs in the paper. The proofs in the supplementary material can be added to an appendix in the paper. It's more comfortable to read.\n2. The cost and implementation challenges of working with copresheaf structures and colimits in large networks are not thoroughly discussed.", "was_revised": true, "final_justification": "Thank you for the detailed clarification. I will keep my score.", "rebuttal": "We thank the reviewer for their assessment and observing that our proposed framework offers a ``powerful abstraction``, ``unifying several neural architectures under a single topological formalism``, in a ``rigorous language``.\nWe address their comments point by point in what follows:\n\n> *W1. Organization of the main paper and supplementary; proofs are in the supplementary.*\n\n**Author Response:**\nWe thank the reviewer for this observation. Due to the separate submission deadlines of the main paper and supplementary materials, we opted to keep the proofs in the supplementary.\nThanks to the extra page allowed, in the final version, we now move these to the appendix -- as far as this is permitted by the venue.\n\n---\n\n> *W2. Computational Complexity. What are the costs and implementation challenges?*\n\n**Author Response:** \nThis practical question will be an important factor in distributing the proposed CTNN framework to the wider community, which we are actively working on. As this was a common point brought up by different reviewers, we kindly refer to the comprehensive analysis of the computational complexity we conducted in response to reviewer uisE Q2/3.\n\n---\n\n> *Q1. Generalization to Sheaves. How does the copresheaf structure generalize to sheaves?*\n\n**Author Response**\nThis is another great practical question; in fact, in Appendix D, we demonstrate that  CTNNs subsume sheaf message passing neural networks (SNN). \nMoreover, CTNNs are strictly more expressive, as we demonstrate in the following result.\n\n**Proposition (Strict Expressivity Superset)**: Let $F_{\\\\text{CTNN}}$, $F_{\\\\text{SNN}}$, and $F_{\\\\text{GNN}}$ denote the function classes realized by Copresheaf Topological Neural Networks (CTNNs), Sheaf Neural Networks (SNNs), and Graph Neural Networks (GNNs), respectively. Then:\n\n$F_{\\\\text{GNN}} \\\\subset F_{\\\\text{SNN}} \\\\subset F_{\\\\text{CTNN}}$, with $F_{\\\\text{CTNN}} \\\\setminus F_{\\\\text{SNN}} \\\\neq \\\\emptyset$.\n\n**Proof**: Any GNN can be viewed as an SNN with all restriction maps set to the identity, so $F_{\\\\mathrm{GNN}} \\\\subseteq F_{\\\\mathrm{SNN}}$, and strictness follows since SNNs can model heterophilic interactions that GNNs cannot. By Theorem 1, every SNN message‐passing update is realized by a CTNN on the bidirected graph, giving $F_{\\\\mathrm{SNN}} \\\\subseteq F_{\\\\mathrm{CTNN}}$. Finally, CTNNs impose no sheaf‐consistency constraints and by Proposition 5 can approximate arbitrary edge‐specific linear maps via MLPs, so there exist functions in $F_{\\\\mathrm{CTNN}}$ not realizable by any SNN. Hence the strict hierarchy holds.\n\nWe include this in Appendix D, where we distinguish our copresheaf CTNN framework from existing SNNs. We thank the reviewer for their comments, now making our paper even stronger."}, {"review_id": "vX4NjFqyTG", "reviewer": "Reviewer_wnSD", "summary": "This paper introduces Copresheaf Topological Neural Networks (CTNNs), a generalization of standard message passing that extends beyond pairwise (node-to-node) interactions. CTNNs operate on Combinatorial Complexes (CCs) by leveraging neighboring functions to define Copresheaf Neighborhood Matrices (CNMs). Unlike traditional binary adjacency matrices, CNMs encode richer local dependencies through copresheaf maps between data associated with the cells of a CC. Building on this foundation, the authors define the Copresheaf Adjacency Matrix (CAM) and Copresheaf Incidence Matrix (CIM), enabling a novel message passing framework that unifies and generalizes across multiple architectures, including CNNs, GNNs, and Transformers. CTNNs provide a flexible and expressive mechanism for capturing complex, multi-scale relationships in structured data. Experiments on standard benchmarks demonstrate the superior expressivity of CTNNs compared to popular baselines such as GCNs and GraphSAGE.", "questions": "1) In your construction, what is the precise base category C over which the copresheaf is defined? Are the cells of the combinatorial complex treated as the objects of C, with morphisms defined by inclusion relations (e.g., x \\subset y) or directed neighborhood links (e.g., x → y)? Moreover, are you modeling a strict copresheaf — that is, a covariant functor F: C → Vect_R — where each object x is assigned a vector space F(x), and each morphism f: x → y is assigned a linear map F(f): F(x) → F(y), such that F(id_x) = id_{F(x)} and F(g o f) = F(g) o F(f)? Or are you using the term \"copresheaf\" more loosely, to describe a set of learned linear maps ρ_{x→y} \\in Hom(F(x), F(y)) without requiring functorial properties like compositionality or identity preservation? Just want to make sure I'm understanding your use of the term correctly.\n\n2) CTNNs operate over combinatorial complexes that include cells of varying dimensions (e.g., vertices, edges, faces). The model defines neighborhood functions like N_adj^(r,k)(x) and N_inc^(r,k)(x), as seen in Definition 8, which explicitly allow message passing between cells of different ranks. This suggests that mixed-rank message aggregation is a fundamental part of the framework.\n\nGiven this, a few important questions arise:\n\n* How are feature spaces F(x) defined and aligned across different cell types or ranks?\n\n* When dim(F(y)) ≠ dim(F(x)), how are the linear maps rho_{y→x} in Hom(F(y), F(x)) implemented to handle these differences in dimensionality?\n\n* In message passing (e.g., Proposition 3), where aggregation is performed over multiple neighborhood functions N_k(x), how are contributions from cells of different types or ranks aggregated meaningfully, especially when tensor products and summations are involved? Clarifying these points would help me better understand how CTNNs handle heterogeneous and multi-scale interactions in practice.\n\n3) You describe the CTNN layer using copresheaf-based neighborhood matrices (CNM, CAM, CIM), which involve learned linear maps for each directed relation. However, the paper does not provide any complexity analysis or runtime/memory benchmarks.\n\n* How does the runtime and memory usage of a CTNN layer scale compared to a standard GNN or Transformer layer on the same data?\n\n* Are there cases where the number of ρ_{y→x} maps becomes a bottleneck (e.g., in dense or high-rank complexes)?\n\n* Have you measured wall-clock time, peak memory usage, or FLOPs to quantify the overhead of this generalization?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "# Technical Strengths\n\n1. **Generalized Theoretical Framework**  \nCTNNs introduce a principled message-passing paradigm rooted in category theory and combinatorial complexes, unifying GNNs, Transformers, and sheaf-based models under a single copresheaf formalism.\n\n2. **Directional and Heterogeneous Representations**  \nBy assigning task-specific feature spaces and learnable maps to each cell, CTNNs move beyond the conventional shared latent space, enabling anisotropic, structured message passing tailored to complex data.\n\n3. **Superior Expressivity and Stability**  \nAcross diverse benchmarks, ranging from physical simulations to graph classification, CTNNs consistently outperform standard models, offering lower errors, improved stability, and enhanced ability to model higher-order structure.\n\n4. **Architecture-Agnostic Versatility**  \nThe copresheaf framework instantiates naturally as Copresheaf GNNs, Transformers, and CNNs, making it broadly applicable across Euclidean and non-Euclidean domains alike.\n\n# Technical Weaknesses\n\n1. **Computational Overhead**  \nThe use of per-edge copresheaf maps introduces additional memory and runtime costs, making scalability a challenge for large or densely connected structures.\n\n2. **Limited Scale of Experiments**  \nMost experiments are conducted on synthetic or small-scale datasets, so the framework’s performance on large, real-world tasks remains untested.\n\n3. **Lack of Ablation or Efficiency Analysis**  \nThe paper doesn’t include detailed ablations to isolate where performance gains come from, nor does it evaluate trade-offs in runtime or parameter count.", "was_revised": true, "final_justification": "I thank the authors for their thoughtful and detailed rebuttal. After carefully considering their responses and discussions with the other reviewers and the area chair, I am increasing my score to 5.\n\nSince all of my core questions were convincingly addressed, and the overall contribution is sound and relevant to the community, I have given significant weight to the improved clarity and completeness offered during the rebuttal.\n\nOverall, I believe this paper makes a meaningful contribution and merits acceptance.", "rebuttal": "We thank the reviewer for their effort in reviewing our work and for the detailed comments. We appreciate that they ranked our paper `excellent` in terms of clarity, significance, and originality. Below, we address their comments point by point.\n\n> *W1/Q3. Computational Overhead*\n\n**Author Response:** \nWe thank the reviewer for pointing this out; indeed, adding additional generality comes at some cost. Based on a recurring emphasis from the reviewers, we have conducted an extensive complexity analysis and included this in the manuscript.\nWe refer the reviewer to our complexity discussion in the response to Reviewer uisE Q2/3. Due to space limits, we omitted that here. \n\nIn summary:\n- **CMPNN complexity**: O(n·k·d² + n·d²) vs. O(n·k·d + n·d²) for standard GNNs\n- **CT complexity**: O(H·n·k·d² + n·d²) vs. O(n²·d + n·d²) for dense Transformers\n- The additional factor of d comes from computing morphisms ρ_{y→x}, with complexity C(ρ) ranging from O(d) for diagonal maps to O(d²) for full matrices\n\nWe will add runtime experiments in the main paper. Empirically, we found that small copresheaf dimensions suffice for most tasks, providing efficiency in practice.\n\n---\n\n> *W2. Limited Scale Experiments*\n\n\n**Author Response:** We appreciate the reviewer’s concern regarding the scope of benchmarks and baselines. The main goal of this work is to *demonstrate the expressive and unifying power of Copresheaf Topological Neural Networks (CTNNs)* by showing how they recover—and meaningfully extend—canonical deep learning architectures: CNNs, GNNs, Transformers, and Sheaf Neural Networks (SNNs). To this end, our experimental design deliberately focused on domains where the structural inductive biases introduced by copresheaf morphisms could be most clearly evaluated.\n\nThe datasets used—including PDE dynamics on grids, synthetic graphs, and MUTAG—were chosen to test these models across spatial, graph-based, and topological settings. While some are smaller-scale, they were sufficient to evaluate generalization, expressivity, and the benefits of directional feature propagation. For example:\n\n* In **Appendix F**, we introduce **CopresheafConv**, a direct generalization of CNNs applied to regular 2D grids. Here, we replace static, translation-invariant convolutional filters with learned, input- and position-conditioned morphisms ρ\\_{y→x}, enabling anisotropic and spatially-varying processing. Unlike CNNs, which impose uniform filters across the grid, CopresheafConv allows local directional adaptation, making it well-suited for data with varying geometry or dynamics.\n\nThis setup allowed us to show not only performance benefits but also how CTNNs naturally recover these architectures as special cases—substantiating the theoretical claims made in Section 3.\n\nWe acknowledge that large-scale datasets such as QM9 or OGB are important for benchmarking at scale and plan to include them in future work. Our current experiments across numerous diverse settings aim to establish CTNNs as a general-purpose, topologically grounded modeling framework.\n\n---\n\n> *W3. Ablations and Efficiency Analysis*\n\n**Author Response:** We thank the reviewer for highlighting the importance of ablation studies and efficiency analysis. We acknowledge that this is an important aspect for understanding the practical implications of our framework. While our current experiments demonstrate consistent improvements across diverse tasks, we agree that more targeted ablations would help isolate the sources of performance gains. Our experiments in Appendix G.1-G.3 provide some insights:\n\n- In Section 5.1 (Physics tasks), replacing standard attention with copresheaf maps yields 50%+ MSE reduction, suggesting the directional transport maps capture anisotropic dynamics better than isotropic averaging (please see the detailed response to Reviewer hjdt W4).\n- In section G.2.1, we see that tasks requiring explicit token-to-token relationships benefit most from copresheaf structures (+38pp on Query-to-Context), while simpler tasks show smaller gains.\n- The comparison between different copresheaf map families (Table 16) shows that expressiveness varies by complexity of $\\rho_{y\\rightarrow x}$.\n\nTo address the reviewer's concern more comprehensively, we now add explicit empirical comparisons in the revised version, comparing the impact of morphism complexities for various tasks, for instance, identity maps, diagonal maps, MLPs, and full linear morphisms, quantifying the contribution of anisotropic transport maps. From our experience, the choice of copresheaf map is highly task dependent, with no morphism strictly outperforming others. \n\n---\n\n> *Q1. Base category and functoriality*\n\n**Author Response:** In CTNNs, the copresheaf $\\mathcal{F}$ is defined over a base category $\\mathcal{C}$, which is the poset category of the directed graph $G_N = (V_N, E_N)$ induced by a neighborhood function $N: X \\to P(X)$ on a combinatorial complex $X$. The objects of $\\mathcal{C}$ are the vertices of $G_N$, i.e., $V_N = X_N \\cup \\bigcup_{x \\in X} N(x)$, where $X_N = \\{ x \\in X \\mid N(x) \\neq \\emptyset \\}$. Morphisms in $\\mathcal{C}$ are directed edges $y \\to x \\in E_N$, where $y \\in N(x)$. The copresheaf $\\mathcal{F}: \\mathcal{C} \\to \\text{Vect}$ assigns vector spaces $\\mathcal{F}(x)$ to vertices and learnable maps $\\rho_{y \\to x}: \\mathcal{F}(y) \\to \\mathcal{F}(x)$ to edges. Unlike Sheaf NNs, these morphisms reflect directional neighborhood links rather than inclusion relations.\n\nThe definition is functorial, but in experiments, we did not enforce functoriality. The maps $\\rho_{y \\to x}$ are learned independently to flexibly model directional, local-to-global relations. Imposing functorial constraints is an interesting future direction.\n\n---\n\n> *Q2. Operating over combinatorial complexes*\n\n**Author Response:** \n\n**(1) Feature space alignment:** The feature spaces $\\mathcal{F}(x)$ need not be aligned initially. For example, vertex, edge, and face features in a mesh may differ in dimension. When $\\dim(\\mathcal{F}(y)) \\neq \\dim(\\mathcal{F}(x))$, even within the same rank, the maps $\\rho_{y \\to x} \\in \\text{Hom}(\\mathcal{F}(y), \\mathcal{F}(x))$ are matrices of size $\\dim(\\mathcal{F}(x)) \\times \\dim(\\mathcal{F}(y))$. These are parameterized using families such as:\n\n* Diagonal MLP: $\\rho_{y \\to x} = \\text{diag}(\\sigma(\\text{MLP}[q_x, k_y]))$\n* Graph Attention: $\\rho_{y \\to x} = \\sigma(\\text{MLP}[q_x, k_y]) \\cdot \\text{Id}$\n* Outer Product: $\\rho_{y \\to x} = W_q q_x (W_k k_y)^\\top$\n\n**(2) Handling dimension mismatch:** The linear maps are implemented as above to handle any $\\dim(\\mathcal{F}(y)) \\neq \\dim(\\mathcal{F}(x))$, ensuring flexibility without requiring uniform dimensions. See also Appendix Table 16.\n\n**(3) Message passing over multiple neighborhoods:** In practice (e.g., Proposition 3), CTNNs use a \"merge node\" to aggregate messages from neighborhoods $N_k(x)$. The target $\\mathcal{F}(x)$ and all incoming $\\rho_{y \\to x}(\\mathcal{F}(y))$ are projected to the same dimension in a CTNN layer. Aggregation (e.g., via sum/tensor ops) is weighted using learnable attention or MLPs. If input dimensions differ, map families like Diagonal MLP or Outer Product project to a common latent space, enabling multi-scale integration."}]

Author rebuttal:

Meta-review: The paper introduces CTNNs: neural layers on combinatorial complexes where each cell $x$ has a local feature space $F(x)$ and each directed relation $y\to x$ carries a learned linear map $\rho_{y\to x}:F(y)\to F(x)$. This yields directional, heterogeneous, higher-order message passing that recovers standard GNNs, CNNs, Transformers, and SNNs as special cases; SNN updates are shown to arise on bidirected graphs via a single copresheaf morphism. Targeted experiments cover PDE dynamics, MUTAG, and CC classification.   

Strengths.  
- Principled unification with explicit operators (CNM/CAM/CIM) and message-passing propositions; clear SNN to CTNN construction.   
- Empirical signal on tasks where anisotropy and directionality matter: sizable MSE reductions on advection/heat and lower variance; consistent MUTAG gains over GCN/GraphSAGE/GIN; improved CC classification.    
- Clear motivation for combinatorial complexes as a rank-aware domain unifying grids, graphs, meshes, and higher-order structures.  

Weaknesses.
- Limited scale and breadth of benchmarks, comparisons to recent SOTA are sparse.
- Efficiency characterization remains light given per-edge transports, notation density can hinder accessibility.  

Reviewers’ main concerns. Mechanism of unification, CTNNs vs SNNs, why combinatorial complexes, complexity and practical overhead, physics-task interpretation, dataset/baseline scope.

Authors’ response.  
 Added a centralized mapping from classical models to CTNN instances, clarified CTNN–SNN relation and expressivity hierarchy $F_{\text{GNN}} \subset F_{\text{SNN}}\subset F_{\text{CTNN}}$, expanded complexity discussion, provided physical interpretation and visualizations for advection/Stokes, acknowledged benchmark-scale limits.

Discussion outcome.  
hjdt’s concerns resolved and score increased; uisE upscored after expressivity/complexity clarifications; bhEu kept acceptance; EnAf and wnSD remained concerned about scale. The core theory and targeted empirical evidence are solid, the rebuttal materially improved clarity.

Decision.  
Accept. The work offers a unifying formalism with concrete operators and empirically relevant gains on tasks aligned with its inductive bias. Despite limited large-scale evaluations, the contribution is above threshold on theoretical depth, conceptual unification, and demonstrated benefits.
Decision: Accept (poster)