SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Similarly,  $e_1^{\mathbf{C}}$  is also transformed into a constant scalar tuple  $\tilde{e}_1^{\mathbf{C}} = (1,0,0)$  through  $\mathcal{F}_{\mathbf{C}}$ . As constant inputs generate constant outputs, we conclude that the derived local scalars can only approximate constant functions. However, since the local frames change as we vary the 3D structure of  $\mathbf{B}$  and  $\mathbf{C}$ , it is evident that  $e_1^{\mathbf{B}} \cdot e_1^{\mathbf{C}}$  is not a constant function of  $(\mathbf{B},\mathbf{C})$ . Thus, we complete the proof by contradiction.
Similarly,  $e_1^{\mathbf{C}}$  is also transformed to a constant scalar tuple  $\tilde{e}_1^{\mathbf{C}} = (1,0,0)$  through  $\mathcal{F}_{\mathbf{C}}$ . As constant inputs generate constant outputs, we conclude that the deduced local scalars can only approximate constant functions. However, since the local frames are changing as we vary the 3D structure of  $\mathbf{B}$  and  $\mathbf{C}$ , it's obvious that  $e_1^{\mathbf{B}} \cdot e_1^{\mathbf{C}}$  is not a constant function of  $(\mathbf{B},\mathbf{C})$ . Therefore, we finish the proof by contradiction.
Further Comments.
1. It is important to note that the counterexample we constructed is not the only valid counterexample. For instance, the norm of any tensors that depend on both  $\mathbf{B}$  and  $\mathbf{C}$  (e.g.,  $\| e_1^{\mathbf{B}} + e_1^{\mathbf{C}}\|$ ,  $\| e_1^{\mathbf{B}}\otimes e_2^{\mathbf{B}}\otimes e_2^{\mathbf{C}}\otimes e_3^{\mathbf{C}}\|$ ) can also serve as valid counterexamples.
2. Let  $R_{\mathbf{BC}}$  denote the frame transition function between  $\mathcal{F}_{\mathbf{B}}$  and  $\mathcal{F}_{\mathbf{C}}$ , defined in the same way as in Equation 5. The proof of Theorem 4.1 implies that all types of invariant interactions  $f_{a}(\mathbf{B}, \mathbf{C})$  can be expressed solely as functions of the union of three sets  $S_{\mathbf{B}}$ ,  $S_{\mathbf{C}}$ , and  $R_{\mathbf{BC}}$ :  $\{S_{\mathbf{B}} \| S_{\mathbf{C}} \| R_{\mathbf{BC}}\}$ . This is because, through  $R_{\mathbf{BC}}$ , the 3D point cloud  $\mathbf{C}$  can be scalarized by the local frame  $\mathcal{F}_{\mathbf{B}}$ . Specifically, let  $\tilde{S}_{\mathbf{C}}^{B}$ ,  $\tilde{S}_{\mathbf{C}}^{C}$  denote the scalarized 3D point cloud of  $S_{\mathbf{C}}$  with respect to  $\mathcal{F}_{\mathbf{B}}$  and  $\mathcal{F}_{\mathbf{C}}$ , respectively. We have the following diagram:
$$
S _ {\mathbf {C}} \xrightarrow {\mathcal {F} _ {\mathbf {C}}} \tilde {S} _ {\mathbf {C}} ^ {C} \xrightarrow {R _ {\mathbf {B C}}} \tilde {S} _ {\mathbf {C}} ^ {B}.
$$
Utilizing formula 15 on the union of the two 3D point clouds,  $(\mathbf{B},\mathbf{C})$ , leads us to this conclusion.
Realizing FT by Equivariant Messages: From the FT definition 5, each element of the  $3 \times 3$  matrix  $R_{ij}$  is calculated by
$$
R _ {i j} (k, l) = \mathbf {e} _ {k} ^ {i} \cdot \mathbf {e} _ {l} ^ {j}. \tag {18}
$$
Now we show how to reproduce  $R_{ij}(k,l)$  through equivariant messages. Let the equivariant message  $\mathbf{m}_i$  be the following:
$$
\mathbf {m} _ {i} := (\mathbf {e} _ {1} ^ {i}, \mathbf {e} _ {2} ^ {i}, \mathbf {e} _ {3} ^ {i}) \cdot \left[ \begin{array}{c c c c c c c c c c} 1 & 1 & 1 & 0 & 0 & 0 & 0 & 0 \\ \hline 0 & 0 & 0 & 1 & 1 & 1 & 1 & 0 & 0 & 0 \\ \hline 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 1 & 1 \end{array} \right]
$$
It's easy to check that  $\mathbf{m}_i\in \mathbf{R}^{3\times 9}$  consists of 9 equivariant vectors (multi-channels). For atom  $j$ $\mathbf{m}_j$  is defined symmetrically. For each node, we also store the scalar messages, e.g.,  $\| \mathbf{e}_k^i\|$  for  $1\leq k\leq 3$ . Flattening the whole matrix  $R_{ij}$  into a  $\mathbf{R}^{1\times 9}$  array, then  $R_{ij}$  is obtained by simple summation and taking the vector norm:
$$
\left\| \mathbf {m} _ {i} + \mathbf {m} _ {j} \right\| = \left\{\left\| \mathbf {e} _ {k} ^ {i} + \mathbf {e} _ {l} ^ {j} \right\| \right\} _ {1 \leq k, l \leq 3},
$$
where the norm is taken for each column of  $\mathbf{m}_i + \mathbf{m}_j$ , such that  $\| \mathbf{m}_i + \mathbf{m}_j\| \in \mathbf{R}^{1\times 9}$ . Then,
$$
R _ {i j} = \frac {1}{2} \left[ \left\| \mathbf {e} _ {k} ^ {i} + \mathbf {e} _ {l} ^ {j} \right\| ^ {2} - \left\| \mathbf {e} _ {k} ^ {i} \right\| ^ {2} - \left\| \mathbf {e} _ {l} ^ {j} \right\| ^ {2} \right].
$$
Our illustration also demonstrates the importance of keeping multi-channel tensor messages.
Relation with Previous Equivariant Update Methods. Following the efficiency principle established in section 4, we don't encode the data of the transition matrices explicitly. Instead, we implement tensor messages to fill in the expressiveness gap. Among the tremendously different designs of equivariant graph neural networks, Schütt et al. [35] is closely related to our equivariant updating method. By the above argument, the inner product operation for node  $i$  (see (9) of Schütt et al. [35])
$$
<   \mathbf {U} \mathbf {v} _ {i}, \mathbf {V} \mathbf {v} _ {i} >
$$
can also be reinterpreted as a realization of the (aggregated) frame transition matrix (5).
Moreover, since the equivariant vectors  $\mathbf{U}\mathbf{v}_i$  and  $\mathbf{V}\mathbf{v}_i$  are both aggregated vector features that belong to the same node  $i$  and the inner product operation between them is performed in the node-wise updating phase, Schutt et al. [35] actually avoids the 2-hop  $O(k^2)$  complexity of computing  $R_{xy}$  for all neighborhood node pairs  $(\mathbf{x},\mathbf{y})$  (while able to express the torsion angle implicitly). For our algorithm, we utilize the scalarization and tensorization in the node-wise updating phase. By the universal approximation theorem 5.1, our method can approximate any inner product operations.
F Related proofs and discussions of Section 5
Equivariant Frames and Higher Order Scalarization and Tensorization. Given an edge  $e_{ij}$  with two atom positions  $(\mathbf{x}_i, \mathbf{x}_j)$ , we define our edge-wise  $SE(3)$  equivariant frames  $\mathcal{F}_{ij}$  as follows:
$$
\left(\mathbf {e} _ {1}, \mathbf {e} _ {2}, \mathbf {e} _ {3}\right) = \left(\frac {\mathbf {x} _ {i} - \mathbf {x} _ {j}}{\| \mathbf {x} _ {i} - \mathbf {x} _ {j} \|}, \frac {\mathbf {x} _ {i} \times \mathbf {x} _ {j}}{\| \mathbf {x} _ {i} \times \mathbf {x} _ {j} \|}, \frac {\mathbf {x} _ {i} - \mathbf {x} _ {j}}{\| \mathbf {x} _ {i} - \mathbf {x} _ {j} \|} \times \frac {\mathbf {x} _ {i} \times \mathbf {x} _ {j}}{\| \mathbf {x} _ {i} \times \mathbf {x} _ {j} \|}\right). \tag {19}
$$
To ensure frame translation invariance, we adopt the approach followed by previous works [59, 60] by restricting the entire 3D conformer space to a linear subspace where the center of mass (CoM) of the system (either the entire system or the sub-cluster to which  $i$  and  $j$  belong) is set to zero. Alternatively, constructing an  $E(3)$  frame is also possible but requires an additional atom position  $\mathbf{x}_k$ , which can be selected using the K-Nearest Neighbor algorithm. If  $(\mathbf{x}_i,\mathbf{x}_j,\mathbf{x}_k)$  spans the 3D space, we obtain an  $E(3)$  equivariant frame by performing Gram-Schmidt orthogonalization. For different constructions of  $E(3)$  frames, readers can refer to Wang and Zhang [61]. However, in this work, we focus on  $SE(3)$  frames since molecular 3D conformers exhibit  $SE(3)$  symmetry but do not possess reflection symmetry.
Once we have an equivariant frame, every vector is a linear combination of the three orthogonal vectors in the frame. Moreover, the unique combination coefficients are exactly the 'scalarized' coordinates in (8). A similar procedure also applies to higher order tensors. Indeed, the vector frame  $\mathcal{F}^1$  extends to a tensor frame  $\mathcal{F}^r$  of arbitrary order  $r > 1$ :
$$
\mathcal {F} ^ {r} := \left\{\mathbf {e} _ {1 _ {1}} \otimes \dots \otimes \mathbf {e} _ {1 _ {r}} \right\} _ {1 \leq i _ {1}, \dots , i _ {r} \leq 3}. \tag {20}
$$
Since the orthonormal frame  $\mathcal{F}^r$  is complete in the sense that it spans the whole tensor space of order  $r$ , every r-th order tensor admits a unique decomposition:
$$
\mathbf {T} = \sum_ {1 \leq i _ {1}, \dots , i _ {r} \leq 3} T ^ {i _ {1}, \dots , i _ {r}} \mathbf {e} _ {i _ {1}} \otimes \dots \otimes \mathbf {e} _ {i _ {r}}. \tag {21}
$$
It's easy to prove that the collection  $\{T^{i_1,\dots,i_r}\}_{1\leq i_1,\ldots,i_r\leq 3}$  consists of invariant scalars. We call the process from  $\mathbf{T}$  to  $\{T^{i_1,\dots,i_r}\}_{1\leq i_1,\ldots,i_r\leq 3}$  scalarization.
Tensorization is the inverse of scalarization, in the sense that it sends scalars  $\{T^{i_1,\dots ,i_r}\}_{1\leq i_1,\ldots ,i_r\leq 3}$  to tensor  $\mathbf{T}$ . Under the same frames we use during scalarization, the following diagram demonstrates the pipeline of producing  $L$  second-order tensors out of  $\{T_j^{i_1,i_2}\}_{1\leq i_1,i_2\leq 3}$ :
$$
\left\{\mathbf {T} _ {1}, \dots , \mathbf {T} _ {L} \right\} = \underbrace {\left\{\left[ \begin{array}{l l l} T _ {1} ^ {1 1} , & T _ {1} ^ {1 2} , & T _ {1} ^ {1 3} \\ T _ {1} ^ {2 1} , & T _ {1} ^ {2 2} , & T _ {1} ^ {2 3} \\ T _ {1} ^ {3 1} , & T _ {1} ^ {3 2} , & T _ {1} ^ {3 3} \end{array} \right] , \dots , \left[ \begin{array}{l l l} T _ {L} ^ {1 1}, & T _ {L} ^ {1 2}, & T _ {L} ^ {1 3} \\ T _ {L} ^ {2 1}, & T _ {L} ^ {2 2}, & T _ {L} ^ {2 3} \\ T _ {L} ^ {3 1}, & T _ {L} ^ {3 2}, & T _ {L} ^ {3 3} \end{array} \right] \right\}} _ {L \text {c h a n n e l s}} \odot \left[ \begin{array}{l l l} \mathbf {e} _ {1} \otimes \mathbf {e} _ {1}, & \mathbf {e} _ {1} \otimes \mathbf {e} _ {2}, & \mathbf {e} _ {1} \otimes \mathbf {e} _ {3} \\ \mathbf {e} _ {2} \otimes \mathbf {e} _ {1}, & \mathbf {e} _ {2} \otimes \mathbf {e} _ {2}, & \mathbf {e} _ {2} \otimes \mathbf {e} _ {3} \\ \mathbf {e} _ {3} \otimes \mathbf {e} _ {1}, & \mathbf {e} _ {3} \otimes \mathbf {e} _ {2}, & \mathbf {e} _ {3} \otimes \mathbf {e} _ {3} \end{array} \right], \tag {22}
$$
where  $\odot$  denotes the element-wise product.
Proof of Theorem 5.1
Proof. The proof relies on the invertibility of Scalarization and Tensorization operations, as demonstrated in Appendix A.5 of Du et al. [30]. This invertibility allows us to establish a commutative
$$
\begin{array}{c} \mathbf {T} ^ {l - 1} \xrightarrow {\rho} \mathbf {T} ^ {l} \\ \text {d i a g r a m a s f o l l o w s :} \\ \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \text {T e n s o r i z e} \\ \tilde {T} ^ {l - 1} \xrightarrow {\mathbf {M L P}} \tilde {T} ^ {l - 1}. \end{array}
$$
This diagram illustrates that for each mapping  $\rho$ , there exists a corresponding "scalarized" mapping  $\tilde{\rho}$  given by:
$$
\tilde {\rho} := \text {T e n s o r i z e} \circ \rho \circ \text {S c a l a r i z e}.
$$
By applying Tensorize, followed by  $\rho$ , and then Scalarize, we obtain an invariant representation  $\tilde{\rho}$ . Since MLP serves as a universal approximator of invariant functions, we can always find an MLP that expresses  $\tilde{\rho}$ . By reversing the arrows, we conclude the proof.
Proof of Equivalence for LEFTNet LeftNet consists of multiple layers of LSE and FTE. LSE is realized by scalarization, and FTE is realized by scalarization and tensorization. Since the invariance of scalarization and the equivariance of tensorization have been proved, we finish the proof.
G Extended Related Work
Table 4: Categorization of representative geometric GNN algorithms. * denotes partially satisfying the requirement.
Method	Symmetry	LSE	FTE	Complexity
SchNet [28]	E(3)-invariant	X	X	O(nk)
EGNN [14]	E(3)-equivariant	X	✓*	O(nk)
GVP-GNN [31]	E(3)-equivariant	X	✓	O(nk)
ClofNet [30]	SE(3)-equivariant	X	X	O(nk)
PaiNN [35]	E(3)-equivariant	X	✓	O(nk)
ComENet [23]	SE(3)-invariant	✓	✓*	O(nk)
TFN [13]	SE(3)/E(3)-equivariant	X	✓	O(nk)
Equiformer [36]	SE(3)/E(3)-equivariant	X	✓	O(nk)
SphereNet [15]	SE(3)-invariant	✓*	✓*	O(nk2)
GemNet [37]	SE(3)-invariant	✓*	✓*	O(nk3)
LEFTNet (Ours)	SE(3)/E(3)-equivariant	✓	✓	O(nk)
Based on the discussions in Section 3 and 4, we identify two essential components for constructing expressive equivariant 3D Graph Neural Networks (GNNs): (1) Local 3D Substructure Encodings (LSE), which enable the local message passing to capture diverse local 3D structures; and (2) Frame Transition Encodings (FTE), which incorporate equivariant coordinate transformations between different local patches into the 3D GNN.
G.1 Modular overview of 3D GNN
Based on the general notion of Local 3D Substructure Encodings (LSE) and Frame Transition Encodings (FTE), we have provided concrete constructions to realize them. However, it is important to note that there are also implicit methods to encode the information of LSE and FTE. In order to provide a comprehensive overview, we review previous 3D Graph Neural Networks (GNNs) that follow this framework and summarize the findings in Table 4. To ensure a fair comparison, we include the computational complexity as it is often a trade-off with expressiveness. A detailed analysis of this trade-off is provided at the end of Section G.2.
Regarding Local 3D Substructure Encodings (LSE), SphereNet [15] and GemNet [37] (implicitly) encode local 3D substructures through a computation-intensive edge-based update. We indicate these two architectures with a * in Table 4 due to the following reasons: In the case of GemNet [37], the message passing framework considers the 1-hop neighborhood with respect to both end points of an edge  $e_{ij}$ , which inherently contains all the nodes of the mutual 3D subgraph  $S_{i-j}$ . However, when compared to SphereNet [15], which encodes all node positions under an angular coordinate system, GemNet [37] only captures partial geometric information. On the other hand, SphereNet [15] utilizes an edge-based update that aggregates 1-hop neighbors with respect to the source node. While this approach considers the immediate neighbors, it may not include all the nodes inside the subgraph  $S_{i-j}$  (e.g., the neighbors of the target node).
Regarding Frame Transition Encodings (FTE), we have demonstrated (see the proof of Theorem 4.1) that the essential information of FTE does not rely on the specific choices of local equivariant frames. Therefore, most 3D GNNs with equivariant vector updates that can incorporate at least three (multi-channel) independent equivariant vectors as edge features are capable of expressing local frame transitions (FT). However, EGNN [14] is an exception as it only updates the position vector (i.e., one channel), which is insufficient for capturing the entire FT. In other words, the power of the update function  $\phi$  in Equation (6) also influences the encoding of FT. Additionally, models that encode torsion angle information partially express FTE, as illustrated in Appendix E. While there are multiple ways to realize LSE and FTE, there is a trade-off between efficiency and expressiveness in
terms of the number of hops considered for message passing, as indicated in the last column of Table 4.
In contrast to our invariant realization of LSE, Batatia et al. [24] constructs their framework by building a complete  $(E(3) +$  permutation) equivariant polynomial basis using spherical harmonics and tensor product, where the monomial variables are a combination of 3D features from different nodes (bodies). On the other hand, we achieve the function of LSE and FTE through edgewise scalarization  $A_{ij}$  and equivariant message passing (see Fig. 3). While Batatia et al. [24] utilize the atomic cluster expansion (ACE) mechanism, we provide an illustration of how to equivariantly realize LSE based on local ACE in the proof of Theorem 3.1. Another work Wang and Zhang [61] uses an ensemble of frames to model the local environment defined through a distance cutoff, which is suitable for pure 3D geometry. However, our local 3D hierarchy incorporates well with the 2D topology. Therefore, whether [61] encodes LSE depends on the radius of the local environment and whether the ensemble of 'frames' purely built as a combination of radical directions (see Equation (6) in [61]) spans  $\mathbf{R}^3$ . In comparison to geometric GNNs like SphereNet [15] and GemNet [37], which encode reflection-antisymmetric torsion angles, and LeftNet, which implements  $SE(3)$  equivariant, reflection-antisymmetric frames, the ensemble of frames in [61] is reflection symmetric and therefore cannot differentiate local chemical isomers. Additionally, Wang and Zhang [61] defines the '(ensembled) frame to frame' projection, which is somehow equivalent to our FTE, and utilizes it to detect the global isomorphism from local observations. In contrast, our FTE is employed to bridge the expressiveness gap of global continuous functions, such as regression tasks on graphs (see the comments in Section E), rather than focusing on detecting global isomorphisms, as in classification tasks on graphs. Moreover, we have demonstrated that using one  $SE(3)$  equivariant frame for each local patch is sufficient for achieving local expressiveness, considering efficiency as well. Finally, we provide another invariant realization of FTE, inspired by [62], which is presented at the end of Section H.
G.2 Relationship to Geometric WL test
Recently, Joshi et al. [22] propose a geometric k-WL test (GWL) to assess the expressiveness of geometric GNN algorithms. In essence, our tree isomorphism aligns with the 1-hop geometric isomorphism introduced in GWL, while the fine-grained triangular isomorphism falls between the 1-hop and 2-hop geometric isomorphism described in GWL. From a model design perspective, we achieve the realization of LSE through local scalarization, which guarantees expressiveness through the Kolmogorov representation theorem ( Zaheer et al. [57]) and the universal approximator property of MLP. Different from our geometry perspective, the fundamental concepts of body order and tensor order, which stem from classical inter-atomic potential theories and exhibit equivariance, play a crucial role in measuring expressive power in Joshi et al. [22]. In addition to the local geometric isomorphism hierarchy, we discover the significance of FTE as the connecting bridge between local invariant scalars and global geometric expressiveness. This realization, along with LSE on mutual 3D substructures, sheds light on the insufficiency of the 1-hop local scalarization implemented in ClofNet ( Du et al. [30]). We further explore the connection between FTE and the neural sheaf Laplacian in Section H.
Computational efficiency of LEFTNet In the analysis of a graph with  $n$  nodes and an average of  $k$  edges per node, the computational efficiency of a message passing-based graph neural network (MPNN) is typically described by the following form:
$$
\mathcal {O} \left(f n + g n \cdot k ^ {c}\right), \tag {23}
$$
where  $f$  represents the computational cost of the node-level updating function (usually an MLP), and  $g$  represents the computational cost of calculating the message for each edge. The power  $c$  is a crucial factor that determines the computational cost of performing each message passing operation. For instance, SphereNet (Liu et al., 2021) aggregates all 1-hop nodes with respect to the source node of an edge to construct a message, resulting in a complexity of  $\mathcal{O}(nk^2)$  for message passing.
To analyze the computational complexity of LEFTNet, we can divide it into the following computational steps:
1. Building the  $S_{i-j}$  and scalarizing with respect to edge frame  $\mathbf{F}_{ij}$ . The 3D subgraph  $S_{i-j}$  is constructed by intersecting the 1-hop neighbors  $\mathcal{N}_i$  and  $\mathcal{N}_j$ , which requires  $\mathcal{O}(nk)$
computations. Additionally, collecting the 1-hop neighbors for each node has a complexity of  $\mathcal{O}(nk)$ . Finally, scalarizing each  $S_{i-j}$  with the edge-wise frame  $\mathbf{F}_{ij}$  (which involves taking inner products) and extending the result to  $m$  channels has a complexity of  $\mathcal{O}(mnk)$ . Therefore, the total complexity for this step remains at  $\mathcal{O}(mnk)$ .
2. Equivariant message passing. From the first step, we get the scalarized  $\tilde{S}_{i-j}$  from the 3D subgraph  $S_{i-j}$ . Then, the edge-wise message is obtained by transforming the scalarized  $\tilde{S}_{i-j}$  with a MLP, and the output invariant scalars is multiplied with both the invariant and equivariant node features (let  $d$  denotes the dimension of the node feature, then the multiplication's complexity is  $\mathcal{O}(dnk)$ . Let  $l$  denotes the complexity of MLP (depends on the depth and hidden dimension of MLP), then the message passing takes  $\mathcal{O}((d+l)nk)$  computation.
3. Node level updating. This step involves nodewise scalarization, applying an MLP, and performing tensorization. Suppose the input and output channel numbers for equivariant tensor features are both  $d$ , then the scalarization using the node frame (similar to step 2) has a complexity of  $\mathcal{O}(dn)$  computations. Similarly, tensorization, which is the inverse operation of scalarization, also has a complexity of  $\mathcal{O}(dn)$ . Therefore, the total complexity for this step is  $\mathcal{O}((l + d)n)$ .
In conclusion, we have analyzed the three steps required for performing one layer of LEFTNet. It is worth noting that the efficiency of LEFTNet is comparable to other 1-hop based GNN algorithms like SchNet [28] ( $c = 1$  in Eq. 23). The final complexity is  $\mathcal{O}(ldn + (d + m + l)nk)$ , which is summarized in Table 4.
H Neural sheaf interpretation
Since our invariant scalarization, utilized in the LSE module, shares similarities with the scalarization technique on vector sheaves (see Hsu [52]), it is natural to explore a potential interpretation of LEFTNET within the framework of neural sheaf diffusion proposed by Bodnar et al. [62]. In this section, we aim to modify the neural sheaf diffusion architecture to preserve  $SE(3)$  equivariance by incorporating our node and edge frames.
We begin by revisiting the concept of a neural sheaf and its associated sheaf Laplacian operator. A cellular sheaf over a discrete graph is a mathematical object that assigns a vector space to each node and edge in the graph and specifies a linear map between these spaces for each incident node-edge pair:
Definition H.1. A cellular sheaf  $(G,\mathcal{F})$  on a graph  $G = (V,E)$  consists of:
- A vector space  $\mathcal{F}(v)$  for each  $v\in V$
- A vector space  $\mathcal{F}(e)$  for each  $e \in E$ .
- A linear map  $\mathcal{F}_{v\leq e}:\mathcal{F}(v)\to \mathcal{F}(e)$  for each incident  $v\triangleleft e$  node-edge pair.
In the case of a 3D graph neural network (GNN), the node-wise features typically consist of tensors (as defined in 10), which naturally form vector spaces. The key aspect of defining a sheaf lies in determining the specific assignments of the linear maps  $\mathcal{F}_{v\leq e}$  for each node  $v$  and each edge  $e$ . For non-geometric GNNs,  $\mathcal{F}_{v\leq e}$  captures how the opinions (represented as vectors in  $\mathcal{F}(v)$ ) manifest in a "discourse space" formed by  $\mathcal{F}(e)$ . In our approach, we equip each node with an equivariant frame denoted by  $F_{v}$  and assign an equivariant frame to each edge denoted by  $F_{e}$ . By definition,  $F_{v}$  and  $F_{e}$  capture the local geometry around the node and the edge, respectively. Thus, a natural choice for  $\mathcal{F}_{v\leq e}$  is given by:
$$
\mathcal {F} _ {v \leq e} := F _ {v} ^ {T} F _ {e} \in \mathcal {O} (3).
$$
To ensure  $SE(3)$  equivariance in the sheaf structure, we define  $\mathcal{F}(v)$  as the set of scalarized vectors obtained by applying scalarization using  $F_{v}$ . It is important to note that the linear map  $\mathcal{F}_{v\triangleleft e}$  between vectors can be extended to a linear map between higher-order tensors through tensor product operations (see Section F). This construction guarantees the entire structure to be  $SE(3)$ -invariant. With a well-defined sheaf structure, we can introduce the sheaf Laplacian operator that defines a
transformation between node features: