SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Here some basic notions from the graph exterior calculus are recalled. More details can be found in, e.g., [11, 51, 64].
Figure 3: A toy graph with six 0-cliques (nodes), six 1-cliques (edges), and one 2-clique.
As mentioned in Section 3, an oriented graph  $\mathcal{G} = \{\mathcal{V},\mathcal{E}\}$  carries sets of  $k$ -cliques, denoted  $\mathcal{G}_k$ , which are collections of ordered subgraphs generated by  $(k + 1)$  nodes. For example, the graph in Figure 3 contains six 0-cliques (nodes), six 1-cliques (edges), and one 2-clique. A notion of combinatorial derivative is then given by the signed incidence matrices  $d_k:\Omega_k\to \Omega_{k + 1}$ , operating on the space  $\Omega_k$  of differentiable functions on  $k$ -cliques, whose entries  $(d_k)_{ij}$  are 1 or -1 if the  $j^{\mathrm{th}}$ $k$ -clique is
incident on the  $i^{\mathrm{th}}(k + 1)$ -clique, and zero otherwise. For the example in Figure 3, these are:
$$
d _ {0} = \left( \begin{array}{c c c c c c} - 1 & 1 & 0 & 0 & 0 & 0 \\ 0 & - 1 & 1 & 0 & 0 & 0 \\ 0 & - 1 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & - 1 & 1 & 0 \\ 0 & 1 & 0 & 0 & - 1 & 0 \\ 0 & 0 & 0 & 0 & - 1 & 1 \end{array} \right), \qquad d _ {1} = (0 \quad 0 \quad 1 \quad 1 \quad 1 \quad 0).
$$
Remark A.1. While the one-hop neighborhood of node  $i$  in  $\mathcal{G}$ , denoted  $\mathcal{N}(i)$ , does not include node  $i$  itself, many machine learning algorithms employ the extended neighborhood  $\overline{\mathcal{N}}(i) = \mathcal{N}(i) \cup \{i\}$ . Since this is equivalent to considering the one-hop neighborhood of node  $i$  in the self-looped graph  $\overline{\mathcal{G}}$ , this modification does not change the analysis of functions on graphs.
It can be shown that the action of these matrices can be conveniently expressed in terms of totally antisymmetric functions  $f \in \Omega_k$ , via the expression
$$
\left(d _ {k} f\right) \left(i _ {0}, i _ {1}, \dots , i _ {k + 1}\right) = \sum_ {j = 0} ^ {k + 1} (- 1) ^ {j} f \left(i _ {0}, \dots , \widehat {i _ {j}}, \dots , i _ {k + 1}\right),
$$
where  $(i_0,\dots,i_{k + 1})$  denotes a  $(k + 1)$ -clique of vertices  $v\in \mathcal{V}$ . As convenient shorthand, we often write subscripts, e.g.,  $(d_kf)_{i_0i_1\dots i_{k + 1}}$ , instead of explicit function arguments. Using  $[S]$  to denote the indicator function of the statement  $S$ , it is straightforward to check that  $d\circ d = 0$ ,
$$
\begin{array}{l} (d _ {k} d _ {k - 1} f) _ {i _ {0}, \ldots , i _ {k + 1}} = \sum_ {j = 0} ^ {k + 1} (- 1) ^ {j} (d _ {k - 1} f) _ {i _ {0}, \ldots , \widehat {i _ {j}}}, \ldots , i _ {k + 1} \\ = \sum_ {j = 0} ^ {k + 1} \sum_ {l = 0} ^ {k + 1} [ l <   j ] (- 1) ^ {j + l} f _ {i _ {0} \dots \widehat {i _ {l}} \dots \widehat {i _ {j}} \dots i _ {k + 1}} \\ + \sum_ {j = 0} ^ {k + 1} \sum_ {l = 0} ^ {k + 1} [ l > j ] (- 1) ^ {j + l - 1} f _ {i _ {0} \dots \widehat {i _ {j}} \dots \widehat {i _ {l}} \dots i _ {k + 1}} \\ = \sum_ {l <   j} (- 1) ^ {j + l} f _ {i _ {0} \dots \widehat {i _ {l}} \dots \widehat {i _ {j}} \dots i _ {k + 1}} \\ - \sum_ {l <   j} (- 1) ^ {j + l} f _ {i _ {0} \dots \widehat {i _ {l}} \dots \widehat {i _ {j}} \dots i _ {k + 1}} = 0, \\ \end{array}
$$
since  $(-1)^{j + l - 1} = (-1)^{-1}(-1)^{j + l} = (-1)(-1)^{j + l}$  and the final sum follows from swapping the labels  $j, l$ . This shows that the  $k$ -cliques on  $\mathcal{G}$  form a de Rham complex [53]: a collection of function spaces  $\Omega_k$  equipped with mappings  $d_k$  satisfying  $\operatorname{Im} d_{k - 1} \subset \operatorname{Ker} d_k$  as shown in Figure 4. When
$$
\Omega_ {0} \xrightarrow {d _ {0}} \Omega_ {1} \xrightarrow {d _ {1}} \Omega_ {2} \xrightarrow {d _ {2}} \dots \xrightarrow {d _ {K - 1}} \Omega_ {K}
$$
Figure 4: Illustration of the de Rham complex on  $\mathcal{G}$  induced by the combinatorial derivatives, where  $K > 0$  is the maximal clique degree.
$K = 3$ , this is precisely the graph calculus analogue of the de Rham complex on  $\mathbb{R}^3$  formed by the Sobolev spaces  $H^{1}$ ,  $H(\mathrm{curl})$ ,  $H(\mathrm{div})$ ,  $L^2$  which satisfies  $\mathrm{div} \circ \mathrm{curl} = \mathrm{curl} \circ \mathrm{grad} = 0$ .
While the construction of the graph derivatives and their associated de Rham complex is purely topological, building elliptic differential operators such as the Laplacian relies on a dual de Rham complex, which is specified by an inner product on  $\Omega_{k}$ . In the case of  $\ell^2$ , this leads to dual derivatives which are the matrix transposes of the  $d_{k}$  having the following explicit expression.
Proposition A.1. The dual derivatives  $d_k^\top : \Omega_{k+1} \to \Omega_k$  adjoint to  $d_k$  through the  $\ell^2$  inner product are given by
$$
\left(d _ {k} ^ {\mathsf {T}} f\right) \left(i _ {0}, i _ {1}, \dots , i _ {k}\right) = \frac {1}{k + 2} \sum_ {i _ {k + 1}} \sum_ {j = 0} ^ {k + 1} f \left(i _ {0}, \dots , [ i _ {j}, \dots , i _ {k + 1} ]\right),
$$
where  $[i_j, \dots, i_{k+1}] = i_{k+1}, i_j, \dots, i_k$  indicates a cyclic permutation forward by one index.
Proof. This is a direct calculation using the representation of  $d_k$  in terms of antisymmetric functions. More precisely, let an empty sum  $\Sigma$  denote summation over all unspecified indices. Then, for any  $g \in \Omega_k$ ,
$$
\begin{array}{l} \langle d _ {k} f, g \rangle = \sum_ {i _ {0} \dots i _ {k + 1} \in \mathcal {G} _ {k + 1}} (d _ {k} f) _ {i _ {0} \dots i _ {k + 1}} g _ {i _ {0} \dots , i _ {k + 1}} \\ = \frac {1}{(k + 2) !} \sum \left(\sum_ {j = 0} ^ {k + 1} (- 1) ^ {j} f _ {i _ {0} \dots i _ {j} \dots i _ {k + 1}}\right) g _ {i _ {0} \dots i _ {k + 1}} \\ = \frac {1}{(k + 2) !} \sum f _ {i _ {0} \dots i _ {k}} \left(\sum_ {i _ {k + 1}} \sum_ {j = 0} ^ {k + 1} (- 1) ^ {j} g _ {i _ {0} \dots [ i _ {j} \dots i _ {k + 1} ]}\right) \\ = \frac {1}{k + 2} \sum_ {i _ {0} i _ {1} \dots i _ {k} \in \mathcal {G} _ {k}} f _ {i _ {0} \dots i _ {k}} \left(\sum_ {i _ {k + 1}} \sum_ {j = 0} ^ {k + 1} (- 1) ^ {j} g _ {i _ {0} \dots [ i _ {j} \dots i _ {k + 1} ]}\right) \\ = \sum_ {i _ {0} i _ {1} \dots i _ {k} \in \mathcal {G} _ {k}} f _ {i _ {0} \dots i _ {k}} \left(d _ {k} ^ {\intercal} g\right) _ {i _ {0} \dots i _ {k}} = \langle f, d _ {k} ^ {\intercal} g \rangle , \\ \end{array}
$$
which establishes the result.
Proposition A.1 is perhaps best illustrated with a concrete example. Consider the graph gradient, defined for edge  $\alpha = (i,j)$  as  $(d_0f)_\alpha = (d_0f)_{ij} = f_j - f_i$ . Notice that this object is antisymmetric with respect to edge orientation, and measures the outflow of information from source to target nodes. From this, it is easy to compute the  $\ell^2$ -adjoint of  $d_0$ , known as the graph divergence, via
$$
\begin{array}{l} \langle d _ {0} f, g \rangle = \sum_ {\alpha = (i, j)} \left(f _ {j} - f _ {i}\right) g _ {i j} = \sum_ {i} \sum_ {(j > i) \in \mathcal {N} (i)} g _ {i j} f _ {j} - g _ {i j} f _ {i} \\ = \frac {1}{2} \sum_ {i} \sum_ {j \in \mathcal {N} (i)} f _ {i} \left(g _ {j i} - g _ {i j}\right) = \langle f, d _ {0} ^ {\intercal} g \rangle , \\ \end{array}
$$
where we have re-indexed under the double sum, used that  $i \in \mathcal{N}(j)$  if and only if  $j \in \mathcal{N}(i)$ , and used that there are no self-edges in  $\mathcal{E}$ . Therefore, it follows that the graph divergence at node  $i$  is given by
$$
\left(d _ {0} ^ {\intercal} g\right) _ {i} = \sum_ {\alpha \ni i} g _ {- \alpha} - g _ {\alpha} = \frac {1}{2} \sum_ {j \in \mathcal {N} (i)} g _ {j i} - g _ {i j},
$$
which reduces to the common form  $(d_0^{\mathrm{T}}g)_i = -\sum_j g_{ij}$  if and only if the edge feature  $g_{ij}$  is antisymmetric.
Remark A.2. When the inner product on edges  $\mathcal{E}$  is not  $L^2$ , but defined in terms of a nonnegative, orientation-invariant, and (edge-wise) diagonal weight matrix  $\mathbf{W} = (w_{ij})$ , a similar computation shows that the divergence becomes
$$
\left(d _ {0} ^ {*} f\right) _ {i} = \frac {1}{2} \sum_ {j \in \mathcal {N} (i)} w _ {i j} \left(f _ {j i} - f _ {i j}\right).
$$
The more general case of arbitrary inner products on  $\mathcal{V},\mathcal{E}$  is discussed in section A.3.
The differential operators  $d_k^\top$  induce a dual de Rham complex since  $d_{k - 1}^{\top}d_{k}^{\top} = (d_{k}d_{k - 1})^{\top} = 0$ , which enables both the construction of Laplace operators on  $k$ -cliques,  $\Delta_k = d_k^\top d_k + d_{k - 1}d_{k - 1}^\top$ , as well as the celebrated Hodge decomposition theorem, stated below. For a proof, see, e.g., [11, Theorem 3.3].
Theorem A.3. (Hodge Decomposition Theorem) The de Rham complexes formed by  $d_k, d_k^\top$  induce the following direct sum decomposition of the function space  $\Omega_k$ ,
$$
\Omega_ {k} = \operatorname {I m} d _ {k - 1} \oplus \operatorname {K e r} \Delta_ {k} \oplus \operatorname {I m} d _ {k} ^ {\intercal}.
$$
In the case where the dual derivatives  $d_k^*$  are adjoint with respect to a learnable inner product which does not depend on graph features, the conclusion of Theorem A.3 continues to hold, leading to an interesting well-posedness result proved in [11] involving nonlinear perturbations of a Hodge-Laplace problem in mixed form.
Theorem A.4. ([11, Theorem 3.6]) Suppose  $\mathbf{f}_k \in \Omega_k$ , and  $g(\mathbf{x}; \xi)$  is a neural network with parameters  $\xi$  which is Lipschitz continuous and satisfies  $g(\mathbf{0}) = \mathbf{0}$ . Then, the problem
$$
\mathbf {w} _ {k - 1} = d _ {k - 1} ^ {*} \mathbf {u} _ {k} + \epsilon g \left(d _ {k - 1} ^ {*} \mathbf {u} _ {k}; \xi\right),
$$
$$
\mathbf {f} _ {k} = d _ {k - 1} \mathbf {w} _ {k - 1} + d _ {k} ^ {*} d _ {k} \mathbf {u} _ {k},
$$
has a unique solution on  $\Omega_k / \mathrm{Ker}\Delta_k$
This result shows that initial-value problems involving the Hodge-Laplacian are stable under nonlinear perturbations. Moreover, when  $\Delta_0$  is the Hodge Laplacian on nodes, there is a useful connection between  $\Delta_0$  and the degree and adjacency matrices of the graph  $\mathcal{G}$ . Recall that the degree matrix  $\mathbf{D} = (d_{ij})$  is diagonal with entries  $d_{ii} = \sum_{j\in \mathcal{N}(i)}1$ , while the adjacency matrix  $\mathbf{A} = (a_{ij})$  satisfies  $a_{ij} = 1$  when  $j\in \mathcal{N}(i)$  and  $a_{ij} = 0$  otherwise.
Proposition A.2. The combinatorial Laplacian on  $\mathcal{V}$ , denoted  $\Delta_0 = d_0^\top d_0$ , satisfies  $\Delta_0 = \mathbf{D} - \mathbf{A}$ .
$$
\begin{array}{l} \left(d _ {0} ^ {\intercal} d _ {0}\right) _ {i j} = \sum_ {\alpha \in \mathcal {E}} \left(d _ {0}\right) _ {\alpha i} \left(d _ {0}\right) _ {\alpha j} = [ i = j ] \sum_ {\alpha \in \mathcal {E}} \left(\left(d _ {0}\right) _ {\alpha i}\right) ^ {2} + [ i \neq j ] \sum_ {\alpha = (i, j)} \left(d _ {0}\right) _ {\alpha i} \left(d _ {0}\right) _ {\alpha j} \\ = [ i = j ] d _ {i i} - [ i \neq j ] a _ {i j} = d _ {i j} - a _ {i j} = \mathbf {D} - \mathbf {A}, \\ \end{array}
$$
where we used that  $\mathbf{D}$  is diagonal,  $\mathbf{A}$  is diagonal-free, and  $(d_0)_{\alpha i}(d_0)_{\alpha j} = -1$  whenever  $\alpha = (i,j)$  is an edge in  $\mathcal{E}$ , since one of  $(d_0)_{\alpha i}, (d_0)_{\alpha j}$  is 1 and the other is -1.
A.2 Bracket-based dynamical systems
Here we mention some additional facts regarding bracket-based dynamical systems. More information can be found in, e.g., [50, 65, 66, 67].
As mentioned before, the goal of bracket formalisms is to extend the Hamiltonian formalism to systems with dissipation. To understand where this originates, consider an action functional  $\mathcal{A}(q) = \int_{a}^{b}L\left(q,\dot{q}\right)dt$  on the space of curves  $q(t)$ , defined in terms of a Lagrangian  $L$  on the tangent bundle to some Riemannian manifold. Using  $L_{q}, L_{\dot{q}}$  to denote partial derivatives with respect to the subscripted variable, it is straightforward to show that, for any compactly supported variation  $\delta q$  of  $q$ , we have
$$
d \mathcal {A} (q) \delta q = \int_ {a} ^ {b} d L (q, \dot {q}) \delta q = \int_ {a} ^ {b} L _ {q} \delta q + L _ {\dot {q}} \delta \dot {q} = \int_ {a} ^ {b} \left(L _ {q} - \partial_ {t} L _ {\dot {q}}\right) \delta q,
$$
where the final equality follows from integration-by-parts and the fact that variational and temporal derivatives commute in this setting. It follows that  $\mathcal{A}$  is stationary (i.e.,  $d\mathcal{A} = 0$ ) for all variations only when  $\partial_t L_{\dot{q}} = L_q$ . These are the classical Euler-Lagrange equations which are (under some regularity conditions) transformed to Hamiltonian form via a Legendre transformation,
$$
H (q, p) = \sup  _ {\dot {q}} \left(\langle p, \dot {q} \rangle - L (q, \dot {q})\right),
$$
which defines the Hamiltonian functional  $\mathcal{H}$  on phase space, and yields the conjugate momentum vector  $p = L_{\dot{q}}$ . Substituting  $L = \langle p,\dot{q}\rangle -H$  into the previously derived Euler-Lagrange equations leads immediately to Hamilton's equations for the state  $\mathbf{x} = (q\quad p)^{\intercal}$ ,
$$
\dot {\mathbf {x}} = \left( \begin{array}{c} \dot {q} \\ \dot {p} \end{array} \right) = \left( \begin{array}{c c} 0 & 1 \\ - 1 & 0 \end{array} \right) \left( \begin{array}{c} H _ {q} \\ H _ {p} \end{array} \right) = \mathbf {J} \nabla \mathcal {H},
$$
which are an equivalent description of the system in question in terms of the anti-involution  $\mathbf{J}$  and the functional gradient  $\nabla \mathcal{H}$ .
An advantage of the Hamiltonian description is its compact bracket-based formulation,  $\dot{\mathbf{x}} = \mathbf{J}\nabla \mathcal{H} = \{\mathbf{x},\mathcal{H}\}$ , which requires only the specification of an antisymmetric Poisson bracket  $\{\cdot ,\cdot \}$  and a Hamiltonian functional  $\mathcal{H}$ . Besides admitting a direct generalization to more complex systems such as Korteweg-de Vries or incompressible Euler, where the involved bracket is state-dependent, this formulation makes the energy conservation property of the system obvious. In particular, it follows immediately from the antisymmetry of  $\{\cdot ,\cdot \}$  that
$$
\dot {\mathcal {H}} = \langle \dot {\mathbf {x}}, \nabla \mathcal {H} \rangle = \{\mathcal {H}, \mathcal {H} \} = 0,
$$
while it is more difficult to see immediately that the Euler-Lagrange system obeys this same property. The utility and ease-of-use of bracket formulations is what inspired their extension to other systems of interest which do not conserve energy. On the opposite end of this spectrum are the generalized gradient flows, which can be written in terms of a bracket which is purely dissipative. An example of this is heat flow  $\dot{q} = \Delta q \coloneqq -[q,\mathcal{D}]$ , which is the  $L^2$ -gradient flow of Dirichlet energy  $\mathcal{D}(q) = (1/2)\int_{a}^{b}|q'|^{2}dt$  (c.f. Appendix A.3). In this case, the functional gradient  $\nabla \mathcal{D} = -\partial_{tt}$  is the negative of the usual Laplace operator, so that the positive-definite bracket  $[\cdot,\cdot]$  is generated by the identity operator  $M = \mathrm{id}$ . It is interesting to note that the same system could be expressed using the usual kinetic energy  $\mathcal{E}(q) = (1/2)\int_{a}^{b}|q|^{2}dt$  instead, provided that the corresponding bracket is generated by  $M = -\Delta$ . This is a good illustration of the flexibility afforded by bracket-based dynamical systems.
Since physical systems are not always purely reversible or irreversible, other useful bracket formalisms have been introduced to capture dynamics which are a mix of these two. The double bracket  $\dot{\mathbf{x}} = \{\mathbf{x}, E\} + \{\{\mathbf{x}, E\}\} = \mathbf{L} \nabla E + \mathbf{L}^2 \nabla E$  is a nice extension of the Hamiltonian bracket particularly because it is Casimir preserving, i.e., those quantities which annihilate the Poisson bracket  $\{\cdot, \cdot\}$  also annihilate the double bracket. This allows for the incorporation of dissipative phenomena into idealized Hamiltonian systems without affecting desirable properties such as mass conservation, and has been used to model, e.g., the Landau-Lifschitz dissipative mechanism, as well as a mechanism for fluids where energy decays but entropy is preserved (see [65] for additional discussion). A complementary but alternative point of view is taken by the metriplectic bracket formalism, which requires that any dissipation generated by the system is accounted for within the system itself through the generation of entropy. In the metriplectic formalism, the equations of motion are  $\dot{\mathbf{x}} = \{\mathbf{x}, E\} + [\mathbf{x}, S] = \mathbf{L} \nabla E + \mathbf{M} \nabla S$ , along with important and nontrivial compatibility conditions  $\mathbf{L} \nabla S = \mathbf{M} \nabla E = \mathbf{0}$ , also called degeneracy conditions, which ensure that the reversible and irreversible mechanisms do not cross-contaminate. As shown in the body of the paper, this guarantees that metriplectic systems obey a form of the first and second thermodynamical laws. Practically, the degeneracy conditions enforce a good deal of structure on the operators  $\mathbf{L}, \mathbf{M}$  which has been exploited to generate surrogate models [29, 68, 31]. In particular, it can be shown that the reversible and irreversible brackets can be parameterized in terms of a totally antisymmetric order-3 tensor  $\xi = (\xi_{ijk})$  and a partially symmetric order-4 tensor  $\zeta = (\zeta_{ik,jl})$  through the relations (Einstein summation assumed)
$$
\{A, B \} = \xi^ {i j k} \partial_ {i} A \partial_ {j} B \partial_ {k} S,
$$
$$
[ A, B ] = \zeta^ {i k, j l} \partial_ {i} A \partial_ {k} E \partial_ {j} B \partial_ {l} E.
$$
Moreover, using the symmetries of  $\zeta$ , it follows (see [67]) that this tensor decomposes into the product  $\zeta_{ik,jl} = \Lambda_{ik}^{m}D_{mn}\Lambda_{jl}^{n}$  of a symmetric matrix  $\mathbf{D}$  and an order-3 tensor  $\Lambda$  which is skew-symmetric in its lower indices. Thus, by applying symmetry relationships, it is easy to check that  $\{\cdot ,S\} = [\cdot ,E] = \mathbf{0}$ .
Remark A.5. In [29], trainable 4- and 3- tensors  $\xi^{ijk}$  and  $\zeta^{ik,jl}$  are constructed to achieve the degeneracy conditions, mandating a costly  $O(N^3)$  computational complexity. In the current work we overcome this by instead achieving degeneracy through the exact sequence property.
A.3 Adjoints and gradients
Beyond the basic calculus operations discussed in section A.1 which depend only on graph topology, the network architectures discussed in the body also make extensive use of learnable metric information coming from the nodal features. To understand this, it is useful to recall some information about general inner products and the derivative operators that they induce. First, recall that the usual  $\ell^2$  inner product on node features  $\mathbf{a},\mathbf{b}\in \mathbb{R}^{|\mathcal{V}|}$ ,  $\langle \mathbf{a},\mathbf{b}\rangle = \mathbf{a}^{\intercal}\mathbf{b}$ , is (in this context) a discretization of the standard  $L^2$  inner product  $\int_{\mathcal{V}}abd\mu$  which aggregates information from across the vertex set  $\mathcal{V}$ . While this construction is clearly dependent only on the graph structure (i.e., topology), any symmetric positive definite (SPD) matrix  $\mathbf{A}_0:\Omega_0\to \Omega_0$  also defines an inner product on functions  $\mathbf{a}\in \Omega_0$  through the equality
$$
(\mathbf {a}, \mathbf {b}) _ {0} := \langle \mathbf {a}, \mathbf {A} _ {0} \mathbf {b} \rangle = \mathbf {a} ^ {\intercal} \mathbf {A} _ {0} \mathbf {b},
$$
which gives a different way of measuring the distance between  $\mathbf{a}$  and  $\mathbf{b}$ . The advantage of this construction is that  $\mathbf{A}_0$  can be chosen in a way that incorporates geometric information which implicitly regularizes systems obeying a variational principle. This follows from the following
intuitive fact: the Taylor series of a function does not change, regardless of the inner product on its domain. For any differentiable function(al)  $E:\Omega_0\to \mathbb{R}$ , using  $d$  to denote the exterior derivative, this means that the following equality holds
$$
d E (\mathbf {a}) \mathbf {b} := \lim  _ {\varepsilon \rightarrow 0} \frac {E (\mathbf {a} + \epsilon \mathbf {b}) - E (\mathbf {a})}{\varepsilon} = \langle \delta E (\mathbf {a}), \mathbf {b} \rangle = (\nabla E (\mathbf {a}), \mathbf {b}) _ {0},
$$
where  $\delta E$  denotes the  $\ell^2$ -gradient of  $E$  and  $\nabla E$  denotes its  $\mathbf{A}_0$ -gradient, i.e., its gradient with respect to the derivative operator induced by the inner product involving  $\mathbf{A}_0$ . From this, it is clear that  $\delta E = \mathbf{A}_0\nabla E$ , so that the  $\mathbf{A}_0$ -gradient is just an anisotropic rescaling of the  $\ell^2$  version. The advantage of working with  $\nabla$  over  $\delta$  in the present case of graph networks is that  $\mathbf{A}_0$  can be learned based on the features of the graph. This means that learnable feature information (i.e., graph attention) can be directly incorporated into the differential operators governing our bracket-based dynamical systems by construction.
The prototypical example of where this technique is useful is seen in the gradient flow of Dirichlet energy. Recall that the Dirichlet energy of a differentiable function  $u: \mathbb{R}^n \to \mathbb{R}$  is given by  $\mathcal{D}(u) = (1/2)\int |\nabla u|^2 d\mu$ , where  $\nabla$  now denotes the usual  $\ell^2$ -gradient of the function  $u$  on  $\mathbb{R}^n$ . Using integration-by-parts, it is easy to see that  $d\mathcal{D}(u)v = -\int v\Delta u$  for any test function  $v$  with compact support, implying that the  $L^2$ -gradient of  $\mathcal{D}$  is  $-\Delta$  and  $\dot{u} = \Delta u$  is the  $L^2$ -gradient flow of Dirichlet energy: the motion which decreases the quantity  $\mathcal{D}(u)$  the fastest as measured by the  $L^2$  norm. It can be shown that high-frequency modes decay quickly under this flow, while low-frequency information takes much longer to dissipate. On the other hand, we could alternatively run the  $H^1$ -gradient flow of  $\mathcal{D}$ , which is motion of fastest decrease with respect to the  $H^1$  inner product  $(u,v) = \int \langle \nabla u,\nabla v\rangle d\mu$ . This motion is prescribed in terms of the  $H^1$ -gradient of  $\mathcal{D}$ , which by the discussion above with  $\mathbf{A}_0 = -\Delta$  is easily seen to be the identity. This means that the  $H^1$ -gradient flow is given by  $\dot{u} = -u$ , which retains the minimizers of the  $L^2$ -flow but with quite different intermediate character, since it functions by simultaneously flattening all spatial frequencies. The process of preconditioning a gradient flow by matching derivatives is known as a Sobolev gradient method (c.f. [69]), and these methods often exhibit faster convergence and better numerical behavior than their  $L^2$  counterparts [70].
Returning to the graph setting, our learnable matrices  $\mathbf{A}_k$  on  $k$ -cliques will lead to inner products  $(\cdot ,\cdot)_{k}$  on functions in  $\Omega_k$ , and this will induce dual derivatives as described in Appendix A.1. However, in this case we will not have  $d_0^* = d_0^\top$ , but instead the expression given by the following result:
Proposition A.3. The  $\mathbf{A}_k$ -adjoints  $d_k^*$  to the graph derivative operators  $d_k$  are given by  $d_k^* = \mathbf{A}_k^{-1} d_k^\top \mathbf{A}_{k+1}$ . Similarly, for any linear operator  $\mathbf{B}: \Omega_k \to \Omega_k$ , the  $\mathbf{A}_k$ -adjoint  $\mathbf{B}^* = \mathbf{A}_k^{-1} \mathbf{B}^\top \mathbf{A}$ .
Proof. Let  $\mathbf{q},\mathbf{p}$  denote vectors of  $k$  -clique resp.  $(k + 1)$  -clique features. It follows that
$$
\left(d _ {k} \mathbf {q}, \mathbf {p}\right) _ {k + 1} = \left\langle d _ {k} \mathbf {q}, \mathbf {A} _ {k + 1} \mathbf {p} \right\rangle = \left\langle \mathbf {q}, d _ {k} ^ {\intercal} \mathbf {A} _ {k + 1} \mathbf {p} \right\rangle = \left\langle \mathbf {q}, \mathbf {A} _ {k} d _ {k} ^ {*} \mathbf {p} \right\rangle = \left(\mathbf {q}, d _ {k} ^ {*} \mathbf {p}\right) _ {k}.
$$