SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Sohir Maskey*
Department of Mathematics,
LMU Munich
maskey@math.lmu.de
Aras Bacho
Raffaele Paolino*
Department of Mathematics & MCML,
paolino@math.lmu.de
Gitta Kutyniok
Graph neural networks (GNNs) have shown state-of-the-art performances in various applications. However, GNNs often struggle to capture long-range dependencies in graphs due to oversmoothing. In this paper, we generalize the concept of oversmoothing from undirected to directed graphs. To this aim, we extend the notion of Dirichlet energy by considering a directed symmetrically normalized Laplacian. As vanilla graph convolutional networks are prone to oversmooth, we adopt a neural graph ODE framework. Specifically, we propose fractional graph Laplacian neural ODEs, which describe non-local dynamics. We prove that our approach allows propagating information between distant nodes while maintaining a low probability of long-distance jumps. Moreover, we show that our method is more flexible with respect to the convergence of the graph's Dirichlet energy, thereby mitigating oversmoothing. We conduct extensive experiments on synthetic and real-world graphs, both directed and undirected, demonstrating our method's versatility across diverse graph homophily levels. Our code is available on GitHub.
Graph neural networks (GNNs) (Gori et al., 2005; Scarselli et al., 2009; Bronstein et al., 2017) have emerged as a powerful class of machine learning models capable of effectively learning representations of structured data. GNNs have demonstrated state-of-the-art performance in a wide range of applications, including social network analysis (Monti et al., 2019), molecular property prediction (Gilmer et al., 2017), and recommendation systems (J. Wang et al., 2018; Fan et al., 2019). The majority of existing work on GNNs has focused on undirected graphs (Defferrard et al., 2016; Kipf et al., 2017; Hamilton et al., 2017), where edges have no inherent direction. However, many real-world systems, such as citation networks, transportation systems, and biological pathways, are inherently directed, necessitating the development of methods explicitly tailored to directed graphs.
Despite their success, most existing GNN models struggle to capture long-range dependencies, which can be critical for specific tasks, such as node classification and link prediction, and for specific graphs, such as heterophilic graphs. This shortcoming also arises from the problem of oversmoothing, where increasing the depth of GNNs results in the node features converging to similar values that only convey information about the node's degree (Oono et al., 2019; Cai et al., 2020). Consequently,
scaling the depth of GNNs is not sufficient to broaden receptive fields, and other approaches are necessary to address this limitation. While these issues have been extensively studied in undirected graphs (Q. Li et al., 2018; G. Li et al., 2019; Luan, M. Zhao, et al., 2019; D. Chen et al., 2020; Rusch et al., 2022), their implications for directed graphs remain largely unexplored. Investigating these challenges and developing effective solutions is crucial for applying GNNs to real-world scenarios.
Over-smoothing has been shown to be intimately related to the graph's Dirichlet energy, defined as
$$
\mathcal {E} (\mathbf {x}) := \frac {1}{4} \sum_ {i, j = 1} ^ {N} a _ {i, j} \left\| \frac {\mathbf {x} _ {i}}{\sqrt {d _ {i}}} - \frac {\mathbf {x} _ {j}}{\sqrt {d _ {j}}} \right\| _ {2} ^ {2},
$$
where  $\mathbf{A} = (a_{i,j})_{i,j=1}^{N}$  represents the adjacency matrix of the underlying graph,  $\mathbf{x} \in \mathbb{R}^{N \times K}$  denotes the node features, and  $d_i \in \mathbb{R}$  the degree of node  $i$ . Intuitively, the Dirichlet energy measures the smoothness of nodes' features. Therefore, a GNN that minimizes the Dirichlet energy is expected to perform well on homophilic graphs, where similar nodes are likely to be connected. Conversely, a GNN that ensures high Dirichlet energy should lead to better performances on heterophilic graphs, for which the nodes' features are less smooth.
This paper aims to bridge the gap in understanding oversmoothing for directed graphs. To this aim, we generalize the concept of Dirichlet energy, providing a rigorous foundation for analyzing oversmoothing. Specifically, we consider the directed symmetrically normalized Laplacian, which accommodates directed graph structures and recovers the usual definition in the undirected case. Even though the directed symmetrically normalized Laplacian has been already used (Zou et al., 2022), its theoretical properties remain widely unexplored.
However, a vanilla graph convolutional network (GCN) (Kipf et al., 2017) implementing this directed Laplacian alone is not able to prevent oversmoothing. For this reason, we adopt a graph neural ODE framework, which has been shown to effectively alleviate oversmoothing in undirected graphs (Bodnar et al., 2022; Rusch et al., 2022; Di Giovanni et al., 2023).
1.1 Graph Neural ODEs
The concept of neural ODE was introduced by Haber et al. (2018) and R. T. Q. Chen et al. (2018), who first interpreted the layers in neural networks as the time variable in ODEs. Building on this foundation, Poli et al. (2021), Chamberlain et al. (2021), and Eliasof et al. (2021) extended the connection to the realm of GNNs, resulting in the development of graph neural ODEs. In this context, each node  $i$  of the underlying graph is described by a state variable  $\mathbf{x}_i(t) \in \mathbb{R}^K$ , representing the node  $i$  at time  $t$ . We can define the dynamics of  $\mathbf{x}(t)$  via the node-wise ODE
$$
\mathbf {x} ^ {\prime} (t) = f _ {\mathbf {w}} (\mathbf {x} (t)), t \in [ 0, T ],
$$
subject to the initial condition  $\mathbf{x}(0) = \mathbf{x}_0\in \mathbb{R}^{N\times K}$ , where the function  $f_{\mathbf{w}}:\mathbb{R}^{N\times K}\to \mathbb{R}^{N\times K}$  is parametrized by the learnable parameters  $\mathbf{w}$ .
The graph neural ODE can be seen as a continuous learnable architecture on the underlying graph, which computes the final node representation  $\mathbf{x}(T)$  from the input nodes' features  $\mathbf{x}_0$ . Typical choices for  $f_{\mathbf{w}}$  include attention-based functions (Chamberlain et al., 2021), which generalize graph attention networks (GATs) (Velickovic et al., 2018), or convolutional-like functions (Di Giovanni et al., 2023) that generalize GCNs (Kipf et al., 2017).
How can we choose the learnable function  $f_{\mathbf{w}}$  to accommodate both directed and undirected graphs, as well as different levels of homophily? We address this question in the following subsection.
1.2 Fractional Laplacians
The continuous fractional Laplacian, denoted by  $(- \Delta)^{\alpha}$  for  $\alpha > 0$ , is used to model non-local interactions. For instance, the fractional heat equation  $\partial_t u + (-\Delta)^{\alpha} u = 0$  provides a flexible and accurate framework for modeling anomalous diffusion processes. Similarly, the fractional diffusion-reaction, quasi-geostrophic, Cahn-Hilliard, porous medium, Schrödinger, and ultrasound equations are more sophisticated models to represent complex anomalous systems (Pozrikidis, 2018).
Similarly to the continuous case, the fractional graph Laplacian (FGL) (Benzi et al., 2020) models non-local network dynamics. In general, the FGL does not inherit the sparsity of the underlying
graph, allowing a random walker to leap rather than walk solely between adjacent nodes. Hence, the FGL is able to build long-range connections, making it well-suited for heterophilic graphs.
1.3 Main Contributions
We present a novel approach to the fractional graph Laplacian by defining it in the singular value domain, instead of the frequency domain (Benzi et al., 2020). This formulation bypasses the need for computing the Jordan decomposition of the graph Laplacian, which lacks reliable numerical methods. We show that our version of the FGL can still capture long-range dependencies, and we prove that its entries remain reasonably bounded.
We then propose two FGL-based neural ODEs: the fractional heat equation and the fractional Schrödinger equation. Importantly, we demonstrate that solutions to these FGL-based neural ODEs offer increased flexibility in terms of the convergence of the Dirichlet energy. Notably, the exponent of the fractional graph Laplacian becomes a learnable parameter, allowing our network to adaptively determine the optimal exponent for the given task and graph. We show that this can effectively alleviate oversmoothing in undirected and directed graphs.
To validate the effectiveness of our approach, we conduct extensive experiments on synthetic and real-world graphs, with a specific focus on supervised node classification. Our experimental results indicate the advantages offered by fractional graph Laplacians, particularly in non-homophilic and directed graphs.
We denote a graph as  $\mathcal{G} = (\mathcal{V},\mathcal{E})$ , where  $\mathcal{V}$  is the set of nodes,  $\mathcal{E}$  is the set of edges, and  $N = |\mathcal{V}|$  is the number of nodes. The adjacency matrix  $\mathbf{A} := \{a_{i,j}\}$  encodes the edge information, with  $a_{i,j} = 1$  if there is an edge directed from node  $j$  to  $i$ , and 0 otherwise. The in- and out-degree matrices are then defined as  $\mathbf{D}_{\mathrm{in}} = \mathrm{diag}(\mathbf{A}\mathbf{1})$ ,  $\mathbf{D}_{\mathrm{out}} = \mathrm{diag}(\mathbf{A}^{\mathrm{T}}\mathbf{1})$ , respectively. The node feature matrix  $\mathbf{x} \in \mathbb{R}^{N \times K}$  contains for every node its feature in  $\mathbb{R}^K$ .
Given any matrix  $\mathbf{M} \in \mathbb{C}^{n \times n}$ , we denote its spectrum by  $\lambda(\mathbf{M}) \coloneqq \{\lambda_i(\mathbf{M})\}_{i=1}^n$  in ascending order w.r.t. to the real part, i.e.,  $\Re \lambda_1(\mathbf{M}) \leq \Re \lambda_2(\mathbf{M}) \leq \ldots \leq \Re \lambda_n(\mathbf{M})$ . Furthermore, we denote by  $\|\mathbf{M}\|_2$  and  $\|\mathbf{M}\|$  the Frobenius and spectral norm of  $\mathbf{M}$ , respectively. Lastly, we denote by  $\mathbf{I}_n$  the identity matrix, where we omit the dimension  $n$  when it is clear from the context.
Homophily and Heterophily Given a graph  $\mathcal{G} = (\mathcal{V},\mathcal{E})$  with labels  $\mathbf{y} = \{y_i\}_{i\in \mathcal{V}}$ , the homophily of the graph indicates whether connected nodes are likely to have the same labels; formally,
$$
\mathcal {H} (\mathcal {G}) = \frac {1}{N} \sum_ {i = 1} ^ {N} \frac {| \{j \in \{1 , \ldots , N \} : a _ {i , j} = 1 \wedge y _ {i} = y _ {j} \} |}{| \{j \in \{1 , \ldots , N \} : a _ {i , j} = 1 \} |},
$$
where the numerator represents the number of neighbors of node  $i \in \mathcal{V}$  that have the same label  $y_{i}$  (Pei et al., 2019). We say that  $\mathcal{G}$  is homophilic if  $\mathcal{H}(\mathcal{G}) \approx 1$  and heterophilic if  $\mathcal{H}(\mathcal{G}) \approx 0$ .
3 Dirichlet Energy and Laplacian for (Directed) Graphs
In this section, we introduce the concept of Dirichlet energy and demonstrate its relationship to a directed Laplacian, thereby generalizing well-known results for undirected graphs.
Definition 3.1. The Dirichlet energy is defined on the node features  $\mathbf{x} \in \mathbb{R}^{N \times K}$  of a graph  $\mathcal{G}$  as
$$
\mathcal {E} (\mathbf {x}) := \frac {1}{4} \sum_ {i, j = 1} ^ {N} a _ {i, j} \left\| \frac {\mathbf {x} _ {i}}{\sqrt {d _ {i} ^ {i n}}} - \frac {\mathbf {x} _ {j}}{\sqrt {d _ {j} ^ {o u t}}} \right\| _ {2} ^ {2}. \tag {1}
$$
The Dirichlet energy measures how much the features change over the nodes of  $\mathcal{G}$ , by quantifying the disparity between the normalized outflow of information from node  $j$  and the normalized inflow of information to node  $i$ .
Definition 3.2. We define the symmetrically normalized adjacency (SNA) as  $\mathbf{L} \coloneqq \mathbf{D}_{in}^{-1/2} \mathbf{A} \mathbf{D}_{out}^{-1/2}$ .
Figure 1: Spectrum  $\lambda (\mathbf{L})$  of common directed real-world graphs. The Perron-Frobenius eigenvalue is  $\lambda_{\mathrm{PF}}\approx 0.94$  for Chameleon, and  $\lambda_{\mathrm{PF}}\approx 0.89$  for Squirrel.
Figure 2: Examples of non-weakly balanced (left), weakly balanced (center), and balanced (right) directed graphs. The Perron-Frobenius eigenvalue of the left graph is  $\lambda_{\mathrm{PF}} \approx 0.97 \neq 1$ , while for the middle and right graphs  $\lambda_{\mathrm{PF}} = 1$ .
Note that  $\mathbf{L}$  is symmetric if and only if  $\mathcal{G}$  is undirected; the term "symmetrically" refers to the both-sided normalization rather than the specific property of the matrix itself.
It is well-known that the SNA's spectrum of a connected undirected graph lies within  $[-1, 1]$  (Chung, 1997). We extend this result to directed graphs, which generally exhibit complex-valued spectra.
Proposition 3.3. Let  $\mathcal{G}$  be a directed graph with SNA  $\mathbf{L}$ . For every  $\lambda \in \lambda (\mathbf{L})$ , it holds  $|\lambda |\leq 1$
Proposition 3.3 provides an upper bound for the largest eigenvalue of any directed graph, irrespective of its size. However, many other spectral properties do not carry over easily from the undirected to the directed case. For example, the SNA may not possess a one-eigenvalue, even if the graph is strongly connected (see, e.g., Figures 1 to 2). The one-eigenvalue is of particular interest since its eigenvector  $\mathbf{v}$  corresponds to zero Dirichlet energy  $\mathcal{E}(\mathbf{v}) = 0$ . Therefore, studying when  $1 \in \lambda(\mathbf{L})$  is crucial to understanding the behavior of the Dirichlet energy. We fully characterize the set of graphs for which  $1 \in \lambda(\mathbf{L})$ ; this is the scope of the following definition.
Definition 3.4. A graph  $\mathcal{G} = (\mathcal{V},\mathcal{E})$  is said to be balanced if  $d_i^{in} = d_i^{out}$  for all  $i\in \{1,\ldots ,N\}$ , and weakly balanced if there exists  $\mathbf{k}\in \mathbb{R}^N$  such that  $\mathbf{k}\neq 0$  and
$$
\sum_ {j = 1} ^ {N} a _ {i, j} \left(\frac {k _ {j}}{\sqrt {d _ {j} ^ {o u t}}} - \frac {k _ {i}}{\sqrt {d _ {i} ^ {i n}}}\right) = 0, \forall i \in \{1, \dots , N \}.
$$
It is straightforward to see that a balanced graph is weakly balanced since one can choose  $k_{i} = \sqrt{d_{i}^{\mathrm{in}}}$ . Hence, all undirected graphs are also weakly balanced. However, as shown in Figure 2, the set of balanced graphs is a proper subset of the set of weakly balanced graphs.
Proposition 3.5. Let  $\mathcal{G}$  be a directed graph with SNA  $\mathbf{L}$ . Then,  $1 \in \lambda(\mathbf{L})$  if and only if the graph is weakly balanced. Suppose the graph is strongly connected, then  $-1 \in \lambda(\mathbf{L})$  if and only if the graph is weakly balanced with an even period.
Proposition 3.5 generalizes a well-known result for undirected graphs:  $-1\in \lambda (\mathbf{L})$  if and only if the graph is bipartite, i.e., has even period. The next result shows that the Dirichlet energy defined in (1) and the SNA are closely connected.
Proposition 3.6. For every  $\mathbf{x} \in \mathbb{C}^{N \times K}$ , it holds  $\mathcal{E}(\mathbf{x}) = \frac{1}{2}\Re\left(\operatorname{trace}\left(\mathbf{x}^{\mathsf{H}}(\mathbf{I} - \mathbf{L})\mathbf{x}\right)\right)$ . Moreover, there exists  $\mathbf{x} \neq \mathbf{0}$  such that  $\mathcal{E}(\mathbf{x}) = 0$  if and only if the graph is weakly balanced.
Proposition 3.6 generalizes the well-known result from the undirected (see, e.g., Cai et al., 2020, Definition 3.1) to the directed case. This result is an important tool for analyzing the evolution of the Dirichlet energy in graph neural networks.
(a) Synthetic cycle graph. The values of the fractional Laplacian can also be negative.
(b) Real-world graphs. We count the number of virtual edges built by the fractional Laplacian based on the distance  $d(i,j)$  in the original graph. The number of virtual edges increases as  $\alpha$  decreases.
Figure 3: Visual representation of long-range edges built by the fractional Laplacian.
4 Fractional Graph Laplacians
We introduce the fractional graph Laplacian through the singular value decomposition (SVD). This approach has two key advantages over the traditional definition (Pozrikidis, 2018; Benzi et al., 2020) in the spectral domain. First, it allows defining the fractional Laplacian based on any choice of graph Laplacian, including those with negative or complex spectrum such as the SNA. Secondly, the SVD is computationally more efficient and numerically more stable than the Jordan decomposition, which would be necessary if the fractional Laplacian was defined in the spectral domain.
Consider a directed graph with SNA  $\mathbf{L}$  and its SVD  $\mathbf{L} = \mathbf{U}\pmb{\Sigma}\mathbf{V}^{\mathsf{H}}$ , where  $\mathbf{U},\mathbf{V}\in \mathbb{C}^{N\times N}$  are unitary matrices and  $\pmb {\Sigma}\in \mathbb{R}^{N\times N}$  is a diagonal matrix. Given  $\alpha \in \mathbb{R}$ , we define the  $\alpha$ -fractional graph Laplacian  $^2$  ( $\alpha$ -FGL in short) as
$$
\mathbf {L} ^ {\alpha} := \mathbf {U} \boldsymbol {\Sigma} ^ {\alpha} \mathbf {V} ^ {\mathsf {H}}.
$$
In undirected graphs, the  $\alpha$ -FGL preserves the sign of the eigenvalues  $\lambda$  of  $\mathbf{L}$  while modifying their magnitudes, i.e.,  $\lambda \mapsto \mathrm{sign}(\lambda)|\lambda|^{\alpha}$ .<sup>3</sup>
The  $\alpha$ -FGL is generally less sparse than the original SNA, as it connects nodes that are not adjacent in the underlying graph. The next theorem proves that the weight of such "virtual" edges is bounded.
Theorem 4.1. Let  $\mathcal{G}$  be a directed graph with SNA L. For  $\alpha >0$ , if the distance  $d(i,j)$  between nodes  $i$  and  $j$  is at least 2, then
$$
\left| \left(\mathbf {L} ^ {\alpha}\right) _ {i, j} \right| \leq \left(1 + \frac {\pi^ {2}}{2}\right) \left(\frac {\| \mathbf {L} \|}{2 (d (i , j) - 1)}\right) ^ {\alpha}.
$$
We provide a proof of Theorem 4.1 in Appendix C. In Figure 3a, we visually represent the cycle graph with eight nodes and the corresponding  $\alpha$ -FGL entries. We also refer to Figure 3b, where we
2We employ the term "fractional graph Laplacian" instead of "fractional symmetrically normalized adjacency" to highlight that, in the singular value domain, one can construct fractional powers of any matrix representations of the underlying graph (commonly referred to as "graph Laplacians"). This fact is not generally true in the eigenvalue domain because eigenvalues can be negative or complex numbers.
3In fact, this property can be generalized to all directed graphs with a normal SNA matrix, including, among others, directed cycle graphs. See Lemma C.1 for more details.
depict the distribution of  $\alpha$ -FGL entries for the real-world graphs Cora (undirected) and Chameleon (directed) with respect to the distance in the original graph. Our empirical findings align with our theoretical results presented in Theorem 4.1.
5 Fractional Graph Laplacian Neural ODE
This section explores two fractional Laplacian-based graph neural ODEs. First, we consider the fractional heat equation,
$$
\mathbf {x} ^ {\prime} (t) = - \mathbf {L} ^ {\alpha} \mathbf {x} (t) \mathbf {W}, \mathbf {x} (0) = \mathbf {x} _ {0}, \tag {2}
$$
where  $\mathbf{x}_0\in \mathbb{R}^{N\times K}$  is the initial condition,  $\mathbf{x}(t)\in \mathbb{R}^{N\times K}$  for  $t > 0$  and  $\alpha \in \mathbb{R}$ . We assume that the channel mixing matrix  $\mathbf{W}\in \mathbb{R}^{K\times K}$  is a symmetric matrix. Second, we consider the fractional Schrödinger equation,
$$
\mathbf {x} ^ {\prime} (t) = i \mathbf {L} ^ {\alpha} \mathbf {x} (t) \mathbf {W}, \mathbf {x} (0) = \mathbf {x} _ {0}, \tag {3}
$$
where  $\mathbf{x}_0, \mathbf{x}(t) \in \mathbb{C}^{N \times K}$  and  $\mathbf{W} \in \mathbb{C}^{K \times K}$  is unitary diagonalizable. Both (2) and (3) can be analytically solved. For instance, the solution of (2) is given by  $\operatorname{vec}(\mathbf{x})(t) = \exp(-t\mathbf{W} \otimes \mathbf{L}^{\alpha})\operatorname{vec}(\mathbf{x}_0)$ , where  $\otimes$  denotes the Kronecker product and  $\operatorname{vec}(\cdot)$  represents the vectorization operation. However, calculating the exact solution is computationally infeasible since the memory required to store  $\mathbf{W} \otimes \mathbf{L}^{\alpha}$  alone grows as  $(NK)^2$ . Therefore, we rely on numerical schemes to solve (2) and (3).
In the remainder of this section, we analyze the Dirichlet energy for solutions to (2) and (3). We begin with the definition of oversmoothing.
Definition 5.1. Neural ODE-based GNNs are said to oversmooth if the normalized Dirichlet energy decays exponentially fast. That is, for any initial value  $\mathbf{x}_0$ , the solution  $\mathbf{x}(t)$  satisfies for every  $t > 0$
$$
\left| \mathcal {E} \left(\frac {\mathbf {x} (t)}{\| \mathbf {x} (t) \| _ {2}}\right) - \min  \lambda (\mathbf {I} - \mathbf {L}) \right| \leq \exp (- C t), C > 0.
$$
Definition 5.1 captures the actual smoothness of features by considering the normalized Dirichlet energy, which mitigates the impact of feature amplitude (Cai et al., 2020; Di Giovanni et al., 2023). Additionally, Proposition 3.6 shows that the normalized Dirichlet energy is intimately related to the numerical range of  $\mathbf{I} - \mathbf{L}$  of the underlying graph. This shows that the Dirichlet energy and eigenvalues (or frequencies) of the SNA are intertwined, and one can equivalently talk about Dirichlet energy or frequencies (see also Lemma D.2). In particular, it holds that
$$
0 \leq \mathcal {E} \left(\frac {\mathbf {x} (t)}{\| \mathbf {x} (t) \| _ {2}}\right) \leq \frac {\| \mathbf {I} - \mathbf {L} \|}{2}.
$$
As seen in Section 3, the minimal possible value attained by the normalized Dirichlet energy is often strictly greater than 0 for directed graphs. This indicates that GNNs on general directed graphs inherently cannot oversmooth to the same extent as in undirected. However, we prove that a vanilla GCN implementing the directed SNA oversmooths with respect to Definition 5.1, see Appendix E.3.
5.1 Frequency Analysis for Graphs with Normal SNA
This subsection focuses on the frequency analysis of FGL-based Neural ODEs for undirected graphs. Most classical GNNs (Kipf et al., 2017; Velickovic et al., 2018) and also graph neural ODEs (Chamberlain et al., 2021; Eliasof et al., 2021) have been shown to oversmooth. Di Giovanni et al. (2023) proved that the normalized Dirichlet energy for GNNs based on (2) with  $\alpha = 1$  can not only converge to its minimal value but also to its maximal possible value. A GNN exhibiting this property is then termed Highest-Frequency-Dominant (HFD).
However, in real-world scenarios, most graphs are not purely homophilic nor purely heterophilic but fall somewhere in between. Intuitively, this suggests that mid-range frequencies might be more suitable. To illustrate this intuition, consider the cycle graph as an example. If we have a homophily of 1, low frequencies are optimal; with a homophily equal to 0, high frequencies are optimal. Interestingly, for a homophily of  $\frac{1}{2}$ , the mid-range frequency is optimal, even though the eigendecomposition is label-independent. More information on this example can be found in Figure 4 and Appendix F. Based on this observation, we propose the following definition to generalize the concept of HFD, accommodating not only the lowest or highest frequency but all possible frequencies.
Figure 4: Eigendecomposition of  $\mathbf{L}$  for the cycle graph  $C_8$  (see Appendix F). The first two rows show the eigenvectors corresponding to the eigenvalues  $\lambda$ . The last row shows how the (label-unaware) eigendecomposition can be used to study homophily, whose definition requires the labels.
Definition 5.2. Let  $\lambda \geq 0$ . Neural ODE-based GNNs initialized at  $\mathbf{x}_0$  are  $\lambda$ -Frequency-Dominant ( $\lambda$ -FD) if the solution  $\mathbf{x}(t)$  satisfies
$$
\mathcal {E} \left(\frac {\mathbf {x} (t)}{\| \mathbf {x} (t) \| _ {2}}\right) \xrightarrow {t \to \infty} \frac {\lambda}{2}.
$$
Suppose  $\lambda$  is the smallest or the largest eigenvalue with respect to the real part. In that case, we call it Lowest-Frequency-Dominant (LFD) or Highest-Frequency-Dominant (HFD), respectively.
In the following theorem, we show that (2) and (3) are not limited to being LFD or HFD, but can also be mid-frequency dominant.