SlowGuess/NeurIPS_2023_OCR huggingface dataset:
When Do Graph Neural Networks Help with Node Classification? Investigating the Impact of Homophily Principle on Node Distinguishability
Sitao Luan $^{1,2}$ , Chenqing Hua $^{1,2}$ , Minkai Xu $^{4}$ , Qincheng Lu $^{1}$ , Jiaqi Zhu $^{1}$ , Xiao-Wen Chang $^{1,\dagger}$ , Jie Fu $^{2,5,\dagger}$ , Jure Leskovec $^{4,\dagger}$ , Doina Precup $^{1,2,3,\dagger}$
{sitao.luan@mail, chenqing.hua@mail, qincheng.lu@mail, jiaqi.zhu@mail, chang@cs, dprecup@cs}.mcgill.ca, {minkai,jure} @cs.stanford.edu, jiefu@ust.hk
<sup>1</sup>McGill University; <sup>2</sup>Mila - Quebec Artificial Intelligence Institute; <sup>3</sup>Google DeepMind; <sup>4</sup>Stanford University; <sup>5</sup>HKUST; † Corresponding Authors
Homophily principle, i.e., nodes with the same labels are more likely to be connected, has been believed to be the main reason for the performance superiority of Graph Neural Networks (GNNs) over Neural Networks on node classification tasks. Recent research suggests that, even in the absence of homophily, the advantage of GNNs still exists as long as nodes from the same class share similar neighborhood patterns [38]. However, this argument only considers intra-class Node Distinguishability (ND) but neglects inter-class ND, which provides incomplete understanding of homophily on GNNs. In this paper, we first demonstrate such deficiency with examples and argue that an ideal situation for ND is to have smaller intra-class ND than inter-class ND. To formulate this idea and study ND deeply, we propose Contextual Stochastic Block Model for Homophily (CSBM-H) and define two metrics, Probabilistic Bayes Error (PBE) and negative generalized Jeffreys divergence, to quantify ND. With the metrics, we visualize and analyze how graph filters, node degree distributions and class variances influence ND, and investigate the combined effect of intra- and inter-class ND. Besides, we discovered the mid-homophily pitfall, which occurs widely in graph datasets. Furthermore, we verified that, in real-work tasks, the superiority of GNNs is indeed closely related to both intra- and inter-class ND regardless of homophily levels. Grounded in this observation, we propose a new hypothesis-testing based performance metric beyond homophily, which is non-linear, feature-based and can provide statistical threshold value for GNNs' the superiority. Experiments indicate that it is significantly more effective than the existing homophily metrics on revealing the advantage and disadvantage of graph-aware modes on both synthetic and benchmark real-world datasets.
Graph Neural Networks (GNNs) have gained popularity in recent years as a powerful tool for graph-based machine learning tasks. By combining graph signal processing and convolutional neural networks, various GNN architectures have been proposed [27, 19, 50, 36, 23], and have been shown to outperform traditional neural networks (NNs) in tasks such as node classification (NC), graph classification, link prediction and graph generation. The success of GNNs is believed to be rooted in the homophily principle (assumption) [42], which states that connected nodes tend to have similar attributes [18], providing extra useful information to the aggregated features over the original node features. Such relational inductive bias is thought to be a major contributor to the superiority of GNNs over NNs on various tasks [4]. On the other hand, the lack of homophily, i.e., heterophily, is considered as the main cause of the inferiority of GNNs on heterophilic graphs, because nodes from different classes are connected and mixed, which can lead to indistinguishable node embeddings, making the classification task more difficult for GNNs [57, 55, 37]. Numerous models have been proposed to address the heterophily challenge lately [46, 57, 55, 37, 5, 32, 7, 54, 21, 34, 31, 51, 35].
Recently, both empirical and theoretical studies indicate that the relationship between homophily principle and GNN performance is much more complicated than "homophily wins, heterophily loses" and the existing homophily metrics cannot accurately indicate the superiority of GNNs [38, 35, 37]. For example, the authors in [38] stated that, as long as nodes within the same class share similar neighborhood patterns, their embeddings will be similar after aggregation. They provided experimental evidence and theoretical analysis, and concluded that homophily may not be necessary for GNNs to distinguish nodes. The paper [35] studied homophily/heterophily from post-aggregation node similarity perspective and found that heterophily is not always harmful, which is consistent with [38]. Besides, the authors in [37] have proposed to use high-pass filter to address some heterophily cases, which is adopted in [7, 5] as well. They have also proposed aggregation homophily, which is a linear feature-independent performance metric and is verified to be better at revealing the performance advantages and disadvantages of GNNs than the existing homophily metrics [46, 57, 32]. Moreover, [6] has investigated heterophily from a neighbor identifiable perspective and stated that heterophily can be helpful for NC when the neighbor distributions of intra-class nodes are identifiable.
Inspite that the current literature on studying homophily principle provide the profound insights, they are still deficient: 1. [38] only consider intra-class node distinguishability (ND), but ignore inter-class ND; 2. [35] does not show when and how high-pass filter can help with heterophily problem; 3. There is a lack of a non-linear, feature-based performance metric which can leverage richer information to provide an accurate statistical threshold value to indicate whether GNNs are really needed on certain task or not.
To address those issues, in this paper: 1. We show that, to comprehensively study the impact of homophily on ND, one needs to consider both intra- and inter-class ND and an ideal case is to have smaller intra-class ND than inter-class ND; 2. To formulate this idea, we propose Contextual Stochastic Block Model for Homophily (CSBM-H) as the graph generative model. It incorporates an explicit parameter to manage homophily levels, alongside class variance parameters to control intra-class ND, and node degree parameters which are important to study homophily [38, 54]; 3. To quantify ND of CSBM-H, we propose and compute two ND metrics, Probabilistic Bayes Error (PBE) and Negative Generalized Jeffreys Divergence ( $D_{\mathrm{NGJ}}$ ), for the optimal Bayes classifier of CSBM-H. Based on the metrics, we can analytically study how intra- and inter-class ND impact ND together. We visualize the relationship between PBE,  $D_{\mathrm{NGJ}}$  and homophily levels and discuss how different graph filters (full-, low- and high-pass filters), class variances and node degree distributions will influence ND in details; 4. In practice, we verify through hypothesis testing that the performance superiority of GNNs is indeed related to whether intra-class ND is smaller than inter-class ND, regardless of homophily levels. Based on this conclusion and the p-values of hypothesis testing, we propose Classifier-based Performance Metric (CPM), a new non-linear feature-based metric that can provide statistical threshold values. Experiments show that CPM is significantly more effective than the existing homophily metrics on predicting the superiority of graph-aware models over graph-agnostic.
We use **bold** font for vectors  $(e.g.,\pmb{v})$  and define a connected graph  $\mathcal{G} = (\mathcal{V},\mathcal{E})$ , where  $\mathcal{V}$  is the set of nodes with a total of  $N$  elements,  $\mathcal{E}$  is the set of edges without self-loops.  $A$  is the symmetric adjacency matrix with  $A_{i,j} = 1$  if there is an edge between nodes  $i$  and  $j$ , otherwise  $A_{i,j} = 0$ .  $D$  is the diagonal degree matrix of the graph, with  $D_{i,i} = d_i = \sum_j A_{i,j}$ . The neighborhood set  $\mathcal{N}_i$  of node  $i$  is defined as  $\mathcal{N}_i = \{j : e_{ij} \in \mathcal{E}\}$ . A graph signal is a vector in  $\mathbb{R}^N$ , whose  $i$ -th entry is a feature of node  $i$ . Additionally, we use  $X \in \mathbb{R}^{N \times F_h}$  to denote the feature matrix, whose columns are graph signals and  $i$ -th row  $X_{i,:} = \pmb{x}_i^\top$  is the feature vector of node  $i$  (i.e., the full-pass (FP) filtered signal). The label encoding matrix is  $Z \in \mathbb{R}^{N \times C}$ , where  $C$  is the number of classes, and its  $i$ -th row  $Z_{i,:}$  is the one-hot encoding of the label of node  $i$ . We denote  $z_i = \arg \max_j Z_{i,j} \in \{1,2,\dots,C\}$ . The indicator function  $\mathbf{1}_B$  equals 1 when event  $B$  happens and 0 otherwise.
For nodes  $i,j\in \mathcal{V}$ , if  $z_{i} = z_{j}$ , then they are considered as intra-class nodes; if  $z_{i}\neq z_{j}$ , then they are considered to be inter-class nodes. Similarly, an edge  $e_{i,j}\in \mathcal{E}$  is considered to be an intra-class edge if  $z_{i} = z_{j}$ , and an inter-class edge if  $z_{i}\neq z_{j}$ .
2.1 Graph-aware Models and Graph-agnostic Models
A network that includes the feature aggregation step according to graph structure is called graph-aware (G-aware) model, e.g., GCN [27], SGC [53]; A network that does not use graph structure information is called graph-agnostic (G-agnostic) model, such as Multi-Layer Perceptron with 2
layers (MLP-2) and MLP-1. A G-aware model is often coupled with a G-agnostic model because when we remove the aggregation step in G-aware model, it becomes exactly the same as its coupled G-agnostic model, e.g., GCN is coupled with MLP-2 and SGC-1 is coupled with MLP-1 as below,
$$
\mathbf {G C N}: Y = \operatorname {S o f t m a x} \left(\hat {A} _ {\text {s y m}} \operatorname {R e L U} \left(\hat {A} _ {\text {s y m}} X W _ {0}\right) W _ {1}\right), \quad \mathbf {M L P - 2}: Y = \operatorname {S o f t m a x} \left(\operatorname {R e L U} \left(X W _ {0}\right) W _ {1}\right)
$$
$$
\mathbf {S G C - 1}: Y = \operatorname {S o f t m a x} \left(\hat {A} _ {\text {s y m}} X W _ {0}\right), \quad \mathbf {M L P - 1}: Y = \operatorname {S o f t m a x} \left(X W _ {0}\right) \tag {1}
$$
where  $\hat{A}_{\mathrm{sym}} = \tilde{D}^{-1/2}\tilde{A}\tilde{D}^{-1/2}, \tilde{A} \equiv A + I$  and  $\tilde{D} \equiv D + I$ ;  $W_0 \in \mathbb{R}^{F_0 \times F_1}$  and  $W_1 \in \mathbb{R}^{F_1 \times O}$  are learnable parameter matrices. For simplicity, we denote  $y_i = \arg \max_j Y_{i,j} \in \{1, 2, \ldots, C\}$ . The random walk renormalized matrix  $\hat{A}_{\mathrm{rw}} = \tilde{D}^{-1}\tilde{A}$  can also be applied to GCN, which is essentially a mean aggregator commonly used in some spatial-based GNNs [19]. To bridge spectral and spatial methods, we use  $\hat{A}_{\mathrm{rw}}$  in the theoretical analysis, but self-loops are not added to the adjacency matrix to maintain consistency with previous literature [38, 35].
To address the heterophily challenge, high-pass (HP) filter [14], such as  $I - \hat{A}_{\mathrm{rw}}$ , is often used to replace low-pass (LP) filter [39]  $\hat{A}_{\mathrm{rw}}$  in GCN [5, 7, 35]. In this paper, we use  $\hat{A}_{\mathrm{rw}}$  and  $I - \hat{A}_{\mathrm{rw}}$  as the LP and HP operators, respectively. The LP and HP filtered feature matrices are represented as  $H = \hat{A}_{\mathrm{rw}}X$  and  $H^{\mathrm{HP}} = (I - \hat{A}_{\mathrm{rw}})X$ . For simplicity, we denote  $\pmb{h}_i = (H_{i,:})^\top$ ,  $\pmb{h}_i^{\mathrm{HP}} = (H_{i,:}^{\mathrm{HP}})^\top$ .
To measure how likely the G-aware model can outperform its coupled G-agnostic model before training them (i.e., if the aggregation step according to graph structure is helpful for node classification or not), a lot of homophily metrics have been proposed and we will introduce the most commonly used ones in the following subsection.
2.2 Homophily Metrics
The homophily metric is a way to describe the relationship between node labels and graph structure. We introduce five commonly used homophily metrics: edge homophily [1, 57], node homophily [46], class homophily [32], generalized edge homophily [26] and aggregation homophily [35], adjusted homophily [47] and label informativeness [47] as follows:
$$
\mathrm {H} _ {\text {e d g e}} (\mathcal {G}) = \frac {\left| \{e _ {u v} \mid e _ {u v} \in \mathcal {E} , Z _ {u , :} = Z _ {v , :} \} \right|}{| \mathcal {E} |}, \mathrm {H} _ {\text {n o d e}} (\mathcal {G}) = \frac {1}{| \mathcal {V} |} \sum_ {v \in \mathcal {V}} \mathrm {H} _ {\text {n o d e}} ^ {v} = \frac {1}{| \mathcal {V} |} \sum_ {v \in \mathcal {V}} \frac {\left| \{u \mid u \in \mathcal {N} _ {v} , Z _ {u , :} = Z _ {v , :} \} \right|}{d _ {v}},
$$
$$
\mathrm {H} _ {\text {c l a s s}} (\mathcal {G}) = \frac {1}{C - 1} \sum_ {k = 1} ^ {C} \left[ h _ {k} - \frac {\left| \{v \mid Z _ {v , k} = 1 \} \right|}{N} \right] _ {+}, \text {w h e r e} h _ {k} = \frac {\sum_ {v \in \mathcal {V} , Z _ {v , k} = 1} \left| \{u u \in \mathcal {N} _ {v} , Z _ {u , :} = Z _ {v , :} \} \right|}{\sum_ {v \in \{v \mid Z _ {v , k} = 1 \}} d _ {v}}, \tag {2}
$$
$$
\mathbf{H}_{\mathrm{GE}}(\mathcal{G}) = \frac{\sum_{(i,j)\in\varepsilon}\cos(\boldsymbol{x}_{i},\boldsymbol{x}_{j})}{|\mathcal{E}|}, \mathbf{H}_{\mathrm{agg}}(\mathcal{G}) = \frac{1}{|V|} \times \left|\left\{v  \Big|\operatorname {Mean}_{u}\bigl(\{S(\hat{A},Z)^{Z_{u,:} = Z_{v,:}}\} \bigr)\geq \operatorname {Mean}_{u}\bigl(\{S(\hat{A},Z)^{Z_{u,:}\neq Z_{v,:}}\} \bigr)\right\} \right|,
$$
$$
\mathrm {H} _ {\mathrm {a d j}} = \frac {\mathrm {H} _ {\mathrm {e d g e}} - \sum_ {c = 1} ^ {C} \bar {p} _ {c} ^ {2}}{1 - \sum_ {k = 1} ^ {C} \bar {p} _ {c} ^ {2}}, \mathrm {L I} = - \frac {\sum_ {c _ {1} , c _ {2}} p _ {c _ {1} , c _ {2}} \log \frac {p _ {c _ {1} , c _ {2}}}{\bar {p} _ {c _ {1}} \bar {p} _ {c _ {2}}}}{\sum_ {c} \bar {p} _ {c} \log \bar {p} _ {c}} = 2 - \frac {\sum_ {c _ {1} , c _ {2}} p _ {c _ {1} , c _ {2}} \log p _ {c _ {1} , c _ {2}}}{\sum_ {c} \bar {p} _ {c} \log \bar {p} _ {c}}
$$
where  $\mathrm{H}_{\mathrm{node}}^v$  is the local homophily value for node  $v$ ;  $[a]_{+} = \max(0, a)$ ;  $h_k$  is the class-wise homophily metric [32]; Mean_u  $(\{\cdot\})$  takes the average over  $u$  of a given multiset of values or variables and  $S(\hat{A}, Z) = \hat{A}Z(\hat{A}Z)^{\top}$  is the post-aggregation node similarity matrix;  $D_c = \sum_{v: z_v = c} d_v$ ,  $\bar{p}_c = \frac{D_c}{2|\mathcal{E}|}$ ,  $p_{c_1, c_2} = \sum_{(u, v) \in \mathcal{E}} \frac{\mathbf{1}\{z_u = c_1, z_v = c_2\}}{2|\mathcal{E}|}$ ,  $c, c_1, c_2 \in \{1, \dots, C\}$ .
These metrics all fall within the range of [0, 1], with a value closer to 1 indicating strong homophily and implying that G-aware models are more likely to outperform its coupled G-agnostic model, and vice versa. However, the current homophily metrics are almost all linear, feature-independent metrics which cannot provide a threshold value [35] for the superiority of G-aware model and fail to give an accurate measurement of node distinguishability (ND). In the following section, we focus on quantifying the ND of graph models with with homophily levels and analyzing their relations.
3 Analysis of Homophily on Node Distinguishability (ND)
The Problem in Current Literature Recent research has shown that heterophily does not always negatively impact the embeddings of intra-class nodes, as long as their neighborhood patterns "corrupt in the same way" [38,6]. For example, in Figure 1 nodes  $\{1,2\}$  are from class blue and both have the same heterophilic neighborhood patterns. As a result, their aggregated features will still be similar and they can be classified into the same class.
However, this is only partially true for ND if we forget to discuss inter-class ND, e.g., node 3 in Figure I is from class green and has the same neighborhood pattern (1/3 orange, 1/3 yellow and 1/3 green) as nodes  $\{1,2\}$ , which means the inter-class ND will be lost after aggregation. This highlights the necessity for careful consideration of both intra- and inter-class ND when evaluating the impact of homophily on the performance of GNNs and an ideal case for NC would be node  $\{1,2,4\}$ , where we have smaller intra-class "distance" than inter-class "distance". We will formulate the above idea in this section and verify if it really relates to the performance of GNNs in section 4. In the following subsection, we will propose a toy graph model, on which we can study the relationship between homophily and ND directly and intuitively, and analyze intra- and inter-class ND analytically.
Node 1 (Class Blue)
Node 3 (Class Green)
Figure 1: Example of intra- and inter-class node distinguishability.
Node 2 (Class Blue)
Node 4 (Class Green)
3.2 CSBM-H and Optimal Bayes Classifier
In order to have more control over the assumptions on the node
embeddings, we consider the Contextual Stochastic Block Model (CSBM) [1]. It is a generative model that is commonly used to create graphs and node features, and it has been widely adopted to study the behavior of GNNs [49, 52]. To investigate the impact of homophily on ND, the authors in [38] simplify CSBM to the two-normal setting, where the node features  $X$  are assumed to be sampled from two normal distributions and intra- and inter-class edges are generated according to two separate parameters. This simplification does not lose much information about CSBM, but 1. it does not include an explicit homophily parameter to study homophily directly and intuitively; 2. it does not include class variance parameters to study intra-class ND; 3. the authors do not rigorously quantify ND.
In this section, we introduce the Contextual Stochastic Block Model for Homophily/Heterophily (CSBM-H), which is a variation of CSBM that incorporates an explicit homophily parameter  $h$  for the two-normal setting and also has class variance parameters  $\sigma_0^2$ ,  $\sigma_1^2$  to describe the intra-class ND. We then derive the optimal Bayes classifier  $(\mathrm{CL}_{\mathrm{Bayes}})$  and negative generalized Jeffreys divergence for CSBM-H, based on which we can quantify and investigate ND for CSBM-H.
$\mathbf{CSBM - H}(\pmb {\mu}_0,\pmb {\mu}_1,\sigma_0^2 I,\sigma_1^2 I,d_0,d_1,h)$  The generated graph consists of two disjoint sets of nodes,  $i\in \mathcal{C}_0$  and  $j\in \mathcal{C}_1$ , corresponding to the two classes. The features of each node are generated independently, with  $\pmb{x}_i$  generated from  $N(\pmb {\mu}_0,\sigma_0^2 I)$  and  $\pmb {x}_j$  generated from  $N(\pmb {\mu}_1,\sigma_1^2 I)$ , where  $\pmb {\mu}_0,\pmb {\mu}_1\in \mathbb{R}^{F_h}$  and  $F_{h}$  is the dimension of the embeddings. The degree of nodes in  $\mathcal{C}_0$  and  $\mathcal{C}_1$  are  $d_0,d_1\in \mathbb{N}$  respectively. For  $i\in \mathcal{C}_0$ , its neighbors are generated by independently sampling from  $h\cdot d_0$  intra-class nodes and  $(1 - h)\cdot d_0$  inter-class nodes. The neighbors of  $j\in \mathcal{C}_1$  are generated in the same way. As a result, the FP (full-pass), LP and HP filtered features are generated as follows,
$$
i \in \mathcal {C} _ {0}: \boldsymbol {x} _ {i} \sim N (\boldsymbol {\mu} _ {0}, \sigma_ {0} ^ {2} I); \boldsymbol {h} _ {i} \sim N (\tilde {\boldsymbol {\mu}} _ {0}, \tilde {\sigma} _ {0} ^ {2} I), \boldsymbol {h} _ {i} ^ {\mathrm {H P}} \sim N \left(\tilde {\boldsymbol {\mu}} _ {0} ^ {\mathrm {H P}}, (\tilde {\sigma} _ {0} ^ {\mathrm {H P}}) ^ {2} I\right),
$$
$$
j \in \mathcal {C} _ {1}: \boldsymbol {x} _ {j} \sim N \left(\boldsymbol {\mu} _ {1}, \sigma_ {1} ^ {2} I\right); \boldsymbol {h} _ {j} \sim N \left(\tilde {\boldsymbol {\mu}} _ {1}, \tilde {\sigma} _ {1} ^ {2} I\right), \boldsymbol {h} _ {j} ^ {\mathrm {H P}} \sim N \left(\tilde {\boldsymbol {\mu}} _ {1} ^ {\mathrm {H P}}, \left(\tilde {\sigma} _ {1} ^ {\mathrm {H P}}\right) ^ {2} I\right), \tag {3}
$$
where  $\tilde{\pmb{\mu}}_0 = h(\pmb{\mu}_0 - \pmb{\mu}_1) + \pmb{\mu}_1$ ,  $\tilde{\pmb{\mu}}_1 = h(\pmb{\mu}_1 - \pmb{\mu}_0) + \pmb{\mu}_0$ ,  $\tilde{\pmb{\mu}}_0^{\mathrm{HP}} = (1 - h)(\pmb{\mu}_0 - \pmb{\mu}_1)$ ,  $\tilde{\pmb{\mu}}_1^{\mathrm{HP}} = (1 - h)(\pmb{\mu}_1 - \pmb{\mu}_0)$ ,  $\tilde{\sigma}_0^2 = \frac{(h(\sigma_0^2 - \sigma_1^2) + \sigma_1^2)}{d_0}$ ,  $\tilde{\sigma}_1^2 = \frac{(h(\sigma_1^2 - \sigma_0^2) + \sigma_0^2)}{d_1}$ ,  $(\tilde{\sigma}_0^{\mathrm{HP}})^2 = \sigma_0^2 + \frac{(h(\sigma_0^2 - \sigma_1^2) + \sigma_1^2)}{d_0}$ ,  $(\tilde{\sigma}_1^{\mathrm{HP}})^2 = \sigma_1^2 + \frac{(h(\sigma_1^2 - \sigma_0^2) + \sigma_0^2)}{d_1}$ . If  $\sigma_0^2 < \sigma_1^2$ , we refer to  $\mathcal{C}_0$  as the low variation class and  $\mathcal{C}_1$  as the high variation class. The variance of each class can reflect the intra-class ND. We abuse the notation  $x_i \in \mathcal{C}_0$  for  $i \in \mathcal{C}_0$  and  $x_j \in \mathcal{C}_1$  for  $j \in \mathcal{C}_1$ .
To quantify the ND of CSBM-H, we first compute the optimal Bayes classifier in the following theorem. The theorem is about the original features, but the results are applicable to the filtered features when the parameters are replaced according to equation 3.
Theorem 1. Suppose  $\sigma_0^2 \neq \sigma_1^2$  and  $\sigma_0^2, \sigma_1^2 > 0$ , the prior distribution for  $\pmb{x}$  is  $\mathbb{P}(\pmb{x} \in \mathcal{C}_0) = \mathbb{P}(\pmb{x} \in \mathcal{C}_1) = 1/2$ , then the optimal Bayes Classifier (CLBayes) for CSBM-H  $(\pmb{\mu}_0, \pmb{\mu}_1, \sigma_0^2 I, \sigma_1^2 I, d_0, d_1, h)$  is
This implies that we generate undirected graphs. See Appendix E.1 for the discussion of directed vs. undirected graphs. See E.2 for the discussion on how to extend CSBM-H to more general settings.
2To avoid unnecessary confusion: we relax  $hd_0$  and  $(1 - h)d_{0}$  to be continuous values so that the visualization in the following sections are more readable and intuitive, especially to show the intersections of the curves.
$$
\operatorname {C L} _ {\text {B a y e s}} (\boldsymbol {x}) = \left\{ \begin{array}{l} 1, \eta (\boldsymbol {x}) \geq 0. 5 \\ 0, \eta (\boldsymbol {x}) <   0. 5 \end{array} , \eta (\boldsymbol {x}) = \mathbb {P} (z = 1 | \boldsymbol {x}) = \frac {1}{1 + \exp (Q (\boldsymbol {x}))}, \right.
$$
where  $Q(\pmb {x}) = a\pmb{x}^{\top}\pmb {x} + \pmb{b}^{\top}\pmb {x} + c,a = \frac{1}{2}\left(\frac{1}{\sigma_{1}^{2}} -\frac{1}{\sigma_{0}^{2}}\right),\pmb {b} = \frac{\pmb{\mu}_{0}}{\sigma_{0}^{2}} -\frac{\pmb{\mu}_{1}}{\sigma_{1}^{2}},c = \frac{\pmb{\mu}_{1}^{\top}\pmb{\mu}_{1}}{2\sigma_{1}^{2}} -\frac{\pmb{\mu}_{0}^{\top}\pmb{\mu}_{0}}{2\sigma_{0}^{2}} +\ln \left(\frac{\sigma_{1}^{F_{h}}}{\sigma_{0}^{F_{h}}}\right)$
3. See the proof in Appendix A.
Advantages of  $\mathbf{CL}_{\mathrm{Bayes}}$  Over the Fixed Linear Classifier in [38] The decision boundary in [38] is defined as  $P = \{\pmb{x}|\pmb{w}^\top \pmb{x} - \pmb{w}^\top (\pmb{\mu}_0 + \pmb{\mu}_1) / 2\}$  where  $\pmb{w} = (\pmb{\mu}_0 - \pmb{\mu}_1) / ||\pmb{\mu}_0 - \pmb{\mu}_1||_2$  is a fixed parameter. This classifier only depends on  $\pmb{\mu}_0, \pmb{\mu}_1$  and is independent of  $h$ . However, as  $h$  changes, the "separability" of the two normal distributions should be different. The fixed classifier cannot capture this difference, and thus is not qualified to measure ND for different  $h$ . Besides, we cannot investigate how variances  $\sigma_0^2, \sigma_1^2$  and node degrees  $d_0, d_1$  affect ND with the fixed classifier in [38].
In the following subsection, we will define two methods to quantify ND of CSBM-H, one is based on  $\mathrm{CL}_{\mathrm{Bayes}}$ , which is a precise measure but hard to be explainable; another is based on KL-divergence, which can give us more intuitive understanding of how intra- and inter-class ND will impact ND at different homophily levels. These two measurements can be used together to analyze ND.
3.3 Measure Node Distinguishability of CSBM-H
The Bayes error rate (BE) of the data distribution is the probability of a node being mis-classified when the true class probabilities are known given the predictors [20]. It can be used to measure the distinguishability of node embeddings and the BE for  $\mathrm{CL}_{\mathrm{Bayes}}$  is defined as follows,
Definition 1 (Bayes Error Rate). The Bayes error rate [20] for  $\mathrm{CL}_{\mathrm{Bayes}}$  is defined as