SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The detailed proof can be found in Appendix D.3. The proposition suggests that aggregated features  $\mathbf{f}$  have better linear separability than the original feature  $x$  when the node  $i$  satisfies  $d_{i} > (p^{(1)} + q^{(1)})^{2} / (p^{(1)} - q^{(1)})^{2}$ . For instance, when  $p^{(1)} = 0.9$  and  $q^{(1)} = 0.1$ , the aggregated features are easier to show better linear separability with  $d_{i} > 1.75$ , which are commonly met in real-world scenarios. This indicates that aggregation is likely to contribute to improved feature separability within the same node subgroup sharing a similar structural property.
We then examine the linear separability for nodes from different classes within different structural patterns, e.g.,  $h_1^{(1)}$  and  $h_2^{(2)}$ . Before diving deep into the rigorous analysis, we first illustrate a special example to show how the structure can lead to worse linear separability. When  $p^{(1)} = q^{(2)}$  and  $q^{(1)} = p^{(1)}$ , we can identify that the  $h_1^{(1)}$  and  $h_2^{(2)}$  are exactly generated from the same distribution. Such distribution can hardly be linearly separable. We then conduct a more rigorous analysis showing a similar observation with the same pattern one. We can summarize the proposition as follows:
Lemma 3 (Linear separability on nodes with different structural patterns). Consider features are from different structural patterns, where  $\mathbf{f}_i^{(1)}$  for  $i\in \mathcal{C}_1$  and  $\mathbf{f}_i^{(2)}$  for  $i\in \mathcal{C}_2$ . For any node  $i$ , the largest-margin linear classifier will have a lower probability to misclassify  $\mathbf{f}_i^{(1)}$  for  $i\in \mathcal{C}_1$  and  $\mathbf{f}_i^{(2)}$  for  $i\in \mathcal{C}_2$  than  $\mathbf{x}_i$  when  $d_i > \frac{(p^{(1)} + q^{(1)})^2}{(p^{(1)} - q^{(2)})^2}$
we can find that aggregated feature show better separability when  $d_{i} > (p^{(1)} + q^{(1)})^{2} / |p^{(1)} - q^{(2)}|^{2}$ . The detailed proof can be found in Appendix D.4. For instance, when  $p^{(1)} = 0.9$ ,  $q^{(1)} = 0.1$  and  $p^{(2)} = 0.2$ ,  $q^{(2)} = 0.8$ , the aggregated features can only show better linear separability with  $d_{i} > 100$  which are hardly met in real-world scenarios. Notice that the only difference with the one in the same pattern one is that the denominator is  $|p^{(1)} - q^{(2)}|^{2}$ , smaller than the  $|p^{(1)} - q^{(1)}|^{2}$  since  $q^{(1)} > q^{(2)}$ . It indicates that only nodes with higher degrees. e.g.,  $d_{i} = 100$ , can achieve improved linear separability.
Based on our analysis, we can draw the conclusion that when nodes are in the same pattern, aggregation can show improved linear separability on  $(p^{(1)} + q^{(1)})^2 / |p^{(1)} - q^{(1)}|^2 < d_i < (p^{(1)} + q^{(1)})^2 / |p^{(1)} - q^{(2)}|^2$  while the ones in the different patterns cannot. It indicates that aggregation can help when nodes are in the same pattern, however, show limitation when nodes are from different patterns. Notice that, such linear separability can be evidence for different behavior on nodes with different patterns. Nonetheless, it cannot directly indicate the performance disparity. Feature separability is conducted based on the ideal classifier with the largest margin. Better feature
separability does not necessarily result in better performance if the classifier is not well-trained with biased training data.
D.2 Linear separability experiment on synthetic CSBM dataset
In this section, we aim to empirically examine the validity of the theoretical analysis results in Section D.1. We generate graph with the CSBM model with the following detailed settings. The class mean distance  $\rho = \| \mu_1 - \mu_2\|$  is set to 0.1. The feature dimension  $d$  and the number of nodes  $n$  are set to 50, and 500, respectively.  $p$  and  $q$  correspond to probabilities of the intra-class and inter-class probability, respectively. The mean aggregation is defined as  $\mathbf{F} = \mathbf{D}^{-1}\mathbf{A}\mathbf{X}$ . It is the key to generating graphs with different structural properties. To generate the homophilic node subgroup with  $p > q$ , we utilize the following settings with  $(p = 0.01, q = 0.005)$ ,  $(p = 0.01, q = 0.003)$ , and  $(p = 0.01, q = 0.001)$ . To generate the heterophilic node subgroup with  $p < q$ , we utilize the following settings with  $(p = 0.001, q = 0.005)$ ,  $(p = 0.001, q = 0.003)$ , and  $(p = 0.001, q = 0.002)$ .
Experiments are conducted to examine how a linear model, logistic regression, fits effectively on the aggregated features. Higher performance on logistic regression indicates better linear separability of features. It is important to note that we focus on the fitting ability rather than the generalization performance. Consequently, we do not evaluate with a test set; instead, we provide all labels for training and assess the performance on all the nodes.
Experiments on the logistic regression are conducted on the homophilic nodes solely, heterophilic nodes solely, and a mixture of homophilic and heterophilic nodes. Typically, we aim to show the linear separability both within the same structural pattern and between homophilic and heterophilic patterns. Experimental results are presented in Table 6. The following observations can be made: (1) When considering nodes following the same structural pattern, either homophilic pattern or heterophilic pattern, better performance can be observed when the probability difference  $|p - q|$  between intra-class and inter-class probability is larger. (2) Comparing the performance considering both structural patterns with the one on a single structural pattern, we reveal a noticeable performance degradation on the one with mixture patterns. This suggests that the nodes with different structural patterns results in decreased linear separability. (3) Comparing the performance among different mixing structural patterns, it is evident that larger homophily ratio differences, e.g.,  $(p = 0.01, q = 0.005)$  compared to  $(p = 0.001, q = 0.005)$ , yield poorer performance than those with smaller homophily ratio differences. The above observations further supports the validity of Lemma 2 and 3, Theorem 1.
Table 6: The performance of logistic regression algorithm on homophilic nodes, heterophilic nodes, and a mixture of homophilic and heterophilic nodes. The results on the first row and first column correspond to the performance on homophilic nodes and heterophilic nodes, solely.
Hete\Homo	-	p=0.01, q=0.005	p=0.01, q=0.003	p=0.01, q=0.001
-	-	74.68±3.19	82.71±1.86	92.08±1.13
p=0.001, q=0.005	79.64±2.11	60.84±0.64	62.08±0.59	81.38±1.02
p=0.001, q=0.003	70.08±1.71	59.72±2.01	61.58±1.08	76.60±0.98
p=0.001, q=0.002	62.08±3.04	65.92±1.95	69.42±1.03	74.16±1.09
D.3 Proof details of linear seperability within the same pattern
In this section, we provide detailed proof of the improved linear separability on aggregated features where nodes from different classes follow the same structural pattern. The following theoretical results indicate that it is more difficult to have improved linear separability when nodes follow different structural patterns. Notably, the following proof is derived based on [13]. For completeness, we show the previous proof as follows, the difference is that we consider a more complicated CSBM-S model with both homophilic and heterophilic patterns rather than the simple CSBM model only describing either homophilic or heterophilic pattern.
Notably, we only show the case  $p^{(1)} > q^{(1)}$  in Appendix D.1 for simplicity. It can be easily extended to  $p^{(2)} < q^{(2)}$ . Proof details can be found as follows.
Consider the vanilla mean aggregation as  $\mathbf{F} = \mathbf{D}^{-1}\mathbf{A}\mathbf{X}$ , the aggregated features with the same structure patterns follow Gaussian distributions:
Then we denote the expectation of the original node features  $\mathbf{x}$  in two classes as  $\mathbb{E}_{c_1}[\mathbf{x}_i]$  and  $\mathbb{E}_{c_2}[\mathbf{x}_i]$ . Similarly, we denote the expectation of the aggregated features as  $\mathbb{E}_{c_1}[\mathbf{f}_i]$  and  $\mathbb{E}_{c_2}[\mathbf{f}_i]$ .
Proposition 2.  $(\mathbb{E}_{c_1}[\mathbf{x}_i],\mathbb{E}_{c_2}[\mathbf{x}_i])$  and  $(\mathbb{E}_{c_1}[\mathbf{f}_i],\mathbb{E}_{c_2}[\mathbf{f}_i])$  share the same middle point.  $\mathbb{E}_{c_1}[\mathbf{x}_i] - \mathbb{E}_{c_2}[\mathbf{x}_i]$  and  $\mathbb{E}_{c_1}[\mathbf{f}_i] - \mathbb{E}_{c_2}[\mathbf{f}_i]$  share the same direction.
The proposition can be calculated through direct calculations. If we consider the feature distributions of two classes, we can observe that both  $\mathbf{x}$  and  $\mathbf{f}$  exhibit a systematic relationship. As a consequence, we can determine the optimal linear classifier for both  $\mathbf{x}_i$  and  $\mathbf{f}_i$ . We then define decision boundary as the hyperplane that is orthogonal to the direction  $\mathbf{w} = \frac{\mu_1 - \mu_2}{||\mu_1 - \mu_2||}$  and passes through the middle point  $\mathbf{m} = (\pmb{\mu}_1 + \pmb{\mu}_2) / 2$  as:
$$
\mathcal {P} = \left\{\mathbf {x} \mid \mathbf {w} ^ {\top} \mathbf {x} - \mathbf {w} ^ {\top} \left(\boldsymbol {\mu} _ {1} + \boldsymbol {\mu} _ {2}\right) / 2 \right\} \tag {9}
$$
We then show the proof details on when  $\mathbf{f}$  has a lower mis-classified probability than  $\mathbf{x}$  with the same decision boundary, indicating better linear separability.
Proof. We provide the proof only for nodes belonging to class  $c_{0}$ , as the case for nodes from class  $c_{1}$  is symmetric and the proof follows the same logic. For a node  $i$  in  $\mathcal{C}_1$ , We have the following:
$$
\begin{array}{l} \mathbb {P} (\mathbf {x} _ {i} \text {i s m i s - c l a s s i f i e d}) = \mathbb {P} \left(\mathbf {w} ^ {\top} \mathbf {x} _ {i} + \mathbf {b} \leq 0\right) \text {f o r} i \in \mathcal {C} _ {1} \\ \mathbb {P} (\mathbf {f} _ {i} \text {i s m i s - c l a s s i f i e d}) = \mathbb {P} \left(\mathbf {w} ^ {\top} \mathbf {f} _ {i} + \mathbf {b} \leq 0\right) \text {f o r} i \in \mathcal {C} _ {1}, \end{array} \tag {10}
$$
we can then scale the decision boundary without changing the original meaning. Then we can have  $\mathbb{P}\left(\mathbf{w}^\top \mathbf{f}_i + \mathbf{b} \leq 0\right) = \mathbb{P}\left(\sqrt{d_i} \mathbf{w}^\top \mathbf{f}_i + \sqrt{d_i} \mathbf{b} \leq 0\right)$  We denote the scaled version of  $\mathbf{f}_i$  as  $\mathbf{f}_i' = \sqrt{d_i} \mathbf{f}_i$ . Then,  $\mathbf{f}_i'$  follows:
$$
\mathbf {f} _ {i} ^ {\prime} \sim N \left(\frac {\sqrt {d _ {i}} \left(p ^ {(j)} \boldsymbol {\mu} _ {1} + q ^ {(j)} \boldsymbol {\mu} _ {2}\right)}{p ^ {(j)} + q ^ {(j)}}, \mathbf {I}\right), \text {f o r} i \in \mathcal {C} _ {1} \tag {11}
$$
Given the scale in Eq. equation 11, the decision boundary for  $\mathbf{f}_i^{\prime}$  is consequently shifted to  $\mathbf{w}^{\top}\mathbf{f}^{\prime} + \sqrt{d_i}\mathbf{b} = 0$ . Now, considering that  $\mathbf{x}_i$  and  $\mathbf{f}_i^{\prime}$  have the same variance, we can compare the misclassification probabilities by merely comparing the distance from their expected values to their corresponding decision boundaries. Specifically, the distances can be described as follows:
$$
\mathrm {d i s} _ {\mathbf {x} _ {i}} = \frac {\left\| \boldsymbol {\mu} _ {1} - \boldsymbol {\mu} _ {2} \right\| _ {2}}{2}
$$
$$
\operatorname {d i s} _ {\mathbf {f} _ {i} ^ {\prime}} = \frac {\sqrt {d _ {i}} \left| p ^ {(j)} - q ^ {(j)} \right|}{p ^ {(j)} + q ^ {(j)}}. \frac {\left\| \boldsymbol {\mu} _ {1} - \boldsymbol {\mu} _ {2} \right\| _ {2}}{2}. \tag {12}
$$
The larger the distance from the expectation to the decision boundary indicates a smaller the misclassification probability. Therefore, when  $\mathrm{dis}_{\mathbf{f}_i'} > \mathrm{dis}_{\mathbf{x}_i}$ ,  $\mathbf{f}_i'$  has a lower probability of being misclassified than  $\mathbf{x}_i$ . By comparing the two distances, we conclude that when  $d_i > \frac{\left(p^{(j)} + q^{(j)}\right)^2}{\left(p^{(j)} - q^{(j)}\right)^2}$ , the mean aggregated feature  $\mathbf{f}_i'$  exhibits a lower probability of misclassification compared to the original node feature  $\mathbf{x}_i$ .
$$
\mathbb {P} \left(\mathbf {f} _ {i} \text {i s m i s - c l a s s i f i e d}\right) <   \mathbb {P} \left(\mathbf {x} _ {i} \text {i s m i s - c l a s s i f i e d}\right) \text {i f} d _ {i} > \left(\frac {p ^ {(j)} + q ^ {(j)}}{p ^ {(j)} - q ^ {(j)}}\right) ^ {2} \tag {13}
$$
D.4 Proof details of linear separability between different structural patterns
In this section, we provide detailed proof of the improved linear separability on aggregated features where nodes from different classes follow different structural patterns. The following theoretical results indicate that it is more difficult to have improved linear separability when nodes follow different structural patterns. Notably, the following proof is derived based on [13]. For completeness, we show the previous proof as follows, the difference is that we consider a more complicated CSBM-S model with both homophilic and heterophilic patterns rather than the simple CSBM model only describing either homophilic or heterophilic pattern
In the following discussion, we only focus on the scenario where nodes in class  $c_{1}$  are in a homophilic pattern with  $p^{(1)} > q^{(1)}$ , and nodes in class  $c_{2}$  are in a heterophilic pattern with  $p^{(2)} < q^{(2)}$ . The other scenario is symmetric and the proof follows the same logic.
Consider the vanilla mean aggregation as  $\mathbf{F} = \mathbf{D}^{-1}\mathbf{A}\mathbf{X}$ , the aggregated features with different structure patterns follow Gaussian distributions:
$$
\mathbf {f} _ {i} ^ {(1)} \sim N \left(\frac {p ^ {(1)} \boldsymbol {\mu} _ {1} + q ^ {(1)} \boldsymbol {\mu} _ {2}}{p ^ {(1)} + q ^ {(1)}}, \frac {\mathbf {I}}{\sqrt {d _ {i}}}\right), \text {f o r} i \in \mathcal {C} _ {1}; \mathbf {f} _ {i} ^ {(2)} \sim N \left(\frac {q ^ {(2)} \boldsymbol {\mu} _ {1} + p ^ {(2)} \boldsymbol {\mu} _ {2}}{p ^ {(2)} + q ^ {(2)}}, \frac {\mathbf {I}}{\sqrt {d _ {i}}}\right), \text {f o r} i \in \mathcal {C} _ {2} ^ {(2)} \tag {14}
$$
Similar to the notation in Proposition 2, we denote the expectation of the original node features  $\mathbf{x}$  in two classes as  $\mathbb{E}_{c_1}[\mathbf{x}_i]$  and  $\mathbb{E}_{c_2}[\mathbf{x}_i]$ . Similarly, we denote the expectation of the aggregated features as  $\mathbb{E}_{c_1}[\mathbf{f}_i]$  and  $\mathbb{E}_{c_2}[\mathbf{f}_i]$ .
Proposition 3.  $(\mathbb{E}_{c_1}[\mathbf{x}_i],\mathbb{E}_{c_2}[\mathbf{x}_i])$  and  $(\mathbb{E}_{c_1}[\mathbf{f}_i],\mathbb{E}_{c_2}[\mathbf{f}_i])$  share the same middle point. When satisfying  $p^{(1)} > q^{(2)}$ ,  $\mathbb{E}_{c_1}[\mathbf{x}_i] - \mathbb{E}_{c_2}[\mathbf{x}_i]$  and  $\mathbb{E}_{c_1}[\mathbf{f}_i] - \mathbb{E}_{c_2}[\mathbf{f}_i]$  share the opposite direction. Specifically, the middle point  $\mathbf{m}$  and the shared direction  $\mathbf{w}$  are as follows:  $\mathbf{m} = (\pmb{\mu}_1 + \pmb{\mu}_2) / 2$ , and  $\mathbf{w} = (\pmb{\mu}_1 - \pmb{\mu}_2) / \| \pmb{\mu}_1 - \pmb{\mu}_2\|_2$ . When satisfying  $q^{(2)} > p^{(1)}$ ,  $\mathbb{E}_{c_1}[\mathbf{x}_i] - \mathbb{E}_{c_2}[\mathbf{x}_i]$  and  $\mathbb{E}_{c_1}[\mathbf{h}_i] - \mathbb{E}_{c_2}[\mathbf{h}_i]$  are in same directions. It indicates that the largest-margin linear model still shares the same discriminative boundary which flips the prediction on class  $c_1$  and  $c_2$ .
Notably, with the assumption  $p^{(1)} + q^{(1)} = p^{(2)} + q^{(2)}$ , the linear classifier can be generally to two cases according to whether the decision boundary direction flips: (1) The decision boundary direction unchanged with  $q^{(2)} > p^{(1)} > p^{(2)} > q^{(1)}$  (2) The decision boundary direction flipped with  $p^{(1)} > q^{(2)} > q^{(1)} > p^{(2)}$ . Notably, the two cases are symmetric with the same conclusion and proof logic. We will focus on the unchanged case with  $q^{(2)} > p^{(1)} > p^{(2)} > q^{(1)}$ .
$$
\mathcal {P} = \left\{\mathbf {x} \mid \mathbf {w} ^ {\top} \mathbf {x} - \mathbf {w} ^ {\top} \left(\boldsymbol {\mu} _ {1} + \boldsymbol {\mu} _ {2}\right) / 2 \right\} \tag {15}
$$
$$
\begin{array}{l} \mathbb {P} (\mathbf {x} _ {i} \text {i s m i s - c l a s s i f i e d}) = \mathbb {P} \left(\mathbf {w} ^ {\top} \mathbf {x} _ {i} + \mathbf {b} \leq 0\right) \text {f o r} i \in \mathcal {C} _ {1} \\ \mathbb {P} (\mathbf {f} _ {i} \text {i s m i s - c l a s s i f i e d}) = \mathbb {P} \left(\mathbf {w} ^ {\top} \mathbf {f} _ {i} + \mathbf {b} <   0\right) \text {f o r} i \in \mathcal {C}. \end{array} \tag {16}
$$
$$
\mathbb {P} \left(\mathbf {f} _ {i} \text {i s m i s - c l a s s i f i e d}\right) = \mathbb {P} \left(\mathbf {w} ^ {\top} \mathbf {f} _ {i} + \mathbf {b} \leq 0\right) \text {f o r} i \in \mathcal {C} _ {1},
$$
$$
\mathbf {f} _ {i} ^ {\prime} \sim N \left(\frac {\sqrt {d _ {i}} \left(p ^ {(1)} \boldsymbol {\mu} _ {1} + q ^ {(1)} \boldsymbol {\mu} _ {2}\right)}{p ^ {(1)} + q ^ {(1)}}, \mathbf {I}\right), \text {f o r} i \in \mathcal {C} _ {1}; \mathbf {f} _ {i} ^ {\prime} \sim N \left(\frac {\sqrt {d _ {i}} \left(p ^ {(2)} \boldsymbol {\mu} _ {1} + q ^ {(2)} \boldsymbol {\mu} _ {2}\right)}{p ^ {(2)} + q ^ {(2)}}, \mathbf {I}\right), \text {f o r} i \in \mathcal {C} _ {2} \tag {17}
$$
Given the scale in Eq. equation 17, the decision boundary for  $\mathbf{f}_i^\prime$  is consequently shifted to  $\mathbf{w}^{\top}\mathbf{f}^{\prime} + \sqrt{d_i}\mathbf{b} = 0$ . Now, considering that  $\mathbf{x}_i$  and  $\mathbf{f}_i^\prime$  have the same variance, we can compare the misclassification probabilities by merely comparing the distance from their expected values to their corresponding decision boundaries. Specifically, the distances can be described as follows:
$$
\operatorname {d i s} _ {\mathbf {x} _ {i}} = \frac {\left\| \boldsymbol {\mu} _ {1} - \boldsymbol {\mu} _ {2} \right\| _ {2}}{2} \\ \sqrt {T} (2) \quad (1) \quad \parallel \quad \parallel \quad \parallel \quad \sqrt {T} (1) \quad (2) \quad \parallel \quad \parallel \tag {18}
$$
$$
\operatorname {d i s} _ {\mathbf {f} _ {i} ^ {\prime}} = \frac {\sqrt {d _ {i}} | p ^ {(2)} - q ^ {(1)} |}{p ^ {(1)} + q ^ {(1)}} \cdot \frac {\| \boldsymbol {\mu} _ {1} - \boldsymbol {\mu} _ {2} \| _ {2}}{2} = \frac {\sqrt {d _ {i}} | p ^ {(1)} - q ^ {(2)} |}{p ^ {(1)} + q ^ {(1)}} \cdot \frac {\| \boldsymbol {\mu} _ {1} - \boldsymbol {\mu} _ {2} \| _ {2}}{2}
$$
The larger the distance from the expectation to the decision boundary indicates a smaller the misclassification probability. Notably,  $q^{(1)} - p^{(2)} = p^{(1)} - q^{(2)}$  with the assumption  $p^{(1)} + p^{(2)} = q^{(1)} + q^{(2)}$ . Therefore, when  $\mathrm{dis}_{\mathbf{f}_i'} > \mathrm{dis}_{\mathbf{x}_i}$ ,  $\mathbf{f}_i'$  has a lower probability of being misclassified than  $\mathbf{x}_i$ . By comparing the two distances, we conclude that when  $d_i > \frac{\left(p^{(1)} + q^{(1)}\right)^2}{\left(p^{(1)} - q^{(2)}\right)^2}$ , the mean aggregated feature  $\mathbf{f}_i'$  exhibits a lower probability of misclassification compared to the original node feature  $\mathbf{x}_i$ .
$$
\mathbb {P} \left(\mathbf {f} _ {i} \text {i s m i s - c l a s s i f i e d}\right) <   \mathbb {P} \left(\mathbf {x} _ {i} \text {i s m i s - c l a s s i f i e d}\right) \text {i f} d _ {i} > \left(\frac {p ^ {(1)} + q ^ {(1)}}{p ^ {(1)} - q ^ {(2)}}\right) ^ {2} \tag {19}
$$
A comparison between linear separability within classes and between classes Notice that the only difference with the one in the same pattern one is that the denominator is  $|p^{(1)} - q^{(2)}|^{2}$ , smaller than the  $|p^{(1)} - q^{(1)}|^{2}$  since  $q^{(1)} > q^{(2)}$ . It indicates that only nodes with higher degrees. For instance, when  $p^{(1)} = 0.9$ ,  $q^{(1)} = 0.1$  and  $p^{(2)} = 0.2$ ,  $q^{(2)} = 0.8$ , the aggregated features can only show better linear separability with  $d_{i} > 100$  when nodes are from different structural patterns. The above condition can be hardly met in real-world scenarios. Based on our analysis, we can draw the conclusion that when nodes are in the same pattern, aggregation can show improved linear separability on  $(p^{(1)} + q^{(1)})^2 / |p^{(1)} - q^{(1)}|^2 < d_i < (p^{(1)} + q^{(1)})^2 / |p^{(1)} - q^{(2)}|^2$  while the ones in the different patterns cannot. It indicates that aggregation can help more when nodes are in the same pattern, however, show limitation when nodes are from different patterns.
E Proof details of the conditional probability difference for nodes with the same feature but different structural patterns
In section 3.1, we examine the discrepancy between nodes 1 and 2 with the same aggregated feature  $\mathbf{f}_1 = \mathbf{f}_2$  but different structural patterns. Typically, we examine the discrepancy with the probability difference of nodes 1 and 2 in class  $c_{1}$ , denoted as  $|p_1(y_u = c_1|\mathbf{f}_u) - p_2(y_v = c_1|\mathbf{f}_v)|$ .  $p_1$  and  $p_2$  are the conditional probabilities of node labels in  $c$  given the feature  $\mathbf{f}$  for nodes on homophilic and heterophilic structural patterns, respectively. The lemma is shown as follows:
Lemma 1. With assumptions (1) A balance class distribution with  $\mathbf{P}(\mathbf{Y} = 1) = \mathbf{P}(\mathbf{Y} = 0)$  and (2) aggregated feature distribution shares the same variance  $\sigma \mathbf{I}$ . When nodes  $u$  and  $v$  have the same aggregated features  $\mathbf{f}_u = \mathbf{f}_v$  but different structural patterns,  $(p^{(1)}, q^{(1)})$  and  $(p^{(2)}, q^{(2)})$ , we can have:
$$
\left| \mathbf {P} _ {1} \left(y _ {u} = c _ {1} \mid \mathbf {f} _ {u}\right) - \mathbf {P} _ {2} \left(y _ {v} = c _ {1} \mid \mathbf {f} _ {v}\right) \right| \leq \frac {\rho^ {2}}{\sqrt {2 \pi} \sigma} \left| h _ {u} - h _ {v} \right| \tag {20}
$$
$\rho = \|\pmb{\mu}_1 - \pmb{\mu}_2\|$  is the original feature separability, independent with structure.  $\mathbf{P}_1$  and  $\mathbf{P}_2$  are the conditional probability of  $y = c_1$  given the feature  $\mathbf{f}$  on structural patterns  $(p^{(1)}, q^{(1)})$  and  $(p^{(2)}, q^{(2)})$ ,
respectively. Lemma 1 implies that nodes with a small homophily ratio difference  $|h_1 - h_2|$  are likely to share the same class, and vice versa.
We first remind the assumptions of the CSBM-S model. There are two assumptions on the models: (1) Nodes from different components share the same feature distribution. In other words, node features within the same class are sampled from the same Gaussian distribution, regardless of different structural patterns. (2) Nodes from different components share similar degree distribution with  $p^{(1)} + q^{(1)} = p^{(2)} + q^{(2)}$ . Notably, our conclusions are still valid without the above assumptions. Those assumptions are not strictly necessary but employed for the elegant expression.
Based on the neighborhood distributions, the mean aggregated features  $\mathbf{F} = \mathbf{D}^{-1}\mathbf{A}\mathbf{X}$  obtained follow Gaussian distributions on both homophilic and heterophilic node subgroups.
$$
\mathbf {f} _ {i} ^ {(j)} \sim N \left(\frac {p ^ {(j)} \boldsymbol {\mu} _ {1} + q ^ {(j)} \boldsymbol {\mu} _ {2}}{p ^ {(j)} + q ^ {(j)}} \text {,} \sigma \mathbf {I}\right), \text {f o r} i \in \mathcal {C} _ {1} ^ {(j)}; \mathbf {f} _ {i} ^ {(j)} \sim N \left(\frac {q ^ {(j)} \boldsymbol {\mu} _ {1} + p ^ {(j)} \boldsymbol {\mu} _ {2}}{p ^ {(j)} + q ^ {(j)}} \text {,} \sigma \mathbf {I}\right), \text {f o r} i \in \mathcal {C} _ {2} ^ {(j)} \tag {21}
$$
Where  $\mathcal{C}_i^{(j)}$  is the node subgroups with structural pattern with  $(p^{(j)},q^{(j)})$  in label  $i$ . Notably, there are typically homophilic pattern with  $p^{(1)} > q^{(1)}$  and heterophilic pattern with  $p^{(2)} < q^{(2)}$ . To simply the assumption for an elegant expression, we denote the aggregated feature mean,  $\frac{p^{(j)}\mu_1 + q^{(j)}\mu_2}{p^{(j)} + q^{(j)}}$  as  $\boldsymbol{\mu}_{ij}^{\prime}$ , where  $i$  and  $j$  correspond to the class and the structural pattern, respectively. For instance,  $\boldsymbol{\mu}_{11}^{\prime}$  represents the mean of nodes in class  $c_{1}$  with the homophilic pattern. We then show the proof details as follows:
Proof. The conditional probability of class  $c_{1}$  given the aggregated feature  $f$ ,  $\mathbf{P}_1(\mathbf{y} = c_1|\mathbf{f}_1)$  can be derived with the Bayes theorem: