I S H ETEROPHILY A R EAL N IGHTMARE F OR G RAPH N EURAL N ETWORKS ON P ERFORMING N ODE C LASSIFICATION ?
Anonymous authors
Paper under double-blind review
A BSTRACT
Graph Neural Networks (GNNs) extend basic Neural Networks (NNs) by using the graph structures based on the relational inductive bias (homophily assumption).
Though GNNs are believed to outperform NNs in real-world tasks, performance advantages of GNNs over graph-agnostic NNs seem not generally satisfactory.
Heterophily has been considered as a main cause and numerous works have been put forward to address it.
In this paper, we ﬁrst show that not all cases of heterophily are harmful 1 for GNNs with aggregation operation.
Then, we propose new metrics based on a similarity matrix which considers the inﬂuence of both graph structure and input features on GNNs.
The metrics demonstrate advantages over the commonly used homophily metrics by tests on synthetic graphs.
From the metrics and the observations, we ﬁnd some cases of harmful heterophily can be addressed by diversiﬁcation operation.
With this fact and knowledge of ﬁlterbanks, we propose the Adaptive Channel Mixing (ACM) framework to adaptively exploit aggregation, diversiﬁcation and identity channels in each GNN layer to address harmful heterophily.
We validate the ACM-augmented baselines with 10 realworld node classiﬁcation tasks.
They consistently achieve signiﬁcant performance gain and exceed the state-of-the-art GNNs on most of the tasks without incurring signiﬁcant computational burden.
I NTRODUCTION
Deep Neural Networks (NNs) (LeCun et al., 2015) have revolutionized many machine learning areas, including image recognition (Krizhevsky et al., 2012),
speech recognition (Graves et al., 2013) and natural language processing (Bahdanau et al., 2014), etc.
One major strength is their capacity and effectiveness of learning latent representation from Euclidean data.
Recently, the focus has been put on its applications on non-Euclidean data (Bronstein et al., 2016), e.g., relational data or graphs.
Combining graph signal processing and convolutional neural networks (LeCun et al., 1998), numerous Graph Neural Networks (GNNs) (Scarselli et al., 2008; Defferrard et al., 2016; Hamilton et al., 2017; Velickovic et al., 2017; Kipf & Welling, 2016; Luan et al., 2019) have been proposed which empirically outperform traditional neural networks on graph-based machine learning tasks, e.g., node classiﬁcation, graph classiﬁcation, link prediction and graph generation, etc.
GNNs are built on the homophily assumption (McPherson et al., 2001), i.e., connected nodes tend to share similar attributes with each other (Hamilton, 2020), which offers additional information besides node features.
Such relational inductive bias (Battaglia et al., 2018) is believed to be a key factor leading to GNNs’ superior performance over NNs’ in many tasks.
Nevertheless, growing evidence shows that GNNs do not always gain advantages over traditional
NNs when dealing with relational data.
In some cases, even simple Multi-Layer Perceptrons (MLPs) can outperform GNNs by a large margin (Zhu et al., 2020b;
Liu et al., 2020; Luan et al., 2020b; Chien et al., 2021).
An important reason for the performance degradation is believed to be the heterophily problem, i.e., connected nodes tend to have different labels which makes the homophily assumption fail.
Heterophily challenge has received lots of attention recently and there are increasing number of models being put forward to address this problem (Zhu et al., 2020b;
Liu et al., 2020; Chien et al., 2021; Zhu et al., 2020a; Yan et al., 2021).
Contributions In this paper, we ﬁrst demonstrate that not all heterophilous graphs are harmful for aggregation-based GNNs and the existing metrics of homophily are insufﬁcient to decide whether the aggregation operation will make nodes less distinguishable or not.
By constructing a similarity matrix from backpropagation analysis, we derive new homophily metrics to depict how much GNNs are inﬂuenced by the graph structure and node features.
We show the advantage of our metrics over the existing metrics by comparing the ability of characterizing the performance of two baseline
GNNs on synthetic graphs of different levels of homophily.
According to the similarity matrix, we observe that diversiﬁcation operation is able to address some harmful heterophily cases, and based on which we propose Adaptive Channel Mixing (ACM)
GNN framework.
The experiments on the synthetic datasets, ablation studies and real-world datasets consistently show that the baseline GNNs augmented by ACM framework are able to obtain signiﬁcant performance boost on node classiﬁcation tasks on heterophilous graphs.
The rest of this paper is mainly organized as follows:
In section 2, we introduce the notation and the background knowledge.
In section 3, we conduct node-wise analysis on heterophily, derive new homophily metrics based on a similarity matrix and conduct experiments to show their advantages over the existing homophily metrics.
In section 4, we demonstrate the capability of diversiﬁcation operation on addressing some cases of harmful heterophily and propose the ACM-GNN framework to adaptively utilize the information from different ﬁlterbank channels for each node to address heterophily problem.
In section 5, we discuss the related works and clarify the differences with our method.
In section 6, we provide empirical evaluations on ACM framework, including ablation study and tests on real-world node classiﬁcation tasks.
P RELIMINARIES
We will introduce the related notation and background knowledge in this section.
We use bold fonts for vectors ( e.g., v ).
Suppose we have an undirected connected graph G =
( V , E , A ) , where V is the node set with |V| = N ; E is the edge set without self-loop; A ∈ R N × N is the symmetric adjacency matrix with A i,j = 1 iff e ij ∈ E , otherwise A i,j = 0 .
We use D to denote the diagonal degree matrix of G , i.e., D i,i =
d i = (cid:80)
j A i,j and use N i to denote the neighborhood set of node i , i.e.,
N i = { j : e ij ∈ E} .
A graph signal is a vector x ∈ R N deﬁned on V , where x i is deﬁned on the node i .
We also have a feature matrix X ∈ R N × F , whose columns are graph signals and whose i -th row
X i, : is a feature vector of node i .
We use Z ∈ R N × C to denote the label encoding matrix, whose i -th row Z i, : is the one-hot encoding of the label of node i .
2. G RAPH L APLACIAN , A FFINITY M ATRIX AND T HEIR V ARIANTS
The (combinatorial) graph Laplacian is deﬁned as L = D − A , which is Symmetric Positive SemiDeﬁnite (SPSD) (Chung & Graham, 1997).
Its eigendecomposition gives L = U Λ U T , where the columns u i of U ∈ R N × N are orthonormal eigenvectors, namely the graph Fourier basis , Λ = diag
( λ 1 , . . .
, λ N ) with λ 1 ≤ · · · ≤ λ N , and these eigenvalues are also called frequencies .
The graph
Fourier transform of the graph signal x is deﬁned as x F = U − 1 x =
U T x = [ u T 1 x , . .
, u TN x ] T , where u
Ti x is the component of x in the direction of u i .
In additional to L , some variants are also commonly used, e.g., the symmetric normalized Laplacian
L sym =
D − 1 / 2 LD − 1 / 2 =
I − D − 1 / 2 AD − 1 / 2 and the random walk normalized Laplacian L
rw =
D − 1 L =
I − D − 1 A .
The graph Laplacian and its variants can be considered as high-pass ﬁlters.
The afﬁnity (transition) matrices can be derived from the Laplacians, e.g.,
A rw =
I − L
rw =
D − 1 A , A sym =
I − L sym
=
D − 1 / 2 AD − 1 / 2 and are considered to be low-pass (LP) ﬁlters (Maehara, 2019).
Their eigenvalues satisfy λ
i ( A rw )
=
λ i ( A sym )
=
1 − λ i ( L sym )
= 1 − λ i ( L rw ) ∈ ( − 1 , 1] .
Applying the renormalization trick (Kipf & Welling, 2016) to afﬁnity and Laplacian matrices respectively leads to ˆ A sym =
˜ D − 1 / 2 ˜
A ˜ D − 1 / 2 and ˆ L sym
=
I − ˆ A sym , where ˜
A ≡ A + I and ˜ D ≡ D + I .
The renormalized afﬁnity matrix essentially adds a self-loop to each node in the graph, and is widely used in Graph Convolutional Network (GCN) (Kipf & Welling, 2016) as follows,
[Equation]
where W 0 ∈ R F × F 1 and W 1 ∈ R F 1 × O are learnable parameter matrices.
GCN can be trained by minimizing the following cross entropy loss
[Equation]
where log( · ) is a component-wise logarithm operation.
The random walk renormalized matrix ˆ A rw =
˜ D − 1 ˜ A , which shares the same eigenvalues as ˆ A sym , can also be applied in GCN.
The corresponding Laplacian is deﬁned as ˆ L rw =
I − ˆ A rw .
The matrix ˆ
A rw is essentially a random walk matrix and behaves as a mean aggregator that is applied in spatial-based GNNs (Hamilton et al., 2017; Hamilton, 2020).
To bridge the spectral and spatial methods, we use ˆ A rw in the paper.
2. M ETRICS OF H OMOPHILY
The metrics of homophily are deﬁned by considering different relations between node labels and graph structures deﬁned by adjacency matrix.
There are three commonly used homophily metrics: edge homophily (Abu-El-Haija et al., 2019; Zhu et al., 2020b), node homophily (Pei et al., 2020), and class homophily (Lim et al., 2021) 2 deﬁned as follows: [Equation] Z v, : } (cid:12) (cid:12) (cid:80) v [Equation]
v |
[Equation]
v,k = [Equation] } d v (3) where [ a ] + = max( a, 0) ; h k is the class-wise homophily metric (Lim et al., 2021).
They are all in the range of [0 , 1] and a value close to 1 corresponds to strong homophily while a value close to 0 indicates strong heterophily.
H edge ( G ) measures the proportion of edges that connect two nodes in the same class; H node ( G ) evaluates the average proportion of edge-label consistency of all nodes; H class ( G ) tries to avoid the sensitivity to imbalanced class, which can cause H edge ( G ) misleadingly large.
The above deﬁnitions are all based on the graph-label consistency and imply that the inconsistency will cause negative effect to the performance of GNNs.
With this in mind, we will show a counter example to illustrate the insufﬁciency of the above metrics and propose new metrics in the following section.
A NALYSIS OF H ETEROPHILY 3. M OTIVATION AND A GGREGATION H OMOPHILY
[Figure]
Figure Heterophily is believed to be harmful for message-passing based GNNs (Zhu et al., 2020b; Pei et al., 2020; Chien et al., 2021) because intuitively features of nodes in different classes will be falsely mixed and this will lead nodes indistinguishable (Zhu et al., 2020b).
Nevertheless, it is not always the case, e.g., the bipartite graph shown in Figure 1 is highly heterophilous according to the existing homophily metrics in equation 3, but after mean aggregation (operated by mean aggregator), the nodes in classes 1 and 2 only exchange colors and are still distinguishable.
Authors in (Chien et al., 2021) also point out the insufﬁciency of H node by examples to show that different graph typologies with the same H node ( G ) can carry different label information.
To analyze to what extent the graph structure can affect the output of a GNN, we ﬁrst simplify GCN by removing its nonlinearity as (Wu et al., 2019).
Let ˆ A ∈ R N × N denote a general aggregation operator.
Then, equation 1 can be simpliﬁed as,
[Equation]
After each gradient decent step ∆ W = γ d L dW , where γ is the learning rate, the update of Y (cid:48) will be (see Appendix C for derivation),
[Equation]
where S ( ˆ A, X ) ≡ ˆ AX ( ˆ AX ) T is a post-aggregation node similarity matrix, Z − Y is the prediction error matrix.
The update direction of node i is essentially a weighted sum of the prediction error, i.e., ∆( Y
[Equation]
[Equation]
Instead of measuring the graph-label consistency, we study the effect of heterophily by considering the post-aggregation node similarity.
To this end, we ﬁrst deﬁne the aggregation similarity score as follows.
Deﬁnition 1.
Aggregation similarity score
[Equation]
where Mean u ( {·} ) takes the average over u of a given multiset of values or variables.
S agg ( S ( ˆ A, X )) measures the proportion of nodes v ∈ V that will put relatively larger similarity weights on nodes in the same class than in other classes after aggregation.
It is easy to see that S agg ( S ( ˆ A, X )) ∈ [0 , 1] .
But in practice, we observe that in most datasets, we will have
S agg ( S ( ˆ A, X )) ≥ 0 .
Based on this observation, we rescale equation 6 to the following modiﬁed aggregation similarity for practical usage,
[Equation]
In order to measure the consistency between labels and graph structures without considering node features and make a fair comparison with the existing homophily metrics in equation 3, we deﬁne the graph ( G ) aggregation ( ˆ A ) homophily and its modiﬁed version as
[Equation]
In practice, we will only check H agg ( G )
when H M agg ( G ) =
As Figure 1 shows, when ˆ A
= ˆ
A rw ,
H agg ( G ) =
H M agg ( G ) = 1 .
Thus, this new metric reﬂects the fact that nodes in classes 1 and 2 are still highly distinguishable after aggregation, while other metrics mentioned before fail to capture such information and misleadingly give value 0.
This shows the advantage of H agg ( G ) and H M agg ( G ) by additionally exploiting information from aggregation operator ˆ A and the similarity matrix.
To comprehensively compare H M agg ( G ) with the metrics in equation 3 on how they can reveal the inﬂuence of graph structure on GNN performance, we generate synthetic graphs with different homophily levels and evaluate SGC (Wu et al., 2019) and GCN (Kipf & Welling, 2016) on them in the next subsection.
3. E MPIRICAL E VALUATION AND C OMPARISON ON S YNTHETIC G RAPHS
In this subsection, we conduct experiments on synthetic graphs with different H M edge ( G ) to empirically verify the effectiveness of H M agg ( G ) and compare it with the existing metrics.
Data Generation & Experimental Setup We ﬁrst generate 280 graphs in total with 28 edge homophily levels varied from 0.005 to 0.95, each corresponding to 10 graphs.
For every generated graph, we have 5 classes with 400 nodes in each class.
For nodes in each class, we randomly generate 800 intra-class edges and [ 800 H edge ( G ) − 800 ] inter-class edges.
The features of nodes in each class are sampled from node features in the corresponding class of 6 base datasets ( Cora, CiteSeer, PubMed,
Chameleon, Squirrel, Film ).
Nodes are randomly splitted into 60%/20%/20% for train/validation/test.
We train 1-hop SGC ( sgc-1 ) (Wu et al., 2019) and GCN (Kipf & Welling, 2016) on the synthetic graphs 3 .
For each value of H edge ( G ) , we take the average test accuracy and standard deviation of runs over 10 generated graphs.
For each generated graph, we also calculate its H node ( G ) , H class ( G ) and H M agg ( G ) .
Model performance with respect to different homophily values are shown in Figure 2.
[Figure]
Comparison of Homophily Metrics
The performance of SGC-1 and GCN are expected to be monotonically increasing with a proper and informative homophily metric.
However, Figure 2(a)(b)(c) show that the performance curves under H edge ( G ) , H node ( G ) and H class ( G ) are U -shaped 4 , while
Figure 2(d) reveals a nearly monotonic curve with a little perturbation around 1.
This indicates that H M agg ( G ) can describe how the graph structure affects the performance of SGC-1 and GCN more appropriately and adequately than the existing metrics.
(See more discussion on aggregation homophily and theoretical results for regular graphs in Appendix B.)
A DAPTIVE C HANNEL M IXING (ACM) F RAMEWORK
[Figure]
-frequency nal, which can be extracted by high-pass ﬁlter, is empirically shown to be useful for addressing heterophily problem (Luan et al., 2020b;
Chien et al., 2021; Bo et al., 2021). In this section, based on the similarity matrix proposed in equation 5, we will ﬁgure out how diversiﬁcation operation i.e., highpass (HP) ﬁlter, is potentially capable to address some cases of harmful heterophily.
From the node-wise analysis, we argue that aggregation (LP ﬁlter) and diversiﬁcation (HP ﬁlter) should be combined together as a ﬁlterbank (Ekambaram, 2014) for feature extraction and different nodes may have different needs for the information processed by different ﬁlters.
Based on the above argument, we generalize ﬁlterbank method and propose Adaptive Channel Mixing framework in subsection 4.2 to address heterophily challenge.
4. H OW D IVERSIFICATION O PERATION H ELPS WITH H ARMFUL H ETEROPHILY
We ﬁrst consider the example shown in Figure 3.
From S ( ˆ A, X ) , we can see that nodes 1,3 assign relatively large positive weights to nodes in class 2 after aggregation, which will make node 1, hard to be distinguished from nodes in class 2.
Despite the fact, we can still distinguish nodes 1,3 and 4,5,6,7 by considering their neighborhood differences: nodes 1,3 are different from most of their neighbors while nodes 4,5,6,7 are similar to most of their neighbors.
This indicates, in some cases, although some nodes become similar after aggregation, they are still distinguishable via their surrounding dissimilarities.
This leads us to use diversiﬁcation operation , i.e., HP ﬁlter I − ˆ A (Ekambaram, 2014) to extract the information of neighborhood differences and address harmful heterophily.
As S ( I − ˆ A, X ) in Figure 3 shows, nodes 1,3 will assign negative weights to nodes 4,5,6,7 after diversiﬁcation operation, i.e., nodes 1,3 treat nodes 4,5,6,7 as negative samples and will move away from them during backpropagation.
This example reveals that there exist some cases that diversiﬁcation operation is helpful while aggregation operation is not.
Based on this observation, we ﬁrst deﬁne diversiﬁcation distinguishability of a node and graph diversiﬁcation distinguishability value as follows to measure the proportion of nodes that diversiﬁcation operation is potentially helpful for.
Deﬁnition 2.
Diversiﬁcation Distinguishability (DD) based on S ( I − ˆ A, X ) .
Given S ( I − ˆ A, X ) , a node v is diversiﬁcation distinguishable if the following two conditions are satisﬁed at the same time,
[Equation]
Then, graph diversiﬁcation distinguishability value is deﬁned as
[Equation]
We can see that DD ˆ A,X ( G ) ∈ [0 , 1] .
Based on deﬁnition 2 , the effectiveness of diversiﬁcation operation on addressing heterophily can be proved for binary classiﬁcation problems under certain conditions, leading us to:
Theorem 1.
(See Appendix E for proof).
For a binary classiﬁcation problem, i.e., C = 2 , suppose X =
Z, ˆ A
= ˆ
A rw .
Then for any I − ˆ A rw , all nodes are diversiﬁcation distinguishable and
DD ˆ A,Z ( G ) = 1 .
Theorem 1 theoretically demonstrates the importance of diversiﬁcation operation on handling heterophily.
Combined with aggregation operation, we can get a ﬁlterbank which uses both LP and HP ﬁlters to distinctively extract the low- and high-frequency information from graph signals.
We will introduce ﬁlterbank in the next subsection.
4. F ILTERBANK AND A DAPTIVE C HANNEL M IXING (ACM) F RAMEWORK
Filterbank For the graph signal x deﬁned on G , a 2-channel linear (analysis) ﬁlterbank (Ekambaram, 2014) 5 includes a pair of low-pass (LP) and high-pass (HP) ﬁlters H LP , H HP , where H LP and
H HP retain the low-frequency and high-frequency content of x , respectively.
Most existing GNNs are under uni-channel ﬁltering architecture (Kipf & Welling, 2016; Velickovic et al., 2017; Hamilton et al., 2017) with either H LP or H HP channel that only partially preserves the input information.
Unlike the uni-channel architecture, ﬁlterbanks with H LP + H HP = I will not lose any information of the input signal, i.e., perfect reconstruction property (Ekambaram, 2014).
Generally, the Laplacian matrices ( L sym , L rw , ˆ L sym , ˆ L rw ) can be regarded as HP ﬁlters (Ekambaram, 2014) and afﬁnity matrices ( A sym , A rw , ˆ A sym , ˆ A rw ) can be treated as LP ﬁlters (Maehara, 2019;
Hamilton, 2020).
Moreover, it can be interpreted that MLPs uses the identity ﬁlterbank with H LP =
I and H HP = 0 that satisfy H LP
+ H HP =
I +
= I .
From Figure 3, we also observe that different nodes may have different needs for the information from different channels, e.g., nodes 1,3 demand information from HP channel while node 2 only needs information from LP channel.
To adaptively leverage the LP, HP and identity channels in
GNNs, we propose the Adaptive Channel Mixing (ACM) architecture in the following part.
Adaptive Channel Mixing (ACM) Framework ACM framework can be applied to lots of baseline
GNNs and in this part, we use GCN as an example to introduce ACM framework in matrix form.
We use H LP and H HP to represent general LP and HP ﬁlters.
The ACM framework includes 3 steps as follows,
Step 1.
Feature Extraction for Each Channel:
[Equation]
Step 2.
Row-wise Feature-based Weight Learning
[Equation]
Step 3.
Node-wise Adaptive Channel Mixing:
[Equation]
[Equation]
The framework with option 1 in step 1 is called ACM framework and with option 2 is named ACMII framework.
In step 1, ACM-GCN and ACMII-GCN implement distinct feature extraction for 3 channels by a set of ﬁlterbank and 3 ﬁltered components H lL , H lH , H lI are obtained.
To adaptively exploit information from each channel, ACM-GCN and ACMII-GCN ﬁrst extract nonlinear information from the ﬁltered signals, then use W l Mix to learn which channel is important or not for each node, leading to the row-wise weight vectors α lL , α lH , α lI ∈ R N × 1 whose i -th elements are the weights for the i -th node. These three vectors are then used as weights in deﬁning the updated H l in step 3.
See Appendix F for the performance comparison with basline models on synthetic datasets.
Complexity Number of learnable parameters in layer l of ACM-GCN and ACMII-GCN is 3 F l − 1 ( F l + 1) + 9 , while it is F l − 1 F l in GCN.
The computation of step 1-3 takes NF l (8 + 6 F l − 1 ) + 2 F l ( nnz ( H LP )+ nnz ( H HP ))
+18 N ﬂops, while GCN layer takes 2 NF l − 1 F l +2 F l ( nnz ( H LP ))
ﬂops, where nnz ( · ) is the number of non-zero elements.
An ablation study and a detailed comparison on running time is conducted in section 6.1.
Limitations of Diversiﬁcation Operation Just like any other method, Diversiﬁcation operation may not work well in all harmful heterophily cases.
For example, when we have more than 2 classes and consider an imbalanced dataset where several small clusters with distinctive labels are densely connected to a large cluster.
In this case, the surrounding differences of nodes in small clusters are similar, i.e., the neighborhood differences are mainly from their connection to the same large cluster, and this possibly makes diversiﬁcation operation fail to discriminate them.
See a more detailed demonstration and discussion in Appendix G.
P RIOR W ORK
We discuss relevant work of GNNs on addressing heterophily challenge in this part.
Authors in (Abu-El-Haija et al., 2019) acknowledge the difﬁculty of learning on graphs with weak homophily and propose MixHop to extract features from multi-hop neighborhood to get more information.
Authors in (Hou et al., 2019) propose measurements based on feature smoothness and label smoothness that are potentially helpful to guide GNNs on dealing with heterophilous graphs.
Geom-GCN (Pei et al., 2020) precomputes unsupervised node embeddings and uses graph structure deﬁned by geometric relationships in the embedding space to deﬁne the bi-level aggregation process to handle heterophily.
H 2 GCN (Zhu et al., 2020b) combines 3 key designs to address heterophily: (1) ego- and neighbor-embedding separation; (2) higher-order neighborhoods; (3) combination of intermediate representations.
CPGNN (Zhu et al., 2020a) models label correlations by the compatibility matrix, which is beneﬁcial for heterophily settings, and propagates a prior belief estimation into GNNs by the compatibility matrix.
FBGNN (Luan et al., 2020b)
ﬁrst proposes to use ﬁlterbank to address heterophily problem, but it does not fully explain the insights behind HP ﬁlters and does not contain identity channel and node-wise channel mixing mechanism.
FAGCN (Bo et al., 2021) learns edgelevel aggregation weights as GAT (Velickovic et al., 2017) but allows the weights to be negative which enables the network to capture the high-frequency components in graph signals.
GPRGNN (Chien et al., 2021) uses learnable weights that can be both positive and negative for feature propagation, it allows GPRGNN to adapt heterophily structure of graph and is able to handle both high- and low-frequency parts of the graph signals.
(See Appendix J for a more comprehensive comparison between ACM-GNNs, ACMII-GNNs and FAGCN, GPRGNN.)
E XPERIMENTS ON R EAL -W ORLD D ATASETS
In this section
[Table]
we evaluate ACM and ACMII framework on real-world datasets.
We ﬁrst conduct ablation studies in subsection 6.1 to validate the effectiveness of different components.
Then, we compare with the state-of-the-arts (SOTA) models in subsection 6.2.
The hyperparameter searching range and computing resources for all experiments are attached in Appendix A
[Table]
6. A BLATION S TUDY & E FFICIENCY
[Table]
We investigate the effectiveness and efﬁciency of adding HP, identity channels and the adaptive mixing mechanism in the proposed framework by ablation study.
Speciﬁcally, we apply the components of ACM to SGC-1 (Wu et al., 2019)
and the components of ACM and ACMII to GCN (Kipf &
Welling, 2016) separately.
We run 10 times on each dataset with the same 60%/20%/20% random splits for train/validation/test used in (Chien et al., 2021) and report the average test accuracy as well as the standard deviation.
We also record the average running time per epoch (in milliseconds) to compare the efﬁciency.
We set the temperature T in equation 11 to be 3 .
From the results we can see that on most datasets, the additional HP and identity channels are helpful, even on strong homophily datasets, such as Cora, CiteSeer and PubMed .
The adaptive mixing mechanism also shows its advantage over the method that directly adds the three channels together.
This illustrates the necessity of learning to customize the channel usage adaptively for different nodes.
As for efﬁciency, we can see that the running time is approximately doubled under ACM and ACMII framework than the original model.
6. C OMPARISON WITH S TATE - OF - THE - ART M ODELS
Datasets & Experimental Setup
In this section, we implement SGC (Wu et al., 2019) with 1 hop and 2 hops (SGC-1, SGC-2), GCNII (Chen et al., 2020), GCNII ∗ (Chen et al., 2020), GCN (Kipf &
Welling, 2016) and snowball networks with 2 and 3 layers (snowball-2, snowball-3) and apply them
[Figure]
amount of improvement of the best ACM-baseline and ACM-baseline over the SOTA models.
See Appendix H for a detailed discussion of the relation between H M agg and the performance of GNNs.
in ACM or ACMII framework 7 : we use ˆ A rw as LP ﬁlter and the corresponding HP ﬁlter is I − ˆ A rw
and they are deterministic. We compare them with several baseline and SOTA GNN models: MLP with 2 layers (MLP-2), GAT (Velickovic et al., 2017), APPNP (Klicpera et al., 2018), GPRGNN (Chien et al., 2021), H 2 GCN (Zhu et al., 2020b), MixHop (Abu-El-Haija et al., 2019), GCN+JK (Kipf & Welling, 2016; Xu et al., 2018; Lim et al., 2021), GAT+JK (Velickovic et al., 2017; Xu et al., 2018; Lim et al., 2021), FAGCN (Bo et al., 2021) GraphSAGE (Hamilton et al., 2017) and Geom-GCN (Pei et al., 2020).
Besides the 9 benchmark datasets Cornell , Wisconsin , Texas , Film ,
Chameleon , Squirrel , Cora , Citeseer and Pubmed used in (Rozemberczki et al., 2021; Pei et al., 2020), we further test the above models on a new benchmark dataset, Deezer-Europe , that is proposed in (Rozemberczki & Sarkar, 2020).
On each dataset used in (Rozemberczki et al., 2021; Pei et al., 2020), we test the models 10 times following the same early stopping strategy, the same random data splitting method 8 and Adam (Kingma & Ba, 2014) optimizer as used in GPRGNN (Chien et al., 2021).
For Deezer-Europe , we test the above models 5 times with the same early stopping strategy, the same ﬁxed splits and AdamW (Loshchilov & Hutter, 2017) used in (Lim et al., 2021).
To better visualize the performance boost and the comparison with SOTA models, in Figure 4, we plot the bar charts of the test accuracy of SOTA models, 3 selected baselines (GCN, snowball-2, snowball-3) and their ACM and ACMII augmented models on 6 most commonly used benchmark heterophily datasets (See table 4 in Appendix A.3 for the full results and comparison).
We can see that after being applied in ACM or ACMII framework, the performance of the 3 baseline models are signiﬁcantly boosted on all tasks and can achieve SOTA performance.
Especially on Cornell, Texas,
Film and Squirrel , the augmented models signiﬁcantly outperform the current SOTA models.
Overall,
It suggests that ACM or ACMII framework can help GNNs to generalize better on node classiﬁcation tasks on heterophilous graphs.
A H YPERPARAMETERS & D ETAILS OF T HE E XPERIMENTS A. H YPERPARAMETER S EARCHING R ANGE FOR S YNTHETIC E XPERIMENTS Hyperparameter
Searching Range
for
Synthetic
Experiments
Models\Hyperparameters
[Table] [Table]
[Table]
A. H YPERPARAMETER S EARCHING R ANGE FOR A BLATION S TUDY
[Table]
Study
[Table]
A. F ULL R ESULTS OF T HE C OMPARISON WITH SOTA M ODELS
The main results of the full sets of experiments 9 with statistics of datasets are summarized in Table 4, where we report the mean accuracy and standard deviation.
We can see that after applied in ACM or
ACMII framework, the performance of baseline models are boosted on almost all tasks and achieve
SOTA performance on 8 out of 10 datasets.
Especially, ACMII-GCN performs the best in terms of average rank (3.40) across all datasets.
Overall, It suggests that ACM or ACMII framework can signiﬁcantly increase the performance of GNNs on node classiﬁcation tasks on heterophilous graphs.
A. H YPERPARAMETER S EARCHING R ANGE FOR GNN S ON R EAL - WORLD D ATASETS
See table 5 for the hyperparameter seaching range of baseline GNNs, ACM-GNNs, ACMII-GNNs and several SOTA models
. A. O PTIMAL H YPERPARAMETERS FOR B ASELINES AND ACM(II)-GNN S ON R EAL - WORLD T ASKS
See the reported optimal hyperparameters for baseline GNNs in table 6 and for ACM-GNNs and
ACMII-GNNs in table 7.
[Table] [Table]
A. D ETAILS OF THE I MPLEMENTATION OF ACM AND ACMII F RAMEWORKS
Unlike some baseline GNN models, in ACM(II) framework, we ﬁrst use dropout operation over the input data.
The implementation of ACM(II)-GCN and ACM(II)-snowball is straightforward, but
SGC-1, SGC-2, GCNII and GCNII* are not able to be applied under ACM(II) framework and we will make an explanation as follows.
• SGC-1 and SGC-2: SGC does not contain nonlinearity, so the option 1 and option 2 in step 1 of equation 11 is the same for ACM-SGC and ACMII-SGC.
Thus, we only implement ACM-SGC.
• GCNII and GCNII*: GCNII: H ( (cid:96) +1)
= σ (cid:16)(cid:16) (1 − α (cid:96) )
ˆ AH ( (cid:96) )
+ α (cid:96) H (0) (cid:17) (cid:16) (1 − β (cid:96) )
I n + β (cid:96) W ( (cid:96) )
(cid:17)(cid:17) GCNII*: H ( (cid:96) +1)
=
σ (cid:16) (1 − α (cid:96) )
ˆ AH ( (cid:96) )
(cid:16) (1 − β (cid:96) )
I n + β (cid:96) W ( (cid:96) ) 1 (cid:17)
+ + α (cid:96) H (0) (cid:16) (1 − β (cid:96) )
I n + β (cid:96) W ( (cid:96) ) 2 (cid:17)(cid:17)
From the above architecture of GCNII and GCNII ∗ we cam see that, without major modiﬁcation, GCNII and GCNII* are hard to be put into ACMII framework.
In ACMII framework, before apply ˆ A , we ﬁrst implement a nonlinear feature extractor σ ( H (cid:96) W ( (cid:96) ) ) .
But in GCNII and GCNII*, before multiplying W (cid:96) ( or W (cid:96) 1 , W (cid:96) 2 ) to extract features, we need to add another term including H (0) , which are not ﬁltered by ˆ A .
This makes the order of aggregator ˆ A and nonlinear extractor unexchangable and thus, incompatible with ACMII framework.
So we did not implement GCNII and GCNII* in ACMII framework.
The open source code is attached in the supplementary material.
A. C OMPUTING R ESOURCES
For all experiments on synthetic datasets and real-world datasets, we use NVidia V100 GPUs with 16/32GB GPU memory, 8-core CPU, 16G Memory.
The software implementation is based on
PyTorch and PyTorch Geometric (Fey & Lenssen, 2019).
[Table]
[Figure] [Table] [Figure] [Figure]
[Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Table]
[Figure]
max -
[Figure]
[Table]
[Figure]
[Table]
.372ms/
[Table]
297s
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table] [Table]
[Figure]
cat
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
Cora
SGC-
[Table] [Table]
B E XPERIMENTAL S ETUP AND F URTHER D ISCUSSION ON S YNTHETIC G RAPHS B. D ETAILED D ESCRIPTION OF D ATA G ENERATION P ROCESS
• For each node v , we ﬁrst randomly generate its degree d v .
• Given d v , we sample hd v intra-class edges and (1 − h )
d v inter-class edges.
More speciﬁcally in
our
synthetic
experiments, for a given h , • we generate node degree d v for nodes in each class from multinomial distribution with numpy.random.multinomial(800/h, numpy.ones(400)/400, size=1)[0] .
• For a sampled d v , we generate intra-class edges from (does not include self-loop) numpy.random.multinomial(h d v , numpy.ones(399)/399, size=1)[0] and inter-class edges from numpy.random.multinomial((1-h) d v , numpy.ones(1600)/1600, size=1)[0] .
We will release the code and the generated data later.
B. F URTHER D ISCUSSION OF A GGREGATION H OMOPHILY ON R EGULAR G RAPHS
We notice that in Figure 2(a), the performance of SGC-1 and GCN both have a turning point, i.e., when H edge ( G ) is smaller than a certain value, the performance will get better instead of getting worse.
With some extra restriction on node degree in data generation process, we ﬁnd that this interesting phenomenon can be theoretically explained by the following proposition 1 based on our proposed similarity matrix which can verify the usefulness of H M agg ( G ) .
We ﬁrst generate regular graphs , i.e., each node has the same degree, as follows,
Generate Synthetic Regular Graphs We ﬁrst generate 180 graphs in total with 18 edge homophily levels varied from 0.05 to 0.9, each corresponding to 10 graphs.
For every generated graph, we have 5 classes with 400 nodes in each class.
For each node, we randomly generate 10 intra-class edges and [ 10 H edge ( G ) − 10 ] inter-class edges.
The features of nodes in each class are sampled from node features in the corresponding class of the base dataset.
Nodes are randomly split into 60%/20%/20% for train/validation/test.
We train 1-hop SGC ( sgc-1 ) Wu et al.
and GCN Kipf & Welling (2016) on synthetic data (see Appendix A.1 for hyperparameter searching range).
For each value of
H edge ( G ) , we take the average test accuracy and standard deviation of runs over 10 generated graphs.
We plot the performance curves in Figure 5.
From Figure 5 we can see that the turning point is a bit less than 0.2.
We derive the following proposition for d -regular graph to explain and predict it.
Proposition 1.
(See Appendix D for proof).
Suppose there are C classes in the graph G and G is a d -regular graph (each node has d neighbors).
Given d , edges for each node are i.i.d.
generated, such that each edge of any node has probability h to connect with nodes in the same class and probability 1 − h to connect with nodes in different classes.
Let the aggregation operator ˆ
A
=
ˆ
A rw .
Then, for nodes v , u 1 and u 2 , where Z u 1 , : =
Z v, : and Z u 2 , : (cid:54) =
Z v, : , we have
[Equation]
and the minimum of g ( h ) is reached at
[Equation]
where d intra = dh , which is the expected number of neighbors of a node that have the same label as the node.
The value of g ( h ) in equation 12 is the expected differences of the similarity values between nodes in the same class as v and nodes in other classes.
g ( h ) is strongly related to the deﬁnition of aggregation homophily and its minimum potentially implies the turning point of performance curves.
In the synthetic experiments, we have d intra = 10 , C = 5 and the minimum of g ( h ) is reached at h = 5 / 27 ≈ 0 .
1852 , which corresponds to the lowest point in the performance curve in Figure 5.
In other words, the H edge ( G ) where SGC-1 and GCN perform worst is where g ( h ) gets the smallest value, instead of the point with the smallest edge homophily value H edge ( G ) = 0 .
This reveals the advantage of H agg ( G ) over H edge ( G ) by taking use of the similarity matrix.
C D ETAILS OF G RADIENT C ALCULATION IN EQUATIONC. D ERIVATION IN M ATRIX F ORM
This derivation is similar to (Luan et al., 2020a).
In output layer [Equation] we have
[Equation]
= − trace( Z T log Y ) where 1 C ∈ R C × 1 , ( · ) − 1 is point-wise inverse function and each element of Y is positive.
Then
[Equation]
Note that
[Equation]
Then,
[Equation]
[Table]
[Equation]
where the 4-th equation holds due to (cid:16) Z (cid:12) (cid:0)
exp( Y (cid:48) )
1 C 1 TC (cid:1) − 1 (cid:17) 1 C 1 TC = (cid:0) exp( Y (cid:48) )
1 C 1 TC (cid:1) − 1 .
Thus, we have
[Equation]
For Y (cid:48) and W , we have
[Equation]
and d L = trace
[Equation]
To get d L dW we have,
[Equation]
C.
C OMPONENT - WISE D ERIVATION
Denote ˜ X
=
XW .
We rewrite L as follows:
[Equation]
(cid:48) (cid:1)(cid:1)
= − trace (cid:0) Z T Y (cid:48) (cid:1) + trace (cid:0) Z T log (cid:0) exp( Y (cid:48) )
1 C 1 TC (cid:1)(cid:1)
= − trace (cid:16) Z T
ˆ AXW (cid:17) + trace (cid:0) Z T log (cid:0) exp
( Y (cid:48) )
1 C 1 TC (cid:1)(cid:1)
= − trace (cid:16) Z T
ˆ AXW (cid:17) + trace (cid:0) 1 TC log (exp( Y (cid:48) ) 1 C ) (cid:1)
=
−
N (cid:88) i = (cid:88)
j ∈N i
ˆ
A
i,j Z i, : ˜ X Tj : +
N (cid:88) i = log  
C (cid:88) c
= exp( (cid:88)
j ∈N i
ˆ
A
i,j ˜ X j,c ) 
 = −
N (cid:88) i = log   exp  
C (cid:88) c
= (cid:88)
j ∈N i
ˆ
A
i,j Z i,c ˜ X j,c   
 +
N (cid:88) i = log  
C (cid:88) c
= exp   (cid:88)
j ∈N i
ˆ
A
i,j ˜ X j,c   
 = −
N (cid:88) i = log exp (cid:32) C (cid:80) c
= (cid:80) j ∈N i
ˆ
A
i,j Z i,c ˜ X j,c (cid:33) (cid:32) C (cid:80) c
=1 exp( (cid:80) j ∈N
[Equation]
[Table]
Z
[Table]
c =
[Table]
for any j .
Consider the derivation of L over ˜ X j (cid:48) ,c (cid:48) :
[Table]
[Equation]
N (cid:88)
[Equation]
Writing the above in matrix form, we have
[Equation]
D P ROOF OF P ROPOSITION
Proof.
According to the given assumptions, for node v , we have ˆ A v,k =
d
+1 , the expected number of intra-class edges is dh (here the self-loop edge introduced by ˆ A is not counted based on the deﬁnition of edge homophily and data generation process) and inter-class edges is (1 − h ) d .
Suppose there are
C ≥ 2 classes.
Consider matrix ˆ AZ ,
Then, we have E (cid:104) ( ˆ AZ ) v,c (cid:105) =
E (cid:20) (cid:80) k
∈V ˆ
[Equation]
Tc
[Equation]
(cid:21)
[Equation]
(cid:80) k ∈V
[Equation]
(cid:105) d
+ , where 1 is the indicator function.
When v is in class c , we have (cid:80)
[Equation]
intra-class edges
[Equation]
self-loop introduced by ˆ A ).
When v is not in class c , we have (cid:80)
[Equation]
inter-class edges uniformly distributed in the other C − 1 classes).
For nodes v, u , we have ( ˆ AZ ) v, : , ( ˆ AZ ) u, : ∈ R C and since elements in ˆ A v,k and ˆ A u,k (cid:48) are independently generated for all k, k (cid:48) ∈ V , we have
[Equation]
Thus,
[Equation]
u, v are in the same class
[Equation]
− 1) 2 ( d +1) 2 , u, v are in different classes
For nodes u 1 , u 2 , and v , where Z u 1 , : = Z v, : and Z u 2 , : (cid:54) =
Z v, : ,
[Equation]
[Equation] [Equation]
Setting g
( h )
= 0 , we obtain the optimal h :
[Equation]
Cd (16)
For the data generation process in the synthetic experiments, we ﬁx d intra , then d = d intra /h , which is a function of h .
We change d in equation 16 to d intra /h , leading to h
[Equation]
It is easy to observe that h satisfying equation 17 still makes g ( h ) = 0 , when d in g ( h ) is replaced by d intra /h .
From equation 17 we obtain the optimal h in terms of d intra : h = d intra
Cd intra +
[Equation]
−
D. A N E XTENSION OF P ROPOSITION
Base on the deﬁnition of aggregation similarity, we have
[Equation]
|V|
[Equation]
|V|
Then,
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
Consider the random variable
[Equation]
Since RV is symmetrically distributed and under the conditions in proposition 1, its expectation is E [ RV ] = g ( h ) as showed in equation 15.
Since the minimum of g ( h ) is 0 and RV is symmetrically distributed, we have P ( RV ≥ 0) ≥ 0 .
5 and this can explain why H agg ( G ) is always greater than 0.
in many real-world tasks.
E P ROOF OF T HEOREM
Proof.
Deﬁne W cv =
( ˆ AZ ) v,c .
Then,
[Equation]
Note that
[Equation]
For any node v , let the class v belongs to be denoted by c v .
For two nodes v, u , if Z v, : (cid:54) =
Z u, : , we have
[Equation]
Then, from equation 18 it follows that
[Equation]
When C = 2 ,
[Equation]
If Z v, : = Z u, : , i.e., c v = c u , we have
[Equation]
[Equation]
Then, from equation 18 it follows that
[Equation]
Thus, if C = 2 , for any v ∈ V , if Z u, : (cid:54) =
Z v, : , we have S ( I − ˆ A, Z ) v,u ≤ 0 ; if Z u, : =
Z v, : , we have S ( I − ˆ A, Z ) v,u ≥ 0 .
Apparently, the two conditions in equation 9 are satisﬁed.
Thus v is diversiﬁcation distinguishable and DD ˆ A,X ( G ) = 1 .
The theorem is proved.
F M ODEL C OMPARISON ON S YNTHETIC G RAPHS
[Figure] [Figure]
datasets
In order to separate the effects of nonlinearity and graph structure, we compare sgc with 1 hop (sgc-1) with MLP-1 (linear model).
For GCN which includes nonlinearity, we use MLP-2 as its corresponding graph-agnostic baseline model.
We train the above GNN models, graph-agnostic baseline models and ACM-GNN models on all synthetic datasets and plot the mean test accuracy with standard deviation on each dataset.
From Figure 6 and Figure 7, we can see that on each H M agg ( G ) level, ACM-GNNs will not underperform baseline GNNs and the graph-agnostic models.
But when
H M agg ( G ) is small, baseline GNNs will be outperformed by graph-agnostic models by a large margin.
This demonstrate the advantage of the ACM framework.
The curves will almost be the same for
ACMII framework.
G D ISCUSSION OF THE L IMITATIONS OF D IVERSIFICATION O PERATION
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table] [Table]
[Table] [Table]
[Figure]
From the black box area of S ( I − ˆ A, X ) in the example in Figure 8 we can see that nodes in class 1 and 4 assign non-negative weights to each other although there is no edge between them; nodes in class 2 and 3 assign non-negative weights to each other as well.
This is because the surrounding differences of class 1 are similar as class 4, so are class 2 and 3.
In real-world applications, when nodes in several small clusters connect to a large cluster, the surrounding differences of the nodes in the small clusters will become similar.
In such case, HP ﬁlter are not able to distinguish the nodes from different small clusters.
H T HE S IMILARITY , H OMOPHILY AND DD ˆ A,X ( G ) M ETRICS AND T HEIR E STIMATIONS
Additional Metrics There are three key factors that inﬂuence the performance of GNNs in realworld tasks: labels, features and graph structure.
The (modiﬁed) aggregation homophily tries to investigate how the consistency of graph structure and labels will inﬂuence the performance of GNNs with features being ﬁxed.
And their correlation is veriﬁed through the synthetic experiments.
Besides graph-label consistency, we need to consider feature-label consistency and aggregatedfeature-label consistency as well to fully investigate the performance of NNs and GNNs.
With aggregation similarity score of the features S agg ( S ( I, X )) and aggregated features S agg (cid:16) S ( ˆ A, X ) (cid:17) listed in table 8, our methods open up a new perspective on analyzing and comparing the performance of graph-agnostic models and graph-aware models in real-world tasks.
Here are 2 examples.
Example 1: People observe that GCN (graph-aware model) underperforms MLP-2 (graph-agnostic model) on Cornell, Wisconsin, Texas, Film .
Based on the aggregation homophily, the graph structure is not the main cause of the performance degradation.
And from Table 6 we can see that
the
S agg (cid:16) S ( ˆ A, X ) (cid:17) for the above 4 datasets are lower than their corresponding S agg ( S ( I, X )) , which implies that it is the aggregated-feature-label inconsistency that causes the performance degradation.
Example 2:
According to the widely used analysis based on node or edge homophily, the graph structure of Chameleon , and Squirrel are heterophilous and bad for GNNs.
But in practice, GCN outperforms MLP-2 on those 2 datasets which means the additional graph information is helpful for node classiﬁcation instead of being harmful.
Traditional homophily metrics fail to explain such phenomenon but our method can give an explanation from different angles:
For Chameleon, its modiﬁed aggregation homophily is not low and its S agg (cid:16) S ( ˆ A, X ) (cid:17) is higher than its S agg ( S ( I, X )) which means its graph-label consistency and aggregated-feature-label consistency help the graphaware model obtain the performance gain; for Squirrel, its modiﬁed aggregation homophily is low but its S agg (cid:16) S ( ˆ A, X ) (cid:17) is higher than its S agg ( S ( I, X )) which means although its graph-label consistency is bad but the aggregated-feature-label consistency is the key factor to help the graph-aware model perform better.
We also need to point out that (modiﬁed) aggregation similarity score, S agg (cid:16) S ( ˆ A, X ) (cid:17) and
S agg ( S ( I, X )) are not deciding or threshold values because they do not consider the nonlinear structure in the features.
In practice, a low score does not tell us the GNN models will deﬁnitely perform bad.
[Table]
Furthermore, in most real-world applications, not all labels are available to calculate the dataset statistics.
In this section, We randomly split the data into 60%/20%/20% for training/validation/test, and only use the training labels for the estimation of the statistics.
We repeat each estimation fortimes and report the mean with standard deviation.
The results are shown in table 8.
Analysis From the reported results we can see that the estimations are accurate and the errors are within the acceptable range, which means the proposed metrics and similarity scores can be accurately estimated with a subset of labels and this is important for real-world applications.
I E XPERIMENTS ON F IXED S PLITS P ROVIDED BY (P EI ET AL ., 2020)
See table 9 for the results and table 10 the optimal searched hyperparameters
[Table]
J A D ETAILED E XPLANATION OF THE D IFFERENCES B ETWEEN ACM-GNN S , ACM(II)-GNN S AND GPRGNN, FAGCN
• Difference with GPRGNN (Chien et al., 2021):
To explain how the channel mixing mechanism makes ACM-GNNs and ACMII-GNNs different from the learning mechanism in GPRGNN, we ﬁrst rewrite GPRGNN as Z = K (cid:80)
[Equation]
K (cid:80) k
=0 diag ( γ k , γ k , . . .
, γ k ) H ( k ) .
The node-wise channel mixing mechanism in GPRGNN form is Z = K (cid:80) k
=0 diag ( γ 1 k , γ 2 k , . . .
, γ Nk ) H ( k ) , where N is the number of nodes and γ ik , i = 1 , . . .
, N are learnable parametric mixing weights for different channels.
ACM and ACMII allow GNNs to learn more diverse parameters in diagonal than GPRGNN and thus, have stronger expressive power than GPRGNN.
• Difference with FAGCN (Bo et al., 2021): instead of using a ﬁxed ˆ A , FAGCN learns a new ﬁlter ˆ A (cid:48) based on ˆ A in a similar way as GAT (Velickovic et al., 2017).
Some people may take ˆ A (cid:48) as a mixing matrix and think it is similar to our channel mixing mechanism, but ˆ A (cid:48) is essentially a learnable aggregator
or LP ﬁlter for LP channel, which is far different from channel mixing.
And ˆ A (cid:48) can be decomposed into ˆ A (cid:48)
=
ˆ A (cid:48) 1 − ˆ A (cid:48) 2 , where ˆ A (cid:48) 1 and − ˆ A (cid:48)represents positive and negative edge (propagation) information respectively.
In our paper, we are not discussing the advantages of using the learned ﬁlter ˆ A (cid:48) over the ﬁxed ﬁlter ˆ A , we are comparing the models with and without channel mixing mechanism.
We believe the empirical results on real-world tasks in table 4 and table 9 is the best way to compare the models with ﬁxed ﬁlter and node-wise channel mixing and the models with learned ﬁlter but without channel mixing
[Table]