Let Your Features Tell The Differences: Understanding Graph Convolution By Feature Splitting
Authors: Yilun Zheng
Xiang Li
Sitao Luan
Xiaojiang Peng
Lihui Chen
Venue: ICLR 2025 Poster (2025)
Keywords: Graph Neural Networks
Graph Homophily
Topological Feature Selection
Primary area: learning on graphs and other geometries & topologies
TL;DR: We propose a new metric to identify GNN-favored and GNN-disfavored features and use topological feature selection to fuse these features into GNNs, which significantly improves GNNs performance without hyper-parameter tuning.

Abstract: Graph Neural Networks (GNNs) have demonstrated strong capabilities in processing structured data. While traditional GNNs typically treat each feature dimension equally important during graph convolution, we raise an important question: **Is the graph convolution operation equally beneficial for each feature?** If not, the convolution operation on certain feature dimensions can possibly lead to harmful effects, even worse than convolution-free models. Therefore, it is required to distinguish convolution-favored and convolution-disfavored features. Traditional feature selection methods mainly focus on identifying informative features or reducing redundancy, but they are not suitable for structured data as they overlook graph structures. In graph community, some studies have investigated the performance of GNN with respect to node features using feature homophily metrics, which assess feature consistency across graph topology. Unfortunately, these metrics do not effectively align with GNN performance and cannot be reliably used for feature selection in GNNs. To address these limitations, we introduce a novel metric, Topological Feature Informativeness (TFI), to distinguish GNN-favored and GNN-disfavored features, where its effectiveness is validated through both theoretical analysis and empirical observations. Based on TFI, we propose a simple yet effective Graph Feature Selection (GFS) method, which processes GNN-favored and GNN-disfavored features with GNNs and non-GNN models separately. Compared to original GNNs, GFS significantly improves the extraction of useful topological information from each feature with comparable computational costs. Extensive experiments show that after applying GFS to $\textbf{8}$ baseline and state-of-the-art (SOTA) GNN architectures across $\textbf{10}$ datasets, $\textbf{90\%}$ of the GFS-augmented cases show significant performance boosts. Furthermore, our proposed TFI metric outperforms other feature selection methods for GFS. These results verify the effectiveness of both GFS and TFI. Additionally, we demonstrate that GFS's improvements are robust to hyperparameter tuning, highlighting its potential as a universally valid method for enhancing various GNN architectures. To facilitate reproducibility and further research, we have made our code publicly available at https://github.com/KTTRCDL/graph-feature-selection.

Published as a conference paper at ICLR 2025
LET YOUR FEATURES TELL THE DIFFERENCES: UN- DERSTANDING GRAPH CONVOLUTION BY FEATURE SPLITTING
Yilun Zheng's Xiang Li Sitao Luan Xiaojiang Peng", Lihui Chen
Nanyang Technological University, Centre for Info. Sciences and Systems, Cooleee of Big Data and Internet, Shenzhen University, 3 Mila- Quebec Artificial Intelligence Institute.
ABSTRACT
Graph Neural Networks (GNNs) have demonstrated strong capabilities in process- ing structured data. While traditional GNNs typically treat each feature dimension equally important during graph convolution, we raise an important question: Is the graph convolution operation equally beneficial for each feature? If not, the convolution operation on certain feature dimensions can possibly lead to harm- ful effects, even worse than convolution-frre models. Therefore, distinguish convolution favored and convolution convolutionddisfvoree features. Traditional feature selection methods mainly focus on identifying informative features or re- ducing redundancy, but they are not suitable for structured data as they overlook graph structures. In graph community, some studies have investigated the per- formance of GNN with respect to node features using feature homophily met- rics, which assess feature consistency across graph topology. Unfortunately, these metries do not effectively align with GNN performance and cannot be reliably used for feature selection in GNNs. To address these limitations, we introduce novel metric, Topological Feature Informativeness (TFI), to distinguish GNN- favored and GNN-disfavored features, where its effectiveness is validated through both theoretical analysis and empirical observations. Based TFI, we propose a simple yet effective Graph Feature Selection (GFS) method, which processes GNN-favored and GiN--disfaoored features with GNNs and non-GNN models separately. Compared to original GNNs, GFS significantly improves the extrac- tion of useful topological information from each feature with comparable com- putational costs. Extensive experiments show that after applying GFS to 8 base- line and state-of-the-art (SOTA) GNN architectures across 10 datasets, 90% the GFS-augmented cases show significant performance boosts. Furthermore, our proposed TFI metric outperforms other feature selection methods for GFS. These results verify the effectiveness of both GFS and TFI. Additionally, we demon- strate that GFS's improvements are robust to hyperparameter tuning, highlighting its potential a universally valid method for enhancing various GNN architec- tures. To facilitate reproducibility and further research, we have made our code publicly available at http::/hhtooooom////ccc///////op
required
INTRODUCTION
Graph Neural Networks (GNNs) are widely used for processing graph-structured data, such as rec-
ommendation systems (Ong al. 2023 Ong & Khong 2024), social networks (Li al. 2023a Awasthi et al.
2023 Luan et al. 2019). elccommmnication (Lu et al. 2024a) and bio-informatics
(Zhang et al. 2021 Kang et al. 2022 Hua et al. (024)). Although graph convolution has been
shown effective to enrich node features with topological information through message propagation, the performance gain is found be restricted by the assumption of homophily, i.e., similar nodes are more likely to be connected in graph (McPherson et al.) (001). On the other hand, when graph exhibits low homophily, i.e., heterophily, the graph convolution operation can lead to performance
"Corresponding Author. Equal contribution. Email address: yilun001 @e.ntu.edu.sp 221021211150 01 4eemii. szu.edu.en. sitao.Iuande @ mail.megill.cs xiaojiangpeng @ s7tu.edu.en, elhchen @ @ntu.edu.ss

---

Published as a conference paper at ICLR 2025
GNN-disfavored Node
GNN-favored Feature
GNN-dislavored Feature
GNN-favored Node
+1
+1
+2
+2
+2
Node level
Feature level
Improvements in GNN performance at node level and feature level. Different colors denote node labels, while the direction and magnitude of arrows denote node features.
Figure 1:
degradation and sometimes even underperform convolution-fiee models, such as Multi-Layer Per- ceptrons (MLPs) (Zhu et al. 2020a Luan ct al. 2022b 2024c). Therefore, to measure the impact of graph convolution operation, label- homophily metries (Pei et 2020a Zhu et al.)
2020a) are
proposed to measure the label consistency along graph topology. However, they neglect the effects of node features, which crucial for graph learning. Feature homophily metrics (Yang al. 2021 Jin et al. 2022) are then proposed to measure the feature consistency along the graph.
Although these existing metries can capture the feature similarity between connected nodes, they overlook that different feature dimensions may exhibit different levels of compatibility with graph structures, and thus may gain different amounts of benefits or negative impacts from graph convolu- tion. For example, as illustrated in Figure (right), the GNN-favored feature exhibits uniform values among intra-class nodes while have different values across inter-class nodes. This characteristic en- ables graph convolution operation to effectively increase the distinguishability among nodes from different classes, as demonstrated in (Luan et al. 2024b). Conversely, graph convolution on GNN- disfavored features may hinder the learning process. This example raises a crucial question: How can we determine whether graph convolution beneficial or not for a specific feature dimension?
To address this issue, in this paper, propose Topological Feature Informativeness (TFI), which measures the mutual information between each dimension of aggregated node features and labels. Compared with previous feature selection metrics that mainly focus selecting the most informa- tive features or reducing redundancy for i.i.d. data (Li et al. 2017), our proposed TFI emphasizes differentiating between GNN-favored and GNN-disfavored features, which is essential for effective graph representation learning. To achieve this, we show that TFI can provably provide an upper bound on the performance gap between graph convolution and convolution-free models, which is supported by both theoretical analysis and empirical observations. Additionally, TFI overcomes the "good heterophily" issue (Ma et al., [02T]] Luan et al. (2024c). which is serious misalignment of existing feature homophily metrics and GNN performance.
Motivated by the principle of "feed the right features to the right model" Luan et al. 2022a). we propose a simple yet effective method called Graph Feature Selection (GFS). GFS first uses TFI to identify GNN-favored and GNN-disfavored features. Then, to enhance the extraction of useful infor- mation, the favorrd features are processed by GNNs, while MLPs handle the GNN-dissavvred features. Last, a final linear layer fuses the embeddings from both models to obtain the final node representation. GFS can be seamlessly integrated into almost any GNN architecture, improving overall model performance with comparable computational costs. Our experiments on real-world benchmark datasets demonstrate that GFS significantly boosts the performance of 8 GNNs across 10 datasets in node classification tasks and the improvement is robust to hyperparameter tuning. Moreover, we demonstrate that TFI outperforms other statistical and optimization-basel metrics for feature selection in GNNs, validating its superiority. Besides, we surprisingly find that GFS is much more effective on node embeddings encoded by Pretrained Language Models (PLMs) than other methods. This implies that the advantages of PLMs in understanding graph-structured data might be rooted in their ability to split the topology-awan and topology-agnostic information into separate feature dimensions. In summary, our main contributions are as follows.
We introduce novel metric, Topological Feature Informativeness (TFI), to distinguish GNN-favored and GNN- din-diifavoredd features. We validate its effectiveness through both empirical observations and theoretical analysis.
2

---

Published as a conference paper at ICLR 2025
We propose Graph Feature Selection (GFS) based on TFI. simple yet powerful method that significantly boosts GNN performance. To the best of our knowledge. we are the first to address feature selection problem based on the splitting of GNN-favored and GNN- disfavored features.
We conduct extensive experiments by applying GFS to 8 baseline and ofte--o--thearn (SOTA) GNN architectures across 10 benchmark datasets. The results show significant performance boost in 90% (72 out of 80) of the cases.
2 PRELIMINARY
We define graph graph, its structure can be represented by an adjacency matrix A E I RNN,, where N is the number of nodes, Au, Au,u indicates the presence of an edge between nodes u and v, i.e. Eues evu E E, otherwise Au, Ae,u = O. The node classification task graph aims predict node labels Y e RN by utilizzig node features X e NxM and topological information, where Xu,m denotes the th feature of node u. The node degree matrix is denoted as D RNNN where D., Evev Au,o v is the degree of node u. The adjacency matrix can normalized as Å D A The neighborhood set of node are denoted by N₁, {ulewo E).
{V,E), where V is the node set and & is the edge set. For undirected
Graph Neural Networks. Graph Neural Networks (GNNs), such GCN (Kipf & Welling 2016),
GAT /Velicković et al. 2017). and GraphSage (Hamilton et al.) (017)), can effectively capture useful topological information from neighbors by message propagation mechanism. Specifically, for node u, its embedding z in the 1-th layer of a GNN can be expressed as:
2, UPDATE 22,-1 AGGREGATE( 1-1 eu)))
(1)
where AGGREGATE(.) represents the aggregation function, and UPDATB(.) is the update function based on the ego node and the aggregated neighbor representations from the previous layer. For the first layer, the initial embedding is set as input node features.
The representative graph convolution can be formulated as - W' denote the node embeddings and learnable parameter matrix at the 1-th layer, and (()) is an acti- vation function. To boost GNN performance, some architectures concatenate ego and neighborhood node embeddings as the input of the next layer (Abu-EllHaija et al.] 2019: /hhu et al. 2020b). i.e.
where H' and
[AH'-1 _H'1] or adding the aggregated node embeddings with the ego node embeddings (Luan et al.
2022a), ie., (Á + I) H which has been shown to improve GNN performance (Platonov 2023), especially in the scenario of heterophily (Luan al. 2024c Lu al. 2024b).
al.
Label Homophily. Homophily, concept originating from social networks, is defined the ten- dency of similar nodes are more likely to connect with each other (McPherson et al.) (00T). In graph learning, people attempt to use homophily as metric to measure whether graph convolution on certain graph is beneficial for GNN not. Higher homophily often implies that the topo- logical structure can provide useful information, which can lead to better GNN performance. The commonly used label-based homophily metrics Luan et al. (2024a) are defined as follows:
hedge (g, Y)= (swvleuveE ( [
(ulueNo,,Y
hnode (9, Y)
1 M Evey
W.
Generally, these metrics only measure label consistency across graph topology. However, they over- look the feature aspect, which also plays critical role in GNN performance (Zheng et al. 2024a). Therefore, some studies focus on the feature consistency across graph topology.
Feature Homophily. To extend the conventional definition of label homophily to node features, feature homophily metries are introduced and the general form
MG9, X X.m) T(...
Σ sim( Xu,my Xy.m)
where 17(-) is a normalization function and sim(.) is a similarity metric. The key difference between different feature homophily metrics lies in the choice of the similarity function sim(.), which could be cosine similarity (Jin et al., (022), dot-product (Yang et af. 2021). or Fuclidean distance (Chen al. 2023). Please refer to Appendix Affor more detailed introduction homophily metries.
3

---

Published as a conference paper at ICLR 2025
Mutual Information. Mutual Information measures the amount of information obtained about one random variable given another variable (Reza 1994). More specifically, given variable X and Y, the mutual information can be expressed as
1(xY)= = Σ EEp(en p(x,)) log p(x.)) y) p(z)p(v)
(4)
yey xex
where p(z, y) is joint probability, and p(r) and pv) are marginal probability distributions.
3 TOPOLOGICAL FEATURE INFORMATIVENESS
As illustrated in Figure 1 not all features in graphs benefit from graph convolution. This raises the question: how can we determine which feature can get benefit from graph convolution? Previous studies (Zheng et al. 2024a; Luan et al. 2024b; Wang et al., 2024a) have demonstrated that it is the stable topological patterns of intra-class nodes that enable effective message aggregation from neighbors, rather than relying solely on seciisteny-bbsseee homophily metrics. To address the lim- itations of feature homophily metrics, we propose Topological Feature Informativenes: (TFI) to measure the informativeness of neighbors in graphs given the feature X.m. which is:
TFLm (YY; (..m)
where X.m = (A)* (Am is the aggregated node features from k-hop neighbors, Á is the adjacency matrix without self-loop, and we set by default. TFL, leverages mutual information to quantify the dependence between variables, including both linear and non- non-linear relationships.
Then, we conduct a theoretical analysis to show how TFLm can effectively measure the benefit of a certain feature gained from graph convolution.
Theorem 1. Given a graph g {V.E) with C' classes and TFI, measured on the m-th feature, the prediction accuracy PA of classifier on node labels Y with aggregated features X.m is upper bounded by:
TFL + log
PA S
(6)
log C'
The proof is given in Appendix B
From Theorem 1, we can see that TFLm can be used to estimate the effect of graph convolution on feature dimension m by relating it to model performance. More specifically, higher TFL, means the prediction accuracy of a classifier with aggregated node feature m has a higher upper bound, which indicates that the graph convolution operation is more likely be beneficial for feature m. To guide feature selection with TFI,, in the following theorem, we explain how it relates to the performance gap between graph-convoluted and convolution- free features.
Theorem 2. The gap between T(Y; X. m. X.m) and 1(Y; X.m)) is upper bounded by TFIm:
I(Y: Xm, X.,m. -imm: X,m) I(Y: X.m Xm,) s I(Y: X,m) TFI
(7)
In Theorem 2, the mutual information I(Y: X. measures how well the ego node features capture the relevant information of labels, without making assumptions about the underlying data distribu- tion. This makes I(Y; X.m) non-parametric estimation of MLP performance. On the other hand, T(Y; X.m, X.,) serves as a non-parametric estimation of GNN performance, which utilizes both the ego node features X.m and the aggregated features X. for prediction The subtraction of X X.,m) and ((Y; X.,m) indicates the information gap between graph-convoluted and convolution-free features w.r.. Y, which is upper bounded by TFIm. In other words, a feature m with larger TFI,,, means that feature is more likely to be GNN-favored; otherwise, GNN-disfavored.
To verify the effectiveness of the above claims about TFI, we conduct experiments on real-world benchmark datasets. Specifically, first compute TFI,, for all features in the graph, sort them,
'For a discrete variable and continuous variable X, mutual information is estimated based on entropy using k-nearest neighbor distances, following Kraskov et al. (2004); Ross (2014). "The combination of X.m and X.m could be either concatenating node embeddings. i.e., [AH'-] -H'- adding the aggregated node embeddings with the ego node emheddings, i.e., (Ā H as introduced in Section
4

---

Published as a conference paper at ICLR 2025
Ancuracy Cap (CCN-MLP
Children
Accuracy Cap (CCN-MLP)
Roman
0.06
0.0020
0.04
0.0015
0.0010
0.02
0.0005
0.00
0.0000
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
TF1
Figure 2: The performance gap between GCN and MLP with the increase of TFI.
and divide them evenly into 10 bins. Next, we train GCN and MLP only with the features in bin respectively, and report the performance difference of GCN and ML..
As shown in Figure 2 GCN underperforms MLP in bins with low TFI and outperforms MLP in bins with high TFI. This observation consistent with Theorem and 2, and indicates that TFI can effectively identifies GNN-favored and GNN-disfavoree features. Note that the estimation of TFI.. does not require of the labels Y. a semi-supervised setting, TFIm can estimated using only the training labels and features, i.e., I (Yarann Xman, without the need for pscudo labels.
In the next section, will introduce how use TFLm for feature selection in GNNs.
4 GRAPH FEATURE SELECTION
In this section, we propose Graph Feature Selection (GFS), a TFI-based feature selection method. is composed of three main components: (1) GNN-favored feature selection, (2) feature embedding, and (3) feature fusion.
High THI
Graph-favored Features
Top
GNN
Linear
Graph-disfavored Features
Bottom 1-r
MLP
TFI
Sort Features
Concat
Figure 3: Framework of GFS with TFI.
As illustrated in Figure for each feature dimension we first measure its corresponding TFI,,. using Eq. (5). Since a higher TFLm indicates stronger GNN performance compared to MLP, we apply threshold to select the set of GNN-favored features where 8(r) is the threshold for the top percentile of all TFI values in features, and r (0, The GNN-disfavored features the remaining 1 of features, defined X-g X {
(XglX.m Xg. TFI,, 2 8(r)),
The GNN-favored features Xg are expected to be more suitable for GNNs, whereas the GNN- disfavored features X-g better suited for MLPs. Thus, we feed Xg into GCN (or any other GNN architecture) and X_g into an MLP to better leverage information from both neighbors and ego nodes. Specifically, in the 1-th layer of GCN or MLP, we have:
GCN: H¹ o (AHb-WA-)) MLP: H'g a (H W-
where H6- X9 and H 0 -g. During message passing, the representations of H. and H are derived independently, ensuring that X² and X²g can be encoded by the most appropriate model without interference.
5

---

Published as a conference paper at ICLR 2025
Lastly, to combine the node information from Xg and X-g together, concatenate the node representation from the last layer L of GNN and MLP, and feed them into a linear layer:
H H H +
where H is the final node representation, W is the weight matrix, and is the bias, respectively. This final node representation is then used for downstream graph tasks.
Complexity Analysis. To compare the time complexity of GNN+GFS against standard GNNs, we use GCN as the backbone GNN and assume the embedding size F is consistent across all layers. First, estimating and sorting TFI introduces a complexity of O(KC + N + log N)), where K is parameter related to mutual information estimation,
the number of classes, and N is the number of nodes. Both K and C' are much smaller than N or the number of edges |E|, making this overhead negligible compared to GNN training complexity as below.
After feature selection, we obtain GNN-favored features Xg E IR Nx(MM and GNN-disfavored fea- tures X-9 R (x))( where [.] is the floor function. For the first layer of GCN+GFS, the time NrMMJF) for the GNN channel and O(N -r)M for the MLP channel. Their summation, O(EE) [ M²), smaller than the complexity of a standard GCN, O(EE M 2M), for < 1. This is because GFS reduces the input feature dimension for GCN.
complexity is ((E)[(M])
In subsequent layers, GCN+GFS has a complexity of O( F + 2N slightly higher than the complexity of GCN O( F F2) by an additional F2 term due to the MLP layers. However, this additional cost F²) is smaller than the O(E F) introduced by graph convolutions. Thus, the overall complexity of GCN+GFS is comparable to that of the original GCN.
EXPERIMENTS
To verify the effectiveness of our proposed Graph Feature Selection (GFS) and Topological Feature Informativeness (TFI), we try to answer the following questions with experiments in this section.
RQ1: Can GFS really enhance the performance of GNNs? Does the improvement always hold?
RQ2: How much does hyperparameter tuning influence the performance gains from GFS?
RQ3: How does TFI compare with other statistic-based and optimization- metrics for GFS?
RQ4: Do the GNN-favored and GNN-disfavored features identified by TFI really fit better with
GNNs or MLPs, respectively?
5.1 EXPERIMENTAL SETUPS
Data Preparation. The datasets used in our experiments include Children, Computers, Filness, History, and Photo from Yan et al. (2023), and Amozon-Ratings Minesweeper, Questions, Roman- Empire, and Tolokers from Platonov et al. (2023). These datasets exhibit varying levels of label homophily across different domains. The dataset statistics and descriptions are shown Appendix For all datasets, we randomly split the data into training, validation, and test sets in a ratio of 50%% 25%% 25% for 10 runs. Note that the node features datasets from Yan al. (0233) are encoded by Pretrained Language Models (PLMs).
Baselines. We implement GFS and compare its performance with 5 baseline GNNs, GCN Kipp & Welling 2016), GAT (Velicković al., 2017). GraphSAGE (SAGE) (Hamilton al., 2017). SGC ( (u et al. 2019) and APPNP (Gasteiger et al. 2018), and 3 SOTA GNNs, Graph Transformer (GT) (Shi et al.) 2020), ACM-GCN (Luan al. 2022a) and FAGCN (Bo al.) 2021). The GFS- augmented GNN is denoted as GNN+GFS throughout this paper. To enhance performance, we incorporate skip connections (He et al. 2016) and layer normalization (Ba et al. 2016) for all the methods, recommended in Platonov al. (203)); Luo et al. (2024). The detailed descriptions of these GNNs are introduced in Appendix C.2 Additionally, we compare our proposed TFI with other cnttstic-baseed metrics, including lGg (Jin et al. 2022). hattr (Yang al. 2021). haim-cos (Chnn et al., 2023), Itgim-eue (Chen et al.) 2023). and hCTF (Lee et al. 2024). We also evalu- ate optimization-aan based metrics such Osoft and 0 Hard within GFS. Detailed definitions of these
statistic- and optimization-bain based metrics can be found in Appendix A and C.4 respectively.
Training. During training, we utilize the Adam optimizer (Kingma & Ba 2014) to update model parameters. Each model is trained for 1000 epochs. Model performance is evaluated node

---

Published as a conference paper at ICLR 2025
classification tasks, measuring accuracy for multi-class datasets and AUC-ROC for binary-class datasets. The searching space for the ratio r is (0.1,0.2.... 0.9] in GF and A = for the number
of the hop of neighbors in TFQ See Appendix C.3 for more training details.
5.2 PERFORMANCE (RQ1)
The results and comparisons of the model performance are shown in Table where the performance gap between GNN+GFS and GNN is denoted as A. We observe that (1) GFS significantly boosts GNN performance on 90%( (0%(72/80) (72/80) cases of the GFS- -uggentgd GNNs, with an average increase of 3.2% in accuracy AUC-ROC on node classification tasks with various homophily levels. This demonstrate the effectiveness of our proposed method. (2) We notice that the improvement of GFS varies by dataset, e.g., in Children and Computers, GCN+GFS achieves 5% average performance improvement, while in Minesweeper and Questions, the improvement is less significant. This may be because most features in Minesweeper and Questions are already GNN-favored. Nevertheless, GFS remains effective on most datasets. (3) We find that GFS demonstrates greater improvements on datasets encoded by Pretrained Language Models (PLMs) (40/40 cases) than those encoded by some traditional methods (32/40 cases), e.g., ont-hot, fasttext or statistics-basee methods. This suggests that, compared to traditional feature encoding methods, which widely used in graph learning, information
PLMs potentially enable the splitting of topology-aware and topology-agnose into separate feature dimensions, which helps GFS work easier. This is surprising discovery and to our knowledge, we are the first to reveal this phenomenon. It can help explain why PLMs can assist the learning of graph models (Ye et al. 2024). In section [.6] we will discuss more about the TFI distribution under PLM vs. traditional feature encoding methods.
Table 1: The performance of GFS on GNN baselines. The improvement is highlighted as bold if there is an increase, i.e.& A > O, after applying GFS.
Model
Children
Comp.
Fituess
History
Photo
Amazon.
Mines.
Questiinss
Roman.
Tolokers
GCN
53.89 33.880337
83.38=1.75 78
87.56+0.79 79
84, 84.26660.38
84.48
51 41.4000449
90.0100.55
76.2811 18
75.51+0.65
19H0..5
GCNIGES
59 2960.31
90.23600 18
93.0950.13
84.99200.38
37.500036
53. 83.32600:1
89 99.9..00.00
76.50e 133
83.30+0.57
85.0120191
+5,41
+6.85
+5.53
+0.73
+2.83
+1.92
-0.02
+0.22
+7.79
+0.82
GAT
53.5440.99
86.5240992
58 23
83.8 83.8340.27
31.0830.00 60
26226 53
84.5720.80
83.415047
GAT-GES
57,7440.35
90.50e0
03.20+011
84. 84.5440
07.56+0.22 78
53.75-0.57
00.2240.66
0700-1111
86.1740.56
84.41+0.77
+4.20
+3.98
+4.97
+2.67
0..04
-0.61
+1.60
+1.00
SAGE
54.6848844
86.08-0.50
586551
84.06+0.42
85.000 64
53,8000.56
70.7440.59
74.91+1000 06
828110.61
82.77+0.35
SAGE+OFS
39, 19.13
90.474024
93,6310.10
84.6810.33
87
87.23.0.51
54.1210.61
90.7110.62
75.3761.33
85.1710.53
83.161
A
+4.46
+4.39
+4.98
+0.62
+2.15
+0.37
.0.03
+0.46
+2.66
+0.39
GT
51 30.20.0.38
16
87.3721.38
83.6110.43
51.30,0:73
90.11
77 17.5761.09
$4.95510.4
83.20u0.00 60
GI+GES
56.0110.35
89572002
92.4480111
84.0920.32
S7415041
52.4780344
8933200.644
89
77.
77.703
36.9920.45
83.46600.7
+4.11
+4.34
+5.07
+0.48
+3.76
-1.17
0.18
+0.13
+2.04
+0.26
SGC
526940.52
52.50:034
83.9840.37
R2 42.79
49.0020.42
764321101
71.7880.81
09.8910.51
786511.11
SGC-GES
55.9340.37
50.2980025
84.1540.23
85 4640033
51,1350.64
86.7920.66
73.5221.10
74.0010.67
82.7610.44
+3.24
+5.28
66.06
+0.17 0
+2.67
+2.43
+12.6
+1.75
44.11
+4.11
APPNP
80.6340.88
836670000
86 11
18
83,3740.25 26
R2 122001111
48.731061
81,6120,79
75.291100
10
798221117
APPNPAGES
36.7140.35
88.5150344
91.220025
84,75400555
86 56
30.7610.62
85.1980997
75
75.06e1100 103
72.3540.69
&
+6.08
+484
+4.6
+1.38
+4.45
+2.03
+1.58
+0.37
+0.87
+3.84
ACMGUN
54.60000500
85 75
89 10
84.22H0.34
84992034
51.1100399
90.9900.5
76.66e1
25,2740357
836110033
ACMOUN4GRS
39,04+0.37
89.
99.5000.19 19
93
04
84 70000000
6661-042
52.81-0755
90.450059
75
99+1.27
87.03+0.57
83.55+1003
43.65
+4.34
+1.62
+0.90
-0.14
-0.67
FAGEN
50,4340.86
79.92+0.99
83.10-0.45
82.0440.62
6767-0.77
46.000.52
78 78.2242.86
60+2.08
62.02+2.98
73,07-0.70
FAGUNAGRS
56.1740.38
87.72=0.49
59.06-0.35
84.3240.36
85
54-033
50,7240822
18.1241.32
21.94e2.03
72.0561.44
82.82+1.25
+5.74
+7.00
+6.36
+2.28
+487
+4.4
+9.90
+13.34
+10.03
+9.75
5.3 SENSITIVITY TO HYPERPARAMETERS (RQ2)
Some recent GNNs (i et al. 2022; Liu et al. 2022) introduce more and more hyperparameters, which could be fragile to hyperparameter tuning (Luan et al. 2024c). To investigate the sensitivity of the superiority of GFS over GNNs to ratio r and other model hyperparameters, we compare the performance between GCN and GCN+GFS under varying settings. First, we test how model performance responds to the changes in ratio r while fixing all other hyperparameters, and Figure 4 shows the results on 4 datasets. Note that GFS collapses to normal GCN when r 1.0 and MLP when r = 0.0, as all features are sent to GNN or MLP, respectively. On Children, Computer, and Roman-Empire, GCN+GFS with r from 0.1 to 0.9 consistently outperforms GCN (r 1.0) and MLP (r = 0.0), indicating that the performance gain obtained from GFS is robust to hyperparameter
"To prevent the behavior of GNN+GFS from closely resembling that of traditional GNNs, exclude 1.0 from our experiments. However, in practice, = 1.0 may occur in GNN+GES when all features are graph-favored, which would make GNN+GFS comparable to GNNs in the worst-case scenario.
"In Appendix D.4 we examine the influence of the number of k-hop neighbors on TFI. Our findings indicate that k 1 is sufficient achieve strong model performance.
7

---

Published as a conference paper at ICLR 2025
We also observe that, although GCN+GFS does not significantly outperform GCN Tolokers, sending 40% (r = 0.4) to 90% (r 0.9) features to GCN is at least not worse than sending all the features into GCN (r 1.0). This again demonstrates the robustness of GCN+GFS on The above results highlight that we do not always need to convolute all the features, as neither r 1.0 nor 0.0 yields optimal results most datasets.
Childron
Comp.
Roman
Tolokers
Acnurayy aUC-Ro0
Accuracy /AUC-OCC
Aceuracy AAUCR00
0.80
Acouracy /UCCROG
0.50
0.80
0.75
0.80
0.53
0.75
0.65
0.50
0.0
0.2
0.4
0.6
0.8
1.0
0.2 0.0
0.1
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.1
0.6 0.8 1.0
Figure 4: The impact of ratio r on the performance of GCN+GFS, where features are sent to MLP and features are sent to GCN. The star represents the best performance.
Secondly, we analyze how other hyperparameters affect GFS performance. Figure shows how GCN, MI.,, and GCN+GFS respond to the changes in the number of layers, dimension of hidden embeddings. learning rate, weight decay, and dropout rate on Computers (homophilic graph) and Amazon-Ratings (heterophilic graph). Results indicate that GCN+GFS consistently outperforms GCN and MLP across various hyperparameter values. This property allows GFS to be easily in- tegrated into most baseline GNNs without much more hyperparameter tuning. Please refer to Ap- pendix D.2 for more results on the sensitivity of GFS to hyperparameters on the other datasets.
Comp.
Neturecy ruucati
Vocumey rubratt
0.5
Noturasy RoCatee
0.9
6.8.
0.7
0.6
0.7
0.6
0.7
MLP
Number of Layers
Eidden Dimenston
Leurzing Hale
Weight Decay
Amezon.
Amaza.
ADC
Accurary: AUC
Accuracy 1B00-AU
0.54
8.52
8.50
0.50
0.52
2.5
Nctureyy
2.48
Accuracy
0.65
3.45
Accuracy
2.45
4.8
62N
40
DUN
0.40
0.48
0.40
Number of
Linerning
Weight
Figure 5: Comparison of the response of GCN+GES, GCN, and MLP to 5 different hyperparameters on Computers and mmzzon-Ratings.
5.4 EFFECTIVENESS OF TFI (RQ3)
We show the superiority of TFI over other metric-based feature selection methods on GCN+GFS, including statistic-based metrics hor Jin al. (2022), /ater Yang al. (202T), hsim-coss haim-cac Chen et al. (2023), and lCTF Lee et al. (2024) and optimization-based methods e.g., Osoft and Hardd where feature selection process learned in an end-to-end manner. All methods follow the shols same hyperparameter tuning process as in GFS. Table 2 shows the performance of different metrics in feature selection for GCN, where TFI achieves the best performance according to average rank across 10 datasets. This is reasonable because other statistic- "good heterophily" issue and TFI overcomes the issue by the measurement of mutual information Platonov et al. (2024). Besides, the optimization-based metrics underperform TFI, indicating the feature selection cannot be effectively handled automatically during the optimization process of find that even for the metrics that inferior to TFI, e.g., hattrs heam-eues and hCTF. applying them within our GFS framework can still improve the performance of baseline GCN, indicating the necessity of feature selection in GNNs.
metrics suffer from explaining
GNNs. In addition,
8

---

Published as a conference paper at ICLR 2025
Table 2: The comparison of feature selection with feature homophily metrics and aptimization-baase
methods.
Metrics
Childron
Comp-
Fitness
History
Photo
Amaron.
Mines
Questions
Roman.
Tolokers
Ang. Rank
TH-
59.29:0.31
20.2520.18
93.0920.13
54 1P.9PeD 38
87.11:5044 42
53.32e9881
$9,99220.00 50
76. 76.50e11.3
83.32+1.57
N5.01=11.91
1.40
AGR
55.00+0 25
63.72+1 32
67.06-0.58 58
31.88-0.25 26
83. 36m0
$1. 51.71+0.55
89.75-0.56 68
75. 75.73+11.53
79. +0.44
85.31+0..7
6.70
hater
55.15e0.32
61
67.88.0.55
$4.22a0.25
83.22290552
51.46a0.70 70
50.15a0: 70
76.40 16.40s1.39
78.50 78.9020.55
S6.0410). 52
4,70
Rame-io
55.1550..51
84.1120.84
67.09=0.96
54.07a0.31
$3.3861.00 (01
51.6820.35 39
89.8726055
75 75.58&1.07
78.98:0.66
83.6910 82
5.95
50.32s0.20 26
W9.00000 20
87.52=1.04
85 85.19999.38
56. 56.71e00 30
51 $1.35at0.49
90.19500.0
76.51=1.17
78:96:1173
34 (4.03all).3 53
3.35
55.04g00.1
59 58.0922.25
$4.32a0.30
83,4668:85
$1.76=0055
89 89.57280.68
76.40%1.33
78.70201.38
M3.38gl).8 59
4,55
Soft
57.04-1 1.7
55. 35.35
50.54 00.54-1.32 32
84.47-0.92
84 04.83e33.33 26
51,70m6.62
89 89.93.0.60 000
75. 75.04+11.25 2.2
70 20.2040:44
84-0++0 54
3.50
Hard
45. 28:06 79
66 0..2224.77 70
68.48.4.11
50.98.5 20.98.0077
72.09 7.2
42.77=0.55
86.6262.22 25
75. 13.5911.29
76 76.9610.61
79.1340.94
8.90
None
53.36e1
83.38a1.76
87.56.0.79 79
14.1360.2
84,0060.77
51,4050.49
89,9160.52
76.24t1.48
75.5130.65
84. 1910.75
5.95
Since TFI is not an unsupervised measurement, we investigate how the percentage of supervision in TFI (percentage of node labels used to calculate TFI in Eq. (5) influences the model performance of GCN+GFS. As shown in Figure 6 as the supervision percentage increases, the model perfor- mance first increases and then stabilizes after 30%. This indicates that 30% of the label supervision achieves similar results as full labels in most datasets and it highlights the effectiveness of TFI in sparse label scenarios. Compared to some other methods (Zheng et al.) [024b] Li et al., 2023b) that require pseudo labels during training. TFI requires less preprocessing in semi-supervised node clas- sification settings. Even with only 10% supervision, TFI can still enhance GCN+GFS performance on most datasets compared to the original GCN. Please refer to Appendix
D33 for more results on
the influence of the supervision percentage TFI on GFS performance.
Children
Comp.
Roman
Tolokers
Accurcy RROC-AU
0.62
Acourso /ROC-UUC
0.91
Accuracy /ROC-AUC
0.82
Accuracy RDC-AUC
0.88
0.60
0.90
0.80
0.86
0.88
0.78
0.84
0.56
0.82
0.52
0.54
CPS
GCN
0.82
0.81
CPS
GCN
0.74
0.76
GPS
GCN
0.80
GCN CPS
0.1
0.9 Supervision Percentage
0.3
0.5
0.7
0.1
0.3
0.5
0.7
0.9
0.1
0.9 Supervision Percenlage
0.3
0.5
0.7
0.1
0.7 0.9 Supervision Percentage
0.3
0.5
Supervision Percenlage
Figure 6: Influence of the supervision percentage in TFI on the model performance of GCN+GFS.
Based on TFI, we extend the feature selection from the raw node features to latent node embeddings, which may also be splitted into GNN-favored and GNN-disfavored. Specifically, or MLP on the training set and then get the node embeddings from the last layer. After that, we train GCN+GFS on these node embeddings following the same process as on the original node features. As shown in Figure 7 applying GCN+GFS on either the original node features, X, or pretrained node embeddings, MLP(X) and GCN(X,A) improves GCN performance on most datasets. Furthermore, for some datasets, such as Computers, Questions, and Romain- Empire, the performance of GCN+GFS could be further improved using pretrained node embeddings compared with original node features. Additionally, it would be interesting to explore the impact of graph feature selection on dynamically updated node embeddings in future research.
train GCN
Accuracy 1AUC ROC
0.9
0.9
0.8
Socureyy AAC
0.8
0.7-
0.7
0.6
0.5
GUNI ues MLNNC
GCK+ GRS OCINKAI
0.5
Orgin Swap
Gcx
0.4
Childrat
Comp.
Finaeed
History
Piatto
Amaon
Mines.
Guestien
Roman
Talekoya
0.4
Childres
Compp
Filadd
Hiltory
Photo
Amazinn
Minst
Questinge
Roman
Talokoro
Figure 7: GCN+GFS node features X and pretrained node embeddings
Figure 8: Performance drop of GCN+GFS by swapping GNN-favored and GNN- disfavored features.
GCN(X, A) and MLP(X).
5.5 GNN-FAVORED AND ENN-DISFAVOREL FEATURES (RQ4) To validate whether features selected by a high TFI are truly selection process by feeding GNN-favored features into MIP and GNN-disfavored features into
we swapped the feature

---

Published as a conference paper at ICLR 2025
GNN. As shown in Figure the performance of GCN+GFS drops significantly across all datasets after swapping. On some datasets, such as Minesweeper, there is a 40% drop in performance. These results show that TFI can reliably identify GNN-favoree and GNN-disfavored features.
Children, 0.02820.016
Comp. 0.09620.022 023
Fitness, 0.18440.022
History 0.041
Photo, 0.068
250
125
100
200
200
100
150
75
100
100
100-
50
5.00
0.05
0.10
0.35
0:20
0:25
0.30
0.25
0.30
00
0.30
0.15
0.05
0.10
0.30
0.091=0 0006
Mines, 0,037=0.
Questions. 0.007=0.001
Roman. 0.067.00.044
0.029=0.014
1.5
20-
20
15-
1.0
10-
10
Figure 9: Comparison of histogram of TFI on 10 datasets.
5.6 TFI DISTRIBUTIONS: PLM vs. TRADITIONAL ENCODING METHODS
To understand why GFS is much more effective datasets with PLM encoded features, conduct detailed analysis on the histogram of TFI distributions for PLM (first row of Figure 9) and traditional encoding methods (second row of Figure 9b. We observe that,
The distributions share similar uni-modal and right-skewed shapes across different datasets. This means that most graph-disfavored feature dimensions are concentrated at the small TFI area. And the graph-favored features are long-tailed with few feature di- mensions containing the most graph- information. Such distribution can help GFS efficiently split graph-aware and graph-agnostic feature dimensions and PLM can consis- tently produce encodings with this pattern for graph-related tasks.
PLM:
Traditional Methods: As shown in Appendix C. Amazon-Rating Questions, Roman- Empire encoded by fasttext, Minesweeper is encoded by one-hot, and Tolokers is en- coded by statistics-based method. Compared to PLM, these methods have lower average TFI values and show irregular TFI distributions. Although fasttext shows good perfor- mance and has an unimodal right- TFI distribution on Roman Roman-Empire, it fails to consistently generate high-quality features across different datasets.
6 CONCLUSION AND FUTURE WORKS
In this paper, attempt to enhance GNN performance by identifying disfavored features. To this end, we introduce Topological Feature Informativeness (TFI), which measures the mutual information between aggregated node features and labels. This metric ef- fectively quantifies the performance gap between GNNs and Multi-Layer Perceptrons (MLPs), as supported by our empirical observations and theoretical analyses. We then propose a simple yet effective method, Graph Feature Selection (GFS), to incorporate TFI to improve the GNN perfor- mance. Our extensive experiments demonstrate that applying GFS 8 GNN architectures across 10 datasets yields a significant performance boost in 90% (72 out of 80) of the cases, with an average increase of 3.2% in accuracy or AUC-ROC on node classification tasks. We also show that the per- formance gains from GFS are robust to hyperparameter tuning, indicating its potential as universal method for enhancing various types of GNNs.
and GNN-
However, the features selected by TFI introduce a new hyperparameter, ratio in GFS. Although GFS improves GNNs performance across a wide range of r. the best performance remains unknown due to the high complexity of the optimization process in GNNs and MLP. Therefore, it is interesting to explore the auto-selection of ratio r in the future. Furthermore, since TFI is a supervised. statistic- based metric and other types of metrics may also enhance GNN performance under GFS. it would be interesting to investigate more types of metrics, both supervised and unsupervised, for feature selection of GNNs in the future.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Sami Abu-EI-Haija, Bryan Perozzi, Amol Kapoor, Nazanin Alipourfard, Kristina Lerman, Hrayr Harutyunyan, Greg Ver Steeg, and Aram Galstyan. Mixhop: Higher-order graph convolutional architectures via sparsified neighborhood mixing. In international conference on machine learn- ing. pp. 21 21-29. PMLR, 2019.
Jason Ansel, Edward Yang, Horace He, Natalia Gimelshein, and Animesh et al. Jain. PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph Compilation. In 29th ACM International Conference on Architectural Support for Program- ming Languages and Operating Systems, Volume 2 (ASPLOS '24). ACM, April 2024. doi: 10.1 10.11445/622065533400666
org/ www.sstsssyttrreeeeee
pdf,
AK Awasthi, Arun Kumar Garov, Minakshi Sharma, and Mrigank Sinha. Gan model based on node classification forecasting in social network. In 2023 International Conference on Artificial Intelligence and Smart Communication (AISC), pp. 1039- 1043. IEEE, 2023.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.064500 2016.
Deyu Bo, Xiao Wang. Chuan Shi, and Huawei Shen. Beyond low- uww-frquency information in graph convolutional networks. In Proceedings of the AAAI conference on artificial intelligence, vol- ume 35, pp. 3950-3957, 2021.
Yuhan Chen, Yihong Luo, Jing Tang, Liang Yang, Siya Qiu, Chuan Wang, and Xiaochun Cao. Lsgnn: towards general graph neural network node classification by local similarity. arXiv preprint arXiv:2305.0424 2023.
John C Davis, Robert J Sampson, et al. Statistics and data analysis in geology, volume 646. Wiley New York, 1986.
Ben Finkelshtein, Xingyue Huang, Michael Bronstein, and Ismail Ilkan Ceylan. Cooperative graph neural networks. arXiv preprint arXiv:2310.01267, 2023.
Johannes Gasteiger, Aleksandar Bojchevski, and Stephan Günnemann. Predict then propagate: Graph neural networks meet personalized pagerank. arXiv preprint arXiv:1810.0523 2018.
Sebastien Gerchinovitz, Pierre Ménard, and Gilles Stoltz. Fano's inequality for random variables. 2020.
Edouard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, and Tomas Mikolov. Learning word vectors for 157 languages. In Proceedings the Eleventh International Conference on Lan- guage Resources and Evaluation (LREC 2018), Miyazaki, Japan, May 2018. European Language Resources Association (ELRA). URL
/ /ccl anthol hottchhooyy.... wrggo.lggll.... 8-1550
Will Hamilton, Zhitao Ying, and Jure Leskovee. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.
Haoyu Han, Juanhui Li, Wei Huang, Xianfeng Tang, Hanqing Lu, Chen Luo, Hui Liu, and Jiliang Tang. Node-wise filtering graph neural networks: A mixture of experts approach. arXiv preprint arXiv:2406.03:: 034644 2024.
Satoshi Hara and Takanori Machara. Enumerate lasso solutions for feature selection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Xiaofei He, Deng Cai, and Partha Niyogi. Laplacian score for feature selection. Advances in neural information processing systems, 18, 2005.
Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. spaCy: Industrial- strength Natural Language Processing in Python. 2020. doi: 1..2281zeendd... 1212303.
11

---

Published as a conference paper at ICLR 2025
Chenqing Hua, Sitao Luan, Minkai Xu, Zhitao Ying, Jie Fu, Stefano Ermon, and Doina Precup. Mudiff: Unified diffusion for complete molecule generation. In Learning on Graphs Conference, pp. 33-1. PMLR, 2024.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterizztioo with gumbel-softmax. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. /
net fett/forrmmidd=rEk..
Di Jin, Rui Wang, Meng Ge, Dongxiao He, Xiang Li, Wei Lin, and Weixiong Zhang. Raw-gnn: Random walk aggregation based graph neural network. arXiv preprint 2rX1v:2206. 13953, 2022.
Chuanze Kang, Han Zhang, Zhuo Liu, Shenwei Huang, and Yanbin Yin. Lr-gnn: A graph neural network based on link representation for predicting molecular associations. Briefings in Bioinfor- matics, 23(1 23(1)bbab5513, 2022.
Seyoung Kim and Eric Xing. Statistical estimation of correlated genome associations to quanti- tative trait network. PLOS genetics, 508)xe 2009.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1oXivv:412.66900 2014.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. arXiv preprint arXiv: 1609,02977 2016.
Alexander Kraskov, Harald Stögbauer, and Peter Grassberger. Estimating mutual information. Phys- ical Review E- StSatiistiaal, Nonlinear, and Soft Matter Physics, 69(6):06133, 2004.
Soo Yong Lee, Sunwoo Kim, Fanchen Bu, Jaemin Yoo, Jiliang Tang, and Kijung Shin. Feature distribution on graph topology mediates the effect of graph convolution: Homophily perspective. International Conference Machine Learning, 2024.
Jure Leskovee and Andrej Krevi. SNAP Datasets: Stanford large network dataset collection. http: 7 7/snap. stw.ford..ooooooo wdt.//orrddddddddeeeeeeeeeeeeeee data June 2014.
David D Lewis. Feature selection and feature extraction for text categorization. In Speech and Natural Language: Proceedings of Workshop Held at Harriman, New York, February 23-26, 1992, 1992.
Jundong Li, Kewei Cheng, Suhang Wang, Fred Morstatter, Robert P Trevino, Jiliang Tang, and Huan Liu. Feature selection: A data perspective. ACM computing surveys (CSUR), 50(6):1-55, 2017.
Xiang Li, Renyu Zhu, Yao Cheng, Caihua Shan, Siqiang Luo, Dongsheng Li, and Weining Qian. Finding global homophily in graph neural networks when meeting heterophily. In International Conference on Machine Learning, pp. 13242-13256. PMLR, 2022.
Xiao Li, Li Sun, Mengjie Ling, and Yan Peng. A survey of graph neural network based recommen- dation in social networks. Neurocomputing, 129412 2023a.
Yayong Li, Jie Yin, and Ling Chen. Informative pseudo-labeling for graph neural networks with few labels. Data Mining and Knowledge Discovery, (7(() (77(1)2228-22) 2023b.
Daniil Likhobaba, Nikita Pavlichenko, and Dmitry Ustalov. Toloker Graph: Interaction of Crowd Annotators, 2023. ltttp: httpa//////oooooooooooooeeeeeeee
Derek Lim, Xiuyu Li, Felix Hohne, and Ser-Nam Lim. New benchmarks for learning on non- homophilous graphs. arXiv preprint arXiv:2 04.00144, 2021.
Huan Liu and Rudy Setiono. Chi2: Feature selection and discretization of numeric attributes. In Proceedings of 7th IEEE international conference on tools with artificial intelligence, pp. 388- 391. Ieee, 1995.
Nian Liu, Xiao Wang, Lingfei Wu, Yu Chen, Xiaojie Guo, and Chuan Shi. Compact Graph Structure Learning via Mutual Information Compression, January 2022. URL http: /tt:://axxii /ttp://xxxxvv///////// abs/ 22011.201.r 05540 arXiv:2201.05:: [es].
12

---

Published as a conference paper at ICLR 2025
Qincheng Lu, Sitao Luan, and Xiao- Wen Chang. Geepnet: Graph wwwvoolutionenhhnnee expectation propagation for massive mimo detection. In IEEE GLOBECOM 2024 Conference Proceedings, 2024a.
Qincheng Lu, Jiaqi Zhu, Sitao Luan, and Xiao Wen Chang. Flexible diffusion scopes with parame- terized laplacian for heterophilic graph learning. arXiv preprint arXiv:2409.0900 09888, 2024b.
Sitao Luan, Mingde Zhao, Xiao- Wen Chang, and Doina Precup. Break the ceiling: Stronger multi- scale deep graph convolutional networks. Advances in neural information processing systems, 32, 2019.
Sitao Luan, Chenqing Hua, Qincheng Lu, Jiaqi Zhu, Mingde Zhao, Shuyuan Zhang, Xiao Wen Chang, and Doina Precup. Revisiting heterophily for graph neural networks. Advances in neural information processing systems, 35:1362-1333 2022a.
Sitao Luan, Mingde Zhao, Chenqing Hua, Xiao Wen Chang, and Doina Precup. Complete the missing half: Augmenting aggregation filtering with diversification for graph convolutional neural networks. arXiv preprint arXiv:22/2. 10822, 2022b.
Sitao Luan, Chenqing Hua, Qincheng Lu, Liheng Ma, Lirong Wu, Xinyu Wang, Minkai Xu, Xiao- Wen Chang, Doina Precup. Rex Ying, et al. The heterophilic graph learning hand- book: Benchmarks, models, theoretical analysis, applications and challenges. arXiv preprint arXiv:2407.09618 2024a.
Sitao Luan, Chenqing Hua, Minkai Xu, Qincheng Lu, Jiaqi Zhu, Xiao-Wen Chang, Jie Fu, Jure Leskovec, and Doina Precup. When graph neural networks help with node classification? investigating the homophily principle on node distinguishability. Advances in Neural Information Processing Systems, 36, 2024b.
Sitao Luan, Qincheng Lu, Chenqing Hua, Xinyu Wang, Jiaqi Zhu, Xiao-Wen Chang, Guy Wolf, and Jian Tang. Are heterophily-specee gnns and homophily metries really effective? evaluation pitfalls and new benchmarks. arXiv preprint arXiv:2409.05755, 2024c.
Yuankai Luo, Lei Shi, and Xiao-Ming Wu. Classic gnns are strong baselines: Reassessing gnns for node classification. arXiv preprint 2xXi::2406.08993 2024.
Yao Ma, Xiaorui Liu, Neil Shah, and Jiliang Tang. Is homophily necessity for graph neural networks? arXiv preprint arXiv:2106.000 06134, 2021.
Miller McPherson, Lynn Smith-Lovin, and James M Cook. Birds of a feather: Homophily in social networks. Annual review of sociology, 27(1):415-444, 2001.
Rongqing Kenneth Ong and Andy WH Khong. Spectrum-based modality representation fusion
graph convolutional network for multimodal recommendation. arXiv preprint arXiv:2412.14978, 2024.
Rongqing Kenneth Ong, Wei Qiu, and Andy WH Khong. Quad-tier entity fusion contrastive repre- sentation learning for knowledge aware recommendation system. Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pp. 1959, 2023.
Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. Geom-gen: Geometric graph convolutional networks. arXiv preprint arXiv:2002.052877 2020a.
Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. Geom-gen: Geometric graph convolutional networks. arXiv preprint arXiv:2002.05287,, 2020b.
Oleg Platonov, Denis Kuznedelev, Michael Diskin, Artem Babenko, and Liudmila Prokhorenkova. A critical look at the evaluation of gnns under heterophily: Are we really making progress? arXiv preprint 302. 11640 2023.
Oleg Platonov, Denis Kuznedelev, Artem Babenko, and Liudmila Prokhorenkova. Characterizing graph datasets for node classification: Homophily- heterophlly dichotomy and beyond. Advances in Neural Information Processing Systems, 36, 2024.
13

---

Published as a conference paper at ICLR 2025
Fazlollah M Reza. An introduction information theory. Courier Corporation, 1994.
Brian C Ross. Mutual information between discrete and continuous data sets. PloS one, 9(2): e87357, 2014.
Yunsheng Shi, Zhengjie Huang, Shikun Feng, Hui Zhong, Wenjin Wang, and Yu Sun. Masked label prediction: Unified message passing model for semi-supervised classification. arXiv preprint arXiv:2009.03509, 2020.
Petar Velicković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv: (70./0033 2017.
Junfu Wang, Yuanfang Guo, Liang Yang, and Yunhong Wang. Understanding heterophily for graph neural networks. arXiv preprint arXivv2401.09123 2024a.
Kun Wang, Guohao Li, Shilong Wang. Guibin Zhang, Kai Wang, Yang You, Junfeng Fang, Xiao- jiang Peng, Yuxuan Liang, and Yang Wang. The snowflake hypothesis: Training and powering gnn with one node one receptive field. In Proceedingy of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. $152-3163, 2024b.
Minjie Wang, Da Zheng, Zihao Ye, Quan Gan, Mufei Li, Xiang Song, Jinjing Zhou, Chao Ma, Lingfan Yu, Yu Gai, Tianjun Xiao, Tong He, George Karypis, Jinyang Li, and Zheng Zhang- Deep graph library: A graph-centric, highly-performant package for graph neural networks. arXiv preprint arXiv:19000 01315, 2019.
Shilong Wang. Hao Wu, Yifan Duan, Guibin Zhang, Guohao Li, Yuxuan Liang, Shirui Pan, Kun Wang, and Yang Wang. All nodes are created not equal: Node-specific layer aggregation and filtration for gnn. arXiv preprini urXiv:2405.07892 2024c.
Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. Sim- plifying graph convolutional networks. In International conference on machine learning, pp. 6861-6871. PMLR, 2019.
Hao Yan, Chaozhuo Li, Ruosong Long, Chao Yan, Jianan Zhao, Wenwen Zhuang. Jun Yin, Peiyan Zhang, Weihao Han, Hao Sun, et al. comprehensive study on text-attributed graphs: Bench- marking and rethinking. Advances in Neural Information Processing Systems, 36: 36:17238-17264, 2023.
Liang Yang, Mengzhe Li, Liyang Liu, Chuan Wang, Xiaochun Cao, Yuanfang Guo, et al. Diverse message passing for attribute with heterophily. Advances in Neural Information Processing Sys- tems, 34:4451-4757663 2021.
Zhilin Yang. William W. Cohen, and Ruslan Salakhutdinov. Revisiting semi-supervised learning with graph embeddings. In Proceedings of the 33rd International Conference on International Conference on Machine Learning Volume 48, ICML' pp. 40-48. JMLR.org, 2016.
Jieping Ye and Jun Liu. Sparse methods for biomedical data. ACM Sigkdd Explorations Newsletter, 14(1):4-15, 2012.
Ruosong Ye, Caiqi Zhang, Runhui Wang, Shuyuan Xu, and Yongfeng Zhang. Language is all a graph needs. In Findings of the Association for Computational Linguistics: EACL 2024, pp- 1955-1973, 2024.
Xiao-Meng Zhang, Li Liang, Lin Liu, and Ming-Jing Tang. Graph neural networks and their current applications in bioinformatics. Frontiers in genetics, 12:690049, 2021.
Tianxiang Zhao, Xiang Zhang, and Suhang Wang. Disambiguated node classification with graph neural networks. In Proceedings of the ACM on Web Conference 2024, pp. 914 923, 2024.
Zheng Zhao and Huan Liu. Spectral feature selection for supervised and unsupervised learning. In Proceedings of the 24th international conference on Machine learning, pp. 1151-1157, 2007.
Yilun Zheng. Sitao Luan, and Lihui Chen. What is missing in homophily? disentangling graph homophily for graph neural networks, arXiv preprint arXiv:2406. 200.:24406.88.4... 2024a.
14

---

Published as a conference paper at ICLR 2025
Yilun Zheng, Jiahao Xu, and Lihui Chen. Learn from heterophily: Heterophilous information- enhanced graph neural network. arXiv preprint arXiv:2403.13351 2024b.
Zhu, Saharon Rosset, Robert Tibshirani, and Trevor Hastie. I-norm support vector machines. Advances in neural information processing systems, 16, 2003.
Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. Beyond homophily in graph neural networks: Current limitations and effective designs. Advances in Neural Information Processing Systems, 33, 2020a.
Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. Beyond homophily in graph neural networks: Current limitations and effective designs. Advances in neural information processing systems, 33:7793-7804, 2020b.
15

---

Published as a conference paper at ICLR 2025
A RELATED WORK
The proposed Graph Feature Selection (GFS) in this paper mainly relates with feature selection and GNNs efficacy. Thus, in this section, we first introduce feature selection methods non-GNNs models and GNNs. Then, we introduce the studies on the effectiveness of GNNs from a graph level, node level, feature level.
Feature Selection. Feature selection crucial for improving model performance, preventing over- fitting, and reducing computational complexity (Li et al. (0117). To preserve data similarity, Lapla- cian Score (He et al., 20055) and SPEC (Zhao & Liu (007) are proposed to select features that best preserve the data manifold structure. To maximize the correlation between feature and class labels, MIM (Lewis) 1992) is proposed based on information theory. To mínimize the fitting errors in clas- sification tasks. Hara & Maehara (2017): /Zhu et al. (2003) propose to select features with larger reduce feature redundancy, T-Score (Davis al.) 1986) and Chi-Square Score (Liu & Setiono 1995) are proposed to access whether features could distinguish different classes. All of the aforementioned methods only consider data in Euclidean space, which cannot be applied for non- -uclliedd data, such graph-structure data. To address this, Ye & Liu (2012): Kiim & Xing (2009) use a Graph Lasso regularizer to select features consistent across connected nodes.
weights models with -norm regularization.
While effective in traditional machine learning, these methods do not identify GNN-favored or GNN-disfavored features, discussed in this paper. Our approach does simply discard GNN- disfavored features; instead, it strategically utilizes them with non-GNN model, driven by the principle of aligning the right features with the right model.
Graph Homophily. Even if traditional GNNs (Kipf & Welling 2016: Velicković al., 2017) are believed to perform well on graph-related tasks, the performance of GNNs could be inferior than non-GNNss models some graphs. Therefore, metrics of graph homophily, such as edge homophily (Zhu et al., (020a), node homophily (Pei et al.) 2020b)) and class homophily (Lim et al. (202T) are proposed determine when GNNs perform well. The definitions of these homophily metries are given as:
(ewoleun fe VY=Y,
(ulueN,,Ya=YY No
hedge (9, Y)
Amode (9, Y)
1 Ever [DT
(10)
1 C 1
Evev,Y (uv UE EN, YoD Eue(n)Yr==c) the
No
helass (G,))
Σ
(11)
N
D² E
hedge (9,
(2
hadj (g, Y)
(12)
E (2)81)z
Generally, these metrics measure the label consistency along the graph topology using indicator function to determine if the connected nodes share the same labels. However, these metrics suffering from the "good heterophily" (Ma al. 202Ij, leading to a misalignment with GNNs performance. Therefore, new metrics, such as label informativeness (Platonov et al. 2024). aggregation homophily (Luan et al. al. [022a)) and classifier-basse based homophily (Luan al.] 2024b) are proposed to mitigate this deficiency. Nevertheless, all these metrics neglect the feature aspect in graphs, which also constitute an important role in GNNs performance (Zheng et al.] (024a), Therefore, some feature homophily metrics, such as generalized edge homophily (Jin et al. 2022) is proposed to measure the feature consistency across the graph topology:
1
X.X, Xxll
haE
X)
(13)
E
|X
ENVEE
16

---

Published as a conference paper at ICLR 2025
Similarly, local similarity (Chen et al. 2023) is proposed to measure feature homophily at the node level by cosine similarity or Euclidean similarity:
Σ X,,,, x
(9,
Σ
[V we du veN.
X
(14)
1
1
(9,
Σ
Σ
(- xx
V
du veN.
Attribute homophily Yang et al. 2021 also consider the feature homophily but with different normalization on each leature:
hattr, (9, x.m) =
{
Σ
Xumm
EVEN, Xu.m du
Ever X
uel
(15)
M
Battr (g, x) =
Σ m==
hattr.m (.Xm
Class-controlled feature homophily (Lee et al. 2024 examines the relationship between graph
topology and feature dependence by analyzing the difference in expected distances between node and its neighbors compared to random nodes, defined as follows:
xcr(g,X,Y)
Σ
Σ (d(e, VMt)) veN,
- d( (u)))
V
ueV du
d(u, V*)
Σ (X.,))) (x,|8))
(16)
V
Σx,-y, X,
x. Y X.
KKolY, = Yo
where X, Y represents class-controlle features and d(-) denotes distance function.
All of these feature homophily metries measure the feature consistency, which still suffer from the "good heterophily" issue as similar as node homophily or edge homophily. Our proposed Topologi- cal Feature Informativeness (TFI) addresses this limitation by the mutual information, which better aligns with GNNs performance on each feature in graphs.
Heterophiil-orrinted GNNs
For graphs with low homophily (heterophily), GNNs are likely to fail and even worse than non-GNNs models (Luan et al. 2024a). Therefore, many approaches (Wang et al. 2024b] Yang et al., [021 Zheng et al. 2024b Lu et al. [022bb] Luan et al.] (024c) are proposed improve the performance of GNNs heterophilous graphs, which be categorized into graph level, node level, or feature level-based GNNs.
For the graph level, FAGCN (Bo et al. 2021) introduces high frequency signals in graph convolution to capture local information to address heterophily. Similarly, ACMGCN (Luan et al., 2022a) and FB-GNNs (Luan al.) 2022b) proposes filterbanks to fuse identify, low-pass, and high-pass filter signals. To mitigate the limitation of local heterophilous neighbors, Mixhop (Abu-El-Haija et al., 2019) and H2GCN Zhu al. (020b) introduces multi-hop neighbors during the graph convolution. To extend local neighbors in heterophilous graphs to global neighbors, GloGNN (Li ct al. 2022) employs a signed and learnable coefficient matrix, while HiGNN (Zheng et al. 2024b) incorporates new neighbors by utilizing neighbor distribution as heterophilous information.
For the node level. SnoH (Wang al. 2024b) modifies the message propagation for each node with varying receptive field in graph convolution. Similarly, CO-GNN (Finkelshtein et al. 2023) assigns each node with different types of directed message propagation in cooperative manner, NoSAF (Wang et al. 2024c) provides each node with node-specific layer aggregation with varying filter weights, and Node-MOE (Han et al. (024) identifies heterophilous nodes with a feature inconsis- tency measurement, then treats these nodes differently with different filters. Different from these approaches, DisamGCL (Zhao et al. 2024) adopts a contrastive learning objective, by identifying ambiguous nodes with historical label predictions and treating these nodes negative samples.
To our best knowledge, only DMP (Yang et al. 202I) proposes to treat each feature differently in heterophily-orriented GNNs. DMP first defines the attribute homophily feature consistency
17

---

Published as a conference paper at ICLR 2025
across the graph topology, then learnable weight to automatic learn the layer-wised weights for each features. However, this kind of attribute homophily suffers from "good heterophily" issue (Ma et al.] 202I) and these homophily values haven't 't been used to guide the feature selection in graphs. Furthermore, these learnable weights in GNNs for feature selection still inferior than our statistic-based metrics, TFI.
B PROOF THEOREM
Theorem 1. Given a graph G = (V.E) with node features X e TE in the m-th dimension and uniform node labels Y e R over V,, we can obtain aggregated features by applying graph con- volution & times, i.e., Xm (A)*Xm For classifier that predicts label Y using the aggregated features Xm its accuracy rate PA is upper bounded by:
PAS I(Y; log2
(17)
log(C')
Proof. For node with its aggregated feature Xm. denote its true label Y and the predicted label as Y = f(Xm), where f() represents the classifier. The occurrence of enror E in the classifier can be expressed as:
iY, E:= 0 -
(18)
Considering the Markov chain
X V, we apply Fano's inequality (Gerchinovitz et al. 2020)
to obtain:
H(YI&) z (p(P)) + Pg 1o) where Pp: is the error rate and Hs(-) is the binary entropy function.
(19)
To express PE, we rearrange the inequality:
PE 2 H(Y (P()))
(20)
log(C
Noting that H(Y18)- =
I(;; -(Y))I(Y;;))))
-
= log(C) -I(Y; and that Ha(Pr) s log 2, we can
substitute these terms into the equation:
I(Y; Xm) log
PE 2
(21)
log( (C)
Finally, converting the expression to the accuracy rate, we find:
1(Y; + log 2
PA 1
(22)
log(C)
This concludes the proof.
C IMPLEMENTATION DETAILS
In this section, we provide all implementation details in Section and Section 5 made the code publicly available, which can be accessed here: feature-selection
We have also
C.I DATASETS
We use Children, Computers, Fitness, History, Photo, Amazon-Ratings, Minesweeper, Questions, Domine Tolokers as mentioned in Table 3 and Squirrel, Chameleon, Actor, Texas, Cornell, Wisconsin, Cora, CiteSeer, PubMed (Yang et al. 2016) mentioned in [4] all experiments. All datasets used in this work are in compliance with the MIT license. Tables 3 and 14 shows the dataset statistics and the values of homophily metrics measured these datasets. The detailed definition of these homophily measurements are shown in Appendix The descriptions main datasets given below:
18

---

Published as a conference paper at ICLR 2025
Table 3: Dataset Statistics
#Nodes
#Vdges
&Features
AClasses
Degress
Domain
Frat. Modeiig
buok-
Bwdo:
hw6
Avg.
Children
76,875
1,554,578
768
24
E-commerce
0.579
0,320
0.2913
0.0255
Comp.
87.229
721,081
commerce
0.5322
0.0208
Fitaeis
1730055
1,773,500
13
10.25
Excommecce
PLMSs
0.0044
0.828
0.366
History
41.551
338,574
768
12
8,63
Ecummerce
PLMs-
0.7812
0.666
0.2644
0.5163
0.296
Photo
48.362
500,939
708
12
10.36
E-commerce
PLMs
0.7992
0.7491
0.7229
0.6892
0.0254
Amazon.
24,492
93,090
100
5
3,80
Eecammeeen
FoefTest
0.3793
0 0.5804
0.127
0.1357
00177
Mines
10,000
39,402
7
2
3,94
Games
One hox
0.6832
0.626
0.094
0.0108
0.0202
Questions
48,921
153,540
301
2
3.14
Wehene
HeafTett
013963
0.83%
0.022
0,2759
0.0049
Roman. 22,662
32,927
300
18
1.45
Weholle
FoeTTest
0.415
0.0699
0.033
-0,078
0.4870
takicors
11,758
$19,000
2
44.14
Social
Statstics
0.331
0.5945
0.186
0,0887
0.0044
Children and History (Yan et al., 2023) datasets are derived from the Amazon-Books dataset, consisting of items with the second-level label "Children" and "History", respec- tively. In both datasets, nodes represent books, and edges indicate frequent co-purchases or co-views between books. Each book's 's label corresponds to its third-level category. The node attributes derived from the book's 's title and description using Pre-trained Language Models (PLMs). The task is to classify the books into 24 categories for Children and 12 categories for History. More details can be found on: https//gghhu..comm//ttterrrc//// Benchmark
Computers and Photo Yan et al. 2023) datasets are extracted from the Amazon-Dlectronics dataset, including products with the second-level label "Computers" and "Photo", respec- tively. Nodes in these datasets represent electronics products, and edges signify frequent co-purchases co-vict voewie The labels correspond to the third-level category of the prod- ucts. User reviews embedded by PLMs were used as the text attributes of the nodes, with the review having the highest number of votes being selected, or, if such review available, a random review was chosen instead. The task is to classify the products into 10 categories for Computers and 12 categories for Photo. More details can be found on: http:///wwggngiiccccoo////ttt//e
Fitness dataset (Yan et al.) (003) is derived from the Amazon- -poztts-Spoos dataset, con- sisting of fitness-related items with the second-level label "Fitness". Nodes repre- sent fitness products, and edges indicate frequent co-purchases or co-views between products, encoded by PLM. The labels are based the third-level category, and the task is to classify items into 13 categories. More details can be found on: https://huggingface http:/ggggigggacccoooo/attttttts SwateeeeeSeeeetotSSS//S//eaaaaaa
Ratinns (Platonov et al. (023) is derived from the Amazon product co-purchasing network metadata, provided by the SNAP (Leskovec & Krevl (2014) Datasets, which nodes represent products and edges connect products that are frequently bought together. Node features are created using the mean of fastText (Grave et al.) 2018) embeddings from prod- uct descriptions. The task is predict the average rating of product, grouped into five classes. More details can be found on: graphs/tree/maiindaaus
Minesweeper (Platonov et al. 2023 is inspired by the classic Minesweeper game and is synthetic in nature. consists of regular 100x100 grid, where each node represents a cell connected to its eight neighboring cells (except for cells the edges, which have fewer neighbors). In this setup, 20% of the nodes are randomly designated as mines. Each node has hot-hoo-ed features representing the number of neighboring mines, with 50% of the nodes having missing features, indicated by a separate binary attribute. The task to recognize a nodes is mine or not. More details can be found https://gitubb.coo//addee
rsscarchhheteroohhiluu-srgee hatps-ggaaaa/////ii/////
Questions (Platonov et al. 2023) is based on user interactions on the Yandex Q question- answering platform. Nodes' labels represent users, and edges connect users if one swered the other's question during one-year period. Node features are the mean of fast- Text (Grave et al., (018)) embeddings from user descriptions, with additional binary feature indicating users without descriptions. The task is to predict which users remained active (i.c.. not deleted or blocked) the end of the period. More details can be found https://gihub..om/yaaddee
Roman- Roman-Empire (Platonov et al.] (023) is constructed from the "Roman Empire" article in the English Wikipedia. Nodes in the graph represent individual words in the article, and edges connect words that either appear consecutively in the text or are related via syn- tactic dependencies. The node's class is syntactic role obtained using spaCy (Honnibal
19

---

Published as a conference paper at ICLR 2025
et al. 2020), and fastText (Grave al., 2018) are used for word embeddings of node features. More details can be found
graphs/trec/mainndaata
Tolokers (Platonov et al. 2023 is based on data from the Toloka crowdsourcing platform (Likhobaba et al. 2023). The nodes represent tolokers (workers) who have participated in at least one of 13 projects and edges connect workers who have collaborated on the same task. Node features based on profile information and task performance statistics. The task is to predict which tolokers have been banned from project. More details can be found https:://gthuu.comt conm/yyadex-rssccchhheeee randomly split the train, validation, and test set 50% 25% 25% for 10
graphs/tree//maiidaaa
For all datasets, runs. Specifically, to investigatethe impact of supervision percentage on the performance of GFS, in Figure 6 we combine the train, validation and test sets for TFI calculation with given ratio r.
C.2 MODEL
The details of all the GNN methods used in our experiments are introduced as follows:
GCN (Kipf & Welling 2016) performs a layer-wise propagation of node features and ag- gregates features from neighboring nodes to capture local graph structures. Each layer of the network updates the node embeddings by applying a convolution operation over the graph, which combines the node's own features with the features of its neighbors. The authors' implementation is available at atttps//githhb..om////////g/
GAT (Velicković al., (2077) leverages self-attention mechanisms perform node clas- sification on graph-structured data. The innovation of GAT lies in its ability to learn different attention coefficients for each neighboring node dynamically. Specifically, the attention coefficient between node i and its neighbor j is computed as: Cij LeakyReLU(a" [W"]" Why]), where h, and h, are the feature vectors of nodes and j. W is shared weight matrix, and a a learnable attention vector. The coefficients are normal- expley)
ized using the softmax function: aij EEex; exp(e.x)) where N; denotes the set of neigh- bors of node i. The final node representation is computed as a weighted sum of its neigh- bors' features: h (Ejo,; a.y Wh; The multi-head attention mechanism improves stability and expressiveness by concatenating or averaging the outputs of multiple indepen- dent attention heads, which is set with num_heads GAT to assign different importances to each neighboring node while maintaining compu- tational efficiency. The authors' implementation is available at https://ittub..cmm/err /GAT
in our experiments. This allows
SAGE (Hamilton et al. 2017) introduces an inductive framework for node embed- dings by aggregating features from node's local neighborhood rather than requiring all nodes to be available during training. Its key innovation lies in the aggregation functions that can efficiently generate embeddings for unseen nodes. The aggregation process involves sampling and aggregating feature information from node's 's neighbor- hood at each layer of the network. The general form of feature aggregation is: ()( a (W( AGGREGATE I (k-1) Vu N(u))) where N(v) represents the set of neigh- bors of node v, and AGGREGATE is set to mean operation in our experiment. This allows the model to generalize across evolving graphs and unseen nodes. The authors' implemen- tation is available at ltttts://githubbb. www.tu....o/////.. coo//willaan www.iiamleii/rrraaSAAAC..
GT (Shi et al. 2020) is a transformer- based architecture designed for graph learning. adapts the traditional Transformer model to graph data by incorporating node and edge features into the attention mechanism. The key innovation of GT is the use of multi- head attention propagate node features across graph edges while also considering edge information. In our experiments, we employ the Graph Transformer with 8 attention exp(LenkyRaLU -e.))
heads. The attention coefficients are calculated as: a,j = ELeNee exp(LeakyRoLU " where q and k, are the query and key vectors for nodes i and j. and eij rep- resents the edge feature between nodes i and j. This allows the model to aggre- gate both node and edge information. The author's implementation is available at https///ghhhcccmm/aaaapideeeeeee
eranplssnoopproppeddnnng
20

---

Published as a conference paper at ICLR 2025
SGC (Simplifying Graph Convolution) (Wu et al. 2019) is a simplified variant of GCN designed to reduce computational complexity while maintaining similar performance. SGC removes the non-lineat activation functions between layers of traditional GCN, collapsing multiple layers into a single linear transformation. This reduces the overall complexity of the model and results in faster training and inference times. The propagation of node features in SGC can be expressed as: H = S XWW where S is the normalized adjacency matrix, X is the input feature matrix, W is the weight matrix, and K the number of propagation steps (or layers). By precomputing S X, the model reduces to simple logistic regression on the preprocessed features, significantly speeding up the training process. In our experiments, we use K 1 for optimal performance. The authors' implementation is available at https://githhb..cmm wtw.ri/uwwwoooomiiieSSSStt
APPNP Gasteiger et al.] 2018) builds upon Graph Convolutional Networks (GCNs) by utilizing personalized PageRank for improved propagation of node features while avoiding oversmoothing. The model separates the neural network prediction from the propagation process, allowing it handle larger neighborhood sizes efficiently. The propagation is controlled by the following iterative equation: Z(k+)) (1-a) az) aH, where Á is the normalized adjacency matrix, a is the teleport probability (set as = 0.1 in our experiments), and H is the initial node feature set. After (set default
2) propa- gation steps, the output node features are computed as: softmax (+)). softmax(Z(*)). This struc- ture permits the model to aggregate information from both local and distant nodes without increasing the depth of the neural network. The authors' implementation is available at https://githu..com http:///oooooo/icaaa//oopppa/a/a
ACMGCN (Luan al.) 2022a) addresses the heterophily problem in graph neural net- works (GNNs) by introducing an Adaptive Channel Mixing (ACM) framework. This model adapts to both homophilic and heterophilic graphs by dynamically learning to balance infor- mation from three channels: aggregation, diversification, and identity. The key innovation lies in its ability to adaptively learn different weights for each node, allowing the model to exploit local graph structure and node feature similarities. The three channels are combined using learned weights, allowing the model to emphasize different types of information for different nodes: H = as os, HT()) + ann H(!)
where H H and HT(") represent the low-pass (aggregation), high-pass (diversification), and identity channels, respectively, and agg ag, are learned weights that balance these channels. This flexible channel mix- ing enables ACMGCN to significantly outperform standard GNN models on heterophilic graphs while maintaining strong performance on homophilic graphs. The authors' imple- mentation is available at https://gihuub..om/// http:/SitaoLaanAAAMMGM...
+ +a, Ha
FAGCN (Bo et al., (021) tackles the limitation of traditional GNNs that primarily focus on low-frequency signals. FAGCN introduces self-gating mechanism to adaptively combine both low-frequency and high-frequency signals, allowing it to handle both assortative and disassortative networks effectively. The key innovation its ability to dynamically adjust the contribution of each frequency type in the message-passing process. The aggregation of node features is expressed as: I eht, + E)je(() aLay (F2h,) + +a where Ft and FH are low-pass and high-pass filters. respectively, and aiij and O'Ha are the learned attention coefficients for each type of signal. This enables FAGCN to adapt to different graph structures and alleviates the over-smoothing problem common in deep GNNs. The authors' implementation available at htttts:/gifuubbcoo//tyy/////////
C.3 TRAINING DETAILS
All models are implemented using the PyTorch (Ansel et al. 2024) framework and the DGL (Wang cet al. 2019) library. We run experiments on machine with 4 NVIDIA RTX A6000 GPUs, each with 48GB of memory. For most of the tables and figures illustrated in Section
5 we perform a grid
search on all dataset and the hyperparameters used for search are list as follow:
Number of layers: {2,3)
Hidden dimension: {128, 256,512]
Learning rate: {3 x 10-5, 10 1 100$,10-3,,000 ,x 10 3,
Weight decay:
Dropout rate: (0.1,0.2,004006600338
21

---

Published as a conference paper at ICLR 2025
Specially, for response of GCN+GFS, GCN, and MLP to number of layers and hidden dimension, as shown in Figure 5 and Figure [2 the hyperparameters range of set are list as follow:
Number of layers: 21,,,3,4,55 6,8,,, 10)
Hidden dimension: {16, 32, 64, 128, 256, 512,1024]
Learning rate: (10-b, 3 ,33 10 1,0",,,0,,,, 10 100 10 ,, 10
Weight decay: {0, 10-9, ,3 10 100
Dropout rate: {0.1, 0.2, 0.3, 0.5,
To enhance the performance of all GNN models, use skip connections (He et al. 2016) and layer
normalization (Ba et al. 2016) across all methods.
Table 4: Hyperparameters for GNN baselines and GNN+GFS on Children.
Dataset
Model
Num of Layers
Hidden Dim
Learning Rate
Weight Decay
Dropout
GCN
256
3e-3
1c-3
0.2
GCN+GFS
256
3e-4
le-4
0.2
GAT
512
3e-5
0
0.2
GAT+GFS
512
3e-4
0
0.2
SAGE
512
3e-5
0
0.2
SAGE+GES
512
3e-5
0
0.2
GT
512
3e-4
1e-3
0.2
GT+GFS
512
3e-4
0
0.4
Children
SGC
512
3--3
0
0.2
SGC+GFS
3e-3
0
0.2
APPNP
512
36-5
0
0.2
APPNP+GES
512
3e-3
0
0.2
ACMGCN
512
3e-5
0
0.2
ACMGCN+GFS
512
3e-4
0.6
FAGCN
512
3e-5
0.2
FAGCN+GFS
512
3e-3
0.2
Table 5: Hyperparameters for GNN baselines and GNN+GFS on Computers.
Dataset
Model
Num of Layers
Hidden Dim
Learning Rate
Weight Decay
Dropout
GCN
2
512
3e-5
0.2
GCN+GFS
3
512
le-4
3e-S
0.2
GAT
128
3e-3
0.2
GAT+GFS
3
512
3e-4
0.4
SAGE
512
3e-5
SAGE+GES
3
512
3e-4
GT
2
512
3e-5
GT+GFS
3
512
3e-5
0.2
Computers
SGC
3
512
3e-4
0.2
SGC+GFS
3
512
3e-3
0.2
APPNP
2
512
3e-5
0.2
APPNP+GES
3
512
3e-4
0.2
ACMGCN
2
512
3e-5
0.2
ACMGCN+GFS
3
512
3e.4
0.4
FAGCN
2
512
3e-S
0.2
FAGCN+GFS
3
512
3e-3
0.2
C.4 OPTIMIZATTIN--ASEE GRAPH FEATURE SELECTION
We implement optimization based metrics Asoa and 0 mentioned in Section 5
Osan: We use two learnable weight matrices, WGNN RM and WMLP ε R.M to guide feature selection for GNN and MLP. For the input node features g NxM we divide them into GNN-favored part Xş and GNN-disfavored part X.g, computed follows:
W
Xg
GNN
W X
(23)
W
W
W
MLP +
W WMPP
X
X
(24)
W2
W2
GNN
ML. +
22

---

Published as a conference paper at ICLR 2025
Table Hyperparameters for GNN baselines and GNN+GFS on Fitness.
Dataset
Model
Num of Layers
Hidden Dim
Learning Rate
Weight Decay
Dropout
GCN
512
3e-5
0.2
GCN+GFS
3
512
3e-4
0.2
GAT
128
3e-3
0
0.2
GAT+GFS
512
3e-4
0.2
SAGE
512
3e-5
0.2
SAGE+GES
512
3e-4
0.2
GT
512
3e-5
0.2
GT+GFS
512
3e-4
0
0.2
Fitness
SGC
512
3e:3
0.2
SGC+GFS
512
3e-3
0.2
APPNP
512
3e-5
0.2
APPNP+GFS
512
3e-4
0.6
ACMGCN
512
3c-5
0.2
ACMGCN+GFS
512
3e-4
0.6
FAGCN
512
3e-5
0.2
FAGCN+GFS
512
3e-3
0.2
Table 7: Hyperparameters for GNN baselines and GNN+GFS on History.
Dataset
Model
Num of Layers
Hidden Dim
Learning Rate
Weight Decay
Dropout
GCN
256
3e-3
0.4
GCN+GFS
512
3e-5
0.2
GAT
512
3e-5
0
0.2
GAT+GFS
2
512
3e-5
0
0.2
SAGE
2
512
3e-5
0
0.2
SAGE+GES
512
3e-5
0.2
GT
512
3e-5
0,2
GT+GFS
512
3c-4
0.2
History
SGC
256
3e-3
0.2
SGC+GFS
512
3e-3
0
0.2
APPNP
512
3e:5
0.2
APPNP+GFS
512
3e-3
0.2
ACMGCN
512
3e-4
0,2
ACMGCN+GFS
512
3e-4
0.6
FAGCN
512
3e-5
0.2
FAGCN+GFS
3
512
3e-3
0.2
Table 8: Hyperparameters for GNN baselines and GNN+GFS on Photo.
Dataset
Model
Num of Layers
Hidden Dim
Learning Rate
Weight Decay
Dropout
GCN
256
3c-3
0.4
GCN+GFS
512
3e-5
0.2
GAT
128
3e-3
0.2
GAT+GFS
512
3e-4
0.2
SAGE
512
3e-5
0
0.2
SAGE+GFS
512
3e-4
0
0.2
GT
512
3e-4
0
0.2
GT+GFS
512
3e.5
0
0.2
Photo
SGC
512
3e-3
le-3
0.2
SGC+GFS
512
3c-3
0.2
APPNP
512
3e-5
0
0.2
APPNP+GFS
3
512
3e-4
0
0.4
ACMGCN
512
3e-3
0
0.4
ACMGCN+GFS
512
3e-4
0.6
FAGCN
512
3e-5
0.2
FAGCN+GFS
512
36-
0.2
23

---

Published as a conference paper at ICLR 2025
Table 9: Hyperparameters for GNN baselines and GNN+GFS on Amazon-Ratings
Dataset
Model
Num of Layers
Hidden Dim
Tearring Rate
Weight Decay
Dropout
GCN
2
512
3e-4
le-3
GCN+GFS
2
512
le-3
512
3e-5
GAT+GFS
3
512
3e-4
SAGE
2
512
3c-5
0.2
SAGE+GFS
3
512
3c44
0.4
GT
2
512
3e-5
0.2
GT+GFS
3
512
3e-5
0.2
Amazon-Ratings
SGC
3
512
3e-3
0.2
SGC+GFS
2
512
3e-3
0.2
APPNP
2
512
3e-5
0.2
APPNP*GES
2
512
3e-3
0.4
ACMGCN
2
512
3c4
0.4
ACMGCN+GES
3
512
3e4
0.6
FAGCN
2
512
3e-5
FAGCN+GFS
2
512
3e-3
Table 10: Hyperparameters for GNN baselines and GNN+GFS on Minesweeper.
Dataset
Model
Num Layers
Hidden Dim
Learning Rate
Weight Decay
Dropout
GCN
2
512
3e-4
0
0.2
GCN+GFS
2
128
le-3
3e-4
0.2
GAT
2
512
3e-4
0
0.2
GAT+GFS
2
512
3e-5
0
0.2
SAGE
2
512
3e-5
0
0.2
SAGE+GFF
2
512
3e-5
0
0.2
GT
2
512
3e-5
0
0.2
GT+GFS
2
512
3e-5
0
0.2
Minesweeper
SGC
2
128
3e-3
0
0.2
SGC+GFS
3
512
3e-3
0
0.2
APPNP
2
512
3e-5
0
0.2
APPNP+GFS
3
512
3e-4
0
0.2
ACMGCN
3
512
3e-5
0
0.2
ACMGCN+GFS
2
512
3e-5
0
0.2
FAGCN
2
512
3e-5
0
0.2
FAGCN+GES
3
512
3e-3
0
0.2
Table 11: Hyperparameters GNN baselines and GNN+GFS on Questions.
Dataset
Model
Num of Layers
Hidden Dim
Learning Rate
Weight Decay
Dropout
GCN
512
3e-4
0.2
GCN+GES
128
1e-4
le-3
0.2
GAT
2
512
3e-4
0
0.2
GAT+GES
512
3c-5
0
0.2
SAGE
512
3c-5
0
0.2
SAGE+GFS
512
3e-5
0
0.2
GT
512
3e-5
0.2
GT+GFS
512
3e-5
0.4
Questions
SGC
3e-3
SGC+GFS
512
3e-5
0.6
APPNP
512
3e-5
0.2
APPNP+GES
3
512
3c-5
0
0.6
ACMGCN
2
512
3e-5
0.2
ACMGCN+GFS
2
512
3e-5
0.2
FAGCN
2
512
3e-5
0.2
FAGCN+GFS
512
3e-3
0.2
24

---

Published as a conference paper at ICLR 2025
Table 12: Hyperparameters for GNN baselines and GNN+GFS Koman-Empiin
Dataset
Model
Num of Layers
Hidden Dim
Learning Rate
Weight Decay
Dropout
GCN
2
512
3e-5
0.2
GCN+GFS
3
128
le-2
le-4
0.3
GAT
2
128
3c-3
0
0,4
GAT+GFS
3
512
3e-4
0
0.6
SAGE
2
512
3e-3
0
0.6
SAGE+GES
3
512
3e-4
0
0.4
GT
2
512
3c-3
0.8
GT+GFS
3
512
3e-4
0.6
Roman-Empire
SGC
2
128
3e-3
0.4
SGC+GFS
2
512
3e-3
0.2
APPNP
2
512
3e-5
APPNP+GFS
2
512
3e-5
0.2
ACMGCN
2
512
3c-3
ACMGCN+GES
3
512
3e-4
FAGCN
2
512
FAGCN+GFS
2
512
3e-3
Table 13: Hyperparameters for GNN baselines and GNN+GFS on Tolokers.
Dataset
Model
Num of Layers
Hidden Dim
Learning Rate
Weight Decay
Dropout
GCN
512
3e-4
1c-5
0.2
GCN+GFS
128
le-3
3e-5
0.2
GAT
512
3e-5
0
0.2
GAT+GFS
512
3e-5
0
0.2
SAGE
512
3e-5
0.2
SAGE+GES
3e-3
GT
512
3e-5
0
0.2
GT+GFS
512
3e-5
0
0.2
Tolokers
SGC
512
3e-3
0
0.2
SGC+GFS
512
3e-3
0
0.2
APPNP
512
3e-5
0.2
APPNP+GFS
512
3e-3
0.2
ACMGCN
3e-4
0.2
ACMGCN+GES
512
3e-4
0
0,2
FAGCN
512
3e-5
0
0.2
FAGCN+GFS
512
3e-3
0
0.2
25

---

Published as a conference paper at ICLR 2025
where is set to 10- to prevent division by zero. In this formulation, WeNN represents the proportion of each feature that is GNN-favored, while WMLP represents the proportion that is MLP-favored. The sum of Xg and g equals the original input X, indicating that the features are divided based on the relative influence of GNN and MI.P preferences. These weight parameters
updated during the training process.
Hard We use learnable weight matrix, W IR RRxx22 guide feature selection for GNN
and MLP, For the input node features X E R R-N we divide them into a Xg and GNN-disfavored part X G. computed as follows:
part
MHad -ofmbee-Softmax (Ward)
(25)
X X MMara 0]
(26)
g = X- (MMaaa((;
(27)
In this case, Mitard RMxx is the output of the Gumbel-Softmax applied to W Hard For
each row in MHadd one value will be and the other will be A value of 1 indicates the selection of that feature, meaning that either the GNN-favored part or the part is chosen for each feature. This binary selection mechanism ensures that each feature is exclusively assigned either Xg X- depending
which component is favored. The use of Gumbel-Softmaxx( (Jang et al. (017)) allows for differentiable sampling, enabling the weight parameters be updated during the training process.
C.5 DETAILS OF PRETRAINED NODE EMBEDDINGS
We implement experiments using both GCN and MLP pretrained features to evaluate their effect when applying GCN+GFS on either the original node features, X, or on the pretrained node em- beddings, MLP(X) and GCN(X, A). The results of this evaluation are presented in Figure 7 in the main text.
MLP(X):: We pretrain a Multi-Layer Perceptron (MLP) model with the following setup: 2 layers, hidden dimension of 1 128, trained for 1000 steps using the ELU activation function. For each of the 10 data splits, the MLP is trained separately with the Adam optimizer, using learning rate of Le 2 and a weight decay of 5e 4. The pretrained MLP is then used to extract MLP(X) corresponding to the training splits, which serves as the node embeddings for the experiment.
GCN(X, A): We pretrain a Graph Convolutional Network (GCN) with 2 layers, a hidden dimension of 128, and trained for 1000 steps. The activation function used is ReLU, with a dropout rate of 0.5. Similarly, GCN trained separately on each of the 10 data splits using the Adam optimizer, with learning rate of Ie 2 and a weight decay of 5e 4. The pretrained GCN then used extract GCN(X, A), corresponding to the training splits, as the node embeddings.
26

---

Published as a conference paper at ICLR 2025
D MORE EXPERIMENTAL RESULTS
D.1 GCN+GFS ON ADDITIONAL DATASETS
We report the performance of GCN+GFS on additional datasets including Squirrel, Chameleon, Ac- tor, Texas, Comelll Wisconsin, Cora, CiteSeer, and PubMed. The dataset statistics and descriptions are shown Figure
Table 14: Addtional Dataset Statistics
Dataset
#Nodes
#Bdgee
#Feetures
#Classes
Ave. Degrees
Rnode
hedge
hetasa
hadi
Avg. TF1
Squirell
2223
46998
2089
21.14
0.1759
0.2072
0.0725
-0.0076
0.0173
Chameleon
2325
0.2273
0.2361
Actor
7600
26659
932
3.51
0.2197
0.2167
0.0074
0.0015
0.0029
Texas
183
279
1703
1.52
0.0748
0.0609
0.0017
-1.4288
0.0348
Cornell
183
1703
1.51
0.1246
0.1227
-1.1984
0.0202
Wisconsin
251
450
1703
1.79
0.1934
0.1778
0.0447
-0.9947
0,0182
Cora
2708
10556
1433
3,90
0.8252
0.8100
0.7657
0.7717
0,0188
CiteSeer
3327
3703
2.77
0.7166
0.7391
0.6267
0.6673
0.0103
PubMed
19717
88651
500
4,50
0.7924
0.8024
0.6641
0.6836
0.0440
the performance of the GCN+GFS model surpasses the standard GCN across additional datasets. The selective feature processing approach provided by GFS allows GNNs to focus beneficial features, leading to improved performance across diverse datasets.
As shown in Figure 10
1.0
Accuracy AUC-ROC
0.8
0.6
0.4
0.2
CCN+GFS
GCN
0.0
Cora
Pubmed
Citeseer
Actor
Chamoleon.
Coreell
Squirral.
Texas.
Wisconsin
Figure 10: Performance on additional datasets.
D.2 SENSITIVITY GFS
Figure shows the response of GCN+GFS
other datasets, where GFS collapses a GNN when r 1.0 or an MLP when 0.0, as all features are sent to GNN or MLP, respectively. Al- though some datasets like Minesweeper and Questions, where all the features may favor GNNs, GFS doesn't 't significantly improve GCN performance, it doesn't diminish GFS's overall effective- ness on most datasets.
Figure 12 shows how GCN, MLP, and GCN+GFS respond to changes in number of layers, dimen- sion of hidden embeddings, learning rate, weight decay, and dropout rate on homophilous graph (Computers) and heterophilous graph (Amazon-Ratings outperforms GCN, but for certain hyperparameter settings on the Questions dataset, GFS does not consistently outperform GCN. This is likely because the node features in these datasets are already highly graph-favores making the additional feature selection offered by GFS less impactful. These results highlight the robustness of GFS under different hyperparameter settings.
other 6 datasets. In most cases, GFS
27

---

Published as a conference paper at ICLR 2025
Fitness
History
Photo
Accuracy /UC-ROC
0.85
Annuracy AUC-ROC
Accuracy /ACC-RO
0.90
0.85
0.85
0.84
0.80
0.80
0.83
0.70
0.75
0.82
0.0
0.2
0.4
1.0
0.2
0.4
0.6
0.2
0.4
0.6
0.8
Amazon.
Mines.
Questions
Ancuracy /AUC-ROC C--
0.52
Accuracy /AC-ROC
Ancuracy /UC-ROC
0.90
0.76
0.51
0.80
0.50
0.74
0.70
0.49
0.60
0.72
0.48
0.50
0.47
0.0
0.2
0.4
0.6
0.8
1.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Figure 11:
The performance GCN+GFS is shown as the ratio r of GNN-favored features TFI increases. The point representing the best performance is highlighted *
D.3 PERCENTAGE OF SUPERVISION IN TFI
In Figure 13 the influence of the supervision percentage in TFI on the performance of GCN+GFS across other 6 datasets is shown. The results highlight that the supervision percentage increases, the performance of GCN+GFS generally improves or stabilizes. Even with only 10% supervision, GCN+GFS outperforms the standard GCN, demonstrating the robustness of the TFI approach. For datasets like Fitness and Amazon, there is a noticeable improvement in performance as the supervi- sion percentage increases, but it stabilizes at around 30%. In datasets such History, Photos, and Mines. the model performance is relatively stable, indicating that a small percentage of supervision sufficient achieve optimal performance. The performance on Questions shows more variability with increasing supervision, possibly due to the graph-favored dataset's 's features. This figure sup- ports the claim that TFI requires minimal supervision to improve GCN+GFS performance, which particularly useful in semi-supervised learning settings.
D.4 THE NUMBER OF K-HOP NEIGHBORS IN TFI
We further investigate the influence of the number of neighbor hops in TFI on the performance of GCN+GFS. We k from (1, 8) as shown Eq. (5). illustrated in Figure 14 increasing the number of neighbor hops does not significantly affect model performance and results in only minor deviations in most datasets. This indicates that 1 I-hop neighbors are sufficient for TFI to select GNN-favored or GNN-disfavoree features.
D.5 PERFORMANCE ON DATASETS WITH PUBLIC SPLITS
We further conduct the experiments these datasets with public splits shown in Table 15 Ac- curacy results on node classification are reported for Cora, PubMed, and CiteSeer, each having one split. For the other datasets, which have ten splits, we provide both accuracy and standard deviation. The results demonstrate that GNN+GF5 outperforms baseline GNN most datasets for types of GNN backbones, which include GCN, GAT, SGC, and GraphSAGE.
28

---

Published as a conference paper at ICLR 2025
Table 15: The performance of graph feature selection on GNN baselines in public split datasets.
Model
Cora
Puhmed.
Citeseer
Actor
Chameleon.
Cornell
Squirrel
Texas.
Wisconsin
GCN
81.28
78.06
33.6840.66
62.41.1.99 62.68+2.02
60.81:6.53
48.41&1.52
66.4916.14
77.65.4.91
GCN+GFS
81.94
78.46
71.74
35.30=0.88
68. 11:147
49.13+1.78
71.57=6.50
80.39+4.89
A
+0.66
+0.40
+1.14
+1.62
+0.27
+7.30
+0.72
+11.08
+2.74
GAT
80,16
76.92
69.88
33.07+0.77 79
65.7942.55
63, 63.24+4.97
51.63+1.47
74.05+4.97
76.47+3.70
GAT+GFS
81.06
77.94
72.24
35.23+0..6 83
66.4942.11
10.1945.88 87 +5.95
82.15+1883
77.8424.38
81,5726.21
+0.90
+2.16
+0.70
+0.52
+3.79
+5.10
SGC
80.74
76,96
69.84
30.6420.75
14.39=1.79 1,79
43,24=7.66 64
39.91#1 566
54.8624.42
53,73+5 64
SGC+GFS
81.48
76.38
71.18
35.6241.22 1.20
54,63&2.14
67.30u4.88 84
42.19a1.12
71.624.22
78.6346.43
A
+0.74
-0.58
+1.34
+4.98
+0.24
+24.06
+2.28
+16.76
+24.90
SAGE
81.06
76.80
68.52
36.20%0.88 81
58.42x2.53
70.00=7.03
39.32 06
74.59
81.18+2.11
SAGE+GFS
71,46
36.47+0.72 +0.27
58.71+1.99
71.08+5.70
39.28+1.49
81.08+5.55 55
83.1445.00
A +0.82
+1.60
+2.94
+0.29
+1.08
-0.04
+6,49
+1.96
D.6 IMPACT OF LABEL HOMOPHILY
To investigate how GSF performs under varying label homophily, we conduct experiments on syn- thetic datasets using CSBM-H (Luan et al. 2024bb Zheng et al. (222a)) to control homophily levels. Specifically, in the CSBM-H model, for node a labeled with y. its features X, RM are sampled from a wlase-wise Gaussian distribution, specifically X,, Ny, (Hy,, Σy.). Each dimension of X independent of each other. To construct the graph structure 9 with a specified homophily degree h. the node has probability h of connecting to nodes with the same label and probability of I-h
C-T of connecting to nodes with different label. We randomly generate 10 graphs with different seeds our experiments to reduce uncertainty. Each graph has 1000 nodes with 10 features and classes. The node degrees are uniformly sampled from [2,8].
We demonstrate how the performance of GCN+GFS varies with label homophily across the range [0.1, ... 0.9] in Table 16 and Figure 15 for node classification tasks. Generally, GCN+GFS outperforms both GCN and MLP across different levels of label homophily, indicating that GFS addresses the limitations of GCN under low homophily and of MLP under high homophily through effective feature selection. Furthermore, the relative increase A (defined as the accuracy gap be- tween GCN+GFS and GCN, divided by the accuracy of GCN) decreases as label homophily in- creases. This finding is expected, as higher homophily levels typically enhance the performance of all graph-au graph-aware aware models, thereby limiting the potential for improvement through GFS.
Table 16: Impact of label homophily on GFS.
Label Hamophily
0.2
0.3
0.4
0.5
0.5
0.7
0.8
0.9
MLP
73.1224.06
21.24
68-4416 93
70.3625.22 30
66.9216.36 36
67
67.5214.64
70.12:5.49 49
69.8814.77
69. 69.3224..6 6.68
GCN
50.12:3.09
50.5613.28
56.00&5.44
61.24a5.30
65.84a5.11
72.32x2.35
80.3223.44
87.6422.23
90.64a2.32
GCN-GPS
73.24=3.85
69 96.9647224
69 69.0446.76
71.36+6.17
69.68+7.69
73.844465
82.76=6.65
89.3654
91. 91,20+6.04
& (relative) 23.12%
19.40%
13.04%
10.12%
3.84%
1.52%
2.44%
1.72%
0.56%
D.7 IMPACT OF FEATURE DIMENSION
We investigate the impact of feature dimension size based on the CSBM-H model (uan et al. 2024b. Zheng et al. (024a), following the setup described in Section D.6 As shown in Table 17 and Figure 16] GFS demonstrates pronounced advantage over GCN as the feature dimension increases. This outcome is expected, as more features allow GFS to exhibit higher fault tolerance during feature selection. In real-world scenarios, where the feature dimensions in datasets reach hundreds or thousands, as indicated in Table 3, GFS is likely to perform well.
Table 17: Impact of size of feature dim on GFS.
#Feature Dim
10
20
30
40
50
MLP
67.5214.64
77.76+3.53
83.40=2.58
87.60=1.89
91.12+1.88
GCN
72.32+2.35
79.20+2.15
21.40
84.64+3.07
8.78
GCN-GFS
73.8414.65
84.1243.70
87.40+3.68
90.8443.51
94.12+3.37
A (relative)
2.06%
5.85%
6.86%
6.83%
6.71%
29

---

Published as a conference paper at ICLR 2025
D.8 IMPACT OF FEATURE SPARSENESS
We investigate the impact of feature sparseness on the CSBM-H model (Laa et al. 2024b Zheng
al., 2024a), following the setup outlined in Section D.6 To control sparseness in synthetic datasets, we randomly mask a certain percentage of node features. As shown in T and Figure
17 the results indicate that the advantage of GFS diminishes as feature sparseness increases. Al- though the relative increase rate of GFS decreases in this context, it still outperforms GCN. This is attributed to the presence of both GNN-favored and GNN-disfavored features, which allows GFS to remain effective in enhancing GNN performance.
Table 18: Impact of sparseness on GFS.
Sparseness
10%
30%
50%
70%
GCN
54.1665.58
16.4883.77
18 38.3333343
45.68a1.90
12. 12a2.88 88
39. 59.52.2.86
34 31.855.559
52.0812.05
29.9212.75
25.1213.07
GON-GFS
67, 99.77.77
39 8884666. 83
60.04:5.44 49
$4.92a5.47
50.6424.15
46.20us 46.20.5.26
39 39.5246.72 72
35.6813.55
31.3611.09
27.24+3.65
A (ralarive)
25.48%
28.74%
22.83%
20.23%
20.23%
16,90%
13.30%
11.22%
4.81%
8.44%
D.9 IMPACT OF NOISES IN FEATURES
To investigate the impact of feature noise on GFS, we conduct experiments on four real-world datasets: Children, Computer, Roman Comann-Emprre, and Tolokers, by introducing noise to the original node features. Specifically, after normalizing the imput features, we add Gaussian noise multiple times. As shown in Table and Figure 18] the relative increase A (defined as the accuracy gap be- tween GCN+GFS and GCN, divided by the accuracy GCN) remains relatively stable, indicating that the performance of GFS is robust against feature noise.
Table 19; Impact of noise on GFS.
Datasets
Noise
10
20
GCN
12.92+1.17
52.93+0.62
52.57=0.75
52.32+0.89
52.20+0.75
Children
GCN-GFS
58.1641.82
57.56+2.64
57.01=3.24
56.31+3.78
55.87+3.94
(relative)
9.89%
8.75%
8.44%
7.65%
7.04%
GCN 84.39+
84,42+0.85
83.07=1.09
82.46+0.77
11.6840.94
Comp.
GCN-GFS
88.87+2.82
88.50+2.25
87.80+3.17
87.10+3.40
85.0145.18
A (relative)
5.31%
4.84%
5.69%
5.63%
4.08%
GCN
84.11+0.330
83.96+0.33
83.6520.40
83.23+0.33
83,00+0.25
Roman.
GCN-GFS
84.85+0.80
84.57+0.77
84.19=0.92
83.73+0.91
83.5940.83
A (relative)
0.87%
0.73%
0.64%
0.60%
0.71%
GCN
1..83+1.41
83.28+0.73
82.81
82.50+0.77
82. 02.1240.62
Tolokers
GCN-GFS
16.52+1.85
85.99+1.79
85.48+1.54
84.99+0.87
84.44+1.75
A (relative) 4.46%
3.25%
3.23%
3.01%
2.81%
D.10 GOOD HETEROPHILY ON FEATURE HOMOPHILY
In this section, we show the phenomenon of "good heterophily" occurs in feature homophily. First, we split node features into 10 bins according to the values of feature homophily. Then, we run GCN on these bins separately to see how the model performance changes with feature homophily. As shown in Table 20 and Figure 19 the GCN performance remains good under a low value of feature homophily, which includes attribute homophily (Yang et al., (021) (hattr). local similarity
(Chen et al. 2023) (hhsim-euc). and class-controlled feature homophily (Lee et al. 2024) (hCTF). This result indicates these consistency-bart based feature homophily cannot align well with GNN perfor- mance, which is similar to the phenomenon in label homophily (Ma et al. 2021 Luan et al. 2024bb Zheng al. 2024a). Conversely, GNN performance consistently increases with the increase of our proposed TFI, implying its effectiveness in selecting GNN-favored or GNN-disfavored features.
D.11 STATISTICAL OF SELECTED FEATURES
As shown Table 21 we present the statistics of values, sparsity, and THI for all features, GNN- favored features, and GNN--disfvored features. The results indicate that: (1) the THI of GNN-
30

---

Published as a conference paper at ICLR 2025
Table 20: GCN performance on different bins of feature homophily metrics.
Roter
31,9410.16
31,51a095
30.2611.82
30.2144.76
30.02=4.99
30.0743.95
29.344-1.30
30.8512.59
29.9613.90
31.4022.26
Ramn-au
31,9210.18
29.5215.15
31.1312.02
31.2121.61
30.7523.65
30.1614.75
31.3811.40
31.1911.86
31.2211.61
30.8913.10
kore
51,9450.16 16
31 93
3014444
30.55a3.37
30.79:2.97
31,55:0.92
30,08:4.67
31.31s1.78
30.75s2.75
31,92s1.74
43.2240.55
favored features is significantly higher than that of GNN-disfavored features; (2) the datasets en-
coded by Pretrained Language Models (PLMs), including Children, Computers, Fitness, History, and Photo, exhibit similar values in features, yet their TFI varies considerably. Notably, higher average TFI across all features correlates with better GFS performance; (3) GFS is less effective datasets with higher sparsity, such as Minesweeper and Tolokers. (4) Datasets with lower ho- mophily tend to identify more features as GNN-disfavored, whereas those with higher homophily identify features as GNN-favored.
Table 21: Summary Statistics of Values, TFI, and Sparseness for Selected Features.
Value
Sparseness
TFI
Value
Sparseness
Value
Sparseness
Children
0:000200 4081
0.00%
0:0776-00155
01558
0.000-0-4088
0.00%
0.606-0.0143
0.0287-0.428 0.402
0,00%
0.01S04 D079
50%
Comp.
0.0201-0
4068
0.005
00065-00222
0.0201 +0 4063
0.09%
0.1010-00050
0.0202-0.4000 4.400
0,00%
0.4785+0.0001 0.001
80%:
Firazs
0.:9550
3928
0.00%
0.844100020
1119540.290 3929
0.095
0.196-660053
0.0195+0.390 3927
0,00%
0.168540.00119
70%
History
082201-0 4113
0.00%
0:412-000261
41220140.4654
0.095
0059416.0288
0.000:10.4044
BL00%
0.02300Q.0807
5US
Photo
0/0202:00 4056
0.00%
0:0680-00027 0278
0.0201 40.4052
0.00%
00812-0:0085
0,020=00 4088
0,00%
0.0475+0.007 0007
60%
Amazun
00003-00573
0.005
0.0914-0 0065
0.003+0.05773
0.00%
00957-0.0040
0.0003-0.0555
0,08%
0.0850+ DM76
60%:
Mines.
0:1429-0 3499
85.71%
0.3169-00:344
0.142940.34P 3499
85.719-
00510-0.0966
0.142540. 3499
85.71%
0.0014+0.0001
80%:
Qucslion
2.0007--0.0510
15.18%%
0
30067-60001
15.19%
00077-0:0010
-0:007-00000
15.095
0.005550..000 0004
90%
Roman.
0:0005:0.0990 0986
0.27%
0:0624-000:66
0.0077
0:1500=0:088
0,0006e0.000 0979
0
0.40 0.011200.016
20%
tolokers
0,3825-0 4567
48.00%
0,0295-00113
0.3626:0-4565
48,00 48.09%
0.049:0000.5
0,182)=0.4570
47.98%
0.0169+0.0000
0.0005
70%
31

---

Published as a conference paper at ICLR 2025
Children
Children
Children
Childrea
Children
0.60
Aecurgy (2000 ADC
Aucumayy ROC-AUC
9.58
8.575
0.50
0.53
0.56
9.050
0.58
Aocemay
0PS GCN MIP
0.525
GP5
0.45
GPS
GPS
9.50
626
0.02
0.500
OCN MIP
0.40
GCN
GCN
0.45
9 5.6.5 an 6.6
Lapyeers
Hidden Dimoniion
Learning
Weight Decay
Dropout Rate
Fitness
Fitzess
Filzess
Fitness
Fineess
Necunes /Hocase
Meunce HRO AtC
Meuncy /RC ROCCAUe
0.90-
Mccuray ROC
0.8
9.8
0.85
0.85
Neturay
5.00
GCN
GrS
0.8
Ge5
Hiddon Dinettiio
Learning
Dropout Rate
History
History
History
History
History
ROC-Ate
0.81
nopaaie
Accuray ROC ADC
Accurey ROC ADC
0.830
0.84
6 0.825
0.890
0.82
0.82
0.775
9.6
0.82
0.750
0.81
Sumber of Layers
Hidden Dincrsion
Learning Rate
Whight Daray
Photo
Photo
Photo
Photo
NOC AUC
Acurses /ROC AUC
Ausurcc ROC-AUC
Aoarruu ROC ABC
0.8
8.8
0.8
0.0
0.7
9.7
GCN
GGN
GCN
MUP
Number of Layses
Hidden Dunataton
Learning Rate
Whight Docay
Rate
Minas
Mines:
Minos.
Aoseeee ROCAUC
Accussy BOC AUC
Accursy 200 AUC
Avourary ROCAUC
Avourer ROCAUD AUC
9.8
0.7
0.7
0,7
0.6
GCN
CKN
0.6
OCN
0.6
OCN
9.
MLP
MEP
8.5
N2.2
0.5
M2.2
Layess
Middden Bimansdon
Learning
Weight Decay
Drogout Rate
Questions
Questions
Quasstions
Ousstions
Quassionss
Accorney ROCAtICC
HOC. HOC.AUC
0.76
0.75
ROC
0.76-
Accurscy/ ROC
8.74
0.70
0.72
0F%
0.73
8u65
GPS
GFF
0.70
024
RON
0.70
0.60
0.70
Hidden Bimerssoo
Wi
Bropout Rate
Roman.
Roman
Roman
Roman.
Roman.
Accursey Poc Atic
Accursey 200 Atfc
Accurae 30C.AM AUC
0.80
0.8
0.89
Accurnyy
3.5
0.75
8.7
0.70
0.70
0.70
3.6
0.85
0.6
0.85
0,0
105
Number
Learning Hate
Weight
Brogout
Tolokers
Tolokers
Tolokors
Tolokers
Tolokers
0.05
Actrreyy PROCAUI
Accurssy RRC AUC
Accuracy AUC
0.85
Accurasy FRO.
8.00
0.60
Accurnny
0.75
8.5
0.75
Mi
5 &
Number of Layser
Midden Timension
Learning Rato
Whightt Docar
Figure 12: Response of GCN+GFS, GCN, and MLP to hyperparameters on more datasets.
32

---

Published as a conference paper at ICLR 2025
Fitness
History
Photo
Accuracy ADC-AUC
0.96
Accuracy /ROC-AUC AC-
Accuracy /ROC-AUC
0.94
0.88
0.90
0.92
0.86
0.88
0.90
0.84
0.86
0.88
GCN
0.82
GCN
0.84
GCN
0.86
GFS
GFS
GFS
0.80
0.82
0.1
0.3
0.5 0.7 0.9
0.1
0.3
0.5 0.7 0.9
0.1
0.3
0.5
0.7 0.9
Supervision Percentage
Supervision Percentage
Supervision Percentage
Amazon.
Mines.
Questions
0.56
Accuracy ROC-AUC
Accuracy ROC-AUUC AUC
0.94
Accuracy ROCCAUUC -UC
0.80
0.54
0.92
0.78
0.52
0.90
0.76
0.50
0.88
0.74
GCN
GCN
GCN
0.48
GFS
0.86
GFS
0.72
GFS
0.1 0.3
0.9
0.1
0.3
0.9
0.1
0.9 Supervision Percentage
0.3
0.7
Supervision Percentage
Supervision Percentage
Figure 13: Influence of the percentage of the supervision in TFI on the model performance of GCN+GFS.
Children
Comp.
Rtress
History
Phote
0.62
Xcunty /00-aM
0.c0-
Zetnnry MABC-A
2,50
0.22
6.86
6.58
0.850-
0.30
0.84
6.925-
0.66
0.000
0.00
6.775
Kioe
K.pep
Hin
KFppp
100
Questions
0.92
0.0
Jccueey RRCCALE
0.75
2.00
0.60
0.60-
0.52
6.62
0.66-
0.30
2.8
0.22
0.20
0.15
K.rop
Figure 14: Influence of the number of neighbor hops in TFI the performance of GCN+GFS.
0.9
0.90
0.8
0.80
0.70
GPS
GPS
GCN
GCN
MLP
0.2
0.4
50
Comparison of the performance of MLP, GCN, and GFS across varying levels of la- bel homophily in synthetic datasets.
Figure 15:
Figure 16: Comparison of the performance of MLP. GCN, and GFS with the increase of the number of feature dimensions.
33

---

Published as a conference paper at ICLR 2025
0.74
0.4
0.3
GCN
0.2
0.0
0.2
0.4
0.6
0.8
Figure 17: Comparison of the performance of MLP, GCN, and GFS with the sparseness in synthetic datasets
Children
Comp.
Roman.
Tolokers
0.60
0.88
0.900
0.58
0.85
0.875
0.86
0.56
0.850
0.84
0.84
0.54
0.825
0.52
OCN
0.800
GCN
0.83
GCN
CCN
20
10
20
20
Figure 18: Comparison of the performance of MLP, GCN, and GFS with the increase of feature noises
0.375
0.375
0.350
0.350
Accuracy
Accuracy
0.325
0.325
0.300
0.300
0.275
0.275
0.250
0.250
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.8
0.9
1.0
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
hattr
haim
0.375
0.50
0.350
Accuracy
Accuracy
0.45
0.325
0.40
0.300
0.35
0.275
0.30
0.250
0.25
0.0
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
0.0
0.1
0.2
0.4
0.5
0.6
0.7
0.8
0.9
hCTF
TFI
Figure 19: Performance of GCN under varying levels of feature homophily or TFL.
34


Reviews:
[{"review_id": "OwW6jWkhPb", "reviewer": "Reviewer_jpAB", "summary": "This paper investigates the problem of feature selection for graph Convolution Networks (GCNs). It begins by introducing TFI, a metric designed to guide the selection of relevant features. Following that, it introduces GFS, a plug-in method that distinguishes between features that benefit graph convolution and those that do not contribute positively or may even have a negative impact. Then, the two sets of features are processed separately using GCN and Multi-Layer Perceptrons (MLP), respectively. Evaluations on node classification tasks demonstrate the effectiveness of the proposed TFI and GFS.", "questions": "See Weaknesses.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1 This paper is well-organized and easy to follow. \n\n2 The figures regarding the design motivation and the proposed framework (Figures 1 and 3) are clear.\n\n3 The theoretical proofs are detailed.", "weaknesses": "1 The proposed feature selection metric TFI, which leverages mutual information between features and labels, is not novel. This is a conventional approach for feature selection, as outlined in [1]. Although the TFI utilizes the features derived from the neighborhood average (AX), the approach is incremental. \n\n2 This paper lacks a comparative analysis of classic feature selection methods, such as [2]. \n\n3 The GPS exhibits limited robustness to the selection of the hyperparameter r.\n\n4 It is unclear how the high-pass filters of FAGCN and ACMGNN, as presented in Table 1, align with the TFI metric with low-pass AX.\n\n5 A significant concern is the applicability of the proposed GFS. The paper asserts that TFI is computed on training nodes. Thus, I am concerned that the unusual dataset division of 50/25/25 for training/validation/testing employed in this paper is crucial for the effectiveness of GFS. The question then becomes: how would the results be affected if a public splitting, such as 20 per class for training in GCN, SGC, GAT, and APPNP, were applied?\n\n[1] Feature Selection: A Data Perspective. ACM Comput. Surv. 50(6).\n\n[2] Multi-label Feature Selection via Global Relevance and Redundancy Optimization. IJCAI 2020.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "X9FV1fHwVb", "reviewer": "Reviewer_iSuK", "summary": "The paper introduces a new method to identify features that are favored and disfavored by GNNs. It uses topological feature selection to integrate these features into GNNs, leading to significant improvements in their performance.", "questions": "1. As shown in Figure 2, the performance gap for the Roman dataset between GCN and MLP in regions with low TFI is minimal. How is the improvement over GCN in Figure 4 so significant regardless of $r$?\n2. Since features are updated in each GNN/MLP layer, is the TFI computed at each layer to get (dis)favored features, or is it only computed at the beginning? This point does not seem clearly explained in the paper.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-motivated and presents a novel approach to distinguish between GNN-favored and GNN-disfavored features, treating them separately to learn with different methods.\n2. The paper is clearly presented and includes solid theoretical guarantees, making it easy to follow.\n3. The authors provide sufficient empirical analysis, including ablation studies and comparisons with state-of-the-art methods.", "weaknesses": "The evaluation method is not fair enough. The performance results in Table 1 should be presented with only the difference of with or without GPS, while keeping the model architecture the same (e.g., number of layers and hidden dimension).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jGzfhE4Spt", "reviewer": "Reviewer_poRT", "summary": "GNNs have strong graph learning capabilities. However, GNNs are not effective at learning all graph data features. In previous work, many metrics based on graph topology and features have been proposed to describe feature homogeneity in order to assess the effectiveness of GNNs. However, there has been no metric that can directly and effectively guide which features GNNs should learn. To fill this gap, this paper proposes an evaluation metric, Topological Feature Informative (TFI), to compute the learnability of features for GNNs. Subsequently, a dual-channel embedding architecture combining GNNs and MLPs is used to embed these features separately.", "questions": "1. How is the decomposition of node features in the graph executed? \n2. What impact does the dimension of the initial node features have?\n3. Can this method only be applied to node features, or can it also be used on edge features, and what would be the effect?\n4. Is it possible to measure what a \"good heterophily\" feature looks like? Can other metrics be used for supplementary description, such as Label Homophily, Feature Homophily, Mutual Information, and so on?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The writing of the paper is clear and easy to understand, with a well-defined theme.\n2. The paper provides a theoretical foundation for using TFI to guide feature selection through simple proofs.\n3. The experimental results on node classification tasks are impressively good.", "weaknesses": "The paper does not provide any statistical analysis of the selected features, which raises some questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WlxgDEZh06", "reviewer": "Reviewer_v9Jq", "summary": "The paper explores the varying impact of graph convolution across different feature dimensions in GNNs. It introduces a novel metric called Topological Feature Informativeness (TFI), designed to identify features that are either favored or disfavored by GNNs. The authors propose a Graph Feature Selection (GFS) method that leverages TFI to process GNN-favored features with GNNs and GNN-disfavored features with MLPs, enhancing overall performance. The experimental results across multiple GNN architectures and datasets demonstrate that GFS improves accuracy with minimal computational overhead. The work emphasizes the importance of feature-aware processing in GNNs, showing that graph convolution is not uniformly beneficial for all feature dimensions.", "questions": "Q1: I am still unclear about how TFI fundamentally differs from other mutual information-based metrics for feature selection in graph learning. Could the authors provide a clearer explanation of its unique contributions and advantages?\n\nQ2: How does the proposed method handle extremely sparse features, which are common in real-world graphs? Are there specific challenges or limitations in this context?\n\nQ3: It appears that the proposed method offers limited advantages on homophilous graphs compared to its performance on heterophilous graphs. Could the authors provide insights to clarify this discrepancy and explain why the method may be less effective in homophilous settings?\n\nQ4: I am curious about the robustness of the proposed method when applied to noisy graphs, where certain features may contain varying levels of noise. Could the authors provide insights or results on its performance in these scenarios?\"", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1: The concept of distinguishing between GNN-favored and GNN-disfavored features is interesting and novel. \n\nS2: The paper is well-written and structured, easy for readers to follow the methodology and findings. \n\nS3:  The experimental results consistently demonstrate the effectiveness of the proposed GFS trick.", "weaknesses": "W1: As far as I am concerned, the theoretical analysis of the proposed strategy is incomplete. The paper does not clearly explain the specific conditions under which distinguishing between GNN-favored and GNN-disfavored features is most effective. \n\nW2: The proposed TFI metric, while beneficial, appears similar to existing mutual information-based methods. The authors need to provide more clarification on how TFI uniquely contributes to feature selection in graph learning, emphasizing its distinct advantages over prior approaches.\n\nW3: Although the experimental results demonstrate strong performance of the GFS-augmented models, I remain somewhat unconvinced. To strengthen the validity of the findings, it would be helpful to include experiments on synthetic datasets, such as those generated by the SBM model with varying levels of heterophily and homophily. Such experiments could offer more reliable insights into the potential advantages of the proposed methodology.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)