GSLB: The Graph Structure Learning Benchmark
Authors: Zhixun Li
Liang Wang
Xin Sun
Yifan Luo
Yanqiao Zhu
Dingshuo Chen
Yingtao Luo
Xiangxin Zhou
Qiang Liu
Shu Wu
Liang Wang
Jeffrey Xu Yu
Venue: NeurIPS 2023 Datasets and Benchmarks Poster (2023)
Keywords: Graph Neural Networks
Graph Structure Learning
Data Mining
TL;DR: We conduct a comparative study of Graph Structure Learning, analyze the pros and cons of it. Then we propose an easy-to-use library, GSLB, to promote further research.

Abstract: Graph Structure Learning (GSL) has recently garnered considerable attention due to its ability to optimize both the parameters of Graph Neural Networks (GNNs) and the computation graph structure simultaneously. Despite the proliferation of GSL methods developed in recent years, there is no standard experimental setting or fair comparison for performance evaluation, which creates a great obstacle to understanding the progress in this field. To fill this gap, we systematically analyze the performance of GSL in different scenarios and develop a comprehensive Graph Structure Learning Benchmark (GSLB) curated from 20 diverse graph datasets and 16 distinct GSL algorithms. Specifically, GSLB systematically investigates the characteristics of GSL in terms of three dimensions: effectiveness, robustness, and complexity. We comprehensively evaluate state-of-the-art GSL algorithms in node- and graph-level tasks, and analyze their performance in robust learning and model complexity. Further, to facilitate reproducible research, we have developed an easy-to-use library for training, evaluating, and visualizing different GSL methods. Empirical results of our extensive experiments demonstrate the ability of GSL and reveal its potential benefits on various downstream tasks, offering insights and opportunities for future research. The code of GSLB is available at: https://github.com/GSL-Benchmark/GSLB.

GSLB: The Graph Structure Learning Benchmark
Zhixun Li
Liang Wang"
Xin Sun
Yifan Luo"
Yanqiao Zhu Dingshuo Chen
Yingtao Luo"
Xiangxin Zhou
Qiang Liu Shu Wu Liang Wang Jeffrey Xu Yu"
Department Systems Engineering and Engineering Management
The Chinese University of Hong Kong
Center for Rescarch on Intelligent Perception and Computing
State Key Lahoratory of Multimodal Artificial Intelligence Systems
Institute of Automation, Chinese Academy of Sciences
School Artificial Intelligence. University of Chinese Academy of Sciences 'Department of Automation, University of Science and Technology of China School of Cyberspace Security, Beijing University of Posts and Telecommunication 'Department of Computer Science. University of California. Los Angeles
Heinz College of Information Systems and Public Policy, Machine Leamning Department School of Computer Science, Carnegie Mellon University
Primary contact: exli@se. cuhk, edu.hk
Abstract
Graph Structure Learning (GSL) has recently garnered considerable attention due to its ability to optimize both the parameters of Graph Neural Networks (GNNs) and the computation graph structure simultancously. Despite the proliferation of GSL methods developed in recent years, there is no standard experimental setting fair comparison for performance evaluation, which creates great obstacle understanding the progress in this field. To fill this gap, we systematically analyze the performance of GSL in different scenarios and develop comprehensive Graph Structure Learning Benchmark (GSLB) curated from 20 diverse graph datasets and 16 distinct GSL algorithms. Specifically, GSLB systematically investigates the characteristics of GSL in terms of three dimensions: effectiveness, robustness, and complexity. We comprehensively evaluate state-of-thh-ar GSL algorithms in node- and graph-level tasks, and analyze their performance in robust learning and model complexity. Further, to facilitate reproducible research, we have developed an easy-to-use library for training, evaluating, and visualizing different GSL methods. Empirical results of our extensive experiments demonstrate the ability of GSL and reveal its potential benefits on various downstream tasks, offering insights and opportunities for future research. The code of GSLB is available at: https: //gi //github. www.coom Benchmark/GGL.B
Introduction
Graphs, structures made of vertices and edges, are ubiquitous in real-world applications. A wide variety of applications spanning social network [51, 9], molecular property prediction [40, 14], fake news detection [45, 1], and fraud detection [23, 27] have found graphs instrumental in modeling complex systems. In recent years, Graph Neural Networks (GNNs) have attracted increasing attention due their powerful ability learn node or graph representations. However, most of the GNNs heavily rely on the assumption that the initial structure of the graph is trustworthy enough to serve as data collection, graph structures are
ground-truth -ruth for training. Due to uncertainty and complexity inevitably redundant, biased, noisy, incomplete, or the original graph structures are even unavailable, which will bring great challenges for the deployment of GNNs in real- cal-world applications.
"Corresponding authors: Qiang Liu (qiang. liu @ @nwpr.a..cccm) Jeffrey Xu Yu (yu@ www......u........
37th Conference on Neural Information Processing Systems (NeurIPS 2023) Track on Datasets and Benchmarks.

---

Table 1: An overview of GSLB. Both algorithms and datasets are divided into three categories: homogeneous node-level, heterogeneous node-level, and graph-level. The evaluation is divided into three dimensions: effectiveness, robustness, and complexity.
Algorithms
LDS [11], GRCN [48], ProGNN [18], IDGL [4), CoGSL [26], SUBLIME [28),
Homogeneous GSL
GEN [39], STABLE [21], NodeFormer [43], SLAPS [10], GSR [54], HES-GSL [42]
Heterogeneous GSL
GTN [49). HGSL [53]
Graph-level GSL
HL [52]. VIB-GSL [36]
Datasets
Cora [47], Citeseer [47], Pubmed [47], ogbn-arxiv [15), Polblogs, Cornell [34],
Homogeneous datasets
Texas [34]. Wisconsin [34], Actor [37]
Heterogeneous datasets
ACM [49], DBLP [49]. Yelp [29]
IMDB-II B 13], IMDB-M [3], COLLAB [46]. REDDIT-B [46]. MUTAG [5],
Graph-level datasets
PROTEINS [2]. Peptides - Fune [8]. Septides-Struct [8]
Evaluations
Homogeneous node classification (Topology Refinemen//Topolgee Inference). Heterogeneous node classification, Gruph-level lasks
Effectiveness
Robustness
Supervision signal robustness, Structure robustness, Feature robustness
Complexity
Time complexity, Space complexity
To mitigate the aforementioned problems, Graph Structure Learning (GSL) [4, 55, 30, 10, 57, 50] has become important theme in graph learning. GSL aims make the computation structure of GNNs more suitable for downstream tasks and improve the quality of the learned representations. While it widespread in different communities and the research enthusiasm for GSL is increasing, there no standardized benchmark that could offer fair and consistent comparison of different GSL algorithms. Moreover, due to the complexity and diversity of graph datasets, the experimental setups in existing work are not consistent, such as varying ratios of the training set and different train/validation/tes. splits. This poses great obstacle holistic understanding of the current research status. Therefore, the development of a standardized and comprehensive benchmark for GSL is an urgent need within the community.
In this work, we propose Graph Structure Learning Benchmark (GSLB), which serves as the first comprehensive benchmark for GSL. Our benchmark encompasses 16 state-of-theearr GSL algorithms and 20 diverse graph datasets covering homogencous node-level, heterogeneous node-level. and graph- level tasks. We systematically investigate the characteristics of GSL in terms of three dimensions: effectiveness, robustness, and complexity. Based on these three dimensions, we conduct an extensive comparative study of existing GSL algorithms in different scenarios. For effectiveness, GSLB provides a fair and comprehensive comparison of existing algorithms on homogeneous node- level, heterogeneous node-level, and graph-level tasks, where we consider both homophilic and heterophilic graph datasets for homogeneous node-level tasks, and cover both Topology Refinement (TR, i.e., refining graphs from data with the original topology) and Topology Inference (TI, i.e., inferring graphs from data without initial topology) settings. For robustness, GSLB evaluates GSL models under three types of noise: supervision signal noise, structure noise, and feature noise. We also compare GSL algorithms with the models specifically designed to improve these types of robustness. For complexity, GSLB conducts a detailed evaluation of representative GSL algorithms in terms of time complexity and space complexity.
Through extensive experiments, we observe that: (1) GSL generally brings performance improvement for node-level tasks, especially on heterophilic graphs; (2) on graph-level tasks, current GSL models bring limited improvement and their performance varies greatly across different datasets; (3) most GSL algorithms (especially unsupervised GSL algorithms) show impressive robustness; (4) GSL models require significant time and memory overhead, making them challenging to deploy on large-scale graphs. In summary, make the following three contributions:

---

Inpest
Graph Learner
Refined computation graph
Post Process
Graph Encoder
G (A
g
(A*,)
DOO
Structure Modeling
ooo
COO
KNNSpanitt
Message Passing
Trask
CDO
DOG
Disentioo
Lnsg
Figure 1: general framework of Graph Structure Learning (GSL). GSL methods start with input features and an optional initial graph structure. Its corresponding computation graph is refined/inferred through a structure learning module. With the learned computation graph, Graph Neural Networks (GNNs) are used to generate graph representations.
We propose GSLB, the first comprehensive benchmark for graph structure learning. We integrate 16 state-of-the-arr GSL algorithms and 20 diverse graph datasets covering homogeneous node-level, heterogeneous node-level, and graph-level tasks. An overview of our benchmark is shown in Table 1.
To explore the ability and limitations GSL, we systematically evaluate existing algorithms from three dimensions: effectiveness, robustness, and complexity. Based on the results, we reveal the potential benefits and drawbacks GSL to assist future research efforts.
To facilitate future work and help researchers quickly use the latest models, develop an easy-to- use open-source library. Besides, users can evaluate their own models or datasets with less effort. Our code is available https: /tte//giitt github.....
2 Problem Definition
In this section, we will briefly review the advances and basic concepts of GSL. Given an undirected graph G (A, where A N is the adjacency matrix, dae if edge (u, v) exists and auv = 0 otherwise, and X E RN*F is the node features matrix, N is the number of nodes, F is the dimension of node features. Given an optional graph G, the goal of GSL is jointly optimize computation graph G = (A* X) and the parameters of graph encoder θ to obtain high-qualin, node representations Z E NxF' for downstream tasks, where A is the refined graph by graph learner.
In general, the objective of GSL can be summarized as the following formula:
LOSt rask (Z', Y) XL AA Z"
(1)
where the first term Lask refers task-specific objective with respect the learned representation Z* and ground-truth Y, the second term Laeg imposes constraints on the learned graph structure and representations, and is a hyper-paramete that controls the trade-off between the two terms. The general framework of GSL is shown in Figure 1.
3 GSLB: Graph Structure Learning Benchmark
In this section, we introduce the overview Graph Structure Learning Benchmark, with considera- tions of algorithms (Section 3.1), datasets (Section 3.2) and evaluations (Section 3.3).
3.1 Benchmark Algorithms
Table 1 shows the overall 16 algorithms integrated in GSLB. They divided into three categories: homogeneous GSL, heterogeneous GSL, and graph-level GSL. We briefly introduce each category in the following, and more details provided in Appendix A.2.
Homogeneous GSL. Most of the existing GSL. algorithms are designed for homogeneous graphs. They assume there is only one type of nodes and edges in the graph. We select 7 TR-oriented

---

algorithms including GRCN [48], ProGNN [18], IDGL [4], GEN [39]. CoGSL [26], STABLE [21], and GSR [54]. For Tl-oriented algorithms, we select SUBLIME [28]. NodeFormer [43], SLAPS [10). and HES-GSI. [42]. It is worth noting that TR-oriented algorithms can only be applied if the original graph structure is available, but we can construct a preliminary graph based on node features (e.g.. KNN graphs or e-graphs).
Heterogeneous GSL. We integrate two representative heterogeneous GSL algorithms: Graph Trans- former Networks (GTN) [49] and Heterogeneous Graph Structure Learning (HGSL) [53], which can handle the heterogeneity and capture complex interactions heterogeneous graphs.
Graph-level GSL. Graph-level GSL algorithms aim to refine each graph structure in datasets. We select two graph-level algorithms: Hierarchical Graph Pooling with Structure Learning (HGP-SL)[S) [52] and Variational Information Bottleneck guided Graph Structure Learning (VIB-GSI.) [36].
3.2 Benchmark Datasets
To comprehensively and effectively evaluate the characteristics of GSL in the field of graph learning, we have integrated a large number of datasets from various domains for different types of tasks. For node-level tasks, evaluate the most mainstream task of GSL, node classification, we use four citation networks (i.e., Cora, Citeseer, Pubmed [47]), and ogbn-arxiv [15], three website networks from WebKB (i.e., Cornell, Texas, and Wisconsin [34]), and cooccurrence network Actor with homophily ratio ranging from strong homophily to strong heterophily. Subsequently, to validate the effectiveness of GSL in heterogeneous node classification, we utilized three heterogeneous graph datasets (i.e., DBLP [49]. ACM [49], and Yelp [29]). To investigate the robustness of GSL, we further incorporate the Polblogs dataset for evaluation. For graph-level tasks, we select six public graph classification benchmark dataset from TUDataset [31] for evaluation, including IMDB-B B [3]. IMDB-M [3], RDT-B [46]. COLLAB [46], MUTAG [5] and PROTEINS [2]. Each dataset is collection of graphs where each graph is associated with a level. Besides, exploring whether GSL can capture long-range information is an exciting topic. Therefore, we have utilized recently proposed
long-range graph datasets: Peptides-func and Peptides-struct [8]. See more details and statistics about
datasets Appendix A.I.
3.3 Benchmark Evaluations
To comprehensively investigate the pros and cons of GSL, our benchmark evaluations encompass three dimensions: effectiveness, robustness, and complexity. For effectiveness, GSLB provides fair and comprchensive comparison of existing algorithms from three perspectives: homogencous node classification, heterogencous node classification, and graph-level tasks. In the case of homogencous node classification, we evaluated them on both homophilic and heterophilic graph datasets, conducting experiments in both TR and TI scenarios. For graph-level, we evaluate graph-level GSL algorithms on TUDataset and long-range graph datasets for exploring the capabilities on graph-level tasks. For most datasets, we use accuracy as our evaluation metric. For robustness, GSLB evaluates three types of robustness: supervision signal robustness, structure robustness, and feature robustness. We control the count of labels to explore the supervision signal robustness of GSL and find that GSL exhibits excellent performance in the scenarios with few labels. We inject random structure noise and graph topology attacks to investigate the structure robustness. We also study the feature robustness by randomly masking a certain proportion of node features. For complexity, we conduct a detailed evaluation of representative GSL algorithms in terms of time complexity and space complexity. It will help to facilitate the deployment of GSL in real-world applications.
Experiments and Analysis
In this section, we systematically investigate the effectiveness, robustness, and complexity of GSL algorithms by answering the following specific questions:
For effectiveness, RQ1: How effective are the algorithms on node-level representation learning (Section 4.2)? RQ2: Can GSL mitigate homophily inductive bias of traditional message-passing based GNNs (Section 4.2)? RQ3: How does GSL perform on heterogeneous graph datasets
(Section 4.3)? RQ4: How effective are the algorithms on graph-level representation learning
(Section 4.4)? RQ5: Can GSL methods capture range information on the graph (Appendix B)?

---

For robustness, RQ6: How robust are GSL algorithms when faced with scarcity of labeled samples? RQ7: How robust are GSL algorithms in the face of structure attack or noise? RQ8:
How is the feature robustness of GSL? (Section 4.5)
For complexity, RQ9: How efficient are these algorithms in terms of time and space (Section 4.6)?
Otherwise, RQ10: What does the learned graph structure look like (Appendix B.2)?
4.1 Experimental Settings
All algorithms in GSLB are implemented by PyTorch [33], and unless specifically indicated, the encoders for all algorithms are Graph Convolutional Networks. All experiments are conducted on Linux server with GPU (NVIDIA GeForce 3090 and NVIDIA A100) and CPU (AMD EPYC 7763), using PyTorch 1.13.0, DGL 1.1.0 [38] and Python 3.9.16.
4.2 Performance on node-level representation learning
For node-level representation learning, we conduct experiments on homogeneous graph datasets under both TR and TI scenarios, and use classification accuracy as evaluation metric. Table 2 shows the experimental results of various GSL algorithms under the standard setting of transductive node classification task in the TR scenario. We observe that: 1) Most GSL algorithms generally show improvements in node classification task, particularly on datasets with high heterophily ratio. Due to the presence of heterophilic connections in heterophily graphs, where nodes are often connected to nodes with different labels, it violates the homophily assumption of message-passing neural networks. As result, traditional GNNs like GCN and GAT exhibit poor performance. However, GSL can improve significantly on heterophily graph datasets by learning new graph structures based on downstream tasks and specific learning objectives. thus enhancing the homophily of the graph and promoting the performance on node-level representation learning. 2) SUBLIME achieves optimal or near-optimal results on most datasets. learns graph structure through contrastive learning in an unsupervised manner. As mentioned in the recent literature [10], optimizing graph structures solely based on label information is insufficient. Leveraging large and abundant amount of unlabeled information can enhance the performance of GSL. 3) The scalability of GSL still needs improvement, as only few models can be trained on large-scale datasets (e.g.. ogbn-arxiv). We will discuss the scalability of GSL algorithms in detail in subsequent section (Section 4.6).
Table 3 shows the experimental results of the transductive node classification task in the TI scenario. Some GSL algorithms are designed for TR scenario (i.e., GRCN, IDGL, etc.). so we use kNN graphs as their original graph structure. As we can observe, on the homophily graph datasets, GSL outperforms basclines, such as MLP, GCN and GAT knn- However, on the heterophily graph datasets, most GSL algorithms often have difficulty achieving better results than baseline models. As mentioned in earlier literature, a network with randomness tends to get better performance utilizing kNN for direct information propagation [17]. Therefore, traditional with KNN graphs demonstrate powerful performance. In addition, as observed in the TR scenario, models that leverage self-supervision to extract abundant unlabeled information often achieve better performance.
pesssage-passing neural networks
4.3 Performance on heterogeneous graph node-level representation learning
In this section, we evaluate the performance GSL algorithms on heterogeneous node classification task and use Macro-Fl and Micro-F1 as our evaluation metrics. Table 4 shows the experimental results on heterogeneous graph datasets. By observing the results, we can find that: 1) Because GTN and HGSL consider both heterogeneity and structure learning, they generally outperform other models on heterogeneous graph datasets. 2) GSL algorithms generally outperform the vanilla GNN models (e.g. GCN and GAT) since they have learned better structures to facilitate message passing, 3) Due to the majority of GSL algorithms not explicitly accounting for heterogeneity, they may exhibit poorer performance on heterogeneous graph datasets. 4) Some datasets (e.g. Yelp) exhibit stronger heterogeneity, and on such datasets, models that consider heterogeneity (e.g. HAN, GTN, and HGSL.) perform significantly better.
5

---

Table Accuracy STD comparison (%) under the standard setting of transductive node classifi- cation task in the Topology Refinement (TR) scenario, which means the original graph structure is available for each method. Performance is averaged from 10 independent repetitions. The highest results are highlighted with bold while the second highest results are marked with underline "OOM" denotes out of memory.
Cora 0.81
Citeseer
Pubmed 0.80
ogbn-arxiv 0.65
Cornell 0.12
Texus 0.06
Wisconsin 0.18
Actor
Edge Hom.
0.74
0.22
GCN
81,46
71.36
79.18
70.77
47.84
57.83
57.45
20.01
GAT
70, a0
77.8 47.85
29,90
46.2226.3
54.05
57.65
28.91
GPRGNN
83.66=a77
71.64
75.99
50.80
76.76=530
85.14
83.33+34z
34.09=109
LDS
83.01
73.55=0.54
0OM
0OM
47.87=7.4
58,92+432
61.70+3.5
31.05+131
GRCN
83.87
62.43=061
78.92
OOM
54.32+824
62.16+7.05
56.08+7.19
29.97=6.1
ProGNN
80.30
68.51
0OM
00M
14.05
48.37
62.54
22.35
IDGL
83.88
72.20
80,00
0OM
50.00
62.43
4914
28. 18.1641141
GEN
80.21
11.15
78.9
OOM
57.02+7.1
65.94
66.07+372
2202
CoGSL
81.76=0044
73.09=0.42
OOM
OOM
52.16+321
59,46+443
58.82+1.2
32.95=129
SUBLIME
83.40=004
72.30=
80,90=094
71.75=0.6
70,54=598
77 +7.03+4223
78.82+635
33.57+008
STABLE
80.20=a0se
68.91+1101
OOM
0OM
44.03+4m5
55.24
53,00
30.18=100
NodeTvommrr
80.28
71.31
78.21
55.40
42.70a551
58,922432
48,434710
1175
GSR
82.48
71.10=025
78.09
0OM
44.32#216
60.81
56.86+124
30.23=0035
Table 3: Accuracy + STD comparison (%) under the standard setting of transductive node classifica- tion task in the Topology Inference (TT) scenario, which means the original graph structure is not available for each method.
Cora 0.81
Citeseer 0.74
Pubmed 0.80
ogbn-arxiv 0.65
Comell 0.12
Texus 0.06
Wisconsin 0.18
Actor
Edge Hum.
0.22
MLP
58.552a30
59.522064
73.00
55.21
71.3528.9
80.27
84.71
35.49-10
(CNN.
66.102244
68.33108
69.23110
75.14
75.95
14.12
32.98_0.4
GAT kan
64.62.1104
68.05
68. 68.761080
55.92am30
74.05
76.49.44
82, 42.16.400
31.67
GPRGNN
69.27-1808
70.292054
68. 19.11
51.39
75.68
81.08
12-133
34.71=1.5
LDS
99.87-19441
72.43
OOM
OOM
72.65
70,20
78. 14:4
32.39
GRCN Keom
69.48-:0.66
00.4
68.96
OOM
71.08+6544
74.32+500
78.63 64.9
30.83-0.7
ProGNN
+0.5
64.55
0OM
OOM
71.35++404
71.89
72. 77.94
31.56-1.14
IDGLenn
69.74+0.57
66.33+004
+4.0
OOM
72.70+4775
75.40
79.21 +3.94
33.07-137
GEN
66.95-:1.40
67.29
69.76
OOM
71.08
74.59
81.76=221
31,28-1006
CoGSLAnn
66.65-2937
68. 68.722084
OOM
OOM
70.27
72.70
76.96252s
34.52-1.55
GSRkan
66.28:20.9
66.77.nm
68.49:1.9
OOM
70.27-1162
74.86
78.62.191
33.73
SLAPS
72.28-2007
70.71211
74.50
55.19
74.591167
79.19
81.9611.6
37.161881
SUBLIME
72.7421.1
72.63
75.08
55.57
72.35
75.51
82, 82.14
32.20
NodeFormer
24.35
45.90
59.8336.59
55.37
42.70a551
58.922432
48 48.24
29.24-1148
HES-GSL
73.68
70.
77.08
56.46:2077
66.22.n19
24.05
79.61
36.73=077
4.4 Performance of GSL algorithms on graph-level tasks
In this section, we conduct graph classification experiments on four social datasets (i.e., IMDB-B, RDT-B, COLLAB, and IMDB-M) and two biological datasets (i.e., MUTAG and PROTEINS). Table 5 shows the experimental results of average accuracy and the standard deviation of 10-fold cross-validation. We can observe that HGP-SL (with GCN as the encoder) consistently outperforms GCN on all datasets. However, find that TIB-GLL exhibits strong instability across different random seeds. And due to the absence of training scripts in the official code?. we performed hyperparameter tuning based the parameter search space (3 E {10-1 10 10 10 10 3,
10 11) provided in the
paper, but we are unable to surpass the performance of the baseline models consistently. Lastly, we conducted an analysis of graph-level GSL algorithms long-range graph dataset [8). For detailed information, please refer to Appendix B.

---

Table 4: Macro-FI and Micro-F1 + STD comparison (%) under the standard setting of heterogeneous node classification task.
ACM
DBLP
Yelp
Method
Macro-Fi Fl
Micro-FI Fl
Macro-F1
Micro-F1
Macro-F1
Micro-FI
GCN
90.27
90. 18 0.61
90.01 1:332
90,99 1.228
78.01:11.8
81,03
GAT
91.52.+0.22
91,46=008
90.22 +0.77
91.13
12.14:11.7
84.5
HAN
91.67=0.30
91.47=0.22
20.53
91. 91.47=0.2
18.49
88. 14
LDS
92.35
92.05
88.1 38.11+000 4:06
88.74
75.98
14
GRCN
93.04.00117
92.94_00118
88.33
89.43::0:4
76.05
80.68
IDGL
91.6921.24
91.63-1.24
89. 60065
90.61 2:056
76,98
79.15:1500
ProGNN
90.57#1.05
90.50=1.29
83. 13.113
84,83+11.6
51.76++146
48.39
GEN
87.91
87.88 =2.1
89.74.10.s
90,65 1.771
80. 13378
82.68 22.68812.844
STABLE
83.54
83.38
75.18 1.99
76.42.11.9
71.4814.7
2..62
GEN
87.91
87.88=2.61
69
90.65
80.43
82.68
SUBLIME
92.42.10.16
92.13 0.337
90.98 4.0.22
91.82.+0.21
79.68 1
82,99
NodeFormer
91.33-20.77
90.60=0095
79
80.5620062
91.6942065
90.59
GSR
92.1421.08
92. 1 =0.99
76.59 45
77.69 =042
83785
85.73
GTN
92.04=0.38
91.94-0.39
90.52++o.5
91.48+60.9
92.98
92.44
HGSL
93.23
93.1310.51
91.58
92.49.:0.55
92.79
92.24
Table 5: Accuracy STD comparison (%) under the setting of graph-level classification task.
Method
IMDB-B
RDT-B
COLLAB
IMDB-M
MUTAG
PROTEINS
GCN
73.20
70. 70.10
76.96
49, 49.85
73.92
67.5226.1
VIB-GSL (GCN)
71.90
68.95
1..14
49.0543.52
68.6345.15
65.68
HGP-SL (GCN)
74.10
OOM
78.06=2.17
51.07
78.07
70.80
GAT
72.30=226
73.55
1.30
48. 2.999
78.7147.51
68.6326624
VIB-GSL (GAT)
72.10
OOM
17.54
49.06
77.13
67.09
SAGE
72.60=3.0
70.20+411
75.58
48.5542.8
68.6544.31
64.47=7.15
VIB-GSI. (SAGE)
73.00
65.75
7..74
48.7945.06
72.81
66.61 -4.88
(GG-SLLAAAGR
21.50
OOM
78.64
49.67:33.9
77.13+3329
73.32=2.0
GIN
73.00=2.67
71.70=5.01
79.86=164
50.30+3.52
87.19
69.07=562
VIB-GSL (GIN)
69.90
75.85
77.25
49.9743.65
85.18411031
75.15
HGP-SL (GIN)
23.50
OOM
80.14
48.67:22.8
73.92+6.24
69.37=3395
4.5 Robustness analysis of GSL algorithms
To investigate the robustness of GSL algorithms, we primarily focus on three aspects: structure robust- ness, feature robustness, and supervision signal robustness. Due limited space, we predominantly investigate the transductive node classification task in our paper. Nevertheless, researchers can utilize our GSLB library to efficiently and conveniently conduct experiments on other tasks as well.
Robustness analysis with re- spect to different supervision signals. We have discovered that GSL maintains surprising performance in scenarios with scarcity of labeled samples.
Cora
Citeseer
70-
Ancuracy (%6)
Accuracy
GCK
GCN
DAGNN
56
DATINN
SulFTritstsp
Sea-Thursa
We varied the number of labels per class in the Cora and Cite- seer datasets and selected two baseline models, DAGNN [25] and Self-Training [22], that per- formed well in scenarios with limited labels. As shown in Fig-
70
Indi.
GREN
GaEN
45
20
10
Labels class
Labels per class
Figure 2: Performance of baselines and GSL algorithms with respect to different numbers of labels per class on Cora and Citeseer.
7

---

Table Accuracy = STD comparison (%) with respect to different perturbation rates. Jaccard and SimPGCN are representative ofate-of-theaart defense GNNs.
Dataset
Pth Rate
GCN
Jaccard
SimPGIN
IDGI
GRCN
ProGNN
STABLE
SUBLIME
0%
83 68
83. 03
82.66
84.69
84,43=026
84,53
33.70
83.84
5%
80.61
81,4410
80.35
18.56
10.34
21.47.044
81.52.08s
79.93....5
Cora
10%
74.38=0..
75.90
76.50
+8.06
77.12
72.61 20.73
78.64
78.71
15%
65. 17=0.09
77. 14
73.77
76.88+0.44
73.74=061
65 199
79.70=1.71
79.34=00l
20%
13
69.0
67.19
6..54
76.44_2447
25.25
0%
76.56=0336
04
+4.35
73.87
76. =0.11
73.36=1.5
72. 72.65=1130
131
5%
72.5 10
70.01
72.99.1.0
72.461047
74.66
71 71.46.00.7
69.66
72.63.0090
Citeseer
10%
71 92=00m
70, 1..
72. +2.68
69.72=0.59
74.06=443
69.03=a0m
72. 71
73.02-20.2
15%
64, Ls
71.74
62.85.11.7
66.46.1117
65.42.11 mo
70
13.90
20%
57.51
67. 67.82
70.06
61.16
69.42.t:114
57.51
71.90
72.55
0%
95.62 m
94.9311%
94.501045
94.83
95.65
95.63.9 94
95.27.10.1
5%
80.57 -:066
78. 78.17
76102+
79.62
93.70:0.15
92.36=044
89.41
93,24-1.9:
Polblogs
10%
71.83 21
71.86+1344
70.12+1.10
74.54=00.9
87.99=1.56
84.66=0.5
89. =008
93.62-4.59
1.5%
66.38 17
69.931206
64.19
75.53
71.85
37.38g031
89.94.m
94.29..4.7
20%
68.19:224
69.22
63.64
71.63:082
71.73:21.5
73.57
87.42:0m
92.60 9..7
GCN
GRCN
LDS
Pro-GNN
STABLE
IDGL
GEN
Cora
Cileseer
Cora
Cileseer
Accuracy (%)
Accurac (%6
Accuraccy (96)
Accuracy cy (96)
0.2
24
02
Edge Deletion Rate
Edge
Deletion Rate
Edge Addition Rate
Edge
Addition Rate
Figure 3: Analysis of robustness when injecting random noise on Cora and Citeseer.
ure 2. we can observe that GSL algorithms (for the sake of brevity, we opted select only three models: GRCN, IDGL, and SUBLIME) achieve the best results in scenarios with fewer labels available. We speculate that this may be because the learned graph structure is denser and exhibits cleaner community boundaries. As result, the supervision signals can propagate more effectively within such a structure.
Robustness analysis with respect to random noise. We randomly remove edges from or add edges to the original graph structures of Cora and Citeseer, then evaluated the performance of GSL algorithms on the corrupted graphs. We change the ratios of modified edges from 0 to 0.9 to simulate different attack intensities. As shown in Figure 3, as the noise intensity increases, the models' performance generally exhibits a downward trend. And we can observe that GSL algorithms commonly demonstrate certain degree of robustness, as they tend to exhibit more stable performance than GCN when random noise is injected. Besides, we also found that, due to variations in the graph modeling process, different algorithms display varying levels of robustness when facing edge deletion and edge addition scenarios. For example, GRCN demonstrates strong robustness in edge deletion scenarios. However, in the edge addition scenarios, it only exhibits slight performance improvements compared to GCN. On the contrary, STABLE exhibits strong robustness in the edge deletion scenario, while showing the opposite trend in edge addition.
Robust analysis with respect to graph topology attack. Following [21, 55], we conduct robust analysis on three graph datasets, i.c., Cora, Citeseer, and Polblogs. First, we select the largest connected component in the graph, and utilize Mettack [58], non-targeted adversarial topology attack method, to generate perturbed graphs. We select the perturbation rate from 0% 20%. Table shows the performance of GSL algorithms on three datasets with respect to various perturbation rates. Surprisingly, we can observe that most GSL algorithms exhibit strong robustness against graph topology attacks, even better than state-of-the-an defense GNNs (e.g.. Jaccard [41] and SimPGCN [19]). GSL can effectively remove the newly added adversarial edges, and recover important edges to promote message passing. As mentioned in Li et al. [21], optimizing graph structures based on either features or supervised signals might not be reliable. We found that self-supervised graph structure modeling methods (c.g., STABLE and SUBLIME) show excellent
8

---

SLAPS
TIl on Core
TR on Citeseer
on Cora
TI Citeseer
Accamay (1)
Accurso (%)
Ancuracy (%)
Mask Feature Rate
Mask Feature Rate
Mask Feature Rate
Mask Feature Rate
Figure 4: Analysis of robustness when injecting random feature noise on Cora and Citeseer.
GCN
LDS
GRON
IDGL
ProGNN
CoGSL
SUBLIME
NodeFormer
Pubmed
64-
Accuracy (%0
Accuracy (%)
Accuracy
07-
80
70
18
167
127
10'
10
Training Time (s)
Training Time (s)
Training Time
Cora
Citeseer
Puhmed
Accuracy (56)
Accuracy (%)
Acmurcc (%0
01
81
20
Training Space (MB)
Training Space (MB)
Training Space (MB)
Figure 5: Training time and space analysis Cora, Citeseer and Pubmed.
performance corrupted graph structure datasets, which means unsupervised representation learning might produce more reliable and high-quality representations to conduct structure modeling.
Robust analysis with respect to feature noise. On the basis of exploring structural robustness, we also study the feature robustness of GSL. We randomly mask a certain proportion of node features by filling them with zeros, to investigate the performance of GSL algorithms when node features are subjected to varying degrees of damage. As shown in Figure 4, we can observe that: 1) the node features play more critical role than the structure on certain datasets. Under the same noise degree, feature noise brings more performance degradation compared with structure noise; 2) Interestingly, while most existing GSL methods rely feature similarity between pairs of nodes learn graph structure. they still exhibit good robustness when facing noisy node features; 3) edge-oriented algorithmss (e.g., ProGNN) show stronger feature robustness, because they optimize adjacency matrix directly, and have less dependence on pairs of node features.
4.6 Efficiency and scalability analysis
In this section, we analyze the efficiency and scalability GSL algorithms on Cora, Citeseer, and Pubmed datasets. For time efficiency, we evaluate the efficiency of the algorithms by measuring the time takes for them converge, i.c.. achieve the best performance on the validation set. For scalability, we set all models to their dense version to ensure a fair comparison. As shown in Figure GSL algorithms generally have higher time and space complexity compared to GCN. This limitation restricts the application of GSL on large-scale graphs. We can observe that some algorithms (e.g., GRCN) can achieve relatively good performance improvement with less complexity increase. Besides, although some algorithms (e.g.. IDGL, LDS, and SUBLIME) achieve remarkable effectiveness improvement, they largely increase the complexity of time and space.
Conclusion and Future Directions
In this paper, we give a brief introduction and overview of graph structure learning. Then we present the first Graph Structure Learning Benchmark (GSLB) consisting of 16 algorithms and 20 datasets

---

for various tasks. Based on GSLB, we conducted extensive experiments to reveal and analyze the performance of GSL algorithms in different scenarios and tasks. Through our comparative study, we find that GSL achieves promising results in heterophily, robustness, etc. The goal of this work is to understand the current state of development of GSL and provide insights for future research.
Notwithstanding the promising results that have been made, there are still some critical challenges and research directions worthy of future investigation.
Insufficient scalability. Most existing works model the existence probability edges based on node pairs, with complexity of O(N²). This makes it challenging to employ GSL in large-scale graphs in real-world applications. Future work should focus on overcoming the limitations GSL in terms of complexity.
Surprising performance with few labels. We have observed that GSL learns denser and more distinct graph structures, which facilitates the propagation of supervision signals. Most existing GNNs that address few label problem are based deep GNNs [25, 13] semi-supervised approaches [6. 35, 20]. without refining the graph structure. In the future, it would be worth exploring the combination of increasing the supervision signals and making the graph structure more suitable for propagating those signals.
Excellent performance of unsupervised GSL in robustness. Some algorithms using self- supervised methods for learning graph structures exhibit excellent performance in robustness, which may be attributed to the avoidance of unreliable supervision signals. In the future, further exploration can be done to utilize unsupervised structure learning for designing defense models.
Hard to apply incomplete graphs. Most existing algorithms rely pairwise node embeddings to generate the probability of edge existence. The underlying assumption is that all attributes of nodes on the graph complete. However, it is common in practice that some nodes all nodes have no features. Future research should address the challenges of structure learning on incomplete graphs.
Acknowledge
This work partially supported by the Research Grants Council of Hong Kong, No. 14202919 and No. 14205520.
References
[1] Tian Bian, Xiao, Tingyang Xu, Peilin Zhao, Wenbing Huang, Yu Rong, and Junzhou Huang. Rumor detection on social media with bi-directional graph convolutional networks. In Proceedings of the AAAI conference artificial intelligence, volume 34, pages 549-556, 2020.
[2] Karsten M Borgwardt, Cheng Soon Ong, Stefan Schönauer, SVN Vishwanathan, Alex J Smola, and Hans-Peter Kriegel. Protein function prediction via graph kernels. Bioinformatics, 2)(supp(_)) 2005. 2, 4
[3] Chen Cai and Yusu Wang. A simple yet effective baseline for non-attributed graph classification. arXiv preprint arXiv:1811.00 033508, 2018. 2,
[4] Yu Chen, Lingfei Wu, and Mohammed Zaki. Iterative deep graph learning for graph neural networks: Better and robust node embeddings. Advances in neural information processing systems, 33:19314-19326 2020. 2, 4, 16
[5] Asim Kumar Debnath, Rosa Lopez de Compadre, Gargi Debnath, Alan J Shusterman, Corwin Hansch, Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. correlation with molecular orbital energies and hydrophobicity. Journal of medicinal chemistry, 34(2):
786-797, 1991. 4
[6] Kaize Ding, Jianling Wang, James Caverlee, and Huan Liu. Meta propagation networks for graph few-shot semi-supervised learning. Proceedings the AAAI Conference on Artificial Intelligence, volume 36, pages 6524-6531, 2022. 10
[7] Paul D Dobson and Andrew Doig. Distinguishing enzyme structures from non-enzymes without alignments. Journal of molecular biology. 330(4):771-883, 2003. 15
10

---

[8] Vijay Prakash Dwivedi, Ladislav Rampášek, Michael Galkin, Ali Parviz, Guy Wolf, Anh Tuan Luu, and Dominique Beaini. Long range graph benchmark. Advances in Neural Information Processing Systems, 11-22326-223400 2022. 6, 15, 17
[9] Wenqi Fan, Yao Ma, Qing Li, Yuan Hc, Eric Zhao, Jiliang Tang, and Dawei Yin. Graph neural networks for social recommendation. In The world wide web conference, pages 417-426, 2019. 1
[10] Bahare Fatemi, Layla El Asti, and Seyed Mehran Kazemi. Slaps: Supe-supn improves structure lcarning for graph neural networks. Advances Neural Information Processing Systems, 34:22667-22681, 2021. 2, 4, 5, 16
[11] Luca Franceschi, Mathias Niepert, Massimiliano Pontil, and Xiao Hc. Learning discrete structures for graph neural networks. In International conference on machine learning, pages 1972-1982. PMLR, 2019.
2, 15, 16
[12] Xiang Gao, Wei Hu, and Zongming Guo. Exploring structure-adaptive graph learning for robust semi- supervised classification. In 2020 ieee international conference on multimedia and expo (ieme), pages 1-6. IEEE, 2020. 16
[13] Johannes Gasteiger, Aleksandar Bojehevski, and Stephan Günnemann, Predict then propagate: Graph neural networks meet personalized pagerank. arXiv preprint arXiv:1810.05999,, 2018. 10
[14] Zhongkai Hao, Chengqiang Lu, Zhenya Huang, Hao Wang. Zheyuan Hu, Qi Liu, Enhong Chen, and Checkong Lee, Asgn: An active semi superviie graph neural network for molecular property prediction. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 731-752, 2020.
[15] Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems, 33:22118 18 2222118-2213,, 2020. 2, 4. 14
[16] Bo Jiang, Ziyan Zhang, Doudou Lin, Jin Tang, and Bin Luo. Semi-supervised learning with graph learning- convolutional networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11313-11320 11320, 2019. 16
[17] Di Jin, Zhizhi Yu, Cuiying Huo, Rui Wang, Xiao Wang, Dongxiao He, and Jiawei Han. Universal graph convolutional networks. Advances in Neural Information Processing Systems, 14:10654-16664 2021. 5
[18] Wei Jin, Yao Ma, Xiaorui Liu, Xianfeng Tang, Suhang Wang, and Jiliang Tang. Graph structure learning for robust graph neural networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 66-74, 2020. 2, 4, 15, 16
[19] Wei Jin, Tyler Derr, Yigi Wang, Yao Ma, Zitao Liu, and Jiliang Tang. Node similarity preserving graph convolutional networks. In Proceedings of the 14th ACM international conference on web search and data mining, pages 148-156, 2021.
[20] Junseok Lee, Yunhak Oh, Yeonjun In, Namkyeong Lee, Dongmin Hyun, and Chanyoung Park. Grafn: Semi- superiuper node classification on graph with few labels via non- parametric distribution assignment. In Proceedings the 45th International ACM SIGIR Conference Research and Development in Information Retrieval, pages 2243 224--224,,775 2022. 10
[21] Kuan Li, Yang Liu, Xiang Ao, Jianfeng Chi, Jinghua Feng, Hao Yang, and Qing He. Reliable representa- tions make a stronger defender: Unsupervised structure refinement for robust gnn. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 925-935, 2022. 2, 4, 8, 16
[22] Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for semi-supervised learning. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018. 7
[23] Zhixun Li, Dingshuo Chen, Qiang Liu, and Shu Wu. The devil is in the conflict: Disentangled information graph neural networks for fraud detection. arXiv preprint arXiv:2210.122 12384, 2022. 1
[24] Derek Lim, Felix Hohne, Xiuyu Li. Sijia Linda Huang. Vaishnavi Gupta, Omkar Bhalerao, and Ser Nam Lim. Large scale learning non non-homon hanhhompphious graphs: New benchmarks and strong simple methods. Advances in Neural Information Processing Systems, 34:20887-209499 2021. 17

---

[25] Meng Liu. Hongyang Gao, and Shuiwang Ji. Towards deeper graph neural networks, In Proceedings of the 26th ACM SIGKDD international conference knowledge discovery data mining, pages 338-348,
2020. 7. 10
[26] Nian Liu, Xiao Wang, Lingfei Wu, Yu Chen, Xiaojie Guo, and Chuan Shi. Compact graph structure learning via mutual information compression. In Proceedings of the ACM Web Conference 2022, pages 1601-1610, 2022. 2,4, 16
[27] Yang Liu, Xiang Ao, Zidi Qin, Jianfeng Chi, Jinghua Feng, Hao Yang, and Qing He. Pick and choose: gnn-based imbalanced learning approach for fraud detection. In Proceedings of the Web Conference 2021, pages 3168-3177, 2021.
[28] Yixin Liu, Zheng, Daokun Zhang, Hongxu Chen, Hao Peng, and Shirui Pan. Towards unsupervised deep graph structure learning. In Proceedings of the ACM Web Conference 2022, pages 1392-1403, 2022. 2, 4, 16
[29] Yuanfu Lu, Chuan Shi, Linmei Hu, and Zhiyuan Liu. Relation structure-aware heterogeneous information network embedding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages
4456-4463, 2019. 2, 4, 14
[30] Dongsheng Luo, Wei Cheng, Wenchao Yu, Bo Zong, Jingchao Ni, Haifeng Chen, and Xiang Zhang. Learning to drop: Robust graph neural network via topological denoising. In Proceedings of the 14th ACM international conference web search and data mining, pages 779-787, 2021. 2, 16
[31] Christopher Morris, Nils M Kriege, Franka Bause, Kristian Kersting, Petra Mutzel, and Marion Neumann. Tudataset: collection of benchmark datasets for learning with graphs. arXiv preprint arXiv:2007.088: 200.06663, 2020. 4
[32] Mark EJ Newman. Mixing patterns in networks. Physical review E, 67(2)026126, 2003. 18
[33] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019. 5
[34] Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. Geom-gen: Geometric graph convolutional networks. arXiv preprint arXiv:2002.05287, 2020. 2,4, 4, 14, 17
[35] Ke Sun, Zhouchen Lin, and Zhanxing Zhu. Multi-stage self-supervised learning for graph convolutional networks on graphs with few labeled nodes. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 5892-5899, 2020.
[36] Qingyun Sun, Jianxin Li, Hao Peng, Jia Wu, Xingcheng Fu, Cheng Ji, and S Yu Philip. Graph structure learning with variational information bottleneck. Proceedings of the AAAI Conference on Artificall Intelligence, volume 36, pages 4165-4174, 2022. 2, 4, 16, 17
[37] Jie Tang, Jimeng Sun, Chi Wang, and Yang. Social influence analysis in large-scal scale networks. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining. pages 807-816, 2009. 2, 14
[38] Minjie Yu Wang. Deep graph library: Towards efficient and scalable deep learning on graphs. In ICLR workshop on representation learning on graphs and manifolds, 2019. 5
[39] Ruijia Wang. Shuai Mou, Xiao Wang, Wanpeng Xiao, Qi Ju. Chuan Shi, and Xing Xie. Graph structure estimation neural networks. In Proceedings of the Web Conference 2021, pages 342-353, 2021. 2, 4, 16
[40] Oliver Wieder, Stefan Kohlbacher, Mélaine Kuenemann, Arthur Garon, Pierre Duerot, Thomas Seidel, and Thierry Langer. A compact review of molecular property prediction with graph neural networks. Drug Discovery Today: Technologies, 1-2, 2020.
[41] Huijun Wu, Chen Wang, Yuriy Tyshetskiy, Andrew Docherty, Kai Lu, and Liming Zhu. Adversarial examples on graph data: Deep insights into attack and defense. arXiv preprint 100.0000000 2019. 8
[42] Lirong Wu, Haitao I. Lin, Zihan Liu, Zicheng Liu, Yufei Huang, and Stan 7.i. Homophily-enhaneedd self-supervision for graph structure learning: Insights and directions. IEEE Transactions on Neural Networks and Learning Systems. 2023. 2, 4, 16
[43] Qitian Wu, Wentao Zhao, Zenan Li, David P Wipf, and Junchi Yan. Nodeformer: A scalable graph structure learning transformer for node classification. Advances in Neural Information Processing Systems. 35:27387-2788 27401, 2022. 2, 4. 16
12

---

[44] Hui Xu, Liyao Xiang. Jiahao Yu, Anqi Cao, and Xinbing Wang. Speedup robust graph structure learning with low-rank information. In Proceedingy of the 30th ACM International Conference on Information & Knowledge Management, pages 2241 -2200, 2021. 16
[45] Weizhi Xu, Junfei Wu, Qiang Liu, Shu Wu, and Liang Wang. Evidence-aware fake news detection with graph neural networks. In Proceedings of the ACM Web Conference 2022. pages 2501-2510, 2022.
[46] Pinar Yanardag and SVN Vishwanathan. Deep graph kernels. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pages 1365 1374, 2015. 2, 4
[47] Zhilin Yang, William Cohen, and Ruslan Salakhudinov. Revisiting semi-supervised learning with graph embeddings. In International conference on machine learning, pages 40-48. PMLR, 2016. 4, 14
[48] Donghan Yu, Ruohong Zhang, Zhengbao Jiang, Yuexin Wu, and Yiming Yang. Graph-revised convolutional network. Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2020, Ghent, Belgium, September 14-18, 2020, Proceedings, Part III, pages 378-393 Springer. 2021. 2, 4,4, 15, 16
[49] Seongjun Yun, Minbyul Jeong, Rachyun Kim, Jaewoo Kang, and Hyunwoo J Kim. Graph transformer networks. Advances in neural information processing systems, 32, 2019. 2, 4. 14, 16, 17
[50] Jinghao Zhang, Yanqiao Zhu, Qiang Liu, Shu Wu, Shuhui Wang, and Liang Wang. Mining latent structures for multimedia recommendation. In Proceedings of the 29th ACM International Conference on Multimedia, pages 3872-3880, 2021.2 2
[51] Yanfu Zhang, Hongchang Gao. Jian Pei, and Heng Huang. Robust self supervie structural graph neural network for social network prediction. In Proceedings of the ACM Web Conference 2022, pages 1352- 1361, 2022.
[52] Zhen Zhang. Jiajun Bu, Martin Ester, Jianfeng Zhang, Chengwei Yao, Zhi Yu, and Can Wang. Hierarchical graph pooling with structure learning. arXiv preprint wrXiv:1911.05954 2019. 2, 4, 16, 17
[53] Jianan Zhao, Xiao Wang, Chuan Shi, Binbin Hu, Guojie Song, and Yanfang Ye. Heterogeneous graph struc- ture learning for graph neural networks. Proceedings of the AAAI conference on artificial intelligence. volume 35, pages 4697-4705, 2021. 2, 4, 16, 17
[54] Jianan Zhao, Qianlong Wen, Mingxuan Ju, Chuxu Zhang, and Yanfang Ye. Sulf-supervised graph structure refinement for graph neural networks. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, pages 159-167, 2023. 2, 4, 16
[55] Cheng Zheng, Bo Zong, Wei Cheng, Dongjin Song, Jingchao Ni, Wenchao Yu, Haifeng Chen, and Wei Wang. Robust graph representation learning via neural sparsification. In International Conference on Machine Learning, pages 1 11458 1 11468. PMLR, 2020. 2, 8, 16
[56] Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. Beyond homophily in graph neural networks: Current limitations and effective designs. Advances in Neural Information Processing Systems, 33:7793-7804, 2020. 17
[57] Yanqiao Zhu Weizhi Xu, Jinghao Zhang, Yuanqi Du Jieyu Zhang, Qiang Liu, Carl Yang, and Shu Wu. A survey on graph structure learning: Progress and opportunities. arXiv e-prints, pages arXiv-2103, 2021.
[58] Daniel Zügner and Stephan Günnemann. Adversarial attacks graph neural networks via meta learning. 2019. 8
13


Reviews:
[{"review_id": "FiHyVXVUUb", "reviewer": "Reviewer_wra6", "summary": "", "questions": "", "limitations": "- The paper offers limited analysis on how to differentiate between the various GSL methodologies.\n- As pointed out by the authors, GSL approaches remain difficult to scale to larger datasets that are used for benchmarking today’s GNNs. \n", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "- Comprehensive benchmarking study, covering all major aspects: performance, time/space requirement and structure refinement/inference capability.\n- It’s a great idea to study the structure refinement capability using a perturbation-based approach.\n- The paper uses a mostly divers set of datasets.\n- The paper makes an effort to quantitatively and intuitively analyze the properties of the learned graph structures comparing, for example, densities and partial adjacency matrices.\n", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "01GDnIVREQ", "reviewer": "Reviewer_QMnH", "summary": "", "questions": "", "limitations": "see above.", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "The comparisons between different methods are clear and easy to understand.\n\nThe authors conduct experiments on various settings including standard node classification, heterogeneous node classification, and graph classification.\n\nThe authors conduct training time and space analyses for different methods.\n", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "UI5KDcSMz6", "reviewer": "Reviewer_KDj1", "summary": "", "questions": "", "limitations": "Yes.", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "1. This paper proposes a standard setting of evaluation metrics for GSL.\n2. This paper conducts extensive experiments, and offers insights and opportunities for further research. Additionally, it provides important evaluation indicators for subsequent research.\n3. This paper evaluates models from multiple dimensions, such as effectiveness, robustness and complexity.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "mJpFcplJZC", "reviewer": "Reviewer_h3AR", "summary": "", "questions": "", "limitations": "N/A", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "Besides the contributions listed above, this paper is:\n1. well-structured and coherent.\n2. The code stored in GitHub is well-documented.\n3. Authors have a good coding practice.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "SadU8WSeSR", "reviewer": "Reviewer_ee9D", "summary": "", "questions": "", "limitations": "NA", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "1. GSLB examines existing GSL methods with various tasks, including homogeneous and heterogeneous node classification and graph classification. The inclusion of heterogeneous node classification is a plus. In addition, the authors include a large dataset, ogbn-arxiv to evaluate the baselines, which is also a plus. \n2. GSLB also examines the robustness of GSL methods in addition to standard node classification, which is also a plus. \n3. The paper is well-organized and in general easy to follow. \n4. The GSLB repository provides good documentation for readers to use, e.g. setting datasets, models, adversarial perturbations, etc.  ", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}]

Author rebuttal:

Meta-review: The paper presents a comparative study of representative graph structure learning methods, benchmarking their performance on various downstream tasks, including both node- and graph-level objectives. The comparison is conducted along three dimensions: effectiveness, robustness, and complexity. This work has received very positive feedback from all reviewers, acknowledging its significant contributions
Decision: Accept (Poster)