Structural Fairness-aware Active Learning for Graph Neural Networks
Authors: Haoyu Han
Xiaorui Liu
Li Ma
MohamadAli Torkamani
Hui Liu
Jiliang Tang
Makoto Yamada
Venue: ICLR 2024 poster (2024)
Keywords: Active Learning
Graph Neural Networks
Structural Fairness
Primary area: learning on graphs and other geometries & topologies

Abstract: Graph Neural Networks (GNNs) have seen significant achievements in semi-supervised node classification. Yet, their efficacy often hinges on access to high-quality labeled node samples, which may not always be available in real-world scenarios. While active learning is commonly employed across various domains to pinpoint and label high-quality samples based on data features, graph data present unique challenges due to their intrinsic structures that render nodes non-i.i.d. Furthermore, biases emerge from the positioning of labeled nodes; for instance, nodes closer to the labeled counterparts often yield better performance. To better leverage graph structure and mitigate structural bias in active learning, we present a unified optimization framework (SCARCE), which is also easily incorporated with node features.  Extensive experiments demonstrate that the proposed method not only improves the GNNs performance but also paves the way for more fair results.

Published as a conference paper at ICLR 2024
STRUCTURAL FAIRNESS- ACTIVE LEARNING FOR GRAPH NEURAL NETWORKS
Haoyu Han', Xiaorui Liu Li Ma', MohamadAlii Torkamani"; Liu',
Jiliang Tang', Makooo Yamada
MMchhiga State University North Carolina State University
SShanghai Jiaotong University
"Amazon OOknaaw Institute of Science and Technology
[hanhaoy I, liuhui7, tangjili @ msu.edu, xliu96 ncsu.edu, mali-cs @ @jtt.eduu.cn alitor@ aoorram.maooomm makoto. yamada @oist.jp
ABSTRACT
Graph Neural Networks (GNNs) have seen significant achievements in semi- supervised node classification. Yet, their efficacy often hinges on access to high- quality labeled node samples, which may not always be available in real-world scenarios. While active learning is commonly employed across various domains pinpoint and label high-quality samples based on data features, graph data present unique challenges due to their intrinsic structures that render nodes non-i.i.d Fur- thermore, biases emerge from the positioning of labeled nodes; for instance, nodes closer the labeled counterparts often yield better performance. To better leverage graph structure and mitigate structural bias in active learning, we present unified optimization framework (SCARCE) which is also easily incorporated with node features. Extensive experiments demonstrate that the proposed method not only improves the GNNs performance but also paves the way for more fair results.
INTRODUCTION
Graphs, representing relationships between entities, are foundational data structures that are wildly used across fields, from recommendation systems (Wu et al.] 2022) and fraud detection (Dou et al.) 2020) to biology (Molho et al.) 2022). Graph Neural Networks (GNNs) have demonstrated impressive capabilities (Gilmer 2017 Kipf & Welling. 2016 Veličković al. 2017] across various graph tasks, particularly in semi supervvid node classification. However, their performance largely depends on the presence of high-quality labeled node samples. Unfortunately, obtaining these samples can be both time-consuming and costly in real-world scenarios. Moreover, even with an equal count of labeled nodes, varied training sets can result in large performance disparities in GNNs (Fey &
Lenssen 2019; Madhawa & Murata 2020).
Active Learning (AL) (Ren et al., 2021: Settles 2011) tackles this issue by identifying the most representative samples for annotation, aiming maximize model performance within given labeling budget. Although AL has seen significant success with i.i.d data like images and text, adapting it to GNNs introduces unique challenges. First, the graph structure brings the interdependencies among nodes, making individual sample selection more intricate. Second, typical AL methods often involve iteratively selecting individual batch of samples and then retraining the model. This iterative approach becomes impractical for GNNs, especially for larger graphs. The reason is that training GNNs doesn't i just involve the training nodes alone; and it also requires consideration of its multi-hop neighbors, adding to the computational demands.
Several AL methods tailored for GNNs have emerged. For instance, AGE Cai al. 2017) addresses the first challenge by incorporating uncertainty and representativeness strategies, commonly found in traditional AL, with graph centrality (Muller al., 1995) for node selection. This approach necessitates multiple model re-training phases, presenting potential scalability concerns. In contrast, FeatProp (Wu et al., 2019) embarks on a distinct path. first propagates original features through the graph and then employs the K- <-Means clustering algorithm on the propagated node features. Nodes closest to cluster centers are then selected. As one-step AL method, FeatProp only needs one node
"This work does not relate to the author's position Amazon

---

Published as a conference paper at ICLR 2024
selection round prior to GNN training, making it notably efficient. GraphPart (Ma et al. 2022) further enhances FeatProp by incorporating it with graph partitioning, subsequently applying FeatProp to each partition. While these methods can leverage the graph structure and avoid retraining GNNs, they heavily rely on the quality of initial node features. However, these features may be unavailable, noisy, or uninformative. As demonstrated in Section 2 FeatProp even underperforms random selection on some datasets.
Graph structure, on the other hand, inherently captures the relationships between individual nodes, setting GNNs apart from many other models. This naturally prompts the question: Can we exploit the graph structure to select representative nodes for GNNs? Current research offers promising insights. For instance, Cai et al. (2017) underscores the potential of graph centrality metrics like Degree and 1998) in aiding node selection, addition, one recent research (Han et al. 2023) uncovers label position bias in GNNs. The label position bias indicates that unlabeled nodes in close proximity to labeled ones tend to exhibit superior performance using GNNs, which would cause the fairness issue for the nodes far away from labeled nodes. It differs from the traditional bias caused by sensitive features, such as gender and race. The label position bias is unique for graphs and related the labeling of graphs. Therefore, strategically selecting nodes for annotation based on their position in graphs can mitigate this fairnsss issue. In Section 2.2] we find that the fair labeling is also related to the model performance. As result, can leverage structural-basee based active learning to boost both performance and fairness.
PageRank (Page et al.
In this paper, we first study the relation between different criteria with the performance of GNNs. To combine these criteria and amplify the GNNs' efficacy, we introduce unified optimization framework tailored for active learning within GNNs. By leveraging the unified framework, we design Structural faimess-aware active learning method (SCARCE) that can concurrently address performance and fairness issues. Additionally, the inherent flexibility of our optimization framework facilitates the integration of node features, which in some contexts, can further enhance the performance. Extensive experiments show that our proposed method not only achieves comparable or even superior performance but also significantly enhances fairness within GNNs.
PRELIMINARY
In this section, we begin by examining whether the original node features are always beneficial for AL. Surprisingly, we find they can even degrade the quality of node selection in certain datasets. This observation leads us consider graph strueture-based criteria for AL. Subsequently,
explore the label position bias, which not only affects fairness but also can reflect the overall performance of GNNs. Moreover, we introduce novel metric grounded in the graph structure, termed "Structure Inertia Score", which gauges the representativeness of node selections. Before diving further, we first establish the notations used in this paper and define active learning under the context of GNNs.
Notations. We use bold upper- letters such as X to denote matrices. X, denotes its i-th row and X:j indicates the i-th row and j-th column element. We use bold lower-case letters such as x to denote column vectors. Let G (V,E) be a graph, where V is the node set with nodes and & the edge set. The graph can represented by an adjacency matrix A E RRax, where Aij 0 indicates that there exists edge between nodes i and uj in G. or otherwise AG, 0. Let D diag( (d1, dg,.... ... dz,)) be the degree matrix, where d, = Σ; Aij is the degree of node v,. We define the normalized adjacency matrix as Å D 1 AD 1, Furthermore, suppose that each node is associated with a d-dimensional feature x and we use X =
E Rexd to denote the feature matrix. We use a binary vector E B as the labeling indicator vector, where t; = node i is a labeled node, or otherwise t; = 0.
Active Learning for GNNs. Consider graph G = (A, accompanied by a small labeled node set, denoted Sn- Given an annotation budget of b, the objective of the active learning is to strategically select node subsets S; across multiple iterations such that |so U s1 U 52 U- U... U srl = b, where T represents the total number of iterations. During each iteration t, a GNN model g: usually trained using the accumulated labeled set s = Sp Us1 U2 J... U Sy to select the 5,+1- The ultimate goal of active learning for GNNs is to minimize the expected classification error of Ộr on the unlabeled nodes. In this work, we focus on the one-step active learning setup for GNNs (Contardo et al., 2017; Wu al. 2019 Ma et al. 2022), where |so] = and T 1. As result, our method is model-agnostic.
2

---

Published as a conference paper at ICLR 2024
2.1 EXAMINATION ON ORIGINAL NODE FEATURES
Both FeatProp (Wu et al. 2019) and GraphPart (Ma et al. 2022) are based on the propagated original node features, such as A X These methods then employ clustering techniques, selecting the nodes nearest to cluster centers as training nodes. Central to these approaches is an underlying belief that the propagated original feature can align with the representations learned by GNNs. Although notable results were observed on several citation datasets, this assumption might not hold on other types of datasets. To verify this, we compare the FeatProp with simple random selection strategy on two wildly used Amazon co-purchase graphs (Shchur et al., 2018), i.e. Computers and Photo. For comprehensive evaluation, we choose two GNN models: GCN (Kipf & Welling 2016) and APPNP (Klicpera et al.] (018)). which cover both coupled and decoupled GNN structure. The overall budget is selected from 20 to 200, and the result of the Computer dataset is shown in Figure T From the result, we can find that the performance of FeatProp is similar to or even worse than the random selection. A similar finding can also be found on the Photo dataset in Appendix A.I As a result, only leveraging the original feature may not be helpful or even degrade the performance of AL. Therefore, we mainly explore the graph structure in this work and then extend our framework with node features.
3000
Figure 1: Random vs. FeatProp on Computers.
Figure 2: GCN Performance vs.
Figure 3: GCN Performance vs. SIS on Cora.
LPS variance on Cora.
2.2 LABEL POSITION BIAS
Label Position Bias (Han al. 2023) arises when the performance of GNN model varies for nodes based on their proximity to labeled nodes in the graph, which would cause the performance fairness issue. A key metric used measure this proximity is the Label Proximal Score defined as follows:
LPS Pt. and P = (1-
where P E R=xx" is the Personalized PageRank matrix, e Ra*1 is given labeling indicator vector, t, = 1 if the node belong to the training set and otherwise and a E (0, 1] presents the teleport probability. As observed by Han et al. (2023), unlabeled nodes that possess higher LPS tend to achieve superior performance in GNNs. They found a low variance of the LPS score, which means all the nodes have similar "distance" improve the performance fairness of GNNs. Thus, we use the LPS variance to measure structural fairness. The study addressed this by modifying the graph structure P, in way that reduced the variance in LPS across nodes. This adjustment paved the way for more consistent and fair predictions across different groups of unlabeled nodes with diverse LPS ranges.
the labeled node, can mitigate the label position bias and
In active learning, strategically choosing labeling nodes, represented by t, can potentially reduce the LPS variance, promoting fairness in GNNs. But does this strategic node selection also elevate the overall performance of the GNNs? To validate this, we conducted experiments with the Cora and CiteSeer (Sen et al. 2008) datasets using both GCN and APPNP models. For each experiment, we randomly selected 20 nodes for training and ran the GNNs through three times for every selection. Figure the relationship between LPS variance and GCN accuracy based on 200 random training selections for the Cora dataset. The results of other methods and datasets are detailed in Appendix A.2 Based on these results, we have the following findings: (1) Node selection can profoundly impact GNN performance. For instance, in Figure we observe accuracy disparity reaching up to 35%.,; (2) Selections with better structural fairness(lower LPS variance) typically correspond superior GNN results. Therefore, beyond just fostering fairness, reduced LPS variance can serve as a crucial criterion for labeling node selection, subsequently enhancing GNN performance.
2.3 STRUCTURE INERTIA SCORE
While ensuring a smaller LPS variance can promote fairness and potentially elevate GNN performance, it's important also emphasize the selection of representative nodes within the graph structure.
3

---

Published as a conference paper at ICLR 2024
Traditionally, graph centrality metrics such Degree and PageRank have been leveraged in AL to identify these representative nodes. However, they select nodes independently. Consequently, if two adjacent nodes both have high Degree or PageRank scores, they are likely to be selected together,
potentially overlooking the broader scope of the graph structure. In recently years, the cluster-based methods (Zhan al.] 2022 [enner & Savarese 2017, Wu al.) 2019 Ma at. 2022), have achieved significant success in AL. The foundation of their efficacy is rooted in the selection of samples that reside close to the cluster centers. These centrally located samples can subsequently serve as representative proxies for other samples within their respective clusters. Inspired by this premise, our objective is to select the most representative nodes within graph structure for GNNs. Inertia, a metric that calculates the sum of distances between individual data points and their correspondigg centroids, frequently employed to measure how well the data is clustered. In graph-based AL., we consider labeled nodes as cluster centers and introduce the Structure Inertia Score (SIS), formulated as:
max P. Pitts SIS Σ
(2)
IEV
where P represents the Personalized PageRank matrix. Each entry, P.y O, captures the structural similarity between node and node j. and indicates the node selections. Thus, the Structure Inertia Score encapsulates the cumulative similarity each node to its proximate labeled node, providing a metric to estimate the representativeness of labeled nodes.
Following the same setting with section 2.2, we explore the relationship between the SIS and the GNNs performance. Figure 3 iillustrates the correlation between SIS and GCN accuracy on the Cora
dataset, and results for alternative methods and datasets are in Appendix A..3 From these results, the labeling selection with a higher SIS tends to have higher performance. Therefore, the SIS can also serve as a structural criterion for labeling node selection.
3 THE PROPOSED METHOD
From the literature and the preliminary studies in Section 2 there are different criteria for selecting labeling nodes. These criteria might not always be overlapped, and using them together could potentially lead to enhanced performance. However, there are several challenges: (1) How to combine these criteria to boost the final performance of GNNs? (2) There is usually no validation set AL, and how can we balance each criterion? To address these issues, we propose unified framework where active learning for GNNs conceptualized as an optimization problem. This design inherently supports the flexible incorporation of diverse criteria. Additionally, we also introduce a gradient balance method to harmonize the different terms. One significant advantage of our framework is that it obviates the need for hyperparameter tuning. This is especially beneficial for active learning: hyperparameter tuning might inadvertently lead to the annotation of suboptimal nodes, thereby leading increased and potentially unnecessary annotation burden. Leveraging our unified optimization framework, we introduce a structural fairness-aware active learning method, namely SCARCE, tailored for GNNs. Additionally, due the fiexibility of the unified framework, it is easy to incorporate node features.
3.1 A UNIFIED OPTIMIZATION FRAMEWORK
To conduct the active learning for GNNs, we seek a binary vector t E Raxx to indicate whether a node is selected for annotation or not. Thus, the active learning for GNNs can be represented as:
minimize f(g.t)
s.t. t1 <b E 10.
where the f= L1+ + L, represents the loss function of all different criteria, and 1,, E R is all one vector. Following/X Xu et al. (2019). we can case this combinatorial optimization problem by relaxing the binary vector t to its convex hull S = [t]t E [0, ",,,,],,,,111 t < <b). which can be solved by the projected gradient descent (PGD) method (Boyd & Vandenberghe 2004):
ITs (k Th V (g. t((t t(t-")]
where & is the current iteration step, 7i represents the learning rate at the step k. and V.f (9, (k 1)) denotes the gradient of the loss function f at (k--)) Besides, [s = arg migmessss ||s t|| is projection operator that projects t into the constraint set S.
4

---

Published as a conference paper at ICLR 2024
Gradient Balance. One essential issue of this optimization process is how to balance different criteria. Due to our focus on the one-step active learning for GNNs, there are no available labeled nodes as the validation set. This absence complicates the fine-tuning of hyperparameters to balance various terms L1, L2, ... Li effectively. Furthermore, each round of hyperparameter tuning may necessitate the annotation of non-ideal nodes. If gradient scale of each criterion might exhibit different scales. As as result, the optimization trajectory may be overwhelmingly influenced by the term with the larger gradient scale, potentially leading to imbalanced learning. Inspired by the multi-task learning (Liu et al. 2021; Guo & Weil 2022). where adaptive gradient mechanisms are employed balance the contribution of multiple tasks, we introduce simple but efficient gradient balance method to balance different terms. Specifically, The process is ViC. twofold: Firstly, we normalize the gradient scale of each term, resulting in u, = V to ensure
the same weight for different criteria, the
unit scale. Subsequently, we uniformly scale every gradient using the factor s min; | As a result, the gradients of different terms should be in the same order of magnitude.
After obtaining the optimal continuous vector t:' the i-th element t represents the probability that the node is selected as the labeling node. Finally, we can generate the binary labeling indicator vector t by sampling from t*
3.2
STRUCTURAL FAIRNESS- AWARE AL FOR GNNS
The studies in Section 2 provide some insights into active learning for GNNs. First, the original node features may not align well with the representations learned by GNNs in some datasets, even after several iterations of feature propagation over graphs. This discrepancy underscores the importance of leveraging the graph structure to select the most representative nodes for GNNs. Second, the label proximity score not only reflects the disparity of GNNs performance in different groups of nodes, where nodes with higher LPS tend to perform better, but its variance can also indicate the performance of GNNs. Specifically, node selections with low LPS variance tend to yield superior GNN performance. Third, drawing inspiration from clustering algorithms, the Structure Inertia Score can serve as an indicator of the representational potency of labeled nodes within the graph structure. Node selections that boast higher SIS generally pave the way for more effective GNN training. Consequently, there's 'ss potential to enhance GNN performance by identifying nodes that exhibit both a low variance in LPS and high SIS.
The unified optimization framework enables us to incorporate both LPS variance and SIS to improve the fairness and performance of GNNs. Therefore, the corresponding objective function can be represented by:
arg min f(C,t) |P cn,l
Σ NEV
max Psyt,
(5)
where (Pt) In represents the mean of label position score Pt. The gradient of the first term respect to can be represented by:
V.G.
= v
Pt
(Pt) 1 =2P
1n
22 Pt
(
(Pt) T 1n 1n
(6)
n
n
and the gradient of the second term with respect to t2 is:
Vt --
Σ max P.jt = Σ LEV
10,00
Pas,
if = l for that i,
(7)
otherwise.
LEV
Finally, we can update the using the PGD in Eq. 4 with the gradient balance method. Building upon the fact that this method solely relies on the graph structure, we name as SCARCE or SCARE CEARCF-Struuuun
The full algorithm is detailed in Algorithm [ First, we uniformly initialize t in line Next, we perform projected gradient descent with gradient balance from lines 4-9, We then sample potential labeling selections using multinomial sampling method. Finally, in line 14, select the binary vector t that yields the smallest overall loss.
3.3 SCARCE WITH NODE FEATURES
Even though the original node features might not always be informative, studies by Wu al. (2019); Ma et al. (022) have demonstrated their potential utility in certain datasets. The flexibility of our
5

---

Published as a conference paper at ICLR 2024
Algorithm 1 Algorithm of SCARCE-Structue
1: Input: The PPR matrix P, budget b, learning rate 7), the iteration numbers K, and the number of
random trials I
2: Output: A binary labeling indicator vector t with |t| b
3: Initialization: {())) 1b/n, b/n,..., b/n]"
4: for = 1,2...., do
5: calculate V (( C1 and V 1ik- C2 based Eq. 6 and Eq.
6: normalize the gradient in 1) = IV Cilli and u1 (k-1)
Faa-1)
V lkk
(t-
7: s = min ((+ II, [(
8: projected gradient descent: t(k) = IIs ((--1) 178 (u (uj-1 + u - 1)=
9: end for
(())
10: normalize
|
11: for = =1 3,.,
do 12: draw binary vector ( by multinomial sampling: sfr) multinoimal((,, 6)
13: end for
14: choose binary vector from { which yields the smallest loss in Eq.
return
unified optimization framework facilitates the seamless integration of these features into SCARCE. Specifically, similar to the Structure Inertia Score, design a Feature Inertia Score (FIS):
cxp (- 1 X x3)
FIS Σ max Styty, S iEV
(8)
ii exp
1g x.) g (Xx))
where the S represents the feature similarity between nodes and j. and g(X) is a feature propagation function. We have the option to exclusively utilize FIS for labeling node selection, namely as SCARCE-Feature, or to integrate it with the structure, referred to as SCARCE-ALL.
3.4 COMPLEXITY ANALYSIS
The inputs in Algorithm include a Personalized PageRank matrix P E Raxn, which can potentially be dense matrix. To address this, adopt the approach of PPRGo (Bojchevski al. 2020) to
calculate sparse approximate PPR matrix P(c), where is the threshold, and each of p( only contains the top largest entries. Therefore, the time complexity to calculate the gradient in Eq. (6)
and Eq. (7) becomes (nk)), which is linear with respect to the number of nodes. Similarly, when incorporating node features, we employ a strategy that preserves only the top k largest entries in each row of S. Overall, the proposed method is applicable to large graphs.
EXPERIMENT
4.1 EXPERIMENTAL SETTINGS
In experiments, we evaluate the performance of various AL methods across multiple datasets and investigate their adaptability with different backbone GNNs for the semi-supervised node classification task. Detailed settings are elaborated upon below.
We perform experiments utilizing six widely used real-world graph datasets, encompassing three citation datasets, i.e., Cora, Citeseer, and Pubmed (Sen al., 2008). two co-purchase datasets from Amazon, i.e., Computers and Photo (Shchur et al. 2018). and one OGB dataset, i.e., ogbn-arxiv (Hu al. 2020). The summary details pertaining to these datasets are provided in Appendix B.1 We also adopt different GNN backbones, i.e., GCN (Kipf & Welling 2016) and APPNP (Klicpera et al. 2018). GAT (Velickovic et al.. 2017) and GCNII (Chen al. 2020
Datasets & Backbone GNNs.
Baselines. We compare our methods with the following baselines, which can be divided into three
categories: (1) General AL methods, including Random, Uncertainty (Settles & Craven 2008), Density (Cai et al., 2017). and CoreSet (Sener & Savarese 2018); (2) Structure only methods,
including Degree and PageRank (Cai et al. 2017); (3) AL methods for GNNs that combine both
structure and features of graph, including AGE (Cai et al. 2017), FeatProp (Wu et al. 2019) and
GraphPart (Ma et al. 2022).

---

Published as a conference paper at ICLR 2024
Active Learning Setup. For each dataset, set the annotation budget to ZC for all methods, where C' is the number of classes in the dataset, and Z is chose from {5, 10, 20). Following the AL setting in Ma et al. 2022). we set the initial seed set So to zero for the one-step AL methods, such as Random, Degree, PageRank, FeatProp, and GraphPart. For the other iterative AL. methods, we randomly choose one-third of the budget the initial seed set So, and let the method select the other two-thirds. Due to the lack of validation set in the AL setup, we train the GNN model with fixed 300 epochs and evaluate over the full graph. For each experiment, we train the backbone GNNs 10 times and report the average performance and standard deviation. For the baselines, we follow the setting in Ma et al. (2022). For our SCARCE, we choose the learning rate 7 from {0.001, 0.01, 0.1} based on the gradient scale of t. More details about the experimental settings and GNN architectures can be found in Appendix B.2
4.2 PERFORMANCE COMPARISON ON BENCHMARK DATASETS
In this subsection, we evaluate the efficacy of the three variations of our proposed SCARCE across all six datasets. Specifically, we examine
which relies solely on the graph structure, SCARCE-Feature, which exclusively employs the propagated features, and SCAR REEACEE-ALLE which integrates all criteria. The performance results for GCN and APPNP are presented in Table [ and Table 2 respectively. From the results, we can make the following observations:
The Active Learning methods designed for graphs, such as AGE, FeatProp, GraphPart, and our SCARCE, usually perform better than the General AL methods. This underscores the importance of the distinctive graph structures.
The proposed SLARREE-Srruttrr consistently outperforms the graph centrality- -ased methods, such as Degree and PageRank. Impressively, this structure-based method can have comparable or even better results compared with the state-of-the-art baselines, such as FeatProp and GraphPatt, which exploit both structure and feature information. Notably, on the Computer and Photo datasets, SCARCE-Structun significantly outperforms other baselines. For example, when 2=5, it achieves an 8.3% and 7.1% 1% relative performance boost over the best baselines on the Computers dataset, when employing the GCN and APPNP models, respectively.
The SCARCE- Feattue can achieve similar performance with FeatProp and GraphPart across all datasets and GNNs. This attests to the effectiveness of our proposed Feature Inertia Score.
By integrating both node features and graph structure, SCARCE-ALL typically enhances per- formance compared to solely leveraging feature or structure on the Cora and CiteSeer datasets. This underscores the potential of combining features and structure in graph-based active learning. However, on the Computers and Photo datasets, SCARCE-ALL performance lags behind the structure-only variant. Besides, the performance of SCARCE-Feature is also subpar, similar to other feature- based methods, such as FeatProp and GraphPart. This observation consistent with our earlier discussions in the preliminary study, which highlighted that the performance of these methods heavily relies on the quality of the original feature.
To further demonstrate the superiority of the proposed SCARCE, we conduct experiments under the noise feature setting and missing feature setting. The results are shown in Appendix B.3
4.3 FAIRNESS COMPARISON
In this subsection, we aim to assess the capability of the proposed SCARCE in alleviating the label position bias issuc, aiming to bolster the fairness of GNNs. We deploy experiments on both the CiteSeer and Computers datasets using GCN and APPNP models. Specifically, following Han et al. (2023), we first sort the nodes by their LPS in Eq. I Subsequently, these nodes are divided into ten distinct sensitivity groups, ensuring each group comprises an equal number of nodes. To evaluate the disparity in performance across these groups, we rely on two commonly used metrics for assessing group bias: the Standard Deviation (SD) and the Coefficient of Variation (CV). Here, CV is defined as the ratio of SD to μ. where μ represents the average accuracy across all groups. The results of CiteSeer and Computers datasets are shown in Figure 4 and Figure 5 respectively. We can find:
The proposed SCARCE consistently yields the lowest SD and CV values across most budget sizes on both the CiteSeer and Computers datasets, irrespective of whether GCN or APPNP is used. Consequently, nodes across diverse sensitivity groups benefit from a more balanced and equitable classification performance when processed by GNNs. This highlights the capability of SCARCE to enhance not just the performance, but also the faimess of GNNs.
7

---

Published as a conference paper at ICLR 2024
Table 1: Semi-supervised node classification accuracy with GCN on benchmark datasets. The bold marker denotes the best performance and the underlined marker denotes the recond-best performance.
Raselnngs Budget C+
Coea 2 10
Chiaser Z- 10
Puhmed 2- 10
2
Z- 20
55
Z 20
Z-3
- 20
Random
4:776
26-48
3.42
81.00 1.40
52.76
7.64
67.42 + 1:94
68.40 1.30
61 43 6.0
4..14+4.11
27:21
1.95
Unceraiity
5.75
70.97 + 3.44
78.12
200
42.02
9.64
54.08 4.43
63.59 2.20
13 5.14
60.71
8.36
72.47
4.05
Density
66.71
74.65
2.65
80.15
2 1:34
52.46
5.13
00.47 a
67.51 1.31
2 6.92
67.92
2.113
74-18
ConsSet
64.06:24.16
74.9
79.73 + 1:19
51.39 6. 3.65
61.73 2:95
68.03
56 6.04
70.27
4.889
24:03:2.00
Degree
64.75
25.99
78.49
2
11.95
47.68
51.7320.42
61,5820.9
6..19
22.70:11:35
Pagerank
6.112211.6
75.11::0:29
82.63
1..19
48.98 0.62
0.224
09.14
57.832021
00.22
2..3
AGEE
62.19:3..0
76.32
82.14
53.42 5.20
68.44 = 1:2:8
99.95
68.77 1.25
62.08
09.33
3.00
77.56 1.59
FeatPop
74.49.20.88
0.
83.96
x 11.20
47.96 0.50
70.62
66.49
71.47
s 0.25
79 0.09
GraphPart
7..59.00.43
81.79 a 0.24
84.20
0:33
63.71 0.50
65.65 0.20
0,
72.18
79,08 6 0.14
SCARCE Structure
76.92.0..4
81.55 0.56
83.30 0.4.1
64.85 1.30
70.26
02.58
22.3140755
75.65 à 0,76
76 08 1.13
SCARCT- Feetuure
7.73
81.09 0.35
84.04 0.17
65.01 1.23
68.06 e 0.53
71.82
0
78,40
6 0.69
80.34 0.29
SCARCE ALL
80.58 0.82
84.8
66.03
11,93:0.40
=
74.01 0.38
77.01 &
Baselines.
Computers 2 10
Photo
Arsiv Z
Budget C
2-5
-0
2-5
Z- 10
2
2-5
2 20
Random
4.894
79.92 2.01
86.18 1.19
76.82 a 10.55
65 60 3.48
88.57 a 3,52
52.85 a 0.91
57.31 a 0.98
60:41
1.02
Unceriiiny
66.79 5.83
15:30
13.14 8.46
52,27 5.13
8 9.133
1.91
53.50 &
58.44
1.22
Density
64.81
12.86 4:2.4
82.16
13,94 6.14
$4,04 = 4.09
89.12
1.65
47.12
2.06
51.92 à:
55.16 1.35
CoreSet
14.32:14.37
12:41
64.05
64.59
18.38
1.17
53.83
Degize
46.85
61.52
40.52
$2,80
18.05
1.78
41.76
46.33 0.92
Pagamin
73.42 2.07
76.26
8.226-68.23
81.07 2.76
1.14
52.83+0.00
57,40
72.50+44.55
75
66.15
69.75 2.85
1.70
FeatPrup
66.68.4:4411
15.30
76.95 11.11
74.65
67.24 0:662
68.41
56.54 54.522064
GraphPart
72.08 5.92
71.19 10:62
1.31
52.52 1.31
55.74
Structure
79.51
90.13 1.01
91.37 0.64
58.32 =
SCARCE-
1.16
2,49
5.36
SCARCE-ALL
26.52+1.61
21.82 1.02
50.35 11.55
35.65 1.332
03.13:=0.75
30.05 0.33
58,98 =
0.35
Table 2: Semi-supervised node classification accuracy with APPNP on benchmark datasets. The bold marker denotes the best performance and the underlined marker denotes the second-bor -estt performance.
Biselines
Corn 2 10
CiteScer 10
Pubhicd 2 10
Budget C+ Z
x- 20
22 5
2 20
2 20
Randou
5.17
7M. 16.16.4 3.75
83.05 * 1.18
54,99 = x
65.37 + 2.38
68.73 = 1.13
65 6.57
58
70.5% + 4.15
78.75 1.80
Uncertrinty
64.49 5.74
75.83
3.26
80
2.42
47.53 e 7.58
55.60 4:47
56-48 0 1.54
58
17
5.19
63:95
R.93
75.44 = 4.27
Density
2..6924.32
2.77
82:31 1:25
54.53 2
31.94
57.05 2 1.52
62.15
2 6.92
70.10 2 6.90
77.58 2.06
CoreSet
2..1222.68
76.37 2.93
R2:31
54,43
6.35
64.19.4..0
68.69 1.46
66.72
6.54
71.35 2 4.42
77.78 2.41
Degres
0..52
77.09
20.26:00.24
53.47
0.75
0.39
62.12 0:40
66.16 0.15
65.28 0.15
Pagerank
0..7
76,92
0.16
0.11
51.51 + 1.30
03.17=0.26
2..21
66.22
0.118
70.49
2: 1123
77.19 2 0.08
AGE
3.10
78.57
1.81
83.43 0.72
52.95
+ 5.45
62:59
4:209
66.83 1.62
59.83 + 7.94
69:40
5.05
79.05 1.46
FeasProp
2..3
$1.47 0.36
84.43 0.112
48.92 0.37
67.98 1
72.47
09.83
75.16
0.04
80.40 0.10
GraphPart
0.13
93.22
0.15
0..1
64.83
0.35
06.39 1 0.22
70.33 0.20
74.56
0.041
0.03 80.34
82.39 0.06
SCARCE Stucture
0.85
83.35
0.33
84.45 0.36
63.94 4
66.09 1
70.83 0.62
74.12
1.64
80.01
81, : 0.43
SCARCE
7 3.06
82.54
0.57
83.91 0.199
71.24
1.55 76.01
72.05
75.12
79.29 0.63
SCARCE-ALL
x2.77
83.17 0,30
84.99
67.15
68.43
7
78.55
80.60 0.37
Biselises
Canputees
Arsiv
Budget
5=5
=
10
Random
76.30
6:00
90.36
54.85
61.91 a 0.68.
Uncertiinty
71.92
79.00
89
54.42
58.75 0.40
Deasity
67.72
74.63 3.17
77.39 10.67
9.00
N7.36 2.67
90.25
48.84 1.68
54.07 1.72
56.22 à 1.22.
ConeSit
66.38
71.47-6.29
76.12 + 5.45
1.
89:08- 3.58
50:31
55.04 0.55
58.79
1.67.
Degroe
44 3.72
51.22
64
6.
57.75 2.55
77.67 13.99
43.51
0.00
0.88
Paperank
74.85 9:50
79.89
3.16
81.56 2 7:31
85,85
1.04
87.08 1
90:49 2 1.16
54.75 0:44
59.42 1: 0.30
62.33 0.30
AGE
7..67
76,91 = 10.89
84.85
2 0.06
$1.47
2 4.95
N7.33 + 1.73
90.92 2 3.03
52.36
2 1.86
50.46 2
59.45 1.06.
Peudrop
74.35
79.16 2.41
80005 1 6.22
53.75
2 1.31
N5.33
2 0.63
91.04 2 0.62
0.77
58.17 + 11.53
51. 0.62
G GraphPart
67.81 1.29
72.60 2.66
W1.76 4.32
84 + 3.23
85.62 + 1.03
R0:81 = 0.74
55.09 = 0.67
56.08 + 0.69
61.00 0.75
SCARCF-Simctuee
0.99
$4.44
0.95
R5J8 à 0.47
85.27 2.53 53
76 1.03
91. -0.90.11 1.01
57.10 0.29
58.05 + 0.34
62.88 = 0.12
SCARCE-Fomie:
65.37 3.30
69.20
1.79
77
60 + 2.20
50.73
= 0.85
87.58
+ 1.73
30 20.34-2 = 1.27
56.01 0.40
60
55 B.22
62 0.15
SCARCE-ALL
22.10
80.33 à 1.09
85.5 à 1.05
$5.05 à 2.65
89.77 + 1.55
91.53 à 1.32
56. 0.24
60.96 8 0.33
62.24
The degree- method usually exhibits high SD and CV values. This can be attributed to the fact that neighbors of these labeled high-degree nodes often possess a higher LPS. Consequently, these nodes tend to outperform others, leading to discrepancies in classification performance across the graph.
4.4 ABLATION STUDY
In this subsection, we investigate the impact of LPS variance and SIS within the SCARCE on GNN performance and faimess. Specifically, we introduce two variants that exclusively incorporate either the LPS variance or SIS into the unified framework. Subsequently, we assess their implications on both accuracy and fairesss using the GCN model. Results for the Cora and CiteSeer datasets across varying budgets are depicted in Fig 6 and Fig 7 respectively. From the results, several observations can be made: (1) Leveraging SIS independently tends to surpass the performance achieved by solely using LPS variance, particularly on the CiteSeer dataset. (2) However, LPS variance usually leads to lower SD, indicating its inherent capacity to promote increased fairness. (3) SCARCE, which combines both SIS and LPS variance, SCARCE can not only elevate overall performance but also attain commendable fairness.
8

---

Published as a conference paper at ICLR 2024
Random Uncortainty
Density CoreSet
Degree PageRank
AGE FeatPro
GraphFart SCARCE
50
20
15
1.
30
39
29
20
10-
10
Budget
Budget Size
Budget
Budget
GCN.
(b) CV GCN.
(c) SD APPNP.
CV APPNP.
Figure 4: The faimness results on the CiteSeer dataset.
50
25
20
40
0
15
30
10
10-
80
100
120
40
60
00
120
140
100
120 140
80
100
120
Budget Size
Budget Sizee
Budget Size
Budget Size
(a) SD on GCN.
(b) CV on GCN.
(c) SD on APPNP.
(d) CV on APPNP.
Figure 5: The fairness results the Computers datasct.
05
USvarera SONCE
60
reriace
15
70
SCHIOL
60
6-
25
60
60
00
100
25
40
80
100
25
40
60
65
150
20
40
65
60
1.30
B..dget S2
Sudgptt Site
Buugper
Buxipe San
Figure 6: Variants of SCARCE Cora.
Figure 7: Variants SCARCE on Citeseer.
RELATED WORKS
Graph Neural Networks (GNNs) (Kipf & Welling 2016 Velickovic et al., 2017) have demonstrated significant potential in graph representation learning across a wide range of tasks (Zhou et al. 2020). However, the success of GNNs often depends on the availability of substantial labeled data, which is typically costly and labor-intensive to obtain. Active Learning (AL) methods are designed to address this challenge (Zhan et al.) 2022 Ren et al. [222)) AL seeks to strategically select the most informative samples for labeling with the goal of enhancing model accuracy while minimizing the amount of labeled data. Various AL strategies have been proposed, ranging from uncertainty- based methods (Beluch et al., 2018), which pick samples that a model predicts with low confidence, to approaches (Gal et (017), which emphasize samples that best represent the unlabeled data. While AL has achieved great success with many deep-learning models, its integration with GNNs remains relatively nascent. Works such as Cai et al. (201)); Gao al.] (2018) have attempted to embed node centrality metrics into traditional AL paradigms, but these require iterative sample selection and model retraining, making the process rather inefficient for GNNs. On the other hand, efforts like Wu et al. (2: Ma et al. (2022) adopt propagated original features for swift node selection, yet their efficacy often hinges on the inherent quality of those features. On the contrary, the proposed SCARCE mainly focuses on the graph structure, which can improve both GNNs performance and fairness.
Cepresentative/Diveriiyys
6 CONCLUSION
In this work, we investigate active learning for GNNs, a domain that remains relatively uncharted. Our exploration begins with an evaluation of various criteria, revealing that while original node features may not be informative, criteria based on graph structures effective indicators of both GNN performance and faireess. We introduce a unified optimization framework for active learning on GNNs. Building on this, present SCARCE, which is both efficient and can easily integrate with node features. Our comprehensive experiments demonstrate that the proposed SCARCE enhances both the performance and fairness of GNNs.

---

Published as a conference paper at ICLR 2024
7 ACKNOWLEDGEMENT
This research is supported by the National Science Foundation (NSF) under grant numbers CNS 2246050, IISI 845081, I1S2212032, IIS2212144, IOS2107215, DUE 2234015, DRL 2025244 and IOS2035472, the Army Research Office (ARO) under grant number W91 INF-21-1-0118, the Home Depot, Cisco Systems Inc, Amazon Faculty Award, Johnson&Johnson, JP Morgan Faculty Award and SNAP. The work is also supported by MEXT KAKENHI Grant Number 20H04243.
REFERENCES
William H Beluch, Tim Genewein, Andreas Nürnberger, and Jan M Köhler. The power of ensembles for active learning image classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3368-9377, 2018.
Aleksandar Bojchevski, Johannes Klicpera, Bryan Perozzi, Amol Kapoor, Martin Blais, Benedek Rózemberczki, Michal Lukasik, and Stephan Günnemann. Scaling graph neural networks with approximate pagerank. In Proceedings of the 26th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, pp. 2464-2473, 2020.
Stephen P Boyd and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004.
Hongyun Cai, Vincent W Zheng, and Kevin Chen-Chuan Chang. Active learning for graph embedding. arXiv preprint arXiv: 705.05085, 2017.
Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. Simple and deep graph convolutional networks. In International conference machine learning, pp. 1725-1735. PMLR, 2020.
Gabriella Contardo, Ludovie Denoyer, and Thierry Artières. A meta-learning approach to one-step active learning. arXiv preprint arXiv: 7777.0600344 2017.
Yingtong Dou, Zhiwei Liu, Li Sun, Yutong Deng, Hao Peng, and Philip S Yu. Enhancing graph neural network- based fraud detectors against camouflaged fraudsters. In Proceedings of the 29th ACM international conference on information & knowledge management, pp. 315-324, 2020.
Matthias Fey and Jan Eric Lenssen. Fast graph representation learning with pytorch geometric. arXiv
preprint arXiv: 1et.:00230002 02428, 2019.
Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep bayesian active learning with image data. In International conference on machine learning, pp. 1 1183-1192. PMLR, 2017.
Li Gao, Hong Yang, Chuan Zhou, Jia Wu, Shirui Pan, and Yue Hu. Active discriminative network representation learning. In IJCAI International Joint Conference on Artificial Intelligene, 2018.
Justin Gilmer, Samuel Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International conference muchine learning, pp.
1263-1272. PMLRR, 2017.
Yiyou Guo and Chao Wei. Multi-task learning using gradient balance and clipping with an application in joint disparity estimation and semantic segmentation. Electronics, 11(8):1217, 2022.
Haoyu Han, Xiaorui Liu, Feng Shi, MohamadAli Torkamani, Charu C Aggarwal, and Jiliang Tang. Towards label position bias in graph neural networks. arXiv preprint arXiv:2305. 15822, 2023.
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems, 33:22118 1111118223333 2020.
Thomas N Kipf and Max Welling, Semi-supervised classification with graph convolutional networks.
arXiv preprint arXiv: (eeiei.o..000 000000000000...... 2016.
Johannes Klicpera, Aleksandar Bojchevski, and Stephan Günnemann. Predict then propagate: Graph
neural networks meet personalized pagerank. arXiv preprint 1rxiv:1810.0599) 2018.
10

---

Published as a conference paper at ICLR 2024
Liyang Liu, Yi Li, Zhanghui Kuang, J Xue, Yimin Chen, Wenming Yang, Qingmin Liao, and Wayne Zhang. Towards impartial multi-task learning. iclr, 2021.
Jiaqi Ma, Ziqiao Ma, Joyce Chai, and Qiaozhu Mei. Partition-based active learning for graph neural networks. arXiv preprint arXiv:2201.09391, 2022.
Kaushalya Madhawa and Tsuyoshi Murata. Active learning for node classification: An evaluation. Entropy, 22(10):1164 2020.
Dylan Molho, Jiayuan Ding. Zhaoheng Li, Hongzhi Wen, Wenzhuo Tang. Yixin Wang, Julian Venegas, Wei Jin, Renming Liu, Runze Su, al. Deep learning in single-cell analysis. arXiv preprint arXiv:2210. 12385, 2022.
Berndt Müller, Joachim Reinhardt, and Michael T Strickland. Neural networks: an introduction. Springer Science & Business Media, 1995.
Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. The pagerank citation ranking: Bring order to the web. Technical report, Technical report, stanford University, 1998.
Pengzhen Ren, Yun Xiao, Xiaojun Chang, Ya-Yao Huang. Zhihui Li, Brij B Gupta. Xiaojiang Chen, and Xin Wang. A survey of deep active learning. ACM computing surveys (CSUR), 54(9):1-40, 2021.
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data. Al magazine, 29(3):93-99) 93, 2008.
Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: core-set approach. arXiv preprint 1rXi::1708.00459 2017.
Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018.
Burr Settles. From theories to queries: Active learning in practice. In Active learning and experimental design workshop in conjunction with AISTATS 20/0, pp. 1-18. JMLR Workshop and Conference Proceedings, 2011 1.
Burr Settles and Mark Craven. An analysis of active learning strategies for sequence labeling tasks. In 2008 Conference on Empirical Methods in Natural Language Processing, EMNLP 2008, Proceedings the Conference, 25-27 October 2008, Honolulu, Hawaii, USA, A meeting of SIGDAT, a Special Interesl Group of the ACL, pp. 1070-1079. ACL, 2008
Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan Günnemann. Pitfalls of graph neural network evaluation. arXiv preprint arXiv: arXiv:/8/ 1.05666 2018.
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. stat, 1050:20, 2017.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks, arXiv preprint arXiv: 10900 2017.
Shiwen Wu, Fei Sun, Wentao Zhang, Xu Xie, and Bin Cui. Graph neural networks in recommender systems: survey. ACM Computing Surveys, 55(5):1-37, 2022.
Yuexin Wu, Yichong Xu, Aarti Singh, Yiming Yang. and Artur Dubrawski. Active learning for graph neural networks via node feature propagation. arXiv preprint arXiv:1910.07567, 2019.
Kaidi Xu, Hongge Chen, Sijia Liu, Pin-Yu Chen, Tsui-Wei Weng, Mingyi Hong, and Xue Lin. Topology attack and defense for graph neural networks: An optimization perspective. arXiv preprint arXiv: 19w.::14900004214,. 2019.
Xueying Zhan, Qingzhong Wang, Kuan-hao Huang, Haoyi Xiong, Dejing Dou, and Antoni B Chan. A comparative survey of deep active learning. arXiv preprint arXiv:2203. 2022.
Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. Graph neural networks: A review of methods and applications. Al open, 1:57-81, 2020.
11

---

Published as a conference paper at ICLR 2024
Appendix
A PRELIMINARY STUDY
A.1 EXAMINATION ORIGINAL NODE FEATCRES
To evaluate the utility of node features in AL., we performed comparative analysis between the FeatProp method and the Random method using Amazon co-purchase graphs, specifically the Computers and Photo datasets (Shchur et al. 2018). We set our analysis across a budget range of 20 to 200, with accuracy results displayed in Figures 8 and For the Random method, nodes are chosen randomly as labeled nodes, based on the pre-determined budget size. In the case of the FeatProp method, the propagated features are obtained through the operation A We then apply the k-means clustering technique, setting the number of clusters equal to the budget size. From each cluster, the node nearest to center is chosen as the labeled node. Our experimental setup primarily employs two GNN models: GCN (Kipf & Welling 2016) and APPNP (Klicpera al. 2018). both used for suner-sueeeeeed node classification tasks. The GCN model features two GCN layers with a hidden size of 16, while the APPNP model boasts two linear layers and ten propagation layers, setting its residual ratio at 0.1. Both models use the ReLU activation function. We train all the models 300 epochs and test the model in the final epoch all nodes in the graph. Our findings suggest that, across most budget sizes, the Random method surpassed FeatProp. This implies that relying solely on original node features may not be beneficial and can, times, hinder AL performance. As result, our study's 's emphasis has been directed towards probing the graph structure.
50
15
Accuracy
Accuracy
60
10-
50
APPUP FestPro2
Aopup Frethrop
202
Budgel Size
Budget Size
Figure 8: Random vs. FeatProp on Computers.
Figure 9; Random vs. FeatProp Photo.
A.2 LABEL POSITION BIAS
In active learning, strategically choosing nodes for labeling can have significant impact on reducing LPS variance, thereby promoting fairnsss within GNNs. To empirically investigate the relationship between strategic node selection and GNN performance, we conducted experiments utilizing the Cora and CiteSeer datasets and employed both GCN and APPNP models. In each experiment, we randomly selected 20 nodes for training and performed three runs of GNNs for each selection. Figure 2 and Figure 10 illustrate the correlation between LPS variance and accuracy, based on 200 random training selections for the Cora dataset using GCN and APPNP model, respectively. Similarly, Figure 11 and Figure [2 depict the same relationship observed for the Citeseer dataset. Our findings reveal that: GNN performance, exemplified in Figure 2 where we observe an accuracy discrepancy of up to 35% and in Figure the accuracy discrepancy up to 30%: (2) Selections characterized by lower LPS variance tend to yield superior GNN results. Thus, beyond its role in fostering fairness, minimizing LPS variance emerges as a pivotal criterion for selecting labeled nodes, ultimately enhancing GNN performance.
(1) Node selection exerts a profound influence
12

---

Published as a conference paper at ICLR 2024
6,822
1300
130M
5,001
03001
Figure 10: APPNP vs. LPS on Figure 11: GCN vs. LPS on Cora.
Figure 12: APPNP LPS
CiteSeer.
CiteSeer.
A.3 STRUCTURE INERTIA SCORE
Adhering to the same setting as discussed in A.2 we investigate the interplay between the SIS and the GNNs performance the Cora and CiteSeer datasets. Figure 3 and Figure [3] shows the the relationship between SIS and accuracy for the Cora dataset, employing GCN and APPNP. respectively, Similarly, Figure [4 and [ present the corresponding analysis for the Citeseer dataset. Notably, our findings suggest that labeling selections with higher SIS tend to yield superior performance. Therefore, SIS can also function as structural criterion for selecting labeled nodes.
Figure 13: APPNP SIS
Figure 14: GCN SIS CiteSeer.
Figure 15: APPNP SIS CiteSeer.
Cora.
B EXPERIMENT
B. DATASETS STATISTICS
In the experiments, the data statisties used in Section 4 are summarized in Table
Table 3: Dataset Statisties.
Dataset
Nodes
Edges
Features
Classes
Partition
Cora
2,708
5,278
1,433
7
CiteSeer
3,327
4.552
3,703
6
14
PubMed
19,717
44,324
500
3
Amazon Computer
13,381
245,778
767
10
9
Amazon Photo
7,487
119,043
745
8
Ogbn-Arxiv
169,343
1,166.243
128
40
5
B.2 HYPERPARAMETERS SETTING
For the experiments in this paper, we mainly utilize two GNN models: GCN (Kipf Welling 2016)
and APPNP (Klicpera al. 2018) for node classification tasks. The GCN model is configured with
13

---

Published as a conference paper at ICLR 2024
two GCN layers, each with hidden size of 16, while the APPNP model consisted of two linear layers and ten propagation layers, with a residual ratio set to 0.1. ReLU activation functions were employed in both GNN models. For the baseline models. we adhere to the settings in (Ma et al. We list some fundamental indicators of the baseline models:
2022).
Learning Rate
Dropout Rate
Weight Decay
Hidden Size
Epochs
Activation Function
0.01
0.5
0.0001
16
300
ReLU
Besides, for the GraphPart (Ma et al. 2022 method, specific partition number is associated with
each dataset. In the case of Cora, CiteSeer, PubMed and Ogbn-Arxiv datasets, we adopt the settings outlined in Ma et al. (2022). For Computer and Photo datasets, we also employ the elhow method to determine the appropriate partition number. Specifically, we conduct the k-means clustering on the graph dataset, exploring range of values for k. typically spanning from 1 to 50. For each value of k, we calculated the sum of squared distances (SSD) between nodes and their respective assigned cluster centers. This metric gauges the proximity of nodes to their cluster centers. Subsequently, we generated a plot that depicts the relationship between the SSD and the number of clusters, k. We identified the elbow point, a juncture where the rate of SSD reduction starts decelerate. This inflection point signifies that increasing the number of clusters beyond this threshold does not yield substantial reduction in SSD and might lead to overfitting. The value of k at this elbow point is deemed the optimal number of clusters for the graph dataset, serving as the designated partition number for the dataset.
For the proposed SCARCE, we choose the learning rate 7) from {0.001, 0.01, 0.1} based on the gradient scale of t. We set the iteration numbers = 100, and the number of random trials R = 3000. The code is available at
B.3 NOISE AND MISSING FEATURE SETTING
In this subsection, we aim to verify the effectiveness of the proposed SCARCE, which can only leverage the graph structure information for active learning, under both noise feature and missing feature settings.
B.3.1 NOISE FEATURES
In the noise feature setting, introduced varying levels of standard Gaussian noise to the original features. Specifically, we modified the features as X = X + k * where e ~ NN,,, 1), and k takes values from the 10.1, 0.3, 0.5, 0.7, 0.9). This experiment was conducted datasets with budget settings of 5C and 10C, where C represents the number of classes in the dataset. We test all the baselines using the GCN and APPNP backbones. The results are shown in Figure Figure 6 and Figure respectively.
the Cora and CiteSeer
Figure 5]
Table 4: The performance of GCN model on the Cora dataset with different levels of noise features.
Budget
sc
IDC
Noree
0.1
0.3
0.5
0.
0.1
B.
0.5
0.7
09
Random
64.32
S1.76
4.19 19
24 To 4.12
11.54x4..1
38.65 = 1.35
74,27
=
61.78
3.42
5493 2
31.30
2.13
48.41 = =1.92
Dagree
2.45
S434
066
0.50
49
1.66
47.37
0.79
74.07
0.35
64.47
5.57
56,00
+ 1162
56.38 = 0.60
Pagenik
60,77
2.522
34.96
2:23
32.36 0.94
12,76 8: 1.27
73.22 à 1:20
60.85
191
S0.33 1:65
44.68
1.005
1..06
PeadPrp
27.512061
4..95
38.50 1.3
32.44 0.3
0.44
55.68
02.54.099
45.03
38.72
2.771
GraphPann
20.22
62.17
45.01 1.85
0024
69.08
56.27
56.34 1.14
51.76
0.44
061
0.78
59.07
Table 5: The performance of GCN model the CiteSeer dataset with different levels of noise
features.
Budget
SC
1DC
Nasee
1.1
0.3
0:3
0.7
10
0.1
B.3
0..
0.7
Random
19.16:3222 3.22
36.36 571
29.53
4.38
27.05 1 2.07
26.34 2 1.85
60,43
44.15
37.951
33.94
1
31,65
0..37
37.27
34.28
0.36
34.53
34.12
0.55
40.20=0.00
39.5012911
35.54 11.54
35.55
0.55
Pagenek
40.34
33.40 2.40
27.09
1.22
2541
1.95
25.08
61.03-0055
41.22+
14.20-0066
12.04 0.66
PeatPop
33.16
26.1
28.13 4.13
25.00 11.11
GraphPPat
52.37
44.63
38.11
35.59
34,42
19.19
20.4% 1.13
37.61
35.45
Sercttuee
59.50
50.35 0.56
41.61 0.83
37.23 + 0.63
35.23
68.23
$1.54
4
41.58
0.99
40.09
From these results, have the following observations:
14

---

Published as a conference paper at ICLR 2024
Table 6: The performance of APPNP model on the Cora dataset with different levels of noise features.
Nouse
0.3
0.3
0.5
0.2
0.1
0.3
0.5
Kansom
60.32
4:4
35.11
35.16
1.00
31.51 444:
77.09
1335
02:31
2:45
14.
2.5
62:38
2.99
60.30 =2335
Degree
67.59 0.92
61.36
0.55
58.98
1::88:0: 0.67
56.29
0.82
55.81
0.88
75.67
=
0.93
66.
66.39 0.41
63.33
0.94
63.49 0.66
Pagenakk
54.90
50.41
48.75 1.99
43.62
2.469
76.23 0.90
67.79 1.72
64.51
60.13 1.22
58.05 = 1.62
PeatPoop
20.17
18.25 1.72
1.6
0.77
25.76 0.86
81005 0.35
1.19
53.96
45.31
39,97 s L.04
GraphPart
=0.68
68.76 1.50
66 1:00
99.14:+...4
54,07 + 2.22
64,05 0
73.79 1.02
66.1+00.59
63.17 1.10
61.91
= 1.19
Seucture
0.66
1:226:00.00
20.10::..21
6.77
82.63 0.59
7
58.73
66.50
66.08 2.58
Table 7: The performance of APPNP model on the CiteSeer dataset with different levels of noise features.
Budget
Nouse
0.1
0.3
0.5
0.1
0.3
0.5
0.7
Kansom
50.91 =
39.3
4:31
3508 5.36
1231 1.47
11.77 44.15
GL.32
4338
2:36
4308 2:42
37.47
38,36 2.07
Degree
50.83 1.38
41.98
1:10
19.77.0..0
36.92
0.38
16.84
50.81 = 2.33
43.26
1000
40.88
0:59
36
0.0.0
37.82 0.81
Pagerank
2.55
21.20::201
$4.63
30.88
2.27
31.75
1.99
2 1.25
46.77
1.40
41.6111011
35.63
= 0.85
RestPop
54.40 1.66
56.12+ 1334
27.69 1.57
25.43
24.31 + 1.53
65.00
1.02
57.86
32.11+1000 1.05
34.88 0.08
27.70 1.05
GraphPPur
60.38
45.39 1.66
40.97 1.14
19.19
38.79
61.59
0.98
4 1000
42.31 126
41.18
39,40
Siucture
62.58
54.76
48.60
45.67 0.55
44.28
a
69.28
0.22
2.3
47.34
173
44.65 2.12
43.55
1.59
With the level of noise increase in the features, the performance of the feature-based methods drops a lot. For example, the FeatProp performs even worse than random selection.
The proposed SCARCE, which only leverages the graph structure in active learning, can outperform baselines by large margin.
B.3.2 MISSING FEATURES
To further test the resilience SCARCE under challenging feature conditions, we conducted experiments in setting where all features are missing. In this scenario, we replaced the original node features with one-hot ID features. We conduct experiments on both Cora and CiteSeer datasets with labeling budgets of SC, 10C, and 20C using both GCN and APPNP models. The results shown in Figure and Figure 9 respectively.
Table 8: The performance GCN model on Cora and CiteSeer dataset without node features.
Dataset
Cora 10C
CiteSeer 10C
Budget
5C
20C
5C
20C
Random
44.76
56.47 + 2.30
64.03
1.74
28.41 2.69
34.78 4 3.06
40.58
2.61
Degree
44.19
51.69 + 2.82
63.26 1.95
27.33
1.97
+
40.99 + 2.02
Pagerank
3.55
47.95 +
63.93 3.85
24.03 + 1.96
32.83
40.35 # 2.02
FeatProp
3.116
$8.84
65.66
29.79 2..32
34.79 + 2.00
41.89 # 2.71
GraphPart
55.37
57.40 4.24
67.60
38.27
2.08
37.40 2.56
43.61
3.15
SCARCE
61.43
68.64 1.94
69.59 = 1.22
37.89
1.94
43.51 + 1.44
48.20 + 1.15
Table 9: The performance of APPNP model on Cora and CiteSeer dataset without node features.
Dataset
Cora 10C
CiteSeer 10C
Budget
5C
20C
5C
20C
Random
62.27 4.86
13.7133.22
75.52 1.65
38.24 4 3.21
46. 2.85
49.98 1.52
Degree
64.48 2.97
65.29 1.63
1.63
38.92 4479
46.93
1.60
49.10
2.16
Pagerank
6.67
1.11
74.15
11.94
42.58 1.39
50.70 1.05
FeatProp
0.99
47.42
0.44
3.59 0.46
29.01 = 0.56
GraphPart
66.40 0.60
0.59
74.15 + 1.38
0.62
4
51.34 + 0.78
SCARCE
72.39 1.54
0.99
76.77 + 0.71
48.06
4.66
52.20 + 2.02
56.21 # 1.92
From the results, we can have similar findings: While the feature-based methods, such as FeatProp, perform poorly under the missing feature setting, our proposed SCARCE have good performance.
In summary, the proposed method SCARCE can perform as expected when the feature quality is low missing, but the ceasede-based active learning methods may perform poorly.
15

---

Published as a conference paper at ICLR 2024
B.4 THE PERFORMANCE WITH MORE BACKBONES
To verify the effectiveness of the proposed SCARCE with different backbones. We further add two representative GNNs: one GAT GATVVelicovvii and the other one is the GCNII Chen et al. (2020). which is deep GNN. For these two methods, follow the hyperparameter settings in their original paper. Specifically, for GCNII, we adopt 64 layers. We test these two backbones using the Cora and CisteSeer datasets with budgets of 5C, 10C, and 20C. The results with GCNII and GAT backbones are shown in Figure T0 and Figure 11 respectively. From these results, we observe that the proposed SCARCE can work well with both GAT and GCNII backbones.
al. (017)) which leverages the attention mechanism,
Table 10: The performance on Cora and CiteSeer dataset with GCNII backbone.
Dataset
Cora 10C
CiteSeer 10C
Budget
SC
20C
5C
20C
Random
69.76 + 6.47
78.29 = 3.99
83.24
1.08
55.25
7.68
65.59 = 1.60
68.25 = 1.28
Uncertainty
68.78 5.64
3.36
82.33 1 1.30
8.70
62.91
5.47
67.97 1.69
Density
72.31 3.10
1.61
83.22
7.049
65,41
2.80
68.07
CoreSet
3.34
16.11
1.97
54122
4..05
67.41 1.02
Degree
6.902.11.99
77.92 0.45
0.41
58.99
6.96
67.41 + 0.50
Pagerank
2.78
76.64
1.27
0.55
2.77
1.00
69.53 0.66
AGE
3.89
#..14
1.22
4..75
6..4
67.59 2.60
FeatProp
2.559
0.58
8.64
2..99
2..9
63.01 0.93
GraphPart
40.45
0.447
83.4
2.22
1..9
65.99 + 1.08
Structure
79.96 1.10
0.54
03498
0.72
0.79
72.05 0.35
Feature
82.05 1.32
83,25
0.59
70.98
72.25 0.74
ALL
82.61
0.51 0.47
6..07
70.91 0.42
72.89 0.44
Table 11: The performance on Cora and CiteSeer dataset with GAT backbone.
Dataset
Cora 10C
CiteSeer 10C
Budget
SC
20C
SC
20C
Random
65.66
3.70
74.06 = 4.19
80,00 : 2.24
51.36 7.53
61.85 + 2.34
64,82 # 2.15
Uncertainty
55.31 + 5.78
62.02 6.56
74.72 3.69
6.35
5.64
60.49
4.00
Density
64.10
70.72
4.544 2.977
64.17 6..70
CoreSet
6.072
72.44
79.48 + 1.52
6.76
58.00
6.154
Degree
4.344 2.22
78.31
1.81
1.22
49.38 2.20
Pagerank
71.84 2.05
80.00
2.44
66.51 0.77 1..72
AGE
66.45
3.89
80.14
65.18 58.97 49.53
FeatProp
73.40 1.39
77.62 1.36
0.53
28.99
62.14
68.08 + 0.54
GraphPart
76.91 1.41
78.29 + 0.88
80.94
58.24
65.10
66.41 + 1.33
Structure
76.36
81.78 0.46
82.58
6.923
0.60
72.36 0.49
Feature
80.56
83,01
0.26
63.11
0.64
72.60 0.38
ALL
80.62 0.50
83.27 + 0.39
84.96 + 0.24
2.77
70.2940.66
72.34 0.40
B.5 THE PERFORMANCE ON OGB PRODUCTS DATASET
To verify the scalability of the proposed SCARCE, we further conducted extensive experiments on the OGB Products dataset, which contains over 2.4 million nodes. We also use the budgets of 5C, 10C, and 20C. The results with both GCN and APPNP backbones are shown in Figure 12
Table 12: Products
Dataset
GCN
APPNP
Budget
5C
10C
20C
5C
10C
20C
Random
60.78 + 0.60
67.38 = 0.74
70.45 : 0.89
66. 16.15 + 1.43
70,23 + 0.42
74.82 # 0.10
Degree
37.13
0.46
0.46
45.82 0.38
50.19
0.99
49,98 0.85
0.74
60.81 0.97
Pagerank
56.33
63.20 0.20
69.00 0.45
61.08 0.32
67.61
72.69 0.18
FeatProp
6..34 2.58
67.48 = 0.97
71.65 0.57
68.64 +.65
69.80 1.20
71.8 0.61
GraphPart
0OT
OOT
OOT
OOT
OOT
OOT
SCARCE
62.25 + 0.37
70.27 # 0.22
74.43 0.22
66.66 +.0
72.63 + 0.20
76.57 # 0.14
16

---

Published as a conference paper at ICLR 2024
Please note the GraphPart method takes more than 24 hours, and treat it out of time (OOT).
The results from our experiments on the large-scale OGB Products dataset clearly demonstrate the efficacy of our proposed SCARCE method. Notably, SCARCE achieves high accuracy with significantly fewer labeled nodes compared to standard training splits. For instance, while the standard split typically uses around 200,000 nodes for training, SCARCE requires only 940 labeled nodes to attain accuracies of 74.43/(75.64 and 76.57/(76.62) for the GCN and APPNP models, respectively, where the value in the (-) represent the accuracy of the standard split. This underscores SCARCE's effectiveness in active learning, particularly in large-scale graph neural network applications.
17


Reviews:
[{"review_id": "CrLZrEeWaA", "reviewer": "Reviewer_BDeb", "summary": "Existing active learning models for GNNs heavily rely on the quality of initial node features and ignore the impact of label position bias in the selection of representative nodes. To address these limitations, this paper proposes a novel framework called SCARCE.", "questions": "Please refer to the weaknesses.\n- There are concerns regarding the fundamental motivation behind active learning. While the primary motivation for active learning lies in the difficulty of obtaining high-quality labels in real-world scenarios, the iterative addition of labels for learned target nodes during the optimization process raises doubts about the original motivation. This creates some contradiction as it suggests that labels for target nodes might be easy to obtain.\n- The improvement compared to baselines seems not statistically significant.\n- They argue that existing methods heavily rely on the quality of initial node features while the proposed framework can mitigate this problem. However, there lack of experimental support. The features of datasets seem typical, lacking any characteristics such as unavailability and noise. There is a need for a quantifiable evaluation to support this point.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "+ They identify the limitations in current active learning methods, specifically the oversight regarding feature quality and position bias.\n+ They propose a novel framework to tackle the aforementioned limitations.\n+ Extensive experiments validate the effectiveness of the proposed framework.", "weaknesses": "- There are concerns regarding the fundamental motivation behind active learning. While the primary motivation for active learning lies in the difficulty of obtaining high-quality labels in real-world scenarios, the iterative addition of labels for learned target nodes during the optimization process raises doubts about the original motivation. This creates some contradiction as it suggests that labels for target nodes might be easy to obtain.\n- The improvement compared to baselines seems not statistically significant.\n- They argue that existing methods heavily rely on the quality of initial node features while the proposed framework can mitigate this problem. However, there lack of experimental support. The features of datasets seem typical, lacking any characteristics such as unavailability and noise. There is a need for a quantifiable evaluation to support this point.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KoAVYAfDgA", "reviewer": "Reviewer_UJDy", "summary": "This paper proposes a unified optimization framework for active learning on graph neural networks (GNNs) that can flexibly incorporate different selection criteria such as structure inertia score (SIS) and label proximity score (LPS) variance. It is empirically demonstrated that SCARCE outperforms existing baselines on node classification tasks across multiple benchmark datasets. In particular, SCARCE achieves higher accuracy than methods like FeatProp and GraphPart while also enhancing fairness by reducing variance in LPS across nodes.", "questions": "(1) I would suggest to add a formal introduction about the fairness notion studied in this paper in Section 2, and add a descriptive discussion in the Introduction accordingly.\n\n(2) If the proposed framework can be easily generalized onto large network data, will the performance superiority still be maintained?\n\n(3) If the proposed framework can be easily generalized onto featureless network data, will the performance superiority still be maintained? Note that in such cases, the feature input of GNNs can be generated following traditional ways.\n\n(4) Can the proposed framework achieve generally good performance across different state-of-the-art GNN backbones? It would be better to adopt more backbones for experiments.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(1) This paper is generally well-organized and easy-to-follow.\n\n(2) The proposed unified optimization framework is flexible and does not require extensive hyperparameter tuning, which is especially useful for active learning. In addition, the scalability seems promising as well.\n\n(3) The superiority on utility and fairness seems significant given the presented results in section 4.3 and 4.4.", "weaknesses": "(1) There lacks a formal introduction of the notion for fairness at the beginning of this paper.\n\n(2) Despite the discussion on scalability, this paper does not perform any experiments on large-scale network datasets.\n\n(3) Only performing experiments on two GNN backbones undermines the superiority of the proposed framework. In addition, one advantage of this paper lies in the applicability on featureless networks, which is not tested in this paper either.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GaFhBTnTLY", "reviewer": "Reviewer_tqC2", "summary": "To leverage graph structure and mitigate structural bias in active learning, the authors present a unified optimization framework.", "questions": "1. Why can we use the relaxation of the binary vector t to its convex hull?\n2. How does the proposed method solve the fairness issue from the theoretical aspect? \n3. What is the formal definition of the structure bias in graphs, and how can we quantify it?\n4. Why do we use active learning to solve the fairness issue in graphs? What if we do not have access to the oracle?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 1, "presentation": 2, "contribution": 1, "strengths": "Originality: The investigation of structure fairness using active learning is something new.\n\nQuality: The technical quality is below average. Many details are missing. For example, label position bias is a new term that was recently proposed, and the authors should elaborate more on it with a more intuitive explanation instead of just some formula. Also, at the beginning of Sec. 3.1, $t$ is a binary vector, and thus it should be $t \\in \\\\{0,1\\\\}^n$. The relaxation in the paper does not make sense to the reviewer.\n\nClarity: In general, it is ok. The reviewer understands how the proposed method works but sometimes fails to see why.\n\nSignificance: The fairness and active learning problems for graphs are important.", "weaknesses": "1. The paper tries to solve the structure fairness problem using active learning. However, the connection between these two is weak, and the reviewer does not find any strong motivations to do so. The authors claim that \"in active learning, strategically choosing labeling nodes, represented by t, can potentially reduce the LPS variance, promoting fairness in GNNs\" in the second para. of Sec. 2.2, but the reviewer does not find any theoretical guarantees to motivate this finding. \n\n2. The goal of active learning is different from mitigating the bias in graphs and the ultimate goal of AL is to use as few as labeled nodes to achieve the best prediction performance. Therefore, the motivation of this work is totally unclear. \n\n3. The paper does not provide any theoretical proof to support the findings or the motivations. The relaxation used in the unified framework is also misleading.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Pvl7ZXW0qi", "reviewer": "Reviewer_RKZS", "summary": "The study focuses on enhancing the performance of GNNs for semi-supervised node classification, even when high-quality labeled samples are scarce. Traditional active learning methods may not work optimally in graph data, given their unique structures and the bias introduced by the positioning of labeled nodes. To address this, the researchers introduce a unified optimization framework called SCARCE, which can be combined with node features. Their experiments confirm that this method not only enhances GNN performance but also helps mitigate structural bias and improve fairness in the results.", "questions": "See Weakness.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Comparison with many baselines in this paper is very good.", "weaknesses": "1. The paper is not easy to follow. I suggest the existing work on fairness in GNN should be discussed. Otherwise, it is very hard to estimate the significance of this work. \n\n2. My major concern is that the fairness definition in this paper is not very clear. We usually use demographic party (DP) or equal odds (EO) to measure fairness. However, this paper only uses the Standard Deviation (SD) and the Coefficient of Variation. Why do the authors consider them instead of DP and EO?\n\n3. Figure 6 and Figure 7 are also not very clear to me. The authors discussed that 'SCARCE, which combines both SIS and LPS variance, SCARCE can not only elevate overall performance but also attain commendable fairness'. However, it is very hard to get this result from these two figures. I suggest the authors provide more details for examination.\n\n4. This paper should focus on fairness instead of classification accuracy. However, Tables 1 and 2 provide more details about the classification accuracy. There should be a trade-off between accuracy and fairness. Only showing accuracy does not make any sense. In addition, how to balance the trade-off between accuracy and fairness in this paper. I do not find any implementation details related to this.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)