Original: T OPOLOGY M ATTERS IN F AIR G RAPH L EARNING : A T HEORETICAL P ILOT S TUDY
Revised: T OPOLOGY M ATTERS IN F AIR G RAPH L EARNING : A T HEORETICAL P ILOT S TUDY

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Recent advances in fair graph learning observe that graph neural networks (GNNs) further amplify prediction bias compared with multilayer perception (MLP), while the reason behind this is unknown.
Revised: Recent advances in fair graph learning observe that graph neural networks (GNNs) further amplify prediction bias compared with multilayer perception (MLP), while the reason behind this is unknown.

Original: In this paper, we conduct a theoretical analysis of the bias amplification mechanism in GNNs.
Revised: In this paper, we conduct a theoretical analysis of the bias amplification mechanism in GNNs.

Original: This is a challenging task since
Revised: This is a challenging task since

Original: GNNs are difficult to be interpreted, and real-world networks are complex.
Revised: GNNs are difficult to be interpreted, and real-world networks are complex.

Original: To bridge the gap, we theoretically and experimentally demonstrate that aggregation operation in representative GNNs accumulates bias in node representation due to topology bias induced by graph topology.
Revised: To bridge the gap, we theoretically and experimentally demonstrate that aggregation operation in representative GNNs accumulates bias in node representation due to topology bias induced by graph topology.

Original: We provide a sufficient condition identifying the statistical information of graph data, so that graph aggregation enhances prediction bias in GNNs.
Revised: We provide a sufficient condition identifying the statistical information of graph data, so that graph aggregation enhances prediction bias in GNNs.

Original: Motivated by this data-centric finding, we propose a fair graph refinement algorithm, named FairGR , to rewire graph topology to reduce sensitive homophily coefficient while preserving useful graph topology.
Revised: Motivated by this data-centric finding, we propose a fair graph refinement algorithm, named FairGR , to rewire graph topology to reduce sensitive homophily coefficient while preserving useful graph topology.

Original: Experiments on node classification tasks demonstrate that FairGR can mitigate the prediction bias with comparable performance on three real-world datasets.
Revised: Experiments on node classification tasks demonstrate that FairGR can mitigate the prediction bias with comparable performance on three real-world datasets.

Original: Additionally, FairGR is compatible with many state-of-the-art methods, such as adding regularization, adversarial debiasing, and Fair mixup via refining graph topology.
Revised: Additionally, FairGR is compatible with many state-of-the-art methods, such as adding regularization, adversarial debiasing, and Fair mixup via refining graph topology.

Original: Therefore, FairGR is a plug-in fairness method and can be adapted to improve existing fair graph learning strategies.
Revised: Therefore, FairGR is a plug-in fairness method and can be adapted to improve existing fair graph learning strategies.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Graph neural networks (GNNs) (Kipf & Welling, 2017; Veli ˇ ckovi ´ c et al., 2018; Wu et al., 2019) are widely adopted in various domains, such as social media mining (Hamilton et al., 2017), knowledge graph (Hamaguchi et al., 2017) and recommender system (Ying et al., 2018), due to remarkable performance in learning representations.
Revised: Graph neural networks (GNNs) (Kipf & Welling, 2017; Veli ˇ ckovi ´ c et al., 2018; Wu et al., 2019) are widely adopted in various domains, such as social media mining (Hamilton et al., 2017), knowledge graph (Hamaguchi et al., 2017) and recommender system (Ying et al., 2018), due to remarkable performance in learning representations.

Original: Graph learning, a topic with growing popularity, aims to learn node representation containing both topological and attribute information in a given graph.
Revised: Graph learning, a topic with growing popularity, aims to learn node representation containing both topological and attribute information in a given graph.

Original: Despite the outstanding performance in various tasks, GNNs still inherit or even amplify societal bias from input graph data (Dai & Wang, 2021).
Revised: Despite the outstanding performance in various tasks, GNNs still inherit or even amplify societal bias from input graph data (Dai & Wang, 2021).

Original: The biased node representation largely limits the application of GNNs in many high-stake tasks, such as job hunting (Mehrabi et al., 2021) and crime ratio prediction (Suresh & Guttag, 2019).
Revised: The biased node representation largely limits the application of GNNs in many high-stake tasks, such as job hunting (Mehrabi et al., 2021) and crime ratio prediction (Suresh & Guttag, 2019).

Original: Hence, bias mitigation that facilitates the research on fair
Revised: Hence, bias mitigation that facilitates the research on fair

Original: GNNs is in urgent need.
Revised: GNNs is in urgent need.

Original: In many real-world graphs, nodes with the same sensitive attribute (e.g., ages) are more likely to connect.
Revised: In many real-world graphs, nodes with the same sensitive attribute (e.g., ages) are more likely to connect.

Original: For example, young people mainly make friends with people of similar ages (Dong et al., 2016).
Revised: For example, young people mainly make friends with people of similar ages (Dong et al., 2016).

Original: We call this phenomenon “topology bias”.
Revised: We call this phenomenon “topology bias”.

Original: Even worse, in GNNs, the representation of each node is learned by aggregating the representations of its neighbors.
Revised: Even worse, in GNNs, the representation of each node is learned by aggregating the representations of its neighbors.

Original: Thus, nodes with the same sensitive attributes will be more similar after the aggregation.
Revised: Thus, nodes with the same sensitive attributes will be more similar after the aggregation.

Original: To get a sense, we visualize the topology bias for three real-world datasets (Pokec-n, Pokec-z, and NBA) in Figure 1, where different edge types are highlighted with different colors for the top- 3 largest connected components in original graphs.
Revised: To get a sense, we visualize the topology bias for three real-world datasets (Pokec-n, Pokec-z, and NBA) in Figure 1, where different edge types are highlighted with different colors for the top- 3 largest connected components in original graphs.

Original: Such topology bias leads to more similar node representation for those nodes with the same sensitive attribute, which is a major source of the graph representation bias.
Revised: Such topology bias leads to more similar node representation for those nodes with the same sensitive attribute, which is a major source of the graph representation bias.

Original: Existing bias mitigation work for GNNs is empirical via adding regularization, adversarial debiasing, or contrastive learning.
Revised: Existing bias mitigation work for GNNs is empirical via adding regularization, adversarial debiasing, or contrastive learning.

Original: These works are motivated by that fact that graph neural networks trained on graphs may inherit the societal bias in data, and the topology of graphs and the message passing in GNNs could even magnify the bias compared with multilayer perception (MLP)
Revised: These works are motivated by the fact that graph neural networks trained on graphs may inherit the societal bias in data, and the topology of graphs and the message passing in GNNs could even magnify the bias compared with multilayer perception (MLP)

Original: (Dai & Wang, 2021).
Revised: (Dai & Wang, 2021).

Original: However, even though fair prediction in GNN can be achieved via a fair training strategy, a
Revised: However, even though fair prediction in GNN can be achieved via a fair training strategy, a

Original: [Figure]
Revised: [Figure]

Original: represent the edge with the same or different sensitive attributes for the connected node pair, respectively.
Revised: represent the edge with the same or different sensitive attributes for the connected node pair, respectively.

Original: We visualize the largest three connected components for each dataset.
Revised: We visualize the largest three connected components for each dataset.

Original: It is obvious that the sensitive homophily coefficients (the ratio of homo edges) are high in practice, i.e., 95 . 30% , 95 .
Revised: It is obvious that the sensitive homophily coefficients (the ratio of homo edges) are high in practice, i.e., 95 . 30% , 95 .

Original: 06%  and 72 .
Revised: 06% , and 72 .

Original: 37% for Pokec-n, Pokec-z and NBA dataset, respectively.
Revised: 37% for Pokec-n, Pokec-z, and NBA dataset, respectively.

Original: fundamental understanding of why large topology bias and message passing algorithm amplifying the bias happens is still missing.
Revised: fundamental understanding of why large topology bias and message passing algorithm amplifying the bias happens is still missing.

Original: A natural question is raised:
Revised: A natural question is raised:

Original: Can we theoretically understand why large topology bias and message passing algorithm amplify the bias from  graph data perspective?
Revised: Can we theoretically understand why large topology bias and message passing algorithm amplify the bias from a graph data perspective?

Original: In this work, we move the first step to understand why large topology bias and message passing algorithm amplify the bias.
Revised: In this work, we move the first step to understand why large topology bias and message passing algorithm amplify the bias.

Original: Specifically, we first define the sensitive homophily coefficient to describe how likely the connected nodes are with the same sensitive attributes.
Revised: Specifically, we first define the sensitive homophily coefficient to describe how likely the connected nodes are with the same sensitive attributes.

Original: Subsequently, we theoretically prove that the GCN-like aggregation in message passing inevitably accumulates representation bias for graphs with large sensitive homophily coefficients.
Revised: Subsequently, we theoretically prove that the GCN-like aggregation in message passing inevitably accumulates representation bias for graphs with large sensitive homophily coefficients.

Original: Second, motivated by our theoretical analysis, we develop a fair graph refinement algorithm, named FairGR, to achieve fair GNN prediction via revising graph topology.
Revised: Second, motivated by our theoretical analysis, we develop a fair graph refinement algorithm, named FairGR, to achieve fair GNN prediction via revising graph topology.

Original: More importantly, FairGR is a plug-in data refinement method and compatible with many fair training strategies, such as regularization, adversarial debiasing, and Fair mixup.
Revised: More importantly, FairGR is a plug-in data refinement method and compatible with many fair training strategies, such as regularization, adversarial debiasing, and Fair mixup.

Original: In short, the contributions can be summarized as follows:
Revised: In short, the contributions can be summarized as follows:

Original: • To the best of our knowledge, it is the first paper to theoretically investigate why the GCN-like message passing scheme amplifies representation bias for large topology bias.
Revised: • To the best of our knowledge, it is the first paper to theoretically investigate why the GCN-like message passing scheme amplifies representation bias for large topology bias.

Original: Specifically, we provide a sufficient condition of graph data to show that GCN-like message passing amplifies representation bias.
Revised: Specifically, we provide a sufficient condition of graph data to show that GCN-like message passing amplifies representation bias.

Original: • Motivated by our theoretical analysis, we propose a graph topology refinement method, named FairGR, to achieve fair prediction.
Revised: • Motivated by our theoretical analysis, we propose a graph topology refinement method, named FairGR, to achieve fair prediction.

Original: • We empirically show that the prediction bias of GNNs is larger than that of MLP on real-world datasets.
Revised: • We empirically show that the prediction bias of GNNs is larger than that of MLP on real-world datasets.

Original: Additionally, the effectiveness of FairGR is experimentally evaluated onthree real-
Revised: Additionally, the effectiveness of FairGR is experimentally evaluated on

Original: world datasets.
Revised: three real-world datasets.

Original: The results show that compared to the state-of-the-art, our FairGR exhibits a superior trade-off between prediction performance and fairness, and is compatible with many fair training strategies, such as regularization, adversarial debiasing, and Fair mixup.
Revised: The results show that compared to the state-of-the-art, our FairGR exhibits a superior trade-off between prediction performance and fairness, and is compatible with many fair training strategies, such as regularization, adversarial debiasing, and Fair mixup.

Original:  P RELIMINARIES 2. N OTATIONS
Revised:  P RELIMINARIES 2. N OTATIONS

Original: We adopt bold upper-case letters to denote matrix such as X , bold lower-case letters such as x to denote vectors or random variables, and calligraphic font such as X to denote set.
Revised: We adopt bold upper-case letters to denote matrices such as X , bold lower-case letters such as x to denote vectors or random variables, and calligraphic font such as X to denote set.

Original: Given a matrix
Revised: Given a matrix

Original: X ∈ R n × d , the i -th row and j -th column are denoted as X i and X · ,j , and the element in i -th row and j -th column is X i,j .
Revised: X ∈ R n × d , the i -th row and j -th column are denoted as X i and X · ,j , and the element in i -th row and j -th column is X i,j .

Original: We use l 1 norm of matrix X as || X
Revised: We use l 1 norm of matrix X as || X

Original: || 

Revised: ||

Original: = (cid:80) ij | X ij | .
Revised: = (cid:80) ij | X ij | .

Original: Let G =
Revised: Let G =

Original: {V , E} be a graph with the node set V
Revised: {V , E} be a graph with the node set V

Original: = { v 1 , · · · , v n } and the undirected edge set E
Revised: = { v 1 , · · · , v n } and the undirected edge set E

Original: = { e 1 , · · · , e m } , where n, m represent the number of node and edge, respectively.
Revised: = { e 1 , · · · , e m } , where n, m represent the number of node and edge, respectively.

Original: The graph structure G can be represented as an adjacent matrix A ∈ R n × n , where A ij = 1 if existing edge between node v i and node v j .
Revised: The graph structure G can be represented as an adjacent matrix A ∈ R n × n , where A ij = 1 if existing edge between node v i and node v j .

Original: N ( i ) denotes the neighbors of node v i and ˜ N ( i )
Revised: N ( i ) denotes the neighbors of node v i and ˜ N ( i )

Original: =
Revised: =

Original: N ( i ) ∪ { v i } denotes the self-inclusive neighbors.
Revised: N ( i ) ∪ { v i } denotes the self-inclusive neighbors.

Original: Suppose that each node is associated with a d -dimensional feature vector and a (binary) sensitive attribute, the feature for all nodes and sensitive attribute are denoted as X ori
Revised: Suppose that each node is associated with a d -dimensional feature vector and a (binary) sensitive attribute, the feature for all nodes and sensitive attribute is denoted as X ori

Original: = R n × d and s ∈ {− 1 , 1 } n .
Revised: = R n × d and s ∈ {− 1 , 1 } n .

Original: I ( s , X ) represents the mutual information between the sensitive attribute and node features.
Revised: I ( s , X ) represents the mutual information between the sensitive attribute and node features.

Original: A ⊙ B represents Hadamard product for matrix element-wise multiplication.
Revised: A ⊙ B represents Hadamard product for matrix element-wise multiplication.

Original: 2. L ABEL AND S ENSITIVE H OMOPHILY C OEFFICIENT IN G RAPHS
Revised: 2. L ABEL AND S ENSITIVE H OMOPHILY C OEFFICIENT IN G RAPHS

Original: The behaviors of graph neural networks have been investigated in the context of label homophily for connected node pairs in graphs (Ma et al., 2021).
Revised: The behaviors of graph neural networks have been investigated in the context of label homophily for connected node pairs in graphs (Ma et al., 2021).

Original: Label homophily in graphs is typically defined to characterize the similarity of connected node labels in graphs.
Revised: Label homophily in graphs is typically defined to characterize the similarity of connected node labels in graphs.

Original: Here, similar node pair means that the connected nodes share the same label.
Revised: Here, similar node pair means that the connected nodes share the same label.

Original: From the perspective of fairness, we also define the sensitive homophily coefficient to represent the sensitive attribute similarity among connected node pairs.
Revised: From the perspective of fairness, we also define the sensitive homophily coefficient to represent the sensitive attribute similarity among connected node pairs.

Original: Informally, the coefficient for label homophily and sensitive homophily are defined as the fraction of the edges connecting the nodes of the same class label and sensitive attributes in a graph (Zhu et al., 2020; Ma et al., 2021).
Revised: Informally, the coefficients for label homophily and sensitive homophily are defined as the fraction of the edges connecting the nodes of the same class label and sensitive attributes in a graph (Zhu et al., 2020; Ma et al., 2021).

Original: We also provide the formal definition as follows:
Revised: We also provide the formal definition as follows:

Original: Definition 1 (Label and Sensitive Homophily Coefficient)
Revised: Definition 1 (Label and Sensitive Homophily Coefficient)

Original: Given a graph G =
Revised: Given a graph G =

Original: {V , E} with node label vector y , node sensitive attribute vector s , the label and sensitive attribute homophily coefficients are defined as the fraction of edges that connect nodes with the same labels or sensitive attributes ϵ label ( G , y )
Revised: {V , E} with node label vector y , node sensitive attribute vector s , the label and sensitive attribute homophily coefficients are defined as the fraction of edges that connect nodes with the same labels or sensitive attributes ϵ label ( G , y )

Original: =
Revised: =

Original: 1 |E| (cid:80) ( i,j ) ∈E 1 ( y i = y j ) , and ϵ sens ( G , s )
Revised: 1 |E| (cid:80) ( i,j ) ∈E 1 ( y i = y j ) , and ϵ sens ( G , s )

Original: = 1 |E| (cid:80) ( i,j ) ∈E 1 ( s i =
Revised: = 1 |E| (cid:80) ( i,j ) ∈E 1 ( s i =

Original: s j ) , where |E| is the number of edges and 1 ( · ) is the indicator function.
Revised: s j ) , where |E| is the number of edges and 1 ( · ) is the indicator function.

Original: Recent works (Ma et al., 2021; Chien et al., 2021; Zhu et al., 2020) aim to understand the relation between the message passing in GNNs and label homophily from the interactions between GNNs (model) and graph topology (data).
Revised: Recent works (Ma et al., 2021; Chien et al., 2021; Zhu et al., 2020) aim to understand the relation between the message passing in GNNs and label homophily from the interactions between GNNs (model) and graph topology (data).

Original: For graph data with a high label homophily coefficient, existing works (Ma et al., 2021; Tang et al., 2020) have demonstrated, either provably or empirically, that the nodes with higher node degree obtain more prediction benefits in GCN, compared to the benefits that peripheral nodes obtain.
Revised: For graph data with a high label homophily coefficient, existing works (Ma et al., 2021; Tang et al., 2020) have demonstrated, either provably or empirically, that the nodes with higher node degree obtain more prediction benefits in GCN, compared to the benefits that peripheral nodes obtain.

Original: As for graph data with a low label homophily coefficient, GNNs do not necessarily lead to better prediction performance compared with MLP since the node features of neighbors with different labels contaminate the node features during feature aggregation.
Revised: As for graph data with a low label homophily coefficient, GNNs do not necessarily lead to better prediction performance compared with MLP since the node features of neighbors with different labels contaminate the node features during feature aggregation.

Original: However, although work (Dai & Wang, 2021) empirically points out that graph data with a large sensitive homophily coefficient may enhance bias in GNNs, the fundamental understanding of message passing and graph data properties, such as sensitive homophily coefficient, is still missing.
Revised: However, although work (Dai & Wang, 2021) empirically points out that graph data with a large sensitive homophily coefficient may enhance bias in GNNs, the fundamental understanding of message passing and graph data properties, such as sensitive homophily coefficient, is still missing.

Original: We provide the theoretical analysis in Section 3.
Revised: We provide the theoretical analysis in Section 4.

Original:  T OPOLOGY E NHANCES B IAS IN GNN S A GGREGATION
Revised:  R ELATED W ORK

Revised: We briefly review the existing work on graph neural networks and fairness-aware learning on graphs. (Please refer to Appendix D for a more comprehensive discussion.). Existing GNNs can be roughly divided into spectral-based and spatial-based GNNs.

Revised: Spectral-based GNNs provide graph convolution definition based on graph theory (Bruna et al., 2013; Defferrard et al., 2016; Henaff et al., 2015).

Revised: Spatial-based GNNs variant are popular due to explicit neighbors’ information aggregation, including Graph convolutional networks (GCN) (Kipf & Welling, 2017), graph attention network (GAT) (Veliˇckovi´c et al., 2018).

Revised: As for fairness in graph data, many works have been developed to achieve fairness in machine learning community (Chuang & Mroueh, 2020), including fair walk (Rahman et al., 2019), adversarial debiasing (Dai & Wang, 2021), Bayesian approach (Buyl & De Bie, 2020), and contrastive learning (Agarwal et al., 2021).

Revised: Much literature empirically shows that GNN or message passing may enhance prediction bias compared with MLP.

Revised: However, the theoretical understanding of why such a phenomenon happens is still unclear.

Revised: In this work, we take an initial step toward theoretically understanding why message passing enhances bias from a data perspective.

Revised: Based on this understanding, we develop a simple yet effective fair graph refinement method to achieve better tradeoff performance.

Revised: More importantly, the proposed FairGR is compatible with many fair training strategies.

Revised:  T OPOLOGY E NHANCES B IAS IN GNN S A GGREGATION

Original: For each node, GNNs aggregate their neighbors’ features to learn its representation.
Revised: For each node, GNNs aggregate their neighbors’ features to learn their representation.

Original: In real-world datasets, we observe a high sensitive homophily coefficient (which is even higher than the label homophily coefficient).
Revised: In real-world datasets, we observe a high sensitive homophily coefficient (which is even higher than the label homophily coefficient).

Original: However, existing message passing schemes tend to aggregate the node features from their neighbors with the same sensitive attributes.
Revised: However, existing message-passing schemes tend to aggregate the node features from their neighbors with the same sensitive attributes.

Original: Thus, the common belief is that the message passing renders node representations with the same sensitive attribute more similar.
Revised: Thus, the common belief is that the message passing renders node representations with the same sensitive attribute more similar.

Original: However, this common belief is heuristic and the quantifiable relationship between the topology bias and representation bias is still missing.
Revised: However, this common belief is heuristic and the quantifiable relationship between the topology bias and representation bias is still missing.

Original: In this section, (1) we rectify such common belief and quantitatively reveal that only sufficient high sensitive homophily coefficient would lead to bias enhancement; (2) we analyze the influence of other graphs statistical information, such as the number of nodes n , the edge density ρ d , and sensitive group ratio c , in term of bias enhancement.
Revised: In this section, (1) we rectify such common belief and quantitatively reveal that only sufficient high sensitive homophily coefficient would lead to bias enhancement; (2) we analyze the influence of other graphs statistical information, such as the number of nodes n , the edge density ρ d , and sensitive group ratio c , in term of bias enhancement.

Original: 3. S YNTHETIC G RAPH
Revised: 4. S YNTHETIC G RAPH

Revised: we consider the synthetic random graph generation using contextual stochastic block model (CSBM) (Fortunato & Hric, 2016), including graph topology and node features generation with Gaussian mixture distribution.

Revised: As a pilot study, we choose the most common-used graph topology with stochastic block model (SBM) and node feature with Gaussian mixture distributions in the analysis.

Revised: The rationale for the choice is due to the fact that SBM is widely used to model and analyze most complex networks (e.g., social networks, World Wide Web, and biological networks) (Van

Original:  Graph Topology Throughout our analysis, we mainly focus on 4 characteristics of graph topology: the number of nodes n , the edge density ρ d , sensitive homophily coefficient ϵ sens , and sensitive group ratio c .
Revised: Der Hofstad, 2016). Graph Topology Throughout our analysis, we mainly focus on 4 characteristics of graph topology: the number of nodes n , the edge density ρ d , sensitive homophily coefficient ϵ sens , and sensitive group ratio c .

Original: Specifically, we consider the synthetic random graph  generation as follows:
Revised: Specifically, we consider the synthetic random graph generation, including graph topology and node features generation as follows:

Original: Definition 2 ( ( n, ρ d , ϵ sens , c ) -graph)
Revised: Definition 2 ( ( n, ρ d , ϵ sens , c ) -graph)

Original: The synthetic random graph G sampled from ( n, ρ d , ϵ sens , c ) - graph satisfies the following properties: 1) the graph node number is n ; 2) the adjacency matrix
Revised: The synthetic random graph G sampled from ( n, ρ d , ϵ sens , c ) - graph satisfies the following properties: 1) the graph node number is n ; 2) the adjacency matrix

Original: A satisfies A ij ∈ { 0 , 1 } and E ij [ P ( A ij = 1)]
Revised: A satisfies A ij ∈ { 0 , 1 } and E ij [ P ( A ij = 1)]

Original: = ρ d ; 3) given connected node pair A ij = 1 , the probability of connected nodes with the same sensitive attribute satisfies P ( s i =
Revised: = ρ d ; 3) given connected node pair A ij = 1 , the probability of connected nodes with the same sensitive attribute satisfies P ( s i =

Original: s j | A ij = 1) =
Revised: s j | A ij = 1) =

Original: ϵ sens ; 4) the binary sensitive attribute s i ∈ {− 1 , 1 } satisfies E
Revised: ϵ sens ; 4) the binary sensitive attribute s i ∈ {− 1 , 1 } satisfies E

Original: i [ P ( s i = 1)]
Revised: i [ P ( s i = 1)]

Original: = c ; 5) independent edge generation.
Revised: = c ; 5) independent edge generation.

Original: Node Features.
Revised: Node Features.

Original: We assume that node attributes in synthetic graph follow Gaussian Mixture Model
Revised: We assume that node attributes in synthetic graph follow Gaussian Mixture Model

Original: GMM ( c, µ 1 , Σ 1 , µ 2 , Σ 2 ) .
Revised: GMM ( c, µ 1 , Σ 1 , µ 2 , Σ 2 ) .

Original: For  node with sensitive attribute s i = − 1 ( s i = 1 ), the node attributes
Revised: For the node with sensitive attribute s i = − 1 ( s i = 1 ), the node attributes

Original: X i follows Gaussian distribution P 1 = N ( µ 1 , Σ 1 ) ( P 2 = N ( µ 2 , Σ 2 ) ), where the node attributes with the same sensitive attribute are independent and identically distributed, and µ i , Σ
Revised: X i follow Gaussian distribution P 1 = N ( µ 1 , Σ 1 ) ( P 2 = N ( µ 2 , Σ 2 ) ), where the node attributes with the same sensitive attribute are independent and identically distributed, and µ i , Σ

Original: i ( i = 1 , 2 ) represent the mean vector and covariance matrix.
Revised: i ( i = 1 , 2 ) represent the mean vector and covariance matrix.

Original: 3. R EPRESENTATION B IAS M EASUREMENT
Revised: 4. R EPRESENTATION B IAS M EASUREMENT

Original: To measure the node representations bias, we adopt the mutual information between sensitive attribute and node attributes I ( s , X ) .
Revised: To measure the node representations bias, we adopt the mutual information between sensitive attribute and node attributes I ( s , X ) .

Original: Note that the exact mutual information I ( s , X ) is intractable to estimate, an upper bound on the exact mutual information is developed as a surrogate metric in the following
Revised: Note that the exact mutual information I ( s , X ) is intractable to estimate, an upper bound on the exact mutual information is developed as a surrogate metric in the following

Original: Theorem 1:
Revised: Theorem 1:

Original: Theorem 1 Suppose the synthetic graph node attribute X is generated based on Gaussian Mixture
Revised: Theorem 1 Suppose the synthetic graph node attribute X is generated based on Gaussian Mixture

Original: Model GMM ( c, µ 1 , Σ 1 , µ 2 , Σ 2 ) , i.e., the probability density function of node attributes for the nodes of different sensitive attribute s = {− 1 , 1 } follows f X
Revised: Model GMM ( c, µ 1 , Σ 1 , µ 2 , Σ 2 ) , i.e., the probability density function of node attributes for the nodes of different sensitive attribute s = {− 1 , 1 } follows f X

Original: ( X i = x
Revised: ( X i = x

Original: | s i = − 1) ∼ N ( µ 1 , Σ 1 ) △ =
Revised: | s i = − 1) ∼ N ( µ 1 , Σ 1 ) △ =

Original: P 1 and f X ( X i = x
Revised: P 1 and f X ( X i = x

Original: | s i = 1) ∼ N ( µ 2 , Σ 2 ) △ =
Revised: | s i = 1) ∼ N ( µ 2 , Σ 2 ) △ =

Original: P 2 , and the sensitive attribute ratio satisfies E
Revised: P 2 , and the sensitive attribute ratio satisfies E

Original: i
Revised: i

Original: [ P ( s i = 1)]
Revised: [ P ( s i = 1)]

Original: = c , then the mutual information between sensitive attribute and node attributes I ( s , X ) satisfies
Revised: = c , then the mutual information between sensitive attribute and node attributes I ( s , X ) satisfies

Original: [Equation]
Revised: [Equation]

Original: Based on Theorem 1, we can observe that lower distribution distance D KL ( P 1 || P 2 ) or D KL ( P 2 || P 1 ) is beneficial for reducing Bias ( s , X ) and I ( s , X ) since the sensitive attribute is less distinguishable based on node representations.
Revised: Based on Theorem 1, we can observe that lower distribution distance D KL ( P 1 || P 2 ) or D KL ( P 2 || P 1 ) is beneficial for reducing Bias ( s , X ) and I ( s , X ) since the sensitive attribute is less distinguishable based on node representations.

Original: 3. W HEN A GGREGATION ENHANCES THE BIAS ?
Revised: 4. W HEN A GGREGATION ENHANCES THE BIAS ?

Original: We focus on the role of message passing in terms of fairness.
Revised: We focus on the role of message passing in terms of fairness.

Original: Suppose the graph adjacency matrix A is sampled for ( n, ρ d , ϵ sens , c ) -graph
Revised: Suppose the graph adjacency matrix A is sampled for ( n, ρ d , ϵ sens , c ) -graph

Original: and we adopt the GCN-like message passing ˜ X =
Revised: and we adopt the GCN-like message passing ˜ X =

Original: ˜
Revised: ˜

Original: AX , where ˜ A is normalized adjacency matrix with self-loop.
Revised: AX , where ˜ A is normalized adjacency matrix with self-loop.

Original: We define the bias difference for such message passing as ∆ Bias = Bias ( s , ˜ X ) − Bias ( s , X ) to measure the role of graph topology.
Revised: We define the bias difference for such message passing as ∆ Bias = Bias ( s , ˜ X ) − Bias ( s , X ) to measure the role of graph topology.

Original: Subsequently, we derive intra-connect and inter-connect probability in Lemma 1.
Revised: Subsequently, we derive intra-connect and inter-connect probability in Lemma 1.

Original: Lemma 1 Suppose the synthetic graph is generated from ( n, ρ d , ϵ sens , c ) -graph, then we obtain the intra-connect and inter-connect probability as follows:
Revised: Lemma 1 Suppose the synthetic graph is generated from ( n, ρ d , ϵ sens , c ) -graph, then we obtain the intra-connect and inter-connect probability as follows:

Original: [Equation]
Revised: [Equation]

Original: Subsequently, we provide a sufficient condition to specify the case that graph topology enhances bias in Theorem 2.
Revised: Subsequently, we provide a sufficient condition to specify the case that graph topology enhances bias in Theorem 2.

Original: Theorem 2 Suppose the synthetic graph node attribute X is generated based on Gaussian Mixture
Revised: Theorem 2 Suppose the synthetic graph node attribute X is generated based on Gaussian Mixture

Original: Model GMM ( c, µ 1 , Σ , µ 2 , Σ ) , and the graph adjacency matrix A is generated from ( n, ρ d , ϵ sens , c ) - graph.
Revised: Model GMM ( c, µ 1 , Σ , µ 2 , Σ ) , and the graph adjacency matrix A is generated from ( n, ρ d , ϵ sens , c ) - graph.

Original: If adopting GCN-like message passing ˜ X = ˜ AX , bias will be enhanced, i.e., ∆ Bias > 0 if the bias-enhance condition holds: ( ν 1 − ν 2 ) 2 min { ζ 1 , ζ 2 } > 1 , where ν 1 − ν 2 < 1 represents the reduction coefficient of the distance between the mean node attributes of the two sensitive attribute group, ζ 1 , ζ 2 mean the connection degree of two sensitive group; the mathematical formulation is given by
Revised: If adopting GCN-like message passing ˜ X = ˜ AX , bias will be enhanced, i.e., ∆ Bias > 0 if the bias-enhance condition holds: ( ν 1 − ν 2 ) 2 min { ζ 1 , ζ 2 } > 1 , where ν 1 − ν 2 < 1 represents the reduction coefficient of the distance between the mean node attributes of the two sensitive attributes groups, ζ 1 , ζ 2 mean the connection degree of two sensitive groups; the mathematical formulation is given by

Original: [Equation]
Revised: [Equation]

Original: where  the node number with the same sensitive attribute n − 1 = n (1 − c ) , n 1 = nc , intra-connect probability p conn =
Revised: where ζ 1 = n − 1 q conn + ( n 1 − 1) p conn + 1 , ζ 2 = n − 1 p conn + ( n 1 − 1) q conn + 1 , the node number with the same sensitive attribute n − 1 = n (1 − c ) , n 1 = nc , intra-connect probability p conn =

Original: E ij [ P ( A ij | s i =
Revised: E ij [ P ( A ij | s i =

Original: s j )] , and inter-connect probability q conn =
Revised: s j )] , and inter-connect probability q conn =

Original: E ij [ P ( A ij | s i s j = − 1)] .
Revised: E ij [ P ( A ij | s i s j = − 1)] .

Original: Based on Theorem 2, we can provide more discussion on the influence of 4 graph data in-formation of graph:the number of nodes n , the edge density ρ d , sensitive homophily coefficient ϵ sens , and sensitive group ratio c for bias enhancement as follows:
Revised: Based on Theorem 2, we can provide more discussion on the influence of 4 graph data in-formation of graph:the number of nodes n , the edge density ρ d , sensitive homophily coefficient ϵ sens , and sensitive group ratio c for bias enhancement as follows:

Original: • Node representation bias is enhanced by message passing for sufficient large sensitive
Revised: • Node representation bias is enhanced by message passing for sufficient large sensitive

Original: homophily ϵ sens .
Revised: homophily ϵ sens .

Original: According to Lemma 1, for large sensitive homophily coefficient ϵ sens → 1 , the inter-connect probability q conn → 0 and intra-connect probability p conn keeps the maximal value.
Revised: According to Lemma 1, for large sensitive homophily coefficient ϵ sens → 1 , the inter-connect probability q conn → 0 and intra-connect probability p conn keeps the maximal value.

Original: In this case, based on Theorem 2, it is easy to obtain that ν 1 = 1 , ν 2 = 

Revised: In this case, based on Theorem 2, it is easy to obtain that ν 1 = 1 , ν 2 =

Original: and the distance for the mean aggregated node representation will keep the same, i.e., ˜ µ 1 − ˜ µ 2 = ( ν 1 − ν 2 )( µ 1 − µ 2 )
Revised: and the distance for the mean aggregated node representation will keep the same, i.e., ˜ µ 1 − ˜ µ 2 = ( ν 1 − ν 2 )( µ 1 − µ 2 )

Original: =
Revised: =

Original: µ 1 − µ 2 .
Revised: µ 1 − µ 2 .

Original: Additionally, the covariance will be diminished after aggregation since ζ 1 and ζ 2 are strictly larger than 1 .
Revised: Additionally, the covariance will be diminished after aggregation since ζ 1 and ζ 2 are strictly larger than 1 .

Original: Therefore, for sufficient large sensitive homophily coefficient ϵ sens , the bias-enhance condition ( ν 1 − ν 2 ) 2 min { ζ 1 , ζ 2 } > 1 holds.
Revised: Therefore, for sufficient large sensitive homophily coefficient ϵ sens , the bias-enhance condition ( ν 1 − ν 2 ) 2 min { ζ 1 , ζ 2 } > 1 holds.

Original: • The bias enhancement implicitly depends on node representation geometric differentiation, including the distance between the mean node representation within the same sensitive attribute and the scale covariance matrix.
Revised: • The bias enhancement implicitly depends on node representation geometric differentiation, including the distance between the mean node representation within the same sensitive attribute and the scale covariance matrix.

Original: Theorem 1 implies that low mean representation distance and concentrated representation (low covariance matrix) leads to fair representation.
Revised: Theorem 1 implies that low mean representation distance and concentrated representation (low covariance matrix) lead to fair representation.

Original: However, GCN-like message passing renders the mean node representation distance reduction ν 1 − ν 2 and concentrated for each sensitive attribute group, which is an “adversarial" effect for fairness and the mean distance and covariance reduction is controlled by sensitive homophily coefficient.
Revised: However, GCN-like message passing renders the mean node representation distance reduction ν 1 − ν 2 and concentrated for each sensitive attribute group, which is an “adversarial" effect for fairness and the mean distance and covariance reduction is controlled by sensitive homophily coefficient.

Original: • The bias is enlarged as node number n being increased.
Revised: • The bias is enlarged as node number n being increased.

Original: For large node number n , the mean distance almostly keeps constant since ν 1 ≈
Revised: For large node number n , the mean distance almost keeps constant since ν 1 ≈

Original: [Equation]
Revised: [Equation]

Original:  and
Revised: , and

Revised: ζ

Original: [Equation]
Revised: [Equation]

Revised: , ζ

Original: [Equation]
Revised: [Equation]

Original: are almost proportional to node number n .
Revised: are almost proportional to node number n .

Original: Therefore, the bias-enhancement condition can be more easily satisfied and ∆ Bias would be higher for large graph data.
Revised: Therefore, the bias-enhancement condition can be more easily satisfied and ∆ Bias would be higher for large graph data.

Original: The intuition is that, given graph density ratio, large graph data represents  higher average degree node.
Revised: The intuition is that, given graph density ratio, large graph data represents a higher average degree node.

Original: Hence, each aggregated node representation adopt more neighbor’s information with the same sensitive attribute, and thus leads to  lower covariance matrix value and higher representation bias.
Revised: Hence, each aggregated node representation adopts more neighbor’s information with the same sensitive attribute, and thus leads to a lower covariance matrix value and higher representation bias.

Original: • The bias is enlarged as graph connection density ρ being increased.
Revised: • The bias is enlarged as graph connection density ρ being increased.

Original: Based on Lemma 1, interconnection probability and inter-connection probability are both proportional to graph connection density ρ d .
Revised: Based on Lemma 1, interconnection probability and inter-connection probability are both proportional to graph connection density ρ d .

Original: Therefore, ν 1 and ν 2 almost keep constant and the distance of mean node representation is constant as well.
Revised: Therefore, ν 1 and ν 2 almost keep constant and the distance of mean node representation is constant as well.

Original: As for  covariance matrix, message passing leads to more concentrated node representation since ζ 1 and ζ 2 are larger for higher graph connection density ρ d .
Revised: As for the covariance matrix, message passing leads to more concentrated node representation since ζ 1 and ζ 2 are larger for higher graph connection density ρ d .

Original: The rationale is similar with the graph node number: given node number, higher graph connection density ρ d means higher average node degree and each aggregated node representation adopt more neighbor’s information with the same sensitive attribute.
Revised: The rationale is similar to the graph node number: given node number, higher graph connection density ρ d means higher average node degree and each aggregated node representation adopts more neighbor’s information with the same sensitive attribute.

Original: • When the sensitive attribute is more balanced (i.e.,) the bias will be enlarged.
Revised: • When the sensitive attribute is more balanced (i.e.,) the bias will be enlarged.

Original: Based on Lemma 1, it is easy to obtain that, given graph connection density ρ d and graph node number n , the intra-connection probability p conn would be high, while  low for inter-connection probability q conn , if the balanced sensitive attribute.
Revised: Based on Lemma 1, it is easy to obtain that, given graph connection density ρ d and graph node number n , the intra-connection probability p conn would be high, while being low for inter-connection probability q conn , if the balanced sensitive attribute.

Original: In other words, intra-connection probability p conn (interconnection probability q conn ) monotonically decreases (increases) with respect to | c − 12 | .
Revised: In other words, intra-connection probability p conn (inter-connection probability q conn ) monotonically decreases (increases) with respect to | c − 12 | .

Original:  L EARNING F AIR G RAPH T OPOLOGY
Revised:  L EARNING F AIR G RAPH T OPOLOGY

Original: The above section provides a data-centric perspective (graph topology) to understanding why message passing many enhance representation bias.
Revised: The above section provides a data-centric perspective (graph topology) to understanding why message passing may enhance representation bias.

Original: In other words, graph topology refinement may assist GNNs model in achieving fair and accurate prediction.
Revised: In other words, graph topology refinement may assist GNNs model in achieving fair and accurate prediction.

Original: However, searching for the optimal graph topology for GNNs is a non-trivial problem due to large-scale discrete optimization.
Revised: However, searching for the optimal graph topology for GNNs is a non-trivial problem due to large-scale discrete optimization.

Original: In this section, motivated by the theoretical analysis, we propose Fair G raph R efinement method, named FairGR , to achieve fair prediction via learning the optimal graph topology.
Revised: In this section, motivated by the theoretical analysis, we propose Fair G raph R efinement method, named FairGR , to achieve fair prediction via learning the optimal graph topology.

Original: Specifically, we formulate the objective functions as three parts, including low sensitive homophily coefficient, high label homophily coefficient, and small topology perturbation.
Revised: Specifically, we formulate the objective functions into three parts, including low sensitive homophily coefficient, high label homophily coefficient, and small topology perturbation.

Original: In this way, FairGR can explicitly reduce the topology bias while preserving useful topology information for prediction.
Revised: In this way, FairGR can explicitly reduce the topology bias while preserving useful topology information for prediction.

Original: Problem Formulation.
Revised: Problem Formulation.

Original: Considering binary sensitive attribute s i ∈ {− 1 , 1 } and binary classification task with label y i ∈ {− 1 , 1 } for i -th node, we aim to modify the graph topology to achieve low sensitive
Revised: Considering binary sensitive attribute s i ∈ {− 1 , 1 } and binary classification task with label y i ∈ {− 1 , 1 } for i -th node, we aim to modify the graph topology to achieve low sensitive

Original: homophily coefficient ϵ sens , high label homophily coefficient ϵ label , and small topology perturbation.
Revised: homophily coefficient ϵ sens , high label homophily coefficient ϵ label , and small topology perturbation.

Original: For sensitive homophily coefficient with the binary sensitive attribute, the determination of any two nodes with the same sensitive attribute can be obtained via H ( ss T ) , where H ( · ) is
Revised: For sensitive homophily coefficient with the binary sensitive attribute, the determination of any two nodes with the same sensitive attribute can be obtained via H ( ss T ) , where H ( · ) is

Original: Heaviside (unit) step function, sensitive attribute vector s ∈ {− 1 , 1 } n × 1 .
Revised: Heaviside (unit) step function, sensitive attribute vector s ∈ {− 1 , 1 } n × 1 .

Original: Based on Definition 1,
Revised: Based on Definition 1,

Original: It is easy to rewrite sensitive homophily coefficient as ϵ sens =
Revised: It is easy to rewrite sensitive homophily coefficient as ϵ sens =

Original: || H ( ss T ) ⊙
Revised: || H ( ss T ) ⊙

Original: A
Revised: A

Original: || ||
Revised: || ||

Original: A ||, where ⊙ represents
Revised: A ||, where ⊙ represents

Original: Hadamard product, and || · || 1 denotes the entry-wise l 1 norms (i.e., the summation over all absolute value of elements).
Revised: Hadamard product, and || · || 1 denotes the entry-wise l 1 norms (i.e., the summation over all absolute value of elements).

Original: Similarly, label homophily coefficient as ϵ label =
Revised: Similarly, label homophily coefficient as ϵ label =

Original: || H ( yy T ) ⊙
Revised: || H ( yy T ) ⊙

Original: A
Revised: A

Original: || ||
Revised: || ||

Original: A ||.
Revised: A ||.

Original: As for graph topology perturbation, we can use the entry-wise l 1 norms of the difference as the measurement.
Revised: As for graph topology perturbation, we can use the entry-wise l 1 norms of the difference as the measurement.

Original: In a nutshell, the objective function to reconstruct graph connections given sensitive attribute s , label y , and graph topology A can be formulated as follows:
Revised: In a nutshell, the objective function to reconstruct graph connections given sensitive attribute s , label y , and graph topology A can be formulated as:

Original: [Equation]
Revised: [Equation]

Revised: Therefore, the rewired graph topology can be obtained via a constrained optimization problem min ˆ A L ( ˆ A | s , y , A ) s.t.

Original:  where α and β are the hyperparameters for label homophily coefficient and graph topology perturbation.
Revised: ˆ A ij ∈ { 0 , 1 } , where α and β are the hyperparameters for label homophily coefficient and graph topology perturbation.

Original: Considering the formulated problem is a large-scale discrete optimization problem, we employ a heuristic optimization method to obtain the modified graph topology.
Revised: Considering the formulated problem is a large-scale discrete optimization problem, we employ a heuristic optimization method to obtain the modified graph topology.

Original: Optimization Strategy.
Revised: Optimization Strategy.

Original: To optimize the formulated problem with constraint, we adopt Proximal
Revised: To optimize the formulated problem with constraint, we adopt Proximal

Original: Gradient Descent (PGD).
Revised: Gradient Descent (PGD).

Original: Specifically, we first adopt gradient descent to update the graph topology using gradient ∂ L (ˆ A | s , y , A ) ∂ ˆ A .
Revised: Specifically, we first adopt gradient descent to update the graph topology using gradient ∂ L (ˆ A | s , y , A ) ∂ ˆ A .

Original: Subsequently, we clip the graph topology ˆ A within { 0 , 1 } in the projection operation of PGD.
Revised: Subsequently, we clip the graph topology ˆ A within { 0 , 1 } in the projection operation of PGD.

Original: Such operation will conduct multiple times to obtain the final graph topology.
Revised: Such operation will conduct multiple times to obtain the final graph topology.

Original: In practice, considering only the sensitive attribute and label for the training set are available, we actually only modify the graph topology within the training nodes.
Revised: In practice, considering only the sensitive attribute and label for the training set are available, we actually only modify the graph topology within the training nodes.

Original: In other words, the three objectives, including sensitive homophily coefficient, label homophily coefficient, and graph topology perturbation, are calculated for the subgraph of training nodes in the training.
Revised: In other words, the three objectives, including sensitive homophily coefficient, label homophily coefficient, and graph topology perturbation, are calculated for the subgraph of training nodes.

Original: In this way, we can avoid information leakage from the test set and reduce the complexity of the optimization problem.
Revised: In this way, we can avoid information leakage from the test set and reduce the complexity of the optimization problem.

Original: [Table]
Revised: Evaluation and Computation Complexity Analysis.

Original: E XPERIMENTS
Revised: For algorithmic evaluation of pre-processing FairGR, we compared the prediction performance (including accuracy and fairness) using original graph topology and rewired graph topology across multiple GNN backbones. Denote the number of training nodes and update iterations as N and T , respectively.

Original: In this section
Revised: Then the computation complexity for gradient computation and projection of PGD are both O ( n 2 train ) . The total computation complexity to obtain the final rewired graph topology is given by O ( Tn 2 train ) . The memory consumption is O ( n 2 train ) due to the storage of graph topology gradient.

Original: [Table]
Revised: [Table]

Original:  we conduct experiments to validate the effectiveness (see Apppendix G for more details.) of  proposed FairGR.
Revised: In this section, we conduct experiments to validate the effectiveness (see Apppendix G for more details.) of the proposed FairGR.

Original: We firstly validate that GCN-like aggregation enhances representation bias for the graph data with large sensitive homophily via synthetic experiments.
Revised: We firstly validate that GCN-like aggregation enhances representation bias for the graph data with large sensitive homophily via synthetic experiments.

Original: For real-world datasets, we conduct experiments to show that the prediction bias of GNN is larger than that of MLP.
Revised: For real-world datasets, we conduct experiments to show that the prediction bias of GNN is larger than that of MLP.

Original: Moreover, we introduce the experimental settings and then evaluate our proposed FairGR compared with several baselines in terms of prediction performance and fairness metrics on real-world datasets.
Revised: Moreover, we introduce the experimental settings and then evaluate our proposed FairGR compared with several baselines in terms of prediction performance and fairness metrics on real-world datasets.

Original: 5. S YNTHETIC E XPERIMENTS
Revised: 6. S YNTHETIC E XPERIMENTS

Revised: In the synthetic experiments, we demonstrate the relation between DP difference across GCN-like message passing operation and sensitive homophily coefficient 2 .

Original: We investigate the influence of graph node number n , graph connection density ρ d , sensitive homophily ϵ sens , and sensitive attribute ration c for bias enhancement of GCN-like message passing.
Revised: Specifically, we investigate the influence of graph node number n , graph connection density ρ d , sensitive homophily ϵ sens , and sensitive attribute ration c for bias enhancement of GCN-like message passing.

Original: For evaluation metric, we adopt the demographic parity (DP) difference during message passing to measure the bias enhancement.
Revised: For evaluation metric, we adopt the demographic parity (DP) difference during message passing to measure the bias enhancement.

Original: For node attribute generation , we first generate node attribute with Gaussian distribution N ( µ 1 , Σ ) and N ( µ 2 , Σ ) for node with binary sensitive attribute, respectively, where µ 1 = [0 , 1] , µ 1 = [1 , 0] and Σ = (cid:20) 10(cid:21) .
Revised: For node attribute generation , we first generate node attribute with Gaussian distribution N ( µ 1 , Σ ) and N ( µ 2 , Σ ) for node with binary sensitive attribute, respectively, where µ 1 = [0 , 1] , µ 1 = [1 , 0] and Σ = (cid:20) 10(cid:21) .

Original: For adjacency matrix generation , we randomly generate edges via  stochastic block model based on the intra-connection and inter-connection probability.
Revised: For adjacency matrix generation , we randomly generate edges via a stochastic block model based on the intra-connection and inter-connection probability.

Original: Figure 2 shows the DP difference during message passing with respect to  sensitive homophily coefficient.
Revised: Figure 2 shows the DP difference during message passing with respect to the sensitive homophily coefficient.

Original: We observe that a higher sensitive homophily coefficient generally leads to larger bias enhancement.
Revised: We observe that a higher sensitive homophily coefficient generally leads to larger bias enhancement.

Original: Additionally, higher graph connection density ρ d , larger node number n , and balanced sensitive attribute ratio c correspond  higher bias enhancement, which is consistent with our theoretical analysis in Theorem 2.
Revised: Additionally, higher graph connection density ρ d , larger node number n , and balanced sensitive attribute ratio c correspond to higher bias enhancement, which is consistent with our theoretical analysis in Theorem 2.

Original: 5. E XPERIMENTS ON R EAL -W ORLD D ATASETS 5.2. E XPERIMENTAL S ETTINGS
Revised: 6. E XPERIMENTS ON R EAL -W ORLD D ATASETS 6.2. E XPERIMENTAL S ETTINGS

Original: Datasets.
Revised: Datasets.

Original: The experiments are conducted on three real-world datasets, including Pokec-z, Pokec-n, and NBA (Dai & Wang, 2021).
Revised: The experiments are conducted on three real-world datasets, including Pokec-z, Pokec-n, and NBA (Dai & Wang, 2021).

Original: Pokec-z and Pokec-n are sampled from a larger social network Pokec (Takac & Zabovsky, 2012) based on  province in Slovakia.
Revised: Pokec-z and Pokec-n are sampled from a larger social network Pokec (Takac & Zabovsky, 2012) based on the province in Slovakia.

Original: We choose region information and the working field of the users as the sensitive attribute and the predicted label, respectively.
Revised: We choose region information and the working field of the users as the sensitive attribute and the predicted label, respectively.

Original: NBA dataset includes around 400 NBA basketball players and is collected from a Kaggle dataset 1 and Twitter.
Revised: NBA dataset includes around 400 NBA basketball players and is collected from a Kaggle dataset 3 and Twitter.

Original: The information of players includes age, nationality, and salary in the 2016-2017 season.
Revised: The information of players includes age, nationality, and salary in the 2016-2017 season.

Original: We choose nationality (U.S. and overseas player) as the binary sensitive attribute, and the prediction label is whether the salary is higher than the median.
Revised: We choose nationality (U.S. and overseas player) as the binary sensitive attribute, and the prediction label is whether the salary is higher than the median.

Original: Evaluation Metrics.
Revised: Evaluation Metrics.

Original: For the node classification task, we adopt accuracy to evaluate the classification performance.
Revised: For the node classification task, we adopt accuracy to evaluate the classification performance.

Original: As for fairness metric, we adopt two most common-used group fairness metrics, including demographic parity and equal opportunity , to measure the prediction bias (Louizos et al., 2015; Beutel et al., 2017).
Revised: As for fairness metric, we adopt two most common-used group fairness metrics, including demographic parity and equal opportunity , to measure the prediction bias (Louizos et al., 2015; Beutel et al., 2017).

Original: Specifically, demographic parity is defined as the average prediction difference over different sensitive attribute group, i.e., ∆ DP = | P (ˆ y = 1 | s = − 1) − P
Revised: Specifically, demographic parity is defined as the average prediction difference over different sensitive attribute groups, i.e., ∆ DP = | P (ˆ y = 1 | s = − 1) − P

Original: (ˆ y = 

Revised: (ˆ y =

Original: | s = 1) | .
Revised: | s = 1) | .

Original: Similarly, equal opportunity is given by ∆ DP = | P (ˆ y = 1 | s = − 1 , y = 1) − P (ˆ y = 1 | s = 1 , y = 1) | , where y and ˆ y represent the ground-truth label and predicted label, respectively.[Figure]
Revised: Similarly, equal opportunity is given by ∆ EO = | P (ˆ y = 1 | s = − 1 , y = 1) − P (ˆ y = 1 | s = 1 , y = 1) | , where y and ˆ y represent the ground-truth label and predicted label, respectively.

Original: Baselines.
Revised: Baselines.

Original: Considering  the proposed FairGR is a pre-processing method, we show that our proposed FairGR can improve many representative GNNs, such as GCN (Kipf & Welling, 2017), GAT (Veliˇckovi´c et al., 2018), SGC (Wu et al., 2019), in many fairness training strategies, such adding regularization, adversarial debiasing.
Revised: Considering that the proposed FairGR is a pre-processing method, we show that our proposed FairGR can improve many representative GNNs, such as GCN (Kipf & Welling, 2017), GAT (Veliˇckovi´c et al., 2018), SGC (Wu et al., 2019), in many fairness training strategies, such adding regularization, adversarial debiasing.

Original: For all models, we train 2 layers  neural networks with 64 hidden units for 300 epochs.
Revised: For all models, we train 2 layers of neural networks with 64 hidden units for 300 epochs.

Original: Implementation Details.
Revised: Implementation Details.

Original: For each experiment, we run 5 times and report the average performance for each method.
Revised: For each experiment, we run 5 times and report the average performance for each method.

Original: We adopt Adam optimizer with 0 .
Revised: We adopt Adam optimizer with 0 .

Original: 001 learning rate and 1 e − 5 weight decay for all models training.
Revised: 001 learning rate and 1 e − 5 weight decay for all models training.

Original: In adversarial debiasing setting, we train the classifier and adversary head with 70 and 30 epochs, respectively.
Revised: In adversarial debiasing setting, we train the classifier and adversary head with 70 and 30 epochs, respectively.

Original: The hyperparameters for adversary debiasing is tuned in { 0 . 0 , 0 . 5 , 1 . 0 , 2 . 0 , 5 . 0 , 8 . 0 , 10 . 0 , 50 . 0 , 100 . 0 } .
Revised: The hyperparameters for adversary debiasing are tuned in { 0 . 0 , 0 . 5 , 1 . 0 , 2 . 0 , 5 . 0 , 8 . 0 , 10 . 0 , 50 . 0 , 100 . 0 } .

Original: For adding regularization, we adopt the hyperparameter set { 0 . 0 , 1 . 0 , 1 . 5 , 2 . 0 , 5 . 0 , 8 . 0 , 10 . 0 , 15 . 0 , 25 . 0 , 50 . 0 , 80 . 0 , 100 . 0 } .
Revised: For adding regularization, we adopt the hyperparameter set { 0 . 0 , 1 . 0 , 1 . 5 , 2 . 0 , 5 . 0 , 8 . 0 , 10 . 0 , 15 . 0 , 25 . 0 , 50 . 0 , 80 . 0 , 100 . 0 } .

Original:  D OES GNN S HAVE A LARGER PREDICTION BIAS THAN MLP?
Revised: 6.2. D OES GNN S HAVE A LARGER PREDICTION BIAS THAN MLP?

Original: To validate the effect of bias enhancement of GNNs, we compare the performance of many representative GNNs over MLP on various real-world datasets, and summarize the results in Table 1 and Figure 6.
Revised: To validate the effect of bias enhancement of GNNs, we compare the performance of many representative GNNs over MLP on various real-world datasets and summarize the results in Table 1.

Original: From these results, we make the following observations:
Revised: From these results, we make the following observations:

Original: • Many representative GNNs have a higher prediction bias compared with MLP model on all three datasets in terms of demographic parity and equal opportunities. 
Revised: • Many representative GNNs have a higher prediction bias compared with MLP model on all three datasets in terms of demographic parity and equal opportunities. For demographic parity, the https:/ www

Revised: com

Original: For demographic parity, the prediction bias of MLP is lower than that of GAT, GCN, and SGC by 32 .
Revised: noahgift -nba [Figure] prediction bias of MLP is lower than that of GAT, GCN, and SGC by 32 .

Original: 53% and 58 .
Revised: 53% and 58 .

Original: 72% on Pokec-z dataset.
Revised: 72% on Pokec-z dataset.

Original: The higher prediction bias comes from the aggregation within the-same-sensitive-attribute nodes and topology bias in graph data.
Revised: The higher prediction bias comes from the aggregation within the same-sensitive attribute nodes and topology bias in graph data.

Original: • For the training behavior, GNNs and MLP models both convergence in terms of accuracy for high label homophily dataset Pokec-n and Pokec-z dataset.
Revised: • FairGR can mitigate bias for GCN and SGC backbone via rewiring graph topology in these three datasets.

Original: For high sensitive homophily dataset Pokec-n and Pokec-z dataset, GNNs demonstrate higher prediction than that of MLP, while theprediction bias difference is relatively small for the low sensitive homophily dataset NBA.
Revised: For GAT backbone, although the bias can be mitigated, the accuracy drop is significant due to the fact that GAT is more sensitive to graph topology rewire.

Original: 5.2. D OES F AIR GR ACHIEVE BETTER TRADEOFF PERFORMANCE IN VARIOUS SETTINGS ?
Revised: 6.2. D OES F AIR GR ACHIEVE BETTER TRADEOFF PERFORMANCE IN VARIOUS SETTINGS ?

Original: Comparison with adversarial debiasing and regularization.
Revised: Comparison with adversarial debiasing and regularization.

Original: To validate that the proposed
Revised: To validate that the proposed

Original: FairGR is compatible with many fairness training strategies, we also show the prediction performance and fairness metric trade-off compared with adversarial debiasing (Fisher et al., 2020) and add demographic parity regularization (Chuang & Mroueh, 2020).
Revised: FairGR is compatible with many fairness training strategies, we also show the prediction performance and fairness metric trade-off compared with adversarial debiasing (Fisher et al., 2020) and add demographic parity regularization (Chuang & Mroueh, 2020).

Original: In adversarial debiasing (Louppe et al., 2017), the output of GNNs is the input of the adversary, and the goal of the adversary is to predict the node sensitive attribute.
Revised: In adversarial debiasing (Louppe et al., 2017), the output of GNNs is the input of the adversary, and the goal of the adversary is to predict the node sensitive attribute.

Original: For these two fair training strategies, we adopt GCN, GAT and SGC as backbones.
Revised: For these two fair training strategies, we adopt GCN, GAT, and SGC as backbones.

Original: We randomly split 50% / 25% / 25% for training, validation, and test dataset.
Revised: We randomly split 50% / 25% / 25% for training, validation, and test dataset.

Original: Figureshows the Pareto optimality curve for all methods, where  right-bottom corner point represents the ideal performance ( 100% accuracy and 0% prediction bias).
Revised: Figureshows the Pareto optimality curve for all methods, where the right-bottom corner point represents the ideal performance ( 100% accuracy and 0% prediction bias).

Original: From the results, we list the following observations as follows:
Revised: From the results, we list the following observations as follows:

Original: [Figure] • For both adversarial debiasing and adding regularization training strategies, our proposed FairGR can achieve a better DP-Acc trade-off compared with that without any graph data refinement for many GNNs.
Revised: • For both adversarial debiasing and adding regularization training strategies, our proposed FairGR can achieve a better DP-Acc trade-off compared with that without any graph data refinement for many GNNs.

Original: In other words, FairGR can effectively reduce training bias and is compatible with many existing fairness training strategies.
Revised: In other words, FairGR can effectively reduce training bias and is compatible with many existing fairness training strategies.

Original: • Topology does matter in GNNs.For
Revised: • Topology does matter in GNNs.

Original: adding regularization

Original: or

Original:  adversarial debiasing , FairGS embrace different tradeoff performance gain on top of different 
Revised: For adding regularization or adversarial debiasing, FairGS embrace different tradeoff performance gain on top of different GNNs.

Original: GNNs

Original: Such observation implies that there are complicated interaction between graph topology and 
Revised: Such observation implies that there is a complicated interaction between graph topology and message passing algorithms.

Original: message passing algorithms.

Original: Additionally, FairGS provide the most tradeoff performance benefit in GAT compared with GCN and SGC
Revised: Additionally, FairGS provide the most tradeoff performance benefit in GAT compared with GCN and SGC.

Original: The high capacity of GAT may energize the message passing algorithm to learn from data.
Revised: The high capacity of GAT may energize the message passing algorithm to learn from data.

Original: Therefore, the tradeoff performance improvement is the highest in adding regularization and adversarial debiasing.
Revised: Therefore, the tradeoff performance improvement is the highest in adding regularization and adversarial debiasing.

Original: R ELATED W ORK

Original: We briefly review the existing work on graph neural networks and fairness-aware learning on graphs.

Original: (Please refer to Appendix D for a more comprehensive discussion.).

Original: Existing GNNs can be roughly divided into spectral-based and spatial-based GNNs.

Original: Spectral-based GNNs provide graph convolution definition based on graph theory (Bruna et al., 2013; Defferrard et al., 2016; Henaff et al., 2015).

Original: Spatial-based GNNs variant are popular due to explicit neighbors’ information aggregation, including Graph convolutional networks (GCN) (Kipf & Welling, 2017), graph attention network (GAT) (Veliˇckovi´c et al., 2018).

Original: As for fairness in graph data, many works have been developed to achieve fairness in machine learning community (Chuang & Mroueh, 2020), including fair walk (Rahman et al., 2019), adversarial debiasin (Dai & Wang, 2021), Bayesian approach (Buyl & De Bie, 2020), and contrastive learning (Agarwal et al., 2021).

Original: Many literature empirically show that GNN or message passing may enhance prediction bias compared with MLP.

Original: However, the theoretical understanding of why such a phenomenon happens is still unclear.

Original: In this work, we take an initial step toward theoretically understanding why message passing enhances bias from a data perspective.

Original: Based on the understanding, we develop a simple yet effective fair graph refinement method to achieve better tradeoff performance.

Original: More importantly, the proposed FairGR is compatible with many fair training strategies.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this work, we theoretically demonstrate that the message passing amplifies node representation bias under the graph data with a large sensitive homophily coefficient, and reveal the role of other graph statistical information in terms of bias amplification.
Revised: In this work, we theoretically demonstrate that the message passing amplifies node representation bias under the graph data with a large sensitive homophily coefficient, and reveal the role of other graphs statistical information in terms of bias amplification.

Original: Additionally, motivated by theoretical understanding, we develop a simple yet effective graph refinement method, named FairGR, to reduce the sensitive homophily while preserving useful information.
Revised: Additionally, motivated by theoretical understanding, we develop a simple yet effective graph refinement method, named FairGR, to reduce the sensitive homophily while preserving useful information.

Original: We conduct synthetic experiments to validate theoretical findings.
Revised: We conduct synthetic experiments to validate theoretical findings.

Original: Experimental results on real-world datasets demonstrate the effectiveness of FairGR in many fair training strategies and GNNs backbones in node classification task.
Revised: Experimental results on real-world datasets demonstrate the effectiveness of FairGR in many fair training strategies and GNNs backbones in node classification tasks.

Original: A P ROOF OF T HEOREM
Revised: A P ROOF OF T HEOREM

Original: We provide a more general proof for categorical sensitive attribute s ∈ { 1 , 2 , · · · , K } and the prior probability is given by P ( s = i ) =
Revised: We provide a more general proof for categorical sensitive attribute s ∈ { 1 , 2 , · · · , K } and the prior probability is given by P ( s = i ) =

Original: c i .
Revised: c i .

Original: Suppose the conditional node attribute x distribution given node sensitive attribute s =
Revised: Suppose the conditional node attribute x distribution given node sensitive attribute s =

Original: i satisfies normal distribution
Revised: i satisfies normal distribution

Original: P i ( x ) △ = N ( µ i , Σ i ) , the distribution of node sensitive attribute is the mixed Gaussian distribution f ( x ) =
Revised: P i ( x ) △ = N ( µ i , Σ i ) , the distribution of node sensitive attribute is the mixed Gaussian distribution f ( x ) =

Original: (cid:80)
Revised: (cid:80)

Original: Ki =1 c i P i ( x ) .
Revised: Ki =1 c i P i ( x ) .

Original: Based on the definition of mutual information, we have
Revised: Based on the definition of mutual information, we have

Original: [Equation]
Revised: [Equation]

Original: where H ( · ) represents Shannon entropy for random variable.
Revised: where H ( · ) represents Shannon entropy for random variable.

Original: Subsequently, we focus on the entropy of the mixed Gaussian distribution H ( x ) .
Revised: Subsequently, we focus on the entropy of the mixed Gaussian distribution H ( x ) .

Original: We show that such entropy can be upper bounded by the pairwise Kullback-Leibler (KL) divergence as follows:
Revised: We show that such entropy can be upper bounded by the pairwise Kullback-Leibler (KL) divergence as follows:

Original: [Equation]
Revised: [Equation]

Original: where KL divergence D KL ( P i || P j ) △ = (cid:82) P i ( x )
Revised: where KL divergence D KL ( P i || P j ) △ = (cid:82) P i ( x )

Original: ln P
Revised: ln P

Original: i ( x ) P j ( x ) d x and cross entropy H
Revised: i ( x ) P j ( x ) d x and cross entropy H

Original: ( P i
Revised: ( P i

Original: || P j )
Revised: || P j )

Original: = − (cid:82) P
Revised: = − (cid:82) P

Original: i ( x ) ln P j ( x )d x .
Revised: i ( x ) ln P j ( x )d x .

Original: The inequality (a) holds based on the variational lower bound on the expectation of a log-sum inequality E (cid:2)
Revised: The inequality (a) holds based on the variational lower bound on the expectation of a log-sum inequality E (cid:2)

Original: ln (cid:80)
Revised: ln (cid:80)

Original: i Z i (cid:3) ≥ ln (cid:2) (cid:80) i e E
Revised: i Z i (cid:3) ≥ ln (cid:2) (cid:80) i e E

Original: [ln Z i ] (cid:3) (Kullback, 1997), and quality (2) holds based on H ( P i || P j )
Revised: [ln Z i ] (cid:3) (Kullback, 1997), and quality (2) holds based on H ( P i || P j )

Original: =
Revised: =

Original: H ( P i )
Revised: H ( P i )

Original: + D KL ( P i
Revised: + D KL ( P i

Original: || P j ) .
Revised: || P j ) .

Original: As a special case for binary sensitive attribute, it is easy to obtain the following results:
Revised: As a special case for binary sensitive attribute, it is easy to obtain the following results:

Original: [Equation]
Revised: [Equation]

Original: B P ROOF OF T HEOREM
Revised: B P ROOF OF T HEOREM

Original: Before going deeper for our proof, we first introduce two useful lemmas on KL divergence and statistical information of graph.
Revised: Before going deeper for our proof, we first introduce two useful lemmas on KL divergence and statistical information of graph.

Original: Lemma 

Revised: Lemma

Original: For two d -dimensional Gaussian distributions P = N ( µ p , Σ p ) and Q = N ( µ q , Σ q ) , the
Revised: For two d -dimensional Gaussian distributions P = N ( µ p , Σ p ) and Q = N ( µ q , Σ q ) , the

Original: KL divergence D KL ( P || Q ) is given by
Revised: KL divergence D KL ( P || Q ) is given by

Original: [Equation]
Revised: [Equation]

Original: where ⊤ is matrix transpose operation and Tr ( · ) is trace of a square matrix.
Revised: where ⊤ is matrix transpose operation and Tr ( · ) is trace of a square matrix.

Original: Proof: Note that the probability density function of multivariate Normal distribution is given by:
Revised: Proof: Note that the probability density function of multivariate Normal distribution is given by:

Original: [Equation]
Revised: [Equation]

Original: the KL divergence between distributions P and Q can be given by
Revised: the KL divergence between distributions P and Q can be given by

Original: [Equation]
Revised: [Equation]

Original: Using the commutative property of the trace operation
Revised: Using the commutative property of the trace operation

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: As for the term I 2 , note that x − µ q = ( x − E P [ x ]) + ( E P [ x ] − µ q ) , we can obtain the following equation:
Revised: As for the term I 2 , note that x − µ q = ( x − E P [ x ]) + ( E P [ x ] − µ q ) , we can obtain the following equation:

Original: [Equation]
Revised: [Equation]

Original: Therefore, the KL divergence D KL ( P || Q ) is given by
Revised: Therefore, the KL divergence D KL ( P || Q ) is given by

Original: [Equation]
Revised: [Equation]

Original: □
Revised: □

Original: Lemma 3 Suppose the synthetic graph is generated from ( n, ρ d , ϵ sens , c ) -graph, then we obtain the intra-connect and inter-connect probability as follows:
Revised: Lemma 3 Suppose the synthetic graph is generated from ( n, ρ d , ϵ sens , c ) -graph, then we obtain the intra-connect and inter-connect probability as follows:

Original: [Equation]
Revised: [Equation]

Original: Proof: Based on Bayes’ rule, we have the intra-connect and inter-connect probability as follows
Revised: Proof: Based on Bayes’ rule, we have the intra-connect and inter-connect probability as follows

Original: [Equation]
Revised: [Equation]

Original: □
Revised: □

Original: Note that the synthetic graph is generated from ( n, ρ d , ϵ sens , c ) -graph.
Revised: Note that the synthetic graph is generated from ( n, ρ d , ϵ sens , c ) -graph.

Original: The sensitive attribute s is generated to with ratio c , i.e., the number of node sensitive attribute s = − 1 and s = 1 are n − 1 = n (1 − c ) and n 1 = nc .
Revised: The sensitive attribute s is generated to with ratio c , i.e., the number of node sensitive attribute s = − 1 and s = 1 are n − 1 = n (1 − c ) and n 1 = nc .

Original: Based on the determined sensitive attribute s , we randomly generate the edge based on parameters ρ d
Revised: Based on the determined sensitive attribute s , we randomly generate the edge based on parameters ρ d

Original: and ϵ sens
Revised: and ϵ sens

Original: and Lemma 1, i.e., the edges within and cross the same group are randomly generated based on intra-connect probability and inter-connect probability.
Revised: and Lemma 1, i.e., the edges within and cross the same group are randomly generated based on intra-connect probability and inter-connect probability.

Original: Therefore, the adjacency matrix A ij is independent on node attribute X i and X j given sensitive attributes s i and s j , i.e., A ij ⊥⊥ ( X i , X j ) | ( s i , s j ) .
Revised: Therefore, the adjacency matrix A ij is independent on node attribute X i and X j given sensitive attributes s i and s j , i.e., A ij ⊥⊥ ( X i , X j ) | ( s i , s j ) .

Original: Similarly, the different node attributes and edges are also dependent for each other given sensitive attributes, i.e., A ij ⊥⊥ A ij | ( s i , s j , s k ) and
Revised: Similarly, the different node attributes and edges are also dependent on each other given sensitive attributes, i.e., A ij ⊥⊥ A ij | ( s i , s j , s k ) and

Original: X i ⊥⊥ X j | ( s i , s j ) .
Revised: X i ⊥⊥ X j | ( s i , s j ) .

Original: Therefore, considering GCN-like message passing ˜ X i = (cid:80) nj =1 ˜
Revised: Therefore, considering GCN-like message passing ˜ X i = (cid:80) nj =1 ˜

Original: A
Revised: A

Original: ij X j , we have the aggregated node attributes expectation given sensitive attribute as follows:
Revised: ij X j , we have the aggregated node attributes expectation given sensitive attribute as follows:

Original: [Equation]
Revised: [Equation]

Original: where ν 1 =
Revised: where ν 1 =

Original: [Equation]
Revised: [Equation]

Original: +
Revised: +

Original: [Equation]
Revised: [Equation]

Original: + n 1 q conn .
Revised: + n 1 q conn .

Original: Similarly
Revised: Similarly

Original: [Equation]
Revised: [Equation]

Original: for the node with sensitive attribute
Revised: for the node with sensitive attribute

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: where ν 2 = ( n − [Equation] − 1) q conn n − 1 q conn +1+( n 1 − 1) p conn .
Revised: where ν 2 = ( n − 1 − 1) q conn n − 1 q conn +1+( n 1 − 1) p conn .

Original: As for the covariance matrix of aggregated node attributes ˜ X given node sensitive attribute s = − 1 and original sensitive attribute
Revised: As for the covariance matrix of aggregated node attributes ˜ X given node sensitive attribute s = − 1 and original sensitive attribute

Original: [Equation]
Revised: [Equation]

Original: note that we can obtain
Revised: note that we can obtain

Original: [Equation]
Revised: [Equation]

Original: where ζ 1 = ( n − 1 − 1) p conn + 1 + n 1 q conn .
Revised: where ζ 1 = ( n − 1 − 1) p conn + 1 + n 1 q conn .

Original: Similarly, given node sensitive attribute s = 1 , we have ˜ Σ 

Revised: Similarly, given node sensitive attribute s = 1 , we have ˜ Σ

Original: =
Revised: =

Original: D
Revised: D

Original: ˜
Revised: ˜

Original: X
Revised: X

Original: i [ ˜ X i | s i = 1] =
Revised: i [ ˜ X i | s i = 1] =

Original: [Equation]
Revised: [Equation]

Original: conn +1+( n 1 − 1) p conn △ = ζ − 1 2 Σ , where ζ 2 = n − 1 q conn + 1 + ( n 1 − 1) p conn .
Revised: conn +1+( n 1 − 1) p conn △ = ζ − 1 2 Σ , where ζ 2 = n − 1 q conn + 1 + ( n 1 − 1) p conn .

Original: In other words, the covariance matrix of the aggregated node attributes is lower than  original one since  “average" operation will make node representation more concentrated.
Revised: In other words, the covariance matrix of the aggregated node attributes is lower than the original one since the “average" operation will make node representation more concentrated.

Original: Note that the summation over several Gaussian random variables is still Gaussian, we define the node attributes distribution for sensitive attribute s = − 1 and s = 1 as P 1 = N ( µ 1 , Σ ) , P 2 = N ( µ 2 , Σ ) , respectively.
Revised: Note that the summation over several Gaussian random variables is still Gaussian, we define the node attributes distribution for sensitive attribute s = − 1 and s = 1 as P 1 = N ( µ 1 , Σ ) , P 2 = N ( µ 2 , Σ ) , respectively.

Original: Similarly, the aggregated node representation distribution follows for sensitive attribute s = − 1 and s = 1 as ˜ P 1 = N (˜ µ 1 , ˜ Σ 1 ) , ˜ P 2 =
Revised: Similarly, the aggregated node representation distribution follows for sensitive attribute s = − 1 and s = 1 as ˜ P 1 = N (˜ µ 1 , ˜ Σ 1 ) , ˜ P 2 =

Original: N (˜ µ 2 , ˜ Σ 2 ) .
Revised: N (˜ µ 2 , ˜ Σ 2 ) .

Original: Note that the sensitive attribute ratio keeps the same after the aggregation and lager KL divergence for these two sensitive attributes group distribution, the bias enhances ∆ Bias > 0 if D KL ( ˜ P 1 || ˜ P 2 ) > D KL ( P 1 || P 2 ) andD KL ( ˜ P 2 || ˜ P 1 ) >
Revised: Note that the sensitive attribute ratio keeps the same after the aggregation and larger KL divergence for these two sensitive attributes group distribution, the bias enhances ∆ Bias > 0 if D KL ( ˜ P 1 || ˜ P 2 ) > D KL ( P 1 || P 2 ) and

Original: D KL (  P 2 || P 1 ) .
Revised: D KL ( ˜ P 2 || ˜ P 1 ) > D KL ( P 2 || P 1 ) .

Original: There fore, we only focus on the KL divergence.
Revised: Therefore, we only focus on the KL divergence.

Original: According to Lemma 2, it is easy to obtain KL divergence for original distribution as follows:
Revised: According to Lemma 2, it is easy to obtain KL divergence for original distribution as follows:

Original: [Equation]
Revised: [Equation]

Original: As for KL divergence for aggregated distribution, similarly, we have
Revised: As for KL divergence for aggregated distribution, similarly, we have

Original: [Equation]
Revised: [Equation]

Original: where inequality ( c ) holds since ln x ≤ x − 1 for any x > 0 .
Revised: where inequality ( c ) holds since ln x ≤ x − 1 for any x > 0 .

Original: Compared with equations (12) and (13), it is seen that D KL ( ˜ P 1 || ˜ P 2 ) > D KL ( P 1 || P 2 ) if ( ν 1 − ν 2 ) 2 ζ 2 > 1 .
Revised: Compared with equations (12) and (13), it is seen that D KL ( ˜ P 1 || ˜ P 2 ) > D KL ( P 1 || P 2 ) if ( ν 1 − ν 2 ) 2 ζ 2 > 1 .

Original: Similarly, we can have
Revised: Similarly, we can have

Original: D KL ( ˜ P 2 || ˜ P 1 ) > D KL ( P 2 || P 1 )
Revised: D KL ( ˜ P 2 || ˜ P 1 ) > D KL ( P 2 || P 1 )

Original: if ( ν 1 − ν 2 ) 2 ζ 1 > 1 .
Revised: if ( ν 1 − ν 2 ) 2 ζ 1 > 1 .

Original: In a nutshell, the bias enhances ∆ Bias >after message passing if ( ν 1 − ν 2 ) 2 min { ζ 1 , ζ 2 } > 1 .
Revised: In a nutshell, the bias enhances ∆ Bias >after message passing if ( ν 1 − ν 2 ) 2 min { ζ 1 , ζ 2 } > 1 .

Original: C T OPOLOGY A MPLIFIES B IAS IN O NE -L AYER GCN
Revised: C T OPOLOGY A MPLIFIES B IAS IN O NE -L AYER GCN

Original: Section 3 demonstrates that GCN-like aggregation operation amplifies node representation bias for graph data with large topology bias.
Revised: Section 4 demonstrates that GCN-like aggregation operation amplifies node representation bias for graph data with large topology bias.

Original: However, it is still unclear whether such observation holds for general GNNs or not.
Revised: However, it is still unclear whether such observation holds for general GNNs or not.

Original: Generally speaking, this problem is quite fundamental and challenging to understand the role of topology in fair graph learning.
Revised: Generally speaking, this problem is quite fundamental and challenging to understand the role of topology in fair graph learning.

Original: In this section, we try to move a step toward this problem via considering one-layer GCN.
Revised: In this section, we try to move a step toward this problem by considering one-layer GCN.

Original: Prior to comparing the prediction for one-layer
Revised: Prior to comparing the prediction for one-layer

Original: GCN and one-layer MLP, we first provide the connection between demographic parity and mutual information of sensitive attributes and predictions.
Revised: GCN and one-layer MLP, we first provide the connection between demographic parity and mutual information of sensitive attributes and predictions.

Original: Then, we theoretically compare the prediction bias of one-layer GCN and one-layer MLP through the lens of mutual information.
Revised: Then, we theoretically compare the prediction bias of one-layer GCN and one-layer MLP through the lens of mutual information.

Original: C. P REDICTION B IAS AND M UTUAL I NFORMATION
Revised: C. P REDICTION B IAS AND M UTUAL I NFORMATION

Original: Here, we only consider binary sensitive attributes s ∈ {− 1 , 1 } and binary labels y ∈ {− 1 , 1 } .
Revised: Here, we only consider binary sensitive attributes s ∈ {− 1 , 1 } and binary labels y ∈ {− 1 , 1 } .

Original: Similarly, we can define ˆ y ∈ {− 1 , 1 } as the binary model predictions.
Revised: Similarly, we can define ˆ y ∈ {− 1 , 1 } as the binary model predictions.

Original: In the fairness community, demographic parity, defined as the average prediction difference among different sensitive attribute groups, is the most commonly used fairness metric, i.e., ∆ DP = | P (ˆ y | s = 1) − P (ˆ y | s = − 1) | .
Revised: In the fairness community, demographic parity, defined as the average prediction difference among different sensitive attribute groups, is the most commonly used fairness metric, i.e., ∆ DP = | P (ˆ y | s = 1) − P (ˆ y | s = − 1) | .

Original: From the mutual information perspective, the correlation between sensitive attributes s and prediction ˆ y can be measured by I ( s ; ˆ y ) .
Revised: From the mutual information perspective, the correlation between sensitive attributes s and prediction ˆ y can be measured by I ( s ; ˆ y ) .

Original: In this subsection, we provide an inherent connection between demographic parity ∆ DP and mutual information I ( s ; ˆ y ) as follows:
Revised: In this subsection, we provide an inherent connection between demographic parity ∆ DP and mutual information I ( s ; ˆ y ) as follows:

Original: Theorem 3 For binary sensitive attributes s ∈ {− 1 , 1 } and binary prediction ˆ y ∈ {− 1 , 1 } , demographic parity ∆ DP and mutual information I ( s ; ˆ y ) satisfies I ( s ; ˆ y ) ≤ 2∆ DP
Revised: Theorem 3 For binary sensitive attributes s ∈ {− 1 , 1 } and binary prediction ˆ y ∈ {− 1 , 1 } , demographic parity ∆ DP and mutual information I ( s ; ˆ y ) satisfies I ( s ; ˆ y ) ≤ 2∆ DP

Original: Proof: For simplicity, we defined the joint probability as α i ∪ j = P
Revised: Proof: For simplicity, we defined the joint probability as α i ∪ j = P

Original: (ˆ y
Revised: (ˆ y

Original: =
Revised: =

Original: i, s = j ) and condition probability as α i
Revised: i, s = j ) and condition probability as α i

Original: | j = P (ˆ y = i | s = j ) .
Revised: | j = P (ˆ y = i | s = j ) .

Original: Considering the log ratio between joint distribution and margin product probability, we havelog P
Revised: Considering the log ratio between joint distribution and margin product probability, we have

Original: [Equation]
Revised: [Equation]

Original: where inequality (d) holds due to log(1 + x ) ≤ x for any x > − 1 .
Revised: where inequality (d) holds due to log(1 + x ) ≤ x for any x > − 1 .

Original: According to the definition of mutual information, we have
Revised: According to the definition of mutual information, we have

Original: [Equation]
Revised: [Equation]

Original: where inequality (f) holds due to (cid:80) i a i b i ≤ (cid:80) i a i (cid:80) i b i for non-negative a i and b i .
Revised: where inequality (f) holds due to (cid:80) i a i b i ≤ (cid:80) i a i (cid:80) i b i for non-negative a i and b i .

Original: □
Revised: □

Original: Theorem 3 shows there is a strong connection between demographic parity and mutual information for binary sensitive attributes and binary labels, i.e., the mutual information is upper bounded by demographic parity multiplied by 2 .
Revised: Theorem 3 shows there is a strong connection between demographic parity and mutual information for binary sensitive attributes and binary labels, i.e., the mutual information is upper bounded by demographic parity multiplied by 2 .

Original: C. P REDICTION B IAS COMPARISON BETWEEN ONE - LAYER GCN AND ONE - LAYER MLP
Revised: C. P REDICTION B IAS COMPARISON BETWEEN ONE - LAYER GCN AND ONE - LAYER MLP

Original: Considering the strong connection between mutual information and demographic parity, we investigate prediction bias comparison between one-layer GCN and one-layer MLP through the lens of mutual information.
Revised: Considering the strong connection between mutual information and demographic parity, we investigate prediction bias comparison between one-layer GCN and one-layer MLP through the lens of mutual information.

Original: For one-layer MLP model, the prediction is given by ˆ y MLP =
Revised: For one-layer MLP model, the prediction is given by ˆ y MLP =

Original: σ ( XW MLP ) , where W MLP is trainable parameter for MLP.
Revised: σ ( XW MLP ) , where W MLP is trainable parameter for MLP.

Original: Similarly, for one-layer GCN, the prediction is given by ˆ y GCN = σ ( ˜ AXW GCN ) , where W GCN is  trainable parameter for GCN.
Revised: Similarly, for one-layer GCN, the prediction is given by ˆ y GCN = σ ( ˜ AXW GCN ) , where W GCN is the trainable parameters for GCN.

Original: Define ˜ X =
Revised: Define ˜ X =

Original: ˜
Revised: ˜

Original: AX , it is easy to seen that one-layer MLP model and one-layer GCN model are almost the same except with different node features.
Revised: AX , it is easy to see that one-layer MLP model and one-layer GCN model are almost the same except with different node features.

Original: Based on Theorem 2, the aggregated node features ˜ X embrace higher presentation bias than that of X .
Revised: Based on Theorem 2, the aggregated node features ˜ X embrace higher presentation bias than that of X .

Original: In other words, the bias of input data for one-layer GCN is higher than that of one-layer MLP.
Revised: In other words, the bias of input data for one-layer GCN is higher than that of one-layer MLP.

Original: For the prediction bias, note that sensitive attributes s , node features X , and prediction ˆ y form a
Revised: For the prediction bias, note that sensitive attributes s , node features X , and prediction ˆ y form a

Original: Markov chain X
Revised: Markov chain X

Original: → X → ˆ y since P (ˆ y | s , X ) =
Revised: → X → ˆ y since P (ˆ y | s , X ) =

Original: P (ˆ y | X ) for the model with vanilla training.
Revised: P (ˆ y | X ) for the model with vanilla training.

Original: Based on data processing inequality, it is easy to obtain
Revised: Based on data processing inequality, it is easy to obtain

Original: [Equation]
Revised: [Equation]

Original: In other words, when I ( s ; X | ˆ y MLP ) =
Revised: In other words, when I ( s ; X | ˆ y MLP ) =

Original: I ( s ; ˜ X
Revised: I ( s ; ˜ X

Original: | ˆ y GCN ) , the higher input data bias will lead to higher prediction bias.
Revised: | ˆ y GCN ) , the higher input data bias will lead to higher prediction bias.

Original: For one-layer MLP and one-layer GCN, if the condition in Theorem 2 are satisfied, the prediction bias of one-layer GCN is also larger than that of MLP.
Revised: For one-layer MLP and one-layer GCN, if the condition in Theorem 2 are satisfied, the prediction bias of one-layer GCN is also larger than that of MLP.

Original: D R ELATED W ORKS
Revised: C. C OMPARISON WITH C ONCENTRATION P ROPERTY IN GNN AND P ERSISTENT H OMOLOGY

Revised: GNN’s concentration property represents all node presentation convergence after stacking of aggregations (Nt & Maehara, 2019; Ma et al., 2022; Baranwal et al., 2021). There are differences between bias enhancement and concentration property in GNN: • Definition.

Revised: Concentration property means that the node representation of all nodes convergence after GNN aggregation.

Revised: Bias enhancement represents the node representation for different sensitive groups that are more distinguished.

Revised: In fact, such two properties are somehow contrary since perfect concentration leads to zero bias.

Revised: • Aggregation.

Revised: For concentration property,

Revised: only the

Revised: normal features and topology are involved in the analysis.

Revised: The high-level interpretation of concentration is that aggregation acts like a low-frequency filter and such an “average" effect leads to node representation convergence.

Revised: In the sensitive homophily coefficient, we would like to clarify that the sensitive attributes are not included for node feature aggregation due to law restrictions.

Revised: Even though the sensitive attribute is included in GNN aggregation, and all node representations are the same, it does not represent the bias enhancement (actually zero bias.).

Revised: The bias in GNN represents the highly different node representations or predictions among different sensitive groups (defined by sensitive attributes).

Revised: • Comparison.

Revised: We also provide the connection between concentration property and bias enhancement in GNN.

Revised: The intuition of why GNN enhances the bias, high sensitive homophily represents that the nodes with the same sensitive attributes are connected with high probability.

Revised: Considering concentration property, the node representation for the same sensitive attribute is more similar after aggregation, therefore leading to highly different representations for different sensitive attribute groups.

Revised: Notice that such behavior only happens for high sensitive homophily coefficients and shallow GNN.

Revised: When the node with different sensitive attribute groups is connected randomly, the bias enhancement would not happen or be insignificant due to random concentration.

Revised: When adopting deep GNNs, all node representations converge and have no bias enhancement.

Revised: Unfortunately, due to concentration property, shallow GNNs are mainly used in practice.

Revised: As for high sensitive homophily coefficient, such a condition is usually satisfied in practice due to natural graph data property.

Revised: Additionally, there are several differences between our proposed optimization scheme and work (Carriere et al., 2021) including definition, dependence, and optimization:

Revised: • Definition: Persistent homology is a method for calculating the importance of topological features in the simplicial complex.

Revised: For example, giving a set of points in a point cloud corresponding to a chair, the task is to detect the object from the points.

Revised: In this case, there is no connection between any pair of points.

Revised: Persistent homology is a tool to identify the topological feature (or connection patterns) via gradually building up the connection between points.

Revised: However, for graphs, they are 1-simplex with explicit connection patterns defined by the set of edges, thus many properties from persistent homology will degenerate to the field of graph theory.

Revised: For example, applying the persistent homology on the graph is equivalent to building maximum spanning trees (MSTs) using Kruskal algorithm (Kleinberg & Tardos, 2006), which is irrelevant to our proposed optimization scheme.

Revised: • Dependence.

Revised: Persistent homology is generally related to sample features, as shown in the example Point cloud optimization of (Carriere et al., 2021).

Revised: In other words, persistent homology somehow represents the topological features of all samples.

Revised: Differentially, in the graph data we focused on, there are node normal attributes, sensitive attributes, and adjacency matrix (graph topology).

Revised: Based on the definition, the sensitive homophily coefficient is related to sensitive attributes and the adjacency matrix.

Revised: However, the optimized adjacency matrix is generally dependent on sensitive attributes.

Revised: • Optimization.

Revised: The main challenge for persistent homology-based optimization is generally undifferentiable except in some special cases.

Revised: (Carriere et al., 2021) develops a general framework to study the differentiability of the persistence of parametrized families of filtrations.

Revised: In this way, under mild assumptions, stochastic subgradient descent algorithms can be applied to such functions to converge almost surely to a critical point.

Revised: For our problem (2), the gradient of loss over topology is differentiable in general.

Revised: The challenge falls in the constraint of an element value.

Revised: In our solution, we use a gradient-based method to update the adjacency matrix via variables relaxation and then adopt project operation to satisfy such constraint

Revised: D R ELATED W ORKS

Original: Graph neural networks.
Revised: Graph neural networks.

Original: GNNs achieve state-of-the-art performance for various real-world applications.
Revised: GNNs achieve state-of-the-art performance for various real-world applications.

Original: There are two categories in GNNs model backbones, including spectral-based and spatial-based
Revised: There are two categories in GNNs model backbones, including spectral-based and spatial-based

Original: GNNs.
Revised: GNNs.

Original: Spectral-based GNNs provide graph convolution operation together with feature transformation (Bruna et al., 2013; Defferrard et al., 2016; Henaff et al., 2015).
Revised: Spectral-based GNNs provide graph convolution operation together with feature transformation (Bruna et al., 2013; Defferrard et al., 2016; Henaff et al., 2015).

Original: Many spatial-based GNNs are also proposed to aggregate the neighbors’ information, including graph attention network (GAT) (Veliˇckovi´c et al., 2018), GraphSAGE (Hamilton et al., 2017), SGC (Wu et al., 2019), APPNP (Klicpera et al., 2019), et al (Gao et al., 2018; Monti et al., 2017).
Revised: Many spatial-based GNNs are also proposed to aggregate the neighbors’ information, including graph attention network (GAT) (Veliˇckovi´c et al., 2018), GraphSAGE (Hamilton et al., 2017), SGC (Wu et al., 2019), APPNP (Klicpera et al., 2019), et al (Gao et al., 2018; Monti et al., 2017).

Original: Fairness-aware learning on graphs.
Revised: Fairness-aware learning on graphs.

Original: Fairness in machine learning has attracted many research efforts to mitigate prediction bias (Chuang & Mroueh, 2020; Zhang et al., 2018; Du et al., 2021;
Revised: Fairness in machine learning has attracted many research efforts to mitigate prediction bias (Chuang & Mroueh, 2020; Zhang et al., 2018; Du et al., 2021;

Original: Yurochkin & Sun, 2020; Jiang et al., 2022; Creager et al., 2019; Feldman et al., 2015).
Revised: Yurochkin & Sun, 2020; Jiang et al., 2022; Creager et al., 2019; Feldman et al., 2015).

Original: Fair walk (Rahman et al., 2019) is a fair version of random walk to learn fair node representation via revising neighborhood sampling.
Revised: Fair walk (Rahman et al., 2019) is a fair version of random walk to learn fair node representation via revising neighborhood sampling.

Original: From the bias mitigation perspective, adversarial debiasing and contrastive learning are also developed for graph data.
Revised: From the bias mitigation perspective, adversarial debiasing and contrastive learning are also developed for graph data.

Original: For example, works (Dai & Wang, 2021; Bose & Hamilton, 2019; Fisher et al., 2020) also adopt the adversary to predict the sensitive attribute given the node representation.
Revised: For example, works (Dai & Wang, 2021; Bose & Hamilton, 2019; Fisher et al., 2020) also adopt the adversary to predict the sensitive attribute given the node representation.

Original: fairness-aware representation learning are also developed via node feature masking, graph topology rewire (Agarwal et al., 2021; Köse & Shen, 2021) for node classification or link prediction tasks (Laclau et al., 2021; Li et al., 2021).
Revised: fairness-aware representation learning is also developed via node feature masking, graph topology rewires (Agarwal et al., 2021; Köse & Shen, 2021) for node classification or link prediction tasks (Laclau et al., 2021; Li et al., 2021).

Original: However, the inherent reason behind the observation that GNNs show higher prediction bias than MLP is still missing.
Revised: However, the inherent reason behind the observation that GNNs show higher prediction bias than MLP is still missing.

Original: In this work, we theoretically and experimentally reveal that many GNNs aggregation schemes boost node representation bias under topology bias.
Revised: In this work, we theoretically and experimentally reveal that many GNNs aggregation schemes boost node representation bias under topology bias.

Original: Furthermore, we develop a simple yet effective graph refinement method, named FairGR, to achieve fair prediction.
Revised: Furthermore, we develop a simple yet effective graph refinement method, named FairGR, to achieve fair prediction.

Original: E D ATASET S TATISTICS
Revised: E D ATASET S TATISTICS

Original: The data statistical information on three real-world datasets, including Pokec-n, Pokec-z, and NBA, are provided in Table 2.
Revised: The data statistical information on three real-world datasets, including Pokec-n, Pokec-z, and NBA, are provided in Table 2.

Original: It is seen that the sensitive homophily are even higher than label homophily coefficient among three real-world datasets, which validates thatthe real-world datasets are usually with large topology bias.
Revised: It is seen that the sensitive homophily is even higher than label homophily coefficient among three real-world datasets, which validates that real-world datasets are usually with large topology bias.

Original: [Table]
Revised: [Table]

Original: F T RAINING A LGORITHMS
Revised: F T RAINING A LGORITHMS

Original: We summarize the training algorithm for FairGR and provide the pseudo codes in Algorithm 1.
Revised: We summarize the training algorithm for FairGR and provide the pseudo codes in Algorithm 1.

Original: Algorithm 1: FairGR Algorithm
Revised: Algorithm 1: FairGR Algorithm

Original: Input : Graph topology A , label y , and sensitive attribute s ; The total epochs T ;
Revised: Input : Graph topology A , label y , and sensitive attribute s ; The total epochs T ;

Original: Hyperparameters α and β .
Revised: Hyperparameters α and β .

Original: Output : The fair graph topology ˜ A .
Revised: Output : The fair graph topology ˜ A .

Original: 1 for epoch from1 to T do
Revised: 1 for epoch from T do

Original: Calculate the gradient 
Revised: Calculate the gradient s y

Original: A s , y ,

Original: A ;
Revised: A ;

Original: Update graph topology
Revised: Update graph topology

Original: A gradient descent;
Revised: A gradient descent;

Original: Project graph
Revised: Project graph

Original: A feasible region; 5 end
Revised: A feasible region; 5 end

Original: G
Revised: G

Original: M ORE E XPERIMENTAL R ESULTS
Revised: M ORE E XPERIMENTAL R ESULTS

Original: G
Revised: G

Original: [Equation]
Revised: [Equation]

Original: E XPERIMENTAL R ESULTS
Revised: E XPERIMENTAL R ESULTS

Original: In this subsection, we provide more experimental results on different covariance matrix.
Revised: In this subsection, we provide more experimental results on different covariance matrix.

Original: Although our theory is only derived for the same covariance matrix, we still observe similar results for the case of different covariance matrix.
Revised: Although our theory is only derived for the same covariance matrix, we still observe similar results for the case of different covariance matrix.

Original: For node attribute generation , we generate node attribute with
Revised: For node attribute generation , we generate node attribute with

Original: Gaussian distribution N ( µ 1 , Σ ) and N ( µ 2 , Σ ) for node with binary sensitive attribute, respectively, where µ 1 = [0 , 1] , µ 1 = [1 , 0] and Σ = (cid:20) 10(cid:21) .
Revised: Gaussian distribution N ( µ 1 , Σ ) and N ( µ 2 , Σ ) for node with binary sensitive attribute, respectively, where µ 1 = [0 , 1] , µ 1 = [1 , 0] and Σ = (cid:20) 10(cid:21) .

Original: We adopt the same evaluation metric and adjacency matrix generation scheme in Section 5.
Revised: We adopt the same evaluation metric and adjacency matrix generation scheme in Section 6.

Original: Figure 5 shows the DP difference during message passing with respect to sensitive homophily coefficient for different initial covariance matrices.
Revised: Figure 4 shows the DP difference during message passing with respect to sensitive homophily coefficient for different initial covariance matrices.

Original: We observe that a higher sensitive homophily coefficient generally leads to larger bias enhancement.
Revised: We observe that a higher sensitive homophily coefficient generally leads to larger bias enhancement.

Original: Additionally, higher graph connection density ρ d , larger node number n , and balanced sensitive attribute ratio c correspond  higher bias enhancement, which is consistent with our theoretical analysis in Theorem 2.
Revised: Additionally, higher graph connection density ρ d , larger node number n , and balanced sensitive attribute ratio c correspond to higher bias enhancement, which is consistent with our theoretical analysis in Theorem 2.

Original: [Figure]
Revised: [Figure]

Revised: G. A BLATION S TUDY FOR F AIR GR

Revised: For investigating the effect of hyperparameters α and β , we conduct experiments with different hyperparameters chosen from α = { 0 . 0 , 0 . 1 , 0 . 5 , 1 . 0 , 5 . 0 , 10 . 0 } and β = { 0 . 0 , 0 . 1 , 0 .

Revised: 5 , 1 . 0 , 5 . 0 , 10 . 0 } while the other one is selected as default.

Revised: The default value for hyperparameters α and β are 0 .

Revised: 1 and 0 . 5 , respectively.

Revised: The results of hyperparamter study with respect to α and β are shown in Figuresand 6, respectively.

Revised: From these results, we can obtain the following observations:

Revised: • Hyprparameter α and β demonstrate different influences on GNN backbone.

Revised: For example, for Pokec-n dataset, hyperparameter α only shows a negligible influence on the accuracy of GCN and GAT, while significant for GAT.

Revised: As for DP, the bias metric is more sensitive to α compared with accuracy.

Revised: • Hyprparameter α and β demonstrate different influences on graph dataset.

Revised: For example, GAT achieves the best accuracy and lowest DP with α = 0 .

Original:  NBA 
Revised: 1 in NBA dataset, while achieving the lowest accuracy and highest DP in Pokec-n dataset. Such observation indicates the importance of hyperparameter tuning for different datasets. [Equation] EHAVIORS FOR GNN S AND MLP

Revised: For vanilla training across different GNN backbones, we plot the training curve with respect to epochs to investigate the training behaviors for GNNs and MLP.

Revised: For the training behavior, GNNs,

Revised: [Figure]

Revised: [Figure]

Revised: and MLP models both converge in terms of accuracy for the high label homophily dataset Pokec-n and Pokec-z dataset.

Revised: For high sensitive homophily Pokec-n and Pokec-z dataset, GNNs demonstrate higher prediction than that MLP, while the prediction bias difference is relatively small for the low-sensitive-homophily dataset NBA.

Original: G. F AIR GR R ESULTS ON V ANILLA T RAINING
Revised: G. F AIR GR R ESULTS ON V ANILLA T RAINING

Original: For vanilla training Figure 6 show the test accuracy and demographic parity curve during training for different GNNs backbones and whether FairGR is adopted for graph topology rewire.
Revised: For vanilla training, Figure 8 shows the test accuracy and demographic parity curve during training for different GNNs backbones and whether FairGR is adopted for graph topology rewire.

Original: From these results we can obtain the following observations:
Revised: From these results, we can obtain the following observations:

Original: • Different GNNs demonstrate different accuracy and demographic parity performance.
Revised: • Different GNNs demonstrate different accuracy and demographic parity performance.

Original: For example, for Pokec-n dataset, GCN has the highest accuracy performance and lowest demographic parity, which implies that message passing also matters even for the same graph topology.
Revised: For example, for Pokec-n dataset, GCN has the highest accuracy performance and lowest demographic parity, which implies that message passing also matters even for the same graph topology.

Original: • Our proposed FairGR consistently achieve lower demographic parity and comparable accuracy performance on all datasets and backbones. 
Revised: • Our proposed FairGR consistently achieves lower demographic parity and comparable accuracy performance on all datasets and backbones. [Figure] [Figure]

Original: G. T RADEOFF PERFORMANCE ON F AIR M IXUP
Revised: G. T RADEOFF PERFORMANCE ON F AIR M IXUP

Original: We also demonstrate that FairGR can achieve better tradeoff performance for different GNN backbones with Fair mixup (Chuang & Mroueh, 2021) in Figure 7.
Revised: We also demonstrate that FairGR can achieve better tradeoff performance for different GNN backbones with Fair mixup (Chuang & Mroueh, 2021) in Figure 9.

Original: Specifically, since input fair mixup requires calculating model prediction for mixed input batch, it is non-trivial to adopt input fair mixup in our experiments on the node classification task.
Revised: Specifically, since input fair mixup requires calculating model prediction for mixed input batch, it is non-trivial to adopt input fair mixup in our experiments on the node classification task.

Original: This is because, for GNN aggregations of neighborhoods’ information, the neighborhood information for the mixed input batch is missing.
Revised: This is because, for GNN aggregations of neighborhoods’ information, the neighborhood information for the mixed input batch is missing.

Original: Instead, we adopt manifold fair mixup for the logit layer in our experiments.
Revised: Instead, we adopt manifold fair mixup for the logit layer in our experiments.

Original: Experimental results show that
Revised: Experimental results show that

Revised: FairGR can achieve better accuracy-fairness tradeoff performance for many GNNs backbones on three datasets.

Revised: H F UTURE W ORK

Revised: There are two lines of follow-up research directions.

Original: [Figure]
Revised: Firstly, The generalization of the theoretical analysis on why aggregation enhances bias in GNN can be further extended. As a pilot study, we theoretically investigate why this phenomenon happens for GCN-like aggregation under random graph topology generated by stochastic block model and Gaussian mixture feature distribution. The more general analysis of other aggregation operations, random graph models, and other feature distributions can be extended. The other line focuses on the graph topology rewire algorithmic perspective, including improving the efficiency of FairGR and effectiveness via designing different objectives for graph topology rewire.