Reassessing Fairness: A Reproducibility Study of NIFA’s Impact on GNN Models
Authors: Ruben Figge
Sjoerd Gunneweg
Aaron Kuin
Mees Lindeman
Venue: Accepted by TMLR ()

Abstract: Graph Neural Networks (GNNs) have shown strong performance on graph-structured data but raise fairness concerns by amplifying existing biases. The Node Injection-based Fairness Attack (NIFA) (Luo et al., 2024) is a recently proposed gray-box attack that degrades group fairness while preserving predictive utility. In this study, we reproduce and evaluate NIFA across multiple datasets and GNN architectures. Our findings confirm that NIFA consistently degrades fairness—measured via Statistical Parity and Equal Opportunity—while
maintaining utility on classical GNNs. However, claims of NIFA’s superiority over existing fairness and utility attacks are only partially supported due to limitations in baseline reproducibility. We further extend NIFA to accommodate multi-class sensitive attributes and evaluate its behavior under varying levels of graph homophily. While NIFA remains effective in multi-class contexts, its impact is more sensitive in mixed and highly homophilic graphs. Although this is not a comprehensive validation of all NIFA claims, our work provides targeted insights into its reproducibility and generalizability across fairness-sensitive scenarios. The codebase is publicly available at: https://github.com/sjoerdgunneweg/Reassessing-NIFA.

Published in Transactions on Machine Learning Research (06/2025)
Reassessing Fairness: A Reproducibility Study of NIFA's Impact on GNN Models
Ruben Figge' University of Amsteriam
ruber.fogge@studentt..uuo
Sjoerd Gunneweg" University of Amsteriam
Aaron Kuin" University of Amsteniam
naron.Kudi@@sttuentt.uuu
Mees Lindeman"
mees. mees. meesllinndemmnnsttddeen....
University of Amsterdam
Reviewed on OpenReview: https: // // openreview. net/ forum? id- 15fXUK
Abstract
Graph Neural Networks (GNNs) have shown strong performance graph-structured data but raise fairness concerns by amplifying existing biases. The Node Injection-based Fair- ness Attack (NIFA) Quo et al. 2024 is recently proposed gray-box allack that degrades group fairness while preserving predictive utility. In this study, we reproduce and evaluate NIFA across multiple datasets and GNN architectures. Our findings confirm that NIFA con- sistently degrades fairness measured via Statistical Parity and Equal Opportunity- while maintaining utility classical GNNs. However, claims of NIFA's 's superiority fairness and utility attacks are only partially supported due to limitations in baseline producibility. We further extend NIFA to accommodate multi-class sensitive attributes and evaluate its behavior under varying levels of graph homophily. While NIFA remains effective in multi-class contexts, its impact is more sensitive in mixed and highly homophilic graphs. Alt Although this is not a comprehensive validation of all NIFA claims, our work provides targeted insights into its reproducibility and generalizability across fairness- sensitive scenar- Reassessing-
existing
ios. The codebase is publicly available at: NIFA
1 Introduction
Graph Neural Networks (GNNs) Gori et al. 2005) have achieved strong performance in tasks involving graph-structured data, with applications in domains such as computer vision, recommendation systems, and social network analysis (Wu et al. 2020 However, fairness remains a critical challenge, as GNNs can amplify biases present in the training data. This poses risks in high-stakes applications such as recruitment, credit scoring, law enforcement, and health healthcare, where biased predictions can lead to unequal and potentially harmful outcomes (Dai & Wang 2021)..
Several recent works have explored fairness attacks GNNs Hussain al. 2022 Kang ct al. 2023 Zhang
et al. 2024b), most of which rely on modifying the graph structure by altering edges between existing nodes. While effective, such interventions may not be realistic in practice due to security or access constraints. To address this, Luo et al. (2024) introduced the Node Injection-based Fairness Attack (NIFA), a gray-box method that injects synthetic nodes into the graph during training. NIFA aims to degrade group fairness
"Equal contribution

---

Published in Transactions on Machine Learning Research (06/2025)
metrics-speciinee speciffcally Statistical Parity and Equal Opportunity accuracy.
whilee while maintaining utility metrics such
In this study, we critically evaluate the main claims of Luo et al. (2024) by reproducing their experiments across multiple datasets and GNN architectures. We also extend their work in two directions: (1) by generalizing NIFA to support multi-class sensitive attributes, and (2) by assessing effectiveness under varying levels of graph homophily. Our reproduction confirms NIFA's fairness-degrading effect on classical GNNs with minimal utility loss, consistent with the original findings. However, results for fairness-aware GNNs diverge in two of the three evaluated cases, and the claim that NIFA outperforms existing attacks is only partially supported due to limited reproducibility of basclines.
Our extensions demonstrate that NIFA remains effective in multi-class fairness settings, but its impact possibly varies with graph structure. In particular, graphs with high homophily or mixed homophilic- heterophilic patterns- conncctions reflcct varying degrocs of similarity between sensitive groups teuu to exhibit more severe fairness degradation. These results highlight the role of both demographic complexity and graph topology in shaping the vulnerabilities of GNNs to fairness attacks.
2 Scope of reproducibility
The objective of this study is to reproduce and validate the key findings presented in the original paper, Are Your Models Still Fair? Fairness Attacks on Graph Nearal Networks via Node Injections, by Luo et al. 2024). The authors introduce NIFA, a gray-box poisoning attack designed to compromise the fairness of Graph Neural Networks (GNNs) while maintaining minimal impact on their utility. In this reproducibility study, we aim to confirm the following claims and explore their implications:
Claim 1: NIFA can significantly degrade fairness in GNNs, measured by increased Asp (Statisti- cal Parity) and Ago (Equal Opportunity), while maintaining negligible utility loss across diverse datasets and models, including fairnoss aware GNNs.
Claim 2: NIFA achieves state-of-the performance with a perturbation rate as low as 1%, ensuring scalability and minimal detoctability.
Claim 3: Extensive experiments confirm that NIFA outperforms existing fairness and utility atlack baselines across multiple datasets.
We reproduce the core experiments of the original NIFA paper to evaluate its primary claims across clas- sical and fairness-aware GNN models. While Luo et al. (024) benchmarked NIFA against six baseline attacks-te -three targeting utility and three targeting fairness we were only able to reliably reproduce two: TDGIA (Zou et al. 2021) and FA-GNN (Hussain et al.] 2022). The remaining baselines lacked accessible or compatible implementations, and integrating them would have required substantial reengineering across codebases.
In addition to this partial reproduction, we exlend NIFA's 's evaluation in two directions to better reflect real-world fairness concerns:
Multi-Class Sensitive Attributes: We successfully extend NIFA to handle datasets with multi- class sensitive attributes, reflecting more complex real-world scenarios. This will enable an evaluation of whether the attack can remain effective and scalable in settings with higher attribute diversity,
Impact of Homophily affect NIFA's 's effectiveness and fairness, focusing the interplay between homophily and bias prop- agation due to node injection. We aim to determine if certain structural properties make networks more susceptible to fairness degradation.
NIFA:
We explore how varying homophily levels in graph structures
This study should be interpreted as a scoped reproduction and extension-not. a comprehensive validation- of NIFA. Our findings offer targeted insights into its generalizability, but do not evaluate all claims or baselines from the original work.
2

---

Published in Transactions on Machine Learning Research (06/2025)
3 Methodology
3.1 Problem Formulation
We adopt notation conventions from Luo et al. 2024 A graph g AV consists of nodes V, an adjacency malrix A E RIMXIVI, and a feature matrix X E RIVVD xD Each node v E V has a label ye E y and a sensitive attribute s. While prior studies limit s to binary values, we generalize s to a k-class setting (s E 0, II....k k - 1), where k denotes the number of classes in a sensitive attribute. GNN-based function fo with parameters 0 is trained predict labels using graph signals.
Fairness-related Concepts. In aligmment with prior works (Dai & Wang, 2021; Dong al. 2022 Ling et al. 2023), the authors focus group fairness. Here, fairness is assessed across groups defined by sensitive attributes. The sensitive attribute divides the node set V into two disjoint groups: Vo, where 8 0, and V1, where = 1. Group fairness is assessed using two primary metrics: Statistical Parity (SP) and Equal Opportunity (EO). These metrics evaluate the independence of predictions and correct predictions, respectively, from the sensitive attribute.
SP. Statistical Parily requires that the model's predictions are independent the sensitive altribute. Formally, for any predicted class y y, the following condition should hold:
P(.. = y|s = 0) P(ù. = y/s =
(1)
where de denotes the predicted label for node v. Statistical Parity ensures that the distribution of predictions is uniform across sensitive groups, regardless of their occurrence the graph.
EO. Equal Opportunity focuses on the model's ability to predict the correct label independently of the sensitive attribute. For any true class y Y, it requires that:
P(j. vly. y. 0) P(g. vly. y. s=
(2)
Where yo represents the true label of node v. Equal Opportunity ensures that the likelihood of correct predictions is consistent across sensitive groups.
The degree to which fairness is violated is quantified using two metrics, Asp and Ago, which measure deviations from the ideal conditions described above. These metrics are defined follows:
ASP (g y 0) P(g = y = 1)1l
(3)
AEO P( = y Me. 0) P(ğ = yu 1)]]
(4)
For both metrics, smaller values indicate better fairness, with Asp and Ago 0 representing perfect fairness.
Attack Definition. Following the line of previous attacks on GNNs (Sun et al. 2020 Zhang et al. 2024a
the authors launch a fairness-targeted attack on GNN models through the application node injection during the training phase. Operating under a gray-box setting, the attacker has access to the graph structure G, node labels V, and sensitive altributes s. However, the attacker lacks knowledge of the model architecture or parameters.
The attack involves injecting malicious nodes V, into the graph during training to degrade fairness while maintaining model accuracy. The modified graph g' AV, X') A','') features and connections. The goal is to maximize Asp and Ago under constraints on the number of injected nodes ([Vy < b) and their connectivity (deg(v) < d). The objective expressed as:
includes injected nodes with trained
maxg
s.t. arg M(fo S deg(u)vev, sd
3

---

Published in Transactions on Machine Learning Research (06/2025)
Original
Injected Nodis Vi
Legend:
Sroup
Surrogate Medel
faren
glaal odax
lijeted
Poisoned graph 6''
Tterative trakiig
Ipedtd elges
Casseiloooo Cotton Looe
Gonotraint
[o] Uncertainty Estimation
(b) Node: Injection
(c) Festure Optimization
Figure Overview of the NIFA framework (Luo et al. 2024
where F()) and M(-) are evaluation functions for fairness and utility, respectively. Constraints on the number of injected nodes ([Vi S. 6) and their connectivity (deg(v)vEv, S d) ensure that the attack remains unnoticeable and deceptive to the defenders, both b and d are prodefined budgets.
3.2
Model framework
The NIFA framework combines node injection and feature optimization to degrade fairness in GNNs. It follows two principles for node injection: (1) pucertaint-mmxxiizatton which uses a Bayesian GNN (Liu
ett al. 2022) to target nodes with high uncertainty near decision boundaries, and (2) Homophily-increms which ensures injected nodes connect exclusively to nodes within the same sensitive group, amplifying intra- group information propagation and increasing the homophily ratio.
Injected nodes' features are optimized using multiple objectives balance fairness degradation and utility preservation. An iterative strategy prevents overfitting to surrogate model parameters. The framework's components componends-umceettainee uncertainty maximization, node injection, and feature optimization are visualized in Figure and detailed below.
Nodes with high uncertainty are selected for injection as their predictions are more vulnerable to adversarial attacks. A Bayesian GNN with Monte Carlo (MC) dropout approximates parameter distributions (Liu et al. 2022), introducing variability predictions through sampled dropout masks. Uncertainty scores are computed the variance of predictions across samples:
Var?" (V,9)).
where T' is the number of samples and fag, (.) is the mapping function with the i-th sampled Bayesian GNN parameters 0B,, Nodes with the top 1% uncertainty within each sensitive group are selected, with k as a tunable hyperparameter.
Homophily-inerease. Injected nodes are distributed among sensitive groups and connect d random target nodes within the same group, where is hyperparameter (Figure Ip). This strategy prevents intra-group information propagation, exacerbating unfairness by increasing node-level homophily:
H. EVEN. 1(su = s
Wall
where Mr, represents the neighbors of node u, 8n is the sensitive attribute of u, and 1(+) is the indicator function.
Feature optimization. Injected nodes' features X, are optimized enhance the effectiveness of NIFA. In gray-box attack setting, attackers lack direct visibility of victim models and instead train surrogate
4

---

Published in Transactions on Machine Learning Research (06/2025)
GNN model S on the poisoned graph g'. The surrogate model S is trained iteratively alongside X, using different objectives, preventing overfitting to specific model parameters. While S follows a standard GNN training process with cross-entropy loss, the optimization of X; incorporates multiple objective functions to maximize unfairness while maintaining utility. Specifically, the optimization framework balances three key components: (1) classification loss to preserve predictive performance, (2) fairness loss to amplify disparities, and (3) a feature constraint to reinforce information differences across sensitive groups. The details of these loss functions and their implementation are presented in Appendix A
3.3 Additional Experiments
The previous sections have outlined the foundational work presented in (Luo et al. 2024). In this section, we provide detailed explanation of the contributions of this study. As in the original NIFA paper, we focus exclusively on node classification tasks, meaning edge classification is not considered in this study.
Multi-class sensitive attributes. In real-world scenarios, sensitive attributes often tend to be non-binary (Duong & Conrad 2023). To adapt the NIFA attack to these complex scenarios, the attack is generalized to operate on multi-class sensitive attributes. This extension involves two main components. Firstly, the fairness metries must. be generalized to accommodate multiple sensitive attribute classes. Secondly, the attack methodology requires adaptation to effectively address multiple sensitive groups.
To maintain consistency with the binary-sensitive attribute metrics established in the original paper, the One-vs-All (OvA) and One-vs-One (OvO) approaches are explored. These methods, commonly used in multi-class classification, allow multi-class problems to be approached by aggregating the results of binary classifiers. In the same way, these lechniques can be used to generalize the binary metrics to multi-class sensitive attributes by aggregating binary metrics. Comparable approaches have been utilized in prior research, where the One-vs-All method has been successfully applied to extend fairness metrics (Roızot ct al. 2022 Krasanakis et al. 2018).
In the One-vs-All approach, the binary metrics, as introduced in Equations I sensitive group against the remainder of the nodes. These binary results are then aggregated into a single metric by computing the expected value over all sensitive groups a E A and all labels y E ):
are computed for each
ASP-OrA E8| P v/s a) PÚù y/s + a)]
(5)
AEOOOA P(O = y. = y. a) P glue = y, 8 + a)]]
(6)
In contrast, the One-vs-One approach computes the expected value for the results of the binary metrics, which are computed for all possible combinations of the sensitive groups (01,02) {(x (z,,))=, y A, # v. x y) for all labels y E Y::
ASP-OvO El = yls a,) - P(ğ = u/s = a2)ll
(7)
A A00-000 = P( = vlv. = y. 01) - P(g = vly. = y. = =-)s=
(8)
Two modifications to the model framework are applied to ensure the attack methodology works. First, the node injection process must be adjusted to ensure an equal proportion injected nodes connect to each sensitive group while maintaining the homophily-increase principle. Second, the most significant change lies in feature optimization, where generalized metrics must be transformed into corresponding loss functions. The existing loss functions, detailed in Appendix A] are extended in line with the generalization of metrics. For the One-vs-All approach, Lsp and LEO are computed for each seusitive group versus the remaining groups, and the mean of these losses is used for optimization. Whereas in the One-vs-Ont approach, the mean losses are calculated across all pairwise combinations of sensitive attributes. The resulting mean losses will replace the LsP and LEO in the optimization objective.
Homophily Evaluation. To investigate the relationship between homophily and the effectiveness NIFA, we adopt a systematic methodology involving graph generation, GNN training, and fairness analysis. Graph structures with varying homophily levels are constructed using synthetic dataset. For these graphs, the
5

---

Published in Transactions on Machine Learning Research (06/2025)
Directed Preferential Attachment with Homophily (DPAH) model (Espin-Noboa et al. 2022 is employed to precisely coltrol homophily parameters, enabling exploration across spectrum of heterophilic to homophilic configurations. To ensure fair comparison, we also calculate the overall homophily rates for both the synthetic and original datascts. These configurations are shown in Figure 2 we explore wide range of homophily rate combinations, detailed Appendix D. building on insights from Ferrara al. (2022).
Beyond these predefined graphs,
(a] Heterophille hM 0.2
[b) Helee/Humo Humno ha 0.2 Amms
(c)
(d) Homoy Hete har 0.8
(c) Homophilic har 0.8 Bmon
Noutral har 0.5 Hmos
Amon
Graph vianalization ion
Figure 2: Graph examples of diffcrent levols of homophily (Espín-Nobon et al. 2022
The standard GCN model used in Luo al. (2024) is used train generated graphs under consistent experimental conditions. Similarly to the original paper, performance is evaluated on their ability to predict node labels. The NIFA framework is then applied across all homophily scenarios, with node injections optimized to target fairness. The perturbation rate is maintained at 1% to ensure minimal detectability, as per the original study. The impact of the attack is assessed by measuring changes in fairness metrics (Asp and Agо) and utility preservation.
Fairness impact is further analyzed by examining how varying homophily levels influence NIFA's ability to isolate minority nodes and exacerbate fairness disparities. The attack's effectiveness is quantified by evaluating the degradation in both fairness and model utility, with attention to the role of structural graph properties in mediating these elfects. Through this comprehensive framework, we aim to provide insights into the interplay between homophily and fairness vulnerabilities, as well as strategies for mitigating adversarial attacks in graph-based models.
4 Experimental Setup and Code
The code to run NIFA on the lour classic GNN models, including dalasets and package requirements, is publicly available on the authors' official code repository[] This code runs without requiring major modifications, although installing the correct packages was nocessary as some library-spocific updates have been made since. Code for reproducing the other experiments, such as rumning fairness-aware models and performing utility and fairness attacks, was not provided. To perform these experiments, their respective code repositories were independently accessed and made compatible with the NIF'A-genrraedd graphs. An updated list of requirements in the code repository. Each reproduction experiment is conducted five times, following the methodology outlined in the original paper.
created to ensure compatibility with our hardware, which is also included
4.1 Datasets
Original Datasets. The experiments conducted in the original paper utilized three different real-world datasets: Pokec-z, Pokec-n and D Pokec-z and Pokec-n are subsets of the Pokec network. Accord-
hitps://githul Caa.aaaabbeoo/CCC--oooooee/////e CCCCLccdee/NIII)
https:/ /swpppssaanood.ddduuu pokec..oooc.ttm
DR... Computer Science Bibliography

---

Published in Transactions on Machine Learning Research (06/2025)
ing to Takac & Zabovsky 20122) Pokec is the most popular social network in Slovakia and is also widely used in the Czech Republic. In this dataset, nodes represent users, while edges represent unidirectional following relationships (Luo et al. 2024).
The DBLP dataset is a citation dataset containing a digital library of comprehensive coverage of database literature (Elmacioglu & Lee 2005). Each node represents an author, and two authors are connected if they have co-authored at least one paper. The node features are derived from words extracted from the published papers of the corresponding author (Luo et al.)
2024). The specifics of cach dataset, well their respective homophily rates, are summarized in Table 4 in Appendix CJ Pokec-z and Pokec-n contain significantly larger numbers of nodes and edges than DBLP, while DBLP has the highest feature dimensionality.
Multi-class Sensitive Attribute Dataset. To the best of our knowledge, there are no graph datasets for node classification with explicit multi-class sensitive attributes. To accommodate for this, the method proposed by Purificctoo et al. (2024) is adopted, which involves augmenting dataset that has binary sensitive attributes to produce multi-class sensitive attributes. This augmentation is achieved by converting other potentially sensitive features into classes, for example, through binning. We follow the approach introduced by Purificato et al.] (0024), who applied this technique to the Pokec dataset by creating a new sensitive attribute. Specifically, the age feature was binned into the following ranges to form five age groups: 0-17, 18-23, 24-28, 29-35, and 36 and above. These experiments aim to compare the OvA and OvO approaches to generalizing the NIFA attack to multi-class sensitive attributes.
Synthetic Dataset. To investigate the impact of homophily rates, we construct synthetic dataset based on the work by [Bspin-Noboa et al. (2022). The dataset consists of synthetic networks generated using the Directed Preferential Attachment with Homophily (DPAH) mode T which integrates both preferential
attachment and homophily. Further details on the model's structure are provided in Appendix D
The dataset lacks explicit node features and only includes binary labels for majority and minority groups. To adapt this dataset for use within the NIFA pipeline, we generate proxy features by introdueing synthetic node classification task. We first assign proxy class labels to the nodes based on two Gaussian distributions. For a given node i, its proxy class label Yi is determined by sampling from two Gaussian distributions N(H1, and W(p2,0²), where the mean vectors μ4 and μ2 represent the centers of the proxy classes, and a² cols the variance within each class. These proxy labels serve as a basis for creating features required for the NIFA pipeline.
To construct the feature space, a n-dimensional feature vector x, R" was generated for each node i, where each feature dimension is sampled from the Gaussian corresponding to its assigned proxy label. Specifically:
o(),, if yı = 0, ((H,,
X;
if y = 1.
A correlation parameter p is introduced to establish a statistical relationship between the Gaussian distribu- tions and the proxy labels. This ensures that the feature space elfectively reflects the proxy labels, making the nodes distinguishable and enabling the creation of a meaningful node classification task. The sensitive attribute s; E {0, 1}, indicating whether a node belongs to the majority (s; = 0) or minority (s, = 1) group, remains binary. We divide the dataset into train (70%), validation (15%), and test (15%) splits. Each split preserves the original distribution of sensitive attributes and proxy labels to minimize sampling bias.
4.2 Model descriptions
Classical GNN Models. To reproduce the original NIEA results on classical models, GCN (Kipf & Welling 2017), GraphSAGE: (Hamilton et al. 2017), APPNP (Gasteiger et al.] (022). and SGC (Wu et al. 2019), the code provided on the GitHub repository was used without significant modifications. Further clarilications on these models are found in Appendix B.
https///gghaab..mmm//eeess//onnf Directed ScaleFree. Networks
ScaleFrre_Networrk
7

---

Published in Transactions on Machine Learning Research (06/2025)
Fairness Aware Models. Reproducing the results of the fairness aware models, specifically FairGNN (Dai & Wang 2021p b, FairVGNN (Wang et al. 2022). and FairSIN (Ju et al. 2023) (see Appendix B.1 for more details), proved to be more complex than reproducing those of the classical models. Adjustments to the code of the laire models were necessary to ensure valid results. Specifically, NIFA's poisoned graphs needed to be stored for each dalaset and loaded through a custom data loading function to maintain compatibility with the fairness model pipelines. Additionally, minor modifications to the training logic, including adjustments to the loss functions, were required. Finally, the exact fairness metrics outlined in subsection 3.1 had to be implemented within each fairness model.
Baseline Attack Models. The authors of the original paper compared NIFA to various competitors, including attacks based on utility: AFGSM (Wang et al. 2020), TDGIA (Zo et al. 2021), G² A2C (Ju ct al., 2023), and attacks based on fairness: FA-GNN (Hussain et al. 2022), FATE Kang et al. 2023 G-Fair Attack (Zhang et al. 2024b). Due to time limitations and the need to reproduce these models independently, two these models were reproduced: the utility-based attack TDGIA and the fairness-based
attack FA-GNN. Their details are found in Appendix B.2
To reproduce the FA-GNN results, two minor modifications were made to the code: the graph-loading process was adjusted, and the fairness metries detailed in subsection 3.1 were implemented. To obtain results for TDGIA, the graphs were poisoned by TDGIA and evaluated on the NIFA fairness metries. Three modifications were made to the codebase: the TDGIA graph-handling method was adjusted, graph transformation step was implemented in the NIFA code, and parts of the NIFA implementat ion were disabled to prevent already poisoned graphs from being reattacked.
4.3 Computational requirements
All our experiments were conducted utilizing NVIDIA A100 GPUs with CUDA 12.1.1. We tried to closely replicate the Python environment specified by the authors to ensure consistency. Specifically, we used Python 3.10.16 and PyTorch 2.4.0. For the Deep Graph Library (DGL), we employed version 2.4.0, which requires PyTorch < 2.4.0, a constraint we adhered to for compatibility. Additionally, we incorporated the PyTorrh Geometric (PyG) library version 2.6.1, along with auxiliary libraries like torch- seatter and torch-sparse, all compatible with CUDA 12.1.1.
5 Results
5.1
Results reproducing the original paper
The reproduced outcomes of the models described in ssubsccction 4.2 are presented in Table 1 results for utility- and fairness-based attacks are presented [Talle 2 In alignment with the original work, we report the mean and standard deviation for all results in this section. The results from the original paper are provided in Appendix E.
The reproduced
The original paper makes three key claims about the effectiveness of NIFA: (1) it significantly degrades fairness (measured via Asr and Ago), (2) it does so without compromising utility, and (3) it outperforms existing fairness and utility attacks across datasets.
Claims and are primarily supported by substantial increases Asp and Ago across classical GNN models, while maintaining high classification accuracy. For example, in the original results on the Pokcc-z dataset, the ASp and AEo values for GCN increased from 7.13% and 5.10% to 17.36% and 15.59% respec- tively. Even on fairness-awan aware models, the impact was reported significant for instance, FairVGNN's A on Pokec-z rose from 2.59% to 9.28%, a nearly fourfold increase. Mcanwhile, utility metrics across models dropped only slightly, usually by few percentage points.
In our reproduction, these claims are partially validated. For classical GNN models, our results closely mirror the original findings. While the absolute values of Asp and Ago are slightly lower our experiments, the relative increase from pre- to post-attack remains consistent. On Pokec-z, for example, all classical models show an approximate twofold increase in both fairness metrics following the attack. Similar trends are
8

---

Published in Transactions on Machine Learning Research (06/2025)
Table 1: Attack performance of NIFA on different victim GNN models. Results are reported in percentages (%). To showcase trends similar to the original paper, we bold the results where NIFA successfully degraded the fairness of victim GNN models.
Pokec-s
Pokec-n
DBLP
Accuracy
Agp
Ago
Aocurnry
Agp
Ago
Aocurnry
Agp
Ago
GCN
Before
71.27 + 0.22
7.74 + 0.30
5,82 + 0.34
71.12 = 0.34
0.74 + 0.47
2.08
0.84
96.13 = 0.76
3.74 1.8
2,31 + 1.32
After
70.99
0.55
15.74
à: 1.03
13.91
1.07
70.38 0.48
1.94
0.71
7.89
0.91
92.96 1.64
11.12
4.37
18.43
4.10
GraphSAGE
Before
70.58 1: 0.76
5.01 #: 1.48
3,50
4: 0.77
08.77
0.34
1.65 2: 1.31
2.34
1.04
96,034 0.37
4.65 a: 2.81
4.40
4: 0.65
After
0.56
5.81
1.19
4.08
0.89
68.93
1.19
9.32
1.88
3.56
1.91
94.42
0.92
8.082
1.24
14:08
1.90
APPNT
Before
09.28
2: 0.83
7:05 4: 0.50
5.20
0.49
08.573
2: 0.37
5.02
1: 0.75
5.34
0.74
96.55 =: 0.33
4.39
1: 0.88
2.94 1: 1.21
After
60.46
0.43
18.88
1,20
17.42
1,33
68.32
0.46
7.74 2.61
7.40
2.58
92.26
1.05
12.61
4.88
18.94
6.33
8GC
Refore
66.65
3.42
6.63 + 1.63
5.38
+ 0.96
66.42
3.25
4.18 2.23
4.00 + 1.903
96.58
0.47
4.07
0.47
2.67 + 0.68
After
67,02
1,56
14.53 +: 2.75
12.99
2.74
67.43 0.89
9,93# 3.00
9.58
2.96
92.81
0.76
10.21
5.55
16.44
7.05
FaiGGNN
Before
72.10
0.12
5.12 4: 0.58
3.04 1: 0.63
70.87 = 0.21
0.91
0.61
1.39
0:46
15.38
0:41
3.27 1: 2.28
3.79 1.16
After
70.58
0.14
5.30 + 0.64
3.17 0.67
71.07 0.28
2.02
1.42
2.86
0.90
05.53
0.20
4.46
1.53
8.54 2.12
VGNN
Refore
06.65
0.91
1.35
0.30
3.48 0.51
64.75 2.81
1.26
0.66
1.31
0.70
96, 08 0.47
3.97
0.98
0.68
After
66.43 +: 0.95
1.95 # 1.16
3.69 1.26
65.67 2.84
2.89
2.12
3.55 1.32
1.12 90:25
11.13
+: 5.75
17.70
7.91
FairSIN
Before
68.3 +
3.79
1.85
3.96
1.67
61.34 3.82
1.67
5.00
1.87
94.32 0.67
0.78
1.48
After
60.14 1: 5.66
4.33
2.81
3.05 4: 2.58
64.51 4: 2.7
1.80
0.95
4.14 A: 1.11
90.30 1.80
13.47 1: 3.09
37.31 5.74
observed across the other two datasets. Moreover, utility degradation is consistently minor, supporting the claim that NIFA is effective without significantly impacting model performance.
However, for fairness-aware GNN models, our results diverge from those reported by Luo et al. 2024). p. In the original study, NIFA causes large increases in Asp and Ago on the Pokec-z and Pokec-n datasets ranging from twofold to over twentyfold changes. In contrast, our reproduction shows only modest increases on these datasets, with fairness metrics even decreasing in three of the evaluated cases. These discrepancies may stem from implementation differences or incompatibilities across fairness-aware model codebases, as discussed in Section 6.1 Based on these results, we conclude that Claims 1 and 2 hold for classical models, but only partially hold for fairness-aware models.
Finally, while the original study also emphasizes NIFA's stealth- by restricting node injection to just 1% of the graph-wa we did not directly evaluate detectability (e.g., via anomaly detection methods). We adhere to the original perturbation budget, but consider the detectability aspect of Claim 2 outside the scope of our study.
Table 2: Accuracy and Fairness performance of attack launched by the different attackers. The results are reported in percentages (%).
Pokec-s
Pokec-n
DBLP
Aocuracy
Asp
Ago
Accuraey
Asp
Ago
Aceuracy
Agp
Ago
Clean
71.27 1 0.22
7.74 1 0.39
5,82 1 0.34
71.12 1 0.31
0.74 1. 0.47
2,08 1 0.81
95.85 1 1.61
3.81 0.31
1.91 1 0.75
E1sl2g stinkks on GNNs
TDGIA
66.56
1: 1.47
1.59
67.09
0.64 0.37
1.31
0.76
0.64
Feirness affaclo on CNN
FA-GNN
71.07 1 0.76
5,89 4: 2.23
4.41 # 1.98
71.11 4: 0.51
3.21 4: 0.62
2.57 4 1.61 96.39 4: 0.90
1.63 = 1:20
3.15 4: 1.48
NIFA
70.90 + 0.55
15.74 1.03
13.91
1.07
70.38
0.48
7.94 0.71
7.89 + 0.91 92.96 + 1.64
11.12 4.37
18.43 :: 4.10
Claim 3, which asserts that NIFA outperforms existing fairness and utility attacks, is supported in the original paper through comparisons with six baselines- three targeting utility (e.g., TDGIA) and three targeting fairness (e.g., FA-GNN).
To assess this claim, we reproduced one attack from each calegory: TDGIA and FA-GNN. Across all three datasets, NIFA consistently resulted in higher fairness degradation than either baseline. For example, on the Pokec-n datasct, the post-attack ASP scores were 0.64% for TDGIA, 3.21% for FA-GNN, and 7.94% for NIFA. These results align with the original paper's s reported trends, suggesting that NIFA remains a strong adversary under fairness mecriss.
However, we do not evaluate the full set of basclines due to the absence of accessible or stable implementations for several attacks. Integrating these into the NIFA pipeline would have required extensive adaptation and might not have reflected the original experimental setup. Moreover, we observed discrepancies in absolute
9

---

Published in Transactions on Machine Learning Research (06/2025)
metric values across all methods, which may be attributed to differences in codebases, evaluation pipelines, or random initialization. As such, we interpret our findings as partial support for the third claim, rather than a comprehensive benchmark.
While our results confirm that NIFA maintains model utility post-attack- consistent with Claim 2 2-we not explicitly assess its detectability. The original paper emphasizes NIFA's 's stealth by limiting perturbation to 1% of the graph. We follow this constraint, but do not evaluate detectability via graph anomaly detection or visibility metrics. We therefore consider the stealth-related component of Claim 2 outside the scope of this reproduction.
we did
5.2 Results beyond the original paper
We conducted two additional experiments to evaluate NIFA's 's attack on fairness: one extending the sensitive altribule to a multi-class setting and another analyzing its effects across different graph structures.
Multi-class sensitive attributes. The experimental results of NIFA on the augmented Pokec dalasets with multi-class sensitive attributes described in section 4.1 are presented in Table 3] These results include both the OvA and OvO approaches, where the mean and standard deviation across five random seeds are reported. Note that the fairness metrics correspond to the respective OvA and OvO versions.
Table 3: NIFA attack performance on augmented Pokec-z and Pokec-n datasets using GCN. Results are in percentages (%) with the best fairness metrics in bold.
Pokec-2
Pokec-n Asp
Accuracy
Asp
B0
Accuracy
A 80
OvA
Before
71.11 + 0.14
39.19
0.38
25.58 = 0.32
70.64
0.22
40.81
+ 1.08
26.51
3.12
After
70.88 + 0.63
42.84 + 0.46
38.89 + 0.76
71.12 + 0.70
46.25 + 0.82
34.52 + 1.37
OyO
Before
71.31 + 0.17
37.74 + 0.39
31.03
- 0.37
70.67 - 0.17
40.17
+ 0.85
26.05 + 2.86
After
71.29 + 0.49
39.88 0.32
39.88 + 1.15
70.67 0.43
43.98 + 0.33
33.36
0.34
The results of NIFA utilizing multi-class sensitive attributes indicate that the authors' claims remain valid in this setting. As observed in Table 3] the model's utility remains comparable to that of the model trained on the clean graph. Simultaneously, the fairness scores demonstrate that the allack was effective, further supporting the authors' claims. However, the absolute values are higher compared to the original results, which we attribute to the use of a different sensitive attribute that changes the feature set of the nodes.
Impact of graph structures. The impact of the NIFA attack on fairness under varying synthetic graph structures and homophily levels, as introduced in section 3.3 is presented in Figure 3. Results are reported as the mean and standard deviation across five random seeds. We focus minority group fraction of f. = 0.3, which most closely reflects the demographic ratios in the original real-world datasets (see subsection 4.1.g. and Appendix C).
We find that fairness degradation due to NIFA is most pronounced in the heterophilic-homophhie configu- ration fletc/Homo, = 0.36). One possible explanation is that this structure introduces conflicting clustering tendencies inter-group mixing in the heterophilic portion and tightly bound intra-group communities in the homophilic portion portion-allowing allowing injected nodes to exploit these dynamics and disrupt group fairness more effectively. However, we also observe high variance in this regime across random seeds, suggesting that structural sensitivity and initialization effects play non-negligible role.
Overall, the ASP and Ago metrics remain relatively stable across most homophily settings, with two ex- ceptions: poterophilic-hompphiie mixtures and highly homophilic graphs. The latter particular exhibits greater variability in fairness degradation, potentially due to the increased density and redundancy in intra- group links, which may amplify stochastic effects in the attack's impact. These findings consistent with
10

---

Published in Transactions on Machine Learning Research (06/2025)
30
20
32
Noom
Reurse
keArd
Hewashar
Ramee
Figure 3: Effect of edge homophily on fairness metrics before and after the NIFA attack (fm 0.3).
NIFA's behavior real datasets and highlight the need for further study of how structural graph properties modulate fairness vulnerability.
6
Discussion
6.1
Discussion of the results
The primary aim of this study was reproduce and critically evaluate the three main claims presented by [uo et al. (2024) in Are Your Models Still Fair? Fairness Allacks Graph Neural Networks via Node Injections. As discussed in iusbsecctio 5.1 we find partial support for Claims and 2, while Claim 3 remains
only partially supported.
Several factors likely contributed to discrepancies between our results and those reported in the original paper. First, only the classical GNN victim models were available in the official NIFA code repository, The fairness-aware models and baseline attacks had to be independently sourced and implemented, poten- tially introducing variations in architectural choices, preprocessing steps, or hyperparameters. Although followed the reference implementations closely, subtle differences may explain the divergence in absolute performance metrics. Second, the original results may have been influenced by undocumented random seeds and environmental configurations, further complicating exact replication.
Our extension of NIFA to multi-class sensitive attributes produces fairness degradation patterns consistent with those observed in the binary case. This suggests that the NIFA attack generalizes to more complex demographic settings and supports the use of extended fairness metrics beyond binary attribute distinctions.
Moreover, analysis of homophily indicates that graph structure significantly influences the succoss of fairness attacks. Graphs with mixed or high homophily exhibited greater fairness degradation, possibly due to denser intra-group connections that amplify bias propagation via injected nodes. These effects were most pronounced in synthetic graphs with heterophilic-homophiil structures, which also showed higher metric variability. These findings raise important questions about the structural factors inftuencing fairness vulnerabilities. Real-world datasets in the original NIFA study also exhibit high homophily, which may partially explain NIFA's 's effectiveness in those settings. However, due to limited data and high sced variance homophilic settings, we refrain from drawing strong conclusions this stage. A more systematic investigation of structural effects is left for future work.
Although NIFA is an adversarial attack method, understanding its behavior across varied structural and demographic conditions can inform the development of more robust, fair, and accountable GNN models. Fairness attacks expose structural weaknesses in deployed systems, particularly in socially sensitive domains such as hiring, lending, and healtheare. By extending evaluation to multi-class sottings and diverse graph structures, our work contributes to the ongoing effort identify and mitigate fairness vulnerabilities in graph-based machine learning.
11

---

Published in Transactions on Machine Learning Research (06/2025)
Given the limited range of bascline attacks assessed, emphasize that this study does not constitute comprehensive validation of NIFA's claimed superiority. While we confirm key trends and broaden the eval- uation scope, our findings should be interpreted as preliminary. Future work should incorporate additional baselines to more thoroughly assess fairness attack performance across scttings.
6.2 Reflection: What was easy? What was difficult?
Once initial dependency issues were resolved, reproducing the attack performance of NIFA on the classical GNN models was relatively straightforward. The official GitHub repository was well-organized, and the README provided clear guidance for setting up and running these experiments.
However, several challenges arose during reproduction. The original codebase required CUDA versions older than 11.6.0 and included deprecated dependencies, particularly in relation to DGL. As result, unable to fully adhere to the stated environment specifications and had to modify portions of the code to ensure compatibility with updated libraries.
Further difficulties emerged when reproducing results for fairness-aware models and baselines not included in the NIFA repository. The original paper referenced additional code repositorics but did not specify how integrate them with the NIFA attack pipeline. While NIFA's 's codebase was well-structured, the external repositories posed challenges, including adapting data-loading procedures for Pokec and DBLP, and modifying utility functions to align with the poisoned graph setup.
A detailed account of these implementation adjustments is provided in subsection 4.2
6.3 Communication with original authors
We contacted the original authors of the paper twice to seek clarification on aspects of their implementation of the fairness-aware models and baseline attack models. The inquiries focused understanding implemen- tation details, such as confirming the configuration of key paramcters, verifying any updates to the official code repository, and clarifying certain techniques used in experiments that were not fully documented in the paper. Both inquiries received prompt and helpful responses, for which their clarifications and timely assistance are greatly appreciated.
12

---

Published in Transactions on Machine Learning Research (06/2025)
References
Enyan Dai and Suhang Wang. Say to the discrimination: Learning fair graph neural networks with limited sensitive attribute information. In Proceedings of the 14th ACM International Conference Web Search and Data Mining, WSDM '21, pp. 680-688, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450382977. doi: 0..146//3437663.4447777 URL httpss//doii ftttp:://do..or/100/ 1145/
3437963 3441752
Yushun Dong, Ninghao Liu, Brian Jalaian, and Jundong Li. Edits: Modeling and mitigating data bias for graph neural networks. In Proceedings of the ACM Web Conference 2022, WWW '22, pp. 1259 1259-1269, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450390965. doi: 10.1145/
1145/34854447 3485447.3512173. URL https://doii /14p://///0..ogg.....111///77777 354111776447333111777 3512173
Manh Khoi Duong and Stefan Conrad. Towards Fairness and Privacy: A Novel Data Pre-processing Op- timization Framework for Non-binary Protected Attributes, pp. 105-120. Springer Nature Singapore, De- cember 2023. ISBN 9789819986965.
doi: 10.1007/978-991-999-998
8. URL http:://x.ddi.oo//// org..1//1........
90079978-981-99-2
Ergin Elmacioglu and Dongwon Lee, On six degrees of separation in dblp-db and more. SIGMOD Rec., 34 22:333-40) June 2005. ISSN 0163-5808. doi: 10.144/108378411037911 URL httt/t/t////oii ftttps://doi.org/1.... 1083784.1087791
Lisette Espín-Noboa, Claudia Wagner, Markus Strohmaier, and Fariba Karimi. Inequality and inequity in network-based ranking and recommendation algorithms. Scientific reports, 12(1):2012, 2022.
Antonio Ferrara, Lisette Espin-Noboa, Fariba Karimi, and Claudia Wagner. Link recommendations: Their impact on network structure and minoritics. In 14th ACM Web Science Conference 2022, WebSei '22 pp. 228 238. ACM, June 2022. doi: 011445/551247.355 583.. URL http: /ttt:://x..r www.ddoi.rrrooo... 1rg/100.114///512241 3531583
Johannes Gasteiger, Aleksandar Bojchevski, and Stephan Günnemann. Predict then propagate: Graph neural networks meet personalized pagerank, 2022. URL http:///aaxxiv..g/abbb////////// 05997
M. Gori, G. Monfardini, and F. Scarselli. A new model for learning in graph domains. In Proceedings. 2005 IEFE International Joind Conference on Neurul Networks, 2005., volume 2, pp. 729-734 734 vol. 2, 2005. doi: 10. (0.1109/ICCNN.000 153554944.0..
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning large graphs. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Newral Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https://proceeddng/ neurips neur.ps.cc/paperr ft1es//pppet/217///ill Beddasoossaaddobbeeerrrrrreeeeed pdf.
Hussain Hussain, Meng Cao, Sandipan Sikdar, Denis Helic, Elisabeth Lex, Markus Strohmaier, and Roman Kern. Adversarial goter- link injection degrades the fairness of graph neural networks. In 2022 IEEE International Conference on Data Mining (ICDM), pp. 975-980, 2022. doi: 10.1109/CCDMM54444422222 00117.
Mingxuan Ju, Yujie Fan, Chuxu Zhang, and Yanfang Ye. Let graph be the go board: Gradient-free node injection attack for graph neural networks via reinforcement learning. Proceedings of the AAAI Conference on Artificial Intelligence, 37)443388 4390, Jun. 2023. doi: 10. 10.1609/aaai, /tt.//aai...3...5.... 25558.. URL https://///// aaai. .rgg index.idexxx/// http//AAA/aatticle/reeee//////
Jian Kang, Yinglong Xia, Ross Maciejewski, Jiebo Luo, and Hanghang Tong. Deceptive fairness attacks on graphs via meta learning, 2023. URL http::::///xxxv.or//b/bbb 2310. 15653.
Thomas N. Kipf and Max Welling Semi-supervised classification with graph convolutional networks, 2017.
URL
13

---

Published in Transactions on Machine Learning Research (06/2025)
Emmanouil Krasanakis, Eleftherios crrromitros-xiouffi Symeon Papadopoulos, and Yiannis Kompatsiaris. Adaptive sensitive reweighting to mitigate bias in fairness-aware classification. In Proceedings of the 2018 World Wide Web Conference, WWW '18, pp. 853 862, Republic and Canton of Geneva, CHE, 2018. International World Wide Web Conferences Stocring Committoe. ISBN 9781450356398. doi: 10.1145/ 3178876.3186133. URL https: http:////oo...///.... 1114/31711886666 3186133
Hongyi Ling, Zhimeng Jiang, Youzhi Luo, Shuiwang Ji, and Na Zou. Learning fair graph representations via automated data augmentations. In The Fleventh International Conference on Learning Representations, 2023. URL
Yang Liu, Xiang Ao, Fuli Feng, and Qing Hc. Ud-gnn: Uncertainty-aware debiased training on semi- homophilous graphs. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '22, pp. 1131-1140, New York, NY, USA, 2022. Association for Computing URL https:///oi.orr:/:
Machinery. ISBN 9781450393850. doi: 011114//5534778.35338333 3534678.35399 3539483
Zihan Luo, Hong Huang, Yongkang Zhou, Jiping Zhang. Nuo Chen, and Hai Jin. Are your models still fair? fairness attacks on graph neural networks via node injections, 2024. URL https://arxii httlaaaiivvoor.oraeee abs//406.
03052
Erasmo Purificato, Ludovico Boratto, and Ernesto William De Luca. Toward a responsible fairness analysis: From binary to multiclass and multigroup assessment in graph neural network-based user modeling tasks. Minds and Machines, 34(3):33, Jul 2024. ISSN 1572-8641. doi: 101007//11022000--9885552 URL
Julien Rouzot, Julien Ferry, and Marie-José Huguet. Learning optimal fair scoring systems for multi-class classification. In 2022 IEEE 94th International Conference on Tools with Artificial Intelligence (ICTAT), pp. 197 204. IEEE, 2022.
Yiwei Sun, Suhang Wang, Xianfeng Tang, Tsung- Yu Hsieh, and Vasant Honavar. Adversarial attacks graph neural nelworks via node injections: A hierarchical reinforcement learning approach. In Proceedings of The Web Conference 2020, WWW '20, pp. 673 683, New York, NY, USA, 2020. As- sociation for Computing Machinery. ISBN 9781450370233. https: org/ 10. 100o00161133333.... 3380149
doi: 10. N0.1145/3366423.333000
URL
Lubos Takac and Michal Zabovsky. Data analysis in public social networks. In International scientific conference and international workshop present day trends of innovalions, volume 1, 2012.
Jihong Wang, Minnan Luo, Fnu Suya, Jundong Lú, Zijiang Yang, and Qinghua Zheng. Scalable allack on graph data by injecting vicious nodes. Data Mining and Knowledge Discovery, 34:1363 1389, 2020.
Yu Wang, Yuying Zhao, Yushun Dong, Huiyuan Chen, Jundong Li, and Tyler Derr. Improving fairness in graph neural networks via mitigating sensitive attribute leakage. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '22 pp. 1938-1948, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450393850. doi: 10.1145 01145/35346688.533900 URL https://doi.org/ 1114445554688.5553888
Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. Simplifying graph convolutional networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, PMLR, 09 15 Jun 2019. URL https: htttps::/proceeedinggg mlr. press/v97/wul9e html
pp.
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A comprehensive survey graph neural networks. IEEE transactions on neural networks and learning systems, 32(1):4-24, 2020.
Cheng Yang, Jixi Liu, Yunhe Yan, and Chuan Shi. Fairsin: Achieving fairness in graph neural networks through sensitive information neutralization, 2024. https://t://arxtt http://aaaiiv..r//////200. 12474
14

---

Published in Transactions on Machine Learning Research (06/2025)
Binchi Zhang, Yushun Dong, Chen Chen, Yada Zhu, Minnan Luo, and Jundong Li. Adversarial attacks fairness of graph neural networks. In The Twelfth International Conference on Learning Represenlations, 2024a. URL http::///peeeeeeww..eettt net/foruuu2ddqq3NNrweee
Binchi Zhang, Yushun Dong, Chen Chen, Yada Zhu, Minnan Luo, and Jundong Li. Adversarial attacks fairness of graph neural networks, 2024b. URL https://arxiy/ www.:oorg/bba/////// wttp://.wwww.bb//....1222... 13822
Xu Zou, Qinkai Zheng, Yuxiao Dong, Xinyu Guan, Evgeny Kharlamov, Jialiang Lu, and Jie Tang. Tdgia: Effective injection attacks on graph neural networks. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Dala Mining, KDD '21, pp. 2461 2471, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450383325, doi: 10. (0.1145/4447648.447733.. URL httpss//dol..rrg
10. 34673444344444144
15

---

Published in Transactions on Machine Learning Research (06/2025)
Appendices
A Feature optimization
The feature optimization background directly adapted from Luo et al. (2024)
Classification loss. To ensure the utility of the victim model remains unaffected and avoid detec- tion, the authors employ the cross-entropy loss the first objective function:
LCE
Σ NE
Yu log hu
where yrr denotes the original training node set,yu is the true label node u, and hy is the output logits of node u.
Fairness loss. To maximize unfairness, two fairness loss functions defined based the metrics Asp and Ao:
LSP = -1
Σ ha
Σ
hall
1 p2
"EVo
"EVT
LEO =
bay
Σ
sEr 0,9
V
VEY
WEVE. 1.1
where h2 E ED represents the raw output for node u, and hu, is the output for class y. V denotes the set of training nodes with sensitive attribute i and label y.
Constraint of feature. To distinguish information introduced by injected nodes between sensitive groups during message propagation, the authors define feature constraint:
Ler = -1 VV.00 nEV1.0
Σ X1,
Σ ueV1a
X x
VV.11
where Vr, is the set of injected nodes connected to sensitive group i. Overall loss. The overall loss for optimizing the injected nodes' features is defined as:
L LoE LCF + B- (LsP LEO),
where a and are hyperparameters controlling the weight of each term.
B Additional model descriptions
B.1 Victim models
A GNN is based on convolutions that operate directly on graphs. All local node information is aggregated to generate graph ixpresentations, enabling semi-supervised learning on graph-structured data.
GCN Kiipf & Welling, 2017):
GraphSAGE (Hamilton et al., 2017): : A framework for inductive node embedding that uses node
features to learn an embedding function that is able to generalize unseen nodes. By integrating node features into the learning algorithm, the model captures the topological structure of each nodc's neighborhood.
APPNP (Gasteiger et al.) 2022): This model uses GCN and PageRank develop framework called personalized propagation of neural predictions (APPNP), which utilizes a large adjustable neighborhood for classification and can be integrated with any neural network.
16

---

Published in Transactions on Machine Learning Research (06/2025)
Motivated by the inherent complexity of GCNs, this model simplifies them by removing non- non-linearitie between layers, reducing the function to a single linear transformation.
SGC (Wu et a 2019):
FairGNN (Dai Wang 2021): : This model is designed to mitigate bias in GNNs while preserving high
node classification accuracy by utilizing graph structures and minimal sensitive information.
FNiN (Wang et a 2022): Motivated by observation of sensitive information leakage in GNNs, FairVGNN creates fair node fealures by automatically detecting and masking sensitive-correlatee fealures, taking the variation of correlations after feature propagation into account.
FairSIN (Yang al., 2024): Current methods address GNNs bias in predictions due to sensitive attributes by filtering out sensitive information. This has the potential of filtering out non-sensitive feature information, resulting in trade-off between predictive performance and fairness. To overcome this, FairSIN incorporates Pairness-facilitate
Features (F3) into node features or representations prior to message passing, with the goal of statistically neutralizing bias while retaining non-sensitive information.
B.2 Baseline attacks
TDGIA (Zou et al., 2021): An injection attack based on topological vulnerability implements a defective edge strategy to identify the original nodes for connection with injected nodes. It then formulates a smooth Teature optimization lo generate features for the injected nodes.
FA-GNN (Hussain et al. 2022): This study highlighted the susceptibility of GNN models to adversarial fairness attacks by empirically identifying several edge injection strategies, which could compromise GNN fairness with only minimal impact on utility.
C Dataset specifics
Table Statistics of the datasets. Adapted from
Luo al. (0024)
Dataset
Pokec-z
Pokec-n
DBLP
# of nodes
67,796
66,569
20,111
# edges
617,958
517,047
57,508
feature dim.
276
265
2530
# of labeled nodes
10,262
8,797
3,196
homophily rate
0.95
0.96
0.79
minority rate
0.35
0.29
0.18
D Synthetic dataset configuration
D.1 Graph generation
First, the number of nodes N was set, and proportion f' of these nodes was randomly assigned to the minority group, while the remainder formed the majority group. Each node received an activity score derived from power-law distribution, parameterized separately for the majority (7M) and minority (Tm) groups, reflecting their differential levels of connectivity potential.
Edges were crcated itcratively until the desired edge density d was achieved. For each itcration, a source node v; was selected according to its activity level, and a target node vj was chosen with probability proportional to the product its in-degree kmm and the homophily hi between the source and target. The homophily values hMM and hyom governed the likelihood of intra-group connections for the majority and minority groups, respectively. The expected number of edges in the graph was scaled by the parameter d, ensuring that the total number of edges approximated d x N (N
17

---

Published in Transactions on Machine Learning Research (06/2025)
D.2 Experimental settings
Multiple combinations of homophily rates and minority fractions explored. Exploration ranges are based on the ranges used in the original paper (Espín-Noboa et al. 2022), well as explorations by Ferrara
et al. 2022).
The homophily values among majorities hMM and minorities hmm were sampled from the set {0.0, 0..,,..,.... 0.2....,1..fff 1.0), while the fraction of minority nodes Sm was set to {0.1, 0.5}. For each combi- nation of these parameters, networks were generated and analyzed under fixed conditions.
The following parameters were held constant across all experiments. The number of nodes set to N = 2000, and the edge density d was maintained at 0.0015, ensuring sparse graphs suitable for modeling real-world networks. The node activity followed power-law out-degree distribution with parameters plo, = plom = 2.5, reflecting heterogeneous connectivity patterns across groups. The number of feature dimensions needed for the proxy task was set to n = 100. Finally, all NIFA attacks were conducted using a perturbation rate of 1,, as per the original paper.
E Results from the original paper
Table 5: Original authors' attack performance of NIFA on different victim GNN models. The results are reported in percentage (%). Adapted from Luo et al. (2024).
Pokee-z
Pokec-n
DBLP
Accussey
Asp
Aso
Accuracy
Aep
Ago
Accuracy
Agp
Ago
GCN
1.13 1.21
+ 1.28
70,02 0.86
0.88 0.62
2.44
96.88
0.34
0.75
Afber
70.50 4: 0.30
17.30
1.16
15.59
1.08
70.12 à: 0.37
10.10 4: 2.10
9.85 4: 1.97
93.37 4: 1.48
13.49 1: 2.83
20.33 1: 3.82
GraphSAGE
Beforo
4.20 0.84
3.46
68.77
0.34
1.65
2.34
1.04
96.58 + 0.29
4.27
1.49
0.91
After
70.05 # 1.25
6.20 + 1.63
4.20 + 1.77
68.93
1.19
3.32 #: 1.88
3.56 #: 1.91
53.92 + 0.74
10.16
2.24
16.65
+ 3.30
APPNP
Before
69.79
0.42
6.63 1: 1,25
5.07
1: 1,26
08.73
0.61
3.39
0.28
3.71 4: 0.28
96.58 1: 0.38
3,98 1: 1.18
2:20 1: 1.08
After
67.83
0.70
18.44
1.41
16.85
1.50
67.00
0.76
+ 13.47
13.52 + 92.46 + 0.94
13.88
3.20
20.20
4.25
SGC
Beloro
69.00 + 0.00
7.28 + 1,50
5.45
+ 1.42
06.95
1.69
2.74 + 0.85
3.21 + 0.78
06.13 + 0.48
4.70 + 1.26
3.11 + 1.24
After
17.65
1.01
16.09
1.06
66.72
1.21
10.59 + 10.67 +
92.86 + 13.88
3.37
20.25
4.44
FairGNN
Belore
68.73 1: 1.12
1.89 1: 0.63
1.51
1: 0.17
69.41 1: 0.66
1.42 1: 0.35
2.32 :: 0.57
93.12 :: 1.23
1.95
1: 0.99
3.09 1: 1.81
After
5.71
4.22
60.97
0.42
5.81
5.77
5.89
3.82
PaNNVGNN
Before
68.57 + 0.45
3.79 + 0.51
2.59 + 0.59
67.77 1.00
1.90 + 1.23
3.10 + 1.20
96.18 + 0.54
1.00 + 0.52
2.01 + 1.05
Afler
07.65
0,38
11.01
2.79
9.28 1: 2.87
65.74 1.42
3.51
1.51
3.65
1.56
91.56 1: 1.18
7.96
L.49
13:57
2.57
FairSIN
Before
67.33 1: 0.22
1.73 1: 1.19
2.61 1: 1.44
67.18
0.30
0.39
0.89
2/40 1: 1.02
94.72 1: 0.62
0:23 1: 0.15
0:45 1: 0.16
After
66.55 0.44
9.48 + 2.62
10.39
1.06
66.20 0.12
11,82 + 0.75
14.58 + 0.22
92.46 + 0.32
10.90
2.12
23.65 7.77
Table 6: Original authors' accuracy and fairness performance of attack launched by the different attackers. The results are reported in percentage (%). Adapted from Luo et al. 20244
Pokec-s
Pokec-n
DBLP
Accuracy
Asp
Ago
Avcuracy
Asp
Ago
Accuracy
Asp
Ágo
Clean
71.22 1: 0.28
7.13 4 1.21
5,10 à: 1.28
70.92 1: 0.66
0.88 1: 0.62
2.44 1: 1.37
95.88 4: 1.61
3.84 4 0.54
1.91 4: 0.75
AFGSM
67.01
0.24
3.07
3.45 0.22
88:21
0:23
5.35 0.15
5.68 0.14
95.38 0.30
5.44
3.48
2.78
TGDIA
62:20 : 0.94
1.66 + 0.16
0.77 + 0.10
63.37
+ 0.08
7,28
0.35
6,96
0.34
93.42
+ 0.20
0.93 + 0.70
1.82 + 0.87
G2A2C
39.41 2: 0.94
6.89 1 0.91
6.11 :: 0.48
31.30 4: 1.71
2.23
1.10
3.76 1.04
86.28
0.25
1.21 4 0.66
3.80 4: 0.12
FA-GNN
69.80
0.48
8.62 + 1.21
8.67 1.28
70.80
0.97
2.64 0.76
3.45 0.54
95.48
0.48
3.32 + 1.65
8.74 +: 1.23
FATE
94.87 1: 0.41
3.62 4. 1.49
2.12 4: 1.01
C-PairAttack
95.12
0.38
6.80
0.59
2.94
1.10
NIFA
70.50 : 0.30
17.36 :: 1.16
15.59 # 1.08
70.12 + 0.37
10.10 + 2.10
9.85 1.97
93.37 +: 1.48
13.49 + 2.83
20.33 =: 3.82
F Exploratory analysis
This scction presents an exploratory analysis investigating how varying levels of homophily and minority group fractions influence fairness outcomes under the NIFA attack. As graph homophily governs the tendency of nodes to connect with others sharing the same attributes, changes in homophily may systematically affect
18

---

Published in Transactions on Machine Learning Research (06/2025)
how injected nodes propagate bias. We assess whether fairness metrics seerriss-spefffccllyyy Aep and Ago roo-respond predictably to these structural changes and whether this response interacts with the proportion of minority group members.
The analysis combines homophily settings (ranging from highly heterophilic to highly homophilic) with vary- ing minority group fractions (Sm 0.3, This design allows us evaluate not only the independent effects of each variable but also potential interaction effects between minority size and structural cohesion.
Minc
0J 02 C4 A2
Figure Impact of homophily and minority fraction on fairness metrics under NTFA.
As shown in Figure 4 no clear monotonic trend was observed in Asp or AgO as a function of homophily level alonc. In particular, fairness degradation did not increase or decrease consistently across the homophily spectrum. This suggests that while homophily affects fairness in some configurations (e.g., mixed or extreme values), its relationship with fairness degradation is not linear straightforward.
In contrast, the minority fraction demonstrated more consistent effect. Graphs with lower minority repre- sentation (Sm = 0.1) exhibited significantly greater shifts in fairness metrics after the NIFA atlack, particu- larly in settings with moderate to high homophily. This aligns with the intuition that when minority nodes are sparsely represented, targeted node injections can more easily skew group-level statistics by amplifying the representation of the majority group. This suggests that group imbalance exacerbates vulnerability to fairness attacks, even when homophily levels are held constant.
Although preliminary, these findings highlight the need for deeper investigation into how structural fac- tors (e.g., community structure, degree distribution) and demographic variables (e.g., group size imbalance) jointly affect fairness robustness. Future work could explore more diverse synthetic graph generators, intro- duce continuous variations in homophily and group sizes, and test interactions with different fairness-aware training objectives.
summary, this analysis provides initial evidence that minority representation plays a larger and more systematic role than homophily in dotermining fairness vulnerability under node injection attacks. However, more granular and statistically powered studies required to draw firm conclusions about these interactions.
19


Reviews:
[{"review_id": "XBmVBbNU4C", "reviewer": "Reviewer_nuQd", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The main contribution of the paper lies in reproducing and extending the recent paper by Luo et al. Fairness in the context of GNNs is an important research topic, and reproducibility studies are certainly valuable in guiding future research. The paper finds that the key claims made by Luo et al. are to a large extent accurate, although it also reports certain inconsistencies. Those findings are certainly relevant for the community going forward, and deserve to be highlighted. Furthermore, the paper conducts some further experiments concerning natural extensions to the paper of Luo et al. Understanding how effective NIFA is in the presence of multi-class sensitive attributes is a natural next step. I also found the study relating the effectiveness of NIFA to the homophily level interesting. Taken as a whole, the paper contains some interesting results that deserve to be highlighted. The paper is well-written, providing adequate background and accurately placing its contributions in the context of the existing literature.\n\nOn the negative side, it is unclear whether the results are significant enough for this venue. There are very few challenges in reproducing the results of Luo et al. Reproducibility studies are more impactful when examining well-established and influential papers; the result of Luo et al. is very recent and not well-cited, so it's unclear whether this paper will have an impact going forward.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper examines the fairness properties of graph neural networks (GNNs), in particular, the node injection-based fairness attack (NIFA) recently proposed by Luo et al. (2024). The paper aims to reproduce the main claims made by Luo et al., most of which are corroborated by this paper. They also extend the work of Luo et al. to account for multi-class sensitive attributes while also investigating the role of homophily in NIFA. They find that NIFA is effective even in the presence of multi-class sensitive attributes. Moreover, they find that homophily correlates with how effective NIFA is.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Two minor points:\n\n- Typo on page 1: \"ampl amplefies\"\n- Eqs. (3) and (4) should have absolute values, right?", "broader_impact_concerns": "No broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "MOmG0yR5q2", "reviewer": "Reviewer_h4eE", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros: This paper is well-written and clear. The message is obvious, and the discussion looks sound.\nCons: This paper provides no new algorithm and only deals with reproducibility issues. I'm afraid this paper's message is not big on ML society. It can be a good paper if the reproducibility issue is considered an important fold.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper aims to check the reproducibility of the NiFA algorithm (attack on the fairness of GNN). The studies address the NIFA paper's claims thoroughly and add some experiments concerning the influence of homophily on graphs. This paper validates the NIFA's claims, except for some setups, such as fairness-aware models. Multi-class and the influence of homophily are examined carefully. It implies the susperiority of NiFA compared to others, and the influence of homophily cannot be little. Furthermore, the paper discusses the reproducibility issues of NIFA in detail.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I suggest the following issues.\n\n1. The analysis of the influence of homophily is too short. More insightful discussion or analysis can strengthen the paper. What's the reason the case of Hete /Homo(0.36) shows the high value in Fig3. \n2. More datasets are required to generalize the NIFA's performance. Therefore, many datasets should be examined if possible.\n3. The scope of the task should be clarified kindly. Is the task a node classification? Is the edge classification valid in this paper?", "broader_impact_concerns": "There is no broad impact statement. However, the fairness algorithm is related to ethical issues such as fairness in decision-making. Also, the topic of NiFA can be related to the security issue.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "olAzzenTuv", "reviewer": "Reviewer_JxUV", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The most notable strength of this reproducibility study is that it goes beyond simple reproduction by extending the evaluation to multi-class sensitive attributes and different graph homophily levels, offering deeper insights into NIFA’s generalizability and robustness in more realistic and diverse scenarios. Another strength of this paper is that it provides a well-documented methodology, including detailed descriptions of dataset handling, model adjustments, evaluation metrics, and attack configurations, which enhances the reproducibility and credibility of their findings. Thus, the paper carefully differentiates between partially and fully reproduced claims, discussing potential reasons for discrepancies, such as differences in code availability or experimental environments.\n\nThe paper also has limitations. Most notably, only two of the six baseline attacks from the original study were reproduced (TDGIA and FA-GNN) due to time constraints. This limits the strength of the comparative evaluation and weakens the assessment of NIFA’s relative performance. An additional observation is that, since some discrepancies for the fairness-aware models and the  running fairness-aware models may be due to differences in code implementation. Thus, as currently structure, the paper does not fully resolve the divergences due to methodological consideration.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The study aims to reproduce the fairness degradation effects of the Node Injection-based Fairness Attack (NIFA) on Graph Neural Networks (GNNs), and to extend its evaluation to multi-class sensitive attributes and varying graph homophily levels.\nThe study evaluates three specific claims of NIFA, a gray-box fairness attack on GNNs, by replicating the original experiments using three real-world datasets (Pokec-z, Pokec-n, DBLP) and four classical GNN models (GCN, GraphSAGE, APPNP, and SGC), as well as fairness-aware models (FairGNN, FairVGNN, and FairSIN). The three claims evaluated are: NIFA's fairness degradation in GNNs while maintaining negligible utility loss, NIFA state-of-the-art performance with scalability and minimal detectability, and NIFA's outperformance over existing fairness and utility attack baselines across multiple datasets. NIFA degrades fairness by injecting synthetic nodes during training, leveraging two key principles: uncertainty maximization using Bayesian GNNs, and increasing homophily by linking injected nodes within the same sensitive group. Features of these nodes are optimized via a multi-objective loss function balancing classification accuracy and fairness degradation. \n\nThis paper also extends the original scope by adapting NIFA to multi-class sensitive attributes using One-vs-All and One-vs-One generalizations of fairness metrics, and by evaluating its effectiveness across synthetic graphs with different homophily levels. The experiments also include comparisons with fairness (FA-GNN) and utility-based (TDGIA) attack baselines. Results are averaged over five runs using a consistent hardware setup, with minor code adjustments due to dependency mismatches. The study assesses both fairness (∆SP and ∆EO) and utility (accuracy), analyzing pre- and post-attack differences to determine NIFA’s robustness and generalizability across configurations.\n\nThe paper confirms that NIFA significantly degrades fairness in classical GNNs with minimal utility loss, though its superiority over baseline attacks is inconclusive. The attack remains effective with multi-class sensitive attributes and under varying homophily, though highly homophilic graphs show increased metric variability, suggesting structural sensitivity.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I would suggest to clarify some details of the study. For instance, indicating the specific obstacles in reproducing the remaining four baseline attacks (e.g., AFGSM, G2A2C, FATE, G-FairAttack) or completing these analysis will provide a more thorough analysis. A time constraint does not sound like a valid scientific justification. Also, it could be helpful to clarify whether pre-trained models or author-provided outputs could be used as a proxy to compare NIFA’s performance. \n\nOther information that could be useful for this analysis are, first, the implementation details or configurations that differed between the original and reproduced fairness-aware models and, second, how sensitive are the fairness metrics (∆SP and ∆EO) to hyperparameters or random seed initialization in these models.", "broader_impact_concerns": "Since this is a reproducibility study, as I indicated above, the main concern is that given the limited set of attacks that are evaluated, the paper can be mistaken as a full validation of NIFA. The details of the atacks studied should be clarified further in the abstract and introduction. This could limit the quality of this paper's broader impact.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)