Fair Bilevel Neural Network (FairBiNN): On Balancing fairness and accuracy via Stackelberg Equilibrium
Authors: Mehdi Yazdani-Jahromi
Ali Khodabandeh Yalabadi
Amirarsalan Rajabi
Aida Tayebi
Ivan Garibay
Ozlem Garibay
Venue: NeurIPS 2024 poster (2024)
Keywords: Fairness
Bilevel optimization
Pareto optimal
Stackelberg
Demographic parity
Fairness-accuracy trade-off
Primary area: fairness

Abstract: The persistent challenge of bias in machine learning models necessitates robust solutions to ensure parity and equal treatment across diverse groups, particularly in classification tasks. Current methods for mitigating bias often result in information loss and an inadequate balance between accuracy and fairness. To address this, we propose a novel methodology grounded in bilevel optimization principles. Our deep learning-based approach concurrently optimizes for both accuracy and fairness objectives, and under certain assumptions, achieving proven Pareto optimal solutions while mitigating bias in the trained model. Theoretical analysis indicates that the upper bound on the loss incurred by this method is less than or equal to the loss of the Lagrangian approach, which involves adding a regularization term to the loss function. We demonstrate the efficacy of our model primarily on tabular datasets such as UCI Adult and Heritage Health. When benchmarked against state-of-the-art fairness methods, our model exhibits superior performance, advancing fairness-aware machine learning solutions and bridging the accuracy-fairness gap. The implementation of FairBiNN is available on https://github.com/yazdanimehdi/FairBiNN.

Fair Bilevel Neural Network (FairBiNN): On Balancing fairness and aceuracy via Stackelberg Equilibrium
Mehdi Hizdani-Jahromi Department of Computer Science University of Central Florida
Ali Khodabandeh Yalabadi Department of Industrial Engineering University of Central Florida Orlando, FL 32816 yal valabadi@ues @ucf edu
Orlando, FL 32816
yazdani@uch edu
AmirArsalan Rajabi Department of Computer Science University of Central Florida
Aida Tayebi
Department of Industrial Engincering University of Central Florida Orlando, FL 32816 ai5307370ucf edu
Orlando, FL 32816 am954283@ucf edu
Tvan Garibay
Ozlem Ozmen Garibay Department of Industrial Engineering University of Central Florida Orlando, FL 32816 ozlemOucf edu
Department of Industrial Engineering
University of Central Florida Orlando, FL 32816 agaribay@uch edu
Abstract
The persistent challenge of bias in machine learning models necessitates robust solutions to ensure parity and equal treatment across diverse groups, particularly classification tasks. Current methods for mitigating bias often result in information loss and an inadequate balance between accuracy and fairness. To address this, we propose novel methodology grounded in bilevel optimization principles. Our deep learning-bar based approach concurrently optimizes for both accuracy and fairness ob- jectives, and under certain assumptions, achieving proven Pareto optimal solutions while mitigating bias in the trained model. Theoretical analysis indicates that the up- per bound on the loss incurred by this method less than or equal to the loss of the Lagrangian approach, which involves adding regularization term to the loss func- tion. We demonstrate the efficacy our model primarily on tabular datasets such as UCI Adult and Heritage Health. When benchmarked against state-of-the-a fair- ness methods, our model exhibits superior performance, advancing fairness- aware machine learning solutions and bridging the accuracy-fairess gap. The implemen- tation of FairBiNN is available on http:///gihbbbcommyyaaaaeee////
Introduction
Artificial intelligence and machine learning models have seen significant growth over the past decades, leading to their integration into various domains such as hiring pipelines, face recognition, financial services, healthcare, and criminal justice. This widespread adoption algorithmic decision-making has raised concerns about algorithmic bias, which can result in discrimination and unfaimesss towards minority groups. Recently, the issue of fairness artificial intelligence has garnered considerable
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

attention from interdisciplinary research communities, addressing these ethical concerns [50]. Several definitions of fairness have been proposed to tackle unwanted bias in machine learning techniques. These definitions generally fall into two categories: individual fairness and group fairness. Individual fairness ensures that similar individuals are treated similarly, with similarities determined by past information [20] 73]. Group fairness, on the other hand, measures statistical equality between different subgroups defined by sensitive characteristics such as race or gender [76] 26]. In this paper, we focus on group faireess, which we will refer to simply fairesss from this point onward.
Fairness approaches in machine learning are commonly categorized into three groups: (1) Pre-process approaches: These methods involve changing the data before training to improve faimesss, such as reweighing labels or adjusting features to reduce distribution differences between privileged and unprivileged groups, making it harder for classifiers to differentiate them [4 42 22 63] Generative
adversarial networks were also utilized to produce unbiased datasets by altering the generator network's s value function to balance accuracy and fairness (53] (2) In-process approaches: These methods modify the algorithm during training, for instance by adding regularization terms to the objective function to ensure fairness. Examples include penalizing the mutual information between protected attributes and classifier predictions to allow a trade-off between fairness and accuracy (35), and adding constraints to satisfy proxy for equalized odds [74] 75). (3) Post-process approaches: These techniques adjust the outcomes after training, such as flipping some outcomes to improve fairness (26). or using different thresholds for privileged and unprivileged groups to optimize the
trade-off between accuracy and fairness [4] [1].
In this work we targeted the in-process bias mitigation category. Traditionally, the fairness multi- criteria problem has been addressed using Lagrangian optimization, wherein the objective function is weighted sum of the primary and secondary loss functions. While this approach allows for the explicit incorporation of fairness constraints through Lagrange multipliers, it may overlook the complex interdependencies between the primary and secondary objectives.
promising alternative to the Lagrangian the bilevel optimization approach which offers several advantages. By formulating the problem as a hierarchical optimization task, we can explicitly model the interactions between the primary and secondary objectives. This allows us to capture the nuanced dynamics of fairness optimization and ensure that improvements in one objective do not come at the expense of the other.
In summary, we introduce novel method that can be trained on existing datasets without requiring any alterations to the data itself (data augmentation, perturbation, etc). Our methodology provides a principled approach to addressing the multi-criteria fairness problem in neural networks. Through rigorous theoretical analysis, we formulated the problem as a bilevel optimization task, proving that it yields Pareto-optimal solutions. We derived an effective optimization strategy that is at least as effective as the Lagrangian approach. Empirical evaluations on tabular datasets demonstrate the efficacy of our method, achieving superior results compared to traditional approaches.
2 Related works
Multi-objective optimization in neural networks involves optimizing two or more conflicting objec- tives simultaneously. Faimess problems are inherently multi-objective in nature, as improvements in one objective (e.g., enhancing fairness) often come at the expense of another objective (e.g.. improving accuracy). Several optimization techniques in neural networks have been employed balance accu- racy and fairness. Classic Methods transform these objectives into a single objective by combining them, typically using a weighted sum where each objective is multiplied by weight that reflects its importance. Adding Regularization and penalty terms are the most common methods that incorporate fairness constraints (c.g.. demographic parity, equal opportunity) directly into the loss function. penalizing disparities in prediction errors across demographic groups or any other unfair behavior. To reduce variation across different groups, Zafar al. (74) proposes "disparate mistreatmen"". new notation for fairness, and standardized the decision bounds of a convex margin- based classifier. Adversarial debiasing and Fair Representation Learning are two examples of these techniques, which encourage the model to generate fair outcomes by introducing penalty term based on an adversarial network or a representation learning framework that invariant to protected attributes, respectively. Zhang et al. [77) addressed bias by limiting an adversary's ability to infer sensitive characteristics from predictions. Avoiding the complexity of adversarial training, Moyer et 447 used mutual infor- mation to achieve invariant data representations concerming specific factors. Song et al. [61] proposed

---

an information-theowee method that leverages both information- theooretict and adversarial approaches to achieve controllable fair data representations. adhering to demographic parity. By incorporating a forget-gat gate similar to those in L.STMs, Jaiswal et al. [0] introduced adversarial forgetting to enhance fairness. Gupta et al. [25] utilized certain estimates for contrasting information to optimize theoretical objectives, facilitating suitable trade-offs between demographic parity and accuracy in the statistical population. Lagrangian optimization techniques are a subset of these techniques that use Lagrange multipliers or other similar techniques incorporate constraints directly into the objective function, turning constrained optimization problems into unconstrained ones. Agarwal et al. 22] proposes an approach for fair classification by framing the constrained optimization problem game where one player optimizes the model parameters, and the other imposes the constraints, and Lagrangian multipliers are used to solve this problem. Cotter et al. [22] expanded this work in more general inequality-constraine to enhance generalizability. They enforce independence by regularizing the covariance between predictions and sensitive variables, which reduces the variation in the relationship between the two. Despite analytic solutions and theoretical assurances, scaling game- theoreic techniques for more complex models remains challenging [9] In addition, these constraints-basee optimizations are data-dependent, meaning the model may exhibit different behavior during evaluation even if con- straints are met during training. Less common approaches including Pareto-based genetic algorithm, Reinforcement Learning, Gradient-Bard have been also utilized in this domain. Mehrabi et al. 4S demonstrated how proxy attributes lead to indirect unfairness using an attention-basse approach and employed a post-processing method to reduce the weight of attributes responsible for unfairness. Perrone et al. 49 introduces a general constrained Bayesian optimization (BO) framework to fine-tune the model's performance while enforcing one or multiple fairness constraints. A probabilistic model is used to describe the objective function, and estimates are made for the posterior variances and means for each hyperparameter configuration. By adding faimness regularization term to meta-learning framework, Slack et al. 660] suggests an adaptation of the Agnest-A Meta-Learning (MAML) (23] algorithm. The primary objective and fairness regularization terms are included in the loss function used to update the model parameters for each task during the inner loop (Leareer). The model parameters are updated in the outer loop (Meta- (Meta-learner) to maximize performance and fairness across all tasks. Although these techniques have achieved a good balance between fairness and accuracy, they might not capture all of the complex interdependencies between these two objectives. In this paper we propose a bilevel optimization approach as an alternative to the Lagrangian approaches. Bilevel optimization is a hierarchical structure in which the context constraints for the "follower" (lower-level) problem are set by the "leader" (upper-level) problem [17]. The leader makes decisions first, and the follower optimizes their decisions based on the leader's choices. This approach can handle more complex and nuanced multi-objective optimization problems in neural networks and is suitable for scenarios where one objective directly influences another and there are complex interactions between the two objectives. In this paper we demonstrate that the bilevel optimization often can achieve better balance and performance compared to classic regularization- based optimization approaches [17] 59 10).
two-player
setting, by simultaneously training each player on two distinct datasets
Methods, and Transfer and Meta- Learning Approaches
Bilevel optimization offers several advantages; by explicitly modeling two-level decision -akiing process, his approach represents the problems in a more natural way where one objective inherently depends on the outcome of another. It provides more flexibility and control over the optimization process by enabling separate optimization of constraints each level. The upper-level optimization can dynamically adjust the lower-level objective based on the current solution, potentially leading to more adaptive and context-sensitive optimization outcomes. Fairness and accuracy objectives can be directly integrated into the optimization framework without the need for additional strategies such as meta-learning.
3 Methodology
In this section we are introducing novel bi-level optimization framework for training neural networks to obtain Pareto optimal solutions when optimizing two potentially competing objectives. Our approach leverages a leader-follower structure, where the leader problem aims to minimize one objective function (e.g. a primary loss), while the follower problem optimizes a secondary objective. We provide theoretical guarantees that our bi-level approach produces Pareto optimal solutions and performs at least as well as, and often strictly better than, the common practice of combining multiple objectives via weighted regularization term in a single loss function. The full statements
3

---

of these theorems and their proofs are provided in the Theoretical Analysis subsection below. Our bi-level approach offers several benefits over regulatization-butt based methods. First, allows for easy customization of the architecture and training algorithm used for each objectíve. The leader and follower problems can utilize different network architectures, regularizers, optimizers, etc. as best suited for each task. Second, the leader problem remains pure minimization of the primary loss, without any regularization terms that may slow or hinder its optimization. Separating out secondary objectives ensures the primary task is learned most effectively. Finally, bi-level training exposes clear interface for controlling the trade-off between objectives. By constraining the follower problem encourage stronger weaker adherence the secondary goal relative to
more or less strictly, the primary one. To realize these benefits, we employ an iterative gradient- gradient-based algorithm to solve the bi-level problem, alternating between updating the leader and follower parameters. We unroll the follower optimization for a fixed number of steps, and backpropagate through this unrolled process to update the leader weights.
3.1
Theoretical Analysis
3.1.1 Problem Formulation
The fairness multi-criteria problem in neural networks can formulated as bi-criteria optimization problem. Let f(Op. f,) denote the primary objective loss function and (00p, f.) denote the secondary objective loss function. Here, 6p θ₂ represents the parameters responsible for optimizing the primary objective, and 8, E O, represents the parameters for the secondary objective. The problem formally stated as:
min (S(0p,
Ope0p.0yE Eo,
3.1.2 Theoretical Foundation: Stackelberg Equilibrium and Pareto Optimality
We leverage the theoretical results of [46]. which investigates the relationship between Stackelberg equilibria and Pareto optimality in game theory. The paper addresses fundamental questions regarding the conditions under which a Stackelberg equilibrium coincides with a Pareto optimal outcome. By proving that bi-level optimization problem satisfies the assumptions outlined in the paper, we establish a strong theoretical foundation for our approach. Specifically, we demonstrate that under certain conditions, the Stackelberg equilibrium of our bi-level optimization problem is equivalent to Pareto optimal solution for the bi-criteria problem of balancing accuracy and fairness objectives, By rigorously verifying these assumptions in the context of our neural network optimization problem, we establish that the Stackelberg equilibrium reached by our bilevel approach indeed corresponds Pareto optimal solution. This theoretical grounding provides confidence that our methodology effectively balances the competing objectives of accuracy and fairness, yielding principled and well-justified solution the problem at hand.
We leverage several key theoretical results to formulate our approach. First, Lemma 3.5 establishes the Lipschitz continuity of the neural network function with respect to subset of parameters. This lemma provides the foundation for analyzing the behavior of the objective functions under parameter variations.
Definition 3.1. A function f RR" such that:
R" called Lipschitz continuous if there exists a constant
Vr, y E R", II, - f(y)ll, s L ||x y/1
(2)
The smallest for which the previous inequality is true is called the Lipschitz constant of f and will be denoted L(f).
Assume that the following assumptions are satisfied:
Assumption 3.2. The primary loss function f(0p.0.)(a) (x)()) is strictly convex in a neighborhood of its local optimum. That is, for any 0p, 8 Op and fixed 0, Oz. if 6p & 0%, and 0p, 0ú are sufficiently close to the local optimum 07 then
S(A0p + (1 1)6%, < AS 0.) + +(1 -))((()),, 0.)
for any À E (0, 1).

---

Assumption 3.3. 16, 10.-0 where e is sufficiently small, i.e., the steps of the secondary parameters are sufficiently small. 03 and 0, represent the parameters for the secondary objective and their updated values, respectively.
Assumption 3.4. Let fi(.) denote the output function of the I-th layer neural network with layers. For each layer E 1,.... L, there exists constant > 0 such that for any input to the 1-th layer:
Iff(z)) c
(4)
where I. denotes suitable norm (e.g., Euclidean norm for vectors, spectral norm for matrices). Refer to Section f.4 common practices in implementing the bounded output assumption.
We recognized the importance of examining how our theory's underlying assumptions apply to real-world applications. For a detailed discussion, refer to Section A.3
Lemma 3.5. Let f(:; 8) be neural network with layers, where each layer is a linear transforma- tion followed by a Lipschitz continuous activation function.
Let 0 the set of all parameters of the neural network, and 0, Then, [1x; 0) is Lipschitz continuous with respect to 0g. [See proof A.4)
any subset of parameters.
We discussed the Lipschitz continuity of common activation functions and popular neural networks, such CNNs and GNNs, in Sections A.5 and A.6 respectively.
Theorems 3.6 and 3.7 further inform our approach. The former establishes conditions under which improvements in the secondary objective lead to improvements in the primary objective, while the latter guarantees the existence of unique minimum solutions for the secondary loss function under certain optimization conditions.
Theorem 3.6. Let f(Op, 0z) for constant 0, be the primary objective loss function and (p 0.) for constant Bg be the secondary objective loss function, where 6, E O and 0g O, are the primary task and secondary task parameters, respectively.
Consider two sets of parameters (0p. 02) and (0p, 02) such that p(0p, 02) S p(0p, 0z). Then [(0pp 0.) < [(0p. 0.) holds based on Lemma 3.5 [See proof
Theorem 3.7. Let 4(0p, 02) be the secondary loss function, where 6p E Op and 0, O2 are the primary and secondary task parameters, respectively. Let (0)) _ denote the parameters at optimization step I, and let (0(+t))0(+++) be the updated parameters obtained by minimizing <p(0))) O.) with respect Og using a sufficiently small step size 11 > i.e.:
(((+1) = o -7V ((!))
(5)
Then, for a sufficiently small step size n, the updated secondary parameters + are the unique minimum solution for the secondary loss function v(0 0o). [See proof A.6
Based on these theoretical insights, we derive our bilevel optimization formulation, described in Theorem 3.8 This theorem establishes the equivalence between the bi-criteria problem and a bilevel optimization problem, allowing us to apply existing theoretical results on Stackelberg equilibrium to the optimization of neural networks.
Theorem 3.8. Under the assumptions stated in Theorems 3.6 and 3.7 ( is equivalent the bilevel optimization problem:
the bi-criteria problem (Eq.
min eoe
f(ep. 07 (0p))
(6)
(0p))
arg min (( p0:000 0.) 0.ce.
(7)
where 87 (0p) denotes the optimal secondary parameters for a given 0p.
Proof. The proof follows from Theorems 3.6 and 3.7 46
5

---

By Theorem 3.6 under the assumptions of strict convexity, Lipschitz continuity, and sufficiently small steps of the secondary parameters, if (0 p. 0&) S (0pp. 02), then f(0p, 0.) S f(0p. 0s).
By Theorem 3.7 under the same assumptions, for each optimization step of the secondary loss function with sufficiently small steps, the updated parameters are the unique minimum solution for the secondary loss function, then the bi-criteria problem T is equivalent to the bilevel optimization problem.
Therefore, the conclusions drawn in the paper 46 can be directly applied to the multi-objectinn optimization problem in neural networks, as the problem is equivalent to the bilevel optimization problem under the stated assumptions.
Theorem 3.9. Assume that the step size in the Lagrangian approach ag is equal to the step size for the outer optimization problem the bilevel optimization approach aj. the scale of the two loss functions should be comparable, the Lagrangian multiplier is equal to the step size for the inner optimization problem in the bilevel optimization approach ag and 6p overparameterized for the given problem. Then, under certain conditions, the overall performance of the primary loss function the bilevel optimization approach may be better than the Lagrangian approach.
Proof. Let f(pp, 0.) denote the primary loss and p(Op, 0,) denote the secondary loss. Assume that both f and yp are differentiable with respect 8g and 0.. Define the Lagrangian function as:
[(pp, 0z, A) p0pp: 0.) + (pp(0p. 0s)
(8)
The update rules for Op and Og in the Lagrangian approach are:
g((+1) = (0() - Vo, @ A) g((+1) = 0(0)) - oz - Vo, (() g(0) A)
(9)
(10)
The update rules for Op and O2 in the bilevel optimization approach are:
0((+1) ((0) a Vo, o(0)
(11)
g(t+1) 8(0) a. Vo, (p(()) 8(())
(12)
Due to the overparameterization of 6p. there exists a set G such that for any 8p E Op B f 0.) f(;;, 0.), where 97 is an optimal solution for the primary loss when 0g is fixed. Suppose that the bilevel optimization approach converges to solution (0 008) and the Lagrangian approach
converges to a solution (0,, 04). Consider the following inequality:
S(0 00)) = 1(0p) 03) + Ap - Xp(
(13)
S f(op) 00) xp(O8, 00) - xpp(0,,0))
(14)
C(0 0 >) - >p(0 00)
(15)
S C(f), 04,2) -xp 04)
(16)
500) 0.:)
(17)
The first inequality holds because (05, 05) ) is the minimizer of s0 0z) the Lagrangian approach.
The second inequality holds because (0) 05) is the minimizer of L(ep, 8,, A) in the Lagrangian approach. Since 05 E Op and 05 Op. we have:
f 0.) F(0 0.) f(0),00)
(18)
Therefore, under the assumptions that ag ay, The sizes of the two loss functions f(Op, 0.) and y0(0,, 0,) should not differ significantly terms of their order of magnitude, À as and Op is overparameterized for the given problem, the bilevel optimization approach may converge to a solution that achieves better performance for the primary loss compared to the Lagrangian approach.

---

3.2 Practical Implementation
To connect the Stackelberg game analysis with practical implementation for datasets, we can formulate a bilevel optimization problem. The upper-level problem corresponds to the accuracy player (leader), while the lower-level problem corresponds the faimess player (follower). We'll use gradient-based optimization techniques to solve the problem.
Let's consider dataset D { di, }> where x; represents the features, dy represents the sensitive attribute, and represents the target variable for the i-th sample.
The optimization problem can be formulated as follows:
min 6a
Laco (f Ba: 03). v1)
(19)
N
s.t. 0J E arg min
Σ 1=1
Loui (Iz; Oar as, Mi)
(20)
N
where (a; 0g) 0y) is the model parameterized by the accuracy parameters 0g and fairness parameters 8,, Luce is the accuracy loss function (e.g.. binary cooss-nnroppy), and Lair is the faimess loss function (e.g.. demographic parity loss). We showed that the demographic parity loss function, when applied to the output of neural network layers, is also Lipschitz continuous (Theorem 3.10).
Demographic Parity Loss Function: The demographic parity loss function DP(J) is defined as:
DP(S) = (B
(21)
where a is sensitive attribute (e.g., race, gender) with two possible values (0 and 1), and p(rla) is the conditional probability distribution of given a.
Theorem 3.10. If ((x) is Lipschitz continuous with Lipschitz constant Ls, then the demographic parity loss function (f(() is also Lipschitz continuous with Lipschitz constant LDp = 2Lg. [See proof A.9
We can easily extend this theorem to include another common fairness metric, equalized odds, as explained in Section A.2
Here's 's practical implementation using gradient-based optimization:
Algorithm 1 Fairness-Accuracy Bilevel Optimization
1: Initialize accuracy parameters Og and fairness parameters Of
2: while not converged or max iterations not reached do
3: Accuracy player's optimization
4: Sample minibatch B, C
5: Compute accuracy loss: La 1 Eies, Loco Oa 01), vi)
18
6:
Update accuracy parameters: 0g x- 02 Uaa Vo, Lo
7:
Fairness player's 's optimization
8:
Sample a minibatch B D
9:
Compute fairness loss: Ly [8, EieB, Lsair (f(xx; Oa 05), as a1.9))
10: Update fairness parameters: 0 t 0, Ty V Ls
end while
In practice, the model fx; fa. fr) can be implemented as neural network with separate layers for accuracy and fairness (Figure 8) The accuracy layers are parameterized by 9a. while the fairness layers are parameterized by 0 The accuracy loss Lace can be chosen based on the task at hand, such as binary cross-entropy for binary classification or mean squared error for regression. The fairness loss I fair can be fairness metric such as demographic parity loss equalized odds loss. The learning rates 7/a and f/ control the step sizes for updating the accuracy and fairness parameters, respectively. They can be tuned using techniques like grid search learning rate scheduling.
7

---

By implementing this algorithm on dataset, we can optimize the model to balance accuracy and fairness, guided by the Stackelberg game formulation. At each iteration. the parameters related to accuracy are optimized while keeping the fairness parameters fixed. Then, with the accuracy parameters held constant, the fairness parameters are optimized. This separate optimization process provides fine-grained control over the trade-off between accuracy and fairness.
4 Experiments
In this section, we contrast our methodology with other benchmark approaches found in the literature.
We employed two metrics for evaluation: accuracy (higher values preferred) for the classification task, and demographic parity differences (DP, lower values preferred) for fairness assessment. Detailed descriptions of all metrics used and implementation settings are available in sections A.9 and A.10 in the appendix, respectively.
We evaluated our method for bias mitigation to various current state-of-the-art approaches. We concentrate on strategies specifically tuned to achieve the best results statistical parity metrics on tabular studies.
4.1 Datasets:
We used two well-known benchmark datasets in this field for experiments which follows: UCI Adult Datasel [19], This dataset is based on demographic data gathered in 1994, including a train set of 30000 and test set of 15,000 samples. The goal is to forecast if the salary more than $50,000 yearly, and the binary protected attribute is the gender of samples gathered in the dataset.
Heritage Health Dataset [.) Predicting the Charleson Index, measure of patient's 10-year mortality. The Heritage Health dataset contains samples from roughly 51,000 patients of which 41000 are in the training set, and 11000 are in the test set. The protected attribute, which has nine potential values, is age.
4.2 Baselines:
We compare our results with the following state-of-the-art methods as benchmarks:
CVIB Achieves fairness using a conditional variational autoencoder.
MIFR 61 :
Optimizes the fairness objective with a mix of information bottleneck factor
and adversarial learning.
FCRL 25]: Uses specific approximations for contrastive information to maximize theoret-
ical goals, facilitating appropriate trade-offss among statistical parity, demographic parity, and precision.
MaxEnt-ARL. [56] Employs adversarial learning to mitigate unfairness.
Adversarial Forgetting [30]: Uses adversarial learning techniques for fairness.
Fair Consistency Regularization (FCR) 4 Aims to minimize and balance consistency
loss across groups.
Robust Fairness Regularization (RFR) 31: Considers the worst-case scenario within the
model weight perturbation ball for each sensitive attribute group to ensure robust fairness.
4.3 Bilevel (FairBiNN) vs. Lagrangian Method
We compare our proposed FairBiNN method with the traditional Lagrangian regularization approach to empirically validate the theoretical benefits of bilevel optimization. Our analysis focuses on the convergence behavior and stability of both methods. For comprehensive details on performance and computational complexity comparison, refer to Section A.7| In the appendix, we have provided the BCE loss plots over epochs for each dataset (Fig. [ and demonstrated the superior performance of the Bi-level approach compared to the Lagrangian approach. We have also presented a comparative analysis these approaches for the trade-off between accuracy and Statistical Parity Difference (SPD) in Figure
8

---

0.200
FORL J02L1
PCL LAA 33331
CID Heur FOOO
08
KBR JeoGEs 20100
Aiverortdl Porperinng 2020
0
Sottke Party Offorencc
Accoraata Forpeting (AMA
KAA onsurpe 2520
TER Piour 2023
0.325
0.300
0000g
0.050
02
0.25
0.0
5.76
0.03
0.73
0.75
0.00
085
Accumay
(a) UCI Adult
(b) Heritage Health
Figure I: Accuracy of various benchmark models compared to the FairBiNN model versus statistical demographic parity for the (a) UCI Adult dataset and (b) Heritage Health dataset. The optimal region on this graph is the bottom right, indicating high accuracy and low DP. The results demonstrate that model (red diamond markers) significantly outperforms other benchmark models the UCI Adult dataset and closely competes with recent state-of t-thher-t models on the Heritage Health dataset.
While Theorem 3.9 in the paper proves that, under certain assumptions, the primary loss function in the bilevel optimization approach is upper bounded by the loss of the Lagrangian approach at the optimal solution, does not analyze or guarantee the convergence behavior of the algorithms. The empirical results for the Health and Adult datasets show that the bilevel approach outperforms the Lagrangian method minimizing the BCE loss. However, further investigation needed to understand the convergence properties of the algorithms and connect the theoretical results with empirical observations. Despite this, the experimental results highlight the potential of the bilevel optimization framework to optimize accuracy and fairness objectives, offering promising approach to address the multi-criteria faimess problem in neural networks.
4.3.1 Benchmark Comparison
We provide average accuracy measure of most probable accuracy and maximum demographic parity as measure of worst-case scenario bias, calculated across five iterations of the training process using random seeds. Unlike Gupta et al. [25,, we did not use any preprocessing feeding to our network. Reported results for our model are Pareto solutions for the neural network during training with different ?y.. Results are reported for methods with multi-layer perceptron classifier with two hidden layers.
the data before
Figures and lb show trade-offs of the statistical demographic parity vs. accuracy associated with various bias reduction strategies in the UCI Adult dataset and Heritage Health dataset, respectively. The ideal area of the graph for the result of method measure how much the curve is located in the lower right corner of the graph, which means accurate and fair results concerning protected attributes. Our results demonstrate that the Bilevel design significantly outperforms competing methods in Adult dataset.
5 Limitations and Future Work
While our results are promising, it's important to acknowledge several limitations our current approach:
One of the most widely used activation functions, softmax, not Lipschitz continuous. This limits the direct application of our method to multiclass classification problems. Future work could explore alternative activation functions or modifications to the softmax that preserve Lipschitz continuity while maintaining similar functionality for multiclass problems.

---

Attention mechanisms, which are widely used in modern language models and other architectures, are not Lipschitz continuous. This presents challenge for extending our method sfate-of-the- art architectures in natural language processing and other domains that heavily rely on attention. However, research into the Lipschitz continuity of attention layers has already begun, with Dasoulas et al. [16] introducing LipschizZNorm, a simple and parameter-free normalization technique applied to attention scores to enforce Lipschitz continuity in self-attention mechanisms. Their experiments on graph attention networks (GAT) demonstrate that enforcing Lipschitz continuity generally enhances the performance of deep attention models.
Our theoretical analysis primarily provides guarantees in comparison to regularization methods. While the results show improvements in fairness overall, the theory does not offer absolute fairness guarantees for the final model. Extending the theoretical framework to include direct fairness guarantees could strengthen the method's s applicability.
This method was not validated on dataset augmentation approaches, which are common in practice for improving model generalization and robustness. Future work should investigate how our method interacts with various data augmentation techniques and whether it maintains its fairness properties under such conditions.
Our current implementation focuses on a single fairness metric (demographic parity). In practice, multiple, sometimes conflicting, fairness criteria may be relevant. Extending our method to handle multiple fairness constraints simultaneously could make it more versatile for real-world applications.
Addressing these limitations presents exciting opportunities for future research. By tackling these challenges, we can further enhance the applicability and effectiveness of fair machine learning methods across a broader range of real-world scenarios and cutting-edge architectures.
6 Discussion and Conclusion
Our primary contribution lies in the thcoretical foundation and general applicability of the proposed framework, rather than extensive ablation studies specific datasets or network configurations. However, we recognize the importance of empirical evaluations. Our work introduces a novel approach to addressing the multi-criteria faimess problem in neural networks, supported by theoretical analysis, particularly Theorem which establishes properties of the optimal solution under certain assumptions, independent of specific datasets architectures. The results on vision and graph datasets (A.11) and ablation studies on the impact of (A.13.I), the position of fairness layers (3.13.3), and different layer types (A.13), presented in the appendix, demonstrate the effectiveness and versatility of our approach. These studies show that the bilevel optimization framework can be successfully applied to various layer types and network architectures, beyond the single lincar layer used in the main experiments. Our experimentation across diverse datasets, including UCI Adult, Heritage Health, and other domains like graph datasets (A.8.I) (POKEC-Z, POKEC-N, and NBA) and vision datasets [A.822 (CelebA), further illustrates the versatility and efficacy of our method. Including these ablation studies in the appendix allows us to maintain the main text's focus on theoretical contributions and the general framework while providing additional empirical evidence to support our claims.
Our results demonstrate the superiority of our model over state-of-the-ar fairness methods in reducing bias while maintaining accuracy, highlighting the potential of our framework to advance fairness- aware machine learning solutions. Notably, our study represents a significant contribution by formulating multi-objective problems neural networks as bilevel design, providing a powerful tool for achieving equitable outcomes across diverse groups in classification tasks. Future research can address our method's limitations and explore potential directions as outlined in Section
References
[1] Alexandre Abraham. Michael Milham, Adriana Martino, Cameron Craddock. Dimitris Samaras, Bertrand Thirion, and Gael Varoquaux. Deriving reproducible biomarkers from multi-site resting-state data: An autism-based example. Neurolmage. 44:7766-455, 2017.
[2] Alekh Agarwal, Alina Beygelzimer, Miroslav Dudfk, John Langford, and Hanna Wallach, A reductions approach to fair classification, In International Conference on Machine Learning, pages 60 69. PMLR, 2018.
10

---

[3] Zeyuan Allen-Zhu, Yuanzhi Li, and Yingyu Liang- Learning and generalization in overparameteriize neural networks, going beyond two layers. Advances in neural information processing systems, 32, 2019.
[4] Bang An, Zora Che, Mucong Ding, and Furong Huang. Transferring faimess under distribution shifts via fair consistency regularization. Advances in Neural Information Processing Systems, 113522822325999 2022.
[5] Alex Beutel, Jilin Chen, Zhe Zhao, and Ed H Chi. Data decisions and theoretical implications when adversarially learning fair representations. arXiv preprint arXiv:1707.00075. 2017.
[6] Avishek Bose and William Hamilton. Compositional fairness constraints for graph embeddings. In International Conference on Machine Learning, pages 715-724, PMLR, 2019.
[7] Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification, In Conference on fairness, accountabilitty and transparency, pages 77-91. PMLR. 2018.
[8] Kristy Choi, Aditya Grover, Trisha Singh, Rui Shu, and Stefano Ermon. Fair generative modeling via weak supervision. In International Conference on Machine Learning, pages 1887-1898. PMLR, 2020.
Ching- Yao Chuang and Youssef Mrouch. Fair mixup: Fairness via interpolation. arXiv preprint 1rXiv:2103.065/0 2021.
[10] Benoît Colson, Patrice Marcotte, and Gilles Savard. An overview of bilevel optimization. Annals of operations research, 153:235-256, 2007.
[11] Sam Corbett- Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. Algorithmic decision making and the cost of fairness. Proceedings of the 23rd aem sigkld international conference on knowledge discovery and data mining, pages 797-806, 2017.
[12] Andrew Cotter, Maya Gupta, Heinrich Jiang, Nathan Srebro, Karthik Sridharan, Serena Wang, Blake Woodworth, and Seungil You. Training well-genneaaiiiing classifiers for fairness metrics and other data- dependent constraints. In International Conference on Machine Learning, pages 1397-1405. PMLR. 2019.
[13] Elliot Creager. David Madras, Jörn-Henrik Jacobsen, Marissa Weis, Kevin Swersky. Toniann Pitassi, and Richard Zemel. Flexibly fair representation learning by disentanglement. In International conference on machine learning, pages 1436-1444 1445. PMLR, 2019.
[14] Enyan Dai and Suhang Wang. Say to the discrimination: Learning fair graph neural networks with limited sensitive attribute information. Proceedings of the 14th ACM International Conference on Web Search and Data Mining, pages 680-688, 2021.
[15] Enyan Dai and Suhang Wang. Learning fair graph neural networks with limited and private sensitive attribute information. IEEE Transactions on Knowledge and Data Engineering, (5(7)7113-77177 2022.
[16] George Dasoulas, Kevin Scaman, and Aladin Virmaux. Lipschitz normalization for self-attention layers with application to graph neural networks. International Conference on Machine Learning, pages 2456-2466. PMIR, 2021.
[17] Stephan Dempe Foundations of bilevel programming. Springer Science Business Media, 2002.
[18] Yuxiao Dong, Omar Lizardo, and Nitesh V Chawla. Do the young live in "smaller world" than the old? age-specific degrees of separation a large-scale mobile communication network. arXiv preprint vrXiv:1606.0755 2016.
[19] Dheeru Dua Casey Graff. UCI machine learning repository, 2017. URL URL http:///rrrriive..c... edu/ml
[20] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pages 314-226, 2012.
[21] Harrison Edwards and Amos Storkey. Censoring representations with an adversary. arXiv preprint arXiv:1511.055 0.8977 2015.
[22] Michael Feldman. Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. Certifying and removing disparate impact. In proceedings of the 21th ACM SIGKDD international conference knowledge discovery and data mining, pages 259-268,11 2015.
11

---

[23] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, pages 1126-1135. PMLR, 2017.
[24] Henry Gouk, Eibe Frank, Bernhard Pfahringer, and Michael J Cree. Regularisation of neural networks by enforcing lipschitz continuity. Machine Learning. 110:393-411 2021.
[25] Umang Gupla, Aaron Ferber, Bistra Dilkina, and Greg Ver Steeg, Controllable guarantees for fair outcomes via contrastive information estimation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 1610-7619. 2021.
[26] Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29:3315-3323, 2016.
[27] Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 778,778, 2016.
[28] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition.
In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[29] Sunhee Hwang, Sungho Park, Dohyung Kim, Mirac Do, and Hyeran Byun. Fairfacegan: Fairess-aware facial image-to-image translation. arXiv preprint arXiv:2012.00226 2020.
[30] Ayush Jaiswal, Daniel Moyer, Greg Ver Steeg, Wael AbdAlmageed, and Premkumar Natarajan. Invariant representations through adversarial forgetting- In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 4272-4279, 2020.
[31] Zhimeng Stephen Jiang, Xiaotian Han, Hongye Jin, Guanchu Wang. Rui Chen, Na Zou, and Xia Hu. Chasing fairness under distribution shift: model weight perturbation approach. Advances in Neural Information Processing Systems, 36, 2024.
[32] Simona Ioana Juvina, Ana Antonia Neacşu, Jérôme Rony, and Ismail Ben Ayed. Training graph neural networks subject to tight lipschitz constraint. Transactions on Machine Learning Research.
Pesquet, Comeliu Burilcanu,
[33] Faisal Kamiran and Toon Calders. Classifying without discriminating. In 2009 2nd international conference on computer, control and communication, pages 1 1-6. IEEE, 2009.
[34] Faisal Kamiran and Toon Calders. Data preprocessing techniques for classification without discrimination. Knowledge and Information Systems, 33(1):1 13,1, 2012.
[35] Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma. Fairness-aa aware classifier with prejudice remover regularizer. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 35 35-50. Springer, 2012.
[36] Ali Khodabandeh Yalabadi, Mehdi Yazdani-Jahromi, Niloofar Yousefi, Aida Tayebi, Sina Abdidizaji, and Ozlem Ozmen Garibay. Fragxsitedti: Revealing responsible segments in gargetarget interaction with transformer-driven interpretation. In International Conference Research in Computational Molecular Biology, pages 68-85. Springer, 2024.
[37] O Deniz Kose and Yanning Shen. Fairgat: Fairess-awaar graph attention networks, ACM Transactions on Knowledge Discovery from Data, 18(7):1-20, 2024.
[38] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
[39] Francesco Locatello, Gabriele Abbati, Thomas Rainforth, Stefan Bauer, Bernhard Schölkoff, and Olivier Bachem. On the fairness of disentangled representations. Advances in Neural Information Processing Systems, 32, 2019.
[40] Vishnu Suresh Lokhande, Aditya Kumar Akash, Sathya N Ravi, and Vikas Singh. Fairalm: Augmented lagrangian method for training fair models with little regret. In European Conference on Computer Vision, pages 365-381 Springer, 2020.
[41] Christos Louizos, Kevin Swersky, Yujia Li. Max Welling, and Richard Zemel. The variational fair autoencoder. arXiv preprint arXiv:1511.00w 2015.
[42] Binh Thanh Luong, Salvatore Ruggieri, and Franco Turini. k-nn as an implementation of situation testing for discrimination discovery and prevention. In Proceedings of the 17th ACM SIGKDD international conference Knowledge discovery and data mining, pages 510, 2011.
12

---

[43] Ninareh Mehrabi, Umang Gupta, Fred Morstatter, Greg Ver Steeg, and Aram Galstyan. Attributing fair decisions with atlention interventions. arXiv preprint urXtv:2109.03952, 2021.
[44] Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR), 54(6):1335, 2021.
[45] Aditya Krishna Menon and Robert Williamson. The cost of fairness in binary classification. In Conference on Fairness, Accountability and Transparency, pages 107-118. PMLR, 2018.
[46] Athanasios Migdalas. When is Stackelberg equilibrium Pareto optimum? Springer, 1995.
[47] Daniel Moyer, Shuyang Gao, Rob Brekelmans, Greg Ver Steeg, and Aram Galstyan. Invariant repre- sentations without adversarial training. Advances Neural Information Processing Systems, volume 31, 9084-9093, 2018.
[48] Bryan Perozzi, Rami AI-Rfou, and Steven Skiena. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 701-710, 2014.
[49] Valerio Perrone, Michele Donini, Muhammad Bilal Zafar, Robin Schmucker, Krishnaram Kenthapadi, and Cédric Archambeau. Fair bayesian optimization. Proceedings of the 2021 AAAI/ACM Conference on Al, Fthics, and Society, pages 854-863, 2021.
[50] Dana Pessach and Erez Shmueli. Algorithmic fairness. arXiv preprint urXiv:2001.0778, 2020.
[51] Novi Quadrianto, Viktoriia Sharmanska, and Oliver Thomas. Discovering fair representations in the data domain. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8227-8236, 2019.
[52] Tahleen Rahman, Bartlomicj Surma, Michael Backes, and Yang Zhang. Fairwalk: Towards fair graph embedding. 2019.
[53] Amirarsalan Rajabi and Ozlem Ozmen Garibay. Tabfairgan: Fair tabular data generation with generative adversarial networks. arXiv preprint arXiv:2109,00660 2021.
[54] Amirarsalan Rajabi, Mehdi Bazdani-Jahronna Ozlem Ozmen Garibay. and Gita Sukthankar, Through fair looking-glass, mitigating bias in image datasets. arXiv preprint arXiv:2209, 086448 2022.
[55] Vikram V Ramaswamy, Sunnie SY Kim, and Olga Russakovsky, Fair attribute classification through latent space de-biasing. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9301-9310, 2021.
[56] Proteek Chandan Roy and Vishnu Naresh Boddeti. Mitigating information lcakage in image representations:
A maximum entropy approach. In Proceedings of the IEFFF/CVF Conference on Computer Vision and Pattern Recognition, pages 2586-2594, 2019.
[57] Prasanna Sattigeri, Samuel C Hoffman, Vijil Chenthamarakshan, and Kush Varshney. Fairness gan: Generating datascts with faimess propertics using a generative adversarial network. IBM Journal of Research and Development, 63(4/5):3 2019.
[58] Viktoriia Sharmanska, Lisa Anne Hendricks, Trevor Darrell, and Novi Quadrianto. Contrastive examples for addressing the tyranny of the majority. arXiv preprint arXiv:2004.066 2020.
[59] Ankur Sinha. Pekka Malo, and Kalyanmoy Deb. A review bilevel optimization: From classical evolutionary approaches and applications. IEEE transactions on evolutionury computation, 22(2):276-295. 2017.
[60] Dylan Slack, Sorelle Friedler, and Emile Givental. Fair meta-learning learning how to learn fairly. arXiv preprint arXiv: 1.04336, 2019.
[61] Jiaming Song, Pratyusha Kalluri, Aditya Grover, Shengjia Zhao, and Stefano Ermon. Learning controllable fair representations. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 2164-2117 PMLR, 2019.
[62] Lubos Takac and Michal Zabovsky. Data analysis in public social networks. In International scientific conference and international workshop present day trends of innovations, volume 1. Present Day Trends of Innovations Lamza Poland, 2012.
13

---

[63] Aida Tayebi, Niloofar Yousefi, Mehdi Yazdani-Jahromi. Elayaraja Kolanthai, Craig J Neal, Sudipta Seal, and Ozlem Ozmen Garibay. Unbiaseddti: Mitigating real-world bias of drug-target interaction prediction by using deep sneemble-balancee learning. Moleculex, 27(9);2980, 2022.
[64] Aida Tayebi, Mchdi Jahromi, Ali Khodabandeh Yalabadi, Niloofar Yousefi, and Ozlem Ozmen Garibay, Learning fair representations: Mitigating statistical dependencies. In International Conference on Cluman-Computee Interaction, pages 105-115. Springer, 2024.
[65] Angelina Wang, Alexander Liu, Ryan Zhang, Anat Kleiman, Leslic Kim, Dora Zhao, Iroha Shirai, Arvind Narayanan, and Olga Russakovsky. Revise: A tool for measuring and mitigating bias in visual datasets. International Journal of Computer Vision, pages 1-21, 2022.
[66] Mei Wang and Weihong Deng. Mitigate bias in face recognition using skewness-aware reinforcement learning. arXiv preprint arXiv: 1911. 1011.00062 2019.
[67] Mei Wang, Weihong Deng. Jiani Hu, Xunqiang Tao, and Yaohai Huang. Racial faces in the wild: Reducing racial bias by information maximization adaptation network. In Proceedings of the ieeelcyf international conference on computer vision, pages 692-702, 2019.
[68] Tianlu Wang, Jieyu Zhao, Mark Yatskar, Kai- Wei Chang, and Vicente Ordonez. Balanced datasets are not enough: Estimating and mitigating gender bias in deep image representations. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5310 5319, 2019.
[69] Tongxin Wang, Zhengming Ding, Wei Shao, Haixu Tang, and Kun Huang. Towards fair cross-domain adaptation via generative learning. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 454-463, 2021.
[70] Xingkun Xu, Yuge Huang, Pengcheng Shen, Shaoxin Li, Jilin Li, Feiyue Huang, Yong Li, and Zhen Cui. Consistent instance false positive improves fairness in face recognition. In Proceedings of the IFFF/CVF conference on computer vision and pattern recognition, pages 578-586, 2021.
[71] Kaiyu Yang, Klint Qinami, Li Fei-Fei, Jia Deng, and Olga Russakovsky. Towards fairer datasets: Filtering and balancing the distribution of the people subtree in the imagenet hierarchy. In Proceedings of the 2020 conference on fairness, accountability, and transporency, pages 547-558, 2020.
[72] Mchdi and Ozlem Ozmen Garibay. Attentionsitedti: an interpretable graph-basee model for drug-targe strgtttgettti interaction prediction using nlp sentence-level relation classification. Briefings Bioinformatics, 23(4):bbac222, 2022.
Niloofar Yousefi, Aida Tayebi, Elayaraja Kolanthai, Craig J Ncal, Sudipta Scal,
[73] Mikhail Yurochkin, Amanda Bower, and Yuckai Sun. Training individually fair ml models with sensitive subspace robustness. arXiv preprint urXiv:1907,00000 00000000 2019.
[74] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P Gummadi. Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In Proceedings of the 26th international conference on world wide web, pages 1 1111-1880, 2017.
[75] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi. Fairness constraints: Mechanisms for fair classification. In Artificial Intelligence and Statisticy, pages 962-970.
PMLR, 2017.
[76] Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning fair representations. In International conference machine learning. pages 325-333. PMLR, 2013.
[77] Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 335-340, 2018.
[78] Lu Zhang, Yongkai Wu, and Xintao Wu. Achieving non-discrimination in data release. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1335-1344, 2017.
[79] Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai- Wei Chang. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. arXiv preprint arXiv:1707.09455 2017.
[80] Dongmian Zou, Radu Balan, and Maneesh Singh. On lipschitz bounds of general convolutional neural networks. IEEE Transactions on Information Theory, 173771173 2019.
14

---

A Appendix supplemental material
A.1 Theoretical proofs
Assume that the following assumptions are satisfied:
Assumption A.1. The primary loss function f(pp, (x)( is strictly convex in a neighhorhood of its local optimum. That is, for any 0p, 8% G and fixed 0g O if 0, + 88, and 0p 66 are sufficiently close to the local optimum 0r. then
S(O0p (1 1)0%'= 0,) < (s(p,, (1 2)5(0/) 0.)
(22)
for any E (0,1).
Assumption A.2. 10, 10.-0,15 where is sufficiently small, i.c., the steps of the secondary parameters sufficiently small. 0g and 0, represent the parameters for the secondary objective and their updated values, respectively.
Assumption A.3. Let ft(.) denote the output function of the 1-th layer in neural network with L layers. For each layer E 1,..., L, there exists constant ạ >> 0 such that for any input iq the 1-th layer:
Ifi(zi)ls a
(23)
where |.| denotes a suitable norm (e.g., Euclidean norm for vectors, spectral norm for matrices).
Lemma A.4. Let f(x;0) 0) be neural network with L layers, where each layer is linear transforma- tion followed by a Lipschitz contimuous activation function.
Let 0 be the set of all parameters of the neural network, and 0g C 0 be any subset of parameters.
Then, f(x; 0) is Lipschitz continuous with respect to 0g-
Proof. Since each activation layer is Lipschitz continuous with Lipschitz constant L we have:
[f 01) fz 0')
(24)
Lilewr + b.) (wfe + b/)l
(25)
Lil(wr wi))i-1 + (b. 8f)
(26)
by the triangle inequality and 24 we have:
[w] w/)ft- + (b) t115 Li - willfi- b(l)
(27)
by assumption A.3 and Eq. 27 we have:
[f((;; 0:) - fi(;; 05 L[(wr - w/la + 16: - 611) - Lic: 0%
(28)
We can write the neural network as the composition of functions of each layer:
fx=;0) fi o fi. (z;
(29)
According to triangle inequality, write:
If 0,) ((; Else c S.t o f(x;0.)) fo o.. f(x
(30)
Since the composition of Lipschitz Continuous functions is Lipschitz continuous with Lipschitz constant equal to the product of individual Lipschitz constants [4 we can write:
Ifz f.41 fl(x; 0.) fr ... f((z;0]1 ( II
Lo) 0,1
(31)
k=it1
by using Eq. 30 and we can write:
1 Laa II Lxcr)10, - -0'1
()=;0) - f(x; 0')
(32)
k=i+1
15

---

with Cauchy-Schwarcd Schwarz inequality we have:
Σ Lecs II Loc+) 0il
(33)
k=i+1
Σ 00 0,)²
E L,)RL3d
(34)
:=1 j=it1
$ 10 0'1
(35)
for all A = (8s, 05) and constant 85 we have:
1f(x;0,) - (1;;6:)< L'10, -8'1
(36)
Theorem A.S. Lef (0pp 0.) for constant 02 be the primary objective loss function and 5p 0.) for constant Og be the secondary objective loss function, where O2 E O and 0g O, are the primary task and secondary task parameters, respectively.
Consider two sets of parameters (0p. 00p:0.) 0.) and (6p, 0.) such that ((pp. S 000p, 0z). Then f(0,. 0.) S f(0p. 0z) holds based on Lemma A.4
Proof. Let (0p. 0z) and (0p. 0.) be two sets of parameters such that (0pp A.)) S cl0,, 0z). By Lemma A.4 f(o,. p. 0₂) is Lipschitz continuous with respect to 8, and 0,.
By Assumption A.2 10 0.15 e, where is sufficiently small. Therefore, applying the Lipschitz
continuity:
|5(0,, 0.) 0.)15 LÍ0, 0.15 Le
(37)
Now, consider the primary loss function f(pp. 0.) for fixed 0z. By Assumption A.I f() is strictly
convex in a neighborhood its local optimum 8p. This means that for 6, and 6, sufficiently close 0::
S(XŌg +1 -1- 1)0,. < (1- A)S(0,,
(38)
for any A E 10,
Since p(Op, 0.) I <p(pp 0,), and the secondary loss function pp.) is used to update the parameters, can assume that 8, is closer to the local optimum 6% than 0z. Therefore, by the strict convexity of f():
f(O,, 02) f(0p, 0.)
(39)
Combining the results from and 39).
f(0 0.) s (f 0.) f(6p. 0.) +.)
Le + f(Op. 0.)
(40)
As is sufficiently small, can conclude that (0,p 0.) S f(0p. 0.). Therefore, under the given assumptions, if ((,,, 0.) S p(0p, 0z), then f(Op, 0.) S f(p. 0z).
Theorem A.6. Let sp(0p, 0.) be the secondary loss function, where O, E O and 0, E O2 are the primary and secondary task parameters, respectively. Let (8% . denote the parameters at optimization step t, and let (olt+1), ,0(++1) be the updated parameters obtained by minimizing p(0%)),,, with respect Og using a sufficiently small step size n > i.e.:
o((+ =((t) V ( 0(9)
(41)
Then, for a sufficienlly small step size n, the updated secondary parameters f(t+1) are the unique minimum solution for the secondary loss function yp(0)) 0.).
16

---

Assumption A.7. p00p, 0.) is smooth and Lipschitz continuous.
Proof. Let 8 be fixed at optimization step i. We consider the optimization problem of minimizing the secondary loss function p(0,)) .8) with respect 6g.
By the assumption, p(())" f2) is smooth and Lipschitz continuous with respect to 0,. This implies that ( (.)) is continuously differentiable and its gradient Vo, (e(b)) ,0,) is Lipschitz continuous with some Lipschitz constant L > 0, i.e., for any 8,. 8¹, E A
Ve, (000) Vo 81 0 - -6,1
(42)
Now, consider the update rule for 8, with sufficiently small step size 17 >
(((+1) (60) VV (0 !!0)
(43)
We want to show that for a sufficiently small 17, + is the unique minimizer of (00)) (.)).
By the Lipschitz continuity of (0, 0,) and the update rule, we have:
- L
((0(),0 01++1)) S (0((t)) 0(() Vo, (( 0 0(
-
(44)
Lij²
= 0!0) V/
0. (0()) ((t) )|
V x ( |1
(45)
2
= =
17 | (1-2) 2 Vo, ,
(46)
If we choose < 2. then (1-
In 2
> 0, and we have:
((0()) (((+1)) <
(47)
This implies that 05 (++1) is strict minimizer of (0 02).
To show that ( is the unique minimizer, suppose there exists another minimizer 0, + 0((+1) By the striet inequality above, we must have:
(00 0.) > 1000), ((++)))
(48)
which contradicts the assumption that 0, a minimizer.
Therefore, for a sufficiently small step size n < Z. the updated secondary parameters (1++)) are the unique minimum solution for the secondary loss function y(0f)) 0.).
Definition A.8. Let (xx be a function that is Lipschitz continuous with Lipschitz constant Ly, i.e. for any 31, 22!
(f())) f((2)lz Ly 02ll
(49)
Demographic Parity Loss Function: The demographic parity loss function DP(() is defined as:
DP(F) = |B EEnspateenn][((0]==]]]]
(50)
where a is sensitive attribute (c.g., race, gender) with two possible values (0 and 1), and p(xla) is the conditional probability distribution x given a.
Theorem A.9. If J (a: Lipschitz continuous with Lipschitz constant L,,, then the demographic parity loss function E (P(f) (f) is also Lipschitz continuous with Lipschitz constant LDP 2Lg.
17

---

Proof. Let fi far and (z) be two functions that are Lipschitz continuous with Lipschitz constant Ly. We want show that:
( - - LDpllfi - -fallo
(51)
where [l.fi - -21--- supal/ii)) f2(x)).
Consider the difference between the demographic parity loss functions:
(eDr(f))(DP(() = E Bx-p(p
- - 1))=
(52)
E s-nn( a 00 [z)) - E Ex-p(ala [f2()))]
B Bemp(sla=0) [fi [2(x) E Exxp(x (x) [2(x)]
(53)
B Bx-p(slemo)) fi f2(x)] || + Bn-p(z/am [1 f2(x))
(54)
Be-p(e)a=0) [4yl /2lloo] + -2=p(=)a==1) | L f2lloo]
(55)
Lyllfi - f2 (Be-p(a)a=0) [1] + E x(
(56)
= = - f2l120
(57)
LD f2lloo
(58)
The first incquality follows from the reverse triangle inequality, the second inequality trivial and the third inequality follows from the Lipschitz continuity of fi and f2. Therefore, the demographic parity loss function EDP(f) is Lipschitz continuous with Lipschitz constant LDp = 2..
A.2 Expansion to Equalized Odds (EO) difference
established the Lipschitz continuity of the demographic parity loss function. This approach can similarly be applied to another widely used fairness loss function, known as the equalized odds loss. The Equalized Odds Difference measures the extent which model's predictions deviate from equalized odds by quantifying differences in true positive rates (TPR) and false positive rates (FPR) across different groups. Mathematically, is defined follows:
In Theorem A.9
For true positive rate (TPR):
TPR Difference 1. - =1= [x)]
(59)
For false positive rate (FPR):
TPR Difference
(60)
The overall EO loss can then be considered as the maximum of these two differences:
EO Difference max(TPR Difference, FPR Difference)
(61)
Following the logic presented Theorem A.9 we can determine the Lipschitz constants LTPR and LFPr for the true positive rate and false positive rate, respectively. The Lipschitz constant for the cqualized odds loss can then expressed as max(LTPR, LFPR).
A.3 Assumption Discussion
Our work on the FairBiNN method introduces novel approach to addressing the faimess fairness-accuran accuracy trade-off in machine learning models through bilevel optimization. The theoretical foundations and empirical results demonstrate the potential of this method to outperform traditional approaches like Lagrangian regularization. However, it's crucial to examine how the underlying assumptions of our theory translate to real-world applications.
18

---

A.3.1 Convexity Near Local Optima
One key assumption in our theoretical analysis is the convexity of the loss function near local optima. In practice, this assumption translates to the behavior of neural networks as they converge during training. While neural network loss landscapes are generally non-convex, recent research suggests that they often exhibit locally convex regions around minima, especially in overparameterized networks 3. In real-world scenarios, as long as the network converges, it will likely encounter these locally convex regions along its optimization path. Our theory applies particularly well these parts of the optimization process. This assumption becomes increasingly valid as the network approaches convergence, which is typically the case for well-designed models trained on suitable datasets. Therefore, practitioners can rely on this aspect of our theory as long their models show signs of convergence on the given data.
A.3.2 Overparameteriiztion
The assumption of overparameeterization in model is another critical aspect that warrants discus- sion. In modern deep learning, overparameterized models - those with more parameters than training samples- increasingly common. This trend aligns well with theoretical framework. In practi- cal terms, as long as the model is capable of overfitting on the training data, this assumption stands. This condition is often met in real-world scenarios, especially with deep neural networks applied to typical dataset sizes. The overparameterization allows for the existence of multiple solutions that can fit the training data, providing the flexibility needed for bilevel optimization approach find solutions that balance accuracy and fairness effectively. However, it's important to note that in some resource-constraaied environments or with extremely large datasets, overparameterization might not always be feasible. In such cases, the applicability of our method may require further investigation or adaptation.
A.3.3 Lipschitz Continuity
The assumption of Lipschitz continuity is crucial for the stability and convergence properties of our optimization process. In our experiments, we ensured that the chosen layers and loss functions satisfy Lipschitz continuity. thus upholding this assumption. For practitioners, we provide rigorous analysis of various layers and activation functions in terms of their Lipschitz properties. This analysis serves as a guide for choosing components that maintain the Lipschitz continuity assumption. Common choices like ReLU activations and standard loss functions (e.g., cross-entropy) are Lipschitz continuous, making this assumption generally applicable in many practical scenarios. However, care must be taken when using certain architectures or custom loss functions. For instance, unbounded activation functions or poorly designed custom losses might violate this assumption. We recommend that practitioners refer to our provided analysis when designing their models to ensure compliance with this crucial property.
A.4 Practical Implications of Bounded Output Assumption
The assumption of bounded layer outputs translates to several practical considerations in neural network design and implementation:
A.4.1 Bounded Activation Functions
In practice, this assumption is often satisfied by using bounded activation functions:
Sigmoid function: bounded between 0 and
Hyperbolic tangent (tanh): bounded between -1 and 1
ReI.U6 variant of ReLU that capped
A.4.2 Normalization Techniques
Various normalization techniques help ensure that the outputs of layers remain bounded:
Batch Normalization: normalizes the output of a layer by adjusting and scaling the activations
19

---

Layer Normalization: similar to batch normalization but normalizes across the features
instead of the batch
Weight Normalization: decouples the magnitude of weight vector from its direction
A.4.3
Regularization
Certain regularization techniques indirectly encourage bounded outputs:
L1/L2 regularization: by penalizing large weights, these methods indirectly limit the magnitude of layer outputs
Dropout: by randomly setting some activations to zero, dropout can help prevent excessively large outputs
By implementing these techniques, practitioners can design neural networks that better align with the theoretical assumption of bounded layer outputs. This alignment potentially leads to more stable training and improved generalization properties, bridging the gap between theoretical guarantees and practical implementations.
A.5 Exploring Lipschitz Continuity in Activation functions
According our theoretical framework, activation functions that are Lipschitz continuous (smooth).
guarantee the pareto front solution when using
Activation functions (f) that are Lipschitz continuous have property where there exists constant L such that for any x, y:
If(x) f(y)ls L| - yll
(62)
As an example, we can prove sigmoid function is Lipschitz continuous. we need show that there exists constant I. such that for all E R:
(o(x)) aw)ls L|| yll
(63)
where (x) is the sigmoid function defined as:
(x)
(64)
1te=
Derivative of the Sigmoid Function
The first step in proving Lipschitz continuity to find the derivative of the sigmoid function, which gives us the rate of change. The derivative of the sigmoid function is:
d'(x) = oz)) -o)))
(65)
Since (xx is always between 0 and 1, the expression (zx) (z (x)) is maximized when azz) = 0.5.
Finding the Lipschitz Constant
The maximum value of a((x) occurs (xx) = 0.5, which gives:
d' d(xx) 0.5 0.5) 0.25
(66)
Therefore, the derivative of the sigmoid function is bounded by 0.25:
0 o' o'(x) < 0.25
(67)
This means that the Lipschitz constant L is 0.25, and Sigmoid Function is smooth.
Similar to this proof, we can show that following common activation functions are also Lipschitz continuous:
20

---

1. Linear: f(z) = I with constant L = 1
2. Hyperbolic Tangent (Tanh): f(x) = tanh(z) with constant L -=1 1
3. ReLU (Rectified Linear Unit): f(æ) - max(0, .) with constant -=]
4. Leaky ReLU: f(x) max(ox, x)) where small positive constant, with constant
L max(1, a)..
5. ELU (Exponential Linear Unit):
= J (ale 1),
if > 0 if S
ELU(x)
The ELU function Lipschitz continuous, but the constant depends on the value of
6. Softplus: (x) log(1 + e²) with constant L.
There some common activation functions that not Lipschitz continuous:
1. Softmmx
2. Binary Step:
11 i 2 0 10 if x <
BinaryStep(c))
3. Hard Tanh:
-1, if <
HardTanh(2))
Φ; 1,
if - if
4. Hard Sigmoid:
0,
if -2. if 2.5
HardSigmoid(a))
1,
0.2x
0.5,
if 2.5 < < 2.5
A.6 Exploring Lipsehitz Continuity in CNNs and GNNs
Convolutional Neural Networks
The Lipschitz continuity of CNN layer can be determined by examining its components: convolution operations, activation functions, and pooling layers. Convolution is a linear operation, and its Lipschitz constant is related the spectral norm of the convolution matrix, typically limited by the of the absolute values of the weights. Activation functions like ReLU and sigmoid are Lipschitz continuous, with constants of and 0.25, respectively. Pooling operations, such as max and average pooling, are also Lipschitz continuous, with max pooling having a constant of I and average pooling having constant dependent on pooling size. Therefore, CNN layer is Lipschitz continuous if all its components are, with the overall Lipschitz constant being the product of the constants of these components.
Zoe et al. 80 developed a linear programming approach to estimate the Lipschitz bound of CNN layers. Their method leverages concepts such as the Bessel bound, discrete signal processing, and the discrete Fourier transform to calculate the Lipschitz constant for each layer in popular architectures like AlexNet and GoogleNet.
Graph Neural Networks
Graph Neural Network (GNN) layers operate graph-structured data through series of message- passing, aggregation, and update steps, each contributing to the Lipschitz continuity of the layer. In the message-pan passing step, functions aggregate information from neighboring nodes and are often linear or involve nonlinearities; linear message-passingg functions are Lipschitz continuous, with constant depending on the weights and the graph's 's maximum degree. Aggregation functions, such as sum, mean, and max, are Lipschitz continuous, with sum and mean being linear, and max having constant of I. Update functions apply neural networks to aggregated information, and if composed
21

---

Hea th Canaset BCE lass
Ingranapan
0.45
SHievel
withour ralmess
Without Halmess
040
0.-
0.35
0.4
025
0.0-
100
200
0
50
125
130
200
bpoch
spach
(a) UCI Adult
(b) Heritage Health
Figure 2: BCE loss over epochs for the Lagrangian, Bi-level, and Without Fairness approaches on (a) the Adult dataset and (b) the Health dataset. These results illustrate that the Bi-level optimization framework achieves lower BCE loss compared to the Lagrangian approach in these experiments, highlighting its potential in optimizing both accuracy and fairness objectives in neural networks.
of Lipschitz continuous operations like linear transformations and activations such as ReLU, they maintain Lipschitz continuity. The overall Lipschitz constant of a GNN layer is influenced by the characteristics of the pessage-passing, aggregation, and update functions, as well as the graph's structure, such as node degrees.
recent study by Juvina et al. bound constraints across various GNN models. To facilitate easier computations, the authors utilize closed-form expressions of tight Lipschitz constant and employ constrained optimization strategy to monitor and control this constant effectively. Although this is not the first attempt to control the Lipschitz constant, the authors successfully reduce the size of the matrices involved by factor of K2 where K is the number of nodes in the graph. While previous works, such as Dasoulas et al. [6]. focused controlling the Lipschitz constant for basic attention-based GNNs, Juvina et al. (2 also extend this approach to enhance the robustness of GNN models against adversarial attacks.
32 presents learning framework designed to maintain tight Lipschitz-
A.7 Direct Comparison: Bilevel (FairBiNN) vs. Lagrangian Method
In this subsection, we present comprehensive comparison between our proposed FairBiNN method and the traditional Lagrangian regularization approach. This comparative analysis serves multiple purposes. Primarily, it aims empirically validate the theoretical advantages of the bilevel optimiza- tion framework outlined in our earlier analysis. By doing so, we demonstrate how the FairBiNN method translates theoretical benefits into practical performance gains in terms of both accuracy and faireess metrics. Furthermore, this comparison provides insight into the convergence behavior and stability of both methods under various hyperparameter settings. It illustrates the flexibility of the FairBiNN approach managing the trade-off between model accuracy and fairness constraints, a erucial aspect in real-world applications of fair machine learning.
We trained both models on the Adult and Health datasets, using the same network architecture, same number of parameters, and optimization settings. Figure 22 displays the BCE loss over epochs for the Adult dataset. The Bi-level approach demonstrates better performance compared to the Lagrangian approach, achieving lower BCE loss of approximately 0.23 by epoch 200, while the Lagrangian approach reaches a loss of about 0.26. Similarly, Figure 2b shows the BCE loss over epochs for the Lagrangian, Bi-level, and Without Fairness approaches the Health dataset.
Through this direct comparison, we aim to bridge the gap between theoretical analysis and practical implementation, showcasing how the principled design of the FairBiNN method leads to tangible improvements fair machine learning tasks. This offers practitioners a clear understanding of when and why they might choose the FairBiNN method over the Lagrangian approach in real-world scenarios. We trained both FairBiNN and Lagrangian models on the Adult and Health datasets, using the same network architecture (Same numeber of parameters) and optimization settings for fair
22

---

0.017
0.
sat.stoo by Dilieerce
2.2230
Soatiko Party Diffrrncc
8.0125
0.2220
02
0.0075
8.0000
9.0025
0.0
5.675
0.700
0.725
0.750
0.775
0.600
0.525
Accracy
(a) UCI Adult
(b) Heritage Health
Figure 3: Comparison of FairBiNN and Lagrangian methods on UCI Adult and Heritage Health datasets
comparison. For each method, we varied the fairness-accuracy trade-off parameter (n for FairBiNN, for Lagrangian) generate a range of models with different accuracy-faimen balances.
Figure 3 presents comparative analysis of the FairBiNN and Lagrangian methods on two benchmark datasets: UCI Adult (Figure Ba) and Heritage Health (Figure (bb). The graphs plot the trade-off between accuracy and Statistical Parity Difference (SPD), a measure of fairness where lower values indicate better fairness. For the UCI Adult dataset (Figure Ba), method consistently outperforms the Lagrangian approach. The FairBiNN curve is closer to the top-left corner, indicating that achieves higher accuracy for any given level of faimess (SPD). The difference particularly pronounced at lower SPD values, suggesting that FairBiNN is more effective maintaining accuracy while enforcing stricter fairness constraints. The Heritage Health dataset results (Figure 3b) show similar trend, but with more dramatic difference between the two methods. The FairBiNN curve dominates the Lagrangian curve across the entire range SPD values. This indicates that FairBiNN achieves substantially higher accuracy for any given fairness level, or equivalently, much better fairness for any given accuracy level. In both datasets, the FairBiNN method demonstrates a smoother, more consistent trade-off between accuracy and fairness. The Lagrangian method, contrast, shows more erratic behavior, particularly in the Heritage Health dataset where its performance degrades rapidly as fairness constraints tighten. These results empirically validate the theoretical advantages of the FairBiNN method discussed earlier the paper. They suggest that bilevel optimization approach is more effective at balancing the competing objectives of accuracy and fairness.
observe that the FairBiNN
A.7.1 Computational Complexity Analysis
Let's define the following variables:
n: number of parameters in 0p
m: number of parameters in 0₂
Cy: cost of computing f and its gradients
Cg: cost of computing á and its gradients
Regularization (Lagrangian) Method
The Lagrangian update rules
= aa 0.) A0(00, 0. = 0, aa-av (f(O), 0,) X0(0p,0.)
(68)
(69)
Computational complexity per iteration: OCC, + Co + +n+
Bilevel Optimization Method
23

---

The bilevel update rules are:
Lower level: 0, = 0, as Vo, ((),,0.) Upper level: 0p 0p o V 10p,
(70)
(71)
Computational complexity per iteration: OCC₁ + Co + +
Empirical Comparison
While the theoretical complexity analysis suggests similar costs for both methods, we conducted empirical tests to compare their actual runtime performance. reppots the average epoch time for both the Adult and Health datasets using the FairBiNN and Lagrangian methods after 10 epochs of warmup.
Table I: Average epoch time (in seconds) for FairBiNN and Lagrangian methods
Method
Adult Dataset (s) Health Dataset (s)
FairBiNN
0.62
1.03
Lagrangian
0.60
1.05
These experiments were conducted on an M1 Pro CPU. As we can observe from the results reported in table []] there is no tangible difference in the average epoch time between the FairBiNN and Lagrangian methods for both datasets. This empirical evidence aligns with our theoretical analysis.
A.8
Related works- Graph and Vision domains
A.8.1
Graph
The message- partgge-pasingggggg structure of GNNs and the topology of graphs both have the potential to amplify the bias. In general, in graphs such as social networks, nodes with sensitive features similar one another are more likely to link to one another than nodes with sensitive attributes dissimilar from one another 18 52. On social networks, for instance, persons of younger generations have a higher tendency to form friendships with others of a similar age [18] This results in the aggregation of neighbors' features in GNN having similar representations for nodes of similar sensitive information while having different representations for nodes of different sensitive features, which leads to severe bias in
in the sense that the predictions are highly correlated with the sensitive attributes of the nodes. GNNs have greater bias due to the adoption of graph structure than models that employ node characteristics [14]. Because of this bias, the widespread use of GNNs in areas such the evaluation of job candidates [44] and the prediction of drug- target interaction [72 (3) would be significantly hindered. As a result, it is essential to research equitable GNNs. The absence of sensitive information presents significant problems to the work that has already been done fair 39 [1 64). Despite the significant amount of work that has been put into developing
models 55] 13
adversarial debiasing 5 21 and fairness constraints 74 75 these models are almost exclusively designed for independently and identically distributed (i.i.d) data, meaning that they are unable to be directly applied to graph data due to the fact that they do not simultaneously take into consideration the bias that comes from node attributes and graph. In recent years, Bose and Hamilton (6), Rahman et al. [52) have been published to learn fair node representations from graphs. These approaches only deal with simple networks that do not have any properties on any of the nodes, and they place their emphasis on fair node representations rather than fair node classifications. Finally, Dai and Wang
fair models through the revision of features 33
34
78), disentanglement [3
14 used graph topologies and a restricted amount of protected attributes and designed FairGNN to reduce the bias of GNNs while retaining high node classification accuracy.
A.8.2 Vision
The challenges caused by bias in computer vision might appear various ways. has been found, for instance, that in action recognition models, when the data include gender bias, the bias is exacerbated by the models trained on such datasets 79 Face detection and recognition models may be less
precise for some racial and gender categories 7 Methods for mitigating bias in vision datasets
are suggested in (55 and [71]. Several researchers have used GANs on image datasets for bias reduction. Sattigeri et al. [57 altered the utility function of GAN in order to generate equitable picture
24

---

proposed fair mixup, which uses data augmentation mitigate bias in Table 2: Summary of Parameter Setting for the fairness layers on tabular datasets
Hyperparameters
UCI Adult
Health Heritage
FC layers before the fairness layers
2
2
Fairness FC layers
1
3
FC layers after the fairness layers
1
1
Epoch
50
50
Batch size
100
100
Dropout
0
0
Network optimizer
Adam
Adam
fairness layers' optimizer
Adam
Adam
classifier layers' learning rate
le-3
le-3
fairness layers' learning rate
le-5
le-5
100
100
datasets. FairFaceGAN 29 provides facial image-to-image translation to avoid unintended transfer of protected characteristics. Roy and Boddeti [56] developed method to mitigate information leakage on image datasets by formulating the problem as an adversarial game to maximize data utility and minimize the amount of information contained in the embedding, measured by entropy. Ramaswamy et al. [55] presents a methodology to generate balanced training data for each protected property by perturbing the latent vector of GAN. Other experiments using GANs to generate accurate data are (8).) Beyond GANs, many strategies have addressed the challenge of Al fairness.
[54 proposed a U-Net for creating unbiased image data. Deep information maximization adaption networks were employed to eliminate racial bias in face vision datasets [67]. while reinforcement
learning utilized for training race- race-balanced network (66). Wang al. 69 offer generative few-shot cross-domain adaptation method for performing fair cross-domain adaptation and enhancing minority category performance. The research in 77 recommends adding penalty term to the softmax loss function to reduce bias and enhance face recognition fairness performance. Quadrianto al. [51] describes a technique for discovering fair data representations with the same semantic information as the original data. There have also been effective applications of adversarial learning for this purpose 668
data.
A.9 Evaluation Metrics
We utilize four metrics to compare our model's performance against baseline models. Average precision (AP) is utilized to gauge classifier accuracy, combining recall and accuracy each point. In tabular and graph datasets, we opt for accuracy to align with existing literature practices.
Fairesss evaluation draws from various criteria, with demographic parity (DP) being widely used. DP quantifies the difference favorable outcomes across protected groups, expressed as P = 1|$
0) - P(Y 1|s 1)l) 44). For scenarios involving more than two groups, DP can be calculated as (a y) = max,,.a, a, Plū 1/a = a;) P 1/a = ay))l (25]. A smaller DP indicates fairer categorization. We also adopt the difference in equality of opportumity (AEO), which measures the absolute difference in true positive rates between gender expressions ([TPR(S 0) -)-TP 1)l)
40 [55) Minimizing AEO signifies fairer outcomes. Demographic parity serves the fairness criterion in our optimization. Discrepancies between EO and DP may occur due to this choice.
A.10
Implementation details
The hyperparameters used in training the models on each dataset can found in the tables and The training was conducted on a computer with NVIDIA GeForce RTX 3090.
25

---

Table 3: Summary of Parameter Setting for the fairness layers on graph datasets
Hyperparameters
POKEC-Z
POKEC-N
NBA
GCN layer before the fairness layers
2
2
Fairness FC layers
1
1
FC layers after the fairness layers
1
Epoch
5000
1000
1000
Batch size
1
Dropout
0
0.5
0.5
Network optimizer
Adam
Adam
Adam
fairness layers' optimizer
Adam
Adam
Adam
classifier layers' learning rate
le-3
le-3
le-2
fairness layers' learning rate r
le-6
le-8
le-5
1000
100
1000
Table 4: Summary of Parameter Setting for the faimess layers on vision dataset
Hyperparameters
CelebA-Attractin CelebA CelebA-Smiling
CelebA WavyHair
Fairness FC layers
1
FC layers after the fairness layers
1
Epoch
30
15
15
Batch size
128
128
128
Dropout
0
0
Network optimizer
Adam
Adam
Adam
fairness layers' optimizer
Adam
Adam
Adam
classifier layers' learning rate
le-3
le-3
1e-3
fairness layers' learning rate "
1e-6
le-5
1e-5
1000
100
100
A.11 Other domains' results
A.11.1 Graph
We compare our suggested framework with some of the cutting-edge approaches for fair classification, and fair graph embedding learning including ALFR (2I), ALFR-e, Debias [77), Debias-e, and FCGE (6) In the ALFR (21] approach, which is pre-processing technique, the sensitive information in the representations created by an MLP-based autoencoder is eliminated using a discriminator. Then the used to train the linear classifier. ALFR-e method make use of the graph structure information and joins the user features in the ALFR with the graph embeddings
debiased representations
discovered by deepwalk [48]. Debias [77), is a fair categorization technique used throughout processing. It immediately applies a discriminator to the predicted likelihood of the classifier. Debias- which is similar the ALFR-e, also includes deepwalk embeddings into the Debias characteristics.
FCGE [ is suggested as a method for learning fair node embeddings in graphs without node characteristics. FairGCN [14), graph convolutional network designed for fairness in graph-based learning. It incorporates fairness constraints during training to reduce disparities between protected groups. FairGAT 37 fairness-aware graph-based learning framework that employs novel attention learning strategy to reduce bias. This framework is grounded a theoretical analysis that identifies sources of bias in GAT-based neural networks used for node classification. NT-FAIRGNN [15] is a graph neural network that aims to achieve fairness by balancing the trade-off between accuracy and fairness. It uses a two-player minimax game between the predictor and the adversary, where the adversary aims to maximize the unfairness. Discriminators screen out the delicate data in the embeddings. We used the Dai and Wang [14 study's 's obtained datasets for our investigation which are as follows:
Pokec [62 is among the most well-known social network datasets Slovakia which resemble Facebook and Twitter greatly. This dataset includes anonymous information from the whole social network of the year 2012. User profiles on Pokec include information on gender, age, interests, hobbies, profession, and more. There millions of users in the original Pokec dataset. Sampled
26

---

Table 5: The comparisons of our proposed method with the baselines Pokec-z
METHOD
ACC(%)
AUC(%)
AD)(%)
A eo (%)
ALFR 21
65.4 #0.3
71.3 #0.3
2.8 #0.5
1.1 #0.4
ALFR-E
48
60.6
70.0
5.8 #0.4
2.8 #0.8
DEBIAS
65.2
71.4
1.96
1.94
DERIAS-E
48
67.5
74.2
4.7
3.0
FCGE G
65.9 +0.2
71.0
3.1
1.7
FAIRGCN
14
40.3
76.7 #0.2
0.9
1.7 +0.2
FAIRGAT
37
#0.1
40.2
0.5
0.8 20.3
NT-FAIRGNN 15
70.0 +0.1
40.3
1.0
10,2
GAT+FAIRBINN (OURS)
70.97 #0.16
77.58=0.11
0.93 #0.44
0.97 #0.40
Table 6: The comparisons of our proposed method with the baselines on Pokec-n
METHOD
ACC(%)
AUC(%)
ADP(%)
Ago (%)
ALFR 21
63.1 #0.6
67.7 #0.5
3.05 20.5
3.9 40.6
ALFR-E
+0.5
71.9 40.3
4..5
41.6
DEBIAS 77
62.6
67.9 #0.7
2.7
+1.0
DEBIAS-E
48
60.6
71.7 #0.7
3.6
4.4
FCGE 6
64.8
69.5 #0.4
4.1
5.5 +0.9
FAIRGCN
14
70.2
74.940.4
0.8
1.1
FAIRGAT
37
70.2
74.9
0.6
+0.2
NT-FAIRGNN
15
70.1 #0.2
74.9 #0.4
0.8 +0.2
1.1 #0.3
GAT+FAIRBINN (OURS)
70.07 20.5
75.8 20.38
20.14
3.0
Pokec-z and Pokec-n datasets based on the provinces that users are from. The categorization task involves predicting the users' working environment. NBA is Kaggle dataset with about 400 NBA basketball players that served as the basis for this extension. Players' 2016-2017 season success statistics, along with additional details like nationality, age, and income are presented. They gathered the relationships between NBA basketball players on Twitter using its official crawling API to create the graph connecting the NBA players. They separated the nationality into two groups, American players and international players, which is sensitive characteristic. The classification job is to predict whether player's 's wage is above the median.
Each experiment was conducted five times, and Tables 5 and 7 report the mean and standard deviation of the runs for Pokec-z, Pokec-n, and NBA datasets, respectively. These results represent the selected Pareto solutions for comparison with the benchmarks. The tables reveal that, in comparison GAT, generic fair classification techniques and graph embedding learning approaches exhibit inferior classification performance, even when utilizing graph information. In contrast, our Bilevel design performs comparably to baseline GNNs. FairGCN close to the baseline, but the FairBiNN approach outperforms it. When sensitive information is scarce (e.g., NBA dataset), baselines exhibit clear bias, with graph-based baselines performing worse. However, our proposed model yields near-zero statistical demographic parity, indicating effective discrimination mitigation.
A.11.2 Vision
We compare our method on vision task with (1) empirical risk minimization (ERM). which accom- plishes training task without any regularization, (2) gap regularization, which directly regularizes the model, (3) adversarial debiasing [77). and (4) Fairmixup 9 To showcase the effectiveness of
our method, we employed the CelebA dataset of face attributes 38 which comprises over 200,000
images of celebrities. Each image in the dataset has been assigned 40 binary attributes, including gender, that were labeled by humans. We selected the attributes of attractive, smiling, and wavy hair and used them in three binary classification tasks, with gender serving as the protected attribute. The reason we chose these three attributes that each of them has group that is sensitive them and receives disproportionately high number of positive samples. We trained a ResNet-15 model [2
27

---

Gapfog
Maup Maniold
Advertarall Debiis
0.10
0.30
Haup
Siaticcal Panto Comeentc
Euelted Oudd Offfrrence
0.25
0.08
0.35
0.04
0.30
0.02
RAYERNICUSS
GapRog
0.00
Kleup
0.78
0.80
0.22
0.24
0.85
0.90
9.75
0.73
0.80
0.82
0.84
0.05
0.88
AP
AP
(a) Attractive AP vs. A DP
(b) Attractive AP vs. A EO
Figure 4: Attractive Attribute of CelebA Dataset as the Target Autribute. (a) reflects the trade-off between Average Precision and Demographic Parity Difference. (b) shows the trade-off between Average Precision and Equalized Odds Difference.
GapReg
Stabvitaal Par 1y (iireeee
0.12
Eartteed Oods Difieecc
0.04
0.00
0.03
0.08
0.02
0.02
0.1
Malp Manfold
GarReg Advertarial Deb
Mhop
ERN
0.00
(a) Smiling AP vs. A DP
(b) Smiling AP vs. A EO
Figure 5: Smiling Attribute of CelebA Dataset as the Target Atribute. (a) reflects the trade-off between Average Precision and Demographic Parity Difference. (b) shows the trade-off between Average Precision and Equalized Odds Difference.
Gapfleg Adivrrania Debias
Capllgg Advertanaa Deb as ERN
0.30
Mixup Manitais
Mhup Manfold
Иaxup
ERM
0.14
3ara4bcc ParRy Ditieree
Equatted 0d5 Offectee
0.12
0.20
0.25
0.00
0.08
0.16
0.2
0.00
0.05
0.70
3.75
0.00
0.05
0.85
0.70
0.75
0.00
0.85
AP
AP
(a) Wavy Hair AP vs. A DP
(b) Wavy Hair AP vs. A EO
Figure 6: Wavy Hair Attribute of CelebA Dataset as the Target Attribute. (a) reflects the trade-off between Average Precision and Demographic Parity Difference. (b) shows the trade-off between Average Precision and Equalized Odds Difference. The FairBiNN method showing competitive results to the baseline.
28

---

Table 7: The comparisons of our proposed method with the baselines NBA
METHOD
ACC(%)
AUC(%)
AD)(%)
A ABo (%)
ALFR
61.3
20.3
2.3 #0.9
3.2 11.5
ALFR-E
48
60.0
12.
4.7
11.7
DEBIAS
7
63.1
71.3
2.5
11.
DEBIAS-E
48
65.6
12.9
5.3
3.1
FCGE G
66.0
73.6
2.9
3.2
FAIRGCN
14
71.1
70.0
1.0
1.24
FAIRGAT
37
70.5
20.7
0.7
0.7
NT-FAIRGNN 15
71.1
+0.3
1.0
1.24
GAT+FAIRBINN (OURS)
77.09 #0.45
77.99 #0.58
0.34 #0.21
12.78 42.9
(a) Without the faimess layers
(b) With the faimess layers &
Figure 7: CelebA Dataset - t-SNE visualization of a and v labeled with gender classes. The invariant encoding ? shows no clustering by gender. These plots are generated using attractive attribute.
for each task and added two additional layers to predict the outcomes. The trade-off between Average Precision (AP), Demographic Parity (DP), and Equality of Opportunity (EO) for attributes "Attrac- tive", "Smiling". and "Wavy Hair" is illustrated in the figures 4 55 6nd method provides a more balanced trade-off between accuracy and faimess. Instead of prioritizing one over the other, our method strikes better balance, ensuring that the trained model is both accurate and fair. Moreover, the FairBiNN model consistently provides better equality of opportunity across various accuracy levels compared to benchmark models. Through empirical validation on multiple benchmarks, we've shown that the FairBiNN approach consistently outperforms other methods in achieving equality of opportunity across various accuracy levels. This indicates that our method can provide fair treatment different protected groups while still maintaining high predictive accuracy.
Our proposed
Furthermore, we demonstrate the power of the bilevel design for fairness by visualizing the t-SNE plot. The t-SNE visualization of (output of the ResNet-18 18 before the classification layer without the fairness layers) and ž (output of the ResNet-18 before the classification layer with the Bilevel fairness) are shown in Figures 7a and 7b] demonstrating that z clusters by gender, but does not. Further insights about ablation study outcomes are detailed in section A.13
A.12 Architecture visualization
To better illustrate the training process outlined in Algorithm I present the network architecture in Figure g
A.13
Ablation Study
A.13.1
Impact of on Model Performance
To understand the sensitivity of our FairBiNN model to the choice of 1), we conducted ablation study on both the Adult and Health datasets. The parameter 7/ controls the trade-off between accuracy and fairness in our bilevel optimization framework.
29

---

Accuracy Weight Optiminer, Loss: RCF
Output Layer (class)
Input Layer
Falmuu paranaties
Acourecy parsaccirss
Fairness player's Layers
Anaray parmenter
Accuracy player's Layers
Figure 8: The FairBiNN network architecture illustrating the process described in Algorithm
0.0
0.200
0.8
0.05
0.30
Accur007
8
0.200
0.075
0.75
0.80
0.000
0
0.78
0.70
11
0.000
500
1005
1500
2005
2500
3000
(a) 17 ablation study on Adult dataset
(b) 17 ablation study on Health dataset
Figure 9: Ablation study on the impact of 7 parameter across two datasets. (a) Results on the Adult dataset showing the effect of different 17 values. (b) Similar analysis conducted on the Health dataset, demonstrating how influences model performance.
A.13.2 Experimental Setup
We varied !) across range values: 1-6000 for Adult dataset, and 1-3000 for Health dataset. For each value of 7), trained the FairBiNN model on both the Adult and Health datasets, keeping all other hyperparameters constant. We evaluated the models based on accuracy and demographic parity (DP).
A.13.3 Results
Figures 9a (bssow the results our ablation study for the Adult and Health datasets, respectively. The results demonstrate clear trade-off between accuracy and fairness varies. For both datasets: As 1) increases, the demographic parity (DP) decreases, indicating improved fairness. However, this improvement fairness comes at the cost of reduced accuracy. The relationship is linear; there are diminishing returns in fairness improvement as increases, especially at higher values. For the Adult dataset, setting 77 1000 appears offer good balance, achieving DP of 0.012 while maintaining an accuracy of 82.9%. For the Health dataset, 7 700 also provides reasonable trade-off with DP of 0.23 and accuracy of 80.2%. These results highlight the importance of carefully tuning to achieve the desired balance between accuracy and fairness. The optimal value may vary depending on the specific requirements of the application and the characteristics of the dataset.
This ablation study demonstrates that our FairBiNN model provides flexible framework man- aging the eecuracy-fairnss trade-off through the y parameter. Practitioners can adjust y based on their specific fairness requirements and acceptable accuracy thresholds. Future work could explore adaptive schemes for setting during training automatically find an optimal balance.
30

---

0.025
0.45
0.40
SIaRIE Tarty biffeeeiee
0.020
0.25
0.0
o
0.035
0.30
0.25
0.010
AA
0.35
8
0.10
0.030
0.805
0.80
0,15
0.820
0.825
0.630
0.75
5.78
5.72
0.34
5.76
0.78
0.80
0.82
Accuracy
Accuracy
(a) Fairness layer position ablation study Adult (b) Fairness layer position ablation study on Health dataset
dataset
Figure 10: Fairness Layers Position (7), where indicates the i - Lh hidden layer
Position of Fairness Layers To understand the impact of the position of fairness layers within the network architecture, we conducted ablation study varying their placement. This study aims to identify the optimal position for faimess layers and provide insights into why certain positions may be more effective.
Experimental Setup
We tested 4 fairness layer positions on the Adult dataset and 5 fairness layer positions the Health dataset. For each configuration, we kept the total number of parameters constant to ensure a fair comparison. We evaluated the models based on accuracy and demographic parity (DP).
Results Figures and 1Oblshow the results of our ablation study for the Adult and Health datasets, respectively.
The results consistently show that placing the fairness layers just before the output layer (in the last hidden layer) yields the best performance in terms of both accuracy and faireess. This configuration achieves the highest accuracy while maintaining the lowest demographic parity on both datasets.
Several factors contribute to the superior performance of faimess layers when placed in the last hidden layer:
Rich Feature Representations: By the time the data reaches the last hidden layer, the network has already learned rich, high-level feature representations. This allows the fairness layers to operate on more informative features, potentially making easier to identify and mitigate biases.
Minimal Information Loss: Placing faimess layers earlier in the network might lead to loss of important information could be useful for the classification task. By positioning them at the end, we ensure that all relevant features are preserved throughout most of the network.
Direct Influence on Output: Being closest to the output layer, fairness layers in this position have the most direct influence the final predictions. This allows for more effective bias mitigation without excessively disturbing the learned representations earlier layers.
Gradient Flow: In backpropagation, gradients from the fairness objective have a shorter path to travel when the fairness layers are near the output. This might lead to more stable and effective updates for bias mitigation.
Adaptability: Fairness layers at the end of the network can adapt to various biases that might emerge from complex interactions in earlier layers, providing a final "correction" before the output.
A.13.4 Number and Type of Fairness Layers
In this subsection, we perform an ablation study to investigate the effects of different functions for the faimess layers. The faimesss layer can be any differentiable function with controllable parameters
31

---

Table 8: Area over the curve of statistical demographic parity and accuracy for model ablation
METHOD
UCI ADULT
HERITAGE HEALTH
ONE LINEAR LAYER
0.411
0,492
TWo LINEAR LAYERS
0.404
0,513
THREE LINEAR LAYERS
0.349
0.531
denoted as 02. We experimented with three configurations for the fairness layers: one linear layer, two linear layers, and three linear layers on tabular datasets. The results of the ablation study are summarized in Table g
For the CelebA dataset, we explored three types of fairness layers: linear layers, Residual Blocks (ResBlocks), and Convolutional Neural Network (CNN) layers. The mean scores of each category of CelebA attributes for each type of fairness layer are provided in Table[9]
The justification for the performance differences between the ResBlock and the fully connected models in our ablation study lies in the proportion of the model occupied by the fairness layers and the specific contributions of these layers to different parts of the network. In particular, there are two primary factors that explain the observed performance differences:
Role in the Network: The ResBlock and the fully connected modules serve different purposes within the network. The ResBlock contributes to the embedding space of the image, which includes feature extraction and representation learning. This enables the model to capture the essential characteristics of the image while minimizing the effect of the protected attributes (e.g.. gender) on the classification task. In contrast, the fully connected module is mainly involved in the classification part of the network, where it contributes to the decision-making process based on the features extracted from the previous layers. This distinction in roles explains why the ResBlock provides more fair results, as it directly affects the representation learning and reduces the influence of the protected attributes on the embeddings.
The flow of data through the ResBlock is different from the flow through the fully connected and CNN modules. ResBlocks have skip connections that allow the input to bypass some layers and directly flow to the subsequent layers. These skip connections help preserving the original information and preventing the loss of critical features during the network's ' forward pass. As result, the ResBlock is more effective in capturing the inherent relationships in the data while mitigating the bias from the protected attributes (28). In contrast, CNNs involve multiple convolution and pooling operations, which can cause the loss of some information relevant to fairness. The fully connected module, with its dense layers, lacks the skip connections present in the ResBlock, which can lead to less effective bias mitigation.
Flow of Data:
In conclusion, our ablation study demonstrates that the choice of layer in the fairness layers can significantly impact the fairnsss and accuracy of the model. essential to strike a balance between fairness and accuracy and to select the appropriate fairness layer for the specific dataset and application at hand.
Table 9: Accumulative comparison between different fairness layers
CNNBlock
AP
ADP
AEO
One Linear Layer
0.646
0.072
0.084
CNN Res Block
0.568
0.04
0.126
CNN Layer
0.617
0.058
0.099
A.14 Ethical & Broader Social Impact
This work introduces a novel bilevel optimization framework for multi-objective optimization in neural networks. While we use fairness as case study, it's important to note that our method is not inherently a fairness research technique, but rather general optimization approach that can be applied to various secondary objectives.
32

---

In the context of our fairesss case study, our FairBiNN method shows promising results in optimizing the trade-off between demographic parity measure of group fairness) and accuracy. While these results are encouraging, it is crucial to consider the broader ethical implications and potential societal impacts of applying this technique to fairness problems.
On the positive side, when applied to fairesss, our approach could help reduce discriminatory outcomes in high-stakes automated decision making systems, promoting more equitable treatment across protected groups in domains like hiring, lending, and healthcare 44 76 By providing
flexible framework to manage the accuracy- fairness trade-off, practitioners can fine-tune models to meet specific fairness requirements mandated by regulations or organizational policies.
However, must also consider potential negative consequences. There is risk that mathematical notions of fairness like demographic parity could provide a false sense of ethical assurance, when fairness is complex social and philosophical concept that cannot be fully captured by simple statistical measures (20) 50) Our focus on group fairness, while important, does not guarantee
individual fairness 773
The mathematical formulation we present, while rigorous, should not be seen as providing absolute ethical guarantees when applied to fairness problems. Over-reliance our method without careful consideration of the broader context could lead to unintended harms 9 39].
As with any machine learning technique, it is the responsibility of practitioners to properly configure and test models for their specific use cases. Our method provides additional tools for optimization but does not absolve practitioners of their ethical obligations. We believe that providing more precise control over trade-off's actually enables more ethical implementations. However, practitioners must be aware that while our approach performs well in our experiments, particularly in balancing demographic parity and accuracy, real-world applications may present unforeseen challenges and edge cases [3 49].
Ultimately. while our work provides a useful tool for multi- colti-oobeectvee optimization neural networks, its application to fairness should not be seen as a complete solution to algorithmic bias. Continued interdisciplinary collaboration between computer scientists, ethicists, policymakers and impacted communities is essential to develop AI systems that are truly fair and beneficial to society 66 60 71]
33

---

1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification:
The abstract and introduction section carefully convey the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We mentioned all limitations and future directions in section 5
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It acknowledged judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer:
[Yes]
Justification: We mentioned all the assumptions and proofs in the methodology 3 and the appendix, section A.]
34

---

Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: All the required details for reproducing this model have been provided. One can use this information to implement the idea even without the code.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (c.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
35

---

Answer: [Yes]
Justification: All the data and code are publicly available. We can provide the link at any time. For the review version, did not include the link keep as an anonymous review. The supplementary Zip file including data and code has been attached.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nnis.co
htapii//guudds//oeeuubbbbii sos.essoubisssioooooooooo for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https for more details.
/ //nips.c//pppp
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please refer section A.10
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
We used four common metrics from the literature that were used for evaluation and comparison. Also, we compared the performances as fair as possible. The significance of our work can be easily interpreted with provided tables and figures.
Justification:
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
36

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Section A.I0 describes the computational power used for this paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
No possible violation exists in this study.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
37

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All the assets in this work, either are original or are publicly available (e.g. datasets).
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt
package should be provided. For popular datasets, paperswi thcode. cww/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
38

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
The link to the Github repository including code and datasets will be provided after acceptance. Due anonymous review, cannot share the link beforehand. The supplementary Zip file including data and code has been attached.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
39


Reviews:
[{"review_id": "kvjoHuu8p6", "reviewer": "Reviewer_J3Wm", "summary": "This paper proposes a novel bilevel optimization framework called FairBiNN for addressing bias and fairness issues in machine learning models while maintaining accuracy. The approach formulates the problem as a Stackelberg game between accuracy and fairness objectives, proving that it yields Pareto-optimal solutions under certain assumptions. Theoretical analysis shows the method performs at least as well as Lagrangian approaches. Experiments on tabular, graph, and vision datasets demonstrate competitive or superior performance compared to state-of-the-art fairness methods in balancing accuracy and fairness metrics like demographic parity and equality of opportunity.", "questions": "1. How sensitive is the method to the choice of hyperparameters, especially $\\eta$?\n2. How does the computational complexity compare to the baseline methods?\n3. Are there any scenarios where the method might not perform as well as traditional approaches?", "limitations": "1. The paper does not explicitly discuss limitations of the proposed approach.\n2. Experiments are limited to a few datasets per domain; more diverse datasets could strengthen generalizability claims.\n3. The approach is not tested on very large-scale datasets or models", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Proposes a novel bilevel optimization framework with theoretical guarantees for fairness-aware machine learning.\n2. Provides rigorous theoretical analysis and proofs for the proposed method.\n3. Demonstrates strong empirical results across multiple domains (tabular, graph, vision) and datasets as shown in both main text and appendix. \n4. Compares against numerous state-of-the-art baselines.\n5. Conducts thorough ablation studies to analyze different components.\n6. Provides clear implementation details and hyperparameters for reproducibility.\n7. Visualizes results effectively through plots and t-SNE visualizations.", "weaknesses": "1. The bilevel optimization framework may be more complex to implement and understand compared to traditional methods.\n2. The theoretical guarantees rely on several assumptions that may not hold in all practical scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful feedback. We have addressed your questions and concerns below. If you have any further questions, we would be glad to discuss them.\n\n## Weaknesses\n### W1\nWhile our approach may initially appear more complex than traditional methods, we believe its benefits significantly outweigh this drawback. Our experiments demonstrate that the proposed method offers greater training stability compared to common adversarial training techniques in fairness research. The optimization can be implemented through a customized training loop with separate optimizers for accuracy and fairness, which is straightforward in modern deep learning frameworks.\n\nThe bilevel framework enhances interpretability by clearly separating accuracy and fairness objectives. This separation allows for more fine-grained control over parameters compared to fairness regularization methods, enabling precise tuning of the trade-off between objectives. This level of control is particularly valuable in sensitive applications where balancing fairness and accuracy is crucial.\n### W2\nWe appreciate your insightful comment regarding the assumptions underlying our theoretical guarantees. We have addressed assumptions' discussion in global rebuttal.\n\n## Questions\n### Q1\nWe have included ablation study in attached PDF (Fig. 1) showing the impact of varying the parameter $\\eta$ on the FairBiNN model. The results revealed a trade-off: increasing $\\eta$ enhanced fairness but decreased accuracy, with diminishing returns at higher values. This underscores the importance of tuning $\\eta$ to achieve an optimal balance between accuracy and fairness, tailored to the specific needs of each application.\n\n### Q2\nIn our paper, we focused our theoretical analysis and direct comparison on the Lagrangian regularization method, as it serves as a natural baseline for our bilevel optimization approach.\nWe'd like to emphasize that the computational complexity of our approach is equivalent to that of the Lagrangian method. Both methods use the same number of parameters and require similar computational resources per iteration. This equivalence in complexity allows for a fair and direct comparison between the two approaches.\nWe have included a detailed theoretical analysis of the computational aspects for both our method and Lagrangian regularization in global rebuttal. Additionally, we report practical execution times for both approaches in our experimental results. These comparisons provide a clear picture of how our method compares to Lagrangian regularization in terms of computational efficiency.\n\nDirect complexity comparisons with other baseline methods are challenging due to their varying approaches to ensuring fairness. By focusing on Lagrangian regularization, we provide a clear comparison of our method’s computational efficiency relative to a similar approach. This comparison highlights the practical benefits of our bilevel optimization framework while maintaining comparable computational efficiency.\n\n### Q3\nWhile our approach has shown strong performance across various scenarios, there are indeed cases where it might face challenges or not perform as well as traditional approaches.\nOne notable scenario is multiclass classification, particularly when using softmax activation in the output layer. The softmax function is not Lipschitz continuous, which is one of the key assumptions in our theoretical framework. This lack of Lipschitz continuity could potentially lead to instability in the optimization process or reduced performance compared to traditional approaches in multiclass settings. Also, in non-stationary environments, Our current framework assumes a static dataset. In scenarios where data distributions change over time, the method might require adaptations to maintain its performance.\n\n## Limitations\n### L1\nWe have added the limitations and future work section in global rebuttal.\n### L2 and L3\nWe appreciate the reviewer's observation regarding the number of datasets used in our experiments. We acknowledge that testing on a wider variety of datasets could indeed strengthen generalizability claims. However, our dataset selection was deliberate and constrained by several factors in the field of fairness research.\n\nWe chose widely used benchmark datasets that are well-established in fairness tasks, such as UCI Adult and Heritage Health for tabular data, Pokec and NBA for graph data, and CelebA for vision. These datasets allow for direct comparison with existing methods. It's important to note that the number of high-quality benchmark datasets specifically designed for fairness tasks is somewhat limited in the field, posing a challenge for all researchers in this area.\n\nAs our primary contribution is a theoretical framework, our main goal was to demonstrate that our theoretical results hold in practice. The selected datasets, covering a range of domains and data types, serve this purpose well. While we agree that more datasets could potentially strengthen generalizability claims, we believe that the diversity of domains we covered provides strong evidence for the broad applicability of our method.\n\nWe are actively seeking to apply our method to additional datasets as they become available as part of our ongoing research. We believe our current results, spanning multiple domains and dataset types, provide a solid foundation for the practical utility of our theoretical framework. However, we welcome and encourage further testing and application of our method on diverse datasets by the research community to further validate its generalizability.\n\n### Flag For Ethics:\nAll datasets used in this study are publicly available, and no human subjects were involved in the research.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lIssujqO7s", "reviewer": "Reviewer_QoV4", "summary": "A common approach to bias mitigating in machine learning is to add a regularization term to the loss function that penalizes deviation from fairness. This is what the current paper calls the Lagrangian approach. It has long been know that this is not an effective approach to multi-objective optimization. The paper proposes to use bilevel optimization to tackle this challenge.", "questions": "*Line 51: \"we introduce a novel method that can be trained on existing datasets without requiring any alterations\". Is this an accurate characterization? There certainly is a modification to the training algorithm?\n*\"Leader-follower\" terminology: is this standard in bilevel optimization? Can you add some references?\n*In assumption 3.3, $\\hat \\theta_s$ appears to be undefined\n*Could you give some example of common activation functions that are Lipschitz continuous? This would help the reader understand how realistic Lemma 3.5 is for common activation functions.\n*Line 208: \"demographic loss function of given layers...\" what do you mean by \"of given layers\"? \n*Is there an equivalent of Theorem 3.11 for other fairness metrics considered in your paper? Like equal opportunity? or Equal odds?\n*Line 219: \"in practice, the model $f$ can be implemented as a neural network with separate layers for accuracy and fairness\". This is one of my main confusions about the paper. How does one decide which layers are accuracy layers and which ones are fairness layers? Some of your experiments concern NNs with just one hidden layer, so then which are accuracy and fairness layers there?\n*Line 298: \"The theoretical analysis, particularly Theorem 4.6, establishes the properties of the optimal solution under certain assumptions, which are not limited to specific datasets or network architectures.\" I could not find Theorem 4.6. I also think your theoretical analysis is very limited due to the lipschitz continuity assumption. Most NNs used in practice woudl violate this assumption?", "limitations": "I would appreciate a further discussion on the limitations of the theoretical analysis. In particular, I believe the theory is highly limited in terms of the type of NNs it can be applied to.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper addresses an important problem in machine learning fairness. Rather than devising a novel fairness metric as is often the focus in this field, this paper studies a more fundamental problem: how do we get solutions on the Pareto front?\n\nThe technique employed, bilevel optimization, has not been previously applied in this context, to the best of my knowledge. (However, I did not get a good understanding of the history of bilevel optimization from this paper.)\n\nThe method, as presented in Algorithm 1, is alluringly simple.", "weaknesses": "The following identified weaknesses could be due to my own misunderstanding:\n\n*I could not understand how one would decide on which are accuracy parameters and which are fairness parameters in a neural network architecture. I could not find guidance on this in the paper.\n\n*Except for the simplest types of neural networks, I can't see how a general neural network can satisfy Assumption 3.4\n\n*Where does bilevel optimization sit in the larger context of multi-objective optimization techniques? I think this needs to be explicitly spelled out.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful feedback. We have addressed your questions and concerns below. If you have any further questions, we would be glad to discuss them.\n\n## Weaknesses\n### W1\nIn our approach, the separation of accuracy and fairness parameters is a key design choice that allows for the bilevel optimization. Typically, accuracy parameters ($θ_a$) are those in the main network layers directly contributing to task performance, while fairness parameters ($θ_f$) are in additional layers specifically designed to address fairness concerns.\nTo address this lack of clarity, we have added a new figure to the paper that visually illustrates how accuracy and fairness parameters are typically separated in different networks (Attached PDF, Fig. 3).\nFor instance, in a simple feedforward network for binary classification, the structure might be:\n\nInput -> Main Layers ($θ_a$) -> Fairness Layers ($θ_f$) -> Output Layer ($θ_a$)\n\nThe architecture of our approach can vary based on the specific task and dataset, and the separation of accuracy and fairness parameters may not always be straightforward. Practitioners may need to experiment with different configurations to find the most effective split for their use case. This flexibility is a strength of our method, allowing adaptation to various network designs. To enhance clarity, we added an ablation study examining the impact of fairness layer positioning within the network architecture (Attached PDF, Fig. 2), offering insights into how design choices affect model performance and fairness. We appreciate the reviewer’s feedback, which improved the clarity and applicability of our work.\n### W2\nWe appreciate the reviewer’s insightful observation regarding Assumption 3.4 and its applicability to general neural networks. While this assumption may initially seem restrictive, we have added a section to the paper explaining how it can be met in practice. Specifically, Assumption 3.4 can be satisfied through common techniques such as:\n* Bounded Activation Functions: Many popular activation functions, such as sigmoid, tanh, and ReLU (when combined with proper weight initialization), naturally bound the output of each layer.\n* Regularization methods\n* Normalization techniques\n\nThese approaches are widely used in modern neural networks, ensuring that many architectures inherently satisfy or can easily be modified to meet this assumption without significantly altering their structure or performance.\n### W3\nThank you for your suggestion! We have modified the related work to place our work better in the existing literature.\n\n## Questions\n### Q1\nYou are correct that our original statement could have been more precise. We have edited it to \".. without requiring any alterations to the data itself (data augmentation, perturbation, etc).\"\n\nUnlike some fairness approaches that require data augmentation or perturbation, our method operates on the original datasets without alterations. This allows for direct application to existing datasets without preprocessing, avoiding additional biases or changes to the data distribution. Our approach focuses on modifying the learning process, not the data, which is a key strength of our method.\n### Q2\nWe have added relevant papers and refined the related work section accordingly.\n### Q3\n$\\hat{\\theta}_s$ represents the updated values of the secondary objective's parameters.\n### Q4\nGreat insight! We have added a discussion and examples of different activation functions and layers in the appendix. In summary, Linear, Sigmoid, Tanh, ReLU, Leaky ReLU, ELU, and Softplus are Lipschitz continuous, while Softmax, Hard Tanh, and Hard Sigmoid are not.\n### Q5\nThe revised statement now reads:\n\"We showed that the demographic parity loss function, when applied to the output of neural network layers, is also Lipschitz continuous (Theorem 3.10)\"\n### Q6\nWe know:\n\\begin{align}\n\\text{EO Difference} = \\max(\\text{TPR Difference}, \\text{FPR Difference})\n\\end{align}\n\nLipschitz constants for true positive rates (TPR) and false positive rates (FPR) can be calculated in the same way as in Theorem 3.10. Therefore, Equalized Odds (EO) possesses a Lipschitz constant, indicating that it is a smooth function.\n### Q7\nWe covered this. (Response to W1)\n### Q8\nWe apologize for the confusion caused by the incorrect theorem number. There is no Theorem 4.6 in our paper; the correct reference should be to Theorem 3.8. This error has been corrected in the text.\n\nWe acknowledge your concern about the limitations of the Lipschitz continuity assumption, as many practical neural networks might appear to violate it. To address this, we have added a section discussing how our theoretical assumptions apply to real-world scenarios, highlighting how neural network design and training practices can satisfy or approximate these assumptions. Additionally, we have included an appendix section exploring the Lipschitz properties of various layers, demonstrating that many commonly used types are Lipschitz continuous or can be modified to be so.\n\nAlthough the Lipschitz continuity assumption may seem restrictive, our analysis shows that it is more applicable to practical neural networks than it initially appears. Many modern architectures, such as CNNs[1], GNNs[2], and GATs[3], either inherently satisfy this property or can be easily adjusted to do so.\n\n\n## Limitations\n### L1\nWe have added limitations and future works to global rebuttal and revised manuscript.\n#### References:\n1. Dongmian Zou, Radu Balan, and Maneesh Singh. On lipschitz bounds of general convolutional neural networks. IEEE Transactions on Information Theory, 66(3):1738–1759, 2019.\n2. Simona Ioana Juvina, Ana Antonia Neacs, u, Jérôme Rony, Jean-Christophe Pesquet, Corneliu Burileanu, and Ismail Ben Ayed. Training graph neural networks subject to a tight lipschitz constraint. Transactions on Machine Learning Research.\n3. Reference [2] in global rebuttal.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "h27CiyI996", "reviewer": "Reviewer_UWLB", "summary": "The paper proposes and justifies a bi-level optimization approach to optimizing empirical risk minimization objectives when additional fairness constraints need to be considered. Using assumptions of Lipschitz continuity and local convexity, they prove that a bi-criteria (accuracy + fairness) problem is equivalent to bi-level optimization. Key to their approach is to having separate parameters for fairness and accuracy. Practically, accuracy and fairness layers are made for neural networks. Extensive experiments are done.", "questions": "1. What is the compute time of approach? Does the bi-level optimization approach take longer? How do the number of parameters compare? Basically, I am wondering if due to the separate set of parameters needed in the bi-level optimization, the proposed approaches architecture has a larger number of parameters than the baselines. (and if so, would increasing parameter sizes of other approaches increase fairness)\n2. What does the overparameterization condition in Theorem 3.9 mean? I am a bit confused about why its needed in Line 186 / 187.", "limitations": "Adequate.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper provides a nice motivation for considering bi-level optimization over the typical fairness regularization approaches.\n- Theorem 3.9 is nice to show that the Largrangian approach (under certain conditions) is an upper bound of the bi-level approach.\n- Experiments seem promising and extensive.", "weaknesses": "- Missing experimental details. Compute time of baseline and approaches not presented. Neither is the number of parameters used in each approach. Additionally, some approaches / baselines in the appendix aren't well specified, eg, FairGAT and FairGCN.\n- The convexity assumption does restrict the theory.\n- Two separate sets of parameters are required. This limits Theorem 3.9's relevance to the typical fairness regularized cases.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful feedback. We have addressed your questions and concerns below. If you have any further questions, we would be glad to discuss them.\n\n## Weaknesses\n### W1\nWe acknowledge that some important information was missing from our initial presentation. To address these concerns, we have made the following additions and clarifications:\n\n1. Compute time and parameter comparison: We have now included a detailed comparison of compute time with the same number of parameters between our method and the Lagrangian approach in the global rebuttal. This comparison is particularly relevant as the Lagrangian method serves as our natural baseline, given the theoretical focus of our work.\n\n2. Focus on Lagrangian comparison: We want to emphasize that our primary goal was to demonstrate the theoretical results in practice. As such, we focused our direct comparisons on the Lagrangian method, which is most closely related to our approach in terms of problem formulation. We have now made this focus clearer in our experimental setup section.\n\n3. Other baselines: While we included results from other approaches for context, we acknowledge that a direct comparison of computational complexity or parameter count with these methods may not be fair or meaningful. These methods approach the fairness problem in fundamentally different ways, making such comparisons non-trivial. We have added a note in our experimental section to clarify this point.\n\n4. Specification of approaches in the appendix: We have added proper citations and brief descriptions of all methods mentioned in the appendix to provide better context. This includes FairGAT, FairGCN, and others.\n\nWe appreciate the reviewer’s feedback, which helped improve the transparency of our experimental setup and the validity of our comparisons, particularly concerning the Lagrangian method.\n\n### W2\nWe'd like to clarify that our theory doesn't assume convexity of the entire neural network optimization problem, but rather convexity near local optima. We have added a new section to the paper discussing how our assumptions translate to real-world scenarios, which addresses this point in detail.\nIn this new section, we explain that while neural network loss landscapes are generally non-convex, recent research [1] suggests they often exhibit locally convex regions around minima, especially in overparameterized networks. Our assumption of convexity near local optima aligns with this understanding.\nIn practice, this assumption translates to the behavior of neural networks as they converge during training. As long as the network converges, it will likely encounter these locally convex regions along its optimization path. Our theory applies particularly well in these parts of the optimization process.\nAs networks approach convergence, our assumption of local convexity becomes increasingly valid, making it applicable to well-designed models trained on suitable datasets. Practitioners can rely on this aspect of our theory if their models show convergence. The clarification and new section in our paper highlight how our theoretical framework aligns with practical neural network optimization. By focusing on local rather than global convexity, we significantly broaden the applicability of our theory to real-world scenarios.\n\n### W3\nWe appreciate the reviewer's insightful observation regarding the separate sets of parameters and their impact on Theorem 3.9's relevance to typical fairness regularized cases. We'd like to clarify that the number of parameters in both our bilevel approach and the regularization method are the same, and both methods optimize parameters within the network (please see Fig. 3 in attached PDF). This allows for a meaningful theoretical comparison between the approaches.\nIn practice, the separation of parameters into accuracy ($θ_a$) and fairness ($θ_f$) sets in our approach allows for more fine-grained control over the optimization process. This can potentially lead to finding better solutions than single-objective regularization methods, as demonstrated in our experimental results.\n\n## Questions\n### Q1\nTo address concerns about experimental details, we have enhanced our paper in two key ways. First, we added a new section that directly compares our method to the Lagrangian approach, demonstrating that our method outperforms it with the same number of parameters. Second, we included a comprehensive analysis that examines the computational complexity of both bilevel and regularization training processes (summary in global rebuttal), along with empirical timing results. These additions provide a clear picture of our method’s efficiency and performance relative to traditional regularization techniques.\n\n### Q2\nThe overparameterization condition refers to a scenario where the model has more parameters than necessary to fit the training data perfectly. In the context of our theorem, specifically in lines 186-187, this condition is needed for the following reasons:\n\n1. Flexibility: Overparameterization provides the model with additional flexibility to find solutions that satisfy both accuracy and fairness objectives simultaneously.\n\n2. Existence of Multiple Optima: It ensures the existence of multiple parameter configurations that can achieve optimal performance on the primary (accuracy) objective. This is crucial for our bilevel optimization approach, as it allows the model to choose among these configurations to optimize the secondary (fairness) objective.\n\n3. Theoretical Guarantees: The condition allows us to establish theoretical guarantees about the performance of our bilevel approach compared to the Lagrangian method.\n\n#### References:\n1. Zeyuan Allen-Zhu, Yuanzhi Li, and Yingyu Liang. Learning and generalization in overparameterized neural networks, going beyond two layers. Advances in neural information processing systems, 32, 2019.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "339vkpSSmM", "reviewer": "Reviewer_dbnD", "summary": "The authors propose a bi-level optimization approach to simultaneously optimize a performance and a fairness loss. They show that this approach is, in theory, superior to a typical approach that regularizes the performance objective with a fairness objective. Using tabular, graph and vision datasets, they demonstrate the Pareto frontiers obtained by their method, compared to multiple (mostly regularization) baselines.", "questions": "- Line 550: how can ensure that $\\hat{\\theta}_p$ is closer to the optimum than $\\theta_p$? Strict improvement at each step seems like a strong assumption to me. Shouldn’t this be spelled out as an extra assumption?\n- How does the assumption of “sufficiently small” $\\eta$ maps to the practical implementation, as we do not know the Lipschitz constant?\n- Similarly, how about the assumption in Theorem 9 of an overparameterization?\n- Line 212, DP: why use $\\theta_1$ and $\\theta_2$? From Assumption 3.10, it suggests that these represent the parameters for $x_1$ and $x_2$. However, the parameters $\\theta$ when computing DP are the same for $a=0$ and $a=1$. I don’t see how we can guarantee that the network’s activations are Lipschitz continuous unless we also make assumptions of the distance between input examples (which can be difficult to estimate). The proof also refers to 2 functions $f_1$ and $f_2$, but we have the same function, only different subsets of inputs $x_{a=0}$ and $x_{a=1}$. Can you clarify please? It is not clear to me how the proof shows that DP is Lipschitz continuous.\n- The discussion seems more like a brief summary with a rebuttal paragraph, rather than a proper discussion. The work is not put in perspective with the literature.\n- Figures 4 and 5 in Appendix: the proposed method is not strictly better compared to the baselines in all cases. Is there a pattern of failure cases? \n\n**Minor**:\n\n- Line 64: is the comment pointing towards distribution shifts in the test data? It is unclear what “depends on the data” means.\n- “Paractical” line 198\n- $\\hat{\\theta}$ is not defined in Assumption 3.3.\n- Equation (1): the description uses the ‘p’ and ‘s’ subscripts while the equation uses ‘a’ and ‘f’. Please correct.\n- Is Assumption 3.10 a repeat of definition 3.1?\n- Line 276: end with “shows”, I am assuming something is missing.\n- Line 545, equation 38: shouldn’t it be $\\phi$ instead of f?\n- Table 4 (Appendix): it looks like FairGAT has smaller delta EO and should be highlighted instead. Overall, I’d suggest highlighting all results within the standard deviation of each other, as it is a bit misleading otherwise.", "limitations": "The authors describe as “NA” the limitations of their approach. I strongly believe that all methods have limitations and these should be discussed. \n- For instance, multiple assumptions are made, but not discussed. \n- It is also unclear how the practical implementation reflects the theoretical results, as it would be reasonable to expect effects of finite sample size, the hyper-parameters selected for each of the fairness / performance layers, …\n- Are any common activation functions not Lipschitz continuous? What about ELU, or other fairness criteria? What would that mean for the derived results?", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "Regularization and adversarial training can be difficult to perform in practice, as the choice of hyper-parameters needs to be performed with care and convergence might not be guaranteed. By leveraging bi-level optimization, the authors provide a novel perspective on the problem (to the best of my knowledge).", "weaknesses": "While I found the idea interesting, I found that the paper lacked clarity, and have a few major comments / concerns at this stage. \n\n**Major concerns**\n\n- I found that there were many assumptions made, basically requiring that all parameters are smooth, change at a limited rate and improve compared to prior steps. While some of these might be reasonable, the paper lacked a discussion of what these assumptions mean, and in which conditions they might not be respected.\n- I am not convinced by Theorem 9 and its proof, as it relies on two functions but DP is computed on a single function (see below).\n- I found that the connection between the theory and its implementation is not well detailed and explored.\n- To me, discussing works that perform constrained or multi-objective optimization of hyper-parameters, or refer to meta-learning for fairness would be relevant.\n- Practically, I see multiple obstacles to the adoption of the proposed method: more hyper-parameters to select, published models cannot be re-used, only one fairness constraint is considered, ... These are not discussed or acknowledged.\n\n**Detailed comments**:\n- Related works: I believe the works who perform the multi-objective or constrained optimization of model’s hyperparameters (e.g. Perrone et al., 2020, https://arxiv.org/pdf/2006.05109 and many others) should be discussed, and potentially considered as baselines. While I understand they do not provide the same guarantees, they are easier to implement.\n- Can the authors discuss their approach with e.g. Slack et al., 2019 (https://arxiv.org/abs/1911.04336), in which the 2 objectives are separated into learners and a meta-learner instead?\n- The related works section describes some related works with one sentence, but does not “group” them into the different avenues they represent, or how they differ from the proposed approach. I would suggest trying to “analyze” the field at a higher level and show what the proposed approach can bring.\n- Overall I found the methodology section dense and not easy to follow. The assumptions are also not discussed, simply enumerated and it is unclear how realistic they are.\n- The experimental baselines seem to encompass multiple techniques, but are not well described. It is unclear how each of these compares to the proposed approach.\n- For the Health dataset, as well as for other datasets (in Appendix), the theoretical results do not seem to hold. This is briefly discussed at the end of the results section, but is a clear limitation of the work. There is also little description of whether the selection of hyper-parameters is more complex for the bi-level case.\n- Relatedly, this is presented as an advantage but to me is a weakness of the approach that the architecture needs to be completely revised to include fairness layers. This means that published models cannot easily be reused, and that the whole process needs to be re-trained for a different fairness constraint.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful feedback. We have addressed your questions and concerns below. If you have any further questions, we would be glad to discuss them.\n## Weaknesses\n### W1\nWe appreciate the reviewer's feedback on the assumptions in our paper. In response, we have added a dedicated discussion section in global rebuttal.\n\n### W2\nThank you for your comment on Theorem 3.9. We believe there may have been a lack of clarity from our part about its purpose and formulation. The theorem compares the performance of the primary objective between our bilevel optimization approach and the Lagrangian approach, not directly addressing the computation of Demographic Parity (DP). It involves two loss functions (See Fig. 3 in attached PDF): one for the main objective, such as accuracy, and another for the fairness constraint, which may be DP but is not limited to it. The theorem concludes that our bilevel approach can achieve better performance on the primary task under certain conditions compared to the Lagrangian method, while still providing Pareto solutions near local minima.\n\n### W3\nWe appreciate your observation about the connection between our theoretical framework and practical implementation. In response, we have added a section comparing bilevel optimization and Lagrangian regularization, presenting empirical results that validate our theoretical claims and show where our approach excels in balancing accuracy and fairness. We’ve also included a section on how our theoretical assumptions apply in real-world scenarios. While our method yields Pareto optimal solutions under certain conditions, we acknowledge that other methods can also produce different Pareto optimal solutions, and we have discussed the scope of our theoretical guarantees in relation to the broader field of fairness-aware machine learning.\n\n### W4\nWe modified the related work section to position our work better in the literature.\n\n### W5\nWe have addressed several key points in our paper. First, we added a section on practical implementation, discussing hyperparameter selection and its role in providing finer control over balancing accuracy and fairness, along with guidelines for choosing these parameters (See a glimpse of it in the attached PDF, Fig. 1 and 2). Second, we clarified that our method can be applied as a downstream task on pre-trained models, allowing for fairness enhancements without full retraining, as demonstrated with ResNet on CelebA. Third, we discussed extending our method to handle other fairness constraints, with theoretical guidance provided in the appendix. Additionally, we included limitations of our work in global rebuttal.\n\n## Comments\nThe Related Work section was thoroughly revised, categorizing classic multi-objective optimization methods into two groups: common regularization-based methods, including adversarial debiasing, fair representation learning, and Lagrangian optimization, and less common approaches, such as gradient-based methods and transfer and meta-learning approaches. Two mentioned papers were added to the latter group and discussed. The advantages of bilevel optimization were summarized at the end of the section.\n\n## Questions\n### Q1\nWe provide a theoretical analysis of the neighborhood around local minima in the loss landscape. Due to the convexity of the loss in this area, a sufficiently small step size can lead to strict improvements in the loss. For more details, please refer to item 1 in the assumption discussion in the global rebuttal.\n\n### Q2\nIn practical scenarios, the learning rate is a tunable hyperparameter specific to the task, and the Lipschitz constant is not required for implementation.\n\n### Q3\nMost modern deep learning architectures satisfy this assumption, as it is met when the network has the capability and parameters to fit all the data. For more details, please refer to item 3 in the assumption discussion in the global rebuttal.\n\n### Q4\nWe apologize for the notation error in the DP definition on Line 212, where  $\\theta_1$  and  $\\theta_2$  should represent the same set of parameters for both  $a=0$  and  $a=1$ . This will be corrected in the revised paper. Regarding the proof of Lipschitz continuity for DP, the proof is correctly formulated, focusing on the difference between DP loss calculated with two parameter sets. Here,  $f_1$  and  $f_2$  refer to the same function evaluated with different parameters, consistent with DP computation practices \\[20,on manuscript\\]. The Lipschitz continuity being proven relates to parameter changes, showing that small parameter adjustments lead to bounded changes in the DP metric. The proof establishes that if the underlying function is Lipschitz continuous with respect to its parameters, then the DP metric is also Lipschitz continuous concerning those parameters.\n\n### Q5\nPlease refer to the global rebuttal. We will include a comprehensive discussion in the revised paper.\n\n### Q6\nOur method’s performance varies due to differences in data handling and augmentation techniques used by baselines, which can affect the loss landscape. While it is not universally superior, our approach consistently demonstrates competitive performance across diverse scenarios, underscoring the complex nature of fairness-accuracy trade-offs in machine learning.\n\n## Minors\nThank you for your meticulous attention to detail. All minor issues have been corrected.\n\n## Limitations\nWe have added Limitations and future works to the global rebuttal and manuscript.\n### L1\nPlease refer to \"Assumption Discussion\" in global rebuttal.\n### L2\nThank you for your comment! We have added several ablation studies to the paper. Figures 1 and 2 in attached PDF provide a glimpse of two of these studies.\n### L3\nYes, Softmax is a key activation function that is not Lipschitz continuous, while ELU and ReLU are. We have also included a theoretical proof of Lipschitz continuity for another common fairness criterion, equalized odds.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
# Global Rebuttal

We sincerely thank the reviewers for their constructive comments, which have significantly contributed to the improvement of our work. We have addressed the reviewers’ concerns regarding our method’s assumptions, limitations, and ablation studies.

## Assumption Discussion
We appreciate the reviewer’s insightful comments on the assumptions behind our theoretical guarantees. We have added a detailed discussion in the paper to address these concerns and relate these assumptions to practical scenarios. Here is the summary of it:

1. Convexity Near Local Optima: While neural networks are generally non-convex, research [1] suggests they exhibit locally convex regions around minima, especially in overparameterized networks. Our theory is particularly applicable as the network approaches convergence, which is common in well-designed models on suitable datasets. (Assumption 3.2)

2. Small Steps for Secondary Parameters: The assumption $|θ_s − \hat{θ}_s| ≤ ε$, where ε is sufficiently small, can be met by choosing an appropriate learning rate. This can be achieved through hyperparameter tuning and grid search, which are standard practices in machine learning.(Assumption 3.3)

2. Overparameterization: This assumption aligns well with modern deep learning trends, where models often have more parameters than training samples. As long as the model can overfit the training data, this condition is met. However, we acknowledge that in resource-constrained environments or with extremely large datasets, this might not always be feasible.

3. Lipschitz Continuity: We have added a rigorous analysis of various layers and activation functions in terms of their Lipschitz properties, serving as a guide for practitioners. Common choices like ReLU activations and standard loss functions (e.g., cross-entropy) satisfy this assumption, making it broadly applicable. (Assumption A.7, A.8)

4. Bounded Output: The assumption of bounded output for each layer can be achieved in practice through the use of bounded activation functions and weight regularization methods, which are common in neural network optimization. (Assumption 3.4)

While these assumptions may not apply universally, they are common in many real-world deep learning applications. Our framework provides valuable insights even when assumptions are only approximately met. Strong empirical results across various datasets support our approach’s practical utility.

We hope this discussion clarifies the applicability of our theoretical guarantees and addresses the reviewer's concerns.

## Limitations and Future Work
While our results are promising, our approach has several limitations. The widely used softmax activation function is not Lipschitz continuous, limiting our method’s application to multiclass classification. Future work could explore alternative activation functions that maintain Lipschitz continuity for multiclass problems.

Additionally, attention mechanisms in modern models are not Lipschitz continuous, posing challenges for extending our method to architectures that rely on attention. Research into enforcing Lipschitz continuity in attention layers, such as using LipschitzNorm [2], a simple and parameter-free normalization technique applied to attention scores, shows potential for enhancing performance in deep attention models, and integrating this with our framework.

Our theoretical analysis mainly provides guarantees in comparison to regularization methods, offering improvements in fairness but not absolute fairness guarantees. Expanding the framework to include direct fairness guarantees would increase its applicability.

We did not validate the method with dataset augmentation, a common practice to improve generalization. Future work should assess how our method interacts with data augmentation and its impact on fairness properties.

Currently, our implementation focuses on demographic parity, but real-world applications often require multiple fairness metrics. Extending our method to address multiple fairness constraints would make it more versatile.

Addressing these limitations presents opportunities for future research to enhance the applicability and effectiveness of fair machine learning methods in diverse scenarios and architectures.

## Computational Complexity Analysis
Let's define the following variables:

- $n$: number of parameters in $\theta_p$
- $m$: number of parameters in $\theta_s$
- $C_f$: cost of computing $f$ and its gradients
- $C_\phi$: cost of computing $\phi$ and its gradients

;where $\theta_p$ and $f$ are related to primary objective and $\theta_s$ and $\phi$ are related to secondary objective.

Based on the Lagrangian and Bilevel update rules, computational complexity per iteration for both approaches: $O(C_f + C_\phi + n + m)$

### Empirical Comparison:

While theoretical complexity analysis indicates similar costs for both methods, we conducted empirical tests to compare their runtime performance on the Adult and Health datasets using FairBiNN and Lagrangian methods after 10 epochs of warmup. These experiments, conducted on an M1 Pro CPU, showed no significant difference in average epoch time between the two methods, supporting our theoretical analysis.

#### Table 1: Average epoch time (in seconds) for FairBiNN and Lagrangian methods

| Method     | Adult Dataset (s) | Health Dataset (s) |
|------------|-------------------|--------------------|
| FairBiNN   | 0.62              | 1.03               |
| Lagrangian | 0.60              | 1.05               |



#### References:
1. Zeyuan Allen-Zhu, Yuanzhi Li, and Yingyu Liang. Learning and generalization in overparameterized neural networks, going beyond two layers. Advances in neural information processing systems, 32, 2019.
2. George Dasoulas, Kevin Scaman, and Aladin Virmaux. Lipschitz normalization for self-attention layers with application to graph neural networks. In International Conference on Machine Learning, pages 2456–2466. PMLR, 2021.

Meta-review: The current paper has its strengths in its theoretical contributions towards introducing a novel bilevel optimization framework to address the fundamental problem of finding solutions on the Pareto front in ML fairness. The theoretical rigor, including guarantees and proofs, are significant contributions and seems to be unanimously agreed upon by the reviewers. The ablation studies presented in the paper, seems to highlight the practical potential of the proposed approach. The broad array of experiments demonstrates the method's efficacy across various domains as well as stability of their approach.

There were concerns raised by the reviewers on the practicality of theoretical assumptions with respect to real-world scenarios as well as lack of discussions towards limitations of the proposed approach. Going over the rebuttal as well as the discussions between the authors and reviewers, I see that the authors seem to have addressed this in the context of current prevalent practices in deep learning implementations. Including them in the revision should bring more clarity over practical utility of the paper. Authors seem to have also provided additional discussion on choice of baselines and comparison with Lagrangian regularization method which seems fair considering their theoretical results. The reviews and rebuttal also highlights that a discussion over limitations of the proposed approach needs to be included in the final version. 

Considering the paper's contributions novel and significant, I **recommend acceptance** with revisions addressing the identified concerns.

Following are the specific items that the authors should address -

1. **Limitations**: Include a brief discussion in the main text and a detailed appendix section outlining the limitations of the current approach.
2. **Ethics**: Add an ethics section to discuss potential ethical implications and considerations related to the proposed method.
3. **Parameter Selection**: Include a figure in main text clarifying parameter selection in neural networks. Explicitly stating correspondence of parameters to notations in theory.
4. **Assumptions**: Add a detailed section in Appendix which clarifies how different components in deep networks fit or do not fit under the proposed assumptions.
5. Computational Complexity Analysis to be added somewhere. 
6. Update the discussion in the experiments section explaining the choices of baselines for comparison with the Lagrangian approach.
7. **Practical Implementation**: Address practical implementation concerns like hyperparameter selection, model reusability, and extension to multiple fairness constraints.

Addressing these concerns and incorporating the suggested revisions will significantly strengthen the paper and enhance its impact.
Decision: Accept (poster)