Fair Bilevel Neural Network (FairBiNN): On Balancing fairness and accuracy via Stackelberg Equilibrium

Authors: mehdi  yazdani-jahromi
ali khodabandeh yalabadi
amirarsalan  rajabi
aida  tayebi
ivan  garibay
ozlem ozmen garibay
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 20:19:04

Abstract: The persistent challenge of bias in machine learning models necessitates robust solutions to ensure parity and equal treatment across diverse groups, particularly in classification tasks. Current methods for mitigating bias often result in information loss and an inadequate balance between accuracy and fairness. To address this, we propose a novel methodology grounded in bilevel optimization principles. Our deep learning-based approach concurrently optimizes for both accuracy and fairness objectives, and under certain assumptions, achieving proven Pareto optimal solutions while mitigating bias in the trained model. Theoretical analysis indicates that the upper bound on the loss incurred by this method is less than or equal to the loss of the Lagrangian approach, which involves adding a regularization term to the loss function. We demonstrate the efficacy of our model primarily on tabular datasets such as UCI Adult and Heritage Health. When benchmarked against state-of-the-art fairness methods, our model exhibits superior performance, advancing fairness-aware machine learning solutions and bridging the accuracy-fairness gap. The implementation of FairBiNN is available on https://github.com/yazdanimehdi/FairBiNN.

Main paper: 
 introduction: 
 Artificial intelligence and machine learning models have seen significant growth over the past decades, leading to their integration into various domains such as hiring pipelines, face recognition, financial services, healthcare, and criminal justice. This widespread adoption of algorithmic decision-making has raised concerns about algorithmic bias, which can result in discrimination and unfairness towards minority groups. Recently, the issue of fairness in artificial intelligence has garnered considerable attention from interdisciplinary research communities, addressing these ethical concerns [50] . Several definitions of fairness have been proposed to tackle unwanted bias in machine learning techniques. These definitions generally fall into two categories: individual fairness and group fairness. Individual fairness ensures that similar individuals are treated similarly, with similarities determined by past information [20, 73] . Group fairness, on the other hand, measures statistical equality between different subgroups defined by sensitive characteristics such as race or gender [76, 41, 26] . In this paper, we focus on group fairness, which we will refer to simply as fairness from this point onward.Fairness approaches in machine learning are commonly categorized into three groups: (1) Pre-process approaches: These methods involve changing the data before training to improve fairness, such as reweighing labels or adjusting features to reduce distribution differences between privileged and unprivileged groups, making it harder for classifiers to differentiate them [34, 42, 22, 63] . Generative adversarial networks were also utilized to produce unbiased datasets by altering the generator network's value function to balance accuracy and fairness [53] . (2) In-process approaches: These methods modify the algorithm during training, for instance by adding regularization terms to the objective function to ensure fairness. Examples include penalizing the mutual information between protected attributes and classifier predictions to allow a trade-off between fairness and accuracy [35] , and adding constraints to satisfy a proxy for equalized odds [74, 75] . (3) Post-process approaches: These techniques adjust the outcomes after training, such as flipping some outcomes to improve fairness [26] , or using different thresholds for privileged and unprivileged groups to optimize the trade-off between accuracy and fairness [45, 11] .In this work we targeted the in-process bias mitigation category. Traditionally, the fairness multicriteria problem has been addressed using Lagrangian optimization, wherein the objective function is a weighted sum of the primary and secondary loss functions. While this approach allows for the explicit incorporation of fairness constraints through Lagrange multipliers, it may overlook the complex interdependencies between the primary and secondary objectives. A promising alternative to the Lagrangian is the bilevel optimization approach which offers several advantages. By formulating the problem as a hierarchical optimization task, we can explicitly model the interactions between the primary and secondary objectives. This allows us to capture the nuanced dynamics of fairness optimization and ensure that improvements in one objective do not come at the expense of the other.In summary, we introduce a novel method that can be trained on existing datasets without requiring any alterations to the data itself (data augmentation, perturbation, etc). Our methodology provides a principled approach to addressing the multi-criteria fairness problem in neural networks. Through rigorous theoretical analysis, we formulated the problem as a bilevel optimization task, proving that it yields Pareto-optimal solutions. We derived an effective optimization strategy that is at least as effective as the Lagrangian approach. Empirical evaluations on tabular datasets demonstrate the efficacy of our method, achieving superior results compared to traditional approaches.In practice, this assumption is often satisfied by using bounded activation functions:• Sigmoid function: bounded between 0 and 1 • Hyperbolic tangent (tanh): bounded between -1 and 1 • ReLU6: a variant of ReLU that is capped at 6Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? Answer: [Yes] Justification: The abstract and introduction section carefully convey the paper's contributions and scope. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
related works: 
 Multi-objective optimization in neural networks involves optimizing two or more conflicting objectives simultaneously. Fairness problems are inherently multi-objective in nature, as improvements in one objective (e.g., enhancing fairness) often come at the expense of another objective (e.g., improving accuracy). Several optimization techniques in neural networks have been employed to balance accuracy and fairness. Classic Methods transform these objectives into a single objective by combining them, typically using a weighted sum where each objective is multiplied by a weight that reflects its importance. Adding Regularization and penalty terms are the most common methods that incorporate fairness constraints (e.g., demographic parity, equal opportunity) directly into the loss function, penalizing disparities in prediction errors across demographic groups or any other unfair behavior. To reduce variation across different groups, Zafar et al. [74] proposes "disparate mistreatment", a new notation for fairness, and standardized the decision bounds of a convex margin-based classifier. Adversarial debiasing and Fair Representation Learning are two examples of these techniques, which encourage the model to generate fair outcomes by introducing a penalty term based on an adversarial network or a representation learning framework that is invariant to protected attributes, respectively. Zhang et al. [77] addressed bias by limiting an adversary's ability to infer sensitive characteristics from predictions. Avoiding the complexity of adversarial training, Moyer et al. [47] used mutual information to achieve invariant data representations concerning specific factors. Song et al. [61] proposed an information-theoretic method that leverages both information-theoretic and adversarial approaches to achieve controllable fair data representations, adhering to demographic parity. By incorporating a forget-gate similar to those in LSTMs, Jaiswal et al. [30] introduced adversarial forgetting to enhance fairness. Gupta et al. [25] utilized certain estimates for contrasting information to optimize theoretical objectives, facilitating suitable trade-offs between demographic parity and accuracy in the statistical population. Lagrangian optimization techniques are a subset of these techniques that use Lagrange multipliers or other similar techniques to incorporate constraints directly into the objective function, turning constrained optimization problems into unconstrained ones. Agarwal et al. [2] proposes an approach for fair classification by framing the constrained optimization problem as a two-player game where one player optimizes the model parameters, and the other imposes the constraints, and Lagrangian multipliers are used to solve this problem. Cotter et al. [12] expanded this work in a more general inequality-constrained setting, by simultaneously training each player on two distinct datasets to enhance generalizability. They enforce independence by regularizing the covariance between predictions and sensitive variables, which reduces the variation in the relationship between the two. Despite analytic solutions and theoretical assurances, scaling game-theoretic techniques for more complex models remains challenging [9] . In addition, these constraints-based optimizations are data-dependent, meaning the model may exhibit different behavior during evaluation even if constraints are met during training. Less common approaches including Pareto-based genetic algorithm, Reinforcement Learning, Gradient-Based Methods, and Transfer and Meta-Learning Approaches have been also utilized in this domain. Mehrabi et al. [43] demonstrated how proxy attributes lead to indirect unfairness using an attention-based approach and employed a post-processing method to reduce the weight of attributes responsible for unfairness. Perrone et al. [49] introduces a general constrained Bayesian optimization (BO) framework to fine-tune the model's performance while enforcing one or multiple fairness constraints. A probabilistic model is used to describe the objective function, and estimates are made for the posterior variances and means for each hyperparameter configuration. By adding a fairness regularization term to a meta-learning framework, Slack et al. [60] suggests an adaptation of the Model-Agnostic Meta-Learning (MAML) [23] algorithm. The primary objective and fairness regularization terms are included in the loss function used to update the model parameters for each task during the inner loop (Learner). The model parameters are updated in the outer loop (Meta-learner) to maximize performance and fairness across all tasks. Although these techniques have achieved a good balance between fairness and accuracy, they might not capture all of the complex interdependencies between these two objectives. In this paper we propose a bilevel optimization approach as an alternative to the Lagrangian approaches. Bilevel optimization is a hierarchical structure in which the context or constraints for the "follower" (lower-level) problem are set by the "leader" (upper-level) problem [17] . The leader makes decisions first, and the follower optimizes their decisions based on the leader's choices. This approach can handle more complex and nuanced multi-objective optimization problems in neural networks and is suitable for scenarios where one objective directly influences another and there are complex interactions between the two objectives. In this paper we demonstrate that the bilevel optimization often can achieve better balance and performance compared to classic regularization-based optimization approaches [17, 59, 10] . Bilevel optimization offers several advantages; by explicitly modeling a two-level decision-making process, his approach represents the problems in a more natural way where one objective inherently depends on the outcome of another. It provides more flexibility and control over the optimization process by enabling separate optimization of constraints at each level. The upper-level optimization can dynamically adjust the lower-level objective based on the current solution, potentially leading to more adaptive and context-sensitive optimization outcomes. Fairness and accuracy objectives can be directly integrated into the optimization framework without the need for additional strategies such as meta-learning.Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: We mentioned all limitations and future directions in section 5.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
methodology: 
 In this section we are introducing a novel bi-level optimization framework for training neural networks to obtain Pareto optimal solutions when optimizing two potentially competing objectives. Our approach leverages a leader-follower structure, where the leader problem aims to minimize one objective function (e.g. a primary loss), while the follower problem optimizes a secondary objective. We provide theoretical guarantees that our bi-level approach produces Pareto optimal solutions and performs at least as well as, and often strictly better than, the common practice of combining multiple objectives via a weighted regularization term in a single loss function. The full statements of these theorems and their proofs are provided in the Theoretical Analysis subsection below. Our bi-level approach offers several benefits over regularization-based methods. First, it allows for easy customization of the architecture and training algorithm used for each objective. The leader and follower problems can utilize different network architectures, regularizers, optimizers, etc. as best suited for each task. Second, the leader problem remains a pure minimization of the primary loss, without any regularization terms that may slow or hinder its optimization. Separating out secondary objectives ensures the primary task is learned most effectively. Finally, bi-level training exposes a clear interface for controlling the trade-off between objectives. By constraining the follower problem more or less strictly, we can encourage stronger or weaker adherence to the secondary goal relative to the primary one. To realize these benefits, we employ an iterative gradient-based algorithm to solve the bi-level problem, alternating between updating the leader and follower parameters. We unroll the follower optimization for a fixed number of steps, and backpropagate through this unrolled process to update the leader weights.The fairness multi-criteria problem in neural networks can be formulated as a bi-criteria optimization problem. Let f (θ p , θ s ) denote the primary objective loss function and φ(θ p , θ s ) denote the secondary objective loss function. Here, θ p ∈ Θ p represents the parameters responsible for optimizing the primary objective, and θ s ∈ Θ s represents the parameters for the secondary objective. The problem is formally stated as:min θp∈Θp,θ f ∈Θ f {f (θ p , θ f ), φ(θ p , θ f )} (1)We leverage the theoretical results of [46] , which investigates the relationship between Stackelberg equilibria and Pareto optimality in game theory. The paper addresses fundamental questions regarding the conditions under which a Stackelberg equilibrium coincides with a Pareto optimal outcome. By proving that our bi-level optimization problem satisfies the assumptions outlined in the paper, we establish a strong theoretical foundation for our approach. Specifically, we demonstrate that under certain conditions, the Stackelberg equilibrium of our bi-level optimization problem is equivalent to a Pareto optimal solution for the bi-criteria problem of balancing accuracy and fairness objectives. By rigorously verifying these assumptions in the context of our neural network optimization problem, we establish that the Stackelberg equilibrium reached by our bilevel approach indeed corresponds to a Pareto optimal solution. This theoretical grounding provides confidence that our methodology effectively balances the competing objectives of accuracy and fairness, yielding a principled and well-justified solution to the problem at hand.We leverage several key theoretical results to formulate our approach. First, Lemma 3.5 establishes the Lipschitz continuity of the neural network function with respect to a subset of parameters. This lemma provides the foundation for analyzing the behavior of the objective functions under parameter variations.EQUATIONThe smallest L for which the previous inequality is true is called the Lipschitz constant of f and will be denoted L(f ).Assume that the following assumptions are satisfied:Assumption 3.2. The primary loss function f (θ p , θ s )(x) is strictly convex in a neighborhood of its local optimum. That is, for any θ p , θ ′ p ∈ Θ p and fixed θ s ∈ Θ s , if θ p ̸ = θ ′ p and θ p , θ ′ p are sufficiently close to the local optimum θ * p , thenEQUATIONfor any λ ∈ (0, 1).Assumption 3.3. |θ s -θs |≤ ϵ, where ϵ is sufficiently small, i.e., the steps of the secondary parameters are sufficiently small. θ s and θs represent the parameters for the secondary objective and their updated values, respectively. Assumption 3.4. Let f l (.) denote the output function of the l-th layer in a neural network with L layers. For each layer l ∈ 1, . . . , L, there exists a constant c l > 0 such that for any input x l to the l-th layer:EQUATIONwhere |.| denotes a suitable norm (e.g., Euclidean norm for vectors, spectral norm for matrices). Refer to Section A.4 for common practices in implementing the bounded output assumption.We recognized the importance of examining how our theory's underlying assumptions apply to real-world applications. For a detailed discussion, refer to Section A.3. Lemma 3.5. Let f (x; θ) be a neural network with L layers, where each layer is a linear transformation followed by a Lipschitz continuous activation function.Let θ be the set of all parameters of the neural network, and θ s ⊆ θ be any subset of parameters. Then, f (x; θ) is Lipschitz continuous with respect to θ s . [See proof A.4] We discussed the Lipschitz continuity of common activation functions and popular neural networks, such as CNNs and GNNs, in Sections A.5 and A.6, respectively.Theorems 3.6 and 3.7 further inform our approach. The former establishes conditions under which improvements in the secondary objective lead to improvements in the primary objective, while the latter guarantees the existence of unique minimum solutions for the secondary loss function under certain optimization conditions. Theorem 3.6. Let f (θ p , θ s ) for constant θ s be the primary objective loss function and φ(θ p , θ s ) for constant θ p be the secondary objective loss function, where θ p ∈ Θ p and θ s ∈ Θ s are the primary task and secondary task parameters, respectively.Consider two sets of parameters (θ p , θ s ) and ( θp , θs ) such that φ( θp , θs ) ≤ φ(θ p , θ s ). Then f ( θp , θs ) ≤ f (θ p , θ s ) holds based on Lemma 3.5. [See proof A.5] Theorem 3.7. Let φ(θ p , θ s ) be the secondary loss function, where θ p ∈ Θ p and θ s ∈ Θ s are the primary and secondary task parameters, respectively. Let (θEQUATIONs ) denote the parameters at optimization step t, and let (θ(t+1) p , θ (t+1) s) be the updated parameters obtained by minimizing φ(θ (t) p , θ s ) with respect to θ s using a sufficiently small step size η > 0, i.e.:EQUATIONThen, for a sufficiently small step size η, the updated secondary parameters θ (t+1) s are the unique minimum solution for the secondary loss function φ(θ(t) p , θ s ). [See proof A.6]Based on these theoretical insights, we derive our bilevel optimization formulation, as described in Theorem 3.8. This theorem establishes the equivalence between the bi-criteria problem and a bilevel optimization problem, allowing us to apply existing theoretical results on Stackelberg equilibrium to the optimization of neural networks. Theorem 3.8. Under the assumptions stated in Theorems 3.6 and 3.7, the bi-criteria problem (Eq. (1)) is equivalent to the bilevel optimization problem:EQUATIONEQUATIONwhere θ * s (θ p ) denotes the optimal secondary parameters for a given θ p .Proof. The proof follows from Theorems 3.6 and 3.7 [46] .By Theorem 3.6, under the assumptions of strict convexity, Lipschitz continuity, and sufficiently small steps of the secondary parameters, if φ( θp , θs ) ≤ φ(θ p , θ s ), then f ( θp , θs ) ≤ f (θ p , θ s ).By Theorem 3.7, under the same assumptions, for each optimization step of the secondary loss function with sufficiently small steps, the updated parameters are the unique minimum solution for the secondary loss function, then the bi-criteria problem (1) is equivalent to the bilevel optimization problem.Therefore, the conclusions drawn in the paper [46] can be directly applied to the multi-objective optimization problem in neural networks, as the problem is equivalent to the bilevel optimization problem under the stated assumptions.Theorem 3.9. Assume that the step size in the Lagrangian approach α L is equal to the step size for the outer optimization problem in the bilevel optimization approach α f , the scale of the two loss functions should be comparable, the Lagrangian multiplier λ is equal to the step size for the inner optimization problem in the bilevel optimization approach α s , and θ p is overparameterized for the given problem. Then, under certain conditions, the overall performance of the primary loss function in the bilevel optimization approach may be better than the Lagrangian approach.Proof. Let f (θ p , θ s ) denote the primary loss and φ(θ p , θ s ) denote the secondary loss. Assume that both f and φ are differentiable with respect to θ p and θ s . Define the Lagrangian function as:EQUATIONThe update rules for θ p and θ s in the Lagrangian approach are:EQUATIONEQUATIONThe update rules for θ p and θ s in the bilevel optimization approach are:EQUATIONEQUATIONDue to the overparameterization of θ p , there exists a set Θ p such that for any θ p ∈ Θ p [3] , f (θ p , θ s ) = f (θ * p , θ s ), where θ * p is an optimal solution for the primary loss when θ s is fixed. Suppose that the bilevel optimization approach converges to a solution (θ B p , θ B s ) and the Lagrangian approach converges to a solution (θ L p , θ L s ). Consider the following inequality:EQUATIONEQUATIONEQUATIONThe first inequality holds because (θ L p , θ L s ) is the minimizer of φ(θ p , θ s ) in the Lagrangian approach. The second inequality holds because (θ L p , θ L s ) is the minimizer of L(θ p , θ s , λ) in the Lagrangian approach. Since θ B p ∈ Θ p and θ L p / ∈ Θ p , we have:EQUATIONTherefore, under the assumptions that α L = α f , The sizes of the two loss functions f (θ p , θ s ) and φ(θ p , θ s ) should not differ significantly in terms of their order of magnitude, λ = α s , and θ p is overparameterized for the given problem, the bilevel optimization approach may converge to a solution that achieves better performance for the primary loss compared to the Lagrangian approach.To connect the Stackelberg game analysis with a practical implementation for datasets, we can formulate a bilevel optimization problem. The upper-level problem corresponds to the accuracy player (leader), while the lower-level problem corresponds to the fairness player (follower). We'll use gradient-based optimization techniques to solve the problem.Let's consider a dataset D = {(x i , a i , y i )} N i=1 , where x i represents the features, a i represents the sensitive attribute, and y i represents the target variable for the i-th sample.The optimization problem can be formulated as follows:EQUATIONEQUATIONwhere f (x; θ a , θ f ) is the model parameterized by the accuracy parameters θ a and fairness parameters θ f , L acc is the accuracy loss function (e.g., binary cross-entropy), and L f air is the fairness loss function (e.g., demographic parity loss). We showed that the demographic parity loss function, when applied to the output of neural network layers, is also Lipschitz continuous (Theorem 3.10).Demographic Parity Loss Function: The demographic parity loss function DP (f ) is defined as:EQUATIONwhere a is a sensitive attribute (e.g., race, gender) with two possible values (0 and 1), and p(x|a) is the conditional probability distribution of x given a. We can easily extend this theorem to include another common fairness metric, equalized odds, as explained in Section A.2.Here's a practical implementation using gradient-based optimization:Algorithm 1 Fairness-Accuracy Bilevel Optimization Compute accuracy loss:L a = 1 |Ba| i∈Ba L acc (f (x i ; θ a , θ f ), y i ) 6:Update accuracy parameters:θ a ← θ a -η a ∇ θa L a 7:Fairness player's optimization 8:Sample a minibatch B f ⊂ DCompute fairness loss:L f = 1 |B f | i∈B f L f air (f (x i ; θ a , θ f ), a i , y i ) 10:Update fairness parameters:θ f ← θ f -η f ∇ θ f L f 11: end whileIn practice, the model f (x; θ a , θ f ) can be implemented as a neural network with separate layers for accuracy and fairness (Figure 8 ). The accuracy layers are parameterized by θ a , while the fairness layers are parameterized by θ f . The accuracy loss L acc can be chosen based on the task at hand, such as binary cross-entropy for binary classification or mean squared error for regression. The fairness loss L f air can be a fairness metric such as demographic parity loss or equalized odds loss. The learning rates η a and η f control the step sizes for updating the accuracy and fairness parameters, respectively. They can be tuned using techniques like grid search or learning rate scheduling.By implementing this algorithm on a dataset, we can optimize the model to balance accuracy and fairness, guided by the Stackelberg game formulation. At each iteration, the parameters related to accuracy are optimized while keeping the fairness parameters fixed. Then, with the accuracy parameters held constant, the fairness parameters are optimized. This separate optimization process provides fine-grained control over the trade-off between accuracy and fairness.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [Yes] Justification: We mentioned all the assumptions and proofs in the methodology 3, and the appendix, section A.1. 
experiments: 
 In this section, we contrast our methodology with other benchmark approaches found in the literature.We employed two metrics for evaluation: accuracy (higher values preferred) for the classification task, and demographic parity differences (DP, lower values preferred) for fairness assessment. Detailed descriptions of all metrics used and implementation settings are available in sections A.9 and A.10 in the appendix, respectively. We evaluated our method for bias mitigation to various current state-of-the-art approaches. We concentrate on strategies specifically tuned to achieve the best results in statistical parity metrics on tabular studies.We used two well-known benchmark datasets in this field for our experiments which are as follows: UCI Adult Dataset [19] , This dataset is based on demographic data gathered in 1994, including a train set of 30000 and a test set of 15,000 samples. The goal is to forecast if the salary is more than $50,000 yearly, and the binary protected attribute is the gender of samples gathered in the dataset.Heritage Health Dataset [1] , Predicting the Charleson Index, a measure of a patient's 10-year mortality. The Heritage Health dataset contains samples from roughly 51,000 patients of which 41000 are in the training set, and 11000 are in the test set. The protected attribute, which has nine potential values, is age.We compare our results with the following state-of-the-art methods as benchmarks:• CVIB [47] : Achieves fairness using a conditional variational autoencoder.• MIFR [61] : Optimizes the fairness objective with a mix of information bottleneck factor and adversarial learning. • FCRL [25] : Uses specific approximations for contrastive information to maximize theoretical goals, facilitating appropriate trade-offs among statistical parity, demographic parity, and precision. • MaxEnt-ARL [56] : Employs adversarial learning to mitigate unfairness. • Adversarial Forgetting [30] : Uses adversarial learning techniques for fairness.• Fair Consistency Regularization (FCR) [4] : Aims to minimize and balance consistency loss across groups. • Robust Fairness Regularization (RFR) [31] : Considers the worst-case scenario within the model weight perturbation ball for each sensitive attribute group to ensure robust fairness.We compare our proposed FairBiNN method with the traditional Lagrangian regularization approach to empirically validate the theoretical benefits of bilevel optimization. Our analysis focuses on the convergence behavior and stability of both methods. For comprehensive details on performance and computational complexity comparison, refer to Section A.7. In the appendix, we have provided the BCE loss plots over epochs for each dataset (Fig. 2 ) and demonstrated the superior performance of the Bi-level approach compared to the Lagrangian approach. We have also presented a comparative analysis of these approaches for the trade-off between accuracy and Statistical Parity Difference (SPD) in Figure 3 . While Theorem 3.9 in the paper proves that, under certain assumptions, the primary loss function in the bilevel optimization approach is upper bounded by the loss of the Lagrangian approach at the optimal solution, it does not analyze or guarantee the convergence behavior of the algorithms. The empirical results for the Health and Adult datasets show that the bilevel approach outperforms the Lagrangian method in minimizing the BCE loss. However, further investigation is needed to understand the convergence properties of the algorithms and connect the theoretical results with empirical observations. Despite this, the experimental results highlight the potential of the bilevel optimization framework to optimize accuracy and fairness objectives, offering a promising approach to address the multi-criteria fairness problem in neural networks.We provide average accuracy as a measure of most probable accuracy and maximum demographic parity as a measure of worst-case scenario bias, calculated across five iterations of the training process using random seeds. Unlike Gupta et al. [25] , we did not use any preprocessing on the data before feeding it to our network. Reported results for our model are Pareto solutions for the neural network during training with different η f . Results are reported for methods with a multi-layer perceptron classifier with two hidden layers.Figures 1a and 1b show trade-offs of the statistical demographic parity vs. accuracy associated with various bias reduction strategies in the UCI Adult dataset and Heritage Health dataset, respectively. The ideal area of the graph for the result of a method is to measure how much the curve is located in the lower right corner of the graph, which means accurate and fair results concerning protected attributes. Our results demonstrate that the Bilevel design significantly outperforms competing methods in Adult dataset. 
limitations and future work: 
 While our results are promising, it's important to acknowledge several limitations of our current approach:One of the most widely used activation functions, softmax, is not Lipschitz continuous. This limits the direct application of our method to multiclass classification problems. Future work could explore alternative activation functions or modifications to the softmax that preserve Lipschitz continuity while maintaining similar functionality for multiclass problems.Attention mechanisms, which are widely used in modern language models and other architectures, are not Lipschitz continuous. This presents a challenge for extending our method to state-of-theart architectures in natural language processing and other domains that heavily rely on attention. However, research into the Lipschitz continuity of attention layers has already begun, with Dasoulas et al. [16] introducing LipschitzNorm, a simple and parameter-free normalization technique applied to attention scores to enforce Lipschitz continuity in self-attention mechanisms. Their experiments on graph attention networks (GAT) demonstrate that enforcing Lipschitz continuity generally enhances the performance of deep attention models.Our theoretical analysis primarily provides guarantees in comparison to regularization methods. While the results show improvements in fairness overall, the theory does not offer absolute fairness guarantees for the final model. Extending the theoretical framework to include direct fairness guarantees could strengthen the method's applicability.This method was not validated on dataset augmentation approaches, which are common in practice for improving model generalization and robustness. Future work should investigate how our method interacts with various data augmentation techniques and whether it maintains its fairness properties under such conditions.Our current implementation focuses on a single fairness metric (demographic parity). In practice, multiple, sometimes conflicting, fairness criteria may be relevant. Extending our method to handle multiple fairness constraints simultaneously could make it more versatile for real-world applications.Addressing these limitations presents exciting opportunities for future research. By tackling these challenges, we can further enhance the applicability and effectiveness of fair machine learning methods across a broader range of real-world scenarios and cutting-edge architectures. 
discussion and conclusion: 
 Our primary contribution lies in the theoretical foundation and general applicability of the proposed framework, rather than extensive ablation studies on specific datasets or network configurations. However, we recognize the importance of empirical evaluations. Our work introduces a novel approach to addressing the multi-criteria fairness problem in neural networks, supported by theoretical analysis, particularly Theorem 3.8, which establishes properties of the optimal solution under certain assumptions, independent of specific datasets or architectures. The results on vision and graph datasets (A.11) and ablation studies on the impact of η (A.13.1), the position of fairness layers (A.13.3), and different layer types (A.13), presented in the appendix, demonstrate the effectiveness and versatility of our approach. These studies show that the bilevel optimization framework can be successfully applied to various layer types and network architectures, beyond the single linear layer used in the main experiments. Our experimentation across diverse datasets, including UCI Adult, Heritage Health, and other domains like graph datasets [A.8.1] (POKEC-Z, POKEC-N, and NBA) and vision datasets [A.8.2] (CelebA), further illustrates the versatility and efficacy of our method. Including these ablation studies in the appendix allows us to maintain the main text's focus on theoretical contributions and the general framework while providing additional empirical evidence to support our claims.Our results demonstrate the superiority of our model over state-of-the-art fairness methods in reducing bias while maintaining accuracy, highlighting the potential of our framework to advance fairnessaware machine learning solutions. Notably, our study represents a significant contribution by formulating multi-objective problems in neural networks as a bilevel design, providing a powerful tool for achieving equitable outcomes across diverse groups in classification tasks. Future research can address our method's limitations and explore potential directions as outlined in Section 5.There are some common activation functions that are not Lipschitz continuous:1. Softmax 2. Binary Step:BinaryStep(x) = 1, if x ≥ 0 0, if x < 03. Hard Tanh:HardTanh(x) =    -1, if x < -1 x, if -1 ≤ x ≤ 1 1, if x > 1 4.Hard Sigmoid:HardSigmoid(x) =    0, if x ≤ -2.5 1,if x ≥ 2.5 0.2x + 0.5, if -2.5 < x < 2.5Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: Please refer to section A.10.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: We used four common metrics from the literature that were used for evaluation and comparison. Also, we compared the performances as fair as possible. The significance of our work can be easily interpreted with provided tables and figures.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA]Justification: NA Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: All the assets in this work, either are original or are publicly available (e.g. datasets).Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: A.1 Theoretical proofs Assume that the following assumptions are satisfied: Assumption A.1. The primary loss function f (θ p , θ s )(x) is strictly convex in a neighborhood of its local optimum. That is, for any θ p , θ ′ p ∈ Θ p and fixed θ s ∈ Θ s , if θ p ̸ = θ ′ p and θ p , θ ′ p are sufficiently close to the local optimum θ * p , thenEQUATIONfor any λ ∈ (0, 1).Assumption A.2. |θ s -θs |≤ ϵ, where ϵ is sufficiently small, i.e., the steps of the secondary parameters are sufficiently small. θ s and θs represent the parameters for the secondary objective and their updated values, respectively. Assumption A.3. Let f l (.) denote the output function of the l-th layer in a neural network with L layers. For each layer l ∈ 1, . . . , L, there exists a constant c l > 0 such that for any input x l to the l-th layer:EQUATION)where |.| denotes a suitable norm (e.g., Euclidean norm for vectors, spectral norm for matrices). Lemma A.4. Let f (x; θ) be a neural network with L layers, where each layer is a linear transformation followed by a Lipschitz continuous activation function.Let θ be the set of all parameters of the neural network, and θ s ⊆ θ be any subset of parameters. Then, f (x; θ) is Lipschitz continuous with respect to θ s .Proof. Since each activation layer is Lipschitz continuous with Lipschitz constant L l we have:EQUATIONby the triangle inequality and 24 we have:EQUATIONby assumption A.3 and Eq. 27 we have:EQUATIONWe can write the neural network as the composition of functions of each layer:EQUATIONAccording to triangle inequality, we can write:EQUATIONSince the composition of Lipschitz Continuous functions is Lipschitz continuous with Lipschitz constant equal to the product of individual Lipschitz constants [24] we can write:EQUATIONby using Eq. 30 and 31 we can write:EQUATIONwith Cauchy-Schwarz inequality we have:EQUATION)EQUATIONfor all θ = (θ s , θ s) and constant θ s we have:EQUATIONTheorem A. EQUATIONNow, consider the primary loss function f (θ p , θ s ) for a fixed θ s . By Assumption A.1, f (.) is strictly convex in a neighborhood of its local optimum θ * p . This means that for θp and θ p sufficiently close to θ * p :EQUATION) for any λ ∈ (0, 1).Since φ( θp , θs ) ≤ φ(θ p , θ s ), and the secondary loss function φ(.) is used to update the parameters, we can assume that θp is closer to the local optimum θ * p than θ p . Therefore, by the strict convexity of f (.):EQUATIONCombining the results from ( 37) and (39):EQUATION)As ϵ is sufficiently small, we can conclude that f ( θp , θs ) ≤ f (θ p , θ s ).Therefore, under the given assumptions, if φ( θp , θs ) ≤ φ(θ p , θ s ), then f ( θp , θs ) ≤ f (θ p , θ s ).Theorem A.6. Let φ(θ p , θ s ) be the secondary loss function, where θ p ∈ Θ p and θ s ∈ Θ s are the primary and secondary task parameters, respectively. Let (θEQUATIONs ) denote the parameters at optimization step t, and let (θ(t+1) p , θ (t+1) s) be the updated parameters obtained by minimizing φ(θ (t) p , θ s ) with respect to θ s using a sufficiently small step size η > 0, i.e.:EQUATIONThen, for a sufficiently small step size η, the updated secondary parameters θ (t+1) s are the unique minimum solution for the secondary loss function φ(θ(t) p , θ s ).Assumption A.7. φ(θ p , θ s ) is smooth and Lipschitz continuous.p be fixed at optimization step t. We consider the optimization problem of minimizing the secondary loss function φ(θ (t) p , θ s ) with respect to θ s .By the assumption, φ(θ (t) p , θ s ) is smooth and Lipschitz continuous with respect to θ s . This implies that φ(θ (t) p , θ s ) is continuously differentiable and its gradient ∇ θs φ(θ (t) p , θ s ) is Lipschitz continuous with some Lipschitz constant L > 0, i.e., for any θ s , θ ′ s ∈ Θ s :EQUATIONNow, consider the update rule for θ s with a sufficiently small step size η > 0:EQUATIONWe want to show that for a sufficiently small η, θ(t+1) sis the unique minimizer of φ(θ(t) p , θ s ).By the Lipschitz continuity of ∇ θs φ(θ(t)p , θ s ) and the update rule, we have:EQUATIONIf we choose η < 2 L , then 1 -Lη 2 > 0, and we have:EQUATIONThis implies that θ (t+1) s is a strict minimizer of φ(θ(t) p , θ s ).To show that θ (t+1) s is the unique minimizer, suppose there exists another minimizer θs ̸ = θ (t+1) s. By the strict inequality above, we must have:EQUATIONwhich contradicts the assumption that θs is a minimizer.Therefore, for a sufficiently small step size η < 2 L , the updated secondary parameters θ (t+1) s are the unique minimum solution for the secondary loss function φ(θEQUATION)Demographic Parity Loss Function: The demographic parity loss function DP (f ) is defined as:EQUATION)where a is a sensitive attribute (e.g., race, gender) with two possible values (0 and 1), and p(x|a) is the conditional probability distribution of x given a. Theorem A.9. If f (x) is Lipschitz continuous with Lipschitz constant L f , then the demographic parity loss function ℓ DP (f ) is also Lipschitz continuous with Lipschitz constant L DP = 2L f .Proof. Let f 1 (x) and f 2 (x) be two functions that are Lipschitz continuous with Lipschitz constant L f . We want to show that:EQUATIONwhere∥f 1 -f 2 ∥ ∞ = sup x |f 1 (x) -f 2 (x)|.Consider the difference between the demographic parity loss functions:EQUATIONEQUATION)The first inequality follows from the reverse triangle inequality, the second inequality is trivial, and the third inequality follows from the Lipschitz continuity of f 1 and f 2 .Therefore, the demographic parity loss function ℓ DP (f ) is Lipschitz continuous with Lipschitz constant L DP = 2L f .In Theorem A.9, we established the Lipschitz continuity of the demographic parity loss function. This approach can similarly be applied to another widely used fairness loss function, known as the equalized odds loss. The Equalized Odds Difference measures the extent to which a model's predictions deviate from equalized odds by quantifying differences in true positive rates (TPR) and false positive rates (FPR) across different groups. Mathematically, it is defined as follows:For true positive rate (TPR):EQUATIONFor false positive rate (FPR):EQUATIONThe overall EO loss can then be considered as the maximum of these two differences:EQUATION)Following the logic presented in Theorem A.9, we can determine the Lipschitz constants L T P R and L F P R for the true positive rate and false positive rate, respectively. The Lipschitz constant for the equalized odds loss can then be expressed as max(L T P R , L F P R ).Our work on the FairBiNN method introduces a novel approach to addressing the fairness-accuracy trade-off in machine learning models through bilevel optimization. The theoretical foundations and empirical results demonstrate the potential of this method to outperform traditional approaches like Lagrangian regularization. However, it's crucial to examine how the underlying assumptions of our theory translate to real-world applications.One key assumption in our theoretical analysis is the convexity of the loss function near local optima. In practice, this assumption translates to the behavior of neural networks as they converge during training. While neural network loss landscapes are generally non-convex, recent research suggests that they often exhibit locally convex regions around minima, especially in overparameterized networks [3] . In real-world scenarios, as long as the network converges, it will likely encounter these locally convex regions along its optimization path. Our theory applies particularly well in these parts of the optimization process. This assumption becomes increasingly valid as the network approaches convergence, which is typically the case for well-designed models trained on suitable datasets. Therefore, practitioners can rely on this aspect of our theory as long as their models show signs of convergence on the given data.The assumption of overparameterization in our model is another critical aspect that warrants discussion. In modern deep learning, overparameterized models -those with more parameters than training samples -are increasingly common. This trend aligns well with our theoretical framework. In practical terms, as long as the model is capable of overfitting on the training data, this assumption stands. This condition is often met in real-world scenarios, especially with deep neural networks applied to typical dataset sizes. The overparameterization allows for the existence of multiple solutions that can fit the training data, providing the flexibility needed for our bilevel optimization approach to find solutions that balance accuracy and fairness effectively. However, it's important to note that in some resource-constrained environments or with extremely large datasets, overparameterization might not always be feasible. In such cases, the applicability of our method may require further investigation or adaptation.The assumption of Lipschitz continuity is crucial for the stability and convergence properties of our optimization process. In our experiments, we ensured that the chosen layers and loss functions satisfy Lipschitz continuity, thus upholding this assumption. For practitioners, we provide a rigorous analysis of various layers and activation functions in terms of their Lipschitz properties. This analysis serves as a guide for choosing components that maintain the Lipschitz continuity assumption. Common choices like ReLU activations and standard loss functions (e.g., cross-entropy) are Lipschitz continuous, making this assumption generally applicable in many practical scenarios. However, care must be taken when using certain architectures or custom loss functions. For instance, unbounded activation functions or poorly designed custom losses might violate this assumption. We recommend that practitioners refer to our provided analysis when designing their models to ensure compliance with this crucial property.The assumption of bounded layer outputs translates to several practical considerations in neural network design and implementation:A.4.Various normalization techniques help ensure that the outputs of layers remain bounded:• Batch Normalization: normalizes the output of a layer by adjusting and scaling the activations• Layer Normalization: similar to batch normalization but normalizes across the features instead of the batch • Weight Normalization: decouples the magnitude of a weight vector from its directionCertain regularization techniques indirectly encourage bounded outputs:• L1/L2 regularization: by penalizing large weights, these methods indirectly limit the magnitude of layer outputs • Dropout: by randomly setting some activations to zero, dropout can help prevent excessively large outputsBy implementing these techniques, practitioners can design neural networks that better align with the theoretical assumption of bounded layer outputs. This alignment potentially leads to more stable training and improved generalization properties, bridging the gap between theoretical guarantees and practical implementations.According to our theoretical framework, we can guarantee the pareto front solution when using activation functions that are Lipschitz continuous (smooth).Activation functions (f ) that are Lipschitz continuous have a property where there exists a constant L such that for any x, y:EQUATIONAs an example, we can prove sigmoid function is Lipschitz continuous. we need to show that there exists a constant L such that for all x, y ∈ R :EQUATIONwhere σ(x) is the sigmoid function defined as:EQUATIONThe first step in proving Lipschitz continuity is to find the derivative of the sigmoid function, which gives us the rate of change. The derivative of the sigmoid function is:EQUATIONSince σ(x) is always between 0 and 1, the expression σ(x)(1-σ(x)) is maximized when σ(x) = 0.5.The maximum value of σ ′ (x) occurs at σ(x) = 0.5, which gives:EQUATIONTherefore, the derivative of the sigmoid function is bounded by 0.25:EQUATIONThis means that the Lipschitz constant L is 0.25, and Sigmoid Function is smooth.Similar to this proof, we can show that following common activation functions are also Lipschitz continuous:1. Linear: f (x) = x with constant L = 1 2. Hyperbolic Tangent (Tanh): f (x) = tanh(x) with constant L = 13. ReLU (Rectified Linear Unit): f (x) = max(0, x) with constant L = 1 4. Leaky ReLU: f (x) = max(αx, x) where α is a small positive constant, with constant L = max(1, α).5. ELU (Exponential Linear Unit):ELU(x) = x, if x > 0 α(e x -1), if x ≤ 0The ELU function is Lipschitz continuous, but the constant depends on the value of α.The Lipschitz continuity of a CNN layer can be determined by examining its components: convolution operations, activation functions, and pooling layers. Convolution is a linear operation, and its Lipschitz constant is related to the spectral norm of the convolution matrix, typically limited by the sum of the absolute values of the weights. Activation functions like ReLU and sigmoid are Lipschitz continuous, with constants of 1 and 0.25, respectively. Pooling operations, such as max and average pooling, are also Lipschitz continuous, with max pooling having a constant of 1 and average pooling having a constant dependent on pooling size. Therefore, a CNN layer is Lipschitz continuous if all its components are, with the overall Lipschitz constant being the product of the constants of these components.Zoe et al. [80] developed a linear programming approach to estimate the Lipschitz bound of CNN layers. Their method leverages concepts such as the Bessel bound, discrete signal processing, and the discrete Fourier transform to calculate the Lipschitz constant for each layer in popular architectures like AlexNet and GoogleNet.Graph Neural Network (GNN) layers operate on graph-structured data through a series of messagepassing, aggregation, and update steps, each contributing to the Lipschitz continuity of the layer. In the message-passing step, functions aggregate information from neighboring nodes and are often linear or involve nonlinearities; linear message-passing functions are Lipschitz continuous, with the constant depending on the weights and the graph's maximum degree. Aggregation functions, such as sum, mean, and max, are Lipschitz continuous, with sum and mean being linear, and max having a constant of 1. Update functions apply neural networks to aggregated information, and if composed A recent study by Juvina et al. [32] presents a learning framework designed to maintain tight Lipschitzbound constraints across various GNN models. To facilitate easier computations, the authors utilize closed-form expressions of a tight Lipschitz constant and employ a constrained optimization strategy to monitor and control this constant effectively. Although this is not the first attempt to control the Lipschitz constant, the authors successfully reduce the size of the matrices involved by a factor of K 2 , where K is the number of nodes in the graph. While previous works, such as Dasoulas et al. [16] , focused on controlling the Lipschitz constant for basic attention-based GNNs, Juvina et al. [32] also extend this approach to enhance the robustness of GNN models against adversarial attacks.In this subsection, we present a comprehensive comparison between our proposed FairBiNN method and the traditional Lagrangian regularization approach. This comparative analysis serves multiple purposes. Primarily, it aims to empirically validate the theoretical advantages of the bilevel optimization framework outlined in our earlier analysis. By doing so, we demonstrate how the FairBiNN method translates theoretical benefits into practical performance gains in terms of both accuracy and fairness metrics. Furthermore, this comparison provides insight into the convergence behavior and stability of both methods under various hyperparameter settings. It illustrates the flexibility of the FairBiNN approach in managing the trade-off between model accuracy and fairness constraints, a crucial aspect in real-world applications of fair machine learning.We trained both models on the Adult and Health datasets, using the same network architecture, same number of parameters, and optimization settings. Figure 2a displays the BCE loss over epochs for the Adult dataset. The Bi-level approach demonstrates better performance compared to the Lagrangian approach, achieving a lower BCE loss of approximately 0.23 by epoch 200, while the Lagrangian approach reaches a loss of about 0.26. Similarly, Figure 2b shows the BCE loss over epochs for the Lagrangian, Bi-level, and Without Fairness approaches on the Health dataset.Through this direct comparison, we aim to bridge the gap between theoretical analysis and practical implementation, showcasing how the principled design of the FairBiNN method leads to tangible improvements in fair machine learning tasks. This offers practitioners a clear understanding of when and why they might choose the FairBiNN method over the Lagrangian approach in real-world scenarios. We trained both FairBiNN and Lagrangian models on the Adult and Health datasets, using the same network architecture (Same numeber of parameters) and optimization settings for fair For each method, we varied the fairness-accuracy trade-off parameter (η for FairBiNN, λ for Lagrangian) to generate a range of models with different accuracy-fairness balances.Figure 3 presents a comparative analysis of the FairBiNN and Lagrangian methods on two benchmark datasets: UCI Adult (Figure 3a ) and Heritage Health (Figure 3b ). The graphs plot the trade-off between accuracy and Statistical Parity Difference (SPD), a measure of fairness where lower values indicate better fairness. For the UCI Adult dataset (Figure 3a ), we observe that the FairBiNN method consistently outperforms the Lagrangian approach. The FairBiNN curve is closer to the top-left corner, indicating that it achieves higher accuracy for any given level of fairness (SPD). The difference is particularly pronounced at lower SPD values, suggesting that FairBiNN is more effective at maintaining accuracy while enforcing stricter fairness constraints. The Heritage Health dataset results (Figure 3b ) show a similar trend, but with a more dramatic difference between the two methods. The FairBiNN curve dominates the Lagrangian curve across the entire range of SPD values. This indicates that FairBiNN achieves substantially higher accuracy for any given fairness level, or equivalently, much better fairness for any given accuracy level. In both datasets, the FairBiNN method demonstrates a smoother, more consistent trade-off between accuracy and fairness. The Lagrangian method, in contrast, shows more erratic behavior, particularly in the Heritage Health dataset where its performance degrades rapidly as fairness constraints tighten. These results empirically validate the theoretical advantages of the FairBiNN method discussed earlier in the paper. They suggest that the bilevel optimization approach is more effective at balancing the competing objectives of accuracy and fairness.Let's define the following variables:• n: number of parameters in θ p• m: number of parameters in θ s• C f : cost of computing f and its gradients• C ϕ : cost of computing ϕ and its gradientsThe Lagrangian update rules are:EQUATIONComputational complexity per iteration:O(C f + C ϕ + n + m)The bilevel update rules are:Lower level:EQUATIONUpper level:EQUATIONComputational complexity per iteration:O(C f + C ϕ + n + m)While the theoretical complexity analysis suggests similar costs for both methods, we conducted empirical tests to compare their actual runtime performance. The message-passing structure of GNNs and the topology of graphs both have the potential to amplify the bias. In general, in graphs such as social networks, nodes with sensitive features similar to one another are more likely to link to one another than nodes with sensitive attributes dissimilar from one another [18, 52] . On social networks, for instance, persons of younger generations have a higher tendency to form friendships with others of a similar age [18] . This results in the aggregation of neighbors' features in GNN having similar representations for nodes of similar sensitive information while having different representations for nodes of different sensitive features, which leads to severe bias in decision-making, in the sense that the predictions are highly correlated with the sensitive attributes of the nodes. GNNs have a greater bias due to the adoption of graph structure than models that employ node characteristics [14] . Because of this bias, the widespread use of GNNs in areas such as the evaluation of job candidates [44] and the prediction of drug-target interaction [72, 36] would be significantly hindered. As a result, it is essential to research equitable GNNs. The absence of sensitive information presents significant problems to the work that has already been done on fair models [5, 13, 39, 41, 64] . Despite the significant amount of work that has been put into developing fair models through the revision of features [33, 34, 78] , disentanglement [13, 41] , adversarial debiasing [5, 21] , and fairness constraints [74, 75] , these models are almost exclusively designed for independently and identically distributed (i.i.d) data, meaning that they are unable to be directly applied to graph data due to the fact that they do not simultaneously take into consideration the bias that comes from node attributes and graph. In recent years, Bose and Hamilton [6] , Rahman et al. [52] have been published to learn fair node representations from graphs. These approaches only deal with simple networks that do not have any properties on any of the nodes, and they place their emphasis on fair node representations rather than fair node classifications. Finally, Dai and Wang [14] used graph topologies and a restricted amount of protected attributes and designed FairGNN to reduce the bias of GNNs while retaining high node classification accuracy.The challenges caused by bias in computer vision might appear in various ways. It has been found, for instance, that in action recognition models, when the data include gender bias, the bias is exacerbated by the models trained on such datasets [79] . Face detection and recognition models may be less precise for some racial and gender categories [7] . Methods for mitigating bias in vision datasets are suggested in [65] and [71] . Several researchers have used GANs on image datasets for bias reduction. Sattigeri et al. [57] altered the utility function of GAN in order to generate equitable picture datasets. FairFaceGAN [29] provides facial image-to-image translation to avoid unintended transfer of protected characteristics. Roy and Boddeti [56] developed a method to mitigate information leakage on image datasets by formulating the problem as an adversarial game to maximize data utility and minimize the amount of information contained in the embedding, measured by entropy. Ramaswamy et al. [55] presents a methodology to generate balanced training data for each protected property by perturbing the latent vector of a GAN. Other experiments using GANs to generate accurate data are [8, 58] . Beyond GANs, many strategies have addressed the challenge of AI fairness.[54] proposed a U-Net for creating unbiased image data. Deep information maximization adaption networks were employed to eliminate racial bias in face vision datasets [67] , while reinforcement learning was utilized for training a race-balanced network [66] . Wang et al. [69] offer a generative few-shot cross-domain adaptation method for performing fair cross-domain adaptation and enhancing minority category performance. The research in [70] recommends adding a penalty term to the softmax loss function to reduce bias and enhance face recognition fairness performance. Quadrianto et al. [51] describes a technique for discovering fair data representations with the same semantic information as the original data. There have also been effective applications of adversarial learning for this purpose [68, 77] . [9] proposed fair mixup, which uses data augmentation to mitigate bias in data.We utilize four metrics to compare our model's performance against baseline models. Average precision (AP) is utilized to gauge classifier accuracy, combining recall and accuracy at each point.In tabular and graph datasets, we opt for accuracy to align with existing literature practices.Fairness evaluation draws from various criteria, with demographic parity (DP) being widely used. DP quantifies the difference in favorable outcomes across protected groups, expressed as (|P (Y = 1|S = 0) -P (Y = 1|S = 1)|) [44] . For scenarios involving more than two groups, DP can be calculated as ∆ DP (a, ŷ) = max ai,aj |P (ŷ = 1|a = a i ) -P (ŷ = 1|a = a j ))| [25] . A smaller DP indicates fairer categorization. We also adopt the difference in equality of opportunity (∆EO), which measures the absolute difference in true positive rates between gender expressions (|T P R(S = 0)-T P R(S = 1)|) [40, 55] . Minimizing ∆EO signifies fairer outcomes. Demographic parity serves as the fairness criterion in our optimization. Discrepancies between EO and DP may occur due to this choice.The hyperparameters used in training the models on each dataset can be found in the tables 2, 3, and 4. The training was conducted on a computer with an NVIDIA GeForce RTX 3090. We compare our suggested framework with some of the cutting-edge approaches for fair classification, and fair graph embedding learning including ALFR [21] , ALFR-e, Debias [77] , Debias-e, and FCGE [6] . In the ALFR [21] approach, which is a pre-processing technique, the sensitive information in the representations created by an MLP-based autoencoder is eliminated using a discriminator. Then the debiased representations are used to train the linear classifier. ALFR-e is a method to make use of the graph structure information and joins the user features in the ALFR with the graph embeddings discovered by deepwalk [48] . Debias [77] , is a fair categorization technique used throughout processing. It immediately applies a discriminator to the predicted likelihood of the classifier. Debiase, which is similar to the ALFR-e, also includes deepwalk embeddings into the Debias characteristics. FCGE [6] , is suggested as a method for learning fair node embeddings in graphs without node characteristics. FairGCN [14] , a graph convolutional network designed for fairness in graph-based learning. It incorporates fairness constraints during training to reduce disparities between protected groups. FairGAT [37] , a fairness-aware graph-based learning framework that employs a novel attention learning strategy to reduce bias. This framework is grounded in a theoretical analysis that identifies sources of bias in GAT-based neural networks used for node classification. NT-FAIRGNN [15] is a graph neural network that aims to achieve fairness by balancing the trade-off between accuracy and fairness. It uses a two-player minimax game between the predictor and the adversary, where the adversary aims to maximize the unfairness. Discriminators screen out the delicate data in the embeddings. We used the Dai and Wang [14] study's obtained datasets for our investigation which are as follows:Pokec [62] is among the most well-known social network datasets in Slovakia which resemble Facebook and Twitter greatly. This dataset includes anonymous information from the whole social network of the year 2012. User profiles on Pokec include information on gender, age, interests, hobbies, profession, and more. There are millions of users in the original Pokec dataset. Sampled Pokec-z and Pokec-n datasets are based on the provinces that users are from. The categorization task involves predicting the users' working environment. NBA is a Kaggle dataset with about 400 NBA basketball players that served as the basis for this extension. Players' 2016-2017 season success statistics, along with additional details like nationality, age, and income are presented. They gathered the relationships between NBA basketball players on Twitter using its official crawling API to create the graph connecting the NBA players. They separated the nationality into two groups, American players and international players, which is a sensitive characteristic. The classification job is to predict whether a player's wage is above the median. Each experiment was conducted five times, and Tables 5, 6 , and 7 report the mean and standard deviation of the runs for Pokec-z, Pokec-n, and NBA datasets, respectively. These results represent the selected Pareto solutions for comparison with the benchmarks. The tables reveal that, in comparison to GAT, generic fair classification techniques and graph embedding learning approaches exhibit inferior classification performance, even when utilizing graph information. In contrast, our Bilevel design performs comparably to baseline GNNs. FairGCN is close to the baseline, but the FairBiNN approach outperforms it. When sensitive information is scarce (e.g., NBA dataset), baselines exhibit clear bias, with graph-based baselines performing worse. However, our proposed model yields near-zero statistical demographic parity, indicating effective discrimination mitigation.We compare our method on vision task with (1) empirical risk minimization (ERM), which accomplishes training task without any regularization, (2) gap regularization, which directly regularizes the model, (3) adversarial debiasing [77] , and (4) Fairmixup [9] . To showcase the effectiveness of our method, we employed the CelebA dataset of face attributes [38] , which comprises over 200,000 images of celebrities. Each image in the dataset has been assigned 40 binary attributes, including gender, that were labeled by humans. We selected the attributes of attractive, smiling, and wavy hair and used them in three binary classification tasks, with gender serving as the protected attribute. The reason we chose these three attributes is that each of them has a group that is sensitive to them and receives a disproportionately high number of positive samples. We trained a ResNet-18 model [27] for each task and added two additional layers to predict the outcomes. The trade-off between Average Precision (AP), Demographic Parity (DP), and Equality of Opportunity (EO) for attributes "Attractive", "Smiling", and "Wavy Hair" is illustrated in the figures 4, 5, and 6 respectively. Our proposed method provides a more balanced trade-off between accuracy and fairness. Instead of prioritizing one over the other, our method strikes a better balance, ensuring that the trained model is both accurate and fair. Moreover, the FairBiNN model consistently provides better equality of opportunity across various accuracy levels compared to benchmark models. Through empirical validation on multiple benchmarks, we've shown that the FairBiNN approach consistently outperforms other methods in achieving equality of opportunity across various accuracy levels. This indicates that our method can provide fair treatment to different protected groups while still maintaining high predictive accuracy.Furthermore, we demonstrate the power of the bilevel design for fairness by visualizing the t-SNE plot. The t-SNE visualization of z (output of the ResNet-18 before the classification layer without the fairness layers) and z (output of the ResNet-18 before the classification layer with the Bilevel fairness) are shown in Figures 7a and 7b , demonstrating that z clusters by gender, but z does not.Further insights about ablation study outcomes are detailed in section A.13.To better illustrate the training process outlined in Algorithm 1, we present the network architecture in Figure 8 .To understand the sensitivity of our FairBiNN model to the choice of η, we conducted an ablation study on both the Adult and Health datasets. The parameter η controls the trade-off between accuracy and fairness in our bilevel optimization framework. We varied η across a range of values: 1-6000 for Adult dataset, and 1-3000 for Health dataset. For each value of η, we trained the FairBiNN model on both the Adult and Health datasets, keeping all other hyperparameters constant. We evaluated the models based on accuracy and demographic parity (DP).show the results of our ablation study for the Adult and Health datasets, respectively. The results demonstrate a clear trade-off between accuracy and fairness as η varies. For both datasets: As η increases, the demographic parity (DP) decreases, indicating improved fairness. However, this improvement in fairness comes at the cost of reduced accuracy. The relationship is not linear; there are diminishing returns in fairness improvement as η increases, especially at higher values. For the Adult dataset, setting η = 1000 appears to offer a good balance, achieving a DP of 0.012 while maintaining an accuracy of 82.9%. For the Health dataset, η = 700 also provides a reasonable trade-off with a DP of 0.23 and an accuracy of 80.2%. These results highlight the importance of carefully tuning η to achieve the desired balance between accuracy and fairness. The optimal value may vary depending on the specific requirements of the application and the characteristics of the dataset. This ablation study demonstrates that our FairBiNN model provides a flexible framework for managing the accuracy-fairness trade-off through the η parameter. Practitioners can adjust η based on their specific fairness requirements and acceptable accuracy thresholds. Future work could explore adaptive schemes for setting η during training to automatically find an optimal balance. Position of Fairness Layers To understand the impact of the position of fairness layers within the network architecture, we conducted an ablation study varying their placement. This study aims to identify the optimal position for fairness layers and provide insights into why certain positions may be more effective.We tested 4 fairness layer positions on the Adult dataset and 5 fairness layer positions on the Health dataset. For each configuration, we kept the total number of parameters constant to ensure a fair comparison. We evaluated the models based on accuracy and demographic parity (DP).Results Figures 10a and 10b show the results of our ablation study for the Adult and Health datasets, respectively. The results consistently show that placing the fairness layers just before the output layer (in the last hidden layer) yields the best performance in terms of both accuracy and fairness. This configuration achieves the highest accuracy while maintaining the lowest demographic parity on both datasets.Several factors contribute to the superior performance of fairness layers when placed in the last hidden layer:• Rich Feature Representations: By the time the data reaches the last hidden layer, the network has already learned rich, high-level feature representations. This allows the fairness layers to operate on more informative features, potentially making it easier to identify and mitigate biases. • Minimal Information Loss: Placing fairness layers earlier in the network might lead to loss of important information that could be useful for the classification task. By positioning them at the end, we ensure that all relevant features are preserved throughout most of the network. • Direct Influence on Output: Being closest to the output layer, fairness layers in this position have the most direct influence on the final predictions. This allows for more effective bias mitigation without excessively disturbing the learned representations in earlier layers. • Gradient Flow: In backpropagation, gradients from the fairness objective have a shorter path to travel when the fairness layers are near the output. This might lead to more stable and effective updates for bias mitigation. • Adaptability: Fairness layers at the end of the network can adapt to various biases that might emerge from complex interactions in earlier layers, providing a final "correction" before the output.In this subsection, we perform an ablation study to investigate the effects of different functions for the fairness layers. The fairness layer can be any differentiable function with controllable parameters 9 .The justification for the performance differences between the ResBlock and the fully connected models in our ablation study lies in the proportion of the model occupied by the fairness layers and the specific contributions of these layers to different parts of the network. In particular, there are two primary factors that explain the observed performance differences:• Role in the Network: The ResBlock and the fully connected modules serve different purposes within the network. The ResBlock contributes to the embedding space of the image, which includes feature extraction and representation learning. This enables the model to capture the essential characteristics of the image while minimizing the effect of the protected attributes (e.g., gender) on the classification task. In contrast, the fully connected module is mainly involved in the classification part of the network, where it contributes to the decision-making process based on the features extracted from the previous layers. This distinction in roles explains why the ResBlock provides more fair results, as it directly affects the representation learning and reduces the influence of the protected attributes on the embeddings. • Flow of Data: The flow of data through the ResBlock is different from the flow through the fully connected and CNN modules. ResBlocks have skip connections that allow the input to bypass some layers and directly flow to the subsequent layers. These skip connections help in preserving the original information and preventing the loss of critical features during the network's forward pass. As a result, the ResBlock is more effective in capturing the inherent relationships in the data while mitigating the bias from the protected attributes [28] .In contrast, CNNs involve multiple convolution and pooling operations, which can cause the loss of some information relevant to fairness. The fully connected module, with its dense layers, lacks the skip connections present in the ResBlock, which can lead to less effective bias mitigation.In conclusion, our ablation study demonstrates that the choice of layer in the fairness layers can significantly impact the fairness and accuracy of the model. It is essential to strike a balance between fairness and accuracy and to select the appropriate fairness layer for the specific dataset and application at hand. This work introduces a novel bilevel optimization framework for multi-objective optimization in neural networks. While we use fairness as a case study, it's important to note that our method is not inherently a fairness research technique, but rather a general optimization approach that can be applied to various secondary objectives.In the context of our fairness case study, our FairBiNN method shows promising results in optimizing the trade-off between demographic parity (a measure of group fairness) and accuracy. While these results are encouraging, it is crucial to consider the broader ethical implications and potential societal impacts of applying this technique to fairness problems.On the positive side, when applied to fairness, our approach could help reduce discriminatory outcomes in high-stakes automated decision making systems, promoting more equitable treatment across protected groups in domains like hiring, lending, and healthcare [44, 76] . By providing a flexible framework to manage the accuracy-fairness trade-off, practitioners can fine-tune models to meet specific fairness requirements mandated by regulations or organizational policies.However, we must also consider potential negative consequences. There is a risk that mathematical notions of fairness like demographic parity could provide a false sense of ethical assurance, when fairness is a complex social and philosophical concept that cannot be fully captured by simple statistical measures [20, 50] . Our focus on group fairness, while important, does not guarantee individual fairness [73] .The mathematical formulation we present, while rigorous, should not be seen as providing absolute ethical guarantees when applied to fairness problems. Over-reliance on our method without careful consideration of the broader context could lead to unintended harms [9, 39] .As with any machine learning technique, it is the responsibility of practitioners to properly configure and test models for their specific use cases. Our method provides additional tools for optimization but does not absolve practitioners of their ethical obligations. We believe that providing more precise control over trade-offs actually enables more ethical implementations. However, practitioners must be aware that while our approach performs well in our experiments, particularly in balancing demographic parity and accuracy, real-world applications may present unforeseen challenges and edge cases [31, 49] .Ultimately, while our work provides a useful tool for multi-objective optimization in neural networks, its application to fairness should not be seen as a complete solution to algorithmic bias. Continued interdisciplinary collaboration between computer scientists, ethicists, policymakers and impacted communities is essential to develop AI systems that are truly fair and beneficial to society [6, 60, 71] .Justification: All the data and code are publicly available. We can provide the link at any time. For the review version, we did not include the link to keep it as an anonymous review.The supplementary Zip file including data and code has been attached.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.38th Conference on Neural Information Processing Systems (NeurIPS 2024). 


Decision: The current paper has its strengths in its theoretical contributions towards introducing a novel bilevel optimization framework to address the fundamental problem of finding solutions on the Pareto front in ML fairness. The theoretical rigor, including guarantees and proofs, are significant contributions and seems to be unanimously agreed upon by the reviewers. The ablation studies presented in the paper, seems to highlight the practical potential of the proposed approach. The broad array of experiments demonstrates the method's efficacy across various domains as well as stability of their approach.

There were concerns raised by the reviewers on the practicality of theoretical assumptions with respect to real-world scenarios as well as lack of discussions towards limitations of the proposed approach. Going over the rebuttal as well as the discussions between the authors and reviewers, I see that the authors seem to have addressed this in the context of current prevalent practices in deep learning implementations. Including them in the revision should bring more clarity over practical utility of the paper. Authors seem to have also provided additional discussion on choice of baselines and comparison with Lagrangian regularization method which seems fair considering their theoretical results. The reviews and rebuttal also highlights that a discussion over limitations of the proposed approach needs to be included in the final version. 

Considering the paper's contributions novel and significant, I **recommend acceptance** with revisions addressing the identified concerns.

Following are the specific items that the authors should address -

1. **Limitations**: Include a brief discussion in the main text and a detailed appendix section outlining the limitations of the current approach.
2. **Ethics**: Add an ethics section to discuss potential ethical implications and considerations related to the proposed method.
3. **Parameter Selection**: Include a figure in main text clarifying parameter selection in neural networks. Explicitly stating correspondence of parameters to notations in theory.
4. **Assumptions**: Add a detailed section in Appendix which clarifies how different components in deep networks fit or do not fit under the proposed assumptions.
5. Computational Complexity Analysis to be added somewhere. 
6. Update the discussion in the experiments section explaining the choices of baselines for comparison with the Lagrangian approach.
7. **Practical Implementation**: Address practical implementation concerns like hyperparameter selection, model reusability, and extension to multiple fairness constraints.

Addressing these concerns and incorporating the suggested revisions will significantly strengthen the paper and enhance its impact.

Reviews: review: paper_summary: paper_summary: This paper proposes a novel bilevel optimization framework called FairBiNN for addressing bias and fairness issues in machine learning models while maintaining accuracy. The approach formulates the problem as a Stackelberg game between accuracy and fairness objectives, proving that it yields Pareto-optimal solutions under certain assumptions. Theoretical analysis shows the method performs at least as well as Lagrangian approaches. Experiments on tabular, graph, and vision datasets demonstrate competitive or superior performance compared to state-of-the-art fairness methods in balancing accuracy and fairness metrics like demographic parity and equality of opportunity.
strength_weakness: strength_weakness: 1. The bilevel optimization framework may be more complex to implement and understand compared to traditional methods.
2. The theoretical guarantees rely on several assumptions that may not hold in all practical scenarios.
questions: questions: 1. How sensitive is the method to the choice of hyperparameters, especially $\eta$?
2. How does the computational complexity compare to the baseline methods?
3. Are there any scenarios where the method might not perform as well as traditional approaches?
limitations: limitations: 1. The paper does not explicitly discuss limitations of the proposed approach.
2. Experiments are limited to a few datasets per domain; more diverse datasets could strengthen generalizability claims.
3. The approach is not tested on very large-scale datasets or models
paper_summary: paper_summary: A common approach to bias mitigating in machine learning is to add a regularization term to the loss function that penalizes deviation from fairness. This is what the current paper calls the Lagrangian approach. It has long been know that this is not an effective approach to multi-objective optimization. The paper proposes to use bilevel optimization to tackle this challenge.
strength_weakness: strength_weakness: The following identified weaknesses could be due to my own misunderstanding:

*I could not understand how one would decide on which are accuracy parameters and which are fairness parameters in a neural network architecture. I could not find guidance on this in the paper.

*Except for the simplest types of neural networks, I can't see how a general neural network can satisfy Assumption 3.4

*Where does bilevel optimization sit in the larger context of multi-objective optimization techniques? I think this needs to be explicitly spelled out.
questions: questions: *Line 51: "we introduce a novel method that can be trained on existing datasets without requiring any alterations". Is this an accurate characterization? There certainly is a modification to the training algorithm?
*"Leader-follower" terminology: is this standard in bilevel optimization? Can you add some references?
*In assumption 3.3, $\hat \theta_s$ appears to be undefined
*Could you give some example of common activation functions that are Lipschitz continuous? This would help the reader understand how realistic Lemma 3.5 is for common activation functions.
*Line 208: "demographic loss function of given layers..." what do you mean by "of given layers"? 
*Is there an equivalent of Theorem 3.11 for other fairness metrics considered in your paper? Like equal opportunity? or Equal odds?
*Line 219: "in practice, the model $f$ can be implemented as a neural network with separate layers for accuracy and fairness". This is one of my main confusions about the paper. How does one decide which layers are accuracy layers and which ones are fairness layers? Some of your experiments concern NNs with just one hidden layer, so then which are accuracy and fairness layers there?
*Line 298: "The theoretical analysis, particularly Theorem 4.6, establishes the properties of the optimal solution under certain assumptions, which are not limited to specific datasets or network architectures." I could not find Theorem 4.6. I also think your theoretical analysis is very limited due to the lipschitz continuity assumption. Most NNs used in practice woudl violate this assumption?
limitations: limitations: I would appreciate a further discussion on the limitations of the theoretical analysis. In particular, I believe the theory is highly limited in terms of the type of NNs it can be applied to.
paper_summary: paper_summary: The paper proposes and justifies a bi-level optimization approach to optimizing empirical risk minimization objectives when additional fairness constraints need to be considered. Using assumptions of Lipschitz continuity and local convexity, they prove that a bi-criteria (accuracy + fairness) problem is equivalent to bi-level optimization. Key to their approach is to having separate parameters for fairness and accuracy. Practically, accuracy and fairness layers are made for neural networks. Extensive experiments are done.
strength_weakness: strength_weakness: - Missing experimental details. Compute time of baseline and approaches not presented. Neither is the number of parameters used in each approach. Additionally, some approaches / baselines in the appendix aren't well specified, eg, FairGAT and FairGCN.
- The convexity assumption does restrict the theory.
- Two separate sets of parameters are required. This limits Theorem 3.9's relevance to the typical fairness regularized cases.
questions: questions: 1. What is the compute time of approach? Does the bi-level optimization approach take longer? How do the number of parameters compare? Basically, I am wondering if due to the separate set of parameters needed in the bi-level optimization, the proposed approaches architecture has a larger number of parameters than the baselines. (and if so, would increasing parameter sizes of other approaches increase fairness)
2. What does the overparameterization condition in Theorem 3.9 mean? I am a bit confused about why its needed in Line 186 / 187.
limitations: limitations: Adequate.
paper_summary: paper_summary: The authors propose a bi-level optimization approach to simultaneously optimize a performance and a fairness loss. They show that this approach is, in theory, superior to a typical approach that regularizes the performance objective with a fairness objective. Using tabular, graph and vision datasets, they demonstrate the Pareto frontiers obtained by their method, compared to multiple (mostly regularization) baselines.
strength_weakness: strength_weakness: While I found the idea interesting, I found that the paper lacked clarity, and have a few major comments / concerns at this stage. 

**Major concerns**

- I found that there were many assumptions made, basically requiring that all parameters are smooth, change at a limited rate and improve compared to prior steps. While some of these might be reasonable, the paper lacked a discussion of what these assumptions mean, and in which conditions they might not be respected.
- I am not convinced by Theorem 9 and its proof, as it relies on two functions but DP is computed on a single function (see below).
- I found that the connection between the theory and its implementation is not well detailed and explored.
- To me, discussing works that perform constrained or multi-objective optimization of hyper-parameters, or refer to meta-learning for fairness would be relevant.
- Practically, I see multiple obstacles to the adoption of the proposed method: more hyper-parameters to select, published models cannot be re-used, only one fairness constraint is considered, ... These are not discussed or acknowledged.

**Detailed comments**:
- Related works: I believe the works who perform the multi-objective or constrained optimization of model’s hyperparameters (e.g. Perrone et al., 2020, https://arxiv.org/pdf/2006.05109 and many others) should be discussed, and potentially considered as baselines. While I understand they do not provide the same guarantees, they are easier to implement.
- Can the authors discuss their approach with e.g. Slack et al., 2019 (https://arxiv.org/abs/1911.04336), in which the 2 objectives are separated into learners and a meta-learner instead?
- The related works section describes some related works with one sentence, but does not “group” them into the different avenues they represent, or how they differ from the proposed approach. I would suggest trying to “analyze” the field at a higher level and show what the proposed approach can bring.
- Overall I found the methodology section dense and not easy to follow. The assumptions are also not discussed, simply enumerated and it is unclear how realistic they are.
- The experimental baselines seem to encompass multiple techniques, but are not well described. It is unclear how each of these compares to the proposed approach.
- For the Health dataset, as well as for other datasets (in Appendix), the theoretical results do not seem to hold. This is briefly discussed at the end of the results section, but is a clear limitation of the work. There is also little description of whether the selection of hyper-parameters is more complex for the bi-level case.
- Relatedly, this is presented as an advantage but to me is a weakness of the approach that the architecture needs to be completely revised to include fairness layers. This means that published models cannot easily be reused, and that the whole process needs to be re-trained for a different fairness constraint.
questions: questions: - Line 550: how can ensure that $\hat{\theta}_p$ is closer to the optimum than $\theta_p$? Strict improvement at each step seems like a strong assumption to me. Shouldn’t this be spelled out as an extra assumption?
- How does the assumption of “sufficiently small” $\eta$ maps to the practical implementation, as we do not know the Lipschitz constant?
- Similarly, how about the assumption in Theorem 9 of an overparameterization?
- Line 212, DP: why use $\theta_1$ and $\theta_2$? From Assumption 3.10, it suggests that these represent the parameters for $x_1$ and $x_2$. However, the parameters $\theta$ when computing DP are the same for $a=0$ and $a=1$. I don’t see how we can guarantee that the network’s activations are Lipschitz continuous unless we also make assumptions of the distance between input examples (which can be difficult to estimate). The proof also refers to 2 functions $f_1$ and $f_2$, but we have the same function, only different subsets of inputs $x_{a=0}$ and $x_{a=1}$. Can you clarify please? It is not clear to me how the proof shows that DP is Lipschitz continuous.
- The discussion seems more like a brief summary with a rebuttal paragraph, rather than a proper discussion. The work is not put in perspective with the literature.
- Figures 4 and 5 in Appendix: the proposed method is not strictly better compared to the baselines in all cases. Is there a pattern of failure cases? 

**Minor**:

- Line 64: is the comment pointing towards distribution shifts in the test data? It is unclear what “depends on the data” means.
- “Paractical” line 198
- $\hat{\theta}$ is not defined in Assumption 3.3.
- Equation (1): the description uses the ‘p’ and ‘s’ subscripts while the equation uses ‘a’ and ‘f’. Please correct.
- Is Assumption 3.10 a repeat of definition 3.1?
- Line 276: end with “shows”, I am assuming something is missing.
- Line 545, equation 38: shouldn’t it be $\phi$ instead of f?
- Table 4 (Appendix): it looks like FairGAT has smaller delta EO and should be highlighted instead. Overall, I’d suggest highlighting all results within the standard deviation of each other, as it is a bit misleading otherwise.
limitations: limitations: The authors describe as “NA” the limitations of their approach. I strongly believe that all methods have limitations and these should be discussed. 
- For instance, multiple assumptions are made, but not discussed. 
- It is also unclear how the practical implementation reflects the theoretical results, as it would be reasonable to expect effects of finite sample size, the hyper-parameters selected for each of the fairness / performance layers, …
- Are any common activation functions not Lipschitz continuous? What about ELU, or other fairness criteria? What would that mean for the derived results?

Comments: title: thank you for the detailed response
Good response to ethical review
Keep Score
Acknowledging response
comment: Thank you for your thoughtful feedback. The selection of accuracy and fairness parameters is indeed treated as a hyperparameter choice, which might appear arbitrary. To address this, we have conducted various ablation studies specifically focused on this matter.
The key point is that we need two disjoint sets of parameters for the two objectives. The assignment of network parts to these sets is a hyperparameter decision.

We appreciate your advice and will clarify the architecture discussion as you suggested.
Thank you for the detailed response. I'm afraid I still find it quite unnatural and confounding to define certain parameters "accuracy" parameters and certain parameters "fairness" parameters. I do not find Figure 3 in the one-page PDF illuminating. It seems completely arbitrary, rather. 

I would also advise that the authors clearly state that they are not modifying existing architectures. Both Reviewer UWLB (question 1) and the ethical reviewer had the impression that the proposed methodology appends new layers/parameters to existing networks.
I find the authors' response to the ethical review very clear and satisfactory.
I thank the authors' response. I think the current response does not change too much of my original impression. I will thus keep my score.
Thank you for your follow-up question and the opportunity to clarify this important aspect of our method.
Regarding your question about the primary and secondary parameter sets:
In our implementation, the parameter sets are indeed disjoint. There is no overlap between the two sets of parameters. The gradients of the two objective functions (primary and secondary) are calculated for the whole network, but the two sets of parameters are optimized separately.
We will add an explicit statement when introducing Equation (1) to specify that $\theta_p$ and $\theta_s$ are disjoint sets.
We agree that exploring our method with more complex architectures like siamese networks or SimCLR would be an interesting direction for future work. While it's beyond the scope of the current paper, we'll add this suggestion to our future work section, highlighting how the disjoint parameter structure might be adapted or reconsidered for these more complex architectures.
We appreciate your thoughtful review and suggestions. Your feedback has been invaluable in helping us improve the clarity of our work.
I thank the authors for providing a thorough response. The additional clarification of W1, Q1, and Q2's response sufficiently answers my question and will bring further clarity on the empirical components of the paper.

### Follow up W3
The clarification on W3 was also helpful alongside the figure provided in the extra pdf. It is an interesting observation that the distinct separation of parameters was shown to be superior in settings. One follow up question regarding this:

**Q:** Do the primary and secondary parameters sets need to be disjoint?

I though it was mentioned somewhere, but I couldn't find it when revisiting the paper. It would be useful to state the answer to this question when initial introducing Eq. (1).
 
This information would be useful in architecture design given the generality of the current results. Although not necessary for this submission, more complicated architectures (eg, siamese-network, SimCLR) beyond the layer position ablation in the pdf would be an interesting follow up work.

### Comment on W2

Just wanted to state that I agree with the response of the authors on this. I was merely stating it as a shared weakness of bilevel optimisation / non-convex optimization. But I don't think it should harm the acceptability of the paper.
We appreciate the thoughtful ethical review of our work. We would like to clarify several points and address the concerns raised:

- **Ethical Foundations**: Our work builds upon established literature on fairness in machine learning. These studies have established the ethical importance and validity of pursuing group fairness in AI systems. Our method contributes to this field without introducing new ethical concerns.

- **Scope and Focus**: As stated in our paper, we focus on group fairness, a well-defined and widely accepted fairness criterion. While individual fairness is important, it was intentionally out of scope. Our paper does not claim to address all aspects of fairness but contributes to group fairness optimization.

- **Model Complexity**: Our bilevel optimization approach does not increase model complexity. It provides a different optimization strategy for existing architectures. The model's parameters and structure remain unchanged; only the training process is modified.

- **Practitioner Responsibility**: Practitioners are responsible for properly configuring and testing models for their specific use cases. Our method provides additional tools for fairness optimization but does not absolve practitioners of their ethical obligations. More precise control over fairness-accuracy trade-offs enables more ethical implementations.

- **Mitigating Unintended Consequences**: While advanced techniques can potentially be misused, our method does not introduce new risks compared to existing fairness approaches. By providing more precise control over fairness constraints, it may help mitigate unintended consequences from less flexible methods.

We have also incorporated a new section into our paper in response to the reviewer's recommendations. The added content reads as follows:

## Ethical \& Broader Social Impact
This work introduces a novel bilevel optimization framework for multi-objective optimization in neural networks. While we use fairness as a case study, it's important to note that our method is not inherently a fairness research technique, but rather a general optimization approach that can be applied to various secondary objectives.
In the context of our fairness case study, our FairBiNN method shows promising results in optimizing the trade-off between demographic parity (a measure of group fairness) and accuracy. While these results are encouraging, it is crucial to consider the broader ethical implications and potential societal impacts of applying this technique to fairness problems.
On the positive side, when applied to fairness, our approach could help reduce discriminatory outcomes in high-stakes automated decision making systems, promoting more equitable treatment across protected groups in domains like hiring, lending, and healthcare[36,65]. By providing a flexible framework to manage the accuracy-fairness trade-off, practitioners can fine-tune models to meet specific fairness requirements mandated by regulations or organizational policies.
However, we must also consider potential negative consequences. There is a risk that mathematical notions of fairness like demographic parity could provide a false sense of ethical assurance, when fairness is a complex social and philosophical concept that cannot be fully captured by simple statistical measures[20,41]. Our focus on group fairness, while important, does not guarantee individual fairness[62].
The mathematical formulation we present, while rigorous, should not be seen as providing absolute ethical guarantees when applied to fairness problems. Over-reliance on our method without careful consideration of the broader context could lead to unintended harms[9].
As with any machine learning technique, it is the responsibility of practitioners to properly configure and test models for their specific use cases. Our method provides additional tools for optimization but does not absolve practitioners of their ethical obligations. We believe that providing more precise control over trade-offs actually enables more ethical implementations. However, practitioners must be aware that while our approach performs well in our experiments, particularly in balancing demographic parity and accuracy, real-world applications may present unforeseen challenges and edge cases[26, *].
Ultimately, while our work provides a useful tool for multi-objective optimization in neural networks, its application to fairness should not be seen as a complete solution to algorithmic bias. Continued interdisciplinary collaboration between computer scientists, ethicists, policymakers and impacted communities is essential to develop AI systems that are truly fair and beneficial to society[6, 59].
#### References:
*. Valerio Perrone,et al. Fair bayesian optimization. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society

All other citations are from the paper itself.
I thank the authors for their careful rebuttal and revisions of the manuscript.

While I believe that the manuscript would benefit from addressing some of the limitations which are currently mentioned as "future work", I agree that this is a novel and interesting take on fairness and the paper is sound and includes a broad array of experiments. Therefore, I have increased my score.