Paper: Fairness Metric Impossibility: Investigating and Addressing Conflicts (ICLR-2024)

Reviewer comment (Evaluation):
The paper adopts a broad framing that treats fairness metrics as complementary and applies a more or less arbitrary selection of them indiscriminately to five datasets, without sufficiently considering that some metrics, such as demographic parity and equalized odds, are fundamentally incompatible and applicable in different scenarios.

Author response:
We thank you for this incredibly valuable feedback regarding the groundedness of our work with respect to the fairness incompatibility theorem. We would like to point out that ManyFairHPO is a general framework that should be specifically applied to different fair machine learning scenarios. In other words, we do not propose that practitioners optimize for an indiscriminate set of fairness metrics, and choose a model that is relatively “fair” in terms of all of them. On the contrary, we believe that the involvement of domain experts in the discussion of which fairness metrics to optimize for is crucial, and should also be accompanied with discussion of what the risks are if one notion of fairness is satisfied by violating another relevant notion. With that being said, we acknowledge that we should be clearer about how ManyFairHPO should be used in practice, and, we have updated our methodology to include a more careful deliberation on how ManyFairHPO should be applied in practice (see Figure 1 and Section 4). We have also provided an exemplary use-case to show the efficacy of our approach in providing solutions in the face of many conflicting fairness objectives and conflict related risks (Section 6.3). We believe that our modified and extended approach clarifies that each FairML problem is categorized by a unique set of objectives and risks, and the MaO problem formulation we propose is extremely useful in such scenarios. To be clear, we do not seek to automate the FairML process, but provide a socio-technical approach that allows practitioners to make informed model selection in FairML problems that are categorized by social objectives that span across multiple conflicting notions of fairness..

Reviewer comment (Theory):
The approach of simultaneously optimizing incompatible fairness metrics is questionable; instead, the reasons for incompatibility should be understood, and stakeholders should deliberate on which fairness conception is appropriate for a given application, rather than combining all metrics in the hope of achieving general fairness.

Author response:
We thank you for this incredibly valuable feedback regarding the groundedness of our work with respect to the fairness incompatibility theorem. We would like to point out that ManyFairHPO is a general framework that should be specifically applied to different fair machine learning scenarios. In other words, we do not propose that practitioners optimize for an indiscriminate set of fairness metrics, and choose a model that is relatively “fair” in terms of all of them. On the contrary, we believe that the involvement of domain experts in the discussion of which fairness metrics to optimize for is crucial, and should also be accompanied with discussion of what the risks are if one notion of fairness is satisfied by violating another relevant notion. With that being said, we acknowledge that we should be clearer about how ManyFairHPO should be used in practice, and, we have updated our methodology to include a more careful deliberation on how ManyFairHPO should be applied in practice (see Figure 1 and Section 4). We have also provided an exemplary use-case to show the efficacy of our approach in providing solutions in the face of many conflicting fairness objectives and conflict related risks (Section 6.3). We believe that our modified and extended approach clarifies that each FairML problem is categorized by a unique set of objectives and risks, and the MaO problem formulation we propose is extremely useful in such scenarios. To be clear, we do not seek to automate the FairML process, but provide a socio-technical approach that allows practitioners to make informed model selection in FairML problems that are categorized by social objectives that span across multiple conflicting notions of fairness..

Reviewer comment (Evaluation):
The manuscript should place greater emphasis on understanding the different fairness metrics and actively deciding the appropriate ones for a given application before using fully automated many-objective optimization on all metrics simultaneously.

Author response:
We thank you for this incredibly valuable feedback regarding the groundedness of our work with respect to the fairness incompatibility theorem. We would like to point out that ManyFairHPO is a general framework that should be specifically applied to different fair machine learning scenarios. In other words, we do not propose that practitioners optimize for an indiscriminate set of fairness metrics, and choose a model that is relatively “fair” in terms of all of them. On the contrary, we believe that the involvement of domain experts in the discussion of which fairness metrics to optimize for is crucial, and should also be accompanied with discussion of what the risks are if one notion of fairness is satisfied by violating another relevant notion. With that being said, we acknowledge that we should be clearer about how ManyFairHPO should be used in practice, and, we have updated our methodology to include a more careful deliberation on how ManyFairHPO should be applied in practice (see Figure 1 and Section 4). We have also provided an exemplary use-case to show the efficacy of our approach in providing solutions in the face of many conflicting fairness objectives and conflict related risks (Section 6.3). We believe that our modified and extended approach clarifies that each FairML problem is categorized by a unique set of objectives and risks, and the MaO problem formulation we propose is extremely useful in such scenarios. To be clear, we do not seek to automate the FairML process, but provide a socio-technical approach that allows practitioners to make informed model selection in FairML problems that are categorized by social objectives that span across multiple conflicting notions of fairness..

Reviewer comment (Evaluation):
It is unclear what exactly INVD measures; specifically, what is "m" in Table 2 in the appendix, and what are the sets being summed over? This lack of clarity makes it hard to interpret results and assess potential similarities with metrics like DDSP.

Author response:
We would like to sincerely thank you for your continued engagement and thoughtful and actionable suggestions! We have incorporated your open questions (new changes highlighted in yellow) as detailed below and updated the main text/appendix as follows:

1. You make a strong point that a more theoretical discussion regarding why fairness metric conflicts occur would improve the comprehensiveness of our overall argument, motivating the complexity of the fairness metric selection problem but also exemplifying how certain dataset characteristics (e.g.. differences in base rates) can help to anticipate certain conflicts. As such, we have incorporated a discussion (Appendix A.4) of the effect of base rates in privileged and unprivileged groups in the context of the Lawschool dataset, as a means to anticipate and explain the conflict between DDSP and INVD. We believe this toy example exemplifies the effect of dataset characteristics on fairness metric conflicts, demystifying the data-dependence of our results, and could potentially lead to future discussions regarding how to use dataset characteristics to anticipate conflict-related risks.
2. 
- In response to your inquiry regarding the effect of model performance on fairness metric conflict strength, we have annotated Figure 2 with the minimum error achieved by each model on each dataset. However, we have not found a clear signal connecting relative model performance and the strength of fairness metric conflicts discovered. Although XGB often outperforms NN and has stronger conflicts, RF sometimes has stronger conflicts (COMPAS) than XGB without increased performance.
- Regarding your inquiry into the symmetry (or lack thereof) of fairness metrics, we agree that this whole must be filled in our argument. We have thus included Appendix Section A.4 explaining an asymmetry we observed on the RF-Lawschool experiment. We also related this asymmetry to the significant difference in base rates of privileged/unprivileged groups (92/8), which as mentioned also responds to your first open question. 

3. Yes we absolutely agree, the incorporation of performance into Figure 4 helps explain how fairness-accuracy trade-offs play into our model selection decision by using a color gradient on our MaO Pareto Front. Indeed, we did not select models from the bottom-left region due to poor model performance. Instead, the preference (w=0.5) towards F1-score in this exemplary use case guides us toward the top right, where models are generally more accurate, and less fair.
4. We thank you for your suggestion to supplement our bias-mitigation experiments (Appendix A.3) in order to increase the trustworthiness of our experimental results. As you suggested, we have incorporated an additional experiment that applies the Equalized Odds version of Exponentiated Gradient Reduction (EGR) and compares this technique to ManyFairHPO’s Pareto Fronts. Now for both Statistical Parity and Equalized Odds metrics (Appendix Figure 7 and 8), we observe that ManyFairHPO Pareto Fronts dominate EGR 9/12 (DDSP) and 7/12 (DEOD) times. This result provides further evidence for the promise of hyperparameter optimization for fairness, and the applicability of ManyFairHPO in real-world scenarios as an alternative to single-metric/output bias-mitigation techniques.

Reviewer comment (Novelty):
The paper does not adequately discuss the significant prior work on fairness-related trade-offs and Pareto frontiers, despite the large existing body of literature; the contributions should be clearly distinguished from previous studies and the related work more extensively covered.

Author response:
Yes, as you said! We appreciate your suggestion to clarify the contribution of our work. We will include your suggestions into our related work section. As we have emphasized in our general reply, we are the first work that treats fairness metrics as conflicting objectives, and thoroughly evaluates this methodology.

Reviewer comment (Experiments):
While the present study appears to provide a useful, practical tool with appropriate metrics for exploring trade-offs in a dataset, this contribution may be limited to small, simple cases with low-dimensional models; this limitation should be discussed explicitly.

Author response:
Yes, as you said! We appreciate your suggestion to clarify the contribution of our work. We will include your suggestions into our related work section. As we have emphasized in our general reply, we are the first work that treats fairness metrics as conflicting objectives, and thoroughly evaluates this methodology.

Reviewer comment (Novelty):
The problem of incorporating multiple fairness objectives is well recognized in the fair ML community, and the adopted method based on multi-objective optimization is also conventional. When one wishes to consider multiple fairness goals, it is a natural attempt to apply multi-objective optimization methods to include all of them. Although there is certainly value in working on the technical details and running experiments to validate performance, there is limited novelty and depth in the proposed ManyFairHPO framework.

Author response:
Despite the natural application of multi-objective optimization, we point out that optimizing for multiple notions of fairness as conflicting objectives has not been thoroughly evaluated in the literature. The closest case of this is [1], who argue that their framework is extensible to multiple fairness metrics, but do not include any experiments. In addition to our validation of the performance of this methodology, we also are working on contextualizing it in a theoretical framework that can allow practitioners to use it in real case scenarios. This represents an additional contribution to our paper.

In addition to the distinction pointed out above, we have focused on the re-framing of ManyFairHPO. In order to do so, we have reformulated our methodology (Figure 1 and Section 4) to provide a start-to-finish framework from fairness metric selection and conflict-related risk identification to Many-Objective model selection, going even deeper into this socio-technical problem than the previously mentioned work [1].

[1] https://www.amazon.science/publications/multi-objective-multi-fidelity-hyperparameter-optimization-with-application-to-fairness

Reviewer comment (Experiments):
The experiments only considered the fair ML methods available from the IBM aif360 library. To consider other fairness definitions or fair ML methods, clarification is needed on how to achieve that with the proposed framework.

Author response:
Incorporating other fairness metrics into the ManyFairHPO framework is quite straightforward from a technical perspective. We decided to use IBM aif360 due to its popularity in the fairness community. However, end-users are encouraged to incorporate any fairness metrics that fit their specific FairML problem.

Reviewer comment (Experiments):
The role of hyperparameter optimization in the ManyFairHPO framework is unclear. It is uncertain whether conventional approaches to hyperparameter tuning are sufficient, or if special techniques are required to handle the new multi-objective setup.

Author response:
Thank you for your question regarding the use of HPO in our framework. In order to go from the MO setting to the MaO setting (three or more objectives) we applied NSGA-III, an extension of the common NSGA-II algorithm that incorporates the concept of reference directions in order to encourage equal exploration of all objectives. However, we note that NSGA-III is a classic HPO method from 2014; our problem formulation thus does not require novel multi-objective optimization algorithms.

Reviewer comment (Experiments):
It is unclear whether there is practical evidence that decision makers will prefer to simultaneously optimize multiple types of fairness, rather than separately study each fairness definition and select the better one.

Author response:
We thank you for your inquiry regarding practical evidence that motivates the demand for our approach. The short answer is that practical evidence is limited due to the lack of approaches that optimize for multiple notions of fairness as conflicting objectives. However, our motivation for this methodology came from considering specific scenarios where multiple conflicting fairness notions might be of value (e.g. statistical parity vs. individual fairness in University admissions) and where conflicts between these notions might produce unwanted risks (e.g. self-fulfilling prophecy from positive discrimination). We believe that if decision makers want to mitigate these risks, they will strongly consider the prospect of optimizing for multiple notions of fairness.

Reviewer comment (Theory):
While the authors propose optimizing model fairness through the Pareto frontier and simultaneous measurement of multiple fairness indicators, they do not provide a theoretical demonstration of how trading off one fairness metric for another could lead to an overall improvement in model fairness. A deeper theoretical exploration in this area could strengthen the paper, offering clearer guidelines on how to navigate fairness trade-offs effectively.

Author response:
Thank you for your insight regarding a theoretical framework on effectively navigating fairness trade-offs. We are actively working on an intended use case which should clear up how MaO model selection works in the context of multiple fairness metrics, conflicts, and conflict-related risks. Regarding your comment regarding overall fairness, we would like to provide a clarification that trading off one fairness metric for another doesn’t lead to an overall improvement in fairness, but rather aims to mitigate the risks of fairness metric incompatibility.

Reviewer comment (Theory):
The paper lacks theoretical analysis on how to select among different Pareto-optimal outcomes, especially when one fairness metric is already at its optimal in one of the Pareto-optimal outcomes, i.e., there is no difference from a single optimization outcome. A theoretical framework or set of criteria for making these choices would be beneficial, providing practitioners with a robust method for decision-making in situations with multiple optimal fairness solutions.

Author response:
We thank you for this extremely valuable insight, and align that our MaO scenario makes model selection especially difficult to imagine. We have elected to incorporate a clarification into our central methodology, and have added a new Many-Objective Model Selection step to ManyFairHPO (Section 4.3), which details the use of objective weight scalarization to transform differing preferences towards multiple fairness metrics into a concrete model selection decision. We also provide a concrete example of our model selection framework on the Lawschool Admissions problem, exemplifying how ManyFairHPO can be used on complex real-world FairML problems with multiple fairness objectives and risks (Section 6.3).

Reviewer comment (Evaluation):
The authors only use one performance metric to evaluate the model performance. In the context of FairML, where the applications are intricate and multifaceted, relying on a single performance metric may not sufficiently capture the model’s overall performance and impact. A diverse set of performance metrics would provide a more holistic view, ensuring a balanced and thorough evaluation.

Author response:
We thank you again for your suggestion. We find it crucial (as in any ML problem) to select appropriate performance metrics that effectively capture the utility of the algorithm in its specific application domain. While we emphasize in newly updated Section 4.1 that performance metric selection is a critical decision (and many different performance metrics can be applied), we maintain that performance-metric trade-offs are not the central focus of our paper, and open the door for future study in this direction. There is also a clear connection between fairness metrics and different aspects of performance (as many fairness metrics can be derived from false positives rate (FPR), false negatives rate (FNR), etc.), and we suggest instead that the fairness-specific impact of models be determined by carefully selecting fairness metrics and thoroughly considering their trade-offs. Having said that, our improved approach now can be viewed as a general one in which many objectives (performance and fairness metrics) can be used with careful consideration by the domain experts.

Reviewer comment (Experiments):
In the experimental section, the authors have not conducted comparisons with existing fairness algorithms. Integrating benchmark comparisons against state-of-the-art fairness algorithms would significantly enhance the paper. It would offer tangible evidence of the proposed method's performance and effectively position the ManyFairHPO framework within the existing FairML research landscape.

Author response:
Thank you for your constructive feedback to include a comparison to specially designed bias-mitigation techniques. We have included a comparison of our MO Pareto Fronts to bias mitigation techniques in Appendix Figure 8, where we postprocess high-accuracy hyperparameter configurations with the SOTA in-processing method Exponentiated Gradient Reduction (EGR). We find that ManyFairHPO is competitive with EGR, finding dominating solutions in many scenarios. Although our study is not concerned with the specific performance of bias-mitigation methods (and rather the overarching problem formulation best suited to FairML problems), we believe that this result validates the performance of ManyFairHPO and motivates the importance of HPO in fairness-aware ML applications.

Reviewer comment (Theory):
In the paper, the authors propose optimizing model fairness through the Pareto frontier by measuring multiple fairness indicators simultaneously. The reviewer requests that the authors provide more theoretical insights or guidelines on how trading off one fairness metric for another could lead to an overall improvement in model fairness. Specifically, the reviewer asks how practitioners should approach situations where improving one aspect of fairness might lead to a decrease in another, and how they can ensure that these trade-offs result in a net positive impact on model fairness.

Author response:
Thank you for your insight regarding a theoretical framework on effectively navigating fairness trade-offs. We are actively working on an intended use case which should clear up how MaO model selection works in the context of multiple fairness metrics, conflicts, and conflict-related risks. Regarding your comment regarding overall fairness, we would like to provide a clarification that trading off one fairness metric for another doesn’t lead to an overall improvement in fairness, but rather aims to mitigate the risks of fairness metric incompatibility.

Reviewer comment (Novelty):
The biggest problem of the paper is the lack of novelty and technical contributions. The proposed ManyFairHPO does not show any technical improvement *adapted to fairness* compared to the prior work on multi-objective optimization or hyperparameter optimization.

Author response:
Thank you for your constructive criticism regarding the contribution of this work. We have addressed this matter in our 'general response' statement; kindly refer to it for details. We made significant changes to our methodology section, which we detail as follows: We have focused our efforts on grounding our approach to real-world applications, and providing a start-to-finish framework to guide practitioners in the use of our approach. ManyFairHPO now contains guidelines on how to select and prioritize fairness metrics and identify conflict-related risks (Section 4.1), and ultimately make model selection decisions (Section 4.3) that successfully incorporate a complex set of fairness objectives and risks. We also provided a case study (Section 6.3) for real-world applications in order to guide practitioners in the effective use of our approach.

Reviewer comment (Experiments):
The authors mentioned constrained optimization (CO) approaches in the related work. However, such a baseline is missing in the empirical evaluation. It would be better to compare CO with MaO empirically and provide more insights into the pros and cons of both approaches.

Author response:
Thank you for your comment regarding the lack of CO baselines in our work. Our line of thinking for not including such baselines was that CO and MO fairness approaches are designed for separate scenarios, and the best choice depends on whether an appropriate set of fairness metrics, meaningful constraints, and achievable values are known ad-hoc. Regardless, the pros and cons of MaO vs. CO, especially in the presence of multiple fairness objectives/constraints is an interesting research question.
We would appreciate your input and thoughts on how we can follow up with experiments in this regard to compare with CO methods (we would also appreciate references to available CO methods, as the two CO papers we cited in our related work section have no public code available).