Paper: Fairness-enhancing mixed effects deep learning improves fairness on in- and out-of-distribution clustered (non-iid) data (ICLR-2024)

Reviewer comment (Writing):
The paper is very poorly written. The structure is not well-presented, and there are many grammar errors.

Author response:
We have made substantial efforts to enhance the clarity of our paper. All of our figures have been refined, new figures 4, 5, and 6 were added. Tables 2 and 3 were updated with two debiasing heads models and new tables 6,7,8,9 were added. Many sections were rewritten for better understanding. We welcome any remaining specific suggestions to improve the paper including grammar modifications.

Reviewer comment (Novelty):
The fairness issue is addressed with domain adversarial neural networks, which is common.

Author response:
While the use of debiasing heads (an adversarial approach) to promote fairness is indeed common in neural networks, their application and suitability to a MixedEffect deep learning framework is novel, and how to do this is non-obvious. Should the FE subnetwork be made fair? Should the RE subnetwork be made fair? Should both be made fair? How to trade off these constraints?  We address these core issues in this paper. This allows us to handle both confounding from cluster effects and improve fairness. Furthermore, we also test Absolute Correlation Loss (ACL) which is distinct from the debiasing heads methods, showcasing our study’s novelty.

Reviewer comment (Theory):
The mathematical definitions are unclear. Only several losses are introduced without any detailed interpretation.

Author response:
We have reviewed our mathematical formulations and believe they are correctly presented. However, we are open to clarifying any specific equations or definitions you find ambiguous, and we will adjust our manuscript accordingly.

Reviewer comment (Theory):
It's unclear how the proposed method identifies new unseen clusters with an adversarial classifier.

Author response:
To clarify, our model uses an adversarial approach to promote fairness, specifically promoting equalized odds fairness. The cluster adversarial network aims to eliminate cluster-specific biases. The z-predictor is a non-adversarial, separately trained predictive model. It maps samples from clusters unseen during training, to a weighted combination of the most similar clusters seen during training. This allows the seen clusters to regularize the interpretation of the sample from the unseen cluster. It allows the MEDL framework to leverage the cluster specific models learned by the RandomEffects subnetwork. This mechanism is detailed in Figure 1.

Reviewer comment (Theory):
There is no justification for why the proposed method can improve equality-of-odds fairness.

Author response:
Our choice of metrics for enforcing equalized odds fairness namely the reduction in standard deviations of TPR and FPR across sites, is grounded in previous literature [1,2]. Specifically, this approach was developed to alleviate the limitations of other measures of fairness. The approach also extends readily to regression (not just classification) and it can be implemented as a neural network. These metrics have been discussed in Section 2.1, and our empirical results demonstrate a significant reduction in these standard deviations, affirming the effectiveness of our approach.

Reviewer comment (Novelty):
The paper seems to predominantly enhance the existing ARMED model by integrating two adversarial debasing components. The modification, while valuable, might not be perceived as groundbreaking, especially when viewed against the backdrop of the existing literature.

Author response:
We acknowledge your concerns about the originality of our contributions. This manuscript notably advances previous work by integrating innovative fairness-enhancing elements into the ARMED framework. Here are some key aspects of our extension: 

* The initial analysis of the ARMED model did not explicitly address fairness. Upon implementation, we discovered that while ARMED ME exhibits marginally better fairness than ARMED FE, the introduction of debiasing heads significantly enhances fairness in both outcomes. This was an unexpected finding, demonstrating the nuanced relationship between the model's components and fairness. 

* Our exploration included both Absolute Correlation Loss (ACL) and debiasing heads. Through comprehensive testing, we found debiasing heads to be consistently more effective. This discovery is pivotal, as it showcases that our framework can efficiently and simultaneously address both fairness and mitigate cluster confounding effects—a significant step forward in machine learning models. 

* Implementing this fairness enhancing MEDL framework was far from straightforward, due to the complexity of balancing multiple losses (MixedEffect, FixedEffect, various adversarial losses, and the Bayesian neural network loss). It required extensive trial and error to develop an effective method for hyperparameter optimization, ensuring stable operation of the framework. This process itself was a substantial undertaking, underscoring the novel challenges and solutions that our work presents. 

Overall, our advancements are not merely iterative but represent a substantial contribution and progression in developing more equitable and confounder-robust machine learning models.

Reviewer comment (Presentation):
Several figures and illustrations used in the paper closely resemble those from the original ARMED paper. This reuse of content, without adequate new context, can create confusion.

Author response:
Regarding figure1. We understand your point. We felt the ARMED authors figure was well done, so while we drew our own figure from scratch, we styled it after the ARMED authors. We apologize if it appeared to be a cut and paste. We have updated the manuscript to use a different style for the figure so that it is clearly shown to be our own, new figure. The new figure removed undefined terms such as "h", "𝛽", and "m".  The new figure is clearer and more easy to understand.

Reviewer comment (Writing):
Some terms, which although might be secondary in the context of this paper, are left undefined. Terms like "h", "𝛽", and "m", for instance, need clear explanations or references, even if briefly, to maintain reader continuity.

Author response:
Regarding figure1. We understand your point. We felt the ARMED authors figure was well done, so while we drew our own figure from scratch, we styled it after the ARMED authors. We apologize if it appeared to be a cut and paste. We have updated the manuscript to use a different style for the figure so that it is clearly shown to be our own, new figure. The new figure removed undefined terms such as "h", "𝛽", and "m".  The new figure is clearer and more easy to understand.

Reviewer comment (Evaluation):
The paper seems to underemphasize the importance of well-established fairness criteria. Metrics like equalized odds or demographic parity should be discussed more prominently, rather than lesser-known metrics such as TPR stdev or FPR stdev's mean and CI.

Author response:
Our choice of metrics was discussed in section 2.1; TPR and FPR std dev are conventionally used to assess equality of odds fairness in binary classification tasks [1]. By definition, equality of odds for binary classification takes the form: P(Ŷ=1 | A=0, Y=y) = P(Ŷ=1 | A=1, Y=y), y ∈ {0,1} where Ŷ is the predicted outcome, y is the ground truth outcome, and A is the sensitive subgroup membership. When y=1, this expression assesses true positive rates; likewise, y=0 is the expression of equalized false positive rates. The use of standard deviation of these rates for each sensitive subclass enabled us to recover a quantitative fairness “score” as described in a similar approach used by Yang et al. [2].

Reviewer comment (Experiments):
Although the results are analyzed thoroughly, the overall experimental setup and methodology appear to lack depth. Without a comprehensive understanding of the experiment's design and employed metrics, the derived results might seem less credible.

Author response:
To provide a more robust validation of our findings, we conducted additional experiments to provide additional baselines including the DFNN and DFNN adv. deb. These additional results appear in Tables 6 and 7. These results show that the proposed fair MEDL framework alone provides both increased fairness (e.g., over DFNN baseline) and improved accuracy (e.g., over the DFNN adv. deb.). This underscores the need for the proposed fair MEDL framework with both FixedEffects and RandomEffects (RE) subnetworks.  

Furthermore, we also added results for the ARMED model incorporating Absolute Correlation Loss (ACL) in Tables 8 and 9. These findings highlight a mixed impact on fairness enhancement and a substantial decrease in predictive accuracy, reaching up to a 6% reduction. This evidence further supports our choice of using debiasing heads, which consistently enhance fairness with only a marginal impact on predictive accuracy, generally under 1%.

Reviewer comment (Presentation):
There's an observable absence of detailed visualizations in the experimental sections, reducing the impact and clarity of results presented.

Author response:
Thank you very much for your advice; we added Figure 4, 5 and 6 to visualize the trade-off between accuracy and fairness enhancement. We observed substantial improvements in both TPR SD and FPR SD, with fairness enhancements reaching up to 67% at a minimal accuracy trade-off of about 1%. Notably, the MixedEffect (ME) models outperform their FixedEffect (FE) counterparts in both accuracy and fairness, underscoring the efficacy of MixedEffect models in balancing fairness with accuracy. We recognize that the larger standard deviations in fairness and accuracy metrics for ARMED models could be attributed to their complexity, resulting in broader convergence areas post-training.

Reviewer comment (Novelty):
Drawing direct visual and textual comparisons to the older ARMED model, without differentiating the advancements made in the current paper, might diminish its perceived value.

Author response:
Additionally, we have revised our figures to better highlight the advancements made in this paper compared to the older ARMED model. We believe these updates more effectively showcase the unique contributions and value of our current work."

Reviewer comment (Evaluation):
In section 3.1, the authors state that “while both Domain Adversarial Debiased (fair DA adv. deb.) and Domain Adversarial with absolute correlation (fair DA ACL) enhance fairness, fair DA adv. deb. exhibits a more consistent fairness improvement across all sensitive variables.” Given the results in Table 1, it is difficult to see that fair DA adv. deb. exhibits a more consistent fairness improvement. For many sensitive variables, the TPR or FPR standard deviation of fair DA ACL is smaller than fair DA adv. deb.. Especially for the Marital-status feature on occupations seen during training and Sex feature on occupations unseen during training, fair DA ACL has both TPR and FPR standard deviation smaller than fair DA adv. deb., which indicates better fairness according to the paper.

Author response:
You correctly noted the discrepancies in our comparison of fairness improvements between the fair DA adv. deb. and fair DA ACL models. We realize that our explanation might have lacked sufficient clarity. Table 5 in the Appendix shows that the drop in ACL is 1.5% in unseen occupations. We also extended ARMED with ACL and include those results in table 8 and 9 in the Appendix. These results  demonstrate the impact of implementing Absolute Correlation Loss (ACL) in the ARMED framework as an alternative to the debiasing heads. While ARMED ACL shows variability in fairness improvement across different sensitive features, it also leads to a significant decrease in prediction accuracy, reaching up to a 6% reduction. Considering these outcomes, we assert that adding debiasing heads, with its more uniform improvement in fairness and negligible impact on accuracy, is a preferable option compared to ACL, at least across all 3 datasets we evaluated in this work.

Reviewer comment (Evaluation):
Authors state that “Moreover, fair DA adv. deb. enhances fairness with minimal reduction in balanced accuracy compared with fair DA ACL—1% vs 1.6%. Given these findings, we chose to incorporate fair DA adv. deb. into the ARMED framework”. The balanced accuracy between fair DA ACL and fair DA adv. deb. seems to be quite small and does not give a convincing reason for choosing fair DA adv. deb. over fair DA ACL.

Author response:
You correctly noted the discrepancies in our comparison of fairness improvements between the fair DA adv. deb. and fair DA ACL models. We realize that our explanation might have lacked sufficient clarity. Table 5 in the Appendix shows that the drop in ACL is 1.5% in unseen occupations. We also extended ARMED with ACL and include those results in table 8 and 9 in the Appendix. These results  demonstrate the impact of implementing Absolute Correlation Loss (ACL) in the ARMED framework as an alternative to the debiasing heads. While ARMED ACL shows variability in fairness improvement across different sensitive features, it also leads to a significant decrease in prediction accuracy, reaching up to a 6% reduction. Considering these outcomes, we assert that adding debiasing heads, with its more uniform improvement in fairness and negligible impact on accuracy, is a preferable option compared to ACL, at least across all 3 datasets we evaluated in this work.

Reviewer comment (Novelty):
I would like to congratulate the authors on publishing their paper on TPAMI, however, ICLR is a decent venue for machine learning too. Advertising ARMED (Sec 2.2) can hardly be part of "Methods" as it is not an innovation/contribution in this paper. Note that unlike many conferences in signal processing, conference papers in top ML venues are not shortened versions of journal papers. You need to be substantially different and innovative from earlier works. Besides, the purpose of double blind is to remove the selection bias in favor of big names. As a reviewer for ICLR, knowing you have a TPAMI paper will not really affect my rating.

Author response:
We understand your concern regarding the similarity of the figure we drew from scratch to that in the TPAMI paper. We have redrawn this figure to make it clear that this work is independent of any prior work. Additionally, this current manuscript is a significant contribution by investigating how fairness-enhancing components could be added to a MEDL framework, and many of these results will apply regardless of the specific implementation of MEDL (ARMED or domain adversarial) as we show in this manuscript. Below we list our specific contributions: 

It was non-obvious that MEDL frameworks, such as but not limited to ARMED, are unfair. They (e.g. ARMED, and domain adversarial) achieve many benefits, it could be the case that they are already fair. This work demonstrates that despite their other benefits in correcting bias due to cluster confounding, MEDL frameworks have unfair. We document and quantify the base level of unfairness. Additionally, we discover that the ME predictions are somewhat more fair than the FE but that adding the debiasing heads greatly improves fairness of both types of predictions. None of these findings were obvious outcomes, that this manuscript produces.  

We tried both ACL and the debiasing heads methodologies and found that the debiasing heads consistently work better. Our works demonstrate that fairness and batch effect mitigation can both be handled simultaneously and efficiently. This is a fundamental achievement that many machine learning researchers and projects could benefit by. 

We faced many challenges implementing the framework; i.e. its implementation is non-trivial. As an example, consider that there are multiple loss terms (MixedEffect fidelity, FixedEffect fidelity, three different adversarial losses, the Bayesian neural network KL-divergence loss). We document how these hyperparameters can be optimized to improve fairness while mitigating the confounding from sample clustering.

Reviewer comment (Experiments):
There are too few baselines to conclude that the experiments are comprehensive enough to demonstrate its fairness.

Author response:
We appreciate your suggestion to incorporate absolute correlation loss as used in fair DA ACL into the full fairness-enhancing ARMED framework for a more comprehensive comparison. These results are in table 8 and 9 in Appendix.  

We also implemented Yang’s method as additional baseline. These results are in table 6 and 7. The results are similar to the FixedEffects subnetwork, with strengthens our belief that making a full MEDL framework fair (with its added RandomEffects subnetwork) is preferable as it improve the model predictive accuracy and handles the confounding simultaneously, compared to Yang’s approach.

Reviewer comment (Experiments):
Authors can try to use the absolute correlation loss mentioned in fair DA ACL for the full fairness-enhancing ARMED framework, then make a comparison with the original proposed full fairness-enhancing ARMED.

Author response:
We appreciate your suggestion to incorporate absolute correlation loss as used in fair DA ACL into the full fairness-enhancing ARMED framework for a more comprehensive comparison. These results are in table 8 and 9 in Appendix.  

We also implemented Yang’s method as additional baseline. These results are in table 6 and 7. The results are similar to the FixedEffects subnetwork, with strengthens our belief that making a full MEDL framework fair (with its added RandomEffects subnetwork) is preferable as it improve the model predictive accuracy and handles the confounding simultaneously, compared to Yang’s approach.

Reviewer comment (Novelty):
**Originality**: this feels like a minor modification of the ARMED framework, especially as the $L_{CCE}(S,S')$ is the same as the loss on Z.

Author response:
While we concur that our work builds upon the ARMED framework,  we respectfully disagree that it is “a minor modification of the ARMED framework”. We have verified that MEDL does NOT intrinsically address fairness. We have quantified that inherent unfairness. We assert that the incorporation on fairness regularizers is a considerable advancement rather than a minor extension. Our exploration of both absolute correlation loss (ACL) and adversarial debiasing heads revealed that debiasing heads are consistently more effective at achieving fairness in a mixed effects context. This finding is a noteworthy discovery and is not an obvious result. The contribution we make and underscores our framework's ability to simultaneously and efficiently address fairness and batch effects. Notably, our approach integrates two debiasing heads, targeting the outputs of both FixedEffect and MixedEffect components. The implementation complexity, involving multiple losses (MixedEffect, FixedEffect, three adversarial losses, and the Bayesian neural network loss), necessitated extensive experimentation and refinement in hyperparameter optimization to achieve stable operation of our enhanced framework.

Reviewer comment (Experiments):
**Quality**: I believe important baselines are missing, as well as a proper discussion. For instance, the `fairness under distribution shift' is an important related field that is not cited here. Baselines from this field could be implemented, including some that include adversarial losses [1].

Author response:
In response to your concern regarding missing baselines, we have included results using Yang’s method in table 6 and 7. The results show that it performs similarly to our FixedEffect subnetwork's outcomes. This similarity bolsters our argument that the addition of the RandomEffect subnetwork marks a substantial improvement. By effectively managing batch effects, our model demonstrates enhanced capacity for fairness enhancement, as indicated in our revised results.  

We also added ARMED ACL as another baseline in table 8 and 9. These baselines show a mix in fairness enhancement, but the drop in predictive accuracy is much bigger – up to 6%. Based on these findings, we are confident that adding debiasing adversary networks is a better way to enhance fairness than using ACL, in the context of mixed effects deep learning.

Reviewer comment (Experiments):
From my understanding, there isn't any baseline implemented outside of ARMED, even though Yang et al., 2023 is referenced.

Author response:
In response to your concern regarding missing baselines, we have included results using Yang’s method in table 6 and 7. The results show that it performs similarly to our FixedEffect subnetwork's outcomes. This similarity bolsters our argument that the addition of the RandomEffect subnetwork marks a substantial improvement. By effectively managing batch effects, our model demonstrates enhanced capacity for fairness enhancement, as indicated in our revised results.

Reviewer comment (Theory):
In terms of the motivation of the method, I have major concerns: the fairness loss is on the FE part of the network. What prevents the RE part of the network from inducing bias? This is actually suggested by the better fairness results from the FE network compared to its ME counterpart.

Author response:
We value your insight regarding the fairness loss being applied solely to the FE part of the network. To address potential bias in the RE component, we conducted additional experiments with dual debiasing heads: one for the FE and one for the RE portion of the mixed effects deep learning (MEDL) framework. The outcomes, presented in updated Tables 2 and 3, exhibit consistent fairness enhancement across both FE and ME outputs. Notably, ME displays superior fairness in seen occupations, both in baseline and fair implementations. However, in unseen occupations, we observed some variability in fairness improvement, suggesting the need for enhancements in our current z-predictor. This aspect is a key focus for our future research.

Reviewer comment (Theory):
The authors mention that they enforce ‘equality of odds’, but they actually enforce that the model is not able to ‘encode’ the sensitive attribute. These are different criteria, and it is possible for models to encode a signal at the same level but display very different equalized odds [2].

Author response:
Regarding the enforcement of equalized odds, we respectfully disagree with the statement that what we "enforce is that the model is not able to ‘encode’ the sensitive attribute.”  The cluster adversary for the FE subnetwork encourages that the conventional model does not encode cluster specific attributes. However, the debiasing heads, are structured quite differently from the cluster adversary. These debiasing heads only take the estimated prediction, y_hat and the ground truth prediction y as input and penalize the corresponding MEDL subnetwork for making errors in a biased way (e.g., more errors for females than males, or for one race versus another). The debiasing heads do not take the intermediate layers or features into consideration, only the errors made in prediction. 

Also, we deliberately chose equalized odds as the fairness approach, over other approaches such as demographic parity because 1) equalized odds was developed to overcome limitations in earlier methods such as demographic parity. 2) Equalized odds is applicable to both classification and regression tasks, 3) Equalized odds can be implemented readily as a neural network, facilitating integration to other deep learning frameworks, such as MEDL. 4) Equalized odds can effectively handle sensitive features encompassing more than two categories. These advantages align well with our framework, which utilizes neural networks for fairness improvements in both classification and regression, often dealing with multi-category sensitive features. Furthermore, our empirical results support the effectiveness of this approach in enhancing fairness across the datasets we examined.

Reviewer comment (Evaluation):
**Significance**: the results seem quite variable (looking at the 95% CI), with obvious overlaps between multiple methods especially when considering unseen clusters. Can the authors discuss the additional complexity of ARMED compared to its variance, and benefit?

Author response:
**Variability of Results in unseen clusters.**
We recognize the variability in our results, particularly with unseen clusters. This variability may stem from the current limitations of our z-predictor, especially when faced with complex site invariance. We are actively working on enhancing our z-predictor by refining hyperparameter optimization and increasing its complexity to better handle such scenarios. These improvements are slated for future development and have been duly noted in the Discussion section of our manuscript.

Reviewer comment (Experiments):
**Scalability**: the authors investigate different numbers of samples, but the number of features remain small (max 19). Can the authors comment on the implications of e.g. using images?

Author response:
**Scalability and Optimization.**
The complexity of the debiasing heads (AF and AM) is not a function of the number of features. Their complexity is fixed with respect to the number of features. Thus our approach does not increase in complexity with the number of features and should scale well with increasing feature count.  

What does increase in complexity with the number of features is the cluster adversary, Ac, and the conventional model. However, in the original MEDL framework that we chose to use here, (ARMED), those authors showed that their model does scale to images and video data.

Reviewer comment (Experiments):
**Optimization**: the loss term includes multiple adversaries, mixes of loss types (e.g. cross-entropy with MSE), each with their own parameter. This seems like a difficult function to optimize, as even one term with an adversary can be challenging to converge. Can the authors comment on this?

Author response:
On the optimization front, we value your recognition of the inherent challenges. Achieving stable operation of our framework involved extensive experimentation. We ultimately employed BOHB [1] for simultaneous optimization of all weight_loss parameters associated with different loss terms. Additionally, we leveraged the Ray [2] framework for efficient parallel simulations. For instance, generating results for Table 2, where approximately 14,000 models were trained in parallel on 16 NVIDIA V100 GPU cards, was accomplished in just a few hours, demonstrating the framework's efficacy and efficiency.   

[1] Stefan Falkner, Aaron Klein, and Frank Hutter. BOHB: Robust and efficient hyperparameter optimization at scale. In International Conference on Machine Learning, pp. 1437–1446. Proceedings of Machine Learning Research, 2018.  

[2] Moritz, Philipp, et al. "Ray: A distributed framework for emerging {AI} applications." 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18). 2018.