FairTune: Optimizing Parameter Efficient Fine Tuning for Fairness in Medical Image Analysis
Authors: Raman Dutt
Ondrej Bohdal
Sotirios A. Tsaftaris
Timothy Hospedales
Venue: ICLR 2024 poster (2024)
Keywords: Fairness
PEFT
Hyperparameter Optimization
Medical Imaging
Primary area: societal considerations including fairness, safety, privacy
TL;DR: FairTune: A framework to optimize Parameter-Efficient Fine-Tuning (PEFT) for fairness in medical image analysis.

Abstract: Training models with robust group fairness properties is crucial in ethically sensitive application areas such as medical diagnosis. Despite the growing body of work aiming to minimise demographic bias in AI, this problem remains challenging. A key reason for this challenge is the fairness generalisation gap: High-capacity deep learning models can fit all training data nearly perfectly, and thus also exhibit perfect fairness during training. In this case, bias emerges only during testing when generalisation performance differs across sub-groups. This motivates us to take a bi-level optimisation perspective on fair learning: Optimising the learning strategy based on validation fairness. Specifically, we consider the highly effective workflow of adapting pre-trained models to downstream medical imaging tasks using parameter-efficient fine-tuning (PEFT) techniques. There is a trade-off between updating more parameters, enabling a better fit to the task of interest vs. fewer parameters, potentially reducing the generalisation gap. To manage this tradeoff, we propose FairTune, a framework to optimise the choice of PEFT parameters with respect to fairness. We demonstrate empirically that FairTune leads to improved fairness on a range of medical imaging datasets. The code is available at https://github.com/Raman1121/FairTune.

Published as a conference paper at ICLR 2024
FAIRTUNE: OPTIMIZING PARAMETER EFFICIENT FINE TUNING FOR FAIRNESS IN MEDICAL IMAGE ANALYSIS
Raman Dutt', Ondrej Bohdal', Sotirios A. Tsaftaris', Timothy Hospedales
'The University of Edinburgh "Samsung AI Center, Cambridge {raman. dutt, ondrej wohdal, s tsaftariss, hospedalesjjedd
ABSTRACT
Training models with robust group fairness properties is crucial in ethically sen- sitive application areas such as medical diagnosis. Despite the growing body of work aiming to minimise demographic bias in AI, this problem remains challeng- ing. A key reason for this challenge is the fairness generalisation gap: High- capacity deep learning models can fit all training data nearly perfectly, and thus also exhibit perfect fairness during training. In this case, bias emerges only dur- ing testing when generalisation performance differs across subgroups. This mo- tivates us to take a bi-level optimisation perspective on fair learning: Optimising the learning strategy based on validation fairness. Specifically, we consider the highly effective workflow of adapting pre-trained models to downstream medical imaging tasks using parameter-efficiees a trade-off between updating more parameters, enabling better fit to the task of interest vs. fewer parameters, potentially reducing the generalisation gap. To manage this tradeoff, we propose FairTune, framework to optimise the choice PEFT parameters with respect to fairness. We demonstrate empirically that FairTune leads to improved fairness range of medical imaging datasets. The code is available at
fine-tuning (PEFT) techniques. There
com/ 12t///aaa111111/11a11
INTRODUCTION
The use of Al in healthcare applications is growing rapidly. Powerful new models enabled by large
datasets (Mei et al., 2022, Ghesu et al.) 2022] Irvin et 2019) are rapidly being developed, lead- ing to highly performant automated diagnosis systems (Tuu et al., 2022) that are increasingly being deployed clinically in clinical practice (Esteva et al. 2021 Dutt et al. 2022 Vats et al.) 2022) However, AI models have repeatedly been shown to exhibit unwanted biases towards various demo- graphic subgroups (Seyyed-Kalantar) et al., 202 Obermeyer et al.) 2019 Larrazabal et al.) 2020 Ricci Lara et al., 2022) for example by providing substantially worse performance on disadvan- taged subgroups defined by protected attributes such gender, race, age, and socioeconomic status. This is obviously socially, ethically, and clinically problematic, especially in potentially life-and- death situations that arise in healthcare.
The issue of biased and inequitable AI systems has prompted a growing body of research striving to analyze the origins of bias and develop interventions mitigate model bias (Xu et al.) (023). Never- theless, recent investigations cast doubt on the extent of progress achieved thus far. Notably, Zietlow al. (2022) postulate that the majority of existing interventions aimed at promoting fairness prove ineffective when applied to deep models, which are commonly utilized for tasks involving images and text data. The reason behind this ineffectiveness lies in the nature of these interventions, such as those proposed by Saaawa et al. (2020) and Zhao et al. (2019), which impose constraints on the training data. For instance, they enforce equal performance across subgroups (Zhao et al., 2019). However, while such constraints can impact the training of shallow models typically employed for tabular data, deep models possess the capability to perfectly fit all training data, rendering these fair- ness constraints automatically satisfied and devoid of any influence the model's learning process. We substantiate this well-documented challenge empirically in Figure [ which illustrates that, in typical medical image analysis scenario, the training data be fitted flawlessly. Consequently, the model is already intrinsically equitable within the training ser. The observed bias in real-world applications emerges during testing, primarily due to differential generalization across subgroups.

---

Published as a conference paper at ICLR 2024
1.0
0.8
AUC
0.6
0.6
0.4
Train AUC
Val AUC
0.2
Train Best AUC
Val Best AUC
Train Worst AUC
Val Worst AUC
0.0
100
20
60 Epoch
100
Epoch
Figure/ 1: Bias arises during train-test generalisation. Left (Training AUROC): High-capacity deep models can exhibit perfect group fairness during training because they can classify all the training data perfectly. Right (Validation AUROC): Bias arises because the disadvantaged subgroup has worse generalisation error than the privileged subgroup. Fine-tuning ViT-Base on the Papila dataset.
Another recent study (Zong et al., 2023) empirically evaluated wide range of fairness interven- tions designed to regularise deep model learning a large suite of medical image analysis tasks. However, they found that prior progress was over-estimated. When subjected to a standardized hyperparameter tuning procedure for fair evaluation, none of the existing fairness interventions exhibited a statistically significant enhancement in fair learning when compared to the conventional approach of supervised learning by empirical risk minimization (ERM).
In this research paper. we introduce a novel approach to fair learning that addresses the challenge highlighted by [Ziellow et al. (002) and depicted in Figure Our method is rooted in the concept of capacity control, and involves introducing a form regularization during the learning process specifically tailored minimize bias in unseen data. To accomplish this, operate within the pre-train/fine-tun framework (Mei et al. 2022 Yosinski et al.] 2014 Tang et al. [022] Zong et al.) 2023). This framework entails initializing models through pre-training extensive external datasets like ImageNet (Deng et al. 2200)., followed by fine-tuning on comparatively smaller medical imag- ing datasets. In this context, as we progressively update the model from its initial pre- pre-trained state, the risk of overfitting to the nuances of the training set increases, leading to the generalization gap illustrated in Figure Hence, the primary challenge lies in restraining the extent of model updates. In this regard, we will illustrate that employing parameter-efficient fine-tuning techniques, which involve the selective updating of a subset of network parameters (Dutt al.) 2023), can result in more equitable generalization. However, this approach poses a critical question: "Which parame- ters should be updated to maximize fairness?" To tackle this question, we introduce our framework named FairTune, designed to search for the optimal parameter update mask. We seek the mask that, when applied to constrain the fine-tuning process, yields a high degree of faimess the validation data. Our empirical findings consistently demonstrate that FairTune outperforms Empirical Risk Minimization (ERM) in terms of fairness across various medical imaging benchmarks.
To summarise contributions: (1) We directly corroborate the conjecture of Zietlow et al. (2022) that bias arises during train-test generalisation (Figure []. (2) In contrast to existing fairness in- terventions, we introduce new fair learning approach that regularises learning so as to optimise validation fairness (cf: existing methods that ineffectively target training fairness). (3) Our empir- ical findings across diverse set of benchmarks consistently demonstrate that FairTune reliably improves performance over ERM.
2 RELATED WORK
2.1 FAIRNESS IN MEDICINE
Bias and unfairness have been widely reported in biomedical AI (Seyyed-Kalantari Kalantari et al., 2021 Ricci Lara et al.) 2022 Obermeyer al., 2019). Biases can arise from complex array of differ- ent underlying causes including dataset imbalance, label noise, and reliance on underlying spurious correlations. A particularly problematic manifestation is that of bias amplification (Lloyd 2018 Hall et al., 2022), where biases that exist in the training set are amplified by the model's predictions during deployment. Measuring fairness is itself complex problem, many different fairness met-
2

---

Published as a conference paper at ICLR 2024
rics have been proposed. with consensus on single preferred metric. For example, optimising for equal performance among demographic subgroups (Dwork et al. 2012] Verma & Rubin 2018)
is intuitive. But this can lead to the levelling down phenomenon (Zietlow et al. 2022), where fair- ness is achieved by decreasing the performance of the advantaged group to match the disadvantaged group potentially even including pathological solutions of reducing both groups' performance to zero. Achieving fairness by levelling down has been criticised as violating the ethical principles of beneficence and non-maleficence [Beauchamp] 2003 Chen et al. al.] 2018 Ustun al. 2201)) We also remark that evaluating systems for fairness is itself complex (Zong et al.) [223]] Verma & Rubin 2018) as fair learning inevitably a multi nulti-obbecctive problem that seeks to simultaneously achieve
potentially conflicting goals of good overall performance and good fairness.
2.2 PREVIOUS ATTEMPTS TO SOLVE FAIRNESS
Fair machine learning has now been widely studied, with numerous methods being proposed that address bias reduction via both pre- prepprocessing (e.g., data re- balanciing and post-processing, as well as interventions aimed at guiding the learning algorithm to generate a fairer predictor. Due to the large volume of the proposed methods in the literature, we refer the readers to comprehensive sur-
veys (Mehrabi et al.) 202T Caton & Haas] 2023; Zong et al. 2023) for more in-depth exploration of the available techniques and their nuances. A crucial observation, however, is that a large family of methods (Sagawa al.) 22200 Zhao et al.| |019)] Agarwal al. [2018b; Beutel et al. 2017]] Diana
et al.
2021
Jeong et al.
et al. 2016 Yii et al. 2022, Donini et al., 2018: Dumoulin Lohaus et al.
ct al.
2016 Kim al.
2019] 2 Kleindessner et al.]
2020:
Martinez et al. 2020)
Padala & Gujar 2020 Wang et al. 2020] Zafar et al. 2017 Wu et al.
2022 Park et al. 2022 rely
on imposing fairness constraints on the training set. As suggested by Zietlow et al. (2022), these
are ineffective in the deep learning regime where constraints are trivially satisfied by a classifier that
achieves 100% training accuracy (Figure [.. Another family of methods endeavours to introduce various forms of regularization during model training, aiming enhance generalization, such as achieving domain independence. While some of these studies initially reported promising outcomes, recent exhaustive benchmarking study (Zong et al.) (2023) has indicated that these assertions were premature. When evaluated across multiple benchmarks, existing methods consistently fall short of systematically outperforming a well-tuned supervised learning baseline for fairness (ERM).
We are inspired by studies such as Zietlow et al. (2022); Zong et al. (2023) to design an algorithm that tunes how to regularise learning with the explicit objective of optimising for validation fairness.
2.3 -ARAMETER-EFFCIIEN FINE- TUNING
Fine-tuning models that have been pre-trained on large datasets is common practice in deep learning (Yosinski et al. 2014 Kornblith et al. 2019). Leveraging a pre-trained initialization enables down- stream tasks to be learned with significantly less data compared to training from scratch. Parameter- Efficient Tuning (PEFT) methods are family of techniques geared towards improving the fine-tuning process. They achieve this by carefully selecting a small subset of parameters for up- dating during fine-tuning while keeping the majority frozen. The underlying concept is that this judicious choice of selective updates should facilitate effective adaptation to the target task (via the minority of updatable parameters) while guarding against overfitting (courtesy of the majority of frozen parameters). A growing number of PEFT methodologies have emerged, each distinguish- ing itself by its specific selection of parameters for updating. These selections may include biases (Ben Zaken et al.) (002), attention matrices (Touvron et al. 2022), or normalization layers (Basu al. (023). Alternatively, some methods introduce and learn specific sets of new parameters, such as low-rank adapters (Hu et al. 2022). all while maintaining the entire pre-trained backbone in frozen state. PEFT techniques have gained wide popularity
mainstream NLP and computer vi- sion applications, although their adoption in medical image analysis tasks remains nascent (Dutt ctt al. 2023 Ma & Wang 2023 Wu et al., 2023 Zhang & Liu 2023).
In this work, we aim to demonstrate that PEFT (Parameter-Efficien Fine-Tuning) offers benefits beyond enhancing traditional generalization capabilities. Specifically, our findings will illustrate that PEFT can enhance faimess by narrowing the generalization gap, especially for disadvantaged subgroups, as depicted in Figure [] Nevertheless, a central challenge persists across all existing PEFT methods, namely, they rely heuristic approaches for partitioning parameters into frozen
3

---

Published as a conference paper at ICLR 2024
and updatable sets. Current methods do not offer principled or learned method for establishing the optimal partition. This becomes particularly crucial, because the ideal PEFT assumption, i.c., the freeze/update partition, may be dataset dependent. For instance, larger datasets might accommodate a more extensive parameter update without suffering from overfitting compared to smaller datasets. The key novelty of this paper lies in approach: instead of prescribing specific PEFT update mask, we introduce a framework designed to autonomously determine the optimal PEFT mask that maximises validation fairness.
METHODOLOGY
3.1 FAIRNESS METRICS
We focus on evaluating the fairness of binary classification of medical images. Given an image we predict its diagnosis label y in a way that aims be independent of any sensitive attribute 8 (age, sex, ethnicity, etc.) so that the trained model is fair and does not unduly disadvantage any particular demographic subgroup. There are plethora of metrics to measure fairness such as equality of opportunity, equal odds, subgroup performance difference, and so on (Verma & Rubin, 2018). Each of these may be more appropriate for different social and economic situations. Our overall framework is agnostic to the choice of fairness metric used, as our contribution is an approach to optimise for any user-specified fairness metric. However, for most of our experiments, we will optimise the metric of moot-disadvvantaeel group performance (Sagawa al. 2020), In this setting we are given loss function (D; 0) (e.g., cross-entropy, or 1 area under ROC curve) for model on dataset D. We assume it can be evaluated for different subgroups of the dataset DD as L(D,; 0). Then the metric for fair learning is
faar
max ((D,;)). sES
We will also report other metrics such as the fairness gap, estimated as the performance difference between the disadvantaged and privileged subgroups, (max, L(D,; min, L(D,;
3.2 PARAMETER-EFFICIEN FINE- TINE-TUNIG
In PEFT, fine-tune only subset of parameters C such that [Ø] < 18]. PEFT strategies be interpreted as specifying sparse binary mask w that determines what parts of should be updated. Given parameters Oo of the pre-trained model and a change Aó to be applied to their values, the fine-tuning process can be described as
AΦ" = arg min basse (Dtrain, Gg + w O Ad).
Ao
where Lbase is standard deep learning loss such cross-entropy for classification.
Different PEFT methods essentially correspond to different structures on the sparsity structure of the binary mask W. For example, BiuFit (Ben Zaken al. (022)) solely updates bias parameters
in a neural network. Attention Tuning (Touvron et al.) 2022) enables updating all the attention matrices in transformer, and on. These methods are generally effective in reducing overfitting when leamingg large models on small datasets thanks to eliminating most parameter updates. We will show that they are also effective in improving generalisation fairness compared to conventional fine-tuning.
There are two key outstanding challenges, however: (1) The optimal PEFT strategy (binary mask w) is dataset-depender For example, a sparser mask w may be preferred for a smaller target task with greater risk of overfitting, and a denser mask may be preferred for a task that is more different to the pre-training task and thus requires stronger adaptation. (2) The optimal PEFT strategy may depend on the ultimate generalisation objective. For example, a sparser mask w might be preferred for fair generalisation compared to conventional overall generalisation. We present solution to both of these issues by introducing an algorithm to optimise the mask w with respect to a fair generalisation objective.
4

---

Published as a conference paper at ICLR 2024
HPO sampling
[1,0,0,..,,,0]],,,
[1,0,0,..,,,0,,,,
Report result to HPO
[1,1,0....111]]]
Mask w = [1,0,0,..,,10]
D
[
f
PEFT using D train
L
Pre-trained 0
Fine-tuned + dep
Sensitive attrributes: gender, age
Figure 2: Illustration that shows how approach optimises the structure of PEFT with respect to fairess.. Hyperparameter optimisation (HPO) selects a mask that decides which components of a pre-trained model 0 are fine-tuned using PEFT. For each sampled mask, the fine-tuned model is evaluated on the validation set to compute the fairness loss L Lreir, which is then reported to the HPO algorithm that decides what masks to sample and which is the final best option.
3.3 OPTIMISING PEFT FOR FAIRNESS
We begin with a pre-trained model f.. dataset (D) split into training, validation and test sets (Direin Deal DDeet), Each dataset D = (X, y. S) contains a set of images X, labels Y and sensi- tive attribute metadata S. We also define search space for PEFT masks w E Q. The goal is to find that leads to the best fair generalisation (Sec 3.1 when conducting PEFT learning (See 3.2).
Bi-level Optimization (BLO): We formalize our problem statement as a bi-level optimization problem consisting of inner and an outer loop. In the inner loop, we fine-tune the pre-trained model on the medical dataset (Dtrain) using a conventional loss L Claase and PEFT mask w. In the outer loop, search for the PEFT mask which leads the inner loop to produce the fairest outcome on the validation set (Dval), ), as measured by Llar More formally, we solve Equation
w* argmin (Dral, (Daaal AΦ"))
such that Ad" arg min bbase (Derain, Og + о Ao).
A6
There are a number of possible strategies for solving BLO problems such as Equation 2 includ- ing meta-gras gradient, evolutionary search, Bayesian Optimisation and others (Hospedales et al. 2021 Sinha et al. 2018, [iu et al. (02T). practice, we adopt hybrid approach with a gradient- free Tree-structured Parzen Estimator (TPE) (Bergstra et al. [011] with successive halving (SH) strategy (Jamieson Talwalkar, 2016) for optimising w" in the outer loop (Akiba et al. 2019). and conven- tional long-horizn gradient-descen fine-tuning in the inner loop. We illustrate the process in Figure and provide full details Algorithm [
Additional details on the HPO are given in Appendix A.5
Besides the selective-update mask the learning rate also provides coarse cue of how much to update. For example a suitably curtailed learning rate would prevent the most egregarious overfitting shown in Figure [
We also optimise along with within the same HPO process of Algorithm
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Architectures: Our experiments adopted the Vision Transformer (ViT) implementation present in the Pytorch Image Models package (Wightman, 2019). A VIT consists of several blocks and each block contains two normalization layers (LNI and LN2), Multi- Self-Attention (MHSA) sub- block and MLP (MLP) sub-components. The normalization layers (LayerNormalizztion Ba et al.
5

---

Published as a conference paper at ICLR 2024
Algorithm Optimizing PEFT for fairness
1: Input: pre-trained model Co. fine-tuning learning rate, number of trials Tn
2: Output: fine-tuned model 0g + w @ Ae and mask w
3: while number of completed trials < Ty do
4: Initialize A = 0 for 6 +- $ C 0o
5: Propose mask w НР
6: while not converged do
7: A AΦ aV a (Dtroin, 0a AΦ)
// PEFT
8: end while
9: Evaluate Lfair (Dval, 0o AΦ) and report the HPO algorithm
10: end while
Return the best (most fair) mask w and fine-tuned model du w O AΦ
2016 are present before and after MHSA. The MLP consists of two fully-connected layers with GELU non-linearity in between. The base variant ViT consists of 12 such blocks.
Baselines: We compare our approach with: (1) full training from scratch. (2) conventional full fine- tuning, as conducted in (Zong et al. 2023) (where it is referred to as ERM) where every layer of an ImageNet pre- pre-trained model adapted the medical image task. (3) linear readout, where the Ima- geNet pre- pre-trained feature extractor is frozen and only the classification head is Iearned, as conducted in (Azizi et al. 2021: Chen al. 2020), (4) PEFT method Attention Tuning (Touvroo et al. 2022) where only attention matrices are fine-tuned, (5) PEFT method Layer Norm Tuning (Basu et al, 2023) where only layer-norm parameters are fine-tuned, (6) FairPrune (Wu et al., 2022), a method that achieves fairness by pruning the model parameters post-training, and (7) Fair Supervised Con- trastive Loss (FSCL) (Park al. 2022) that inherits the properties of supervised contrastive learning and penalizes the usage of sensitive attribute information in representation for improving fairness.
We remark that the thorough benchmark in (Zong al.) 2023) already dismissed suite of algo- rithms designed for the purpose of fair learning as equal worse than SLMM/FineTTuiingg (Vapnik) 1999), including DomainInd Wang et al. (2020), LAFTR Madras et al. (2018), CFair /Zhao et al.
(2019) LNL Kim et al.] (2019), EnD Tartaglino et al. (2221) ODR |aarhan et al.) (2020), Group-
DRO Sagawa et al. (2020), SWaD Cha et al. (2021) )), and SAM Foret et al. (2021).
Search Space: We define PEFT search space that consists of the choice to fine-tune or freeze each module within a 12-layer VIT, where each VIT layer consists of MHSA, MLP, and LN modules.
Thus our main PEFT search space 12 is the space of 36-bit binary marks f2 E {0, 1].3, 1336 This search space contains Attention Tuning (Touvron et al.) 2022). Layer Norm Tuning (Basu et al. 2023), linear readout, and full-fine tuning (Zong et al. 2023) as special cases. As ablations, we also consider 12-bit search spaces that solely search for the combination of layer-norm and attention layers to tune,
as sparser alternatives (Touvron al., 2022 Basu et al. 2023).
Our experiments include seven frequently adopted medical image analysis datasets. The selection of datasets was based five integral factors: a) the presence of sensitive attributes, b) the presence of different potential sources of bias, c) the representation of different anatomical regions (domains), d) varying size, and e) public availability for reproducibility. Following these factors, we included Fitzpatrick17K (Groh et al. 2021; 2022), HAM10000 [Tschand]] 2018). Papila (Kovalyk) al. 2022). OL31 (Zambrano Chaves al. 2021 OASIS-1 (Mareus et al. 2007). 3Fr300 (Luo et al. 2023). and CheXpert Irvin et al. (2019). Following the settings in Zong et al.) (2023),
Datasets:
the preprocessing steps for all datasets included the binarization of the sensitive attributes (skin type, age, and sex) and the classification label along with the removal of studies with missing information. More details on data preprocessing are presented in Appendix A.4
MatageNet-pre-triing ViT-Base model for 30 epochs with a linear warmup for 10 epochs and cosine annealing learning rate schedular [oohchilovv & Hutter 2016). The batch size is set to 512 and the optimizer used is AdamW Loshchilov & Hutter 2018). We consider one sensitive attribute at a time and note the overall performance, the worst subgroup performance and the gap between the best and worst subgroup.
Experimental Settings: All experiments fine-tune
PEFT Mask Search and Hyperparameter Optimisation: For PEFT mask search, we rely on "rre-ssrructured Parzen Estimator (TPH) (Bergstra al. 2011) for sampling hyperparameter values.

---

Published as a conference paper at ICLR 2024
We also employ a pruning strategy, successive halving (Jamieson & Talwalkar 2016). for early termination of unpromising trials. Our search space includes the binary mask (36 or 12 bits) along with the learning rate. For large-scale CheXpert dataset 10% subsampling to accelerate the HPO, and then the full train set for actual training.
random atttrbbte balanced
Since the medical image analysis tasks usually severely imbalanced, cross-entropy loss or accuracy as the meta-objective [fair for the search. Since we use a gradient- free outer-loop optimizer, it is not necessary for the meta-objective be differentiable.
AUC rather than
As remarked in (Zong et al. 2023). existing fairness methods generally did not specify any val- idation criteria. They found that when conducting common fairness-driven HPO, the previously claimed differences between state-of-the of--tr--ttt-tttt methods vanished, and none outperformed ERM (full fine-tuning baseline, in our case). Thus we carefully ensure that all competitors optimise their learn- ing rates based on Llair, while only our FairTune further optimisise the PEFT mask based on the same criterion. Our main HPO objective is disadvantaged subgroup AUC (Sagawa al.) (2020). as We will also experiment with overall AUC as ablation for comparison.
discussed in Sec. 3.1
4.2
MAIN RESULTS
We report our main results in Table in terms of over- all test AUROC, most disadvantaged subgroup AUC, and the gap between advantaged and disadvantaged subgroups. From the table we can draw several con- both training
0.8
clusions: (1) Fine-tuning improves
0.6
from scratch and linear readout of the frozen features.
0.4
As discussed in (Zong et al., 2023), this is very strong baseline which many state-of-the-art purpose- designed fair learning were not able surpass when combined with proper hyperparamettr tuning. (2) Nevertheless, the state-of-the-ar PEFT methods At- tention Tuning (Tourron et al.) 2022) and LN Tun- ing (Basu et al., 2023) surpass this baseline, although they are not purpose designed for faimess at all. We attribute this to their reducing the base architecture's adaptation capacity, limiting its ability to overfit to the advantaged subgroup, and thus limiting the generalisa- tion gap (Figure [] (3) Recent fairness interventions
Val AUC
0.2
Val Best AUC
Val Worst
0
10
20
30
40
50 Epoch
60
70
80
90 100
Figure FairTune leads to stable fine- tuning with reduced differences between the best and worst performing subgroups compared to conventional fine-tuning from Figure]
FairPrune (Wu et al. 2022) and FSCL (Park et al.) (202) also underperform overall. (4) Finally, FairTune generally achieves the best test performance overall by all metrics, with consistently good performance across all the benchmarks and sensitive attributes. (5) The second best in terms of AUC gap training from scratch, but this corresponds unacceptably poor performance overall, an example of the leveling- down phenomenon to be avoided (Zietlow et al. 2022).
As qualitative illustration how Fair Fuir influences the fine-tuning process, we repeat the initial illustrative experiment in Figure T but now with the FairTune discovered mask for the same Papila we can see that the FairTune discovered mask leads to much fairer fine-tuning with much performance for the disadvantaged subgroup, and reduced gaps between the groups compared to the vanilla fine-tuning shown in Figure
dataset. From the results in Figure 3
We further show in the Appendix that FairTune leads to strong performance also when using (1) alternative fairness metries (Table 3). (2) self-supervised pre-trained ViT-Base (He et al. 2022)
(Table 4 and (3) overlapping sensitive attributes (Table
4.3 ABLATION STUDY ON THE DESIGN OF FAIRTUNE
We analyse alternative design choices for FairTune in Table reporting test score average across Fitzpatrick17K. HAM10000, Papila, OL31, OASIS-1 datasets. We first ask What is the impact of our choice of meta-objective foör as introduced in Sec 3.1 Comparing the results for Tuin (Min AUC) and FairTune (Overall AUC), we see that the min min-group performance and corresponding AUC gap are clearly improved. This directly demonstrates the value of tuning model adaptation
7

---

Published as a conference paper at ICLR 2024
Train Scrsich
Full FT
Lincar Readout
Tuning
LayerNorm Tuning
(Ours)
HAMMO000 Fixiiee
Overall AUC
73.5
95.9
92.5
88.4
96.7
Skin
Min. ALC
72.3
94.9
59.6
91.3
52.7
%6.1
AUC
1.6
1.4
3.5
688
2.4
Overall AUC
74.3
86.8
$4.9
91.9
91.1
76.2
71.8
94.0
Min. ALC
66.3
79.2
75,4
85.9
85.2
64.5
90.1
AUC
10.1
12.5
10.9
10,9
13,2
4,9
Ovzrall AUC
54.4
86.8
85.8
91.5
91.5
64.4
68.9
94.8
Gender
Man. ALC
63.7
86.3
85.0
91.9
90.9
63.9
66.3
94.4
AUC
1.7
2.0
1.8
3.6
1.7
1.2
4.8
0,9
Overall AUC
47.5
$6.1
62.2
K1.8
$1.4
73.1
78.3
88.6
Age
Min. ALC
19.3
$1.2
60.7
73.6
65.2
71.2
72.7
K5.2
Papli
Cap AUC
2.5
6.5
29.0
7.7
18.3
7.4
7.6
4.0
Overall AUC
39.7
88.9
64.7
85.4
58.4
79.5
77.9
91.8
Gender
Min. ALC
28.9
88.8
79.5
80.4
$1.0
74.4
71.4
19.2
Cop AUC
24.7
0.2
8.7
9.2
11.4
8.1
8.8
3.6
Overall AUC
61.9
67.4
64.4
64.3
65.3
66.1
64.4
72,6
Age
Min. ALC
54.5
62.4
54.8
62.4
62.0
63.5
60.8
70.1
0L31
Gap AUC
7.9
8.4
14.5
4.7
5.3
4.4
5.7
3.6
Overall AUC
63.8
65.2
62.8
74.8
72.9
65.2
75.
Gender
Min.
60.0
62.5
60.6
69.3
60.4
15.4
Cap
2.1
7.5
4.3
6.5
3.7
57.8
74.5
Age
57.7
60.0
55.6
73.2
Oatic
Cap
4.6
9.0
4.8
1.7
70.8
61.8
65.5
5.2
3,0
3.2
6.2
F2.3
64.7
74.4
86.4
Hanard GP3330
Age
71.7
84.4
4.8
3.2
Overall AUC
13.4
80.2
57.5
74.1
68.4
Gender
Min. ALC
69.4
79.5
$2.9
79.4
85.8
73.5
78.1
86.7
2.5
1.9
Ovzrall AUC
72.9
79.3
53.5
85.2
65.4
74.4
81.5
87.1
Race
Min. AEC
67.4
72.7
79.7
80.8
70.6
76.1
62.4
AUC
6.9
7.3
7.6
5.2
6.5
Overall AUC
63.4
85.5
51.7
86.1
62.8
78.9
79.2
87.5
CheXper
Age
Min. ALC
75.5
82.3
77.3
K2.3
79.7
75.6
77.9
K3.N
AUC
6.1
5.9
4.8
5.3
4.2
4.2
6.2
5.0
Overall AUC
$4.1
85.3
$1.7
86.1
83.2
80.2
79.5
68.2
Gender
Min. ACC
$1.5
$6.1
60.7
65.0
80.6
78.5
77.6
86.5
Cup AUC
4.8
2.3
2.6
2.5
4.1
3.2
4.2
3.2
Aug. Overall Score
68.1
79.9
78.1
61.5
$1.2
72.9
74.2
85.7
Avg Min. Scone
64.0
76.7
73.4
77.9
76.6
89.1
20.3
R3.1
Aeg: Gap Scom:
6.6
4.6
7.1
5.7
6.4
5.4
6.1
3.6
Arg. Overal Rank
7.1
3.5
5.1
3.3
3.4
6.5
6.1
1.0
Aug. Min. Bank
6.9
3.6
5.2
3.1
3.7
6.4
61
1.00
Avg. Clpp Rack
4.9
4.1
4.4
5.2
4.7
4.3
2.7
Table 1: Evaluation of fair generalisation across medical imaging benchmarks. We report the Area Under ROC curve (AUROC) [t. %] across the whole test set (overall) and for the most disadvantaged subgroup (min). We also report the AUROC gap [4, %] between the advantaged and disadvantaged subgroups (Gap). All results are based on ImageNet pre-trained ViT-B, except Train from Scratch.
capacity with a fairness-specific objective rather than general purpose validation objectives. We next ask What is the impact of our PEFT search space, as introduced in Sec 3.2p We compare our full 36-bit search space (which includes full fine-tuning, Attention Tuning, LayerNorm tuning, and Linear Readout special cases), with two smaller alternative 12-bit search spaces that correspond to searching for the subset of attention and layer-norm parameters to update. Between the two search spaces, Attention Tuning is better overall, but also introduces larger AUC gap. However, the full 36-bit Tune space is better than both of these subspaces. Nevertheless, all FairTune variants are better than the Fine-Tune baseline, in terms of Min AUC demonstrating the value of tuning model adaptation capacity. We report the full set of results in Table in the Appendix.
Fair 12-bit, Attention Min Group AUC
FairTune
FairTune 36-bit, Full Overall AUC
FairTune 36-bit, Full
Metrie
Fine- Tune
12-bit, LayerNorm Min Group AUC
Min Group AUC
Overall (1)
78.5
82.4
81.0
83.5
84.7
Min (1)
75.1
79.5
78.0
80.4
82.2
Gap (4)
3.7
4.6
4.2
5.0
3.4
Table 2: Ablation study FairTune design including search space Ω and objective Lfair, Average test AUC over various datasets. Our 36-bit search space surpasses 12 bit alternatives. Min Group AUC as the objective leads to improved fairness compared to the conventional overall AUC.
8

---

Published as a conference paper at ICLR 2024
4.4 ANALYSIS OF MASKS
We finally study what FairTune has learned by analysing the estimated PEFT masks, using the same subset of datasets as for our earlier analysis. We split the analysis by normalization, attention layers and MLP components. For each block, we visualize the proportion of the number of times (over the datasets and sensitive attributes) the given component was selected for fine-tuning, We further compare the masks derived from the different optimization objectives: a) Optimizing for overall per- formance, and b) optimizing the performance the most disadvantaged subgroup. From the plots, we can observe that: (1) The strategies selected are non-trivial without simple preference for either one layer type or initial vs. later layers, as expected by prior intuitively motivated work (Touvron) et al. 2022; Basu et al.) 2023). This demonstrates the value of automated selection of layers for updating. (2) Furthermore, the high-variability of selection for some blocks over datasets/attributes, as indicated by probabilities close to 0.5, shows the importance of learning dataset/attribute cetttrbbuu--specifieee fair tuning strategies, rather than relying on any single task-agnostic recipe. (3) The overall perfor- mance and min-subgroup performance objectives lead substantially different masks, explaining their differing empirical performance earlier. (4) While there is substantial dataset/attribute speci- ficity, there are some general common trends. For example, the min-subbroup objective consistently leads to freezing the first normalization layer, as well as the last four MLP layers. Meanwhile, a clear difference between the overall and min-subgroup objectives the comparatively increased tendency of the overall objective to unfreeze the last four MLP layers.
Normalization Layers
Attention Layers
MLP Layers
0.8-
0.9
0.8
0.7
0.7
0.7
0.6
0.5
0.5
Frequency
0.4
0.5
0.4
0.3
0.3
0.2
0.2
10
2
Block
Block
Block
Obj: Overall Performance
Obj: Min Sub-group Pertormance
Figure 4: Frequency of selecting specific component for fine-tuning across different scenarios.
5 DISCUSSION
Potential Limitations The improvement downstream fairness performance comes at a compu- tational cost it requires us try various configurations of the masks, each of which corresponds to model re-training. For example, our full FairTune pipeline takes 48GPUh on the Fitzpatrick 1k dataset, compared about 1h for unoptimized training, and 14GPUh for our HPO-tuned fine-tuning baseline. As pointed out in Zong et al. (2023), even for conventional models, proper HPO is required for optimising fairness. So the cost of well-tuned model is inevitably much larger than a single training run. Conveniently, the cost per HPO iteration can be substantially lower in our PEFT regime, than in typical train-from ftrom-ccraeh HPO (Feurer& & Hutter 2019). and it could be further alleviated via using efficient techniques such as ASHA (Li et 2020)) or PASHA (Bohdal et al.) (2023) that sup- port parallelization. Future work could also study gradient- based meta-learning (Hospedales al.] 2021 to more efficiently search higher-ding sisher-dimensional masks.
6 CONCLUSION
We provide an empirical demonstration to show that controlling the capacity of deep neural net- works, particularly through the use of Parameter-Efficien Fine-Tuning methods, can lead to im- proved fairness on downstream tasks. Building on this finding, we introduce a framework, FairTune, that is fairness metric-agnostic and provides guidance-free selection of model components to be fine-tuned. Through extensive ablation studies involving different datasets, sensitive attributes and fine-tuning strategies, we established our framework leads consistent gains against standard fine- tuning baselines and vanilla PEFT approaches. Finally, the analysis of the selected masks has shown non-trivial scenario-dependent strategies are learned, showing the need for proposed algorithm.

---

Published as a conference paper at ICLR 2024
REFERENCES
Alekh Agarwal, Alina Beygelzimer, Miroslav Dudik, John Langford, and Hanna Wallach. A reduc- tions approach fair classification. In ICML, 2018a.
Alekh Agarwal, Alina Beygelzimer, Miroslav Dudik, John Langford, and Hanna Wallach. Learning adversarially fair and transferable representations. In ICML, 2018b.
Alekh Agarwal, Miroslav Dudik, and Zhiwei Steven Wu. Fair regression: Quantitative definitions and reduction-based algorithms. ICML, 2019.
Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A next-generation hyperparameter optimization framework, In KDD, 2019.
Shekoofeh Azizi, Basil Mustafa, Fiona Ryan, Zachary Beaver, Jan Freyberg, Jonathan Deaton, Aaron Loh, Alan Karthikesalingam, Simon Kornblith, Ting Chen, al. Big self-supervised models advance medical image classification. In CVPR, 2021.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv, 2016.
Solon Barocas, Moritz Hardt, and Arvind Narayanan. Faimess and machine learning. fairmlbook. org, 2019.
Samyadeep Basu, Daniela Massiceti, Shell Xu Hu, and Soheil Feizi. Strong baselines for parameter efficient few-shot fine-tuning. arXiv, 2023.
Tom L Beauchamp. Methods and principles in biomedical ethics. Journal of Medical ethics. 29(5), 2003.
Elad Ben Zaken, Yoav Goldberg, and Shauli Ravfogel. BitFit: Simple parameter-efficient fine-
tuning for transformer-based masked iinuagg-models, In ACL, May 2022.
James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. Algorithms for hyper-paraments optimization. NeurlPS, 2011.
Alex Beutel, Jilin Chen, Zhe Zhao, and Ed Huai hsin Chi. Data decisions and theoretical implications when adversarially learning fair representations. arXiv, 2017.
Ondrej Bohdal, Lukas Balles, Martin Wistuba, Beyza Ermis, Cedric Archambeau, and Giovanni Zappella. PASHA: Efficient HPO and NAS with progressive resource allocation. In ICLR, 2023.
Simon Caton and Christian Haas. Fairness in machine learning: A survey. ACM Computing Surveys, 2023.
Junbum Cha, Sanghyuk Chun, Kyungjae Lee, Han-Cheol Cho, Seunghyun Park, Yunsung Lee, and Sungrae Park. Swad: Domain generalization by seeking flat minima. NeurIPS, 2021.
Irene Chen, Fredrik D Johansson, and David Sontag. Why is my classifier discriminatory? NeurIPS, 2018.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In ICML, 2020.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
Emily Diana, Wesley Gill, Michael Kearns, Krishnaram Kenthapadi, and Aaron Roth. Minimax group fairness: Algorithms and experiments. In AIES, 2021.
Jeff Donahue, Philipp Krähenbühl, and Trevor Darrell. Adversarial feature learning. arXiv, 2016.
Michele Donini, Luca Oneto, Shai Ben-David, John Shawe-Taylor, and Massimiliano Pontil. Em pirical risk minimization under fairness constraints. NeurlPS, 2018.
Vincent Dumoulin, Ishmael Belghazi, Ben Poole, Olivier Mastropietro, Alex Lamb, Martin Ar- jovsky, and Aaron Courville. Adversarially learned inference. arXiv, 2016.
10

---

Published as a conference paper at ICLR 2024
Raman Dutt, Dylan Mendonca, Huai Ming Phen, Samuel Broida, Marzyeh Ghassemi, Judy Gi- choya, Imon Banerjee, Tim Yoon, and Hari Trivedi. Automatic localization and brand detection of cervical spine hardware radiographs using weakly supervised machine learning. Radiology: Artificial Intelligence, 4(2)x211009, 2022.
Raman Dutt, Linus Ericsson, Pedro Sanchez, Sotirios A Tsaftaris, and Timothy Hospedales. Parameter-efficient fine-tuning for medical image analysis: The missed opportunity. arXiv, 2023.
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairess through awareness. In Proceedings the 3rd innovations theoretical computer science confer- ence, 2012.
Andre Esteva, Katherine Chou, Serena Yeung, Nikhil Naik, Ali Madani, Ali Mottaghi, Yun Liu, Eric Topol, Jeff Dean, and Richard Socher. Deep learning-enabled medical computer vision. NPJ digital medicine, 4(1), 2021.
Matthias Feurer and Frank Hutter. Hyperparameter optimization. Automated machine learning: Methods, systems, challenges, 2019.
Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimiza- tion for eficiently improving generalization. In ICLR, 2021.
Florin C Ghesu, Bogdan Georgescu, Awais Mansoor, Youngjin Yoo, Dominik Neumann, Prag- neshkumar Patel, RS Vishwanath, James M Balter, Yue Cao, Sasa Grbic, et al. Self-supervised learning from 100 million medical images. arXiv, 2022.
Matthew Groh, Caleb Harris, Luis Soenksen, Felix Lau, Rachel Han, Aerin Kim, Arash Koochek, and Omar Badri. Evaluating deep neural networks trained on clinical images in dermatology with the fitzpatrick 17k dataset. In CVPR, 2021.
Matthew Groh, Caleb Harris, Roxana Daneshjou, Omar Badri, and Arash Koochek. Towards trans- parency in dermatology image datasets with skin tone annotations by experts, crowds, and an algorithm. arXiv, 2022.
Melissa Hall, Laurens van der Maaten, Laura Gustafson, and Aaron B. Adcock. A systematic study of bias amplification. ArXiv, 2022.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In CVPR, 2022.
Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey, IEEE transactions on pattern analysis and machine intelligence, 44(9), 2021.
Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. ICLR, 2022.
Jeremy Irvin, Pranay Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-llcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In AAAI, 2019.
Kevin Jamieson and Ameet Talwalkar. Non-stochastic best arm identification and hyperparameter optimization. In AISTATS, 2016.
Jiwoong J Jeong, Brianna L Vey, Ananth Bhimireddy, Thomas Kim, Thiago Santos, Ramon Correa, Raman Dutt, Marina Mosunjac, Gabriela Oprea-llies, Geoffrey Smith, et al. The emory breast imaging dataset (embed): A racially diverse, granular dataset of 3.4 million screening and diag- nostic mammographic images. Radiology: Artificial Intelligence, 5(1), 2023.
Byungju Kim, Hyunwoo Kim, Kyungsu Kim, Sungjin Kim, and Junmo Kim. Learning not to learn: Training deep neural networks with biased data. In CVPR, 2019.
Matthäus Kleindessner, Samira Samadi, Muhammad Bilal Zafar, Krishnaram Kenthapadi, and Chris Russell. Pairwise faimess for ordinal regression. In AISTATS, 2022.

---

Published as a conference paper at ICLR 2024
Simon Kornblith, Jonathon Shlens, and Quoc V Le. Do better imagenet models transfer better? In CVPR, 2019.
Oleksandr Kovalyk, Juan Morales-Sánchez, Rafael Verdi-Monndeero, Inmaculada Sellés-Navarro Ana Palazón-Cabanes, and José-Luis Sancho-Gómez. Papila: Dataset with fundus images and clinical data of both eyes of the same patient for glaucoma assessment. Scientific Data, 9(1,, 2022.
Agostina Larrazabal, Nicolás Nieto, Victoria Peterson, Diego H Milone, and Enzo Ferrante. Gen- der imbalance in medical imaging datasets produces biased classifiers for computer-aided diag- nosis. Proceedings of the National Academy of Sciences, 117(23), 2020.
Liam Li, Kevin Jamieson, Afshin Rostamizadeh, Ekaterina Gonina, Jonathan Ben-Tzur, Moritz Hardt, Benjamin Recht, and Ameet Talwalkar. A system for massively parallel hyperparameter tuning. MLSys, 2020.
Risheng Liu, Jiaxin Gao, Jin Zhang, Deyu Meng, and Zhouchen Lin. Investigating bi-level opti- mization for learning and vision from unified perspective: survey and beyond. arXiv, 2021.
Kirsten Lloyd. Bias umplification in artificial intelligence systems. arXiv, 2018.
Michael Lohaus, Michael Perrot, and Ulrike Luxburg. Too relaxed to be fair. In ICML, 2020.
Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv, 2016.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2018.
Yan Luo, Yu Tian, Min Shi, Tobias Elze, and Mengyu Wang. Harvard glaucoma fairness: A retinal nerve disease dataset for fairness learning and fair identity normalization. arXiv, 2023.
Jun Ma and Bo Wang. Segment anything in medical images, arXiv, 2023.
David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. Learning adversarially fair and transferable representations. ICMI., 2018.
Daniel S Marcus, Tracy H Wang, Jamie Parker, John G Csemannsy, John C Morris, and Randy L Buckner. Open access series of imaging studies (oasis): cross cross-sectional mri data in young, middle aged, nondemented, and demented older adults. Journal of cognitive neuroxcience, 19(9), 2007.
Roman C Maron, Michael Weichenthal, Jochen Utikal, Achim Hekler, Carola Berking, Axel Hauschild, Alexander H Enk, Sebastian Haferkamp. Joachim Klode, Dirk Schadendorf, et al. Systematic outperformance 112 dermatologists in multiclass skin cancer image classification by convolutional neural networks. European Journal of Cancer, 119, 2019.
Natalia Martinez, Martin Bertrana, and Guillermo Sapiro. Minimax pareto fairness: multi objec-
tive perspective. In ICML, 2020.
Ninareh Mehrabi, Fred Morstatter, Nripsuta Ani Saxena, Kristina Lerman, and Galstyan. A survey on bias and fairness in machine learning. ACM Computing Surveys, 2021.
Xueyan Mei, Zelong Liu, Philip M Robson, Brett Marinelli, Mingqian Huang. Amish Doshi, Adam Jacobi, Chendi Cao, Katherine E Link, Thomas Yang, et al. Radimagenet: An open radiologic deep learning research dataset for effective transfer learning. Radiology: Artificial Intelligence, (5), 2022.
Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(6464), 2019.
Manisha Padala and Sujit Gujar. Fnne: Achieving faimess through neural networks. In IJCAI, 2020.
Sungho Park, Jewook Lee, Pilhyeon Lee, Sunhee Hwang. Dohyung Kim, and Hyeran Byun. Fair contrastive learning for facial attribute classification. In CVPR, 2022.
12

---

Published as a conference paper at ICLR 2024
María Agustina Ricci Lara, Rodrigo Echeveste, and Enzo Ferrante. Addressing faimesss in artificial intelligence for medical imaging. Nature Communications, 13(1), 2022.
Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks. In ICLR, 2020.
Mhd Hasan Sarhan, Nassir Navab, Abouzar Eslami, and Shadi Albarqouni. Fairness by learning orthogonal disentangled representations. In ECCV, 2020.
Laleh Seyyed-Kalantari, Haoran Zhang, Matthew BA McDermott, Irene Y Chen, and Marzyeh Ghassemi. Underdiagnosis bias of artificial intelligence algorithms applied chest radiographs in under-served patient populations. Nature medicine. 27(12). 2021.
Jae-hun Shim, Kyeongbo Kong, and Suk-Ju Kang. Core-set sampling for efficient neural architecture search. arXiv, 2021.
A. Sinha, P. Malo, and K. Deb. A review on bilevel optimization: From classical to evolutionary approaches and applications. IEEE Transactions on Evolutionary Computation, 22(2), 2018.
Yucheng Tang, Dong Yang, Wengi Li, Holger R Roth, Bennett Landman, Daguang Xu, Vishwesh Nath, and Ali Hatamizadeh. Self-supervised pre-training of swin transformers for 3d medical image analysis. In CVPR, 2022.
Enzo Tartaglione, Carlo Alberto Barbano, and Marco Grangetto. End: Entangling and disentangling deep representations for bias correction. In CVPR, 2021.
Ekin Tiu, Ellie Talius, Pujan Patel, Curtis P Langlotz, Andrew Y Ng. and Pranav Rajpurkar. Expert- level detection of pathologies from unannotated chest x-ray images via self-supervised learning. Nature Biomedical Engineering, 2022.
Hugo Touvron, Matthieu Cord, Alaaeldin El-Nouby, Jakob Verbeek, and Hervé Jégou. Three things everyone should know about vision transformers. In ECCV, 2022.
Philipp Tschandl. The HAM10000 dataset, large collection of multi-source dermatoscopic images of common pigmented skin lesions, 2018.
Berk Ustun, Yang Liu, and David Parkes. Fairness without harm: Decoupled classifiers with prefer- guarantees. In ICMI., 2019.
Vladimir N Vapnik. An overview of statistical learning theory. IEEE transactions on neural net- works, 10(5), 1999.
Vanshika Vats, Aditya Nagori, Pradcep Singh. Raman Dutt, Harsh Bandhey, Mahika Wason, Rakesh Lodha, and Tavpritesh Sethi. Early prediction of hemodynamic shock in pediatric intensive care units with deep learning on thermal videos. Frontiers in Physiology. 13:862411,2000
Sahil Verma and Julia Rubin. Fairness definitions explained. In International workshop on software fairness (fairware). 2018.
Savan Visalpara, Krishnateja Killamsetty, and Rishabh Iyer. A data subset selection framework for efficient hyper-parameter tuning and automatic machine learning. In ICMI. Workshops, 2021.
Zeyu Wang, Klint Qinami, loannis Christos Karakozis, Kyle Genova, Prem Nair, Kenji Hata, and Olga Russakovsky. Towards fairness in visual recognition: Effective strategies for bias mitigation. In CVPR, 2020.
Hilde Weerts, Miroslav Dudik, Richard Edgar, Adrin Jalali, Roman Lutz, and Michael Madaio. Fairlearm: Assessing and improving fairness ai systems. JMLR, 2023.
Ross Wightman. Pytorch image models.
https://github. http:/hwh..om////a////eeea wwt.w/wwwww.......... http://thhabbccmmm/ii.aaaaaa///t
pytorch- sanrcch-image-ee www.imaaamaa....oo
2019.
Junde Wu, Rao Fu, Huihui Fang, Yuanpei Liu, Zhaowei Wang. Yanwu Xu, Yueming Jin, and Tal Arbel. Medical sam adapter: Adapting segment anything model for medical image segmentation. arXiv, 2023.
13

---

Published as a conference paper at ICLR 2024
Yawen Wu, Dewen Zeng, Xiaowei Xu, Yiyu Shi, and Jingtong Hu. Fairprune: Achieving fairness through pruning for dermatological disease diagnosis. International Conference on Medical Image Computing and Computer-Assister Intervention, 2022.
Zikang Xu, Jun Li, Qingsong Yao, and S. Kevin Zhou. Progress and prospects for fairness in healthcare and medical image analysis. arXiv, 2023.
Fabian SL Yii, Raman Dutt, Tom MacGillivray, Baljean Dhillon, Miguel Bernabeu, and Niall Strang. Rethinking retinal image quality: Treating quality threshold as tunable hyperparameter. In International Workshop Ophthalmic Medical Image Analysis, 2022.
Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? In NIPS, 2014.
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez. and Krishna P Gummadi. Fair- ness constraints: Mechanisms for fair classification. In AISTATS, 2017.
Juan Zambrano Chaves, Akshay Chaudhari, Andrew L. Wentland, Arjun D Desai, Imon Baner- jee, Robert D Boutin, David J Maron, Fatima Rodriguez, Alexander ' Sandhu, R Brooke Jeffrey, et Opportunistic assessment of ischemic heart disease risk using abdominopelvic computed tomography and medical record data: multimodal explainable artificial intelligence approach. medRxiv, 2021.
Kaidong Zhang and Dong Liu. Customized segment anything model for medical image segmenta- tion. arXiv, 2023.
Han Zhao, Amanda Coston, Tameem Adel, and Geoffrey J Gordon. Conditional learning of fair representations. In ICLR, 2019.
Dominik Zietlow, Michael Lohaus, Guha Balakrishnan, Matthäus Kleindessner, Francesco Lo- catello, Bernhard Schölkopf, and Chris Russell. Leveling down in computer vision: Pareto inef- ficiencies in fair deep classifiers. In CVPR, 2022.
Yongshuo Zong, Yongxin Yang, and Timothy M Hospedales. Medfair: Benchmarking faimess for medical imaging. In ICLR, 2023.
14

---

Published as a conference paper at ICLR 2024
A
APPENDIX
A.1 ADDITIONAL ANALYSES
We include several further analyses, including the study of fairness transferability to other metrics, evaluation using a self-supervised pre-trained model, overlapping sensitive attributes and others.
Impact on Other Fairness Metrics Our implementation of FairTune targets minAUC the meta- objective for selecting PEFT masks that optimize this notion of fairness. One might reasonably ask how the resulting models perform in terms of other notions of fairness such as Equalized Odds Difference (EOddsD) Agaawal et al., 2018a) and Demographic Parity Difference (DPD) (Agarwal al. 20 2019 Barocas et al. (211)). We emphasise that although these metries are common, they have also been widely criticised in the literature for being pareto inefficient and potentially violating ethical non-maleficence (Beauchampl 2003) Chen et al. 2018 Ustun al. 2019 Zietlow iet al. 2022). For example, it is possible to fully satisfy these criteria by providing zero-accuracy for all subgroups, which would be strictly worse than the status quo. Therefore we followed the recommendation of GroupDRO (Sagawa et al. (020) and the recent MEDFAIR (Zong et al. 2023), and focused our evaluation on the most disadvantaged subgroup metric (minAUC).. This metric not vulnerable to the potential pathological outcomes that satisfy EOddsD and DPD.
Neveretheless, for completeness we evaluate our minAUC optimised models in terms of EOddsD and DPD metrics in Table D
The results show FairTune does quite good job of satisfying the EOddsD and DPD objectives, even though our algorithm optimises for minAUC. Where other meth- ods outperform FairTune on these metrics, they are worse both overall and minAUC, thus being pareto dominated by FairTune.
Finally, we remark that while we recommend minAUC objective and metric, the design of the Fair- Tune algorithm treats specific choice of optimization metric as a hyperparameter. Therefore users casily able to plug-and-pla. EOddsD, any other fairness metric as the target for FairTune to optimize (cf: Section 4.3 and Table
Combination with Other Pre-Trained Models Our main results are based on fine-tuning a VIT- В pre-trained by supervised learning ImageNet. We also study the impact of fine-tuning self- supervised pre-trained model, namely Masked AutoEncoder (MAE) (He et al. 2022). The results in Table 4 confirm that FairTune obtains both excellent overall performance and fairness, including in terms of the alternative fairness metrics.
Overlapping Sensitive Attributes Our main exepriments used binary sensitive attributes. We now perform an additional experiment to further test the efficacy of FairTune in a scenario with multiple overlapping sensitive attributes. Specifically we used the CheXpert dataset and defined new sensitive attributes based the intersection of the annotated Age and Gender attributes. This resulted in distinct categories: (1) Patients with ages between and 60 and Male gender, (2) Patients with ages 60 and above and Male gender, (3) Patients with ages between O and 60 and Female gender, and (4) Patients with ages 60 and above and Female gender.
The results of this experiment are presented in Table 5 We can see that FairTune leads excellent fairness and overall performance also when using overlapping sensitive attributes. The results also indicate that good performance consistent for supervised and self-supervised base models.
Analysis of Outer Loop Convergence Figure illustrates the outer loop convergence process for our datasets when using Min AUC objective on the 12-bit Attention Tuning Search Space. All datasets saturate within the given number of outer-loop iterations showing that the HPO search for sufficient number of trials to reach the optimal objective value.
Full details of mejavvbeective comparison from Table 2 the main paper.
Table 6 reports the full details of the ablation study
15

---

Published as a conference paper at ICLR 2024
4444
050
Teal
(a) Fitzpatrick
(b) Papila
(c) OL31
Figure 5: Optimization trajectory for the outer loop PEFT mask search. The plots shown here are for the 12-bit Attention Tuming PEFT search space. The saturation in the objective value demonstrates that the objective saturated within 200 outer loop iterations trials.
A.2 DATASET DETAILS
In this section, we report the dataset details. All datasets are publicly accessible from the URLs shown in Table 7 Dataset statistics shown in Table 8 Tables 9 13 report the specific sensitive attribute splits used for each dataset.
We categorized the three partition labels into binary labels, specifically "benign" and "malignant." " Within this categorization, we considered "non-neoplastic" and "benign" as be- longing the benign label category, while "malignant" remained in the malignant label category. Additionally, we utilized Fitzpatrick skin type labels as the sensitive attributes for our analysis.
Fitzpatrick17k:
HAM10000: We categorized the 7 diagnostic labels into binary labels, specifically "benign" and "malignant," in accordance with the methodology outlined by Maron et al. (2019). The "benign" cat- egory encompasses basal cell carcinoma (bcc), benign keratosis-like lesions (including solar lentig- ines, seborrheic keratoses, and lichen-planus like keratoses, bkl), dermatofibroma (df), melanocytic nevi (nv), and vascular lesions (comprising angiomas, angiokeratomas, pyogenic granulomas, and hemorrhage, vasc). On the other hand, the "malignant" category includes Actinic keratoses and intraepithelial carcinoma (Bowen's 's disease, akiec), and melanoma (mel). Images lacking recorded sensitive attributes were excluded from the dataset, resulting in a total of 9948 retained images.
Papila: this dataset, we have excluded the "suspect" label class and have focused on binary clas- sification tasks using images labeled as either "glaucomatous" or "oongllaucomatous." The dataset includes both right-eye and left-eye images of the same patients. For the purpose of splitting the dataset into training, validation, and test sets, we have followed specific proportion, with a split of 70% for training, 10% for validation, and 20% for testing. Additionally, it is worth noting that ensure that images from the same patient not shared across these splits. This practice helps maintain the independence of the data subsets used for training, validation, and testing, which is crucial for evaluating the model's performance effectively.
OL3I: The Opportunistic L.3 computed tomography slices for Ischemic heart disease risk assess- ment (OL3I) dataset comprises 8139 axial computed tomography (CT) slices acquired at the third lumbar vertebrae (L3) level of various individuals. The primary objective of this dataset is to develop a predictive model for determining whether an individual will receive diagnosis of ischemic heart disease within one year following the CT scan, based on the provided labels, which represent the prognosis. In this analysis, both the sex and age of the individuals are considered sensitive attributes, taking into account potential disparities in the risk assessment related to these attributes.
This dataset comprises rross-sectional assortment of 416 individuals spanning an age range from 18 to 96 years old. For each of these individuals, the dataset includes 3 or 4 individ- ual Tl-weighted MRI scans, all acquired during single sean sessions. The participants encompass individuals who are right-handed, and the dataset includes both male and female subjects. Among the included subjects, 100 individuals who are over the age of 60 have received clinical diagnoses ranging from very mild to moderate Alzheimer's 's disease (AD). Furthermore, the dataset also incor- porates a reliability dataset, which contains imaging data from 20 individuals who are not diagnosed with dementia. These individuals underwent subsequent MRT session within 90 days of their ini-
Oasis:
16

---

Published as a conference paper at ICLR 2024
tial imaging session for the purposes of assessing data reliability and consistency. The dataset is originally in 3D which has been converted to 2D by selecting the central slices from the MRI scan. We have categorized the labels into two categories by mapping the 'CDR' labels O, 0.5, and into one category and label 2.0 into the other category. While selecting the slices from MRI, we have selected 10% of the central slices for the first category and 25% in the case of the second category.
HF3300d-GF3300: The dataset has been designed for fairness learning and contains 3300 2D and 3D retinal nerve samples from 3300 patients. The dataset is balanced in terms of racial groups and contains information on three different types of sensitive attributes: age, sex, and race. The classification label determines if a patient has glaucoma or not (binary classification). As pre- processing step, we have binarized the sensitive attribute age and race, following the steps in Zong et al. 2023).
CheXpert: This dataset contains 224,316 chest radiographs of 65,240 patients. Each image can have one or more from set of 14 labels depicting 14 different observations. For preprocessing, we have binarized the sensitive attribute age and used the "No Finding" label for training, validation and testing, as done in Zong et al. (2023).
A.3 METRICS
The metries employed in the study are explained below:
Area under the receiver operating characteristic curve (AUROC) is the standard metric for evaluation of the performance of binary classification tasks. The metric remains unaffected by the potential imbalance in class labels. Our assessment involves the computation of both the average AUC and the AUC for individual subgroups. Noteworthy emphasis is placed on the AUC gap and the worst-case AUC, serving as crucial indicators in the evaluation of group fairness and max-min fairness.
AUC:
Equalized Odds Difference (EOddsD): This metric measures if a machine learning system works equally well on different subgroups by determining the true positive and false positive rates across different subgroups. A classifier, denoted as h, is deemed to satisfy equalized odds within a distribution over (X, Y) ) if the prediction h(X) exhibits conditional independence with respect to the sensitive attribute A, given the label Y. Agarwal et al. 2018a) define this as E = Y = y] E (x) = y). In our experiments, we provide Equalized Odds Difference from the Fairlearn package Weerts al. 2023) that returns the larger of the true positive rate difference and false positive rate difference.
Demographic Parity Difference (DPD): Demographic parity, as fairness metric, aims guaran- tee that predictions made by a machine learning model remain impartial with regard to membership in a sensitive group. Simply put, achieving demographic parity signifies that the likelihood of specific prediction is not contingent upon membership in sensitive group. In the context of binary classification, demographic parity specifically entails maintaining equal selection rates across dif- ferent groups. A classifier h satisfies demographic parity under distribution over [X, A, A, if its prediction IxX) is statistically independent of the sensitive feature Agaawal et al. (2018a) define
this [']((X)] A = a) (X)). We used the Fairlearn package Weerts et al. (2023) for DPD implementation that reports the absolute difference between the highest and lowest selection rates E A.
A significant limitation of EOddsD and DPD metrics that they can be trivially satisfied when the performance is 0. More broadly they do not necessitate strong performance while achieving fairness, which can often lead to undesirable performance losses. minAUC does not suffer from such limitation. That is, perfect minAUC is sufficient condition for utopia (all subgroups solved perfectly), while EOddsD and DPD are only necessary, but not sufficient conditions.
A.4 DATA PREPROCESSING AND EXPERIMENTS
We follow the data preprocessing steps as in Zong et al. 2023 Specifically, we created random splits of the entire dataset into training, validation and test with proportions of 80%. 10%, 10%. Next, we binarized the prediction labels and the sensitive attributes. Detailed instructions with examples given here
17

---

Published as a conference paper at ICLR 2024
A.4.1 SENSITIVE ATTRIBUTES
Sensitive Attribute Skin Type: In relation to the sensitive attribute skin type we delineate two groups. The first group encompasses samples with skin types ranging from 0 to 2, while the second group comprises samples with skin types exceeding 2.
Sensitive Attribute Age: For the datasets HAM10000, Papila. OL3I, Oasis, Burvard-GF3300 and CheXpert, we establish two distinct categories categorize the sensitive attribute age: one encompassing ages ranging from O to 60, and the other comprising individuals with ages exceeding 60.
Sensitive Attribute Race:
We created two categories with 'Black' patients belonging to the first and 'Asian' or 'White' patients belonging to the second.
Overlapping Sensitive Attributes (Age+Gender): We created new category of sensitive at- tributes by combining Age and Gender.
Category Patients with ages between 0 and 60 and Male gender. Category 2: Patients with ages 60 and above and Male gender.
Category Patients with ages between 0 and 60 and Female gender.
Category 4: Patients with ages 60 and above and Female gender.
A.4.2 LABELS
The first group includes samples labelled malignant, while the second group contains samples with all other remaining labels.
Fitzpatrick17k:
We convert the original labels into two categories: benign and malignant. The former contained samples with original labels 'bec', 'bkl', 'dermatofibroma' 'nv", 'vasc' while the latter category contained samples labelled as 'akiec', and 'mel'.
HAM10000:
Papila: We used samples belonging to 'healthy' and 'glaucoma' categories while samples from the 'suspect' class was excluded.
OL3I: The original dataset already contains binary categories.
Oasis-1: The 'CDR' labels 0.5 and into one category and label 2.0 into the other category.
Harvard-GF3300: The original labels are already categorized into two categories: Glaucoma and Non-Glaucoma.
Originally, each sample in the dataset can correspond to one or more of the 14 labels. We used the 'Noo-Fidding' label for training, validation and testing (as done in Zong al. (0223) since it is the only binary classification label.
CheXpert:
A.5 EXPERIMENTAL SETTINGS FOR HYPERPARAMETER OPTIMIZATION (HPO)
Details of HPO Pruning: We employed the Successive Halving Pruner provided in the Optuna package Akiba et al. (2019). Successive Halving (SH) is a bandit-based algorithm that identifies the best configuration amongst set of configurations. The min resource parameter was set auto to automatically determine the number of minimum resources to be allocated to trial, and we set the reduction factor parameter to 4 (default value). This indicates that after the end of each rung. .J. of the most promising trials would be promoted. For the remainder of the parameters (min.early-stoppel stoppiystopi .rate and bootstrap
we used the default value O.
Details the Parameter Sampler: We employed the TPE sampler that uses the Tree-structured Parzen Estimator algorithm to sample hyperparameter values. Specifically, each trial, TPE fits a Gaussian Mixture Model (GMM) to the parameter values (binary mask, in our case) that lead the best values for the objective metric (min sub-group AUC, overall sub-group AUC, etc). Hence with each trial, sample better parameters (mask) that lead to improved value for the objective.
Details on BLO using TPE and SH: In each trial, the TPE sampler samples some values for the binary mask that is employed in fine-tuning. At the end of fine-tuning, obtain value for
18

---

Published as a conference paper at ICLR 2024
the objective metric that is associated with the sampled values. Over numerous trials, the sampler determines which values better optimize the objective metric and gives them a higher preference during the sampling process. At the same time, Successive Halving (SH) prunes certain trials that do not show sufficient promise, thus saving both time and compute. More specifically, in our case, only quh of the trials (sampled mask values) promoted for consideration in the next rung. For the HPO search, we used different number of trials for different datasets depending on their size. Specifically, we used 40 trials for Fitzpatrick I 17k, 60 trials for HAM10000, Oasis, and Harvard- GF3300, 100 trials for Papila and OL3I, and 40 trials for CheXpert. Notably, we ran HPO 10% of the total samples in CheXpert owing its size.
Scalability to Large Datasets: We employed CheXpert Irvin et al. 2019) in order to test the scalability of our proposed framework to large datasets. The original size of CheXpert around 220,000 samples. We randomly selected 10% (20,000) samples for conducting the HPO search. The mask obtained through this search was employed during fine-tuning on the entire original dataset. During subset selection, ensured maintaining the ratios of sensitive attributes. Subsampling for HPO search has been shown to be effective previously (Shim et al. 2021 Visalpara et al. 2021 and results reveal the same.
19

---

Published as a conference paper at ICLR 2024
Thibssr
Atte:
Metric
Scratch
Trsin From
FT
Lincar Readout
Atiention Tuning
LayerNorm Tuning
FairPronc
ESCI.
FairTone (Ours)
ObexallAAle
13.5
95.9
98.3
94.1
92.5
83.4
89.1
96.7
Flott 7k
Min. ALC
72.3
94.9
99.6
93.2
91.5
K2.7
84.2
96.1
Skin Type
Gup AUC
1.6
3.8
2,8
1.4
3.5
6.8
5.6
2.4
BO
174
153
10.1
21.9
1.2
48.4
26.2
10.0
DPD
12.2
4,4
6.8
5.1
6.6
21.3
9.3
59
Overall
74.3
26.8
$4,9
93.9
91.1
76.2
77.8
94.0
Min. ALC
66.3
79.2
75.4
K3.9
53.2
64.3
67.3
90.1
HAMH0000
Age
Gap AUC
10.1
11.5
8.5
9.1
11.1
12.5
10.9
8,3
59.3
10.9
41.2
13.2
26.7
14.8
10.1
4.9
DPD
23.1
12.9
11.9
7.2
12.7
23.1
16.5
13.2
54.4
14.8
Min.
83.7
36.3
85.0
91.9
90.9
63.9
66.3
91.4
Gender
AUC
1.7
1.8
3.6
1.2
0.9
67.3
86.9
70.6
32.1
5.6
3.3
1.1
AUC
47.5
77.1
65.2
71.2
85.2
Age
2.5
18.3
7.4
40
Pepilo
BQalaD
45.1
36.1
41.8
42.8
57.1
39.2
34.5
30.3
11.4
5.1
3.6
39.2
12.5
10.0
9.7
7.3
4.9
61.9
65.3
71.6
54.5
63.5
70.1
Age
7.9
5.3
4.4
3.6
BOodsD
OL.11
63.5
65.2
75.2
62.0
60.4
75.4
Gender
AUC
2.1
4.3
6.5
3.7
DOGISD
21.3
24.5
51.2
18.5
12.3
$2.4
4.4
Ovzaal AUC
61.0
64.4
62.6
66.0
66.8
57.8
$1.4
74.5
Min. AEC
57.7
60.1
60.0
55.6
73.2
AUC
4.6
9.0
4.8
1.7
B06Hd
41.4
46.7
43.8
98.2
74.3
96.2
44.7
Oasis
DPD
41.1
35.6
43.6
38.2
45.3
31,3
38.2
35.0
Overall AUC
60.7
64.6
62.3
65.1
63.4
64.3
62.3
70,8
Min. ACC
56.9
60.1
99.7
61.5
61.4
61.8
60.8
65.5
Gender
AUC
5.2
4.8
3.7
488
3.0
3.2
62
52.2
58.3
91.2
63.3
64.2
71.4
84.2
S8.3
DPD
25.0
31.3
15.8
18.3
20.7
22,1
22.3
20.6
Overall AUC
72.3
$2.3
$3.6
51.7
$4.7
74.4
80.4
M5.4
Min. ALC
67.4
79.1
$1.7
77.5
$1.5
71.7
78.3
84.4
Age
Cup ALC
6.4
2.5
3,5
48
3.8
48
3.2
3.2
EDalaD
34.2
23.1
23.8
20.3
21.2
53.2
27.2
18.1
Haraad GF3300
DPD
34.4
3012
30.3
24.7
31.3
$1.4
32.1
26.5
Overall AUC
73.4
60.2
83.4
60.1
87.6
74.1
81.1
63.4
Min. ACC
69.4
79.5
62.0
70.4
65.8
73.5
76.
86.7
Gender
AlCC AUIC
7.8
2.8
2.8
2.9
2.5
3.1
2.4
1.9
DOSlsD
22.3
13:3
4.4
13.2
7.2
21.3
11.2
9.4
DPD
9.1
7.7
3.4
8.5
5.7
8.1
7.3
6.3
Overall AUCC
72.9
79.3
$3.5
k5.2
5544
74.4
81.5
87.1
Min. ALC
67.4
72.7
51.1
70.7
50.8
70.6
76.
H2.4
Rice
Gup AUC
6.9
7.3
4.6
7.8
7.6
5.2
6.5
B044dD
18.1
9.3
14.2
6.2
10.1
25.1
12.2
13.2
DPD
17.2
6.2
11.3
11.4
11.2
16.0
11.4
13.3
Overall AUC
63.4
85.5
$1.7
86.1
52.8
78.9
79.2
N7.5
Min. ALC
78.5
82.3
71.3
$2.3
79.7
75.6
77.9
83.8
Age
Gap AUC
6.1
3.9
4.8
5.3
4.2
4.
5.0
Chelkprrt
DOddsD
23.3
2333
22.1
28.9
25.4
$1.6
$1.2
23.4
DPD
9.1
7.4
6.6
5.4
18.2
$1.1
36.2
17.1
80.2
Min. AUC
81.5
$6.1
$0.7
85.0
80.6
78.5
86.5
Gender
Gap AUC
4.8
2.6
2.5
4.1
1.2
3.2
19.2
11.3
19.3
13.3
10.1
DPD
4.1
13.4
8.1
Aug: Overall Score
79.9
R1.5
73.9
85.7
Aug. Stoee
64.0
76.7
77.9
76.5
68.1
R3.1
Avg. Gup Scmce
6.6
6.4
5.4
3.6
Arg. EDMdD Same
20.4
29.5
29.5
35.7
44.7
23,9
Aug DPD Score
20.8
153
14.4
16.3
17.7
21.4
20.9
16.0
Avg. Ovenal Rank
Avg. Min, Rask
3.7
6.4
1.0
Avg. Cap
4.7
4.3
3.7
Avg.
6.4
3.6
Ang. DPD
6.9
4.9
5.6
Table 3: Evaluation of fair generalisation across medical imaging benchmarks. We report the Area Under ROC curve (AUROC) [t. %] across the whole test set (overall) and for the most disadvantaged subgroup (min). We also report the AUROC gap [L. %] between the advantaged and dinadvantaged subgroups (Gap), Difference of Equalized Odds [L, %, Agarwal et al. (2018a) and Demographic Parity Difference [L. %. Agarwal al. (201 Sa 2019); [arocaa et al. (2019)). All results are based on ImageNet pre-trained ViT-B, except Train from Scratch.
20

---

Published as a conference paper at ICLR 2024
Datuset
Attr:
Metric
Seratch
Readout
Attention Tuning
LayerNorm Tuning
FairTune (Ours)
Overall AUC
73.5
91.7
85.6
73.6
92.1
95.3
Fiize 7%
Min. AUC
72.3
93.5
84.1
69.1
91.3
94.3
Typo
3.7
EOAddD
17.1
15.3
26.2
1.3
9.9
DPD
12.2
5.4
4.5
11.0
5.2
3.5
Overall AUC
74.3
87.9
80.6
82.5
90.6
92.2
Mín. AUC
66.3
81.7
HAM1000
Age
Gap
EOddaD
10.1
11.5
21.8
4.0
11.7
11.7
7.1
DPD
23.1
0.7
12.1
7.4
10.4
0.6
Ovenall AUC
84.4
91.3
67.6
92.1
91.2
94.1
Gendrr
Gap
1.7
2.8
2.2
1.9
67.3
13.8
5.2
14,1
11.8
3.0
DPD
5.6
2.2
1.9
3,9
2.0
0.4
Overall AUC
47.5
83.6
79.5
83.8
82.6
84.4
77.1
Age
Gap AUC
2.5
6.1
16.4
13.1
8.6
7.0
EOddaD
36.1
31.2
18.8
37.3
29.3
25,4
Pepils
DPD
45.1
35.6
11.8
38.2
34.4
33.0
Overall AUC
39.7
78.2
82.6
81.8
74.8
Gender
Gap ALC
24.7
11.1
10.3
7.0
8.0
3.6
EDdda
20.5
16.7
33.3
18.3
22.6
15.5
DPD
8.3
0.7
16.0
Min. AUC
54.5
58.6
64.3
64.1
Age
Gap ALC
7.9
8.3
4.2
5.5
9.1
7.9
ED3daD
33.1
34,9
26.8
23.2
60.0
36.
OLS
DPD
43.1
Min. AUC
62.0
70.3
57.7
60.2
67.5
71.8
Gender
Gap AUC
2.1
2.1
4.9
3.7
5.7
4.5
EOddaD
21.3
19.7
16.6
13.8
11.0
11.8
AUC
67.1
68.8
Min. AUC
57.5
53.7
60.5
57.8
51.1
68.4
Age
Gap ALC
4.6
25.4
9.8
18.8
17.7
6.5
EDStd
46.0
20.3
Ouais
Overall AUC
60.7
70.9
67.2
58.6
61.8
72.6
Min. AUC
56.9
57.8
55.1
$7.4
56.3
66.4
Gender
Cap ALC
5.2
14.4
12.3
3,8
7.8
7.3
EDdtd
$2.2
1.0
21.4
DPD
25.0
35.3
3.8
0.3
19.5
Overall AUC
72.3
84.5
75.2
75.5
85.1
86.1
Min. AUC
67.4
82.9
74,2
72.1
83.4
84,0
ALC
6.4
7.0
3.7
4.9
2.9
EO3lD
Harvar C GF3300
DPD
34.4
24.4
33.4
17.4
29.3
21.3
Overall AUC
73.4
85.1
76.1
71.4
79.1
86.1
Min. AUC
69.4
84.3
75.3
70.2
78.1
85.4
Gender
7.8
2.7
3.8
EDdlaD
71.0
DPD
9.1
3.1
6.1
4.7
10.4
2.4
Overall AUC
72.9
84.3
78.5
70.2
$4.1
85.7
Min. AUC
67.1
80.5
75.6
65.5
79.1
83,5
7.5
18.1
11.3
11.9
DPD
17.2
6.4
8.5
2.7
10.5
9.9
Ovenal AUC
83.4
84.3
81.3
85.2
83.2
87.5
Min. AUC
78.5
76.2
81.5
80.5
Gap
CheXpert
EOddaD
23.3
34.1
38.9
28.1
27.6
14.4
DPD
9.1
10.2
10.5
8,3
18.9
9.2
Overall AUC
84.1
81.1
80.5
85.4
82.5
86.7
Min. AUC
79,4
Gap
2.5
2.6
4.2
EOdddD
13.2
12.6
10.3
10.3
19.7
10.1
DPD
4.1
8.9
10.2
7.6
12.4
7.5
Overall Score
68.1
75.6
75.8
80.6
83.9
Avg.
Avg. Gap Score
6.6
7.7
7.3
7.3
6.9
Avg. BOddD Score
29.4
28.4
24.0
22.6
24.5
20.8
DPD Score
20.8
13.7
13.8
8.9
19.2
12.7
Arg.
Avg. Min. Rank
5.3
4.7
3.8
3.1
Avg. Cap Rank
3.2
3.6
4.1
3.9
3.9
2.1
Avg. FO4 DirT Rank
4.5
4.1
3.1
3.3
3.4
2.6
DPD Rank
3.4
4.4
Table 4: Evaluation of fair generalisation across medical imaging benchmarks when using self- supervised pre- training (masked autoencoder MAE). FairTune obtains the best average ranking in terms of all fairness metrics.
21

---

Published as a conference paper at ICLR 2024
Model
Full-FTT Linear-Readout
Afteation- Tuning
LayerNorm-Tuning ESCL
Supervised VIT Buse
Overall AUC
83.6
80.3
84.4
83.2
80.7
86.8
76.3
Gap
8.1
7.7
9.5
8,8
6.1
8.1
EOddsD
22.9
22.6
18.3
12.6
DPD
11.3
3.7
11.5
Overall
85.5
Min AUC
79.1
74.9
77.4
75.1
79.5
80.1
Gap
7.1
7.7
9.5
7,8
7.1
EOddsD
11,4
DPD
12.5
4.1
10.8
6.1 9.5
8.5
Table Table showing the results for the VIT Base model pre-trained in a supervised (ImageNet) and self-superrised (MAE) fashion for the overlapping sensitive attributes in the CheXpert dataset. FairTune achieves the best performance in terms of the AUC (overall and min sub-group) and dif- ference of equalized odds (EOddsD) across both the pre-training scenarios.
FairTune: Attention Tuning (12-bit)
FairTune: LayerNorm Tuning (12-bit)
FairTune:
FairTune:
Dataset
Sens Attr
Metric
(36-bit)
(36-bit)
Obj: Overall AUC
Obj: Min Group AUC
Fitz
Overall
95.9
94.0
96.1
96.7
Type
93.4
95.2
96.1
Gap
3.8
2.2
3.2
2.4
Overall
93.9
93.0
92.8
94.0
HAM10000
Age
Min.
87.6
88.4
86.1
90.1
Gup
5.8
9.3
Overall
93.6
91.6
91.7
94.8
Gender
Min.
93.4
91.1
93.2
94.4
Gap
0.9
0.7
1.0
0.9
Overall
84.2
88.1
88.2
88.6
Age
84,5
85.4
Papila
Gup
2.2
4.4
3.0
4.0
Overall
88.4
90.1
92.3
91.8
Gender
Min.
85.3
83.1
87.2
90.2
Gap
8.0
5.5
3.6
Overall
71.4
68.6
71.4
72.6
Age
Min.
65.8
67.3
68.4
70.1
OL31
Gap
9.8
1.9
6.2
3.6
Overall
76.8
72.9
75.6
78.2
Gender
70.1
72.0
75.4
Gup
2.7
4.0
6.1
3.7
Overall
69.3
66.5
73.8
74.5
Age
Min.
67.7
63.1
72.1
73,2
Onsis
Gap
5.0
1.9
1.7
Overall
64.4
69.2
70.8
Gender
Min.
62.4
61.3
63.9
65.5
Gup
7.5
6.2
8.6
6.2
Avg. Overall Rank
2.8
3.7
2.3
1.1
Avg. Min. Rank
3.6
2.3
1.1
Avg. Gap
2.3
3.2
Table 6: Ablation study on the design of the FairTune algorithm full of results. Results high- lighted in bold indicate the best minimum sub-group performance for each dataset and sensitive attribute.
22

---

Published as a conference paper at ICLR 2024
Dataset
Access
Fitzpatrick177 17k
HAM10000
PAPILA
OL31
Oasis-1
Harvard-GF3300
CheXpert
Table 7: URLs access the datasets.
Dataset
Modality
Size
Sensitive Attribute
PAPILA
Fundus Image (2D)
420
Age, Gender
OL311
Heart CT (2D)
8139
Age, Gender
HAM10000
Skin Dermatology (2D)
9948
Age, Gender
OASIS
Brain MRI (3D ->2DD
12156
Age, Gender
Fitzpatrick 17K
Skin Dermatology (2D)
16012
Skin Type
4F3300
Retinal Nerves (2D)
3300
Age, Gender, Race
CheXpert
Chest Radiographs (2D)
222,793
Age, Gender
Table 8: Table representing detailed statisties of the datasets including the modality, dataset size obtained after filtering for incomplete studies, and sensitive attributes present in the metadata.
Label
Label
Label 1
Split
Attribute
Skin Type
7538
1312
Train
Skin Type
3541
418
Skin Type
940
159
Validation
Skin Type
456
46
Skin Type 0
934
180
Test
Skin Type
443
45
Table 9: Sensitive Attribute v/s Label Distribution for Fitzpatrick dataset.
Label
Label
Label
Label
Split
Label
Label
Split
Attribute
Attribute
Age Group 0
5218
545
Male
3609
725
Train
Train
Age Group
1618
586
Female
3224
406
Age Group 0
604
69
Male
419
104
Validation
Validation
Age Group
219
97
Female
39y
Age Group 0
653
70
Male
464
79
Test
Test
Age Group
208
71
Female
395
62
(a) Sensitive Attribute: Age
(b) Sensitive Attribute: Gender
Table 10: Sensitive Attribute v/s Label Distribution for HAM10000 dataset.
23

---

Published as a conference paper at ICLR 2024
Label
Label
Attribute
Label o
Label
Split
Attribute
Label
Label
Split
Age Group 0
Age Group
3836
1716
2392
1768
Train
Female
Male
4076
1476
1924
2236
Train
Age Group
468
312
Validation
Male
208
Validation
Age Group 1
312
104
Female
572
312
Age Group 0
364
260
Male
260
104
Test
Test
Age Group
260
Female
(a) Sensitive Attribute: Age
(b) Sensitive Attribute: Gender
Table 11: Sensitive Attribute v/s Label Distribution for PAPILA dataset.
Attribute
Label
Label 0
Label
Split
Attribute
Label
Label
Label
Split
Age Group
Male
1984
Trin
Train
Age Group 1
1487
141
Female
3015
117
Age Group 0
830
13
Validation
Male
498
32
Validation
Age Group
Female
Age Group 0
1060
Male
39
Age Group
478
Test
Female
909
32
Test
(a) Sensitive Attribute: Age
(b) Sensitive Attribute: Gender
Table 12: Sensitive Attribute v/s Label Distribution for OL3I dataset.
Attribute
Label
Label
Label
Split
Attribute
Label
Split
Age Group 0
3836
1768
Male
1476
1924
Train
Train
Age Group
Female
4076
2236
Age Group 0
Age Group I
468
312
312
104
Validation
Male
Female
572
208
312
104
Validation
Group 0
364
260
Male
260
104
Test
Age Group
260
364
Female
364
520
(a) Sensitive Attribute: Age
(b) Sensitive Attribute: Gender
Table 13: Sensitive Attribute v/s Label Distribution for OASIS dataset.
24


Reviews:
[{"review_id": "DpHMiNZWVa", "reviewer": "Reviewer_Dgwu", "summary": "The paper introduces FairTune, a fine-tuning method for pre-trained models that aims to improve fairness with respect to sensitive attributes. The contribution lies in developing a technique that minimizes disparities in model performance between different demographic groups while maintaining high overall predictive accuracy. The method is demonstrated across various datasets and benchmarks, particularly in medical imaging, using the AUROC metric for evaluation.", "questions": "1. The code link is not available.\n2. Can the authors examine the proposed FairTune on dataset with larger \"Gap\"? In Table 1, the Gaps for the datasets are relatively small.  Some improvements were limited, compared with full fine-tune. \n3. Can the authors provide insights into the computational overhead introduced by FairTune compared to traditional fine-tuning methods?\n4. What are the scalability considerations for applying FairTune to very large datasets or models?\n5. How sensitive is FairTune to the choice of sensitive attributes, and can it adapt to scenarios with multiple overlapping sensitive categories?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. FairTune provides a new pathway and improvement in reducing bias in AI models.\n2. The paper conducted extensive testing over multiple datasets.\n3. It leverages an ablation study to show the effectiveness of each component of the tuning process.", "weaknesses": "The paper may not fully address the computational costs or scalability issues associated with FairTune. Please see the questions for more details.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "piuike8Gwd", "reviewer": "Reviewer_gkxu", "summary": "This paper addresses the challenge of minimizing demographic bias in AI models used for medical diagnosis. The authors highlight the fairness generalization gap, where deep learning models can fit training data perfectly and exhibit fairness during training but show bias during testing when performance differs across subgroups. To tackle this issue, they propose a bi-level optimization approach called FairTune. FairTune optimizes the learning strategy based on validation fairness by adapting pre-trained models to medical imaging tasks using parameter-efficient fine-tuning techniques. The authors demonstrate empirically that FairTune improves fairness on various medical imaging datasets.", "questions": "Please refer to point 1, 2, and 3 in the weaknesses to provide more convincing empirical evidence.\n\nMoreover, I noted that the code repository mentioned in the abstract has not been established. Providing access to the implementation code would greatly enhance the comprehensibility of this research during the review process.\n\n----------------\nPost-rebuttal comments: Thank the authors for addressing my concerns. The additional empirical evidence enhances the experiments in this work. Therefore, I would raise my rating to 'marginally above the acceptance threshold.'", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper recognizes the fairness generalization gap, where deep learning models exhibit perfect fairness during training but bias emerges during testing when generalization performance differs across subgroups.\n\n2. This work introduce a parameter-efficient fine-tuning technique as an effective workflow for adapting pre-trained models to downstream medical imaging tasks.\n\n3. The paper is easy to follow.", "weaknesses": "1. The absence of widely-used fairness metrics, such as Demographic Parity Difference [1,2,3] and Difference of Equalized Odds [1], in this work raises concerns about the completeness of the evaluation. Including these fairness metrics is essential for making the results more convincing.\n\n2. The benchmarking presented in the study appears to be incomplete. To provide a comprehensive comparison, it is advisable to include at least two additional fairness-aware methods in the experiments: Fair Supervised Contrastive Loss [4] and Group Distributionally Robust Optimization [5].\n\n3. Considering the relevance of MedFair [6], which evaluates fairness across various datasets, especially the significant CheXpert dataset for assessing fairness in medical applications, it would be beneficial to adhere to the experimental protocol and employ CheXpert for evaluating the proposed FairTune.\n\n4. It is worth noting that bi-level optimization can be computationally intensive and time-consuming due to the iterative optimization required in both inner and outer loops.\n\n5. When optimizing for fairness during fine-tuning, there is a potential concern regarding the impact on generalization performance, especially for unseen data or different subgroups. It would be valuable to clarify whether there are mechanisms in place to mitigate any adverse effects on generalization.\n\nReferences:\n\n[1] Alekh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, and Hanna M. Wallach. A reductions approach to fair classification. In ICML, volume 80 of Proceedings of Machine Learning Research, 60–69. PMLR, 2018.\n\n[2] Alekh Agarwal, Miroslav Dudík, and Zhiwei Steven Wu. Fair regression: quantitative definitions and reduction-based algorithms. In ICML, volume 97 of Proceedings of Machine Learning Research, 120–129. PMLR, 2019.\n\n[3] Solon Barocas, Moritz Hardt, and Arvind Narayanan. Fairness and Machine Learning. fairmlbook.org, 2019.\n\n[4] Sungho Park, Jewook Lee, Pilhyeon Lee, Sunhee Hwang, Dohyung Kim, Hyeran Byun; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 10389-10398\n\n[5] Sagawa, S., Koh, P. W., Hashimoto, T. B., & Liang, P. (2019, September). Distributionally Robust Neural Networks. In International Conference on Learning Representations.\n\n[6] Zong, Y., Yang, Y., & Hospedales, T. (2022, September). MEDFAIR: Benchmarking Fairness for Medical Imaging. In The Eleventh International Conference on Learning Representations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "R3jMGQPtuU", "reviewer": "Reviewer_XV7U", "summary": "The paper focuses on an important field in AI, which is achieving group fairness in models, especially in medical diagnosis. They argue that this is essential but challenging due to the fairness generalisation gap where bias emerges during testing. The authors introduce a bi-level optimisation approach called FairTune, which optimises parameter-efficient fine-tuning (PEFT) techniques to balance model fit and fairness generalisation. The empirical results in the paper show that the proposed method enhances fairness across multiple medical imaging datasets.", "questions": "Please see weakness.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "The paper proposed a new method to finetune the pretrained model, which is potentially benefits and convenient to the current hype of foundation models or large models that require large-scale pretraining. \n\nThe proposed PEFT achieves the best performance when compared with other fairness finetuning approaches. \n\nThe paper is well written and motivates clearly as well.", "weaknesses": "Given the model is proposed for finetuning a pre-trained model, could the authors provide some results that using the proposed approach on finetuning Masked Autoencoder or MOCO to see if this can improve fairness for self-supervised pretraining? \n\nFor evaluation metrics in fairness, DPD and DEOdds are very common to validate an algorithm's fairness, could the authors evaluate their methods on some of the datasets using those two metrics? \n\nThe datasets compared only contains a limited number of attributes, could the authors compare their approaches to fairness medical dataset containing more sensitive attributes such as the \"Luo, Yan, et al. \"Harvard Glaucoma Fairness: A Retinal Nerve Disease Dataset for Fairness Learning and Fair Identity Normalization.\" arXiv preprint arXiv:2306.09264 (2023).\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pd70CuFVSA", "reviewer": "Reviewer_iDy7", "summary": "This paper aims to promote the group fairness of the deep learning model in medical image analysis, with a specific focus on validation fairness. The authors propose a parameter-efficient fine-tuning method to update parameters regarding fairness. The proposed method is validated on five medical imaging datasets and outperforms compared methods.", "questions": "- Why the metric for fair learning is to minimize the largest loss of a subgroup instead of pursuing a uniform loss distribution among subgroups?\n- How to explain the differences between masks by using different optimizing objectives?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- This paper studies promoting group fairness, which is an important topic.\n- The motivation is well demonstrated.\n- The overall framework design is easy to follow.\n- The proposed method outperforms the compared methods.", "weaknesses": "- In the method part, the authors limit the method fairness in binary classification. It has not mentioned how to extend the methodology for multi-classification.\n- The first challenge of PEFT is related to the dataset itself, which is not a challenge for fairness.\n- The method details are not clear. E.g., how to solve the BLO problem by using TPE with SH.\n- It is not clear how to split the train/val/test data.\n- Since this method utilizes validation data to tune the model, it is not proper to report the validation AUC; instead, test AUC should be reported.\n- The experiment only validates the AUC within subgroups, more comprehensive metrics are expected (e.g., equal opportunity.)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)