FairSeg: A Large-Scale Medical Image Segmentation Dataset for Fairness Learning Using Segment Anything Model with Fair Error-Bound Scaling
Authors: Yu Tian
Min Shi
Yan Luo
Ava Kouhana
Tobias Elze
Mengyu Wang
Venue: ICLR 2024 poster (2024)
Keywords: Medical Segmentation
Medical Imaging
Fairness Learning
Health Equity
Deep Learning
Trustworthy AI
Primary area: societal considerations including fairness, safety, privacy
TL;DR: We curate and release the first large-scale fairness dataset for medical segmentation.

Abstract: Fairness in artificial intelligence models has gained significantly more attention in recent years, especially in the area of medicine, as fairness in medical models is critical to people's well-being and lives. High-quality medical fairness datasets are needed to promote fairness learning research. Existing medical fairness datasets are all for classification tasks, and no fairness datasets are available for medical segmentation, while medical segmentation is an equally important clinical task as classifications, which can provide detailed spatial information on organ abnormalities ready to be assessed by clinicians. In this paper, we propose the first fairness dataset for medical segmentation named Harvard-FairSeg with 10,000 subject samples. In addition, we propose a fair error-bound scaling approach to reweight the loss function with the upper error-bound in each identity group, using the segment anything model (SAM). We anticipate that the segmentation performance equity can be improved by explicitly tackling the hard cases with high training errors in each identity group. To facilitate fair comparisons, we utilize a novel equity-scaled segmentation performance metric to compare segmentation metrics in the context of fairness, such as the equity-scaled Dice coefficient. Through comprehensive experiments, we demonstrate that our fair error-bound scaling approach either has superior or comparable fairness performance to the state-of-the-art fairness learning models. The dataset and code are publicly accessible via https://ophai.hms.harvard.edu/datasets/harvard-fairseg10k.

Published as a conference paper at ICLR 2024
FAIRSEG: A LARGE-SCALE MEDICAL IMAGE SEGMENTATION DATASET FOR FAIRNESS LEARNING USING SEGMENT ANYTHING MODEL WITH FAIR ERROR-BOUND SCALING
Yu Tian Min Shi Yan Luo Ava Kouhana' Tobias Elze! Harvard Ophthalmology Al Lab, Harvard University {ytian11, mshii, yluo16 akouhana,
Mengyu Wang'
toobias tobias_elll belze, mengyu Sanggyu_wang]oooo @meei hww.ahar
ABSTRACT
Fairness in artificial intelligence models has gained significantly more attention recent years, espocially the area of medicine, as fairness in medical models is critical to people's well-being and lives. High- quality medical fairness datasets are needed to promote fairness learning research. Existing medical fairness datasets are all for classification tasks, and no fairness dalasets are available for medical segmentation, while medical segmentation is an equally important clinical task as classifications, which can provide detailed spatial information ready to be assessed by clinicians. In this paper, we propose the first fairness dataset for medical segmentation named Harvard-FairSeg with 10,000 subject samples. In addition, we propose a fair error- error-bound scaling approach to reweight the loss function with the upper arror-bound in each identity group, using the segment anything model (SAM). We anticipate that the segmentation performance equity tackling the hard cases with high training errors each identity group. To utilize novel equity-scaled sogmentation
organ abnormalities
be improved by explicitly
facilitate fair comparisons, performance metric to compare segmentation metrics in the context of fairness, such as the equity-scaled Dice coefficient. Through comprchensive experiments, we demonstrate that our fair error-bound scaling approach either has superior or comparable fairness performance to the state-of the art. fairness learning models. The dataset and code are publicly accessible via httpss///phaaii hms. harvard, edu/ daadddatttataaaaarrrrrraa wwwaaavaadd---iraeglllll fairseg10k
INTRODUCTION
As the use of artificial intelligence grows in medical image diagnosis, it's vital to ensure the fairness of these deep learning models and delve into hidden biases that could arise in complex real-world scenarios. Regrettably, machine learning models can inadvertently incorporate scnsitive attributes (like race and gender) associated with medical images, which could influence the model's ability to differentiate abnormalitics. This challenge has spurred significant efforts to investigate biases, champion fairness, and launch new datascts in the fields of machine learning and computer vision.
To date, only a few public fairness dalasets have been proposed for studying the fairness classification Dressel Farid (2018); Asuncion & Newman (207 y Wightman (1998 Miao
(2010): Kuzilek et al. 2)1 Rugg les et. al. (2015); Zhhang el al
2017 Zong et al. 2022
Irvin et al. 2009 Johnson
al. (2019): Kovalyk et al. (2022
Predominantly, most of
those datasets Dressel & Farid (2018): Asuncion & Newman 20077: Wightman 1998 ;
Miao 2010 Kuzilek et al. (2017): Ruggles et al. (2015); Zhang et al. (2017) consist
of tabular data, making them ill-suited for developing fair computer vision models that
demand imaging data. This gap is concerning, especially considering the rise of impactful
"Contributed equally as co-first authors

---

Published as a conference paper at ICLR 2024
deep-learning models dependent on such data. In the field of medical imaging, only handful of datasets Irvin et al. (2019) ; Johnson et al. (2019); Kovalyk et al. (2022) have been used for fairness learning. However, most of These datasets were not explicitly crafted with fairness modeling. They often include only limited range of sensitive attributes, like age, gender, and race, thus constricting the scope of examining fairness among varied demographic groups. Additionally, they also lack a thorough benchmarking framework. More importantly, while those previous datasets and approaches offer solutions for medical classification, they overlook the arguably more critical field of medical segmentation.
Medical segmentation Liu et al. 20022); Tian et al. (2023a); Chen et al. (0021 Tian et al. (202bb;; Zhou et al. (2018); Wang et al. 2022a] provides detailed spatial information on specific anatomical structures, essential for personalized treatments, monitoring discase progression, and enhancing diagnostic accuracy. In contrast to classification's broad categorization, segmentation's 's precise delincation offers in-depth information about organ abnormalities, facilitating better patient care from diagnosis to therapeutic interventions. Despite its significance, there's a glaring absence of public datasets for studying fairness in medical segmentation across diverse sensitive attributes. Given the importance of ensuring fairness medical segmentation and the special characteristics of medical data, we argue that a large-scale medical segmentation dataset that is designed to study fairness and provide a playground for developing algorithmic debiasing approaches crucial. However, creating such new benchmark for fairness learning presents multiple challenges. First, there's a scarcity of large-scale, high-quality medical data paired with manual pixcl-wise annotations, both of which are labor-intensing and timo-consuming to collect and annotate. Second, existing methods Park et al. (2022); Wang al.] (2022b); Beutel al. (2017) are primarily designed for medical classilication, and the performance remains questionable when adapting to segmentation tasks. It is also uncertain whether the unfairness that exists in the segmentation tasks can be effectively alleviated by algorithms. Lastly, a universal metric for evaluating the fairness of medical segmentation models remains elusive. Furthermore, adapting existing fairness metrics designed for classification to segmentation tasks can also be challenging.
In order to address these challenges, we propose the first large-scale fairness dataset for medical segmentation, named Harvard-Fairde cup segmentation from SLO fundus images for diagnosing glaucoma, as shown in Figure [] Glaucoma, prominent cause of irreversible global blindness, has a preva lence of 3.54% in the 40-80 age bracket, affecting roughly 80 million individuals Tham et al. (2014); Luo et al. (2023b); Shi et al. (2023a); Luo et al. (2023a); Shi et al. (2023c). Despite its significance, carly glaucoma often remains asymptomatic, emphasizing the need for timely professional
which is designed for fairness optic disc and
tests. Accurate segmentation of the optic disc and cup is crucial for early glaucoma diagnosis by healthcare professionals. It's important note that Black individuals face a doubled risk of developing glaucoma compared to other groups, yet the segmentation accuracy is often lowest for this demographie. This motivates us curate dataset for studying segmentation fairness issues before the practical use of any segmentation models in the real world. Particularly, the highlights of our proposed Harvard-FairSeg dataset are follows: (1) The first fairness learning dataset for medical segmentation. The dataset provides optic disc and cup segmentation with SLO fundus imaging data; (2) The dataset is equipped with six sensitive attributes collected from real-world clinical scenarios for the study of fairness learning problem; (3) We evaluate multiple SOTA fairness learning algorithms on our proposed new dataset with various segmentation performance metrics including Dice coefficient and intersection over union (IoU).
In addition to our valuable dataset, we develop a fair error-bound scaling (FEBS) approach as an add-on contribution to demonstrate that the fairnsss challenges in medical segmentation can indeed be tackled. The core idea of our FEBS approach is to rescale the loss function with the upper training error-bound of each identity group. The rationale is that the hard cases in each identity group may be the driving factors for the underlying performance disparities, and the hard cases in each identity group may be due to pathophysiological and anatomical differences between identity groups. For instance, Asians have more angle-closure glaucoma compared with Whites, and Blacks have a larger cup-to-disc ratio compared with other races. Explicitly tackling the hard cases by using upper crror-bound in each identity group may help reduce model performance incquity. We
2

---

Published as a conference paper at ICLR 2024
Table 1: Public Fairness Datasets Commonly Used in Medical Imaging.
Dataset
Modality
Senstive Attribute
Imunges Sugmentation
ChuNoert
Clese X-ow
MXMI-CCR
Chest X-w (1D)
3701563
PAPILA
Fusius Emsgg (2D)
Agn. Set
120
Sain Demnstolog (2U)
Agn, Set
9,415
Sain Demailskg (2D)
Siin type
16,012
Hort 290)
Age.
5,330
COVID-CT-MD
Lung
Aue.
ADNI
Baaio
Age.
ADNI ST
Bruin MR3 (SD)
Age,
110
Pariend-P SI. Fandus Imagge (2T) Age. Gemdee, Roos, Rénniriy, Proiered langregh and Marital Stotts
10,000
subsequently integrate our proposed FEBS (Fair Error-Bound Scaling) approach with the recent segmentation foundation model, the Segment Anything Model (SAM) Kirillov et al. (222)) to explore whether FEBS enhances segmentation fairness across various sensitive attributes.
To facilitate the comparison between different fairness learning models, we propose equily- scaled performance metrics. More specifically, for instance, the ES-Dice calculated as the overall Dice coefficients divided by the sunmation of the relative disparity between the overall Dice coefficients and the group Dice coefficients. This equity-scaled segmentation performance metric provides more straightforward evaluation and is easier to interpret by climicians than existing fairness metrics such as demographic parity difference (DPD) and difference in equalized odds (DEOdds).
Our core contributions are summarized as follows:
We introduce the first fairness dataset for medical segmentation and benchmarked it with the state-of-tho-art (SOTA) fairness learning approaches.
Major:
Minor 1: We develop novel fair error-bound scaling approach to improve segmentation performance equity.
equity-scaled segmentation performance metric to facilitate fair comparisons between different fairness learning models for medical segmentation.
Minor 2: We design
2 RELATED WORK
Medical Fairness Datasets. Healthcare disparities in disadvantaged minority groups have been a significant concern due to heightened disease susceptibility and underdiagnosis. Although deep learning provides solutions for addressing this by automating disease detection, it is essential to address the potential for performance biases. Existing medical fairness datasets such as CheXpert Irvin et al. (2019), MIMIC-CXR Johnson al. (2119). and Fitzpatrick1 7k Groh et al. (2021) i focus on image classification and often neglect the vital domain of medical segmentation. Additionally, their limitod sot of sensitive attributes like age, sex, and race reduces versatility in fairness system development. In this work, we introduce the first pegnentationnfocuusee medical fairness dataset, encompassing attributes like age, gender, race, and ethnicity in segmentation tasks.
Fairness Learning. Biases in computer vision datasets arise from data inequalities, leading to skewed predictions. Strategies to mitigate these biases are categorized into pre-processing, in-processing, and procespro.sssigg Pre- Pre-processing methods, like those in Quadrianto let al. (2019): RRamaswamy et al. (2021): Zhang Sang (2020); Park et al. (2022), ), de-
bias training data, but may compromise computational efficiency. In-processing methods integrate fairness during model training but might sacrifice accuracy due to loss function manipulations Beutel et al. (2]222117) Roh et al. (2020): Sarhan et al. (2020); Zafar et al. (2017); Zhang et al. (2018); (Shi et al. (202bb) Post- cest-proeessing methods ofter corrective measures but have limitations during testing Wang et al. (2022b); Kim et al. (2019; Lohia et al. (2019). Our work proposes novel fair orror-bound scaling approach, hypothesizing that addressing hard cases within identity groups could enhance model fairness performance.
Fairness Metrics. Fairness definitions vary based on application context. Group fairness, as described in Verma & Rubin (2018), ensures impartiality within demographic groups but may sometimes compromise individual fairness. This trade-off conflict with
3

---

Published as a conference paper at ICLR 2024
medical ethics, emphasizing the need for methods that prioritize both group and individual
fairness
Beauchamp (2003).
Some argue that Max-Min fairness complements group fairness
metrics Zong el. al. [2022].
Common fairness metrics include Demographic Parily Difference
(DPD) Bickel et al.
1975 Agarwal et al. (2018) 2019), Difference in Equal Opportunity (DEO) Hardt et al. (2066) and Dillerence in Equalized Odds (DFOdds) Agarwal et al.
(2018). In crucial medical scenarios, prioritizing group fairness over accuracy not. viable. We introduce an equity scaling mechanism, aligning segmentation accuracy with group
fairness, offering comprehensive perspective on both performance and fairness.
3 FAIR DISC-CUP SEGMENTATION
Disc-Cup Segmentation Framework. In ophthalmologe disc-cup segmentation serves as a foundational step in assessing the optic nerve head structures and diagnosing glaucoma in its early stage. We represent a scanning laser ophthalmoscopy (SL.)) fundus image as x C HR xc, where x W denotes the spatial resolution and C signifies the channel count. Our objective is to predict a segmentation map, S, with a resolution of H x W. Every pixel in this map is associated with class from the set y = {yo, y1, 12 where yo represents the background, y/ denotes the optic disc, and y/2 indicates the cup. The overall
objective is to construct model fo Rd S S capable of predicting this segmentation mask 8, where 0 denotes the segmentation model's parameters. The segmentation model fo can be implemented using various SOTA models, such as SAM Kirillov et al. 2023 and
TransUNet Chen et al. (2021).
This task is important bocause vertical cup-to-disc ratio (CDR) Jonas al. (2000) is accepted and commonly used by clinicians as a prime metric for the early detection of glaucoma. is computed by determining the ratio of the vertical cup diameter (VCD) to the vertical disc diameter (VDD). An SLO fundus image vividly portrays the optic disc (OD) distinguishable bright. white/gra oval in its center, split into the inner bright optic cup (OC) and the outer neuroretinal rim. An increasing CDR generally hints al a larger risk of glaucoma, underscoring the importance of precise disc and cup segmentation.
Setting out with an ambition of fair segmentation, our FairSeg framework aims to ensure consistent and unbiased disc-cup delineation across diverse demographic groups. In practice, we observe significant bias between different demographic groups, and this can be caused by many factors (e.g., healthy subjects from the Black racial group tend to obtain slightly larger CDR than other groups). This venture is not merely a technical pursuit but carries profound societal implications, making it crucial to infuse fairness in our methodology.
Incorporating Fairness in Segmentation. Attaining accurate segmentations remains pivotal; however, ensuring the system's fairesss across various demographic groups also amplifies its clinical importance. Thus, in our Harvard-FairSeg dataset, each image comes with an associated sensitive attribute a E A. This attribute embodies critical social demographics such as race A = (Asian, Black or African American, White or Caucasian), gender A = (Female, Male), ethnicity A = (Non-Hispanic, Hispanic), and preferred language A (English, Spanish, Others). For simplicity, we digitize these categories to A = (0, 1,2] or A = {0. Incorporating the fairness learning paradigms into disc-cup segmentation, goal extends to training fo not only for segmentation prccision but also to ensure equitable performance across the diverse sensitive attributes.
DATASET ANALYSIS
Data Collection and Quality Control. Our institute's institutional review board (IRB) approved this study, which followed the principles of the Declaration of Helsinki. Since the study was retrospective, the IRB waived the requirement for informed consent from patients.
The subjects tested between 2010 and 2021 are from a large academic eye hospital. There are three lypes of data to be released in this study: (1) SLO fundus imaging scans; (2) patient demographics; and (3) disc-cup masks annotated automatically from the OCT machine and manually graded by professional medical practitioners. Particularly, the pixel annotations of disc and cup regions are first acquired from the OCT machine, where the disc border in 3D
4

---

Published as a conference paper at ICLR 2024
Layer Segmentation Boundaries
Derive and and Cup Borders
Ground Truth Segmentation Boundaries
Leal to dead TUARERs
OCT B-scans
Single OCT B-scan
OCT Fundus Image
SLO Fundus Image
The process to obtain ground truth disc and cup boundaries on the SLO fundus image. The OCT and SLO fundus images have been previously registcred using NiftyReg.
Figure 1:
OCT is segmented as the Bruch's membrane opening by the OCT manufacturer software, and the cup border is detected as the intersection between the inner limiting membrane (LLM) and the plane that results in minimum surface area from the intersection and disc border the plane Mitsch et al. (2019); Everett & Oakley (2015). Approximately, cup border can be considered as the closest location on the ILM to the disc border, which is defined as the Bruch's 's membranc opening. Both Bruch's 's membranc opening and the internal limiting membrane can be easily segmented due to the high contrast between them and the background. Since the OCT manufacturer software leverages 3D information, the disc and cup segmentation is generally reliable. comparison, 2D disc and cup segmentation on fundus photos can be challenging due to various factors including attenuated imaging signal and blood vessel obstruction. However, OCT machines are fairly expensive and less prevalent in primary care, therefore, we propose to migrale those annotations from 3D OCT to 2D ST.O fundus for potentially broader impact in early-stage glaucoma screening in the primary care domains. Specifically, we first align the SLO fundus images with OCT- derived fundus (OCT fundus) images utilizing the NiftyReg registration tool Modat et al. (201)). Subsequently, the affine metric from NiftyReg is applied to the disc-cup masks of the OCT fundus images, aligning them to the SLO fundus images. This procedure effectively produces a plethora of high-caliber SLO fundus mask annotations, sidestepping the labor- intensive manual pixel annotation process. It's noteworthy that this registration operation demonstrates considerable precision in real-world scenarios, as evident from our empirical observations that highlight approximate 80% success rate in registrations. Post this automated process, the generated masks undergo rigorous examination and are hand-graded by a panel of five medical professionals to ascertain the precise annotation of the disc-cup regions and exclude masks with incorrect disc or cup locations and failed registrations.
Data Characteristics. Our Harvard- FairSee dataset contains 10,000 samples from 10,000 subjects. We divide our data into the training set with 8,000 samples, and the set with 2,000 samples. The dataset's collective age average is 60.3 16.5 years. Within this dataset, six sensitive attributes including age, gender, race, ethnicity, preferred language, and marital status, are included for in-depth fairness learning studics. In particular, regarding racial demographics, the dataset includes samples from three major groups: Asian, with 919 samples; Black, with 1,473 samples; and White, with 7,608 samples. Gender-wise, females constitute 58.5% of the subjects, with the remainder being males. The ethnic distribution highlighted by 90.6% Non-Hispanic, 3.7% Hispanic, and 5.7% unspecified. In terms of preferred language, 92.4% of the subjects prefer Euglish, 1.5% prefer Spanish, 1% prefer other languages, and 5.1% remain unidentified. From a marital status perspective, 57.7% are in marriage partnered, 27.1% are single, 6.8% have experienced divorce, 0.8% are legally separated, 5.2% are widowed, and 2.4% are not specified.
5 METHODOLOGY FOR UNDERSTANDING DEBIASING
In this section, we introduce our proposed fair error-bound scaling approach, which modifies the Dice loss to adjust pixel-wise weights of samples from different sensitive groups. Moreover, after recognizing the potential shortcomings of standard fairness metrics in medical contexts is primarily their potential discord with overall performance (i.e., Dice for segmentation AUC for classification) and their sensitivity to data quality, we further
5

---

Published as a conference paper at ICLR 2024
advocate for an equity scaling metric for segmentation tasks. This metric integrates demographic-deeendeen equity considerations with primary segmentation performance metrics, including Dice and loU.
Fair Error-Bound Scaling. As aforementioned, we denote an image x E X C RHHWWC and our objective is to prodict a segmentation map, S, with a resolution of H W. Every pixel in this map is associated with a class from the set y = {Bo. V1, 32 where Yo represents the background, 8/1 denotes the optic disc, and 322 indicates the cup. Each image comes with an associated sensitive altribute E A. This attribute embodies critical social demographics that have been introduced in Section. 3 We hypothesize that the group of samples that obtain smaller overall dice loss means the model learns better for the samples from that certain group, and therefore, requires smaller weights for these groups of samples. While, in contrast, the group with larger overall dice loss (i.e., hard cases) may lead to worse generalization and cause more algorithmic bias, which requires larger learning weights for these groups of samples. Therefore, we propose a new fair grror-bount scaling method for scaling the dice losses between different demographic groups during training. We first define the standard Dice loss between predicted pixel scores 9 and the ground truth targets y is represented as:
D(0,y) = 11 2 x ((D.8) +e
(1)
where the intersect function I calculates the clement-win wise multiplication between the predicted pixel scores and the target y.
To ensure fairness across different attribute groups, we augment the Dice loss above with novel Fair Error-Bound Scaling mechanism. This results in the loss function below:
Liair Conir v. a)
1 D
Σ
24 x D(j x Wa, W:);
(2)
where 24 denotes the weight for target class i, and Wa is the weight list corresponding to particular attribute. The weight list W for the sensitive attribute E A can be formulated
(mmin))) 121 D D(H(a)..lol.)) )
=a= = tanh
(3)
ΣP' D(g(a)a yla)))
By modulating predicted pixel scores with these altribute weights, this loss ensures that different attribute groups contribute balancedly to the loss function during model training, thereby promoting fairness.
Equity-Scaled Metric for Fair Segmentation. In the field of medical image segmentation, the need to address demographic fairness has become increasingly paramount. Traditional segmentation metrics like Dice and loU offer insights into the segmentation performance but may not capture the fairness across diverse demographic groups effectively. Considering this, our goal is to present a new metric that encapsulates both the efficiency of the segmentation and its fairness across different demographics. This gives rise to holistic perspective, ensuring models that are both accurate and fair.
Let I E (Dice, ToU, represent segmentation metric. Conventional segmentation metrics, like Dice and ToU, can be expressed as M({(2',y))) where they assess the overlap between the predicted segmentation z' and the ground truth y. However, these metrics often overlook the demographic-sennstive attribute information of the samples. Inspired by Luo et al. (2023c), to incorporate group-wise fairness, we need to evaluate the performance across each demographic group individually. Let's denote the set of all demographic groups as A. To integrate demographic fairness, we first define a performance discrepancy A for segmentation metrics, similar to Equation (4) as follows:
A
Σ ([((E')))) I([(2' a, AA]]
(4)
AEA
Here, A measures the aggregale deviation of each demographic group's performance from the overall performance. It. approaches zero when all groups achieve similar segmentation accuracy.

---

Published as a conference paper at ICLR 2024
Table 2: Optic Cup and Rim segmentation performance on the Harvard-FairSer dataset
with race as the sensitive attribute.
Overall ES-Dine
Overall ES-laU"
Method
Dice"
InU]
Diss
Dics"
Diae
InU]
IaU]
laU!"
SAMee
0,5612
0.867]
0.745
0.7813
0,5085
0.5730
0,5070
0.7885
0.7005
0.7905
IADY
0.7702
0.7840
0.5000
05501
0.8705
0.7008
6.7822
0.7849
SAMad ChospDRO
S PAM
018556
(16671
0.7935
0.8567
0.7716
6.7822
0.791
Noe
0.5281
0J441
0.7300
0.7532
0
05380
0,8505
0.7277
0.7576
0.7551
The
0,8216
0.7012
0.8215
05117
85226
0.7266
6:2382
0.783
UNet PEDS
0,8255
0.8864
0.7355
0.7497
0.5484
0.5054
0.7347
0,7570
0.7013
SAMee
0,7478
0.221
0.7217
0,7800
077255
0.5644
0.6745
0,6687
0.7213
SA33e
0,7234
05216
0.7138
0.7801
0.7091
05206
0.0636
0.722
SAMlii-Cherppiit
S SAMed4 PRRS
0.7529
0J5325
0.6451
0.7280
0.7852
0.7789
0
0.6825
0.0820
0,7459
Noe
0.7031
0.7977
0.8845
0.6766
0.7157
0.7317
0.508
0.816
0.5001
0.6913
Noe ADV
0.7030
(17866
8.5055
0,863
6.111
0.786
8.987
0.8166
6.5882
0.6884
0.770
UNt PEDS
0.7050
0.7000
0.6735
0.7335
0.8180
0.6185
0.6020
0.0935
When we consider the fairness across different groups, we need to compute the relative disparity between the overall segmentation accuracy and that within each demographic groups. With this, the Equity-Scaled Segmentation Performance (ESSP) metric, ESSP, is defined as:
I(((=',
ESSP =
(5)
1 1+4
This formulation ensures that ESSP is always less than equal to I. As A diminishes (indicating equitable segmentation performance across groups), ESSP converges to the traditional metric I. Conversely, a higher A signifies greater disparity in segmentation performance across demographics, resulting in lower ESSP score. This approach allows us to evaluate segmentation models not only for their accuracy (as measured by Dice, IoU, etc.) but also for their fairness across different demographic groups. This makes the ESSP scoring function a pivotal metric in ensuring both segmentation accuracy and fairness in medical imaging tasks. Please note that an alternative method for gauging equity- scaled segmentation ion performance involves directly utilizing standard deviation to quantify disparities across different groups. However, this approach, reliant on standard deviation, tends to yield only subtle variations between overall and equity-scaled performances. Such minor discrepancies can restrict its effectiveness in accurately quantifying fairness across various algorithms. For comprehensive insights, we also include results derived from this standard beaied metric in the appendix.
EXPERIMENT & ANALYSIS
6.1 ALGORITHMS
Segmentation Models. We select two segmentation models to investigate the fair
segmentation problem, including the recent SOTA SAMed Zhang & Liu (2023) model and
a classic TransUNet Chen et al. (2021). The SAMed approach Zhang & Liu 2023) offers
novel solution for medical image segmentation, drawing inspiration from the expansive capabilities of the recent. Segment Anything Model (SAM) Kirillov et al. 2023 Pioneering new paradigm, SAMed tailors the large-scale SAM image encoder to cater specifically to medical images. By employing the Low-rank- based (LoRA) fine-tuning strategy, SAMed efficiently adapts the SAM image encoder for medical scenarios. The LoRA modules are integrated into both the prompt encoder and mask decoder, and the model is fine-tuned on labeled medical image segmentation datasets. TransUNet Chen et al. (202)) represents a harmonious integration of transformer architectures with the established U-Net design principles, tailored specifically for medical imaging. It combines the strength of CNN backbones for spatial information extraction with the power of transformers to capture long- range dependencics in images. By utilizing the self-attention mechanism of transformers, TransUNet ensures that even distant regions in an image are effectively related, while its U- Net architecture helps preserve vital spatial hicrarchies, making it potent tool for medical image segmentation.
Fairness Algorithms. For our proposed fair segmentation, we employed several state-of- the-art. fairness techniques as baselines:
7

---

Published as a conference paper at ICLR 2024
Table 3: Optic Cup and Rim segmentation performance on the Harvard- dataset with gender as the sensitive attribute.
Overall RS-Diew
Overall Dioe
Overall FS-IoU7
Overall TolF
Male Diee]
Pemale Diow
Male ToU]
Female
Methor
Toll
SAMed
0.8623
0.5671
0.7737
0.7515
0.8617
0.5703
0.7783
0:7855
SAMwA-ATV
0 0.8655
0 3.667
0.7780
0.7585
0.8911
0.9675
0.7791
0:7550
0.5089
0-6071
0.7800
0.7505
0.8672
0.9670
0.7804
0:781
FFM
0.8622
0
0.7756
0.7829
0
0.87358
0.7851
0.7879
0.8436
0.5481
0.7000
0.7582
0.8458
018515
0.7608
0.7564
0,8312
0.8345
0.7268
0.7306
0.8344
0.8365
0.798:
0,7550
Traee GroupDDDD
0.8438
0 3178
0.383
0.2522
0.8611
0.8528
07883
1575
TransUNe++
0.8484
0.8:89
0.7492
0.7590
0.8494
0.5514
0.756
0.7556
SAMod
0.7217
0.8819
SAMod-ADV
0.8244
0.7226
0,8312
0.7276
0.7181
SAMed GroDDRO
0.8320
0.7258
0.7293
RAMod--FF
0.8277
0.7225
TranslNaa
0.7883
0.7927
0.9650
0.6700
0.7961
0.6736
Trans UNa
0.7754
0.7452
0.0630
0.61290
0.0584
Trans UNet RR0
0.7803
0.7017
0.6678
0.0699
0.7930
0.77000
0.0716
0.6677
TransUNet
0.7851
0.7158
0.0653
0,0608
0,7924
0.7932
0.6078
0,0258
Table 4: Optic Cup and Rim segmentation performance on the Harvard-FairSee dataset
with preferred language as the sensitive attribute.
Overoll ES-Dicet
Overall Diec
Overoll ES-loU-
Overall IoUT
English
Spaaish
Otlers Dieci
English IoUi
Spanish loU?
Otlaces
Method
IoU
SAM
0.855
0.671
0.717%
0.7813
0.8022
0/1677
0.581
0.7711
0.831
0.4401
SA ADV
05157
09656
07M6
0.7830
0.8955
0.9131
0.0000
0.7915
0.8432
0.7763
0,8750
05202
87370
0.7847
0
01485
0,8849
0.7825
0.3353
0.8019
SAMod FEDS
0.7405
0.7826
0.5070
0.0011
0.7714
0.7810
0.5481
0.7012
Not ADV
0.7850
08812
01817
0.7225
0.226
0.5815
BS335
0.7901
6.2000
0.7855
Not Gioudit
0,7915
0.416
B12122
0.7422
0.825
0.5864
0.8571
0.7423
0.7005
0.7005
0,4040
0.8611
47000
0.7535
0.0011
0.9012
0.791
0.500P
0.7619
SAMI
0,7952
0.8911
510000
0.7217
0.5810
0.5531
0.7511
0.7465
0.8871
SA ADV
0 7351
0
0.7217
0,897
0.5315
0.7511
0.7053
0.0003
0.7961
0.8111
0.6015
0.723
0,8822
0.5215
0.50U5
0.7258
0.741
0.6954
SAMod FEDS
0.7892
0,5318
0.0600
0.7244
0.5928
0.7912
0.7213
0.5915
0.5105
0.6050
Tand ATV
0.7521
(17881
0.0936
0.7905
0L7961
0.7501
0.0937
0.5717
0.695
Thand GeemmbD
0.7577
0.7857
0.06 5
0.7867
0.9157
0.7076
0.8915
0.0010
0.7551
0.7895
0U6379
0.8958
0.7905
0.5105
0.7861
0.8686
0.0905
0.6421
1. Adversarially Fair Representations (ADV): Introduced by Madras el al. Madras el al. (2018), ADV aims to curate unbiased representations. It refines the model such that sensitive attributes are difficult to deduce from its representations, effectively minimizing inherent biases.
2. Group Distributionally Robust Optimization (GroupDRO): Proposed by Sagawa et al. [Sagawa et al. (2019). GroupDRO seeks to enhance model robustness. It minimizes the maximum training loss across all groups by strategically increasing regularization, ensuring the model doesn't exhibit bias towards any particular group.
These fairness algorithms are incorporated into segmentation methods, specifically SAMed and TransUNet, to assess their capability in enhancing fairness and reducing bias across diverse demographic groups during segmentation.
6.2 TRAINING SETUP
Datasets. As we introduce in Section 4 8,000 SLO fundus images are used for training, and 2,000 SLO fundus images are used for evaluation, with the patient-level split. Apart from the pixel-wise annotation for disc and cup regions, each sample also includes six different sensitive attributes. In this paper, we select four sensitive attributes that obtain the largest group-wise discrepancy in practice, including Race, Gender, Language, and Ethnicity.
Training and Implementation Details. For both SAMed and TransUNer a combined loss function comprising both cross entropy and dice losses is employed as the training loss. The loss function can be defined as
(6) Here, CE and Dice stand for the cross entropy loss and Dice loss, respectively. The loss weights a and A2 help alternate a balance between the two loss terms. In the initial stages of training, we leverage a warmup strategy to stabilize the model training with our Harvard- FairSeg data. We use the AdamW optimizer with exponential learning rate decay. For the backbone of SAMed and TransUNet, we select. the standard ViT-B architecture. We use learning rate of 0.01, momentum of 0.9, and weight decay 1e-4 for TransUNet. For SAMed, set the learning rate to 0.005, with momentum and weight decay set to 0.9 and 0.1,
L C, D(S)) + XsDice(S,
8

---

Published as a conference paper at ICLR 2024
Table 5: Optic Cup and Rim segmentation performance on the Harvard-Fairte dataset with ethnicity as the sensitive attribute.
Overall RS-Diow
Overall Dios
Overall FS-IoU7
Overall TolF
Hisponic Dien]
Non- Hon-Hlepanic Diow
Rieqanic loU1
Non- Non-Hlppanic Toll
Method
SAMed
0.8539
0:5671
0.2575
0,7515
0.8653
0,85884
0.775
0,8100
SAMwA-ATV
0.8400
0 2077
0.7505
0.7514
0.8911
0.5885
0.7791
0.8060
0,8550
0 2018
0.7007
0.7540
0.8622
0.5855
0.78:9
0.8031
FFMMd-FKRSS
0.8560
0.8685
0.7628
0.7845
0.8704
0.8831
0.7904
0.8070
0.8251
0.5481
0.7200
0.1582
0.865
0,8704
0.7508
0.7826
0.8112
0.7085
0.7315
0.83%
[U836]
0.7254
0,7622
TraneUNee- GroupDDD
0.8332
0 83887
0355
0.2006
0.8468
0.9655
0.256
0.7735
TramsUNe++
0.8370
0.8483
0.7355
0.7542
0.8501
0.7515
0.7784
SAMod
0.7217
0.7200
7.7
SAMad-ADV
0,5123
0.7257
0.7241
0.7342
SAMed GroDDRO
0.8212
0.7224
8AMod-3
Trans/Naa
0.7815
0,8057
0.6815
Tram-UNa
0.7774
0.6442
0,7829
0.6000
UransUNe GempDRD
0.7904
0.0738
0.0726
0.9644
TransUNet
0.78G7
0.7939
0.0692
0.6754
0.7943
0.6097
0,6789
respectively. The SAMed model is trained for a total of 130 epochs with an early stopping at the 70 epoch. The TransUNet is trained for 300 epochs without early stopping, following their original implemennatton. We use a batch size of 42 for training both models. When combining with the fairness algorithms, we use the same setup as aforementioned. For our proposed fair -roor scaling, we set y = 1.0 for all our experiments.
6.3 SEGMENTATION & FAIRNESS EVALUATION
Race. Table 6 shows the segmentation performance with respect to race, particularly among Asian, Black, and White groups, suggesting that SAVed+ADV, AeMed+GroupDeee and our method yicld relatively comparable results. SAM slightly outperforms others in terms of Dice scores among the White subgroup and the overall ES-Dice and ES-IoU. For the Rim, our proposed SAMed+FEBB achieves the best ES-Dice of 0.7529 and the best ES-IoU of 0.6451, which surpasses other methods based SAMed. TransUNet-baan based methods achieve relatively worse results than SAMed- based approaches. Notably, our methods with Trans as the backbone can surpass other TransUNet-based approaches in terms of ES-Dice and ES-ToU.
When gender serves as the sensitive attribute, the distinctions between methods become subtler, as shown Table 3 Our mothod combined with the SAMed and TransUNet model consistently outperforms the bascline and its counterpart in terms of ES-Dice and ES-IoU on both Cup and Rim. This showcases the efficacy of our technique in handling gender disparities. Similar to race, TransUNettbaaser methods also perform worse than the SAMed-based approaches.
Gender.
Language. For results with language attribute in Table SAMed+ ADV achieves the best. performance and fairness for the Cup and rim segmentation when compared with methods based on SAMed. However, the performance of our approaches achieves the best based on UNansUNet in terms of ES-Dice and ES-IoU. The disparity between different language groups is the second largest when compared to that of gender and ethnicity.
Ethnicity. When considering ethnicity as the sensitive attribute, our method combined with both two segmentation models surpasses the rest terms of ES-Dice and EI- scores. The details of results with ethnicity are shown in Table 9 The performance gaps between Hispanics and Non- -os-iiiaas are larger than gender.
7 CONCLUSION
In this paper, we propose the first datasct to study fairness in medical segmentation. The proposed FairSeg benchmarks are equipped with many SOTA fairness algorithms using different segmentation backbones. Our innovative fair bruor- scaling technique introduces a new fairness dice loss for segmentation tasks. Furthermore, a new equity-scaled evaluation metric provides novel scoring functions for evaluating the fairness of segmentation between different demographic groups. We hope that our contributions not only stimulate further research in this new domain but also serve as a catalyst for the adoption of fairness considerations in medical AI applications universally.
9

---

Published as a conference paper at ICLR 2024
REFERENCES
Alekh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, and Hanna Wallach. A reductions approach to fair classification. In International Conference Machine Learning, pp. 60-69. PMLR, 2018.
Alekh Agarwal, Miroslav Dudik, and Zhiwei Steven Wu. Fair regression: Quantitative definitions and reduction-based algorithms. In International Conference on Machine Learning, pp. 120 129. PMLR, 2019.
Arthur Asuncion and David Newman. Uci machine learning repository, 2007.
Tom L Beauchamp. Methods and principles in biomedical ethics. Journal of Medical ethics, 29(5):269 274, 2003.
Alex Beutel, Jilin Chen, Zhe Zhao, and Ed H Chi. Data decisions and Cheoretical implications when adversarially learning fair representations. arvie preprinl Xix: 10:::70700000 007.00075, 2017.
Peter Bickel, Eugene A Hammel, and J William O'Connell. Sex bias in graduate admissions: Data from berkeley: Measuring bias is harder than is usually assumed, and the evidence is sometimes contrary to expectation. Science, 187 399-4444 1975.
Jieneng Chen, Yongyi Lu, Qihang Yu, Xiangde Luo, Ehsan Adeli, Yan Wang, Le Lu, Alan Yuille, and Yuyin Zhou. Transunet: Transformers make strong encoders for medical image segmentation. Nrw preprint ar Xiv:22102. Xri::2200.043300 2021.
Julia Dressel and Hany Farid. The accuracy, fairness, and limits of predicting recidivism. Science advances, (())aea0,,580, 2018
Matthew J Everett and Jonathan D Oakley. Automated analysis of the optic nerve head: measurements, methods and representations, August. 11 2015. US Patent 9,101,293.
Matthew Groh, Caleb Harris, Luis Soenksen, Felix Lau, Rachel Han, Acrin Kim, Arash Koochek, and Omar Badri. Evaluating deep neural networks trained on clinical images in dermatology with the fitzpatrick 17k dataset. In Proceedings of the IEEE/CH /CVF Conference Computer Vision and Pattern Recognition, pp. 1820-18230 2021.
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. Advances in newral information processing syslems, 29, 2016.
Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-Ilcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp. 590-597, 2019.
A Alistair EW Johnson, Tom J Pollard, Nathaniel R Greenbaun, Malthew P Lungren, Chih- ying Deng, Yifan Peng, Zhiyong Lu, Roger G Mark, Seth J Berkowitz, and Steven Horng. Mimic-cxr-jper jpg. a large publicly available database of labeled chest radiographs. arXiv preprint NKK::1901.070.. 2019.
Jost B Jonas, Antonio Bergua, Paul Schmitz- Valckenberg, Konstantinos Papastathoopolles, and Wido M Budde. Ranking of optic disc variables for detection of glaucomatous optic nerve damage. Innestigative Ophthalmology & Visual Science, (7(7):1764-177 1176,, 2000.
Michael P Kim, Amirata Ghorbani, and James Zou. Multiaccuracy: Black-box post- processing for fairness in classification. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp. 247-254, 2019.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tele Xiao, Spencer Whitehead, A Alexander C Berg, Wan-Yee Yen Lo, et al. Segment anything. Xio preprint or arx::230440066 02649, 2023.
10

---

Published as a conference paper at ICLR 2024
Oleksandr Kovalyk, Juan
Rafael Verdi-Monedero, Inmaculada. Sellés- Navarro, Ana Palazón-Cabanes, and José-Luis Sancho-Gómez, Papila: Dataset with fundus images and clinical data of both eyes of the same patient for glaucoma assessment. Scientific Dala, 911)22 2022.
Jakub Kuzilek, Martin Hlosta, and Zdenek Zdrahal. Open university learning analytics dataset. Scientifie data, 4(1):1 8, 2017.
Yuyuan Liu, Yu Tian, Chong Wang, Yuanhong Chen, Fengbei Liu, Vasileios Belagiannis, and Gustavo Carneiro. Translation consistent semi-supervised segmentation for 3d medical images. arXiv preprint arXiv:2203. 113200..14550 14523, 2022.
Pranay K Lohia, Karthikeyan Natesan Ramamurthy, Manish Bhide, Diptikalyan Saha, Kush Varshney, and Ruchir Puri. Bias mitigation post-processing for individual and group fairness. In Teassp 2019-2019 seee international conference acoustics, speech and signal processing (icassp), pp. 2847-288 2851. TEFE, 2019.
Yan Luo, Min Shi, Yu Tian, Tobias Elze, and Mengyu Wang. Harvard glaucoma detection and progression: A multimodal multilask dataset and generalization-vinffoee semi- Proceedings of the TEER/CVF International Conference on Computer Vision, pp. 20471-20482, 2023a.
supervised learning-
Yan Luo, Yu Tian, Min Shi, Tobias Elze, and Mengyu Wang. Eye fairness: A large-scale 3d imaging dataset for equitable eye diseases screening and fair identity scaling. arXiv preprint 0rvi::23100022 02492, 2023b.
Yan Luo, Yu Tian, Min Shi, Tobias Elze, and Mengyu Wang. Harvard glancoma fairness: A retinal nerve discase dataset for fairness learning and fair identity normalization. arXiv preprint orXiv:2306.0922 2023c.
David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. Learning adversarially fair and transferable representations. In International Conference on Machine Learning, pp. 3384-3393. PMLR, 2018.
Weiwen Miao. Did the results of promotion exams have disparate impact on minorities? using statistical evidence in ricci v. destefano. Journal of Statistics Education, 18(3), 2010.
Christoph Mitsch, Stephan Holzer, Lorenz Wassermann, Hemma Resch, Susanne Urach, Barbara Kiss, Anton Hommer, Clemens Vass, and Ursula of spectralis and cirrus spectral domain optical coherence tomography for the objective morphometric assessment of the neuroretinal rim width. Graefe Archive for Clinical and Experimental Ophthalmology, 257:1265 -21111665511.5. 2019.
Comparison
Mare Modat, David M Cash, Pankaj Daga, Gavin P Winston, John S Duncan, and Sébastien Ourselin. Global image registration using a symmetric block-matchin approach. Journal of medical imaging, 1 1(2):024003 024003, 2014.
Sungho Park, Jewook Lee, Pilhyeon Lee, Sunhee Hwang, Dohyung Kim, and Hyeran Byun. Fair contrastive learning for facial attribute classification. In Proceedings of the IEEE/CH /CVF Conference on Compuder Vision and Pattern Recognition, pp. 10389 10398, 2022.
Novi Quadrianto, Viktoriia Sharmanska, and Oliver Thomas. Discovering fair representations in the data domain. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8227 8236, 2019.
Vikram V Ramaswamy, Sunnie SY Kim, and Olga Russakovsky, Fair attribute classification through latent space de-biasing. In Proceedings of the TEEE/CVP conference on computer vision and pattern recognition, pp. 9301-9310, 2021.
Yuji Roh, Kangwook Lee, Steven Whang, and Changho Suh. Fr-train: A mutual basee approach to fair and robust training. In International Conference on Machine Learning, pp. 8157-8157. PMLR, 2020.
11

---

Published as a conference paper at ICLR 2024
Steven Ruggles, Robert McCaa, Matthew Sobek, and Lara Cleveland. The ipums collaboration: integrating and disseminating the world's population microdata. Journal of demographic economics, 81(2):203 216, 2015.
Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint Axi::911100873) 2019.
Mhd Hasan Sarhan, Nassir Navab, Abouzar Eslami, and Shadi Albarqouni. Fairness by learning orthogonal disentangled representations. In Computer Vision-FCCC 2020: 16th European Conference, Glasoww, UK, August 23 23-28, 2020, Proceedings, Part XXIX 16, pp. 746-761. Springer, 2020.
Min Shi, Anagha Lokhande, Mojtaba S Fazli, Vishal Sharma, Yu Tian, Yan Luo, Louis R Pasquale, Tobias Elze, Michael V Boland, Nazlee Zebardast, et al. Artifact-tolerant clustering-guided contrastive embedding learning for ophthalmic images in glaucoma. IEEE Journal of Biomedicul and Health Informalics, 2023a.
Min Shi, Yan Luo, Yu Tian, Lucy Q Shen, Tobias Elze, Nazlce Zebardast, Mohammad Eslami, Saber Kazeminasab, Michael V Boland, David S Friedman, et al. Equitable artificial intelligence for glaucoma screening with fair identity normalization. medRain, pp. 2023-12, 2023b.
Min Shi, Jessica A Sun, Anagha Lokhande, Yu Tian, Yan Luo, Tobias Flze, Lucy Q Shen, and Mengyu Wang. Artifact correction in retinal nerve fiber layer thickness maps using deep learning and its clinical utility in glaucoma. Translational Vision Science & Technology, 12(11):12 12, 2023c.
Yih-Chung Tham, Xiang Li, Tien Y Wong, Harry A Quigley, Tin Aung, and Ching- Yu Cheng. Global prevalence of glaucoma and projections of glancoma burden through 2040: systematic review and aeta--nnayssi. Ophthalmology, (11(1)):2081 2290,, 2014.
Yu Tian, Fengbei Liu, Guansong Pang, Yuanhong Chen, Yuyuan Liu, Johan W Verjans, Rajvinder Singh, and Gustavo Carnciro. Self-supervised pscudo multi-class pro-training for unsupervised anomaly detection and segmentation in medical images. Medical Image Analysis, 90: 102930, 2023a.
Yu Tian, Guansong Pang, Yuyuan Liu, Chong Wang, Yuanhong Chen, Fengbei Liu, Rajvinder Singh, Johan W Verjans, Mengyu Wang, and Gustavo Carneiro. Unsupervised anomaly detection in medical images with a memory- augget-ed multi-level cross- attentional masked autoencoder. In International Workshop on Machine Learning in Medical Imaging, pp. 11-21. Springer, 2023b.
Sahil Verma and Julia Rubin. Fairness definitions explained. In Proceedings of the international workshop on software fuirness, pp. 1-7, 2018.
Risheng Wang, Tao Lei, Ruixia Cui, Bingtao Zhang, Hongying Meng, and Asoke K Nandi. Medical image segmentation using deep learning: A survey. IET Image Processing, 16(5): 1243 1267, 2022a.
Zhibo Wang, Xiaowei Dong, Henry Xue, Zhifei Zhang, Weifeng Chin, Tao Wei, and Kui Ren. Pairness-aware adversarial perturbation towards bias mitigation for deployed deep models. In Proceedings of the TEFE/CVE Conference on Computer Vision and Pattern Recognition, pp. 19379910388,0. 2022b.
Linda F Wightman. Lsac national longitudinal bar passage study. Isac research report series. 1998.
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna Gummadi. Fairness constraints: Mechanisms for fair classification. In Artificial intelligence and statistics, pp. 962-970. PMLR, 2017.
Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Miligating unwanted biases with adversarial learning. In Proceedings of the 2018 A AAAT/ACM Conference on AT, Ethics, and Sociely, pp. 335-340, 340, 2018.
12

---

Published as a conference paper at ICLR 2024
Kaidong Zhang and Dong Liu. Customized segment anything model for medical image segientation.. orXiv preprinl urx:::2304 11372334413785, 2023.
Yi Zhang and Jitao Sang. Towards faiiress paradox: Adversarial example-based data augmentation for visual debiasing. In Proceedings of the 28th ACM International Conference on Multimedia, pp. 4346 4354, 2020.
Zhifei Zhang, Yang Song, and Hairong Qi. Age progression/regression by conditional adversarial autoencoder. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5810 5818, 2017.
Zongwei Zhou, Md Mahfuzur Rahman Siddiquee, Nima Tajbakhsh, and Jianming Liang. Unet++:: A nested u-net architecture for medical image segmentation. In Deep Learning in Medical Image Analysis and Mullimodal Learning for Clinical Decision Support: 4th International Workshop, DLMTA 2018, and 8th. International Workshop, MT-CDD CDS 2018, Held Conjunction with MICCAT 2018, Granada, Spain, September 20, 2018, Proceedings 4, pp. 3-11. Springer, 2018.
Yongshuo Zong, Yongxin Yang, and Timothy Hospedales. Medfair: Benchmarking fairness for medical imaging. Xiv preprint ar Xxi::2210.01792, 2022.
13

---

Published as a conference paper at ICLR 2024
A APPENDIX
Equity-Scaled Metric Using Standard Deviation. A modification to the original equity-scaled metric involves calculating the standard deviation Stdev of the segmentation metric for various demographic groups. Let I E {Dice, !oU..... represent a segmentation metric. Conventional segmentation metrics, like Dice and IoU, can be expressed as M(f(2', ,))),, where they asscss the overlap between the predicted segmentation z' and the ground truth Let's denote the set of all demographic groups as A.
Given this, the Equity-Scaled Segmentation Performance with standard deviation (ESSP- Sudev) metric for segmentation, ESSP-Sudev, is defined as:
ESSP-Stdev Z(1(2', x))(
1 Stdev
The denominator in Equation naturally penalizes variations in the segmentation metric across demographics. As Stdey approaches zero, indicating uniform performance across groups, ESSP converges to I. Conversely, the variability (Stdev) increases, ESSP will diminish, thereby reflecting a penalty for demographic disparity.
A potential limitation of this metrie is that it may yield only marginal differences between Equity-Scaled performance and the overall segmentation performance. This subtle distinction could pose a challenge in discerning the effectiveness of varions de-biasing approaches in enhancing algorithmic fairness. Nonetheless, for reference purposes, we include results derived from such metrics in this appendix.
Table 6: Optic Cup and Rim segmentation performance on the Harvard-FairSeg datase
with race as the sensitive attribute.
Overall ES-Dioe!
Overall
Overall ES-Lat-
Overall InU]
Axian
Black
White Dioei
Asiin InU]
Black
White
Method
Dioe"
InUl"
0.9675
05710
0.704
0.7955
0.8075
0.8855
0.9017
6.7k19
0.805
0:7715
SAMdd
0.503
0.9771
0.729
0.7815
0.868
0.8730
3.5070
0.7885
6.7005
0.7905
SAMae SAM
0.5640
0.5698
0.7000
0.7840
0.5090
0.5955
0
0.7705
0.782
0.7566
0.835
05701
0,7711
0.7642
SAMed
0
0.5587
Trand/Nee
0.8372
0.881
0.7409
0,7532
0.8270
0JM489
0.5005
0.7277
0.7576
0,7551
Thand Nee-ADV
018325
05110
0.755
0.1132
(18215
05417
05576
0.7260
0.7882
0.763
0.3155
05312
0.719
05309
05311
0.7212
6.1829
07.815
UNat PIEDS
0,5215
0.7241
0.7013
0,5035
0.3155
0.0099
0.7211
0,2030
0.7822
0.5000
0.6854
0.0639
0.7297
SAMae
0,5030
0.5211
RI2L5
0.227
0,890
0.7115
0.5644
0.6745
C.6887
0.200
0.7015
05216
0.7901
0.7011
05206
0.0636
0.7825
SAMai-
SA SAMmd+PRBS
0.4956
0.8325
0.5865
0,7260
0.7852
0.7769
0.8473
0.6825
0.6620
0,7459
(L7934
(U1927
0.6766
0.715
0.7217
0.500
0.8160
0.5001
0.6013
Ban ADV
0,797
0.706
019538
0.713
0,726
05887
0.6116
0.082
01885
0.777
TransUNet: PEEDS
0.7628
0.7950
0.6410
0.6725
0.7385
0.8150
0.6185
0.0020
0.0935
Expansion of Evaluation Metrics. To provide a more thorough assessment of segmentation accuracy, in Table. 10 we provide the segmentation performances on our Harvard-FairSee datasot with race as the sensitive attribute in terms of Hausdorff Distance (HD95), Average Surface Distance (ASD), and Normalized Surface Distance (NSD).
Table 7: Optic Cup and Rim segmentation performance on the Harvard-FairSe datasct
with gender as the sensitive attribute.
Oversil Ei-
Overall Dios
Overall ES-1oU7
Overall Ioll
Male Dicw!
Ferrade Dies
Mals
Forndds
Mothod
IoU1
Ioll
SAMal
0 0:077
0.7733
0.7515
0.5703
07783
0.7855
SAMaI-ATI
08858
0.7787
0.7505
0.8051
0.5675
0.7791
0.7650
0:8770
0.7803
0.7508
0.8672
0.5670
0.7801
0.7514
SAMod
0.5078
0,8702
0.7807
0.7528
0.7861
0.7879
ThanstiNes
1775/2
0.7532
0.7508
0.7564
TraneUNe+AATY
TransUNe GGomppDD
0.8130
0.7522
0.8480
0.7530
SAMad
0.8251
0.7217
0.7169
SAMw--ATIV
0.8265
01 0.3518
0.7185
0.7230
0.8512
015263
(17277)
0.7181
0.8274
0.332)
0.7205
0.7255
0.6558
(18274
0.7292
0.7188
SAMmi+ FRBS
0.8280
0,8918
0.7227
0.7258
0.8338
0,8280
0.7274
0.7225
TransUNi
0.7866
0.7427
0.9673
0.0706
0.7951
0.7894
0.6736
0.6665
Trans A
0.7753
0 0.7822
0.6553
0,0530
0.7935
017779
0.6029
0.6534
Trans UNe G20
0.7901
0.7017
0.8681
0,9509
0.7930
0.7200
0.6T16
0.6677
Tra FRBS
0.7903
017808
0,8694
0.6698
0.7923
0.79352
0.0078
0.0853
14

---

Published as a conference paper at ICLR 2024
Table 8: Optic Cup and Rim segmentation performance on the Hlavard-FaiirSer dataset with preferred language as the sensitive attribute.
Overall
Overall
Overall
Overall
English
Speaish
Others
English Ink]]
Spanish
Otlers
Ink]
SANIs
0,5/50
0.677
07703
0.7813
0.5622
0.8677
D.501
0.7791
0.5001
A
05855
0966
87386
0.7830
0.905
0.9131
0.9210
0.7915
0.8432
0.71622
SAMed FEDS
0.5527
0.7640
0.5714
0.7810
0.7007
Tox
0.8255
0.5481
0.223
0.7632
0.5409
0.8972
0.831
0.7516
0.8100
0.7001
Nes ADV
0.907]
0.812
07006
0.7225
0.5236
0.8835
0.2255
0.7901
0.700
0.7875
Thandl Deampbia
0.8211
05116
T7211
071113
0,835
0.811
0J8571
0.7:31
0.7963
0.7011
SAMI
0,8179
0,2291
0.7217
0.5915
0.5534
0.7511
0.7465
0:8871
SAM
0.5057
0.8255
0.000
0.2233
055337
0,8225
0.5015
0.7231
0.2015
0,6003
CD
0.5130
0.8111
0.7075
0.8222
0.5005
0.7215
0.0954
SAMed+
Ton
0.7721
0.7927
0.6925
0.6700
0.7940
0.5105
0.7633
0.6721
0.0950
0.6216
Band Yee-ANN AFV
0.7090
0.7811
0.0001
0.0956
0.7933
0
0.6887
0.6717
0.6915
Thand Greepphit
0.7091
3
0.7991
0.7018
0.0903
045333
Table 9: Optic Cup and Rim segmentation performance the Harvard-Fairde dataset with ethnicity as the sensitive attribute.
Overall ES-Diow
Overall Dice
Overall FS-1o17
Overall Iol]
Hisponic Diee!
Non-Hispanic Diin
Hisganic loU?
Non-Hispanic:
Motborl
Tol:1
SAMad
0.559
0.3671
0.7665
0,7513
0.8633
0.7750
0.5100
SAMod-ADV
0.8548
0.775
0.7057
0.7814
0.8601
0.8553
0.7791
0
0,8098
0.7718
0.7540
0.8623
0.5855
0.7819
SAMol43
0
0.3055
0.7758
0.7845
0.8704
0.5231
0.7904
0,8070
Tha
0.833
0.7365
0.7522
0.8663
0.8704
0.7008
0.7820
Thans UNi.-
0.8171
0.5120
0.7315
0.830%
0.7254
Thans//Ncc RD
0,7026
0.8668
0.7007
0,7725
TransUNet+
0.5388
0.5453
0.7412
0.7542
0.5501
0.7515
0,7764
SAMai
0.8221
0,7217
0.7200
0.7317
SAMad-ADV ADV
0.82511
0,5121
0.7257
0.7241
0.7342
SAMed 94Mee-ChoosDDB
0.8237
0,8259
0.7224
0.7308
0.7298
SAMed+ FEDS
0.8296
0.8131
0.7215
0,7242
0.8349
0,540S
0.7278
0,7329
TranslNee
0.7818
0,2027
0.0050
0,6706
0.7914
0,8057
0.0055
0. 0.6815
0.7758
0. 0.7841
0.6570
0,6642
0,7822
0.77115
0U6T20
0.6658
Thans GampDDRU
0.7924
0.7943
0.9234
0,0748
0.7336
0.7700
0.6728
0,66-4
Trats FEBS FEBS
0.7854
0.7920
0.6710
0.6754
0.7943
0,5000
0.6097
0,6789
Table 10: Optic Cup and Rim HD95, ASD, and NSD performances on the Harvard- FairSeg dataset with race the sensitive attribute.
Mechod
Ovezall HD061
Astan
Black
White HT285]
Overall
Astan
Black ASD1
White ASD1
Overall NSDT
Astan NSD1
Black NSDJ
White
HT285]
HD86]
ASD
ASD1
NSD1
SAMed
11.0005
11.01:2
0.3885
3.5050
4.7755
4.7743
3.7210
0/6825
0.7838
SAMed+ +DDY
9.6591
11,0170
11.1193
5.9479
4.9236
3.7821
0.7405
0.7145
0.6054
0.7537
DRO
9.4535
11,0419
3.9491
4.582
3.8855
0.7415
0.7000
0.7547
PEDS
10.5176
3.5006
0.7289
0.7038
0.7015
0.7518
Tres
0.9422
Tran--NNet FiRS
SAMed
0.0370
114889
11.5071
0.4357
3.5053
4.5013
4.5473
3.7476
0.7481
0/7313
0/7215
07756
4AMed+ADV
8.8316
10. 10.
10.3361
8.5582
3.5525
3:90.56
3.8075
3:1700
0.5083
0.7091
0.70%
0
S4Med+
8.7609
10-4655
10/0322
8.9076
3.5175
3.8345
3,9490
$:1702
0.8078
0.7700
0.7006
08191
SA MaMi PERS
8.7080
10.5107
10.5107
8.5082
3/174
4.0053
4.0605
$:2206
0.3043
0.7782
0.77M
0
TNos
4.6484
T,4186
5.0055
4.1961
1.7084
1.5643
1.5941
1.5800
0.9601
0.9036
0.93
0-9688
Tasaa ADV
4.63301
5,0065
1.4000
4.5560
1.8110
2.3085
2,0885
1.721.
0.9654
0.9265
0.5221
0.9676
TrasUNe Gro
4.5268
T,4243
1.2200
0.2942
1.74U8
1.3683
2,0017
1.67 02
0.9036
0.9022
0.501T
0.9683
Trans + EEDS
4,5311
7,5251
1.2805
0,2082
1,8921
2,0642
2.0771
1.7564
0.9581
0.5006
0.5015
0 0.9606
15


Reviews:
[{"review_id": "uYaRh16bmB", "reviewer": "Reviewer_Suah", "summary": "This paper proposes a publicly available medical fairness segmentation dataset (FairSeg) that contains 10,000 subject samples of 2D SLO Fundus images. The paper also proposes equity-scaled segmentation performance metrics to facilitate fair comparisons.", "questions": "1. Why validation set is not constructed/used in selecting models in training?\n\n2. It would be helpful to report Hausdorff distance and average surface distance along with Dice to better evaluate the methods.\n\n3. The details of how standard deviation is computed need to be elaborated. Is it computed across the mean of for each group?\n\n4. How is the training/testing split performed? Is it just randomly sampled without considering sensitive attributes at patient level?\n\n5. It would be helpful to discuss the importance of registration in preprocessing using NiftyReg.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. The fairness concern is an important topic, especially in medical images and the lack of segmentation dataset is a big issue. The motivation of the proposed dataset is strong.\n\n2. The dataset contains a large amount of segmentation ground truths (10,000) and is well evaluated by authors with several SOTA learning algorithms.\n\n3. As described by the authors, the segmentation seems to undergo a rigorous process including a hand-graded annotation by a panel of five medical professionals after initial registration.", "weaknesses": "1. The accuracy of the Nifty reg needs to be investigated since it might not be the SOTA for image registration.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "w9QoDjt3kw", "reviewer": "Reviewer_5cUh", "summary": "In this work, the authors introduced the new FairSeg dataset, designed to address fairness concerns in the domain of medical segmentation. Their innovative methodology centers on a fair error-bound scaling technique, which recalibrates the loss function by considering the upper error-bound within each identity group. Furthermore, they designed a new equity-scaled segmentation performance metric to facilitate fair comparisons between different fairness learning models for medical segmentation. Extensive experimentation underscores the efficacy of the fair error-bound scaling approach, demonstrating either superior or comparable fairness performance when compared to state-of-the-art fairness learning models. Furthermore, The related dataset and code are both made publicly accessible by the authors.", "questions": "1. In equation (1), a parenthesis is missing in the formula.\n2. The authors proposed a new the Dice loss with a novel Fair Error-Bound Scaling mechanism, however there are experiment results to show the differences between the new dice loss and common one.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "+ The paper is well-written and easy to follow. \n+ The proposed framework is technically sound.\n+ The experiments are comprehensive.", "weaknesses": "There is no visualization comparison between different methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uQKzOl8mxK", "reviewer": "Reviewer_b2i2", "summary": "This paper proposed a fundus image dataset for benchmarking the fairness of medical image segmentation methods, which is the first dataset and benchmark in this field. The authors also proposed to rescale the loss function with the upper training error-bound of each identity group to tackle the fairness issue.", "questions": "- The dataset was released as npz format. Could you please also release the original format?\n\n- It would be great if you could release the trained models as well.\n\n- Where do you plan to host this benchmark? CodaLab could be a good platform.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- Novel Dataset: The paper introduced FairSeg, a new dataset for medical image segmentation with a focus on fairness. The creation of such a dataset is valuable as it addresses a gap in the current availability of medical datasets with fairness considerations.\n\n- Fairness-Oriented Methodology: The authors proposed a fair error-bound scaling approach and an equity scaling metric. These methods represent an advanced effort to integrate fairness directly into the model training process, which could lead to more equitable healthcare outcomes.\n\n- Open Access: I like that the author released the dataset and code for reproducibility and further research, which is a strong aspect of this work.", "weaknesses": "- Dice and IoU are equivalent (https://www.sciencedirect.com/science/article/pii/S1361841521000815), which are not necessary to be reported simultaneously. Instead, please add NSD which is suggested by metrics reloaded (https://arxiv.org/abs/2206.01653).\n\n- nnUNet is still the state-of-the-art in many segmentation tasks. It would be great to evaluate it on your dataset.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TywQCUSi08", "reviewer": "Reviewer_f8dv", "summary": "This paper proposed a dataset for retinal disc/cup segmentation with several pre-defined attributes, which should be useful for studying the fairness problem in the medical domain. Furthermore, the authors set a baseline for the problem and define the evaluation metrics in this scenario. Overall, this work is sound and meaningful.", "questions": "See the above weaknesses", "limitations": "", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "[1] Providing a dataset for fairness-related research is meaningful for the current community, along with its baseline and evaluation setting.\n\n[2] Good writing and clear motivation", "weaknesses": "[1] ICLR might not be the best place for this paper. Other medical journals or conferences would be more suitable.\n\n[2] There are many evaluation ways to assess the fairness problem. The selected metrics might not be the most suitable one. Please elaborate more on the motivation of baseline setting and evaluation.\n\n[3] Some current works should be included to make the experiments sufficient. See: FairAdaBN: Mitigating unfairness with adaptive batch normalization and its application to dermatological disease classification\n\n[4] Since most of the attributes are only for the patient level, why use the pixel-wise weights?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)