The Disparate Benefits of Deep Ensembles
Authors: Kajetan Schweighofer
Adrian Arnaiz-Rodriguez
Sepp Hochreiter
Nuria M Oliver
Venue: ICML 2025 poster (2025)
Keywords: Deep Ensembles
Disparate Benefits
Predictive Diversity
Algorithmic Fairness
Post-Processing
Primary area: social_aspects->fairness
TL;DR: We uncover the disparate benefits effect of Deep Ensembles, analyze its cause and evaluate approaches to mitigate its negative fairness implications.

Abstract: Ensembles of Deep Neural Networks, Deep Ensembles, are widely used as a simple way to boost predictive performance. However, their impact on algorithmic fairness is not well understood yet. Algorithmic fairness examines how a model's performance varies across socially relevant groups defined by protected attributes such as age, gender, or race. In this work, we explore the interplay between the performance gains from Deep Ensembles and fairness. Our analysis reveals that they unevenly favor different groups, a phenomenon that we term the disparate benefits effect. We empirically investigate this effect using popular facial analysis and medical imaging datasets with protected group attributes and find that it affects multiple established group fairness metrics, including statistical parity and equal opportunity. Furthermore, we identify that the per-group differences in predictive diversity of ensemble members can explain this effect. Finally, we demonstrate that the classical Hardt post-processing method is particularly effective at mitigating the disparate benefits effect of Deep Ensembles by leveraging their better-calibrated predictive distributions.

The Disparate Benefits of Deep Ensembles
Kajetan Schweighofer'
Adrian Arnaiz-Rodriguee
Sepp Hochreiter' Nuria Oliver
Abstract
Fairface
Ensembles Deep Neural Networks, Deep En- sembles, widely used simple way to boost predictive performance. However, their impact algorithmic fairess is not well understood yet. Algorithmic faimess examines how a model's per- formance varies across socially relevant groups defined by protected attributes such as age, gen- der, or race. In this work, we explore the interplay between the performance gains from Deep Ensem- bles and faimess. Our analysis reveals that they unevenly favor different groups, a phenomenon that we term the disparate benefits effect. We empirically investigate this effect using popular facial analysis and medical imaging datasets with protected group attributes and find that it affects multiple established group fairness metries, in- cluding statistical parity and equal opportunity. Furthermore, we identify that the per-group differ- ences in predictive diversity of ensemble members can explain this effect. Finally, we demonstrate that the classical Hardt post-prose proccssocs method is particularly effective at mitigating the disparate benefits effect of Deep Ensembles by leveraging their better-calibrated predictive distributions.
Falmess Perforancce
Accunn
Ma
g
voo-
600
CheXpert
Ensemble Members
Ensemble Members
Ensemble Members
Figure 1: Unfairness caused by the disparate benefits effect of Deep Ensembles. Adding members to the ensemble increases its performance (Accuracy. AUROC) but decreases its faimess (1-SPD, I-EOD, I-AOD). We find evidence this effect across multiple standard fairness metries (cols) and vision datascts (rows).
it is crucial to examine how these models perform across different groups that are defined by protected attribute (e.g., gender, age, race, etc.) which is the focus of the field of Algorithmic Fairness (Barocas al., 2023). Ensuring equitable operation of these models across protected groups is imperative, as they can significantly impact individuals and communities, potentially widening existing disparities if not adequately addressed. Although the differences in performance across protected groups (group fairness vio- lations) of individual DNNs has been thoroughly studied (Zhang et al., 2018; Sagawa et al., 2020; Zhang et al., 2022; Amaiz-Rodrigus & Oliver, 2024), the impact on fairness due ensembling these networks remains underexplored.
1. Introduction
Deep Ensembles (Lakshminarayanan) et al., 2017) have demonstrated their efficacy as a simple and robust method to improve the performance of individual Deep Neural Net- works (DNNs). Their superior performance has made them popular choice for real-world applications (Bhusal al., 2021; Dolezal al., 2022), including high-stakes scenar- ios where the impact on people's 's lives of Machine Learning (ML) supported decisions can be profound, such as in health- care, education, finance or the law. In such applications,
In this paper, our aim is to fill this gap by conducting an ex- tensive empirical study of the fairness implications of Deep Ensembles, analyzing their underlying causes, and exploring mitigation strategies. Our empirical study is based on two popular facial analysis datasets and widely used medical imaging dataset, each with multiple binary target variables and protected group attributes. We evaluate a total of fifteen tasks across five different DNN model architectures and three standard group faimess metrics. Our analyses reveal that Deep Ensembles unevenly benefit different protected groups in what we refer as the disparate benefits effect (cf.
'ELLIS Unit LIT AI Lab, Institute for Machine Learning, JKU Linz, Austria "ELLIS Alicante, Alicante, Spain "NXAI GmbH, Linz, Austria. Correspondence to: Kajetan Schweighofer
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

The Disparate Benefits of Deep Ensembles
Fig.1). 1). We further investigate the causes of this disparate benefits effect and find evidence that differences in the pre- dictive diversity of ensemble members across groups can explain why ensembling benefits groups differently. Finally, we explore potential approaches to mitigate the negative im- pact on fairness caused by the disparate benefits effect. We find that Deep Ensembles are more sensitive to the predic- tion threshold than individual models due to their improved calibration. This makes post-processing methods suitable approach to mitigate the fairness violations. In fact, our results show that Hardt post-processing (Hardt et al., 2016) is very effective, resulting in fairer predictions while pre- serving the improved performance of Deep Ensembles. In sum, the main contributions of this paper are threefold:
or the model's decision rule (post-processing) (Hardt et al., 2016; Cruz & Hardt, 2024). In this paper, we focus on group algorithmic faimess and analyze the impact of Deep Ensembles on group faimess.
Deep Ensembles. Deep Ensembles (Lakshminarayanan et al., 2017) are known as a simple and effective method to boost the performance of DNNs and to estimate predic- tive uncertainty (Ovadia et al., 2019; Ashukha et al., 2020; Schweighofer et. 2023). They mainly rely on the stochas- ticity of the initialization and optimization procedure for diversity (Fort et al., 2019). However, obtaining more di- verse Deep Ensembles is still an active area of research (Rame & Cord, 2021; Lee et al., 2023; Pagliardini et al., 2023). Furthermore, the exact mechanisms that produce the performance improvements observed in Deep Ensembles re- main an open research question (Abe et al., 2022b; Jeffares et al., 2023; Abe et al., 2024).
1. We empirically analyze how the performance gains of Deep Ensembles distributed across groups defined by protected attributes (Sec. 5). Our findings reveal that Deep Ensembles yield disparate benefits across groups, often benefiting the already advantaged group.
Ensemble Fairness. Prior work at the intersection of algo- rithmic fairness and ensembling has investigated the effect of model multiplicity (Marx et al., 2020; Coston et al., 2021; Black et al., 2022a;b; Long et al., 2023; Cooper et al., 2024), and has reported that ensembling decreases the multiplic- ity of predictions, thus being less arbitrary than individual models. Shallow model ensembles (i.e., models that are not DNNs) have been used to improve the fairness of out- comes (Kamiran & Calders, 2012). Kenfack et al. (2021) considered a fairnsss-aware weighting of shallow model en- sembles. Similarly, Gohar et al. (2023) investigates multiple research questions regarding ensembles of shallow models. Theoretical considerations on ensemble faimess have been investigated by Grgié- Hlgić-Hlača et al. (2017). Based on these in- sights, Bhaskaruni et al. (2019) proposed a fair ensembling strategy adopting the AdaBoost framework. However, we are not aware of any work that has investigated the impac of Deep Ensembles on group fairness metrics.
2. We investigate the potential causes for the disparate ben- efits effect (Sec. 6). Our analysis identifies per-group differences in the predictive diversity of ensemble mem- bers as a key contributing factor.
3. We evaluate approaches to mitigate unfairness due to the disparate benefits effect (Sec. 7). We find that Deep Ensembles are more sensitive to the prediction threshold due to their improved calibration. Thus, Hardt post- processing (Hardt et al., 2016) is found to be very effec- tive, ensuring more fair predictions while preserving the improved performance of Deep Ensembles.
2. Related Work
Algorithmic Fairness. Algorithmic fairness is defined by means of various ethical and legal concepts (Barocas & Selbst, 2016; Corbett-Davies et al., 2017: Binns, 2018), re- sulting in diverse statistical and causal notions of equality between tasks and contexts (Kusner et al., 2017; Mehrabi et al., 2021). We focus on group fairness metrics -sttistii- cal discrimination metries for classification (Carey & Wu, 2023)- that measure error rate differences between groups defined by protected attributes (Hardt et al., 2016; Zafar et al., 2017). Several metrics quantify group fairness by im- posing independence conditions on the joint distribution of targets, predictions, and protected attributes (Barocas et al., 2023), capturing performance disparities due to varying in- put and target distributions among protected groups (Garg et al., 2020: Pombal al., 2022). Consequently, multitude of ML techniques have emerged over the past decade to promote group algorithmic faireess (Mchrabi et al., 2021) by modifying the data (pre-processing) (Kamiran & Calders, 2012; Arnaiz-Rodne Rodriguz & Oliver, 2024), the learning pro- cess (in processsig) (Agarwal et al., 2018; Jung et al., 2023);
The most closely related previous work to ours that by Ko et al. (2023), which investigates the effect of Deep Ensem- bles subgroup performance and served as inspiration for our work. However, their focus and methodology are dif- ferent from ours. For most of their experiments, the group variable of interest A is defined as a subset of the full target space y, i.e., of the worst and best performing targets. In our experiments with real-world data, groups are defined by the values of protected attribute, such as age, gender, or race. Furthermore, Ko et al. do not consider established group faimess metrics we do, focusing instead on per- group changes in accuracy. Finally, Ko et al. conclude that Deep Ensembles have exclusively positive impact, while we show that they can negatively affect group fairness. In addition, we investigate potential causes for this effect and analyze mitigation strategies that preserve fairness while maintaining the performance gains of the ensembles.
2

---

The Disparate Benefits of Deep Ensembles
3. Background
demographic parity. It is formally defined as
We consider the canonical setting of binary classifica- tion with imputs a E R.,, targets y E {0, 1}, and group attributes a { defined according to pro- tected or sensitive variables, such as gender, age, or race. We further consider DNNs as the models to map an in- put æ to the 1-dimensional probability simplex A' = ((50, $1) E R2² so 2 0,81 2 00 + $ 1]. We define this mapping as fw : RD + A for a model with pa- rameters un The output of this mapping defines the dis- tribution parameters of the predictive distribution of the model, denoted by p(y æ; w). A training dataset D (az,, 8)) is used to determine the model parameters by minimizing the cross-entropy loss. The final prediction 9 is given by the argmax over the predictive distribution.
P(ύ=1|A=1) = P(Y=1|A-0)
(2)
Second, equal opportunity (Hardt et al., 2016), which de- fines fairness as predicting the positive outcome inedpen- dently of the protected group attribute, but conditioned on the observed outcome being positive. Equal opportunity is therefore formally defined as
p(-=1 =1 A=1,Y=1) = P(L=1 A= (3)
Third, equalized odds (Hardt et al., 2016), which is stricter version of equal opportunity where the predictive indepen- dence must hold conditioned both positive and negative observed outcomes. Equalized odds is formally defined as
P(C =1 A=1,Y=y) = p(Ý=1 A=0,Y=5) (4)
Deep Ensembles. Deep Ensembles (Lakshminarayanan et al., 2017) are an ensemble method that uses DNNs as the base learners. While shallow learners often aggregate predic- tions in the ensemble via majority voting, Deep Ensembles typically average the output distributions of individual mem- bers. Furthermore, individual models are generally trained independently on the same data using different random seeds for initialisation and training. Deep Ensembles are widely recognized as a way to perform approximate sampling from the posterior distribution p(w D)= = p(D | w w))(ww)/ /P( (Wilson & Izmailov, 2020; Ashukha al., 2020), often pro- viding the most faithful posterior approximations (Izmailov et al., 2021). The predictive distribution of an ensemble with N members is given by
Vy {0. 1}. These measures are particularly relevant be- cause they operationalize antidiserimination principles, such as disparate impact in U.S. law (Feldman et al., 2015). Sta- tistical parity focuses on ensuring equal outcomes, while equal opportunity and equalized odds balance error rates to promote equity across groups. All operationalized notions of fairness have limitations such that it is not necessarily guaranteed that changing the model predictions to satisfy these conditions will actually lead perfectly fair outcomes in the real world (Selbst et al., 2019; Liu et al., 2018). Fur- thermore, some notions of fairness can be incompatible with each other, such as statistical parity and equalized odds if A and are not independent (Chouldechova, 2017; Kleinberg et al., 2017). Nevertheless, these metrics are a meaningful and widely used tool to quantify group faimesss
/ p(y x. w) p(w| D) dw w
p(y x. D)-
4. Experimental Setup
ply x. wn), wn p(w D))
Datasets. In our experiments, we evaluated Deep Ensem- bles on three different vision datasets. First, two facial anal- ysis datasets, namely FairFace (Karkkainen & Joo, 2021) and UTKFace (Zhang et al., 2017). For these datasets, all models were trained on the training split of FairFace and evaluated on the official test split of FairFace and the full UTKFace dataset. Protected group attributes were binarized, except for gender which was already binary. For the at- tribute age, we defined young and old, where a person is considered old from 40 years onwards to obtain a roughly balanced age distribution. For the attribute race, we bina- rized into white vs. non-white. We trained the models using one of the attributes as the target variable and evaluated with the remaining two attributes as protected group variables for all possible combinations of the target and protected group attributes. Second, the CheXpert medical imaging dataset (Irvin et al., 2019) using the recommended targets provided by Jain et al. (2021) and protected group attributes provided by Gichoya et al. (2022). The no finding target was
Thus, it is an approximation of the posterior predictive dis- tribution. The prediction of the Deep Ensemble equivalent to single model given by ğ = argmax p(y x, D).
Group Fairness. The group fairness desiderata are based the statistical dependencies between the random variables of the predicted outcomes Y, the observed outcomes Y and the protected group attribute A. Following widespread convention, we consider binary outcomes and protected groups, with Y Y = 1 being the positive outcome and A = the advantaged group. We focus on three well- established notions of group fairness as follows (Mehrabi et al., 2021: Caton & Haas, 2023).
First, statistical parity (Dwork et al., 2012; Kamishima et al., 2012), according to which fairness is achieved when the positive outcome is predicted independently of the pro- tected group attribute. Statistical parity is also known as
3

---

The Disparate Benefits of Deep Ensembles
used train and evaluate the models. Samples without all protected group attributes have been removed. A random
Table I: Disparate Benefits:
Change in performance and fair- ness violations due to ensembling, Significant differences (A) between the Deep Ensemble (ef. Tab 3) and the average ensem- ble member (ef. Tab. 4) are highlighted in bold (t-test, five runs, p 0.05). Gray cells denote that faimess violations >> 0.05 for hoth the Deep Ensemble and the average ensemble member.
subset of 1/8 was split as test dataset. Protected group at- tributes were binarized as for the facial analysis datasets. Additional details are given in Apx. D.1.
Models and training. We used five different DNN architec- tures, namely ResNet1 8/34/50 (He et al., 2016), RegNet-Y Y 800MF (Radosavovic et al., 2020) and EfficientNerV2-S (Tan & Le, 2021) for our evaluation, due to their widespread adoption and competitive performance in vision tasks. The models that were trained on the FairFace training dataset were trained for 100 epochs using SGD with momentum of 0.9 with a batch size of 256 and learning rate of le-2. Furthermore, a standard combination of linear (from factor I to 0.1) and cosine annealing schedulers was used. The models that were trained on the CheXpert training dataset were trained for 30 epochs given that the training dataset is roughly three times the size of FairFace, resulting in a similar number of gradient steps and a similar learning rate schedule. We independently trained 10 models for archi- tectures on 4 target variables with 5 seeds. Thus, total of 1,000 individual models were obtained for our evalua- tion. The results discussed in the main paper correspond to the use of ResNet50 as the model architecture. Additional results for the other model architectures are provided in Apx. F.2 and Apx. F.3.
D' Target Group
Accuracy (1)
SPD (J)
EOD (4) A AOD (4)
age/ gender
.0222.001
.0222.003
.0172.004
.0172.003
age / raace
.0224.001
.0094.003
.0122.004
..07
FF gender / age
.0144.001
-0011 001
-.007+.001
+00044.002
FF gender / race
0144..01
2.01
.000_.00
000
-0002 002
FF race /age
.01 .0154.001
.0044.001
.0054.002
2001
FF race/ gender
.0154.001
.000+.002
-008- 000
.002 004
UTK age gender
.0154.001
.0172.001
.0154.002
.0122.001
UTK age race
.0152.001
.010
.0101.001
.0041.002
UTK gender /age UTK :acee sace
.0094.001
.001 001
000061.000
.0003..001
.0094.001
.000g.001
0001_002
.001=.001
UTK race/ UTK race/ raooee//eeeerr gender
.021
.0134.001
.0074.002
.000_.01
.021 4.01
.003 002
-000 003
-.002 002
D' Group
A AUROC (f)
A SPD (4)
A F.D (4)
AOD (L)
CX
age
.0084.000 .0014.000 .00
.003+.001
CX
gender
.00
.000+.001
.001-.004 004
.001
CX
.005
.00224.00
.000+ 003
-0001 002
where A=R is the false positive rate, calculated for the respective group partitions of the test dataset. Due to our as- sumption that = the advantaged group, all measures are consequently E [0, 1]. where is the most fair. More details on Eq. (5) (7) are given in Apx. A.
Performance Metrics. We utilized accuracy as the perfor- mance metric on the FairFace and UTKFace datasets. In the case of CheXpert, we measured performance using the AU- ROC as established in previous work on this dataset (Zhang et al., 2022; Zong et al., 2023).
5. The Disparate Benefits of Deep Ensembles
In this section, we study the disparate benefits effect of Deep Ensembles using the experimental setup described in Sec. 4. First, we investigate the disparate benefits effect on the FairFace (FF) test dataset. Second, we apply the same models trained on FF to the UTKFace (UTK) dataset. UTK contains facial images similar to those of FF but from a different source, representing realistic setting for facial analysis under slight distribution shifts. Third, we inves- tigate the disparate benefits effect on the CheXpert (CX) medical imaging dataset to assess whether the impact on fairness of Deep Ensembles also occurs in other domains than facial analysis. Our analysis examines two primary facets of the disparate benefits effect: (i) the relationship between the number of ensemble members and the changes in performance and fairness violations (Fig. I); and (ii) for which targets and protected group attributes a statistically significant disparate benefits effect observed (Tab. 1).
Group Fairness Metrics. We measured group faimess us- ing empirical estimators for the fairness desiderata given by Eq. (2)- (4). Statistical Parity Difference (SPD) esti- mates the violation of the condition given by Eq. (2) and is computed as
PR
PRA=
where PR is the positive rate calculated on the partition of the test dataset D' = {(Ix, Vk, {x)] with the corre- sponding protected group attribute a. Equal Opportunity Difference (EOD) estimates the violation of the condition given by Eq. (3) and is computed as
A-1 TPR
(6)
where APR is the true positive rate, calculated for the respective group partitions of the test dataset. Average Odds Difference (AOD) (Bellamy et al., 2018) is an estimator of a relaxation of equalized odds (cf. Eq. (4)) computed as
Facial analysis (FF). The top row of Fig. I shows results for FF, where models were trained on target age and evaluated under the protected group attribute gender. We find that performance increases while fairness decreases when adding ensemble members. In particular, the largest decrease in faimess occurs when the first member is added to the Deep Ensemble. Tab. presents the change (A) in performance
1 APR 1-1 -TPR oo + APR -FPR A-PRA-ol (7)
1
2
4

---

The Disparate Benefits of Deep Ensembles
and fairness violations between individual models and
Deep Ensemble of 10 members for all tasks. While perfor- mance always increases for the Deep Ensemble (positive A), fairess does not necessarily increase after ensembling. We observe a disparate benefits effect with significant changes in the fairness metrics for four out of six target protected group combinations. This occurs primarily when individ- ual members already exhibit substantial fairness violations (gray cells in Tab. 1). The strongest disparate benefits effect (largest absolute A) has negative impact, thus decreas- ing fairness, i.e. increasing fairness violations. However, there are also cases where the Deep Ensemble is a more fair classifier than individual models (negative A).
Ensemble Members
Ensemble Members
Ensemble Members
Change in PR, TPR and FPR when adding mem- bers to the ensemble. Members trained on target variable age, evaluated on the FF test dataset with gender as protected group attribute, The advantaged group A = (male) has higher TPR and lower FPR, resulting in a net zero change in PR. The disadvantaged group A
Figure 2:
0 (female) has lower FPR and thus lower PR.
Facial analysis under a distribution shift (UTK). The middle row of Fig. 1 depicts the results the UTK dataset, with the same target and protected group for FF. Indi- vidual ensemble members exhibit higher fairness violations than for FE, which can be explained by the distribution shift between FF and UTK. However, the magnitude and behav- ior of the disparate benefits effect when adding ensemble members are similar to those observed with the FF dataset. The results for all target/ / group combinations are presented in Tab. 1. The results for UTK are generally similar those reported for the FF dataset. A notable exception is that the difference in SPD with target variable r race and protected group attribute a age is of opposite sign and larger for UTK than for FF.
Overall, our results demonstrate that Deep Ensembles can decrease faimess. Therefore, we next investigate the reason for their disparate benefits and explore mitigation strategies.
6. What is the Reason for Disparate Benefits?
In this section, we investigate the potential causes of the disparate benefits effect. The considered faireess metrics (Eq. -5)-Eq. (7)) are based on the per-group PR, TPR and FPR metrics. Therefore, as a first step. we analyze how these change when adding additional ensemble members. Although this analysis offers insight into why the disparate benefits effect occurs, it does not directly explain its under- lying cause. We hypothesize that the effect arises from dif- ferences in predictive diversity among ensemble members. Our empirical results support this hypothesis, indicating that a gap in the average predictive diversity between groups is a key driver of the effect. To validate this further, we designed two synthetic experiments to investigate this hypothesis in a controlled setting.
Medical imaging (CX). The bottom row of Fig. I shows the results on the CX dataset with age as protected group attribute. The disparate benefits effect also occurs in this task, but with a smaller magnitude, which is explained by the smaller performance gains of Deep Ensembles on this dataset. Similarly as with the facial dataset, the change in fairnsss after adding the first ensemble member is the most pronounced. The complete results for all protected groups are presented in Tab. 1. For the protected group age, the disparate benefits effect occurs under all fairness metrics. Moreover, there is significant difference in SPD for the protected group race, although individual models do not have substantial SPD and vice versa for EOD.
Changes to predictions for increasing ensemble size. We begin by examining how the metrics PR. TPR, and FPR for each group change when ensemble members are added, since the considered fairness metrics (Eq. (5) - Eq. (7)) are based on these. Fig. 2 shows these changes for the model trained on FF with age as target variable and gender as protected group, evaluated on the FF test dataset. The results show that the increase in SPD results from a decrease in the PR of the disadvantaged group when adding members to the ensemble, while the PR of the advantaged group remains stable (A R 0). The TPR of the disadvantaged group stays constant, but the TPR of the advantaged group increases, so the Deep Ensemble improves in correctly predicting Y = 1 only for the advantaged group, resulting in a higher EOD, The FPR of both groups decreases, more so for the disadvan- taged group, thus the Deep Ensemble improves in correctly predicting Y = 0 (since FPR is one minus the true negative rate). However, this does not offset the TPR disparity, re- sulting in higher AOD. The results for the remaining tasks are provided in Fig. 12 14 in the appendix.
Additional results. We investigate the influence of the model size of the individual ensemble members in Apx. F.2. Our results show that for tasks where the disparate benefits effect occurs, it increases with model size. Furthermore, we also analyze the disparate benefits effect under different model architectures. The results and more details are given in Apx. F.3, finding that the results provided in the main paper are consistent across architectures. Finally, we also show the disparate benefits effect for heterogeneous Deep Ensembles in Apx. F.4. Complementary to main investi- gation, we explore the notion of minimax fairness (Martinez et al., 2020) within our experiments in Apx. F.I.
5

---

The Disparate Benefits of Deep Ensembles
Disparate Benefits
Y=0,
Y=0, (21%)
Y=0, (71%)
0.6
Y=0, A-1 (29%)
0.8
Y=a, A-1 (23%)
Y=O, A-1 (12%)
Y=1, Am0 (21%)
Y=1, AmD (26%)
0.03
Y=1, Amo (14%)
DIV
Y-1, (24%)
DIV
0.6
Y=1, (29%)
Y-1, A-1 (2%)
0.02
0.4
0.4
0.01
0.3
(a) FF: Y= age, A gender
(b) UTK: Y= age, gender
(c) CX: A age
1.0
0.04
Y=0, A=0 (38%)
0.6
Y=0, A=0 (27%)
Y=0, A=0 (34%)
Equal Benefits
0.8
Y-a, (43%)
Y-a, (30%)
Y=D, A-1 (49%)
Y-1, A-0 (9%)
Y-1, (20%)
0.03
Y-1, A-0 (7%)
DIV
Y-1, A-1 (10%)
DIV
0.4
Y-1, (22%)
DIV
A-1 A-1, (10%)
0.02
0.2
0.2
0.01
X
I
1
0
1
0
Y
(d) FF: Y= race, A gender
(e) UTK: Y= race, A = gender
(f) CX: A gender
Figure 3: Average predictive diversity (DIV) per group A and target Y. Exemplary results for datasets FF, UTK and CX. Arrows indicate per-target group differences. Top row (a) (c): Signiticant disparate benefits (ef. Tab. 1) occur when DIV differences between groups are large. Bottom (d) (f): No significant disparate benefits (i.e. equal benefits) occur when DIV differences small.
Predictive diversity of ensemble members. The ensem- ble predictive distribution (Bq. (1)) is an average over the predictive distributions of its members. Therefore, the dis- parate benefits effect must stem from the characteristics of the predictive distributions of individual members. Previ- work investigated the predictive diversity of individual members the driving mechanism for the increase in the performance of Deep Ensembles (Abe et al., 2022b; Jeffares et al., 2023; Abe et al., 2024). Only individual members have different predictive distributions, combining them can lead to an ensemble that performs better than the individual models. While previous work investigates predictive diver- sity for individual inputs 2t, we interested in the average predictive diversity (potentially for given group) the test dataset. Following from the definition of predictive di- versity by Jeffares et al. (2023) (Theorem 4.3), the average predictive diversity DIV thus given by
is measure of how different individual ensemble members predict. Thus if there higher DIV for one group, this group has more potential to benefit from ensembling.
Consequently, we hypothesize that differences the aver- age predictive diversity per group cause the disparate bene- fits effect. To investigate this hypothesis, we consider two sets of tasks for FF, UTK and CX, respectively: those where the disparate benefits effect occurs and those where it does not occur (ef. Tab. 1). The results are depicted in Fig. 3, showing the average predictive diversity DIV per combina- tion of the target variable Y and the protected group attribute A. In agreement with our hypothesis, tasks showing the dis- parate benefits effect (Fig. 3a-c) have substantial differences in average predictive diversity between groups, while tasks without the effect (Fig. 3d-f) show only minimal differences. Results on all tasks are shown in Fig. 15 17 in the appendix.
Controlled experiment. To further test our hypothesis of the differences in average predictive diversity between groups causing the disparate benefits effect, we conducted controlled experiment. We use the FashionMNIST (Xiao et al., 2017) dataset and create binary classification prob- lmm with two targets: "T-shirt/top" (Y 0) vs "Shirt" (Y 1), and two groups, A O where the same image of the same target is concatenated twice and A where two different images of the same target are concatenated. This is done for both the train and test datasets. An illustration of inputs e for both targets and groups is given in Fig. 4a. Nat- urally, having an input consisting of two different images (A 1) should lead more diverse ensemble members, they may learn the top image, the bottom image
K
1
1 N 12
DIV =
log
ply = Us Ik: wr)
(8)
K
Ensemble Log-Li Likelliiiaood
log p(y = Dgg w.)) )
N
1),
n=1
Average Member Log-Likelihood
for a test dataset D' = {(xx, yk. ax) ) 1= and set of N models with parameters [w, J.v In Apx. C we provide further discussion about the average predictive diversity and how it arises as a natural measure of interest from a Bayesian perspective, Intuitively, the average predictive diversity DIV
6

---

The Disparate Benefits of Deep Ensembles
54
.94
96
1 -ADD (-+)
-92
.95
90
.22
Faimess
Performance
.90
86-
Ensemble Members
# Ensemble Members
# Ensemble Members
(a) Inputs per target and group.
(b) Change in performance and fairness for increasing sizes of the Deep Ensemble.
0.04
0.05
7w0 AnD 1298)
1299)
Tel And 125%
.075
.000
A-1 (50%)
A-o (50%)
TPR
FRR
0.03
.020
0.02
.000
-040
-020
0.01
.025
-060
Ensemble Members
Ense Ensemble Members
Ensemble Members
(c) DIV per Y and A.
(d) Change in PR, TPR and FPR for increasing sizes of the Deep Ensemble.
Figure 4: Controlled experiment. (a) Overview of the dataset, showing examplary input for different target group combinations. (b) The performance (accuracy) increases whereas fairness (I-SPD, I-EOD, I-AOD) decreases when adding more members to the ensemble. (c. d) The disparate benefits effect is caused by increased PR and TPR, as well as decreased FPR for the group with higher average predictive diversity A = 1. For the group with smaller average predictive diversity A = 0, there are no significant changes in PR, TPR and FPR.
or any combination of features from both. The concate- nation of two identical images (A = 0) does not provide additional information and therefore should not lead to increase in diversity of the ensemble members. This intu- ition is experimentally confirmed by having higher DIV for A = (Fig. 4c). We observe the same behavior regard- ing the change in performance, fairness (Fig. 4b) and PR, TPR and FPR (Fig. 4d) as for the real-world datasets that we investigate throughout the rest of the paper. In sum, the synthetic dataset (Fig. 4a) enforces more predictive diver- sity for one group (Fig. 4c), leading the disparate benefits effect (Fig. 4b, d).
The results are shown in Fig. 6. In order to summarize the average predictive diversity into a single number, we cal- culate a diversity score as [DIV DDIV==1.==1 DIV DiVy=1.A=ol + DDV 0 DIVY= the sum of the lengths of the arrows in Fig. 3, 4c (also in Fig. 15, 16, 17), shown in the rightmost plot in Fig. 6. We observe that for increasing o, the diversity score increases. Furthermore, we find that the changes (A) in accuracy, SPD, EOD and AOD due to ensembling increase as well, being highly correlated with the average predictive diversity. We provide the absolute accuracies, SPDs, BODs and AODs for individual ensemble members, the Deep Ensemble and the differences between those in Tab. 2 in the Appendix. In sum, for this second controlled experiment, we find that the higher the predictive diversity per group, the stronger the disparate benefits effect.
Intuitively speaking, this is
Controlling for predictive diversity. Next, we want to alter the level of predictive diversity and further investigate its relationship with the observed changes fairness met- rics. We used a similar setup as in the previous controlled experiment, i.e., the FashionMNIST dataset and the same targets. However, we used a different methodology to define the groups, in order to vary the level of predictive diversity in the advantaged group A = 1. We defne an input for the disadvantaged group A- = 0 as the original image con- catenated with uniform random noise of the same size (each pixel is drawn independently). Furthermore, we define an input for the advantaged group A = 1 as the original image concatenated with a linear interpolation between different image of the same target and uniform random noise. The linear interpolation coefficient is a, where = 0 results in solely uniform random noise (A = 0 and A = 1 are equiv- alent then) and a = results in two concatenated images from the same label. Thus, for a = 1, A = 1 is equivalent to how it was defined in the previous controlled experiment. An illustration of inputs for both targets and groups under different values of is given in Fig. 5.
7. Mitigating the Unfairness caused by the Disparate Benefits Effect
In this section, we investigate strategies to mitigate the un- faimess due to the disparate benefits effect in the cases where faimess decreases due to ensembling. We focus on in- terventions that can be applied to trained ensemble members, thus operate in a post-processing manner. This allows to leverage the existing ensemble members as opposed to pre- and in in-processing methods that would require expensive re-training of individual members.
First, we analyze whether it would be possible non- uniformly weight ensemble members to attain better trade- off between performance and fairness violations in the Deep Ensemble. Second, we examine the characteristics of the predictive distribution of the Deep Ensemble. We find that
7

---

The Disparate Benefits of Deep Ensembles
a=)===
a
0.2
(c)
0.4
(d) (d)== a=
Figure 5: Controlling for predictive diversity. Inputs per larget Y and group A for different levels of linear interpolation factor
Accuracy
0.03
0.04
0.02
0.05
Diversity
0.02
SPD
4 AOD
0.05
0.02
0.01
0.01
0.00
0.00
0.00
0.0
0.4
1.0
0.0
0.2
0.4
1.0
0.0
0.4
1.0
0.0
0.2
0.4
0.0
0.2
0.4
1.0
Figure 6: Change (A) in accuracy, SPD, EOD and AOD due ensembling, as well as the diversity score for different levels of linear interpolation factor a. The disparate benefits effect stronger for experimental conditions with higher average predictive diversity.
Deep Ensembles are more calibrated than individual mem- bers our considered tasks and consequently more sensi- tive to the selected prediction threshold. Inspired by this finding, we investigate a group-dependent threshold opti- mization approach (Hardt et al., 2016), often simply referred to as Hardt post- prot-psocsssing (HPP) in the algorithmic fairness literature, to mitigate the negative impact of the disparate benefits effect of Deep Ensembles. The results show that HPP is highly effective in ensuring fairer predictions while maintaining the enhanced performance of Deep Ensembles.
ual predictive distributions (Ovadia et al.. 2019; Seligmann et al., 2023). We empirically validate this finding considered datasets by evaluating the Expected Calibration Error (ECE) (Naeini et al., 2015). The results are given in Fig. 7a, showing that Deep Ensembles are indeed more calibrated (lower ECE) than individual members for all con- sidered datasets with all available targets Y. Being more calibrated means that the predicted probabilities correspond better to the actual outcomes. Importantly, better calibration increases the sensitivity to the prediction threshold, even slight changes can significantly impact predictions (Cohen & Goldszmidt, 2004). Representative results are shown in Fig. 7b. For Deep Ensembles (Fig. 7b, left), there are clear optimal values for prediction thresholds for each group (dashed lines) that are consistent across runs. For individual members (Fig. 7b, right), there is no clear optimal value. Any threshold between 0.2 and 0.8 leads to similar accuracy and the optimal value has a high variance between runs. The complete results and analysis are provided in Apx. F.6.
Weighting the ensemble members. We analyze whether it is possible to improve the performance fairness trade-off of Deep Ensembles by assigning different weights to each en- semble member, as opposed the standard uniform weights reflected in Eq. (1). Although the results, shown in Fig. 28 in the appendix, suggest the possibility of better trade-offss, developing method that systematically identifies the op- timal weights to achieve significantly improved outcomes remains challenging. Specifically, we tried two approaches: (i) selecting the best weighting on the validation set and (ii) weighting the individual ensemble members proportional to their fairness violations. Both methods lead to ensembles that average in between the performance and fair- ness violations of the Deep Ensemble with standard uniform weighting and individual models, with high variance. A detailed discussion is provided in Apx. F.5.
Hardt Post-Processing (HPP). The sensitivity of Deep Ensembles the selected threshold suggests that group- specific threshold optimization could be an effective unfair- mitigation strategy. A commonly used approach for this purpose in the algorithmic fairness literature is Hardt post- processing (HPP) (Hardt et al., 2016). As a post- post-processing method, HPP can be applied to the Deep Ensembles predic- tive distribution without changing how individual models are trained. Moreover, HPP was shown to be Pareto domi- nant addressing equalized odds fairness constraints com- pared to other fairness interventions (Cruz & Hardt, 2024), while adding minimal computational overhead. Notably, al- though HPP is classical method to improve the fairness of ML models, it hasn't 't been investigated for Deep Ensembles.
Better calibration leads to more sensitivity to the predic- tion threshold. Next, we analyze the predictive distribution of Deep Ensembles to identify mechanisms to mitigate the additional unfairness caused by the disparate benefits ef- fect. Deep Ensembles are known to be better calibrated than individual models because they average over individ-
8

---

The Disparate Benefits of Deep Ensembles
Ensemble
Individual Members
0.20
Hembers Doep E
0.825
A=1 (19%)
Accuracy (+)
ECE [+)
0.15
UTK
CX
0.800
0.10
0.775
0.750
0.05
0.725
0.00
0.700
gender
race
age Target
gender
race
n.t.
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
Threshold
Threshold
(a) ECE Deep Ensemble and members
(b) Threshold sensitivity (FF, Y= age, == race).
Figure 7: Calibration & prediction threshold. (a) Deep Ensembles are more calibrated than individual members, thus have lower ECE for all considered tasks. (b) As a result, they are more sensitive to the selection of the prediction threshold.
0.200
0,150
0.150
0,100
Members
SPD
0.100
EOD
0.100
AOD
0.080
Deep Ensemble rest-precessing
0,060
0.050
0.050
0.040
0.80
0.81
0.82
0.80
0.81
0.82
0.79
0.80
0.81
0.82
Accuracy (-)
Accuracy (-)
Accuracy (-)
Figure 8: Impact of applying HPP on the individual members and the Deep Ensemble on FE, Models are trained on target variable age, evaluated using protected attribute gender. Dotted lines indicate average fairness violation of individual members on the validation set, dashed line indicates 0.05 faimess violation. Arrows denote applying HPP the Deep Ensemble (red) or the individual members (grey) with those two target fairesss violations. The Deep Ensemble maintains or improves in accuracy while also improving in fairness.
We apply HPP to the Deep Ensembles considering each of the three group fairness metrics (SPD, EOD and AOD) with the aim of satisfying the fairness desiderata given by Eq, (2) - Eq. (4). Representative results for the FF dataset with target variable age and protected group attribute gender are depicted in Fig. 8. The complete results for all tasks are given in Tab. 5 19 in the appendix. As seen in Fig. 8, after applying HPP, the Deep Ensemble (red dots) attains the same level of fairness violation (y-axis) as individual ensemble members (gray dots) exhibit on average, without sacrificing any performance (x-axis). This achieved by setting the desired fairness violation for HPP to the aver- age violation of the individual members on validation set (dotted line). Noteworthy, the Deep Ensemble's 's accuracy even increases slightly when optimizing the decision thresh- olds through HPP to values different from 0.5, which is the implicit threshold when using the argmax. Furthermore, we compare the Deep Ensemble and individual ensemble members after applying HPP with target faimess viola- tion of 0.05 (dashed line). Here, the performance of the Deep Ensemble remains robust, even the performance of individual members declines.
igation strategies. Specifically, we demonstrated the exis- tence of the effect in experiments on three vision datasets, considering 15 different tasks and five model architectures. We investigated potential causes for this effect, our findings suggesting differences in the predictive diversity of the en- semble members an explanation. Finally, evaluated different approaches to mitigate the additional unfairness due to the disparate benefits effect. We find that Deep En- sembles are better calibrated than the individual ensemble members and thus more sensitive to the prediction thresh- old. Consequently, the classical HPP method is particularly suited to this setting, as it leverages the better-calibrated predictive distribution of the Deep Ensemble. Remarkably, HPP is thus very effective in mitigating unfairness for Deep Ensembles while preserving their superior performance.
The main limitations of our study are that we focus on vision tasks, and hence on ensembles of convolutional DNNs. Fur- thermore, the three considered group fairness metries, while widely used, are not sufficient guarantee fair outcomes, as fairness can't be reduced to satisfying any single metric alone. In future work, we thus plan explore other notions of fairness, such as individual fairness, and extend our anal- ysis to other types of models and datasets, for instance, in the language domain. Furthermore, we intend to investigate the disparate benefits effect Deep Ensembles where pre- or in processing faimness interventions have been applied individual ensemble members.
8. Conclusion
In this work, reveal the disparate benefits effect of Deep Ensembles, analyze its underlying reason, and explore mit-

---

The Disparate Benefits of Deep Ensembles
Impact Statement
sary? In Advances in Neural Information Processing Sys- tems, volume 35, pp. 33646-33660. Curran Associates, Inc., 2022b.
Our study unveils the disparate benefits effect of Deep En- sembles. which potentially causes socially harmful predic- tions. Although we investigate its origin and explore way to mitigate it, this intervention alone not guarantee fair outcomes. Researchers and practitioners should keep in mind that the fairness of predictions of any ML model ap- plied in the real world can't it be reduced to any single metric and must be carefully assessed depending on the application.
Abe, T., Buchanan, E. K., Pleiss, G., and Cunningham, J. P. Pathologies of predictive diversity in deep ensembles. Transactions on Machine Learning Research, 2024.
Agarwal, A., Beygelzimer, A., Dudík, M., Langford, J., and Wallach, H. A reductions approach to fair classification. In International conference on machine learning, pp. 60- 69, 2018.
Acknowledgements
We thank Julien Colin, Vihang Patil and Aditya Gulati for their constructive feedback on our work. Further- more, we thank André Cruz for his help with adapting the error-parity package we use for our post-processing experiments.
Arnaiz-Rodrigue A. and Oliver, N. Towards algorithmic fairness by means of instance-level data re-weighting based on shapley values. In ICLR 2024 Workshop on Data-centric Machine Learning Research (DMLR), 2024.
Ashukha, A., Lyzhov, A., Molchanov, D., and Vetrov, D. Pitfalls of in-domain uncertainty estimation and ensem- bling in deep learning. In International Conference on Learning Representations, 2020.
The ELLIS Unit Linz, the LIT AI Lab, the Institute for Machine Learning, are supported by the Federal State Up- per Austria. We thank the projects FWF AIRI FG 9-N (10.55776/FG9), AlAGreeeHeeaingGridss (FFG- 899943), Stars4Waters
Barocas, S. and Selbst, A. D. Big data's disparate impact. California Law Review, 104:671, 2016.
FWF Bilateral Artificial Intelligence (10.55776/C0E12). We thank NXAI GmbH, Audi AG, Silicon Austria Labs (SAL). Merek Healthcare KGaA, GLS (Univ. Waterloo), TŪV Holding GmbH, Software Competence Center Hagen- berg GmbH, dSPACE GmbH, TRUMPF SE Co. KG.
Barocas, S., Hardt, M., and Narayanan, A. Fairness and Machine Learning: Limitations and Opportunities. MIT Press, 2023.
The ELLIS Unit Alicante Foundation acknowledges sup- port from Intel corporation, a nominal grant received at the ELLIS Unit Alicante Foundation from the Regional Govern- ment of Valencia Spain (Convenio Singular signed with Generalitat Valenciana, Conselleria de Innovación, Indus- tria, Comercio y Turismo, Dirección General de Innovación) and grant by the Banc Sabadell Foundation. In addition, this work is partially funded by the European Union EU HE ELIAS - Grant Agreement 101 102377. Views and opin- ions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Health and Digital Executive Agency (HaDEA).
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., Nagar, S., Ramamurthy, K. N., Richards, J., Saha, D., Sattigeri, P., Singh, M., Varshney, K. R., and Zhang, Y. Ai fairness 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias. ArXiv, 1810.01943, 2018.
Bhaskaruni, D., Hu, H., and Lan, C. Improving prediction fairness via model ensemble. In 2019 IEEE 31st Inter- national Conference on Tools with Artificial Intelligence (ICTAI), pp. 1810-1814, 2019.
Kajetan Schweighofer acknowledges travel support from ELISE (GA no 951847).
Bhusal, N., Shukla, R. M., Gautam, M., Benidris, M., and Sengupta, S. Deep ensemble learning-based approach to real-time power system state estimation. Interna- tional Journal of Electrical Power & Energy Systems, 129:106808 2021.
References
Abe, T., Buchanan, E. K., Pleiss, G., and Cunningham, J. P. The best deep ensembles sacrifice predictive diversity. In I Can't Believe It's 's Not Better Workshop: Understanding Deep Learning Through Empirical Falsification, NeurIPS, 2022a.
Binns, R. Fairness in machine learning: Lessons from politi- cal philosophy. In Conference on fairness, accountability and transparency, pp. 149-159, 2018
Black, E.,, Leino, K., and Fredrikson, M. Selective ensem- bles for consistent predictions. In International Confer- ence on Learning Representations, 2022a.
Abe. T., Buchanan, E. K., Pleiss, G., Zemel, R., and Cun- ningham, Deep ensembles work, but are they neces-
10

---

The Disparate Benefits of Deep Ensembles
Black, E., Raghavan, M., and Barocas, S. Model multi- plicity: Opportunities, concerns, and solutions. In Pro- ceedings of the 2022 ACM Conference Fairness, Ac- countability, and Transparency, FAceT '22, pp. 850-863. Association for Computing Machinery, 2022b.
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., and Zemel, R. Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Confer- ence, ITCS '2, pp. 214-226. Association for Computing Machinery, 2012.
Carey, A. N. and Wu, X. The statistical fairness field guide: perspectives from social and formal sciences. Al and Ethics, 3(1):1-23, 2023.
Feldman, M., Friedler, A.. Moeller, J., Scheidegger, C., and Venkatasubramanian, S. Certifying and removing dis- parate impact. In Proceedings of the 21th ACM SIGKDD International Conference Knowledge Discovery and Data Mining, KDD '15, pp. 259-268. Association for Computing Machinery, 2015.
Caton, S. and Haas, C. Fairness in machine learning: A survey. ACM Computing Surveys, 2023.
Chouldechova, A. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data, 5(2):153-163 2017.
Fort, S., Hu, H., and Lakshminarayanan, B. Deep ensembles: A loss landscape perspective. ArXiv, 1912.02757, 2019.
Cohen, I. and Goldszmidt, M. Properties and benefits of cal- ibrated classifiers. In European conference on principles of data mining and knowledge discovery, pp. 125-136. Springer, 2004.
Garg, P., Villasenor, J., and Foggo, V. Fairness metries: A comparative analysis. 2020 IEEE international conference on big data (Big Data), pp. 3662-3666, 2020.
Gichoya, J. W., Banerjee, I., Bhimireddy, A. R., Burns, J. L., Celi, L. A., Chen, L.-C., Correa, R., Dullerud, N., Ghassemi, M., Huang, S.-C., Kuo, P.C.., Lungren, M. P., Palmer, L. J.,, Price, B. J., Purkayastha, S., Pyrros, A. T., Oakden-Rayner, L., Okechukwu, C., Seyyed-Kalanten L., Trivedi, H., Wang, R., Zaiman, Z., and Zhang, Ai recognition of patient race in medical imaging: mod- elling study. The Lancel Digital Health, 466):4006e414, 2022.
Cooper, A. F., Lee, K., Choksi, M. Z., Barocas, S., De Sa, C., Grimmelmann, J., Kleinberg, J., Sen, S., and Zhang, B. Arbitrariness and social prediction: The confounding role of variance in fair classification. Proceedings of the AAAI Conference on Artificial Intelligence, 38(20): 2024.
Corvett-Davies, S., Pierson, E., Feller, A., Goel, S., and Huq, A. Algorithmic decision making and the cost of fair- ness. In Proceedings of the 23rd acm sigkdd international conference on knowledge discovery and data mining, pp- 797-806, 2017.
Gohar, U., Biswas, S., and Rajan, H. Towards understanding fairness and its composition in ensemble machine learn- ing. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), pp.
Coston, A., Rambachan, A., and Chouldechova, A. Char- acterizing fairness over the set of good models under selective labels. In Proceedings of the 38th International Conference on Machine Learning, volume 139 Pro- ceedings of Machine Learning Research, pp. 2144-2155, 2021.
2023.
Grgić-Hlača, N., Zafar, M. B., Gummadi, K. P., and Weller, A. On fairness, diversity and randomness in algorithmic decision making. ArXiv, 10208, 2017.
Hardt, M., Price, E., Price, E., and Srebro, N. Equality of opportunity in supervised learning. In Advances in Neu- ral Information Processing Systems, volume 29, Curran Associates, Inc., 2016
Cruz, A. and Hardt, M. Unprocessing seven years of algo- rithmic fairness. In The Twelfth International Conference on Learning Representutions, 2024.
Diana, E., Gill, W., Kearns, M., Kenthapadi, K., and Roth, A. Minimax group fairness: Algorithms and experiments. In Proceedings of the 2021 AAAI/ACM Conference on Al, Ethics, and Society, AIES '21, pp. 66-76. Association for Computing Machinery, 2021.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.
Irvin, J., Rajpurkar. P., Ko, M., Yu. Y., Ciurea-Hluss, S.. Chute, C., Marklund, H., Haghgoo, B., Ball, R., Sh- panskaya, K., Seekins, J., Mong, D. A., Halabi, S. S., Sandberg, J. K., Jones, R., Larson, D. B., Langlotz, C. P. Patel, B. N., Lungren, M. P., and Ng, A. Y. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. Proceedings of the AAAI Conference on Artificial Intelligence, 33:590 2019.
Dolezal, M., Srisuwananukom,, A., Karpeyev, D., Ramesh, S., Kochanny, S., Cody, B., Mansfield, A. S., Rakshit, S., Bansal, R., Bois, M. C., Bungum, A. O., Schulte, J. J., Vokes, E. E., Garassino, M. C., Husain, A. N., and Pear- son, A. T. Uncertainty-informes deep learning models enable high-confidence predictions for digital histopathol- ogy. Nature Communications, 13 13(1):6572, 2022.

---

The Disparate Benefits of Deep Ensembles
Izmailov, P., Vikram, S., Hoffman, M. D., and Wilson, A. G. G. What bayesian neural network posteriors really like? In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 1629-4644) 2021.
Lakshminarayanan, B., Pritzel, A., and Blundell, C. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Process- ing Systems, volume 30. Curran Associates, Inc., 2017.
Lanitis, A., Taylor, C., and Cootes, T. Toward automatic simulation of aging effects on face images. IEFE Trans- actions Pattern Analysis and Machine Intelligence, 24 (4):442 455, 2002.
Jain, S., Smit, A., Truong, S. Q.. Nguyen, C. D., Huynh, M.-T., Jain, M., Young, V. A., Ng, A. Y., Lungren, M. P.. and Rajpurkar, P. Visualchexbert: addressing the discrep- ancy between radiology report labels and image labels. In Proceedings of the Conference Health, Inference, and Learning, pp. 105-115. Association for Computing Machinery, 2021.
Lee, Y., Yao, H., and Finn, C. Diversify and disambiguate: out-of-distribution robustness via disagreement. In The Eleventh International Conference Learning Repre- sentations, 2023.
Jeffares, A., Liu, T., Crabbé, J., and van der Schaar, M. Joint training of deep ensembles fails due to learner collusion. In Advances in Neural Information Processing Systems, volume 36, pp. 13559-13589. Curran Associates, Inc., 2023.
Liu, L. T., Dean, S., Rolf, E., Simchowitz, M., and Hardt, M. Delayed impact of fair machine learning. In International Conference on Machine Learning, pp. 3150-3155 2018.
Long, C. X., Hsu, H., Alghamdi, W., and Calmon, F. P. Arbitrariness lies beyond the fairness-accuracy frontier. ArXiv, 3306.09425, 2023.
Jung, S., Park. T., Chun, S., and Moon, T. Re-weighting based group fairness regularization via classwise robust optimization. In The Eleventh International Conference on Learning Representations, 2023.
Martinez, N., Bertran, M., and Sapiro, G. Minimax pareto fairness: A multi objective perspective. In Proceedings of the 37th International Conference on Machine Learn- ing, volume 119 of Proceedings of Machine Learning Research, pp. 6755-6764, 2020.
Kamiran, F. and Calders, T. Data preprocessing techniques for classification without discrimination. Knowledge and information systems, 33(1): 2012.
Kamishima, T., Akaho, S., Asoh, H., and Sakuma, Fairnes-awwaee classifier with prejudice remover regu- larizer. In Machine Learning and Knowledge Discovery in Databases, pp. 35-50. Springer Berlin Heidelberg, 2012.
Marx, C., Calmon, F., and Ustun, B. Predictive multiplicity in classification. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Pro- ceedings of Machine Learning Research, pp. 676566677777 2020.
Karkkainen, K. and Joo, J. Fairface: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation.. In Proceedings of the IEEF/CVF Winter Conference on Applications of Computer Vision, pp. 1548- 1558, 2021.
Mchrabi, N., Morstatter, F., Saxena, N., Lerman, K., and Galstyan, A. A survey on bias and fairness in machine learning. ACM Computing Surveys, 54(6), 2021.
Naeini, M. P., Cooper, G., and Hauskrecht, M. Obtaining well calibrated probabilities using bayesian binning. In Proceedings of the AAAI conference on artificial intelli- gence, volume 29, 2015.
Kenfack, P. J., Khan, A. M., Kazmi, S. A., Hussain, R., Oracevic, A., and Khattak, A. M. Impact of model en- semble on the fairness of classifiers in machine learning. In 2021 International Conference on Applied Artificial Intelligence (ICAPAI), pp. 1-6, 2021.
Ovadia, Y., Fertig, E., Ren, J., Nado, Z., Sculley, D., Nowozin, S., Dillon, J.. Lakshminarayanan, B., and Snoek, Can you trust your model's uncertainty? evaluat- ing predictive uncertainty under dataset shift. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
Kleinberg, J., Mullainathan, S., and Raghavan, M. Inherent trade-offss in the fair determination of risk scores. ArXiv, 1609.05807, 2017.
Ko, W.-.., D' D'souza, D., Nguyen, K., Balestriero, R., and Hooker, S. Fair-ensemble: When fairness naturally emerges from deep ensembling. ArXiv, 2303.00586, 2023.
Pagliardini, M., Jaggi, M., Fleuret, F., and Karimireddy, S. Agree to disagree: Diversity through disagreement for better transferability. In The Eleventh International Conference on Learning Representations, 2023.
Kusner, M. J., Loftus, J., Russell, C., and Silva, R. Counter- factual fairness. Advances in neural information process- ing systems, 30, 2017.
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z.,, Gimelshein, N., Antiga,
12

---

The Disparate Benefits of Deep Ensembles
L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, I.,, Bai, J., and Chintala, S. Pytorch: An imperative style, high-pest hegh-performance deep learning library. Advances in Neural Information Processing Systems, 32, 2019.
Xiao, H., Rasul, K., and Vollgraf, R. Fashion-mniss novel image dataset for benchmarking machine learning algorithms. ArXiv, 1708.07747, 2017.
Zafar, M. B., Valera, I., Gomez Rodriguez, M., and Gum- madi, K. P. Fairness Beyond Disparate Treatment &; Disparate Impact: Learning Classification without Dis- parate Mistreatment. In International Conference World Wide Web, pp. 1171-118800 2017.
Pombal, J., Cruz, A. F., Bravo, J., Saleiro, P., Figueiredo, M. A. T., and Bizarro, P. Understanding unfairness in fraud detection through model and data bias interactions. ArXiv, 2207.06273, 2022.
Zhang, B. H., Lemoine, B., and Mitchell, M. Mitigating un- wanted biases with adversarial learning. In AIES' '78, pp. 335-340. Association for Computing Machinery, 2018.
Radosavovic, I., Kosaraju, R. P., Girshick, R., He, K., and Dollar, P. Designing network design spaces. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Zhang, H., Dullerud, N., Roth, K., Oakden-Rayner, L., Pfohl, S., and Ghassemi, M. Improving the fairness of chest x-ray classifiers. In Proceedings of the Conference Health, Inference, and Learning, volume 174 of Pro- ceedings of Machine Learning Research, pp. 204-233. 2022.
Rame, A. and Cord, M. DICE: Diversity in deep ensembles via conditional redundancy adversarial estimation. In International Conference on Learning Representations, 2021.
Sagawa, S., Koh, P. W., Hashimoto, T. B., and Liang. P. Distributionally robust neural networks. In International Conference on Learning Representations, 2020.
Zhang, Z., Song, Y., and Qi. H. Age progression/regres- sion by conditional adversarial autoencoder. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
Schweighofer, K., Aichberger, L.., Telanskyi, M., Klambauer, G., and Hochreiter, S. Quantification of uncertainty with adversarial models. Advances in Neural Information Processing Systems, volume 36, pp. 19446-19484. Cur- ran Associates, Inc., 2023.
Zietlow, D., Lohaus, M.. Balakrishnan, G., Kleindessner, M., Locatello, F., Schölkopf, B., and Russell, C. Lev- eling down in computer vision: Pareto inefficiencies in fair deep classifiers. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10400-10411, 2022.
Selbst, A. D., Boyd. D., Friedler, A.. Venkatasubrama- nian, S., and Vertesi, J. Fairness and abstraction in ciotechnical systems. In Proceedings of the conference on fairness, accountability, and transparency, pp. 59-68, 2019.
Zong, Y., Yang, Y., and Hospedales, T. MEDFAIR: Bench- marking faimess for medical imaging. In The Eleventh International Conference on Learning Representations, 2023.
Seligmann, F., Becker, P., Volpp, M., and Neumann, G. Beyond deep ensembles: A large-scale evaluation of bayesian deep learning under distribution shift. Ad- vances in Neural Information Processing Systems, vol- ume 36, pp. 29372-29442 Curran Associates, Inc., 2023.
Tan, M. and Le, Q Efficientnetv2: Smaller models and faster training. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Pro- ceedings of Machine Learning Research, pp. 10096- 10106, 2021.
Wilson, A. and Izmailov, Bayesian deep learning and probabilistic perspective of generalization. In Advances in Neural Information Processing Systems, volume 33, pp. 4697-4708. Curran Associates, Inc., 2020.
Xia, I.,, Chen, C., and Aggarwal, J. View invariant hu- man action recognition using histograms of 3d joints. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2012 IEEE Computer Society Conference on, pp. 20-27, 2012.
13

---

The Disparate Benefits of Deep Ensembles
A. Details on Computing Group Fairness Metrics
Group fairness metrics, as previously discussed, are based on assumptions related the independence the prediction with respect to the protected attribute and the target. For completeness, we present below how to estimate the metrics given in Eq. (5) - Eq. (7) with samples. We start by defining the number of correct (TP, TN) and wrong decissions (FP, FN) of model:
K TP =
K
Σ (1(())) > 8 1yk = 1},
TN =
Σ ([(()))
< 1 = 0]
k=1
==1
FP
1[f(x)) > t 1yv O].
FN Σ [[f())) < 1ly 1].
Here, D' = {(Tx Ve, [=))== the test dataset; a datapoint (æ', Uk. 02) consists of input features, observed outcome and protected group attribute; f((2x) is the model's predicted value for zg; and is the classification threshold. To compute these metrics for specific value of protected group attribute A (e.g., male for gender), we add the term [ al to each computation, resulting in group-specific true positives TP A=a. negatives TN A= false positives FPA=a. and false negatives FNA=a-
Once all these buliding blocks are computed, the group-specific Positive Rate (PRA=a) is given by
TPA=a + A=a
PR = P( = A= TPA= + FPA= + TN + FN
Finally, equal opportunity and equalized odds depend the conditional true/false negative/positive rates, depending the values of the protected group attribute A and are calculated as:
TP TPA=a
TPR = P(K = Y = =
TP APA= + FN
TNA=o
TNR P(Ý 0 Y 0, 0)
FAA= + TNA=a A=a
FP
FPR p(ÿ = 0,4 0)
FP A=o + TN
FN
ANR = P( 0 Y = 1,A a)
TP TP + FN
A.1. Group fairness metries factorization of P(Y, Y A).
In order to analyze the trade-offss and connections between different statistical group faimesss metries, common approach is to use the factorization of P(Y,Y Y A), which offers a clear intuition of the incompatibilities between some of them. Then, all the introduced metrics are related as per:
P( Y,A = X P(Y
1A
P(Y Ý, A = PÝ A 1)
P( Y,A= P( A = 0)
P(Y Ý, A= 0)
P( A= 0)
(9)
Separation
Prevalence Fq.
Sufficiency YLAY
Independence
y PLAYY
YLA
Y 14
AOD, BOD
e.g. SPD
For instance, it suggests that, if the target prevalence is different across groups and the model is perfectly calibrated (sufficiency), then separation and independence conditions cannot be satisfied simultaneously.
B. Biases and Group Unfairness
Biases induced by datasets have been studied in Pombal et al. (2022). They consider the joint distribution P(X, Y, A). Generally there is bias under distribution shift with P*(X, Y, A) # P(X, Y, A), where the distribution after the shift P
14

---

The Disparate Benefits of Deep Ensembles
the model is applied is different the distribution P the training data was sampled from. Furthermore, Pombal et al. (2022) consider biases in the training data distribution. A bias arises if
P(X, Y) + P(X,Y Y A))
(10)
as well as if P(A) is not uniform distribution. Note that P(X,Y Y A) can be factorized into
P(X,Y Y A) P(X Y, P(X A)
(11)
P(Y X, A) P(Y A)
Different parts of the factorization in Eq. (11) can lead to unfairness:
P(Y) # P(Y A) corresponds to a prevalence disparity, i.e., the class probability depends on the protected attribute. present in FairFace dataset since has been specifically curated to avoid this problem (Karkkainen
This imbalance & Joo, 2021). However, observe it in the UTKFace and CheXpert datasets.
P(X Y) + P(X Y, A) reflects group-wise disparity of the class-conditional distribution, and indicates that the feature space is distributed differently depending the protected attribute, which is undesirable, since the likelihood of p(D w)) could vary across protected groups, leading to potentially different per-group error rates and hence unfairness. The experimental results in Fig. (3) illustrate differences in the likelihood of the dataset for different (A,
P(Y X) + P(Y X, A) represents noisy targets. In this case, the distribution of Y given X depends on the protected group attribute. The classification experiments in Tab. 1, Fig. I and Fig. 2 analyze metrics related to P(Y X, A) and the resulting accuracy and fairness violations.
C. Bayesian Perspective on the Average Predictive Diversity
In this section, motivate the average predictive diversity DIV (ef. Eq. (8)) from Bayesian perspective. Given are training dataset D = {( = as well as a test dataset D = {x yx) yx)) the protected attribute is omitted for brevity in this section. Furthermore, we are given prior distribution p(w) on the model parameters.
Marginal Likelihood. Through Bayes' rule, obtain posterior distribution over the model parameters given the training dataset p(w D)= p(D
Recall that the marginal likelihood is given by p(D) -Sw p(D w)p( w)ww, ie., the expected likelihood on the dataset over all models according to their prior distribution. Intuitively, the marginal likelihood thus measures how well possible models represent the given dataset.
The disparate benefits effect occurs on a test dataset D'. Consequently, we are interested in the marginal likelihood under the test dataset p(D"). For the test dataset D', the posterior distribution given the training dataset p(w D) is the new prior distribution p(w). The marginal likelihood under the test dataset thus given by
K
N
K plu
P(D') = / II ys æ.k un) p(w) dun R k=1
I
Dk. w.)
(12)
N
with w, drawn according to p(w) = p(w D). In practice the set of model parameters (w, ) obtained from the training of the Deep Ensemble used approximate the integral.
Likelihood Ratio. If the likelihood under the posterior predictive distribution
P(D') II
w ply yk 0xg w) p(w D) dw
II
1
ply yk
Tb. w.)
(13)
again with W, drawn according to p(w) p(w D). does not differ from the marginal likelihood, there is no difference between predicting with a single model sampled according to the posterior and predicting with the ensemble of all sampled models. Thus, we investigate the likelihood ratio p(D')/ //()))) as a natural measure of diversity the predictions of the models that make up the ensemble.
15

---

The Disparate Benefits of Deep Ensembles
For practical purposes, it is more convinient work with log-likelihoods rather than likelihoods, the products in Eq. (13) and Eq. (13) become sums. Therefore, we consider the logarithm of the likelihood ratio, leading to
p(D') ((D))
log
= log P(D) - log p (D').
(14)
Inserting Eq. (12) and Eq. (13) into Eq. (14) we obtain
F(D')
N ply = yk
log
log
BRs w..)
Σ
log ply = yk Iks W..)
(15)
((D))
k=1
A=1
= K DIV,
with DIV as defined in Eq. (8), which what we wanted to show. Eq. (15) Ex= DIV, with the predictive diversity DIV given by Theorem 4.3 in Jeffares al. (2023). To mitigate the impact of different dataset sizes, common practice to divide log-likelihoods by the number of datapoints in the dataset K when comparing between datasets of different sizes. Doing for the logarithm of the likelihood ratio, 1/K log (D(D))/((D)) is an approximation of the Jensen gap (Eq. (5) in Abe et al. (2022a) and Eq. (3) in Abe et al. (2024)) with K samples in the dataset D'
D. Details of the Experimental Setup
The code to reproduce our experiments is available at https://github.comm com/://muuoooommmmm////////mm hotps//ooooooom/////s/aaatttttt benefits.
D.1. Datasets
We conducted all our experiments on facial analysis and medical imaging datasets. In the following, we provide details about the datasets.
Facial Analysis. We used two widely used facial analysis datasets, FairFace (Karkkainen & Joo, 2021) (License: CC BY 4.0) and UTKFace2 (Zhang et al., 2017) (License: research only, not commercial). FairFace created for advancing research in fairness, accountability and transparency in computer vision as it addresses the lack of diversity in existing face datasets used for research purposes. The FairFace dataset comprises 108,501 facial images collected from publicly available sources, such as Flickr and Google Images, and covers a diverse range of demographics, including various ethnicities, ages, genders, and skin tones. The dataset includes annotations for gender, age, and ethnicity. UTKFace contains over 20,000 facial images of individuals collected from the publicly available datasets UTKinect (Xia et al., 2012) and FGNET (Lanitis et al., 2002), well images scraped from the internet. It includes annotations for three demographic attributes: age, gender, and ethnicity.
Medical Imaging. We used the medical imaing dataset Che (Irvin et al., 2019) (License: Stanford University Dataset Research Use Agreement). It consists of a large publicly available dataset of 224,316 316 chest X-rays along with associated ladiolggist-labeled annotations for the presence or absence of 14 different thoracic pathologies. It is designed to address the challenges of class imbalance and target noise commonly encountered medical image classification tasks. CheXpert has become a widely used benchmark dataset the field of medical imaging and has been instrumental in advancing research on automated chest radiograph interpretation, particularly in the context of deep learning approaches. We use the recommended targets provided by Jain et al. (2021) (visualCheXbert targets) and group attributes provided by Gichoya al. (2022)'.
D.2. Models and Training
We used the ResNet ResNet18/24/50 8/24/50, Hyperparameters reported in the main paper were the result of an initial manual tuning the respective validation
RegNet-Y 800MF and EfficientNetV2-S implementations of Pytorch (Paszke et al., 2019).
'Obtained from ht httppe://githbbb.cmm///////
using the [Padding=0.25) version.
2Obtained from htt original source hatt:p/:///uussnnggggttttboboot does no longer work.
jaaaaaijikkauuaa hhtttkk--a face- cropped as the download link on the
h OOtainedd from http:://saattoocann..eneeeeesss http::/eissiiitteeeeeettaaaeeeee net
4 f 7ebe2, user account required.
Obtained from https::/sttaffoddiinn 992bb8 fc,, user account required.
htwp://ssstteeeeeeettttttteeeeee 92ada ao- 1aada7c-4443-- 6666 basb-88bbb11
16

---

The Disparate Benefits of Deep Ensembles
Table Results for controlled experiments. Performance and faimesss violations individual ensemble members, the Deep Ensemble as well as the change in performance and faimess violation due to ensembling. Gray cells denote the results of the controlled experiment
in Sec. 6 in the main paper.
Individual Ensemble Members
Setting
Accuracy (1)
SPD (4)
EOD (4)
AOD (4)
Main Paper (Fig. 4a)
0.894
0.04810.011
0.0801..0016
0.06410.00s
a 0.0 (Fig. 5a)
00.44
0.029+0.005
0.015+0.000
0.016=0.006
0.2 (Fig. 5b)
0.86010.005
0.02440.016
0.033
0.038
& = 0.4 (Fig. 5c)
0.801s
0.039
0.06944.016
0.060
a 1.0 (Fig. 5d)
0.88040.006
0.041
0.079+0.027
0.068
Deep Ensemble
Setting
Accuracy (1)
SPD (4)
EOD (4)
AOD (4)
Main Paper (Fig. 4a)
0.92440.002
0.057
0.103
10.004
0.0 (Fig. 5a)
0.84940.003
0.03320.004
0.010
0.015
a: = 0.2 (Fig. 5b)
0.87610.002
0.034
0.047:0.011
0.043
a 0.4 (Fig. 5c)
0.89640.002
0.054+0.008
0.105++0.013
0.084
1.0 (Fig. 5d)
0.91010.003
0.058
0.133
0.108
Difference (A) between Deep Ensemble and individual members
Setting
A Accuracy (1)
A SPD (4)
A EOD (4)
A AOD (4)
Main Paper (Fig. 4a)
0.030
0.009
0.054
0.04710.000
a = 0.0 (Fig. 5a)
0.00540.001
0.004
00.004400007
-0.001
a 0.2 (Fig. 5b)
0.017+0.003
0.010+0.006
0.014+0.011
0.005
0.4 (Fig. 5c)
0.02510.002
0.015+0.009
0.037
0.024
a = 1.0 (Fig. 5d)
0.03040.002
0.017:0.009
0.055
0.040
sets, but mostly align with commonly utilized hyperparameters for classical image datasets such as CIFAR1O. The raw performance on the task was not of extreme importance, but is comparable to previous studies on the same datasets with similar network architectures (Karkkainen & Joo, 2021; Zhang et al., 2022; Zong et al., 2023).
D.3. Computational Cost
For training the models, we utilized a mixture P100, RTX 3090, A40 and A100 GPUs, depending on availablility in our cluster. Training single model took around 3 hours on average over all considered model architectures and datasets, resulting in 3,000 GPU-hours. Evaluating these models on the test datasets accounted for approximately 150 additional GPU-hours.
E. Complete Experimental Results
The experimental results included in the main paper describe a subset of all the considered tasks. In this section, we provide the results of the complete set, along with additional supporting tables and figures.
Detailed Results on Controlled Experiments. The detailed results for the controlled experiments the main paper (Sec. 6) are provided in Tab. We report the absolute accuracies. SPDs, EODs and AODs for individual ensemble members, the Deep Ensemble and the differences between those.
Performance and fairness violation of Deep Ensemble and individual members. Tab. 4 and Tab. 3 contain the performance and fairness violations of individual ensemble members and the resulting Deep Ensemble, respectively.
The disparate benefits effect of Deep Ensembles. Fig. 9 11 depict the change performance and fairness violations when adding individual ensemble members for all considered tasks.
17

---

The Disparate Benefits of Deep Ensembles
Table Performance and fairness violations of Deep Ensembles (10 members). Statistics are obtained from five independent runs.
D'
Target / Group
Accuracy (1)
SPD (L)
EOD (J)
AOD (4)
FF
age/ / /ender
0.812
0. 0..99
0.165+0.010
0.12640.000
FF
age / race
0.812+0.007
0.112=0.008
0.063+0.011
0.075+0.008
FF
gender / age
0.90910.004
0.142 0.003
0.1005
0.065
FF
gender / race
0.90940.001
0.009=0.008
0.003+0.001
0.00440.003
FF
race age
0.88540.004
0.035_0.000
0.03840.014
0.025gn.006
FF
race / gender
0.88540.004
0.005=0.004
0.025+0.010
0.01540.005
UTK
age/ / gender
0.793+0.005
0.309=0.009
0.252+0.009
0.20440.008
UTK
age / race
0.793
0.211 0.006
0.18810.007
0.10610.005
UTK
gender / age
0.923
0. 0.180=0.00s
0.083
0.054440000
UTK
gender race
0.92340.003
0.002=0.002
0.023+0.003
0.029
UTK
race / age
0.84040.006
0.129=0.000
0.079
0.044
UTK
race / gender
0.840+0.006
0.010=0.004
0.02440.008
0.01440.000
D'
Group
AUROC (1)
SPD (L)
EOD (U)
AOD (J)
CX
age
0.943
0. 1.090.002
40.006
0.10440.003
CX
gender
0.943
0.000=0.001
0.024
0.014
CX
race
0.943
0.040=0.001
0.092+0.005
0.048+0.002
Changes PR, TPR and FPR. Fig. 12 14 display the change in PR, TPR and FPR per group when adding individual ensemble members for all considered tasks.
Difference average predictive diversity. Fig. 15 17 depict the differences in average predictive diversity per target and protected group.
Hardt post-processing (HPP). Tab. 5 19 contain the results of mitigating unfaimess by means of HPP (Hardt et al., 2016) on all considered tasks. HPP was either applied with the threshold set to the average fairness violation of the individual ensemble members on the validation set (val) or to 0.05. Note that for some tasks, the original fairness violation of both the Deep Ensemble and its members was already lower than 0.05, where HPP leads to an increase in unfairness up to the desired threshold. Experiments on FairFace and CheXpert use the respective validation sets learn the group dependent thresholds in HPP. For experiments on UTKFace, the FairFace validation set was used learn the thresholds, it was designed to emulate a real-world distribution shift scenario. Also for UTKFaac, the same conclusions for the FairFace experiments described in the main paper hold, i.e., while HPP is very effective mitigate unfairness in the Deep Ensembles, the desired fairness violation (0.05) is not reached due to the distribution shift. Note that the balanced accuracy was used as the performance metric for Cperp because the metric investigated in the main paper, the AUROC, does not consider selecting threshold.
18

---

The Disparate Benefits of Deep Ensembles
0.795
0.830
0.890
0.850
-0790
Accurag o
SPD
0.820
EOD
0.840
0.880
Perfer
0.785
Acc
0.810
0.780
0.800
(a) Y age, A gender
0.960
0.940
0,900
0.795
1PSPO (+)
0.950
0.890
E-EDD
0.940
0.930
0.925
Ralmess
Perfor mance
0.790
Accuras Oy
0.785
0.920
0.880
0.780
0.920
(b) Y age, A = race
0.862
0.940
0.925
0.860
1 AOD (+)
0.890
0.935
-0922
0.858
SPD
EOD
0.920
0.885
Acco
0.854
0.852
0.880
0.925
0.915
(c) gender, age
0.995
1.000
0.998
0.925
0.993
0.922
SPD
0.990
E-D0D
0.995
AOD
Rammsss Perför mance
0.920
07
0.988
0.994
0.917
0.992
(d) Y gender, A race
0.845
0.970
0.840
0.980
0.980
SPD
EDD
ADD
Tarmess
0.835
0.965
0.975
Perfor mance
0.960
0.830
0.825
0.965
0.940
(e) Y = race, A = age
1.010
1.040
0.995
0.845
SPD [+)
1.030
0.990
1.005
ADD
0.835
1.020
0.985
1.000
0.825
1.000
Ensemble Members
Ensemble Members
# Ensemble Members
() Y race, A = gender
Figure 9: The disparate benefits effect Deep Ensembles. The performance increases, when adding more members to the ensemble. Models are trained and evaluated on the FF dataset, Statistics are computed based on five independent runs.
fairness changes, often decreasing,
19

---

The Disparate Benefits of Deep Ensembles
0.770
0.795
0.710
--ADD [+]
0.810
0.760
-0.900
A Accurao 0
SPD
0.700
1- E.DDDD
0.750
0.800
Perfer
0.785
0.690
0.740
0.790
0.780
0.680
(a) Y age, A gender
0.910
00.95
IPSPD (a)
0.790
Accura Cy
0.900
Ralmess
Perfor mance
0.790
-0.785
0.890
0.780
0.780
(b) Y age, A = race
0.948
0.925
SPD (+)
0.823
-EDD
0.915
AOD
0.946
0.922
0.920
Acco
0.818
0.910
0.942
0.915
0.905
0.940
(c) gender. age
1.030
1.002
0.925
1.028
0.922
1.000
I.SPPD
0.998
EDD
1.022
0.970
Rarmess
Perfor mance
0.920
0.917
Ac
0.996
1.020
0.968
(d) Y gender, A race
0.878
0,930
0.845
SPD (+)
0.875
EED [+)
AOD [+]
D.840
0.873
0.955
Tarmess
D.833
Perfor mance
0.870
0.910
0.950
0.830
0.868
0.825
0.865
0.945
(e) Y = race, A = age
0.645
SPD (+)
1.030
A.ADD (+)
0.990
EDD
Ralmess
0.835
1.020
0.985
0.825
Ensemble Members
Ensemble Members
# Ensemble Members
() Y race, A = gender
Figure disparate benefits effect of Deep Ensembles. The performance increases. also the fairness changes, often decreasing. when adding more members to the ensemble. Models are trained on FF and evaluated on the UTK dataset. Statistics are computed based on five independent runs.
20

---

The Disparate Benefits of Deep Ensembles
Table 4: Performance and faimesss violations of individual members. Statistics are obtained from five independent runs.
D
Target / Group
Accuracy (t)
SPD (L)
EOD (U)
AOD (L)
FF
age gender
0.79440.001
0.173=0.001
0.15340.002
0.11340.000
FF
age / race
0.704
0.107
0.058
0.072
FF
gender / age
0.89940.001
0.142=0.001
0.11440.001
0.068++.000
FF
gender race
0.89940.001
0.010=0.001
0.00340.001
0.00640.001
FF
race / age
0.87310.000
0.040=0.001
0.040
0.02910.002
FF
race gender
0.87310.000
0.004_0.002
0.01910.003
0.01310.002
UTK
age/gender
0.78210.001
0.296_0..om
0.24010.008
0.195_n.003
UTK
age race race
0.782+0.001
0.207=0.002
0.182+0.003
0.10440.002
UTK
gender age
0.91640.001
0. 0.180=0.002
0.087
0.05640.001
UTK
gender / race
0.91611.001
0.002 00001
0.0231..000
0.02810.000
UTK
race age
0.822+0.002
0. 0.118=0.001
0.073
0.04340.000
UTK
race gender
0.82210.002
0.008
0.021
0.015
D
Group
AUROC (1)
SPD (1)
EOD (1)
AOD (4)
CX
age
0.94040.000
0. 0.138_0.001
0.17440.003
0.101
CX
gender
0.940+0.000
0.000
0.02440.008
0.01440.001
CX
0.940
0.041 0.000
0.091 10.003
0.049
0.866
0.830
0.902
0.864
0.943
SPD
0.862
EDD
0.820
0.897
0.942
0.860
0.895
0.941
0.815
Falmness
-0940
0.810
Performanze
(a) A age
0.985
1.001
-0.44
1- SPD [+]
0.988
0.943
1.000
EOD
AOD
0.942
0.985
0.941
0.999
0.983
0.940
0.980
(b) 4 = gender
0.961
0.954
0.944
0.960
0.910
0.943
SPD
EDD
A.DDD
0.942
0.959
0.905
0.950
0.941
Fainness
Parformance
# Ensemble Members
# Ensemble Members
# Ensemble Members
(c) race
Figure 11: The disparate benefits effect of Deep Ensembles. The performance increases. also the fairness changes, often decreasing. when adding more members to the ensemble. Models are trained and evaluated on the CX dataset. Statistics are computed based on five independent runs.
21

---

The Disparate Benefits of Deep Ensembles
0.000
A-0 (47%)
0.000
0.020
A= (53%)
PR
-0.010
5 TPR
0.010
FPR
-0.020
0.000
0.030
40,020
-0.010
0.040
-0.330
0.020
10
2
10
(a) Y age, A = gender
0.005
0.000
A-0 (81%)
0.030
A-1 (19%)
00.010
PR
00005
TPR
0.020
0.010
0.030
-0.015
0.000
-0.040
10
(b) Y = age, A = race
0.000
A (55%)
A-1 (45%)
-0.005
TPR
FPR
0.0110
0.015
0.000
10
2
10
(c) Y gender, A age
0.000
A-O (81%)
0.020
-0.005
A=1 (19%)
0.005
0.015
0.0110
0.000
-0.055
0.005
0.000
-0.025
10
2
10
(d) Y gender. A = race
0.040
0.000
A-O (55%)
-0.005
A=1 (45%)
PR
A TPR
-0.010
.0.110
0.000
-0.015
-0.020
-0.020
00.020
10
10
2
10
(c) Y = race, A = age
0.000
0.000
A-O (47%)
-0.005
-0.005
A-1 (53%)
PR
0.010
TPR
0.000
FPR
.0.010
-0.010
-0.015
-0.020
-0.020
.0.330
10
Ensemble Members
# Ensemble Members
#
Ensemble Members
(f) Y race, A gender
Figure 12: Changes in PR, TPR and FPR for Deep Ensemble (10 members) on the FF dataset. Statistics are computed based on five independent runs.
22

---

The Disparate Benefits of Deep Ensembles
0.020
0.030
0.000
A-0 (48%)
0.010
0.020
A= (52%)
B TPR
A FPR
0.010
0.010
0.000
-0.020
10
2
10
(a) Y age, A = gender
0.020
0.030
0.000
A-O (57%)
A (43%)
0.010
0.020
PR
A TPR
FPR
0.000
0.010
0.000
-0.020
0.010
-0.010
-0.030
10
(b) Y = age, A = race
0.000
A (45%)
A-1 (55%)
0.020
PR
0.005
TPR
FPR
-0.005
0.010
-0.010
0.000
0.000
00.015
10
2
10
(c) Y gender, A age
0.010
0,020
A-O (57%)
A=1 (43%)
0.007
0.015
-0,003
TPR
4 FPR
0.005
0.005
0.010
4.008
0.005
0.000
.00010
0.000
10
2
10
(d) Y gender. A = race
0.000
0.010
A-O (45%)
-0.005
A=1 (55%)
0.005
TPR
0.020
-0.010
0.005
0.010
-0.020
-0.010
0.000
-0.025
10
2
10
(c) Y = race, A = age
0.000
A-O (48%)
0.005
0.030
-0.005
A-1 (52%)
PR
0.000
TPR
00.010
0.010
0.005
-0.020
0.000
0.010
0.025
10
Ensemble Members
# Ensemble Members
# Ensemble Members
(f) Y race, A gender
Figure 13: Changes in PR, TPR and FPR for Deep Ensemble (10 members) on the UTK datasct. Statistics are computed based on five independent runs.
23

---

The Disparate Benefits of Deep Ensembles
0.002
A-o (85%)
0.005
0.020
0.000
A=1 (15%)
TPR
0.010
0.002
0.000
0.000
0.004
-0.006
-0.020
10
(a) age
0.004
0.020
Amo (41%)
0.002
0.000
0.010
0.000
TPR
-0.002
0.002
0.004
-0.004
00010
0.006
10
(b) gender
0.004
0.020
A=0 (63%)
0.002
0.000
A-1 (37%)
0.010
0.000
TPR
-0.002
00.022
0.000
-0.004
0.004
-0.010
0.006
-0.020
Ensemble Members
Ensemble Members
* Ensemble Members
(c) = race
Figure 14: Changes in PR, TPR and FPR for Deep Ensemble (10 members) the CX dataset. Statistics computed based five independent runs.
F
0.6
Y=0, A-1 (29%)
0.20
0.45
Y=1, A-0 (21%)
0.5
Y-1, A-1 (24%)
0.15
DN
Y-0, A-0 (45%)
0
Y-0, A-0 (26%)
0.4
0.35
Y=0, A=1
0.10
Y=1, A=0 (36%)
$
A=0 A=0 (29%)
0.3
Y=1, A-1 (9%)
$
Y=1, A=1 (24%)
0.30
0.05
(a) Y age, A gender
(b) Y age, A
Y gegdeer, A age
1.0
1.0
0.16
Y=a, A-D (38%)
Y-0,A-0 A-0 (45%)
Y=0, A-0 (38%)
Y-0, A-1 (9%)
Y-0,A-1 (36%)
Y-0, A-1 (43%)
Y=1,
Y-1,A-0
Y=1, A=1 (10%)
0.6
Y=1, (9%)
0.6
Y=1, A=1 (10%)
DN
0.4
0.4
I
0.2
0.2
0.13
0
1
1
(d) Y gender, A race
(c) Y race, A age
( Y = race, gender
Figure 15: Average predictive diversity DIV for each value of the protected attribute A and target variable Y on the FF dataset. Statistics are obtained from five independent runs.
24

---

The Disparate Benefits of Deep Ensembles
Y=0,A=0
0.175
0.8
Y=0, A=1 (23%)
0.7
Y=0, A=1 (19%)
Y=1, A-0 (26%)
0.6
$
Y=1, (32%)
0.150
Y-0, A-D (11%)
DIv
Y=1, (29%)
0.5
$
Y=1, (23%)
DW
0.125
Y-0, A-1 (66%
Y-1, (3%%
0.4
0.4
0.100
0.2
0.075
(a) Y age, A gender
(b) Y age, A
(c) Y -egderdr A ago
0.16
Y=0, A= (27%)
Y=0, A=0 (26%)
0 Y=0, A=0 (27%)
(20%)
0.6
Y-D,
(30%)
f
Y=1. A-D (30%)
Y-1,
(20%)
Y-1, Am1 (22%)
Y=1,A-1 A-1 (23%)
Y-1, Am1 (22%)
DIV
0.12
0.4
0.10
0.2
0.08
(d) Y gender, A = race
(c) Y = race, A = age
(f Y = race, A gender
Figure Average predictive diversity DIV for each value of protected attribute and target variable on the UTK dataset. Statistics are obtained from five independent runs.
0.04
0.04
Y-0, A-0 (71%)
Y=0, A-0
Y-D, A-0 (53%)
(12%)
Y-0, Am1
Y-0,
0.03
Y=1, A= (14%)
Y=1, A= (7%)
0.03
Y=1, A=0 (10%)
0
Y=1, A=1 (2%)
Y=1, A= (10%)
0
Y=1, A=1 (6%)
0.02
0.02
DN
0.02
0.01
0.01
0.01
(a) A = age
(b) A = gender
(c) A = race
Figure Average predictive diversity DIV each value of protected attribute and target variable are obtained from five independent runs.
CX dataset. Statistics
25

---

The Disparate Benefits of Deep Ensembles
Table 5: HPP results (aceuracy and faimess violation metries) on FF. Models trained on target variable age, evaluated using protected attribute gender. Statistics are obtained from five independent runs, and additionally over all individual ensemble members if applicable.
Before HPP
Acc (1)
SPD (4)
Acc (t)
EOD (4)
Acc (1)
AOD (4)
Members
0.7944.003
0.1731.007
0.1531.012 0.7941.003
0.7941.003
0.1131.008
Deep Ensemble
0.816 002
0.1944.001
0.816 0022
0.171
0.816 2.02
0.129
After HPP
HPP-SPD (L)
HPP-EOD (L)
HPP-AOD (L)
Deep Ens. (val)
0.818
0.1764.011
0.818+.001
0.157
0.8184.001
0.114
Deep Ens. (0.05)
0.818 001
0.057
0.815+.00 .002
0.067
0.816 0022
0.062
Members (0.05)
0.789
0.0561.022
0.792 .00s
0.0551.021
0.793 0.0541.015
Table 6: HPP results results (accuracy and fairness violation metrics) on FF. Models are trained on target variable age, evaluated using protected attribute race. Statistics are obtained from five independent runs, and additionally over all individual ensemble members if applicable.
Before HPP
Acc (1)
SPD (4)
Ace (t)
EOD (4)
Acc (1)
AOD (L)
Members
0.7944.003
0.107 .007
0.794+.00 .003
0.0584.011
0.794+.000 003
0.072+.007
Deep Ensemble
0.8161.001
0.1161.006
0.8161.001
0.070
0.8161.0m1
0.0791.006
After HPP
HPP-SPD (L)
HPP-EOD (L)
HPP-AOD (L)
Deep Ens. (val)
0.8184.001
0.070
0.8184.001
0.818 0.041
0.032
Deep Ens. (0.05)
0.818 o01
0.063
0.818+.001
0.033+.013
0.818
0.032
Members (0.05)
0.795
0.061..0115
0.7951.004
0.0494.028
0.795_1.004
0.054_.008
Table 7: HPP results (accuracy and faimess violation metries) on FF. Models are trained on target variable gender, evaluated using protected attribute age. Statistics are obtained from five independent runs, and additionally over all individual ensemble members if applicable.
Before HPP
Acc (t)
SPD (4)
Acc (t)
EOD (4)
Acc (t)
AOD (L)
Members
0.142+.00 0.899
0.899
0.114+.007
0.899
0.068
Deep Ensemble
0.913
0.9131.001 0.1422.002
0.107
0.9131.001
0.0644.001
After HPP
HPP-SPD (L)
HPP-EOD (L)
HPP-AOD (L)
Deep Ens. (val)
0.913
0.116g.015
0.913
0.0844.015
0.9131.0m1
0.067
Deep Ens. (0.05)
0.911
0.055
0.913+.001
0.0544.00x
0.913
0.067
Members (0.05)
0.8944.004 .004 0.048
0.897+.004
0.048+
0.8984.003 0.072+.005
Table 8: HPP results (accuracy and fairness violation metrics) on FF. Models are trained target variable gender, evaluated using protected attribute race. Statisties are obtained from five independent runs, and additionally over individual ensemble members applicable.
Before HPP
Acc (1)
SPD (L)
Acc (t)
EOD (4)
Acc (1)
AOD (L)
Members
0.899
0.010
0.899+.00x .003
0.003
0.8994.uus
0.006
Deep Ensemble
0.913 .001
0.009
0.913+.001
0.003+.002
0.913 001
0.004
After HPP
HPP-SPD (L)
HPP-EOD (L)
HPP-AOD (4)
Deep Ens. (val)
0.912
0.037+.005
0.912+.001
0.0044.007
0.912
0.007
Deep Ens. (0.05)
0.912+.000 0011
0.009+.002
0.912+.000
0.024+ 007
0.912+.001
0.032+.008
Members (0.05)
0.898 003
0.002+.013
0.898 003
0.007+.01
0.898
0.017#.012
26

---

The Disparate Benefits of Deep Ensembles
Table 9: HPP results (accuracy and faimess violation metrics) on FF. Models are trained on target variable race, evaluated using protected attribute age. Statistics obtained from five independent runs, and additionally over all individual ensemble members if applicable.
Before HPP
Acc (1)
SPD (4)
Acc (t)
EOD (4)
Acc (1)
AOD (4)
Members
0.8731.002
0.0401..06
0.8731.002
0.040
0.8731.002
0.0291.009
Ensemble
0.888 0001
0.0362.000
0.888
0.045
0.888 0.028+.002
After HPP
HPP-SPD (L)
HPP-EOD (L)
HPP-AOD (()
Deep Ens. (val)
0.887
0.040
0.030+.o11 0.888
0.887
0.025#.006
Deep Ens. (0.05)
0.052
0.888+.001
0.054 006
0.887+.001
0.052
Members (0.05)
0.873
0.030 0224
0.8731.004
0.8741.004
0.038
Table 10: HPP results (accuracy and fairness violation metrics) on FF. Models are trained on target variable race, evaluated using protected attribute gender. Statistics are obtained from five independent runs, and additionally over all individual ensemble members if applicable.
Before HPP
Acc (1)
SPD (4)
Ace (t)
EOD (4)
Acc (1)
AOD (L)
Members
0.004 0.873+.002
.005
0.873+.00 .002
0.0194.016
0.873+.002
0.013+.006
Ensemble
0.888 15881..001
0.0051.002
0.8881.001
0.0271.005
0.888
0.0161.002
After HPP
HPP-SPD (L)
HPP-EOD (L)
HPP-AOD (L)
Deep Ens. (val)
0.8884.001
0.0122.003
0.888
0.888 0.005
0.016
Deep Ens. (0.05)
0.888 002
0.013
0.888
0.017
0.888
0.019
Members (0.05)
0.873
0.0044.025
0.8731.004
0.0031.044
0.8731.004
0.029
Table 11: HPP results (accuracy and fairness violation metrics) on UTK. Models are trained on target variable age, evaluated using protected attribute gender. Statistics are obtained from five independent runs, and additionally over all individual ensemble members if applicable.
Before HPP
Acc (t)
SPD (4)
Acc (t)
EOD (4)
Acc (t)
AOD (L)
Members
0.7822.004
0.2962.008
0.240+.012 0.782+.004
0.782..004
0.195
Ensemble
0.7964.001
0.313
0.25511004 0.7964.001
0.7961.001
0.20711003
After HPP
HPP-SPD (L)
HPP-EOD (L)
HPP-AOD (L)
Deep Ens. (val)
0.796
0.299
0.7964.002
0.2451.011
0.7951..02
0.194
Deep Ens. (0.05)
0.795 004
0.211 +.00
0.796+.000 .003
0.175+.007
0.797+.u04
0.155+.006
Members (0.05)
0.777
0.202 0211
0.778+.004
0.163+
0.778+.004 0.145
Table 12: HPP results (accuracy and fairness violation metrics) on UTK. Models are trained target variable age, evaluated using protected attribute race. Statisties are obtained from five independent runs, and additionally over all individual ensemble members applicable.
Before HPP
Acc (1)
SPD (L)
Acc (t)
EOD (4)
Acc (1)
AOD (L)
Members
0.782
0.207
0.782+ .004
0.182+.00
0.782+.u04
0.104
Deep Ensemble
0.7964.001
0.217
0.191 0.796+.001
0.796
0.108
After HPP
HPP-SPD (L)
HPP-EOD (L)
HPP-AOD (4)
Deep Ens. (val)
0.791
0.188+.000 .008
0.792 0011
0. 0.168 0006
0.791
0.085
Deep Ens. (0.05)
0.791 0001
0.183 .005
0.791
0.163+ 010
0.791+
0.085
Members (0.05)
0.774 000
0.173
0.1764.021 0.777+.005
0.777+.005
0.092
27

---

The Disparate Benefits of Deep Ensembles
Table 13: HPP results (aceuracy and faimess violation metrics) on UTK. Models are trained on target variable gender, evaluated using protected attribute age. Statistics are obtained from five independent runs, and additionally over all individual ensemble members if applicable.
Before HPP
Acc (1)
SPD (4)
Acc (t)
EOD (J)
Ace (1)
AOD (J)
Members
0.9164.002
0.180
0.916+.oo .002
0.916 +007
0.056=.003
Deep Ensemble
0.9264.001
0.181
0.9264.001
.001
0.081+.003
0.9264.001
0.0522.001
After HPP
HPP-SPD (L)
HPP-EOD (L)
HPP-AOD (4)
Deep Ens. (val)
0.925
0.161 1.011
0.9251.0011
0.0601.013
0.925
1.00
Deep Ens. (0.05)
0.920
0.117#.001
0.923+.000
0.037
0.925
0.051
Members (0.05) 0.9104.001
0.111
0.911+.001
0.0344.011
0.9144.001 0.057
Table 14; HPP results (accuracy and faimess violation metrics) on UTK. Models are trained on target variable gender, evaluated using protected attribute race. Statistics are obtained from five independent runs, and additionally over all individual ensemble members if applicable.
Before HPP
Acc (t)
SPD (4)
Ace (1)
EOD (4)
Acc (t)
AOD (L)
Members
0.002+.003 0.9164.002
0.9164.002
0.0234.004
0.9164.002
0.028+.003
Deep Ensemble
0.926
0.0021.001
0.9261.001
0.0221.002
0.9261.001
0.0291.001
After HPP
HPP-SPD (L)
HPP-EOD (L)
HPP-AOD (L)
Deep Ens. (val)
0.926..001
0.021
0.9241.001
0.029
0.9251.001
0.034_.003
Deep Ens. (0.05)
0.924 001
0.012
0.923+.002
0.049
0.922
0.053+.005
Members (0.05) 0.914 0002
0.006
0.914+.002
0.035+.015
0.9144.002 0.039
Table 15: HPP results (accuracy and fairness violation metries) on UTK. Models are trained on target variable race, evaluated using protected attribute age. Statistics are obtained from five independent runs, and additionally over all individual ensemble members if applicable.
Before HPP
Acc (1)
SPD (L)
Ace (1)
EOD (4)
Acc (1)
AOD (L)
Members
0.822+.006
0.118 .009
0.822+ .006
0.073 016
0.822 006
0.0432.007
Deep Ensemble
0.843 .002
0.132
0.843 1.00
0.080
0.843 1.002
0.043_.002
After HPP
HPP-SPD (L)
HPP-EOD (L)
HPP-AOD (L)
Deep Ens. (val)
0.857=.001
0.131 ..00
0.8584.002
0.0664.00s
0.858
0.042
Deep Ens. (0.05)
0.8564.002
0.149
0.858+.002
0.0864.005
0.857+.003
0.057
Members (0.05) 0.8164.014
0.118 1.038
0.8161.015
0.0781.047
0.8171.014
0.055_.029
Table 16: HPP results (accuracy and fairness violation metrics) on UTK. Models are trained on target variable race, evaluated using protected attribute gender. Statistics are obtained from five independent runs, and additionally over all individual ensemble members if applicable.
Before HPP
Ace (t)
SPD (4)
Ace (t)
FOD (4)
Acc (1)
AOD (L)
Members
0.822 0.008 0.8221.006
0.021 1.019
0.822...00
0.015
Ensemble
0.843
0.843+.002 0.011
0.023
0.843..u02
0.013
After HPP
HPP-SPD (L)
HPP-EOD (L)
HPP-AOD (L)
Deep Ens. (val)
0.8584.003
0.0391.002
0.8584.001
0.000
0.8594.003
0.016
Deep Ens. (0.05)
0.859
0.038+.013
0.859
0.859 0.0194.019
0.019
Members (0.05)
0.8164.014
0.0091.044
0.8164.015
0.8161.014 0.0021.049
0.0301.032
28

---

The Disparate Benefits of Deep Ensembles
Table 17: HPP results (balanced accuracy and faimesss violation metries) CX. Models are evaluated using protected attribute age. Statistics are obtained from five independent runs, and additionally over all individual ensemble members applicable.
Before HPP
BAce (1)
SPD (4)
BAcc (1)
EOD (4)
BAcc (1)
AOD ())
Members
0.783 .008
0.138
0.783+.00 008
0.174+.010
0.783+.008
0.101
Deep Ensemble
0.786
0.139
0.786
0.1824.004
0.786
0.1044.002
After HPP
HPP-SPD (L)
HPP-EOD (L)
HPP-AOD (L)
Deep Ens. (val)
0.122 0.801
0.800+.004
0.1254.048
0.800
0.073+.030
Deep Ens. (0.05)
0.788 004
0.057=.002
0.798+.000
0.052+.007 .07
0.800
0.038+.010
Members (0.05)
0.7822.010
0.060
0.789
0.063
0.7901.011
0.049
Table 18: HPP results (balanced accuracy and fairness violation metrics) on CX. Models are evaluated using protected attribute gender.
Statistics are obtained from five independent runs, and additionally over all individual ensemble members if applicable.
Before HPP
BAce (t)
SPD (4)
BAce (t)
EOD (4)
BAce (t)
AOD (4)
Members
0.783 0008
0.000
0.783
0.0244.010
0.7831.u00
0.014
Deep Ensemble
0.786=.004 0044
0.000z.000
0.786+ 000
0.025+.000 0022
0.7864.004
0.015
After HPP
HPP-SPD (L)
HPP-EOD (L)
HPP-AOD (L)
Deep Ens. (val)
0.801#.006 006
0.002+.001
0.798+.007
0.005 .20
0.798 0077
0.014+.005
Deep Ens. (0.05)
0.7964.005
0.001
0.798+.006
0.009
0.7964.005
0.020
Members (0.05)
0.792:
0.001
0.792+.012
0.018
0.792
0.0222..013
Table HPP results (balanced accuracy and fairness violation metrics) on CX. Models are evaluated using protected attribute race. Statistics are obtained from five independent runs, and additionally over all individual ensemble members applicable.
Before HPP
BAcc (t)
SPD (4)
BAcc (1)
EOD (4)
BAcc (t)
AOD (L)
Members
0.783 ouu
0.041
0.783+..o .008
0.091
0.783
0.049
Deep Ensemble
0.786...ul
0.040..001
0.7864.004
0.0914.004
0.786
0.047
After HPP
HPP-SPD (L)
HPP-EOD (L)
HPP-AOD (4)
Deep Ens. (val)
0.801 1.007
0.037
0.083 0.8021.007
0.8021..07
0.0441.006
Deep Ens. (0.05)
0.802+.007 0077
0.039
0.802 .08
0.078 008
0.799+.001
0.053
Members (0.05)
0.793
0.0384.o0n
0.073 0.793
0.793
0.047=.016
29

---

The Disparate Benefits of Deep Ensembles
F. Additional Investigations
This section presents additional investigations that are complementary to those presented main section of the manuscript. First, we analyze the complementary notion of min-max fairness. Second, we investigate how the disparate benefits effect behaves for different model sizes of the individual ensemble members. Third, we conduct the same investigation different model architectures. Fourth, study whether the disparate benefits effect also occurs for heterogeneous Deep Ensembles composed of members with different model architectures. Fifth, report an alternative approach to mitigate the negative impact on faireess due to Deep Ensembling by means of weighting individual members differently in the ensemble. Finally, we study the calibration of the Deep Ensemble and its individual members and the resulting sensitivity of their threshold used to make the prediction.
E.. Minimax Fairness
The notions group fairness discussed throughout the paper (Eq. (2) (4)) control for the gap between group characteristics such as their PR, TPR or FPR. Another notion often considered recent work is minimax faimess (Martinez et al., 2020; Diana et al., 2021; Zietlow et al., 2022), where the characteristics the worst group are of importance. For instance (Zietlow al., 2022) showed, that the accuracy and TPR of both the minority and majority group decrease when using standard in-processing interventions in facial analysis tasks similar to FF and UTK in experiments. Therefore, investigate the minimax fairness impact of Deep Ensembles. Specifically, we discuss the TPR, FPR and accuracy.
The results for TPR and FPR are given in Fig. 12- 14. We observe, that for none of the considered tasks, there as a significant negative change of the TPR due to ensembling. Similarly, we find that for none of the considered tasks, there is a significant positive change of the FPR due to ensembling, which is desired as better classifier should have a lower FPR. The results accuracy are given in Fig. 18 20. We find, that the accuracies of both groups significantly increase for all considered tasks. In sum, while we find that Deep Ensembles have disparate benefits effect, where one group benefits more than the other, thus increases unfairness w.r.t. disparity based group faimesss metrics, the predictive performances of both groups increase thus improve fairness under a minimax fairness perspective.
0.030
(47%)
A= (81%)
0.020
A=O (55%)
(53%)
A=1 (19%)
A=1 (45%)
Accuracy
Accuracy
Accuracy
0.015
0.020
0.020
0.010
0.010
0.005
0.000
0.000
0.000
(a)
age.
gender
(b) Y age, A race
(c) Y gender, age
0.020
A-0 (81%)
0.020
A-0 (55%)
0.015
A-0 (47%)
A-1 (19%)
A-1 (45%)
A-1 (53%)
Accuracy
0.015
Accuracy
0.015
Accuracy
0.010
0.010
0.005
0.005
0.005
0.000
0.000
0.000
2
10
2
10
8
#I Ensemble Members
Ensemble Members
Ensemble Members
(d) Y = gender, A = race
(c) Y = race, A = age
(f) Y = race, A = gender
Figure 18: Change in Accuracy for Deep Ensemble (10 members) the FF dataset, Statistics computed based five independent runs.
F.2. Model Size
The experiments in the main paper were conducted using ResNet50 models. In this section we investigate whether the size of the models plays major role in determining the existence and strength of the disparate benefits effect. The results are
30

---

The Disparate Benefits of Deep Ensembles
0.020
A=0 (48%)
A=0 (57%)
0.015
A=0 (45%)
A=1 (52%)
0.020
A=1 (43%)
A=1 (55%)
Accuracy
A Accuracy
Accuracy
0.015
0.010
0.010
0.005
0.005
0.005
0.000
0.000
0.000
(a) age, A gender
(b) Y = age, A = race
(c) Y gender, A = age
0.025
A=0 0 (57%)
A=0 (45%)
A=0 (48%)
A=1 (43%)
0.020
A=1 (55%)
0.020
A=1 (52%)
Accuracy
0.010
Accuracy
Accuracy
0.015
0.010
0.005
0.005
0.005
0.000
0.000
0.000
2
4
10
2
4
8
10
2
8
10
# Ensemble Members
# Ensemble Members
# Ensemble Members
(d) Y = gender, A race
(e) Y = race, A = age
(l) Y race, A = gender
Figure Change Accuracy Deep Ensemble (10 members) on the UTK dataset, Statistics are computed based on five independent runs.
0.008
A-0 D (85%)
0.005
A-0 (41%)
0.006
A-O (63%)
A=1 (15%)
A-1 (59%)
A=1 (37%)
Accuracy
0.006
Accuracy
Accuracy
0.004
0.003
0.002
0.001
0.000
0.000
2
10
10
8 # Ensemble Members
# Ensemble Members
# Ensemble Members
(a) age
(b) A gender
(c) race
Figure 20: Change in Accuracy for a Deep Ensemble (10 members) on the CX dataset, Statistics are computed based on five independent runs.
shown in Fig. 21 23.. As seen in the Figures, the majority of cases the performance gains due ensembling slightly increase for larger model classes. The fairness violations however increase larger degree, see e.g. Fig. 21 (a) and (b), Fig. 22 (a), (b) and (c) well Fig. 23 (a). Generally, observe increase in the magnitude of the change in fairesss violations with larger model classes for all tasks that exhibit significant disparate benefits (ef. Tab. 1).
E.3. Model Architecture
In this section investigate the role of the specific model architecture the existence and strength of the disparate benefits effect. The results are shown in Fig. 24 26. In the majority of cases, disparate benefits occur throughout all considered model architectures. Especially for EfficientNetV2-5 we observe significant disparate benefits for some cases where we do not observe them in the main investigation based on ResNet50. For example for UTK, target race, group age under AOD (Fig. 25c) or CX, group race under EOD and AOD. Overall, we do not find a systematic difference of the results for different model architectures.
31

---

The Disparate Benefits of Deep Ensembles
( (/-curryy (+
4 SPD SPD (-11a Accuracy
E0D A.Accuryyy
Afarreo Wadlan
Natemmercc
resnet18
resnet34
resnet50
resnet18
resnet34
resnet50
resnet18
resnet34
resnet50
(a) Y age, A gender
A SPD (+15 Accuracy [+:
0.020
400 (+1) Accurayy
0.020
6,0D (-) Accurayy
0.020
0.000
0.000
0.000
(b) Y age, A = race
SPD (-1/A Accuracy (+:
8.DDD Accurscy
16 Accurscy
Harness Vicanas
Peffrmne
0.010
0.010
0.010
0.000
0.000
0.000
-0.010
-0.010
-0.010
resnet18
resnet34
resnet50
resnet18
resnet34
resnet50
resnet18
resnet34
resnet50
(c) Y
gender, age
SWD (+) a Accuracy [+
0.015
EOD (=)) Accuracy
0.015
(+D((=))&)) Accurasy
0.015
Viliion
ferimuuee
0.010
0.010
0.011
0.005
0.005
0.005
0.000
0.000
0.000
.0.005
-0.005
-0.005
resnet18
reshet34
resnet50
resnet18
resnet34
resnet50
resnet18
resnet34
reshet50
(d) Y = gender, A = race
SPD ( Accunacc [+)
Accuracy
5OD 11 Accursc)
AOD (11-1111 Accurscy
Faimess
Ferfomance
0.000
-0.010
-0.010
-0.010
resnet18
resnet34
resnet50
resnet18
resnet34
resnet50
resnet18
resnet34
resnet50
Y race, age
a SPD (-1/a Accurry [+
EoD (+))a Accuracy
(.ADD(+) & (c)uuA Accurscy
4 Nivess Vilates
Pefomane
0.010
0.010
0.010
0.000
0.000
0.000
-0.010
-0.010
-0.010
resnet18
resnet34 Model
resnet50
resnet18
resnet34 Model
resnet50
resnat18
resnet34 Model
resnet50
( Y race, A gender
Figure 21: The disparate benefits effect of Deep Ensembles for different model sizes. Models are trained and evaluated on the FF dataset. Statistics are computed based on five independent runs.
32

---

The Disparate Benefits of Deep Ensembles
4 SPD (-1/A Accuracy [+
E0D (+) (-)))13.0 Acccuaayy
100D(-11& A/&uracr
Afarroo Wadlan
Natemeerce
0.015
0.015
0.015
0.010
0.005
0.005
0.005
resnet18
resnet34
resnet50
resnet18
resnet34
resnet50
resnet18
resnet34
resnet50
(a) Y age, A gender
SPD (+1/ Accurity [+;
0.015
0.015
4DD (-11/AAAccraaa Accuraay
0.015
0.010
0.010
0.010
0.005
0.005
0.005
50D
0.000
0.000
0.000
(b) Y age, A = race
SPD (-1/A Accuracy (+:
0.010
ADD (4) Accurscy
Hirress Visamas
Peformne
&.DDD (4 Accuracy
0.005
0.005
0.005
0.000
0.000
0.000
-0.005
-0.005
-0.005
resnet18
resnet34
resnet50
resnet18
resnet34
resnet50
resnet18
resnet34
resnet50
(c) Y
gender, age
6:50 (s-)6 Accundyy
0.010
E0D --V Accurssy
0.010
ADDxx Accuracy
0.010
Viliian
ferimuuee
0.007
0.007
0.007
0.005
0.005
0.005
0.000
0.000
0.000
resnet18
resnet34
resnet50
resnet18
resnet34
resnet50
resnet18
resnet34
reshet50
(d) Y = gender, A = race
COD & Accuracy (+
Faimees
Ferforance
(-)/b Accuracy
4D00 (-114 Accurscy
0.010
0.000
0.000
0.000
resnet18
resnet34
resnet50
resnet18
resnet34
resnet50
resnet18
resnet34
resnet50
Y race, age
(>1/A Acarney (+
EGD (+)/d Accuratc [-
0.020
0.020
6,000 Accuracy
0.020
Nirees Vilates
Fefomane
0.010
0.010
0.010
0.000
0.000
0.000
resnet18
resnet34 Model
resnet50
resnet18
resnet34 Model
resnet50
resnat18
resnet34 Model
resnet50
( Y race, A gender
Figure 22: The disparate benefits effect of Deep Ensembles for different model sizes. Models are trained and evaluated on the UTK dataset. Statistics are computed based on five independent runs.
33

---

The Disparate Benefits of Deep Ensembles
(++
500 (-178 1-17/AURK AUROC [-)
ADD a AUROC [-)
0.005
0.005
0.005
0.000
0.000
0.000
resnet18
resnet34
resnet50
resnet18
resnet34
resnet50
resnet18
resnet34
resnet50
(a) age
SPD (-)/A AUROC (-4)
F0D1-114 A (+)
Afaman Vellin
AUROC
0.005
AAD (+
0.000
0.000
0.000
-0.005
-0.005
-0.005
resnet18
resnat34
resnet50
resnet18
resnet34
resnet50
resnet18
resnet34
resnet50
(b) A gender
SPD (+176 AUROC [+)
FOD (-) AUROC [-]
(+)) AURDC (URCC
0.004
0.004
0.004
0.002
0.002
0.002
0.000
0.000
0.000
-0.002
-0.002
0..04
Model
Model
Model
(c) A race
Figure The disparate benefits effect Deep Ensembles different model sizes. Models trained evaluated on the CX dataset. Statistics are computed based on five independent runs.
F.4. Heterogenous Ensembles
The results presented Fig. in the main paper are obtained from a homogeneous Deep Ensemble composed of ResNet50 models. The results presented in Fig. consider same target protected group combinations the same datasets using heterogeneous Deep Ensemble ResNer18/34/50 models. We observe the disparate benefits effect for heterogencous ensembling similar extent than for homogeneous ensembling.
F.5. Deep Ensemble Weighting
In this section, we study whether there exist weightings to combine the individual models the Deep Ensemble that perform better than standard uniform averaging in Eq. (1). The approximation in Eq. (1) thus changes to
pxly x, D)
R
Xm ply xc, w.)
(16)
À satisfies Σ An = 1 and Aa 0 Vn. Note that Eq. (16) results in Eq. (1) if An = 1/N Vn. We consider ~ Dir(a1.... Q with On = Vn. Weightings are thus drawn uniformly at random from N dimensional probability simplex. In our empirical investigation, we sampled 2,000 weightings À and evaluated the resulting ensembles on the three tasks. The results are given in Fig. 28, showing individual members and the different resulting ensembles, as well as their convex hull. In the case the FF and UTK datasets. there apprears be strong correlation between fairness violations and performance, and the weights hardly provide more Pareto optimal models. However, regarding the CX dataset, we observe that there are many weightings that would yield more favorable outcome than uniform averaging as generally done by Deep Ensembles. In the following, we outline two methods to choose such weighting. However, both methods did no lead to a significantly better outcome than uniform averaging. Nevertheless, we include qualitative discription our experiments as guidance for future research.
Weight selection based on the validation set. The simplest approach to identify more favorable set of weights consists of
34

---

The Disparate Benefits of Deep Ensembles
SPD (-1/A Accuracy (+
(-01 (008-=11Accraas Accuracy (+
Afarroo Wadlan
Aattemernce
0.030
0.020
0.020
0.020
(+) --)18
& (0D((+))
0.010
0.010
0.010
resnet50
regnet
efficientnet
resnet50
regnet
efficientnet
resnet50
regnet
efficientnet
(a) Y age, A gender
& SPD0 SPO(=1 Accuracy [+:
400 Accuray
(<1 Accuraay
0.000
regnet
efficientnet
regnet
regnet
(b) Y age, A = race
SPD (+1/A Accuracy (+:
16 Accurscy
Harres Vicanas
Pefrrane
BOD 1 Accurscy
0.010
resnet50
regnet
efficientnet
resnet50
regnet
efficientnet
resnet50
regnet
efficientnet
(c) Y gender, age
Accuuayy L-x
A EOD (errt(--..... Accurasy (+
ADD ( Accuracy (-
ferimmnne
6.5P0 (-0 Accunsty
0.010
0.010
0.010
resnet50
regnet
efficientnet
resnet50
regnet
efficientnet
resnet50
regnet
efficientnet
(d) Y = gender, race
SPD (-)/ (-)) Accurary
A COD (+11 & Accuracy (+
Falmess
Ferforance
AD0 1 Accurscy
0.000
0.000
resnet50
efficientnet
resnet50
efficientnet
resnet50
efficientnet
Y race, age
a SPD (-1/a Acauory t+
EGD (+))a Accuracy
AODD(=))) ( Accurcyy
4 Nirees Vilates
Pefomane
0.010
0.010
0.010
0.000
-0.010
resnet50
regnet Model
efficientnet
resnet50
regnet Model
efficientnet
resnat50
regnet Model
efficientnet
( Y race, A gender
Figure 24: The disparate benefits effect of Deep Ensembles for different model architectures. Models are trained and evaluated on the FF dataset. Statistics are computed based on five independent runs.
35

---

The Disparate Benefits of Deep Ensembles
SPD (+)1a Accuracy (+
A- (0000=11Acccmm Accurccy (+
Afarroo Wedlan
Nafeemercc
0.020
0.020
0.020
0.015
0.010
0.010
0.010
0.005
0.005
0.005
resnet50
regnet
efficientnet
resnet50
regnet
efficientnet
resnet50
regnet
efficientnet
(a) Y age, A gender
8 SPD (+1/-) Accuracy [+:
6<00 ( (=1/AAccuraa Accurayy
0.015
0.015
0.015
0.010
0,010
0.010
0.005
0.005
0.005
0.000
0.000
0.000
efficientnet
regnet
(b) Y age, A = race
SPD (+1/A Accuracy (+:
0.010
0.010
ADD (-) Accurscy(L
0.010
Harress Vicanas
Pefrrmnn
0.005
0.005
0.005
BOD (
0.000
0.000
0.000
.0.005
0.005
0.005
resnet50
regnet
efficientnet
resnet50
regnet
efficientnet
resnet50
regnet
efficientnet
(c) Y gender, age
6 (-01 Accunaty 1+x
A EOD (-)/& ( Accurscy (+
ADD ( Accuray
Vilucon
ferirmuns
0.005
0.005
0.005
0.000
resnet50
regnet
efficientnet
resnet50
regnet
efficientnet
resnet50
regnet
efficientnet
(d) = gender, race
(-)/6 Accuraty [++)
0.020
COD (A)/A Accurscy (+
0.020
0.020
Falmees
Ferforance
ADD 1 Accurscy
0.010
0.010
0.010
0.000
0.000
0.000
resnet50
efficientnet
resnet50
efficientnet
resnet50
efficientnet
Y race, age
a SPD (-1/a Accurrcy [+
0.020
0.020
(=)) (-01 Accuacy Acuucyy
0.020
& Nirres Vilates
a Pefomann
0.010
0.010
0.010
EOO (e
1
0.000
0.000
0.000
-0.010
-0.010
resnet50
regnet Model
efficientnet
resnet50
regnet Model
efficientnet
resnat50
regnet Model
efficientnet
( Y race, A gender
Figure 25: The disparate benefits effect of Deep Ensembles for different model architectures. Models are trained and evaluated on the UTK dataset. Statistics are computed based on five independent runs.
36

---

The Disparate Benefits of Deep Ensembles
5PD (+)/6 AUROC (+)
FOD ( AUROE
0.010
0.010
(-1/A
0.010
0.005
0.005
0.005
0.000
0.000
0.000
(a) A age
6SP (+-/6 AUROC (+)
0.005
4 (-1//AA AUROC
0.005
4,500 AUUCC
0.005
0.000
0.000
0.000
00.005
.0.005
-0.005
(b) A gender
A.PD 114 (++)
A FGD (-)] J AUROCI [-))
FFamest Welkon
0.005
0.005
AOD (->/s AUROCI
0.005
0.000
resmet50
regnet
efficientnet
resnet50
regnet Model
efficientnet
resnet50
regnet Model
efficientnet
Model
(c) race
Figure 26: The disparate benefits effect of Deep Ensembles for different model architectures. Models are trained and evaluated on the CX dataset. Statistics are computed based on five independent runs.
selecting as a hyperparameter. In our experiments, we sampled À uniformly at random as described before and selected the Pareto optimal weighting the validation However, found that the selected weights did not improve performance the test dataset, neither for the UTK dataset where could expected due the distribution shift datasets, where the validation and test datasets are drawn from the same distribution. Notably, the selected solutions were very close to the uniform weighting that is usually used in Deep Ensembles.
the FF and CX
Fairness-based weighting. Furthermore, we leveraged the information about the fairness violation of the individual members to define the weights and yield a fairer ensembling. This is similar to the approach reported in Kenfack et al. (2021), yet for neural networks base models. Given fairness violation measure F, [0, for each ensemble member, we define the weighting factor
Xn exp(-Fal1 Σn (xp(-F//7}
(17)
for Eq. (16), where T E RR, is a temperature hyperparameter. For high values of the temperature parameter 7 + oc, Eq. (16) becomes equivalent Eq. (I). For low values the temperature parameter 7 - 0, the fairness-weighted predictive distribution given by Eq. (16) approaches the predictive distribution of the model with lowest fairness violation. We calculated the fairness measure validation dataset. In experiments, the proposed fairness- fainhess-weighted Deep Ensemble to the uniform weighting. Notably, the selected solutions were either close to the individual models or to uniform averaging, thus exhibiting extremely high variance. In further analysis, we found that performance and fairness violations are extremely dependent on the selected temperature, both being non-smooth functions of the temperature. On the considered datasets and models, the best temperatures were usually found around le-2.
additional held "fairesss" dataset. The temperature parameter was selected on the
significantly Pareto dominant
F.6. Calibration and Threshold Selection
As elaborated the main part of the paper, we find that the Deep Ensemble is better calibrated than individual members (Fig. 7a). Here we provide a more detailed analysis that looks into the decrease in ECE per protected group for each target protected group attribute pair (task) we consider throughout our experiments. The results are provided in Fig. 29, showing
37

---

The Disparate Benefits of Deep Ensembles
Fairface
0.850
0.820
0.885
0.820
T.SPO
0.815
1.E00
0.840
AOD
0.880
PAAL Perfor mance
Ralmiss
0,810
0.835
0.810
0.830
0.805
0.800
0.805
0.825
0.720
0,780
0.820
SPD [+]
-.8000
0.710
0.770
EOD
0.760
AOD
Petfer
Faimess
-0.795
0.800
0.690
0.750
0.790
0,740
0.790
CheXpert
0.868
0.850
0.946
0.845
0.910
0.866
SPD
0.840
AUROC
0.864
0.835
0.905
Performance
0.862
0.942
0.825
0.900
0.860
10
30
30
# Ensemble Members
# Ensemble Members
Ensemble Members
Figure 27: The dangers of the disparate benefits effect for heterogeneous (Re5Net18/34/50) Deep Ensembles. The performance increases, but the fairness decreases when adding members the ensemble. The models evaluated on the FairFace test dataset and UTKFace dataset are trained to predict age the target variable and are evaluated using gender (male female) as protected attribute to define the groups. CheXpert models are trained predict whether there was a finding regarding set of medical conditions or not are evaluated using age (young/ / old) as the protected attribute to define the groups. Statistics are obtained from five independent runs.
that for some tasks, the ECE significantly differs per group, but the Deep Ensemble is more calibrated than individual members, regardless of the protected group attribute.
Finally, we report the results of analyzing the dependency of the Deep Ensemble and individual ensemble members on selecting the threshold for prediction. When using the usual argmax, implicitly a threshold of 0.5 is used. In the post- processing experiments we found that applying the method even under an additional fairness constraint can improve the performance. We evaluated all trained models on their respective validation datasets. Results are depicted in Fig. 30. The results show that the Deep Ensemble more sensitive to the threshold on the FF dataset, especially for target variable age. Regarding the CX dataset, the balanced accuracy exhibits roughly the same behavior under varying thresholds for the Deep Ensemble than for individual members. However, the spread of the optimal threshold is much smaller throughout all experiments.
38

---

The Disparate Benefits of Deep Ensembles
Fairface
.200
.180
.130
.190
.170
1800
.160
.120
SPO
AOD
.150
.110
.140
160
.100
.130
79.0
80.0
81.0
79.0
80.0
81.0
79.0
80.0
81.0
Accuracy (-)
Accuracy (+)
Accuracy
UTKFace
.310
.250
.200
I .300
SPD
*
.240
AOD
.290
.230
.880
.180
77.5
78.0
78.5
79.0
79.5
77.5
78.0
78.5
79.0
79.5
77.5
78.0
78.5
79.0
79.5
Accuracy (-)
Accuracy
Accuracy
CheXpert
144
.110
.190
1142
.105
140
AOD
SPD
.138
.100
.170
.136
.134
93.6
93.8
94.0
94.2
94.4
93.6
93.8
94.0
94.2
94.4
93.6
93.8
94.0
94.2
94.4
AUROC (-)
AUROC (+)
AUROC
Figure 28: Convex hull of performance and fairness violations for possible weightings to aggregate members of the Deep Ensemble. Ensemble weights are drawn uniformly at random from a dimensional simplex. Grey points represent individual models, the black star corresponds to their average performance and fairness violation. The red star represents the standard Deep Ensemble with uniform weighting.
Members
A-C
0.20
Deep Ensemble
0.15
FF
UTK
CX
ECE
0.05
0.00
age [gender]
age [raiee]
gender [age]
gender [race]
age [gender]
age [race]
gender [race]
n.f. [age]
n.f. [race]
[age]
[gender)
[age]
[age]
[gender]
[gender]
Target [Protected Group Attribute]
Figure 29: Expected Calibration Error (ECE) per group (group denoted by the hatches) for individual ensemble members and the Deep Ensemble for all considered target protected attribute combinations. Statistics are computed based on five independent runs.
39

---

The Disparate Benefits of Deep Ensembles
Ensemble
Individual
Ensemble
Individual Members
(079)
0
A- (53%)
A- 129%)
0.80
0.800
Accuracy
0.775
0.75
0.750
0.725
0.700
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0,50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
Threshold
Threshald
Threshold
Threshald
(a) FF: Y = age. A gender
(b) FF: Y = age, A race
Ensemble
Individual Me Members
Ensemble
Individual Members
0.95
0.90
0.90
Accuracy
Accuracy
0.85
0.80
0.80
0.75
(559)
0.75
A- (619)
A- (45%)
A- (1.9%)
0.70
0.70
0.2
0.4
0.6
0.8
0.8
1.0
0.0
0.2
1.0
Threshold
Threshold
Threshold
Threshold
(c) FF: gender, age
(d) FF: Y gender, A race
Individual Members
Individual
3 1909)
147M A=1 153%)
0.90
= 149%)
0.90
0.85
Accuracy
0.80
0.80
0.75
0.70
0.70
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Threshold
Threshold
Threshold
Threshold
(c) FF: Y race, A ago
(f) FF: Y race, A gender
Ensemblee
Individual Members
Ensemble
Individual Members
Balanced Balanced Accuracy [-]
Salanced Accuracy
0.85
0.85
0.80
0.70
0.6
1.0
Threshold
Threshold
Threshold
Threshold
(g) CX: A = age
(h) CX: 4 = gender
Ensemble
Individual Members
Balanced Accuracy [-)
0-196) 4 130%)
0.80
0.75
0.70
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4 Threshold
0.6
0.8
1.0
Threshold
(i) CX: A race
Figure 30: (Balanced) Accuracy depending on the chosen threshold for the FF and CX validation datasets. Vertical lines and shading denote optimal threshold per protected group. Statistics are computed based on five independent runs.
40


Reviews:
[{"review_id": "GIkfIB7jol", "reviewer": "Reviewer_zbgH", "summary": "This paper explores the impact of Deep Ensembles on algorithmic fairness. The authors find that Deep Ensembles, while improving overall performance, can unevenly benefit different groups, a phenomenon they call the \"disparate benefits effect.\" The paper suggests that differences in predictive diversity among ensemble members across different groups could potentially explain this effect. The paper also shows that the classical post-processing method (Hardt et al., NIPS 2016) can mitigate the disparate benefits effect by utilizing the better-calibrated predictive distributions of Deep Ensembles.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thorough assessment of our work.  We are pleased that it was found to be well written and to offer valuable insights. In response to the questions raised:\n\n> Limited generalizability of findings due to focus on vision datasets\n\nWe agree with your observation and acknowledge this limitation in the final paragraph of the paper:  *“The main limitations of our study are that we focus on vision tasks, and hence on ensembles of convolutional DNNs. [...]”*. That said, we aim to partially mitigate this limitation by including a diverse range of vision tasks spanning facial analysis (including under distribution shift) and medical imaging. These are application domains where Deep Neural Networks and Deep Ensembles are commonly deployed, making them highly relevant for studying their fairness implications. Moreover, we note that Deep Ensembles are rarely used in the tabular settings often featured in fairness research, where tree-based models remain dominant (Shwartz-Ziv, Ravid, and Amitai Armon. \"Tabular data: Deep learning is not all you need.\" Information Fusion 81 (2022): 84-90). This context motivated our focus on vision data, where Deep Ensembles are widely considered.\n\n> The paper does not introduce its own fairness algorithm\n\nWe agree that our paper does not propose a new fairness algorithm—this was not our objective. The main contribution of our paper is to show the existence of the disparate benefits effect of Deep Ensembles (Section 5) and to investigate its underlying causes (Section 6). While we do not claim to introduce a novel fairness method, we believe that presenting this new challenge without discussing potential remedies would leave the analysis incomplete. Therefore, in Section 7, we explore ways to mitigate the negative consequences of disparate benefits. We focus on post-processing methods that can be applied on the already trained individual ensemble members for practicality. Specifically: (1) we show that weighting members non-uniformly does not improve the performance-fairness trade-off whereas (2) Hardt Post-processing (HPP) is very effective in improving fairness while maintaining the utility of Deep Ensembles; and (3) we provide insights into why this is the case (improved calibration / threshold sensitivity).\n\n> [Hardt Post-processing] is not specifically designed for ensembling. This also weakens the connection between the intuition in Section 6 and the mitigation strategy.\n\nIndeed, HPP is not specifically designed to operate on ensembles.  However, its model-agnostic nature allows it to operate on any predictive distribution, making it broadly applicable. Since Deep Ensembles produce predictions by averaging the output probabilities of individual models (Eq. 1), HPP can be applied in exactly the same way as with individual ensemble members.\n\nRegarding “the intuition in Section 6”, we note that predictive diversity is the reason that ensembles predict differently than individual models - if there were zero predictive diversity, all models would agree and the ensemble would predict exactly the same as every ensemble member.  The averaging of diverse, and sometimes conflicting, predictive distributions leads to improved calibration (as shown in Figure 5). We show that HPP can effectively leverage this property: Deep Ensembles are more sensitive to thresholding decisions than individual models, and HPP, which relies on optimizing group-specific thresholds under fairness constraints, is particularly well-suited to take advantage of this threshold sensitivity. Thus, the improved calibration and diversity in ensembles directly support the effectiveness of HPP as a mitigation strategy.\n\n> Comparison with various types of fairness algorithms\n\nWe agree that this is an important direction for future work. We explicitly acknowledge this in the final sentence of the paper: *“Furthermore, we intend to investigate the disparate benefits effect for Deep Ensembles where pre- or in-processing fairness methods have been applied to individual ensemble members.”* Given the wide variety of fairness interventions available, we believe this direction warrants a dedicated study. Expanding the scope further within the 8-page limit would have come at the expense of depth in our current analysis.\n\nWe thank the reviewer again for their valuable feedback and thoughtful remarks. We hope that our rebuttal provides a more complete perspective on the contributions and scope of our study, and kindly ask the reviewer to consider a reassessment of their evaluation in light of these points.", "claims_and_evidence": "The paper's central finding is the potential for unfairness in deep ensembles. Empirical results highlight this issue, though the reliance on vision datasets may limit the generalizability of these findings.", "methods_and_evaluation_criteria": "The paper shows that an existing post-processing method (Hardt et al., NIPS 2016) can mitigate unfairness in deep ensembles, but it does not introduce a novel algorithm for this purpose. Comparing more diverse fairness algorithms would enhance the paper by providing a deeper understanding of different fairness approaches within ensemble learning.", "theoretical_claims": "The paper does not contain theoretical results.", "experimental_designs_or_analyses": "While the paper's diverse experimental results offer various insights, it would benefit from the inclusion of different dataset types (e.g., non-image) and a more comprehensive set of fairness algorithms, as discussed above.", "relation_to_broader_scientific_literature": "This paper focuses on potential unfairness issues in deep ensemble learning, which may offer some insights for applications of ensemble methods in scientific literature.", "essential_references_not_discussed": "Mostly well-discussed.", "other_strengths_and_weaknesses": "Strengths:\n- The paper is well-written, and the overall results provide valuable insights.\n- The paper offers an interesting intuition into unfairness in deep ensemble learning.\n\nWeaknesses:\n- Although the paper provides diverse experimental results, its reliance on vision datasets may limit the generalizability of the findings.\n- Additionally, the paper does not introduce its own fairness algorithm. While Hardt post-processing is a good existing algorithm that can mitigate this issue, it is not specifically designed for ensembling. This also weakens the connection between the intuition in Section 6 and the mitigation strategy. \n- Furthermore, a broader comparison with various types of fairness algorithms would be beneficial.", "other_comments_or_suggestions": "NA", "questions_for_authors": "Questions are included in the above sections.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hDfi7Qv0bu", "reviewer": "Reviewer_iGve", "summary": "The paper discusses two relations between deep ensembles and fairness notions. The first relation is the effect of the number of members on the overall fairness performance of the system. The second is what the authors call 'disparate benefit', which is the difference in fairness violation between the ensemble and the average fairness violation of the members. This is achieved by conducting several large scale experiments on three datasets. The results of the initial experiments are further analyzed by conducting new experiments investigating the relation between the disparate benefit phenomenon and the predictive diversity and expected calibration error.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for engaging deeply with our work and for providing their thoughtful and positive assessment. We’re pleased that you found the experimental design sound, the analyses sufficiently elaborate, and the results well-supported. Your detailed comments helped us reflect on both the limitations and opportunities for improvement in the framing and positioning of our work. We would like to respond in more detail to two specific comments:\n\n> The paper obscures quite some information with regards to fairness by only considering binary sensitive attributes (numerical and categorical attributes are mapped to be binary). This limits the scope of the analysis significantly and this choice could be communicated at an earlier stage of the paper.\n\nWe fully agree that explicitly stating the scope of the fairness setting—particularly the use of binary sensitive attributes and labels—is crucial for readers to assess our findings. We chose to focus on the canonical setting of binary labels and attributes to make our findings easily accessible and interpretable. However, exploring non-binary labels and group attributes (also possibly intersectional) is indeed an important direction for future work.\n\nWhile we currently state our setting in the opening paragraphs of Sections 3 (formal setup) and 4 (dataset description), we acknowledge that this may come too late for some readers. To make this clearer upfront, we propose updating the final paragraph on the first page to state: *“... each with multiple* **binary** *target variables and protected group attributes.”* to ensure that readers are immediately aware of our setup before engaging with the technical content.\n\n> Relation To Broader Scientific Literature\n\nThank you for highlighting several important and relevant works on fairness in ensemble learning. While our Related Work section focused primarily on studies most closely related to the disparate benefit phenomenon—such as Ko et al. (2023)—we fully agree that placing our contribution within the broader literature on ensembles and fairness will strengthen the paper.\n\nIn the revised version, we will expand the Related Work section to include the studies you suggested (Gohar et al., 2023; Kenfack et al., 2021; Mishra & Kumar, 2023; Tayebi & Garibay, 2023) as well as additional relevant works (e.g., Grgić-Hlača et al., Bhaskaruni et al.). For example:\n\n* Tayebi & Garibay apply pre-processing interventions to ensemble members of a Deep Ensemble (on tabular data), offering a complementary perspective to our work.\n* Gohar et al. investigate multiple research questions around the fairness impact of different ensembling techniques for classical models (e.g. SVMs, logistic regression, naive bayes), extending the discussion beyond deep learning.\n\nWe believe these additions will further clarify our paper’s relationship to the broader fairness literature and underscore its specific contributions.\n\n\nOnce again, we sincerely thank the reviewer for their valuable feedback. We appreciate your support of our work and are committed to addressing these points to improve the clarity and contextualization of our contribution in the final version. Please don’t hesitate to suggest any additional revisions that might further strengthen the paper.\n\n---\n\nGrgić-Hlača, N., Zafar, M. B., Gummadi, K. P., & Weller, A. (2017). On fairness, diversity and randomness in algorithmic decision making. arXiv:1706.10208.\n\nBhaskaruni, D., Hu, H., & Lan, C. (2019, November). Improving prediction fairness via model ensemble. In 2019 IEEE 31st International conference on tools with artificial intelligence (ICTAI) (pp. 1810-1814).", "claims_and_evidence": "The claims made in the paper are supported by experiments. The main body of the paper only provides results for one architecture, however more results are provided in the appendix.", "methods_and_evaluation_criteria": "As concluded by the authors, the scope of the current experiments is rather limited, as the only use case is on image datasets and only one type of bias mitigation method is applied. The datasets used for the experiments are appropriate for this type of fairness research and the Hardt method is one of the strongest. The paper obscures quite some information with regards to fairness by only considering binary sensitive attributes (numerical and categorical attributes are mapped to be binary). This limits the scope of the analysis significantly and this choice could be communicated at an earlier stage of the paper.", "theoretical_claims": "No theoretical claims were made in the paper.", "experimental_designs_or_analyses": "The overal experimental design is sound with many variables and several random seeds to account for variance in the results. The analyses are elaborate enough when also accounting for the results provided in the appendix. The reporting of the figures would benefit from \nequalizing the ranges on the axes. For example in Figure 3 the reporting would improve if the y-axes were the same for the plots as the difference in length of the arrows between the top and bottom row is requied for this figure. The controlled experiment is a nice touch to validate the authors' suspicion with regard to the role that predictive diversity plays in affecting fairness violation behaviour.", "relation_to_broader_scientific_literature": "The field of fairness research has in recent years started to investigate ensembles as an ML framework which might have some significant influences on the fairness of a system. The paper itself does not discuss many other fairness papers that focus on ensembles specifically. The only comparison seems to be made to the work of Ko et al. (2023). The main bulk of fairness research cited by the paper are \nmore general papers that are relevant for most fairness research. Examples of fairness research on ensembles includes the works of Gohar et al. (2023), Kenfack et al. (2021), Mishra and Kumar. (2023), and Tayebi and Garibay (2023). \n\n\nUsman Gohar, Sumon Biswas, Hridesh Rajan. Towards Understanding Fairness and its Composition in Ensemble Machine Learning (2023). ICSE '23\n\nPatrik Joslin Kenfack, Adil Mehmood Khan, S.M. Ahsan Kazmi, Rasheed Hussain, Alma Oracevic, Asad Masood Khattak. Impact of Model Ensemble On the Fairness of Classifiers in Machine Learning. ICAPAI '21.\n\nGargi Mishra, Rajeev Kumar. An individual fairness based outlier detection ensemble. (2023). Pattern Recognition Letters\n\nAida Tayebi, Ozlem Ozmen Garibay. Improving Fairness via Deep Ensemble Framework Using Preprocessing Interventions. (2023). HCII '23", "essential_references_not_discussed": "The reviewer is not aware of any contradacting research results.", "other_strengths_and_weaknesses": "This is an overall well-written research paper that focuses on the behaviour of fairness notions in the context of fairness violations. The results are abundantly communicated through the figures and also discussed in the main body of the paper.  The scope of datasets and bias mitigation methods applied limit the validity of the paper, which was also acknowledged by the authors.", "other_comments_or_suggestions": "/", "questions_for_authors": "/", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "921a2K5HpF", "reviewer": "Reviewer_rDWq", "summary": "This paper studies the fairness of deep ensembles on three image datasets. They find that ensembling tends to reinforce unfair behavior of ensemble members. The authors provide an explanation in that the predictive diversity causes this effect, which they call disparate benefits effect and give empirical evidence. Finally, the authors suggest some strategies to mitigate the effect.\n\n## Update after rebuttal\n\nI found the arguments risen by reviewer mL9u very reasonable and compelling. I hadn't seen it from that viewpoint. I still think that the paper makes interesting points, but will adjust my scoring below the acceptance threshold.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their very positive assessment of our work. We are pleased that they consider our experiments to be well designed, described and sufficient to support our claim that ensembling can have a negative impact on fairness. \n\n## Questions\n**Q1**\n\nWe acknowledge that the sample size (5 ensembles per task) is relatively small, primarily due to the computational cost of training large, fully independent ensembles (e.g., 5 seeds × 10 models = 50 models, totaling ~3000 GPU hours for our experiments). This way we can assure independence of individual samples. To give more insight into the normality condition, we performed the Shapiro-Wilk test. While we acknowledge that normality is difficult to verify with small samples, the results generally support the use of the t-test. Noteworthy, there is one case rejecting normality at p < 0.05: SPD on CX (age). We will make the respective entry in Tab 1 non-bold and provide the following additional table in the final version of the paper.\n\n**Shapiro-Wilk test p-value**\n|  | Accuracy | SPD | EOD | AOD |\n|-------|-------|-------|-------|-------|\n| FF (a/g) | 0.44  | 0.62  | 0.82  | 0.20  |\n| FF (a/r) | 0.44  | 0.28  | 0.97  | 0.35  |\n| FF (g/a) | 0.76  | 0.96  | 0.85  | 0.55  |\n| FF (g/r) | 0.76  | 0.31  | 0.48  | 0.63  |\n| FF (r/a) | 0.86  | 0.95  | 0.82  | 0.28  |\n| FF (r/g) | 0.86  | 0.28  | 0.30  | 0.73  |\n| UTK (a/g) | 0.30  | 0.21  | 0.07  | 0.15  |\n| UTK (a/r) | 0.30  | 0.88  | 0.20  | 0.90  |\n| UTK (g/a) | 0.64  | 0.40  | 0.31  | 0.57  |\n| UTK (g/r) | 0.64  | 0.17  | 0.76  | 0.21  |\n| UTK (r/a) | 0.22  | 0.32  | 0.06  | 0.82  |\n| UTK (r/g) | 0.22  | 0.40  | 0.61  | 0.58  |\n| CX (a) | 0.47  | **0.01**  | 0.18  | 0.22  |\n| CX (g) | 0.47  | 0.29  | 0.33  | 0.14  |\n| CX (r) | 0.47  | 0.10  | 0.68  | 0.59  |\n\nWe also considered increasing the sample size by combining members across seeds, but this would introduce dependencies between ensemble members, which we believe would compromise the statistical validity more than the limited sample size does. That said, we are open to alternative suggestions for robust significance testing if the reviewer has a preferred method in mind.\n\n**Q2**\n\nYes, we selected according to Tab 1 as the disparate benefits effect is not directly visible in Fig 3. The purpose of Fig 3 is to illustrate how the disparate benefits effect emerges when there are differences in the predictive diversity between groups (top row vs bottom row in Fig 3). These differences are highlighted by the black arrows in the figure, showing greater disparities in predictive diversity precisely in the tasks where we have found disparate benefits as per Tab 1. \n\nFor clarity, as explained in the caption of Fig 3, the top and bottom rows show examples of datasets where disparate benefits are and are not present, respectively. In the top row, we observe significant differences in predictive diversity, while in the bottom row, there are no or residual differences. This contrast demonstrates how predictive diversity differences correlate with the presence or absence of the disparate benefits effect. We will revise the caption and the accompanying text to clearly explain the distinction between the effect itself (shown in Tab 1) and its explanation (shown in Fig 3).\n\n## Remarks\n**Causation between predictive diversity and disparate benefits**\n\nWe fully agree that causality is very hard to establish. The real world experiments are observational in nature. Therefore, we conducted the interventional synthetic experiment. Note that we provide an extension of the experiment in Apx F.1., where we change the feature diversity per group (the $\\alpha$ parameter). There we see strong correlations between the differences in predictive diversity and the disparate benefits effect (see Tab 19 and Fig 16).\n\n**Definition of \"disparate benefits effect\"**\n\nThank you very much for this feedback. We agree with your assessment that the definition of the disparate benefits effect in the abstract is the best one and will adapt the description in the introduction accordingly, providing a more formal definition.\n\n**Effect size**\n\nThe disparate benefits effect depends on the unfairness of the base models. For example, the absolute increase shown in Table 1 of SPD in 2.2% for FF (age/gender) corresponds to a 10% relative increase in unfairness. Furthermore, the change in unfairness is closely linked to the increase in accuracy due to ensembling, which is in the range of 1-3% on our considered tasks. Therefore, the absolute effect size of the disparate benefits can not be expected to be substantially larger than the effect of ensembling itself. \n\n**Scatter plot**\n\nThank you for this suggestion, we will provide such plots in the final version of the paper.\n\nWe thank the reviewer again for this thoughtful and constructive feedback. We hope to have properly addressed the questions and concerns.", "claims_and_evidence": "The paper claims that ensembling neural networks tends to have a negative impact on fairness. A well-designed and described set of experiments on three datasets sufficiently supports the claim. They further claim that this effect is caused by predictive diversity, for which they do NOT provide evidence, since they only show that the two phenomena have a significant correlation in the studied data (but both could have a common cause and be independent given that cause).", "methods_and_evaluation_criteria": "Several evaluations are conducted throughout the study, all of them properly designed and executed except maybe the sample size of 5 and the statistical significance test using a t-test. The evaluation criteria are reasonable to me.", "theoretical_claims": "No theoretical claims are being made.", "experimental_designs_or_analyses": "I already commented on this above.", "relation_to_broader_scientific_literature": "Excellent", "essential_references_not_discussed": "None I am aware of.", "other_strengths_and_weaknesses": "None", "other_comments_or_suggestions": "- I think that while Table 1 is interesting, it would be important to have a scatter plot with the absolute violations of a single member (x-axis) and the ensemble (y-axis).\n- The paper coins the term \"disparate benefits effect\" but fails to properly introduce and maybe formalize it. I searched with Ctrl + F and found that curiously the abstract (not the introduction) gives the best intuition.\n- Also, while the results show a consistent tendency to reduce fairness by ensembling, the effect is not very pronounced. Statistical significance doesn't mean that it is particularly relevant. I mean even the highest difference is only 0.022, not particularly high.", "questions_for_authors": "- As far as I can see, you have only five ensembles (seeds) for each setup; I doubt that (i) the t-test is a proper way of measuring statistical significance here (why would you believe that the assumptions are satisfies?).\n- In Fig. 3, is the disparate benefits effect visible in the Figure? Or do we only know this from the selection based on table 1? I am inclined to the second option but would like to confirm this.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2sjQJR7Gr0", "reviewer": "Reviewer_mL9u", "summary": "This paper presents an empirical study of the fairness properties of deep ensembles. On several image datasets, the authors explore the fairness/accuracy tradeoff of deep ensembles at varying numbers of members, and explore a hypothesis which relates decreasing fairness metrics with varying amounts of predictive diversity between groups. They demonstrate that common post-processing methods can be helpful for improving the fairness of the ensemble.\n\n\n###########\nUPDATE AFTER REBUTTAL:\nThanks for the rebuttal. I'm not totally sure what to do with this paper as I still have some reservations about the central argument and I see I'm the main holdout here. I'm also not totally sure what to do with the extra table in the last comment - I'm looking at the left column which corresponds to the distribution over \"Y=0,A=0 / Y=0,A=1 / Y=1,A=0 / Y=1,A=1\" - however it seems like the base rates are equal between groups at all values in the left hand column: e.g.  \"0.22 / 0.22 / 0.28 / 0.28\" yields a 0.28/0.22 ratio in both group A=0 and 1. I tried flipping around the A and Y values to make sense but I don't quite see where the base rates variation comes in.\n\nI'm going to keep my score as is but acknowledge that I may just be missing something here given the scores of the other reviewers - apologies if I didn't understand something obvious about the paper.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their critical assessment of our work and the insightful questions. We first address the main question, followed by responses to the specific concerns outlined in the review.\n\n## Main Question\nWe appreciate the request for clearer evidence linking predictive diversity to the disparate benefits effect. As you correctly noted, with real-world datasets we cannot directly control predictive diversity. This limits us in Fig 3 to comparing tasks where the disparate benefits effect is observed (top row) versus not observed (bottom row) as per the results in Tab 1. \n\nHowever, believe the experiment on synthetic data reported in Apx F.1. aligns with your suggestion. There, we systematically varied the level of predictive diversity using a parameter $\\alpha$, while keeping all other factors constant. As shown in Fig 16 and Tab 19, the disparate benefits effect increases with higher diversity and vanishes entirely when $\\alpha = 0$. Importantly, this setup uses equal base rates ($p(Y=y, A=a) = 0.25$ for all combinations), ruling out base rate differences as the driver of the observed disparities.\nBased on your feedback, we will move this experiment into the main paper in the camera-ready version to strengthen our core argument.\n\n## Comments\n**Base rates**\n\n We provide the base rates in the legends of Fig 3 (see Fig 13-15 for all tasks), as well as Fig 4c for the synthetic experiment. Notably, there are some tasks (e.g. FF Y=age, A=gender) with very balanced base rates exhibiting significant levels of disparate benefits (see Tab 1). Also, we designed the synthetic task with equal base rates to eliminate this possible confounding factor.\n\n**Fairness/accuracy tradeoff**\n\nWe appreciate the concern about a potential fairness/accuracy tradeoff. However, we do not believe the disparate benefits effect merely reflects such a tradeoff. In some tasks (Fig 7 and 8), both accuracy and fairness improve through ensembling. Moreover, for tasks where fairness declines, we find that applying Hardt Post-processing (HPP) to the Deep Ensemble (DE) restores fairness to the level of individual models, without sacrificing the ensemble’s accuracy gains (Fig 6, red dot directly above the dotted line). This suggests the effect is not intrinsically an accuracy/fairness tradeoff.\n\n**Single model**\n\nThank you very much for your observation. We inspected the average predictive diversity per target and group for all 5 model variants and obtained similar results to those reported in Fig 3. Based on your feedback, we will add a new set of Figures in the revised version of the paper, summarizing these results. \n\n**Hardt post-processing**\n\nWe agree to the point that DEs are not different from any other probabilistic classifier regarding HPP. In essence, HPP takes any predictive distribution and group attribute as input independent of how this distribution was obtained. However, we believe that our finding regarding the improved calibration of the ensemble and thus its heightened sensitivity to the threshold is an interesting and novel finding.\n\n**Fig 6**\n\nWe thank the reviewer for pointing out the lack of clarity regarding the two red dots in Fig 6, which correspond to different levels of fairness tolerance to be achieved by the post-processing of the DE.\n\n* The first level of fairness tolerance is indicated by the dotted line and corresponds to the average fairness of individual ensemble members (gray dot next to the dotted line). The red dot closest to the dotted line corresponds to applying HPP to the DE to achieve the same level of fairness as the individual members (dotted line). Note how this is achieved while increasing accuracy. Thus, it illustrates that the DE remains Pareto-dominant by increasing both fairness and accuracy. \n* The second level of fairness tolerance is marked by the dashed line and it corresponds to 0.05, which is a commonly used value in the literature. In this case, the DE is also Pareto-dominant when compared to the individual models after post-processing both to the same tolerance level. \n\nWe acknowledge that Fig 6 contains a lot of information and will revise its caption and description for clarity.\n\n\nWe believe these revisions and clarifications directly address the reviewer’s central concerns—particularly regarding predictive diversity, base rate confounding, and the distinct fairness behavior of ensembles. We hope this supports a more favorable reassessment of our work.", "claims_and_evidence": "The central thesis of this paper is that deep ensembles \"unevenly favor different groups\", which the authors call the disparate benefits effect. The key evidence relies on demonstrating that as models are added to an ensemble, we see accuracy rise and fairness get worse.\n\nI am not sure I am convinced of this thesis by the arguments in this paper. In particular, I don't think the authors dispel the possibility that this is simply a property of improving model (read: ensemble) accuracy. We frequently observe such a fairness/accuracy tradeoff across many types of models, in particular if group base rates differ (in this case, we must observe such a tradeoff for a metric like SPD).\n\nVarious pieces of evidence are proposed, which I do not think necessarily demonstrate the argued conclusion:\n- argument around average predictive diversity: the argument here is that if diversity differs between groups then the benefits of ensembling would mostly accrue to that group. However, this seems to lead more naturally to an argument about subgroup accuracy than equalized odds type metrics, and certainly wouldn't apply to the SPD pattern seen here\n- Fig 4: this synthetic example does not necessarily demonstrate the desired pattern - it just constructs a case where there is differing DIV and disparate benefits. To show the pattern, one would need to also construct the counterfactual faces where there is equal DIV and equal benefits. Current data is consistent with the hypothesis  that there is nothing special about ensembles e.g. difference in base rates is driving the disparity\n- Fig 3 does demonstrate the desired pattern, however it is just observed for a single model and I would want to see this pattern shown more systematically to support the claim\n- The fact that Hardt post-processing is successful is also concordant with the view that ensembles are just like any other classifier, and can have their fairness properties improved with smart post-processing. The point about thresholding in 5b is interesting but I'm not sure it distinguishes ensemble fairness that clearly; in fact, Fig 6 itself shows that post processing applies to individual models helps just as much as applied to the ensemble!", "methods_and_evaluation_criteria": "I would be interested to know what the underlying base rates are on each dataset\n\nI think the OOD-style experiment using UTKFace is an interesting idea, enjoyed seeing that included", "theoretical_claims": "n/a", "experimental_designs_or_analyses": "see first section", "relation_to_broader_scientific_literature": "connection to fairness + ensemble literature is fine", "essential_references_not_discussed": "n/a", "other_strengths_and_weaknesses": "As stated previously, I find the overall argument of the paper to not be particularly convincing. However, I do think the direction here is an interesting place to explore, and I found the overall clarity of the writing to be pretty good.", "other_comments_or_suggestions": "Fig 6: it's not totally clear to me why there are 2 red dots downstream of the starting point on each plot", "questions_for_authors": "I would like to see evidence which tells me specifically how the fairness properties of ensembles behave differently from the fairness properties of individual models - e.g. in the DIV hypothesis, a more fully fleshed out series of observations along the lines of what's in Fig 3, or a clearer case where you construct a counterfactual example synthetically with/without high DIV that results in different levels of benefit disparity", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper presents an empirical study about the fairness impacts of deep ensembles. Experimental results show that deep ensembles unevenly helps boost performance of groups, which furthers the parity in prediction accuracy between groups. The authors explain this may be linked to predictive diversity for each group. The paper received mixed ratings (1,2,3,4). While the reviewers appreciated the empirical findings and potential underlying causes, they also had concerns about the validity of the claim and questioned whether the provided evidence can be tightly connected to the main argument. While the concerns exist, the paper's findings can provide a new insight about the property of deep ensembles.
Decision: Accept (poster)