ICML 2025

The Disparate Benefits of Deep Ensembles
Kajetan Schweighofer 1 Adrian Arnaiz-Rodriguez 2 Sepp Hochreiter 1 3 Nuria Oliver 2
Abstract
Ensembles of Deep Neural Networks, Deep En-
sembles, are widely used as a simple way to boost
predictive performance. However, their impact
on algorithmic fairness is not well understood yet.
Algorithmic fairness examines how a model’s per-
formance varies across socially relevant groups
defined by protected attributes such as age, gen-
der, or race. In this work, we explore the interplay
between the performance gains from Deep Ensem-
bles and fairness. Our analysis reveals that they
unevenly favor different groups, a phenomenon
that we term the disparate benefits effect. We
empirically investigate this effect using popular
facial analysis and medical imaging datasets with
protected group attributes and find that it affects
multiple established group fairness metrics, in-
cluding statistical parity and equal opportunity.
Furthermore, we identify that the per-group differ-
ences in predictive diversity of ensemble members
can explain this effect. Finally, we demonstrate
that the classical Hardt post-processing method is
particularly effective at mitigating the disparate
benefits effect of Deep Ensembles by leveraging
their better-calibrated predictive distributions.
1. Introduction
Deep Ensembles (Lakshminarayanan et al., 2017) have
demonstrated their efficacy as a simple and robust method
to improve the performance of individual Deep Neural Net-
works (DNNs). Their superior performance has made them
a popular choice for real-world applications (Bhusal et al.,
2021; Dolezal et al., 2022), including high-stakes scenar-
ios where the impact on people’s lives of Machine Learning
(ML) supported decisions can be profound, such as in health-
care, education, finance or the law. In such applications,
1ELLIS Unit, LIT AI Lab, Institute for Machine Learning,
JKU Linz, Austria 2ELLIS Alicante, Alicante, Spain 3NXAI
GmbH, Linz, Austria. Correspon...