Distributive Justice as the Foundational Premise of Fair ML: Unification, Extension, and Interpretation of Group Fairness Metrics
Abstract
Group fairness metrics are an established way of assessing the fairness of predictionbased decision-making systems.
However, these metrics are still insufficientlylinked to philosophical theories, and their moral meaning is often unclear.
Wepropose a general framework for analyzing the fairness of decision systems basedon theories of distributive justice, encompassing different established “patternsof justice” that correspond to different normative positions.
We show that themost popular group fairness metrics can be interpreted as special cases of ourapproach.
Thus, we provide a unifying and interpretative framework for groupfairness metrics that reveals the normative choices associated with each of themand that allows understanding their moral substance.
At the same time, we providean extension of the space of possible fairness metrics beyond the ones currentlydiscussed in the fair ML literature.
Our framework also allows overcoming severallimitations of group fairness metrics that have been criticized in the literature, mostnotably (1) that they are parity-based, i.e., that they demand some form of equalitybetween groups, which may sometimes be harmful to marginalized groups, (2) thatthey only compare decisions across groups, but not the resulting consequences forthese groups, and (3) that the full breadth of the distributive justice literature is notsufficiently represented.
Introduction
Supervised machine learning (ML) is increasingly being used for prediction-based decision makingin various consequential applications, such as credit lending, school admission, and recruitment.
Recent work has shown that the use of algorithms for decision making can reinforce existing biasesor introduce new ones [7].
Consequently, fairness has emerged as an important desideratum forautomated decision making.
As recent cases in practice have shown, this is crucial in order to mitigateunjustified disadvantages towards certain demographic groups (see, e g., [2, 43, 20, 37]).
However,quantifying the fairness of decision making systems is not straightforward as any morally appropriatenotion of fairness heavily depends on the given context.
Many different measures have emerged in the algorithmic fairness literature to assess and mitigateunfairness towards marginalized groups in decision making systems.
Many of the proposed notions offairness are in the category of so-called group fairness criteria [6], some of which are mathematicallyincompatible in practice Kleinberg et al.
[30], Chouldechova [13].
Therefore, satisfying such afairness criterion comes at the expense of not being able to satisfy others Kleinberg et al.
[29], Wong[51].
All existing group fairness criteria demand equality of a certain value between different sociodemographic groups [11], which is in contrast to the comparison of individuals, as it is done withother types of fairness such as individual fairness [17] or counterfactual fairness Kusner et al.
Readers unfamiliar with group fairness may refer to [36, Chapter 2], [49], and [6] for an overviewof the topic.
We briefly introduce and formally define the most-discussed group fairness criteria in Appendix A.
Much of the algorithmic fairness literature evolves around a limited set of group fairness metricsand is often not clearly linked to the many philosophical theories of justice that have been welldiscussed.
Kuppler et al.
[31] find that there is little to no overlap between philosophical theoriesof justice and metrics in the algorithmic fairness literature and conclude that “apparently, the fairmachine learning literature has not taken full advantage of the rich and longstanding literature ondistributive justice” [31, p. 17].
Therefore, the definitions of group fairness could be described asquite narrow when viewed from a philosophical perspective.
This becomes evident when thinkingabout an example: Group fairness metrics typically demand that groups are equal with respect
tosome metric.
Demanding equality between groups often makes sense, but consider a case in which wecould increase the utility of one group without harming another: Should we do this?
While we cannotsay that this is always a good idea, it at least seems to be a reasonable objection to group fairnessmetrics, which demand equality at all costs.
Therefore, this paper asks whether group fairness metricscan be extended to compare groups in other ways.
As of today, only a limited number of fairness metrics have been discussed, forcing stakeholders tochoose between a set of pre-defined metrics that they then have to justify for their context.
This paper,in contrast, presents a general framework for the derivation of targeted and context-specific fairnessmetrics, starting from values and moral views, and connects these to the philosophical literature, inparticular to theories of distributive justice.
Our main contributions can be summarized as follows:
We propose a general framework for assessing the fairness of prediction-based decisionsystems, based on theories of distributive justice, and allowing for different established“patterns of justice” that correspond to different normative positions.
The framework isbased on an analysis of how utility is distributed between groups.
“Pattern of justice” refersto normative ideas of what constitutes a just distribution. We show
that the
most popular group fairness metrics can be interpreted as special cases
of
this
approach,
which
thus
establishes a
unifying
framework that includes established
metrics,
but also shows how new ones can be constructed
We first present existing literature on group fairness (including its limitations) in Section 2.
In
Section 3, we present our unified framework for utility-based definitions of group fairness.
We focuson the mathematical formalization of different aspects of the distributive justice literature whilekeeping the review of the philosophical side short.
More details about the philosophical side can befound in the companion paper [3].
Section 4 then demonstrates that existing group fairness metricsare special cases of our utility-based approach.
Finally, we discuss the implications of this andpossible future work in Section 5.
Limitations of current group fairness criteria
Existing group fairness criteria pursue an egalitarian approach.
This means that they demand equalityof a certain value between different socio-demographic groups [11].
The fulfillment of these criteriais easy to assess, as this only requires access to a few variables (e.g., to check whether statisticalparity is satisfied, we only need the decisions and the group membership of individuals).
However,they also come with several limitations:
The "leveling down objection" As has been shown by [26], in some cases, enforcing groupfairness criteria can yield worse results for all groups in order to ensure parity between the groups.
This is what is known as the "leveling down objection", which is often brought forward to challengeegalitarianism in philosophical literature [38, 16]: In a case in which equality requires us to worsenthe outcomes for everyone, should we really demand equality or should we rather tolerate someinequalities?
As criticized by Cooper and Abrams [14], Weerts et al.
[50], existing definitions ofgroup fairness lack this differentiation as they always minimize inequality.
No consideration of consequences As pointed out by Hertweck et al.
[23] and Weerts et al.
[50],a large part of the existing work on fairness criteria seems to focus on an equal distribution offavorable decisions and not on the consequences of these decisions.
Binns [10] notes that thesecriteria "[assume] a uniform valuation of decision outcomes across different populations" [10, p. 6],and notes that this assumption does not always hold.
Whether a loan approval has a positive effect onone’s life or not arguably depends on one’s ability to repay this loan (and possibly on other individualattributes).
This narrow focus on the algorithm’s decisions instead of its consequences makes itdifficult to use existing group fairness criteria for a moral assessment of unfairness in decision makingsystems.
Parity-based criteria that only consider the decisions but not their consequences do not allowus to deliberately give positive decisions to a larger share of the disadvantaged group as this wouldbe a form of unequal treatment.
However, Kasy and Abebe [28] argue that in such a case, unequaltreatment can be required by justice to reduce overall inequalities.
Several works have therefore takena utility-based view of fairness.
Heidari et al.
[21]’s utility-based definitions of fairness focus on theeffects of decisions while [12] developed a method that follows the Rawlsian leximin principle toincrease the welfare of the worse off groups.
However, none of them provides a general frameworkthat encompasses different theories of distributive justice.
Limited set of fairness definitions Another limitation of existing group fairness criteria is thatthey represent a limited set of alternatives.
One has to choose one over the others, as they aremathematically incompatible [30, 13].
[44, 27] have highlighted that the criteria differ with respectto underlying moral values.
Thus, solely choosing one among the limited set of criteria might failadequately represent a morally appropriate definition of fairness for a given context.
Heidari et al.[22] show how existing group fairness criteria can be viewed as instantiations of the equality ofopportunity (EOP) principle.
Similarly, [9] show that they can be viewed as special cases of amore general principle of fairness they call fair equality of chances (FEC).
This way, they provide aframework through which the existing fairness criteria can be viewed.
However, the conditions underwhich the existing fairness criteria map to EOP (or to FEC, respectively) are not always given.
Wecannot expect every application to fall neatly into one of these conditions and thus cannot expect tofind a fitting fairness criterion among the ones already proposed in the group fairness literature.
These more general notions of fairness might be suitable to grasp the different existing notions ofgroup fairness.
However, they do not adequately represent the complexity of the distributive justiceliterature Kuppler et al.
In this paper, we want to bridge the gap between fair machine learningand philosophical theories of distributive justice.
A framework for fairness evaluations based on distributive justice
As discussed in Section 2, current group fairness criteria have some serious shortcomings.
Clearly,they do not reflect the full breadth of the literature on distributive justice [31].
To address this issue(at least partially), we propose a utility-based extension of group fairness.
This section introduces thisapproach from a rather technical perspective.
More details on its links to the literature on distributivejustice can be found in [3].
Our approach is based on the observation that each decision systemcreates a distribution of utility among individuals and groups.
Theories of distributive justice areconcerned with the questions when such a distribution can be considered just.
As will later show,some of these theories can be mapped to classical group fairness concepts from the fair ML literature(see Section 4).
We consider a decision making system that takes binary decisions D on decision subjects DS ofa given population P , based on a decision rule r .
The decision rule assigns each individual i ∈
Pa binary decision d i ∈ { 0 , 1 } , applying the decision rule to some input data, which includes anunknown but decision-relevant binary random variable Y .
It does not matter how the decision rulefunctions.
It could, for example, be an automated rule that takes decisions based on predictions of Yfrom an ML model or the decisions could be made by humans.
We further assume that at least twosocial groups are defined, denoted with different values for the sensitive attribute A .
3.1 Utility of the decision subjects
As previously discussed, current definitions of group fairness only consider the decisions themselves,but not their consequences — even though the same decision could be beneficial for some and harmful for others [50].
Our approach explicitly considers the consequences of decisions, i.e., the resultingutility (or welfare ), which could be positive in the case of a benefit or negative in the case of a harm.
We model the consequences with a utility function u which, in our binary context, may depend onboth the decision
d i and the value
y i of Y .
The utility u DS,i of a decision subject i is given by:
[Equation]
where the utility weights w dy denote the four different utility values that might be realized for thefour combinations of the random variables Y and D .
The utility u DS,i is a realization of a random variable U DS .
For assessing the fairness of a decisionrule, we are interested in systematic differences of utilities between groups.
This means that we areinterested in the expectation value E ( U DS ) of the individual utility, for different groups in A .
3.2 Relevant groups to compare
Theories of distributive justice are typically concerned with individuals [45] while group fairness isconcerned with socially salient groups.
Group fairness focuses on comparisons of different groupsas this is what theories of discrimination are concerned with [1].
This poses the question of howthe comparison of individuals in distributive justice and the comparison of socially salient groups ingroup fairness can be combined?
John Rawls’s concept of "relevant positions" [39, §16, pp.
81-86]is a concept that unites both ideas.
We view "relevant positions" as the groups whose expectedutility we want to compare and refer to them as the relevant groups (to compare).
2 As defined in [3],relevant groups to compare have comparable moral claims 3 to receive the same utility, but probablydo not receive the same utility.
Our approach thus views the theories of distributive justice, which weintroduced in Section 2, from the perspective of relevant groups to compare.
To be more specific, relevant groups are defined by two concepts: (1) claims differentiator J : Whatmakes it the case that some people have the same claims to utility while others have different claimsto utility?
; (2) causes of inequality (resulting in socially salient groups A ):
What are the most likelycauses of inequalities?
As described in [3], the claims differentiator identifies people who have equal moral claims.
In otherwords, the utility should be distributed equally between these people.
This means we only considerpeople with equal claims for our fairness evaluation.
Within the group of all individuals that haveequal claims to utility (i.e., that are equal in their value for J ) we specify groups that are unlikelyto end up receiving equal utility, on average, based on the known causes of inequality (i.e., that aredifferent in their value for A , which is sometimes also referred to as protected attribute ).
J and Adefine the relevant groups that group fairness criteria compare.
For simplicity, we will assume thatthere are only two groups
A = { 0 , 1 } that are unlikely to receive the same utility.
It is, for example,common to expect individuals of a different race or gender to not derive the same utility from decisionsystems.
In the next step, we want to compare the utilities of the relevant groups.
Specifically, we willcompare the expectation value of utility over all decisions made for a given population under a givena decision rule.
We denote this as the expected utility that takes the relevant groups into account,
E ( U DS | J = j, A = a ) , where J denotes the claims differentiator and j corresponds to a possiblevalue of the variable J , and a ∈ A denotes the different socially salient group to be compared witheach other.
In our framework, assessing fairness means comparing relevant groups with the same j ,but different a , with respect to the distribution of utility.
3.3 Patterns for a just distribution of utility
The claims differentiator J tells us which individuals have equal moral claims to the utility distributedby the decision process.
However, in some cases, an equal distribution of utility among the relevantgroups (defined by J and A ) may not be the primary concern for justice (see below).
Our approachoffers different choices, which we refer to as patterns of justice .
For each of them, we will brieflyexplain their normative view of what constitutes justice.
For each pattern, we formulate a fairnessconstraint and a fairness metric : A fairness constraint is a mathematical formalization of a patternof justice, which can either be satisfied or not.
A fairness metric F , on the other hand, can measurethe degree to which this criterion is fulfilled.
Note that we construct fairness metrics for a binary
A = { 0 , 1 } .
Therefore, all patterns of justice that we present compare the expected utility oftwo relevant groups: A =
∧ J = j (i.e., E ( U DS | J = j, A = 0) ) and A = 1 ∧ J = j (i.e.,
E ( U DS | J = j, A = 1) ).
However, the patterns of justice that we introduce here (egalitarianism,maximin, prioritarianism, sufficientiarianism) can easily be translated to cases of more groups.
In the following, we introduce only a few patterns of justice (representing fairness principles for theallocation of goods) that are widely discussed in philosophical literature.
However, our utility-baseddefinition of group fairness should in no way be seen as limited to these patterns.
Our approachcan easily be extended to other patterns of justice and one may also implement their own pattern ofjustice.
Our goal here is simply to highlight a few popular patterns of justice and how they can beembedded in our approach.
3.3.1 Egalitarianism
Egalitarianism – as the name suggests – demands equality [5].
Egalitarianism as a broad concept doesnot, however, specify what should be equalized.
This is subject of the equality of what debate initiatedby Sen [45].
One could, for example, aim to equalize the opportunities (equality of opportunity) oroutcomes (equality of outcomes).
Fairness criterion
The egalitarian fairness criterion is satisfied if the expected utility is equal forthe relevant groups:
[Equation]
Fairness metric The degree to which egalitarianism is fulfilled is measured as the absolute difference between the two groups’ expected utilities (lower values are better):
[Equation]
[Equation]
3.2 Maximin
Maximin describes the principle that among a set of possible distributions, the one that maximizesthe expected utility of the relevant group that is worst-off should be chosen [33].
In contrast toegalitarianism, inequalities are thus tolerated if the worst-off group benefits from them.
This has beendefended by Rawls in the form of the "difference principle" [39, 40].
Fairness criterion
The maximin fairness criterion is satisfied if there is no other possible distribution that would lead to a greater expected utility of the worst-off relevant group, which we denote by
U
[Equation]
It thus requires that the decision rule r ′ (whichrepresents the decision taken for each individual) results in a U worst − off DS
[Equation]
that is greater or equalthan the U worst − off DS ( r ) for any other decision rule r from the set of all possible decision rules R :
[Equation]
Fairness metric The degree to which maximin is fulfilled is measured as the value of the lowestexpected utility between all relevant groups (higher values are better):
[Equation]
[Figure]
3.3.3 Prioritarianism
Prioritarianism describes the principle that among a set of possible distributions, the one that maximizes the weighted sum of utilities across all people [25].
In contrast to egalitarianism, inequalitiesare thus tolerated if they increase this weighted sum of expected utilities.
In this weighted sum, theexpected utility of the worst-off relevant groups is given a higher weight (the maximin principle canbe seen as the extreme version of this as an infinite weight is given to the worst-off relevant groups).
Fairness criterion
The prioritarian fairness criterion is satisfied if there is no other possibledistribution that would lead to a greater overall expected utility, which is measured as a weightedaggregation of the relevant groups’ expected utilities, where the expected utility of the worst-offrelevant group is given a higher weight.
It thus requires that the decision rule r ′ results in a weightedutility ˜ U DS ( r ′ [Equation] =
[Equation]
DS
[Equation]
( r ′ ) that is greater or equal than the ˜ U DS ( r ) forany other decision rule r from the set of all possible decision rules R :
[Equation]
(6) where ˜ U DS denotes the sum of decision subject utilities for all groups with a weight k > 1 applied tothe worst-off group.
Fairness metric The degree to which prioritarianism is fulfilled is measured as an aggregate of the(weighted) expected utilities (higher values are better):
[Equation]
3.3.4 Sufficientarianism
Sufficientarianism [47] describes the principle that there is a minimum threshold of utility that shouldbe reached by everyone in expectation.
Inequalities between relevant groups above this minimumthreshold are acceptable according to this principle.
Inequalities are thus tolerated as long as allgroups achieve a minimum level of utility in expectation.
Fairness criterion The sufficientarian fairness criterion is satisfied if all groups’ expected utilitiesare above a given threshold t :
[Equation]
Fairness metric The degree to which sufficientarianism is fulfilled is measured as the number ofgroups whose expected utility is above the given threshold t (higher values are better):
[Equation]
where [Equation]
3.4 Extension of group fairness
Based on the mathematical framework outlined in this section, we suggest an extension of thecurrent understanding of group fairness as described in Section 2.
Instead of seeing group fairnessas demanding equality between socio-demographic groups with respect to some value, we insteadpropose the following definition:
Definition 1 (Group fairness) .
Group fairness is the just distribution of utility among relevant groups.
What makes a distribution just depends on the pattern of justice.
Thus, our extended understandingof group fairness does not necessarily require equal expected utilities across groups.
Furthermore,our definition ensures that only relevant groups are being compared (in the most familiar case, thesecorrespond to socio-demographic groups).
Group fairness criteria , in our sense, specify when group fairness is satisfied by a decision-makingsystem.
From this, it follows that there are more group fairness criteria than previously acknowledged.
This extension of group fairness criteria alleviates some of the criticisms of currently popular groupfairness criteria as we will show in Section 5.
Relation to existing group fairness criteria
Existing group fairness criteria are special cases of the utility-based extension we propose.
In thissection, we formally show under which conditions our approach maps to existing group fairnesscriteria (see Table 1 for a summary of the results).
In particular, we look at well-known groupfairness criteria: (conditional) statistical parity, equality of opportunity, false positive rate (FPR)parity, equalized odds, predictive parity, false omission rate (FOR) parity, and sufficiency.
Themathematical definitions of these criteria can be found in Table 2 in Appendix A. Furthermore, weshow how the utility-based group fairness metrics relate to existing ones.
In this section, we onlydemonstrate when our utility-based approach results in one of three often discussed group fairnesscriteria: statistical parity, equality of opportunity, and predictive parity.
We refer the interested readerto the Appendix B.2 where we provide a similar mapping for other existing group fairness criteria.
The findings we present in this section extend the ones of [22], [34], and [9].
While [22] consider thedistribution of undeserved utility (what they call the difference between an individual’s actual andeffort-based utility ), [34] and [9] use the decision subject utility U DS to derive a morally appropriategroup fairness definition.
This is similar to our approach presented in this paper; however, they onlyconsider two options U DS = D and U
DS = Y , while our approach allows for arbitrary functions ffor the utility: U DS = f ( D, Y ) .
Statistical parity (also called demographic parity or group fairness [17]) is defined as P ( D = 1 | A =0)
=
P ( D = 1 | A = 1) .
For specific decision subject utility weights w dy and without any claimsdifferentiator J , the condition of our utility-based fairness criteria derived from our framework isequivalent to statistical parity:
Proposition 2 (Statistical parity as utility-based fairness) .
If the utility weights of all possibleoutcomes (as described in Section 3.1) do not depend on the group membership ( w dy ⊥ a ), andw 11 = w 10 ̸ = w 01 = w 00 , then the egalitarian pattern fairness condition with J = ∅ is equivalent tostatistical parity.
The formal proof of Proposition 2 can be found in Appendix B.1.1.
We use w 1 y 6 to denote the decision subject utility associated with a positive decision ( D = 1 ) andw 0 y to denote the decision subject utility associated with a negative decision ( D = 0 ).
As we showedabove, requiring statistical parity can be equivalent to requiring the fulfillment of a utility-based groupfairness criterion.
However, even if the two criteria are equivalent, this is not necessarily true if wecompare the group fairness metrics that specify the degree to which these two criteria are fulfilled, i.e.,if we compare the degree to which statistical parity is fulfilled with the degree to which a utility-basedfairness metric is fulfilled:
Corollary 3 (Partial fulfillment of statistical parity in terms of utility-based fairness) .
Suppose thatthe degree to which statistical parity is fulfilled is defined as the absolute difference in decision ratiosacross groups, i.e., | P ( D = 1 |
A = 0) − P ( D = 1 | A = 1) | .
If the utility weights of all possibleoutcomes do not depend on the group membership ( w dy ⊥ a ), and w 11 =
w 10 ̸ = w 01 =
w 00 (i.e.,w 1 y ̸ = w 0 y ), and J = ∅ , then the degree to which egalitarianism is fulfilled is equivalent to thedegree to which statistical parity is fulfilled, multiplied by | w 1 y − w 0 y | .
The formal proof of Corollary 3 can be found in Appendix B.1.2.
Intuitively, F egalitarianism , which isderived from the utility-based fairness approach and represents the degree to which egalitarianism isfulfilled, can be seen as the degree to which statistical parity is fulfilled, weighted by the absolutedifference in utility for the decision received (decision subject utility for a positive versus a negativedecision).
Equality of opportunity (also called TPR parity) is defined as P ( D = 1 | Y = 1 , A = 0)
=
P ( D =1 | Y = 1 , A = 1) , i.e., it requires parity of true positive rates (TPR) across groups
a ∈ A [19].
Proposition 4 (Equality of opportunity as utility-based fairness) .
If w 11 and w 01 do not depend onthe group membership ( w d 1 ⊥ a ), and w 11 ̸ = w 01 , then the egalitarian pattern fairness conditionwith J = Y and j = { 1 } is equivalent to equality of opportunity. [Table]
The formal proof of Proposition 4 can be found in Appendix B.1.3.
Compared to statistical parity,equality of opportunity only requires equal acceptance rates across those subgroups of A who areof type Y = 1 .
This corresponds to the claims differentiator j = { 1 } for J = Y .
Thus, we simplyrequire the utility weights w 11 and w 01 to be unequal and independent of a .
As is the case forstatistical parity, there are differences when looking at the degree to which the two notions of fairnessare fulfilled (equality of opportunity and the utility-based fairness under the conditions specified in Proposition 4):
Corollary 5 (Partial fulfillment of equality of opportunity in terms of utility-based fairness) .
Supposethat the degree to which equality of opportunity is fulfilled is defined as the absolute difference indecision ratios for individuals of type Y = 1 across groups, i.e., | P ( D = 1 | Y = 1 , A = 0) − P
( D =1 | Y = 1 ,
A = 1) | .
If w 11 and w 01 do not depend on the group membership ( w d 1 ⊥ a ), w 11 ̸ = w 01 ,
J = Y , and j = { 1 } , then the degree to which egalitarianism is fulfilled is equivalent to the degreeto which equality of opportunity is fulfilled, multiplied by | ( w 11 − w 01 ) | .
The formal proof of Corollary 5 can be found in Appendix B.1.4.
Predictive parity (also called PPV parity [8] or outcome test [48]) is defined as P ( Y = 1 | D = 1 , A =0) =
P ( Y = 1 | D = 1 , A = 1) , i.e., it requires parity of positive predictive value (PPV) rates acrossgroups a ∈ A .
Proposition 6 (Predictive parity as utility-based fairness) .
If w 11 and w 10 do not depend on thegroup membership ( w 1 y ⊥ a ), and w 11 ̸ = w 10 , then the egalitarian pattern fairness condition with
J = D and j = { 1 } is equivalent to predictive parity.
The formal proof of Proposition 6 can be found in Appendix B.1.5.
Compared to equality ofopportunity, predictive parity requires an equal share of individuals to be of type Y = 1 amongthose subgroups of A who receive the decision D = 1 .
This corresponds to the claims differentiatorj = { 1 } for J = D .
Thus, we simply require the utility weights w 11 and w 10 to be unequal andindependent of a .
As is the case for the other group fairness criteria, there are differences regardingthe degree to which the two notions of fairness are fulfilled (predictive parity and the utility-basedfairness under the conditions specified in Proposition 6):
Corollary 7 (Partial fulfillment of predictive parity in terms of utility-based fairness) .
Suppose thatthe degree to which predictive parity is fulfilled is defined as the absolute difference in the ratio ofindividuals that are of type Y = 1 among all those that are assigned the decision D = 1 acrossgroups, i.e., | P ( Y = 1 | D = 1 , A = 0)
− P
( Y =
| D = 1 ,
A = 1) | .
If w 11 and w 10 do not dependon the group membership ( w 1 y ⊥ a ), w 11 ̸ = w 10 , J = D , and j = { 1 } , then the degree to whichegalitarianism is fulfilled is equivalent to the degree to which predictive parity is fulfilled, multipliedby | w 11 − w 10 | .
The formal proof of Corollary 7 can be found in Appendix B.1.6.
Considering Table 1, we see that existing group fairness criteria have a narrow understanding of utilityand do not tolerate inequalities, which can ultimately be harmful to already marginalized groups asprevious work has shown [26].
Moreover, existing group fairness criteria embed assumptions aboutwho has equal or different moral claims to utility.
If we were to, for example, demand equalizedodds for credit lending (where D is the bank’s decision to either approve a loan ( D = 1 ) or reject it( D = 0 ), and Y is the loan applicant’s ability to repay the loan ( Y = 1 ) or not ( Y = 0 )), we would make the following assumptions: People who are different in their ability to repay their loans havedifferent claims to utility.
We must thus equalize the expected utilities between people who are ableto repay their loans and we must also equalize the expected utilities between people who are notable to repay their loans.
However, the assumptions listed in Table 1 may not be met for all decisionmaking systems.
Our utility-based extension is thus necessary to implement other views of justice.
Discussion
As we have seen, existing group fairness criteria are special cases of our utility-based approach.
Thisapproach addresses several of the limitations of existing group fairness criteria that we discussed in
Section 2.
The "leveling down objection" The "leveling down objection" is a prevalent anti-egalitarianismargument [38, 16] saying that less inequality is not desirable if this requires lowering the better-offgroup’s welfare to match the one of the worse-off group.
On this basis, choosing egalitarianism as thepattern of justice has been criticized in the algorithmic fairness literature (see, e.g., [34, 26, 50]).
Ourapproach allows using other patterns of justice, such as maximin, prioritarianism, or sufficientarianism(see Section 3.3).
Other patterns that can be formalized as mathematical formulas may also be used.
One could, for example, combine several patterns into one and require equal expected utilities acrossgroups as long as none of the groups is better off than it would be without any fairness requirement.
This would represent a combination of egalitarianism and a group-specific baseline threshold (similarto sufficientarianism), making a "leveling down" of the better-off group impossible and adheringto the Pareto principle.
Therefore, our approach links group fairness to a much larger part of theliterature on distributive justice than current group fairness criteria.
No consideration of consequences Existing group fairness criteria only consider the distributionof either D or Y .
This could be interpreted as analyzing the distribution of utility but assumingthat utility is equivalent to either D or Y instead of, for example, the combination of D and Y .
Existing group fairness criteria thus represent a very confining definition of utility.
Our approachacknowledges that the utility of the decision subjects does not only depend on the decision itself butalso on other attributes such as one’s ability to repay a loan or one’s socioeconomic status (see, e.g.,[23, 50, 10].
This is represented through the utility function described in Section 3.1.
Limited set of fairness definitions
Previous attempts to guide stakeholders in choosing appropriatefairness criteria have taken on the form of explicit rules, such as in [42, 35, 41].
Such rules, however,presuppose a limited set of fairness definitions between which stakeholders can choose.
Instead,we provide a method to construct ad-hoc fairness criteria that reflect the values decided on by thestakeholders by combining the definition of the utility function for decision subjects (Section 3.1), therelevant groups to compare (Section 3.2) and the pattern for a just distribution of utility (Section 3.3).
Many important questions remain and may be the subject of future research: What are relevanttrade-offs when imposing utility-based group fairness criteria as requirements?
Optimal decisionrules for existing group fairness criteria have been derived by [19, 15, 8] – do they change for thefairness criteria defined by our approach?
Moreover, there is a risk that decision makers simplyuse our approach to bluewash their decision making system, which they may claim to be "fair" and"unbiased" after coming up with a fairness criterion that neatly fits their own goals.
This is an issuewith other fairness criteria as well.
Therefore, it is important to make the process of defining fairnesscriteria accessible to the public, so that decision subjects can get involved and hold decision makersaccountable.
This raises the question: with utility functions being notoriously hard to define [46, 18],how could our approach be accessible enough for practical use?
What may be needed is a process foreliciting values from stakeholders.
One may object that this makes group fairness criteria similarlydifficult to implement as individual fairness and counterfactual fairness.
Our response to this is thatexisting group fairness criteria might seem easier to use, but they still embed values and assumptions
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] The limitations are described inSection 5(c)
Did you discuss any potential negative societal impacts of your work?
[Yes]
The
potential negative effect of decision makers misusing our approach for bluewashing
is briefly discussed in Section 5.
However, it should be noted that this is a potential
negative effect of all approaches to measuring fairness.
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Sections 3,
4, and B.
(b) Did you include complete proofs of all theoretical results?
[Yes] See Appendix B.3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[N/A]
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[N/A]
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[N/A]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[N/A]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[N/A]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A](b)
Did you describe any potential participant risks, with links to Institutional ReviewBoard (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
[N/A]
A Existing group fairness criteria
Here, we briefly introduce the most discussed group fairness criteria.
Table 2 list the parity requirements associated with these criteria.
Statistical parity demands that the share of positive decisionsis equal between socio-demographic groups (defined by the sensitive attribute A
= { 0 , 1 } ) [17] –this is only required for a set of so-called legitimate attributes l ∈ L for the criterion conditionalstatistical parity [15].
Equality of opportunity , similarly, demands equal shares of positive decisionsbetween socio-demographic groups, but only for those whose target variable is positive ( Y = 1 ) [19]– thus, it is sometimes also referred to as true positive rate (TPR) parity.
Equalized odds – sometimesalso called separation – requires both equality of opportunity and FPR parity (which is similar toequality of opportunity, however, it is limited to individuals of type Y = 0 ).
In contrast, predictiveparity demands equal shares of individuals of type Y = 1 across socio-demographic groups, but onlyfor those who received a positive decision D = 1 – thus, it is sometimes also referred to as positivepredictive value (PPV) parity.
Sufficiency requires both PPV parity and false omission rate (FOR)parity (which is similar to PPV parity, however, it is limited to individuals who received a negativedecision D = 0 ).
[Table]
B Mapping existing group fairness criteria to our utility-based approachB.1 Omitted proofsB.1.1 Proof of Proposition 2
Recall that the utility-based fairness following the pattern of egalitarianism requires equal expectedutilities between groups:
[Equation]
Since there is no claims differentiator (i.e., J = ∅ ), this can be simplified to:
[Equation]
B.10)
For w 11 = w 10 and w 01 =
w 00 , the decision subject utility (see Equation 1) is:
[Equation]
.11) where w 1 y denotes the decision subject utility associated with a positive decision ( D = 1 ) and w 0 ydenotes the decision subject utility associated with a negative decision ( D = 0 ).
Thus, the expectedutility for individuals of group a can be written as:
[Equation]
If the utility weights of all possible outcomes do not depend on the group membership ( w dy ⊥ a ), andw 1 y ̸ = w 0 y 7 , then the utility-based fairness following the pattern of egalitarianism (see Equation B.10)requires:
[Equation]
.13) where the last line is identical to statistical parity.
B.1.2 Proof of Corollary
Recall that the degree to which egalitarianism is fulfilled is defined as F egalitarianism =
| E ( U DS | J =j, A = 0) − E ( U DS | J = j, A = 1) | (see Equation 3).
If the utility weights of all possible outcomesdo not depend on the group membership ( w dy ⊥ a ), and w 11 =
w 10 ̸ = w 01 =
w 00 (i.e., w 1 y ̸ = w 0 y ),
J = ∅ , this can be written as (see Equations B.10 and B.12):
[Equation] [Equation]
where the last line corresponds to a multiplication of | w 1 y − w
y | with the degree to which statisticalparity is fulfilled.
B.1.3 Proof of Proposition 4
Recall that the utility-based fairness following the pattern of egalitarianism requires equal expectedutilities between groups:
[Equation]
Since the claims differentiator is the same as the attribute Y = 1 , i.e., J = Y and the only morallyrelevant value of Y is 1 (i.e., j = { 1 } ), this can be simplified to:
[Equation]
For y i = 1 , the decision subject utility (see Equation 1) is:
[Equation] [Equation]
Thus, the expected utility for individuals of type Y = 1 in group a can be written as:
[Equation]
If w 11 and w 01 do not depend on the group membership ( w d 1 ⊥ a ), and w 11 ̸ = w 01 8 , then theutility-based fairness following the pattern of egalitarianism (see Equation B.16) requires:
w
[Equation]
( w
[Equation]
01 ) · P ( D = 1 | Y
[Equation]
1 , A
[Equation]
[Equation]
01 + ( w
[Equation]
[Equation]
P ( D
[Equation]
1 | Y = 1 , A = 1)
[Equation]
[Equation]
B.19) where the last line is identical to equality of opportunity.
B.1.4 Proof of Corollary 5
Recall that the degree to which egalitarianism is fulfilled is defined as F egalitarianism =
| E ( U DS | J =j, A = 0) − E ( U DS | J = j, A = 1) | (see Equation 3).
If w 11 and w 01 do not depend on the groupmembership ( w d 1 ⊥ a ), w 11 ̸ = w 01 , J = Y , and j = { 1 } , this can be written as (see Equations B.16and B.18):
[Equation]
where the last line corresponds to a multiplication of | w 11 − w 01 | with the degree to which equalityof opportunity is fulfilled.
B.1.5 Proof of Proposition
Recall that the utility-based fairness following the pattern of egalitarianism requires equal expectedutilities between groups:
[Equation]
Since the claims differentiator is the same as the decision D = 1 , i.e., J = D and the only morallyrelevant value of D is 1 (i.e., j = { 1 } ), this can be simplified to:
[Equation]
For d i = 1 , the decision subject utility (see Equation 1) is:
[Equation]
B
[Equation]
[Equation]
Thus, the expected utility for individuals in group a that are assigned the decision D = 1 can bewritten as:
[Equation]
If w 11 and w 10 do not depend on the group membership ( w 1 y ⊥ a ), and w 11 ̸ = w 10 9 , then theutility-based fairness following the pattern of egalitarianism (see Equation B.22) requires:
[Equation]
where the last line is identical to predictive parity.
B.1.6 Proof of Corollary 7
Recall that the degree to which egalitarianism is fulfilled is defined as F egalitarianism =
| E ( U DS | J =j, A = 0) − E ( U DS | J = j, A = 1) | (see Equation 3).
If w 11 and w 10 do not depend on the groupmembership ( w 1 y ⊥ a ), w 11 ̸ = w 10 , J = D , and j = { 1 } , this can be written as (see Equations B.22and B.24): [Equation] = 0))
[Equation]
where the last line corresponds to a multiplication of | w 11 − w 10 | with the degree to which predictiveparity is fulfilled.
B.2 Mapping to other group fairness criteria
In Section 4, we mapped our utility-based approach to the three group fairness criteria statistical parity,equality of opportunity, and predictive parity.
Here, we additionally show under which conditions ourutility-based approach is equivalent to other group fairness criteria: conditional statistical parity, falsepositive rate parity, equalized odds, false omission rate parity, and sufficiency.
B.2.1 Conditional statistical parity
Conditional statistical parity is defined as P ( D = 1 | L = l, A = 0)
=
P ( D = 1 | L = l, A = 1) ,where L is what [15] refer to as the legitimate attributes.
Thus, conditional statistical parity requiresequality of acceptance rates across all subgroups in A = 0 and A = 1 who are equal in their value lfor L , where L can be any (combination of) feature(s) besides D and A .
Proposition 8 (Conditional statistical parity as utility-based fairness) .
If the utility weights of allpossible outcomes do not depend on the group membership ( w dy ⊥ a ), and w 11 =
w 10 ̸ = w 01 =
w 00 ,then the egalitarian pattern fairness condition with J = L is equivalent to conditional statistical parity.
The proof of Proposition 8 is similar to the one of Proposition 2.
Under these conditions, the degree to which F egalitarianism is fulfilled is equivalent to the degree towhich conditional statistical parity is fulfilled, multiplied by | w 1 y − w
y | .
This could easily be proved– similar to the proof of Corollary 3 but with the conditions of the utility-based fairness stated in
Proposition 8.
B.2.2 False positive rate (FPR) parity
FPR parity (also called predictive equality [15]) is defined as P ( D = 1 | Y = 0 , A = 0)
=
P ( D =1 | Y = 0 , A = 1) , i.e., it requires parity of false positive rates (FPR) across groups a ∈ A .
Proposition 9 (FPR parity as utility-based fairness) .
If w 10 and w 00 do not depend on the groupmembership ( w d 0 ⊥ a ), and w 10 ̸ = w 00 , then the egalitarian pattern fairness condition with J =
Yand j
= { 0 } is equivalent to FPR parity.
For y i = 0 , the decision subject utility (see Equation 1) is:
[Equation]
Thus, the expected utility for individuals of type Y = 0 in group a can be written as:
[Equation]
Hence, we simply require the utility weights w 10 and w 00 to be unequal and independent of a .
Then,the proof of Proposition 9 is similar to the one of Proposition 4.
If w 10 and w 00 do not depend on the group membership ( w d 0 ⊥ a ), and w 10 ̸ = w 00 , then the degreeto which F egalitarianism is fulfilled is equivalent to the degree to which FPR parity is fulfilled, multipliedby | w 10 − w 00 | .
This could easily be proved – similar to the proof of Corollary 5.
B.2.3 Equalized odds
Equalized odds (sometimes also referred to as separation [6]) is defined as P ( D = 1 | Y
=
y, A =0)
=
P ( D = 1 | Y =
y, A = 1) , for y ∈ { 0 , 1 } .
Proposition 10 (Equalized odds as utility-based fairness) .
If the utility weights of all possibleoutcomes do not depend on the group membership ( w dy ⊥ a ), w 11 ̸ = w 01 , and w 10 ̸ = w 00 , then theegalitarian pattern fairness condition with J = Y and j = { 0 , 1 } is equivalent to equalized odds.
The conditions under which the utility-based fairness criteria is equivalent is shown separately forequality of opportunity (see Proposition 4) and FPR parity (see Proposition 9).
Since equalized oddsrequires equality of opportunity and FPR parity, the the conditions for both fairness criteria mustbe met (i.e., w dy ⊥ a ), w 11 ̸ = w 01 , w 10 ̸ = w 00 , J = Y , and j = { 0 , 1 } ), so that the utility-basedfairness constraint is equivalent to equalized odds.
B.2.4 False omission rate (FOR) parity
FOR parity is defined as P ( Y = 1 | D = 0 , A = 0)
=
P ( Y = 1 | D = 0 , A = 1) , i.e., it requiresparity of false omission rates (FOR) across groups a ∈ A .
Proposition 11 (FOR parity as utility-based fairness) .
If w 01 and w 00 do not depend on the groupmembership ( w 0 y ⊥ a ), and w 01 ̸ = w 00 , then the egalitarian pattern fairness condition with J = D ,and j = { 0 } is equivalent to FOR parity.
For d i = 0 , the decision subject utility (see Equation 1) is:
[Equation]
Thus, the expected utility for individuals in group a that are assigned the decision D = 0 can bewritten as:
[Equation]
Hence, we simply require the utility weights w 01 and w 00 to be unequal and independent of a .
Then,the proof of Proposition 11 is similar to the one of Proposition 6.
If w 01 and w 00 do not depend on the group membership ( w 0 y ⊥ a ), and w 01 ̸ = w 00 , then the degreeto which F egalitarianism is fulfilled is equivalent to the degree to which FoR parity is fulfilled, multipliedby | w 01 − w 00 | .
This could easily be proved – similar to the proof of Corollary 7.
B.2.5 Sufficiency
Sufficiency is defined as P ( Y = 1 | D =
d, A = 0)
=
P ( Y = 1 | D = d, A = 1) , for d ∈ { 0 , 1 } [6].
Proposition 12 (Sufficiency as utility-based fairness) .
If the utility weights of all possible outcomesdo not depend on the group membership ( w dy ⊥ a ), w 11 ̸ = w 10 , and w 01 ̸ = w 00 , then the egalitarianpattern fairness condition with J = D and j = { 0 , 1 } is equivalent to sufficiency.
The conditions under which the utility-based fairness criteria is equivalent is shown separately forpredictive parity (see Proposition 6) and FOR parity (see Proposition 11).
Since sufficiency requirespredictive parity and FOR parity, the the conditions for both fairness criteria must be met (i.e.,w dy ⊥ a ), w 11 ̸ = w 10 , w 01 ̸ = w 00 , J = D , and j = { 0 , 1 } ), so that the utility-based fairnessconstraint is equivalent to sufficiency.