Adjusting Machine Learning Decisions for Equal Opportunity and Counterfactual Fairness
Authors: Yixin Wang
Dhanya Sridhar
David Blei
Venue: Accepted by TMLR ()

Abstract: Machine learning (ML) methods have the potential to automate
high-stakes decisions, such as bail admissions or credit lending, by
analyzing and learning from historical data. But these algorithmic
decisions may be unfair: in learning from historical data, they may
replicate discriminatory practices from the past.  In this paper, we
propose two algorithms that adjust fitted ML predictors to produce
decisions that are fair.  Our methods provide post-hoc adjustments to
the predictors, without requiring that they be retrained.  We consider
a causal model of the ML decisions, define fairness through
counterfactual decisions within the model, and then form algorithmic
decisions that capture the historical data as well as possible but
are provably fair.  In particular, we consider two definitions of
fairness.  The first is ``equal counterfactual opportunity,'' where
the counterfactual distribution of the decision is the same regardless
of the protected attribute; the second is counterfactual fairness.  We
evaluate the algorithms, and the trade-off between accuracy and
fairness, on datasets about admissions, income, credit, and
recidivism.

Published in Transactions on Machine Learning Research (07/2023)
Adjusting Machine Learning Decisions for Equal Opportunity and Counterfactual Fairness
Yixin Wang University of Michigan
yizinu@uumich..e edu
Dhanya Sridhar Mila- Qula-Qu AI Institute and Université de Montrénl
dhanya.rridharrmmsttl.. quebee
David M. Blei
david. wlw..bleeecllubb....e edu
Columbia University
Reviewed on OpenReview: ofpeerevee net/ forum? P6NCCPbbb
Abstract
Machine learning (ml) methods have the polential to aulomate high-stakes decisions, such as bail admissions or credit lending, by analyzing and learning from historical data. But these almirithmic decisions may be unfair: in learning from historical data, they may replicate discriminatory practices from the past. In this paper, we propose two algorithms that adjust fitted MT. predictors to produce decisions that are fair. Our methods provide post-hoc adjustments the predictors, without requiring that they be retrained. We consider causal model of the ML decisions, define fairness through counterfactual decisions within the model, and then form algorithmic decisions that capture the historical data as well possible, but are provably fair. In particular, we consider two definitions of fairness. The first is "cqual cunterfactuull opportunity," where the counterfactual distribution of the decision is the same regardless the protected attribute; the second is counterfactual fairness. We evaluate the algorithms, and the trado-off between accuracy and fairness, on datasets about admissions, income, credit, and recidivism.
1 Introduction
There is growing interest in using machine learning (ml) methods to automate important decisions about people by analyzing and learning from historical data. For example, in the criminal just.ice system, mi algorithms are routinely used assess a defendant's risk of recidivism inform pretrial release and parole decisions (Brennan al., 2009 Larson et al., 2016 Lin et al. 2020). ml algorithms were also used by university to help determine which college applicants should be reviewed (Waters & Miikkulainen 2014).
As rumning example, consider an admissions committee that decides which applicants to accept to a university. Given historical data about applicants' traits and the admission decisions, an ml algorithm can learn to predict who will be admitted and who will not, and the resulting ml prcision-maker will accurately simulate the committec's decisions. But while these algorithmic decisions would save time and effort, they will also inherit some of the undesirable properties of the admissions committee (Corbett-Daviee Mitchell al. 2021 If the committee unfairly biased then its ml replacement will be well.
al.) 2017
In this paper, we develop two methods that adjust existing ML decision-maker to produce algorithmic decisions that are both accurate and fair. We prove that these algorithmic decisions maintain as much fidelity as possible to the decisions of the historical committee, but are adjusted to correct for the biases inherent in past decisions. We demonstrate these himc decisions on datasets about income, credit, and recidivism.
A key contribution of the proposed methods is that they do not require refitting the ML method data. One train ML. decision-maker usual, but then use the proposed methods to adjust the resulting

---

Published in Transactions on Machine Learning Research (07/2023)
algorithmic decisions to be fair. Consequently, these methods can be employed by practitioners who have pre-trained ml model but without the original data, when the ml model is too expensive to refit.
Main idea. To quantify the fairness of a decision maker, we use two causal criteria. Causal criteria for fairness were first introduced in Kusner et al. (017 and have been operationalized in many algorithms (Zhang & Bareinboim 2018a Zhang et al. 2017 Wu al.) 2019a b Coston et al. 2020 Kilbertus et al. 2017 2019 Chiappa 2019). Causal fairness criteria require encoding our assumptions about bias using causal models. While these assumptions require careful scrutiny Klohler-Hausmani 2018), causal models enable to derive fairness measures to capture different mechanisms of bias. In contrast to causal criteria, statistical measures of fairness cannot distinguish between different mechanisms of unfairness (Hardt et al. 2016).
First, we introduce the equal counterfactual opportunity (eco) criterion. With an eco decision-maker, individuals that have the same attributes will receive the same probability of admission, regardless of their demographic group. In particular, for an algorithmic decision that satisfics cco, there is no causal relationship between the protected altributes (such as race or gender) and the decision. The eco criterion is individual-level; it requires that each decision is fair.
Second, we consider the counterfactual fairness (cf) criterion proposed in Kusner et al. (2019) Under cf, the probability of admission is not affected by (hypothetical) interventions on protected attributes, including when we account for how those interventions might subsequently change the other attributes. For example, minority applicants may have been historically denied university admission due to lower test scores, but these low scores might be consequences of systemic disadvantages that the minority applicants face. A cf decision-maker adjusts individuals' attributes (such as test score) to account for such disadvantages. Similar to the eco criterion, the cf is also individual-level.
With these two criteria in place, the methods we develop draw on the ideas and methods of causal infer- ence (Pearl 2009: Peters et al. 2017). First we require that the historical decision-makin process is captured by a causal model. This model reflects domain knowledge and assumptions about the process, including the probabilistic relationship between an individual's altributes and the decision, and relationships between the attributes themselves. The causal model might be elicited from experts, informed from data, or a combination.
Contimuing the example, causal model of admissions represents the applicants' traits and the committee's decisions. In this paper we use the model illustrated in Fig. 2 There are protected allributes, such as gender, and cro-pproeeted attributes, such as test score. The model posits that the (historical) decision was affected by both, e.g., the committee considered the test score but also unfairly considered gender. The model also posits that the non- con-protected attributes may be affected by the protected ones, e.g., gender may affect the test score through systematic gender bias in schooling. The methods we develop do not require this particular causal model, but it will be running example.
Given the causal model of historical decisions, then frame algorithmic decisions new causal variables in the same model, ones whose values are functions of the individuals' attributes. This framing is important because it allows us to consider counterfactuals about the algorithms, c.g., "what would the algorithm decide for this applicant if she was male and had achieved the same lest score?" Probabilistic properties of such counterfactuals help formally define the mathematical criteria for eco and cf.
Finally, we derive methods to produce algorithmic docisions that meet these criteria. The first step to use the historical data to fit an ml method to predict. the decision, such as with a simple logistic regression or neural network. (Note we can then use the causal model and causal criteria to assess the fairness of the ML -nkksi)n-maker.) We then show how to adjust the trained ml decision-maker to satisfy cco, and we show how to adjust the eco decision- maker to exercise cf. We will see that both adjustment procedures involve estimating causal quantities based on both the assumed causal model and the original ml predictions.
We emphasize again that this method uses a fitted ml decision- maker but does not require retraining it or re-analyzing the historical data, which could require significant computational cost. We prove that, under the assumed causal model, the fair decision-makers we derive theoretically optimal-they maintain as much fidelity to the historical data as possible, while still being eco-fair ef-fair.
2

---

Published in Transactions on Machine Learning Research (07/)023)
We study these approaches on simulated admissions data and on three public datasets, about income, credit, and recidivism. We find that classical ml decisions are the most accurate, but they may be unfair. The fair preiison-mmaees deviate from the ml decisions, but provide decisions that meet the fairness criteria. Compared to other approaches that satisfy eco and ef (Chen et al. 2019 Kusner et al. 2017 the method developed here provides decisions with higher accuracy while remaining lair.
Contributions. 1) We develop algorithms that modify existing ml predictors to be fair, without requiring retraining. 2) We prove that these decisions optimally recover ml predictors while satisfying the criteria for eco and CF. 3) We conduct empirical studies on real and simulated settings to show that our algorithms produce better decisions than existing approaches that satisfy the same fairness criteria.
Related work. This paper contributes to research about causal models for defining and implementing fair ML. Kusner et al. (017)) introduce counterfactual fairness (the ef criterion above) and the FairLearning algorithm to satisfy it. This paper provides cco-fair decisions and cf-fair dccisions that minimally adjust the fitted ml decisions. Like FairLearingg Kusner et al. (2017), the ef-fair decisions also satisly counterfactual fairness. But in contrast to FairLearning, which omits descendants of the protected attributes, the ef prciisionmmkke in this paper uses all available attributes and achieves higher fidelity to historical decisions.
Kilbertus et al. 2017] considers more fine-grained causal notions of fairness and relates paths and variables in causal models to violations of eco and cf. Other work studies path-specific fairness, where some paths in the causal model are deemed unfair. These works typically propose new training objectives that satisfy
fairness (Chiappa 2019 Zhang et al.
2017
Nabi & Shpitser 2018 [Zhang & Barcinboim 2018b Wu
et al. 2019b; Mhasawade & Chunara 2021 Dutta et al. 2021 Madras et al.) 2019). In contrast to the methods developed for path-spocific fairness, this paper targets individual-level fairness, where cach decision is guaranteed to be fair. Moreover, the algorithmic decisions we propose in this paper do not require re-training the ml decision- makers.
A number of recent papers further extend the idea of counterfactual fairness. Kusner ct al. (2019): Mishler &
Kennedy 22021) develop decision-makers that larget multiple decisions or multiple fairness criteria. Coston
et al. (2020] extends the use of counterfactuals for fair risk assessment scores. Galhotra et al. (0021 Parafita
& Vitria [2021 Dai al. (2021); Foster et al. [2021]: | Artelt et al. (021): Karimi et al. [2021 |t Veitch et al.
(2021) develop algorithms for counterfactual invariance and counterfactual fairness in model explanation and
representation learning. Mishler et al. (022) extends counterfactual fairness to counterfactual equalized odds
and propose post-processing optimization programs for adjustment. Unlike these works, this paper focuses on developing post-hoc fairness adjustments with closed forms, without requiring retraining.
Alongside these causal approaches to algorithmic fairness, there a vast. literature on statistical criteria for fairness, We refer the readers to Mitchell et al. (2021); ) Corbett-Davies & Goel (2018); Pessach & Shamueli 2022 Berk al.
2021 for surveys of these related ideas.
The literature on fair MT, contrasts fairness criteria that apply to each individual and those that apply to groups. The definitions of eco and of in this paper are individual-level critcria. Similarly, Dwork et al. (2012) define a notion of eco without causality called individual fairness. Given a distance metric between individuals, it requires similar individuals receive similar decisions. The eco-fair decisions (of Eq. 1 recover this requirement without relying on an explicit distance metric, which can be difficult to construct. Speicher 2018); Chouldechova & Roth 2019) study the tradeoffss between group- and invididual-level criteria.
et al.
Finally, the method proposed here crucially relies on the theory and algorithms behind causal inference Pearl 2009: Peters et al.] 2017). In particular, it requires that the causal model is accurate, that the necessary couttefaactuls are identifiable, and that they can be estimated. In practice, these assumptions are strong and have the potential to be misused Kasirzadeh & Smart, 2021 Kohler- Hausmann, 2018). To alleviate
some of this burden, subsequent research can extend this method to employ recent ideas about causal fairness
under uncertain causal models (Kilbertus et al. 2018 Russell et al. 2017 Kilbertus et al. 2019 Wu et al.
2019a
De Lara. et al. 2021
3

---

Published in Transactions on Machine Learning Research (07 2203))
2 Assessing fairness with counterfactuals
Consider automating the admissions process at university. By using dataset of past admissions, the goal is to algorithmically compute the admissions decision for new applicants. The dalasel contains applicants, each with attributes and the committee's decision about whether to admit them. Some attributes are deemed protected, such as gender, religion, or ethnicity; others are not protected, such as a grade point average or a score on a standardized test.
To illustrate the ideas, we will consider a simple setting. Suppose there are only two attributes for each applicant, their gender and their score on test. (For case of exposition, consider two attributes and binary gender variable, but the methods in this paper easily handle multiple attributes and non- non-binary gender.) The gender is protected attribute; the test score is not. Fig. 1shows an example of such data. In the figure, the first 5,000 applicants are (simulated) historical data applicants' attributes and the decisions made by a committee. The three applicants A, B, and C are new applicants for whom we will make decisions.
A causal model of the decisions. The methods we develop require a causal model (Pearl 2009 of the historical admissions process. This model should capture the decision, the inputs to the decision, and the causal relationships among these variables. Consider the model in Fig. 2 It assumes that an applicant's 's gender A and test score S can affect the admissions decision Y; the admissions committee might unfairly prefer to admit males. It also assumes the applicant's gender A can affect the test score S; femalc applicants might. receive fewer opportunities for test preparation, disparily that results in lower scores. (There are also other variables in Fig. 2] which we will discuss below.) Note that positing a causal model is a strong assumption; is important for the user of the methods to consult domain knowledge and provide a plansible model of the historical decisions. Many causal models are special cases of the causal model in Fig. 2 if we have multiple (or multi-dimensional) protected and unprotected attributes.
We discuss these cases in $ 3
A causal model implies a distribution of counterfac- tuals, how each variable would change under hypo- thetical intervention on other variables [Pear] 2009 (Note this "intervention" is only a mathematical con- struction, a hypothetical change that helps articulate counterfactuals. It may be about an immutable char- acteristic (Greiner& & Rubin 1)) Denote (a s) to be the counterfactual decision of the ith applicant when we set their test score to be S; = s and gender to be A, = a. For example, consider the counter- factual docision of a female applicant had she been a male and had the same lest score. A key idea of this paper is that we can use the assumed model to ask counterfactual questions about any decisions, either those made by a committee or those made by an algorithm. We first show how we can interpret fairness through properties of such counterfactual de- cisions. We then show how to adjust an algorithmic decision maker, such as one produced by classical ml, to produce decisions that satisfy fairness properties.
ID
Sex
Test
Admit
fml
yeco
yref
54
yes
2
m
66
no
5000
44
no
A
f
85
0.67
0.77
0.78
B
85
0.84
0.77
0.76
C
65
?
0.57
0.69
0.70
Figure 1: Simulated data of past university applicants (1-5000) and their admissions decisions. The (fictional) admissions committee violates eco and does not enforce cf. For three new applicants (A, B, C), a machine learning classifier trained on this biased data yields unfair decisions, denoted by f For example, a male and female applicant with the same test score receive different admission probabilities, violating eco. We algorithmically adjust this decision to produce ÿ decisions that are eco-fair. This decision provides the equally qualified two candidates A and B an equal probability of being admitted. From this &eco decision, we derive yet, an algorithmic decision that enforces cf. It increases the probability of admission for applicant C, by accounting for her systematic disadvantage.
The equal counterfactual opportunity criterion. A decision maker provides equal counterfactual oppor- tunity (eco) when equally qualified people reccive the same decisions regardless of their prolected at- tributes. In the causal model, we say that a decision maker satisfics eco if changing the protected attribute does not change its distribution of the decision Y. Consider an applicant with gender a and test score s. An eco decision maker gives the same probability of admission even after changing the gender to a' (but keeping the test score at s). In Fig. 1 candidates A and
4

---

Finally, eco criterion is also equivalent to conditional demographic parity @work et al. 2012 conditional on all remaining attributes, if all remaining attributes (the unprotected attributes) are descendents of the protected attribute in the cansal graph. For example, if the protected attribute current monthly income, and the unprotected attributes include the location of the individual's first. job (which is an ancestor of the protected attribute), then ECO-fairness not equivalent the conditional demographic parity given all remaining altributes. However, suppose the unprotected attributes only include the descendants of the is equivalent to Published in Transactions on Machine Learning Research (07/2023)
B have the same test score but different genders; if given equal opportunities, they should receive the same probability of admission.
Definition (Equal counterfactual opportunity (eco)), A decision Y sotisfies eco in the protected attribute A if. for all possible values of a, a' and 8,
((a,s)](A=a, a, = _ (d {A a,
= ss..
(The notation = R means Q and R are equal in distribution.)
This criterion requires that, for any individual with protected attributes A a and unprotected attributes S = 8, their counterfactual decisions Y(a', s) must have the same distribution if they had the same unprotected attributes, no matter what the protected attribute value A = a' may be.
An algorithmic decision that satisfies eco asserts that there is no causal relationship between the protected attributes and the decision Y. Tf a committee or algorithm produces eco-fair decisions then the causal model contains no arrow between A and Y; two applicants from different demographic groups, but who are otherwise similarly qualified, will have the same probability of admission.
fml
feco
Sct
A
Y
Y
Proco
Grel
S
n
Other research has posited mathematical criteria to capture the legal notion of oqual opportunity. HHardt ct al. (016)) proposed the equality of opportunity cri- terion evaluate the fairness of decisions predicted by a classifier. Their criterion requires the misclassi- fication rates for the disadvantaged group to be the same as those for the other group. In contrast, the eco criterion is finer-grained; it requires that the deci- sions are fair under each possible configuration of the attributes that are input into the decision algorithm; all individuals with the same unprotected attribute must reccive the same dccision probability regardless of interventions on their protected attributes. An- other fine-grained criterion appears in Dwork et al.) 20022, which defines a notion of cqual opportunity (without causality) called individual fairness. Given distance metric between individuals, it requires that similar individuals receive similar decisions. Algorithmic decisions that satisfy the eco criterion also recover this requirement, but with distance metric implied by the causal model.
Figure 2: The causal model reflects domain knowl- edge and assumptions about an admissions deci- sion process: protected attributes (A), such as gender, can affect both the admissions decision (Y) and the remaining attributes (S), such as test scores. Algorithmic decisions (Yrml, yrcco yref) are variables in the causal model, produced by fitted algorithms (fml, Soco: fct) and we can inspect their biases. Classical ml reproduces the historical data it was trained on, inheriting the unfairness of a past admissions committee. eco decisions adjust ml, eliminating any problematic dependence on the protected attribute. of decisions further ad- just eco decisions to exercise affirmative action based on the protected attributes.
protected attribute, e.g. the individual's monthly expenditure. Then the conditional demographic parity.
The counterfactual fairness criterion. The counterfactual fairness (cf) criterion was introduced by Kusner et al. (2017). A decision maker that targets corrects for historical disadvantages that may stem from protected attributes. Female applicants may have fewer opportunities for test preparation, leading to lower test scores and lower likelihood of admission. Had they been male, they may have had more opportunities for preparation, achieved higher score, and been admitted.
5

---

Published in Transactions on Machine Learning Research (07/2023)
One of the goals of cf is to ensure that, all else being equal (such as effort aptitude), an applicant would receive the same decision had they not been in disadvantaged group. This goal is different from that of eco because it accounts for how the protected attribute affects the other attributes, how an applicant's gender affects their test score. Moreover, eco prohibits direct impacts of protected attributes on decisions but allows an indirect influence through unprotected attributes. In contrast, cf prohibits both types of influences, direct. or indirect.
Mathematically, counterfactuals help capture this idea. Consider applicant with test score s and gender Had they belonged to different group d', they may have achieved different test score S(a') {A = a, s). Notice this counterfactual score is conditional on the attributes of the applicant, including their (factual) score. The variable ((male) {A = female, S = high score} can capture that a female applicant with a high test score will have an even higher test score had she been male.
The cf critcrion further uses nested counterfactual. The variable Y(a', ((a') | {A= a =) involves an intervention on the protected altribute along with a resulting change in the unprotected attribute. An ef-fair decision is one where this counterfactual does not change the distribution of the decision.
Definition 2 (Counterfactual fairness (Kusner et al. 2017 ) A decision Y exercises counterfactual fairness (cf) if, for all possible values of a, a' and s,
Via, (4=
E {Ad
This criterion requires that, for any individual with protected altributes A= a and unprotected attributes S = their counterfactual decisions (ad' S(a')) must have the same distribution as if they had different protected attribute value A A=a a', together with the corresponding counterfactual unprotected attribute under this different protected attribute S(a').
Algorithmic decisions that satisfy of are not affected by (hypothetical) interventions on protected attributes, including accounting for how those interventions might subsequently change the other attributes. They use the protected attribute to correct for historical disadvantages.
3 Constructing fair algorithmic decisions
Dets. and provide criteria for fairness in terms of distributions of counterfactual decisions. Using the causal model of the decision-making process, these criteria help evaluate the fairness of its outcomes. The critcria can be used to evaluate decisions produced by human process, such as admissions committee, produced by an algorithm, such as fitted ml model.
Denote an algorithmic decision as Y, a causal variable that depends on the attributes {a, s). It comes from a fitted probabilistic decision maker, y ~ f(a s), where f()) is a probability density function. For example, a binary admissions decision Y ~ ({,,s) )) is drawn from a Bernoulli that depends on the altributes (e-g., a logistic regression). By considering algorithmic decisions as variables in the causal model, we can infer algorithmic counterfactuals Ý ((a s) and confirm whether they satisfy the and of criteria.
Now consider a classical ml model that is fit to emulate historical admissions data. If the historical decisions did not exercise eco or of then neither will the decisions produced by ml. Below we develop fair ml decision makers, algorithms that adjust the decisions made by classical ml to be eco-fair or cf-fair, i.e., to satisfy the eco and of criteria. We will show that these adjusted decisions are as accurate as possible relative to the historical data, while still producing oco-fair or cf-fair decisions.
Machine learning (ML) decisions. An machine learning (ml) decision maker uses historical data to accurately predict the decision Y from the protected attribute A and unprotected attribute S. Admissions involves binary decisions, and so logistic regression is a common choice. The decision maker fai(a, s) draws the decision from a Bernoulli, yml {A- = S = =} ~ Bern(a s + BA a + Bu)), where o--) is the logistic function and the coefficients are fit. to maximize the observed data likelihood. When included in the causal model, fmi and ml are illustrated in Fig. 2
The ml decision ml will accurately mimic the historical data, But it will also replicate harmful discriminatory practices. If the committee did not give equal opportunities to applicants of different genders then the
6

---

Published in Transactions on Machine Learning Research (07/2023)
decision yrml will violate eco. If the committee did not correct for the disparate impact of gender on the test score, then ÿmm will not exercise cf.
Return to the illustrative simulation ig.. We fit a logistic regression to the training data, which finds coefficients close to the mechanism that generated the data. Consequently, when used to form algorithmic decisions, it replicates the unfair committee. Consider female applicant (a -a=f, = 85) and male applicant. В (a m. 85) and the classical ml decisions for each. For A, her probability of being admitted 67%. For B, his probability of admission is 84%. Despite identical scores, the female applicant is 17% less likely to get in.
Algorithmic decisions that satisfy eco. We use the ml decision maker ful to produce an eco-fair decision maker Soen, one whose decisions satisfy eco. Consider an ml decision maker smi(a, 8) and an applicant with attributes (anwws Snew]. Her eco decision is ÿremo (anew; (nee ~ feco(Suew) where
Seco (Soo()new)= = Smi(a, &new (hew)p(a) da.
(1)
The eco decision probability holds the unprotected attribute Snew fixed and takes a weighted average of the ml decision maker for the different values of the protected attribute The weights are determined by the proportions of each group in the whole population. Note that the ml decision maker is fixed; it does not need to be retrained.
Return to Fig. We use the fitted logistic regression fmi(a, s) to produce the fair decision maker. This data has equal numbers of women and men so the weighted average is faco s) 0.5 (male, 0.5m (female, s). Using eco-fair decisions, applicants A and B both have a 77% probability of admission.
The eco decision yrece ~ feco (Snew) satisfies the eco criterion: applicants with the same score will have the same chance of admissions regardless of their gender. The intuition is that the eco decision preserves the causal relationship between the test score S and the ml decision ÿmml, but it ignores the possible effect of the protected attribute A. In the do notation for interventions (Pearl 2009), what this means is that P(yeco; do(a, +)) P(ýml,; do())) ((os)) for all a. The weighted average is the adjustment formula. (Pearl 2009 which calculates p((ss.
Finally, Thm. shows the theoretical optimality of the eco decision makers among all those that satisfy eco.
Theorem (eco-fairness and optimality of eco decisions). eco decisions salisfy the follow properties:
The decision fr Seco(a,3) s) eco-fair.
2. Among all eco-fair decisions, P-E marimally recovers the ML decision ý'l
PEGO
E + (A))))
arg min yεсogyεCO
where EECO the set of eco-fair decisions and the expectation is taken over P(A) (Proof in App. A.
Thm. shows that PECO is the eco-fair decision maker that closest to the ML decision maker in KL divergence, over a population where the protected and unprotected attributes are independent.
Thm. focuses on probabilistic decision makers gml even though the optimal classifier that minimizes the misclassification rate is the deterministic Bayes classifier, which output decisions that are deterministic function of the protected and unprotected attributes. The reason is that the ground truth outcomes are uncertain, c.g. an outcome of whether a student will be admitted to an Ivy League school entails randomness in the process. Given probabilistic ground iruth outcomes, the optimal probabilistic decision maker pml minimizes the KL divergence to the ground truth outcomes, which is equivalent to minimizing the commonly used cross-cntropy loss a binary classification setting. While probabilistic decision makers yml may not be as accurate with respect to the misclassification rate as deterministic decision makers, they can reflect the uncertainty in the predictions, remaining faithful to the uncertainty in ground truth outcomes. In contrast,
7

---

Published in Transactions on Machine Learning Research (07/2023)
deterministic classifier ignores this uncertainty. We thus consider probabilistic decision makers for both ML and fair decision makers.
A reader may ask: in Thm. 1 why do we define optimality in terms of the KL distance between fair decision makers and ML decision makers, given that probabilistic ML decision makers could be suboptimal in classification? Why not minimize the distance to the ground truth outcomes? As it turns out, minimizing a decision maker's KL divergence the optimal probabilistic decision maker ÿ will also minimize its KL divergence to the ground truth outcomes (if the class of probabilistic ML decision makers is flexible enough). The reason is that the optimal probabilistic ML decision maker (in terms of KL divergence to the ground truth outcomes) must have the same distribution as the ground truth outcomes,
Gm arg
arg min (a ( ())))((((a
Va,
(a - ((, Va,
Thus, considering the KL divergence to optimal probabilistic decision makers makes a meaningful metric; a decision maker that minimizes this distance implicitly minimizes the KL divergence to the (random) ground truth outcomes.
FTU, Greco and why consider the population of P(A) in Thm. An
1.0
alternative method to satisfy eco is ftu (Kusner et al 2017 ftu satisfies eco by fitting an ml model from the unprotected attribute S to the decision Y, and completely omitting the protected attribute A; ftu decisions are also eco-fair. While both ftu and the eco decision makers satisfy the eco criterion, ftu may still indirectly correlate with the protected attributes if an unprotected attribute correlates with the protected attribute the population. In contrast, the eco decision maker would not.
0.8
Decision
6.8
0.2
Maie
0.0
As an example, we simulate additional umprotected attribute that has 0.9 correlation with gender. We further subsample the adult dataset such that the correlation between the protected attribute and the decision is 0.7. We then compare the ftu decision and the cco. While both cco-fair, Fig. 3 shows that the ftu decisions remain highly correlated with gender while the eco decision has substantially lower correlation. The eco decision avoids inheriting discrimination patterns when unprotected attributes are highly correlated with the protected attribute. Such high correlation between protected and unprotected attributes do not appear in the adult dataset; the prediction accuracy of ftu and feco are similar. That said, Fig. 3 delincates a sctting where the eco docision maker may be desired.
Figure 3: fairness through un- awareness (ftu) vs, feco ; The eco decision avoids inheriting dis- crimination patterns when un- protected attributes are highly correlated with the protected at- tribute. While both eco fair, ftu decisions remain highly corre- lated with gender while the eco decision has a substantially lower correlation.
This distinction between ftu and the eco decision makers also suggests we consider the population of P(S) in Thm. I The theorem shows that the eco decision maker is closest in expected KL to the ML decision maker, among all fair decision makers. Note the expectation is taken with respect to P(A) distribution where there is no information 'leaked' from the protected attribute to the unprotected attributes. The reason is that if there is correlation between them, then we could consider "fair" decision makers that are not actually fair they might make decisions based on unprotected attributes but really be capitalizing on correlations to the protected ones. The optimality the fair decision maker is under the setting where we only pick attributes that are uncorrelated to the protected one.
A Algorithmic decisions that exercise cf. We now show how to adjust the eco-fair decision maker satisfy counterfactual fairness.
Consider the eco-fair decision maker (s) and an applicant with attributes {anows Snew }. Her cf-fair decision is of ~ (Anew: Snew), where
fer Snow)) )) ffo (s(a)) p(s(a) Onow, Snew) ds(a) (a)aaad.a da.
Thus we form an of decision of by drawing from a mixture: (a) sample a gender from the population distribution p(a); (b) sample a test. score from its counterfactual distribution s(a) ~ p(s(a) Gnew, Snow );
8

---

Published in Transactions on Machine Learning Research (07/2023)
(c) sample the eco-fair decision for that counterfactual test score Ged feco We repeat this sampling over different values of and s(a) to form the ef decision yre..
One subtlety of ÿd is step (b). It draws the counterfactual test score under an intervened gender, but conditions on the observed gender and test score. This requires abduction (Pearl 2009), where unobserved noise variables are inferred conditioned on observed variables, allowing us to sample counterfactual cutcomes and construct CF-ffir decision makers. For example, consider a model where females have fewer opportunities for test preparation. If we observe female applicant with high test score, the unobserved noise might capture that this applicant is particularly gifted. Under this value of noise, her counterfactual test score will be even higher.
Put differently, the ef-fair decision maker uses the eco-fair decision maker (s), which only depends on the test score (and averages over the gender). However, the ef-fair decision maker also corrects for the effect on the test score due to the gender of the applicant. It replaces the current test score Suew with the adjusted score s(a) {anew: Snew } under a ~ p(a). With this corrected score, it produces an cf-fair decision.
0.80
0.1-
Paco
0.70
-0.2
0.65
-0.3
0.60
Blas due to sox R.
Blas cun sex
Fairicaming
For the simulated data in Fig. 1 we calculate fat(a, s) from feco(a, s). This decision maker requires abduc- tion, calculating the test score cach applicant would have achieved had their gender been different. Con- sider applicant C. Her eco-fair probability of accep- tance is 69%, but when exercising of it increases to 70%. This adjustment corrects for the (simulated) systemic difficulty of females to receive test prepa- ration and, consequently, higher scores.
0.00
0.80
-0.05
0.75
-0.10
--.15
-0.20
0.65
-0.25
0.30
0.60
0.00
0.25
0.75
0.25
0.50
Hetoriccl deadventage
Ilistonccll disadvantage
The cf-fair decision maker fa((a,) s) accounts for his- torical disadvantage because it adjusts the appli- cant's test score (and their resulting admissions decision) to their counterfactual test scores under intervention on gender. Further, each element is computable from the dataset. (We discuss how to calculate these decisions below.) Note that cf-fair decisions can be formed from any eco-fair decision maker.
Figure 4: Measures of decision-qualityy in the simulated admissions data (error bars indicate = 1 sd). The eco decision maker JECO is the most accurate one that achieves eco-fairness; the cf decision maker fer is the most accurate one that satisfies cf-fairness. The classical ML decision maker fml is most accurate overall but violates both fairness measures. We vary the historical disadvantage 8, from 0.0 to +5.0 and vary direct gender discrimination À from 0.0 to +0.8.
The ef-fair decision maker depends on the protected attribute, and thus the cf-fair decision ýee is not eco-fair. But among all ff-r decisions, it is closest to the eco-fair decision; we prove this fact in Thm. Theorem (cf-fairness and optimality of the of decisions ). of decisions satisfy the following properties:
1. The decision &et far(a, 8) is cf-fair.
2. Among all of decisions, the of decision maker minimally modifies the marginal distribution of Y
get arg min E [KL(P() ec0 ( (A.5))))(Y"" yeg
where yet are all of decisions and the expectation is over P(A) It also preserves the marginal distribution of the eco decision maker, P(Yel)- = P(yecco) (Proof in App. B.)
hm..3 also says that fair decisions preserve the marginal distribution of the eco-fair decisions. If the eco decision maker admits 20% of the applicants then the of decision maker will also admit 20%. (The of
9

---

Published in Transactions on Machine Learning Research (07/2023)
decision maker will likely admit a different set of applicants from the eco decision maker.) That it preserves the marginal distribution makes the cf-fair decision maker applicable as a decision policy, such as when there is a fixed budget for admissions.
Finally, cfif decisions also satisfy demographic parity, a group-level statistical criteria that has been used as a measure of cf (Dwork et al. 2012). Demographic parity requires equal decision distributions for the advantaged and disadvantaged groups of the protected attribute. (It does not involve causal model.) Kusner
et al. (217)) shows that, assuming the causal model in satisfy demographic parity.
Fig. 2 decisions that satisfy counterfactual cf also
Why are the eco and ef decisions optimal? TTims. and 2 establish the optimality of the the eco and of decisions; they rely on the following observation. The polential for unfairness arises when individuals have different values in their protected attributes. In the admissions example, applicants have different genders (protected attribute), which may causally affect their admission decisions (outcome) and lead to unfairness. If all applicants had the same gender then any decision maker would be "fair" in that there is no possibility for discrimination.
This observation lcads to the theorems. The eco and of decision makers predict in fictitious world where all applicants had the same gender A'. They take an existing decision maker and ask: what would its counterfactual prediction be if each applicant had the gender A'? The eco decision [F.. 1) averages over a distribution P(A'). Thm. I says that the eco decisions are closest to the ML decisions when A' is distributed as the gender distribution the data. Thm. 2 says that the of decision maker closest to (i.e. "minimally modifies") the eco decision maker.
The decisions are probabilistic, so closeness in the two theorems is measured by the average KL distance. The average is over target population where the protected A and unprotected S are independent; this mimics an ideal setting that fair learning algorithms target, i.e. where no applicants are disadvantaged.
Calculating eco and ef decision makers. The eco and of decision makers can be calculated from data. The eco decision maker uses the fitted ML decision maker ful: but marginalizes out the protected attribute. The ef decision maker uses the eco decision maker foco: including an abduction step [Pearl] 2009) to calculate the counterfactual value of the attribute s(a').
Algorithm 1 The eco and of decision makers (for additive-error models).
Input: Data D {(ai, 8i, vi)] where os is pro-
tected, is not, and yi is the decision.
Output: Decision makers {miia s), foco(s), Sar(a, s)}
We focus on settings where the counterfactual deci- sions are identifiable from observational data. We assume: (1) The protected attributes A and unpro- tected attributes S follow the causal graph in Fig. 2 That is, there is no unobscrved confounding between A and S, between S and Y, and between A and Y. (2) The structural equation model for A, S, Y needs to be correct so that structural equation can be iden- tified from observational dala. These two conditions enable the identification of counterfactual decisions.
From the data D, Smi(a, 8), p(a), and g(a) = E [[S A a] (e.g., with regression).
The ml decision maker fmila, s) draws from
gml ~ fmi(a, s).
The eco decision maker Soco(s) draws from Eq. I a' d'~ goco Smilla,
Given the necessary assumptions for identifiability, the counterfactual calculation requires modeling the effect of the protected attribute A (gender) on the other attribute (test score). With linear models, this calculation involves combining the residual on the observed data with prediction about. the coun- terfactual setting. In the data of Fig. 1 the abduction infers what applicant C's test would have been had she been male, given her test score (as female) was 65. The algorithu can also be applied to nonlinear
The of decision maker Ser(a, 8) draws from Fq. 2
a' p(a): s' = =((aa)) + (s - -(aa));; get ~ fcco(aa,
models; see
App. C for details.
10

---

Published in Transactions on Machine Learning Research (07/2023)
As concrete example, Alg. 1 provides the algorithm for calculating and cf decision makers (Eqs. and 2) from fitted ml decision maker in the case of additive-error models, e.g. 8= s + for some function g and some random variable e. (We discuss the identification of the eco and ef decision makers and prove the correctness of Alg. 1 in App. CD)
The FairLearning Kusner et al. 2017 algorithm, which also satisfies cf-fairness performs the same abduction step as fer compute residuals E, =, -.- L A= a:} but then fits decision maker using only the residuals and non-descendants of the protected attributes. For example, in admissions, after calculating the residual it does not include the test score in its decision maker. In contrast, Alg. I uses all available attributes and still is cf-fair. The eco decision maker in
Alg. 1 also makes use of all attributes; it does not omit protected
attributes as done in ftu (Kusner et al. 2017 Empirically,
Alg. provides more accurate decisions than
these existing eco and cf algorithms.
Multiple protected and umproteeted attributes. A g.. can be generalized to scttings with multiple protected and unprolected attribules. Note that both A and S can be a vector, so we can collect all protected attributes into the A variable and all umprotected attributes into the S variable. As long the resulting attributes still follow the causal graph in Fig. 2 Alg. T is still correct. That said, Alg. I may not apply in other cases, e.g. when the following three conditions simultancously hold: (1) the multiple unprotected attributes are chained; (2) we consider interventions on only a subset of the unprotected attributes; (3) we focus on a subpopulation by conditioning on a descendant of the intervened unprotected attributes. In this case, the corresponding counterfactual is not identifiable.
4 Empirical studies
We study algorithmic decisions both simulated and real datasets, examining the tradeoff between fairness and prediction quality. We compare the eco and ef decision-makers to existing algorithmic decision-makers fairness through unawareness (ftu) and FairLearning (Kusner et al. -that target the same cco and cf fairness criteria. Throughout the empirical studies, we consider ML decision-makers that are (generalized) linear models. We use linear regression models when the outcome is real-valued, and logistic regression when the outcome binary. (The supplement provides software that reproduces the studics.)
We find the following. (1) As expected, the classical ml decision ÿ is accurate but unfair. (2) The eco decision gre is less accurate than ýmml, but is eco-fair; it is more accurate than ftu. (3) The of decision yee is less accurate than the cco-fair decision, but is ef-fair and achieves demographic parity; it is more accurate than FairLearning.
Evaluating algorithmic decision-makers. For each decision maker, we measure the eco and of fairness of its decisions, and evaluate its fidelity relative to the historical data.
One way to measure fairness is to compare distributions of counterfactual decisions. Sup- pose a protected attribute A with values disadvantaged group). As one metrie, we consider the average violations of eco and of over the population as metrics: Peco = 1 E [ [a (;),s)));)) (q, - (a', st) a, a,,s Pet =
(for the advantaged group) and a' (for the
E E (a, s(a)) az 8: E (f(a's(aa)) s(a')) ai 8:
]] The sum is over n samples in a held-out test set;
expectations are with respect to the decision-makes
The eco motric contrasts the decision probabilitics when changing the protected attribute but holding other attributes fixed. The ef metric contrasts the decision probabilities when changing the protected attribute A and allowed other attributes to counterfactually vary. When the metric is 0.0, the decision- maker achieves eco-fairness; when it is greater than 0.0, there is bias towards the disadvantaged group; when it is less than 0.0, the decksion-maker favors the disadvantaged group. The same interpretation applies to cf.
Another measure of ef fairness demographic parity Dwork et al. (012)). We measure demographic parity with the symmetric Kullbeck-Lcibler (kl) divergence between prediction distributions for the docision P( |4 = a) and P( | A = a'); it is zero for decision-maker that achieves demographic parity. (We evaluate the symmetric kl by binning the values of predictions.)
11

---

Published in Transactions on Machine Learning Research (07/2023)
Finally, we measure the fidelity of the decision-make to the historical data which it was fit. The metric is the prediction score: for binary outcomes, it is the mean accuracy of out-of-sample predictions; for real-valued outcomes, it is the coefficient of determination It² of the predictions.
Simulated admissions.
We first study simu-
lated datasets about an unfair admissions com- mittee, which is from the following structural model, a, ~ (ernoulli(0.5):: s; a; max(0, min(A- a; + 100 E, 100)); ~ Uniform O0, 1]:3% | 0, Bernoull((o(-110- B, 8 . Ba a))-
Metrics (x10²)
on Adult
Peco
Pdt
KL
Prediction
fml
2.5(1.9)
16.2(12.0)
15.0
78.6
ftu
0(0)
14.8(7.9)
12.2
77.3
(We threshold the Lest. score to mimic real-world lest scores that are usually bounded.) We fix the effect of test score on admissions to 3. = 2.0. We generate multiple datasets by varying the gender bias Sa and the historical disadvantage on test score X.
Seco
0(0)
(8.3)
12.6
77.4
FL Set
-14.8(9.1)
0(0)
5.3
75.1
-9.1
0(0)
1.5
77.1
Figure 5: the Adult dataset, algorithmic decision- makers decide which individuals are loanworthy based on their income and examine their fairness relative to gender. The eco and ef decision makers are fair lowards females while remaining the most accurate. Both the eco decision maker feeo and ftu achieve eco- fairness as mcasured by the eco-metric. The of decision- maker fet and FairLearning (FL) (Kusner el al. 2017 are cf-fair and achieve demographic parity (close-to-zere KL). The ml decision maker fml is the most accurate overall but the eco decision maker is the most accurate among the eco-fair decision-makers. We report mean values across individuals with the standard deviation in parentheses. eco and cf metric st standard deviations are < 0.1 and V 0.11, respectively. We also report the KI. divergence and prediction accuracy, which are distributional metrics.
Fig 4 show how eco-fairness and prediction qual- ity trade off as the bias S. increases. Only the eco decision-makers fECO and ftu achieve eco fair- ness. Although both decision-maker are less accu- rate than classical ml, the eco periisoonmaker is more accurate than ftu. Fig. 4 shows how cf-fairness and prediction quality trade off the historical disad- vantage À increases. Only the fet and FairLearning baaision-makers achieve cf-fairness. Among these cf-fair decision-makers, the ef decision-maker of this paper is more accurate.
Case studies. We study eco and ef proision-makers simulated admissions dataset and three real datasets of sensitive decisions about people. The adult income data (Dua & Grafi 201 7a) and the German credit data (Dua & Grafi 2017b contain data about people and decisions about which are loan worthyH
ProPublica's COMPAS data contains information about criminal defendants and decisions about their recidivism score. Each dataset contains protected and unprotected attributes, and in the studies adjust classical ml income data in this section and defer the results of the German crodit data and the COMPAS data to App. F
to achieve fairness relative to the protected attributes. We focus on the adult
Fig. 5) summarize the fairness and prediction quality from the adult income data. In the adult income data, race and gender are the protected attributes. We discuss the fairness metrics Peco and Per relative to gender and report the mcan (and s.d.) of all the metrics. The findings are consistent with the simulation studies and generalize across the studies.
Although the classical ml decision-maker is the most accurate, its decisions are biased against the disadvantaged group. Positive values in the E0-metric Peco reveal direct discrimination in classical ml decisions; males and white individuals receive higher probability of being decided as loan-worthy or lower recidivism score than their equivalent female or non-white counterparts. Positive values in the AA-metric Pef indicate that the classical ml mecision-maker does not exercise cf. It provides lower decision probabilities to females or non- white individuals than their male or white counterparts whose attribute values also reflect the historical advantage.
1 Readers who familiar with these datasets know that they contain scores, as opposed decisions. We these scores to "decisions" in order evaluate the methods of this paper binary decisions.
12

---

Published in Transactions on Machine Learning Research (07 2222))
The eco and ftu decisions are eco-fair (Peco is zero) and the eco decisions are more accurate. The eco-fair decision-makee makers provide equal decision probabilities to individuals that are equal all other attributes, regardless of gender or race. Neither counterfactual of nor FairLearning achieve eco-fairness, but this is by design. These decisions produce negative values for the metric, showing advantage for the disadvantaged group. The ef-fair decision-makers correct for historical biases, which leads to differ- ent decisions for the disadvantaged and advantaged groups.
1.0
Female CF Decision Prob.
0.8
0.6
fLL
FTU
fEco
0.2
FairLearning
fcr
0.0
0.00
0.25
0.50
0.75
1.00
Male CF Decision Prob.
The ef and FairLcarning decisions both achieve cf- lairness (Pef zero) and demographic parity (kl is zero). They succeed in providing equal decision probabilities to females or non-white individuals af- ter considering their counterfactual attribute values had they been male or white. Among these cf-fair decision-makers, for is more accurate.
(a) Assessing
1.0
fNL
Female CF Decision Prob.
FTU
0.8
feco
0.6
FairLearning
fcr
Measuring decisions in the adult income data. We study the algorithmic decisions in more depth. Using the adult income data, we examine the metrics of Fig. (The results below exhibit similar patterns in all three datasets.)
0.4
0.0
0.00
0.25
0.50
0.75
1.00
We first discuss eco-fairness for gender. F ig. veals the same patterns of eco-fairness that Fig.
6a
re-
Male CF Decision Prob.
showed. Both eco and ftu decision probabilities align with the diagonal; they are eco-fair in providing equal decision probabilities to equally qualified individuals regardless of gender. None of the other algorith- mie decision makers is eco-fair. Fig. 68 shows that classical ml decisions are biased against females, pro- ducing lower decision probabilities for them than their male counterparts. In contrast, the cf-fair de- cision makers for and FairLearning are less eco-fair than fml but provide an advantage to females after accounting for historical disadvantagos. Further, ef and PairLearning are less eco-fair than classical ML.
(b) Assessing cf
Figure 6: Comparing individual decision probabilities under (hypothetical) "intervention" on gender (in the adult income data). (a) To study eco, the other at- tributes remain at their original values and we plot p(i(a, s) a, s) against p(i(a', s) a, s). The fECo deci- sion maker provides eco females; produces deci- sion probabilities that align along the diagonal. (b) To study cf, the remaining attributes vary with the inter- vention gender and we plot p(b(a, s) a, s) against. P(O(a,, s(a')) a, s). The decision maker fat exercises of; it provides equal decision probabilities to fermales as their male counterparts after correcting for the histor- ical disadvantage they facc. In contrast, the classical mi decision maker ful demonstrates both direct and in- direct discrinination against females; it provides lower decision probabilities to females compared to their male counterparts in both comparisons.
We next discuss cf-fairness for gender. Fig. 6b com- pares the docision probabilitics received by individu- als if they were females to those they receive if they were males, with the resulting changes to their other attributes. An algorithmic decision maker is ef-fair if the decision probabilities align with the diagonal. The decision makers fa and TairLearning produce decision probabilities that align with the diagonal and are thus cf fair. The remaining decision makers do not exercise cf; eco decisions provide a greater degree of cf-fnrnness than ftu decisions while classical ml decisions the most unfair in exercising cf.
We next inspect demographic parity. Fig. 7 illustrates demographic parity by plotting the decision probabilities for both male and female individuals. The left panel comes from a classical ml decision maker; the right panel
13

---

Published in Transactions on Machine Learning Research (07/2023)
comes from fei While the classical ml decision maker produces noticeably different decision probabilities for the two groups, the decision probabilities produced by the of decision maker Jat are nearly identical.
We now turn to the predictive performance. Fig. reports the predictive performance of all methods. Though it is not fair by either criterion, classical ml produces the "best" predictions, closest to the historical dataset. Among the eco-fair methods, eco produces better predictions than ftu. This corroborates Thm. Moreover, eco's prediction scores are close classical ml, so its fairness comes with little cost. Among the ef-fair methods, of predicts better than FairLearning; this corroborates Thm. 2 But since it alters the decision probability of individuals from disadvantaged groups, it predicts less well than the ml methods.
5 Discussion
We develop fair ML algorithms that modify fitted ML predictors to make them fair. We prove that the resulting predictors are eco-fair or cf-fair, and they otherwise maximally recover the fitted MT. predictor.
3.5
Gander - Famale
Gender Femalo
3.0
There are a few limitations of this work. One limita- tion in the scope of fairness notions being studied. We focus two counterfactual notions of fairness in this work; the proposed algorithms may only be fair with respect to these not ions; they may not im- prove fairness with respect to other fairness notions. Develop post-hoc algorithms that simultancously en- force multiple fairness notions is an interesting direc- tion.
2.0
1.5
1.0
0.5
1,0
0.0
1.0
Dexision Prob.
Dectiion
(a) of decision maker
(b) ml decision maker
Figure The adult income data: distributions of decision probabilities. (a) The cf-fair decision maker fet produces equal decision probabilities for females and males, achieving demographic parily with respect to gender. (b) The classical ml decision maker ful does not satisfy demographic parity.
Another limitation lies in the assumptions required by counterfactull approaches to fairness. The meth- ods in this paper rely on correct causal model, the ability to estimate its parameters, and the ability to estimate the necessary counterfactuals. We rec- ognize that these are strong pettrrements-psstttiii
and fitting causal models requires both domain expertisc and statistical care. Morcover, even with the correct model in hand, issues of unobserved confounding can affect the ability to properly estimate its parameters the counterfactuals. Developing fairness algorithms robust to violations of these assumptions is another worthwhile avenue of future work.
14

---

Published in Transactions on Machine Learning Research (07 2023)
References
Artelt, A., Vaquet, V., et al. (2021). Evaluating robustness of counterfactual explanations. orXic preprint arXiv:2103.0253
Berk, R., Hcidari, H., Jabbari, S., Kearns, M., & Roth, A. (2021). Fairness in criminal justice risk assessments: The state of the art. Sociological Melhods & Research, 50(1), 3 44.
Bremnan, T., Dicterich, W., & Ehret, B. (2009). Evaluating the predictive validity of the compas risk and needs assessment system. Criminal Justice and Behavior, 36(1), 21-40.
Chen, J., Kallus, N., Mao, X., Svacha, G., & Udell, M. (2019). Fairness under unawareness: assessing disparity when protected class is unobserved. Fairness, Accountabilityy and Transparency.
Chiappa, S. (2019). Path-specific counterfactual fairness. Proceedings of the AAAI Conference on Artificial Intelligence, 33, 7801-7808.
Chouldechova, A. & Roth, A. (2019). The frontiers fairness in machine learning. Communications of the ACM.
Corbett-Davies, S. & Goel, S. (2018). The measure and mismeasure of fairness: A critical review of fair machine learning. arXiv preprint ar arXiv: 1at.1100...02. 00023.
Corbett-Davics, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017). Algorithmic decision making and the cost of Tairness. In Proceedings of the 23rd acm sigkdd international conference on knowledge discovery and data mining (pp. 197-806).
Coston, A., Mishler, A., Kennedy, E., Chouldechova, A. (2020). Counterfactual risk assessments, evaluation, and fairness. FAT*.
Dai, J., Upadhyay, S., Bach, S. H., & Lakkaraju, H. (2021). What will it take to generate fairness-rreerving explanations? arXiv preprint arXiv:2106. 13344.
De Lara, L.,, González-Sanz, A., Asher, N., & Lonbes, J.-M. (2021). Transport-based counterfactual models. arXiv preprint 0rXh::2108.19025
Dua, D. Graff, C. (2017a). UCI machine learning repository: Adult data set.
Dua, D. & Graff, C. (2017b). UCI machine learning repository: Statlog (german credit data) data set.
Dutta, S., Venkatesh, P., Mardziel, P., Datta, A., Grover, P. (2021). Fairness under feature exemptions: Counterfactual and obscrvational measures. IEEE Transactions on Information Theory, 67(10), 6675 6675-
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012). Fairness through awareness. Innovations in Theoretical Computer Science Conference.
Foster, A., Vezér, Á., al. (2021). Contrastive mixture of posteriors for counterfactual inference, data integration and fairness. orXiv preprinl ar arXiv:2106.0811
Galhotra, S., Pradhan, R., & Salimi, B. (2021). Explaining black-box algorithms using probabilistie contrastive counterfactuals. In Proceedings of the 2021 International Conference on Management of Data (pp. 577 377-590).
Greiner, D. J. & Rubin, D. B. (2011). Causal effects of perceived immutable characteristics. Review of Economics and Statistics, 93(3), 775-785.
Hardt, M., Price, E., Srebro, N., et al. (2016). Equality of opportunity in supervised learning. Neural Information Processing Ssystems.
Karimi, A.H.,, Schölkopf, B., Valera, (2021). Algorithmic recourse: from counterfactual explanations interventions. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (pp. (53-362).
15

---

Published in Transactions on Machine Learning Research (07/2023)
Kasirzadeh, A. & Smart, A. (2021). The use and misuse of counterfactuals in ethical machine learning. In Proceedings of the 2021 ACM Conference on Fairness, Accounlability, and Transporency (pp. 228 228-236).
Kilbertus; N., Ball, P., Kusner, M., Weller, A., Silva, R. (2019). The sensitivity of counterfactual fairness unmeasured confounding. Uncertainty Artificial Intelligence.
Kilbertus, N., Carulla, M. R., et al. (2017). Avoiding discrimination through causal reasoning. Newral Information Processing Systems.
Kilbertus, N., Gascon, A., al. (2018). Blind justice: Fairness with encrypted sensitive attributes. Interna- lional Conference Machine Learning.
I. (2018). Eddic murphy and the dangers of counterfactual causal thinking about detecting racial discrimination. Nw. UL Rev., 113, 1163.
Kusner, M., Russell, C., Loftus, J., & Silva, R. (2019). Making decisions that reduce discriminatory impact. International Conference on Machine Learning.
Kusner, M. J., Loftus, J., Russell, C., & Silva, R. (2017). Counterfactual fairness. Neural Information Processing Systems.
Larson, J., Mattu, S., Kirchner, L., & Angwin, J. (2016). How analyzed the compas recidivism algorithm. ProPublica (5 2016), 9.
Lin, Z. "., Jung, J., Gool, S., Skeem, J. (2020). The limits of human predictions of recidivism. Science Advances, 6(7).
Madras, D., Creager, E., Pitassi, T., & Zemel, R. (2019). Fairness through causal awareness: Learning causal latent-variable models for biased data. In Proceedings of the conference on fairness, accountability, and transparency (pp. 349 358).
Mhasawade, V. & Chunara, R. (2021). Causal multi-level fairness. In Proceedings of the 2021 AAAT/ACM Conference on AI, Elhics, and Sociely (pp. 784-794 784-794).
Mishler, A. & Kennedy, E. (2021). Fade: Fair double ensemble learning for observable and counterfactual outcomes. Xix preprint ar *xXx:2109.001722
Mishler, A., Kennedy, F. H., & Chouldechova, A. (2021). Fairness in risk assessment instruments: Post- processing to achieve counterfactual equalized odds. In Proceedings of the 2021 ACM Conference Fairness, Accountabilily, and Transparency (pp. 386 400).
Mitchell, S., Potash, E., Barocas, S., D' D'Amour, A., Lum, K. (2021). Algorithmic fairness: Choices, assumptions, and definitions. Annual Review of Statistics and Its Application, 8, 141-163.
Nabi, R. & Shpitser, I. (2018). Fair inference on outcomes. AAAI Conference Artificial Intelligence.
Parafita, A. & Vitria, J. (2021). Deep causal graphs for causal inference, black-box explainability and fairness. In Artificial Intelligence Research and Development: Proceedings of the 23rd International Conference of the Catalan Association for Artificial Intelligence, volume 339 (pp. 415).: 1OS Press.
Pearl, J. (2009). Causality. Cambridge University Press.
Pessach, D. & Shmueli, E. (2022). A review on fairness in machine learning. ACM Computing Surveys (CSUR), 55(3), 1 44.
Peters, J., Janzing, D., & Schölkopf, B. (2017). Elements of causal inference: foundations and learning algorithms. MIT press.
Russell, C., Kusner, M., Loftus, J., & Silva, R. (2017). When worlds collide: Integrating different counterfactual assumptions in fairness. Neural Information Processing Systems.
16

---

Published in Transactions on Machine Learning Research (07/2023)
Speicher, T., Heidari, H., et al. (2018). A unified approach to quantifying algorithmic unfairness: Measuring individual & group unfairness via inequality indices. Knowledge Discovery and Dala Mining.
Veitch, V., D'Amour, A., Yadlowsky, S., & Eisenstein, J. (2021). Counterfactual invariance spurious correlations in text classification. Advances in Neural Information Processing Systems, 34.
Waters, A. & Mikkulainen, R. (2014). Grade: Machine learning support for graduate admissions. AT
Magazine, (pp.64).
WI,, Y., Zhang, L.,, & Wu, X. (2019a). Counterfactual fairness: Unidentification, bound and algorithm. In Proceedings of the Twehty-Eighth International Joint Conference on Artificial Intelligence.
Wu, Y., Zhang, L., Wu, X., & Tong, H. (2019b). Pe-fairness: A unified framework for measuring causality- based fairness. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' d'Alcé-BBuc, E. Fox, & R. Garnett (Eds.), Advances in Newral Information Processing Systems, volume 32: Curran Associates, Inc.
Zhang, J. Bareinboim, E. (2018a). Equality of opportunity in classification: A causal approach. Newral Information Processing Systems.
Zhang, J. & Bareinboim, F. (2018b). Fairness in decision-making the causal explanation formula. AAAT Conference on Artificial Intelligence.
Zhang, L., Wu, Y., & Wu, X. (2017). A causal framework for discovering and removing direct and indirect discrimination. Proceedings of the 26th International Joint Conference on Artificial Intelligence, (pp. 3929 3935).
17


Reviews:
[{"review_id": "VcFgu3EjA2", "reviewer": "Reviewer_s9RH", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n---------\nThis paper is well written and generally easy to follow. The empirical studies use sensible baseline methods, and they appropriately use both simulated data (where the counterfactuals are known) and a range of datasets that appear frequently in the fairness literature.\n\nWeaknesses\n----------\nI have concerns about both the technical soundness of the proposed methods as well as their interest for TMLR readers.\n\nRegarding technical soundness, my most serious concern is that Theorem 1, which describes an optimality property of the ECO-fair predictor, appears to be incorrect. The first two lines of the proof in the appendix suggest an assumption that the sensitive feature $S$ and the remaining features $A$ are independent, but they are not independent under the causal graph in Figure 2, which is the causal structure assumed throughout the paper. (It would be a very different setting if they were independent.) If they are not independent, then equation (8) requires $P(S \\mid A)$ in place of $P(S)$, and the result in equation (15) is $\\int P(\\widehat{Y}^{\\text{ml}}(S, a))P(S \\mid A = a)dS$.\n\nIndeed, it's easy to come up with a toy example that shows that their predictor doesn't minimize the expected KL-divergence as described in Theorem 1; the predictor that does is $\\widehat{Y} \\sim \\text{Bern}(f(a))$, where $f(a) = \\int P(\\widehat{Y}^{\\text{ml}}(S, a))P(S \\mid A = a)dS$.\n\nSince the CF-fair predictor takes the ECO-fair predictor of Theorem 1 as a starting point, these issues propagate to the other method and to Theorem 2, which means they are relevant to all the results in the paper.\n\nRegarding interest to TMLR's audience, I'm not sure that the optimality property described in the two theorems is actually ever desirable, and if not then I'm not sure what interest these methods hold.\n\nMore specifically, the authors assume that an initial model $f_{\\text{ml}}: \\mathcal{S} \\times \\mathcal{A} \\mapsto [0, 1]$ is trained to approximate the historical decision process represented in the data. This model induces a randomized classifier $\\widehat{Y}^{\\text{ml}}(s, a) \\sim \\text{Bern}(f_{\\text{ml}}(s, a))$, and their two theorems say that their ECO-fair and CF-fair predictors minimize the expected KL divergence with respect to $\\widehat{Y}^{\\text{ml}}(S, A)$. However, it is well known that the optimal classifier is the deterministic Bayes classifier $\\widehat{Y}(S, A) = 1[\\mathbb{E}[Y \\mid S, A] \\geq 0.5]$, not a randomized classifier like $\\widehat{Y}(S, A) \\sim \\text{Bern}(\\mathbb{E}[Y \\mid S, A])$. Since randomized classifiers are never the most accurate possible classifiers, it is unclear what the value is of minimizing divergence from $\\widehat{Y}^{\\text{ml}}(S, A)$ rather than the underlying decision process. Once again, it is easy to construct toy examples where their ECO-fair classifier is far less accurate than a simple deterministic ECO-fair classifier constructed via Fairness Through Unawareness, so it's at least clear that these methods aren't optimal in terms of accuracy.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes two methods for training fair classifiers (aka decision processes), one designed to satisfy a fairness criterion they call Equal Counterfactual Opportunity (ECO) and one designed to satisfy Counterfactual Fairness (CF) as proposed by Kusner et al. (2017).\n\nEach method proceeds by first training an unconstrained probabilistic classifier and then applying a simple modification that gives rise to a new, fair probabilistic classifier. Each method is supported by a theorem which shows that the method is \"optimal,\" in the sense that among all methods that satisfy the corresponding criterion, their method minimizes the KL divergence between the initial probabilistic classifier and the resulting fair classifier.\n\nThey compare their methods empirically on a simulated dataset and three real datasets, under causal assumptions sufficient to render identifiable the counterfactual quantities required for Counterfactual Fairness. They compare their ECO-fair method to Fairness Through Unawareness (FTU), which also trivially achieves ECO-fairness; and they compare their CF method to the original FairLearning method proposed by Kusner et al. (2017).", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "- **[critical]** Unless I have seriously misunderstood something, the proofs need to be checked/corrected, and then the results need to be re-run using the corrected method described above. \n- **[critical]** I think a discussion is required about why the divergence property described in the theorems (which should hold with the corrected method) is desirable/meaningful. Or perhaps there are other desirable properties that these methods have that can be presented instead or in addition.\n- **[critical]** In the empirical studies, what kind of structural model do they assume? Do they assume linear models throughout?\n- **[critical]** Figure 5 is too small to read fully.\n- **[would strengthen the work]** ECO-fairness is equivalent to conditional demographic (aka statistical) parity, conditional on all remaining features. It's worth acknowledging that while causal reasoning might plausibly lead to ECO-fairness, causal assumptions and causal inference techniques are not required for their ECO-fair method. This equivalence also suggests using other techniques as a baseline, since there are myriad techniques for demographic parity which can be easily extended for conditional demographic parity. \n- **[would strengthen the work]** \"The eco criterion is individual-level; it requires that each decision is fair... Similar to the ECO criterion, the CF is also individual-level.\" This makes it sound like these criteria operate at an unusually very fine-grained level, but they just modify distributions of decisions conditional on the inputs to the decision algorithm. While it's common in the fairness literature to characterize these types of criteria as \"individual-level\" fairness criteria rather than \"group\" fairness criteria, it's a very strong assertion to say that \"each decision is fair.\" I find this semantically at odds with the fact that the resulting classifiers are randomized, so my concern is that this phrasing will be confusing for other readers.", "broader_impact_concerns": "No concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "aWL0O81bsk", "reviewer": "Reviewer_FtKh", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Overall, I really enjoyed reading this paper. \n\nS1) Counterfactual fairness is a tricky topic to cover correctly, and I found that the authors did a clear job of motivating the work, setting up the problem, and presenting meaningful results. One of the best parts of the paper is the clear engagement of the limitations of this work, for example the explicit need for a correct causal model of the world and no unobserved confounding. \n\nS2) The experiments were clear and well-designed. Certainly followup work could explore cases where the proposed model might run into technical errors (e.g., finite sample issues), but the presented empirical work is compelling.\n\nW1) The diferences in metrics on Figure 4 can be quite small, and it was unclear how to think about the effect of sample size. Confidence intervals or bootstrapping data splits would help clarify. \n\nW2) The notation for Definition 1 and 2 is nonstandard, and I found it very difficult to parse the text and the notation without an explicit statement about what the Y(s,a) | {.} or A(s') | {.} notations meant. One additional line in the text would have cleared that up.\n\nW3) Additionally, I spotted a few typos that are no terrible but should be amended:\n - p7, \"is particularly gifted. under this value of noise\" -> \". Under\"\n - p7, \"also corrects for the affect on the test score\" -> \"effect\"", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose a method for post-hoc adjustment of machine learning predictions using counterfactual reasoning. In particular, the authors introduce equal counterfactual opportunity (ECO), a definition centered around equivalent decision distributions agnostic to the protected attribute s. The authors show that their decision makers that optimally modify the marginal distribution of the corresponding decision Y for ECO and counterfactual fairness (Kusner et al 2017). Lastly, they present empirical experiments using a college admissions example comparing their defined bias metric, KL divergence between the proposed decision distribution and the vanilla ML model distribution, and the overall model performance.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "See W1-W3", "broader_impact_concerns": "With the topic of counterfactual fairness, it is always important to specify the underlying assumptions. This paper does an excellent job of that. No other concerns about broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "TawKYHF4XC", "reviewer": "Reviewer_qyZv", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n(i) The paper provides an easy-to-follow discussion of two reasonable causal fairness criteria (ECO and CF). It provides simple post-processing techniques to achieve these criteria, and provides extensive empirical studies on standard datasets to show their performance relative to existing methods in the causal fairness literature.\n\nWeaknesses:\n(i) The conditions for Theorem 1 are not useful -- in particular, it defines the expectation over $P(S) P(A)$ i.e., assuming that $S$ is independent of $A$. But, as the authors discuss at length in the beginning of the paper, we are often worried that there is a causal effect of $S$ on $A$ in which case $S$ is not independent of $A$. So, how is this then a useful guarantee if it does not apply to the actual data-generating process/causal graph at hand? The same weakness also applies to their construction of the CF-fair prediction function and its guarantee in Theorem 2. \n\n(ii) Assuming for simplicity we care about squared-loss, the \"best'' FTU prediction function is simply $E[Y \\mid A]$ (i.e., the true conditional expectation of $Y$ given the unprotected attributes $A$). The \"best'' unconstrained prediction function would simply be $E[Y \\mid A, S]$, which we can think of as the ideal ML decision-maker to use the authors' language. \nFrom this perspective, we can think of both the FTU prediction function and the authors' procedure as post-processing the ideal ML decision-maker. The FTU prediction function applies iterated expectations $\\hat{Y}^{FTU}(a) = \\int E[Y \\mid A = a, S = s] P(s \\mid a) ds$, and weights the ML decision maker according to the conditional distribution $p(s \\mid a)$. In contrast, the ECO prediction function weights the ML decision maker according to the marginal distribution $p(s)$. \nSo these only differ by their choice of weights to apply to the ideal ML decision-maker. \n\nFurthermore, based on the proof of Theorem 1 and this intuition, it seems like if we actually computed the KL-divergence under the true joint distribution $P(A, S)$, we may then find that the FTU prediction function actually provides the best approximation to the ideal ML learner. Is that conjecture correct? If not, it would be value to discuss. If so, then it is not clear why we should use the authors' post-processing algorithm in the first place. Similarly, since the fairlearning algorithm also satisfies CF-fairness, does the analogous result hold for it?\n\n(iii) I found the authors' discussion of FTU to be unconvincing on pg. 7. In particular, the authors argue that FTU will satisfy to some other, undefined notion of fairness (i.e., \"undesirable consequences if an unprotected attribute correlates...\"). It is odd to criticize FTU for something that it doesn't claim to achieve (and something the authors don't even formally define).\n\n(iv) Does the abduction step referred to on pg. 8 refer to the sampling procedure in the construction of the CF fair predictor? If so, that should be mentioned. Otherwise, the abduction step is not defined in the paper, and I was confused by what the authors were referring to.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper develops methods to post-process a prediction function to achieve two causal fairness criteria: \"equal counterfactual opportunity'' and \"counterfactual fairness.''\n\nThe authors consider the following setting: let $s_i$ denote the protected attribute, $a_i$ denote the unprotected attributes, and $y_i$ denote the decision. For example, $s_i$ may be gender, $a_i$ may be test scores, and $y_i$ may be an admissions decision made by a committee. The authors assume that the setting is summarized by a causal model in which (i) both $s_i, a_i$ affect $y_i$; and (ii) $s_i$ may affect $a_i$ (Figure 2). Let $Y_i(s, a)$ denote the counterfactual/potential decision and $A(s)$ denote the counterfactual/potential characteristics. \n\nIn this setting, the authors define:\n* Equal counterfactual opportunity (ECO): the decision $Y$ satisfies ECO if $Y(s, a) | \\{S = s, A = a\\} \\sim Y(s', a) \\mid \\{S = s, A = a\\}$. In words, group membership has no direct causal effect on the decision among similarly qualified individuals.\n* Counterfactual fairness (CF): the decision $Y$ satisfies CF if $Y(s, A(s)) \\mid \\{S = s, A = a\\} \\sim Y(s', A(s')) \\mid \\{S = s, A = a\\}$. In words, group membership has no effect on the decision accounting for the possible causal effects it has on the other unprotected attributes, effectively \"correcting'' for historical disadvantages/disparities that may be reflected in the unprotected attributes.\n\nThe authors next propose post-processing techniques to post-process an existing prediction function to satisfy ECO and CF:\n* To satisfy ECO, the authors propose to construct a new prediction function that takes a weighted average of the existing prediction function across different values of the protected attribute at each $A = a$. The authors show that the resulting predictor $\\hat{Y}^{ECO}$ provides the best approximation (in the sense of minimizing the KL-divergence between the new predictions and the existing predictions) to the existing prediction function among all that satisfy ECO (Theorem 1).\n* To satisfy CF, the authors propose to construct a new prediction function that post-process an ECO-fair prediction function based on the sampling procedure described on pg. 7. The authors show that the resulting CF prediction function provides the best approximation (again in the KL-divergence minimizing sense) to the ECO-fair prediction function.\n\nThe authors illustrate their proposed methods in simulations, and applications to the adult income data, Compas data, and German credit data.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "See my discussion of the weaknesses above. \n\nMy main concerns are (i), (ii) above. These concerns could be addressed by either (i) showing that the results extend to the case in which we evaluate KL-divergence over the underlying joint distribution or (ii) explicitly discussing the limitations of this result per my comments above. \n\nAddressing the remaining weaknesses by adding some further discussion will strengthen the work.", "broader_impact_concerns": "I have no concerns about the ethical implications of this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: My recommendation is somewhere between "Accept with minor revision" and "Reject" (but encourage resubmission).  Based on the reviews, author responses, discussion, and my own reading of the revised manuscript, I am taking a chance and recommending **Accept with (*significant but let's call it*) minor revision**.  Please see my specific suggestions for improving the manuscript in the **Claims and Evidence** and **Audience** sections above.  Specifically, while in its current form I think Sections 1 - 3 of the paper lack the clarity on ideal world vs real world and FTU vs. the ECO decision-maker, and are thus of limited interest to the TMLR audience, I believe the authors can address this concern in "minor" revision.  

**Summary of reviewer discussion/recommendations**

Two of the reviewers recommended reject or leaning reject, while only one recommended "leaning accept".  My recommendation therefore goes against the general inclination of the three reviewers.  Quoting some concerns accompanying the recommendations:

"While I have no remaining concerns about the technical soundness of their results, I remain unconvinced of the paper's interest to TMLR readers. As the authors point out, the "reality of decision making" in the present world is that the protected feature S often affects the unprotected features A, which is part of what gives rise to unfairness. This is a large part of the motivation for this paper, and for many algorithmic fairness methods in general. However, Theorems 1 and 2 provide guarantees about their methods in a world in which S and A are independent, which the authors describe as an "ideal" decision-making process. Furthermore, the authors acknowledge that a fairness-through-unawareness predictor, which satisfies their ECO-fairness criterion, is optimal in the actual world."

"Although the authors addressed all of my direct concerns, I am very concerned by [...] Theorem 1's assumption of independence between A and S, which seems contradictory of the problem setup. Theorem 1's flaw undermines the entire premise of the paper.

The authors responded to this concern by stating that they're explicitly assuming an idealized distribution where A and S are independent. I am unconvinced based on their reasoning and their proof. A satisfactory proof would include the introduction of this idealized world distribution and a separate proof analysis that the reasoning about the idealized world distribution would result in the optimal reasoning for the existing world distribution."
Decision: Accept (poster)