Title: Causal Fairness for Outcome Control
Venue: NeurIPS 2023
Accepted: yes
Keywords: Fair Machine Learning

Causal Inference

Decision-Making
Abstract: As society transitions towards an AI-based decision-making infrastructure, an ever-increasing number of decisions once under control of humans are now delegated to automated systems. Even though such developments make various parts of society more efficient, a large body of evidence suggests that a great deal of care needs to be taken to make such automated decision-making systems fair and equitable, namely, taking into account sensitive attributes such as gender, race, and religion. In this paper, we study a specific decision-making task called outcome control in which an automated system aims to optimize an outcome variable $Y$ while being fair and equitable. The interest in such a setting ranges from interventions related to criminal justice and welfare, all the way to clinical decision-making and public health. In this paper, we first analyze through causal lenses the notion of benefit, which captures how much a specific individual would benefit from a positive decision, counterfactually speaking, when contrasted with an alternative, negative one. We introduce the notion of benefit fairness, which can be seen as the minimal fairness requirement in decision-making, and develop an algorithm for satisfying it. We then note that the benefit itself may be influenced by the protected attribute, and propose causal tools which can be used to analyze this. Finally, if some of the variations of the protected attribute in the benefit are considered as discriminatory, the notion of benefit fairness may need to be strengthened, which leads us to articulating a notion of causal benefit fairness. Using this notion, we develop a new optimization procedure capable of maximizing $Y$ while ascertaining causal fairness in the decision process.

Causal Fairness for Outcome Control

Abstract

As society transitions towards an AI-based decision-making infrastructure, an everincreasing number of decisions once under control of humans are now delegated to automated systems. Even though such developments make various parts of society more efficient, a large body of evidence suggests that a great deal of care needs to be taken to make such automated decision-making systems fair and equitable, namely, taking into account sensitive attributes such as gender, race, and religion. In this paper, we study a specific decision-making task called outcome control in which an automated system aims to optimize an outcome variable Y while being fair and equitable. The interest in such a setting ranges from interventions related to criminal justice and welfare, all the way to clinical decision-making and public health. In this paper, we first analyze through causal lenses the notion of benefit , which captures how much a specific individual would benefit from a positive decision, counterfactually speaking, when contrasted with an alternative, negative one. We introduce the notion of benefit fairness, which can be seen as the minimal fairness requirement in decision-making, and develop an algorithm for satisfying it. We then note that the benefit itself may be influenced by the protected attribute, and propose causal tools which can be used to analyze this. Finally, if some of the variations of the protected attribute in the benefit are considered as discriminatory, the notion of benefit fairness may need to be strengthened, which leads us to articulating a notion of causal benefit fairness. Using this notion, we develop a new optimization procedure capable of maximizing Y while ascertaining causal fairness in the decision process.

1 Introduction

Decision-making systems based on artificial intelligence and machine learning are being increasingly deployed in real-world settings where they have life-changing consequences on individuals and on society more broadly, including hiring decisions, university admissions, law enforcement, credit lending, health care access, and finance [19, 26, 6]. Issues of unfairness and discrimination are pervasive in those settings when decisions are being made by humans, and remain (or are potentially amplified) when decisions are made using machines with little transparency or accountability. Examples include reports on such biases in decision support systems for sentencing [1], face-detection [7], online advertising [37, 10], and authentication [35]. A large part of the underlying issue is that AI systems designed to make decisions are trained with data that contains various historical biases and past discriminatory decisions against certain protected groups, and such systems may potentially lead to an even more discriminatory process, unless they have a degree of fairness and transparency.

In this paper, we focus on the specific task of outcome control , characterized by a decision D which precedes the outcome of interest Y . The setting of outcome control appears across a broad range of applications, from clinical decision-making [13] and public health [15], to criminal justice [23] and various welfare interventions [9]. In general, outcome control will cover settings in which an

institution (such as a hospital or a social service) may attempt to maximize an outcome (such as survival or well-being) using a known control (such as surgery or a welfare intervention) 1 . Importantly, due to historical biases, certain demographic groups may differ in their distribution of covariates and therefore their benefit from treatment (e.g., from surgery or intervention) may be lower than expected, leading to a lower allocation of overall resources. We next discuss two lines of related literature.

Firstly, a large body of literature in reinforcement learning [36, 38] and policy learning [11, 33, 21, 18, 2] analyzes the task of optimal decision-making. Often, these works consider the conditional average treatment effect (CATE) that measures how much probabilistic gain there is from a positive versus a negative decision for a specific group of individuals when experimental data is available. Subsequent policy decisions are then based on the CATE, a quantity that will be important in our approach as well. The focus of this literature is often on developing efficient procedures with desirable statistical properties, and issues of fairness have not traditionally been explored in this context.

On the other hand, there is also a growing literature in fair machine learning, that includes various different settings. One can distinguish three specific and different tasks, namely (1) bias detection and quantification for currently deployed policies; (2) construction of fair predictions of an outcome; (3) construction of fair decision-making policies. Most of the work in fair ML falls under tasks (1) and (2), whereas our setting of outcome control falls under (3). Our work also falls under the growing literature that explores fairness through a causal lens [22, 20, 28, 44, 43, 41, 8, 32].

For concreteness, consider the causal diagram in Fig. 1 that represents the setting of outcome control, with X the protected attribute, Z a possibly multidimensional set of confounders, W a set of mediators. Decision D is based on the variables X,Z , and W , and the outcome Y depends on all other variables in the model. In this setting, we also assume that the decision-maker is operating under budget constraints.

Previous work introduces a fairness definition that conditions on the potential outcomes of the decision D , written Y d 0 , Y d 1 , and ensures that the decision D is independent of the protected attribute X for any fixed value of Y d 0 , Y d 1 [14]. Another related work, in a slightly

Z

X

W

Y

D

different setting of risk assessment [9], proposes conditioning on the potential outcome under a negative decision, Y d 0 , and focuses on equalizing counterfactual error rates. Finally, the work [25] is also related to our setting, and proposes that fairness should be assessed with respect to three causal effects of interest: (i) total effect X → Y ; (ii) total effect D → Y ; and (iii) total effect X → D . Our framework, however, integrates considerations of the above effects more coherently, and brings forward a new definition of fairness based on first principles.

However, the causal approaches mentioned above take a different perspective from the policy learning literature, in which policies are built based on the CATE of the decision D , written ❊ [ Y d 1 -Y d 0 | x, z, w ] , which we will refer to as benefit 2 . Focusing exclusively on the benefit, though, will provide no fairness guarantees apriori. In particular, as can be seen from Fig. 1, the protected attribute X may influence the effect of D on Y in three very different ways: (i) along the direct pathway X → Y ; (ii) along the indirect pathway X → W → Y ; (iii) along the spurious pathway X glyph[arrowaxisleft]glyph[axisshort]glyph[axisshort]glyph[arrowaxisright] Z → Y . Often, the decision-maker may view these causal effects differently, and may consider only some of them as discriminatory. Currently, no approach in the literature allows for a principled way of detecting and removing discrimination based on the notion of benefit, while accounting for different underlying causal mechanisms that may lead to disparities.

In light of the above, the goal of this paper is to analyze the foundations of outcome control from a causal perspective of the decision-maker. Specifically, we develop a causal-based decision-making framework for modeling fairness with the following contributions:

(i) We introduce Benefit Fairness (BF, Def. 2) to ensure that at equal levels of the benefit, the protected attribute X is independent of the decision D . We then develop an algorithm for achieving BF (Alg. 1) and prove optimality guarantees (Thm. 2),

(ii) We develop Alg. 2 that determines which causal mechanisms from X to the benefit (direct, indirect, spurious) explain the difference in the benefit between groups. The decision-maker can then decide which causal pathways are considered as discriminatory.

(iii) We define the notion of causal benefit fairness (CBF, Def. 3), which models discriminatory pathways in a fine-grained way. We further develop an algorithm (Alg. 3) that ensures the removal of such discriminatory effects, and show some theoretical guarantees (Thm. 4).

1.1 Preliminaries

We use the language of structural causal models (SCMs) as our basic semantical framework [30]. A structural causal model (SCM) is a tuple M := ⟨ V, U, F , P ( u ) ⟩ , where V , U are sets of endogenous (observable) and exogenous (latent) variables respectively, F is a set of functions f V i , one for each V i ∈ V , where V i ← f V i (pa( V i ) , U V i ) for some pa( V i ) ⊆ V and U V i ⊆ U . P ( u ) is a strictly positive probability measure over U . Each SCM M is associated to a causal diagram G over the node set V , where V i → V j if V i is an argument of f V j , and V i glyph[arrowaxisleft]glyph[axisshort]glyph[axisshort]glyph[arrowaxisright] V j if the corresponding U V i , U V j are not independent. Throughout, our discussion will be based on the specific causal diagram known as the standard fairness model (SFM, see Fig. 1). Further, an instantiation of the exogenous variables U = u is called a unit . By Y x ( u ) we denote the potential response of Y when setting X = x for the unit u , which is the solution for Y ( u ) to the set of equations obtained by evaluating the unit u in the submodel M x , in which all equations in F associated with X are replaced by X = x .

2 Foundations of Outcome Control

In the setting of outcome control, we are interested in the following decision-making task:

Definition 1 (Decision-Making Optimization) . Let M be an SCM compatible with the SFM. We define the optimal decision problem as finding the (possibly stochastic) solution to the following optimization problem given a fixed budget b :

The budget b constraint is relevant for scenarios when resources are scarce, in which not all patients possibly requiring treatment can be given treatment. In such a setting, the goal is to treat patients who are most likely to benefit from the treatment, as formalized later in the text. We next discuss two different perspectives on solving the above problem. First, we discuss the problem solution under perfect knowledge, assuming that the underlying SCM and the unobserved variables are available to us (we call this the oracle's perspective). Then, we move on to solving the problem from the point of view of the decision-maker, who only has access to the observed variables in the model and a dataset generated from the true SCM.

2.1 Oracle's Perspective

The following example, which will be used throughout the paper, is accompanied by a vignette that performs inference using finite sample data for the different computations described in the sequel. We introduce the example by illustrating the intuition of outcome control through the perspective of an all-knowing oracle:

Example (Cancer Surgery - continued) . A clinical team has access to information about the sex of cancer patients ( X = x 0 male, X = x 1 female) and their degree of illness severity determined from tissue biopsy ( W ∈ [0 , 1] ). They wish to optimize the 2-year survival of each patient ( Y ), and the decision D = 1 indicates whether to perform surgery. The following SCM M ∗ describes the data-generating mechanisms (unknown to the team):

(a) Units in the oracle example.

X

W

Y

D

(b) Causal diagram for the running example.

where the f D mechanism is constructed by the team.

The clinical team has access to an oracle that is capable of predicting the future perfectly. In particular, the oracle tells the team how each individual would respond to surgery. That is, for each unit U = u (of the 500 units), the oracle returns the values of

Having access to this information, the clinicians quickly realize how to use their resources. In particular, they notice that for units for whom ( Y d 0 ( u ) , Y d 1 ( u )) equals (0 , 0) or (1 , 1) , there is no effect of surgery, since they will (or will not) survive regardless of the decision. They also notice that surgery is harmful for individuals for whom ( Y d 0 ( u ) , Y d 1 ( u )) = (1 , 0) . These individuals would not survive if given surgery, but would survive otherwise. Therefore, they ultimately decide to treat 100 individuals who satisfy

since these individuals are precisely those whose death can be prevented by surgery. They learn there are 100 males and 100 females in the (0 , 1) -group, and thus, to be fair with respect to sex, they decide to treat 50 males and 50 females. □

The space of units corresponding to the above example is represented in Fig. 2a. The groups described by different values of Y d 0 ( u ) , Y d 1 ( u ) in the example are known as canonical types [3] or principal strata [12]. Two groups cannot be influenced by the treatment decision (which will be called 'Safe' and 'Doomed', see Fig. 2a). The third group represents those who are harmed by treatment (called 'Harmed'). In fact, the decision to perform surgery for this subset of individuals is harmful. Finally, the last group represents exactly those for whom the surgery is life-saving, which is the main goal of the clinicians (this group is called 'Helped').

This example illustrates how, in presence of perfect knowledge, the team can allocate resources efficiently. In particular, the consideration of fairness comes into play when deciding which of the individuals corresponding to the (0 , 1) principal stratum will be treated. Since the number of males and females in this group is equal, the team decides that half of those treated should be female. The approach described above can be seen as appealing in many applications unrelated to to the medical setting, and motivated the definition of principal fairness [14]. As we show next, however, this viewpoint is often incompatible with the decision-maker's perspective.

2.2 Decision-Maker's Perspective

We next discuss the policy construction from the perspective of the decision-maker:

Example (Cancer Surgery - continued) . The team of clinicians constructs the causal diagram associated with the decision-making process, shown in Fig. 2b. Using data from their electronic health records (EHR), they estimate the benefit of the treatment based on x, w :

❊ ❊ In words, at each level of illness severity W = w , the proportion of patients who benefit from the surgery is the same, regardless of sex. In light of this information, the clinicians decide to construct

(a)

(q)

U-spaceforOracle

U-spaceforDecision-Maker

II

II

0.5

×

×

Sex

M

0.0

WO

0

0

II

II

0.5

×

×

0.0

0.00

0.25

0.50

0.75

1.00

uy(resilience)

Canonical Type:·Doomed·Helped·Safe

Benefit

the decision policy f D such that f D ( W ) = 1 ( W > 1 2 ) . In words, if a patient's illness severity W is above 1 2 , the patient will receive treatment.

After implementing the policy and waiting for the 2-year follow-up period, clinicians estimate the probabilities of treatment within the stratum of those helped by surgery, and compute that

indicating that the allocation of the decision is not independent of sex. That is, within the group of those who are helped by surgery, males are more likely to be selected for treatment than females. □

The decision-making policy introduced by the clinicians, somewhat counter-intuitively, does not allocate the treatment equally within the principal stratum of those who are helped, even though at each level of illness severity, the proportion of patients who benefit from the treatment is equal between the sexes. What is the issue at hand here?

To answer this question, the perspective under perfect knowledge is shown in Fig. 3a. In particular, on the horizontal axis the noise variable u y is available, which summarizes the patients' unobserved resilience. Together with the value of illness severity (on the vertical axis) and the knowledge of the structural causal model, we can perfectly pick apart the different groups (i.e., principal strata) according to their potential outcomes (groups are indicated by color). In this case, it is clear that our policy should treat patients in the green area since those are the ones who benefit from treatment.

In Fig. 3b, however, we see the perspective of the decision-makers under imperfect knowledge. Firstly, the decision-makers have no knowledge about the values on the horizontal axis, since this represents variables that are outside their model. From their point of view, the key quantity of interest is the conditional average treatment effect (CATE), which we call benefit in this context 3 , defined as

Importantly, in our setting, the benefit ∆ can be uniquely computed (identified) from observational data, implying that we may use the notion of benefit for practical data analyses:

Proposition 1 (Benefit Identifiability) . Let M be an SCM compatible with the SFM in Fig. 1. Then the benefit ∆( x, z, w ) is identifiable from observational data and equals:

for any choice of x, z, w .

Algorithm 1 Decision-Making with Benefit Fairness

Inputs: Distribution P ( V ) , Budget b

1: Compute ∆( x, z, w ) = ❊ [ Y d 1 -Y d 0 | x, z, w ] for all ( x, z, w ) . 2: If P (∆ > 0) ≤ b , set D ∗ = 1 (∆( x, z, w ) > 0) and RETURN ( D ∗ ) . 3: Find δ b > 0 such that

5: Construct the policy D ∗ such that:

The benefit is simply the increase in survival associated with treatment. After computing the benefit (shown in Eq. 11), the decision-makers visualize the male and female groups (lighter color indicates a larger increase in survival associated with surgery) according to it. It is visible from the figure that the estimated benefit from surgery is higher for the x 0 group than for x 1 . Therefore, to the best of their knowledge, the decision-makers decide to treat more patients from the x 0 group.

The example illustrates why the oracle's perspective may be misleading for the decision-makers. The clinicians can never determine exactly which patients belong to the group

that is, who benefits from the treatment. Instead, they have to rely on illness severity ( W ) as a proxy for treatment benefit ( ∆ ). In other words, our understanding of treatment benefit will always be probabilistic, and we need to account for this when considering the decision-maker's task. A further discussion on the relation to principal fairness [14] is given in Appendix A.

2.3 Benefit Fairness

To remedy the above issue, we propose an alternative definition, which takes the viewpoint of the decision-maker:

Definition 2 (Benefit Fairness) . Let ∆ denote the benefit of treatment in Eq. 14 and let δ be a fixed value of ∆ . We say that the pair ( Y, D ) satisfies benefit fairness (BF , for short) if

The above definition pertains to the population-level setting, but without oracle knowledge. In this setting, the objective function in Eq. 1 is expected to be lower than in the oracle case. Further, in finite samples, the constraint from Def. 2 is expected to hold approximately. Notice that BF takes the perspective of the decision-maker who only has access to the unit's attributes ( x, z, w ) , as opposed to the exogenous U = u 4 . In particular, the benefit ∆( x, z, w ) is estimable from the data, i.e., attainable by the decision-maker. BF then requires that at each level of the benefit, ∆( x, z, w ) = δ , the rate of the decision does not depend on the protected attribute. The benefit ∆ is closely related to the canonical types discussed earlier, namely

The values of c, d in Eq. 21 are covariate-specific, and indicate the proportions of patients helped and harmed by the treatment, respectively, among all patients coinciding with covariate values x, z, w (i.e.,

4: Otherwise, define

Algorithm 2 Benefit Fairness Causal Explanation

Inputs: Distribution P ( V ) , Benefit ∆( x, z, w ) , Decision policy D

1: Compute the causal decomposition of the resource allocation disparity into its direct, indirect, and spurious contributions [44, 31]

2: Compare the distributions P (∆ | x 1 ) and P (∆ | x 0 ) .

3: Compute the causal decomposition of the benefit disparity

❊ (∆ | x 1 ) -❊ (∆ | x 0 ) = DE + IE + SE . (23) 4: Compute the counterfactual distribution P (∆ C | x ) for specific interventions C that remove the direct, indirect, or total effect of X on the benefit ∆ .

all u ∈ U s.t. ( X,Z,W )( u ) = ( x, z, w ) ). The proof of the claim in Eq. 21 is given in Appendix B. Using this connection of canonical types and the notion of benefit, we can formulate a solution to the problem in Def. 1, given in Alg. 1. In particular, Alg. 1 takes as input the observational distribution P ( V ) , but its adaptation to inference from finite samples follows easily. In Step 2, we check whether we are operating under resource scarcity, and if not, the optimal policy simply treats everyone who stands to benefit from the treatment. Otherwise, we find the δ b > 0 which uses the entire budget (Step 3) and separate the interior I of those with the highest benefit (all of whom are treated), and the boundary B (those who are to be randomized) in Step 4. The budget remaining to be spent on the boundary is b -P ( I ) , and thus individuals on the boundary are treated with probability b -P ( I ) P ( B ) . Importantly, male and female groups are treated separately in this random selection process (Eq. 20), reflecting BF. The BF criterion can be seen as a minimal fairness requirement in decision-making, and is often aligned with maximizing utility, as seen from the following theorem:

Theorem 2 (Alg. 1 Optimality) . Among all feasible policies D for the optimization problem in Eqs. 1-2, the result of Alg. 1 is optimal and satisfies benefit fairness.

A key extension we discuss next relates to the cases in which the benefit itself may be deemed as discriminatory towards a protected group.

3 Fairness of the Benefit

As discussed above, benefit fairness guarantees that at each fixed level of the benefit ∆ = δ , the protected attribute plays no role in the treatment assignment. However, benefit fairness does not guarantee that treatment probability is equal between groups, i.e., that P ( d | x 1 ) = P ( d | x 0 ) . In fact, benefit fairness implies equal treatment allocation in cases where the distribution of the benefit ∆ is equal across groups (shown formally in Appendix F), which may not always be the case:

Example (Cancer Surgery - continued) . After applying benefit fairness and implementing the optimal policy D ∗ = 1 ( W > 1 2 ) , the clinicians compute that P ( d | x 1 ) -P ( d | x 0 ) = -50% , that is, females are 50% less likely to be treated than males. □

In our example benefit fairness results in a disparity in resource allocation. Whenever this is the case, it implies that the benefit ∆ differs between groups. In Alg. 2 we describe a formal procedure that helps the decision-maker to obtain a causal understanding of why that is, i.e., which underlying causal mechanisms (direct, indirect, spurious) lead to the difference in the benefit. In Appendix G we discuss in detail how the terms appearing in the decompositions in Eqs. 22 and 23 are defined. The key subtlety here is how to define counterfactuals with respect to the random variable ∆ . In the appendix we define this formally and show that the variable ∆ can be considered as an auxillary variable in the structural causal model. Furthermore, we show that the notions of direct, indirect, and spurious effects are identifiable under the SFM in Fig. 1 and provide the identification expressions for them, allowing the data analyst to compute the decompositions in Alg. 2 in practice (see Appendix G). We next ground the idea behind Alg. 2 in our example:

(a) Density of benefit

∆

by group.

(b) Density of counterfactual benefit

∆

C

.

2.0

Distribution Density

1.5

1.0

0.5

F0°0

0.0

0.1

0.2

0.3

Benefit

Distribution

|xo (Male)

△|x1 (Female)

2.0

Distribution Density

1.5

1.0

0.5

0°0

0.0

0.1

Benefit

0.2

0.3

Distribution

△|xo

x, Wx0lX1

Example (Decomposing the disparity) . Following Alg. 2, the clinicians first decompose the observed disparities into their direct, indirect, and spurious components:

showing that the difference between groups is entirely explained by the levels of illness severity, that is, male patients are on average more severely ill than female patients (see Fig. 4a). Direct and spurious effects, in this example, do not explain the difference in benefit between the groups.

Based on these findings, the clinicians realize that the main driver of the disparity in the benefit ∆ is the indirect effect. Thus, they decide to compute the distribution of the benefit P (∆ x 1 ,W x 0 | x 1 ) , which corresponds to the distribution of the benefit had X been equal to x 0 along the indirect effect. The comparison of this distribution, with the distribution P (∆ | x 0 ) is shown in Fig. 4b, indicating that the two distributions are in fact equal. □

In the above example, the difference between groups is driven by the indirect effect, although generally, the situation may be more complex, with a combination of effects driving the disparity. Still, the tools of Alg. 2 equip the reader for analyzing such more complex cases. The key takeaway here is that the first step in analyzing a disparity in treatment allocation is to obtain a causal understanding of why the benefit differs between groups. Based on this understanding, the decision-maker may decide that the benefit ∆ is unfair, which is what we discuss next.

3.1 Controlling the Gap

A causal approach. The first approach for controlling the gap in resource allocation takes a counterfactual perspective. We first define what it means for the benefit ∆ to be causally fair:

Definition 3 (Causal Benefit Fairness) . Suppose C = ( C 0 , C 1 ) describes a causal pathway from X to Y which is deemed unfair, with C 0 , C 1 representing possibly counterfactual interventions. The pair ( Y, D ) satisfies counterfactual benefit fairness (CBF) if

To account for discrimination along a specific causal pathway (after using Alg. 2), the decision-maker needs to compute an adjusted version of the benefit ∆ , such that the protected attribute has no effect along the intended causal pathway C . For instance, C = ( { x 0 } , { x 1 } ) describes the total causal effect, whereas C = ( { x 0 } , { x 1 , W x 0 } ) describes the direct effect. In words, CBF requires that treatment benefit ∆ should not depend on the effect of X on Y along the causal pathway C , and this condition is covariate-specific, i.e., holds for any choice of covariates x, z, w . Additionally, the decision policy D should satisfy BF, meaning that at each degree of benefit ∆ = δ , the protected attribute plays no

role in deciding whether the individual is treated or not. This statement is for fixed value of ∆ = δ , and possibly considers individuals with different values of x, z, w . CBF can be satisfied using Alg. 3. In Step 1, the factual benefit values ∆ , together with the adjusted, counterfactual benefit values ∆ C (that satisfy Def. 3) are computed. Then, δ CF is chosen to match the budget b , and all patients with a counterfactual benefit above δ CF are treated 5 , as demonstrated in the following example:

Example (Cancer Surgery - Counterfactual Approach) . The clinicians realize that the difference in illness severity comes from the fact that female patients are subject to regular screening tests, and are therefore diagnosed earlier. The clinicians want to compute the adjusted benefit, by computing the counterfactual values of the benefit ∆ x 1 ,W x 0 ( u ) for all u such that X ( u ) = x 1 . For the computation, they assume that the relative order of the illness severity for females in the counterfactual world would have stayed the same (which holds true in the underlying SCM). Therefore, they compute that

for each unit u with X ( u ) = x 1 . After applying Alg. 1 with the counterfactual benefit values ∆ C , the resulting policy D CF = 1 (∆ C > 1 4 ) has a resource allocation disparity of 0 . □

The above example illustrates the core of the causal counterfactual approach to discrimination removal. BF was not appropriate in itself, since the clinicians are aware that the benefit of the treatment depends on sex in a way they deemed unfair. Therefore, to solve the problem, they first remove the undesired effect from the benefit ∆ , by computing the counterfactual benefit ∆ C . After this, they apply Alg. 3 with the counterfactual method (CF) to construct a fair decision policy. Notably, the causal approach to reducing the disparity relies on the counterfactual values of the benefit ∆ C ( u ) , as opposed to the factual benefit values ∆( u ) . As it turns out, measuring (or removing) the covariate-specific direct effect of X on the benefit ∆ is relatively simple in general, while the indirect effect may be more challenging to handle:

Remark 3 (Covariate-Specific Direct Effect of X on Benefit ∆ is Computable) . Under the assumptions of the SFM, the potential outcome ∆ x 1 ,W x 0 ( u ) is identifiable for any unit u with X ( u ) = x 0 for which the attributes Z ( u ) = z, W ( u ) = w are observed. However, the same is not true for the indirect effect. While the counterfactual distribution of the benefit when the indirect effect is manipulated, written P (∆ x 0 ,W x 1 | x 0 ) , is identifiable under the SFM, the covariate-level values ∆ x 0 ,W x 1 ( x 0 , z, w ) are not identifiable without further assumptions.

A utilitarian/factual approach. An alternative, utilitarian (or factual) approach to reduce the disparity in resource allocation uses the factual benefit ∆( u ) , instead of the counterfactual benefit ∆ C ( u ) used in the causal approach. This approach is also described in Alg. 3, with the utilitarian (UT) method. Firstly, in Step 5, the counterfactual values ∆ C are used to compute the disparity that would arise from the optimal policy in the hypothetical, counterfactual world:

The idea then is to introduce different thresholds δ ( x 0 ) , δ ( x 1 ) for x 0 and x 1 groups, such that they introduce a disparity of at most M . In Step 6 we check whether the optimal policy introduces a disparity bounded by M . If the disparity is larger than M by an ϵ , in Step 7 we determine how much slack the disadvantaged group requires, by finding thresholds δ ( x 0 ) , δ ( x 1 ) that either treat everyone in the disadvantaged group, or achieve a disparity bounded by M . The counterfactual (CF) approach focused on the counterfactual benefit values ∆ C and used a single threshold. The utilitarian (UT) approach focuses on the factual benefit values ∆ , but uses different thresholds within groups. However, the utilitarian approach uses the counterfactual values ∆ C to determine the maximum allowed disparity. Alternatively, this disparity can be pre-specified, as shown in the following example:

Example (Cancer Surgery - Utilitarian Approach) . Due to regulatory purposes, clinicians decide that M = 20% is the maximum allowed disparity that can be introduced by the new policy D . Using Alg. 3, they construct D UT and find that for δ ( x 0 ) = 0 . 21 , δ ( x 1 ) = 0 . 12 ,

which yields P ( d UT ) ≈ 50% , and P ( d UT | x 1 ) -P ( d UT | x 0 ) ≈ 20% , which is in line with the hospital resources and the maximum disparity allowed by the regulators. □

Algorithm 3 Causal Discrimination Removal for Outcome Control

Inputs: Distribution P ( V ) , Budget b , Intervention C , Max. Disparity M , Method ∈ { CF,UT }

1: Compute ∆( x, z, w ) , ∆ C ( x, z, w ) for all ( x, z, w ) .

2: If P (∆ > 0) ≤ b , set D = 1 (∆( x, z, w ) > 0) and RETURN ( D ) .

3: Find δ CF > 0 such that

4: If Method is CF, set D CF = 1 (∆ C ( x, z, w ) ≥ δ CF ) and RETURN ( D CF ) .

5: If M not pre-specified, compute the disparity

6: Find δ UT such that P (∆ ≥ δ UT ) = b . If

set D UT = 1 (∆( x, z, w ) ≥ δ UT ) and RETURN ( D UT ) . 7: Otherwise, suppose w.l.o.g. that P (∆ ≥ δ b | x 1 ) -P (∆ ≥ δ b | x 0 ) = M + ϵ for ϵ > 0 . Define l := P ( x 1 ) P ( x 0 ) , and let δ ( x 0 ) lb be such that

Set δ ( x 0 ) = max( δ ( x 0 ) lb , 0) , and δ ( x 1 ) s.t. P (∆ ≥ δ ( x 1 ) | x 1 ) = b P ( x 1 ) -1 l P (∆ ≥ δ ( x 0 ) | x 0 ) . 8: Construct and RETURN the policy D UT :

Finally, we describe the theoretical guarantees for the methods in Alg. 3 (proof given in Appendix C):

Theorem 4 (Alg. 3 Guarantees) . The policy D CF is optimal among all policies with a budget ≤ b that in the counterfactual world described by intervention C . The policy D UT is optimal among all policies with a budget ≤ b that either introduce a bounded disparity in resource allocation | P ( d | x 1 ) -P ( d | x 0 ) | ≤ M or treat everyone with a positive benefit in the disadvantaged group.

We remark that policies D CF and D UT do not necessarily treat the same individuals in general. In Appendix D, we discuss a formal condition called counterfactual crossing that ensures that D CF and D UT treat the same individuals, therefore explaining when the causal and utilitarian approaches are equivalent [29]. In Appendix E we provide an additional application of our outcome control framework to the problem of allocating respirators [5] in intensive care units (ICUs), using the MIMIC-IV dataset [17].

4 Conclusion

In this paper we developed causal tools for understanding fairness in the task of outcome control. We introduced the notion of benefit fairness (Def. 2), and developed a procedure for achieving it (Alg. 1). Further, we develop a procedure for determining which causal mechanisms (direct, indirect, spurious) explain the difference in benefit between groups (Alg. 2). Finally, we developed two approaches that allow the removal of discrimination from the decision process along undesired causal pathways (Alg. 3). The proposed framework was demonstrated through a hypothetical cancer surgery example (see vignette) and a real-world respirator allocation example (see Appendix E). We leave for future work the extensions of the methods to the setting of continuous decisions D , and the setting of performing decision-making under uncertainty or imperfect causal knowledge.

References

J. Angwin, J. Larson, S. Mattu, and L. Kirchner. Machine bias: There's software used across the country to predict future criminals. and it's biased against blacks. ProPublica , 5 2016. URL https://www.propublica.org/article/ machine-bias-risk-assessments-in-criminal-sentencing .

S. Athey and S. Wager. Policy learning with observational data. Econometrica , 89(1):133-161, 2021.

A. Balke and J. Pearl. Counterfactual probabilities: Computational methods, bounds and applications. In Uncertainty Proceedings 1994 , pages 46-54. Elsevier, 1994.

N. Bennett, D. Pleˇ cko, I.-F. Ukor, N. Meinshausen, and P. Bühlmann. ricu: R's interface to intensive care data. arXiv preprint arXiv:2108.00796 , 2021.

E. L. D. Biddison, R. Faden, H. S. Gwon, D. P. Mareiniss, A. C. Regenberg, M. Schoch-Spana, J. Schwartz, and E. S. Toner. Too many patients. . . a framework to guide statewide allocation of scarce mechanical ventilation during disasters. Chest , 155(4):848-854, 2019.

T. Brennan, W. Dieterich, and B. Ehret. Evaluating the predictive validity of the compas risk and needs assessment system. Criminal Justice and Behavior , 36(1):21-40, 2009.

J. Buolamwini and T. Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification. In S. A. Friedler and C. Wilson, editors, Proceedings of the 1st Conference on Fairness, Accountability and Transparency , volume 81 of Proceedings of Machine Learning Research , pages 77-91, NY, USA, 2018.

S. Chiappa. Path-specific counterfactual fairness. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 33, pages 7801-7808, 2019.

A. Coston, A. Mishler, E. H. Kennedy, and A. Chouldechova. Counterfactual risk assessments, evaluation, and fairness. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency , pages 582-593, 2020.

A. Datta, M. C. Tschantz, and A. Datta. Automated experiments on ad privacy settings: A tale of opacity, choice, and discrimination. Proceedings on Privacy Enhancing Technologies , 2015 (1):92-112, Apr. 2015. doi: 10.1515/popets-2015-0007.

M. Dudík, J. Langford, and L. Li. Doubly robust policy evaluation and learning. In Proceedings of the 28th International Conference on International Conference on Machine Learning , ICML'11, page 1097-1104, Madison, WI, USA, 2011. Omnipress. ISBN 9781450306195.

C. E. Frangakis and D. B. Rubin. Principal stratification in causal inference. Biometrics , 58(1): 21-29, 2002.

M. A. Hamburg and F. S. Collins. The path to personalized medicine. New England Journal of Medicine , 363(4):301-304, 2010.

K. Imai and Z. Jiang. Principal fairness for human and algorithmic decision-making. arXiv preprint arXiv:2005.10400 , 2020.

T. R. Insel. Translating scientific opportunity into public health impact: a strategic plan for research on mental illness. Archives of general psychiatry , 66(2):128-133, 2009.

A. Johnson, L. Bulgarelli, T. Pollard, S. Horng, L. A. Celi, and R. Mark. Mimic-iv. PhysioNet. Available online at: https://physionet. org/content/mimiciv/1.0/(accessed August 23, 2021) , 2020.

A. E. Johnson, L. Bulgarelli, L. Shen, A. Gayles, A. Shammout, S. Horng, T. J. Pollard, B. Moody, B. Gow, L.-w. H. Lehman, et al. Mimic-iv, a freely accessible electronic health record dataset. Scientific data , 10(1):1, 2023.

N. Kallus. Balanced policy evaluation and learning. Advances in neural information processing systems , 31, 2018.

A. E. Khandani, A. J. Kim, and A. W. Lo. Consumer credit-risk models via machine-learning algorithms. Journal of Banking & Finance , 34(11):2767-2787, 2010.

N. Kilbertus, M. Rojas-Carulla, G. Parascandolo, M. Hardt, D. Janzing, and B. Schölkopf. Avoiding discrimination through causal reasoning. arXiv preprint arXiv:1706.02744 , 2017.

T. Kitagawa and A. Tetenov. Who should be treated? empirical welfare maximization methods for treatment choice. Econometrica , 86(2):591-616, 2018.

M. J. Kusner, J. Loftus, C. Russell, and R. Silva. Counterfactual fairness. Advances in neural information processing systems , 30, 2017.

J. Larson, S. Mattu, L. Kirchner, and J. Angwin. How we analyzed the compas recidivism algorithm. ProPublica (5 2016) , 9, 2016.

M. K. Lee, A. Jain, H. J. Cha, S. Ojha, and D. Kusbit. Procedural justice in algorithmic fairness: Leveraging transparency and outcome control for fair algorithmic mediation. Proceedings of the ACM on Human-Computer Interaction , 3(CSCW):1-26, 2019.

D. Madras, E. Creager, T. Pitassi, and R. Zemel. Fairness through causal awareness: Learning causal latent-variable models for biased data. In Proceedings of the conference on fairness, accountability, and transparency , pages 349-358, 2019.

J. F. Mahoney and J. M. Mohen. Method and system for loan origination and underwriting, Oct. 23 2007. US Patent 7,287,008.

L. J. Modra, A. M. Higgins, V. S. Abeygunawardana, R. N. Vithanage, M. J. Bailey, and R. Bellomo. Sex differences in treatment of adult intensive care patients: a systematic review and meta-analysis. Critical Care Medicine , 50(6):913-923, 2022.

R. Nabi and I. Shpitser. Fair inference on outcomes. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 32, 2018.

H. Nilforoshan, J. D. Gaebler, R. Shroff, and S. Goel. Causal conceptions of fairness and their consequences. In International Conference on Machine Learning , pages 16848-16887. PMLR, 2022.

J. Pearl. Causality: Models, Reasoning, and Inference . Cambridge University Press, New York, 2000. 2nd edition, 2009.

D. Pleˇ cko and E. Bareinboim. Causal fairness analysis. arXiv preprint arXiv:2207.11385 , 2022. (To appear in Foundations and Trends in Machine Learning ).

D. Pleˇ cko and N. Meinshausen. Fair data adaptation with quantile preservation. Journal of Machine Learning Research , 21:242, 2020.

M. Qian and S. A. Murphy. Performance guarantees for individualized treatment rules. Annals of statistics , 39(2):1180, 2011.

J. Richens, R. Beard, and D. H. Thompson. Counterfactual harm. Advances in Neural Information Processing Systems , 35:36350-36365, 2022.

J. Sanburn. Facebook thinks some native american names are inauthentic. Time , Feb. 14 2015. URL http://time.com/3710203/facebook-native-american-names/ .

R. S. Sutton and A. G. Barto. Reinforcement learning: An introduction . MIT press, 1998.

L. Sweeney. Discrimination in online ad delivery. Technical Report 2208240, SSRN, Jan. 28 2013. URL http://dx.doi.org/10.2139/ssrn.2208240 .

C. Szepesvári. Algorithms for Reinforcement Learning . Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool Publishers, 2010.

J. L. Vincent, R. Moreno, J. Takala, S. Willatts, A. De Mendonça, H. Bruining, C. Reinhart, P. Suter, and L. G. Thijs. The sofa (sepsis-related organ failure assessment) score to describe organ dysfunction/failure: On behalf of the working group on sepsis-related problems of the european society of intensive care medicine (see contributors to the project in the appendix), 1996.

D. B. White and B. Lo. A framework for rationing ventilators and critical care beds during the covid-19 pandemic. Jama , 323(18):1773-1774, 2020.

Y. Wu, L. Zhang, X. Wu, and H. Tong. Pc-fairness: A unified framework for measuring causality-based fairness. Advances in neural information processing systems , 32, 2019.

H. Wunsch, A. D. Hill, N. Bosch, N. K. Adhikari, G. Rubenfeld, A. Walkey, B. L. Ferreyro, B. W. Tillmann, A. C. Amaral, D. C. Scales, et al. Comparison of 2 triage scoring guidelines for allocation of mechanical ventilators. JAMA network open , 3(12):e2029250-e2029250, 2020.

J. Zhang and E. Bareinboim. Equality of opportunity in classification: A causal approach. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31 , pages 3671-3681, Montreal, Canada, 2018. Curran Associates, Inc.

J. Zhang and E. Bareinboim. Fairness in decision-making-the causal explanation formula. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 32, 2018.

Acknowledgements This research was supported in part by the NSF, ONR, AFOSR, DoE, Amazon, JP Morgan, and The Alfred P. Sloan Foundation.

Supplementary Material for Causal Fairness for Outcome Control

The source code for reproducing all the experiments can be found in our code repository. Furthermore, the vignette accompanying the main text can be found here.

A Principal Fairness

We start with the definition of principal fairness:

Definition 4 (Principal Fairness [14]) . Let D be a decision that possibly affects the outcome Y . The pair ( Y, D ) is said to satisfy principal fairness if

for each principal stratum ( y d 0 , y d 1 ) , which can also be written as D ⊥ ⊥ X | Y d 0 , Y d 1 . Furthermore, define the principal fairness measure (PFM) as:

The above notion of principal fairness aims to capture the intuition described in oracle example in Sec. 2.1. However, unlike in the example, the definition needs to be evaluated under imperfect knowledge, when only the collected data is available 6 . An immediate cause for concern, in this context, is the joint appearance of the potential outcomes Y d 0 , Y d 1 in the definition of principal fairness. As is well-known in the literature, the joint distribution of the potential outcomes Y d 0 , Y d 1 is in general impossible to obtain, which leads to the lack of identifiability of the principal fairness criterion:

Proposition 5 (Principal Fairness is Not Identifiable) . The Principal Fairness (PF) criterion from Eq. 34 is not identifiable from observational or experimental data.

The implication of the proposition is that principal fairness, in general, cannot be evaluated, even if an unlimited amount of data was available. One way to see why PF is not identifiable is the following construction. Consider an SCM consisting of two binary variables D,Y ∈ { 0 , 1 } and the simple graph D → Y . Suppose that we observe P ( d ) = p d , and P ( y | d 1 ) = m 1 , P ( y | d 0 ) = m 0 for some constants p d , m 1 , m 0 (additionally assume m 0 ≤ m 1 w.l.o.g.). It is easy to show that these three values determine all of the observational and interventional distributions of the SCM. However, notice that for any λ ∈ [0 , 1 -m 1 ] the SCM given by

(1

-

D

)

1

(

U

Y

∈

[

m

1

, m

1

+

λ

])

,

satisfies P ( d ) = p d , P ( y | d 1 ) = m 1 , and P ( y | d 0 ) = m 0 , but the joint distribution P ( y d 0 = 0 , y d 1 = 1) = m 1 -m 0 + λ depends on the λ parameter and is therefore non-identifiable.

A.1 Monotonicity Assumption

To remedy the problem of non-identifiability of principal fairness, [14] proposes the monotonicity assumption:

Definition 5 (Monotonicity) . We say that an outcome Y satisfies monotonicity with respect to a decision D if

In words, monotonicity says that for every unit, the outcome with the positive decision ( D = 1 ) would not be worse than with the negative decision ( D = 0 ). We now demonstrate how monotonicity aids the identifiability of principal fairness.

Proposition 6. Under the monotonicity assumption (Eq. 39), the principal fairness criterion is identifiable under the Standard Fairness Model (SFM).

Proof. The main challenge in PF is to obtain the joint distribution P ( y d 0 , y d 1 ) , which is nonidentifiable in general. Under monotonicity, however, we have that

Therefore, it follows from monotonicity that

thereby identifying the joint distribution whenever the interventional distributions P ( y d 0 ) , P ( y d 1 ) are identifiable.

In the cancer surgery example, the monotonicity assumption would require that the patients have strictly better survival outcomes when the surgery is performed, compared to when it is not. Given the known risks of surgical procedures, the assumption may be rightfully challenged in such a setting. In the sequel, we argue that the assumption of monotonicity is not really necessary, and often does not help the decision-maker, even if it holds true. To fix this issue, in the main text we discuss a relaxation of the PF criterion that suffers from neither of the above two problems but still captures the essential intuition that motivated PF.

B Canonical Types & Bounds

Definition 6 (Canonical Types for Decision-Making) . Let Y be the outcome of interest, and D a binary decision. We then consider four canonical types of units:

(i) units u such that Y d 0 ( u ) = 1 , Y d 1 ( u ) = 1 , called safe,

(ii) units u such that Y d 0 ( u ) = 1 , Y d 1 ( u ) = 0 , called harmed,

(iii) units u such that Y d 0 ( u ) = 0 , Y d 1 ( u ) = 1 , called helped,

(iv) units u such that Y d 0 ( u ) = 0 , Y d 1 ( u ) = 0 , called doomed.

In decision-making, the goal is to treat as many units who are helped by the treatment, and as few who are harmed by it. As we demonstrate next, the potential outcomes Y d 0 ( u ) , Y d 1 ( u ) depend precisely on the canonical types described above.

Proposition 7 (Canonical Types Decomposition) . Let M be an SCM compatible with the SFM. Let D be a binary decision that possibly affects the outcome Y . Denote by ( s, d, c, u )( x, z, w ) the proportion of each of the canonical types from Def. 6, respectively, for a fixed set of covariates ( x, z, w ) . It then holds that

Proof. Notice that we can write:

Therefore, we have that

where the first line follows from the law of total probability, and the second by definition. Similarly, we have that

thereby completing the proof.

The proposition shows us that the degree of benefit ∆( x, z, w ) captures exactly the difference between the proportion of those helped by the treatment, versus those who are harmed by it. From the point of view of the decision-maker, this is very valuable information since higher ∆( x, z, w ) values indicate a higher utility of treating the group corresponding to covariates ( x, z, w ) . This insight can be used to prove Thm. 2, which states that the policy D ∗ obtained by Alg. 1 is optimal:

Proof. Note that the objective in Eq. 1 can be written as:

Eq. 55 follows from the law of total probability, and Eq. 56 from the consistency axiom. Now, note that Y d 0 , Y d 1 ⊥ ⊥ D | X,Z,W , from which it follows that

By noting that P ( D = 0 | x, z, w ) = 1 -P ( D = 1 | x, z, w ) , we can rewrite the objective as

Only the second term in Eq. 60 can be influenced by the decision-maker, and optimizing the term is subject to the budget constraint:

Such an optimization problem is a simple linear programming exercise, for which the policy D ∗ from Alg. 1 is a (possibly non-unique) optimal solution.

Finally, as the next proposition shows, the values of P ( y d 1 | x, z, w ) , P ( y d 0 | x, z, w ) can be used to bound the proportion of different canonical types:

Proposition 8 (Canonical Types Bounds and Tightness) . Let ( s, d, c, u )( x, z, w ) denote the proportion of each of the canonical types from Def. 6 for a fixed set of covariates ( x, z, w ) . Let m 1 ( x, z, w ) = P ( y d 1 | x, z, w ) and m 0 ( x, z, w ) = P ( y d 0 | x, z, w ) and suppose that m 1 ( x, z, w ) ≥ m 0 ( x, z, w ) . We then have that (dropping ( x, z, w ) from the notation):

In particular, the above bounds are tight, meaning that there exists an SCM M , compatible with the observed data, that attains each of the values within the interval. Under monotonicity, the bounds collapse to single points, with d = 0 and c = m 1 -m 0 .

0

1

1

1

s

(

x, z, w

)

c

(

x, z, w

)

d

(

x, z, w

)

Proof. There are three linear relations that the values s, d, c, u obey:

On top of this, we know that s, d, c, u are all non-negative. Based on the linear relations, we know that the following parametrization of the vector ( s, d, c, u ) holds

which represents a line in the 3-dimensional space ( s, d, c ) . In particular, we know that the values of ( s, d, c ) have to lie below the unit simplex in Fig. 5 (in yellow). In particular, the red and the blue planes represent the linear constraints from Eq. 64-65. The line parametrized in Eq. 67 lies at the intersection of the red and blue planes. Notice that d ∈ [0 , min( m 0 , 1 -m 1 )] since each of the elements in Eq. 67 is positive. This bound on d also implies that c ∈ [ m 1 -m 0 , m 1 ] . Finally, we need to construct an f Y mechanism that achieves any value within the bounds. To this end, define

which is both feasible and satisfies the proportion of canonical types to be ( s, d, c, u ) .

C Proof of Thm. 4

Proof. The first part of the theorem states the optimality of the D CF policy in the counterfactual world. Given that the policy uses the true benefit values from the counterfactual world, we apply the argument of Thm. 2 to prove its optimality.

We next prove the optimality of the D UT policy from Alg. 3. In Step 2 we check whether all individuals with a positive benefit can be treated. If yes, then the policy D UT is the overall optimal policy. If not, in Step 6 we check whether the overall optimal policy has a disparity bounded by M . If this is the case, D UT is the overall optimal policy for a budget ≤ b , and cannot be strictly improved. For the remainder of the proof, we may suppose that D UT uses the entire budget b (since we are operating under scarcity), and that D UT has introduces a disparity ≥ M . We also assume that the benefit ∆ admits a density, and that probability P (∆ ∈ [ a, b ] | x ) > 0 for any [ a, b ] ⊂ [0 , 1] and x .

Let δ ( x 0 ) , δ ( x 1 ) be the two thresholds used by the D UT policy. Suppose that ˜ D UT is a policy that has a higher expected utility and introduces a disparity bounded by M , or treats everyone in the disadvantaged group. Then there exists an alternative policy D UT with a higher or equal utility that takes the form

with δ ( x 0 ) ′ , δ ( x 1 ) ′ non-negative (otherwise, the policy can be trivially improved). In words, for any policy D UT there is a threshold based policy that is no worse. The policy D UT is also a threshold based policy. Now, if we had

it would mean policy D UT is using a larger budget than D UT . However, D UT uses a budget of b , making D UT infeasible. Therefore, we must have that

We first handle the case in Eq. 74. In this case, the policy D UT introduces a larger disparity than D UT . Since the disparity of D UT is at least M , the disparity of D UT is strictly greater than M . Further, note that δ ( x 0 ) ′ > δ ( x 0 ) ≥ 0 , showing that D UT does not treat all individuals with a positive benefit in the disadvantaged group. Combined with a disparity of > M , this makes the policy D UT infeasible.

For the second case in Eq. 75, let U ( δ 0 , δ 1 ) denote the utility of a threshold based policy:

Thus, we have that

where the last line follows from the fact that D UT has a budget no higher than D UT . Thus, this case also gives a contradiction.

Therefore, we conclude that policy D UT is optimal among all policies with a budget ≤ b that either introduce a bounded disparity in resource allocation | P ( d | x 1 ) -P ( d | x 0 ) | ≤ M or treat everyone with a positive benefit in the disadvantaged group.

D Equivalence of CF and UT Methods in Alg. 3

Anatural question to ask is whether the two methods in Alg. 3 yield the same decision policy in terms of the individuals that are selected for treatment. To examine this issue, we first define the notion of counterfactual crossing:

Definition 7 (Counterfactual crossing) . We say that two units of the population u 1 , u 2 satisfy counterfactual crossing with respect to an intervention C if

(i) u 1 , u 2 belong to the same protected group, X ( u 1 ) = X ( u 2 ) .

(ii) unit u 1 has a higher factual benefit than u 2 , ∆( u 1 ) > ∆( u 2 ) ,

(iii) unit u 1 has a lower counterfactual benefit than u 2 under the intervention C , ∆ C ( u 1 ) < ∆ C ( u 2 ) .

In words, two units satisfy counterfactual crossing if u 1 has a higher benefit than u 2 in the factual world, while in the counterfactual world the benefit is larger for the unit u 2 . Based on this notion, we can give a condition under which the causal and utilitarian approaches are equivalent:

Proposition 9 (Causal and Utilitarian Equivalence) . Suppose that no two units of the population satisfy counterfactual crossing with respect to an intervention C , and suppose that the distribution of the benefit ∆ admits a density. Then, the causal approach based on applying Alg. 1 with counterfactual benefit ∆ C , and the utilitarian approach based on factual benefit ∆ and the disparity M defined in Eq. 29, will select the same set of units for treatment.

Proof. The policy D UT treats individuals who have the highest benefit ∆ in each group. The D CF policy treats individuals with the highest counterfactual benefit ∆ C . Importantly, the policies treat the same number of individuals in the x 0 and x 1 groups. Note that, in the absence of counterfactual crossing, the relative ordering of the values of ∆ , ∆ C does not change, since

Thus, since both policies pick the same number of individuals, and the relative order of ∆ , ∆ C is the same, D UT and D CF will treat the same individuals.

E Experiment

Weapply the causal framework of outcome control to the problem of allocating mechanical ventilation in intensive care units (ICUs), which is recognized as an important task when resources are scarce [5], such as during the COVID-19 pandemic [40, 42]. An increasing amount of evidence indicates that a sex-specific bias in the process of allocating mechanical ventilation may exist [27], and thus the protected attribute X will be sex ( x 0 for females, x 1 for males).

To investigate this issue using the tools developed in this paper, we use the data from the MIMIC-IV dataset [17, 16] that originates from the Beth Israel Deaconess Medical Center in Boston, Massachusetts. In particular, we consider the cohort of all patients in the database admitted to the ICU. Patients who are mechanically ventilated immediately upon entering the ICU are subsequently removed. By focusing on the time window of the first 48 hours from admission to ICU, for each patient we determine the earliest time of mechanical ventilation, labeled t MV . Since mechanical ventilation is used to stabilize the respiratory profile of patients, for each patient we determine the average oxygen saturation in the three-hour period [ t MV -3 , t MV ] prior to mechanical ventilation, labeled O 2 -pre. We also determine the oxygen saturation in the three-hour period following ventilation [ t MV , t MV + 3] , labeled O 2 -post. For controls (patient not ventilated at any point in the first 48 hours), we take the reference point as 12 hours after ICU admission, and calculate the values in three hours before and after this time. Patients' respiratory stability, which represents the outcome of interest Y , is measured as follows:

Values of oxygen saturation above 97 are considered stable, and the larger the distance from this stability value, the higher the risk for the patient. We also collect other important patient characteristics before intervention that are the key predictors of outcome, including the SOFA score [39], respiratory rate, and partial oxygen pressure (PaO 2 ). The data loading is performed using the ricu R-package [4], and the source code for reproducing the entire experiment can be found here.

(a) Disparity in

D

curr

decomposed.

(b) Benefit fairness criterion on D curr .

0.03

0.02

Value

0.01

0.00

CausalFairnessMeasure

0.25

P(mechanical ventilation)

Sex

Female

0.20

Male

0.15

0.10

0.05

D1

D2

D3

D4

D5

D6

D7

D8

D9

D10

BenefitDecile

Step 2: Analyze the current policy using Alg. 2. Then, we perform an analysis of the currently implemented policy D curr , by computing the disparity in resource allocation, P ( d curr | x 1 ) -P ( d curr | x 0 ) , and also the causal decomposition of the disparity into its direct, indirect, and spurious contributions:

The results are shown in Fig. 6a, with vertical bars indicating 95% confidence intervals obtained using bootstrap. The decomposition demonstrates that the decision to mechanically ventilate a patient has a large direct effect of the protected attribute X , while the indirect and spurious effects explain a smaller portion of the disparity in resource allocation. We then compute

across the deciles of the benefit ∆ . In order to do so, we need to estimate the conditional potential outcomes Y d 0 , Y d 1 , and in particular their difference ❊ [ Y d 1 -Y d 0 | x, z, w ] . We fit an xgboost model which regresses Y on D,X,Z, and W , to obtain the fit ̂ Y . The learning rate was fixed at η = 0 . 1 , and the optimal number of rounds was chosen via 10-fold cross-validation. We then use the obtained model to generate predictions

from which we can estimate the benefit ∆ . The results for the probability of treatment given a fixed decile are shown in Fig. 6b. Interestingly, at each decile, women are less likely to be mechanically ventilated, indicating a possible bias.

Step 3: Apply Alg. 1 to obtain D ∗ . Our next step is to introduce an optimal policy that satisfies benefit fairness. To do so, we make use of the benefit values. In our cohort of 50,827 patients, a total of 5,404 (10.6%) are mechanically ventilated. We assume that the new policy D ∗ needs to achieve the same budget. Therefore, we bin patients according to the percentile of their estimated benefit ∆ . For the percentiles [90% , 100%] , all of the patients are treated. In the 89 -90 percentile, only 60% of the patients can be treated. We thus make sure that

Due to the construction, the policy D ∗ satisfies the benefit fairness criterion from Def. 2.

(a) Disparity in

D

∗

decomposed.

(b) Benefit ∆ disparity decomposition.

0.010

Value

0.005

0.000

CausalFairnessMeasure

0.4

0.3

Value

0.2

0.1

0.0

CausalFairnessMeasure

Step 4: Apply Alg. 2 to analyze D ∗ . We next decompose the disparity of the new policy D ∗ , and also decompose the disparity in the benefit ∆ . We obtain the following results:

The two decompositions are also visualized in Fig. 7. Therefore, even after applying benefit fairness, some disparity between the sexes remains. The causal analysis reveals that males require more mechanical ventilation because they are more severely ill (indirect effect). They also require more mechanical ventilation since they are older (spurious effect), although this effect is not significant. Finally, males also seem to benefit more from treatment when all other variables are kept the same (direct effect, see Fig. 7b). We note that using Alg. 1 has reduced the disparity in resource allocation, with a substantial reduction of the direct effect (see Eq. 93 vs. Eq. 97).

Step 5: Apply Alg. 3 to create D CF . In the final step, we wish to remove the direct effect of sex on the benefit ∆ . To construct the new policy D CF we will make use of Alg. 3. Firstly, we need to compute the counterfactual benefit values, in the world where X = x 1 along the direct pathway, while W attains its natural value W X ( u ) ( u ) . That is, we wish to estimate ∆ x 1 ,W X ( u ) for all patients in the cohort. For the computation of the counterfactual values, we make use of the xgboost model developed above. In particular, we use the fitted model to estimate the potential outcomes

The adjusted potential outcomes allow us to estimate ∆ x 1 ,W X ( u ) , after which we obtain the policy D CF that satisfies the CBF criterion from Def. 3.

After constructing D CF , we have a final look at the disparity introduced by this policy. By another application of Alg. 2, we obtain that

Therefore, we can see that the removal of the direct effect from the benefit ∆ resulted in a further decrease in the overall disparity. The comparison of the causal decompositions for the original policy D curr , optimal policy D ∗ obtained from Alg. 1, and the causally fair policy D CF is shown in Fig. 8.

3.0%

2.0%

Value

1.0% 4

0.0%

TV

DE

IE

SE

FairnessMeasure

Policy

Dcurr

D

#

DCF

F Connection of Benefit Fairness and Demographic Parity

In this appendix, we discuss the connection of the benefit fairness (BF) criterion with demographic parity (DP). The connection is given in the following formal result.

Proposition 10 (Benefit Fairness & Demographic Parity) . Suppose that the distribution of the benefit ∆ is equal between the groups x 0 , x 1 , that is

Then any decision policy D satisfying benefit fairness also satisfies demographic parity.

Proof.

implying demographic parity.

G Practical Considerations for Decompositions in Alg. 2

In this appendix, we discuss some practical details about how to perform the decompositions of the treatment disparity and benefit disparity as described in Alg. 2. In particular, we begin by showing that the benefit ∆ can be considered as variable in the causal diagram.

Proposition 11 (Connection of Benefit with Structural Causal Models) . Let M be an SCM compatible with the SFM in Fig. 1. Let the benefit ∆ be defined as in Eq. 14. Let f Y be the structural mechanism of the outcome Y , taking X,Z,W,D as inputs, together with the noise variable U Y . The structural mechanism fo the benefit ∆ is then given by:

f ∆ ( x, z, w ) = ❊ u y [ f Y ( x, z, w, d 1 , u y )] -❊ u y [ f Y ( x, z, w, d 0 , u y )] , (105) where ❊ u y integrates over the randomness in U Y . Therefore, the benefit ∆ is a deterministic function of X,Z,W , and we can add it to the causal diagram as follows:

.

Z

X

W

∆

Therefore, after adding ∆ to the causal diagram as in the above proposition, we can view it as another outcome variable. In particular, it also allows us to define various counterfactuals with respect to it, via the standard definitions for counterfactuals used in structural causal models. For instance, the potential outcome ∆ x 0 (which is a random variable) is simply given by f ∆ ( x 0 , Z, W ) , where Z, W are random variables but X = x 0 is fixed. Other, possibly nested counterfactuals are also defined analogously.

Equipped with a structural understanding of how benefit relates to the original causal model, we can now explain in more detail how the decompositions in Alg. 2 are performed. In particular, the decompositions follow the previously proposed approaches of [44, 31]:

Proposition 12 (Computing Decompositions in Alg. 2) . Let M be an SCM compatible with the SFM in Fig. 1. Let the benefit ∆ be defined as in Eq. 14 and suppose it is added to the causal diagram as in Prop. 11. The decompositions of the disparity in treatment allocation, and the disparity in expected

benefit, can be computed as follows:

Furthermore, under the assumptions of the SFM in Fig. 1 both of the decompositions are identifiable and their identification expressions are given by

where the random variable F is either replaced by the treatment decision D to obtain the direct, indirect, and spurious terms in Eqs. 106-108 or by the benefit ∆ to obtain the terms in Eqs. 109-111.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors analyze a new dimension of decision making: outcome control and benefit from decisions. Specifically, this work uses causal analysis to introduce new definitions of fairness on benefits and outcome control. The authors also propose new algorithms to study fairness and design fair algorithms for outcome fair decisions.\n\nSTRENGTHS:\n* The paper proposes new fairness notions for outcome control, that is an important aspect of many decision making settings\n* The authors use a simple example to walk through all the different aspects and notions, that greatly helps the readability of the paper.\n* Through the synthetic use case, the authors show when and how benefit fairness is important, and how causal path analysis could also become critical for decision making while accounting for fairer outcomes.\n\nWEAKNESSES:\n* While the appendix provides additional results on a real-world COVID dataset, the main paper has limited results on various different settings.\n* While the authors assume causal knowledge, the paper does not discuss limitations explicitly for when computing counterfactuals and especially path-specific counterfactuals might become challenging in practice.\n* Section 3 should have better explained which counterfactuals we compute to look at the path-specific factors and how the 3 different components are found out by decomposition.\n\nQUESTIONS:\n* Two works [1, 2] seem somewhat related to the authors’ notions and framework. How does the framework of decision outcome control relate to the thinking of fairness of “treatments” in [1]? Similarly, how does the notion of “benefit” the authors introduce relate to the notion of “harm” in [2]? \n\n[1] Madras, David, et al. \"Fairness through causal awareness: Learning causal latent-variable models for biased data.\" Proceedings of the conference on fairness, accountability, and transparency. 2019. [2] Richens, Jonathan, Rory Beard, and Daniel H. Thompson. \"Counterfactual harm.\" Advances in Neural Information Processing Systems 35 (2022): 36350-36365.\n\nLIMITATIONS:\n* While the paper’s primary focus is on introducing a new theoretical framework of fairness in decision making related to outcome control and benefit, more analysis and results on different experimental settings might have been an added benefit.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new fairness notion called benefit fairness that considers fairness in the outcomes of the decisions. In order to avoid the unidentifiability issue of the principle fairness, this paper proposes to condition on the conditional average treatment effect (CATE). The paper provides an algorithm that is proven to be optimal and satisfy benefit fairness. The paper in addition provides an algorithm to satisfy both benefit fairness and counterfactual benefit fairness.\n\nSTRENGTHS:\nThe proposed new fairness notion is quite interesting. It is different from traditional fairness notions, yet it is well-motivated and -explained using a real-world example. \n\nThe proposed algorithms come with theoretical analysis that shows their optimality and fairness satisfaction. \n\nA budget is involved in the problem formulation that makes the algorithms more generally applicable.\n\nWEAKNESSES:\nThe connection between benefit fairness and traditional fairness notions other than principle fairness, such as demographic parity, equal opportunity, etc., is not clear. Are they consistent or do they conflict to some extent? A rigorous study may be out of the scope of this paper but some discussions will be helpful.\n\nThe measure of benefit fairness depends on the observation of the outcome Y. Can we generally assume that Y is available when we construct the decision policy? In the running example, Y is observed after a 2-year period. How can we estimate Y when we construct the decision policy before the 2-year period?\n\nSome statements and notations are not very clear. For example, in Definition 3, $y_C$ is not defined and the definition of the pathway is not clear. My understanding is that the paper attempts to formulate path-specific counterfactual fairness, but the notations are quite confusing. In addition, $\\Delta_{C}$ is not defined either. The two terms CBF and BFC may easily confuse the readers.\n\nQUESTIONS:\nAlthough I didn’t check the correctness of the proof, Theorem 2 seems quite strong. Are there any assumptions about the policy or the causal model in Theorem 2?\n\nMy understanding of benefit fairness is that, it considers the causal effect of the link X->Y. If we remove this link, then $\\Delta(x_0,z,w)$ is always equal to $\\Delta(x_1,z,w)$, and benefit fairness degrades to the conditional demographic parity. Thus, the meaning of benefit fairness is to make the causal effect of the link X->D match the causal effect of the link X->Y? Is this understanding correct? \n\nIf my above understanding is correct, does that mean that CBF cannot be satisfied if the link X->Y exists, because CBF generally means that  the decision should not depend on X?\n\nLIMITATIONS:\nNone.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem where a decision maker must allocate a treatment $D$ to optimize an outcome variable $Y$ while ensuring that the decision is fair, formulating fairness as the protected attribute $X$ not having an influence on $D$. It uses a clinical decision-making process (a running example on Cancer Surgery, Fig. 2(b)) to showcase its proposed method.\n \nThe paper proposes the notion of benefit based on the potential outcomes (PO) framework. $Y_{d_0}$ denotes the outcome of a patient that didn’t undergo surgery while $Y_{d_1}$ denotes the outcome of a patient that did undergo surgery. Under PO, only one of the two potential outcomes are observed per patient. Benefit is defined as $Y_{d_1} - Y_{d_0}$. Hence, a decision maker will derive a $D^*$ that maximizes overall benefit: i.e., those patients for which $Y_{d_0}=0$ (died without surgery) and $Y_{d_1}$ (survived with surgery).\n \nThrough the running example, the paper shows how a decision maker (under imperfect information) can be unfair via $D$ despite optimizing for the benefit. It then introduces a set of algorithms to reach fair benefit.\n\nSTRENGTHS:\n[S1] The topic is relevant and novel within (causal) fairness. Under a heterogenous population of individuals, it is reasonable to expect that not all individuals will benefit from a treatment (captured by $D$ and its causal effect on $Y$ in the paper). The notion of benefit becomes relevant as (i) it formalizes which individuals need $D$ and which do not; (ii) it raises the potential issue of what happens when those that benefit under $D$ don’t embody societal representational fairness goals.\n\n[S2] The use of the running example was very useful for understanding the paper’s goal.\n\n[S3] Fig. 3 illustrates well the problem of allocating a treatment without knowing the type of patient w.r.t. to $D$.\n\n[S4] Def. 2 (Benefit Fairness), putting aside the estimation issues, is a nice extension to other group-level fairness definitions (like Equal Opportunity). It states that individuals across the protected attribute $X$ that have the same (potential) benefit $\\delta$ to gain from treatment should have the same probability of getting treated.\n\nWEAKNESSES:\n[W1] PO limitations should be discussed further: Given the use of the PO framework, there’s a lack of treatment on how the benefit ($\\Delta = Y_{d_1} - Y_{d_0}$) can be estimated in a consistent and unbiased way, at least, at the individual level. For each individual, we can only observe $Y_{d_0}$ or $Y_{d_1}$: for each individual, the one observed is the factual and the other one the counterfactual. However, the paper lacks a meaningful discussion on how we can estimate the counterfactuals in the non-oracle setting, which is the one of practical interest. Even in the running example, isn’t there a risk that the clinicians are using a biased $\\Delta$? If so, how would that affect the proposed algorithms? The limitations should be discussed explicitly.\n\nIn particular, the derivation and justification of Eq. (16), which is central to the algorithm(s) is not clear. Is it a statement on the same tuple $(x, z, w)$: if so, how can the same individual be benefited and harmed at the same time by the treatment? Or is it a “match” between two individuals? Also, what is $z$ in the running example (this random variable is only presented and used in Fig. 1)?\n\nCounterfactual generation/estimation is not exploited in the example. For instance, if we have access to the SCM model, couldn’t we generate the counterfactual distribution via Pearl’s abduction, action, and prediction steps for $Y$. We could then identify the “doomed” and “safe”? This is important as Algorithm 1 can only operate meaningfully if it can identify the borders of the types of individuals (w.r.t. the notion of benefit). Line 1 in Algorithm 1 needs to be discussed further.\n\n[W2] Stronger Section 2: This section could be tighter. For instance, what’s the role of the budget $b$ from Def. 1? Even in the oracle setting, it is possible that we can’t treat all 100 patients if the budget is low enough, no? Conversely, if the budget is high enough and $\\Delta \\geq 0$, we could treat all patients, no?\n\nFurther, as a follow up to W1, if there is the pair \\{$Y_{d_0}$, $Y_{d_1}$\\} for each individual then there is also a $\\Delta$ parameter. Now, under the oracle, we can see through the future and allocate $D$ based on $\\Delta$. In the non-oracle setting, there’s a brief discussion about using $W$ as a proxy that leads to unfairness: how does that translate into estimating $\\Delta$? Otherwise, if we can estimate it, then why are we even considering the proxy $W$? What I’m hinting at here – do correct me if I’m wrong – is that Def. 2 needs to have some measure of uncertainty to highlight that the non-oracle allocation will have some error w.r.t. to the oracle allocation (for a fixed $b$). Otherwise, it seems like we are still in the oracle setting.\n\n[W3] Limited evaluation: The running example is essentially the only use case. Although it shows the proposed methodology, it would’ve helped to, e.g., test the algorithms under different parameters for the same synthetic data. Similarly, could the algorithms handle other variables on top $W$ or under a second protected attribute like race? The evaluation, even for the single use case, could’ve been pushed further to show the robustness of the approach.\n\n[W4] Some definitions are unclear or not fully explained. For instance, \nIn Def. 1, the role of the budget $b$ is not explained. It would’ve also helped to formulate it under the PO framework explicitly. \nIn Def. 2, please define $\\Delta$ and $\\delta$ within the definition itself. These are presented later.\nIn Def. 3, is it at the individual level? Under the SCM presented (Fig. 2, but also Fig. 2), how are the conditionals the same under the interventions captured by the causal pathways. Don’t we need to update for downstream effects under $\\mathcal{C}$?\n\nQUESTIONS:\nSee my questions in the Weaknesses section. I’m willing to increase my score if these questions are addressed.\n\nLIMITATIONS:\nFurther discussion on $\\Delta$ is missing. It’s a considerable limitation for the application of this definition and it’s not evaluated here theoretically.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper focuses on outcome control from a causal perspective of the decision-maker. The authors introduce benefit fairness taking the perspective of the decision maker and provide theoretical guarantee that the algorithmic result is optimal and satisfies benefit fairness. To support the decision maker in indicating potential discrimination early on, algorithm 2 evaluates the difference in benefit to demographic groups. And the paper defines causal benefit fairness with an accompanying algorithm and a guarantee of optimality.\n\nSTRENGTHS:\nS1 - In fairness research, the need for more causal perspectives has been a growing discussion. This research is significant in contributing to an underdeveloped area.\n\nS2 - The research problem is well developed and simple guarantees provided with proofs in the appendix.\n\nWEAKNESSES:\nExposition clarity needs minor improvement. For instance, what is the relationship between the first and second paragraph of the introduction? What do the authors see as the connection between outcome control and historical biases?\n\nQUESTIONS:\nThe authors do not actually provide a clear definition of “outcome control” as used in this paper. Based on the content, I assumed they are using the high level definition put forth by [Procedural Justice in Algorithmic Fairness, CSCW 2019], which defines outcome control as enabling the correctability and possible recourse against an individual decision. But this paper is not cited in the present paper. The authors reference [Causal Fairness Analysis, 2022], which is an ICML tutorial where section 5.3.3 contains exact wording and sentences from the present paper lines 35-38. The tutorial includes more insight clarifying that outcome control setting requires that the institution and the individual have the same utility function. It would be helpful to clarify this in the main text of the present paper. What is the definition of “outcome control” being used in this paper?\n\nLIMITATIONS:\nIt is not clear how generalizable this framework is given the demonstration on clinical examples.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-10"}