SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Causal structures imply conditional independencies. For example, if the only causal relationships are that  $X$  causes  $Y$  and  $Y$  causes  $Z$ , then we know that  $X \perp Z \mid Y$ . On a directed acyclic graph (DAG), following Pearl and Dechter [38], we say that a variable  $Y$  dependence-separates or d-separates  $X$  and  $Z$  if  $X$  and  $Z$  are connected via an unblocked path (i.e., no unconditioned collider in which two arrows point directly to the same variable) but are no longer connected after removing all arrows that directly connect to  $Y$  [22]. So in this example,  $Y$  d-separates  $X$  and  $Z$ , which is equivalent to the conditional independence statement. A selection variable  $S$  in a rectangle indicates an observed variable that is conditioned on in the data-generating process, which induces an association between its parents and thereby does not block the path as an unconditioned collider would. For Theorem 2, Corollary 2.1, and Corollary 2.2, we make the usual assumption of faithfulness of the causal graph, meaning that the only conditional independencies are those implied by d-separation, rather than any causal effects that happen to be precisely counterbalanced. Specifically, we assume faithfulness between the protected class  $A$ , the label  $Y$ , and the component of  $X$  on which the predictor  $f(X)$  is based. If the component is not already its own node in the causal graph, then faithfulness would apply if the component were isolated into its own node or nodes.
For a predictor, these implied conditional independencies can be group fairness metrics. We can restate conditional independencies containing  $X_A^\perp$  as containing  $f(X)$  because, if  $f(X)$  is a counterfactually fair predictor, it only depends on  $X_A^\perp$ , the component that is not causally affected by  $A$ . In Theorem 2, we provide the correspondence between counterfactual fairness and the three most common metrics: demographic parity, equalized odds, and calibration.
Theorem 2. Let the causal structure be a causal DAG with  $X_{Y}^{\perp}$ ,  $X_{A}^{\perp}$ ,  $Y$ , and  $A$ , such as in Figure 2. Assume faithfulness between  $A$ ,  $Y$ , and  $f(X)$ . Then:
1. Counterfactual fairness is equivalent to demographic parity if and only if there is no unblocked path between  $X_A^\perp$  and  $A$ .
2. Counterfactual fairness is equivalent to equalized odds if and only if all paths between  $X_A^\perp$  and  $A$ , if any, are either blocked by a variable other than  $Y$  or unblocked and contain  $Y$ .
3. Counterfactual fairness is equivalent to calibration if and only if all paths between  $Y$  and  $A$ , if any, are either blocked by a variable other than  $X_A^\perp$  or unblocked and contain  $X_A^\perp$ .
Similar statements can be made for any group fairness metric. For example, the notion of false negative error rate balance, also known as equal opportunity [23], is identical to equalized odds but only considers individuals who have a positive label  $(Y = 1)$ . The case for this metric is based on false negatives being a particular moral or legal concern, motivated by principles such as "innocent until proven guilty," in which a false negative represents an innocent person  $(Y = 1)$  who is found guilty  $(f(X) = 0)$ . False negative error rate balance is assessed in the same way as equalized odds but only with observations that have a positive true label, which may be consequential if different causal structures are believed to obtain for different groups.
Table 1 shows the ten group fairness metrics presented in Verma and Rubin [48] that can be expressed as conditional independencies. Theorem 2 specified the correspondence between three of these (demographic parity, equalized odds, and calibration), and we extend to the other seven in Corollary 2.1.
We have so far restricted ourselves to a binary classifier  $f(X) \in \{0,1\}$ . Here, we denote this as a decision  $f(x) = d \in D = \{0,1\}$  and also consider probabilistic classifiers that produce a score  $s$  that can take any probability from 0 to 1, i.e.,  $f(x) = s \in S = [0,1] = \mathbb{P}(Y = 1)$ . The metrics of balance for positive class, balance for negative class, and score calibration are defined by Verma and Rubin [48] in terms of score. Verma and Rubin [48] refer to calibration for binary classification (Definition 3) as "conditional use accuracy equality." To differentiate these, we henceforth refer to the binary classification metric as "binary calibration" and the probabilistic classification metric as "score calibration."
Table 1: Group fairness metrics from Verma and Rubin [48].
Name	Probability Definition	Independence Definition
Demographic Parity	\(\mathbb{P}(D=1 \mid A=0)=\mathbb{P}(D=1 \mid A=1)\)	D⊥A
Conditional Demographic Parity	\(\mathbb{P}(D=1 \mid A=0,L=l)=\mathbb{P}(D=1 \mid A=1,L=l)\)	D⊥A|L=l
Equalized Odds	\(\mathbb{P}(D=1 \mid A=0,Y=y)=\mathbb{P}(D=1 \mid A=1,Y=y)\)	D⊥A|Y
False Positive Error Rate Balance	\(\mathbb{P}(D=1 \mid A=0,Y=0)=\mathbb{P}(D=1 \mid A=1,Y=0)\)	D⊥A|Y=0
False Negative Error Rate Balance	\(\mathbb{P}(D=0 \mid A=0,Y=1)=\mathbb{P}(D=0 \mid A=1,Y=1)\)	D⊥A|Y=1
Balance for Negative Class	\(\mathbb{E}[S \mid A=0,Y=0]=\mathbb{E}[S \mid A=1,Y=0]\)	S⊥A|Y=0
Balance for Positive Class	\(\mathbb{E}[S \mid A=0,Y=1]=\mathbb{E}[S \mid A=1,Y=1]\)	S⊥A|Y=1
Conditional Use Accuracy Equality (i.e., Calibration)	\(\mathbb{P}(Y=y \mid A=0,D=d)=\mathbb{P}(Y=y \mid A=1,D=d)\)	Y⊥A|D
Predictive Parity	\(\mathbb{P}(Y=1 \mid A=0,D=1)=\mathbb{P}(Y=1 \mid A=1,D=1)\)	Y⊥A|D=1
Score Calibration	\(\mathbb{P}(Y=1 \mid A=0,S=s)=\mathbb{P}(Y=1 \mid A=1,S=s)\)	Y⊥A|S
Corollary 2.1. Let the causal structure be a causal DAG with  $X_{Y}^{\perp}, X_{A}^{\perp}, Y$ , and  $A$ , such as in Figure 2. Assume faithfulness between  $A, Y$ , and  $f(X)$ . Then:
1. For a binary classifier, counterfactual fairness is equivalent to conditional demographic parity if and only if, when a set of legitimate factors  $L$  is held constant at level  $l$ , there is no unblocked path between  $X_{A}^{\perp}$  and  $A$ .
2. For a binary classifier, counterfactual fairness is equivalent to false positive error rate balance if and only if, for the subset of the population with negative label (i.e.,  $Y = 0$ ), there is no path between  $X_{A}^{\perp}$  and  $A$ , a path blocked by a variable other than  $Y$ , or an unblocked path that contains  $Y$ .
3. For a binary classifier, counterfactual fairness is equivalent to false negative error rate balance if and only if, for the subset of the population with positive label (i.e.,  $Y = 1$ ), there is no path between  $X_A^\perp$  and  $A$ , a path blocked by a variable other than  $Y$ , or an unblocked path that contains  $Y$ .
4. For a probabilistic classifier, counterfactual fairness is equivalent to balance for negative class if and only if, for the subset of the population with negative label (i.e.,  $Y = 0$ ), there is no path between  $X_A^\perp$  and  $A$ , a path blocked by a variable other than  $Y$ , or an unblocked path that contains  $Y$ .
5. For a probabilistic classifier, counterfactual fairness is equivalent to balance for positive class if and only if, for the subset of the population with positive label (i.e.,  $Y = 1$ ), there is no path between  $X_{A}^{\perp}$  and  $A$ , a path blocked by a variable other than  $Y$ , or an unblocked path that contains  $Y$ .
6. For a probabilistic classifier, counterfactual fairness is equivalent to predictive parity if and only if, for the subset of the population with positive label (i.e.,  $D = 1$ ), there is no path between  $Y$  and  $A$ , a path blocked by a variable other than  $X_{A}^{\perp}$ , or an unblocked path that contains  $X_{A}^{\perp}$ .
7. For a probabilistic classifier, counterfactual fairness is equivalent to score calibration if and only if there is no path between  $Y$  and  $A$ , a path blocked by a variable other than  $X_A^\perp$ , or an unblocked path that contains  $X_A^\perp$ .
Table 2: Experimental results: Robust prediction
Source Accuracy	Target Accuracy
Naive	FTU	CF	Naive	FTU	CF	Target Trained
Measurement Error	0.8283	0.7956	0.7695	0.8031	0.8114	0.8204	0.8674
Selection on Label	0.8771	0.8686	0.8610	0.8566	0.8652	0.8663	0.8655
Selection on Predictors	0.8659	0.656	0.8658	0.8700	0.8698	0.8699	0.8680
This general specification can be unwieldy, so to illustrate this, we provide three real-world examples shown in Figure 2: measurement error, selection on label (i.e., "post-treatment bias" or "selection on outcome" if the label  $Y$  is causally affected by  $X$ ), and selection on predictors. These imply demographic parity, equalized odds, and calibration, respectively.
Corollary 2.2. Assume faithfulness between  $A, Y$ , and  $f(X)$ .
1. Under the graph with measurement error as shown in Figure 2a, a predictor achieves counterfactual fairness if and only if it achieves demographic parity.
2. Under the graph with selection on label as shown in Figure 2b, a predictor achieves counterfactual fairness if and only if it achieves equalized odds.
3. Under the graph with selection on predictors as shown in Figure 2c, a predictor achieves counterfactual fairness if and only if it achieves calibration.
We conducted brief experiments in a semi-synthetic setting to show that a counterfactually fair predictor achieves robust prediction and group fairness. We used the Adult income dataset [3] with a simulated protected class  $A$ , balanced with  $P(A = 0) = P(A = 1) = 0.5$ . For observations with  $A = 1$ , we manipulated the input data to simulate a causal effect of  $A$  on  $X$ :  $P(\text{race} = 0\text{other}) = 0.8$ . With this as the target distribution, we produced three biased datasets that result from each effect produced with a fixed probability for observations when  $A = 1$ : measurement error ( $P = 0.8$ ), selection on label ( $P = 0.5$ ), and selection on predictors ( $P = 0.8$ ).
On each dataset, we trained three predictors: a naive predictor trained on  $A$  and  $X$ , a fairness through unawareness (FTU) predictor trained only on  $X$ , and a counterfactually fair predictor based on an average of the naive prediction under the assumption that  $A = 1$  and the naive prediction under the assumption  $A = 0$ , weighted by the proportion of each group in the target distribution.
Theorem 3. Let  $X$  be an input dataset  $X \in \mathcal{X}$  with a binary label  $Y \in \mathcal{Y} = \{0,1\}$  and protected class  $A \in \{0,1\}$ . Define a predictor:
$$
f _ {n a i v e} := \underset {f} {\operatorname {a r g m i n}} \mathbb {E} [ \ell (f (X, A), Y) ]
$$
where  $f$  is a proper scoring rule. Define another predictor:
$$
f _ {C F} := \mathbb {P} (A = 1) f _ {n a i v e} (X, 1) + \mathbb {P} (A = 0) f _ {n a i v e} (X, 0)
$$
If the association between  $Y$  and  $A$  is purely spurious, then  $f_{CF}$  is counterfactually fair.
For robust prediction, we show that the counterfactually fair (CF) predictor has accuracy in the target distribution near the accuracy of a predictor trained directly on the target distribution. For group fairness, we show that the counterfactually fair predictor achieves demographic parity, equalized odds, and calibration, corresponding to the three biased datasets. Results are shown in Table 2 and Table 3, and code to reproduce these results or produce results with varied inputs (number of datasets sampled, effect of  $A$  on  $X$ , probabilities of each bias, type of predictor) is available at https://github.com/jacyanthis/Causal-Context.
Table 3: Experimental results: Fairness metrics
Demographic Parity Difference (CF)	Equalized Odds Difference (CF)	Calibration Difference (CF)
Measurement Error	-0.0005	0.0906	-0.8158
Selection on Label	0.1321	-0.0021	0.2225
Selection on Predictors	0.1428	0.0789	0.0040
In this paper, we provided a new argument for counterfactual fairness—that the supposed trade-off between fairness and accuracy [12] can evaporate under plausible conditions when the goal is accuracy in an unbiased target distribution (Theorem 1). To address the challenge of trade-offs between different group fairness metrics, such as their mutual incompatibility [30] and the variation in costs of certain errors such as false positives and false negatives [19], we provided a conceptual tool for adjudicating between them using knowledge of the underlying causal context of the social problem (Theorem 2 and Corollary 2.1). We illustrated this for the three most common fairness metrics, in which the bias of measurement error implies demographic parity; selection on label implies equalized odds; and selection on predictors implies calibration (Corollary 2.2), and we showed a minimal example by inducing particular biases on a simulated protected class in the Adult income dataset.
There are nonetheless important limitations that we hope can be addressed in future work. First, the counterfactual fairness paradigm still faces significant practical challenges, such as ambiguity and identification. Researchers can use causal discovery strategies developed in a fairness context [5, 20] to identify the causal structure of biases in real-world datasets and ensure theory like that outlined in this paper can be translated to application. Second, a key assumption in our paper and related work has been the assumption that associations between  $Y$  and  $A$  are "purely spurious," a term coined by Veitch et al. [47] to refer to cases where, if one conditions on  $A$ , then the only information about  $Y$  in  $X$  is in the component of  $X$  that is not causally affected by  $A$ . This has provided a useful conceptual foothold to build theory, but it should be possible for future work to move beyond the purely spurious case, such as by articulating distribution shift and deriving observable signatures of counterfactual fairness in more complex settings [36, 39].
We thank Bryon Aragam, James Evans, and Sean Richardson for useful discussions.
[1] Kartik Ahuja et al. "Conditionally Independent Data Generation". In: Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence. Ed. by Cassio de Campos and Marloes H. Maathuis. Vol. 161. Proceedings of Machine Learning Research. PMLR, July 2021, pp. 2050–2060.
[2] Julia Angwin et al. "Machine Bias". In: ProPublica (2016). (Visited on 08/20/2023).
[3] Barry Becker and Ronny Kohavi. Adult. 1996. DOI: 10 .24432 / C5XW20. (Visited on 08/21/2023).
[4] Andrew Bell et al. "The Possibility of Fairness: Revisiting the Impossibility Theorem in Practice". In: 2023 ACM Conference on Fairness, Accountability, and Transparency. Chicago IL USA: ACM, June 2023, pp. 400-422. DOI: 10.1145/3593013.3594007. (Visited on 08/05/2023).
[5] Ruta Binkyté-Sadauskiené et al. Causal Discovery for Fairness. June 2022. arXiv: 2206.06685 [cs, stat]. (Visited on 05/02/2023).
$$
f^{*}(X):= \operatorname *{argmin}_{f\in \mathcal{F}^{CF}}\mathbb{E}_{P}[\ell (f(X),Y)]
$$
Proof. Counterfactual fairness is a case of counterfactual invariance. By Lemma 3.1 in Veitch et al. [47], this implies  $X$  is  $X_A^\perp$ -measurable. Therefore,
$$
\operatorname *{argmin}_{f\in \mathcal{F}^{\mathrm{CF}}}\mathbb{E}_{P}[\ell (f(X),Y)] = \operatorname *{argmin}_{f\in \mathcal{F}^{\mathrm{CF}}}\mathbb{E}_{P}[\ell (f(X^{\frac{1}{A}}),Y)]
$$
Following the same reasoning as Theorem 4.2 in Veitch et al. [47], it is well-known that under squared error or cross entropy loss the risk minimizer is  $f^{*}(x_{A}^{\perp}) = \mathbb{E}_{P}[Y\mid x_{A}^{\perp}]$ . Because the target distribution  $Q$  has no selection (and no confounding because  $A$  is exogenous in the case of counterfactual fairness), the risk minimizer in the target distribution is the same as the counterfactually fair risk minimizer in the target distribution, i.e.,  $\mathbb{E}_Q[Y\mid x] = \mathbb{E}_Q[Y\mid x_A^\perp ]$ . Thus our task is to show  $\mathbb{E}_P[Y\mid x_A^\perp ] = \mathbb{E}_Q[Y\mid x_A^\perp ]$ .
Selection on label is shown in Figure 2b. Because  $X_{A}^{\perp}$  does not d-separate  $Y$  and  $A$ ,  $f^{*}(X)$  depends on the marginal distribution of  $Y$ , so we need an additional assumption that  $P(Y) = Q(Y)$ . We can use this with Bayes' theorem to show the equivalence of the conditional distributions,
$$
\begin{array}{l} Q (Y \mid X _ {A} ^ {\perp}) = \frac {Q \left(X _ {A} ^ {\perp} \mid Y\right) Q (Y)}{\int Q \left(X _ {A} ^ {\perp} \mid Y\right) Q (Y) \mathrm {d} y} (A.1) \\ = \frac {P \left(X _ {A} ^ {\perp} \mid Y\right) Q (Y)}{\int P \left(X _ {A} ^ {\perp} \mid Y\right) Q (Y) \mathrm {d} y} (A.2) \\ = \frac {P \left(X _ {A} ^ {\perp} \mid Y\right) P (Y)}{\int P \left(X _ {A} ^ {\perp} \mid Y\right) P (Y) \mathrm {d} y} (A.3) \\ = P \left(Y \mid X _ {A} ^ {\perp}\right), (A.4) \\ \end{array}
$$
where the first and fourth lines follow from Bayes' theorem, the second line follows from the causal structure  $(X$  causes  $Y)$ , and the third line follows from the assumption that  $P(Y) = Q(Y)$ . This equality of distributions implies equality of expectations.
Selection on predictors is shown in Figure 2c. Because  $X_A^\perp$  d-separates  $Y$  and  $A$ ,  $f^*(X)$  does not depend on the marginal distribution of  $Y$ , so we immediately have an equality of conditional distributions,  $Q(Y \mid X_A^\perp) = P(Y \mid X_A^\perp)$ , and equal distributions have equal expectations,  $\mathbb{E}_P[Y \mid x_A^\perp] = \mathbb{E}_Q[Y \mid x_A^\perp]$ .
1. Counterfactual fairness is equivalent to demographic parity if and only if there is no unblocked path between  $X_A^\perp$  and  $A$ .
2. Counterfactual fairness is equivalent to equalized odds if and only if all paths between  $X_{A}^{\perp}$  and  $A$ , if any, are either blocked by a variable other than  $Y$  or unblocked and contain  $Y$ .
Figure 3: Examples of an unblocked path and a blocked path in a causal DAG.
3. Counterfactual fairness is equivalent to calibration if and only if all paths between  $Y$  and  $A$ , if any, are either blocked by a variable other than  $X_A^\perp$  or unblocked and contain  $X_A^\perp$ .
Proof. For a predictor  $f(X)$  to be counterfactually fair, it must only depend on  $X_A^\perp$ .
1. By Definition 1, demographic parity is achieved if and only if  $X_{A}^{\perp} \perp A$ . In a DAG, variables are dependent if and only if there is an unblocked path between them (i.e., no unconditioned collider in which two arrows point directly to the same variable). For example, Figure 3a shows an unblocked path between  $A$  and  $X_{A}^{\perp}$ .
2. By Definition 2, equalized odds are achieved if and only if  $X_A^\perp \perp A \mid Y$ . If there is no path between  $X_A^\perp$  and  $A$ , then  $X_A^\perp$  and  $A$  are independent under any conditions. If there is a blocked path between  $X_A^\perp$  and  $A$ , and it has a block that is not  $Y$ , then  $X_A^\perp$  and  $A$  remain independent because a blocked path induces no dependence. If there is an unblocked path between  $X_A^\perp$  and  $A$  that contains  $Y$ , then  $Y$  d-separates  $X_A^\perp$  and  $A$ , so  $X_A^\perp$  and  $A$  remain independent when conditioning on  $Y$ . On the other hand, if there is a blocked path between  $X_A^\perp$  and  $A$  and its only block is  $Y$ , then conditioning on the block induces dependence. If there is an unblocked path that does not contain  $Y$ , then  $X_A^\perp$  and  $A$  are dependent.
3. With Definition 3, we can apply analogous reasoning to the case of equalized odds. Calibration is achieved if and only if  $Y \perp A \mid X_A^\perp$ . If there is no path between  $Y$  and  $A$ , then  $Y$  and  $A$  are independent under any conditions. If there is a blocked path between  $Y$  and  $A$ , and it has a block is not  $X_A^\perp$ , then  $Y$  and  $A$  remain independent because a blocked path induces no dependence. If there is an unblocked path between  $Y$  and  $A$  that contains  $X_A^\perp$ , then  $X_A^\perp$  d-separates  $Y$  and  $A$ , so  $Y$  and  $A$  remain independent when conditioning on  $X_A^\perp$ . On the other hand, if there is a blocked path between  $Y$  and  $A$  and its only block is  $X_A^\perp$ , then conditioning on the block induces dependence. If there is an unblocked path that does not contain  $X_A^\perp$ , then  $Y$  and  $A$  are dependent.
Corollary 2.1. Let the causal structure be a causal DAG with  $X_{Y}^{\perp}$ ,  $X_{A}^{\perp}$ ,  $Y$ , and  $A$ , such as in Figure 2. Assume faithfulness between  $A$ ,  $Y$ , and  $f(X)$ . Then:
1. For a binary classifier, counterfactual fairness is equivalent to conditional demographic parity if and only if, when a set of legitimate factors  $L$  is held constant at level  $l$ , there is no unblocked path between  $X_{A}^{\perp}$  and  $A$ .
2. For a binary classifier, counterfactual fairness is equivalent to false positive error rate balance if and only if, for the subset of the population with negative label (i.e.,  $Y = 0$ ), there is no path between  $X_A^\perp$  and  $A$ , a path blocked by a variable other than  $Y$ , or an unblocked path that contains  $Y$ .
3. For a binary classifier, counterfactual fairness is equivalent to false negative error rate balance if and only if, for the subset of the population with positive label (i.e.,  $Y = 1$ ), there is no path between  $X_{A}^{\perp}$  and  $A$ , a path blocked by a variable other than  $Y$ , or an unblocked path that contains  $Y$ .
4. For a probabilistic classifier, counterfactual fairness is equivalent to balance for negative class if and only if, for the subset of the population with negative label (i.e.,  $Y = 0$ ), there is no path between  $X_A^\perp$  and  $A$ , a path blocked by a variable other than  $Y$ , or an unblocked path that contains  $Y$ .
5. For a probabilistic classifier, counterfactual fairness is equivalent to balance for positive class if and only if, for the subset of the population with positive label (i.e.,  $Y = 1$ ), there is no path between  $X_A^\perp$  and  $A$ , a path blocked by a variable other than  $Y$ , or an unblocked path that contains  $Y$ .
6. For a probabilistic classifier, counterfactual fairness is equivalent to predictive parity if and only if, for the subset of the population with positive label (i.e.,  $D = 1$ ), there is no path between  $Y$  and  $A$ , a path blocked by a variable other than  $X_A^\perp$ , or an unblocked path that contains  $X_A^\perp$ .
7. For a probabilistic classifier, counterfactual fairness is equivalent to score calibration if and only if there is no path between  $Y$  and  $A$ , a path blocked by a variable other than  $X_{A}^{\perp}$ , or an unblocked path that contains  $X_{A}^{\perp}$ .
Proof. Each of these seven metrics can be stated as a conditional independence statement, as shown in Table 1, and each of the seven graphical tests of those statements can be derived from one of the three graphical tests in Theorem 2. Note that the graphical test for a binary classifier is the same as that for the corresponding probabilistic classifiers because the causal graph does not change when  $f(X_{A}^{\perp})$  changes from a binary-valued (i.e.,  $f(X) \in \{0,1\}$ ) function to a probability-valued function (i.e.,  $f(X) \in [0,1]$ ).
From demographic parity:
1. Conditional demographic parity is equivalent to demographic parity when some set of legitimate factors  $L$  is held constant at some value  $l$ .
From equalized odds:
2. False positive error rate balance is equivalent to equalized odds when considering only the population with negative label (i.e.,  $Y = 0$ ).
3. False negative error rate balance is equivalent to equalized odds when considering only the population with positive label (i.e.,  $Y = 1$ ).
4. Balance for negative class is equivalent to equalized odds for probabilistic classifiers when considering only the population with negative label (i.e.,  $Y = 0$ ).
5. Balance for positive class is equivalent to equalized odds for probabilistic classifiers when considering only the population with negative label (i.e.,  $Y = 1$ ).
From binary calibration:
6. Predictive parity is equivalent to binary calibration when considering only the population with positive label (i.e.,  $D = 1$ ).
7. Score calibration is equivalent to binary calibration for probabilistic classifiers.
Proof. By Theorem 2: