Fairness and robustness in anti-causal prediction
Authors: Maggie Makar
Alexander D'Amour
Venue: Accepted by TMLR ()

Abstract: Robustness to distribution shift and fairness have independently emerged as two important desiderata required of modern machine learning models. While these two desiderata seem related, the connection between them is often unclear in practice.  Here, we discuss these connections through a causal lens, focusing on anti-causal prediction tasks, where the input to a classifier (e.g., an image) is assumed to be generated as a function of the target label and the protected attribute. By taking this perspective, we draw explicit connections between a common fairness criterion - separation - and a common notion of robustness - risk invariance.  These connections provide new motivation for applying the separation criterion in anticausal settings, and inform old discussions regarding fairness-performance tradeoffs. In addition, our findings suggest that robustness-motivated approaches can be used to enforce separation, and that they often work better in practice than methods designed to directly enforce separation. Using a medical dataset, we empirically validate our findings on the task of detecting pneumonia from X-rays, in a setting where differences in prevalence across sex groups motivates a fairness mitigation. Our findings highlight the importance of considering causal structure when choosing and enforcing fairness criteria.

Published in Transactions on Machine Learning Research (12/2022)
Fairness and robustness in anti-causal prediction
Maggic Makar
mmakar@uuicch..c edu
Computer Science and Engineering University of Michigan Ann Arbor, MT
Alexander D'Amour
dexdaamou44ggoggee.cmm
Google Research Cambridge, MA
Reviewed on OpenReview: https: // openreview. net/ forum? id= EhOUSGl
net/forum?
Abstract
Robustness to distribution shift and fairness have independently emerged as two important desiderata required of modern machine learning models. While these two desiderata seem related, the connection between them often unclear in practice. Here, we discuss these connections through causal lens, focusing on anti-causal prediction tasks, where the input to classifier (e.g., image) is assumed to be generated the protected attribute. By taking this perspective, we draw explicit connections between a common fairness criterion- sean---cppatiin-annle a common notion of robustness-rin invari- ance. These connections provide new molivation for applying the separation criterion in anticausal settings, and inform old discussions regarding fhrness-performane tradeoffs. In addition, our findings suggest that robustness-motivan separation, and that they often work better in practice than methods designed directly force separation. Using medical dataset, we empirically validate our findings the task of detecting pneumonia from X-rays, in setting where differences in prevalence across sex groups motivates a fairness mitigation. Our findings highlight the importance of considering causal structure when choosing and enforcing fairness criteria.
function of the target label and
approaches can be used to enforce
1 Introduction
In real world applications of machine learning, there strong desire enforce the intuitive criterion that models should not depend inappropriately sensitive attributes. Such desire is often formalized in terms of specific quantitative notions of fairness. Fairness criteria often focus on equalizing model behavior across different population subgroups defined by sensitive attribute. When designing fair systems, practitioners face a number of choices: among them are the choice of fairness criterion, and how to implement it. For example, a practitioner may wish to choose between simply using an unconstrained model that maximizes overall performance, a model that makes predictions independently of the sensitive groups (independence). or model that makes prodictions independently sensitive groups given the ground truth label (scparation) (see Barocas al. 2019 Chapter 2).
Here, show that the causal structure of problem can be useful piece of context for choosing and enforcing a fairness criterion in given application. We present this argument by making a connection to some recent insights on the relevance of causal structure to robust machine learning. In principle the connection between robustness and fairness should not be surprising. Similar to fairness methods, robustness methods also soek to regulate how classifiers depend on particular features; however, the details of the connection depend on the particular causal structure of the problem hand. In particular, we focus on an anti-causal prediction setting, where the input to classifier (e.g., image) is assumed to be generated function

---

Published in Transactions on Machine Learning Research (12/2022)
of the label and the sensitive attribute (see Figure 1). In this context, show that the connection between fairness and robustness can provide new motivations and methods for enforcing fairness criteria in practice.
Concretely, we use the example of detecting pneurnonia from chest X-rays as a motivating example, inspired by Jabbour et al. (2020). In this example, we assume that both the presence of pneumonia (Y) and the patient's s sex (V) causally influence the X-ray image (X) used to make the diagnosis. In this context, practitioners may aim to make diagnoses that respect certain fairness criteria across sex groups. As show, under the causal structure of this problem, some of these fairness criteria map neatly onto robustness criteria that seek to preclude the classifier from using image features influenced only by V.
x
Figure 1: Causal DAG of the setting in this paper. The main label Y and the sensitive attribute V gener- ate observed input X, but Y only affects X through the sufficient statistic X".
Informed by this causal structure, we provide a description of how fairness criteria and distributional nosust criteria align, and discuss practical impli- cations of this alignment for motivating and enforcing fairness criteria in this setting. Specifically, we focus on alignment between the separation fairness criterion-tlan that the distributions of predictions within the positive and nega- tive classes should be independent of the sensitive group group-and and risk invariance as a robustness criterion that the predictive risk of a model remain invariant across a family of distribution shifts. We show that:
The separation fairness criterion implies risk invariance across family of distribution shifts that change the basc rate of the label Y within sensitive groups V, but respoct the causal structure of the problem. This provides new perspective on discussions of perfosssperformmnce tradeoffs when applying the separation criterion in practice.
In practice, algorithms designed to enforce risk invariance also enforce the separation criterion, in some cases more effectively than algorithms that attempt to directly incorporate the separation criterion as a regularizer. We explore properties of one particular algorithm, presented in Makar et al. (2021) which uses a weighted regularizer based on the maximum mean discrepancy (MMD), and compare it to MMD-based implementation of an empirical separation criterion.
For contrast, we show conflict between the independence fairness criterion (often measured in terms of equalized predictions between V groups, or demographic parity) and the risk invariance property.
A number the connections that we describe here between fairness, robustness, and causal structure have been noted before in work aimed at improving robustness under input perturbations or distribution shift,, most notably Veitch et al. (2021) and Makar al. (2021). Our main contribution to draw out the practical implications of these results for fairness in a concrete setting. Our findings support paying increased attention to the causal structure of a problem to inform the choice and implementation of fairness criteria in practice, even if these metrics are on their face "oblivious", or independent of causal structure. This causal focus can have the advantage of highlighting connections to other desiderata and methodology.
2 Related work
Robustness to distribution shift and fairness are closely related, and many lines of work have aimed to highlight formal and empirical connections between them. For example, [aaaaaa et. al. (2019) explored applying distributionally robust optimization to address worst-subgroup performance for under-represented groups. As another example, Adragua et al. (2020) show that using methods meant to induce robustness leads to "more fair" classifiers for internet comment toxicity. Along similar lines, Pruksachatkun et al. (2021) found that certified robustness approaches designed to ensure robustness of NLP methods against word substitution attacks can be used to reduce violations to the equalized odds criterion.
A key perspective in our work that many In this sense, our work is most similar and complementary to Veitch et al. (2021), which the authors derive implications of counterfactual invariance to certain input perturbations, and show that these implications depend strongly on the causal structure of the problem at hand. Here, focus narrower setting,
relationships are mediated by causal structure.
2

---

Published in Transactions on Machine Learning Research (12/2022)
and highlight concrete conclusions about the relationship between easily measurable robustness and fairness metrics that are often used for evaluation in practice, with a greater focus on implications for fairness.
Notably, our use of causal ideas is distinct from another body of work that defines fairness criteria directly in terms of the causal model. These include definitions of fairness that revolve around direct causal effects of sensitive attributes on outcomes (Kilbertus al.) 2017: Nabi & Shpitser 2018 Zhang & Bareinboim, 2018), or discrepancies between counterfactual outcomes (Kusner et al. 2017]] Chiappa 2019). By contrast, focus "oblivious" fairness criteria that not themselves a function of causal structure (Hardt et al, 2016 but show that the causal structure of problem can still inform when and how to such criterion.
3 Background and preliminaries
Sctup We consider a supervised learning sctup where the task is to construct a predictor f(X) that predicts a label Y (e.g,, pneumonia) from an input X (e.g., chest X-ray). In addition, we have a protected attribute V (e.g. patient sex) available only at training time. Throughout, we will use capital letters to denote variables, and small letters to denote their value. Our training data consist of tuples D {(x,, yi, e:) drawn from source training distribution Ps. We restrict our focus to the case where Y and V are binary and f is a classifier. Specifically, we will consider functions f of the form f = h(o(x)), where & is a representation mapping and h is the final classifier.
In this context, a practitioner may be interested in ensuring that the classifier f(X) treats individuals from different groups fairly. Fairness is operationalized by enforcing constraints on how the distribution of model predictions can differ across individuals from different groups (that is, with different values of V) or individuals in different label classes (Chat is, with different values of Y).
We focus two well-established fairness criteria (see Barocas et al. 2019 Chapter 2 for full discussion).
While these criteria are typically defined with respect to the predicted class (i.c., Y= = 1{f(X) > 8) for some threshold 8) we consider stronger fairness notions defined with respect to the predicted probabilities S(X) (see " "Analogues with Scores" in Mitchell et al., 2021). This focuses the exposition on issues relating to the quality of f(X) independently of the choice of 6.
The first criterion, separation, requires that the distribution of predictions f(X) be the same across groups V conditional on the ground truth label Y, that is f(X) 1 V Y. Separation is often evaluated in terms of equalized odds (EO) (Hardt et al. 2016), which examines whether the conditional expectations of the predictions for each ground truth class Y are the same across sensitive groups V, i.e., that
(x,( | 0, Y y] Ex, 1, yl Vy E (0,1).
The second criterion, independence, requires that the distribution of predictions be the same overall across groups V. Independence is often evaluated in terms of demographic parity (DP), which examines whether the expected prodiction is the same across groups, i.c., that
Ep, (X() V = 0] = Ep, (/(X) V= =1]
Unlike EO which requires equality conditional Y, DP requires marginal form of equality.
In practice, separation and independence are often enforced at training time using empirical measures of distributional discrepancies, while the effectiveness of these interventions are often evaluated by measuring how closely the model respects the EO and DP critcria on test data.
Causal assumptions The separation and independence criteria are "oblivious" criteria: they are con- straints on the observable distributions of labels and predictions that do not depend on the form of the classifier or the mechanisms of the data generating process (Hardt et al. 2016). However, the implications of enforcing these criteria do depend on the causal structure of the problem. Here, we outline the causal assumptions that underlie the anti-causal prediction setting that we study, and show how connects to our running example of detecting pneumonia from chest. X-rays.
3

---

Published in Transactions on Machine Learning Research (12/2022)
We assume that P. has generative structure shown in Figure I in which the inputs X are generated by the labels (Y, V). We assume that the labels Y and V are are correlated, but not causally related; that is, an intervention on V does not imply a change in the distribution of Y, and vice versa. Such correlation often ariscs through the influence of an unobscrved third variable such as the environment from which the data is collected. We represent this in Figure | with the dashed bidirectional arrow.
In addition to the specific DAG, we assume that there is a sufficient statistic X such that Y only affects X through X", and X can be fully recovered from X via the function X' := e(X). However, we assume that the sufficient reduction e e(X) is unknown, we denote X as unobserved in Figure !
While anti-causal and sufficiency assumptions are somewhat strong, they are also reasonable in a number of important contexts where machine learning is applied. For example, the chest X-ray example plausibly satisfies these conditions. First, the disease (pneumonia) is the true cause of abnormal findings reflected in the chost X-ray. Sccondly, there are few interactions between the presentation of pucumonia and sex in the radiograph, so it is plausible to assume that the sufficient features of the X-ray that are influenced by pneumonia X' can be recovered from the input image X. More generally, these assumptions are likely to apply in settings where the goal is to recover pre-existing ground truth label Y from an input X, and Y and V to not interact substantially in generating X.
These assumptions are, of course, not universal. For example, in many prediction settings, the label Y is more plausibly caused by the input X. This situation is often referred to as a causal prediction setting, One example of a causal prediction task occurs when predicting the risk of cardiovascular discase (Y) from environmental risk factors (X) in settings where sensitive attribute affects those environmental risk factors. In this case, Y could be function of blood pressure (X"), which is affected by environmental risk factors, which are in turn affected by the the sensitive attribute. In this setting, the relationship between fairness and robustness criteria differs substantially from the relationships we outline below (see discussion in Section 4.2) Similarly, in many anti-causal prediction settings, there is substantial interaction between the sensitive group V and the target label Y in generating the input X. For example, if Y represents a heart condition, V the patient'ss 's age, and X waveform from an electrocadiogram (ECG), the presentation of heart conditions in an ECG are very different depending on whether the patient is a child or an adult. In this case, our charactcrization of the optimal risk invariant predictor (introduced below) would be different.
Finally, when we discuss the properties of learning algorithms, we make an overlap assumption on the source distribution, P.. Specifically we assume that the support of P.(V)).(Y) is contained in the support of Pv(V,Y). Intuitively, this assumption implics that we observe all combinations of Y and V during training time that will also appear any larget test distribution. Absent such an assumption, the behavior of ʃ on unobserved combinations is unlearnable using the observed data.
3.1 Robustness as Risk Invariance under Shifts in Dependence
Here, review some robustness criteria that have been introduced before in our anti-causal setting, with the goal of drawing formal connection to fairness criteria. Robustness criteria and faimess criteria often have distinct motivations. While fairness is often motivated by a principle of non-discrinination, robust or invariant predictive methods are often motivated by generalization to out-of-distribution settings.
Of particular interest here are scenarios where a "shortcut" " association between inputs and target label changes between training and deployment time (Geirhos et al. 2020). A canonical example of shortcut learning is presented in the context of classifying animals from images in Beery al. 2018)) where classifiers often failed when animals in the test set appeared in new and unusual locations: for example, cows, which frequently appeared in grassy locations in the training set, were misclassified when they appeared on beach backgrounds in the test set. Whilc any model might be prone to shorteut lcarning, recent empirical findings have shown that deep neural networks are especially prone to relying on shortcuts (Beery et al. 2018 Sagawa
et al.) 2019 2020 Ilyas et al. 2019 Azulay & Weiss 2018 Geirhos et al., 2018 D'Amour et al. 2020). A
flurry of recent papers suggest methods create robust models that give predictions that are invariant to the
spurious correlations or shorteuts (Sagawa et al. 2019 Creager et al. 2021 Arjovsky et al.
2019 Krueger
et al.
2021
4

---

Published in Transactions on Machine Learning Research (12/2022)
In this paper, we focus particular notion of robustness called risk invariance. Following the robustness literature, we assume that the model f is trained a source distribution Ps. and measure its predictive risk on one or many target distributions Pt. We write the generalization risk of a function f on distribution P as Rp = Ex,y ( that follow). We say that a model f is risk invariant across a family of distributions P its predictive risk is the same for each target distribution Pt in that family. Here, we consider families of target distributions that can be generated from Pa by interventions on the causal DAG in Figure [ Specifically, we consider interventions on the dependence between Y and V that keep the marginal distribution of Y constant. Each distribution in this family can be obtained by replacing the source conditional distribution P5(V (Y) with a target conditional distribution Pt(V] | (Y):
where is a predictive loss (we define this as the logistic loss in arguments
P = (P.(X X (XP (YP((Y) (Y)),
(1)
This family allows the marginal dependence between and V to change arbitrarily.
For our analysis, one distribution contained in the set P will be important: the distribution where Y TL V. i.e., := Px(X X*, (X* (V)) We refer to po as the idealized distribution. In the chest x-ray example, P is the distribution where a pneumonia patient is equally likely to be female or male, and similarly a healthy patient is equally likely to be female or male. We formally define this notion of robustness next.
Definition 1 (Risk Invariance). Given the family P defined in equation 1 we say that a set of pre- dictors FRob is risk invariant if they have the same risk for all P. E P, i.e., if FRob = {f (f) = (J) PY P).
Risk invariance arises naturally as a criterion for checking whether a model's prediction depends on "shortcut" features in X that are influenced by V but not by Y. In our setting, any predictor that does not rely on such shorteuts will necessarily be risk invariant (Makar et al. 2021 Veitch al., 2021).
4 Fairness and robustness in the anti-causal setting
In this section, we discuss conmections between fairness and robustness that are specific to our anti-causal setting. These connections inform considerations about how practitioners may wish to choose between using an unconstrained model, or enforcing separation or independence. In particular, we show that there is substantial alignment between risk invariance and separation in this setting, which can provide motivation for preferring separation over unconstrained model model that enforces independence.
4.1 Separation and Predictive Performance
Often the most difficult choice in dociding to apply fairness criteria. to a classification problem is whether to constrain the model at all. Some of the oldest discussions in applying fairness to machine learning center on tradeoffs between parities enforced by coistase models and overall predictive performance of unconstrained models (see, e.g., Coobett-DDavess et al.) 2017 Mitchell et al., 2021). Here, we revisit this discussion in our anti-causal setting, and show that separation can be motivated on purely performance- oriented grounds if the notion of performance is expanded to include predictive risk under distribut ion shifts. In addition, we show that the optimal risk invariant predictor in this setting satisfies separation, suggesting that algorithms that target the optimal risk invariant predictor can be effective for learning models that are fair according to the separation criterion.
Our discussion here revolves around two results. First, in the anti-causal setting in Figure D separation implies risk invariance. Secondly, the optimal risk invariant predictor in this setting satisfics separation. We present these two results formally in turn, then discuss their implications. We include the proofs of these
"his notion of risk invariance could be generalized to include would introduce some notational overhead. would require that re-weighted risk be invariant across such family. 2 See also, discussions that pre-date machine learning, such as discussions of fairness in education assessment reviewed in Hutchinson Mitchell 20119.
where the marginal distribution of Y also changes, but
5

---

Published in Transactions on Machine Learning Research (12/2022)
statements to highlight how the results depend on our core causal assumptions (anti-causal structure and the sufficiency of )*)
Proposition 1. In the anti-causal setting shown in Figure 1 suppose that predictor f satisfies seporation in the training distribution, that is, f(X) LL P.V Y. Then f is risk-ineariant with respect to the family of target distributions P defined in
Proof. We show this by docomposing the risk of the model f on any target distribution Pt E P in terms of the conditional risk given V and Y. Within the family P, P.(e(J(X),)) Y V. is the same for all P: E P; thus, we can write the risk conditional on Y and V independently of the target distribution. Let Rey == Ep, ((((())) Y) V = u, Y = yl for any P. E P. The overall risk on a target distribution P. can be written as the weighted average of these subgroup risks
R
Σ PA = v) 01 Y = DE(0,1)
=Y=))Rop Roy + P.(V = Y 1 Rigl
(2)
Now, the separation criterion states that f(X) 1 V Y, which implies ((f(X),)) 1T V Y, which implies that Roy Rig for y E (0, 13. Thus, the terms in the summation (2) are the same regardless of the = - Y = y) factors, and thus the risk is invariant for all E.E.P
Proposition 2. In the anti-causal setting shown in Figure 7 (a) the optimal risk invariant predictor with respect to P has the form f² B|Y X*) F and (b), this predictor salisfies seporation.
Proof. Part (a) is shown as Proposition in Makar et al. (0021). The key points of the proof are that (1) E[Y X*] is representable by function (XX) under the assumptions made in Section 3 (namely, the assumption that X" can be written as e(X); (2) under the uncorrelated distribution P E X*] is Bayes
optimal; and (3) B[Y X*] is risk invariant. (2) and (3) imply that no other risk invariant predictor can have lower risk across P.
Part (b) follows from the fact that X is d-separated from conditional on Y in the DAG in Figure Thus, E[Y 1
Y for all P. E P.
Remark 1. Veitch ct al. (2021) also provide a number of related results in their study of the implications of counterfactual invariance in anti-causal settings. Counterfactual invariance requires that the predictions of a model be invariant across preserrrss counterfactual versions of the input, such as the counterfactual that we would observe if the sensitive attribute were different. Our findings concern narrower case where there exists a sufficient statistic X", which Veitch et al. (2021) refer to this as the "purely spurious" case. In their findings, Veitch et al. (2021) show that counterfactual invariance implies the separation criterion generally in anticausal scttings, and that the optimal counterfactually invariant predictor in also minimax optimal in the purely spurious case. Here, our results speak more directly to the implications of fairness criteria in the purely spurious setting, and, by focusing on the weaker risk invariance property, we make the connection without the conceptual ambiguity of defining counterfactuals with respect to sensitive attributes (see, e.g. Hausmann 2018 for discussion of this point).
These results have several implications in the motivation and practical application of the separation criterion. tradeoffs. In particular,
First, they provide a counterpoint to standard discussions of they highlight the importance of specifying the distribution (or distrributions) on which fairness and perfor- mance are definod. On one hand, in our sctting, the Bayes optimal in-distribution predictor, Ep, [Y] X), does not in general satisfy separation (this follows because X is not d-separated from V, so EE X] is not inde- pendent of V). This implies that model satisfying separation must have thwrr-than-optimal in in-distribution accuracy. On the other hand, because Proposition 1 shows that separation implics risk invariance, the per- formance of a model that satisfies separation cannot degrade if the model is deployed in a causally consistent scenario included in P. And in fact, as Proposition 2 shows, the best possible risk invariant model satis- fies separation, implying that there is no tradeoff between optimal invariant performance and this fairness critcrion. Thus, if model performance boyond the training distribution P, is important in an application,
Note that F. (V| | X*] the same for all P. P. s0 we omit. the distribuutonal subscript on this expectation.

---

Published in Transactions on Machine Learning Research (12/2022)
risk invariance can provide purely performanco-orientee motivation for enforcing separation criterion. We note in passing that since separation and sulficiency are at odds, risk invariant models in the anticausal settings will not satisfy sufficiency.
2 suggests that, in our anti-causal setting, criteria designed for targeting optimal risk invariant predictors may be effective for learning classifiers that satisfy separation, even if they do not imply separation directly. For example, consider the approach proposed in Makar et al. (2021) that targets the optimal risk invariant predictor by enforcing an ideal distribution independence (IDI) criterion.
Secondly, Proposition
Definition 2 (Ideal Distribution Independence). model f satisfies IDI iff f(X) 1 poV; i.e., if its predictions (XX are independent of the sensitive attribute V under the ideal distribution Po, where Y IL p.
While the IDI critcrion does not itself imply separation, Makar et al. (2021) show that enforcing IDI at training time can lead to more efficient recovery of the optimal risk invariant predietor, which does satisfy separation. This suggests that enforcing risk invariance in practice also help to close fairness gaps. We demonstrate this empirically in our chest X-ray application presented in section G
4.2 Separation versus Independence
It useful note that considering the robustness properties of predictor in its particular causal setting also distinguishes between different fairness criteria. In particular, in non-trivial instances of our anti-causal setting, there is conflict between the optimal risk invariant predictor and independence.
Proposition 3. Let (x = P. In addition, assume that E[S(X) = y] is non-trivial function of y, i.e., that the value of Y actually affects the expectation of the sufficient statistic X*. Then for any distribution P. & P in P, (X) & V and independence is not satisfied.
ElY x*) be the optimal risk invariant predictor with respect to
Proof. Note that for each v, (x) | V =] E (X) Y v)P:(Y = -vv For any Pt where Y 1 V, the weights P.(Y = = V in this summation will differ as a function of v. By assumption, the expectations (X) Y = v] differ for different values of y, so changing their weights in the summation will change the sum. Thus, for + v', Ep V u| Ep,lf V = ='],, which implics (X) V.
=
If downstream performance of a model on shifted distributions is an important consideration in an applica- tion, this result provides motivation against using independence fairness criterion, because it implies tradeoff with the best achievable invariant risk. We demonstrate the practical implications of this tradeoff in our chest X-ray example in section
Importantly, we note that this conflict between the risk of a model on downstream applications and indepen- dence can change depending on the causal structure of the problem. In particular, Veitch et al. (2021) show that in some causal prediction scttings, where V causes X, but X causes Y, independence is compatible with the optimal predictor that satisfies related robustness criterion that they call counterfactual invari- ance (Veitch et al. 20211, see Theorem 4.2). This highlights the importance of considering causal structure when deriving the implications of enforcing particular fairness criteria in a given application. However, we leave further discussion of the causal prediction scenario, and the related implications for independence and counterfactual invariance criteria for future work.
5 Enforcing Separation
The discussion in section 4 implics that there are two ways to enforce scparation during training. First, it can be enforced directly by encouraging equality between representation distributions conditional on Y. Alternatively, as shown in section 4 it can be enforced indirectly by minimizing predictive risk subject to the IDI critcrion, by encouraging equality between the marginal distributions of Iearned representations & under the ideal distribution P In this section, we discuss the practical considerations around different implementation schemes and highlight that learning algorithms that enforce separation directly face distinct
7

---

Published in Transactions on Machine Learning Research (12/2022)
technical challenges from those that encourage separation through enforcing the IDI robustness criterion. Because of this distinction, as we show in Section B the IDI-motivated algorithm hm can achieve better separation in practice.
We center this discussion on a specific family of approaches that relies on estimates of distributional discrep- ancies to enforce statistical independences. Specifically, we focus the Maximum Mean Discrepancy (MMD) since it is a popular choice in the fairness literature (Prost & Qian 2019 Madras al., 2018 Louizos al.
2015), and the robustness literature (Makar et al., 2021 Guo al. 2021p. The MMD defined as follows:
Definition 3. Let Z Pz, and Z' Pz', be two arbilrary variables. And let Q be class of funclions w Z 1 + R, MMD(22, Pz, Pz.) sup (Epzw(Z) -EPz,w(Z')),
When Z is set to be a general reproducing kernel Hilbert space (RKHS), the MMD defines a metric probability distributions, and is equal to zero if and only if P2 Pz.. We take Q to be the RKHS and drop it from our notation. MMD-based regularization ion methods enforce statistical independences at training time by penalizing discrepancies between distributions that would be equal the independence held. The MMD penalty can be applied to the final layer f or to the learned representation Φ. Both methods induce the required invariances. We follow the literature in imposing the penalty on the representation ø.
The strategy for directly enforcing separation penalizes discrepancies in representation distributions condi- tional Y Prost et al. 2019). Such strategy is translated to learning objective as follows:
2 hC-wDD PC-NND argminn Σ ((b(o(x:)) vi) + Σ, MMD (Poo..' Poin)
where Po.. (()(X)IV = =. y), a is parameter picked through cross-validation MMD is an estimate of MMD2. In the experiments below, we use the V-statistic estimator for the MMD presented in Gretton let al. (012)). This estimator relies on the radial basis function (RBF), which requires a bandwidth parameter y picked through cross-validation.
While this strategy is forward, it has some practical limitations, especially when training using mini-batches of data in stochastic gradient descent. Within each batch, C-MMD requires first dividing the population based on Y then estimating the MMD within each subgroup. This effectively reduces the sample size used for MMD estimation, making the estimates more volatile and less reliable, especially when batch sizes are small.
Meanwhile, a strategy for enforcing separation indirectly through IDI, developed by Makar et al. 2022] makes use of weighted marginal MMD discrepancy (WM-MMD). WM-MMD requires marginal estimate of the MMD computed with respect to p rather than the observed Ps. This strategy uses reweighting to manipulate the observed data such that it mimics desirable independencies in ps. Specifically, it uses the P_(Y=y)P,(V==)) following weights: u(y,v) v) P. =o, such that for each example, u4 u(y,, v.). This weighting scheme maps expectations under Ps to expectations under pe. The final objective becomes:
fuw--MDD PWR-HMD argming, Σn ((((h((())))))) + a- MMD (Pag Pay),
where MMD (Pag. ) a weighted estimate of the MMD. This strategy also has practical challenges. While WM-MMD does not require this data-slicing, if base rates are too skewed within groups, the weights may be highly variable, and introduce volatility into the regularization. This is a typical risk with weighted estimators (see, for example (Cortes et al. 2010
Ultimately, the better strategy to employ to enforce scparation depends on the context. For example, we find that WM-MMD is far more stable in our experiments on chest X-rays, and that the instability due to data-slicing induced by C-MMD leads it to exhibit a fairness- perress-prffommacce tradeoff in practice that WM- MMD is less prone to. This highlights the value of expanding the toolbox of fairness regularizers include regularizers targeted at causally compatible robustness criteria. For more general use, we present concrete heuristic to choose between the two methods in given application.
8

---

Published in Transactions on Machine Learning Research (12/2022)
By comparison to those two approaches, approach that encourages independence rather than separation can be implemented by penalizing the unweighted prediction loss and the unweighted marginal MMD as follows:
"En ((h(6(x))). ta MMD (Po.: P..).
hy- PH-2ND = argmin n i=
(5)
6 Experiments
In this section, we empirically demonstrate the implications of the relationships between fairness criteria and robustness criteria in an anti-causal setting. In this demonstration, we show that several of the arguments made at the population level in Section | bear out in a practical application. Specifically, following Jabbour et al. (2020), we consider the task of predicting pueumonia (Y) from chest x-rays (X) considering sex to be protected altribute (V). In this context, we demonstrate the close practical connection ween separation (as measured with EO) and risk invariance. First, models that better satisfy EO exhibit greater risk invariance (Proposition D. Secondly, we show that training procedures that target risk invariance can yield models that satisfy EO (Proposition 2) and somewhat surprisingly, they perform better than models that explicitly trained to satisfy separation. Third, no such alignment holds between independence (as measured with DP) and robustness (Proposition 3
Data and Test Distribution Design. We conduct this analysis a publicly available dataset, CheXpert [Irvin et al.) 2019). The data contain 224,316 chest x-rays of 65,240 patients. Each chest x-ray is associated with 12 labels corresponding to different pulmonary conditions. Each label encodes if the corresponding condition is confidently present, confidently absent or might be present with some uncertainty. We select patients who have pncumonia (confidently present), or have no finding (i.c., all conditions are confidently absent). For the former group, we give them the label pneumonia (i.e., y/ = 1), for the latter group we give them the label "healthy" (i.e., y = 0). We exclude patients who do not have pneumonia but might have other pulmonary conditions from our analysis since other conditions such as Atelectasis or Lung Edema can be visually similar to pncumonia, making them hard to distinguish from pncumonia without additional information (e.g,, medical charts). This leaves us with a lotal of 21,232 unique x-ray studies. We split the dataset into 70% examples used for training and validation, while the rest is held out for testing.
At training time, we undersample patients recorded as women who did not have pncumonia to create sctting where a naive predictor might have systematic errors for the sampled group. Specifically, we sample the data such that 30% of the population has pneumonia, i.c., P& = 1) = 0.3, and the majority of women patients have pneumonia whereas the majority of men patients do not have pneumonia i.e., P.(V = 1|Y 1) = PV 0|Y 0) 0.9. In this setting, we show later in figure 2 clep neural network trained in the usual way learns to rely on sex of the patient to predict pneumonia.
Here, the family of target distributions is the family of distributions that are compatible with the DAG in Figure [ where the base rates of pneumonia were systematically skewed between sex groups by selective sampling. This set includes all distributions where Pt(V = Y = y) is allowed to take on any value between and for each y. To test our models, we generated test distributions from this family. We denote these test distributions P.- (Pa.1, Po.3; Po.s: Po.7, Po.9, Po.95), where Pμ. is generated such that Pa(V 1 Y = 1) P.V 0| Y 0) μ with P. = 1) held constant.
Implementation. We use DensNet-121 (Huang et al. 2017), pretrained on ImageNet, and fine tuned for our specific task. We use DenseNet because it was shown to outperform other commonly used architectures on the CheXpert dataset (Irvin et al. 2019 Ke et al., 2021 Jabbour et al., 2020 All models are implemented in TensorFlow (Abadi et al.) 2015). We train all models using a batch size of 64, and image sizes 256 x 256 for 50 epochs. We used publicly available code provided by the authors in Makar et al. (2021
In addition to C-MMD, WM-MMD described in scction 5 we implement deep neural network (DNN) without any robustness or fairness penalties. We note that C-MMD is the same as MinDiff, which is the approach to enforce EO in Tensorfow
httt::///gthuub.ccm///a/aaaa///e wtwp://wwwwwallloootttttttteete motivated shortcut reeoova removal
remediation

---

Published in Transactions on Machine Learning Research (12/2022)
For the three MMD based models, need to pick the free parameter which controls how strictly we enforce the MMD penalty, and y, which is the kernel bandwidth needed to compute the MMD. We follow the cross-validation procedure outlined in Makar et al. (2021). Specifically, we split the training and validation data into 75% for training and 25% for validation. We further split the validation data into 5 "folds". We compute the MMD on each of the folds. The MMD is computed the same way as training, meaning for the conditional MMD penalty, we compute the conditional MMD on each of the folds, and similarly for the weighted marginal MMD, and the marginal MMD. We then compute the mean and standard deviation of the MMD estimate over the 5 folds. Using T-test, we test if the mean MMD is statistically significantly different from zero. We exclude hyperparameters that yield non-zero MMD. Out of the remaining hyperparameters we pick the model that has the best performance, i.e., the lowest logistic loss. For the DNN, we perform I2-regularivation. We pick the regularization parameter based on the validation loss. Additional details about hyperparameters are included in the appendix.
6.1 Results
Sex as a shortcut in pneumonia prediction In our first set of results, we examine the extent to which each model utilizes information about sex to detect pncumonia. To do this, we mcasure the performance of each model across our set of shifted test sets Pt. Models that encode information about sex to predict pucumonia (i.c., use sex as a short- cut) will experience a degradation in performance test distributions where the marginal correlation between label Y and sex V has changed. Meanwhile, model that does not use this shortcut should have invariant performance across these test sets.
0.9
I
0.8-
AUROC
0.7-
DNN + C-MMD I M-MMD
0.6
0.5-
+ WM-MMD
0.4-
I
For each of the distributions in Pt, we compute the the area under the receiver operating curve (AU- ROC) of each predictor on each of the test distri- butions as mcasure of its performance across dis- tribution shifts. To estimate the uncertainty in the AUROC, we create 1000 bootstraps of the test set, and compute the means and standard deviations of the AUROC.
0.2
0.4
0.8
P(V = 1 Y 1) P(V oJY 0) at test time
Figure 2: Test distribution the x-axis, and perfor- mance measured by AUROC on the y-axis. Dashed line shows training distribution. DNN, which does not incorporate any fairness/ risk invariance criteria has the best in-distribution performance but its perfor- mance quickly deteriorates across distribution shifts. M-MMD over-penalizes the correlation between Y, V. While WM-MMD achieves worse in-distribution per- formance compared to DNN and C-MMD, it has has better out of distribution performance for distributions most dissimilar to the training distribution.
Figure 2 shows the results of this analysis. The x axis shows P(V = 1 Y 1) P(V = 0 Y 0) at test time, while the y-axis shows the corre- sponding mean AUROC. The vertical dashed line shows the conditional probability at training time. Notably, DNN, which does not incorporate any fair-
ness/ risk invariance criteria has the most severe dependence on the shorteut, and by relying on sex as a shortcut, achieves the best in-distribution performance. However, such performance quickly deteriorates on test distributions where the strength of the shortcut association is smaller or even "flipped". While C- MMD should not, in theory, encode information about sex, we see that its practical implementation yields some dependence sex, albeit less severe than that of the DNN, As we show later in this section, this somewhat surprising behaviour of C-MMD is because of its poor finite sample properties, which arise due to data-slicing. Meanwhile, M-MMD uses the shortcut to satisly the fairness criterion, which leads to en- coding an opposite dependence on sex. Finally, while WM-MMD achieves worse in-distribution performance compared to DNN and C-MMD, it has has better out of distribution performance for distributions most dissimilar to the training distribution. This suggests that WM-MMD has the least dependence on sex as a shortcut compared to the other predictors.
10

---

Published in Transactions on Machine Learning Research (12/2022)
Alignment between fairness and robustness in anti-causal settings. Here, we examine the extent to which different fairness criteria jeeparatioon/nndeeedeene and their implementation align with robustness across distributions.
We empirically measure robustness by computing:
Robustness = max Rp, (f) min Rp, (f)l. PAEPi
PAEP
where Rp, is the logistic loss achieved on the distribution Pμ E P,. Lower values imply higher robustness. We measured the violations to the EO and DP criteria by generating a test set that is drawn from the same distribution as the training set, and computing:
EO violation max Ep, (f()) y, V 1] E = y. O]], and
y
DP violation = [Er] = -1]- (x. O|/-
Figure (left, middle) show our mcasure of robustness on the x-axis. Each subplot shows a different fairness criterion on the y-axis. To estimate the uncertainty in the fairness and robustness metrics, we create 1000 bootstraps of the test set, and compute the means and standard deviations of each of the metrics. Both subplots confirm that the DNN which does not incorporate any fairness/robustare information about sex in the learned representation, and is neither robust nor fair by any metric of fairness. In addition, the results confirm our analysis in section tracks with less violations to the EO criterion in figure 3 (left). The same is not true for the DP criterion in figure 3 (middle): M-MMD achieves lower violations to the DP criterion compared to WM-MMD yet the former is less robust than the latter. While both C-MMD and WM-MMD are expected to perform favorably, we see that WM-MMD consistently performs better on robustness and fairness. We revisit this comparison in dotail below.
penalties indeed encodes
see that better robustness (i.e., risk invariance)
Risk invariance and its empirical relationship to EO and DP. Here, seek to empirically validate the analysis presented in section g Recall from Definition [ that our robustness criterion, risk invariance, requires that a model achieve the same risk (i.e., the same logistic loss) on test sets sampled according to different distributions with varying correlation between pneumonia and sex. To measure risk invariance empirically, we computed the logistic loss on each the test distributions in the set Pe (defined above), and measured risk invariance by computing:
Robustness = max Rp, (f) min P,EP,
(/)+
P.EP.
where Rr, the logistic loss on the distribution Pp. Lower values imply higher robustness. We also mcasured the violations the EO and DP criteria by generating a test set that is drawn from the same distribution as the training set, and computing:
Ep, ((x) y, V 1| (S()) = v. =o/. and
EO violation
DP violation = Er, = - = of|
Figure 3 shows our measure of robustness the x-axis. Each subplot shows different fairness criterion the y-axis. The results also presented in a table format in the appendix. We estimate the uncertainty in the fairness and robustness metrics by bootstrapping the test set, as described before. All subplots confirm that the DNN which does not incorporate any about sex in the learned representation, and is neither robust nor fair by any metric fairness. In addition, the results confirm our analysis in section we see that better robustness (i.c., risk invariance) tracks with less violations to the EO criterion in figure 3 (left). The same not true for the DP criterion in figure 3 (right): M-MMD achieves lower violations to the DP criterion compared to WM-MMD yet the former is less
penalties indeed encodes information
11

---

Published in Transactions on Machine Learning Research (12/2022)
robust than the latter. We note that despite the fact that the robustness of M-MMD and WM-MMD seems overlapping, a simple two sided ttest rejects the null (that the robustnesss of the two models is the same) with p-value < 0.001
While both C-MMD and WM-MMD are expected to perform favorably, we see that WM-MMD consistently achieves better on robustness and fairness suggesting that the latter is more stable. We will revisit this comparison in detail later.
Violations BO oriterion
Violations oritorion
0.5
0.4
0.4
0.3
0.3
+
DNN
0.2
C-MMD
0.1
M-MMD WM-MMD
0.1
0.0
0.0
0.0
0.2
0.4
0.6
0.8
1.0
1.2
0.0
0.2
0.4
0.6
0.8
1.0
1.2
Most robust
Robustancs
+ Leust robust
Must robust
Robustraes
Leasst robust
Figure Robustness on the x-axis, and different fairness criteria on the y-axis. Left plot shows violations to EO. Right plot shows violations to DP. Models with higher robustness achieve lower fairness violations for EO, but not DP which is consistent with the analysis in section 4 The simple DNN relies sex shorteut achieving the worst robustness and fairness properties. WM-MMD outperforms C-MMD achieving better robustness and lower EO violations.
WM-MMD is more stable than C-MMD. We now investigate the surprising discrepancy in per- formance between WM-MMD and C-MMD: while in theory they both target risk invariance, empirical performance suggests that WM-MMD is more effective.
This discrepancy can be explained, in part, by how well each optimized optimized MMD criterion gen- cralizes to test data. Figure g shows the estimated MMD on the training, validation and lesting data broken down by the target label. Here the testing data is sampled from the same distribution as the training data to highlight estimation error rather than errors due to data shift. The plot shows several important findings. First, the estimated MMD at training time is a more reliable estimate of the test- set MMD when weighted marginal MMD penalties are used training and validation time, signaling that data slicing in C-MMD leads to unreliable es- timates. Second, the difference between the MMD among the group defined by Y = compared to the group defined by Y = O is smaller when using WM- MMD. Smaller difference in the MMD between the two groups is important to ensure that the outcomes for both groups defined by the target label do not vary based on the protected attribute.
Valiatios
n CAMDD
WMM MMND
Figure 4: Estimated conditional MMD on the train- ing, validation and testing data for WM-MMD and C-MMD. Estimates of the MMD are more stable for the WM-MMD objective than C-MMD.
This analysis can be repeated in practice choose between the two penalties: the estimated MMD on the validation data reliable proxy for the estimate's generalizability. In practice, if there is large discrepancy between the training and validation C-MMD estimates, WM-MMD might be better option and vice versa.
12

---

Published in Transactions on Machine Learning Research (12/2022)
7 Discussion
In this work, we showed that by taking into account the specific causal structure of a prediction problem, we are able to draw deep connections between notions of robustness and fairness. We established a practical near-equivalence between separation and risk-invariance in an anti-causal prediction setting. Specifically, in one direction, separation implics risk invariance, while in the other direction, algorithms that are used to obtain performant risk invariant prodictors yield predictors that approximately satisfy separation. This connection provides performaace-orientel motivation for applying the separation criterion in an important class of problems, and provides new set of tools, borrowed from the robustness literature, to enforce the criterion in practice.
More generally, our findings demonstrate that the understanding causal structure of a given problem can yield useful practical insights, both for understanding how different modeling constraints impact the real world behavior of models, and for implementing more sample efficient models that comply with practitioners' stated goals. As models are deployed in more complex settings, causality can be a useful tool for motivating, expressing, and satisfying ever-more detailed practitioner requirements.
Broader Impact Statement
This work aimed to help practitioners and researchers reason about one aspect of the broader impact of ML models, particularly those that are constrained to satisfy certain statistical parities. This work could provides motivation for enforcing parities like separation when there is reason to believe that the causal assumptions hold. However, this motivation is necessarily incomplete, and should serve one of many different factors when considering how to manage the broader impact of an MT. system. In particular, fairness and other socially salient goals should be determined based on the context of the application, including how the model influences real decision processing, the potential for harm, and how tradcoffs interact with existing social disparities.
Acknowledgments
We are thank the anonymous reviewers, and Victor Veitch for their feedback. This work was partially funded by NSF grant No. 2153083. Any opinions, findings, and conclusions or recommendationn expressed in this material are those of the author(s) and do not necessarily reflect the views the National Science Foundation.
References
Martin Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Cor- rado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Trving, Michael Isard, Yangqing Jia, Rafal Jozefowiez, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: Largo-scale machine learning on hetcrogencous systems, 2015. URL https://wwi: tensorfloo. org/ Software available from tensorflow.org,
Robert Adragna, Elliot Creager, David Madras, and Richard Zemel. Fairness and robustness in invariant learning: A case study in toxicity classification. arXiv preprint arXiv:2011.006,5,, 2020.
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXie:1907.02880 2019.
Aharon Azulay and Yair Weiss. Why do deep convolutional networks generalize poorly to small image transformations? arXiv preprinl ar Xrv: 100.1805.1211 12177, 2018
Solon Barocas, Moritz Hardt, and Arvind Narayanan. Fairness and Machine Learning. fairmlbook.org, 2019. http://aww.fairmlbboo.
13

---

Published in Transactions on Machine Learning Research (12/2022)
Sara Beery, Grant Van Horn, and Pietro Perona. Recognition in terra incognita. In Proceedings of the European Conference on Compuler Vision (ECCV), pp. 456-473, 2018.
Silvia Chiappa. Path-specific counterfactual fairness. In Proceedings of the AAAT Conference on Artificial Intelligence, volume 33, pp. 7801 7808, 2019.
Sam Corbott-Davics, Emma Pierson, Avi Feller, Sharad Gool, and Aziz Huq. Algorithmic decision making and the cost. of fairness. In Proceedings of the 23rd aem sigkdd internalional conference on knowledge discovery and data mining, pp. 797-806, 2017.
Corinna Cortes, Yishay Mansour, and Mchryar Mohri. Learning bounds for importance weighting. Tn Nips, volume 10, pp. 442 450. Citeseer, 2010.
Elliot Creager, Hoern-Henrik Jacobsen, and Richard Zemel. Environment inference for invariant learning, In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 2189 2200. PMLR, 18 24 Jul 2021. URL htttp::/procceddngg... mlr http://www....rsssseeeeeee hteess/vv3//eee ppress//133/creagerrl.... creager21a..eeeeeeeeee
Alexander D'Amour Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Dealon, Jacob Eisenstein, Malthew D Hoffman, et al. Underspecificatio presents chal- lenges for credibility in modern machine learning. arXiv preprint or arxiv:201 1. 1.03395, 2020.
Robert Geirhos, Carlos RM Temme, Jonas Rauber, Heiko H Schütt, Matthias Bethge, and Felix A Wich- mann. Generalisation in humans and deep neural networks. In Advances in neural information processing systems, pp. 7538-7550, 2018.
Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Malthias Bethge, and Felix A Wichmann. Shortcut learning in deep neural networks. arXiv preprint arXiv:2004.07990 2020.
Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. kernel two-sample test. The Journal of Machine Learning Research, 13(1):723-773, 2012.
Ruocheng Guo, Pengchuan Zhang, Hao Liu, and Emre Kiciman. Out-of-distribution prediction with invariant risk minimization: The limitation and an effective fix. arXiv preprint arXim:2101.07733 2021.
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. Advances in neuraa information processing systems, 29:3315- 19:3315-3323, 2016.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Denscly connocted convolu- tional networks. In Proceedings of the IEFE conference on computer vision and pallern recognilion, pp. 4700-4708, 2017.
Ben Hutchinson and Margaret Mitchell. 50 years of test (un) fairness: Lessons for maching learning. In Proceedings of the Conference on Foirness, Accountability, and Transparency, pp. 49 58, 2019.
Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. In Advances Neural Information Processing Systems, pp. 125 136, 2019.
Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-llcus, Chris Chute, Henrik Mark- lund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In Proceedings of the AAAI conference on artificial intel- ligence, volume 33, pp. 590-597, 2019.
Sarah Jabbour, David Fouhcy, Ella Kazerooni, Michael W Sjoding, and Jenna Wiens. Deep learning applied chest x-rays: Exploiting and preventing shorteuts. In Machine Learning for Healthcare Conference, pp. -82. PMLRR 2020.
14

---

Published in Transactions on Machine Learning Research (12/2022)
Alexander Ke, William Ellsworth, Oishi Banerjee, Andrew Y Ng, and Pranav Rajpurkar. Chextransfer: performance and parameter efficiency of imagenet models for chest x-ray interpretation. In Proceedings of the Conference on Health, Inference, and Learning, pp. 116 124, 2021.
Niki Kilbertus, Mateo Rojas- Carulla, Giambattista Parascandolo, Moritz Hardt, Dominik Janzing, and Bern- hard Schölkopf. Avoiding discrimination through causal reasoning. In Proceedings of the 31st International Conference Newral Informalion Processing Systems, pp. 556-666- 2017.
Issa Gohler-Hausmann. Eddic murphy and the dangers of counterfactual causal thinking about detecting racial discrimination. No. UL Rev., 113:1163, 2018.
David Krueger, Ethan Caballero, Joern-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of-distrbuution generalization via risk extrapolation (rex). In International Conference on Machine Learning, pp. 5815-5826. PMLR, 2021.
Matt J Kusner, Joshua R Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. arXiv preprint arXiv:1703, 11703.00866, 2017.
Christos Louizos, Kevin Swersky, Yujia Li, Max Welling, and Richard Zemel. The variational fair autoen- coder. arXiv preprinl 0Xix:1511.00830 2015.
David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. Learning adversarially fair and trans- ferable representations. arXiv preprint arXiv:1802.00 06309, 2018
Maggie Makar, Ben Packer, Dan Moldovan, Davis Blalock, Yoni Halpern, and Alexander D'Amour. Causally- motivated shortcut removal using auxiliary labels. arXiv preprint arXiv:2105. 200,:2620,,,2, 2021.
Shira Mitchell, Eric Potash, Solon Barocas, Alexander D' D'Amuur, and Kristian Lum. Algorithmic fairness: Choices, assumptions, and definitions. Annual Review of Statistics and Its Application, 8:141-163, 2021.
Razich Nabi and Ilya Shpitser. Fair inference on outcomes. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
Flavien Prost and Hai Qian. H chi, jilin chen, and alex beutel. 2019. In Toward a better trade-off between performance and fairness with kernelbased distribution matching."Mi with Guarantees" workshop 33rd Conference on Neural Information Processing Systems, 2019.
Flavien Prost, Hai Qian, Qiuwen Chen, Ed H Chi, Jilin Chen, and Alex Beutel. Toward a better trade-off be- tween performance and fairness with kernel-based distribution matching. ar preprint arx:::1911 1100110.777.. 11779, 2019.
Yada Pruksachatkun, Satyapriya Krishna, Jwala Dhamala, Rahul Gupta, and Kai-Wei Chang. Does robust- ness improve fairness? approaching fairness with word substitution robustness methods for text classifi- cation. arXiv preprinl or Xix:222006. 10826, 2021.
Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Perey Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprinl arXiv:1911.0877 2019.
Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, and Percy Liang. An investigation of why overparame- terization exacerbates spurious corrclations. arXiv preprint arXiv:2005.04344 04345,, 2020.
Victor Veitch, Alexander D'Amour, Steve Yadlowsky, and Jacob Eisenstein. Counterfactual invariance to spurious correlations: Why and how to pass stress tests. arXie preprint arXiv:2106.00543 2021.
Junzhe Zhang and Elias Barcinboim. Fairness in docision-making-taae the causal explanation formula. In Thirlg-Second AAAI Conference on Artificial Intelligence, 2018
15


Reviews:
[{"review_id": "DVPci7aMHV", "reviewer": "Reviewer_XoML", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n1. Explained the casual prediction setting in a detailed manner\n2. Risk Invariance is explained well \n\nWeakness\n1. Some assumptions, such as pneumonia and patient sex influence changes, are highlighted well, but I don't see enough historical evidence that such assumptions are valid.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The research paper aims to draw a parallel between distribution shifts and fairness. The author assumes the input to be the function of the target label and sensitive attributes. The dataset used for the experiment is the X-RAY images dataset. The goal is to predict the presence of pneumonia wherein, also considering sex as one of the prevalent factors to ensure fairness mitigation. The author also provides a fresh perspective on fairness and performance tradeoffs. The author also compares his approach with MMD-based implementation.  ", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I think if you could add more explanation on MMD-based implementation, that would give readers a baseline to compare it against the right approach. ", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "7LL3sCVMRJ", "reviewer": "Reviewer_UgDX", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n1. The connection  between robustness and fairness criterion are interesting and important questions to study.\n2. The specific results on separation inducing training leading to risk-invariance and optimal risk-invariant predictor leading to separation are useful observations.\n3. The paper is well-motivated, well-organised, and easy to read.\n4. The paper talks about applicability and limitation of the anti-causal setting under construction. It also elaborates clearly the restricted family of distributions that it considers to study risk-invariance. It also justifies the choice.\n5. The experiments are well-designed to proof the central hypothesis, i.e. the (near-)optimal risk-invariant predictor inducing separation.\n\nWeakness:\n1. The results are never rigorously proven, not even in the appendix. It is imperative to do so as they seem like the core contribution of the paper.\n2. The paper uses a training mechanism inducing risk invariance to ensure separation. But such a result is never proven. As the proofs are not rigorous, it is not clear why or why not it can be proved that a predictor trained using risk invariance will (or will not) ensure separation. This is not trivial as the predictor obtained through WM-MMD is not the optimal risk-invariant predictor but an approximation of it. Then, the other question is if we have only a near-optimal risk-invariant predictor, how much can we lose in separation?\n3. The paper never describes M-MMD in detail. It should be described clearly.\n4. As M-MMD is said to improve independence, it is supposed to go worse separation. What is it we want to find from experiments except this intuitive fact?\n5. The paper claims that risk-invariance inducing mechanisms induce better fairness performance than separation inducing algorithms. This claim is never validated as there is no comparison with any state-of-the-art separation inducing algorithms. It should be important to validate this claim.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper connects robustness, in the sense of risk invariance over label-sensitive attribute correlations, and fairness, in terms of separation, for anti-causal prediction tasks. Specifically, the paper show two interesting results: a) a predictor satisfying separation is risk-invariant in this specific setting, b) the optimal risk invariant predictor in this setting satisfies separation. Additionally, the optimal risk invariant predictor does not satisfy independence for any label-sensitive attribute distribution except the uncorrelated one.\nThe paper uses WM-MMD proposed by (Makar et al., 2021) to show that an Ideal Distribution Independence inducing loss with a good estimator of MMD can help to achieve separation and the proposed risk-invariance simultaneously. Specifically, the claim is that an effiicient risk-invariance inducing algorithm, such as WM-MMD, can help us to achieve good fairness performance, in terms of separation. The claim is empirically validated with experiments on the CheXpert dataset.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Technical: Please look into the weakness and corresponding questions.\n2. Formatting: Except that there are minor typos, such as \"my\" -> \"by\", \"excmaple\"-> \"example\"", "broader_impact_concerns": "The paper studies the connection between a specific sense of robustness and fairness, namely separation conditions. It is an interesting question for the community as developing fair predictors has social consequences, and training them efficiently and accurately is a natural question connected to it. But presently, the paper uses a risk-invariance inducing training mechanism to ensure separation. Given the theoretical and empirical evidence, it seems like a promising direction but it is hard to conclude anything more about efficiency/accuracy.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "SpZqCafnF9", "reviewer": "Reviewer_QUzX", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This paper is well-organized and clearly written. The authors study a novel setting of fair supervised learning in the transportability setting. The authors analyzed two popular fairness measures and reached a concise, yet interesting conclusion. That is, an optimal robust predictor is generally obtainable under separation but not independence. Given the popularity of these two common fairness measures and the relevance of transportability in many practical settings, this paper could be helpful for AI researchers across disciplines. \n\nAs for the weakness, more details could be included fo experiments. For instance, the authors claim that \"we see that better robustness (i.e., risk invariance) tracks with less violations to the EO criterion in figure 3 (left). The same is not true for the DP criterion in figure 3 (right).\" However, in Figure 3, the plots of M-MMD and WM-MMD are very close to each other, seemingly within the margin of error. It is unclear if this simulation is direct evidence corroborating the authors' conclusion.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper evaluates two popular fairness measures in the transportability setting, including the equalized odds  (Hardt et al., 2016), called separation in this paper, and the demographic disparity, called independence. The authors consider an anti-causal setting, which is graphically described in Fig. 1, where $Y$ is the true label, $X$ is the observed input features, V is the sensitive attribute, and $X^*$ is an latent sufficient statistics summarizing the causal influence of Y on X. The goal of the leaner is to obtain an optimal predictor $\\hat{Y} \\gets f(X)$ that could estimate the true label $Y$ given input feature $X$ while satisfying the pre-specified fairness constraints.\n\nThe novelty of the authors' treatment is that they consider the fairness prediction in the transportability setting, where the generative distribution associated with the sensitive attribute $P(V|Y)$ could be different in the target domain. The challenge is to learn a risk invariant predictor $f(X)$ whose predictive performance is robust across different parametrizations of $P(V|Y)$. The authors prove that it is generally infeasible to obtain an optimal risk invariant predictor while satisfying independence. However, such an optimal predictor is still obtainable under the separation constraint. Simulation results corroborate the authors' findings.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Could the authors provide more details on the experiments? Particularly, could the authors provide a table containing detailed values of fairness violations and robustness measures in Figure 3?", "broader_impact_concerns": "The authors have included a discussion addressing the broader impact of this paper. Overall, this work helps researchers understand the challenges of fair ML in practical settings, particularly those that are constrained to satisfy certain statistical parities. This work could provides\nmotivation for enforcing parities like separation under the anti-causal setting.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ZDiXJQbKJq", "reviewer": "Reviewer_9GZG", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n1. The work provably gives the connections between the non-discrimination criterion, separation, and the risk invariance. \n2. It’s good to consider practical situation of detecting pneumonia from chest X-rays across different population subgroups defined by a sensitive attribute to validate the results.\n3. They also discuss the relationship between risk invariant and independence.\n\nWeaknesses:\n1. I have some concern about the contribution from method side since the authors use the MMD method from [1].\n2. The non-discrimination fairness criteria typically include independence, separation and sufficiency in [2]. The authors consider the relationship between robustness and independence. But how about the relationship between robustness and sufficiency.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper investigates the connections between “separation” fairness criterion and “risk invariance” robustness notion under anti-causal settings. The authors theoretically give the relationships of robustness and fairness in Proposition 1 and Proposition 2. They also empirically validate their findings on the task of detecting pneumonia from X-rays and show that robustness-motivated approaches can be used to enforce separation, and that they often work better in practice than methods designed to directly enforce separation. ", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please address my previous concerns ", "broader_impact_concerns": "n/a", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work is about fair supervised learning under the transportability setting.
Two popular fairness measures have been analyzed, concluding that an optimal robust predictor is generally obtainable under separation but not independence. Given the popularity of these fairness measures and the relevance of transportability in many practical settings, this paper could be helpful for AI researchers across disciplines. The paper already incorporated some changes asked by the reviewers in their reviews, thanks to the responsiveness of the authors.
Decision: Accept (poster)