Exchanging Lessons Between Algorithmic Fairness and Domain Generalization

1 INTRODUCTION . Machine learning achieves super-human performance on many tasks when the test data is drawn from the same distribution as the training data . However , when the two distributions differ , model performance can severely degrade to even below chance predictions ( Geirhos et al. , 2020 ) . Tiny perturbations can derail classifiers , as shown by adversarial examples ( Szegedy et al. , 2014 ) and common-corruptions in image classification ( Hendrycks & Dietterich , 2019 ) . Even new test sets collected from the same data acquisition pipeline induce distribution shifts that significantly harm performance ( Recht et al. , 2019 ; Engstrom et al. , 2020 ) . Many approaches have been proposed to overcome model brittleness in the face of input distribution changes . Robust optimization aims to achieve good performance on any distribution close to the training distribution ( Goodfellow et al. , 2015 ; Duchi et al. , 2016 ; Madry et al. , 2018 ) . Domain generalization on the other hand tries to go one step further , to generalize to distributions potentially far away from the training distribution . The field of algorithmic fairness meanwhile primarily focuses on developing metrics to track and mitigate performance differences between different sub-populations or across similar individuals ( Dwork et al. , 2012 ; Corbett-Davies & Goel , 2018 ; Chouldechova & Roth , 2018 ) . Like domain generalization , evaluation using data related to but distinct from the training set is needed to characterize model failure . These evaluations are curated through the design of audits , which play a central role in revealing unfair algorithmic decision making ( Buolamwini & Gebru , 2018 ; Obermeyer et al. , 2019 ) . While the ultimate goals of domain generalization and algorithmic fairness are closely aligned , little research has focused on their similarities and how they can inform each other constructively . One of their main common goals can be characterized as : Learning algorithms robust to changes across domains or population groups . Achieving this not only allows models to generalize to different and unobserved but related distributions , it also mitigates unequal treatment of individuals solely based on group membership . In this work we explore independently developed concepts from the domain generalization and fairness literatures and exchange lessons between them to motivate new methodology for both fields . Inspired by fairness approaches for unknown group memberships ( Kim et al. , 2019 ; Hébert-Johnson et al. , 2018 ; Lahoti et al. , 2020 ) , we develop a new domain generalization method that does not require domain identifiers and yet can outperform manual specification of domains ( Table 1 ) . Leveraging domain generalization insights in a fairness context , we show the regularizer from IRMv1 ( Arjovsky et al. , 2019 ) optimizes a fairness criterion termed “ group-sufficiency ” which for the first time enables us to explicitly optimize this criterion for non-convex losses in fair classification . The following contributions show how lessons can be exchanged from the two fields : • We draw several connections between the goals of domain generalization and those of algorithmic fairness , suggesting fruitful research directions in both fields ( Section 2 ) . • Drawing inspiration from recent methods on inferring worst-case sensitive groups from data , we propose a novel domain generalization algorithm—Environment Inference for Invariant Learning ( EIIL ) —for cases where training data does not include environment partition labels ( Section 3 ) . Our method outperforms IRM on the domain generalization benchmark ColorMNIST without access to environment labels ( Section 4 ) . • We also show that IRM , originally developed for domain generalization tasks , affords a differentiable regularizer for the fairness notion of group sufficiency , which was previously hard to optimize for non-convex losses . On a variant of the UCI Adult dataset where confounding bias is introduced , we leverage this insight with our method EIIL to improve group sufficiency without knowledge of sensitive groups , ultimately improving generalization performance for large distribution shifts compared with a baseline robust optimization method ( Section 4 ) . • We characterize both theoretically and empirically the limitations of our proposed method , concluding that while EIIL can correct a baseline ERM solution that uses a spurious feature or “ shortcut ” for prediction , it is not suitable for all settings ( Sections 3 and 4 ) . 2 DOMAIN GENERALIZATION AND ALGORITHMIC FAIRNESS . Here we lay out some connections between the two fields . Table 2 provides a high-level comparison of the objectives and assumptions of several relevant methods . Loosely speaking , recent approaches from both areas share the goal of matching some chosen statistic across a conditioning variable e , representing sensitive group membership in algorithmic fairness or an environment/domain indicator in domain generalization . The statistic in question informs the learning objective for the resulting model , and is motivated differently in each case . In domain generalization , learning is informed by the properties of the test distribution where good generalization should be achieved . In algorithmic fairness the choice of statistic is motivated by a context-specific fairness notion , that likewise encourages a particular solution that achieves “ fair ” outcomes ( Chouldechova & Roth , 2018 ) . Empty spaces in Table 2 suggest areas for future work , and bold-faced entries suggest connections we show in this paper . Notation Let X be the input space , E the set of environments ( a.k.a . “ domains ” ) , Y the target space . Let x , y , e ∼ pobs ( x , y , e ) be observational data , with x ∈ X , y ∈ Y , and e ∈ E . H denotes a representation space , from which a classifier w ◦ Φ ( that maps to the pre-image of ∆ ( Y ) via a linear map w ) can be applied . Φ : X → H denotes the parameterized mapping or “ model ” that we optimize . We refer to Φ ( x ) ∈ H as the “ representation ” of example x. ŷ ∈ Y denotes a hard prediction derived from the classifier by stochastic sampling or probability thresholding . ` : H× Y → R denotes the scalar loss , which guides the learning . The empirical risk minimization ( ERM ) solution is found by minimizing the global risk , expressed as the expected loss over the observational distribution : CERM ( Φ ) = Epobs ( x , y , e ) [ ` ( Φ ( x ) , y ) ] . ( 1 ) Domain Generalization Domain generalization is concerned with achieving low error rates on unseen test distributions . One way to achieve domain generalization is by casting it as a robust optimization problem ( Ben-Tal et al. , 2009 ) where one aims to minimize the worst-case loss for every subset of the training set , or other well-defined perturbation sets around the data ( Duchi et al. , 2016 ; Madry et al. , 2018 ) . Other approaches tackle domain generalization by adversarially learning representations invariant ( Zhang et al. , 2017 ; Hoffman et al. , 2018 ; Ganin et al. , 2016 ) or conditionally invariant ( Li et al. , 2018 ) to the environment . Distributionally Robust Optimization ( DRO ) ( Duchi et al. , 2016 ) , seeks good performance for all nearby distributions by minimizing the worst-case loss supq Eq [ ` ] s.t.D ( q||p ) < , whereD denotes similarity between two distributions ( e.g . χ2 divergence ) and is a hyperparameter . The objective can be computed as an expectation over p via per-example importance weights γi = q ( xi , yi ) p ( xi , yi ) . Recently , Invariant Learning approaches such as Invariant Risk Minimization ( IRM ) ( Arjovsky et al. , 2019 ) and Risk Extrapolation ( REx ) ( Krueger et al. , 2020 ) were proposed to overcome the limitations of domain invariant representation learning ( Zhao et al. , 2019 ) by discovering invariant relationships between inputs and targets across domains . Invariance serves as a proxy for causality , as features representing “ causes ” of target labels rather than effects will generalize well under intervention . In IRM , a representation Φ ( x ) is learned that performs optimally within each environment—and is thus invariant to the choice of environment e ∈ E—with the ultimate goal of generalizing to an unknown test dataset p ( x , y|etest ) . Because optimal classifiers under standard loss functions can be realized via a conditional label distribution ( f∗ ( x ) = E [ y|x ] ) , then an invariant representation Φ ( x ) must satisfy the following Environment Invariance Constraint : E [ y|Φ ( x ) = h , e1 ] = E [ y|Φ ( x ) = h , e2 ] ∀h ∈ H ∀ e1 , e2 ∈ E . ( EI-CONSTR ) Intuitively , the representation Φ ( x ) encodes features of the input x that induce the same conditional distribution over labels across each environment . Because trivial representations such as mapping all x onto the same value may satisfy environment invariance , other objectives must be introduced to encourage the predictive utility of Φ. Arjovsky et al . ( 2019 ) propose IRM as a way to satisfy ( EI-CONSTR ) while achieving a good overall risk . As a practical instantiation , the authors introduce IRMv1 , a gradient-penalty regularized objective enforcing simultaneous optimality of the same classifier w ◦ Φ in all environments.1 Denoting by Re = Epobs ( x , y|e ) [ ` ] the per-environment risk , the objective to be minimized is CIRM ( Φ ) = ∑ e∈E Re ( Φ ) + λ||∇w|w=1.0Re ( w ◦ Φ ) ||2 . ( 2 ) Krueger et al . ( 2020 ) propose the related Risk Extrapolation ( REx ) principle , which dictates a stronger preference to exactly equalize Re ∀ e ( e.g . by penalizing variance across e ) , which is shown to improve generalization in several settings.2 Fairness Early approaches to learning fair representations ( Zemel et al. , 2013 ; Edwards & Storkey , 2015 ; Louizos et al. , 2015 ; Zhang et al. , 2018 ; Madras et al. , 2018 ) leveraged statistical independence regularizers from domain adaptation ( Ben-David et al. , 2010 ; Ganin et al. , 2016 ) , noting that marginal or conditional independence from domain to prediction relates to the fairness notions of demographic parity ŷ ⊥ e ( Dwork et al. , 2012 ) and equal opportunity ŷ ⊥ e|y ( Hardt et al. , 2016 ) . Recall that ( EI-CONSTR ) involves an environment-specific conditional label expectation given a data representation E [ y|Φ ( x ) = h , e ] . Objects of this type have been closely studied in the fair machine learning literature , where e now denotes a “ sensitive ” indicating membership in a protected demographic group ( age , race , gender , etc . ) , and the vector representation Φ ( x ) is typically replaced by a scalar score S ( x ) ∈ R. E [ y|S ( x ) , e ] can now be interpreted as a calibration curve that must be regulated according to some fairness constraint . Chouldechova ( 2017 ) showed that equalizing this calibration curve across groups is often incompatible with a common fairness constraint , demographic parity , while Liu et al . ( 2019 ) studied “ group sufficiency ’ ‘ —satisfied when E [ y|S ( x ) , e ] = E [ y|S ( x ) , e′ ] ∀e , e′—of classifiers with strongly convex losses , concluding that ERM naturally finds group sufficient solutions without fairness constraints . Because Liu et al . ( 2019 ) consider convex losses , their theoretical results do not hold for neural network representations . However , by noting the link between group sufficiency and the constraint from ( EI-CONSTR ) , we observe that the IRMv1 regularizer ( applicable to neural nets ) in fact minimizes the group sufficiency gap in the case of a scalar representation Φ ( x ) ⊆ R , and when e indicates sensitive group membership . It is worth noting that Arjovsky et al . ( 2019 ) briefly discuss using groups as environments , but without specifying a particular fairness criterion . Reliance on sensitive demographic information is cumbersome since it often can not be collected without legal or ethical repercussions . Hébert-Johnson et al . ( 2018 ) discussed the problem of mitigating subgroup unfairness when group labels are unknown , and proposed Multicalibration as a way of ensuring a classifier ’ s calibration curve is invariant to efficiently computable environment splits . Since the proposed algorithm requires brute force enumeration over all possible environments/groups , Kim et al . ( 2019 ) suggested a more practical algorithm by relaxing the calibration constraint to an accuracy constraint , yielding a Multiaccurate classifier.3 The goal here is to boost the predictions of a pre-trained classifier through multiple rounds of auditing ( searching for worstcase subgroups using an auxiliary model ) rather than learning an invariant representation . A related line of work also leverages inferred subgroup information to improve worst-case model performance using the framework of DRO . Hashimoto et al . ( 2018 ) applied DRO to encourage long-term fairness in a dynamical setting where the average loss for a subpopulation influences their propensity to continue engaging with the model . Lahoti et al . ( 2020 ) proposed Adversarially Reweighted Learning , which extends DRO using an auxiliary model to compute the importance weights γi mentioned above . Amortizing this computation mitigates the tendency of DRO to overfit its reweighting strategy to noisy outliers . Wang et al . ( 2020 ) proposed a group DRO method for adaptively estimating soft assignments to groups suitable for the setting where group labels are noisy . 1w ◦ Φ yields a classification decision via linear weighting on the representation features . 2Analogous to REx , Williamson & Menon ( 2019 ) adapt Conditional Variance at Risk ( CVaR ) ( Rockafellar & Uryasev , 2002 ) to equalize risk across demographic groups . 3Kearns et al . ( 2018 ) separately utilize boosting to equalize subgroup errors without sensitive attributes . Limitations of generalization-first fairness One exciting direction for future work is to apply methods developed in the domain generalization literature to tasks where distribution shift is related to some societal harm that should be mitigated . However , researchers should be wary of blind “ solutionism ” , which can be ineffectual or harmful when the societal context surrounding the machine learning system is ignored ( Selbst et al. , 2019 ) . Moreover , many aspects of algorithmic discrimination are not simply a matter of achieving few errors on unseen distributions . Unfairness due to task definition or dataset collection , as discussed in the study of target variable selection by Obermeyer et al . ( 2019 ) , may not be reversible by novel algorithmic developments .

Summary: The main contribution of the paper is to highlight the similarity between two active areas in ML namely "domain generalization" and "fairness". Further, the paper proposes an approach inspired by recent developments in the fairness literature for domain generalization. The high-level idea is that similarly to the way that fair algorithm are able to improve the worst-case accuracy of predictors across different groups without knowing the sensitive attributes, perhaps we can use these ideas to domain generalization when environment partitions are not known to the algorithm. In some sense, in both of these research areas the goal is to design robust algorithms. Similarly, the paper uses the idea from domain generalization to design fair algorithms w.r.t. a notion called "group sufficiency". The idea is to somehow infer the "worst-case" subgroup (i.e., the one that our algorithm has the worst accuracy on it) and then using a round of auditing improve the performance of the algorithm across all subgroups.