Statistical inference for individual fairness

1 INTRODUCTION . The problem of bias in machine learning systems is at the forefront of contemporary ML research . Numerous media outlets have scrutinized machine learning systems deployed in practice for violations of basic societal equality principles ( Angwin et al. , 2016 ; Dastin , 2018 ; Vigdor , 2019 ) . In response researchers developed many formal definitions of algorithmic fairness along with algorithms for enforcing these definitions in ML models ( Dwork et al. , 2011 ; Hardt et al. , 2016 ; Berk et al. , 2017 ; Kusner et al. , 2018 ; Ritov et al. , 2017 ; Yurochkin et al. , 2020 ) . Despite the flurry of ML fairness research , the basic question of assessing fairness of a given ML model in a statistically principled way remains largely unexplored . In this paper we propose a statistically principled approach to assessing individual fairness ( Dwork et al. , 2011 ) of ML models . One of the main benefits of our approach is it allows the investigator to calibrate the method ; i.e . it allows the investigator to prescribe a Type I error rate . Passing a test that has a guaranteed small Type I error rate is the usual standard of proof in scientific investigations because it guarantees the results are reproducible ( to a certain degree ) . This is also highly desirable in detecting bias in ML models because it allows us to certify whether an ML model will behave fairly at test time . Our method for auditing ML models abides by this standard . There are two main challenges associated with developing a hypothesis test for individual fairness . First , how to formalize the notion of individual fairness in an interpretable null hypothesis ? Second , how to devise a test statistic and calibrate it so that auditors can control the Type I error rate ? In this paper we propose a test motivated by the relation between individual fairness and adversarial robustness ( Yurochkin et al. , 2020 ) . At a high-level , our approach consists of two parts : 1. generating unfair examples : by unfair example we mean an example that is similar to a training example , but treated differently by the ML models . Such examples are similar to adversarial examples ( Goodfellow et al. , 2014 ) , except they are only allowed to differ from a training example in certain protected or sensitive ways . 2. summarizing the behavior of the ML model on unfair examples : We propose a loss-ratio based approach that is not only scale-free , but also interpretable . For classification problems , we propose a variation of our test based on the error rates ratio . 1.1 RELATED WORK . At a high level , our approach is to use the difference between the empirical risk and the distributionally robust risk as a test statistic . The distributionally robust risk is the maximum risk of the ML model on similar training examples . Here similarity is measured by a fair metric that encodes our intuition of which inputs should be treated similarly by the ML model . We note that DRO has been extensively studied in the recent literature ( Duchi et al. , 2016 ; Blanchet & Murthy , 2016 ; Hashimoto et al. , 2018 ) , however outside of the fairness context with the exception of Yurochkin et al . ( 2020 ) ; Xue et al . ( 2020 ) . Yurochkin et al . ( 2020 ) focus on training fair or robust ML models instead of auditing ML models . Xue et al . ( 2020 ) also use the difference between the empirical and distributionally robust risks as a test statistic , but their test is only applicable to ML problems with finite feature spaces . This limitation severely restricts the applicability of their test . On the other hand , our test is suitable for ML problems with continuous features spaces . We note that the technical exposition in Xue et al . ( 2020 ) is dependant on the finite feature space assumption and in this work we develop a novel perspective of the problem that allows us to handle continuous feature spaces . 2 GRADIENT FLOW FOR FINDING UNFAIR EXAMPLES . In this section , we describe a gradient flow-based approach to finding unfair examples that form the basis of our suite of inferential tools . Imagine an auditor assessing whether an ML model is fair or not . The auditor aims to detect violations of individual fairness in the ML model . Recall Dwork et al . ( 2011 ) ’ s definition of individual fairness . Let X ⊂ Rd and Y ⊂ Rd be the input and output spaces respectively , and f : X → Y be an ML model to audit . The ML model f is known as individually fair if dy ( f ( x1 ) , f ( x2 ) ) ≤ Lfairdx ( x1 , x2 ) for all x1 , x2 ∈ X ( 2.1 ) for some Lipschitz constant Lfair > 0 . Here dx and dy are metrics on X and Y respectively . Intuitively , individually fair ML model treats similar samples similarly , and the fair metric dx encodes our intuition of which samples should be treated similarly . We should point out that dx ( x1 , x2 ) being small does not imply x1 and x2 are similar in all aspects . Even if dx ( x1 , x2 ) is small , x1 and x2 may differ much in certain attributes , e.g. , protected/sensitive attributes . Before moving on , we comment on the choice of the fair metric dx . This metric is picked by the auditor and reflects the auditor ’ s intuition about what is fair and what is unfair for the ML task at hand . It can be provided by a subject expert ( this is Dwork et al . ( 2011 ) ’ s original recommendation ) or learned from data ( this is a recent approach advocated by Ilvento ( 2019 ) ; Wang et al . ( 2019 ) ; Mukherjee et al . ( 2020 ) ) . Section 4 provides details of picking a fair metric in our empirical studies . To motivate our approach , we recall the distributionally robust optimization ( DRO ) approach to training individually fair ML models ( Yurochkin et al. , 2020 ) . Let f : X → Y be an ML model and ` ( f ( x ) , y ) : Z → R+ be any smooth loss ( e.g . cross-entropy loss ) . To search for differential treatment in the ML model , Yurochkin et al . ( 2020 ) solve the optimization problem max P : W ( P , Pn ) ≤ ∫ Z ` ( f ( x ) , y ) dP ( z ) , ( 2.2 ) where W is the Wasserstein distance on probability distributions on feature space induced by the fair metric , Pn is the empirical distribution of the training data , and is a moving budget that ensures the adversarial examples are close to the ( original ) training examples in the fair metric . Formally , this search for differential treatment checks for violations of distributionally robust fairness . Definition 2.1 ( distributionally robust fairness ( DRF ) ( Yurochkin et al. , 2020 ) ) . An ML model h : X → Y is ( , δ ) -distributionally robustly fair ( DRF ) WRT the fair metric dx iff supP : W ( P , Pn ) ≤ ∫ Z ` ( z , h ) dP ( z ) − ∫ Z ` ( z , h ) dPn ( z ) ≤ δ . ( 2.3 ) The optimization problem ( 2.2 ) is an infinite-dimensional problem , but its dual is more tractable . Blanchet & Murthy show that the dual of ( 2.2 ) is max P : W ( P , Pn ) ≤ EP [ ` ( f ( x ) , y ) ] = min λ≥0 { λ + EPn [ ` cλ ( x , y ) ] } , ( 2.4 ) ` cλ ( xi , yi ) , max x∈X { ` ( f ( x ) , yi ) − λd2x ( x , xi ) } . ( 2.5 ) In practice , since ( 2.5 ) is highly non-convex in general , auditors use gradient-based optimization algorithm to solve it and terminate the algorithm after few iterations . As a result , one can not guarantee optimality of the solution . However , optimality is required to establish convergence guarantees for DRO algorithms . This issue is typically ignored in practice when developing training algorithms , e.g . as in Yurochkin et al . ( 2020 ) , but it should be treated with care when considering limiting distribution of the related quantities required to calibrate a test . We note that Xue et al . ( 2020 ) needed discrete feature space assumption due to the aforementioned concern : when the feature space is discrete , it is possible to solve ( 2.5 ) optimally by simply comparing the objective value at all points of the sample space . In this paper we adapt theory to practice , i.e . analyze the limiting distribution of ( 2.5 ) optimized for a fixed number of gradient steps . The effects of early termination can be characterized by a continuous-time approximation of adversarial dynamics , which we called gradient flow attack . Given a sample ( x0 , y0 ) , the gradient flow attack solves a continuous-time ordinary differential equation ( ODE ) { Ẋ ( t ) = ∇x { ` ( f ( X ( t ) ) , y0 ) − λd2x ( X ( t ) , x0 ) } , X ( 0 ) = x0 , ( 2.6 ) over time t ≥ 0 . For fixed penalty parameter λ and stopping time T > 0 , Φ : X × Y → X is the unfair map Φ ( x0 , y0 ) , X ( T ) . ( 2.7 ) Here the map Φ is well-defined as long as g ( x ) , ∇x { ` ( f ( x ) , y0 ) − λd2x ( x , x0 ) } is Lipschitz , i.e. , ‖g ( x1 ) − g ( x2 ) ‖2 ≤ L‖x1 − x2‖2 for some L > 0 . Under this assumption , the autonomous Cauchy problem ( 2.6 ) has unique solution and thus Φ : X × Y → X is a one-to-one function . We call Φ an unfair map because it maps samples in the data to similar areas of the sample space that the ML model performs poorly on . We note that data in this case is an audit dataset chosen by the auditor to evaluate individual fairness of the given model . The audit data does not need to be picked carefully and could be simply an iid sample ( e.g . testing data ) . The unfair map plays the key role as it allows us to identify areas of the sample space where model violates individual fairness , even if the audit samples themselves reveal no such violations . In the rest of the paper , we define the test statistic in terms of the unfair map instead of the optimal point of ( 2.5 ) . This has two main benefits : 1. computational tractability : evaluating the unfair map is computationally tractable because integrating initial value problems ( IVP ) is a well-developed area of scientific computing ( Heath , 2018 ) . Auditors may appeal to any globally stable method for solving IVP ’ s to evaluate the unfair map . 2. reproducibility : the non-convex nature of ( 2.5 ) means the actual output of any attempts at solving ( 2.5 ) is highly depend on the algorithm and the initial iterate . By defining the test statistic algorithmically , we avoid ambiguity in the algorithm and initial iterate , thereby ensuring reproducibility . Of course , the tractability and reproducibility of the resulting statistical tests comes at a cost : power . Because we are not exactly maximizing ( 2.5 ) , the ability of the test statistic to detect violations of individual fairness is limited by the ability of ( 2.7 ) to find ( unfair ) adversarial examples .

Summary: The paper introduces a framework to statistically test whether a given model is individually fair or not. In particular, given a model, a distance metric over individuals, and a data point z, the authors propose an algorithm that finds a new data point z' such that z' is similar to z but their corresponding losses are different under the model -- if the model is not individually fair. They provide experimental results to show how their proposed method can detect unfairness in practice.