Fair Classification with Adversarial Perturbations

1 Introduction . It is increasingly common to deploy classifiers to assist in decision-making in applications such as criminal recidivism [ 50 ] , credit lending [ 21 ] , and predictive policing [ 34 ] . Hence , it is imperative to ensure that these classifiers are fair with respect to protected attributes such as gender and race . Consequently , there has been extensive work on approaches for fair classification [ 32 , 26 , 28 , 17 , 63 , 62 , 48 , 24 , 27 , 1 , 13 ] . At a high level , a classifier f is said to be “ fair ” with respect to a protected attribute Z if it has a similar “ performance ” with respect to a given metric on different protected groups defined by Z . Given a fairness metric and a hypothesis class F , fair classification frameworks consider the problem of finding a classifier f ? ∈ F that maximizes accuracy constrained to being fair with respect to the given fairness metric ( and Z ) [ 8 ] . To specify fairness constraints , these approaches need protected attributes of training data to be known . However , protected attributes can be erroneous for various reasons ; there could be uncertainties during data collection or data cleaning process [ 20 , 52 ] , or the attributes could be strategically misreported [ 46 ] . Further , protected attributes may be missing entirely , as is often the case for racial and ethnic information in healthcare [ 20 ] or when data is scraped from the internet as with many image datasets [ 22 , 66 , 35 ] . In these cases , protected attributes can be “ imputed ” [ 18 , 36 , 16 ] , but this can also introduce errors [ 12 ] ; further , imputation by machine-learning-based methods is known to be fragile to imperceptible changes in the inputs [ 29 ] and to have correlated errors across samples [ 49 ] . Perturbations in protected attributes , regardless of origin , have been shown to have adverse effects on fair classifiers , affecting their performance on both accuracy and fairness metrics ; see e.g. , [ 16 , 7 ] . Towards addressing this problem , several recent works have developed fair classification algorithms for various models of errors in the protected attributes . [ 44 ] consider an extension of the “ mutually contaminated learning model ” [ 53 ] where , instead of observing samples from the “ true ” joint distribution , distributions of observed group-conditional distributions are stochastic mixtures of their true counterparts . [ 6 ] consider a binary protected attribute and Bernoulli perturbations that are 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . independent of the labels ( and of each other ) . [ 14 ] consider the setting where each sample ’ s protected attribute is independently flipped to a different value with a known probability . [ 59 ] considers two approaches to deal with perturbations . In their “ soft-weights ” approach , they assume perturbations follow a fixed distribution and one has access to an auxiliary data containing independent draws of both the true and perturbed protected attributes . In their distributionally robust approach , for each protected group , its feature and label distributions in the true data and the perturbed data are a known total variation distance away from each other . Finally , in an independent work , [ 40 ] study fair classification under the Malicious noise model [ 56 , 39 ] in which a fraction of the training samples are chosen uniformly at random , and can then be perturbed arbitrarily . Our perturbation model . We extend this line of work by studying fair classification under the following worst-case adversarial perturbation model : Given an η > 0 , after the training samples are independently drawn from a true distribution D , the adversary with unbounded computation power sees all the samples and can use this information to choose any η-fraction of the samples and perturb their protected attributes arbitrarily . This model is a straightforward adaptation of the perturbation model of [ 31 ] to the fair classification setting and we refer to it as the η-Hamming model . Unlike perturbation models studied before , this model can capture settings where the perturbations are strategic or arbitrarily correlated as can arise in the data collection stage or during imputation of the protected attributes , and in which the errors can not be “ estimated ” using auxiliary data . In fact , under this perturbation model , the classifiers outputted by prior works can violate the fairness constraints by a large amount or have an accuracy that is significantly lower than the accuracy of f ? ; see Section 5 and Supplementary Material D.2 . Taking these perturbed samples , a fairness metric Ω , and a desired fairness threshold τ as input , the goal is to learn a classifier f with the maximum accuracy with respect to the true distribution D subject to having a fairness value , ΩD ( f ) , of at least τ with respect to the true distribution D. Our contributions . We present an optimization framework ( Definition 4.1 ) that outputs fair classifiers for the η-Hamming model and comes with provable guarantees on accuracy and fairness ( Theorem 4.3 ) . Our framework works for multiple and non-binary protected attributes , and the large class of linear-fractional fairness metrics ( that capture most fairness metrics studied in the literature ) ; see Definition 3.1 and [ 13 ] . The framework provably outputs a classifier whose accuracy is within 2η of the accuracy of f ? and which violates the fairness constraint by at most O ( η/λ ) additively ( Theorem 4.3 ) , under the mild assumption that the “ performance ” of f ? on each protected group is larger than a known constant λ > 0 ( Assumption 1 ) . Assumption 1 is drawn from the work of [ 14 ] for fair classification with stochastic perturbations . While it is not clear if the assumption is necessary in their model , we show that Assumption 1 is necessary for fair classification in the η-Hamming model : If λ is not bounded away from 0 , then no algorithm can give a non-trivial guarantee on both accuracy and fairness value of the output classifier ( Theorem 4.4 ) . Moreover , we prove the near-tightness of our framework ’ s guarantee under Assumption 1 : No algorithm can guarantee to output a classifier with accuracy closer than η to that of f ? and any algorithm that violates the fairness constraint by less than η/ ( 20λ ) additively has an accuracy at most 19/20 ( Theorems 4.5 and A.21 ) . Finally , we also extend our framework ’ s guarantees to the Nasty Sample Noise model ( Supplementary Material A.1.5 ) . The Nasty Sample Noise model is a generalization of the η-Hamming model , which was studied by [ 11 ] in the context of PAC learning ( without any fairness considerations ) , where the adversary can choose any η-fraction of the samples , and can arbitrarily perturb both their labels and features . We implement our framework for logistic loss function with linear classifiers and evaluate its performance on COMPAS [ 3 ] , Adult [ 23 ] , and a synthetic dataset ( Section 5 ) . We generate perturbations of these datasets admissible in the η-Hamming model and compare the performance of our approach to several baselines [ 44 , 6 , 59 , 14 , 40 ] with statistical rate and false-positive rate as fairness metrics.1 On the synthetic dataset , we compare against a method developed for fair classification under stochastic perturbations [ 14 ] and demonstrate the comparative strength of the η-Hamming model ; our results show that [ 14 ] ’ s framework achieves a significantly lower accuracy than our framework for the same statistical rate . Empirical results on COMPAS and Adult show that the classifier output by our framework can attain better statistical rate and false-positive rate than the accuracy maximizing classifier on the true distribution , with a small loss in accuracy . Further , our framework has a similar ( or better ) fairness-accuracy trade-off compared to all baselines we consider in a variety of settings , and is not dominated by any other approach ( Figure 1 and Figures 7 and 8 in Supplementary Material E.2 ) . 1Let q ` ( f , SR ) ( respectively q ` ( f , FPR ) ) be the fraction of positive predictions ( respectively false-positive predictions ) by f in the ` -th protected group . f ’ s statistical rate ( respectively false-positive rate ) is the ratio of the minimum value to the maximum value of q ` ( f , SR ) ( respectively q ` ( f , FPR ) ) over all protected groups . Techniques . The starting point of our optimization framework ( Definition 4.1 ) is the “ standard ” optimization program for fair classification in the absence of any perturbations : Given a fairness metric Ω and a desired fairness threshold τ as input , find f ? ∈ F that maximizes the accuracy on the given data Ŝ constrained to a fairness value at least τ on the given data . However , when Ŝ is given to us by an η-Hamming adversary , this standard program , which imposes the fairness constraints with respect to the perturbed data Ŝ , may output a classifier with an accuracy/fairness-value worse than that of f ? when measured with respect to D. But , observe that the difference in accuracies of a classifier when measured with respect to the given data Ŝ and data sampled fromD is at most η . Thus , if f ? ∈ F is feasible for the standard optimization program , this observation ( used twice ) implies that the accuracy of the output classifier measured with respect to D is within 2η of the accuracy of f ? measured with respect D ( Equation ( 8 ) ) . However , without any modifications , the classifier output by the standard optimization program could still have a fairness value much lower than τ with respect to D ( see Example A.27 ) . To bypass this , we introduce the notion of s-stability that allows us to lower bound the fairness value of a classifier with respect to D given its fairness value on Ŝ . Roughly , f ∈ F is said to be s-stable with respect to a fairness metric if for any Ŝ that is generated by an η-Hamming adversary , the ratio of fairness value of f with respect to D and with respect to Ŝ is between s and 1/s ( see Definition 4.7 ) . It follows that any s-stable classifier that has fairness value τ ′ > 0 with respect to Ŝ , has fairness value at least s · τ ′ with respect to D. Hence , an optimization program that ensures that all feasible classifiers are s-stable ( for a suitable choice of s ) and have fairness value at least τ ′ > 0 with respect to Ŝ , comes with a guarantee that any feasible classifier has a fairness value at least s · τ ′ ( with respect to D ) . If such an optimization program could further ensure that f ? is feasible for it , then by arguments presented above , the classifier output by this optimization program would satisfy required guarantees on both fairness and accuracy ( Lemma 4.9 ) . The issue is that , to directly enforce s-stability , one needs to compute the fairness values of classifiers with respect to D , but this is not possible in the absence of samples from D. We overcome this by present a “ proxy ” constraint on the classifier ( Equation ( 5 ) ) that involves only Ŝ and ensures that any classifier that satisfies it is s-stable . Moreover , f ? satisfies this constraint under Assumption 1 . Overall , modifying Program ( 2 ) to include this constraint ( Equation ( 5 ) ) with a suitable value of s , and setting an appropriate fairness threshold τ so that f ? remains feasible , leads us to our framework .

Summary: This work studies learning fair classification under the assumption that an arbitrary eta-fraction of training samples are perturbed on the protected attributes. The authors propose an optimization framework for eta-Hamming model which has the provable guarantees on accuracy and fairness. The experiments with linear models on synthetic and real-world datasets are used to evaluate the proposed approach.