Fair Classification with Adversarial Perturbations

1 Introduction . It is increasingly common to deploy classifiers to assist in decision-making in applications such as criminal recidivism [ 50 ] , credit lending [ 21 ] , and predictive policing [ 34 ] . Hence , it is imperative to ensure that these classifiers are fair with respect to protected attributes such as gender and race . Consequently , there has been extensive work on approaches for fair classification [ 32 , 26 , 28 , 17 , 63 , 62 , 48 , 24 , 27 , 1 , 13 ] . At a high level , a classifier f is said to be “ fair ” with respect to a protected attribute Z if it has a similar “ performance ” with respect to a given metric on different protected groups defined by Z . Given a fairness metric and a hypothesis class F , fair classification frameworks consider the problem of finding a classifier f ? ∈ F that maximizes accuracy constrained to being fair with respect to the given fairness metric ( and Z ) [ 8 ] . To specify fairness constraints , these approaches need protected attributes of training data to be known . However , protected attributes can be erroneous for various reasons ; there could be uncertainties during data collection or data cleaning process [ 20 , 52 ] , or the attributes could be strategically misreported [ 46 ] . Further , protected attributes may be missing entirely , as is often the case for racial and ethnic information in healthcare [ 20 ] or when data is scraped from the internet as with many image datasets [ 22 , 66 , 35 ] . In these cases , protected attributes can be “ imputed ” [ 18 , 36 , 16 ] , but this can also introduce errors [ 12 ] ; further , imputation by machine-learning-based methods is known to be fragile to imperceptible changes in the inputs [ 29 ] and to have correlated errors across samples [ 49 ] . Perturbations in protected attributes , regardless of origin , have been shown to have adverse effects on fair classifiers , affecting their performance on both accuracy and fairness metrics ; see e.g. , [ 16 , 7 ] . Towards addressing this problem , several recent works have developed fair classification algorithms for various models of errors in the protected attributes . [ 44 ] consider an extension of the “ mutually contaminated learning model ” [ 53 ] where , instead of observing samples from the “ true ” joint distribution , distributions of observed group-conditional distributions are stochastic mixtures of their true counterparts . [ 6 ] consider a binary protected attribute and Bernoulli perturbations that are 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . independent of the labels ( and of each other ) . [ 14 ] consider the setting where each sample ’ s protected attribute is independently flipped to a different value with a known probability . [ 59 ] considers two approaches to deal with perturbations . In their “ soft-weights ” approach , they assume perturbations follow a fixed distribution and one has access to an auxiliary data containing independent draws of both the true and perturbed protected attributes . In their distributionally robust approach , for each protected group , its feature and label distributions in the true data and the perturbed data are a known total variation distance away from each other . Finally , in an independent work , [ 40 ] study fair classification under the Malicious noise model [ 56 , 39 ] in which a fraction of the training samples are chosen uniformly at random , and can then be perturbed arbitrarily . Our perturbation model . We extend this line of work by studying fair classification under the following worst-case adversarial perturbation model : Given an η > 0 , after the training samples are independently drawn from a true distribution D , the adversary with unbounded computation power sees all the samples and can use this information to choose any η-fraction of the samples and perturb their protected attributes arbitrarily . This model is a straightforward adaptation of the perturbation model of [ 31 ] to the fair classification setting and we refer to it as the η-Hamming model . Unlike perturbation models studied before , this model can capture settings where the perturbations are strategic or arbitrarily correlated as can arise in the data collection stage or during imputation of the protected attributes , and in which the errors can not be “ estimated ” using auxiliary data . In fact , under this perturbation model , the classifiers outputted by prior works can violate the fairness constraints by a large amount or have an accuracy that is significantly lower than the accuracy of f ? ; see Section 5 and Supplementary Material D.2 . Taking these perturbed samples , a fairness metric Ω , and a desired fairness threshold τ as input , the goal is to learn a classifier f with the maximum accuracy with respect to the true distribution D subject to having a fairness value , ΩD ( f ) , of at least τ with respect to the true distribution D. Our contributions . We present an optimization framework ( Definition 4.1 ) that outputs fair classifiers for the η-Hamming model and comes with provable guarantees on accuracy and fairness ( Theorem 4.3 ) . Our framework works for multiple and non-binary protected attributes , and the large class of linear-fractional fairness metrics ( that capture most fairness metrics studied in the literature ) ; see Definition 3.1 and [ 13 ] . The framework provably outputs a classifier whose accuracy is within 2η of the accuracy of f ? and which violates the fairness constraint by at most O ( η/λ ) additively ( Theorem 4.3 ) , under the mild assumption that the “ performance ” of f ? on each protected group is larger than a known constant λ > 0 ( Assumption 1 ) . Assumption 1 is drawn from the work of [ 14 ] for fair classification with stochastic perturbations . While it is not clear if the assumption is necessary in their model , we show that Assumption 1 is necessary for fair classification in the η-Hamming model : If λ is not bounded away from 0 , then no algorithm can give a non-trivial guarantee on both accuracy and fairness value of the output classifier ( Theorem 4.4 ) . Moreover , we prove the near-tightness of our framework ’ s guarantee under Assumption 1 : No algorithm can guarantee to output a classifier with accuracy closer than η to that of f ? and any algorithm that violates the fairness constraint by less than η/ ( 20λ ) additively has an accuracy at most 19/20 ( Theorems 4.5 and A.21 ) . Finally , we also extend our framework ’ s guarantees to the Nasty Sample Noise model ( Supplementary Material A.1.5 ) . The Nasty Sample Noise model is a generalization of the η-Hamming model , which was studied by [ 11 ] in the context of PAC learning ( without any fairness considerations ) , where the adversary can choose any η-fraction of the samples , and can arbitrarily perturb both their labels and features . We implement our framework for logistic loss function with linear classifiers and evaluate its performance on COMPAS [ 3 ] , Adult [ 23 ] , and a synthetic dataset ( Section 5 ) . We generate perturbations of these datasets admissible in the η-Hamming model and compare the performance of our approach to several baselines [ 44 , 6 , 59 , 14 , 40 ] with statistical rate and false-positive rate as fairness metrics.1 On the synthetic dataset , we compare against a method developed for fair classification under stochastic perturbations [ 14 ] and demonstrate the comparative strength of the η-Hamming model ; our results show that [ 14 ] ’ s framework achieves a significantly lower accuracy than our framework for the same statistical rate . Empirical results on COMPAS and Adult show that the classifier output by our framework can attain better statistical rate and false-positive rate than the accuracy maximizing classifier on the true distribution , with a small loss in accuracy . Further , our framework has a similar ( or better ) fairness-accuracy trade-off compared to all baselines we consider in a variety of settings , and is not dominated by any other approach ( Figure 1 and Figures 7 and 8 in Supplementary Material E.2 ) . 1Let q ` ( f , SR ) ( respectively q ` ( f , FPR ) ) be the fraction of positive predictions ( respectively false-positive predictions ) by f in the ` -th protected group . f ’ s statistical rate ( respectively false-positive rate ) is the ratio of the minimum value to the maximum value of q ` ( f , SR ) ( respectively q ` ( f , FPR ) ) over all protected groups . Techniques . The starting point of our optimization framework ( Definition 4.1 ) is the “ standard ” optimization program for fair classification in the absence of any perturbations : Given a fairness metric Ω and a desired fairness threshold τ as input , find f ? ∈ F that maximizes the accuracy on the given data Ŝ constrained to a fairness value at least τ on the given data . However , when Ŝ is given to us by an η-Hamming adversary , this standard program , which imposes the fairness constraints with respect to the perturbed data Ŝ , may output a classifier with an accuracy/fairness-value worse than that of f ? when measured with respect to D. But , observe that the difference in accuracies of a classifier when measured with respect to the given data Ŝ and data sampled fromD is at most η . Thus , if f ? ∈ F is feasible for the standard optimization program , this observation ( used twice ) implies that the accuracy of the output classifier measured with respect to D is within 2η of the accuracy of f ? measured with respect D ( Equation ( 8 ) ) . However , without any modifications , the classifier output by the standard optimization program could still have a fairness value much lower than τ with respect to D ( see Example A.27 ) . To bypass this , we introduce the notion of s-stability that allows us to lower bound the fairness value of a classifier with respect to D given its fairness value on Ŝ . Roughly , f ∈ F is said to be s-stable with respect to a fairness metric if for any Ŝ that is generated by an η-Hamming adversary , the ratio of fairness value of f with respect to D and with respect to Ŝ is between s and 1/s ( see Definition 4.7 ) . It follows that any s-stable classifier that has fairness value τ ′ > 0 with respect to Ŝ , has fairness value at least s · τ ′ with respect to D. Hence , an optimization program that ensures that all feasible classifiers are s-stable ( for a suitable choice of s ) and have fairness value at least τ ′ > 0 with respect to Ŝ , comes with a guarantee that any feasible classifier has a fairness value at least s · τ ′ ( with respect to D ) . If such an optimization program could further ensure that f ? is feasible for it , then by arguments presented above , the classifier output by this optimization program would satisfy required guarantees on both fairness and accuracy ( Lemma 4.9 ) . The issue is that , to directly enforce s-stability , one needs to compute the fairness values of classifiers with respect to D , but this is not possible in the absence of samples from D. We overcome this by present a “ proxy ” constraint on the classifier ( Equation ( 5 ) ) that involves only Ŝ and ensures that any classifier that satisfies it is s-stable . Moreover , f ? satisfies this constraint under Assumption 1 . Overall , modifying Program ( 2 ) to include this constraint ( Equation ( 5 ) ) with a suitable value of s , and setting an appropriate fairness threshold τ so that f ? remains feasible , leads us to our framework .

Summary: They consider the problem offFair classification in the adversarially perturbed setting — that is, the adversary gets to choose eta fraction of the sample arbitrarily and change their protected attributes arbitrarily.   They show an optimization problem that ensures fairness and accuracy guarantees under a mild assumption and show a mild assumption is necessary for those guarantees.   Finally, they have some experiments in which they consider two well thought out adversaries.