Certifying Robustness to Programmable Data Bias in Decision Trees

1 Introduction . The proliferation of machine-learning algorithms has raised alarming questions about fairness in automated decision-making [ 4 ] . In this paper , we focus our attention on bias in training data . Data can be biased due to societal inequities , human biases , under-representation of minorities , malicious data poisoning , etc . For instance , historical data can contain human biases , e.g. , certain individuals ’ loan requests get rejected , although ( if discrimination were not present ) they should have been approved , or women in certain departments are consistently given lower performance scores by managers . Given biased training data , we are often unable to de-bias it because we do not know which samples are affected . This paper asks , can we certify ( prove ) that our predictions are robust under a given form and degree of bias in the training data ? We aim to answer this question without having to show which data are biased ( i.e. , poisoned ) . Techniques for certifying poisoning robustness ( i ) focus on specific poisoning forms , e.g. , label-flipping [ 31 ] , or ( ii ) perform certification using defenses that create complex , uninterpretable classifiers , e.g. , due to randomization or ensembling [ 23 , 24 , 31 ] . To address limitation ( i ) , we present programmable bias definitions that model nuanced biases in practical domains . To address ( ii ) , we target existing decision-tree learners—considered interpretable and desirable for sensitive decision-making [ 32 ] —and exactly certify their robustness , i.e. , provide proofs that the bias in the data will not affect the outcome of the trained model on a given point . We begin by presenting a language for programmatically defining bias models . A bias model allows us to flexibly specify what sort of bias we suspect to be in the data , e.g. , up to n % of the women may have wrongly received a negative job evaluation . Our bias-model language is generic , allowing ∗ Author ’ s name in native alphabet : ú Gñ « Q . Ë @ ð @ 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . us to compose simpler bias models into more complex ones , e.g. , up to n % of the women may have wrongly received a negative evaluation and up to m % of Black men ’ s records may have been completely missed . The choice of bias model depends on the provenance of the data and the task . After specifying a bias model , our goal is to certify pointwise robustness to data bias : Given an input x , we want to ensure that no matter whether the training data is biased or not , the resulting model ’ s prediction for x remains the same . Certifying pointwise robustness is challenging . One can train a model for every perturbation ( as per a bias model ) of a dataset and make sure they all agree . But this is generally not feasible , because the set of possible perturbations can be large or infinite . Recall the bias model where up to n % of women may have wrongly received a negative label . For a dataset with 1000 women and n = 1 % , there are more than 1023 possible perturbed datasets . To perform bias-robustness certification on decision-tree learners , we employ abstract interpretation [ 12 ] to symbolically run the decision-tree-learning algorithm on a large or infinite set of datasets simultaneously , thus learning a set of possible decision trees , represented compactly . The crux of our approach is a technique that lifts operations of decision-tree learning to symbolically operate over a set of datasets defined using our bias-model language . As a starting point , we build upon Drews et al. ’ s [ 16 ] demonstration of poisoning-robustness certification for the simple bias model where an adversary may have added fake training data . Our approach completely reworks and extends their technique to target the bias-robustness problem and handle complex bias models , including ones that may result in an infinite number of datasets . Contributions . We make three contributions : ( 1 ) We formalize the bias-robustness-certification problem and present a language to compositionally define bias models . ( 2 ) We present a symbolic technique that performs decision-tree learning on a set of datasets defined by a bias model , allowing us to perform certification . ( 3 ) We evaluate our approach on a number of bias models and datasets from the fairness literature . Our tool can certify pointwise robustness for a variety of bias models ; we also show that some datasets have unequal robustness-certification rates across demographics groups . Running example . Consider the example in Fig . 1 ; our goal is to classify who should be hired based on a test score . A standard decision-tree-learning algorithm would choose the split ( predicate ) score 6 3 , assuming we restrict tree depth to 1.2 As shown in Fig . 1 ( middle ) , the classification depends on the data split ; e.g. , on the right hand side , we see that a person with score > 3 is accepted , because the proportion ( “ probability ” ) of the data with positive labels and score > 3 is 4/5 ( > 1/2 ) . Now suppose that our bias model says that up to one Black person in the dataset may have received a wrongful rejection . Our goal is to show that even if that is the case , the prediction of a new test sample x will not change . As described above , training decision trees for all possible modified datasets is generally intractable . Instead , we symbolically learn a set of possible decision trees compactly , as illustrated in Fig . 1 ( right ) . In this case the learning algorithm always chooses score 6 3 ( generally , our algorithm can capture all viable splits ) . However , the proportion of labels on either branch varies . For example , on the right , if the highlighted sample is wrongly labeled , then the ratio changes from 0.8 to 1 . To efficiently perform this calculation , we lift the learning algorithm ’ s operations to interval arithmetic and represent the probability as [ 0.8 , 1 ] . Given a new test sample x = 〈race=Black , score=7〉 , we follow the right branch and , since the interval is always larger than 0.5 , we certify that 2Other predicates , e.g. , score 6 4 , will yield the same split . We choose a single split for illustrative purposes here . ( The implementation considers all possible splits that yield distinct partitions , so it would consider score 6 3 and score 6 4 as a single entity . ) the algorithm is robust for x . In general , however , due to the use of abstraction , our approach may fail to find tight intervals , and therefore be unable to certify robustness for all robust inputs . 2 Related work . Ties to poisoning . Our dataset bias language captures existing definitions of data poisoning , where an attacker is assumed to have maliciously modified training data . Poisoning has been studied extensively . Most works have focused on attacks [ 6 , 11 , 25 , 33 , 38 , 39 , 40 ] or on training models that are empirically less vulnerable ( defenses ) [ 3 , 9 , 19 , 28 , 31 , 36 ] . Our work differs along a number of dimensions : ( 1 ) We allow programmatic , custom , composable definitions of bias models ; notably , to our knowledge , no other work in this space allows for targeted bias , i.e. , restricting bias to a particular subgroup . ( 2 ) Our work aims to certify and quantify robustness of an existing decision-tree algorithm , not to modify it ( e.g. , via bagging or randomized smoothing ) to improve robustness [ 23 , 24 , 31 ] . Statistical defenses show that a learner is robust with high probability , often by modifying a base learner using , e.g. , randomized smoothing [ 31 ] , outlier detection [ 36 ] , or bagging [ 22 , 23 ] . Nonstatistical certification ( including abstract interpretation ) has mainly focused on test-time robustness , where the vicinity ( e.g. , within an ` p norm ) of an input is proved to receive the same prediction [ 2 , 21 , 30 , 35 , 37 , 1 ] . Test-time robustness is a simpler problem than our train-time robustness problem because it does not have to consider the mechanics of the learner on sets of datasets . The only work we know of that certifies train-time robustness of decision trees is by Drews et al . [ 16 ] and focuses on poisoning attacks where an adversary adds fake data . Our work makes a number of significant leaps beyond this work : ( 1 ) We frame data bias as programmable , rather than fixed , to mimic real-world bias scenarios , an idea that has gained traction in a variety of domains , e.g. , NLP [ 41 ] . ( 2 ) We lift a decision-tree-learning algorithm to operate over sets of datasets represented via our bias-model language . ( 3 ) We investigate the bias-robustness problem through a fairness lens , particularly with an eye towards robustness rates for various demographic groups . Ties to fairness . The notion of individual fairness specifies that similar individuals should receive similar predictions [ 18 ] ; by contrast , we certify that no individual should receive different predictions under models trained by similar datasets . Black and Fredrickson explore the problem of how individuals ’ predictions change under models trained by similar datasets , but their concept of similarity is limited to removing a single data point [ 7 ] . Data bias , in particular , has received some attention in the fairness literature . Chen et al . suggest adding missing data as an effective approach to remedying bias in machine learning [ 10 ] , which is one operation that our bias language captures . Mandal et al . build on the field of distributional robustness [ 5 , 27 , 34 ] to build classifiers that are empirically group-fair across a variety of nearby distributions [ 26 ] . Our problem domain is related to distributional robustness because we certify robustness over a family of similar datasets ; however , we define specific data-transformation operators to define similarity , and , unlike Mandal et al. , we certify existing learners instead of building empirically robust models . Ties to robust statistics . There has been renewed interest in robust statistics for machine learning [ 13 , 14 ] . Much of the work concerns outlier detection for various learning settings , e.g. , estimating parameters of a Gaussian . The distinctions are two-fold : ( 1 ) We deal with rich , nuanced bias models , as opposed to out-of-distribution samples , and ( 2 ) we aim to certify that predictions are robust for a specific input , a guarantee that can not be made by robust-statistics-based techniques [ 15 ] . 3 Defining data bias programmatically . We define the bias-robustness problem and a language for defining bias models programmatically . Bias models . A dataset T ⊆ X ×Y is a set of pairs of samples and labels , where Y = { 0 , . . . , n−1 } . For a dataset T , we will use TX to denote { x | ( x , y ) ∈ T } . A bias model B is a function that takes a dataset and returns a set of datasets . We call B ( T ) ⊆ 2X×Y a bias set . We assume that T ∈ B ( T ) . Intuitively , B ( T ) represents all datasets that could have existed had there been no bias . Pointwise data-bias robustness . Assume we have a learning algorithm A that , given a training dataset T , deterministically returns a classifier hT from some hypothesis classH . Fix a dataset T and bias model B . Given a sample x ∈ X , we say that A is pointwise robust ( or robust for short ) on x iff there is a label i such that for all T ′ ∈ B ( T ) , we have hT ′ ( x ) = i ( 1 ) Basic components of a bias model . We begin with basic bias models . Missing data : A common bias in datasets is missing data , which can occur via poor historical representation of a subgroup ( e.g. , women in CS-department admissions data ) , or from present-day biases or shortsightedness ( e.g. , a survey that bypasses low-income neighborhoods ) . Using the parameter m as the maximum number of missing elements , we formally define : MISSm ( T ) = { T ′ | T ′⊇T , |T ′\T | 6 m } MISSm ( T ) defines an infinite number datasets when the sample space is infinite ( e.g. , R-valued ) . Example 3.1 . Using T ex from Fig . 1 , MISS1 ( T ex ) is the set of all datasets that are either T ex or T ex plus any new element ( x , y ) with arbitrary race , score , and label . Label flipping : Historical data can contain human biases , e.g. , in loan financing , certain individuals ’ loan requests get rejected due to discrimination . Or consider employee-performance data , where women in certain departments are consistently given lower scores by managers . We model such biases as label flipping , where labels of up to l individuals in the dataset may be incorrect : FLIPl ( T ) = { T ′ | |T |=|T ′| , |T\T ′| 6 l , TX = T ′X } Example 3.2 . Using T ex from Fig . 1 and a bias model FLIP1 , we have FLIP1 ( T ex ) = { T ex0 , . . . , T ex3 , T ex5 , . . . , T ex 9 , T ex } , where T exi is T ex with the label of the element with score = i changed . Fake data : Our final bias model assumes the dataset may contain fake data . One cause may be a malicious user who enters fraudulent data into a system ( often referred to as poisoning ) . Alternatively , this model can be thought of as the inverse of MISS , e.g. , we over-collected data for men . FAKEk ( T ) = { T ′ | T ′⊆T , |T\T ′| 6 k } Example 3.3 . Using T ex from Fig . 1 and a bias model FAKE1 , we get FAKE1 ( T ex ) = { T ex↓0 , . . . , T ex↓3 , T ex↓5 , . . . , T ex ↓9 , T ex } where T ex↓i is T ex such that the element with score = i has been removed . Targeted bias models . Each bias model has a targeted version that limits the bias to a specified group of data points . For example , consider the missing data transformation . If we suspect that data about women is missing from an HR database , we can limit the MISS transformation to only add data points with gender = female . Formally , we define a predicate g : X × Y → B , where B = { true , false } . MISSgm ( T ) = { T ′ | T ′ ⊇ T , |T ′ \ T | 6 m , and g ( x , y ) is true ∀ ( x , y ) ∈ ( T ′ \ T ) } Targeted versions of label-flipping and fake data can be defined in a similar way . Example 3.4 . In Fig . 1 ( right ) , we used bias model FLIPg1 , where g targets Black people with negative labels . This results in the bias set FLIPg1 ( T ex ) = { T ex1 , T ex8 , T ex } , where T exi is T ex with the label of the element with score = i changed ( recall that scores 1 and 8 belong to Black people in T ex ) . Composite bias models . We can compose basic components to generate a composite model . Specifically , we define a composite model B as a finite set of arbitrary basic components , that is , B = [ MISSg1m1 , . . . , MISS gj mj , FLIP gj+1 l1 , . . . , FLIP gj+p lp , FAKE gj+p+1 k1 , . . . , FAKE gj+p+q kq ] ( 2 ) B ( T ) is generated from T by applying the basic components of B iteratively . We must apply the constituent components in an optimal order , i.e. , one that generates all datasets that can be created through applying the transformers in any order . To do this , we apply components of the same type in any order and apply transformers of different types in the order MISS , FLIP , FAKE ( see Appendix ) . Example 3.5 . Suppose B = [ MISSg12 , FAKE1 ] . Then B ( T ) is the set of all datasets obtained by adding up to 2 arbitrary data points that satisfy g1 to T , and then removing any up to 1 data point .

Summary: This paper introduces and explores a framework for certifying that classification models produced by a machine learning algorithm are making predictions on examples that are robust to bias in the training data.  Certification is per example: a certification for an example means that the same prediction would have been made on that example if the training data used to train that model had been perturbed in ways that correspond to potentially eliminating a specified type of bias in the dataset.   The paper presents an algorithm aimed at certifying robustness for a standard decision tree learner and proves that it is sound.  It explores the algorithm in a series of experiments on standard datasets, with different bias definitions and across different demographic groups.