Certifying Robustness to Programmable Data Bias in Decision Trees

1 Introduction . The proliferation of machine-learning algorithms has raised alarming questions about fairness in automated decision-making [ 4 ] . In this paper , we focus our attention on bias in training data . Data can be biased due to societal inequities , human biases , under-representation of minorities , malicious data poisoning , etc . For instance , historical data can contain human biases , e.g. , certain individuals ’ loan requests get rejected , although ( if discrimination were not present ) they should have been approved , or women in certain departments are consistently given lower performance scores by managers . Given biased training data , we are often unable to de-bias it because we do not know which samples are affected . This paper asks , can we certify ( prove ) that our predictions are robust under a given form and degree of bias in the training data ? We aim to answer this question without having to show which data are biased ( i.e. , poisoned ) . Techniques for certifying poisoning robustness ( i ) focus on specific poisoning forms , e.g. , label-flipping [ 31 ] , or ( ii ) perform certification using defenses that create complex , uninterpretable classifiers , e.g. , due to randomization or ensembling [ 23 , 24 , 31 ] . To address limitation ( i ) , we present programmable bias definitions that model nuanced biases in practical domains . To address ( ii ) , we target existing decision-tree learners—considered interpretable and desirable for sensitive decision-making [ 32 ] —and exactly certify their robustness , i.e. , provide proofs that the bias in the data will not affect the outcome of the trained model on a given point . We begin by presenting a language for programmatically defining bias models . A bias model allows us to flexibly specify what sort of bias we suspect to be in the data , e.g. , up to n % of the women may have wrongly received a negative job evaluation . Our bias-model language is generic , allowing ∗ Author ’ s name in native alphabet : ú Gñ « Q . Ë @ ð @ 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . us to compose simpler bias models into more complex ones , e.g. , up to n % of the women may have wrongly received a negative evaluation and up to m % of Black men ’ s records may have been completely missed . The choice of bias model depends on the provenance of the data and the task . After specifying a bias model , our goal is to certify pointwise robustness to data bias : Given an input x , we want to ensure that no matter whether the training data is biased or not , the resulting model ’ s prediction for x remains the same . Certifying pointwise robustness is challenging . One can train a model for every perturbation ( as per a bias model ) of a dataset and make sure they all agree . But this is generally not feasible , because the set of possible perturbations can be large or infinite . Recall the bias model where up to n % of women may have wrongly received a negative label . For a dataset with 1000 women and n = 1 % , there are more than 1023 possible perturbed datasets . To perform bias-robustness certification on decision-tree learners , we employ abstract interpretation [ 12 ] to symbolically run the decision-tree-learning algorithm on a large or infinite set of datasets simultaneously , thus learning a set of possible decision trees , represented compactly . The crux of our approach is a technique that lifts operations of decision-tree learning to symbolically operate over a set of datasets defined using our bias-model language . As a starting point , we build upon Drews et al. ’ s [ 16 ] demonstration of poisoning-robustness certification for the simple bias model where an adversary may have added fake training data . Our approach completely reworks and extends their technique to target the bias-robustness problem and handle complex bias models , including ones that may result in an infinite number of datasets . Contributions . We make three contributions : ( 1 ) We formalize the bias-robustness-certification problem and present a language to compositionally define bias models . ( 2 ) We present a symbolic technique that performs decision-tree learning on a set of datasets defined by a bias model , allowing us to perform certification . ( 3 ) We evaluate our approach on a number of bias models and datasets from the fairness literature . Our tool can certify pointwise robustness for a variety of bias models ; we also show that some datasets have unequal robustness-certification rates across demographics groups . Running example . Consider the example in Fig . 1 ; our goal is to classify who should be hired based on a test score . A standard decision-tree-learning algorithm would choose the split ( predicate ) score 6 3 , assuming we restrict tree depth to 1.2 As shown in Fig . 1 ( middle ) , the classification depends on the data split ; e.g. , on the right hand side , we see that a person with score > 3 is accepted , because the proportion ( “ probability ” ) of the data with positive labels and score > 3 is 4/5 ( > 1/2 ) . Now suppose that our bias model says that up to one Black person in the dataset may have received a wrongful rejection . Our goal is to show that even if that is the case , the prediction of a new test sample x will not change . As described above , training decision trees for all possible modified datasets is generally intractable . Instead , we symbolically learn a set of possible decision trees compactly , as illustrated in Fig . 1 ( right ) . In this case the learning algorithm always chooses score 6 3 ( generally , our algorithm can capture all viable splits ) . However , the proportion of labels on either branch varies . For example , on the right , if the highlighted sample is wrongly labeled , then the ratio changes from 0.8 to 1 . To efficiently perform this calculation , we lift the learning algorithm ’ s operations to interval arithmetic and represent the probability as [ 0.8 , 1 ] . Given a new test sample x = 〈race=Black , score=7〉 , we follow the right branch and , since the interval is always larger than 0.5 , we certify that 2Other predicates , e.g. , score 6 4 , will yield the same split . We choose a single split for illustrative purposes here . ( The implementation considers all possible splits that yield distinct partitions , so it would consider score 6 3 and score 6 4 as a single entity . ) the algorithm is robust for x . In general , however , due to the use of abstraction , our approach may fail to find tight intervals , and therefore be unable to certify robustness for all robust inputs . 2 Related work . Ties to poisoning . Our dataset bias language captures existing definitions of data poisoning , where an attacker is assumed to have maliciously modified training data . Poisoning has been studied extensively . Most works have focused on attacks [ 6 , 11 , 25 , 33 , 38 , 39 , 40 ] or on training models that are empirically less vulnerable ( defenses ) [ 3 , 9 , 19 , 28 , 31 , 36 ] . Our work differs along a number of dimensions : ( 1 ) We allow programmatic , custom , composable definitions of bias models ; notably , to our knowledge , no other work in this space allows for targeted bias , i.e. , restricting bias to a particular subgroup . ( 2 ) Our work aims to certify and quantify robustness of an existing decision-tree algorithm , not to modify it ( e.g. , via bagging or randomized smoothing ) to improve robustness [ 23 , 24 , 31 ] . Statistical defenses show that a learner is robust with high probability , often by modifying a base learner using , e.g. , randomized smoothing [ 31 ] , outlier detection [ 36 ] , or bagging [ 22 , 23 ] . Nonstatistical certification ( including abstract interpretation ) has mainly focused on test-time robustness , where the vicinity ( e.g. , within an ` p norm ) of an input is proved to receive the same prediction [ 2 , 21 , 30 , 35 , 37 , 1 ] . Test-time robustness is a simpler problem than our train-time robustness problem because it does not have to consider the mechanics of the learner on sets of datasets . The only work we know of that certifies train-time robustness of decision trees is by Drews et al . [ 16 ] and focuses on poisoning attacks where an adversary adds fake data . Our work makes a number of significant leaps beyond this work : ( 1 ) We frame data bias as programmable , rather than fixed , to mimic real-world bias scenarios , an idea that has gained traction in a variety of domains , e.g. , NLP [ 41 ] . ( 2 ) We lift a decision-tree-learning algorithm to operate over sets of datasets represented via our bias-model language . ( 3 ) We investigate the bias-robustness problem through a fairness lens , particularly with an eye towards robustness rates for various demographic groups . Ties to fairness . The notion of individual fairness specifies that similar individuals should receive similar predictions [ 18 ] ; by contrast , we certify that no individual should receive different predictions under models trained by similar datasets . Black and Fredrickson explore the problem of how individuals ’ predictions change under models trained by similar datasets , but their concept of similarity is limited to removing a single data point [ 7 ] . Data bias , in particular , has received some attention in the fairness literature . Chen et al . suggest adding missing data as an effective approach to remedying bias in machine learning [ 10 ] , which is one operation that our bias language captures . Mandal et al . build on the field of distributional robustness [ 5 , 27 , 34 ] to build classifiers that are empirically group-fair across a variety of nearby distributions [ 26 ] . Our problem domain is related to distributional robustness because we certify robustness over a family of similar datasets ; however , we define specific data-transformation operators to define similarity , and , unlike Mandal et al. , we certify existing learners instead of building empirically robust models . Ties to robust statistics . There has been renewed interest in robust statistics for machine learning [ 13 , 14 ] . Much of the work concerns outlier detection for various learning settings , e.g. , estimating parameters of a Gaussian . The distinctions are two-fold : ( 1 ) We deal with rich , nuanced bias models , as opposed to out-of-distribution samples , and ( 2 ) we aim to certify that predictions are robust for a specific input , a guarantee that can not be made by robust-statistics-based techniques [ 15 ] . 3 Defining data bias programmatically . We define the bias-robustness problem and a language for defining bias models programmatically . Bias models . A dataset T ⊆ X ×Y is a set of pairs of samples and labels , where Y = { 0 , . . . , n−1 } . For a dataset T , we will use TX to denote { x | ( x , y ) ∈ T } . A bias model B is a function that takes a dataset and returns a set of datasets . We call B ( T ) ⊆ 2X×Y a bias set . We assume that T ∈ B ( T ) . Intuitively , B ( T ) represents all datasets that could have existed had there been no bias . Pointwise data-bias robustness . Assume we have a learning algorithm A that , given a training dataset T , deterministically returns a classifier hT from some hypothesis classH . Fix a dataset T and bias model B . Given a sample x ∈ X , we say that A is pointwise robust ( or robust for short ) on x iff there is a label i such that for all T ′ ∈ B ( T ) , we have hT ′ ( x ) = i ( 1 ) Basic components of a bias model . We begin with basic bias models . Missing data : A common bias in datasets is missing data , which can occur via poor historical representation of a subgroup ( e.g. , women in CS-department admissions data ) , or from present-day biases or shortsightedness ( e.g. , a survey that bypasses low-income neighborhoods ) . Using the parameter m as the maximum number of missing elements , we formally define : MISSm ( T ) = { T ′ | T ′⊇T , |T ′\T | 6 m } MISSm ( T ) defines an infinite number datasets when the sample space is infinite ( e.g. , R-valued ) . Example 3.1 . Using T ex from Fig . 1 , MISS1 ( T ex ) is the set of all datasets that are either T ex or T ex plus any new element ( x , y ) with arbitrary race , score , and label . Label flipping : Historical data can contain human biases , e.g. , in loan financing , certain individuals ’ loan requests get rejected due to discrimination . Or consider employee-performance data , where women in certain departments are consistently given lower scores by managers . We model such biases as label flipping , where labels of up to l individuals in the dataset may be incorrect : FLIPl ( T ) = { T ′ | |T |=|T ′| , |T\T ′| 6 l , TX = T ′X } Example 3.2 . Using T ex from Fig . 1 and a bias model FLIP1 , we have FLIP1 ( T ex ) = { T ex0 , . . . , T ex3 , T ex5 , . . . , T ex 9 , T ex } , where T exi is T ex with the label of the element with score = i changed . Fake data : Our final bias model assumes the dataset may contain fake data . One cause may be a malicious user who enters fraudulent data into a system ( often referred to as poisoning ) . Alternatively , this model can be thought of as the inverse of MISS , e.g. , we over-collected data for men . FAKEk ( T ) = { T ′ | T ′⊆T , |T\T ′| 6 k } Example 3.3 . Using T ex from Fig . 1 and a bias model FAKE1 , we get FAKE1 ( T ex ) = { T ex↓0 , . . . , T ex↓3 , T ex↓5 , . . . , T ex ↓9 , T ex } where T ex↓i is T ex such that the element with score = i has been removed . Targeted bias models . Each bias model has a targeted version that limits the bias to a specified group of data points . For example , consider the missing data transformation . If we suspect that data about women is missing from an HR database , we can limit the MISS transformation to only add data points with gender = female . Formally , we define a predicate g : X × Y → B , where B = { true , false } . MISSgm ( T ) = { T ′ | T ′ ⊇ T , |T ′ \ T | 6 m , and g ( x , y ) is true ∀ ( x , y ) ∈ ( T ′ \ T ) } Targeted versions of label-flipping and fake data can be defined in a similar way . Example 3.4 . In Fig . 1 ( right ) , we used bias model FLIPg1 , where g targets Black people with negative labels . This results in the bias set FLIPg1 ( T ex ) = { T ex1 , T ex8 , T ex } , where T exi is T ex with the label of the element with score = i changed ( recall that scores 1 and 8 belong to Black people in T ex ) . Composite bias models . We can compose basic components to generate a composite model . Specifically , we define a composite model B as a finite set of arbitrary basic components , that is , B = [ MISSg1m1 , . . . , MISS gj mj , FLIP gj+1 l1 , . . . , FLIP gj+p lp , FAKE gj+p+1 k1 , . . . , FAKE gj+p+q kq ] ( 2 ) B ( T ) is generated from T by applying the basic components of B iteratively . We must apply the constituent components in an optimal order , i.e. , one that generates all datasets that can be created through applying the transformers in any order . To do this , we apply components of the same type in any order and apply transformers of different types in the order MISS , FLIP , FAKE ( see Appendix ) . Example 3.5 . Suppose B = [ MISSg12 , FAKE1 ] . Then B ( T ) is the set of all datasets obtained by adding up to 2 arbitrary data points that satisfy g1 to T , and then removing any up to 1 data point .

Summary: The paper focuses on robustness against bias in training data. In particular, the main goal of the paper is to develop a method that can certify models produced are pointwise-robust to potential dataset biases. Authors focus on decision-tree learning due to the interpretable nature of the models. Authors propose using a novel symbolic technique to evaluate a decision-tree learner on a large number of datasets, certifying that each and every dataset produces the same prediction for a specific test point.