Zero-shot Fairness with Invisible Demographics

1 INTRODUCTION . Machine learning is already involved in decision-making processes that affect peoples ’ lives such as in screening job candidates ( Raghavan et al. , 2020 ) and in pricing credit ( Hurley & Adebayo , 2017 ) . Efficiency can be improved , costs can be reduced , and personalization of services and products can be greatly enhanced – these are some of the drivers for the widespread development and deployment of machine learning algorithms . Algorithms such as classifiers , however , are trained from large amount of labeled data , and can therefore encode and even reinforce past discriminatory practices that are present in the data . The classifier might treat some groups of individuals unfavorably , for example , denying credit on the grounds of language , gender , age and their combined effect . Algorithmic fairness aims at building machine learning algorithms that can take biased datasets and outputs fair/unbiased decisions for people with differing protected attributes , such as race , gender , and age . A typical setting of algorithmic fairness is as follows . We are given a training set of observations x ∈ X , their corresponding protected attributes s ∈ S , and the target label y ∈ Y for learning a classifier . In a statistical notion of algorithmic fairness e.g . ( Kamiran & Calders , 2012a ; Hardt et al. , 2016 ; Zafar et al. , 2017 ) , we control the discrepancy of a classifier ’ s loss for a small number of demographic groups defined on protected attributes . Recently , several works have considered the setting where protected attributes are unknown ( Kearns et al. , 2018 ; Hashimoto et al. , 2018 ; Khani et al. , 2019 ) . They aim to control the losses of groups whose size is greater than some predefined value . These works focus on an abstract worst-off group rather than demographic groups . It has been noted that the implied worst-off groups may differ from well-specified demographic groups who are known to suffer from past discriminatory practices ( Hashimoto et al. , 2018 ) . We are interested in the setting that is in between having complete annotations for demographic groups and having none . In this paper , we introduce algorithmic fairness with invisible demographics . Who are the invisible demographics ? In the context of machine learning systems , those are individuals with thin or non-existent labeled training data . The invisible population is primarily composed of individuals with certain protected attributes ( Hendricks , 2005 ; Abualghaib et al. , 2019 ; Perez , 2019 ) . We now elaborate on several algorithmic decision scenarios involving invisible demographics . One scenario is when we observe partial outcomes for some of the demographic groups , e.g . we have labeled training data for males ( with positive and negative outcomes ) , but for the group of females , we only observe the one-sided labels ( negative outcome ) . Another scenario is when we do not observe any outcome for some of the demographic ( sub ) groups , e.g . we have training samples for white-skinned and dark-skinned males , and white-skinned females , but we have zero labeled data for dark-skinned females . An extreme version of the last scenario is when we do not observe any outcome for females regardless of their skin colors , e.g . we only have training samples for males and no training examples for females . To summarize , in the invisible demographics problem , we define the demographics groups that are expected to be seen , so they are not abstract . However , not all of the demographics are observed ( labeled ) during training , forming missing or invisible demographics . This paper presents learning disentangled representations in the presence of invisible demographics . Our source of supervision is motivated by the observation that we want to deploy our classifier to the eventual real-world population . This deployment dataset will contain individuals from all demographics . We thus consider the setting where unlabeled data is available for learning disentangled representation . We call this data a context set and this context set is much like the deployment dataset , it is unlabeled but it contains all demographics including the invisible ones . We aim to convert our unlabeled context set into a perfect dataset ( Kleinberg et al. , 2016 ; Chouldechova , 2017 ) , a dataset in which the target label and protected attribute are independent ( i.e . y ⊥ s ) . We will then use this perfect dataset as the inductive bias for learning disentangled representations . How do we construct this perfect dataset without labels ? We assume that the number of demographic groups ( hence clusters ) is known a priori corresponding to the diverse demographic groups in the real-world population in which our machine learning system will be deployed . We use unsupervised kmeans clustering , or a supervised clustering based on rank statistics ; the latter one allows to form the clusters that also support annotations in the train data . Once the clusters have been found , we can equalize the cluster size to form a perfect dataset and use it as an input for learning a disentangled fair representation . See fig . 1 for an overview of our learning with invisible demographic framework . Specifically , our paper provides the following main contributions : 1 . A problem of algorithmic fairness with invisible demographics where we have zero data for some of demographics and we still have to make predictions for those groups . 2 . Applying clustering methods to the task of transforming unlabeled context dataset into a perfect dataset . 3 . Theoretical and experimental justification that the disentangled model with the perfect dataset as an inductive bias provides a well-disentangled fair representation , one component captures the demographic factors and another component is invariant to them . Related work We describe related work in three areas : zero-shot learning , semi-supervised learning , and disentangled representation learning . On zero-shot learning . The setting with incomplete training data , where we aim to account for seen and unseen outcomes is also known as generalized zero-shot learning . Traditionally , zero-shot learning transfers knowledge from classes for which we have training data to classes for which we do not have via auxiliary knowledge , e.g . via prototype examples ( Larochelle et al. , 2008 ) , intermediate class description such as semantic attributes ( Lampert et al. , 2009 ; Xian et al. , 2018 ) , word2vec embeddings ( Bucher et al. , 2019 ) . Our method similarly uses a context set as a source of auxiliary knowledge but in in contrast to generalized zero-shot learning , our context set is an unlabeled pool of data , where class descriptions are unknown . On semisupervised learning . Wick et al . ( 2019 ) proposed a semi-supervised method that can successfully harness unlabeled data to correct for the selection bias and label bias in the training data . The unlabeled data , despite not containing the target label y , is labeled in terms of the protected variable s. Our setting is significantly harder because there is no label information about y and s in the context set . On disentangled representations learning . Locatello et al . ( 2019a ) suggested that disentanglement in representation learning may be a useful property to encourage fairness when protected variables are not observed . In order for disentangled representations to improve fairness measure without the knowledge of protected attribute s , they have to assume that the target label y and the protected attribute s are independent , i.e . y ⊥ s. Though , in fairness settings , the variable s is correlated with the variable y , and therefore unsupervised methods are not suitable for fairness ( Jaiswal et al. , 2018b ; 2019 ) . Indeed , experiments in ( Locatello et al. , 2019a ) were wholly done with procedurally generated synthetic datasets involving 2D and 3D shapes . Without some supervision or inductive bias , disentangled representation methods would not solve the issue of algorithmic fairness with invisible demographics ( Locatello et al. , 2019b ) . 2 METHODOLOGY . 2.1 THEORETICAL BACKGROUND . In this section , we first formulate mathematically the problem of invisible demographics and its associated issue of algorithmic fairness . We then motivate theoretically the idea of perfect dataset for achieving fairness , and its use for an inductive bias in learning disentangled representations . Invisible demographics and algorithmic fairness . Let S denote the set of discrete-valued protected attributes with an associated domains S. S can take the values taken by a single protected attribute , or , S = S1 × S2 × . . .× Sp with S1 , . . . , Sp be discrete-valued protected attributes more generally . X , with the associated domain X , represents other attributes of the data . Let Y denote the space of class labels for a classification task ( Y = { 0 , 1 } for binary classification or Y = { 1 , 2 , . . . , Ccls } for multi-class classification ) . For ease of exposition , we assume that we have multiple sourcesM of samples , one for each combination of class label y and protected attribute s. That is , we have : Mys , ∀y ∈ Y , ∀s ∈ S , ( 1 ) where , for example , the source My=0 , s=0 contains all data points with class label y = 0 and protected attribute s = 0 . As in a standard supervised learning task , we have access to a training set Dtr = { ( xi , si , yi ) } , that is used to learn a model M : X → Y . Dtr is composed of several sources . This labeled training dataset , however , lacks samples from some of the sources : ∃y ∈ Y , ∃s ∈ S : Dtr ∩Mys = ∅ . ( 2 ) For example , we might not have samples from two sources : My=0 , s=0 andMy=1 , s=0 . In binary classification , this corresponds to zero-labeled data for the invisible demographic group s = 0 . Or we only observe a negative outcome for the invisible demographic s = 0 , i.e . we haveMy=1 , s=0 = ∅ . Once the model M is trained , we deploy it to the real-world population with diverse demographic groups . That is , we have a deployment set , Dt = { ( xi ) } which has overlap with all sources : Dt ∩Mys 6= ∅ ∀y ∈ Y , ∀s ∈ S. ( 3 ) If the model relies only on the incomplete training set , it is not unreasonable to expect that the model to easily misunderstand the invisibles . We can all agree that this sounds unfair , and we would like to rectify this . We will be precise shortly about the adopted mathematical definitions of fairness . We propose to alleviate the issue of unfairness to the invisibles by mixing labeled with unlabeled data , which is usually much cheaper to obtain . In this paper , we call this unlabeled data a context set Dctx = { ( xi ) } . This context set has overlap with all sources : Dctx ∩Mys 6= ∅ ∀y ∈ Y , ∀s ∈ S ( 4 ) The context set is much like the deployment set : it has no information about class labels y or the protected attributes s. We adopt a statistical notion of algorithmic fairness in which it balances a certain condition between groups of individuals with different protected attributes . The term ȳ below is the prediction of a machine learning model M . Several statistical fairness criteria have been proposed ( Kamiran & Calders , 2012a ; Hardt et al. , 2016 ; Zafar et al. , 2017 ; Chouldechova , 2017 ; Raghavan et al. , 2020 ) ( shown below for the case where s and y are binary ) : Pr ( ȳ = 1|s = 0 ) = Pr ( ȳ = 1|s = 1 ) ( equality of acceptance rate ) ( 5 ) Pr ( ȳ = 1|s = 0 , y ) = Pr ( ȳ = 1|s = 1 , y ) ( equality of true positive/negative rate ) ( 6 ) Pr ( y = 1|s = 0 , ȳ ) = Pr ( y = 1|s = 1 , ȳ ) ( equality of positive/negative predicted value ) ( 7 ) Generally , those statistical notions can be expressed in terms of different ( conditional ) independence statements between the involved random variables ( Barocas et al. , 2019 ) : ȳ ⊥ s ( equation 5 ) , ȳ ⊥ s | y ( equation 6 ) , and y ⊥ s | ȳ ( equation 7 ) . If our training set has no positive outcome for the demographic s = 0 , i.e . My=1 , s=0 = ∅ , the true positive rate for this group will suffer , and therefore we will likely not be able to satisfy , among others , equality of true positive rate . Perfect dataset . We call a dataset for which y ⊥ s holds , a perfect dataset ( Chouldechova , 2017 ; Kleinberg et al. , 2016 ) . If we have access to a perfect dataset , we could equalize true positive/negative rates ( eq . 6 ) and also equalize positive/negative predicted values ( eq . 7 ) for all demographic groups . This can be shown by using the sum and product rule of conditional probabilities , e.g . ( Kannan et al. , 2019 ) . Let ’ s consider a binary-valued protected attribute , s′ versus s′′ . For s′ , we can compute : Pr ( y = 1|ȳ = 1 , s′ ) = Pr ( ȳ=1|y=1 , s′ ) Pr ( y=1|s′ ) / ( Pr ( ȳ=1|y=1 , s′ ) Pr ( y=1|s′ ) +Pr ( ȳ=1|y=0 , s′ ) ( 1−Pr ( y=1|s′ ) ) ) , and accordingly for s′′ . The conditional probability on the left hand side is a positive predicted value , and this quantity can be expressed in terms of true positive/negative rates and the base ( prior ) rate , shown on the right hand side . If we have a perfect dataset ( y ⊥ s holds , which means equal base rates Pr ( y = 1|s′ ) = Pr ( y = 1|s′′ ) ) , an equality in the true positive/negative rates will give us an equality in the positive/negative predicted values . Similarly , with a perfect dataset , we can equalize true positive/negative rates ( eq . 6 ) and also acceptance rates ( eq . 5 ) for all demographic groups . From the sum probability rule , we have : Pr ( ȳ = 1|s′ ) = Pr ( ȳ = 1|y = 1 , s′ ) Pr ( y = 1|s′ ) + Pr ( ȳ = 1|y = 0 , s′ ) ( 1− Pr ( y = 1|s′ ) ) for s′ value , and accordingly for s′′ value . Here , an acceptance rate on the left hand side is related to true positive/negative rates and the base ( prior ) rate as shown on the right hand side . In general , however , our given dataset is likely to be imperfect . In this paper , we pursue learning a fair classifier for all demographics as learning disentangled representations with an approximately perfect dataset . Disentangled representation . Disentanglement learning aims to find a split representation of a data point x and a mapping function f such that f ( x ) = ( z1 , z2 , . . . , zp ) where z1 , z2 , . . . , zp are p distinct ( independent ) factors of variations . We can mathematically formalize this intuitive definition using group and representation theories Higgins et al . ( 2018 ) , or using structural causal models Suter et al . ( 2019 ) . Specifically in this paper , we would like to split representation of data into two factors as f ( x ) = ( zy , zs ) where zy contains factors that are relevant for y-prediction and zs contains factors related to demographic group s. As noted by Jaiswal et al . ( 2018a ; 2019 ) ( also vide sec . 1 ) , since the protected variable s is correlated with the class label y , we need annotations of undesired nuisance variable s to be successful in using disentanglement learning methods for fairness . We only have annotations of variable s in the training set Dtr = { ( xi , si , yi ) } , however , crucially , this set contains missing demographic groups . We have all demographic groups in the context set Dctx = { ( xi ) } ( also in the deployment set Dtr = { ( xi ) } ) , though , the challenge is we should not expect annotations of protected variable s at the deployment time . The next section will show that we can still leverage the context set for learning the disentangled representations . Disentanglement with a perfect dataset . Our framework for learning the disentangled representations comprises four core modules : 1 ) an encoder function f that embeds x into a bipartite space f ( x ) → ( zy , zs ) ; 2 ) a decoder function g that learns the inverse of f , mapping back from the embedded space into the input domain g ( zy , zs ) → x̃ ; 3 ) a predictor function l that predicts y from zy , and 4 ) a discriminator function h that classifies whether a given batch of samples embedded in zy derives from the either context set or the training set ; this marks a significant departure from the typical GAN discriminator , which takes as input batches of data and yields a prediction for each sample independently of the other samples in the batch . Fig . 2a shows our framework , where the training signal comes from the perfect dataset . Formally , given the training set , Dtr and samples from the balanced ( i.e . perfect – see section 2.2 for how this details on how this can be practically achieved ) context set Xperf , our learning objective can be written as : Lmatch = ∑ x∈Xtr ⋃ Xperf Lrecon ( x , g ( zs , zy ) ) + λ1 ∑ x∈Xtr Lsup ( y , l ( zy ) ) + + λ2 ( log h ( f ( zy ⊂ Xperf ) ) + log ( 1− h ( f ( zy ⊂ Xtr ) ) ) , ( 8 ) where Lrecon and Lsup denote the reconstruction loss , and supervised loss , respectively , and λ1 and λ2 are pre-factors . In practice , this objective is computed over mini-batches , B , and the discriminator h is trained via the standard JSD loss ( Goodfellow et al. , 2014 ) to map a batch of data points from the training set and the context set to a binary label : 1 if the batch is judged to have been sampled from the context set , 0 otherwise . Its goal is to effectively estimate the probability that a batch of samples , as a set , has been sampled from one distribution or the other . Since the task is a set-prediction one , we require that the function it defines respects the exchangeability of the batch dimension – that is the discriminator ’ s predictions should take into account dependencies between samples in a batch but should be invariant to the order in which they appear , i.e . we have h ( { zy } Bb=i ) = h ( { zy } Bb=π ( i ) ) for all permutations π ∈ Π . To render the entirety of the function h composed of sub-functions h1 ( h2 ( h3 ... ) ) ) , it requires only the innermost , sub-function , ρ in the chain to have this property . While there are a number of choices when it comes to defining ρ , we choose a weighted average ρ = 1B ∑ i ( { attention ( zy ) } Bb=i ) , with weights computed according to a learned attention mechanism . It takes the form of the scaled dot-product attention ( Vaswani et al. , 2017 ) attention ( Q , K , V ) : = softmax ( QKT / √ d ) V , , weighting values ( V ) according to the similarity between the associated key ( K ) and query ( Q ) matrices , as measured by their dot-product . Q , K , and V are used after they have been embedded into linear subspaces by matrix-multiplication with learned weight matrices of dimension Rm×d . We found that defining K and V as zy , and Q as the mean of zy over B , yielded good results and leave it to future work to explore more sophisticated methods . The result of ρ is then processed by a series of fully-connected layers , following the DeepSets ( Zaheer et al. , 2017 ) paradigm , which ultimately computes a single prediction for the current batch . We know that the independence condition y ⊥ s holds in the perfect set , but not in the training set due to sampling bias . To do well , the discriminator should rely on this knowledge . More concretely , since the context and training set have differing support over S×Y , namely ( Str×Ytr ) ( ( Sperf×Yperf ) , that support serves as an indicator of the distribution from which the data has been drawn . The scenarios we consider dictate Ytr = Yperf , making the disentangling well-posed . However , since we wish to use Sctx × Yctx \ Str × Ytr as the training signal for the encoder , and not the relative frequency of the target classes , it is important that , like the context set , we weight the samples of the training set such that p ( str|ytr ) p ( ytr ) is equal for all str , ytr ∈ Str × Ytr . To guide the network towards the desired solution , we supplement this implicit constraint with the explicit constraint that zy be predictive of y , which we achieve using a linear predictor l ; whenever we have dim ( S ) > 1 ( in our experiments this corresponds to the partial outcomes setting ) we also impose the same constraint on zs , but with respect to s. With these conditions met , to fool the discriminator , the encoder must separate out information pertaining to S into the embedded space zs not part of the discriminator ’ s input , leaving only unprotected information in zy .

Summary: This paper tackles a fair classification problem with an invisible demographic, a situation where the records who have some specific target labels and sensitive attributes are missing. In this setting, the authors introduce a disentangled representation learning framework to make the resultant classifier fair by taking advantage of the additional dataset, context dataset. They demonstrate by the empirical evaluations that the proposed disentangled representation learning algorithm success to mitigate unfair bias by utilizing the perfect dataset, a dataset in which the target label and sensitive attribute are independent. Usually, the perfect dataset is unavailable; hence, they introduce a method to convert the context dataset into the perfect dataset. The authors also show that even if the context dataset is not perfect, the presented method successes to mitigate an unfair bias.