Title: THE FAIRNESS-ACCURACY LANDSCAPE OF NEURAL CLASSIFIERS

Abstract: That machine learning algorithms can demonstrate bias is well-documented by now. This work confronts the challenge of bias mitigation in fully-connected feedforward neural nets through the lens of multiobjective optimisation. Specifically, recognising that fairness and accuracy are competing objectives, the proposed methodology uses techniques from multiobjective optimisation to ascertain the fairness-accuracy landscape of a neural net classifier. Along the way, a new causal notion of fairness is introduced that is particularly suited to giving a nuanced treatment when data is collected under unfair practices. In particular, special attention is paid to subjects whose covariates could appear with substantial probability in either value of the sensitive attribute. Experimental results suggest that the proposed method produces neural net classifiers that distribute evenly across the Pareto front of the fairness-accuracy space and is more efficient at finding non-dominated points than an adversarial approach.

1. INTRODUCTION:

There is increasing concern over the ethics of machine learning algorithms. The issue of machine bias was prominently featured in ProPublica's 2016 eponymous article (Angwin et al., 2016) where the investigation uncovered prejudice against African-Americans in COMPAS (Correctional Offender Management Profiling for Alternative Sanctions), a recidivism prediction tool developed by Northpointe. Efforts to mitigate machine bias has received steadily increasing attention from stakeholders in a wide array of arenas including academia, industry research labs, and advocacy groups.

The question of fairness has a long history (Sen & McMurrin, 1980) and its definition continues to be debated (Dwork et al., 2012; Chouldechova, 2016; Joseph et al., 2016) . For instance, Corbett-Davies et al. (2017) and Dieterich et al. (2016) argue that COMPAS can be regarded as fair with respect to certain fairness notions. Works such as Hardt et al. (2016) and Kleinberg (2018) have investigated this type of ambiguity by showing that certain fairness criteria cannot be simultaneously satisfied.

Having fixed a definition of fairness, most works in algorithmic fairness proceed by imposing constraints during the training process in an attempt to enforce fairness (Hardt et al., 2016; Joseph et al., 2016; Zafar et al., 2017a; b) . Recent works can handle very complex learning algorithms such as neural networks (Beutel et al., 2017; Wadsworth et al., 2018; Madras et al., 2018; Manisha & Gujar, 2018) . However none of the works in the literature seek to provide a holistic view of the fairness-accuracy landscape of the algorithm.

What we mean is this -though it is desirable that the neural network maintain high predictive accuracy while simultaneously remaining fair with regards to a sensitive variable such as race or gender, these two objectives often compete. Given this, it is helpful to cast the balancing act between fairness and accuracy as a multiobjective optimisation task and look at the fairness-accuracy Pareto front. This can give a bird's eye view of the algorithm and can be useful for comparing two algorithms based on, say, the "volume" of the Pareto front (Li et al., 2015) . In this work, we propose a methodlogy for estimating the fairness-accuracy Pareto front of a fully-connected feedforward network.

Contributions This is the first work in algorithmic fairness that specifically addresses estimation of the fairness-accuracy Pareto front of a fully-connected feedforward network. The framework presented is flexible enough to allow for user-supplied accuracy and fairness measures. Along the way, we introduce a causal measure of fairness which emphasises fairness amongst subjects with the most overlap in observed covariates across the different values of a sensitive attribute. Finally, the methodology can be used to enforce fairness in all intermediate representations of the neural network. This has potential benefits for downstream transfer learning tasks.

2. RELATED WORK:

In this section, we review the broad categories of existing methods in algorithmic fairness. The major discernible classes of fair learning methods can be trifurcated according to the stage in which action is taken. The first class of methods attempts to remove bias from the input data itself. These methods rest on the premise that once proper preprocessing is accomplished, any classifier can be used to subsequently produce fair predictions (Kamiran & Calders, 2012; Feldman et al., 2015; Calmon et al., 2017; Johndrow & Lum, 2019) .

On the other hand, post-processing techniques directly operate on the classifier output and are amenable to any classifier. The technique in Hardt et al. (2016) for instance seeks to learn a monotone transformation of the classifier's output to remove unfairness with regard to either demographic parity or equalised odds.

The third type of algorithmic fairness method directly intervenes in the stage of training. Many of these methods are specific to certain classifiers and certain notions of fairness. Generally speaking, train-time methods minimise predictive error while enforcing some fairness constraint (Calders & Verwer, 2010; Kamishima et al., 2011; Zafar et al., 2017a; b; c; Bechavod & Ligett, 2017; Agarwal et al., 2018; Narasimhan, 2018) .

The proposed methodology falls into this category. However, rather than placing fairness constraints on the output of the classifier, our method nudge internal representations in the neural network to be less biased. It is akin to performing a sequence of supervised pre-processing to the input data, one in each layer of the neural net.

We will later implement in Section 6 an adversarial alternative to estimating the fairness-accuracy Pareto front. This is also a train-time algorithmic fairness method which employs concepts from adversarial learning. The main classifier is engaged in a game with an adversary adversary that tries to predict the sensitive attribute from the output of the predictor (Ganin et al., 2016; Beutel et al., 2017; Wadsworth et al., 2018; Zhang et al., 2018) .

3. THE FAIRNESS-ACCURACY PARETO FRONT:

This section introduces the fairness-accuracy Pareto front of a general learning algorithm. Suppose the inputs live in some space X, the sensitive attribute (e.g. race or gender) in A, and the response in Y. Let (X, A, Y) be a measurable space and P be a probability measure on it. Let F be a class of functions from X to Y. Given a loss function L : Y × Y → R, we may define the risk, R(f

; P ) = E (x,a,y)∼P L(f (x), y).

Suppose F is chosen to be the family of functions f θ : X → Y parametrised by a fully-connected feedforward neural network with parameter θ ∈ Θ. For a probability measure P on (X, A, Y), define R(θ; P ) = R(f θ ; P ). Let U (θ; P ) be a measure of the unfairness of f θ , in a manner to be made precise in Section 4. Since we wish for the learning algorithm f θ to be both accurate and fair, we wish to minimise, over θ, the vector objective function

EQUATION

)

The optimisation is made difficult by the competing nature of the individual components. Take for instance as one extreme of the spectrum -performing classification completely at random. Then the resulting classifier will certainly be fair with respect to the sensitive attribute, by almost all measures of fairness. The other extreme might be to obtain a perfect classifier. But for datasets collected in a biased way, this will result in the classifier being unfair. For instance, a recidivism dataset would contain a disproportionate number of re-offenses amongst some group if police routinely target that group.

When the individual components of a vector objective compete, as they do in equation 1, it is unlikely that a parameter value exists which simultaneously minimises the individual objectives. This lack of total ordering necessitates optimisation according to a partial order. For a, b ∈ R p , we say a ≤ b if and only if every component of a is less than or equal to the corresponding component of b. Suppose we have p objective functions J 1 , . . . , J p where each is a function from the parameter space Θ to R. Then θ ∈ Θ is Pareto optimal if and only if there does not exist any θ ∈ Θ such that (J 1 ( θ), . . . , J p ( θ)) ≤ (J 1 (θ), . . . , J p (θ)) with at least one strict inequality. The Pareto front is the set of all Pareto optimal points.

A basic technique for approximating the Pareto front is to scalarise the vector objective function.

Let λ ∈ [0, 1]. One possible scalarisation scheme for equation 1 forms the convex combination (1 -λ)R(θ; P ) + λU (θ; P ). An important caveat is that scalarisation in this manner only allows for recovery of points on the convex hull of the Pareto front (Das & Dennis, 1997) . A scalarisation scheme that avoids this issue is the so-called Chebyshev method (Ehrgott, 2000; Giagkiozis & Fleming, 2015) in which we take

EQUATION

The Chebyshev scalarisation enjoys several desirable properties. It guarantees solutions that are at least weakly Pareto optimal for any λ ∈ [0, 1]. The term weakly refers to replacing the nonstrict inequality in the Pareto optimal definition with a strict inequality. A further property of the Chebyshev scalarisation is that any Pareto optimal solution can be obtained for some λ.

Estimation of the Pareto front Now we describe a general technique for estimating the set {θ λ : λ ∈ [0, 1]} where θ λ is given by equation 2. Let (x 1 , a 1 , y 1 ), . . . , (x n , a n , y n ) be independent copies of (x, a, y) drawn from (unknown) distribution P model . Define the empirical measure as Pdata = (3)

1 n n i=1 δ (xi,ai,yi) . Let R(θ; Pdata ) be the plug-in estimator of R(θ; P model ), i.e. R(θ; Pdata ) = E (x,a,y)∼ Pdata L(f (x), y) = 1 n n i=1 L(f (x i ), y i ). Let U (θ; Pdata ) be an estimate of U (θ; P model ), but

Evaluation To assess the quality of our Pareto front approximation, it will be helpful to have unbiased estimators of R( θλ n ; P model ) and U ( θλ n ; P model ). Suppose we have a test set

V = {(x * i , a * i , y * i )} where (x * 1 , a * 1 y * 1 ), . . . , (x * m , a * m , y * m )

are another set of independent copies of (x, a, y) drawn from distribution P model . Define the corresponding empirical measure as

Ptest = 1 m m i=1 δ (x * i ,a * i ,y * i ) .

The risk of θλ n can be assessed using the out-of-sample average loss R( θλ

n ; Ptest ) = E (x,a,y)∼ Ptest L(f (x; θλ n ), y) = 1 m m i=1 L(f (x * i ; θλ n ), y * i ).

The unfairness can be also assessed on the test set, let's denote it U ( θλ n ; Ptest ). The appropriate loss function involved in R(θ; P model ) will be context-specific. Since we will be interested in binary classification, we will limit future discussion to the cross-entropy loss. How fairness should be defined is much more controversial. We will discuss various existing notions of fairness in the next section and advocate for a new causal measure of fairness that is especially adept at handling inherent biases in the dataset.

4. A NEW CAUSAL FAIRNESS MEASURE:

Definitions of fairness can be largely divided into two camps. On one hand, we have non-causal fairness notions which typically operate by conditioning on the levels of the sensitive variable. For instance, enforcing equalised odds (Hardt et al., 2016) on a classifier f θ amounts to enforcing two conditional distributions are the same, e.g. p(f θ (x) | a = a, x = x).

In the non-causal category, two fundamental definitions of fairness are demographic parity and conditional parity. The classifier f θ (x) is said to exhibit demographic parity with the sensitive attribute a if f θ (x) ⊥ ⊥ a, where the shorthand ⊥ ⊥ means independence. Intuitively, demographic parity assesses if the predicted score does not depend on the sensitive variable. For example, a classifier predicting if a convicted criminal will re-offend exhibits demographic parity with respect to race if the distribution of f θ (x) is the same irrespective of race. The drawbacks to demographic parity are well-documented (Hardt et al., 2016; Kleinberg, 2018) . Essentially, when the base rates differ across values of the sensitive attribute, satisfying demographic parity can come at the cost of discrimination.

A more flexible framework of fairness is given by conditional parity, a term coined in Ritov et al. (2017) . Let u be a random vector. The prediction score f θ (x) is said to exhibit conditional parity with sensitive attribute a conditional on u if f θ (x) ⊥ ⊥ a | u. Under the umbrella of conditional parity, Ritov et al. ( 2017) unified various fairness definitions. For instance, the notion of equalized odds, introduced in Hardt et al. (2016) , is recovered by setting u to the true target class membership itself.

Intuitively, conditional parity asks for class predictions that are independent of the sensitive variable a conditioned on u. For example, one could consider a classifier predicting if an applicant should be admitted to graduate school. Here, one may desire admission decisions to be independent of sex (demographic parity), or, for conditional parity, independent of sex conditional on a particular university department. That the notions of demographic and conditional parity can strongly differ and may lead seemingly paradoxical results was strikingly illustrated in Bickel et al. (1975) for graduate admissions at UC Berkeley.

4.1. CAUSAL FAIRNESS IN THE OVERLAP POPULATION:

Taking the causal approach to defining fairness means replacing the question "Is the learning algorithm (conditionally) dependent on the sensitive attribute?" with the question "Does the sensitive attribute have a causal effect on the algorithm's predictions?" In an ideal world, we could intervene on the sensitive attribute by manipulating their values in an experiment and recording the outcomes. However, we usually only have access to observational data. Fortunately, causal inference tools can be used to glean causal effects from observational data. The tools are based on posing hypothetical questions about counterfactuals or potential outcomes: "What would have been the prediction outcome in a parallel universe where the only thing that changed about this subject was the value of the sensitive attribute?" Our approach differs from previous works in causal fairness (Kusner et al., 2017; Kilbertus et al., 2017; Khademi et al., 2019) mainly in the causal estimand we use. Also, notably, we do not make use of structural equation models. A good review on causal approaches to algorithmic fairness can be found in Loftus et al. (2018) .

Our new causal fairness notion is based on a special case of the weighted average treatment effect (WATE) (Hirano et al., 2003) whose development is motivated by the fact that in many situations due to selection bias, the study population may be different from the target population. To make valid causal statements, samples are weighed according to the covariate distributions of some target population.

We will use a special case of the WATE to measure and then penalise the causal link between the sensitive attribute and the intermediate representation in some layer of the neural network. Suppressing the dependence on the layer, let h denote values in the hidden layer. Adopting the potential outcome framework of Imbens & Rubin (2015) where we assume the Stable Unit Treatment Value Assumption, each intermediate representation h takes on one of two potential outcomes, h(0), h(1) depending on whether a = 0 or a = 1. Note that h = ah(1) + (1 -a)h(0), i.e. we can only ever observe one of the two potential outcomes.

Note that there are subtleties entailed by immutable variable such as race or gender. Kilbertus et al. (2017) argue for a workaround based on the idea of potential proxies. For instance the immutable characteristic of race has proxies such as name, visual features, languages spoken at home that can be conceivably manipulated.

Under unconfoundedness, i.e. a is independent of {h(0), h(1)} conditional on x, WATE is a class of causal estimands τ g : R m → R m parametrised by a function g : R p → R given by

EQUATION

where

µ 1 (x) = E(h(1) | x = x) and µ 0 (x) = E(h(0) | x = x).

This form reveals WATE is indeed a measure of the causal effect in the target population specified by g(x). Note when g(x) = 1 for all values of x, WATE reduces to the standard conditional average treatment effect (CATE),

τ CAT E = E (x,a,y)∼P model (h(1) -h(0) | x = x).

Henceforth, we focus our discussion on the case when g(x) = e(x)(1 -e(x)) where e(x) = P (a = 1 | x = x) is also known as the propensity score. The propensity score is typically understood to be the probability of treatment given the covariate x. (Recall in our case, the sensitive variable a plays the role of treatment.) The WATE corresponding to g(x) = e(x)(1 -e(x)) was called the average treatment effect for the overlap population (ATO) in Li et al. (2018a) . The term overlap refers to the fact that the ATO articulates the causal effect among the overlap population which consists of subjects whose covariates could appear with substantial probability in either value of the sensitive attribute.

Let ê(x) be the estimated propensity score function. In our experiments, we used a neural network to estimate the propensity score function. We actually also calibrated the predicted probabilties using the temperature scaling procedure of Guo et al. (2017) .This neural net for learning the propensity score is trained once and for all on the training set.

A consistent estimator of the ATO (Li et al., 2018b) based on data

EQUATION

where w i are the so-called overlap weights (Li et al., 2018a) given by

w i = 1 -ê(x i ) if a i = 1 ê(x i ) if a i = 0.

Overlap weights derive their name from an emphasis on subjects with the most overlap in observed covariates x across the treatments (in our case the treatment is the sensitive attribute). Besides the nice interpretation of the ATO, there is also an important practical reason to adopt it as our causal estimand of choice. The overlap weights smoothly down-weigh subjects in the tails of the propensity score distribution, thereby mitigating the common problem of extreme propensity scores.

For precise conditions under which the estimator τAT O is consistent, we refer the reader the set of regularity assumptions (called Assumption 1 to 5) in Hirano et al. (2003) . These conditions are regulated to the distribution of x and distribution of h(0) and h(1). There is also a condition on the smoothness of the propensity score e(x).

5. METHODOLOGY:

In this section, we present a methodology for approximating the fairness-accuracy Pareto front of a fully-connected feedforward neural net classifier for the cross-entropy loss and the ATO fairness measure. The available data include a single binary sensitive variable a, input variables x ∈ R p , and binary response y indicating class membership. The input x is further standardised to mean 0 and variance 1. All discrete variables are dummy encoded.

Let w (l) ∈ R m l ×m l-1 and b (l) ∈ R m l , l = 1, . . . , L be the parameters in the l-th layer of a fullyconnected feedforward neural network with L layers. Let h (l) : R m l-1 → R m l be the affine transformation h (l) = w (l) v (l-1) + b (l) , l = 1, . . . , L where v (0) = id is the identity function and m 0 = p. The activation function σ (l) : R m l → R m l is applied to obtain v (l) = σ (l) • h (l) , l = 1, . . . , L. The activation function in the final layer, σ (L) , is restricted to the sigmoid function since we wish the classifier to output scores between 0 and 1. We use the ReLU activation function in all other layers for our experiments. Let h (l) i be shorthand for the application of the function h (l) to input feature x i , i.e. h (l) i = h (l) (x i ). Collect all parameters w (l) and b (l) for l = 1, . . . , L into the parameter vector θ. The neural network is simply the function f θ : R p → [0, 1] given by f θ (x) = v (L) (x).

We will employ the binary cross-entropy loss L : [0, 1] × {0, 1} → R given by L(ŷ, y) = y log ŷ + (1 -y) log(1 -ŷ). Our interest is to determine, for the network f θ , the fairness-accuracy Pareto front associated to the (unknown) vector objective function

EQUATION

The first component measures classification error while the second component determines unfairness with respect to the ATO measure. (We would like both components to have low values.) To estimate the Pareto front of equation 6, we will use the strategy laid out in Section 3. Namely, we estimate each component of equation 6, scalarise the vector objective function using the Chebyshev method, and finally optimise the scalarised objective.

Estimation of R(θ; P model ) is straightforward; we simply use the plug-in estimator R(θ;

Pdata ) = -1 n n i=1 [y i log f θ (x i ) + (1 -y i ) log(1 -f θ (x i ))

] . Now we turn our attention to estimating τ AT O ((f θ (x)), the average effect of the sensitive attribute on the prediction for the overlap population. To achieve a low value for τ AT O ((f θ (x)), we could directly constrain the network to learn final predictions with low ATO. However, it may be preferable to penalise the ATO in the hidden layers of the network. This way, downstream analyses that involve transfer learning are also safeguarded against bias. See the exposition in Madras et al. (2018) for further benefits of learning fair internal representations. To keep the notation simple, let's say we penalise the hidden units in the some layer l. We then calculate the ATO in that layer to obtain

U (θ; Pdata ) = τAT O (h (l) ) = n i=1 a i h (l) i (1 -ê(x i )) n i=1 a i (1 -ê(x i )) - n i=1 (1 -a i )h (l) i ê(x i ) n i=1 (1 -a i )ê(x i )

We can generalise U to penalise all layers in the neural network by summing over l. We will report on this experiment in Section 6.

Ideally, we would finely sample λ in [0, 1] to find the set { θλ n : λ ∈ [0, 1]} where θλ n is given by equation 3. But realistically, it might only be possible to perform the optimisation in equation 3 over a coarse grid of λ's. It turns out that evenly distributed λ's in the interval [0, 1] can often produce solutions that form clumps on the Pareto front, i.e. evenly distributed λ's in [0, 1] do not produce evenly distributed points in the multi-objective space. Future work might seek to adaptively select the λ's by implementing methods such as the Normal-Boundary-Interactive (Das & Dennis, 2000) .

We also found it necessary to ensure the two terms in equation 6 are comparable in scale, so we standardised each term as follows. First, we ran the optimisation for λ = 0 and recorded the minimum R min and maximum R max of R(θ; Pdata ). Similarly we then ran the optimisation for λ = 1 to obtain U min and U max . Then we standardised by (R -R min )/(R max -R min ) for the expected loss and analogously for the unfairness measure.

Suppose we have available to us a testing set {(x * i , a * i , y * i )} m i=1 . We note that for evaluation, we assess

R( θλ n , Ptest ) U ( θλ n , Ptest ) on the final prediction f θλ n (x * i )

where

U ( θλ n , Ptest ) = m i=1 a * i f θλ n (x * i )(1 -ê(x * i )) m i=1 a * i (1 -ê(x * i )) - m i=1 (1 -a * i )f θλ n (x * i )ê(x * i ) n i=1 (1 -a * i )ê(x * i )

.

6. EXPERIMENTS:

In this section, we will apply the proposed methodology to two benchmarking datasets in the algorithmic fairness literature -the UCI adult income dataset and the ProPublica recidivism dataset. The two datasets are briefly summarised in Table 1 in Appendix A. Missing values were pre-processed according to the accompanying code. Our goals are as follows. In the UCI dataset, we wish to predict whether an individual has income above 50, 000 USD while remaining fair with respect to gender. Separately, we wish to perform the same prediction task while remaining fair with respect to race. In the recidivism dataset, we wish to predict whether an individual will recommit a crime in two years while remaining fair with respect to race. Then in each of the 100 training sets, for a collection of 15 λ's in the interval [0, 1], we find θλ n according to the Chebyshev scalarisation scheme (left panel) and θλ n according to the adversarial approach (right panel). Thus there are a total of 1500 learned θ's in each plot and the magenta boundary is the Pareto frontier culled from these 1500 candidates. We can see the estimated Pareto front by the proposed methodology spans the fairness-accuracy space more than the adversarial approach.

We will achieve these goals by estimating the fairness-accuracy Pareto front of a binary classifier given by a fully-connected feedforward neural network. Note that we are conducting the analysis for the UCI dataset separately for race and gender. Future work should address fairness with respect to multiple sensitive attributes simultaneously; this would require an extension of the ATO to multiple "treatments" which was suggested as feasible future work in Li et al. (2018a) .

Each dataset is split into a training set and a held-out test set, with the split reported in Table 1 . First, the propensity scores are estimated using a neural net. Details of the propensity score network are given in Appendix A. For the neural net architecture defining f θ , the number of fully-connected layers and number of hidden nodes in each layer (held constant over the layers) were tuned for each dataset with the goal of not incurring over-fitting in the held-out test set. Each fully-connected layer is interspersed with a dropout layer with dropout probability 0.2. The resulting architecture is reported in Table 2 . The ReLU activation function is used in all intermediate layers while the sigmoid function is used in the output layer.

To learn the network, we use the ADAM optimisation algorithm (Kingma & Ba, 2014) . The initial learning is fixed at 0.001. We reduce the learning rate when the training loss has stopped decreasing by using the ReduceLROnPlateau scheduler in PyTorch, setting the factor and patience variables to 0.9 and 10, respectively. All training took place over 500 epochs. Mini-batch size was chosen to be around 5% of the training set size; 150 and 1000 minibatch sizes were used in the recidivism and UCI datasets, respectively.

For a given dataset and a sensitive attribute of interest, we repeatedly split the data into training and testing sets, creating in total 100 sets of each. Then, in each of the 100 training sets, we find θλ n , according to equation 3, for a collection of 15 λ's in the interval [0, 1]. The quality of the approximation is assessed using the corresponding test set. Thus we produce a total of 1500 learned network parameters and each one can be plotted in the fairness-accuracy space. Each column of Figure 1 shows 1500 θλ n 's as well as the Pareto front culled from these 1500 Pareto candidates. The culling simply checks which of the 1500 points are non-dominated points. As we sweep from the top-left corner to the bottom-right corner, we move from neural networks exhibiting high-accuracylow-fairness to those exhibiting low-accuracy-high-fairness.

Next, we investigate the visual effect of dialling λ between 0 and 1. In Figure 2 , we display the distribution of the classifier's prediction in the recidivism dataset broken down by class label and sensitive attribute. Each panel of Figure 2 is a different λ value. In addition to reporting the ATO measure of fairness, we also indicate other non-causal fairness metrics including Equalised Odds, Equal Opportunity, and Demographic Parity. Similar visualisation for the UCI (gender) and UCI (race) dataset can be found in Figures 3 and 4 in the Appendix, respectively. We also repeated this experiment by modifying the fairness measure to penalise intermediate representations in all layers, i.e. set U (θ; Pdata ) = L i=1 τAT O (h (l) ) , for which the results are reported in Figure 6 in Appendix A. It seems that penalising all layers makes training the neural network more difficult as we encounter many iterations where we end up in a local optimum. For simplicity, we had used the same network architecture as in the case of penalising one internal layer which may have prompted the performance issue.

Comparison to alternatives We did not find other works in the algorithmic fairness literature that address the specific task of finding the fairness-accuracy Pareto front of a neural network. Given this, we instead looked for methods where there was some type of tuning parameter that controls the trade-off between fairness and accuracy. By dialling this tuning parameter, one could hope to sweep out a set of classifiers that live in different parts of the fairness-accuracy landscape.

Given the diversity of fairness methods, due in part to the fairness definition used, we decided to implement the adversarial training technique proposed in Louppe et al. (2017) which is not based on a specific fairness criterion. The idea is intuitive; the classifier and adversarial are engaged in a zero-sum game. The classifier network, call its parameters θ clf , attempts to make the best binary classification. The adversary, on the other hand, attempts to make the best prediction of the binary sensitive attribute based on the classifier's prediction. Let θ adv denote the parameters of the adversarial network. The overall objective is θλ

n = arg min θ clf [Loss y (θ clf ) -λLoss a (θ clf , θ adv )] .

where the first loss measures the prediction of y and the second loss measures the prediction of the sensitive attribute a. Both losses were chosen to be the binary cross-entropy loss.

Our implementation of Louppe et al. (2017) is based on GoDataDriven's post on fairness in machine learning with adversarial networks. Following their choice of epochs, we alternate the following steps over 200 epochs: (1) train the adversarial network for a single epoch, holding the classifier network fixed and (2) train the classifier network on a single sampled mini batch, holding the adversarial network fixed. Details on the adversarial network architecture are provided in Appendix A. For the classifier network, we employed the same network as above in our proposed methodology and kept all training choices, such as the optimisation algorithm and mini-batch size, the same. The classifier was pretrained for 2 epochs.

We modified GoDataDriven's code so that we can sweep θλ n over the parameter λ. The result of the adversarial approach is shown in the right column of Figure 1 . Once again, using the same 100 training and testing pairs as above, we find solutions to θλ n for a set of 15 λ's in [0, 1], for a total of 1500 Pareto candidates. We can immediately see that compared to the proposed methodology, the adversarial is less capable of finding a Pareto front that spans the fairness-accuracy space. Indeed the set of dominated (non-dominated) points in the adversarial approach is larger (smaller) relative to the proposed approach, see Table 3 .

A APPENDIX:

In this section, we provide additional details on the experiments conducted in Section 6. First, in Table 1 , we summarise the datasets on which all experiments were conducted. The feedforward architecture used in the proposed methodology is given in Table 2 for each of the three data settings.

Next, we describe the neural net we employed to estimate the propensity scores P (a = 1 | X). We actually used the same neural net for all three data settings in Table 1 . Each of the 3 fully-connected layer, with 32 hidden units each, is interspersed with a dropout layer with dropout probability 0.2. The ReLU activation function is used in all intermediate layers while the sigmoid function is used in the output layer. The cross-entropy loss is used between the estimated scores and the true labels dictated by a. To learn the network, we use the ADAM optimisation algorithm (Kingma & Ba, 2014) . The initial learning is fixed throughout at 0.001. Training took place over 100 epochs. Mini-batch size was chosen to be around 5% of the training set size; 150 and 1000 minibatch sizes were used in the recidivism and UCI datasets, respectively. After the propensity neural network is trained, we apply the calibration technique proposed in Guo et al. (2017) to calibrate the probability predictions. We used their GitHub code with no modification.

Figure 5 shows the additional output from the experiment that produced Figure 1 on the UCI dataset with race as the sensitive attribute. We also repeated the experiment in Section 6 by penalising the ATO in all layers. The results are shown in Figure 6 . The adversarial approach we compared the proposed methodology against used a network with 4 hidden layers with 32 hidden units in each. ReLU activations were used throughout except in the final layer where the sigmoid function is used. The adversarial network was pretrained for 5 epochs. Optimisation used ADAM and minibatch sizes described in Table 1 . 

:



Figures and Tables:

not necessarily a plug-in estimator. Then we consider the empirical version of equation 2 given by θλ n = arg min θ max{(1 -λ)R(θ; Pdata ), λU (θ; Pdata )}.

Figure 2: Distributions of the predicted probabilities in the test set of the recidivism dataset are plotted for λ = 0 (left) and λ = 0.712 (right). The distributions are broken down by different values of the true target label y and value of the sensitive attribute z. Besides the causal fairness notion we introduced in this work, we also indicate equalised odds (mv EO), equality of opportunity (mv EOpp) and demographic parity (mv DP).

Figure 3: Prediction probabilities in the test set of UCI (race) for varying values of λ, indicated by p penalty in the heading of each plot.

Figure1: Each column of plots corresponds to a dataset and a sensitive attribute of interest. In all panels, we repeatedly split the data into training and testing sets, creating in total 100 sets of each.

Table content: <table><tr><td/><td/><td>UCI (gender)</td><td/><td/><td/><td/><td>Recidivism (race)</td></tr><tr><td>U ( θλ n , Ptest)</td><td>0.0 0.1 0.2</td><td/><td/><td>Pareto frontier Chebyshev θλ n</td><td>U ( θλ n , Ptest)</td><td>0.00 0.05</td><td/><td>Pareto frontier Chebyshev θλ n</td></tr><tr><td>( θλ n , Ptest)</td><td>0.1 0.2</td><td>R( θλ n , Ptest)</td><td/><td>Pareto frontier adversarial θλ n</td><td>( θλ n , Ptest)</td><td>0.05</td><td>R( θλ n , Ptest)</td><td>Pareto frontier adversarial θλ</td></tr><tr><td>U</td><td>0.0</td><td/><td/><td/><td>U</td><td>0.00</td><td/></tr><tr><td/><td>0.4</td><td>0.5 R( θλ n , Ptest)</td><td>0.6</td><td>0.7</td><td/><td/><td colspan="2">0.600 0.625 0.650 0.675 0.700 0.725 0.750 R( θλ n , Ptest)</td></tr></table>

Dataset descriptions

Table content: <table><tr><td/><td/><td>dataset features</td><td/><td/><td/></tr><tr><td colspan="2">Dataset dim(x)</td><td>binary outcome y</td><td colspan="4">sensitive a training size testing size minibatch size</td></tr><tr><td>Recidivism</td><td colspan="2">12 Reoffend in 2 years?</td><td>binary race</td><td>3086</td><td>3086</td><td>150</td></tr><tr><td>UCI</td><td colspan="2">93 Income above 50K?</td><td>binary race</td><td>15470</td><td>15470</td><td>1000</td></tr><tr><td>UCI</td><td colspan="3">93 Income above 50K? binary gender</td><td>15470</td><td>15470</td><td>1000</td></tr></table>

Peer review (iclr2020):
This paper proposes a method to approximate the *fairness-accuracy landscape* of classifiers based on neural networks. The key idea is to set up a multi-dimensional objective, where one dimension is about prediction accuracy and another about fairness. The fairness component relies on a definition of fairness based on causal inference, relying on the idea that a sensitive attribute should not causally affect model predictions. I found the causal idea intriguing, since it makes sense that we don*t want a sensitive attribute to have a causal effect. However, there may be several problems with this approach: 1) For a causal estimate to be valid we need several assumptions. For example, we need A (the sensitive attribute) to be independent of potential outcomes conditional on X --- the so-called *unconfoundedness assumption* in causal inference. We also need *positivity*, i.e., that 0< P(A=1|X) <1. These assumptions are not discussed in the paper. Furthermore, the particular context of the paper, where the treatment is actually an immutable characteristic, makes such discussion much more subtle. What will we do, for instance, if there are no A=1 in the sample when X = ...? 2) The authors seem to assume that the propensity score model is well specified. This can be tested, e.g., using [1]. What do we do when this fails? 3) Why do we want U to be small, i.e., why do we want the causal effect of A to be small, is never justified. In particular, its relation to *fairness* is never fleshed out, but just assumed to be so. This can be problematic when, say, we are missing certain important X that are important for A. Then, there will be a measurable causal effect of A on h(). Some other problems: - What is the reason for focusing on *neural classifiers*? There is nothing specific in the method or analysis that relates to neural networks, except for the use of the causal estimand in a *hidden layer*. - In the Introduction, the authors could cite the works of Amartya Sea, etc., on fairness. Certainly the study of fairness problems did not start in 2016. - What exactly is a *sensitive attribute*? If we don*t want to bias our predictions, then why include it in the analysis? - It is unclear what is new and what is related work in page 3. - Sec. 4: The claim that *causal inference is about situations where manipulation is impossible* discards voluminous work in causal inference through randomized experiments. In fact, many scientists would agree that causal inference is impossible without manipulation. - As mentioned above, why this particular estimand leads to more *fairness* is never explained. - Do we need a square or abs value in Eq (5)? - The experimental section is weak and does not illustrate the claims well. It would be important to explain the choice of the particular causal estimand, the choice of the hidden layer to put the estimand in, to explore the choice of the objective, and so on. Currently, none of these choices/design aspects are being investigated. [1] *A specification test for the propensity score using its distribution conditional on participation* (Shaikh et al, 2009)