The fairness-accuracy landscape of neural classifiers
Authors: Susan Wei
Marc Niethammer
Venue:  (2020)

Abstract: That machine learning algorithms can demonstrate bias is well-documented by now. This work confronts the challenge of bias mitigation in feedforward fully-connected neural nets from the lens of causal inference and multiobjective optimisation. Regarding the former, a new causal notion of fairness is introduced that is particularly suited to giving a nuanced treatment of datasets collected under unfair practices. In particular, special attention is paid to subjects whose covariates could appear with substantial probability in either value of the sensitive attribute.  Next, recognising that fairness and accuracy are competing objectives, the proposed methodology uses techniques from multiobjective optimisation to ascertain the fairness-accuracy landscape of a neural net classifier. Experimental results suggest that the proposed method produces neural net classifiers that distribute evenly across the Pareto front of the fairness-accuracy space and is more efficient at finding non-dominated points than an adversarial approach.

Under review as a conference paper at ICLR 2020
THE FAIRNESS-ACCURACY LANDSCAPE OF NEURAL CLASSIFIERS
Anonymous authors
Paper under double-blind review
ABSTRACT
That machine learning algorithms can demonstrate bias is well-documented by now. This work confronts the challenge of bias mitigation in fully-connected feedforward neural nets through the lens of multiobjective optimisation. Specif- ically. recognising that fairesss and accuracy are competing objectives, the pro- posed methodology uses techniques from multiobjective optimisation to ascertain the fairness-accuracy landscape of neural net classifier. Along the way, a new causal notion of fairness is introduced that is particularly suited to giving nu- anced treatment when data collected under unfair practices. In particular, special attention is paid to subjects whose covariates could appear with substantial prob- ability in either value of the sensitive attribute. Experimental results suggest that the proposed method produces neural net classifiers that distribute evenly across the Pareto front of the fairness-accuracy space and is more efficient at finding non-dominated points than an adversarial approach.
INTRODUCTION
There is increasing concern over the ethics of machine learning algorithms. The issue of machine bias was prominently featured in ProPublica's s 2016 eponymous article (Angwin al,, 2016) where the investigation uncovered prejudice against African-Americans in COMPAS (Correctional Of- fender Management Profiling for Alternative Sanctions), a recidivism prediction tool developed by Northpointe. Efforts to mitigate machine bias has received steadily increasing attention from stake- holders in a wide array of arenas including academia, industry research labs, and advocacy groups.
The question of fairness has long history (Sen & McMurrin 1980) and its definition continues to be debated (Dwork et al. 2012) Chouldechova [016)] Joseph et al. (016)). For instance, instance,Corrett Corbett- Davies be regarded as fair with respect to
et al. (017)) Dieterich al. (2016) argue that COMPAS
certain fairness notions. Works such as Hardt et al. (2016) and Kleinberg (2018) have investigated this type of ambiguity by showing that certain fairness criteria cannot be simultaneously satisfied.
Having fixed a definition of fairness, most works in algorithmic fairness proceed by imposing con- straints during the training process in an attempt to enforce fairness (Hardt al.) 2016) Joseph et al. 2016 Zafar et al. 2017aa b). Recent works can handle very complex learning al algorithms such as neural networks (Beutel et 2017: Wadsworth al., 2018 MMdras et al. 2018 Manisha & Gujar 2018). However none of the works in the literature seek to provide a holistic view of the fairness-an accuracy landscape of the algorithm.
What we mean is this though it is desirable that the neural network maintain high predictive accu- racy while simultaneously remaining fair with regards to a sensitive variable such as race or gender, these two objectives often compete. Given this, it is helpful to cast the balancing act between fair- ness and accuracy as a multiobjective optimisation task and look at the fairness-accuracy Pareto front. This can give a bird's eye view of the algorithm and can be useful for comparing two algo- rithms based on, say, the "volume" of the Pareto front (Li et al. 2015). In this work, we propose methodlogy for estimating the fairness-accurac Pareto front of fully-connected feedforward network.
Contributions This is the first work in algorithmic faimess that specifically addresses estimation of the fairness-an fairress-accuracy accuracy Pareto front of a fully-connected feedforward network. The framework presented is flexible enough to allow for user-supplied accuracy and fairness measures. Along the

---

Under review as a conference paper at ICLR 2020
way, introduce causal measure of fairesss which emphasises fairness amongst subjects with the most overlap in observed covariates across the different values of a sensitive attribute. Finally, the methodology be used enforce faimess in all intermediate representations of the neural network. This has potential benefits for downstream transfer learning tasks.
2 RELATED WORK
In this section, review the broad categories of existing methods in algorithmic fairness. The major discernible classes of fair learning methods can be trifurcated according to the stage in which action is taken. The first class of methods attempts remove bias from the input data itself. These methods rest on the premise that once proper preprocessing is accomplished, any classifier can be used to subsequently produce fair predictions (Kamiran & Calders 2012 Feldman al.) 2015
Calmon et al. 2017 Johndrow & Lum 2019).
On the other hand, post-processing techniques directly operate on the classifier output and are amenable to any classifier. The technique in Hardt et al. (2016) for instance seeks to learn mono- tone transformation of the classifier's output to remove unfairness with regard either demographic parity or equalised odds.
The third type of algorithmic fairness method directly intervenes in the stage of training. Many of these methods are specific to certain classifiers and certain notions of fairness. Generally speaking, muin-time methods minimise predictive error while enforcing some fairness constraint (Calders &
Verwer 2010; Kamishima et al. 2001 Zafar et al. 2017a blc Bechavod & Ligett 2017 Agarwal
et al. 2018) Narasimhan 2018).
The proposed methodology falls into this category. However, rather than placing faimess constraints on the output of the classifier, our method nudge internal representations in the neural network to be less biased. It is akin to performing sequence of supervised pre-processing to the input data, one in each layer of the neural net.
We will later implement in Section an adversarial alternative estimating the faimess-accuracc Pareto front. This is also train-time algorithmic fairness method which employs concepts from adversarial learning. The main classifier is engaged in game with adversary adversary that tries to predict the sensitive attribute from the output of the predictor (Ganin et al. 2016 Beutel et al.,
2017 Wadsworth et al. 2018 Zhang et
2018
3 THE SAIRNESS-ACCURAEE PARETO FRONT
This section introduces the fairness-accurac Pareto front of a general learning algorithm. Suppose the inputs live in some space X, the sensitive attribute (e.g. race or gender) A, and the response in Y. Let (X, A, Y) be a measurable space and P be probability measure on it. Let F be class of functions from X to Y. Given a loss function L Y x Y - R, we may define the risk, R(;; P) P[
Suppose is chosen to be the family of funetions So X + Y parametrised by fully-connected feedforward neural network with parameter 0 E O. For a probability measure P on (X, A, W), define R(O; P) = Rife; P). Let (0; P) be a measure of the unfairness of fe. in manner to be made precise in Section Since wish for the learning algorithm fo to be both accurate and fair, wish to minimise, over 0, the vector objective function
R(;; P) U (0; P)
(I)
The optimisation is made difficult by the competing nature of the individual components. Take for instance as one extreme of the spectrum performing classification completely at random. Then the resulting classifier will certainly be fair with respect to the sensitive attribute, by almost all measures of fairess.. The other extreme might be to obtain a perfect classifier. But for datasets collected in biased way. this will result in the classifier being unfair. For instance, recidivism dataset would contain disproportionate number of re-offenses amongst some group if police routinely target that group.
2

---

Under review as a conference paper at ICLR 2020
When the individual components of vector objective compete, they do in equation it is un- likely that a parameter value exists which simultancously minimises the individual objectives. This lack of total ordering necessitates optimisation according partial order. For b E R', we say a S if and only if every component of a is less than or equal to the corresponding component of b. Suppose we have p objective functions J1,-.-,d Jp where each is function from the parameter space to R. Then E θ is Pureto optimal if and only if there does not exist any 0 such that ...........,
(J1 (0).... Tp)) S (J,( with at least one strict inequality. The Pareto front the set of all Pareto optimal points.
A basic technique for approximating the Pareto front is to scalarise the vector objective function. Let À E [0, II. One possible scalarisation scheme for equation forms the convex combination (1 - R(6; P) + (U; P). An important caveat is that scalarisation in this manner only allows for recovery of points
the convex hull of the Pareto front (Das & Dennis 1997). A scalarisa-
tion scheme that avoids this issue is the so-called Chebyshev method (Khrgott 2000 Giagkiozis &
Fleming 2015) in which we take
0² = arg min max - A) P), AU,
The Chebyshev scalarisation enjoys several desirable properties. It guarantees solutions that are least weakly Pareto optimal for any À [0, I]. The term weakly refers to replacing the non- strict inequality in the Pareto optimal definition with a stricl inequality. A further property of the Chebyshev scalarisation that any Pareto optimal solution can be obtained for some A.
Estimation of the Pareto front Now we describe a general technique for estimating the set {0> À E [0, where 0 is given by equation 2 Let (x1: (x,,81, Y1)....... (Xn, & Yn) be inde- pendent copies of (x, y) drawn from (unknown) distribution P Pmodel. Define the empirical mea- Σ (xx)) Let R(0; Pasta) be the plug-in estimator of R(0; P Pmodel). i.c. ((f(x).y) = 1 Em= [(f(x)). Let U(e; Paata) be an estimate of U(; P Pmodel), but not necessarily plug-in estimator. Then we consider the empirical version of equation given by
sure as Paata
R(0; Palta) =
(x..,)))Paa (x.
d arg min max (1 A)R(0; Pasta), (U; Paata)).
Evaluation To assess the quality of our Pareto front approximation, it will be helpful to have unbiased estimators of R(O,; Pmodel) and U(2:; Pmodel.. Suppose have test set V = {(x,, yf)) where afyi......... (xin am,.", Ym)) are another set of independent copies of a. drawn from distribution Pmodel- Define the corresponding empirical measure as Prest m Ei= ((x7,0737)) The risk of 07 can be assessed using the out-of-sample average loss R(OA; Prest) = (x..))) (X((x; @A).y) y) 1 E (xf(x); (x); y?). The unfairness can be also assessed the test set, let's denote it U()); Pust).
The appropriate loss function involved in R(0 Pmodel) will be context-specific. Since we will be interested in binary classification, will limit future discussion the cross-entropy loss. How fairness should be defined is much more controversial. We will discuss various existing notions of fairness in the next section and advocate for new causal measure of fairness that is especially adept at handling inherent biases in the dataset.
4 A NEW CAUSAL FAIRNESS MEASURE
Definitions of fairness can be largely divided into two camps. On one hand, we have non-causal faimess notions which typically operate by conditioning the levels of the sensitive variable. For instance, enforcing equalised odds (Hardt et al. 2016) on a classifier fo amounts to enforcing two conditional distributions the same, e.g. p(fo
0, ===)) I).
In the non-causal category, two fundamental definitions of fairness are demographic parity and con- ditional parity. The classifier fo(x) is said to exhibit demographic parity with the sensitive attribute a fx( IL a, where the shorthand 1 means independence. Intuitively, demographic parity as- sesses if the predicted score does not depend on the sensitive variable. For example, classifier
3

---

Under review as a conference paper at ICLR 2020
predicting if convicted criminal will re-offend exhibits demographic parity with respect to race the distribution of (x) is the same irrespective of race. The drawbacks to demographic parity
well-documented (Hardt et all, 2016, Kleinberg 2018). Essentially, when the base rates dif- fer across values of the sensitive attribute, satisfying demographic parity can come at the cost of discrimination.
A more flexible framework of fairness is given by conditional parity, a term coined in Ritov et al. (2017). Let be random vector. The prediction score fo fx( said to exhibit conditional parity with sensitive attribute a conditional on u if (x( 1 u. Under the umbrella of conditional parity, Ritov al. (2017) unified various fairness definitions. For instance, the notion of equalized odds, introduced in Hardt et al. (2016), is recovered by setting u to the true target class membership itself.
Intuitively, conditional parity asks for class predictions that are independent of the sensitive variable conditioned on u. For example, one could consider a classifier predicting if an applicant should be admitted to graduate school. Here, one may desire admission decisions to be independent of sex (demographic parity), or, for conditional parity, independent sex conditional on particular university department. That the notions of demographic and conditional parity can strongly differ and may lead scemingly paradoxical results was strikingly illustrated in Bickel et al. (1975) for graduate admissions at UC Berkeley.
4.1 CAUSAL FAIRNESS IN THE OVERLAP POPULATION
Taking the causal approach to defining fairness means replacing the question "Is the learning algo- rithm (conditionally) dependent on the sensitive attribute?" with the question "Does the sensitive attribute have causal effect on the algorithm's 's predictions?" In an ideal world, we could intervene on the sensitive attribute by manipulating their values in an experiment and recording the outcomes. However, we usually only have access to observational data. Fortunately, causal inference tools can be used to glean causal effects from observational data. The tools are based on posing hypothetical questions about counterfactuals potential outcomes: "What would have been the prediction out- come in a parallel universe where the only thing that changed about this subject was the value of the sensitive attribute?"
Our approach differs from previous works in causal fairness (Kusner et al., 2017 Kilbertus et al,, 2017 Khademi al. 2019) mainly in the causal estimand use. Also, notably, do not make use of structural equation models. A good review on causal approaches to algorithmic fairness can be found in Loftus et al. (018).
Our new causal fairness notion is based on a special case of the weighted average treatment effect (WATE) (Hirano al., 2003) whose development is motivated by the fact that in many situations due to selection bias, the study population may be different from the target population. To make valid causal statements, samples are weighed according to the covariate distributions of some target population.
We will use special case of the WATE to measure and then penalise the causal link between the sensitive attribute and the intermediate representation in some layer of the neural network. Sup- pressing the dependence on the layer, let h denote values in the hidden layer. Adopting the potential outcome framework of Imbens & Rubin (201)) where we assume the Stable Unit Treatment Value Assumption, each intermediate representation h takes on one of two potential outcomes, h()), h(1) depending whether a 0 observe one of the two potential outcomes.
1. Note that h ah(1) + (1 aO), i.c. can only ever
Note that there are subtleties entailed by immutable variable such as race gender. Kilbertus et al. (2017) argue for workaround based on the idea of potential proxies. For instance the immutable characteristic of race has proxies such as name, visual features, languages spoken at home that can be conceivably manipulated.
Under unconfonndedness, i.c. is independent of (h(0), h(1) conditional on x, WATE is class of causal estimands Ty m - IR parametrised by a function RP -> IR given by
Tg (h) = (x(x,,)) Pamodaa P (x())()) (x)
(x)))
(4)
(xx,, ) [x(x)]
4

---

Under review as a conference paper at ICLR 2020
where /1 (x) = E(h(1) x= z) and (o(x:) E(h(O) x) This form reveals WATE is indeed
a measure of the causal effect in the target population specified by g(z). Note when g(x) =
for all values of 2, WATE reduces to the standard conditional average treatment effect (CATE),
TCATE = -xx.a.)))/Pmmoal (h(1) h(0) x x).
Henceforth, we focus our discussion on the case when g(z) e(x)(1-e(a)) where e(x) = P(a -
1|x x . is also known the propensity score. The propensity score is typically understood to be the probability of treatment given the covariate x. (Recall in our case, the sensitive variable a plays the role of treatment.) The WATE corresponding to g(x) c(x) e(x)) called the average treatment effect for the overlap population (ATO) in Li et al. (2018).. The term overlap refers to the fact that the ATO articulates the causal effect among the overlap population which consists of subjects whose covariates could appear with substantial probability in either value of the sensitive attribute.
Let é(z) be the estimated propensity score function. In our experiments, we used a neural network to estimate the propensity score function. We actually also calibrated the predicted probabilties using the temperature scaling procedure of Guo et al. (2017). This neural net for learning the propensity score is trained once and for all on the training set.
A consistent estimator of the ATO Li al.
2018b) based on data {(x;, a, h;) is
fATO (h) = Ei= a,h,w;
(1 a h,
(5)
E a,w;
a. )
where w, are the so-called overlap weights (Li et al.
2018a) given by
1 ε(x;)
if a; = 1
00;
(xxx))
if a; = O.
Overlap weights derive their name from an emphasis on subjects with the most overlap in observed covariates across the treatments (in our case the treatment is the sensitive attribute). Besides the nice interpretation the ATO, there is also an important practical reason to adopt it as our causal estimand of choice. The overlap weights smoothly down-weigh subjects in the tails of the propensity score distribution, thereby mitigating the common problem of extreme propensity scores.
For precise conditions under which the estimator FATO is consistent, of regularity assumptions (called Assumption I to 5) in Hirano et al. (2003). These conditions are regulated the distribution of x and distribution of h(0) and h1)) There is also a condition on the smoothness of the propensity score e(r).
refer the reader the set
5 METHODOLOGY
In this section, we present a methodology for approximating the fairness-accuracy Pareto front of fully-connected feedforward neural net classifier for the cross-entropy loss and the ATO fairness measure. The available data include a single binary sensitive variable a, input variables x E R", and binary response y indicating class membership. The input x is further standardised to mean 0 and variance 1. All discrete variables are dummy encoded.
Let w E m and ((0) E R'I," 1,..., connected feedforward neural network with L layers. Let h (1) Rm - R be the affine trans- formation
L be the parameters in the I-th layer of fully-
10 = (4) (- + 1..0,.......
where (0) id the identity function and mg p. The activation function d() : + R is applied to obtain
( = の h h1
L.
The activation function in the final layer, o (L.), is restricted to the sigmoid function since we wish the classifier to output scores between and 1. We for our experiments. Let (n))) be shorthand for the application of the function ht (1) to input feature x,, iee. h, (x;). Collect all parameters (!)) and ((9) for I = 1...., L into the parameter vector 0. The neural network is simply the function fo : RP [0, 1] given by fo(x) o(2)
the ReLU activation function in all other layers
5

---

Under review as a conference paper at ICLR 2020
We will employ the binary cross-entropy loss L: [0. 1] x {0,1} - R given by L(g. y) y log +
(1 y) log(1 - 9). Our interest is to determine, for the network fo. the fairesss-accuree Pareto front associated to the (unknown) vector objective function
R(O; Pmoddl) (U() P Pmoddl)
= ( ("ix log fx(x) + (1 log(1 -
(6)
(for)(f((x)]
The first component measures classification error while the second component determines unfairness with respect to the ATO measure. (We would like both components have low values.) To estimate the Pareto front of equation 6 we will use the strategy laid out in Section 3 Namely, we estimate each component of equation 66 scalarise the vector objective function using the Chebyshev method, and finally optimise the scalarised objective.
Estimation of R(O; Pmodee) is straightforward; we simply use the plug-in estimator R(O; P Panta) log (x,) + (1 - yi) log(1 - fo [xx)]]] Now we turn our attention to estimating (ff (x)), the average effect of the sensitive attribute on the prediction for the overlap pop- ulation. To achieve a low value for (ATO(((o(x)) we could directly constrain the network to learn final predictions with low ATO. However, may be preferable to penalise the ATO in the hidden layers of the network. This way. downstream analyses that involve transfer learning are also safe- guarded against bias. See the exposition in Madras al. (2018) for further benefits of learning fair internal representations, To keep the notation simple, let's 's say we penalise the hidden units in the some layer I. We then calculate the ATO in that layer to obtain
1 n E yy
ai= a, (0
(1-(x)) E(x,))
Ei= (1 a, E(x;)
U(0 Pasta) |
E - -1xx((x))
Eh (1 a (xx,)
We can generalise U to penalise all layers in the neural network by summing over L. We will report on this experiment in Section 6
Ideally, we would finely sample À in [0, 1] to find the set (0=- > E [0, 1|} where 0# is given by equation 3 But realistically, it might only be possible to perform the optimisation in equation a coarse grid of X's. It turns out that evenly distributed A's in the interval [0, 1] can often produce solutions that form clumps on the Pareto front, i.c. evenly distributed X's in [0, 1] do not produce evenly distributed points in the multi-objective space. Future work might seek to adaptively select the A's by implementing methods such as the Normal- -oundary
(Das & Dennis, 2000).
We also found it necessary to ensure the two terms in equation 6 are comparable in scale, so we standardised each term as follows. First, we ran the optimisation for À = 0 and recorded the minimum Rmin and maximum Ryaz of R(0; Paata). Similarly we then ran the optimisation for x 1 to obtain Umi and Umar. Then we standardised by (R / expected loss and analogously for the unfairness measure.
Rmin) for the
Suppose we have available to us a testing set ax We note that for evaluation, we
R Pusat) on the final prediction faz (x²) where
assess
U (O,),, Prost.)
Σm m
aj (x² ) {(x;))
ETh (1 1 az) (x))
U(A), no
Prest)
D1T a (1-é(x;))
>1 ag )x(x))
EXPERIMENTS
In this section, we will apply the proposed methodology to two benchmarking datasets in the algo- rithmic fairness literature- the UCI adult income dataset and the ProPublica recidivism dataset. The two datasets are briefly summarised in Table in Appendix [A] Missing values were pre-processed according to the accompanying code. Our goals are as follows. In the UCI dataset, we wish to predict whether an individual has income above 50, 000 USD while remaining fair with respect to wish to perform the same prediction task while remaining fair with respect to race. In the recidivism dataset, we wish to predict whether an individual will recommit a crime in two years while remaining fair with respect to race.
gender. Separately,

---

Under review as a conference paper at ICLR 2020
UCI (gender)
Recidivism (tace)
0.2
U Pown
Pareto frontier Chebyshev 6
0.1
Chebyshev d
0.05
0/00
0.0
0,00
R(62, Powi)
2 Puse)
0.2
Pareto frontier
U0R Prw))
adversurial 8
0.05
adversuuiil 6
0.1
0/68
0.0
0.625
0.700
0.750
n1 P
HL92 Puoa)
Figure 1: Each column of plots corresponds to a dataset and a sensitive attribute of interest. In all panels, repeatedly split the data into training and testing sets, creating in total 100 sets of each. Then in each of the 100 training sets, for collection of 15 X's in the interval [o, 1]. we find 0 according the Chebyshev scalarisation scheme (left panel) and 0 according to the adversarial approach (right panel). Thus there are a total of 1500 learned 0's in each plot and the magenta boundary the Pareto frontier culled from these 1500 candidates. We can see the estimated Pareto front by the proposed methodology spans the fairness-accuracy space more than the adversarial approach.
We will achieve these goals by estimating the fairness-accuracy Pareto front of a binary classifier given by fully-connected feedforward neural network. Note that the UCI dataset separately for race and gender. Future work should address fairness with respect to multiple sensitive attributes simultaneously; this would require extension of the ATO multiple "treatments" which was suggested as feasible future work in Li et al. (2018)).
conducting the analysis for
Each dataset is split into training set and a held-out test set, with the split reported in Table I First, the propensity scores are estimated using a neural net. Details of the propensity score network given in Appendix A For the neural net architecture defining fe, the number of fully-connected layers and number of hidden nodes in each layer (held constant over the layers) were tuned for each dataset with the goal of not incurring over-fitting in the held-out test set. Each fully-connected layer is interspersed with a dropout layer with dropout probability 0.2. The resulting architecture reported in Table 2 The ReLU activation function is used all intermediate layers while the sigmoid function is used in the output layer.
To learn the network, we use the ADAM optimisation algorithm (Kingma & Baj 2014). The initial learning is fixed at 0.001. We reduce the learning rate when the training loss has stopped decreasing by using the ReduceLROnn lat scheduler in PyTorch, setting the factor and patience variables to 0.9 and 10, respectively. All training took place over 500 epochs. Mini-batch size was chosen to be around 5% of the training set size; 150 and 1000 minibatch sizes were used in the recidivism and UCI datasets, respectively.
For a given dataset and a sensitive attribute of interest, we repeatedly split the data into training and testing sets, creating in total 100 sets of each. Then, in each of the 100 training sets, find 0A, according to equation approximation is assessed using the corresponding test set. Thus we produce a total of 1500 learned network parameters and each one can be plotted in the fairness-accuracy space. Each column of Figure shows 1500 f% 's as well the Pareto front culled from these 1500 Pareto candidates. The culling simply checks which of the 1500 points are non-dominated points. As we sweep from the top-left corner to the bottom-right corner, we move from neural networks exhibiting high-aceuracy- low-fairness to those exhibiting low-accuracy hhihrrua--ihghh-aainaea
for collection of 155 X's in the interval 1]. The quality of the
Next, we investigate the visual effect of dialling À between and In Figure 2 we display the distribution of the classifier's prediction in the recidivism dataset broken down class label and sensitive attribute. Each panel of Figure 2 is a different À value. In addition to reporting the ATO measure of faimess, Equal Opportunity, and Demographic Parity. Similar visualisation for the UCI (gender) and UCI (race) dataset can be found in Figures [ad in the Appendix, respectively.
also indicate other non-causal faimesss metrics including Equalised Odds,
7

---

Under review as a conference paper at ICLR 2020
Epoch 459: teaking 4xiprethst:: 00000
tpoch 489: testing dispproknst: -0.
cacsalmo
0244 ECm4 E.pp-Opppp.. 4,900, (P-13.700:
E0pp-66000 GP 15,291,
y=0003; protected 7-0
---- y=0.03; protected z-0
y=0 protected
y=0,00: protected
1=10 protected 2=0
y=1.0% protected 2=0
15.5
y-1 00: protected 2=1
y- 1.00: protocted 2=1
12.5
3
7.5
0
0.3
Figure 2: Distributions of the predicted probabilities in the test set of the recidivism dataset are broken down by different
plotted for À O (left) and x 0.712 (right). The distributions values of the true target label y and value of the sensitive attribute z. Besides the causal fairness notion we introduced in this work, we also indicate equalised odds (mv (mv_EO), equality of opportunity (mv_EOpp) and demographic parity (Mv_DP).
We also repeated this experiment by modifying the fairness measure to penalise intermediate repre- sentations in all layers, i.e, set U@: Pasta) Σl= |Aro (h()). for which the results are reported Figurelolin Appendix [A] It seems that penalising all layers makes training the neural network more difficult as we encounter many iterations where we end up in a local optimum. For simplicity, we had used the same network architecture as in the case of penalising one internal layer which may have prompted the performance issuc.
Comparison alternatives We did not find other works in the algorithmic fairness literature that address the specific task of finding the fairness-accuracy Pareto front of neural network. Given this, we instead looked for methods where there was some type of tuning parameter that controls the trade-off between fairness and accuracy. By dialling this tuning parameter, one could hope sweep out a set of classifiers that live in different parts of the faimess--cccracy landscape.
Given the diversity of fairness methods, due in part the faimnsss definition used, we decided to implement the adversarial training technique proposed in Louppe et al. (2017) which is not based on specific fairesss criterion. The idea is intuitive; the classifier and adversarial are engaged in zero-sum game. The classifier network, call its parameters Oay, attempts to make the best binary classification. The adversary, on the other hand, attempts to make the best prediction of the binary sensitive attribute based on the classifier's prediction. Let Oadv denote the parameters of the ad- versarial network. The overall objective is 0** = arg minlga, [Lossy (0assy(0ar) - ALoss (Octs Oude) )] where the first loss measures the prediction of y and the second loss measures the prediction of the sensitive attribute a. Both losses were chosen to be the binary cross-entropy loss.
Our implementation of Louppe et al. (201)) is based on GoDataDriven's post on fairness in machine learning with adversarial networks. Following their choice of epochs, we alternate the following steps over 200 epochs: (1) train the adversarial network for a single epoch, holding the classifier network fixed and (2) train the classifier network on single sampled mini batch, holding the adver- sarial network fixed. Details on the adversarial network architecture are provided in Appendix A For the classifier network, we employed the same network as above in our proposed methodology and kept all training choices, such the optimisation algorithm and mini-batch size, the same. The classifier was pretrained for 2 epochs.
We modified Driven'' code so that we can sweep 0% over the parameter X. The result of 12 the adversarial approach is shown in the right column of Figure I Once again, using the same 100 training and testing pairs as above, we find solutions to 0² for a set of 15 X's in j0, IJ. for total of 1500 Pareto candidates. We can immediately see that compared to the proposed methodology. the adversarial is less capable of finding Pareto front that spans the faimess-accuume accuracy space. Indeed the set of dominated (onddominated) points in the adversarial approach is larger (smaller) relative to the proposed approach, Table
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Alekh Agarwal, Alina Beygelzimer, Miroslav Dudik, John Langford, and Hanna M. Wallach. A reductions approach to fair classification. In Proceedings of the 35th International Conference Machine Learning, ICML 2018, Stockholmsmüssan, Stockholm, Sweden, July 10-15, 2018, pp- 60 60-69,20 2018. URL https//t/tttpppprreeeeiaago// mmr
httpp:///ppooeeeea.......eeeeeea wttpp//ss//////a/aallal..0.. 8tm
Angwin, Larson,
Mattu,
and L. Kirchner.
Machine
bias.
ProP-
ublica, 2016. bachine-biast wawiiskkassssanne aeeessssssssessssnneeee htw.errmmmmm--sssteeeennnnn
URL https:/
Yahay Bechavod and Katrina Ligett. Learning fair classifiers: regularization--nspired approach. CoRR, 0bs//177700044 2017.
Alex Beutel, Jilin Chen, Zhe Zhao, and Ed Huai hsin Chi. Data decisions and theoretical implications when adversarially learning fair representations. CoRR, 0etss/777.00077. 2017.
Peter Bickel, Eugene A Hammel, and William O' O'Connell. Sex bias in graduate admissions: Data from berkeley. Science, 187(4175):398 -04, 1975.
Toon Calders and Sicco Verwer. Three naive bayes approaches for discrimination-ffee classification. Data Mining and Knowledge Discovery, 21 Sep 2010. ISSN 1573-756X. doi: 10. 1007/s10611 10100000111001000111011110111111
htRlihhtps:///tt wrg. 100 11t11111111000/0
Flavio Calmon, Dennis Wei, Bhanukiran Vinzamuri, Karthikeyan Natesan Ramamurthy, and Kush R Varshney. Optimized pre-processing for discrimination prevention. In Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Feeguus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems 30, pp. 3992 4001. Curran Associates. Inc.. 2017. URL httt:://aapess. www. wwtppppp...ppp.....ppppppppppppe for
6988 - 6988-opt wtt.::iimmmeeeeeeeeeeeeeeeeeeeee
pdf
Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data, 5, 10 2016. doi: 2t..0////.g.226.00000
Sam Corbeut-Davees, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. Algorithmic de- cision making and the cost of fairness. In Proceedings of the 23rd ACM SIGKDD Interna- tional Conference on Knowledge Discovery and Data Mining, KDD '17, pp. 797-806, New York, NY, USA, 2017. ACM. ISBN 978 978-14500-488744 doi: 10. 10.1145:30999833309999 URL 3098095
http::/doo acm, org /10
I. Das and J. E. Dennis. closer look at drawbacks of minimizing weighted sums of objectives for pareto set generation in multicriteria optimization problems. Structural optimization, 14(1):
63-69, Aug 1997. ISSN 1615-1488, doi: 10.1007/BF011995559 URL htttttt////dd htt.//ii.ogg///oooooooooo 10. 100. BF0
Indraneel Das and J Dennis. Normal-boundee intersection: A new method for generating the pareto surface in nonlinear multicriteria optimization problems. SIAM Journal on Optimization, 8, 07 2000. doi: 10.1137/5105222249633377711
William Dieterich, Christina Mendoza, and Tim Brennan. Compas risk scales: Demonstrating curacy equity and predictive parity. Technical report, Northpointe Inc., 2016.
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness
through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Con- ference, ITCS '12, pp. 216-226, New York, NY, USA, 2012. ACM. ISBN 978-1-4503-1115 URL http: doi acm. wwt//0..//000 1ttp:/10.111//299022200
1. doi: 0111452209023622000225
2090255
M. Ehrgott. Multicriteria optimization, Lecture Notes in Economics and Mathematical Systems. Springer-Verlat 2000.

---

Under review as a conference paper at ICLR 2020
Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasub raan. Certifying and removing disparate impact. In Proceedings of the 21th ACM SIGKDD Knowledge Discovery and Data Mining, KDD '15, pp. 259-268, New York, NY, USA, 2015. ACM. ISBN 978-1 -4004-363364-22 doi: 10. 10.1145/27835588277837 URL http::/tp///o.... acm www.orrgg./.... 11w...1152228788 2783311
International Conference
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. lanainaadversarill training of neural networks. J. Mach. Learn. Res., 17(1):209--0033 January 2016. ISSN 1532-4435. URL http:// acm. org/ httg/ciiee tat inn
2944666652244744
I. Giagkiozis and P.J. Fleming. Methods for multi-objective optimization: An analysis. In- formation Sciences, 293:338 - 350, 2015. ISSN 0020-0255. doi: https://doi.org/10..o.
ins.2014.08.071. URL http: /wwp//wwww...// www.ssccienneiieeeoeomoe hwt.///ciiinee/erriiile//aatiaie 50C2002551400901
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning Volume 70, ICML' 17, pp. 1321-1330. JMLR.org, 2017. URL http:///t/ wwl.laam.com org/ citatiii......... cfm? 3fmmidd33003B1 3305518
Moritz Hardt, Eric Price, and Nathan Srebro. Equality of opportunity in supervised learning. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS' 16, pp. 3323-3331, USA, 2016. Curran Associates Inc. ISBN 978-1-5108-3881-9, URL http::/dl.... acm. wwt.oorg//oooooooooooo wwt. on wtt..ccmmmm..11112.2....
Keisuke Hirano, Guido W. Imbens, and Geert Ridder. Efficient estimation of average treatment effects using the estimated propensity score. Econometrica, 71(4):1161-1199 2003. ISSN 00129682, 14680262. URL http: //wp:////www//. www.:ssstorrrroooo http:/oo.oo//tttae/e//eee//..//.
Guido W. Imbens and Donald B. Rubin. Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction. Cambridge University Press, New York, NY, USA, 2015. ISBN 0521885884, 9780521885881
James E. Johndrow and Kristian Lum. An algorithm for removing sensitive information: Application to race-independent recidivism prediction. Ann. Appl. Stat., 13(1):189-25) 03 2019. doi: 10.
1214/1--A0AS1201 URL https::/t///. ://oii....oo// www.oo.g...... 121411...1111111 100111//11AAA120000
Matthew Joseph, Michael Kearns, Jamie H Morgenstern, and Aaron Roth. Fairness in learning: Classic and contextual bandits. In D. D. Lee, M. Sugiyama, U. V. Luxburg, Guyon, and Garnett (eds.), Advances in Neural Information Processing Systems 29, Curran Associates, Inc., 2016. URL http:://app:rr nips. cc/paper/ pddf
pp. 325-333.
6355- fai rness- o http:///alaasiiiaaooooooonaaaaaa
Faisal Kamiran and Toon Calders. Data preprocessing techniques for classification without discrim- ination. Knowledge and Information Systems, 33( 33(1): Oct 2012. ISSN 0219-3116. doi: 10.
1007//0011--011-046--8 URL /
011 0463--0433-11
Toshihiro Kamishima, Shotaro Akaho, and Jun Sakuma. Fairness-aware learning through regular- ization approach. In Proceedings of the 201 IEEE / Ith International Conference on Data Mining Workshops, ICDMW '11, pp. 643-650 Washington, DC, USA, 2011. IEEE Computer Society. doi: 10.1109/CDMW22011..83 URL huttps://das org/10.
ISBN 978-0-7695-4409-00
1109/ICDM 2011.83
Aria Khademi, Sanghack Lee, David Foley, and Vasant Honavar. Fairness in algorithmic decision making: An excursion through the lens of causality. In The World Wide Web Conference, WWW $19, pp. 2907-2914, New York, NY, USA, 2019. ACM. ISBN 978- 1188--11503-6674--5 doi: 10.1145/ 3308558.33 335599.. Uttttt////Gooi acm. org/ 110.01111533300 htt.://.....................1//. 3313559
Niki Kilbertus, Mateo Rojas-Carulla, Giambattista Parascandolo. Moritz Hardt, Dominik Janzing, and Bernhard Schölkopf. Avoiding discrimination through causal reasoning. In Proceedings of the 31st International Conference Neural Information Processing Systems, NIPS' 17, pp. 656- 666. USA, 2017. Curran Associates Inc. ISBN 778-1-5100-6006-11 org/citat: on wfw.:dd=mmmmm1111 77..4477.11329934 3294834
URL http::///// acm.
10

---

Under review as a conference paper at ICLR 2020
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. International Conference on Learning Representations, 12 2014.
Jon Kleinberg. Inherent trade-offs in algorithmic fairness. In Abstracts of the 2018 ACM Interna- tional Conference on Measurement and Modeling of Computer Systems, SIGMETRICS '18, pp. 40-40, New York, NY, USA, 2018. ACM. ISBN 978- 1 1-550--586----0 doi: 10.1145/3219617.
3219634. URL htttp://eee /tt.////iii// acm. org /1C. 1C. 1145/32196111 3219634
Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fair- ness. In I. Guyon, U. V. Luxburg, S. Bengio, Н. Wallach, R. Fergus, S. Vish- wanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems 30, pp- 4066 Curran Associates, Inc., 2017. URL http::/paperr: nips cc/paper/
6995 wtt-=-uuunnerfcttaaaeeee fairness.
Fan Li, Kari Lock Morgan, and Alan M. Zaslavsky. Balancing covariates via propensity score weighting. Journal of the American Statistical Association, 1 113(521):390- 400, 201 doi: 10. (11.2255.0....16222.
URL
2016.
1260466
Fan Li, Laine E Thomas, and Fan Li. Addressing Extreme Propensity Scores via the Overlap Weights. American Journal of Epidemiology, 188(1))250-257, 09 2018b. ISSN 0002-9262. doi: 10. h0.10993aje/kwy201... URL StLLttts::.. /tttts::/do...rrgg///. www.e/rrggo.oo.. http:////.1..1///////////o/o
Miqing Li, Shengxiang Yang. and Xiaohui Liu, A performance comparison indicator for pareto front approximations in many-objectivy optimization. In Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, GECCO 15, pp. 703-710, New York, NY, USA, doi: 10.11455273948022554677. URL URL http:://ooi
2015. ACM. ISBN 978 177--7----000 4500--3472-3 acm. wtt..oo/11..115/7335511101 -T7-115555733988.775477777
Joshua R. Loftus, Chris Russell, Matt J. Kusner, and Ricardo Silva. Causal reasoning for algorithmic fairness. CoRR, /e5/1805.058855 2018. Utl.ittt.//aaxxv..rrr/b/////////
Gilles Louppe, Michael Kagan, and Kyle Cranmer. Learning to pivot with adversarial net- works. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vish- wanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems 30, pp. 981-990. Curran Associates, Inc., 2017. URL http::/ppapera nips cc/paper/ 6699 lew.eeeennn------iooooo with
-tt.ooo-wiiiaaaaaerrarrraaa adversarrall-nexwoksss.u net.-rks.
adversarial
David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. Learning adversarially fair and transferable representations. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp- 3384-3393, Stockholmsmssen Stockholm Sweden, 10-15 Jul 2018. PMLR. URL www.s/pessssvsee press//800mmdrrass 8a ht.ml
P. Manisha and Sujit Gujar. A neural network framework for fair classifier. CoRR, abs/ abs/181/ 181 11881.000244 1.00247. 2018. URL http:///wwxxvv.o.aaa/a///aaa/
org
C0247
Harikrishna Narasimhan. Leamingg with complex loss functions and constraints. In Amos Storkey and Fernando Perez-Cruz (eds.), Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, volume 84 of Proceedings of Machine Learning Research, pp. 1644-1654, Playa Blanca, Lanzarote, Canary Islands, 09-11 Apr 2018. PMLR. URL httt:::: //proceedings mlr. press / v8 4
html
Va'acov Ritov, Yuekai Sun, and Ruofei Zhao. On conditional parity as notion of non- discrimination in machine learning. To appear in Statistical Science, 2017.
Amartya Sen and S McMurrin. Equality of what? Tanner Lectures on Human Values, Volume 1, 1980.
Christina Wadsworth, Francesca Vera, and Chris Piech. Achieving faimess through adversarial learning: an application to recidivism prediction. CoRR, abssl180..001111 807999999,,.. 2018. URL http:
//arxiv.org w/rrx//.org//bb//807.. 001

---

Under review as a conference paper at ICLR 2020
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez- -oorizzZddri,uzz and Krishna P. Gummadi. Fair- ness constraints: Mechanisms for fair classification. In Proceedings of the 20th International Conference Artificial Intelligence and Statistics, AISTATS 2017, 20-22 April 2017, Fort Laud- erdale, FI., USA, pp. 962-970, 2017a. URL htt.p::/procceediiiiit zafar 7a. html
r.press/v5A,
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P. Gummadi. Fair- ness beyond disparate treatment &#38; disparate impact: Learning classification without dis- parate mistreatment.
Proceedings of the 26th International Conference on World Wide Web, WWW '17, pp. 1171-1180, Republic and Canton of Geneva, Switzerland, 2017b. Interna- tional World Wide Web Conferences Steering Committee. ISBN 978-1-4503-4913-0 doi: 00.1145/303891133055666 /ttps:::://do
htt.//i.oorggg// www.ooggg///. 11w.11445/2 11t.....11111111 011.588811113302662 3052660
Muhammad Bilal Zafar, Isabel Valera, Manuel Hoomz-Rooriguue Krishna P. Gummadi, and Adrian Weller. From parity to preference-based notions of fairness in classification. In NIPS, 2017c
Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adversar- ial learning. CoRR, abs/1801.0770 007593, 2018. URL http: /ttppiv///xvvv..//a org/ abs 110//11111111 07593
A APPENDIX
In this section, we provide additional details on the experiments conducted in Section 6 First, in Table [] we summarise the datasets on which all experiments were conducted. The feedforward architecture used in the proposed methodology is given in Table 2 for each of the three data settings.
Next, we describe the neural net we employed to estimate the propensity scores P(a = X We actually used the same neural net for all three data settings in Table I Each of the 3 fully-connected layer, with 32 hidden units each, is interspersed with a dropout layer with dropout probability 0.2. The ReLU activation function is used in all intermediate layers while the sigmoid function is used in the output layer. The cross-entropy loss is used between the estimated scores and the true labels dictated by To learn the network, 2014). The initial learning is fixed throughout at 0.001. Training took place over 100 epochs. Mini-batch size was chosen be around 5% of the training were used in the recidivism and UCI datasets, respectively. After the propensity neural network is trained, we apply the calibration technique proposed in Guo al. (2017) calibrate the probability predictions. We used their GitHub code with no modification.
the ADAM optimisation algorithm (Kingma & Ba
size; 150 and 1000 minibatch sizes
Figure the additional output from the experiment that produced Figure on the UCI dataset with race the sensitive attribute. We also repeated the experiment in Section 6 by penalising the ATO in all layers. The results are shown in Figure 6 The adversarial approach we compared the proposed methodology against used a network with 4 hidden layers with 32 hidden units in each. ReLU activations were used throughout except the final layer where the sigmoid function is used. The adversarial network pretrained for 5 epochs. Optimisation used ADAM and minibatch sizes described in Table I
Table 1: Dataset descriptions
dataset features
Dataset
dim(x)
binary outcome y
sensitive
training size
testing size
minibatch size
Recidivism
12
Reoffend in years?
binary race
3086
3086
150
UCI
93
Income above 50K?
binary race
15470
15470
1000
UCI
93
Income above 50K?
binary gender
15470
15470
1000
12

---

Under review as a conference paper at ICLR 2020
Table 2: Network architecture
neural network features
Dataset
layers L
hidden nodes
Recidivism
4
UCI
32
10
UCI
32
10
Epoch aptprobhot): 0.000ty-00000 cassaleo 1272; 80-15.021
texting apgigoonntts p.penatt--.3000 Boie
my E.ppm 5.500 20-200.44; eerace
my E.ppe0 565, my SP-17.461; rerace
y=0.03; protected
y=0.03; protected
y protected 1-1
y=0.00: protected 2-1
20
protected
y=1.02) protected 2=1
y=1.03; protected z-1
13-
13-
0.0
0.2
1.0
0.0
D4
5.8
Epoch 4033 toal na die probieeti: 5177
Epech 400: tod nn dee probiretic perully-a.150 any B0xp0xpp-o 0.030, my www.pp_0000000.0 9,801; z-race
www.E0pg-3..0 3,17% DP 15.765; 2-577882288 zarace
y=0.00; protected 1=0
y=0 00: protected z-0
y=0.00: protected
y= protected
y-1.03; protected 2-0
1=1.03: protected z-0
y= protected
y=
30-
23
0.
0.2
0.5
1.0
0.4
0.
Figure 3: Prediction probabilities in the test set of UCI (race) for varying values of X, indicated by p-penalty in the heading of each plot.
Table A comparison between the proposed methodology the adversarial technique for finding the Pareto front in each of the three data settings of Figure Reported are the number of non-dominated points. Higher is better.
UCI (gender)
UCI (race)
Recidivism
Proposed
44
33
89
Adversarial
13
9
27
13

---

Under review as a conference paper at ICLR 2020
Epoch 459, astorcthsti pe.0ity-0.000
Epoch astorsbnetii penalty-0.050
crugalan 1/2 my. 20177 2000
caucalidd 181;; my 20-25. 180
y=0.02: protected 2=10
y=0.02 protected z=0
y=0.03; protected
20-
y=0.03; protocted
y=1.02: protected ZwO
y=1.02; protected zw0
y- 00; protected 1-1
y-1.00; protected z-1
3-
3-
10-
4--
54
2-
0
Epech ap:loobhstt: 0..49 cassulwo p44; ED 20-83.225 23.131
teating pisloronott: penaaity-0770 000; toleo oc0 E0pomo.000 mi
my 10mp- 140, oP 078;
y=0.03; protected
y=0.03; protected z-0
y 03: protected 1-1
y=0 protected 2-1
1=1:03:00
y=1.00: protected
y-1 02: protected 2=1
y=1.03, protected z-1
15-
20-
10-
0.0
0.2
044
0.
1.0
0.0
04
56
Figure 4: Prediction probabilities in the test set of UCI (gender) for varying values of X, indicated by p. penalty in the heading of each plot.
UCI (race)
Pareto frontier Chobyshev 0%.
U( P
0.1
0.0
R(O2 n Punt)
Pareto frontier
Ui Poot)
0.1
ad adversarial 0
0.0
0.4
0.5
0.6
0.7
R(ØA, np Punt)
Figure 5: This reports the result for UCI (race) in the same experiment that produced Figure
14

---

Under review as a conference paper at ICLR 2020
UCI (gender): Penalising all internal representatane lations
0.20
Pareto frontier
Chebyshev 0
0.15
UiA Poss
0.10
0.05
0.00
0.4
0.5
0.6
0.7
R(O Pisa)
UCT (race): Penalising all internal representations
0.150
Pareto frontier
Chebyshex A
0.125
0.100
best
U(A P
0.075
0.050
0.025
0.000
0.4
0l5
0..
0.7
R(O P(oa)
Recidivism (race): Penalising all internal representations
Pareto frontier Chebyshev 0%
0.08
0.06
U ( Punt)
0.04
0.02
0.00
0.600
0.625
0.650
0.675
0.700
0.725
0.750
R(0), Pine)
Figure 6: We repeated the experiment in Figure I changing only the faimess penalty to penalise the ATO in all layers. There is drop in the quality of the Pareto front estimation compared to penalising just one internal layer. Namely, more of the candidate points are dominated points this modified experiment where we've penalised ATO in all layers. It seems that should have perhaps also tuned for a brand new architecture given this new penalty.
15


Reviews:
[{"review_id": "rJgwE8_xcS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose a novel joint optimisation framework that attempts to optimally trade-off between accuracy and fairness objectives, since in its general formal counterfactual fairness is at odds with classical accuracy objective. To this end, the authors propose optimising a Pareto front of the fairness-accuracy trade-off space and show that their resulting method outperforms an adversarial approach in finding non-dominated pairs.\n\nAs main contributions, the paper provides:\n* A Pareto objective formulation of the accuracy fairness trade-off\n* A new causal fairness objective based on the existing Weighted Average Treatment Effect (WATE) and Average Treatment Effect for the Overlap Population (ATO)\n\nOverall, I think the paper makes an interesting contribution to the field of fairness and that the resulting method seems quite attractive for a real-world practitioners. However, I found the writing / notation imprecise at times and the experimental section too small (lacking an extensive set of baselines, and only on two datasets). For these reasons, I give it a Weak Accept.\n\nSome feedback on notation / writing:\n* Typo on page 2, the loss L should be defined on X x Y and not Y x Y\n* In page 5, h is being used without being introduced first \n* the justification for using ATO in the internal layers of the network is a bit insufficient\n\nIn terms of suggestions, I think the experimental section needs to be extended and that the various modelling choices need to be explored and/or be further justified.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "r1xy_56sFr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a method to approximate the \"fairness-accuracy landscape\" of classifiers based on neural networks.\nThe key idea is to set up a multi-dimensional objective, where one dimension is about prediction accuracy and another \nabout fairness. The fairness component relies on a  definition of fairness based on causal inference, relying on the \nidea that a sensitive attribute should not causally affect model predictions.\n\nI found the causal idea intriguing, since it makes sense that we don't want a sensitive attribute to have a causal effect.\nHowever, there may be several problems with this approach:\n\n1) For a causal estimate to be valid we need several assumptions. For example, we need A (the sensitive attribute) \nto be independent of potential outcomes conditional on X --- the so-called \"unconfoundedness assumption\" in causal inference.\nWe also need \"positivity\", i.e., that 0< P(A=1|X) <1. \nThese assumptions are not discussed in the paper. Furthermore, the particular context of the paper, where the treatment is actually an immutable characteristic, makes such discussion much more subtle. \nWhat will we do, for instance, if there are no A=1 in the sample when X = ...?\n\n\n2) The authors seem to assume that the propensity score model is well specified. This can be tested, e.g., using [1].\nWhat do we do when this fails? \n\n\n3) Why do we want U to be small, i.e., why do we want the causal effect of A to be small, is never justified.\nIn particular, its relation to \"fairness\" is never fleshed out, but just assumed to be so.\nThis can be problematic when, say, we are missing certain important X that are important for A. \nThen, there will be a measurable causal effect of A on h(). \n\nSome other problems:\n- What is the reason for focusing on 'neural classifiers'? There is nothing specific in the method or analysis \nthat relates to neural networks, except for the use of the causal estimand in a 'hidden layer'.\n\n- In the Introduction, the authors could cite the works of Amartya Sea, etc., on fairness. \nCertainly the study of fairness problems did not start in 2016.\n\n- What exactly is a \"sensitive attribute\"? If we don't want to bias our predictions, then why include it in the analysis?\n\n- It is unclear what is new and what is related work in page 3.\n\n- Sec. 4: The claim that \"causal inference is about situations where manipulation is impossible\" discards \nvoluminous work in causal inference through randomized experiments. In fact, many scientists would \nagree that causal inference is impossible without manipulation.\n\n- As mentioned above, why this particular estimand leads to more \"fairness\" is never explained.\n\n- Do we need a square or abs value in Eq (5)?\n\n- The experimental section is weak and does not illustrate the claims well. \n   It would be important to explain the choice of the particular causal estimand, the choice of the hidden layer to put the estimand in, to explore the choice of the objective, and so on. Currently, none of these choices/design aspects are being investigated.\n\n\n\n[1] \"A specification test for the propensity score using its distribution conditional\non participation\" (Shaikh et al, 2009)", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "BJexY-Q4cr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "General:\nThe paper proposed to use a causal fairness metric, then tries to identify the Pareto optimal front for the vectorized output, [accuracy, fairness]. While the proposed method makes sense, I am not sure what exactly their contribution is. It is kind of clear that Pareto optimal exists, and what they did is to run the experiments multiple times with multiple \\lambda values for the Chebyshev method and plot the Pareto optimal front. \n\nPro:\nRan multiple experiments and drew the Pareto optimal front for the considered dataset. \n\nCon & Question:\nThe so-called causal fairness metric does not seem to be any more fundametal than the other proposed metrics. It seems like they worked with another metric. \nAfter defining the fairness metric, everything else seems straightforward. Just use test (validation) set to estimate the accuracy & fairness, then plot the results on the 2d plane. \nCan we identify the Pareto optimal front without running all 1500 experiments? What happens when running a model takes long to train? Then, the proposed method cannot be practical. \nFigure fonts are very small and hard to see. ", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This manuscript investigates and characterizes the tradeoff between fairness and accuracy in neural network models. The primary empirical contribution is to investigate this tradeoff for a variety of datasets.

The reviewers and AC agree that the problem studied is timely and interesting. However, this manuscript also received quite divergent reviews, resulting from differences in opinion about the novelty of the results. IN particular, it is not clear that the idea of a fairness/performance tradeoff is a new one. In reviews and discussion, the reviewers also noted issues with clarity of the presentation. In the opinion of the AC, the manuscript is not appropriate for publication in its current state. 
Decision: Reject