Fairness with Wasserstein Adversarial Networks
Authors: serrurier Mathieu
Loubes Jean-Michel
Edouard Pauwels
Venue:  (2020)

Abstract: Quantifying, enforcing and implementing fairness emerged as a major topic in machine learning. We investigate these questions in the context of deep learning. Our main algorithmic and theoretical tool is the computational estimation of similarities between probability, ``\`a la Wasserstein'', using adversarial networks. This idea is flexible enough to investigate different fairness constrained learning tasks, which we model by specifying properties of the underlying data generative process. The first setting considers bias in the generative model which should be filtered out. The second model is related to the presence of nuisance variables in the observations producing an unwanted bias for the learning task.  For both models, we devise a learning algorithm based on approximation of Wasserstein distances using adversarial networks. We provide formal arguments describing the fairness enforcing properties of these algorithm in relation with the underlying fairness generative processes. Finally we perform experiments, both on synthetic and real world data, to demonstrate empirically the superiority of our approach compared to state of the art fairness algorithms as well as concurrent GAN type adversarial architectures based on Jensen divergence.

Under review as a conference paper at ICLR 2020
FAIRNESS WITH WASSERSTEIN ADVERSARIAL NET- WORKS
Anonymous authors Paper under double-blind review
ABSTRACT
Quantifying, enforcing and implementing fairness emerged as major topic in machine learning. We investigate these questions in the context of deep learning. Our main algorithmic and theoretical tool is the computational estimation of similarities between probability, "à la Wasserstein", using adversarial networks. This idea is flexible enough to investigate different fairness constrained learning tasks, which we model by specifying properties of the underlying data generative process. The first setting considers bias the generative model which should be filtered out. The second model is related to the presence of nuisance variables in the observations producing an unwanted bias for the learning task. For both models, we devise a learning algorithm based on approximation of Wasserstein distances using adversarial networks. We provide formal arguments describing the fairess enforcing properties of these algorithm in relation with the underlying fairness generative processes. Finally we perform experiments, both on synthetic and real world data, demonstrate empirically the superiority of approach compared to state of the art fairesss algorithms as well as concurrent GAN type adversarial architectures based on Jensen divergence.
Keywords: Fair learning, Wasserstein distance, Adversarial Networks.
INTRODUCTION
Along the last few years, much emphasis has been laid on fairness issues in machine learning. Actually, when the learning sample presents biases, these are learnt by algorithms based on loss functions promoting closeness observed data. Using such models for decision making generalizes biases to the whole population. This drawback of machine learning. also known as unfairness, has become major challenge in the domain. For recent survey this topic refer to Dwork et al. (2012) Zemel et al. (2013) or Friedler et al. and references therein.
Fairness usually deals with situations where algorithm exhibits different behavior for two different subgroups of the population, while these subgroups should not play any role in its outcome. This situation is often modeled follows the algorithm should aims forecasting a variable Y based on observations X. Fairness is then defined with respect to protected variable, called protected attribute, S which represents membership each population subgroup. The algorithm is called fair if its predictions does not depend too much on S.
Defining and quantifying this notion of dependency is complicated task and has received much attention. One of the main tools is the so-called disparate impact which measures if the decision taken by an algorithm differs from one group another. Absence of disparate impact is called demographic parity. Another measure of fairness is given by the dependency of prediction error with respect to S. The independent case is form of fairness called equality of odds. We refer for instance to Chouldechoval (017)) Friedler al. (2016) or Besse et al. (2018) and references therein. Both situations amounts to considering that either the distribution of the prediction, or its conditional distribution given the target variable Y, does not depend S. Hence faimess quantification can be naturally implemented using distance between conditional distributions.
This point of view has been extensively studied when trying "repair" data sets as described for instance in Feldman et al. (2015). Johndrow & Lum (2017). Hacker & Wiedemann (2017) or Friederrr et al. (209)). This solution consists in changing the input data that predictability of the protected

---

Under review as a conference paper at ICLR 2020
attribute is impossible. The data will be blurred in order to obtain fair treatment of the protected class. The natural distance to measure the difference between the conditional distributions is the so-called Wasserstein distance, which provides alternative framework to measure the dependency of the decision rule with respect to the protected attribute as shown in Barrio et al. (2019a) or [Barrio eet al. (2019h). Yet previous methods face the difficulty of computing the Wasserstein distance which is a challenging task as shown in Peyré & Cuturi (2019).
In this work, we aim at building fair classifiers by considering a Wasserstein type constraint. Adding constraints to the classifiers to get fair behavior has been studied in several papers. We refer to Friedler et al. (2019), Zafar al. (2017a) and references therein. Our approach. yet sharing some similarities with Edwards & Storkey (2016), based on Ganin et (2016). is more flexible and enables to solve wider classes of fairness problems based on different adversarial architecture resulting in more suited loss functions. Wasserstein constraint for fairness has also been considered in Jiang] al. (2019) for binary logistic regression. In the following, we provide algorithms which, for both demographic parity and equality of odds, can incorporate fairness constraints based on the I-Wasserstein distance. Here we will consider two different mathematical models, describing the relationships between the variables X the target variable Y and the protected variable S. Computing Wasserstein type constraints is difficult, we use neural networks as they have been proved useful to estimate Wasserstein type distances as discussed in Arjovsky al. (017)) The proposed approach can be combined with any kind of neural network predictor. Hence we are able to manage a large variety of input data structure (e.g. images) as well as output labels (multiclass, regression, images ...). We demonstrate on fairness benchmark datasets that the proposed Wasserstein approximation framework outperforms both classical fair algorithms (e.g fair SVM) as well as similar adversarial architectures based on Jensen GAN losses very close to the approaches described in Beutel et al.
(017) Madras et al. (2018).
The paper falls into the following parts. Section 2 is devoted to the presentation of Wasserstein distance, approximation schemes and applications to fair modeling. Section 3 describes a first model of fairness related to demographic parity. Section 4 considers a second option connected to equality of odds. For both models, we propose an adversarial network methodology to obtain a fair classifier for each type of faimess. Section studiee these algorithms real benchmark data sets as well as synthetic simulations.
2 FAIRNESS: DEFINITIONS AND METRICS
2.1 FRAMEWORK
The statistical model we consider is the following. The problem consists in forecasting binary variable Y E (0, 1}, using observed covariates X E Rd 2 1. We assume moreover that the population can be divided into two categories that represent a bias, modeled by a variable S E 10. This variable is called the protected, or sensitive, attribute which takes the values S 0 for the minority" class and 5 1 for the majority class. [] We observe n joint realizations of these variables D = {(X;, S, Y)), i E [1....,n]},
..). We use the following notations Dj {(X,, Y,), i E f1,..., n). S. = l} for I = 0, D [X2, i E (1.....n). S. = L,Y, = k} for = 0, 1, 1: = 0,
The fair classification problem aims at predicting Y from the variables X, using a family of binary classifiers E 9 Id - (0, }} without using the information conveyed by S. For every g E 9, the outcome of the classification will be the prediction Y = 9(X).
We consider in the following that the classifier g comes from a score given by a predictor F:; Rd +
such that Y 1(X)>> for chosen threshold r) >
Different criteria have been proposed for measuring the fairness of f depending of the context. The disparate impact (DI) measures the sensitivity of the predicted values y with respect to S.
DI(Ý
p(L 1|$ = 0) p(Ý 1/s = 1)).
Note that in the case where S is not a binary variahle but multidimensional or multi-class, can consider one versus one faimess identifying in each case "minority""
2

---

Under review as a conference paper at ICLR 2020
The most favorable situation in terms of fairness with respect to the protected attribute S, is achieved when DI(Y, SS) 0 (i.e. P(Ý = 1| = 0) P(Ý 1|5 = =))) which corresponds to the situation known as demographic parity. In this case, the predicted class is independent from S.
While its interpretation is clear, the mathematical properties of the disparate impact measure not favorable, in particular it lacks robustness and smoothness features which would be necessary to blend algorithmic practice and mathematical theory. In the following, we propose an alternative measure of equality of opportumity which features smoothness properties and comes with strong mathematical background. Given score function F, we set (x))) = ((F(X)|S (SF(X) = 0) the laws of conditional distribution of the score for each class and denote the corresponding quantile functions by Qo. and II. Independence of the decision with the variable S would entail that the repartition of the scores is similar for the two subgroups. So the distance between the quantiles of these two distributions acts as a measure of fairness measuring that the repartition of the score is spread in similar ways whatever the values of the protected attribute, hence acting as a sensitivity index of the predicted values F F(X) with respect to S. Namely define
1) and
(o
= -f, (Qo,(()) - (t)ett)
EMD(F(X), = W(C(((())
(1)
which corresponds to the so-called earth or Wi Wasserstein distance between the conditional distributions. Clearly (FF)( Li (F(X)) = 0 implies that DI(Y. 5) = O. So Wasserstein distance appears in this framework as a smooth criterion to assess the sensitivity w.r.t to the protected variable. This criterion corresponds to the quantity that used to measure fairness in [Barrio et al. (2019b) and Barrio et al. (2019a). Note that Wasserstein distance for fairness has been also considered in the seminal paper by Feldman al. (2013).
Another important criterion is the equality of odds, which measures the influence of the S on the accuracy of the algorithm. For this the prediction errors across the different class groups compared
and this notion of fairness is achieved when P(ý 1|Y 1,5 0) P( 1 = 1, S = 1) and P( O|Y- 0, = 0) p(ý OJY 0, S 1). Here again, this condition can be interpreted as a notion of independence of the conditional distributions defined for (i, () E {0, 1) as (X))) the distribution of the random variable (J(X)JY = S = s). Hence as we exposed for the notion of equality of opportunity, fairness will be assessed through the computations of the Wasserstein distances W(CB (X)), (F)))) and (F (F)))).
Note that in some cases, are only interested in equality of opportunities. This corresponds to the
case where we only require that P(Ý 1|Y S.S 0) P( 1|Y = 1, = 1) pointed out in Hardt et al. (2016). Hence in this case it amounts to control only ((C)( (F)))))
2.2 WASSERSTEIN DIVERGENCES USING NEURAL NETWORKS AND PROPERTIES
The earth-mover, or Wasserstein-l distances between probability distribution defined as follows: :
inf W(41,C2) TEI(E,,))) X,Y
X Y
where II(C,, is the set of all probability measures on X, Y with marginals Cn and C2. Disparate Impact is closely related to the notion of unpredictability of the variable S. Hence the aim in this case is to The distance associated to these notions is the total variation distance drv (L1, L2 but due intractability of this distance, it has been replaced in the machine learning literature by W. Clearly the independent case is obtained when the distance is null and the decrease of W leads to smaller DI as shown in the experiments in Barrio et al. (0119)). Hence a constraint on the Wasserstein distance promotes fairness, also in terms of Disparate Impact.
Although the infimum in Equation (2) is not tractable in general, it can be approximated by a neural network. The first step is to reformulate (2) using the Muntorovich-Rubineee duality Villani (2008)
W(C1, L2)
22)= = sup Ex~e, Ex~c,((X) SEFi
where Fg denotes the space of Lipschitz function. As second step, the approach proposed in Arjovsky et al.] (2017) is based on estimation of the supremum in (3) by replacing F, by the set of functions described by fixed neural network architecture with spectral normalization Miyato et al.
3

---

Under review as a conference paper at ICLR 2020
(2018). This provides general methodology estimate and optimize divergences à la Wasserstein and leads to interesting empirical results Arjovsky et al. (2017). Furthermore, we demonstrate empirically that this approach allows to control to some extent the empirical EMD divergence introduced above. The two examples of fairness measures which we have introduced are based on distributional divergence measured using Wasserstein distance and we propose to handle these divergence terms computationally using the dual formulation presented in this paragraph.
One specificity of the fairness problem which we consider is that, empirically, we only have access to finite number of samples for each values of the protected attribute (S E {0, 1J). For example, we only have access Lo and L1 through a fixed finite sample and the Wasserstein terms which we manipulate are only computed on finitely many samples. This raises the following comments.
Other divergences, such Sonsen-Shannen Goodfellow al. (2014), KL divergence or total variation Arjovsky et al.] (2017) can be approximated using neural networks. These divergences reflect similarities between mutually absolutely continuous probability measures. However they degenerate when considering singular measures. For the problems which we intend to attack in this work, aim at enforcing equality of distributions using only a fixed number samples. Entropy or total variation based divergences fail to capture dissimilarity between singular measures, and in particular they degenerate when considering disjoint finite sample sets. On the other hand, Wasserstein metric well defined and does not degenerate empirical distributions given by finite samples.
second favorable property of this metric its continuity features. When considering parametric distributions, this translate into continuity of the metric in the parameter space remarked by Arjovsky et al. (2017), resulting in numerically more favorable situations compared to discontinuous problems. Another important consequence of the continuity properties of Wasserstein distance is that it translates into stable approximation of distribution divergence in the limit of large i.i.d samples (see Appendix). This last property very desirable since all we can do from an empirical perspective is limited to finite samples.
3 TYPE FAIR LEARNING DEMOGRAPHIC PARITY
The first case where fairness is desirable corresponds to the situation where the target variable is biased (....). For instance, it is well-known that the income of a people is biased by the gender. The situation doesn't 't arise from biased gathering of data but from bias that exist in the real data and that we don't t want to reproduce in our model. Thus, a fair model in this case will change the prediction in order make them independent from the protected variable. A suitable objective for this problem is to obtain of disparate impact (or the SDI) as close as possible to 1.
Y
Y"
Y
Figure 1: Fairness type
Figure 2: Fairness type
This situation is formally represented in Figure I In this situation we require that
X IL S/Y
and
Y Il SJY
where Y* is not observed. Note that, as it intuitively expected, Y is not independent from S (even conditionally to X). In this example, Y could represent an ideal case where the income level reflect the proficiency and not the gender.
3.1 TYPE FAIR NEURAL NETWORKS
The following can be applied either in multivariate regression, Y E Rd or classification Y (0, }}. consider the type configuration described in Figure [ We propose neural network model with adversarial Wasserstein constraints on the output as described in Figure this networks, the function F is classifier regressor. In order have prediction independent from S, add
4

---

Under review as a conference paper at ICLR 2020
Aa
W(Z 21(Z))
X
F
(((F(())
A
X
T
F
UOY, F(T(X))
((Co(F(X)), (((((X))
W(C 412))
Figure 3: Networks for fairness type 1
Figure 4: Fair networks for faimeess type 2
Algorithm Type Fair learning algorithm
Require: the learning rate, À the fairness constraint, m the batch size, nb_iter, the number of iterations, the number of iterations for the Wasserstein estimators.
1: for & in nb_iter do
2: for j in The do
L(Do), and {2/, v mm L(Di) 4: update A with an by gradient ascent : V Ao 1 E (A(F(z))) -(((()))((((())
3: sample iid (1i,
5: Normalize A as in Miyato et al. (2018).
6: end for
7: sample iid, D2 = {d, = (x,, s,, Dc) ) L(D)
8: update F by gradient descent:
(F((.)).
Σ A(F(x;))
Σ A(F(z;))
172
DD Dol d:EDo
D& n Dil
d,ED1
9: end for
Wasserstein penalization reflecting the dependency between F(X) and S, we obtain the following optimization:
inf Exx((X Y)] AVCC)((())))
where the loss function for the problem. Note that the Wasserstein penalty term in (4) is exactly the EMD fairness measure which introduced in (I). Applying the approximation scheme described in Section 2.2 we obtain the following saddle point problem:
inf sup E (x(((((X))) V)) + A F [A(F(X)) F. [A(F(X)))) XWLD
AEF.
X~Li
are hyper-parameters and F. represents the of functions encoded by fixed architecture neural network with spectral normalization. Approximating expectations using empirical sample, the learning process for this model is described in Alg. [ As explained in Section 2.2 in the limit of large samples, the maximization in A provide a proxy for the Wasserstein distance between the two conditional distributions. Note that we can use a similar architecture for equality of opportunities.
4 TYPE 2 FAIR LEARNING: EQUALITY OF ODDS
The second case where fairness is desirable corresponds to the situation where the data are subject to bias nuisance variable which in principle of help for the learning task hand and which influence should be removed. On famous example is the dog wolf problem exposed in Ribeiro et al. (2016). In this example, that data was heavily biased by the presence, for the wolfs, and the absence, for the dogs, of snow in the picture. Although the presence of snow is not independent from the presence of wolfs, we prefer model that focuses on animal features rather than background. More generally, this kind of situation appears when the descriptors target variables show dependency with the protected variable due to a biased data collection process or when we plan to use the model on data that have different distribution with respect the protected variable (this could be the
5

---

Under review as a conference paper at ICLR 2020
if we want to detect wolfs and dogs in free area). The equality of odds is suitable objective for this type of fairness.
We represent the underlying data generation process formally in Figure B require the following conditional independence:
X II S/Y,
Y 1 S|X
where X is not observed. Note that, it is intuitively expected, Y and X are independent from S' (even conditionally on any other variable in the model). In this context, we suppose that there representation of the data X from which we can build model to predict Y which will be independent of S given Y. Back to our example for wolf and dog. X could be the pictures, X could be physical features of the animal. model learnt from this X could predict Y (dog, wol from picture independently of the presence of snow (even if the probability of observing snow is greater when the animal on a picture a wolf).
4.1 TYPE 2 FAIR NEURAL NETWORKS FOR BINARY CLASSIFICATION
In the following, we consider the case where Y E {0. 1] and type 2 configuration. We propose neural network model with adversarial Wasserstein constraints as described Figure 3.1
In this networks, the function F o T(X) (or F(Z) with Z T(X)) is classifier constructed in two steps a transformation T' : + T such that Z has the same properties as X (i.e. X 1L S)Y). In order achieve this goal, constraint the distribution Z conditionally to Y to be independent S. Based on this idea, we obtain the following optimization problem:
Z and a classifier F: 1 F(2) E 0, 1. We expect to build
inf F.
Ex ((F((((X)) +) X(WCC)(((X))) AWA (5)
WCIT(())
where a given loss function (binary cross entropy our setting). We then apply the approximation
procedure of Wasserstein distance described in Section 2.2 and obtain the following saddle point problem:
inf F.T
Ex ((F(T(X)),
+1 sup Ao(T(X)) E Ao(T(X)) + A(T(X)) - AT((())) 2I(X)
AIEE <B(X)
4)(X)
cI(x)
where A are hyper-parameters of the method and Fs describes all functions generated by a given fixed architecture neural network with spectral normalization Miyato et al. (2018). Based on finite sample approximation of the various expectations in this formulation, the learning process similar to Alg. and is fully described in the appendix. The supremum over Ao and A₁ for finite sample approximation of the expectations a proxy to the Wasserstein distance between the two conditional distributions of interest in 2.1 Moreover, Property [ states that in the limit of Wasserstein distance between conditional distribution set O the latent space Z = T(X) satisfies the same properties of conditional Independence as X Furthermore any classifier build a posteriori Z will satisfy equal opportunities with respect to Y.
Proposition 1 Assume that the deterministic map T satisfies
W
((CIT(
= and
then, we have T(X) IL S|Y. for any measurable map G. G(T(X)) 11 SJY.
Proof sketch: Both are expressions of equality in distribution. Nullity of Wasserstein distance entails T(X) IL S|Y. This implies that for any deterministic map G. G(T(X)) IL SJY. Note that, contrary to other repair procedures for which the transformation must be recomputed for any observations (in Barrio et al. (2019a) the transformation relies on the optimal transport map which depends on the observations). here the optimal transformation can be used directly for all new observations.
D

---

Under review as a conference paper at ICLR 2020
Table 1: Accuracy, Disparate impact and EMD for adult, bank and CelebA under demographic parity and equality of odds constraint
Type 1: demographic parity
data
Adult gender
Bank
CelebA
ACC
DI
EMD
ACC
DI
EMD
ACC
DI
EMD
Alg
UnfairClf
84.5[0.5]
0.19
0.18
90.3[0.2]
0.14
0.16 [0.7]
77.9 [0.7]
0.46
0.38
C-SVM
79.4 [0.6]
0.02
90.0 [0.2]
0.02
Wass. Alg1 X = 0.9
80.7 [0.7]
0.01
0.02
90.0 [0.2]
0.01
0.01
71.2 [0.1]
0.06
0.05
Gan, AlgT X 3
80.3 [0.7]
0.03
0.04
87.2 [0.9]
0.07
0.05
74.1 [0.3]
0.18
0.17
Wass. Alg1 X 0.65
83.2 [0.5]
0.07
0.03
90.2 [0.2]
0.01
0.02
74.6 [0.2]
0.18
0.17
Gan. Alg1. À
83.9 [0.4]
0.09
0.09
90.4 [0.2]
0.09
0.08
75.4 [0.3]
0.23
0.22
Type 2:
equality odds
ACC
DIy
EMDy
ACC
Dly
EMDY
ACC
Diy
EM
UnfairClf
84.5[0.5]
0.19
0.18
89.9[0.5]
0.07
0.08
78.1 [0.1
0.48
0.38
Wass. Alg1 À
81.0 [0.6]
0.03
0.05
89.2 [0.2]
0.04
0.05
77.2 [0.7]
0.27
0.23
Gan. Alg1 À 2
81.9 [0.7]
0.09
0.08
89.1 [0.2]
0.04
0.05
77.1 [0.6
0.27
0.23
EXPERIMENTATION
We show this section, empirical results supporting our theoretical expectations. For simplicity and reproducibility purposes, we keep neural networks as simple as possible and try to use similar architectures as much as we can. For all the experiments, we set the learning rate of Adam to 1e 1e-1 and nw to 10 (see algorithm description in Sections 3 and All experiments have been implemented with keras/tensorflow.
We consider three state of the art fairness benchmarks (i) impact of gender in adult database (predict income>50K, 48842 examples, 16 attributes) (ii) impact age (boolean 25 age > 60) in the bank database (predict credit acceptance, 45211 examples, 17 attributes) (iii) impact of the gender on the attractivity in a subset of the celebA dataset (64x64 rgb images, 19670 examples).
Fairness benchmarks:
Concurent methods: We compare results with the C-SVM implementation proposed by Zafar (et al. 22017a b) and a classifier based on our neural network architecture without fairness constraint (UnfairCIf) We also compare approach with GAN type Jensen adversarial in Beutel et al. (2017): Madras et al. (2018). Note that the architectures that we use are slightly different from the purpose ensure objective comparison with approach. Indeed keep our adversarial architectures and only replace Wasserstein loss and network by a classifier with binary cross entropy and GAN trick for training.
original papers, this
Results: Table D reports accuracy (ACC), DI and EMD in a 70% train-3nnnttes scheme with 10 ( repetitions assess variability in both demographic parity and equality of odds scenarios. For equality of odds, we aggregate fairness measures DI and EM D, conditioning on Y and summing over 0, 1. these aggregated measures are denoted by DIy and EMDy. For the adversarial approaches we report one result with high fairness constraint and one result with a lesser constraint (obtained by considering different values of A). For the demographic parity constraint, all examples, algorithm reduces the DI close to 0 with an acceptable accuracy decrease. Figure 5Jillustrates that (see Equation (4)) controls the trade-off between accuracy and fairness
our last both in wasserstein and GAN configurations. Our approach clearly dominates concurrent methods in all situations in terms of both accuracy and fairness level. This is further illustrated in the third part of Figure 5 where the accuracy fairness tradeoff is very favorable to wasserstein approach compared to more traditional GAN methods. Finally Table illlustrates difficulties for GAN models enforce hard faimess constraints (DI close to 0). For equality of odds, method and GAN approach perform similarly.
SStandard deviation is only reported for accuracy because it negligihle for other quantities
7

---

Under review as a conference paper at ICLR 2020
gunder wasiesttin
cender qun
ACC
ACC
EMD
01/5
0.84
EMD
0.135
0.150
0.8
0.12
0.23
0.100
0.100
0.1
0.1
0.050
0.000
0.40
0.025
0.00
8b
2.04
0.0
CGU
0.0
Figure 5: Accuracy/ fairness tradeofI's between our Wasserstein approach and more traditional GAN approaches similar to Beutel et al. (2017); Madras et al. (2018) for demographic parity.
a)
b)
c)
d)
Figure 6: Fair auto encoder.a) T-shirt, b) fair representation of T-shirt, c) shirt, d)fair representation of shirt
Learning based on fair representations: To illustrate Proposition and the fact that our fairness constrinat can be applied to other type of problem than classification, consider classification of T-shirts versus shirts (Y = {I's shirt, shirt)) in the fashion-MNIST Mataset[Xiao al. 2017 (12000 training examples, 2000 test examples). These to distinguish in this dataset (accuracy around 0.9). We bias the dataset by adding a color (S (lurquoise, yellow) correlated the class variable Y: P(S yellow|Y Y T -shirl) shirl) = 0.9. We apply the following experimental process: train on the biased dataset, and compare validation performances both the biased test set and the same biased test set, with switched colors: P(S = yellow]Y = sshrt) = 0.1 and P(S = Turquoise = shirt) 0.1.
classes known to be the most chal challengigg
0.9,
P(S = Turquoise)
We train fair auto-encoder with two Wasserstein adversarial networks constraining equality of odds for the decoded images. We observe in Figure that equality of odds is achieved by assigning the same color to all transformed images. We train first network (unfairClf) on the biased database. We train a similar network on the fair database, and construct a fair classifier (fairClf) by composition of the second trained classifier and the auto-encoder. As expected unfairCIf generalizes better on the biased test set (accuracy 0.94 versus 0.87). However when switching the test color distribution fairClf is far more robust (accuracy 0.82 versus 0.60). This demonstrates that in addition to building a representation which looks fair, our auto-encoder approach is robust to fluctuations of the bias variable distribution.
CONCLUSIONS
This work tackles the challenge of incorporating constraints to deal with bias issues in machine learn- ing. We show that Wasserstein is an appropriate choice of distance between conditional distributions to control faimess using adversarial neural networks. We also explicit mathematical models providing abstract frameworks to understand and apply two types of fair constraints (demographic parity and equality of odds). The predictor obtain prove efficient well known fairness benchmarks
8

---

Under review as a conference paper at ICLR 2020
well as synthetic problems. Our experiments designed with minimal hand tuning overcome reproducibility issues. As expected adversarial wasserstein constraints are more efficient to enforce fairness than their traditional GAN counterparts.
REFERENCES
Martín Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein gan. CoRR, 2017.
Eustasio Del Barrio, Fabrice Gamboa, Paula Gordaliza, and Jean-Michel Loubes. Obtaining faimess using optimal transport theory. Proceedings of ICML. Conference, 2019a.
Eustasio Del Barrio, Paula Gordaliza, and Jean-Michel Loubes. A central limit theorem for I_p transportation cost with applications to fairness assessment in machine learning. Information and Inference journal of IMA, 2019b.
Philippe Besse, Eustasio Del Barrio, Paula Gordaliza, and Jean-Michel Loubes. Confidence intervals 88700066666... 2018.
for testing disparate impact in fair learning, arXiv preprint
Alex Beutel, Jilin Chen, Zhe Zhao, and Ed H Chi. Data decisions and theoretical implications when adversarially learning fair representations. arXiv preprint arXiv.1707.00075. 2017.
Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data, 5(2):1533166, 2017.
E. del Barrio, E. Giné, and C. Matrán. Central limit theorems for the Wasserstein distance between the empirical and the true distributions. Ann. Probab., pp. 1009-1071, 1999.
Eustasio del Barrio and Jean-Michel Loubes. Central limit theorems for empirical transportation cost in general dimension. Ann. Probab., 47(2):926-99) 9511, 03 2019.
Dobrić and J. Yukich. Asymptotics for transportation cost in high dimensions. Journal of Theoretical Probability, 8, 1995.
R.M. Dudley. Real Analysis and Probability. Cambridge University Press, 2002.
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel+. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science confer- ence, pp. 214-226. ACM, 2012.
Harrison Edwards and Amos Storkey. Censoring representations with adversary. In International Conference on Learning Representations, 2016.
Michael Feldman, Sorelle Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubra- manian. Certifying and removing disparate impact. In Proceedings of the 21th ACM SIGKDD International Conference Knowledge Discovery and Data Mining, pp. 259-268. ACM, 2015.
Sorelle Friedler, Carlos Scheidegger, Suresh Venkatasubramannian Sonam Choudhary, Evan P. Hamil- ton, and Derek Roth. A comparative study of fairness-enhancing interventions in machine learning. ArXiv e-prints.
Sorelle Friedler, Carlos Scheidegger, and Suresh Venkatasubramaniaa.. On the (im)possibility of fairness. arXiv e-prints, art. arXiv:1609.07236. Sep 2016.
Sorelle A Friedler, Carlos Scheidegger, Suresh Venkatasuurammanian, Sonam Choudhary, Evan P. Hamilton, and Derek Roth. A comparative study of fairness- enhanccng interventions in machine learning. In Proceedings of the Conference Fairness, Accountability, and Transparency, FAT* '19, pp. 329-338, New York, NY, USA, 2019, ACM.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario March, and Victor Lempitsky. aovesiiadversaaial training neural networks. Journal of Machine Learning Research, 17 2016.

---

Under review as a conference paper at ICLR 2020
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley. Sherjil Ozair, Aaron Courville, and Yoshua Bengio, Generative adversarial nets. Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence. and K. Q. Weinberger (eds.), Advances in Neural Information Process- ing Systems 27, pp. 2672-2680. Curran Associates, Inc., 2014.
Philipp Hacker and Emil Wiedemann. A continuous framework for fairness. arXiv e-prints, art. arXiv:1712.07924, Dec 2017.
Moritz Hardt, Eric Price, Nati Srebro, al. Equality of opportunity in supervised learning. In Advances in neural information processing systems, pp. 3315-3323, 2016.
Ray Jiang, Aldo Pacchiano, Tom Stepleton, Heinrich Jiang, and Silvia Chiappa. Wasserstein fair classification. In Uncertainty in AI, 2019.
James E Johndrow and Kristian Lum. An algorithm for removing sensitive information: application to race-independent recidivism prediction. arXiv preprint arXiv: 04957, 2017.
David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. Learning adversarially fair and transferable representations. In International Conference on Machine Learning, pp. 3381-3390, 2018.
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. In International Conference Learning Representations, 2018.
Gabriel Peyré and Marco Cuturi. Computational optimal transport. Foundations and Trends in Machine Learning, 1(5-6))355-607, 2019.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "why should trust you?": Explaining the predictions of any classifier. In Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16, pp. 1135-1144, New York, NY, USA, 2016.
ACM.
G.R. Shorack. Probability for Statisticians. Springer, 2000.
Cédric Villani. Optimal Transport: Old and New. Grundlehren der mathematischen Wissenschaften. Springer Berlin Heidelberg, 2008.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017.
Muhammad B. Zafar, Isabel Valera, Manuel G. Rodriguez, and Krishna P. Gummadi. Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In Proceedings of the 26th International Conference World Wide Web, pp. 1771-1180 International World Wide Web Conferences Steering Committee, 2017a.
Muhammad B. Zafar, Isabel Valera, Manuel G. Rodriguez, and Krishna P. Gummadi. Fairness Constraints: Mechanisms for Fair Classification. In Aarti Singh and Jerry Zhu (eds.), Proceed- ings of the 20th International Conference on Artificial Intelligence and Statistics, volume 54 of Proceedings of Machine Learning Research, pp. 962-970, 2017b.
Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning fair representations. In Proceedings the 30th International Conference Machine Learning, volume 28, pp. 325-333, 2013.
10

---

Under review as a conference paper at ICLR 2020
This is supplementary file for the paper Fairness with Wasserstein Adversarial Networks.
A SUPPLEMENTARY
A.1 CENTRAL LIMIT THEOREM FOR EMPIRICAL WASSERSTEIN IN DIMENSION
The behavior of the empirical transportation cost is key issue and heavily relies on the dimension of the problem. For general dimension we refer to Dobrić & Yukich (1995) for the asymptotic limit of transportation costs. The asymptotic behavior is provided for quadratic cost in general dimension in del Barrio & Loubes (2019). For the dimensional case, which corresponds to the situation encountered here since the constraint is imposed on score F(X) E R. it is possible to control more precisely for general costs the asymptotic behavior of the empirical Wasserstein's 's distance. The following theorem is an extension for the two sample case and W of the theorem found in Barrio al. 2019b). We provide it for sake of completeness.
Theorem Assume that F and G satisfies the following assumption
L FF(t)(1 - F(t))dt < O0
(6)
AF = G) = O where À is Lebesgue measure on R. Assume that atm - (0, Then
n n+ (FW) Gm) - Wi((F,)) t (1 -
with of(F. = -o Gi(,; G)dd (So (1; (I:F)) and
(()
(tt Ftt = (F-) sgn - (F(s)) <<t<l.
Proof sketch: This results follows the proofs Barrio al. 2019b). can be deduced from the
one sample case which states that
+ / uF(x)de,
G))
G)
where UF(x) ((F(z)) if F(x) G(z), (z -B(F(z)) if F(x) G(x). of of(x) and B is Brownian bridge on [0, 1]. In particular, then
(F(, G) (F,FG)) - N(0, (F,
Note that Wasserstein distance expressed through the quantile functions but when = 1, we have by simple arguments that
W(Pa, Q) (x)()) - G(x))de,
which allows to deal with the empirical transportation cost through the consideration of the process
(x) (T(FF( -(())), x E
Under the assumption
F(L((1 F()))dt <0
have that af converges weakly in L1 (R) to BF, centered Gaussian process R with covariance function
Cov (BF (x), Bd (v)) F(x Ay) F(x)
see Theorem 2.1 in del Barrio et al. 1999). By the Skorohod-Dudie Dudley-Wichura Wichura Theorem (see, e.g., Theorem 11.7.2 in Dudley 2002)), we can, therefore, consider versions of of and B such that |o BFI - 0 a.s.. Now,
Vn(Wi -V = (2:)dx, -
11

---

Under review as a conference paper at ICLR 2020
where un (z) (f(((()) G(z) a (z)/v - [F(x) - G(z)l). We introduce (x) vn((F(x)) G(z)+ + (3)/ (F(x) G(x)) and v(z) B (z) F(z) > G(x), v(z)
-B" if F(x) < ((z) and v(x) = (f G(z). We note that [u, Un (z) S
la (x) - Bf (1)), which implies that
1 un (2)x)d
S un (x))dd s $1a - B 11, +0
with probability one.
Now, if F(x) ((x) then (x) will eventually equal B (x)),, while F(x) G(z) then
(x) = = B (x) for large enough n. Hence, (2) =(xx) pointwise. On the other hand,
|0, (x)] S (B²(x)) (x))l-
This shows that we can apply dominated convergence to conclude that
Je un (1)()) / o(x)dz.
(8)
Combining (7) and 8 we see that (T(W G) WI(F,)), + SR v(2)dz. To conclude we note that 8 has the same distribution (-F(-) with B standard Brownian bridge on [0, 1]. Normality and the expression for the variance when A(F = G) = 0 follow from the fact that, that case,
[ vlz)do / ((F(z))(()ddz
with h(x)== I[F))) - - < G(x)). This last integral is a centered Gaussian r.v. with variance
{ (F(xAy) F(+))
L: H²(()dt - -O. H(()a))
where H(t) = SE 1(t)
114) hss)ds (the last equality follows, from instance, from Proposition 7.4.2,
117 in Shorack 22000). Finally, we note that F(x) > G(x) if and only G- > x and also that = G (F())) if and only if G(x) 2 F(x) and G(y) < F(x) for every < But then Gzz) = F(x) unless G is not continuous at x. But this can happen at most for a countable collection of z. This means that [(F(x)) > G(x)) [[G (F(x)) > x) and, under the assumption X(F = G) = 0, that [(F(x) < G(z)) = I(G (F(x)) < x) for a.e x. This completes the proof.
A.2 ASYMPTOTICS IN THE MULTIVARIATE SETTING
Proposition 2 Given Xn Li and Xi...., ,,,' L2 be two tid sample Rd of com- pactly supported distributions Li and L2. Then we have
n'
Σ f(X)) SEFi
sup
1 + f(X)
W(C1, C2).
W
Proof sketch: Let L1,n and L2,n' be the two empirical measures associated to our two samples. From the law of large numbers, Li,n - L1 and L2,n' 3 L2 where the limit understood as n
convergence in distribution of probability measures (or weak convergence of probability distributions). The result follows combining the equivalence between (2) and 33) with the fact that W metrizes this notion of convergence [Villani 2008 Corollary 6.9).
B ALGORITHM
C ARCHITECTURES
C.I GENERAL PROCESS
We use RELU for all the hidden layers, and implement neither dropout nor batch normalization (except for the celebA classifier which use batch normalization).
12

---

Under review as a conference paper at ICLR 2020
Algorithm 2 Type 2 Fair learning algorithm
Require: a, the learning rate, À the fairness constraint, m the batch size, nb_ ab_iter, the number of iteration, the the number of iterations for the Wasserstein estimators.
nb for nb_ cdbepochhee do
2: for j in Tay do
3: sample idd { ~ (X). and {-)) (X)
4: update Ag by gradient ascent V Ao - (Ao(T(z))-AAo((3(
5: Scale A as in Miyato et al. (2018)
6: repeat for A1 and the corresponding sampling
7: end for
8: sample iid Dr = {d, = (I,, 8i y)) ~ L(D)
9:
update T and F an by gradient descent:
m
Σ V FET m
(FT((
Σ
Ao(T(zi))
Σ Ao(T(z;))
D& Di
ID DDD d.D.
d,D
Σ
AT(((x))
Σ AT(((z)) (9)
1i Dil d
1Dx n HHH dGED)
end for
C.2 TYPE ARCHITECTURES
We
present the architecture of the network F for the unfairelf, Type and
1
type 2 architectures used for the Adult and bank
dataset
ID
Layer Kernel Stride Activation Normalization Output
Inputo Input
None None
nb variables
densel dense
RELU None
128 1
dense2
dense
RELU None
64 x
dense3
dense
RELU None
32 x
OutputF
Dense
sigmoid None
We
present the architecture of the network F for the unfair- architectures used for the CelebA
elf,
Type and type 2
dataset
Layer ID
Kernl Stride Activation Normalization Output
InputO
Input
None None RELU
64 x 64 x 3
Conv1
Convolutional Convolutional max pooling
3 3
True
64 x 64 x 32
Cony2
3 3
RELU
True
64 x 64 x 32
poolo
None
32 x 32 x 32
Conv3
Convolutional
3
RELU
True
32 x 32 64
Conv4
Convolutional
3 3
RELU
True
32 x 32 x 64
Conv5
Convolutional max pooling
3x3
RELU
True
64
pooll
None
16 x 16 x 64
Conv6
Convolutional
3x3
RELU
True
x 16 128
Conv7
Convolutional
3x3
RELU
True
16 x 16 x 128
Conv8
Convolutional max pooling
3x3
RELU
True
316 x 16 x 128
pool2
None
8 x 8 X 128
Conv9
Convolutional
3x3
RELU
True
8 8 x 256
Conv 10
Convolutional
3x3
RELU
True
8 8 x 256
Conv 11
Convolutional
3x3
RELU
True
8 8 x 256
pool3
pooling flatten
None
4 4 x 256
Flat O
None
4096 x 1
densel
dense
RELU
None
32 x 1
OutputF
Dense
sigmoid
None
All the wasserstein and GAN constraint networks use the following for the Type architectures
13

---

Under review as a conference paper at ICLR 2020
ID
Layer
Kernel
Stride
Activation
Normalization
Output
InputO
Input
None
None
densel
dense
RELU
spectral for wass 64 x
dense2
dense
RELU
spectral for wass
32 x
OutputA
Dense
linear (signoid for GAN)
spectral for wass
The constraint are always added on the output layer of the F network.
C.3
FAIR AUTO ENCODER
We present the architecture the network T for the fair auto-encoder
ID
Layer
Kernel
Stride
Activation
Normalization
Output
InputO
Input
None
None
28
3
ConvO
Convolutional max pooling
3 x
RELU
None
28 x 28
64
poolo
None
14 14
32
Conv1 I
Convolutional
3 3
RELU RELU
None
14 x 14 128
Conv2
Convolutional
3
None
14 x 14 256
pool2
max pooling
None
7 7 x 256
Conv3
Convolutional
3
RELU
None
7 X X 256
up0
upsampling
None
14 14 256
Conv4
Convolutional
3
RELU
None
14 X 14 X 128
up0
upsampling
None
28 28 128
Conv5
Convolutional
3x3
RELU
None
28 x 28 x 128
Conv6
Convolutional
3x3
RELU
None
28 28 128
OutpurT
Convolutional
3 x
RELU
None
28 x 28 3
We present the architecture of the network A for the fair auto-encoder
ID
Layer
Kernel
Stride
Activation
Normalization
Output
InputO
Input
None RELU
None
28 28
ConvO
Convolutional
x 3
spectral
28 x 28 x 32
poolo
max pooling
spectral
14 14 x 32
Conv1
Convolutional
3 x
RELU RELU
spectral
14 14 x 64
Conv2
Convolutional max pooling
3
spectral
14 14 128
pool2
spectral
x x 128
Flat O
flatten
spectral
6272
dense2
dense
RELU linear
spectral
32 X 1
OutputA
Dense
spectral
14


Reviews:
[{"review_id": "rkgtvbuH9S", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a variant of adversarial learning to achieve some of the popular group fairness definitions. The main novelty is the idea of minimizing Wasserstein distance between the conditional distributions of classifier predictions given different values of the protected attribute.\n\nMy main concern is the approximation of a simple 1d Wasserstein distance with a neural network. Wasserstein distance between two discrete distributions in 1d can be computed in closed form (simple function of order statistics). That is, eq. (1) is simple to evaluate for two empirical distributions. There is no need to use a neural network for approximation, and even if authors choose to do so, some discussion on how well it approximates actual Wasserstein distance is needed. I think the proposed algorithm could be more interesting if authors can work out the optimization problem with the actual Wasserstein distance.\n\nOn the theoretical/motivation side, it is not enough to say that demographic parity is achieved when the corresponding Wasserstein distance is 0. What is needed is that demographic parity difference is bounded from above by the corresponding Wasserstein distance (I don't know if it is true or not, but would like to know). Then minimizing Wasserstein distance to achieve demographic parity could be justified.\n\nFinally, the paper is quite poorly written. The description of fairness in the introduction is very vague. Authors essentially describe demographic parity as fairness, while it is simply one of the several definitions of group fairness. There is also individual fairness (the paper by Dwork et al. is cited, but not properly discussed) and prior work emphasizing certain deficiencies of group fairness [1] along with several recent papers studying individual fairness [2,3], some also utilizing Wasserstein distance [4].\nAuthors also provided incorrect definition of disparate impact. Equation in the bottom of page 2 corresponds to statistical parity difference, while disparate impact is the ratio.\n\"Equality of opportunity\" on the top of page 3 seems to be a typo\n\"the mathematical properties of the disparate impact measure are not favorable, in particular it lacks robustness and smoothness features which would be necessary to blend algorithmic practice and mathematical theory\" - I don't think this claim makes sense. There are many prior works studying disparate impact and proposing algorithms to achieve it, e.g. the cited work of Feldman et al. Authors should be more specific regarding what mathematical properties they consider not favorable.\n\nThere are a lot of typos and grammatical mistakes, e.g.\nin the 1st paragraph of section 2.2, the sentence \"Hence the aim in this case is to” is unfinished.\nin the 1st paragraph of section 3, the first sentence seems to be unfinished.\n\n[1] Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016). Inherent trade-offs in the fair determination of risk scores.\n[2] Kearns, M., Roth, A., & Sharifi-Malvajerdi, S. (2019). Average Individual Fairness: Algorithms, Generalization and Experiments.\n[3] Jung, C., Kearns, M., Neel, S., Roth, A., Stapleton, L., & Wu, Z. S. (2019). Eliciting and Enforcing Subjective Individual Fairness.\n[4] Yurochkin, M., Bower, A., & Sun, Y. (2019). Learning fair predictors with Sensitive Subspace Robustness.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "HygO3_32YH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the authors proposed a fairness-aware learning method.\nIn particular, the authors considered two kinds of fairness problem and designed two regularizers accordingly.\nEssentially, both of these two strategies learn classifiers and calibrate the distributions conditioned on protected variables jointly. \nThe calibration of the distribution is achieved in the framework of optimal transport.\n\nThis work is a natural extension of the optimal transport-based method shown in (Barrio et al, 2019a,b). The main differences include 1) instead of calibrating distributions after learning classifiers, the proposed method achieves calibration and learning jointly, replacing the primal Wasserstein barycenter problem with the dual form of Wasserstein distance (Arjovsky et al. 2017); 2) the proposed method considers two types of fairness problem. \n\nCompared with vanilla GAN, the potential advantage of WGAN on distribution matching is well-known. It seems unfair that the authors compared the vanilla GAN-based regularizer with the proposed WGAN-based regularizer just on EMD because EMD corresponds to the proposed regularizer directly. In Table 1, although the DI of vanilla GAN is higher than that of WGAN, its ACC is also higher than that of WGAN as well. In Figure 5 (a, b), if we set lambda=0.6 for WGAN and lambda=1 for vanilla GAN, both of them can achieve ~0.838 ACC and ~0.100 DI. In Figure 5(c), what do the points represent? Why not use DI as the x-axis? Because of the issues in experiments, it is hard to evaluate the improvements of the proposed method.\n\nAdditionally, the proposed method always causes the degradation of ACC when improving DI. However, the method in (Barrio et al, 2019a) just applies a Wasserstein barycenter-based post-processing but can suppress the degradation on ACC greatly. Could the authors discuss the differences and the advantages of the proposed method in detail?  Could the authors consider more recent work as their baselines?\n\nIn summary, the method makes sense, but its novelty is limited and the improvements are incremental.\n\nMinors:\nPage 6, Line 3: Figure 3 —> Figure 2.\nI suggest swapping Figure 2 and Figure 3.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "BJerxTNAYr", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose a method for adding an approximate disparate impact loss to a classification objective, and show that optimizing a classifier for this loss leads to \"fairer\" predictions with little or no accuracy loss.\n\nThe authors first formulate two notions of fairness in terms of earth mover distance between the distribution of scores conditioned on either value of a protected variable. They then show that the dual formulation of the earth mover distance can be approximated (specifically, lower-bounded) by optimizing the parameters of a neural network under spectral norm constraints. This leads to a min-max global optimization scheme to learn a fair classifier.\n\nStrengths: The proposed method does better on the considered fairness metric than a GAN model with similar accuracy.\n\nWeaknesses: The paper is difficult to read, glosses over some important details, and contains some inaccuracies.\n\n-- Clarity: \n--- The authors need to better describe the assumptions (or lack thereof) made on the joint distribution of X, S, and Y. \n--- Measures such as the quantiles or probability laws need to be formally defined before they are used in definitions. \n--- The \\mathcal{L} notation is overloaded (it is used for probability laws, conditional and unconditional, as well as marginals, with \\mathcal{L}_1 referring to both!), leading to potential confusion.\n--- The domain of X and Y in equation (2) is not defined anywhere, neither is the distance.\n--- Similar lack of consistency with the use of F / \\mathcal{F} / \\hat{f}, without any explicit parameterization\n--- Figure 2 needs to be in Section 4, and Table 1 needs to be trimmed to size\n\n-- Overlooked problems:\n--- In the dual formulation, the optimization is done over a sub-set of Lipschitz function, hence approximation of \\mathcal{W} is a lower bound at every step. Minimizing a lower bound on a loss can be justified, but requires more discussion\n--- The trade-off inherent in the choice of n_w in algorithm 1 needs to be further discussed, especially in the case of large datasets where a full epoch of SGD in the inner loop of the optimization process is impractical\n\n-- Inaccuracies:\nThe graphical models in Figures 1 and 2 and conditional independences written in the text are not consistent:\n--- In Figure 1,  X is NOT independent of S given Y (neither is Y*) (see: V structures in a directed graphical model)\n--- In Figure 2, X* is independent of S regardless of conditioning on Y\n\nConsidering all of the above issues, the paper is not currently ready for publication", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents an approach to enforce statistical fairness notions using adversarial networks. The reviewers point out several issues of the paper, including 1) their approach does not provably enforce criteria such as demographic parity, 2) lack of novelty and 3) poor presentation.
Decision: Reject