Fair Mixup: Fairness via Interpolation
Authors: Ching-Yao Chuang
Youssef Mroueh
Venue: ICLR 2021 Poster (2021)
Keywords: fairness
data augmentation

Abstract: Training classifiers under fairness constraints such as group fairness, regularizes the disparities of predictions between the groups. Nevertheless, even though the constraints are satisfied during training, they might not generalize at evaluation time. To improve the generalizability of fair classifiers, we propose fair mixup, a new data augmentation strategy for imposing the fairness constraint. In particular, we show that fairness can be achieved by regularizing the models on paths of interpolated samples  between the groups. We use mixup, a powerful data augmentation strategy  to generate these interpolates. We analyze fair mixup and empirically show that it ensures a better generalization for both accuracy and fairness measurement in tabular, vision, and language benchmarks.

Published as a conference paper at ICLR 2021
FAIR MIXUP: FAIRNESS VIA INTERPOLATION
Ching-Yao Chuang" CSAIL, MIT
Youssef Mroueh IBM Research Al mr mrouehlus i.mm com
cychuangemit..cm t. edu
ABSTRACT
Training classifiers under fairness constraints such as group fairesss, regularizes the disparities of predictions between the groups. Nevertheless, even though the constraints are satisfied during training, they might not generalize at evaluation time. To improve the generalizability of fair classifiers, we propose fair mixup, new data augmentation strategy for imposing the faimess constraint. In partic- ular, we show that fairness can be achieved by regularizing the models on paths of interpolated samples between the groups. We use mixup, a powerful data aug- mentation strategy to generate these interpolates. We analyze fair mixup and em- pirically show that ensures a better generalization for both accuracy and fairness measurement in tabular, vision, and language benchmarks. The code is available at https: pectts://githhn /github.tttt.t com/ chingyaoc/ fair mair-mixup.
INTRODUCTION
Fairness has increasingly received attention in machine learning, with the aim of mitigating unjus- tified bias in learned models. Various statistical metrics were proposed to measure the disparities of model outputs and performance when conditioned on sensitive attributes such as gender or race. Equipped with these metrics, can formulate constrained optimization problems impose fair- ness as a constraint. Nevertheless, these constraints do not necessarily generalize since they are data-dependent, i.e they are estimated from finite samples. In particular, models that minimize the disparities on training sets do not necessarily achieve the same faimess metric on testing sets (Cotter al., 2019). Conventionally, regularization is required improve the generalization ability of model (Zhang et al., 2016). On one hand, explicit regularization such as weight decay and dropout constrain the model capacity. On the other hand, implicit regularization such as data augmentation enlarge the support of the training distribution via prior knowledge (Hernández- Garcíaa & König, 2018).
In this work, we propose data augmentation strategy for optimizing group fairness constraints such as demographic parity (DP) and equalized odds (EO) (Barocas al., 2019). Given two sensitive groups such as male and female, instead of directly restricting the disparity, we propose to regularize the model on interpolated distributions between them. Those augmented distributions form a path connecting the two sensitive groups. Figure/ 1 provides an illustrative example of the idea. The path simulates how the distribution transitions from one group to another via interpolation. Ideally, if the model is invariant to the sensitive attribute, the expected prediction of the model along the path should have a smooth behavior. Therefore, we propose a regularization that favors smooth transitions along the path, which provides stronger prior on the model class.
We adopt mixup (Zhang et al., 2018b), a powerful data augmentation strategy, to construct the in- terpolated samples. Owing to mixup's simple form, the smoothness regularization we introduce has a closed form expression that can be easily optimized. One disadvantage of mixup is that the inter- polated samples might not lie on the natural data manifold. Verma et al. (2019) propose Manifold Mixup, which generate the mixup samples in a latent space. Previous works (Bojanowski et al., 2018; Berthelot et al., 2018) have shown that interpolations between pair of latent features corre- spond to semantically meaningful, smooth interpolation in the input space. By constructing the path in the latent space, we can better capture the semantic changes while traveling between the sensitive groups and hence result in a better fairness regularizer that we coin fair mixup. Empirically, fair
Work done during an intersship IBM Research AT

---

Published as a conference paper at ICLR 2021
Unregularized Fair Mixup
Expected Parity
Zn
Pb
Pa
ADP
Female
Path
Male
ADP
Pg
Pi
(a) Interpolations between Groups
(b) Smoothness leads to Faireess
Figure 1: (a) Visualization of the path constructed via mixup interpolations between groups that have distribution Po and Pt. respectively. (b) Fair mixup penalizes the changes in model's expected prediction with respect to the interpolated distributions. The regularized model (blue curve) has smaller slopes comparing to the unregularized one (orange curve) along the path from Po to Pi, which eventually leads to smaller demographic parity ADP.
mixup improves the generalizability for both DP and EO on tabular, computer- vision, and natural language benchmarks. Theoretically, we prove for particular case that fair mixup corresponds to Mahalanobis metric in the feature space in which perform the classification. This metric ensures group faireess the model, and involves the Jacobian of the feature map as we travel along the path.
In short, this work makes the following contributions:
We develop fair mixup, a data augmentation strategy that improves the generalization of group faimess metrics;
We provide theoretical analysis to deepen understanding of the proposed method;
We evaluate approach via experiments on tabular, vision, and language benchmarks;
2 RELATED WORK
Machine Learning Fairness To mitigate unjustified bias in machine learning systems, various fairnsss definitions have been proposed. The definitions can usually be classified into individual fairness or group fairness. system that is individually fair will treat similar users similarly, where the similarity between individuals can be obtained via prior knowledge al., 2012; Yurochkin et al., 2019). Group fairness metrics measure the statistical parity between subgroups defined by the sensitive attributes such gender race (Zemel et al., 2013; Louizos et al., 2015; Hardt et al., 2016). While fairness can be achieved via pre- or post-processing. optimiz- ing fair metrics training time lead to the highest utility (Barocas et al., 2019). For instance, Woodworth et al. (2017) impose independence via regularizing the covariance between predictions and sensitive attributes. Zafar al. (2017) regularize decision boundaries of convex margin-based classifier to minimize the disparaty between groups. Zhang et al. (2018a) mitigate the bias via minimizing adversary's ability predict sensitive attributes from predictions.
metric learning (Dwork
Nevertheless, these constraints are data-dependent, even though the constraints are satisfied during training. the model may behave differently at evaluation time. Agarwal al. (2018) analyze the generalization eтгor of fair classifiers obtained via two-player games. To improve the generalizabil- ity, Cotter et (2019) inherit the two-player setting while training each player on two separated datasets. In spite of the analytical solutions and theoretical guarantees, game-theone theoretic approaches could be hard to scale for complex model classes. In contrast, our proposed fair mixup, is general data augmentation strategy for optimizing the faimeess constraints, which is easily compatible with any dataset modality or model class.
Data Augmentation and Regularization Data augmentation expands the training data with ex- amples generated via prior knowledge, which can be seen as implicit regularization (Zhang al., 2016; Hernández García & König, 2018) where the prior is specified as virtual examples. Zhang et al. (2018b) proposes mixup, which generate augmented samples via convex combinations of pairs of examples. In particular, given two examples 2;, E Rd where z could include both input and label, mixup constructs virtual samples (z + t)2, for [0, 1]. State-of-the-ar results
2

---

Published as a conference paper at ICLR 2021
obtained via training mixup samples in different modalities. Verma al. (2019) introduces manifold mixup and shows that performing mixup in a latent space further improves the generaliza- tion. While previous works focus general learning scenarios, show that regularizing models on mixup samples can lead to group fairness and improve generalization.
3 GROUP FAIRNESS
Without loss of generality, we consider the standard fair binary classification setup where obtain inputs X C Id labels E y = {0, 1]. sensitive attribute A E {0, 1}, and prediction score Y E [0. 1] from model f : Rd - [0, 1]. We will focus on demographic parity (DP) and equal- ized odds (EO) in this work, while approach also encompasses other fairness metrics (detailed discussion in section 5). DP requires the predictions Y be independent of the sensitive attribute A, that is, P(Ý
0) P 1). However, DP ignores the possible correlations between Y and A and could rule out the perfect predictor if Y Ví A. EO overcomes the limit of DP by conditioning
the label Y. In particular, EO requires Y and A to be conditionally independent with respect to Y. that is, P( Y difficulty of optimizing the independency constraints, Madras et al. (2018) propose the following relaxed metrics:
P(Y 0, Y y) for {0, 10, Given the
ADP(f) = Pa Ex
AEO(F) = E (0)1)
Be f(x) - Ea f(x)
where we define P. = P(IIA = a) and P = P(IA= a, y). a,y E 10. We denote the joint distribution of X and Y by P. Similar metrics have also been used in Agarwal et al. (2018), Wei et al. (2019), and Taskesen et al. (2020). One can formulate a penalized optimization problem regularize the fairness measurement, for instance,
(Gap Regularization): min ((e.))
AADP(S,
where l is the classification loss. In spite of its simplicity, experiments show that small training values of ADP do not necessarily generalize well at evaluation time (See section 6). To improve the generalizability, introduce data augmentation strategy via dynamic form of group fairness metrics.
4 DYNAMIC FORMULATION OF FAIRNESS: PATHS BETWEEN GROUPS
For simplicity, will first consider ADP as the fairness metric, and extend our development to AEO section 5. ADP provides a static measurement by quantifying the expected difference at Po and Pt. In contrast, one can consider a dynamic metric that measures the change of ý while transitioning gradually from Po to P1. To convert from the static to the dynamic formulations, start with a simple Lemma that bridges two groups with an interpolator T T(ro, 31, t), which generates interpolated samples between 2a and 21 based step t.
Lemma 1. Let T X² [0, 1]
X be a function continuously differentiable w.t.t t such that D1, 0) = 2g and T(xo, 21. 1) = Tz. For any differentiable function f, we have
T(*o, Fz,
ADP(F) S.' dt [
(dd =: =I[[' dt (t
(2)
interpolation
where we define py (1) = Pa [TT the expected output of f with respect to T( (1,0,,1, 1).
Figure provides an illustrative example of the idea. Lemma relaxes the binary sensitive attribute into a continuous variable t e [0, 1], where μ captures the behavior of f while traveling from group to group 1 along the path constructed with the interpolator T. In particular, given two examples zo and æ1 drawn from each group, T generates interpolated samples that change smoothly with respect to
3

---

Published as a conference paper at ICLR 2021
For instance, given two racial backgrounds in the dataset, Hy simulates how the prediction of changes while the data of one group smoothly transforms to another. We can then detect whether there are "unfair" changes in j4f along the path. The dynamic formulation allows to
di 7dr((0
(1
1
(y((0)
(1y(1)
-y(0
ADP
measure the sensitivity of f with respect to a relaxed con- tinuous sensitive attribute via the derivative (t))).. Ideally, if f is invariant to the sensitive attribute. d (t) should be small along the path from tantly, a small ADP does not imply de (t)) small for t E [0, 1] since the derivative could fluctuate it can be seen in Figure 2.
0 to 1. Impor-
0.0
0.2
0.4
0.6
0.8
1.0
d
Figure 2: The expected output (y gradually changes t - 1. Even when ADP is small, l [t)l could still be large along the path.
4.1 SMOOTHNESS REGULARIZATION
To make / invariant to I, we propose to regularize the derivative along the path:
Rf) S.' dt (r((t dt.
(Smoothness Regularizer):
(3)
Interestingly, Rr(()) is the are length of the curve defined by (1)(1) for E [0, Now, we can interpret the problem from geometric point of view. The interpolator T defines a curve (1y(1)
[0, 1] + I. and ADP(f) = (0) He (1) is the Euclidean distance between points t = 0 and 1. AfP(f) fails capture the behavior f while transitioning from Po to Pt. In contrast, regularizing the are length Rf(()) favors a smooth transition from i = 0 to 1, which constrains the fluctuation of the function as the sensitive attributes change. By Jensen's inequality, ADP(f) S (f) for any f. which further justifies the validity of regularizing ADP(f) with (f)
5 FAIR MIXUP: REGULARIZING MIXUP PATHS
remains determine the interpolator T. good interpolater shall (1) generate meaningful inter- polations, and (2) the derivative of u (-) with respect to i should be easy to compute. In this section, we show that mixup (Zhang et al., 2018b), a powerful data augmentation strategy, is itself a valid interpolator that satisfies both criterions.
Input Mixup We first adopt the standard mixup (Zhang et al., 2018b) by setting the interpolator the linear interpolation in input space: (1, two (1 =t)11 It can be verified that Tmixup satisfies the interpolator criterion defined in Lemma 1. The resulting smoothness regularizer has the following closed form expression':
RDP uiisup (f) - - (
(Vaf(tro + (1 (al-t)an t)=1), 2o x1) (Po( dt.
The regularizer can be easily optimized by computing the Jacobian of f on mixup samples. Jacobian regularization is a common approach to regularize neural networks (Drucker & LeCun, 1992). For instance, regularizing the norm of the Jacobian can improve adversarial robustness (Chan et al., 2019; Hoffman et al., 2019). Here, regularize the expected inner product between the Jacobian on mixup samples and the difference 20 - 21
Manifold Mixup One disadvantage of input mixup is that the curve is defined with mixup samples,
which might not lie on the natural data manifold. Verma et al. (2019) propose Manifold Mixup, which generate the mixup samples in the latent space Z. In particular, manifold mixup assumes compositional hypothesis f fog where X - 2 is the feature encoder and the predictor f 2 - y takes the encoded feature to perform prediction. Similarly, we can establish the equivalence between
exchange the derivative and integral via the Dominated Convergence Theorem
4

---

Published as a conference paper at ICLR 2021
ADP and manifold mixup:
ADP(S g) = I: d di
[[lg(x))+
Manifold Mixup
which results the following smoothness regularizer:
RDP mixup (fo gg = = -fo // (V= (V=f(tg(2))) + ()g(21)). - g(x1)) dPo(zo) (x1) dt.
Previous works (Bojanowski et al., 2018; Berthelot et al., 2018) have showed that interpolations between a pair of latent features correspond semantically meaningful, smooth interpolations in input space. By constructing curve in the latent space, we can better capture the semantic changes while traveling from Po to Pt.
Extensions and Implementation The derivations presented far, can be easily extended to Equalized Odds (EO). In particular, Lemma can extended to AEO by interpolating Po and P for y 10, AEO Σ
Joa d |ITT/SO, x( (20)) dt
BE (0.1)
The corresponding mixup regularizers can be obtained similarly by substituting Po and P1 in Rauxup and Rm-mixup with Po and P.
mixup ( = Σ [V (1 ()x1))))..))))))))) zo xι) dPP (20 )dP (a1) (mi)
[0,1)
Our formulation also encompasses other fairness metrics that quantify the expected difference be- tween groups. This includes group fairness metrics such as accuracy equality which compares the mistreatment rate between groups (Berk et al., 2018). Similar to equation (I), we formulate penal- ized optimization problem to enforce fairness via fair mixup:
(Fair Mixup): min ((ax)))(((((z)) y)] + R (f)-
(4)
Implementation-wise follow Zhang et al. (2018h) where only is sampled per batch to perform mixup. This strategy works well in practice and reduce the computational requirements.
5.1 THEORETICAL ANALYSIS
To gain deeper insight, we analyze the optimal solution fair mixup in a simple case. In particular, we consider the classification loss L( (ff()),)) y) = =x)((x) and the following hypothesis class:
(f](()) (v, ((=)) 12Th X -
Define my = Ex-F, ((x), the label conditional mean embeddings, and ma = Eg-r, ((x) and my Ez-z, (x)) the group mean embeddings. We then define the expected difference 54 m.--m m__ and 80.1 = my m To derive an interpretable solution, we will consider the L.2 variants of the penalized optimization problem. The following proposition gives the analytical solution when we regularize the model with ADP.
Proposition 2. (Gap Regularization) Consider the following minimization problem
À ADP(f)² + A2 l1f
min
JEH
2
2
For a fixed embedding 4 the optimal solution f corresponds to v' given by the following closed form:
01 -0=
1
>a
projso, (8+)
where proj is the soft projection defined proje
5

---

Published as a conference paper at ICLR 2021
The solution u² can be interpreted as the projection of the label discriminating direction d= on the subspace that orthogonal to the group discriminating direction 60,1. By projecting to this orthogonal subspace, directions when performing prediction. Interestingly, the projection trick has been used in Zhang al. 2a), where they subtract the gradient of the model parameters in each update step with its projection on unfair directions. We then prove the optimal solution of fair mixup with the same setup as above. Similarly, we introduce an L2 variant of the fair mixup regularizer defined as follows:
prevent the model from using group specifie directions, that are unfair
Reissppp (5) = [n- (Vaf(tro- + (1 - t)æ1). xo - 21) dPP(aa)dP)) (x1) dt, 10=00
where we consider the squared absolute value of the derivative within the integral, in order to get closed form solution.
Proposition 3. (Fair Mixup) Consider the following minimization problem
A2 min (Z (( + À1 R (1) 2 11 SER
Let P (otto + (1 t)x1)] be the dependent mean embedding, and rine its derivative with respect to Let be a positive-semi definite matrix defined as follows: D So nh, 8 rilll Given an embedding 4 the optimal solution v² has the the following form:
v" (A1 A22m) 8+
Hence the optimal fair mixup classifier can be finally written as
f(x) = (61, (A, + xz1m))1((x)))
which means that fair mixup changes the geometry of the decision boundary via new dot product in the feature space that ensures group fairness, instead of simply projecting on the subspace orthogonal to single direction in gap regularization. This dot product leads to Mahalanobis distance in the feature space that is defined via the covariance of time derivatives of mean embeddings of intermediate densities between the groups. To understand this better, given points zo in group and 21 in group 1, by the mean value theorem, there exists Ie such that:
(zo) = f(1) + f zo - =1) = f(11) (6+, (N1 + X J. x1))
Note that D) provides the correct average conditioning for (xx ( x1), this can be seen from the expression of th (D is covariance of J& (xx x1)). This conditioned Jacobian ensures that the function does not fluctuate lot between the groups, which matches our motivation.
EXPERIMENTS
We now examine fair mixup with binary classification tasks on tabular benchmarks (Adult), visual recognition (CelebA), and language dataset (Toxicity). For evaluation, we show the trade-offs be- tween average precision (AP) and fairness metrics (ADP/AEO) by varying the hyper-parameter À in the objective. We evaluate both AP and fairness metrics testing set to assess the generalizability of learned models. For a fair comparison, we will compare fair mixup with baselines that optimize the fairness constraint training time. In particular, risk minimization (ERM) that trains the model without regularization,
compare method with (a) empirical (b) gap regularization, which directly regularizes the model as given in Equation (I), and (c) adversarial debiasing (Zhang al., 201 8a) introduced in section 2. Details about the baselines and experimental setups for each dataset can be found in appendix.
6.1 ADULT
UCI Adult dataset (Dua & Graff, 2017) contains information about over 40,000 individuals from the 1994 US Census. The task is to predict whether the income of a person is greater than S50k given attributes about the person. We consider gender the sensitive attribute to measure the fairness the algorithms. The models are two-layer ReLU networks with hidden size 200. We only evaluate input mixup for Adult dataset the network is not deep enough to produce meaningful

---

Published as a conference paper at ICLR 2021
0.75
0.77
0.05
0100
0.76
8 0.74
Expeste
-0.55
0.72
FRM
13EM
0.75
GapRes
-0.15
GiyRog
Mixap
Mixup
0.05
0.10
0.15
0.08
0.10
0.15
0.0
0.2
0.4
0.0
1.0
DP
2
(a) Demographic Parity
(b) Equalized Odds
(c) Path Visualization
Adult Dataset. (a.b) The tradeoff between AP and ADP/SEO. (c) Visualization of the mixup path for models that regularize ADP with different algorithms. We plot the calibrated curve u'y()) :- pt(() (0) for a better visualization. In this case, 1í (0) = 0 and [n)( = ADP for all the calibrated curves p.y. Therefore, we can compare the ADP of each method with the absolute value of the last points (f = 1). The flatness of the path is highly correlated with the ADP.
Figure 3:
0.900
0.98
0.55
0.875
0.830
0.96
0.30
0.800
Gapting
0.4
0.775
Manifold Minop
0.1
8.0
02
0.05
DP
A
A DP
0.982
0.85
0.80
0.976
0.20
0.974
0.75
0.972
(a) Attractive
(b) Wavy Hair
(c) Smiling
Figure 4: CelebA Dataset. The tradeoff between AP and ADP/AEO shown in the first/second row for each task. Manifold mixup consistently outperforms the baseline across tasks.
latent representations. We retrain each model 10 times and report the mean accuracy and fairness measurement. In each trial, the dataset randomly randomly split into a training, validation, and testing set with partition 60%, 20%, and 20%, respectively. The models then selected via the performance on the validation set.
that fair mixup consistently achieves a better tradeoff compared to the baselines. We then show the tradeoff between AP and AEO figure 3 (b). For this metric, fair mixup performs slightly better than directly regularizing the EO gap. Interestingly, fair mixup even achieves a better AP compared to ERM, indicating that mixup regularization not only improves the generalization of fairness constraints but also overall accuracy. To understand the effect of fair mixup, we visualize the expected output /4 along the path for each method (i.e μy as function of t). For fair comparison, we select the models that have similar AP for the visualization. As can see in figure 3 (c), the flatness of the path is highly correlated to ADP. Traininig without any regularization leads to the largest derivative along the path, which eventually leads to large ADP. All the fairnes-- algorithms regularize the slope to some extent, nevertheless, fair mixup achieves the shortest are length and hence leads to the smallest ADP.
Figures 3 (a) shows the tradeoff between AP and ADP. We

---

Published as a conference paper at ICLR 2021
FRM
0.3
0.3
Figure 5: Visualization of calibrated paths on at- tractive classification task for ADP regularized mod- els. The flatness of both in- put and latent path plays an important role in regulariz- ing ADP.
Gaples
Mixi
02
M-Mixup
0.
Expected
0.1
0.1
0.
0.
0.0
0.2
0.4
0..6
0.8
1.0
0.0
0.2
0.4
0.5
08
1.
(a) Input Path
(b) Latent Path
6.2
CELEBA
Next, we show that fair mixup generalizes well to high-dimensional tasks with the CelebA face attributes dataset (Liu et al.). CelebA contains over 200,000 images of celebrity faces, where each image is associated with 40 human-labelee binary attributes including gender. Among the attributes, we select attractive, smile, and wavy hair and use them to form three binary classification tasks while treating gender as the sensitive attribute². The reason choose these three attributes is that there exists in all these tasks, a sensitive group that has more positive samples than the other one. For each task, train ResNet-18 18 (He al., 2016) along with two hidden layers for final prediction. To implement manifold fair mixup, we interpolate the representations before the average pooling layer.
The first row in figure shows the tradeoff between AP and ADP for each task. Again, fair mixup consistently outperforms the baselines by a large margin. We also observe that manifold mixup further boosts the performance for all the tasks. The tradcoff's between AP and AEO are shown in the second row of figure 4. Again, both input mixup and manifold mixup yields well generalizing classifiers. To gain further insights, we plot the path in both input space and latent space figure 5 (a) and (b) for the "attractive" attribute classification task. Fair mixup leads to a smoother path in both cases. Without mixup augmentation, gap regularization and adversarial debiasing present similar paths and both have larger ADP. We also observe that the expected output μz in the latent path is almost linear with respect to the continuous sensitive attribute manifold mixup being the curve with the smallest slope and hence smallest ADP.
6.3 TOXICITY CLASSIFICATION
Lastly, consider comment toxicity classification with Jigsaw toxic comment dataset (Jigsaw, 2018). The data was initially released by Civil Comments platform, which was then extended to a public Kaggle challenge. The task is predict whether comment is toxic not while being fair across groups. A subset of comments have been labeled with identity attributes, including gender and race. has been shown that some of the identities (e.g., black) correlated with the toxicity label. In this work, we consider race as the sensitive attribute and select the subset of comments that contain identities black asian, as these two groups have the largest gap in terms of probability of being associated with a toxic comment. We use pretrained BERT embeddings (Devlin et al., 2019) to encode each comment into vector of size 768. three layer ReLU network then trained to perform the prediction with the encoded feature. We directly adopt manifold mixup since input mixup is equivalent to manifold mixup by simply setting the encoder g to BERT. Similarly, we retrain each model 10 times using randomly split training, validation, and testing sets, and report mean accuracy and fairness measurement.
Figures 6 (a) and (b) show the tradeoff between AP and ADP/AEO, respectively. Again, fair mixup consistently achieves a better tradeoff for both ADP and AEO. We then show the visualization of calibrated paths for ADP-regularized models in Figure 6 (c). We can see that even with the powerful BERT embedding, all the baselines present fluctuated paths with similar patterns. In contrast, fair mixup introduces a nearly linear curve with small slope, which eventually leads the smallest ADP.
Disclaimer: the attractive experiment is illustrative example and such classifiers of subjective attributes ethical.
8

---

Published as a conference paper at ICLR 2021
0.37
0.42
Expected Cuiput
0.00
0.36
0.40
ERM
-0005
DRM
Cappaa
GapRez
Moxu
-0.10
Maup
0.34
0.00
0U12
11.44
0,16
0L88
0.85
0.10
0.15
0.20
0.
0.2
0.4
0.6
0.5
1.0
DP
BO
Demographic Parity
(b) Equalized Odds
(c) Path Visualization
Figure 6: Toxic Classification (a,b) The tradeoff between AP and ADP/AEO. (c) Visualization of the calibrated paths for models that regularize ADP with different algorithms. Interestingly, fair mixup presents a nearly linear curve with small slope, while the baselines introduce "inverted-U" shaped curves.
CONCLUSION
In this work, we propose fair mixup, a data augmentation strategy to optimize fairness constraints. By bridging sensitive groups with interpolated samples, fair mixup consistently improves the gen- eralizabiliiy of fairness constraints across benchmarks with different modalities. Interesting future directions include (I) generating interpolated samples that lie on the natural data manifold with gen- erative models or via dynamic optimal transport paths between the groups (Benamou & Brenier, 2000), (2) extending fair mixup to other group fairness metries such as accuracy equality, and (3) estimating the generalization of fairness constraints (Chuang et al., 2020).
REFERENCES
Alekh Agarwal, Alina Beygelzimer, Miroslav Dudik, John Langford, and Hanna Wallach. re- ductions approach to fair classification. In International Conference Machine Learning, pp. 60-69, 2018.
Solon Barocas, Moritz Hardt, and Arvind Narayanan. Fairness and Machine Learning. fairml- book.org, 2019, http://t/eeee //ww://www.///..////////////wo// fai rm wlm.lboo...rr
Jean-David Benamou and Yann Brenier. computational fluid mechanics solution to the Monge- Kantorovich mass transfer problem. Numerische Mathematik, 2000.
Richard Berk, Hoda Heidari, Shahin Jabbari, Michael Kearns, and Aaron Roth. Fairness in crim- inal justice risk assessments: The state of the Sociological Methods & Research, pp. 0049124118782533 2018.
David Berthelot, Colin Raffel, Aurko Roy, and Ian Goodfellow. Understanding and improving inter- polation in autoencoders via adversarial regularizer. In International Conference on Learning Representations, 2018.
Piotr Bojanowski, Armand Joulin, David Lopez-Pas, and Arthur Szlam. Optimizing the latent space of generative networks. In International Conference on Machine Learning, pp. 600-609, 2018.
Alvin Chan, Yi Tay, Yew Soon Ong, and Jie Fu. Jacobian adversarially regularized networks for robustness. In International Conference Learning Representations, 2019.
Ching-Yar Yao Chuang, Antonio Torralba, and Stefanie Jegelka. Estimating generalization under distri- bution shifts via domain-invariant representations. International Conference on Machine Learn- ing. 2020.
Andrew Cotter, Maya Gupta, Heinrich Jiang, Nathan Srebro, Karthik Sridharan, Serena Wang, Blake Woodworth, and Seungil You. Training well- well-generalizing classifiers for fairness metrics and other data-dependent constraints. In International Conference on Machine Learning, pp. 1397-1405, 2019.

---

Published as a conference paper at ICLR 2021
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLI (1), 2019.
Harris Drucker and Yann LeCun. Improving generalization performance using double backpropa- gation. IEEE Transactions on Neural Networks, 1992.
Dheeru Dua and Casey Graff. UCI machine learning repository, 2017. URL URLhtt:://archie
ics. uci edu/ml.
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings the 3rd innovations theoretical computer science confer- ence, pp. 211-21122 214-226,70 2012.
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. In Advances in neural information processing systems, pp. 3315-33233 2016.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Alex Garran and Peter König. Data augmentation instead of rxpiiciteeuarriizttioi arXiv preprint arXiv:1806.038: 03852, 2018.
Judy Hoffman, Daniel A Roberts, and Sho Yaida. Robust learning with jacobian regularization. arXiv preprint arXiv: 19t::02289999999 2019.
Jigsaw. Jigsaw unintended bias in toxicity classification, 2018. URL httht:h/twww/i kaggle.. com/c c/ c/jigsaw unintended bias- -in www.oiiitt-lssssiiiiiiiieoeaeeee
Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Large-scale celebfaces attributes (celeba) dataset.
Christos Louizos, Kevin Swersky, Yujia Li, Max Welling, and Richard Zemel. The variational fair autoencoder. arXiv preprint arXivv1511.00880 2015.
David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. Learning adversarially fair and transferable representations. International Conference on Machine Learning, 2018.
Bahar Taskesen, Viet Anh Nguyen, Danicl Kuhn, and Jose Blanchet. A distributionally robust approach to fair classification. arXiv preprint arXiv:2007.095300 09500, 2020.
Vikas Verma, Alex Lamb, Christopher Beckham, Amir Najafi, Ioannis Mitliagkas, David I Lopez- Paz, and Yoshua Bengio. Manifold mixup: Better representations by interpolating hidden states. In International Conference on Machine Learning, pp. 6438-6447. PMLR, 2019.
Dennis Wei, Karthikeyan Natesan Ramamurthy, and Flavio du Pin Calmon. Optimized score trans- formation for fair classification. arXiv preprint arXiv: 000666 2019.
Blake Woodworth, Suriya Gunasekar, Mesrob Ohannessian, and Nathan Srebro. Learning non- discriminatory predictors. In Conference on Learning Theory, pp. 1920-1953. 2017.
Mikhail Yurochkin, Amanda Bower, and Yuekai Sun. Training individually fair ml models with sensitive subspace robustness. In International Conference on Learning Representations, 2019.
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi. Fair- ness constraints: Mechanisms for fair classification. In Artificial Intelligence and Statistics, pp- 962-970. PMLR, 2017.
Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning fair representations. In International Conference on Machine Learning, pp. 325-333, 2013.
10

---

Published as a conference paper at ICLR 2021
Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adver- sarial learning. In Proceedings of the 2018 AAAI/ACM Conference on Al, Ethics, and Society, pp- 335-340, 2018a.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. arXiv preprint 16X1::1611.0350 2016.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations, 2018b.

---

Published as a conference paper at ICLR 2021
A PROOFS
A.1 PROOF OF LEMMA 1
Lemma 1. Let T X² [0, 1] X be a function continuously differentiable w.r.t t such that
T(ro, I1, 0) 20 and To00, 21, = Ij. For any differentiable function f. we have
!
d di (
ADP())
interpolation
Proof. The result follows from the fundamental theorem of calculus. In particular, given an inter- polator T, we first rewrite the ADP with the T:
ADP(f) P2 fx) Egn P. fx))
= Ezo 01 P. f(xo)= - fx1))
Exo~Po ( #1. 0))
1))l
Not that Bag-Paaas ((T 11 t)) is real-valued continuous funetion on [0, 1]. Therefore, we have the following equivalence via the fundamental theorem of calculus:
ADP(f)
Ez Po.iiN. (T 11, 0)) Ezo P f (T 21, 1))
d E PI ((T(
dt
A.2 PROOF OF PROPOSITION 2
Proposition 2. (Gap Regularization) Consider the following minimization problem
min P(c.a)-P(((((())u)) + A JEH
2 + X2
2
For fixed embedding o the optimal solution f" corresponds to v" given by following closed form:
A2
8+ projhu.. (6+)
XI
A2
where proj is the soft projection defined proj
TM
Proof. The problem above can be written as follows:
((v) ((u, 82)) + À1 2
Xa 2
min
(w, 60.1) 2
||011
Setting first order condition to zero
V.L(v) -8 A1 & 60,1 + A2p 0,
obtain
(A160,1 @ do,1 + Aetm)u = S+.
By inverting and applying the Sherman-Morrison Lemma, we have
(A100,1 80,1 )22m)) -1
A2 Im + 60.1 3 60.1
8
\
(2g 200, @ 50,1
x
Im
84
1 1100,111
A2
1
Sa.1 2 60.1
In
64.
A
Aa X1
+ 1180.1
12

---

Published as a conference paper at ICLR 2021
Note the soft projection vector is defined as follows:
proj (x) | + a.
follows that
22 8+ - proj 60.1 (6+) A2 which can be interpreted as the projection of the label discriminating direction åf on the subspace that is orthogonal to the group discriminating direction do,
A.3 PROOF OF PROPOSITION
Proposition 3. (Fair Mixup) Consider the following minimization problem
min ( ( (x)( [)] + À 12maaa (f) A2 FEN
11513
Define m, = Po (ttr + (1 ()x:)] } be the t dependent mean embedding, and rive its derivative with respect Let D be a positive-semi definite matrix defined as follows: D So my @ dt, Given an embedding 2 the optimal solution v has the the following form:
(Ai A21m) 10z
Proof. For t E [0, 1]. we note by the distribution of T F1, t), for To Pos I2 ~ Pi, and note my Ezup, (xr) and my its time derivative. We consider the lz variant of (f)(f) f) in the analysis:
L.¹ [. ku) dl (v, (L," ni @ (ind))))
dt () dt =/[/ at
| (o. dt
RF(()
We then expand mn when is mixup:
m, dt Be 2(tro + (1 - t)z1)= = Ero ~P, J ( + (1 t)x1)( - x1)
where J denotes the Jacobian. Note that D = So my @ m,dt, hence for the classification with fair mixup regularizer, the problem is equivalent to:
+ A1 Du) +
A2 2 ||u|
min C(v) ((v,
2
Setting first order condition we obtain (Ai + 22 d2, which gives the optimal solution
v² (A, A2 84.
The corresponding optimal fair mixup classifier can be finally written as
f(x) = D + )21m)
B EXPERIMENT DETAILS
Adult We follow the preprocessing procedure of Yurochkin al. (2019) by removing some fea- tures in the dataset?. We then encode the discrete and quantized continuous attributes with one-hot encoding. We retrain each model 10 times with batch size 1000 and report the mean accuracy and fairess measurement. The models are selected via the performance the dataset is randomly split into training and testing set with partition 80% and 20%, respectively. The models optimized with Adam optimizer (Kingma & Ba, 2014) with learning rate 1 e For DP, we sample 500 datapoints for each A E {0, 1] to form a batch. Similarly, for EO, we sample 250 datapoints for each (A, Y) pair where A, Y E (0.
validation set. In each trial,
13

---

Published as a conference paper at ICLR 2021
CelebA Model-wise, extract the feature of size 512 after the average pooling layer of ResNet- 18. two-layer ReLU network with hidden size 512 then trained to perform prediction. Per- centage of positive-labeled datapoints for attractive, wavy hair, and smiling that is male are 22.7%, 18.36%, and 34.6%, respectively. We use the original validation set of CelebA to perform model selection and report the accuracy and fairness metrics the testing set. The visualization paths are also plotted with respect to the testing data. To implement manifold mixup, we interpolate the spatial features before the average pooling layer. Similarly, all the models are optimized with Adam optimizer with learning rate 1 x e
Toxicity Classification We download the Jigsaw toxic comment dataset from Kaggle website Percentage of positive-labeled datapoints for black and asian are 18.8% and 6.4%. respectively, which together results dataset size 22835. We retrain each model 10 times with batch size 200 and report the mean accuracy and faimess measurement. The models selected via the per- formance on validation set. The batch- sampligg and data splitting procedure is the same as the one for Adult dataset. The models again optimized with Adam optimizer with learning rate 1
C ADDITIONAL EXPERIMENTS
C.I TRAINING PERFORMANCE
In Figure 7, we show the training performance for Adult dataset. As expected, GapReg outperforms Fair mixup on the training set since it directly optimizes the fairness metrie. The results also support our motivation: the constraints that are satisfied during training might not generalize at evaluation time.
0.90
0.800
0.775
0.78
ERM
AdDDehias
0.76
Capplog
0.700
Misup
0.00
A
A
(a) Demographic Parity
(b) Equalized Odds
Figure 7: Training Performance Adult Dataset. The tradeoff between AP and ADP/AEO
training set.
C.2 EVALUATION METRIC
The relaxed evaluation metric could overestimate the performance when the predicted confidence is significantly different between groups. For instance, a classifier f can be completely unfair while satisfying this condition: f(x) 1 w.p. 60%, O w.p. 40% on Po. and f(x) 0.6 w.p. 100% on Pt. This satisfies this expectation-base condition. However, highly unfair we binarize prediction by setting the threshold 0.5.
To overcome this issue, let ft the binarized predictor f((z) = 1(f(x) 2 t), we evaluate the model with average ADP (ADP) defined as follows:
ADP())
Σ
[e-Pos(()) EEWA fe(x));
T
IET
ΣЕδ = T
Σ
E Be-po f((x) - E En f(())
FET (E
14

---

Published as a conference paper at ICLR 2021
where T is set of threshold values. ADP averages the ADP with binarized predictions derived via different thresholds. For instance, by averaging the ADP with thersholds = [0.1, 0.2, .... 0.9),
ADP 0.5 instead of O for the example above, which captures the unfairness between groups. We report ADP for each methods with thersholds T 0011, 0.2, ., 0.9) in Figure 8. Similarly, fair mixup exhibits the best tradeoff comparing to the baselines for demographic parity. For equalized odds, the performances of fair mixup and GapReg similar, where fair mixup achieves better tradeoff when AEO is small.
0.78
0.76
0.76
0.75
X 0.74
ERM
0.74
AdvDehias
GupReg Mixup
0.72
0.72
0.050
0.075
0.100
0.125
0.150
0.05
0.10
0.15
ADP
AED
(a) Demographic Parity
(b) Equalized Odds
Figure 8: Average ADP and AEO Adult Dataset.
C.3 SMALLER MODEL SIZE
To examine the effect of model size, we reduce the hidden size from 200 50 and show the result in Figure 9. Overall, the performance does not vary significantly after reducing the model size. We can again observe that fair mixup outperform the baselines for ADP. Similar to the results in section C.2, the performances of fair mixup and GapReg are similar, where fair mixup achieves better tradeoff when AEO is small.
0.76
0.74
0.74
Advieebis
Gaplezz
0.72
Mixip
0.05
0.30
0.15
0.05
0.15
DP
A0
(a) Demographic Parity
(b) Equalized Odds
Figure Reducing the Model Size on Adult Dataset.
15


Reviews:
[{"review_id": "vbP0DQITaTE", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes \"fair mixup\" for training fair classifiers.  Inspired by the mixup algorithm, which was presented to improve the generalization performance in Zhang et al., 2018b, fair mixup pick two samples from two different sensitive groups.  Instead of regularizing the gap (e.g., \\delta DP), the authors regularized the derivative along the path between two samples.  This algorithm can be applied either to the input space or to the feature space.  The idea of using mixup for training fair classifiers is interesting, and the paper is well written and easy to read.  However, I have a few critical concerns.\n \nConcerns: \n- E_{x from P0} f(x) = E_{x from P1} f(x) does not imply fairness:  My first concern is the choice of this relaxed metric, which the authors borrowed from Madras et al. (2018).  Most surrogate conditions are sufficient conditions for the target fairness condition.  For instance, zero mutual information between the sensitive attribute and the classification output implies demographic parity.  The empirical covariance, used in Zafar et al., is also similar.  However, this one is neither necessary nor sufficient.  A classifier can be completely unfair while satisfying this condition: f(x) = 1 w.p. 60%, 0 w.p. 40% on P0, and f(x) = 0.6 w.p. 100% on P1.  This satisfies this expectation-based condition.  However, it is highly unfair against P0 as P(Y=1|P0) = 0.6 and P(Y=1|P1) = 1, assuming the threshold is 0.5.  I would make sense if this metric helps impose the actual fairness conditions we care about. Still, the authors only reported \\Delta DP and \\Delta EO, without reporting the actual DP/EO differences or multiplicative gaps.  \n\n- Does fairness condition not generalize?:  First of all, I haven't seen large gaps between the level of fairness measured in the train set and the test set.  I believe that it really depends on the choice of training algorithm, and most of the methods I have seen in the literature do not exhibit huge gaps.  Even when there is such a gap, one can choose the best model based on the validation performance (accuracy/fairness).  The authors may want to add more evidence on the lack of fairness generalization of existing algorithms to justify the considered problem.\n\n- Other baseline algorithms with early stopping:  The authors seem not to use any validation or early-stoping to maximize the test performance.  This is commonly done in most work in the literature, so please clarify this or do so.  Most importantly, related to the above concern, GapReg's performance might be improved if used with proper validation and early stopping.  Also, please report the train performance of GapReg.  Its train performance should be better than Fair mixup in every aspect.  AdvDebias is not a good baseline algorithm as it's not optimized for a target task.  Please add proper baseline algorithms.  \n\n===\n\npost-rebuttal:  The authors have addressed some of my concerns, but the experimental results are still missing several important baselines.   Raising my score from 4 to 5.", "extra_scores": {}, "extra_text": {}}, {"review_id": "LJX4XNAV8vS", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper provided a fair mixup strategy to improve the generalizability of fair classifiers. The authors also provide theoretical understanding of the proposed methods. Extensive experimental results are presented to verify the effectiveness of proposed method. Mainly, I have two questions that need to be answered.\n\n1) The authors claimed that \"the proposed fair mixup can improve the generalization of group fairness metrics\", and they \"provide a theoretical analysis to deepen the understanding of the proposed method\". However, it can not be seen from either proposition 1 or proposition 2 to show the improvement of generalization. Either proposition 1 or proposition 2 does not show the generalization bound. Could you please give some explanations? \n\n2) In the theoretical analysis, the authors considered $f(x)$ as a linear function of $\\Phi(x)$ where $\\Phi$ is an embedding function. However, this is not true in many applications. Even for the experiments in this paper, this is not true. For example, two-layer ReLu networks (this is not a linear function) is used in section 6.1. If so, the theoretical results can not fully deepen the understanding of the proposed method, which makes the results less important. On the other hand, if general non-convex function $f(x)$ (for example, NN with ReLu activation function) is considered, then the optimal $v^*$ is very hard to obtain since this is a NP-hard for general non-convex minimization problem.\n\nFinally, I am wondering if the proposed methods can be extended to multi-class classification since this paper is studying binary classification problem?", "extra_scores": {}, "extra_text": {}}, {"review_id": "IF-ebW-xth", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: This paper proposed a data augmentation method for training a classifier which is intended to have predictive parity between two identified groups. It is based on the “mixup” idea – samples from the two groups are interpolated between, and the smoothness of this path is encouraged. The authors recommend doing the interpolation in latent space. An optimal solution in a constrained setting is derived, and experiments show the empirical success of the model at this task on 3 datasets.\n\nRecommendation: I recommend acceptance of this paper. The application of mixup to this task is sensible, the paper is clearly written, and the theoretical and empirical work both seem solid.\n\nStrengths:\n-\tUsing mixup for a task like this seems reasonable. There is a literature on data augmentation methods for this task\n-\tExplanation of the mixup method is clear and the theoretical work seems good\n-\tThe empirical results are pretty strong, beating each method at each task. Showing the tradeoff is good, this is the right type of diagram to use here\n-\tI like that this method is non-adversarial – easier to train and more reliable imo\n\nWeaknesses + Clarifications:\n-\tThe question of the latent variable model seems relevant and interesting. It seems that the mixup method is only as good as the model, and also the trained model might add its own biases to the classification task. It would be nice to see some discussion of this in the paper\n-\tI am surprised that mixup improves precision on the adult task. It would be good to see some exploration of this\n-\tFor experiments, are all runs shown? Or just the Pareto fronts.\n-\tA number of hyperparameters (e.g. regularization) are not given \n-\tFor all the latent path figures (eg Fig 3) why is the y value at x= 0 always 0? Is it normalized to this? Be clear in your description (or maybe I missed it)\n-\tI would be interested in seeing some further analysis on this model, perhaps using the interpolations themselves\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "74Jmt-gLXj", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Brief Summary:\n\nThis paper introduces a method to improve fairness generalization.  The authors utilize a modified version of mixup where they regularize in favor of invariance of classifier prediction for interpolations between sensitive groups.  The idea is that we expect changes to sensitive attributes not to affect classification decisions.  They introduce a number of regularization terms for both demographic parity and equal opportunity.  Because the mixup interpolations are initially taken as linear interpolations between data --- and thus may not be naturally occurring --- they also introduce a version where samples are taken from a latent space.  They perform assessments on a number of datasets and modalities including Adult, CelebA, and  Jigsaw toxic comments.  They find their methods improve fairness generalization compared to a few baselines, including directly regularizing the model for fairness.\n\nQuestions + Comments:\n\nSection 1:\n- Figure 1 is a nice example. It could be good if Expected Output was written out a bit more explicitly or make it so that $\\Delta DP$ is clearly the difference between the two points.  When I glanced at this graph at first, I thought $\\Delta DP$ was being plotted --- consider that readers haven't reached the methods section at this point.  I was somewhat confused about what was being shown at first. \n\nSection 4:\n- When introducing lemma 1, I'd recommend stating explicitly that $T(x_0, x_1, t)$ corresponds to a function that outputs an interpolated sample between $x_0$ and $x_1$ at step $t$. This is clarified immediately below the lemma, but currently, the reader is left guessing when they first read the lemma as to what $T(x_0, x_1, t)$.  I found it slightly confusing while reading and ask the authors to consider stating it upfront. \n- Why is  $P_0 (x_0)$ what we're integrating over? What is this notation meant to say? I notice that this is equivalent to the expected value of drawing $x_0 \\sim P_0$, so is this meant to indicate the likelihood of each $x_0$ as well? \n\nSection 5:\n- When describing manifold mixup, there's an assumption that interpolations in the latent space $\\mathcal{Z}$ will map nicely onto the set of legitimate data instances. If the encoder-decoder pair is an autoencoder trained without any specific regularization, it could be the case that many instances for linear interpolations could be off manifold.  I don't think this assumption is poor by any means, but I think describing the desired properties of the latent space could be a nice addition. \n\nSection 6:\n\n- Subsection 6.1: The two layer ReLU model with hidden size of $200$ is very large for the adult data set.  Is it necessary to use such a large model for this method and train for $1000$ iterations, which also feels like a long time for such a small data set? Its surprising that mixup improves overall performance on the data so noticeably, and I'm wondering if the ERM model has overfit the data at this point --- though such training regimes may be needed for fair mixup given that only one $t$ is sampled iteration. Could the authors try something like (1) decreasing the training time or size of the model or (2) just add another baseline such as a random forest or logistic regression to this graph? I think this would help us understand the effects of fair mixup a bit better. \n\n- Subsections 6.2: For selecting the models, did you always choose the model with best validation accuracy for both ERM and fairmixup? Further how long did you train? \n\n- Subsection 6.3: Was model selection done in this experiment?\n\nOverall Experimental questions:\n\n- I'm curious as to the training requirements imposed by the use of this method. The authors compare ERM models trained at as many iterations as the fairmixup models.  However, given that only one $t$ is sampled per batch, it might take quite a few iterations to get a good fairmixup model.  Is this the correct intuition? Will the ERM models produce optimal solutions long before fairmixup? Further, if we increase the number of $t$'s, will the fairmixup models converge quicker at the cost of computation time? I'd appreciate if the authors add a bit of discussion related to this.  If the question is unfounded, I'd appreciate clarification. \n\nFor the experimental selection, some model selection for the adult data sets and toxicity classification would add the experiments and help disentangle the effects of overfitting (unless I'm missing something) -- particularly in the case of the adult data set because right now, I'm inclined to think the model is overfitting.\n\nOverall, I'm fairly convinced this method is working well and appreciate the breadth of data modalities and experiments the authors performed performed. \n\nOverall thoughts:\n\nI think this is a nice paper with a useful contribution to fairness generalization.  If it is really the case that this method is flexible enough to improve both fairness and classification generalization, this method could be quite useful in general. I have a couple clarification questions from the body of the paper, and I'd appreciate answers from the authors.  Further, I'd also appreciate it if the authors provided clarification to the experimental decisions I asked about above. \n\nOne last point for the authors to consider is that for CelebA, the authors build classifiers to preform attractiveness classification. Though they do so from the perspective of discouraging classifiers to exhibit biases, it still could be worthwhile to include a brief discussion in the appendix on the ethical considerations of such models.\n\n \n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: # Paper Summary

The goal of this paper is to improve generalization of fairness metrics by borrowing ideas from "mixup", which attempts to improve generalization in the non-fairness setting by introducing convex combinations of training examples as virtual examples.

They adapt this idea by interpolating between protected *groups*, and adding a regularizer that forces the classifier to vary smoothly along this interpolation path. To this end, they show that, for a particular interpolation function, the (empirical) disparity in the fairness metric is upper bounded by their proposed regularizer (which depends both on the fairness metric, and the interpolation function). They consider two fairness metrics (disparate impact and equalized odds) and two interpolation functions (convex combinations in the feature space, or in a latent space).

As Reviewer 4 points out, the above is not a complete explanation for why their regularizer works: they've only really shown that it upper bounds the empirical disparity in the fairness metric (and we could have regularized this empirical disparity directly, and indeed they do so, as a baseline, in their experiments). Presumably the intuition is that their regularizer is improving generalization by (implicitly) depending on virtual examples, but this isn't made explicit.

In a "theoretical analysis" section, they give closed form solutions using classification loss, along with L2 regularization and either (i) a regularizer penalizing the true disparity of impact or (ii) their proposed regularizer (which upper bounds the former). Both reviewer 4 and I seem to doubt if this adds much insight (the other reviewers didn't discuss this section).

They close with experiments on Adult, CelebA, and Jigsaw Toxicity, all of which show dramatic performance gains using their regularizer. However, they only compare to one external baseline (adversarial debiasing).

# Pros

1. Reviewers agreed that the paper was well-written
1. The derivation of their regularizer is somewhat complex, but is described step-by-step, and very clearly
1. Adapting mixup to the problem of improving fairness generalization seems natural and intuitive, but this intuition is maybe given short shrift in the later sections
1. Experiments show impressive results

# Cons

1. Reviewer 1 notes that having the expected value of the classification function be equal for both protected groups does not imply fairness, since the classification function would presumably be thresholded to make hard classification decisions
1. Reviewer 4 points out that they do not actually explain why their regularizer will improve generalization better than the "usual" disparity regularizer. Instead, they only show that it upper-bounds the empirical disparity in the fairness metric. Presumably, the intuition is that their mixup regularizer is doing something like adding "virtual samples"
1. I would like to see a more detailed explanation of how their regularizer is implemented, in the main text (they only say that it "can be easily optimized by computing the Jacobian of f on mixup samples")
1. Reviewers 1 and 2 would like more external baselines (there is only one at the moment, "adversarial robustness"), with reviewer 1 suggesting early stopping. The authors added a new early stopping experiment on CelebA to the appendix, but it would be nice to have this baseline included in all experiments in the main text

# Conclusion

Three of the four reviewers recommended acceptance, with the "reject" reviewer scoring it "5: weak reject". This reviewer had three main criticisms: (i) matching expected classification functions is not the same as matching classification *decisions*, (ii) fairness problems might not have a generalization problem to begin with, and (iii) the experiments don't include enough external baselines. I disagree with the second point, but agree with the other two. I think the third is the most critical, since the first could be solved in many cases by e.g. sampling instead of making hard deterministic decisions.

Overall, my opinion is that this is a borderline paper, but that it falls on the "accept" side of the boundary. The idea is intuitive, and exposition is clear, the derivation is quite interesting, and the experimental results are (aside from not having enough baselines) impressive.
Decision: Accept (Poster)