Learning Antidote Data to Individual Unfairness
Authors: Peizhao Li
Ethan Xia
Hongfu Liu
Venue: Submitted to ICLR 2023 (2023)
Keywords: Individual Fairness
Antidote Data
Machine Learning Fairness

Abstract: Fairness is an essential factor for machine learning systems deployed in high-stake applications. Among all fairness notions, individual fairness, following a consensus that `similar individuals should be treated similarly,' is a vital notion to guarantee fair treatment for individual cases. Previous methods typically characterize individual fairness as a prediction-invariant problem when perturbing sensitive attributes, and solve it by adopting the Distributionally Robust Optimization (DRO) paradigm. However, adversarial perturbations along a direction covering sensitive information do not consider the inherent feature correlations or innate data constraints, and thus mislead the model to optimize at off-manifold and unrealistic samples. In light of this, we propose a method to learn and generate antidote data that approximately follows the data distribution to remedy individual unfairness. These on-manifold antidote data can be used through a generic optimization procedure with original training data, resulting in a pure pre-processing approach to individual unfairness, or can also fit well with the in-processing DRO paradigm. Through extensive experiments, we demonstrate our antidote data resists individual unfairness at a minimal or zero cost to the model's predictive utility.

Under review as a conference paper at ICLR 2023
LEARNING ANTIDOTE DATA TO INDIVIDUAL UNFAIR- NESS
Anonymous authors
Paper under double-blind review
ABSTRACT
Fairness is an essential factor for machine learning systems deployed in high-stake applications. Among all fairness notions, individual fairness, following a consen- sus that 'similar individuals should be treated similarly.' is vital notion to guar- antee fair treatment for individual cases. Previous methods typically characterize problem when perturbing sensitive at-
individual faimess as prediction-invariae tributes, and solve it by adopting the Distributionally Robust Optimization (DRO) paradigm. However, adversarial perturbations along direction covering sensitive information do not consider the inherent feature correlations or innate data con- straints, and thus mislead the model to optimize at off- manifold and unrealistic samples. In light of this, we propose method to learn and generate antidote data that approximately follows the data distribution to remedy individual unfairness. These on- manifold antidote data can be used through generic optimization pro- cedure with original training data, resulting individual unfairness, or can also fit well with the in in-processing DRO paradigm. Through extensive experiments, we demonstrate our antidote data resists individ- ual unfairness at minimal or zero cost to the model's predictive utility.
pure pre-processing approach
INTRODUCTION
Unregulated decisions could reflect racism, ageism, and sexism in high-stakes applications. such as grant assignments (Mervis) 2022), recruitment (Dastin 2018), policing strategies (Gelman al.] 2007]. and lending services (Bartlett et al. 2022). To avoid societal concerns, fairness, as one of the fundamental ethical guidelines for AI, has been proposed to encourage practitioners to adopt Al responsibly and fairly. The unifying idea of fairness articulates that MI systems should not dis- criminate against individuals or any groups segmented by regally-proeecter and sensitive attributes, therefore preventing disparate impact in automated decision-making (Barocas & Selbst 2016).
Kusner et al., 2017 Hashimoto al. 2018). Group faimess is currently the most influential notion in the fairness com- munity, driving different groups to receive equitable outcomes regardless of their sensitive attributes, in terms of statistics like true positive rates positive rates (Hardt et al.. 2016). However, these statistics describe the average of a group, hence lacking guarantees on the treatments of individ- ual cases. Alternatively, individual fairness established upon consensus that 'similar individuals should be treated similarly,' shift force to reduce the predictive gap between conceptually similar instances. Here, 'similar' means two instances have close profiles regardless of their different sensi- tive attributes, and usually have customized definitions upon domain knowledge. We invite readers look into Section for more concrete establishment on individual fairness.
Many notions have been proposed to specify AI Fairness (Dwork et al. 201 2
Previous methods solve the individual fairness problem mainly by Distributionally Robust Optimiza- tion (DRO) (Yurochkin et al., 20200 Yurochkin & Sun 202T RRoss et al. [020 Yeom & Fredrikson) 2021. They convert the problem to optimize models for invariant predictions towards original data and their perturbations, where the perturbations are adversarially constructed to mostly change the sensitive information in a sample. However, one use case of DRO in model robustness is to adversar- ially perturb a sample by a small degree. The perturbations can be regarded as local perturbations, and the adversarial sample is still the data manifold. In contrast, perturbing sample for individ- ual fairness purposes, e.g., directly flipping its sensitive attributes like gender from male to female, cannot be regarded as a local perturbation. These perturbations may violate inherent feature cor-

---

Under review as a conference paper at ICLR 2023
relations, e.g., some features are subject to gender but without notice, thus driving the adversarial samples leaving the data manifold. Additionally, perturbations in a continuous space could break the innate constraints from tabular, e.g., discrete features should be in one-hot format. Consequently, these adversarial samples for fairness are unrealistic and do not match the data distribution. Taking these data can result in sub-optimal tradeoffs between utility and individual fairness.
In this work, we address the above limitations and propose an approach to rectify models for in- dividual fairness from a pure data-centric perspective. Following the high-level idea of the DRO paradigm, and by giving a concrete setup for similar samples, we learn the data manifold through generative models, and continue to construct on mo-maniifold samples with different sensitive attributes as antidote data to mitigate individual unfairness. We launch two ways to use the generated anti- dote data: simply inserting antidote data into the original training set and training models through regular optimization, or equipping antidote data to the DRO pipeline an in-processing approach. Our approach works for multiple sensitive attributes, and each sensitive attribute can have multi- ple values. We conduct experiments census, criminological, and educational datasets, compared to standard classifiers and several baseline methods. Compared to baseline methods, our method greatly mitigates individual unfaireess and has minimal or zero side effects to model utility.
2 INVIDIVUAL FAIRNESS AND COMPARABLE SAMPLES
Notations Let fo denote a parameterized probabilistic classifier, X' and y denote input and output space with instance x and label y. respectively. For tabular datasets, we assume every input instance contains three parts of features: sensitive features s = [s1, s2. .sN.]. continuous features c c= (2, c. and discrete features d = [dl, d2. ... dNzl. with N denoting the number of features in each parts. We assume these three parts of features are exclusive, i.e., C, and d do not share any feature or column. We use dg to denote the discrete features of instance x, and the same manner for other features. For simplification we shall assume discrete features d contain categorical features before one-hot encoding, continuous features с contain features in a unified range like [o. 1] after some scaling operations, and all data has the same feature dimension. We consider sensitive attributes in a categorical format. Any continuous sensitive attribute can be binned into discrete intervals fit our scope. We use + to denote vector-vector or vector-scalar concatenation.
Individual Fairness: Concept and Practical Usage The concept of individual fairness is firstly raised in Dwork et al. (2012). Following consensus that 'similar individuals should be treated sim- ilarly,' the problem is formulated as a Lipschitz mapping problem. Formally, for arbitrary instances and x' E X, individual fairness is defined as (Dx, Dy) Diisschht property of classifier fo: S Dx(x,
Dy (y(f)
(1)
where +x(::)) and Dy(:,)) :) are some distance functions respectively defined in the input space X' and output space V and shall be customized upon domain knowledge. However, for general problem, it could be demanding to carry out a concrete and interpretable (-.-.)) and (-)),), hence makes individual faimeess impractical in many applications. To simplify this problem from a continuous Lipschitz constraint, some works evaluate individual fairness of models with binary distance function: xx' = for two different samples x and x' if they are exactly the same except sensitive attributes, i.e., = c', d = d', and A s' (Yurochkin et al.) 2020 Yurochkin & Sun 202T). Despite the interpretability, this constraint can be too harsh to find sufficient comparable samples since other attributes may correlate with sensitive attributes. For empirical studies, these studies can only simulate the experiments with semi-synthetic data where they flip onc's sensitive attribute to construct sample and evaluate the predictive gap. Note that for tabular data, simply discarding the sensitive attributes could be a perfectly individually fair solution to this simulation.
In this work, consider relaxed version of the above individual faimess definition for imper- fect classifier. We present Definition 2.T to characterize in what conditions we shall consider two samples are comparable. When two samples and x' are coming be comparable, their predictive gap [x( - Jo(=') should be minimized for the individual fairness purpose.
Definition 2.1 (comparable samples). Given Ta, To E R22, and x' are comparable iff all con- straints are satisfied: 1. N, [d,, # d3 < Ta: 2. max{lc. - <11 s Vi,isis No: and 3.
y == n'
Remark 2.1. For some thresholds Ta and Te, two samples are considered comparable iff 1. there are at most TĄ features differing in discrete features; 2. the largest disparity among all continuous features is smaller or equal to T., and 3. two samples have the same ground-truth label.
2

---

Under review as a conference paper at ICLR 2023
allows two samples to be slightly different in discrete and continuous features, and arbitrarily different in sensitive attributes. The definition is also flexible to extend if users want to enforce some crucial features to be identical for comparable samples, and this does not affect our model design. Our comparable samples are highly interpretable and semantically rich. For example, in lending data, certify individual fairness for two samples, we set discrete features to the history of past payment status (where value 1 indicates a complete payment, and value O indicates missing payment), and continuous features to the monthly amount of bill statement. Two samples are considered to be comparable if they have a determinate difference in payment status and amount of bills. In what follows we shall build models and evaluate individual fairness by Definition 2.1 and mostly consider comparable sumples with different sensitive attributes.
Definition 2.1
3 LEARNING ANTIDOTE DATA TO INDIVIDUAL UNFAIRNESS
Motivation Several methods solve the individual fairness problem through Distributionally Ro- bust Optimization (DRO) Yurochkin et al. 22200 Yurochkin & Sun 2021 Ruoss et al.) 2020 Yeom
& Fredrikson| 2021). The high-level idea is to optimize a model at some samples with perturbations that dramatically change their sensitive information. The solution can be summarized as:
min (Fo())) and min B(ax) max e(f(xr + +), xte~Dai
(2)
fo
where the first term is standard empirical risk minimization, and the second term is for loss mini- mization over adversarial samples. Dsen is some customized distribution offering perturbations to specifically change one's sensitive information. For example, Yurochkin et all (2020) character- izes Dsen as a subspace called sensitive subspace learnt from logistic regression, which contains the most predictability of sensitive attributes. [Ruoss et al.] (2020) find out this distribution via logical constraints. Though feasible, we would like to respectfully point out that (1) Perturbations violate feature correlations could push adversarial samples leave the data manifold. An intuitive example is treating age a sensitive attribute. Perturbations can change a person's 's age arbitrarily to find an optimal age that encourage the model predict the most differently. Such perturbations ignore the correlations between the sensitive feature and other features like education or annual income, resulting in an adversarial sample with age or 10 but holding a doctoral degree or getting $80K annual income. (2) Samples with arbitrary continuous perturbations can easily break the nature of tabular data. There are only one-hot discrete values for categorical variables after one-hot encoding, and potentially fixed range for continuous variables. For example, the adversarial samples may in half bachelor degree and half doctoral degree. These two observations make the adversarial samples from Deen unrealistic and leaving the data manifold, thus distorting the following DRO paradigm, and resulting in sub-optimal tradeoff' between fairness and utility.
In this work, we address the above issues related to Dsen and propose generate on-manifold data for individual fairness purposes. The high-level philosophy is, by giving an original training sample, generate its comparable samples with different and reasonable sensitive attributes, and the generated data should fit into existing data manifold and obey the inherent feature correlations or innate data constraints. We name the generated data as antidote data. The antidote data can either mix with original training data to be pre- pre-processing technique, or either serve as Dsen in Equation (2) as an in in-processing approach. By taking antidote data, a classifier would give individually fair predictions.
3.1 ANTIDOTE DATA GENERATOR
We start by elaborating on the generator of antidote data. The purpose of antidote data generator go is, given a training sample x, generating its comparable samples with different sensitive attributees). To ensure the generations have different sensitive features, we build go as a conditional generative model to generate a sample with pre-defined sensitive features. Given sensitive attributes 8 A s. (recall s is the sensitive attributes of instance x),, the objective is:
go: (z. 2) + 2, with S S. x and d satisfy Definition 2.1
where z ~ No0 1) is drawn from a standard multivariate normal distribution as a noise vector. The generation & should follow the data distribution and satisfy some innate constraints from dis- crete or continuous features, i.e., the one-hot format for discrete features and a reasonable range for continuous features. In the following, we shall elaborate the design and training strategy for go-
3

---

Under review as a conference paper at ICLR 2023
Encoding Continuous Values For continuous features, we adopt mode-specific normaliza- tion (Xu et al. 2019) to encode every column of continuous values independently. We use Vari- ational Bayesian to estimate the Gaussian mixture in the distribution of one continuous feature. This approach will decompose the distribution into several modes, where each mode is a Gaussian distri- bution with unique parameters. Formally, given a value Ci in the i-th column of continuous feature and j-th row in the tabular, the learned Gaussian mixture is P(ct,) = Σx=1 WC,NN Pr, R, where wa is the weight of k-th mode in i-th continuous feature, and μ4g. and 0% are the mean and standard deviation of the normal distribution of k-th mode. We use the learned Gaussian mixture to encode every continuous value. For each value C.jj we estimate the probability from each mode via Ps. (c,,( = (et, k oi and sample one mode from the discrete probability distribution P; with K; values. Having a sampled mode k. we represent the mode of Cr,j using a one-hot mode indicator vector, an all-zero vector Cú,r except the k-th entry equal to 1. We use scalar to represent the relative value within k-tth mode: Vi,r = (Ca,j / By encoding all continuous values, we have re-representation ō to substitute as as the input for antidote data generator ge:
i = (v1.z ① C1, ① + UN... 2 eN...) @ d₂ 1 Sz.
(4)
Recall ( denotes vector-vector vector-scalar concatenation. To construct comparable sample èt, the task for continuous features is to classify the mode from latent representations, i.e., estimate C, and predict the relative value U/ We can decode U/,a and C1 back to continuous value using the learned Gaussian mixture.
Structural Design The whole model is designed in Generative Adversarial Networks GGodfell- low et al. 2014) style, consisting of a generator Jo and a discriminator da.
The generator go takes the re-representation म. pre-defined sensitive feature s, and noisy vector z as input. The output from ge will be a vector with the same size as E including Ug. eg, d,, and sg. To ensure all discrete features are in a one-hot manner so that the generations will follow tabular distribution, we apply Gumbel softmax (Jang et al.) 2017) as the final activation to each discrete feature and obtain dg. Gumbel softmax is differentiable operation to encode a continuous distribution over simplex and approximate it to a categorical distribution. This function controls the sharpness of output via hyperparameter called temperature. Gumbel softmax is also applied to sensitive features 83 and mode indicator vectors eş to ensure the one-hot format.
The purpose for the discriminator model dg is to distinguish the fake generations from real samples, and we also build discriminator to identify generated samples in terms of its comparability from real comparable samples. Through discriminator, the constraints from comparable samples are implicitly encoded into the adversarial training. We formulate the fake sample for discriminator as t @ (é- i). and real samples as á' O â @ (x' - .), where E' is the re-representation of a comparable sample x' to drawn from the training data. The third term f 3 is encoded to emphasize the different between two comparable samples. Implicitly regularizing the comparability leaves full flexibility to the generator to fit with various definitions of comparable samples, and avoid adding complicated penalty terms, as long as there are real comparable samples prepared for training.
Training Antidote Data Generator We train generator and discriminator iteratively through the following objectives with gradient penalty Guirajani et al. (0117) to ensure stability:
min E Ex,x'' g0
(c (S2, sx) do (ā @ s + z)).
(5)
min E Be,x'rDamp
(go (ē Ss + 2z)) do()'),
(6)
where D Deomp is the distribution describing the real comparable samples in data, fce is cross entropy loss to penalty the prediction of every sensitive attribute in Sa with Sy as the ground-truth.
3.2 LEARNING WITH ANTIDOTE DATA
We elaborate two ways to easily apply the generated antidote data for the individual fairness purpose.
In practice, it is not strictly guaranteed that ge will produce comparable samples submitting to Def- inition 2.1 Some samples may be incompatible with some pre-defined sensitive features coming from the violations of neural networks. Thus, we apply post-processing step Post to filter out comparable samples from all the generations. Given a dataset X, for one iteration of sampling, we input every with all possible sensitive features (except s.)) the generator, collect raw generations
4

---

Under review as a conference paper at ICLR 2023
Algorithm DRO-Anti: DRO with Antidote Data for Individual Fairness
1: Input: Training data T = {(r w/) learning rate v, loss function
2: Train Comparable Sample Generator go with {at} } and comparable constraints
3: Sample antidote data X using ge
4: repeat
5: - 0 - ( (V (o(maxaE( ((,, ((,, w))l
D {0 t- the set of M
comparable samples of and { post(X)
6: until convergence
X. and apply Post(X) (X) to get the antidote data. The label y for antidote data is copied from the original data. We may have multiple iterations of sampling to enlarge the pool of antidote data.
The first way antidote data is to simply insert all antidote data the original training set:
min fo
[((o(z)..) x E X + PX)t(X).
Since we only add additional training data, this approach is model-agnostic, flexible any model op- timization procedure. and fits well with well well-developed data analytical toolkits such as sklearn (Pe- dregosa al. 2011). We consider the convenience as a favorable property for practitioners.
The second way is to apply antidote data with Distributionally Robust Optimization. We present the training procedure in Algorithm [ In every training iteration, except the optimization at real data with E(z, y), we add an additional step to select z's comparable samples in antidote data with the highest loss incurred by the current model's parameters, and capture gradients from { ) [(.. y) to update the model. The algorithm is similar to DRO with perturbations along some sensitive directions, but instead we replace the perturbations with on-manifold gener- ated data. The additional loss term in Algorithm [ can be upper bounded by gradient smoothing regularization term. Taking Taylor expansion, we have:
max (a.) (..) (m
f(x,, ()
max BE (2:) m x-
[l(E, v) E(x,,)]
= =((r,y) + max KV V y), (+ x))l + O(82)
(8)
(E; c-
< <1(r, v) + + max V E(x, + max V E1x, y) + N max V,, ((, + 0669)
Recall Ta and Te the thresholds for discrete and continuous features in Definition 2.1 ( is the higher-order from Taylor expansion. The last inequality is from Definition 2. The three gra- dients discrete, continuous, and sensitive features serve gradient regularization and encourage the model to have invariant loss with regard to comparable samples. However, the upper bound is only sufficient but not necessary condition, and our solution encodes real data distribution into the gradient regularization to solve individual unfairness with more favorable tradc-offs.
4 EXPERIMENTS
4.1
EXPERIMENTAL. SETUP
Datasets We involve censual datasets Adult (Kohavi & Becker 1996) and Dutch (Van der Laan 2000), educational dataset Law School (Wightman) 1998) and Oulad (Kuzilek al. (017)) and criminological dataset Compas (Angwin et al. 2016) in our experiments. For each dataset, we select one two attributes related ethics as sensitive attributes which expose a significant individual unfairness in base model like neural networks. We report their details in Appendix A
Protocol For all datasets, we transform discrete features into one-hot encoding, and standardize the features by removing the mean and scaling to unit variance. We transform continuous features into the range between and I. We construct pairs of comparable samples for both training and testing sets. In experiments, different from (Yurochkin & Sun 2021 Yurochkin al. 2020). our evaluations on tabular datasets are sampled from real testing data but not simulated. We evaluate both the model utility and individual fairness in experiments. For utility, consider the under the Receiver Operating Characteristic Curve (ROC), and Average Precision (AP) to characterize the precision of probabilistic outputs in binary classification. For individual faimess, consider
5

---

Under review as a conference paper at ICLR 2023
Table Experimental results on Adult dataset
ROC1 t
A t
Pos. Comp. (Mean/Q3) 4
Neg. Comp. (Mean/Q3) +
LR (Basc)
90.04
75.72
31.75/ / 43.55
10.25 18.37
L.RProoj
81.40 lo 60%
62.19 -17.87%
25.10-20.95% -20955 34.83 -20.02%
23.29 4127.17% / 33.03 +79818
LR+Dis
89.95 0.10%
75.59
30.81 -294% / /41.0 5.62%
9.40 9.29% 17.78
LR+Anti
89.72 -035%
75.04
24.72 22.133 / 30.84 129.18%
8.66 15.56%
14.64 -20.29%
LR+Anti+Dis
89.56 .0.3%%
74.83
23.02 -27.9%%
26.61 -38.90%
8.12
20.76/
13.91 -24.28%
NN (Base)
88,18
70.09
33.21 / 47.84
13.03
23.37
NN+Proj
87.42 L036%
68.51 -2.25%
32,38 -2.52% / 46.455 -2.91%
13.59 +4.36% / 23.69 +1.375
NN+Dis
88.15 0.04%%
70.27 +0.20%
32,90 -099% / 44,77 6.37%
11.83 9.17% / 23.36 -0.0%%
SenSR
86,01 -2.4%%
66.19 -5.5%%
28,68 13.63% / 44.22 -7.59%
14.88 +14.20% / 23.07 -1.31%
SenSel
86,42 2000%
66.08 -5.72%
27.92 11594% / 33,92 -24.91%
13.22 +1.53% / 26.01 +11.28%
LCIFR
87.35 4s9r
68.52 2.24%
32.51 2.13% / 44.84 6626%
12.97 -.11% 26.49 +13.35%
NN+Anti
87.95 0.26%
69.51 0.83%
26.05 21.57% / 35,76 25.26%
10.42 19.97%
16,88 27.880%
ANtAntiDDis
87.79 0.44%
69.40
24.40
26.53% 32. 12 32.85%
9.56
26.63%
15.54 -33.51%
DRO-Anti
87.91 -031%
71.08 41%%
17.46 -47.4%% 20.04 15%
5.48 -5.966 6.87 -70.595
the gap in probabilistic scores between comparable samples when both two samples have the same positive or negative label (abbreviated Pos. Comp. and Neg. Comp.). We evaluate unfairness for positive and negative comparable samples in terms of the arithmetic mean (Mean) and upper quartile (Q3). The upper quartile can show us the performance of some porse-performed pairs. For a base model with randomness like NN, we ran the experiments five times and report the average results.
Baselines We consider two base models: logistic regression (LR), and three-layers neural net- works (NN). We use logistic regression from Scikit-learn (Pedregosa et al.) 2011). and our antidote data is compatible with this mature implementation since it does not make change to the model. Approaches involving DRO currently do not support this LR pipeline, but will be validated through neural networks implemented with PyTorch. We have the following five baselines in experiments: 1. Discard sensitive features (Dis). This approach simply deletes the appointed sensitive features in the input data; 2. Project (Proj) (Yurochkin et al. (2020). Project finds a linear projection via logistic regression which minimizes the predictability of sensitive attributes in data. It requires an extra pre- processing step to project input data. 3. SenSR (Yurochkin et al. (2220). SenSR is based on DRO. It finds sensitive subspace through logistic regression which encodes the sensitive information most, and generates perturbations on this sensitive subspace during optimization. 4. SenSel (Yurochkin & Sun (021). SenSeI also uses the DRO paradigm, but involves distances penalties on both input and model predictions to construct perturbations; 5. LCIFR (Ruoss et al. (020)). LCIFR computes adversarial perturbations with logical constraints, and optimizes representations under the attacks from perturbations. We basically follow the default hyperparameter setting from the original imple- mentation but fine-tune some parameters to avoid degeneration in some cases. For our approaches, we use Anti to denote the approach that simply merges original data and antidote data, use 'Anti.' to denote adding Dis to original and antidote data, and use DRO-Anti to denote antidote data with DRO. We standardized basclines with the same base model in experiments.
4.2 How ANTIDOTE DATA MITIGATE UNFAIRNESS
We present our numerical results on Table I Table 2 and Figure I and defer more to Appendix From these results we have the following major observations.
Antidote Data Show Good Performance Across all datasets, with antidote data, our models mostly perform the best in terms of individual fairness, and with only minimal drop or sometimes even a slight improvement on predictive utility. For example, on Law School dataset, our AN+Anti mitigates individual unfairness by 70.38% and 63.36% in terms of the Mean in Pos. Comp. and Neg. Comp., respectively, with improvements on ROC by 0.47% and AP by 0.07%. On this dataset, other methods typically bring a 0.1 %-255% drop in utility, and deliver less mitigation on individual unfairness. In some cases, some baseline methods do give better individual faimess, e.g., LCIFR for Neg. Comp., but their fairness is not consistent for positive comparable samples, which is usually achieved at significant cost on utility (up to 13.03% drop in ROC).
Improvements from DRO-Anti Our DRO-Anti outperforms base models that learn with antidote data through regular optimizations. This model gets fairer results and slightly better predictive utility. This is because DRO-Anti introduces antidote data into every optimization iteration and selects the worst performed data instead of treating them equally. The typical DRO training has an

---

Under review as a conference paper at ICLR 2023
Table 2: Experimental results on Law School dataset
ROC 1
AP
Pos. Comp. (Mean/Q3)
Neg. Comp. (Mean/Q3)
I. (Base)
86.14
97.80
3.67 / 5.39
11.70/ 15.21
LR+Proj
85.84 0.35%
97.74
2.23 39.35% / 2,48 54.07%
8.66 25.9% / 11.40
LR+Dis
86.18
97.79
2.04
2.32 -56.90
7.33
11.25
LR+Anti
86.22 +0.8%
97.80
1.79 -51.00 2.20 -59.14%
6.56
8.64 43..1%
LR+Anti+Dis
86.20 v0.06%
97.80
-0.00%
1.76 52.08%
-59.96%
46.3%
8.36 45.00
NN (Base)
85.70
97.72
5.38 8.22
12.55
16.47
NN+Proj
85.89 40.22%
97.76 -0.4%%
2.04 -62.0% / 2.27 -72.39%
5.52 -6.0%%
6,46 40077%
NN+Dis
85.99
97.78
1.97 63336% / 2.22 -72.98%
5.34 57.42% 6,45
SenSR
84,49
97.55
2.58
3.23
5.56 55.68%
7.81 -2.57%
SenSel
84.59 -1.30%
97.49 -0.24%
7.01 +30.33%
10.83 +31.645
18.22 -45
24.99 451.729
LCIFR
74.53 133.03%
95.28 -2.50%
2.63 51,05% 3.06 -62,79%
3.35 -3.28% 3.78 -7707%
NN+Anti
86.11 +0.47%
97.79
+0.07%
1.59 -70.38%
1.94 -76.44%
4.60
46.36% 6.31 61.00%
NN+Anti+Dis
86.07 +0.13%
97.79
+0.06%
1.54
77131%
1.80 -78.05%
4.44 61.66%
5.47 66.78%
DRO-Anti
86.56 +1.0%%
97.88
+0.16%
1.52
-71.55% / 1.82 -77.82%6
4.10
67.34%
5.54
66.33%
0.40
0.40
Gap in Pos. Comp.
0.30
0.30
0.20
0.20
0.10
0.10
Anti+Dis
Proj
SenSR
SenSel
LCIFR
Anti
Anti+Dis
DRO
Gap in Neg. Comp.
0.20
0.20
0.10
0.10
0.00
0.00
LR
Proj
Dis
Anti
Anti+Dis
NN
Proj
Dis
SenSR
SenSel
LCIFR
Anti
Anti+Dis
DRO
ROC
0.70
0.65
Proj
Proj
Dis
SenSR
0.73
0.70
0.70
0.65
LR
Proj
Dis
Anti
Anti+Dis
NN
Proj
Dis
SenSR
SenSel
LCIFR
Anti
Anti+Dis
DRO
ROC
Figure I: Box plots for experimental results on Compas dataset. Experiments in the left three figures use Logistic Regression as the base model, and the right three figures use Neural Networks. The top two rows plot the results in individual fairness, while the bottom two rows plot the model's utility. Since we set two sensitive attributes for Compas dataset, we plot three situations for comparable sampley upon sensitive attributes for these two samples, and use logical expressions to denote them. We use 'and' to indicate none of the sensitive attributes is same between a pair of comparable samples, use "or" to denote at least one sensitive attribute is different, and use 'not" to indicate both two sensitive attributes are consistent. The dash line the box plots indicate the arithmetic mean.
iterative optimization in every epoch to search for good perturbations. In contrast, DRO-Anti omits the inner optimizations but only evaluates every antidote data in each round.
Binding well with Dis. Removing sensitive features from input data generally improves individ- ual fairness. In Law School dataset, discarding sensitive features can bring up 44.32% 63.36% mitigation in individual fairness. But once sensitive features are highly correlated with other fea- tures, the mitigation is not guaranteed. In Adult dataset. removing sensitive features only gets 0.93% 2.94% improvements across these two models. Regardless of the varying performance from Dis, antidote data bind well with sensitive features discarding. On Adult dataset, LR+Anti plus

---

Under review as a conference paper at ICLR 2023
0.88
0.88
0.88
0.86
Comparability Ratio
0.84
ROC
0.82
ROC
0.82
0.60
Discrete Continuous
0.80
0.80
Sensitive
0.40
0.78
SenSel
0.78
SenSel
LCIFR
LCIFR
0.76
Anti.
Anti.
0.20
DRO-Anti.
DRO-Anti.
0.74
0.74
0.00
0.30
0.20
0.10
0.15
0.10
0.05
100
200
300
400
500
Gap in Pos, Comp.
Gap in Neg. Comp.
Training Epoch
Figure 2: A & B: The tradeoffss between utilliy and fairness on Adult dataset. For SenSel we iterate the controlling hyperparameter in (le+3, 5e+3, le+4, 5e+4, le+5, 2e+5, Se+5). For LCIFR, we iterate the weight for fairness in (0.1, 1.0, 10.0, 50.0, 100.0). For Anti, we have the proportion of antidote ratio at 0%, 45%, 90%, 134%, 180%. 225%, 270%, 316%, 361% and 406%. For DRO- Anti, we have the proportion of antidote ratio at 45%, 90%, 136%, 180%, 225%. Every point is plotted with variances, and the variance for our models is too small to observe in this figure. C: The convergence terms of the comparability ratio during the training of the generator.
Dis boosts individual faimess in Pos. Comp. by 5.36%, where solely discarding sensitive features only has 0.94% improvements. This number consistent in NN, ie.,, 4.96% compared to 0.93%.
Algorithmic Tradeoff's In Figure 2 A & B, show the tradcoff's between utilitty and fairness. We have two major observations: (1) Models with antidote data perform better tradeoffs, i.e., with more antidote data, have lower individual unfairness, and less drop in model utility. DRO-Anti has the best tradeoffs and achieves individual fairness with an inconspicuous sacrifice of utility even when the amount of antidote data goes up. (2) Our models enjoy lower variance with different random seeds. For baseline methods, when we turn up the hyperparameters controlling the tradeoffs, there an instability the final results and a sigmificant variance. However, as our model optimized on approximately real data, and with no change on model from Anti. and minimal change in optimization from DRO-Anti, there is no observational variance in the final results.
Convergence In Figure C, we show the change of the comparability ratio, i.e., the rate of com- parable samples from the entire generated samples, during training for different types of features. The comparability ratio of sensitive features quickly converged to since we have direct supervision. The ratio of discrete and numerical features converged around the 500-th iteration due to the implicit supervision from the discriminator. The ratio of continuous features is lower than discrete features due to more complex patterns. Due to the imperfect comparability ratio, we add an additional step Post() to filter out incomparable samples.
4.3 MODELING THE DATA MANIFOLD
Compare to Randomly Generated Compa- rable Samples In Table we compare ran- domly generated comparable samples phasize the benefit of data manifold modeling. We sample the random comparable samples such: (1) Uniformly sample discrete features
Table 3: Comparing random generated compa- rable samples on Adult dataset
ROC f
Pos. Comp. (Mean/Q3) |
NN
88.18
33.21 / 47.84
+100.0% Rand.
88.25
31.33 / 44.69
+200.0% Rand.
88.18
30.16 / 42.51
and perturb them into a random value the
+300.0% Rand.
88.19
29.48 / 39.77
current feature. The total number of perturbed
+500.0% Rand.
88.08
27.94 / 39.31
features is arbitrary [0, Ta]. (2) Uniformly sample values from [-Tc. Tel. and add the per-
+44.5% 5 Anti
87.95
26.05 / 35.76
turbations to continuous features. We clip the perturbed features in [0, 1]. (3) Randomly perturb an arbitrary number of sensitive features. We add these randomly generated comparable samples to the original training set. From the results in T we observe that with only 44.5% antidote data, the model outperforms the one with 500% randomly generated comparable samples in terms of individ- ual fairness. By surpassing 10x data efficacy, the results demonstrated that modeling on-manifold comparable samples greatly helpful mitigate individual unfairness.
Learning Efficacy Antidote Data In Table we study the model binary classification perfor- mance by training only generated data. We use Accuracy (Acc.), Bal. Acc. (Balance Accuracy),
8

---

Under review as a conference paper at ICLR 2023
and F1 Score (F1) for evaluation. We construct a synthetic training set that has the same amount of data as the original training set. We use two baselines. Random Data: the randomly generated data fit the basic constraints from tabular. Pert. in SenSel: we collect perturbations from the original data in every training iteration of SenSel, and uniformly sample from these perturbations.
Within expectation, results in Table 4 show that our antidote data suffer from a performance drop compared to the original data because the generator cannot perfectly fit the data manifold. Even so, antidote data surpass random data and perturbations from SenSel, indicating that anti- dote data are closer to the original training data.
Table 4: Learning efficacy Adult dataset
Acc.
Bal. Acc. f
F1 f
Original Data
84.64
76.16
65.55
Random Data
30.48
40.25
29.59
Pert. in SenSel
53.81
67.83
50.36
Antidote Data
78.48
74.03
59.84
5 RELATED WORK
Machine Learning Fairness AI Fairness proposes ethical regulations to rectify algorithms not discriminating against any party or individual. To quantify the goal, the concept "group fairness' asks for equalized outcomes from algorithms across sensitive groups in terms of statistics like true positive rate or positive rate (Hardt et al. (016)) Similarly, minimax fairness (Hashimoto al. 2018) characterizes the algorithmic performance of the worst-performed group among all. Though appeal- ing, both of these two notions guarantee poorly on individuals. To compensate for the deficiency, counterfactual fairness (Kusner et al., 2017) describes the consistency of algorithms on one instance and its counterfacts when sensitive attributes got changed. However, this notion and corresponding evaluations strongly rely on the casual structure (Glymour et al.) 2016) which originates from the data generating process. Thus, in practice, an explicit modeling is usually unavailable. Individual fairness (Dwork et al. 2012) describes the pair-wise predictive gaps between similar instances, and it is feasible when the constraints in input and output spaces are properly defined.
Individual Fairness Several methods have been proposed for individual fairness. Sharifi- Malvajerdi et al.] (2019) study Average Individual Fairness. They regulate the average error rate for individuals on series of classification tasks with different targets, and bound the rate for the worst-performed individual. Yurochkin er al. (2020); Yurochkin & Sun (202T): b): [Ruoss et al. (2020); Yeom & Fredrikson (202I) develop models via DRO that iteratively optimized at samples which violate fairness at most. To overcome the hardness for choosing distance functions, Mukkerree et al. 2020) inherit the knowledge of similar/dissimilar pairs of inputs, and propose to learn good similar- ity metrics from data. [Ilvento (2020) learns metrics for individual fairness from human judgements, and construct an approximation from a limited queries to the arbiter. Petersen et al. (2021 I) propose graph smoothing approach to mitigate individual bias based on similarity graph. Lahoti et al.)
(2019) propose a probabilistic mapping from input to low-rank representations that reconcile indi-
vidual fairness well. To introduce individual fairness to more applications, Vargo et al. (2021) study individual fairness in gradient boosting, and the model is able to work with non-smooth models such as decision trees. [woork et al. (2020) study individual fairness in a multi-stage pipeline. Maity al.
(2011: John et al. (2020) study model auditing with individual fairness.
Crafting Adversarial Samples Beyond regular adversary (Madry et al. 2018), using genera- tive models to craft on-manifold adversarial samples is an attractive technique for model robust- ness (Xiao et al. 2018 Zhao al., 20188] Kos al. 20188] Song et al., 2018). Compared to general adversarial samples without too many data-dependent considerations, generative samples are good approximations the data distribution and can offer attacks with rich semantics. Experimentally. crafting adversarial samples is in accordance with intuition and has shown to boost model general-
ization capacity (Stutz et al. 2019 Raghunathan et al. 2019).
CONCLUSION
In this paper we studied individual fairness on tabular datasets, and focused on an individual fairness definition with rich semanties. We proposed an antidote data generator to learn on-manifold compa- rable samples, and used the generator to produce antidote data for the individual fairness purpose. We provided two approaches equip antidote data to regular classification pipeline or a distribu- tionally robust optimization paradigm. By incorporating generated antidote data, we showed good individual fairness as well as good tradeoffs between predictive utility and individual fairness.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. Machine bias. In Ethics of Data and Analytics, 2016.
Michelle Bao, Angela Zhou, Samantha A Zottola, Brian Brubach, Sarah Desmarais, Aaron Seth Horowitz, Kristian Lum, and Suresh Venkatasubrammnnia. It's COMPASlicated: The messy rela- tionship between RAI datasets and algorithmic fairesss benchmarks. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1), 2021. URL
htttps://openreval net forum? SeeerummidgggM5Bvviiii 8whnpxX
Solon Barocas and Andrew Selbst. Big data's disparate impact. California Law Review, 2016.
Robert Bartlett, Adair Morse, Richard Stanton, and Nancy Wallace. Consumer-lending discrimina- tion in the fintech era. Journal of Financial Economics, 2022.
Jeffiey Dastin, Amazon scraps secret ai recruiting tool that showed bias against women, 2018.
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of The 3rd Innovations in Theoretical Computer Science Conference, 2012.
Cynthia Dwork, Christina Ilvento, and Meena Jagadeesan. Individual fairness pipelines. In 1st Symposium Foundations of Responsible Computing, 2020.
Andrew Gelman, Jeffrey Fagan, and Alex Kiss. An analysis the new york city police department's "stop-and-frisk" policy in the context of claims of racial bias. Journal of the American statistical association, 2007.
Madelyn Glymour, Judea Pearl, and Nicholas P Jewell. Causal inference in statistics: A primer. John Wiley & Sons, 2016.
Ian Goodfellow, Jean Pouget-Abadie, Mchdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Infor- mation Processing Systems, 2014.
Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Im- proved training of wasserstein gans. In Advances in Neural Information Processing Systems, 2017.
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. In Advances in Neural Information Processing Systems, 2016.
Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographics in repeated loss minimization. In Proceedings of the 35th International Conference on Machine Learning. 2018.
Christina Ilvento. Metric learning for individual fairness. In /st Symposium on Foundations of Responsible Computing, 2020.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. In International Conference on Learning Representations, 2017.
Philips George John, Deepak Vijaykeerthy, and Diptikalyan Saha. Verifying individual fairness in machine learning models. In Conference Uncertainty in Artificial Intelligence, 2020.
Ronny Kohavi and Barry Becker. Adult data set, 1996.
Jernej Kos, Ian Fischer, and Dawn Song. Adversarial examples for generative models. In IEEE Security and Privacy Workshops, 2018.
Matt Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. In Advances in Neural Information Processing Systems, 2017.
10

---

Under review as a conference paper at ICLR 2023
Jakub Kuzilek, Martin Hlosta, and Zdenek Zdrahal. Open university learning analytics dataset. Scientific Data, 2017.
Preethi Lahoti, Krishna P Gummadi, and Gerhard Weikum. ifair: Learning individually fair data representations for algorithmic decision making. IEEE 35th International Conference on Data Engineering, 2019.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Reprexentations, 2018.
Subha Maity, Songkai Xue, Mikhail Yurochkin, and Yuekai Sun. Statistical inference for individual fairness. In International Conference on Learning Representations, 2021.
Jeffrey Mervis. Nsf grant decisions reflect systemic racism, study argues, 2022.
Debarghya Mukherjee, Mikhail Yurochkin, Moulinath Banerjee, and Yuekai Sun. Two simple ways to learn individual fairness metrics from data. International Conference on Machine Learning. 2020.
Fabian Pedregosa, Gaäl Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-learn: Machine learning in python. Journal of machine learning research, 2011.
Felix Petersen, Debarghya Mukherjee, Yuckai Sun, and Mikhail Yurochkin. Post-processing for individual fairness. In Advances Neural Information Processing Systems, 2021.
Aditi Raghunathan, Sang Michael Xie, Fanny Yang. John C Duchi, and Percy Liang. Adversarial training can hurt generalization. arXiv preprint arXiv:I906 06032, 2019.
Anian Ruoss, Mislav Balunovic, Mare Fischer, and Martin Vechev. Learning certified individually fair representations. In Advances in Neural Information Processing Systems, 2020.
Saeed Malvarri,,, Michael Kearns, and Aaron Roth. Average individual fairness: Algo- rithms, generalization and experiments. In Advances in Neural Information Processing Systems, 2019.
Yang Song, Rui Shu, Nate Kushman, and Stefano Ermon. Constructing unrestricted adversarial examples with generative models. In Advances Neural Information Processing Systems, 2018.
David Stutz, Matthias Hein, and Bernt Schiele. Disentangling ad adversarial robustness and generaliza- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.
Paul Van der Laan. The 2001 census in the netherlands. In Conference The Census Population, 2000.
Alexander Vargo, Fan Zhang, Mikhail Yurochkin, and Yuckai Sun. Individually fair gradient boost- ing. In International Conference on Learning Representations, 2021.
Linda F Wightman. Lsac national longitudinal bar passage study. LSCA Research Report Series, 1998.
Chaowei Xiao, Bo Li, Yan Zhu, Waren He, Mingyan Liu, and Dawn Song. Generating ad- versarial examples with adversarial networks. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, 2018.
Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante and Kalyan Veeramachaneni. Modeling tabular data using conditional gan. In Advances in Neural Information Processing Systems, 2019.
Samuel Yeom and Matt Fredrikson. Individual faimess revisited: transferring techniques from ad- versarial robustness. In Proceedings of the Twenty-Ninth International Conference on Interna- fional Joint Conferences Artificial Intelligence, 2021.

---

Under review as a conference paper at ICLR 2023
Mikhail Yurochkin and Yuckai Sun. Sensei: Sensitive set invariance for enforcing individual fair- ness. International Conference on Learning Representations, 2021.
Mikhail Yurochkin, Amanda Bower, and Yuckai Sun. Training individually fair ml models with sensitive subspace robustness. In International Conference on Learning Representations, 2020.
Zhengli Zhao, Dheeru Dua, and Sameer Singh. Generating natural adversarial examples. In Inter- national Conference on Learning Representations, 2018.
12

---

Under review as a conference paper at ICLR 2023
A DATASET
Adult dataset
The Adult dataset contains census personal records with attributes like age, educa- tion, race, etc. The task is determine whether a person makes over $50K year. We 45.25% antidote data for Anti, and 225.97% antidote data for DRO-Anti. We set Ta = 1 and To 0.025 for the constraints of comparable samples.
Compas dataset The Compas dataset is criminological dataset recording prisoners' information like criminal history, jail and prison time, demographic, sex, etc. The task to predict recidivism antidote data for Anti, and 184.89% antidote data for DRO-Anti. We set Ta = and T. 0.025. Note that from (Bao et al. 2021), Compas dataset may not be the ideal dataset for demonstrating algorithmic fairness.
risk score for defendants. We use 148 14.555
Law School dataset The Law School dataset dataset contains law school admission records. The goal is to predict whether candidate would the bar exam, with available features like sex, race, and student's decile, etc. We use 56. 18% antidote data for Anti, and 338.50% antidote data for DRO-Anti. We set Ta and T. 0.1.
Oulad The Open University Learning Analytics (Oulad) dataset contains information of students and their activities in the virtual learning environment for seven courses. It offers students' gender, region, age, and academic information to predict students' final results in a module mndule-presentation. We use 523.23% antidote data for Anti, and 747.8 14..85% antidote data for DRO-Anti. We set TĄ and T, = 0.025.
Dutch dataset The Dutch dataset dataset shows people profiles in Netherlands in 2001. It provides information like sex, age, household, citizenship, etc., and aim to predict a person's occupation. We remove 8,549 duplication in the test set and reduce the size 6.556. We use 205.449 antidote data for Anti, and 770.65% antidote data for DRO-Anti. We set Ta 1 and To 0.025.
Table 5: Dataset Statistics. We report data statistic including sample size as well as the number of positive and negative comparable samples in training / testing set, respectively.
Dataset
#Samplc
#Dim.
Sensitive Attribute
#Pos. Comp.
#Neg. Comp.
Adult
30,162 / 15,060
103
marital-status
739 193
38,826 10,412
Compas
4,626 / 1,541
354
racc + sex
24,292 / 2,571
8,116 1,020
Law School
15,598 / 5,200
23
race
13,425 1,530
1,068 118
Oulad
16,177 5,385
age
age_band band
33,747 3,927
5,869 608
Dutch
45,315/ 6,556
61
sex
1,460,028 6,727
1,301,376 9,390
B IMPLEMENTATION DETAILS
We elaborate the architecture of our model details by using the hidden representations.
h Rett///anccheemmmcl.laaaeeesceee ez
h2 Reti///accchhermmlc((leseesssss) hn
hg = Rett///acchhormmllllnnneeeeeee (h2))
9o
0 tanh(ttineerr- for e]]) 4 5 S No
é; gumbell. (hnddeiex lindex for K())) AA is No
d = gumbelo. (handdixxx for d())) VOsisM, Na
h = DeepouteggLaayyaaUUliCCLnne,ee,, 3i0 a)))
dg
h2 = DropoutegLLaayyeelllllinieenssss score Linear-1(hh)
We use Adam optimizer. We set the learning rate for generator go 2e-4, for discriminator de to 2e-4, weight decay for go to le-6, for de to O. We set batch size to 4096 and training epochs to 500.
13

---

Under review as a conference paper at ICLR 2023
C ADDITIONAL RESULTS
We present experimental results on Dutch dataset in Table and on Oulad dataset in Table 7 and tradcoff's study in Figure 3 Similar conclusions can be drawn in Section 4.2 with antidote data, our models Anti and DRO-Anti achieve good individual faimess and favorable tradeoff's between fairness and model predictive utility.
Table 6: Experimental results on Dutch dataset
ROC
1
Pos. Comp. (Mean/Q3) 4
Neg. Comp. (Mean/Q3) 4
LR (Base)
89.55
87.87
17.84 / 24.15
21.81 / 30.29
LR+Proj
86,62 -3.28%
85.13 -3.13%
7.74 56.00% / 8.21
8.01 -63.30% 8.55 -71.79%
LR+Dis
87.51 -2.28%
85.71 -2.4%%
8.44 -52.67% 9.003 66.63%
9.11 -5.22% 11.29 .62.72%
LR+Anti
85,41 -1.63%
83.38
-5.11%
9.55 46.47% 10.37 -57.05%
10.74 -50.77% 12.70 -58.09%
LR+AntiDDis
87.40 -2.40%
85.51
-2.09%
7.08 66.32% / 7.06 -70.66%
7.10 -67.44% / 7,48 -75.31%
NN (Base)
90.22
88.93
15.88/ 20.85
21.42/ 31.68
NN+Proj
88.18 -2 2696
86.94 -2.23%
8.1 -4895% / 19.44 -54 75%
7.65 -64.99% 9.73 49 28.28%
NN+Dis
88,21 .2.2%
86,92 -2.25%
8.18 -18.5% / 19.41 -54.8%%
8,18 66.80% / 10.53 666.76%
SenSR
87.78 -2,70%
86.68 -2.52%
8.54 -66.0% / 19.71 -53.46%
7.72 -6394% 8.61 -72.83%
SenSel
89.91 -0.34%
88.34 .066%%
16.21 +2.0% 21.12 +1.29%
21.98 +2.65% / 31.25 -1.35%
LCIFR
88,04 -2.42%
86.54
8.12 / 9930
8.41 60.73% / 10.61 -66.50%
NN+Anti
87.05 -3.51%
85,59 -3755%
8.71 145.13% 10.50 -19.67%
9,49 55.70% 13.23 58.23%
Ai+Anti+Dis
87.80 -2 6866
86.37 -288%
6.78 -5.300 / 7.37 64.65%
6.32 -70.88% 7.15 -37.44%
DRO
88.00 2..1%%
87.13 -2.00%
6.34 60.01% / 6.06 -70.93%
4.91 77.00%
5.41 -82.9%%
Table 7: Experimental results on Oulad dataset
ROC
AP
Pos. Comp. (Mean/Q3)
Neg. Comp. (Mean/Q3) )
LR (Base)
63.04
76.73
8.41 / 12.09
9.08 / 12.97
LR+Proj
65.20 +3.41%
79.29
+3.34%
5.33 36.61% / 7.50 -37.9%
5,46 29.88% / 7.69
LR+Dis
62.52 -0.8%
76.39
5.42 .35.00 7.89 -34.7%%
5.89
/ 8.74
LR+AAnti
62.17 -1.38%
76.24 0.64%
23.61% 99.19 -24.0%%
7.10 21.76% 1.778 -2463%
LR+AAti+Dis
60.82 -3.52%
75.07
1717
5.10 -39311 6.95 -42,33%
5.81 -36.45 / 8.67 -33.178
NN (Base)
65,80
79.72
6.63 9955
6.81 /9.73
NN+Proj
65,42 -0.57%
79.49 -0,29%
4.76 28.266
6.70 -0011%
4.65 -31688 / 6.71 -31.00%
NN+Dis
65.51 .043%
79.59
Jn. 1696
4.78 -27.94%
6.84 -28.75%
4.75 27% / 6.94 -28.66%
SenSR
65,58 -0.3%%
79.57 0.19%
4.96 25.17% / 7..0 -7.08%
4,23 -3785% / 6.16 -36.77%
SenSel
64.14 -2.52%
78.68
-1.31%
5.53 -16.9%% 8.13 -15.22%
5.50 .19 18% 7.99 -17
LCIFR
65.21 -0.89%
79.40
0.40%
4.13 37.611 / 5,66 -41.01%
3,70 45.70% / 5.26 45594%
Anti
64.75 -1.59%
79.08
0.80%
4.09 33.32% 5.82 -30
4.51 60% / 6.30 235
Ai+
64.97 -1.26%
79.20 00.6%%
4,00 -39.70% / 5.53 44.,33%
4.18
/ 5.97 .3.61%
DRO
64.38 2.16%
78.62 -1.38%
2.86
-5.800%
3.71 -61.34%
3.97
-11.44 5.22
46631%
0.70
0.70
0.60
ROC
0.60
0.55
0.55
SenSel
LCIFR
LCIFR
0.50
0.50
Anti.
ARO-Anti.
A
DRO-Anti
0.00
Gap in Pos. Comp.
Gap in Neg. Comp.
Figure 3: The tradeoff's between utility and fairness Compas dataset. For SenSel iterate the controlling hyperparameter in (1e+3, 5e+3, le+4, 5e+4, le+5, 2e+5, 5e+5). For LCIFR, we iterate the weight for fairness in (0.1, 1.0, 10.0, 50.0. 100.0). For Anti, have the proportion of antidote ratio at 10%, 130%, 1 150%, 167%, 185%. 206%. For DRO-Anti, we have the proportion of antidote ratio 129%. 146%, 167%. 184%, 201% %, 222%. Every point is plotted with variances.
14


Reviews:
[{"review_id": "zxygBYlXo_2", "reviewer": "Reviewer_pdNt", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose a technique for improving distributionally robust techniques for individual fairness by generating antidote samples. These adversarial samples are constrained to be on the manifold of the data distribution. The antidote data generator is a generative adversarial network (GAN) model that takes samples from data distribution and generates comparable samples, where their non-sensitive attributes differ within some thresholded values, whereas their sensitive attributes could differ arbitrarily. The authors then incorporate the generated antidote data into a DRO optimization by forcing the optimization objective not only to optimize the loss w.r.t. original data but also a wors-case loss w.r.t. generated antidote data. Finally, the authors demonstrate the benefit of the model in several public fairness datasets.", "summary_of_the_review": "It's an interesting paper overall. But I have some concerns about novelty and quality. Therefore, I recommend weak rejection.", "strength_and_weaknesses": "Strengths:\n- The technique makes sure that the adversarial samples do not fall far from the data manifold.\n- Interesting GAN-based data generators.\n- The antidote technique is applicable to both DRO and non-DRO models.\n- The experiments demonstrate the benefit of the model compared to baselines.\n\nWeakness:\n- The proposed method looks like a variation of data augmentation methods, whereby the samples can be perturbed freely within a boundary without changing the label. The perturbation scheme makes sure that non-sensitive attributes are kept in check, i.e., they should not fall too far from the original data. However, the sensitive attributes can change without any restriction. \nTo me, this construction looks more similar to a group fairness constraint rather than individual fairness. The groups are represented by the discrete values in the sensitive attributes. The data augmentation scheme suggests the model outputs the same prediction regardless what the value of the sensitive attributes. This is very similar to the concept of demographic parity in group fairness, where the probability of positive prediction should be equalized across multiple groups. \n- The metrics chosen for the comparison are different than the metrics used in previous works (e.g., Yurochkin et al. (2020)). For example, the authors use AP rather than the standard accuracy metric for the performance metrics and the positive/negative comparable samples for the fairness metrics. As the 'positive/negative comparable samples' are specifically designed for the 'comparable' concept presented in the paper, this may give an unfair advantage to the proposed model compared to the baselines. I suggest the authors use standard metrics for the evaluation, such as the one used by Yurochkin et al. (2020), for a better comparison with baselines.\n- In Table 3, DRO-Anti achieves the worst ROC result. Why is it marked as bold?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "nQjjrRWADMb", "reviewer": "Reviewer_rYKn", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors proposed a data-augmentation method to improve a measure of individual fairness in classification. They propose to learn an \"antidote data generator\" using GANs, which is intended to learn \"comparable\" samples to any given sample from another sensitive class. This data could then be added directly to the training data or incorporated using a form of DRO. Experimentally, they show that this provides better results in some individual fairness metrics, without losing too much predictive utility.", "summary_of_the_review": "I think this paper is probably not quite there in terms of the clarity of the ideas and novelty. Due to these issues, the experimental section, which is pretty thorough, doesn't quite have the impact that I think it could. As such, I'm recommending rejection.", "strength_and_weaknesses": "Strengths:\n- generative data approaches to individual fairness questions are sensible and promising\n- experimental evaluation is pretty extensive and thorough\n\nWeaknesses:\n- The notion of \"comparable samples\" feels a little bit off to me, and may not capture some important cases. For instance, there could be some non-sensitive feature which nonetheless correlates strong with a sensitive feature, and when trying to find a comparable sample for a given sample, we may have to change that feature by a large absolute amount. However, this type of change is disallowed by Def 2.1, and in my opinion, really restricts the cases that we could apply this model to.\n- The setup around the notation and \"comparable samples\" definition feels a little bit ad-hoc, and doesn't totally define some cases. what if we have ordinal features? are they considered continuous or discrete in this framework? are sensitive attributes s continuous or discrete? In Def 2.1: not sure why the discrete features are constrained by a sum, but the continuous features are constrained by a max\n- I don't quite understand all the choices made in the data generating setup. For instance, if we're using GANs I don't understand why we first need to model the continuous data as a mixture of Gaussians - there are some implicit assumptions here that I think should be stated. I'm also not sure why you can't guarantee that your outputs will be comparable according to Def 2.1 by, for instance, clipping the output to the required range.\n- some experimental details are a little bit unclear to me - I can't quite follow exactly where only real samples are used, where only generated samples are used, and where a mixture is used. It would be good to have clear delineation of these regimes. Also, it seems that some detail/hyperparameters are missing around the antidote generation - for instance how many comparable samples are generated at each DRO step.\n- some confusion in the additional experiments: 1. I can't find the comparability ratio defined anywhere 2. I'm not sure why you would need an additional Post(x) step\n\n\nNotes:\n- the paper discusses IF only in terms of sensitive attributes (see the abstract), but there is a large literature which discusses it without reference to sensitive attributes (e.g. https://arxiv.org/pdf/1905.10607.pdf) - would be good to note this early on, in the introduction\n- bottom of p1: \"primary use case of DRO in model robustness is to adversarially perturb a sample by a small degree\" - don't agree with this statement, there is a long line of work in statistics, recently popular in ML (particularly see John Duchi + collaborators' work eg. https://arxiv.org/abs/1810.08750) which uses a different approach to DRO based on perturbing distributions rather than samples\n- the inequality in the last line of Eq 8 loses me, I think a little bit more explanation would help\n- in general, I recommend against using the COMPAS dataset - it's not a great dataset for demonstrating points about algorithmic fairness - see https://openreview.net/forum?id=qeM58whnpXM for some commentary\n- it would be good to list the hyperparameters which are \"default for other methods\" in your paper itself, just so it's self contained.\n- a graphical representation of Table 1 might be helpful\n- Figure 1 and 2 would benefit from having a black-and-white friendly visualization\n- a group fairness type baseline - e.g. post-processing, would be interesting here, just to note that it isnt' helpful for your objective\n- bottom of p8 - total number of pertubed features should be in (0, T_d)\n- bottom of p8 - adding noise from [-Tc, Tc] seems a lot, wouldn't this result in a lot of extreme values?\n- the \"Learning Efficacy\" experiment is interesting, but I'd be very interested to see what happens as you add antidote data to the original dataset", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "CDpLF8iUGJ", "reviewer": "Reviewer_rsds", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper presents a method for learning a GAN for tabular data that performs conditional generation with semantic constraints for the structural properties in the tabular dataset. This model is used to generate perturbed samples that are, it is argued, intuitively performing a similar role to the perturbations in distributionally robust optimization. These models are created for three standard fairness datasets (Adult, COMPAS, Law School) and empirical results are given.", "summary_of_the_review": "Tabular data in particular needs more attention in the robustness literature, and I believe in the generative modeling literature as well, and it is good to see the authors' effort in this direction. I have several concerns about the paper in its current form. It seems that only a narrow slice of the DRO literature is discussed, leaving some more mainstream DRO methods out of the work entirely. Well-known effective baselines for tabular data are also not discussed in the work or compared against in the experiments. It is also not clear whether the proposed approach is really equivalent to \"robust optimization\" at all. The experimental results are a bit difficult to parse due to differences across datasets, but seem inconclusive. Finally, I have some reproducibility concerns. Detailed comments above.", "strength_and_weaknesses": "\n## Major Comments\n\n* The paper is missing discussion of, and comparison against, many of the more canonical works in the DRO literature. It seems to me that all of these should be discussed in the paper (they are not) and more importantly, compared against. These papers present formal approaches to solving the DRO problem and would be important baselines for the current work (which claims to be \"similar to DRO\" but is in fact not DRO, since there is no notion of worst-case distributional shift being solved for here -- I think the connection to DRO here is not entirely clear, it seems more like adversarial training). For example:\n\n- Sagawa, Shiori, et al. \"Distributionally Robust Neural Networks.\" International Conference on Learning Representations. 2019.\n\n- Levy, Daniel, et al. \"Large-scale methods for distributionally robust optimization.\" Advances in Neural Information Processing Systems 33 (2020): 8847-8860. (This paper has a nice git repo with implementations.)\n\n- Namkoong, Hongseok, and John C. Duchi. \"Stochastic gradient methods for distributionally robust optimization with f-divergences.\" Advances in neural information processing systems 29 (2016).\n\n* Since the paper is explicitly focused on tabular datasets, it should compare against *strong* baselines for tabular data. This is widely considered to be methods such as XGBoost and LightGBM (see e.g. below):\n\n- Shwartz-Ziv, Ravid, and Amitai Armon. \"Tabular data: Deep learning is not all you need.\" Information Fusion 81 (2022): 84-90.\n\n- Grinsztajn, Léo, Edouard Oyallon, and Gaël Varoquaux. \"Why do tree-based models still outperform deep learning on tabular data?.\" arXiv preprint arXiv:2207.08815 (2022).\n\nI will note that *any* of these tabular methods, it seems, could be used alongside \"antidote\"-augmented datasets, and an augmented dataset should ideally be able to achieve improved fairness without degrading the accuracy of the best possible model.\n\n* A major issue with GAN-style models, and DRO methods, is both are extremely sensitive to hyperparameters. Combining them seems an even more fragile exercise -- please clearly describe the process for hyperparameter tuning/selection (not just the final values, given in B) and what checks were performed to ensure that the results were not sensitive to these parameterizations. This will also be important when the canonical DRO methods described above are used.\n\n* Similarly, it is not clear at all how the percentages of antidote data were chosen: Why do you use 45.25% antitode data for Anti, 225.97% for DRO-Anti on Adult, and 56.18% Angi/338.50% on Law School? How sensitive are the results to these values?\n\n\n## Minor Comments\n\n* It seems that different sets of results are presented for each dataset (besides the tables). For example, Fig. 1 cocntains extra results for COMPAS, Fig. 2 shows a different set of results for Adult. It would be really nice to have identical plots, for all three datasets, side by side to compare.\n\n* From Figure 1, it looks like none of the methods have significant differences. \n\n* I found the introduction very difficult to read, I would suggest revising for clarity.\n\n## Typos etc.\n\nThere are several typos in the paper. Here are a few.\n\nP2: \"impracticable\"\n\nP2: \"may correlated with\"\n\nP2: \"are come to be comparable\"\n\nP3: \"by given an original training sample\" -> given an original training sample\n\nP5: \"censual\" - I am not sure this is a word\n\n* The shading in the tables is very difficult to see.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "7IQn0-EgMFS", "reviewer": "Reviewer_Q48M", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a method to learn and generate \"antidote data\", which are comparable samples, to resist individual unfairness at minimal cost to model predictivity. The antidote data is generated using a GAN type generator. The paper discusses applications to various datasets and observes the effect of antidote data on fairness for specific sensitive data.", "summary_of_the_review": "Overall, the paper is interesting but still has shortcomings in its quality and clarity; please find detailed comments in the above sections.", "strength_and_weaknesses": "Some comments on strengths of the paper:\n1. The paper addresses an important topic of fairness within machine learning.\n2. The paper contains good examples of the importance of fairness, as well as good comments on why certain practices in the literature may fail to generate adversarial samples that remain on the data manifold.\n\nSome comments on weaknesses of the paper:\n1. Overall, the clarity of the paper can be improved. Many technical details in Section 3 were vague or unclearly defined, see points listed below in 3. for some comments. \n2. How are the thresholds Tc and Td chosen for the experiments? Are the results consistent for a wide selection of different threshold values? The numbers shown in Appendix A seem to be quite arbitrary --- are the reported numbers a consequence of choosing a \"better\" set of thresholds?\n3. Some definitions and wordings are unclear and may benefit from a revision. For example:\n  - definition 2.1 might benefit from a rephrasing emphasizing \"comparable samples given thresholds Td and Tc\".\n  - In Equation (3), the notions of $\\bar{s}$ and $s_{\\hat{x}}$ are not immediately clear, although one can trace back to it after reading page \n    4 to have a better idea. It may be helpful to include clarifications of definitions early on to aid clarity.\n  - The $\\oplus$ operator used in (4), (5) and (6) is not explained and causes confusion when understanding the algorithms in page 4.\n  - It would also be helpful to be consistent in notation, for example, variations such as \"DRO-Anti\" and \"Dro-Anti\" are repeatedly seen.\n  - The small term $\\delta$ is not clearly defined in (8).\n4. What do the experiment numbers look like for other sensitive features? Are there any intuitive explanations to why LCIFR is better in terms of Neg. Comp. on the Law School dataset?\n5. It would be helpful to add discussions in the main text to clarify how the generated data using the algorithms of Section 3.1 are \"on-manifold\": from the paragraph right above Section 3.1, a vague logic/goal of \"the generated data should fit into existing data manifold and obey the inherent feature correlations or innate data constraints\" is stated. Can this be elaborated? More precisely, why would \"comparable samples\" fit into \"existing data manifold\" for general datasets?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject