Sample Selection for Fair and Robust Training
Authors: Yuji Roh
Kangwook Lee
Steven Euijong Whang
Changho Suh
Venue: NeurIPS 2021 Poster (2021)
Keywords: trustworthy AI
fairness
robustness
sample selection

Abstract: Fairness and robustness are critical elements of Trustworthy AI that need to be addressed together. Fairness is about learning an unbiased model while robustness is about learning from corrupted data, and it is known that addressing only one of them may have an adverse affect on the other. In this work, we propose a sample selection-based algorithm for fair and robust training. To this end, we formulate a combinatorial optimization problem for the unbiased selection of samples in the presence of data corruption. Observing that solving this optimization problem is strongly NP-hard, we propose a greedy algorithm that is efficient and effective in practice. Experiments show that our method obtains fairness and robustness that are better than or comparable to the state-of-the-art technique, both on synthetic and benchmark real datasets. Moreover, unlike other fair and robust training baselines, our algorithm can be used by only modifying the sampling step in batch selection without changing the training algorithm or leveraging additional clean data.

Sample Selection for Fair and Robust Training
Yuji Roh KAIST yuji. rogiirro@kaist.cco kc.
Kangwook Lee
University of Wisconsin-Madison kangwook leewwisc.edu
Steven Euijong Whang" KAIST
Changho Suh KAIST chsuhōkaist kr
swhang@kaist.ag ac.kr
Abstract
Fairness and robustness are critical elements Trustworthy AI that need to be addressed together. Fairness is about learning unbiased model while robustness is about learning from corrupted data, and it known that addressing only one them may have adverse affect the other. In this work, propose sample selection-based algorithm for fair and robust training. To this end, we formulate a combinatorial optimization problem for the unbiased selection of samples in the presence of data corruption. Observing that solving this optimization problem is strongly NP-hard, we propose greedy algorithm that efficient and effective in practice. Experiments show that our algorithm obtains faimess and robustness that are better than or comparable the state-of of-the-art technique, both on synthetic and benchmark real datasets. Moreover, unlike other fair and robust training baselines, our algorithm can be used by only modifying the sampling step batch selection without changing the training algorithm or leveraging additional clean data.
1 Introduction
Trustworthy Al is becoming essential for modern machine learning. While traditional focus of machine learning is to train the most accurate model possible, companies that actually deploy AI including [Mirrosofi] [2021] Google [2020], and IBM (2020) are now pledging to make their systems fair, robust, interpretable, and transparent well. Among the key objectives, we focus on fairness and robustness because they are closely related and need to be addressed together. Fairness is about learning an unbiased model while robustness is about learning resilient model even noisy data, and both issues root from the same training data. A recent work (Roh et al. 2020) shows that improving fairness while ignoring noisy data may lead to worse accuracy- cocuracy-fiimess tradcoff. Likewise, focusing on robustness only may have an adverse affect on fairness as we explain below.
There are several possible approaches for supporting fairness and robustness together. One is an in- processing approach where the model architecture itself modified optimize the two objectives. In particular, the state-of-the-an -ft-t---tth-tttt-e approach called FR-Train (Roh et al. 2020 uses adversarial learning to train a classifier and two discriminators for fairness and robustness. However, such an approach requires the application developer to use a specific model architecture and is thus restrictive. Another approach is to preprocess the data Kamiran and Calders 2011] and remove any bias and noise before model training, but this requires modifying the data itself. In addition, data cleaning method that designed to explicitly improve model fairness and robustness together.
not aware of general
Instead, we propose an adaptive sample selection approach for the purpose of improving fairness and robustness. This approach can easily be deployed as batch selection method, which does not require
"Corresponding author
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

1.5
%
V1
By
12.9 %
1 NO
40.0 %
2 1
Loss
1.0
X1
6.8 %
4g: YO
40.3 %
0.5
3o
500
1000
1500
2000
2500
3000
Epochs
(a) The size ratios of (label y, sensitive group 2) sets within the COMPAS dataset.
(b) Tracing the model losses on (label y. sensitive group sets when training logistic regression on the COMPAS dataset.
Figure 1: The ProPublica COMPAS dataset (Angwin al. 2016 has bias where the set (y = 1, 1) has the smallest size in Figure la. However, when training a logistic regression model on this dataset, the same set has the highest loss every epoch as shown in Figure 1b. Blindly applying clean selection at any point will thus discard (y = 1, z = 1) samples the most, which results in even worse bias and possibly worse fairness.
modifying the model data. Our techniques build on top of two recent lines of sample selection research: clean selection for robust training [Rousseeuw 1984, Song et al. 2019 Shen and Sanghavi 2019) and batch selection for faimess (Roh et al.] 2021 Clean selection is standard approach for discarding samples that have high-loss values and are thus considered noisy. More recently, batch
selection for fairness has been proposed where the idea is to adaptively adjust sampling ratios among sensitive groups (e.g., black or white populations) so that the trained model is not discriminative.
We note that state-of-the-ard clean selection techniques like Iterative Trimmed Loss Minimization (ITLM) [Shen and Sanghavi] 2019] are not designed to address fairness and may actually worsen the data bias when used alone. For example, Figure shows how the ProPublica COMPAS dataset (used by U.S. courts to predict criminal recidivism rates [Angwin et al. (0166) can be divided into four sets where the label y is either 0 or 1, and the sensitive attribute is either 0 or 1. The sizes of the sets biased where (y = 1, = 1) is the smallest. However, the model loss on (y = 1, z = 1) is the highest among the sets shown in Figure Tb Hence, blindly applying clean selection to remove high-loss samples will discard (y 1, 1) samples the most, resulting in a worse bias that may negatively affect fairness
demonstrate in Sec. [2]
We thus formulate a combinatorial optimization problem for fair and robust sample selection that achieves unbiased sampling in the presence of data corruption. To avoid discrimination on a certain group, we adaptively adjust the maximum number of samples per (y. 2) set using the recently proposed FairBatch system [Roh et al. 2021], which solves bilevel optimization problem of unfairness mitigation and standard empirical risk minimization through batch selection. We show that our optimization problem for sample selection can be viewed as a multidimensional knapsack problem and is thus strongly NP-hard [Garey and Johnson| 1979]. We then propose an efficient greedy algorithm that is effective in practice. Our method supports prominent group fairness measures: equalized odds [Hardt et al. 2016) and demographic parity [Feldman et al. (2015). For data corruption, currently assume noisy labels (Song al. 2020), which can be produced by random flipping or adversarial attacks.
Experiments synthetic and benchmark real datasets (COMPAS (Angwin et al. 2016 and Adult- Census [Kohavi (996)) show that our method obtains accuracy and fairness results that are better than baselines using fairness algorithms [Zafar et al. 2017a b Roh et al.) 2021 and on par with and sometimes better than FR-Train [Roh et al. 2020 which leverages additional clean data.
Notation Let 0 be the model weights, X be the input feature to the classifier, y E Y be the true class, and ý be the predicted class where ý a function of (x, 0). Let E Z sensitive attribute, e.g., race or gender. Let d y) be training sample that contains feature and label. Let S be a selected subset from the entire dataset D, where the cardinality D is n. In the model training. we use loss function (-(-) where smaller value indicates more accurate prediction.
2 Unfairness in Clean Selection
We demonstrate how clean selection that only focuses on robust training may lead to unfair models. Throughout this paper, use ITLM (Shen and Sanghavi 2019 as representative clean selection

---

ITLM
ITLM
0.3
ITLM
Accuracy
0.5
B0 Dispo
0.3
DP Disp.
LR
LR
LR
0.4
0.1
10%
1526
27%
10%
1856
20%
25%
10%
18%
27%
25%
Noise Rate
Noise Rate
Noise Rate
Accuracy.
(h) Equalized odds disparity-
(c) Demographic parity disparity.
Figure 2: Performances of logistic regression with and without ITLM (ITLM and LR, respectively) on the ProPublica COMPAS dataset while varying the noise rate using label flipping (Paudice et al.) 2018). Although ITLM improves accuracy (higher is better), the faimess worsens according to the equalized odds and demographic parity disparity measures (lower is better).
method for its simplicity. although other methods can be used as well. We first introduce ITLM and demonstrate how ITLM can actually worsen fairness.
ITLM Cleaning samples based on their loss values is traditional robust training tech- nique [Rousseeuw] 1984) that is widely used due its efficiency. Recently, ITLM was proposed as scalable approach for sample selection where lower loss samples are considered cleaner. The following optimization problem is solved with thcoretical guarantees of convergence:
Σ (s(s))
min S
S
where S is the selected set, s; a sample in S, and 7 the ratio of clean samples in the entire data.
Potential Unfairness We show how applying ITLM may lead to unfairness through a simple experiment. We use the benchmark ProPublica COMPAS dataset [Angwin et al. 2016] where the label y indicates recidivism, and the sensitive attribute z is set to gender. We inject noise into the training using label flipping techniques [Paudice et al., 2018). For simplicity, ITLM knows the clean ratio 7 of the data and selects [Tn] samples. We train a logistic regression model on the dataset, and ITLM iteratively selects clean samples during the training.
assume that
show the accuracy and fairness performances logistic regression with and without ITLM while varying the noise rate (i.e., portion of labels flipped) in Figure 2 As expected, ITLM increasingly improves the model accuracy for larger noise rates as shown Figure 2a At the same time, however, the fairness worsens according to the two measures use in this paper (see definitions in Sec.
equalized odds (Hardt et al. 2016) disparity and demographic parity [Feldman et al., 2015) disparity 2c respectively. The results confirm our claims in Sec.
shown in Figures 2b and
3 Framework
We provide an optimization framework for training models on top of clean selection with fairness constraints. The entire optimization involves three joint optimizations, which explain in piecewise fashion: (1) how to perform clean selection with fairness constraints, (2) how to train a model on top of the selected samples, and (3) how generate the faimess constraints. In Sec. 4 we present a concrete algorithm for solving all the optimizations.
3.1 Clean Selection with Fairness Constraints
The first optimization is to select clean samples such that the loss is minimized while fairness constraints satisfied. For now, assume that the faimess constraints are given where the selected samples certain set (y = y. the class y = y. Notice that Σ, A(y.s) = 1, Vy E Y. The fairness constraints are specified as upper bounds because using equality constraints may lead to infeasible solutions. We explain how the X
2) cannot exceed portion Aty.a) of the selected samples of

---

values are determined in Sec.
3.3
We thus solve the following optimization for sample selection:
min
Lo(ds) Pi
(1)
s.t.
Pi S Tn
Σ Ps S (v. Syl, ]E
(2)
Pr {0. 1), = 1,...,
where ps indicates whether the data sample d, is selected not, (gy.)) is an index set of the (y,z) 2) class, and Sy is the selected samples for constraint comes from ITLM while the second constraint is for the fairness.
Note that |Syl = Ejet, Pj- In Eq. 2 the first
An important detail is that we use inequalities all of our constraints, unlike ITLM. The reason is that we may not be able to find a feasible solution of selected samples if we only use equality constraints. For example, ((,0)) X(1.1) 0.5, but we have zero (y = 1, 0) samples, then the fairnsss equality constraint for (y = 1, z = 0) can never be satisfied.
3.2 Model Training
The next optimization is to train a model on top of the selected samples from Sec. 3.1 the sample selection optimization only uses incquality constraints, which means that for some set (y = y, Z = 2), its size [S(ye) may be smaller than \(y,=)|S,), However, we would like to train the model as if there were A(g,z)|5 [Syl samples for the intended fairness performance. We thus reweight
Recall that
ASy and perform weighted empirical risk minimization (ERM) for model training. We then solve the bilevel optimization of model training and sample selection in Sec. as follows:
each set (y y, = 2) by [S(y.ay]
min Σ
Σ
T ( (8:)
lo(s:)
(3)
(((
S,ESx
(3,
Sx =
arg min S [d:1p=1)::a.
Eq.
(4)
2
where À is the set of A(y.2) for all Z. YEY.
3.3 Fairness Constraint Generation
To determine the Xy. values, we adaptively adjust sampling ratios of sensitive groups using the techniques in FairBatch [Roh et al. 2021]. For each batch selection, we compute the fairness of the intermediate model and decide which sensitive groups should be sampled more (or less) in the next batch. The sampling rates become the A(g.=) values used in the fairness constraints of Eq. 2
FairBatch supports all the group fairess measures used in this paper. As an illustration, we state FairBatch's 's optimization when using equalized odds disparity (see Sec. A.1 for the optimization using demographic parity disparity):
min max ((y. L(g. 2 # 1',,,,, 2, E y Y
(5)
where (yy=) 1/1Sx(r.=)l Esxe.., (s(s;) and Sx(y.=) is subset of S from Eq. for the
y. 2 = 2) set. FairBatch adjusts À each epoch via a signed gradient-based algorithm the direction of reducing unfairness. For example, if the specific sensitive group is less accurate the samples in the epoch (i.e., z is discriminated
terms of equalized odds), FairBatch selects more data from the (y y, z 2) set to increase the model's 's accuracy that set:
(t)) = (a.2) sign (()==') L(y,)), # z'
(6)
where a the step size of updates. To integrate FairBatch with our framework, we adaptively adjust À as above for an intermediate model trained on the selected set Sx from Eq.

---

Algorithm 1: Greedy-Based
Algorithm 2: Overall Fair and Robust Model Training
Clean and Fair Sample Selection
Input: train data (Tirains Itrain); clean ratio т, loss function lo(:)
Input:
loss (o))), fair ratio
lambda X, clean ratio 7
d t (Frrain Mrrain)
0 initial model parameters
p (c) (o(
{A 1y E Y, E 2 +- random sampling ratios
sortIdx = (pro(profit)
for each epoch do
S S-11
= Algorithm((ff(d) 2, Ty)
for idx in sortldx do if diax does not violate any constraint in Eq. 8 w..... X and r then Append data S O
Draw minibatches from S w.r.t. (0,21Sy/||((u.s)
for each minibatch do
Update model parameters 0 according the minibatch
Update VAy, E À using the update rule as in Eq.
Output model parameters 0
4 Algorithm
We present our algorithm for solving the optimization problem in Sec. 3 We first explain how the clean selection with fairness constraints problem in Sec. 3.] can be converted to multi-dimensional knapsack problem, which has known solutions. We then describe the full algorithm that includes the three functionalities: clean and fair selection, model training, and fairness constraint generation.
The clean and fair selection problem
be converted to equivalent problem by (1) maximizing the sum of (max(f( - (d(d)) instead of minimizing the sum of (d4)'s where d is the set of d,' 's and (2) re-arranging the fairness constraints that the right-hand side expressions become constants (instead of containing the variable Syy as follows:
knapsack
max (max(ee - Lo(d:)) Pb
(7)
if & Dy if Dy and D2, e
s.t. DEIPi S Ex= WiP: S Tn, where ui
Ayyss)
2 A(a
if d, D(y,))
(8)
ps E (0,1), i == 1,..., n
where D(y.z) is a subset for the (y,2) 2) class. More details on the conversion are in Sec. A.2.
The multi-dimensional knapsack problem is known to be strongly NP-hard Garey and Johnson 1979]. Although exact algorithms have been proposed, they have exponential time complexi- ties [Klllerer et al. [004]] and are thus impractical. There is polynomial-time approximation scheme (PTAS) [Caprara et al., 2000), but the actual computation time increases exponentially for more accurate solutions [Klllerer et al. 2004]. Since our method runs within single model training, we cannot tolerate such runtimes and thus opt for a greedy algorithm that efficient and known to return reasonable solutions in practice (Akçay et al., 2007]. Algorithm ttakes a greedy approach by sorting all the samples in descending order by their (max((e(d)
(d())) values and then sequentially selecting the samples that do not violate any of the constraints. The computational complexity is thus On log n) where n is the total number of samples.
We now present our overall model training process for weighted ERM in Algorithm 2 For each epoch, we first select clean and fair sample set S using Algorithm T We then draw minibatches from S according to the (y = y, 2) 2--w weights described in Sec. 3.2 and update the model parameters via stochastic gradient descent. A minibatch selection with uniform sampling can be considered as an unbiased estimator of the ERM, so sampling each (y= y, z) set in proportion to its weight results an unbiased estimator of the weighted ERM. Finally, at the end of the epoch, we update the À values using the update rules in Eq. 6 We discuss about convergence in Sec. A.3.
5

---

5 Experiments
We evaluate our proposed algorithm. We use logistic regression for all experiments. We evaluate our models on separate clean test sets and repeat all experiments with 5 different random seeds. We use PyTorch, and all experiments are run on Intel Xeon Silver 4210R CPUs and NVIDIA Quadro RTX 8000 GPUs. More detailed settings are in Sec. B.I.
Fairness Measures We focus on two representative group fairness measures: (1) equalized odds (EO) (Hardt et al.] 2016], whose goal is to obtain the same accuracy between sensitive groups conditioned on the true labels and (2) demographic parity (DP) [Feldman al., 2015], which is satisfied when the sensitive groups have the same positive prediction ratio. We evaluate the fairness
disparities over sensitive groups as follows: EO disparity max Pr(y = 1|z = 2, y = y)
Pr(ý- = 1 y) and DP disparity maX:ez |Pr(ý = z 2) Pr(ý = 1) Note that the classi- fier perfectly fair when the fairness disparity zero.
Datasets We use a total of three datasets: one synthetic dataset and two real benchmark datasets. We generate a synthetic dataset using a similar method to Zafar al. 2017a]. The synthetic dataset has 3,200 samples and consists of two non-sensitive features (x1. X2), one sensitive feature z, and one label class y. Each sample (X1, X2. y) is drawn from the following Gaussian distributions: (X1, X2) y = 1 ~ [[11; 1], [5, 1, and (x1, x2)]y y 0 ~ 1[=1; 11]] [10, 1; 1,3).. We add the sensitive
feature z to have a biased distribution: Pr(z = 1) = 7 Pr((() x,)ly = 1) 1)/(7Pr((xx,, x,)ly
1)
Pr((x}, x6)ly 0)] where (x1, x,) = (x1 cos(s /5) - sin( XI 55n( + X2 (0s((7/5) (T/5)) We visualize the synthetic dataset Sec. B.2. We utilize two real datasets, ProPublica COMPAS [Angwin
et al. 2016] and AdultCensus [Kohavi 1996). and use the same pre- pre-processing in IBM Fairness
360 Bellamy et al., 2019]. COMPAS and AdultCensus consist of 5,278 and 43,131 samples, respectively. The labels in COMPAS indicate recidivism, and the labels in AdultCensus indicate each customer's s annual income level. For both datasets, we use gender as the sensitive attribute. Our experiments do not use any direct personal identifier, such name date of birth.
Noise Injection We use two methods for noise injection: (1) label flipping (Paudice et al. 2018). which minimizes the model accuracy (Label Flipping) and (2) targeted label flipping |Roh et al.) 2020]. which flips the labels of specific group SGroup-Targeter group in (2), we choose the one that, when attacked, results in the lowest model accuracy all groups. The two methods represent different scenarios for label flipping attacks. In Secs. 5_1 5.2 5.4 we flip 10% of labels in the training data, and in Sec. 5.3 we flip 10% to 20% of labels (i.e., varying the noise rate) in the training data.
Label Flipping). When targeting
Baselines We compare our proposed algorithm with four types of baselines: (1) vanilla training
using logistic regression (LR)); (2) robust only training using ITLM [Shen and Sanghavi 2019];
fair only training using FairBatch [Roh et al. 2021] and (4) fair and robust training where
evaluate two baselines and a state-of-the-an method called FR-Train [Roh et al. 20200. The two
baselines are as follows where each runs in two steps:
ITLM-FB: Runs ITLM and then FairBatch on the resulting clean samples.
ITLM-+Penall Runs ITLM and then an in in-processing fairness algorithm (Zafar et al. 2017 7 on the clean samples. The fairness algorithm adds a penalty term to the loss lunction for reducing the covariance between the sensitive attribute and the predicted labels.
FR-Train [Roh et al. 2020] performs adversarial training between a classifier and fair and robust discriminators. For the robust training. FR-Train relies separate clean validation set. Since FR-Train benefits from additional data, its performances can be viewed as an upper bound for our algorithm, which does not utilize such data.
In Sec. B.3, we also compare with other two-step baselines that improve the faimesss both during and after clean sample selection.
Hyperparameters We choose the step size for updating À (i.e., a in Eq. 6) within the candidate set (0.0001, 0.0005, 0.001) } using cross volsdvaaii.tion. We assume the clean ratio is known for any dataset. If the clean ratio is not known in advance, can be inferred using cross-validation [Liu and Yu al. 2018 For all basclines, start from candidate set of hyperparameters and
Tao 2015

---

Table 1: Performances on the synthetic test set w.r.t. equalized odds disparity (EO Disp.) and demographic parity disparity (DP Disp.). Disp....) We compare our algorithm with four types of baselines: (1) vanilla training: LR: (2) robust training: ITLM (Shen and Sanghavi 2019]; (3) fair training: FB [Roh et al.) 202T); and (4) fair and robust training:
FILM-FFB, ITLM Penalty (Zafar al. 2017a b).
and FR-Train (Roh et al.) 2020). We flip 10% of labels in the training data. Experiments are repeated
5 times. We highlight the best and second-best performances among the fair and robust algorithms.
Label Flipping
Group-Targes Label Flipping
Method
Acc.
EO Disp.
Acc.
DP Disp.
Acc.
EO Disp.
Acc.
DP Disp.
LR
.665..003
_557=.015
.665..003 400=.0000
.600=.002 .405=.008
.600=.002 .300=.006
ITLM
.716+.001 .424+.003
.716+.001 .380=.002
.719=0001 .315=.017
.719=.001
.287=.009
FB
.509+0017 .051+0018
.683+0002
.063+.0019
.544 .005 .072+-010
.526+.003
.082+0002
ITLM-
.718:1.003 .199:t.020
.725+.002
.089=0032
.77==0001
.108=1.000
.704:1003
.067=.027
-Penall
.5511.051 .172:1.046
.774..012
..68:.014
.706=.001
.0801.004
668:1.004
.044:.004
Ours
.727+-002 .064+.005
.720+-001
.006=.001
.726 .040+-002
,720+-001
.039-.0007
FR- Train
.722:1005 .0781.010
.7111.003 .057=.027
.715=.003 .0461.011 .7061.011 .079=.030
Table 2: Performances on the COMPAS test set w.r.t. equalized odds disparity (EO Disp.) and
demographic parity disparity (DP Disp.). Other experimental settings are identical to Table
Label Flipping
Label Flipping
Method
Acc. EO Disp.
Acc.
DP Disp.
Acc.
EO Disp.
Acc.
DP Disp.
LR
503:.002 .23::.021
5031.002 .093=.020
5513=000 .668:..000
513:1000 .648=.000
ITLM
53661000 .145+.003
536:.000 .944=.033
.539=002
.593:.019
.32910002 .547=.015
.5091.001 .0832.013
503=.002
.046=.010
.525=.004
.093=.003
.507=.004
.059=.010
ITLM-FB
.5201.001
.0641.013
5231.003
.0399.000
538=,008
.315:10074
.5201.013
.130=.105
.523+0003
.077
.25+.001
.059=.088
.517 014 432
.514+.000 .00
.094=-022
Ours
5211.000 .044:..00
.544
.025=.000
.545=009
.084:.012
.521=.000
.024=.000
FR- Train
6201.0099 .0541.015
.6071.016 .51
.611-029 .02
.5971.039 .045=.014
use cross-validation choose the hyperparameters that result in the best fairness while having an accuracy that best aligns with other results. More hyperparameters are described in Sec. B.1.
5.1 Accuracy and Fairness
We first compare the accuracy and fairness results of our algorithm with other methods on the synthetic dataset in Table We inject noise into the synthetic dataset either using label flipping or targeted label flipping. For both cases, our algorithm achieves the highest accuracy and fairness results compared to the baselines. Compared to LR, our algorithm has higher accuracy and lower EO and DP disparities, which indicates better fairness. Compared to ITLM and FB, our algorithm shows much better fairness and accuracy, respectively. FB's accuracy is noticeably low a result of worse accuracy-fairres tradeoff in the presence of noise. We now compare with the fair and robust training methods. Both ITLM- and ITLM- usually show worse accuracy and fairness compared to our algorithm, which shows that noise and bias cannot easily be mitigated in separate steps.
Compared to FR-Train, our algorithm surprisingly has similar accuracy and better fairness, which suggests that sample selection techniques like ours can outperform nn-processing techniques like FR-Train that also rely on clean validation sets. We also perform an accuracy- feurrcs-fairness trade-off comparison between our algorithm and FR- Train in Sec. B.4, where the trends are consistent with the results in Table
We now make the same comparisons using real datasets. We show the COMPAS dataset results here in Table 2 and the AdultCensus dataset results in Sec. B.5 as they are similar. The results for LR, ITLM, and FB are similar to those for the synthetic dataset. Compared the fair and robust algorithms ITLM->FB, ITLM->Penalty, and FR-Train, our algorithm usually has the best or second-best accuracy and fairness values as highlighted in Table Unlike the synthetic dataset, FR-Train obtains very high accuracy values with competitive fairness. We suspect that FR-Train is benefiting from its clean validation set, which motivates us to perform a more detailed comparison with our algorithm in the next section.
7

---

Table Detailed comparison with FR- Train on the COMPAS test set using 10% label flipping. We use the following three different validation sets for FR- Train::: (1) noisy; (2) noisy, but cleaned with ITLM; and (3) clean.
Method
Acc.
EO Disp.
Acc.
DP Disp.
FR-Train with clean val. set
.220=.009 0744.005
.0771.016 .050..013
FR-Train with noisy val. set cleaned with ITLM FR-Train with noisy val. set
.531=.033 008720088
.304.022 .0591.018
.542+.034
.073=.021
5131.005 .411.0024
Ours
55011000
.0442.000
.544.000 .255..000
0.8
0.4
Accuracy
ITLM
DP Disp.
0.G
FR-Train (9.. Clron Val.
0.2
0.4
0.0
10%
15% Noise Rate
20%
10%
15%
20%
Noise Rate
(a) Accuracy.
(b) Demographic parity disparity.
Figure 3: Performances of LR, ITLM, FR-Train, and our algorithm (Ours) on the synthetic data while varying the noise rate using label flipping Pauuiice et al. 2018
5.2 Detailed Comparison with FR-Train
We perform a more detailed comparison between our algorithm and FR-Train. In Sec. 5.1 we observed that FR-Train takes advantage of its clean validation set to obtain very high accuracy values on the COMPAS dataset. We would like to see how that result changes when using validation sets that are less clean. In T we compare with two scenarios: (1) FR- TRaTrain has no clean data, so the validation set is just as noisy as the training set with 10% of its labels flipped and (2) the validation set starts as noisy, but is then cleaned using ITLM, which is the best we can do if there no clean data. As a result, our algorithm is comparable to the realistic scenario (2) for FR-Train.
We also compare runtimes (i.e., wall clock times in seconds) of our algorithm and FR-Train on the syn- thetic and COMPAS datasets w.r.t. equalized odds. As a result, the synthetic dataset, but a bit slower on the COM- PAS dataset. Our algorithm's 's runtime bottleneck is the greedy algorithm (Algorithm[I [ for batch selection. FR-Train trains slowly because it trains three models (one classifier and two discriminators) together. Hence, there is no clear winner.
Table 4: Runtime comparison (in seconds) of our algorithm and FR-Train on the syn- thetic and COMPAS test sets w.r.t. equal- ized odds disparity. Other experimental set- tings are identical to Table [
that our algorithm is faster on
Method
Synthetic
COMPAS
FR-Train Ours
90.986 44.746
230.304
241.291
5.3 Varying the Noise Rate
We compare the algorithm performances when varying the noise rate (i.e., portion of labels flipped) in the data. Figure shows the accuracy and fairness (w..t. demographic parity disparity) of LR. ITLM, FR-Train, and our algorithm when varying the noise rate in the synthetic data. Even the noise rate increases, the relative performances among the four methods do not change significantly. Our algorithm still outperforms LR and ITTLM in terms of accuracy and fairness and comparable to FR-Train having worse accuracy, but better fairness. The results w.r.t. equalized odds disparity similar and can be found in Sec. B.6.
5.4 Ablation Study
we conduct an ablation study to investigate the effect of each component in our algorithm. We consider three ablation scenarios: (1) remove the fairness constraints in Eq. 2 which reduce discrimination in sample selection, (2) remove the weights Eq. 3 which ensure fair model training, and (3) remove both functionalities (i.e., same as ITLM). As result, each ablation scenario leads
In Table 5
8

---

Table 5: Ablation study on the synthetic and COMPAS test sets using 10% label flipping. We consider the following three ablation scenarios: (1) w/o faimess constraints; (2) w/o ERM weights; and (3) w/o both.
Synthetic
COMPAS
Method
Acc.
EO Disp.
Acc.
DP Disp.
Acc.
EO Disp.
Acc.
DP Disp.
W/o both
.716:20011 .224.003
.716:.009
,380:1.002
.5362.000
_145=.003
.5362.000 .094:.003
W/o fairness const.
.718:+002 .070+.004
.723=-001 .036+.002
.520=1002 .110=0005
.520=.002 .073:+0002
W/o ERM weights
.727+.001
.233+.014
.727=000 .201=0009
.531=.004
.152=0018 .529:+.007 .058+.020
Ours
.727+-002 .044+0005
.720+0001 .006++0001
.521+.000 ,044=0000 .544+.000 .025+.000
to either worse accuracy and fairness or slightly-better accuracy, but much worse faireess. We thus conclude that both functionalities are necessary.
Related Work
Fair & Robust Training We cover the literature for fair training, robust training, and fair and robust training, in that order.
For model faimess, many definitions have been proposed to address legal and social issues Narayanan 2018). Among the definitions, we focus on group fairness: equalized odds [Hardi et al. 2016) and demographic parity [Feldman et al. 2015). The algorithms for obtaining group fairness can be
categorized as follows: (1) pre- pre-processing techniques that minimize bias in data [Kamiran and
Calders 2011 Zemel et al. 2013 Feldman et al., 20155 du Pin Calmon et al., @ al.
2020 Jiang and Nachum 2020]. (2) in in-processing techniques that revise the training process to
prevent the model from learning the bias [Kamishima et al. 2012 Zatar et al. 2017a Agarwal
eet al. 2018 Zhang et al. 2018 Cotter et al.] and (3) post-processing techniques that modify
the outputs of the trained model to ensure fairess [Kamiran al. 2012 Hardt et al.] 20166] Pleiss
jet al. 2017 Chzzhen et al. 2019). However, these fairness-only approaches do not address robust
training as we do. Beyond group fairness, there are other important fairness measures including
individual fairness [Dwork et al. 2012) and causality-based fairness (Kilbertus et al. 2017 Kusner
al. 2017 Zhang and Bareinboim) 2018 Nabi and Shpitser 2018 Khademi et al. 2019] Extending
our al gorithm to support these measures is an interesting future work.
Robust training focuses training accurate models against noisy adversarial data. While data can be problematic for many reasons [Xiao et al., 2013). most of the robust training literature assumes label noise (Song et al. 2020 and mitigates it by using (1) model architectures that are resilient
against the noise [Chen and Gupta 2015] Jiddal et al. 2016 Bekker and Goldberger, [206)] Han et al.
2018a], (2) loss regularization techniques Goodfellow al. 2015 Pereyra al. [2017] Tanno et al.
2019 Hendrycks et al. 2019 Menon et al. 2020]. , and (3)
loss correction techniques Patrini et al.
2017 Chang al. 2017 7 Ma al. 2018 Arazo et al. 2019 However, these works typically do not
focus on improving fairness.
A recent trend is to improve both fairness and robustness holistically [Lee et al. 202 The closest work to our algorithm is FR- Train [Roh et al.) 2020), which performs adversarial training among a classifier, fairness discriminator, and robustness discriminator. In addition, FR-Train relics clean validation set for robust training. In comparison, our algorithm is a sample selection method that does not require modifying the internals of model training and does not rely on a validation set either. There are other techniques that solve different problems involving faimess and robustness. A recent study observes that feature selection for robustness may lead to unfaireess (Khani and Liang] [021] and another study shows the limits of fair learning under data corruption [Konstantinov and Lampert 202T]. In addition, there techniques for (1) fair training against noisy or missing sensitive group
information [Hashimoto et al. 2018 Lamy et al. 2019 Lahoti et al. 2020 Wang et al., 2020 Awasthi
al. 2020 Celis et al. [002]] ]. (2) fair training with distributional robustness [Mandal et al. 2020
Rezaci et al. 2021). (3) robust fair training under sample selection bias [Du and Wu 2021]. and (4)
fairness-reducing attacks [Chang et al. 2020 Solans et al. 2020). In comparison, we treat fairness
and robustness as equals and improve them together.
Sample Selection While sample selection is traditionally used for robust training, it is recently being used for fair training as well. The techniques are quite different where robust training focuses

---

on selecting or fixing samples based on their loss values (Han et al. 20186 Jiang et al. 2018 Chen al. 2019 Shen and Sanghavi 2019), while fair training more about balancing the sample sizes among sensitive groups to avoid discrimination [Roh et al.) 2021]. In comparison, we combine the two approaches by selecting samples while keeping balance among sensitive groups.
7 Conclusion
We proposed to our knowledge the first sample selection- algorithm for fair and robust training. A key formulation is the combinatorial optimization problem of unbiased sampling in the presence of data corruption. We showed that this problem is strongly NP-hard and proposed an efficient and effective algorithm. In the experiments, fairness compared to other baselines. In comparison to the state-of-the-an FR-Train, our algorithm is competitive and casier to deploy without having modify the model training or rely a clean validation set.
algorithm shows better performances in accuracy and
Societal Impact & Limitation Although we anticipate our research have positive societal impact by considering fairness in model training, it may also have some negative impacts. First, our faimess-aware training may result in worse accuracy. Although we believe that faimesss will be indispensable in future machine learning systems, we should not unnecessarily sacrifice accuracy. Second, choosing the right fairness measure challenging. and poor choice may lead to undesirable results. In applications like criminal justice and finance where fairness issues have been thoroughly discussed, choosing the right measure may be straightforward. On the other hand, there are new applications like marketing where fairness has only recently become an issue. Here one needs to carefully consider the social context to understand what it means to fair.
We also discuss limitations. Fairness is a broad concept, and our contributions are currently limited prominent group fairness measures. There other important notions of fairesss like individual fairness, and we believe that extending our work to address them is an interesting future work. In addition, our robust training is currently limited to addressing label noise. However, ITLM is also known to handle ad adversarial perturbation on the features, so we believe our algorithm can be extended other types of attacks as well. Another avenue of research is to use other robust machine learning algorithms Ren et al. (2018) in addition to ITLM.
Acknowledgements
Yuji Roh and Steven E. Whang were supported by a Google AI Focused Research Award and by the Engineering Research Center Program by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. NRF-201 8R ASA and NRF- 2021R 1021RICICI005999) Kangwook Lee was supported by NSF/Intel Partnership on Machine Learning for Wireless Networking Program under Grant No. CNS-20031 29 and by the Understanding and Re- ducing Inequalities Initiative of the University of Siiconsin-Madioone Office of the Vice Chancellor for Research and Graduate Education with funding from the Wisconsin Alumni Research Foundation. Changho Suh was supported by Institute for Information communications Technology Planning & Evaluation (ITTP) grant funded by the Korea government (MSIT) (No. 2019-0-01396, Development of framework for analyzing, detecting, mitigating of bias in Al model and training data).
References
A. Agarwal, A. Beygelzimer, M. Dudfk, J. Langford, and H. M. Wallach. A reductions approach to fair classification. In ICML, pages 60 69, 2018
Y. Akçay, H. Li, and S. H. Xu. Greedy algorithm for the general multidimensional knapsack problem. Annals of Operations Research, 150(1): 2007.
Angwin, J. Larson, S. Mattu, and L. Kirchner. Machine bias: There's software used across the country to predict future criminals, And its biased against blacks. ProPublica, 2016.
E. Arazo, D. Ortego, P. Albert, N. O'Connor, and K. McGuinness. Unsupervised label noise modeling and loss correction. In /CML, pages 122-321,22 2019.
10

---

P. Awasthi, M. Kleindessner, and J. Morgenstern. Equalized odds postprocessing under imperfect group information. In AISTATS, pages 1770-1780, 2020.
A. J. Bekker and J. Goldberger. Training deep pages 2682 2686, 2016.
based unreliable labels. In IEEE ICASSP.
R. E. Bellamy, K. Dey, Hind, S. Hoffiman, S. Houde, Kannan, Lohia, J. Martino, S. Mehta, A. Mojsilovic, S. Nagar, K. Ramamurthy, J. T. Richards, D. Saha, P. Sattigeri, M. Singh, K. R. Varshney, and Y. Zhang. Al fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias. IBM J. Res. Dev., (3(4/5)::1-4:111 2019.
A. Caprara, H. Kellerer, U. Pferschy, and D. Pisinger. Approximation algorithms for knapsack problems with cardinality constraints. EJOR, 123(2):3333345, 2000.
E. Celis, L. Huang, V. Keswani, and N. K. Vishnoi. Fair classification with noisy protected attributes: framework with provable guarantees. In JCML, 2021.
Chang, T. D. Nguyen, S. K. Murakonda, E, Kazemi, and Shokri. On adversarial the robustness of fair machine learning. ArXiv, 2020.
H.-S. Chang, E. Learned-Miller, and A. McCallum, Active bias: Training more accurate neural networks by emphasizing high variance samples. In NeurlPS, 2017.
P. Chen, B. B. Liao, G. Chen, and S. Zhang. Understanding and utilizing deep neural networks trained with noisy labels. In ICML, pages 1062-107,, 2019.
X. Chen and A. Gupta. Webly supervised learning of convolutional networks. In ICCV, pages 1431-1439, 2015.
K. Choi, Grover, T. Singh, Shu, and Ermon. Fair generative modeling via weak supervision. In JCML, 2020.
Chzhen, C. Denis. M. Hebiri, Oneto, and Pontil. Leveraging labeled unlabeled data consistent fair binary classification. In NewrlPS, pages 12760-12770. 2019.
A. Cotter, H. Jiang, and K. Sridharan. Two-player games for efficient non-convex constrained optimization. In ALT, pages 300-332, 2019.
W. Du and X. Wu. Robust fairnes--wware learning under sample selection bias. arXiv preprint wow.::::ii....I... 2021.
F. du Pin Calmon, D. Wei, B. Vinzamuri, K. N. Ramamurthy, and K. R. Varshney. Optimized pre- pre-processing for discrimination prevention. In NeurlPS, pages 3995-4004, 2017.
C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. Zemel. Fairness through awareness. In ITCS, pages 214-226, 2012.
M. Feldman, S. Friedler, J. Moeller, C. Scheidegger, and Venkatasubramannian. Certifying and removing disparate impact. In SIGKDD, pages 259-268, 2015.
M. R. Garey and D. S. Johnson. Computers and Intractabiliry: A Guide to the Theory of NP-completeness. Mathematical Sciences Series. W. H. Freeman, 1979.
I. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.
Google.
Responsible ai practices. sesponsibee-ai-pratiice 2020.
httpp://ai..goggarrrpooobbbbbbis
B. Han, J. Yao, N. Gang, M. Zhou, I. Tsang, Y. Zhang, and M. Sugiyama. Masking: perspective of noisy supervision. In NeurlPS, pages 5839-5849, 2018a.
Han. Q. Yao, Yu. G. Niu, M. Xu, W. Tsang, and Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In NeurIPS, pages 8535-8545, 2018b.
M. Hardt, E. Price, and N. Srebro. Equality of opportunity in supervised learning. In NeurlPS, pages 3315-3323, 2016.
T. Hashimoto, M. Srivastava, H. Namkoong, and P. Liang. Fairness without demographics in repeated loss minimization. In ICML, pages 1929- 1938,1988,1100 2018.

---

D. Hendrycks, K. Lee. and M. Mazeika. Using pre-training can improve model robustness and uncertainty. In ICML, pages 2712-2721, 2019.
IBM. Trusting ai. https://www.researr wtt..mmm.mmmmmmm......m.......tt wtt../c/maaticiiii.
http://iiicaainttlllleeeee//eeee httpae/eaaaaatttttaaaaaaaaaaaaaa aii/ 2020.
Jiang O. Nachum, Identifying correcting label bias in machine learning. A/STATS, pages 702-712, 2020.
Jiang, Zhou, T. Leung. L.-J. Li. Fei-Fei. Mentorne:: Learning data-driven curriculum for very deep neural networks on corrupted labels. In ICML, pages 2304-2313, 2018.
L. Jindal, M. Nokleby, and X. Chen. Learning deep networks from noisy labels with dropout regularization. In IEEE ICDM, pages 967-972, 2016.
E. Kamiran and T. Calders. Data preprocessing techniques for classification without discrimination. Knowl. Inf. Syst. 33(1):1 33, 2011.
F. Kamiran, A. Karim, and X. Zhang. Decision theory for discrimination aware classification. In IEEE ICDM, pages 224-929, 2012.
T. Kamishima, S. Akaho, H. Asoh, and J. Sakuma. Fairness-aware classifier with prejudice remover regularizer. In ECML PKDD, pages 35-50, 2012.
H. Kellerer, U. Pferschy, and D. Pisinger. Knapsack Problems. Springer Nature Book Archives Millennium. 2004.
A. Khademi, S. Lee. D. Foley, and V. Honavar. Fairness in algorithmic decision making: An excursion through the lens of causality. In WWW, pages 2907-2914, 2019.
Khani and P. Liang. Removing spurious features can hurt accuracy and affect groups disproportionately In ACM FAceT, pages 196-205, 2021.
N. Kilbertus, Rojas-Carulla, Parascandolo, M. Hardt, D. Janzing, and B. Schölkopf. Avoiding discrimina- tion through causal reasoning. In NewrlPS, pages 656-666, 2017.
Kohavi. Scaling up accuracy of naive-bayes classifiers: decision-tree hybrid. In SIGKDD, pages 202 207, 1996.
N. Konstantinov and C. H. Lampert. Faimess-aware learning from corrupted data. ArXiv, 2021.
M. Kusner, J. Loftus, C. Russell, and R. Silva. Counterfactual fairness. In NeurlPS, pages 4066-4076. 2017.
Lahoti, A. Beutel, J. Chen, K. Lee, F. Prost, N. Thain, X. Wang, and E, Chi. Fairness without demographics through adversarially reweighted leamning. In NeurIPS, pages 740, 2020.
A. Lamy, 7. Zhong, A. K. Menon, and Verma. Noise-tolerant fair classification. In NewrlPS, 2019.
J.-G. Lee, Y. Roh, H. Song, and S. E. Whang. Machine learning robustness, faimesss, and their convergence. In SIGKDD, 2021.
T. Liu and D. Tao. Classification with noisy labels by importance reweighting. IEEE TPAMI, 38(3)444-611 2015.
X. Ma, Y. Wang, M. E. Houle, S. Zhou, S. Erfani, S. Xia, S. Wijewickrema, and J. Bailey. Dimensionnllty-ddiien learning with noisy labels. In ICML, pages 3355-3366 2018.
D. Mandal, S. Deng, S. Jana, J. Wing, and D. J. Hsu. Ensuring faimess beyond the training data. In NeurIPS, pages 1044--8456, 2020.
A. K. Menon, A. S. Rawat, Reddi, and S. Kumar. Can gradient clipping mitigate label noise? In ICLR, 2020.
Microsoft. Responsible ai principles from microsoft. responsible-aii 2021.
www.:mmccoooooooo httpss//ww..mcroofff.cooo. us/ us/ai/
Nabi and I. Shpitser. Fair inference on outcomes. In AAAI, pages 1931-1940, 2018
A. Narayanan. Translation tutorial: 21 fairness definitions and their politics. In ACM FAceT, volume 1170, 2018.
G. Patrini, A. Rozza, A. Krishna Menon, R. Nock, and L. Qu. Making deep neural networks robust to label noise: loss correction approach. In CVPR, pages
12

---

A. Paudice, L. Munioz-Gnnzälez, and E. C. Lupu. Label sanitization against label flipping poisoning attacks. In ECML PKDD, pages 5-15, 2018.
G. Pereyra, G. Tucker, J. Chorowski. F.. Kaiser, and G. Hinton. Regularizing neural networks by penalizing confident output distributions. 2017.
G. Pleiss, M. Raghavan, F. Wu, J. M. Kleinberg, and K. Q. Weinberger. On fairness and calibration. In NeurlPS, pages 5684-5693, 2017.
M. Ren, W. 7eng. B. Yang, and R. Urtasun. Learning to reweight examples for robust deep learning. In ICMI., pages 4331-4340, 2018.
A. Rezaci, A. Liu, O. Memarrast, and B. D. Ziebart. Robust faimesss under covariate shift. AAAI, pages 9419-9427, 2021.
Y. Roh, K. Lec, S. E. Whang, and C. Sub. FR-Train: A mutual informatton-basses approach fair and robust training. In ICML, pages 8147-8157, 2020.
Y. Roh, K. Lee, S. E. Whang, and C. Suh. FairBatch: Batch selection for model faimesss. In ICLR, 2021.
P. J. Roussecuw, Least median of squares regression. Journal of the American Statistical Association, 79(388): 371-880, 1984.
Y. Shen and S. Sanghavi. Learning with had training data via iterative trimmed loss minimization. In ICMI., pages 5739-5748, 2019.
D. Solans, B. Biggio, and C. Castillo. Poisoning attacks on algorithmic fairness. In ECML PKDD, pages 162 2020.
H. Song, M. Kim, and J.-G. Lec. Selfie: Refurbishing unclcan samples for robust deep learning. In ICML, pages 5907-5915, 2019.
H. Song, M. Kim, D. Park, and J.-G. Lee. Learning from noisy labels with deep neural networks: A survey. ArXiv, 2020.
R. Tanno, A. Sacedi, S. Sankaranarayanan, D. C. Alexander, and N. Silberman, Learning from noisy labels by regularized estimation of annotator confusion. In CVPR, pages 11244-11253, 2019.
S. Wang. W. Guo, H. Narasimhan, A. Cotter, M. Gupta, and M. Jordan. Robust optimization for fairness with noisy protected groups. In NeurlPS, pages 5190-5203, 2020.
T. Xiao, T. Xia, Y. Yang. C. Huang. and X. Wang. Learning from massive noisy labeled data for image classification. In CVPR, pages 2691-2699 2015.
X. Yu, T. Liu, M. Gong, K. Batmanghelich, and D. Tao. An efficient and provable approach for mixture proportion estimation using linear independence assumption. In CVPR, pages 4480 -480, 2018.
M. B. Zafar, I. Valera, M. Gomez-Rodrigun and K. P. Gummadi. Fairness constraints: Mechanisms for fair classification. In AISTATS, pages 962-99 270.970.222 2017a.
M. B. Zafar, I. Valera, M. Gomez-Rodriguez, and K. P. Gummadi. Faimess beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In wwW, pages 1171-1180, 2017b.
R. S. Zemel, Y. Wu, K. Swersky, T. Pitassi, and C. Dwork. Learning fair representations. In JCML, pages 325-333, 2013.
B. H. Zhang, B. Lemoine, and M. Mitchell. Mitigating unwanted biases with adversarial learning. In AIES. pages 335-340, 2018.
J. Zhang and E. Bareinboim. Fairness in decision-making the causal explanation formula. In AAAI, pages 2037-2045, 2018.
13


Reviews:
[{"review_id": "XPmgcmFl05p", "reviewer": "Reviewer_GddM", "summary": "This paper proposes a sample selection-based method for training a robust and fair model under data corruption (label flipping). At a high-level, the proposed method combines the essence of ITLM (sample selection method for robustness) and FairBatch (sample selection method for fairness) in a clever way. The method involves three optimizations: (1)minimizing the loss of the selected samples while satisfying fairness constraints; (2)training a model on samples selected from (1); (3) adjusting sampling ratios of each subgroup. \nFor evaluation, the author compares the proposed method with the state-of-the-art baseline method FR-Train as well as variations of the proposed method on one synthetic dataset and two widely used datasets. ", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n- well-written\n- comprehensive empirical comparison with baselines and variations\n- decent performance without the usage of extra data\n\nCons:\n- the proposed method is inferior to FR-Train when clean validation data is used for FR-Train\n\n\nThe paper is fairly well-written and proposes a method that cleverly combining the essence of two existing methods for robust and fair training. It also conducts an in-depth comparison with the baseline FR-Train and shows that the proposed method can achieve on par results when the extra validation data used for FR-Train is noisy. My remaining concern is that will the proposed method performs better if, say, the extra data is also used? Or alternatively what if the validation data is not given to FR-Train? For example, a subset of the training data can be held out for validation purpose so no extra data advantage will be given to FR-Train to compare with the proposed method.\n\n\nQuestion:\n- If no extra data is used for FR-Train or extra data is given to the proposed method, will the proposed method outperforms FR-Train?\n- How is the proposed method compared with FR-Train in terms of computational cost?\n ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5"}, "extra_text": {}}, {"review_id": "MUfLJ3tjPZx", "reviewer": "Reviewer_75jQ", "summary": "The paper presents a technique for subsampling training data in order to simultaneously ensure fairness with respect to protected characteristics and robustness in the face of corrupted data. As formulated the problem is NP-hard, so the authors devise a greedy heuristic for solving it and show that applying the technique to logistic regression gives competitive empirical results.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "There are common techniques for subsampling datasets to improve performance, but they may actually make unfairness worse by removing samples in a way related to protected characteristics. So the idea here is to subsample a dataset while constraining the choice to ensure that no (protected attribute, label) group is sampled more than some thresholds (which are set adaptively during the training loop).\n\nThe general idea is then that training with subsampling of data given these fairness constraints represents a bilevel optimization problem -- the top level is the model minimizing loss, and the bottom level is subsampling in order to maximize the top level. (Although both are minimizing almost the same loss, so it might seem like it's just a single joint minimization problem, the loss the model actually trains on is reweighted in such a way that the problem is general sum, so I think it really does need to be thought of as bilevel.)\n\nThis problem is then converted into a knapsack problem, which in turn can be approximately solved using a greedy heuristic. Experiments are run on synthetic datasets and COMPAS, with datasets corrupted by different types of label flipping. \n\nOriginality:\n\nThe paper mostly combines existing approaches and techniques, but to solve a new problem which it convincingly shows has not been previously addressed.\n\nClarity:\n\nThe paper is clearly written, there are no issues with presentation.\n\nQuality:\n\nThe experiments make reasonable choices and are fairly convincing and thorough, although on only one non-synthetic dataset. The definitions of fairness are reasonable choices from the literature.\n\nSignificance:\n\nThe technique is a solution to a reasonable but pretty narrow problem: when one wants to improve performance using subsampling while also guaranteeing fairness constraints.\n\nUpdate after rebuttal: the authors have convinced me that this combination of fairness/robustness due to dataset corruption is of more general interest than I had first thought.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors responsibly discuss the limitations of quantitative fairness definitions and the dangers of carelessly applying them in settings which might not make sense.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3.5"}, "extra_text": {}}, {"review_id": "9RTt3w1Nbk1", "reviewer": "Reviewer_5Ear", "summary": "The paper introduces an algorithm for training models that are both fair and robust to noise in the data. A number of experiments shows the effectiveness of the method for improving both fairness and robustness.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "$\\textbf{Originality}$\n\nTo my awareness the presented algorithm is novel and studying practical algorithms for designing models that are both robust and fair is an underexplored area. The proposed algorithm consists of multiple steps that are related to prior methods, which is properly acknowledged in the text.\n\nRegarding related work, I find that a few recent papers that deal with the interplay between robust and fair ML can be discussed in addition to those presented in the paper. In particular, [1] studies the limits of fair learning under data corruption and [2,3,4,5] are additional recent works that study fairness under label/attribute noise.\n\n$\\textbf{Quality}$\n\nThe work appears technically sound, with the algorithm and the experimental setups discussed in sufficient detail.\n\n$\\textbf{Clarity}$\n\nOverall, the paper is well-written and easy to follow. As a minor suggestion, on page 3, equation (2) can include $S_y$ written explicitly in terms of the variables $p_j$ to improve clarity.\n\n$\\textbf{Significance}$\n\nDesigning algorithms that improve both fairness and robustness at the same time is certainly important. The presented method seems to be relatively effective in achieving this, although the performance appears relatively inconsistent between datasets. Nevertheless, since there is little prior work on the topic, I still find the presented results to be useful and of interests. Future work can compare this algorithm to other baselines for example running Iterative Trimmed  Loss  Minimization or another robust ML algorithm on a fairness-regularized objective.\n\n\n$\\textbf{References}$\n\n[1] N. Konstantinov and C. H. Lampert. Fairness-aware learning from corrupted data, ArXiv, 2021\n\n[2] A. L. Lamy et al. Noise-tolerant fair classification. NeurIPS, 2019.\n\n[3] P. Awasthi, M. Kleindessner and J. Morgenstern. Equalized odds postprocessing under imperfect group information. In: AISTATS, 2020.\n\n[4] A. Mehrotra and L. E. Celis. Mitigating Bias in Set Selection with Noisy Protected Attributes. In: Conference on Fairness, Accountability, and Transparency, 2021.\n\n[5] L. Celis, L. Elisa, L. Huang and N. K. Vishnoi. Fair Classification with Noisy Protected Attributes. ArXiv, 2020", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "None that I can foresee.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "7klaEhRK5nV", "reviewer": "Reviewer_pcux", "summary": "This paper deals with unfairness in the clean sample selection method, a method to make the learned model robust against the outliers. The authors develop a mechanism to enforce fairness to the clean sample selection method. Their method is formulated as a combinatorial optimization problem; however, the optimization problem is strongly NP-hard and is difficult to solve. To overcome this difficulty, they introduce a greedy algorithm that works well in practice. The empirical evaluations demonstrate that their method outperforms the baselines and approaches to the ideal performance. ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper introduces some interesting ideas for fair and robust learning. The results are surprising and interesting; even if we employ the fair classification method, the fairness can be worsened by applying the clean sample selection method. The empirical results demonstrate that the present method overcomes such an issue. Hence, my current decision is acceptance.\n\nThe design of the optimization problem of Eq. 1 and Eq. 2 is unclear for me. Specifically, I'm wondering why the second constraint in Eq. 2 is upper bounds on the sample sizes and is not lower bounds. The lower bound constraints of the sample sizes are understandable because if there is an extremely small group, the estimation error of the fairness measure can be large. Hence, the lower bound constraints are justified as a requirement for accurate estimation of the fairness score. However, I cannot come up with a justification for the upper bound constraint.\n\nThe parameter choices of the existing fair learning algorithm are not provided. The compared methods, e.g., the method from Zafar et al. 2017, have a parameter that controls the trade-off between fairness and accuracy. The choice of the trade-off parameter highly influences the fairness performance of these methods. I concern that comparison might be unfair if the parameter choice is inappropriate. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The limitations and potential societal impact are adequately addressed.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Reviewers were unanimous in appreciating the paper's contribution to a topical problem, namely ensuring model fairness and robustness. The motivating analysis of the potential adverse effects of clean-sample selection for fairness were seen as particularly interesting. The reviews did bring up some areas for potential improvement, including citing a broader line of work on fairness under corruption, and clarifying some technical details in the algorithm. The authors are encouraged to incorporate these for an update version of the paper.
Decision: Accept (Poster)