Sample Selection for Fair and Robust Training
Authors: Yuji Roh
Kangwook Lee
Steven Euijong Whang
Changho Suh
Venue: NeurIPS 2021 Poster (2021)
Keywords: trustworthy AI
fairness
robustness
sample selection

Abstract: Fairness and robustness are critical elements of Trustworthy AI that need to be addressed together. Fairness is about learning an unbiased model while robustness is about learning from corrupted data, and it is known that addressing only one of them may have an adverse affect on the other. In this work, we propose a sample selection-based algorithm for fair and robust training. To this end, we formulate a combinatorial optimization problem for the unbiased selection of samples in the presence of data corruption. Observing that solving this optimization problem is strongly NP-hard, we propose a greedy algorithm that is efficient and effective in practice. Experiments show that our method obtains fairness and robustness that are better than or comparable to the state-of-the-art technique, both on synthetic and benchmark real datasets. Moreover, unlike other fair and robust training baselines, our algorithm can be used by only modifying the sampling step in batch selection without changing the training algorithm or leveraging additional clean data.

Sample Selection for Fair and Robust Training
Yuji Roh KAIST yuji. rogiirro@kaist.cco kk.
Kangwook Lee
University of Wisconsin-Madison kangwook leewwisc.edu
Steven Euijong Whang" KAIST
Changho Suh KAIST chsuhōkaist kr
swhang@kaist.ag ac.kr
Abstract
Fairness and robustness are critical elements Trustworthy AI that need to be addressed together. Fairness is about learning unbiased model while robustness is about learning from corrupted data, and it known that addressing only one them may have adverse affect the other. In this work, propose sample selection-based algorithm for fair and robust training. To this end, we formulate a combinatorial optimization problem for the unbiased selection of samples in the presence of data corruption. Observing that solving this optimization problem is strongly NP-hard, we propose greedy algorithm that efficient and effective in practice. Experiments show that our algorithm obtains faimess and robustness that are better than or comparable the state-of of-the-art technique, both on synthetic and benchmark real datasets. Moreover, unlike other fair and robust training baselines, our algorithm can be used by only modifying the sampling step batch selection without changing the training algorithm or leveraging additional clean data.
1 Introduction
Trustworthy Al is becoming essential for modern machine learning. While traditional focus of machine learning is to train the most accurate model possible, companies that actually deploy AI including [Mirrosofi] [2021] Google [2020], and IBM (2020) are now pledging to make their systems fair, robust, interpretable, and transparent well. Among the key objectives, we focus on fairness and robustness because they are closely related and need to be addressed together. Fairness is about learning an unbiased model while robustness is about learning resilient model even noisy data, and both issues root from the same training data. A recent work (Roh et al. 2020) shows that improving fairness while ignoring noisy data may lead to worse accuracy- cocuracy-fiimess tradcoff. Likewise, focusing on robustness only may have an adverse affect on fairness as we explain below.
There are several possible approaches for supporting fairness and robustness together. One is an in- processing approach where the model architecture itself modified optimize the two objectives. In particular, the state-of-the-an -ft-t---tth-tttt-e approach called FR-Train (Roh et al. 2020 uses adversarial learning to train a classifier and two discriminators for fairness and robustness. However, such an approach requires the application developer to use a specific model architecture and is thus restrictive. Another approach is to preprocess the data Kamiran and Calders 2011] and remove any bias and noise before model training, but this requires modifying the data itself. In addition, data cleaning method that designed to explicitly improve model fairness and robustness together.
not aware of general
Instead, we propose an adaptive sample selection approach for the purpose of improving fairness and robustness. This approach can easily be deployed as batch selection method, which does not require
"Corresponding author
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

1.5
%
V1
By
12.9 %
1 NO
40.0 %
2 1
Loss
1.0
X1
6.8 %
4g: YO
40.3 %
0.5
3o
500
1000
1500
2000
2500
3000
Epochs
(a) The size ratios of (label y, sensitive group 2) sets within the COMPAS dataset.
(b) Tracing the model losses on (label y. sensitive group sets when training logistic regression on the COMPAS dataset.
Figure 1: The ProPublica COMPAS dataset (Angwin al. 2016 has bias where the set (y = 1, 1) has the smallest size in Figure la. However, when training a logistic regression model on this dataset, the same set has the highest loss every epoch as shown in Figure 1b. Blindly applying clean selection at any point will thus discard (y = 1, z = 1) samples the most, which results in even worse bias and possibly worse fairness.
modifying the model data. Our techniques build on top of two recent lines of sample selection research: clean selection for robust training [Rousseeuw 1984, Song et al. 2019 Shen and Sanghavi 2019) and batch selection for faimess (Roh et al.] 2021 Clean selection is standard approach for discarding samples that have high-loss values and are thus considered noisy. More recently, batch
selection for fairness has been proposed where the idea is to adaptively adjust sampling ratios among sensitive groups (e.g., black or white populations) so that the trained model is not discriminative.
We note that state-of-the-ard clean selection techniques like Iterative Trimmed Loss Minimization (ITLM) [Shen and Sanghavi] 2019] are not designed to address fairness and may actually worsen the data bias when used alone. For example, Figure shows how the ProPublica COMPAS dataset (used by U.S. courts to predict criminal recidivism rates [Angwin et al. (0166) can be divided into four sets where the label y is either 0 or 1, and the sensitive attribute is either 0 or 1. The sizes of the sets biased where (y = 1, = 1) is the smallest. However, the model loss on (y = 1, z = 1) is the highest among the sets shown in Figure Tb Hence, blindly applying clean selection to remove high-loss samples will discard (y 1, 1) samples the most, resulting in a worse bias that may negatively affect fairness
demonstrate in Sec. [2]
We thus formulate a combinatorial optimization problem for fair and robust sample selection that achieves unbiased sampling in the presence of data corruption. To avoid discrimination on a certain group, we adaptively adjust the maximum number of samples per (y. 2) set using the recently proposed FairBatch system [Roh et al. 2021], which solves bilevel optimization problem of unfairness mitigation and standard empirical risk minimization through batch selection. We show that our optimization problem for sample selection can be viewed as a multidimensional knapsack problem and is thus strongly NP-hard [Garey and Johnson 1979]. We then propose an efficient greedy algorithm that is effective in practice. Our method supports prominent group fairness measures: equalized odds [Hardt et al. 2016) and demographic parity [Feldman et al. al.] (2015). For data corruption, currently assume noisy labels (Song al. 2020), which can be produced by random flipping or adversarial attacks.
Experiments synthetic and benchmark real datasets (COMPAS (Angwin et al. 2016 and Adult- Census [Kohavi (996)) show that our method obtains accuracy and fairness results that are better than baselines using fairness algorithms [Zafar et al. 2017a b Roh et al.) 2021 and on par with and sometimes better than FR-Train [Roh et al. 2020 which leverages additional clean data.
Notation Let 0 be the model weights, X be the input feature to the classifier, y E Y be the true class, and ý be the predicted class where ý a function of (x, 0). Let E Z sensitive attribute, e.g., race or gender. Let d y) be training sample that contains feature and label. Let S be a selected subset from the entire dataset D, where the cardinality D is n. In the model training. we use loss function (-(-) where smaller value indicates more accurate prediction.
2 Unfairness in Clean Selection
We demonstrate how clean selection that only focuses on robust training may lead to unfair models. Throughout this paper, use ITLM (Shen and Sanghavi 2019 as representative clean selection

---

ITLM
ITLM
0.3
ITLM
Accuracy
0.5
B0 Dispo
0.3
DP Disp.
LR
LR
LR
0.4
0.1
10%
1526
27%
10%
1856
20%
25%
10%
18%
27%
25%
Noise Rate
Noise Rate
Noise Rate
Accuracy.
(h) Equalized odds disparity-
(c) Demographic parity disparity.
Figure 2: Performances of logistic regression with and without ITLM (ITLM and LR, respectively) on the ProPublica COMPAS dataset while varying the noise rate using label flipping (Paudice et al.) 2018). Although ITLM improves accuracy (higher is better), the faimess worsens according to the equalized odds and demographic parity disparity measures (lower is better).
method for its simplicity. although other methods can be used as well. We first introduce ITLM and demonstrate how ITLM can actually worsen fairness.
ITLM Cleaning samples based on their loss values is traditional robust training tech- nique [Rousseeuw] 1984) that is widely used due its efficiency. Recently, ITLM was proposed as scalable approach for sample selection where lower loss samples are considered cleaner. The following optimization problem is solved with thcoretical guarantees of convergence:
Σ (s(s))
min S
S
where S is the selected set, s; a sample in S, and 7 the ratio of clean samples in the entire data.
Potential Unfairness We show how applying ITLM may lead to unfairness through a simple experiment. We use the benchmark ProPublica COMPAS dataset [Angwin et al. 2016] where the label y indicates recidivism, and the sensitive attribute z is set to gender. We inject noise into the training using label flipping techniques [Paudice et al., 2018). For simplicity, ITLM knows the clean ratio 7 of the data and selects [Tn] samples. We train a logistic regression model on the dataset, and ITLM iteratively selects clean samples during the training.
assume that
show the accuracy and fairness performances logistic regression with and without ITLM while varying the noise rate (i.e., portion of labels flipped) in Figure 2 As expected, ITLM increasingly improves the model accuracy for larger noise rates as shown Figure 22 At the same time, however, the fairness worsens according to the two measures use in this paper (see definitions in Sec.
equalized odds (Hardt et al. 2016) disparity and demographic parity [Feldman et al., 2015) disparity 2c respectively. The results confirm our claims in Sec.
shown in Figures 2b and
3 Framework
We provide an optimization framework for training models on top of clean selection with fairness constraints. The entire optimization involves three joint optimizations, which explain in piecewise fashion: (1) how to perform clean selection with fairness constraints, (2) how to train a model on top of the selected samples, and (3) how generate the faimess constraints. In Sec. 4 we present a concrete algorithm for solving all the optimizations.
3.1 Clean Selection with Fairness Constraints
The first optimization is to select clean samples such that the loss is minimized while fairness constraints satisfied. For now, assume that the faimess constraints are given where the selected samples certain set (y = y. the class y = y. Notice that Σ, A(y.s) = 1, Vy E Y. The fairness constraints are specified as upper bounds because using equality constraints may lead to infeasible solutions. We explain how the X
2) cannot exceed portion Aty.a) of the selected samples of

---

values are determined in Sec.
3.3
We thus solve the following optimization for sample selection:
min
Lo(ds) Pi
(1)
s.t.
Pi S Tn
Σ Ps S (v. Syl, ]E
(2)
Pr {0. 1), = 1,...,
where ps indicates whether the data sample d, is selected not, (gy.)) is an index set of the (y,z) 2) class, and Sy is the selected samples for constraint comes from ITLM while the second constraint is for the fairness.
Note that |Syl = Ejet, Pj- In Eq. 2 the first
An important detail is that we use inequalities all of our constraints, unlike ITLM. The reason is that we may not be able to find a feasible solution of selected samples if we only use equality constraints. For example, ((,0)) X(1.1) 0.5, but we have zero (y = 1, 0) samples, then the fairnsss equality constraint for (y = 1, z = 0) can never be satisfied.
3.2 Model Training
The next optimization is to train a model on top of the selected samples from Sec. 3.1 the sample selection optimization only uses incquality constraints, which means that for some set (y = y, Z = 2), its size [S(ye) may be smaller than \(y,=)|S,), However, we would like to train the model as if there were A(g,z)|5 [Syl samples for the intended fairness performance. We thus reweight
Recall that
ASy and perform weighted empirical risk minimization (ERM) for model training. We then solve the bilevel optimization of model training and sample selection in Sec. as follows:
each set (y y, = 2) by [S(y.ay]
min Σ
Σ
T ( (8:)
lo(s:)
(3)
(((
S,ESx
(3,
Sx =
arg min S [d:1p=1)::a.
Eq.
(4)
2
where À is the set of A(y.2) for all Z. YEY.
3.3 Fairness Constraint Generation
To determine the Xy. values, we adaptively adjust sampling ratios of sensitive groups using the techniques in FairBatch [Roh et al. 2021]. For each batch selection, we compute the fairness of the intermediate model and decide which sensitive groups should be sampled more (or less) in the next batch. The sampling rates become the A(g.=) values used in the fairness constraints of Eq. 2
FairBatch supports all the group fairess measures used in this paper. As an illustration, we state FairBatch's 's optimization when using equalized odds disparity (see Sec. A.1 for the optimization using demographic parity disparity):
min max ((y. L(g.silh 2 # 1',',,, 2, E y Y
(5)
where (yy=) 1/1Sx(r.=)l Esxe.., (s(s;) and Sx(y.=) is subset of S from Eq. for the
y. 2 = 2) set. FairBatch adjusts À each epoch via a signed gradient-based algorithm the direction of reducing unfairness. For example, if the specific sensitive group is less accurate the samples in the epoch (i.e., z is discriminated data from the (y y, z set to increase the model's 's accuracy that set:
terms of equalized odds), FairBatch selects more
(t)) = (a.2) sign (()==') L(y,)), # z'
(6)
where a the step size of updates. To integrate FairBatch with our framework, we adaptively adjust À as above for an intermediate model trained on the selected set Sx from Eq.

---

Algorithm 1: Greedy-Based
Algorithm 2: Overall Fair and Robust Model Training
Clean and Fair Sample Selection
Input: train data (Tirains Itrain); clean ratio т, loss function lo(:)
Input:
loss (o))), fair ratio
lambda X, clean ratio 7
d t (Frrain Mrrain)
0 initial model parameters
p (c) (o(
{A 1y E Y, E 2 +- random sampling ratios
sortIdx = (pro(profit)
for each epoch do
S S-11
= Algorithm((fg(d) 2, Ty)
for idx in sortldx do if diax does not violate any constraint in Eq. 8 w.r.t. X and r then Append data S O
Draw minibatches from S w.r.t. (0,21Sy/||((u.))
for each minibatch do
Update model parameters 0 according the minibatch
Update VAy, E À using the update rule as in Eq.
Output model parameters 0
4 Algorithm
We present our algorithm for solving the optimization problem in Sec. 3 We first explain how the clean selection with fairness constraints problem in Sec. 3.] can be converted to multi-dimensional knapsack problem, which has known solutions. We then describe the full algorithm that includes the three functionalities: clean and fair selection, model training, and fairness constraint generation.
The clean and fair selection problem
be converted to equivalent problem by (1) maximizing the sum of (max(f( - (d(d)) instead of minimizing the sum of (d4)'s where d is the set of d,' 's and (2) re-arranging the fairness constraints that the right-hand side expressions become constants (instead of containing the variable Syy as follows:
knapsack
max (max(ee - Lo(d:)) Pb
(7)
if & Dy if Dy and D2, e
s.t. DEIPi S Ex= WiP: S Tn, where ui
Ayyss)
2 A(a
if d, D(y,))
(8)
ps E (0,1), i == 1,..., n
where D(y.z) is a subset for the (y,2) 2) class. More details on the conversion are in Sec. A.2.
The multi-dimensional knapsack problem is known to be strongly NP-hard Garey and Johnson 1979]. Although exact algorithms have been proposed, they have exponential time complexi- ties [Klllerer et al. [004]] and are thus impractical. There is polynomial-time approximation scheme (PTAS) [Caprara et al., 2000), but the actual computation time increases exponentially for more accurate solutions [Klllerer et al. 2004]. Since our method runs within single model training, we cannot tolerate such runtimes and thus opt for a greedy algorithm that efficient and known to return reasonable solutions in practice (Akçay et al., 2007]. Algorithm ttakes a greedy approach by sorting all the samples in descending order by their (max((e(d)
(d())) values and then sequentially selecting the samples that do not violate any of the constraints. The computational complexity is thus On log n) where n is the total number of samples.
We now present our overall model training process for weighted ERM in Algorithm 2 For each epoch, we first select clean and fair sample set S using Algorithm T We then draw minibatches from S according to the (y = y, 2) 2--w weights described in Sec. 3.2 and update the model parameters via stochastic gradient descent. A minibatch selection with uniform sampling can be considered as an unbiased estimator of the ERM, so sampling each (y= y, z) set in proportion to its weight results an unbiased estimator of the weighted ERM. Finally, at the end of the epoch, we update the X values using the update rules in Eq. 6 We discuss about convergence in Sec. A.3.
5

---

5 Experiments
We evaluate our proposed algorithm. We use logistic regression for all experiments. We evaluate our models on separate clean test sets and repeat all experiments with 5 different random seeds. We use PyTorch, and all experiments are run on Intel Xeon Silver 4210R CPUs and NVIDIA Quadro RTX 8000 GPUs. More detailed settings are in Sec. B.I.
Fairness Measures We focus on two representative group fairness measures: (1) equalized odds (EO) (Hardt et al.] 2016], whose goal is to obtain the same accuracy between sensitive groups conditioned on the true labels and (2) demographic parity (DP) [Feldman al., 2015], which is satisfied when the sensitive groups have the same positive prediction ratio. We evaluate the fairness
disparities over sensitive groups as follows: EO disparity max Pr(y = 1|z = 2, y = y)
Pr(ý- = 1 y) and DP disparity maX:ez |Pr(ý = z 2) Pr(ý = 1) Note that the classi- fier perfectly fair when the fairness disparity zero.
Datasets We use a total of three datasets: one synthetic dataset and two real benchmark datasets. We generate a synthetic dataset using a similar method to Zafar al. 2017a]. The synthetic dataset has 3,200 samples and consists of two non-sensitive features (x1. X2), one sensitive feature z, and one label class y. Each sample (X1, X2. y) is drawn from the following Gaussian distributions: (X1, X2) y = 1 ~ [[11; 1], [5, 1, and (x1, x2)]y y 0 ~ 11=1; 11]] [10, 1; 1,3).. We add the sensitive
feature z to have a biased distribution: Pr(z = 1) = 7 Pr((() x,)ly = 1) 1)/(7Pr((xx,, x,)ly
1)
Pr((x}, x6)ly 0)] where (x1, x,) = (x1 cos(s /5) - sin( XI 55n( + X2 (0s((7/5) (T/5)) We visualize the synthetic dataset Sec. B.2. We utilize two real datasets, ProPublica COMPAS [Angwin
et al. 2016] and AdultCensus [Kohavi 1996). and use the same pre- pre-processing in IBM Fairness 360 Bellamy et al., 2019]. COMPAS and AdultCensus consist of 5,278 and 43,131 samples, respectively. The labels in COMPAS indicate recidivism, and the labels in AdultCensus indicate each customer's s annual income level. For both datasets, we use gender as the sensitive attribute. Our experiments do not use any direct personal identifier, such name date of birth.
Noise Injection We use two methods for noise injection: (1) label flipping (Paudice et al. 2018). which minimizes the model accuracy (Label Flipping) and (2) targeted label flipping |Roh et al.) 2020]. which flips the labels of specific group SGroup-Targeter group in (2), we choose the one that, when attacked, results in the lowest model accuracy all groups. The two methods represent different scenarios for label flipping attacks. In Secs. 5_1 5.2 5.4 we flip 10% of labels in the training data, and in Sec. 5.3 we flip 10% to 20% of labels (i.e., varying the noise rate) in the training data.
Label Flipping). When targeting
Baselines We compare our proposed algorithm with four types of baselines: (1) vanilla training
using logistic regression (LR)); (2) robust only training using ITLM [Shen and Sanghavi 2019];
fair only training using FairBatch [Roh et al. 2021] and (4) fair and robust training where
evaluate two baselines and a state-of-the-an method called FR-Train [Roh et al. 20200. The two
baselines are as follows where each runs in two steps:
ITLM-FB: Runs ITLM and then FairBatch on the resulting clean samples.
ITLM-+Penall Runs ITLM and then an in in-processing fairness algorithm (Zafar et al. 2017 7 on the clean samples. The fairness algorithm adds a penalty term to the loss lunction for reducing the covariance between the sensitive attribute and the predicted labels.
FR-Train [Roh et al. 2020] performs adversarial training between a classifier and fair and robust discriminators. For the robust training. FR-Train relies separate clean validation set. Since FR-Train benefits from additional data, its performances can be viewed as an upper bound for our algorithm, which does not utilize such data.
In Sec. B.3, we also compare with other two-step baselines that improve the faimesss both during and after clean sample selection.
Hyperparameters We choose the step size for updating À (i.e., a in Eq. 6) within the candidate set (0.0001, 0.0005, 0.001) } using cross volsdvaaii.tion. We assume the clean ratio is known for any dataset. If the clean ratio is not known in advance, can be inferred using cross-validation [Liu and Yu al. 2018 For all basclines, start from candidate set of hyperparameters and
Tao 2015

---

Table 1: Performances on the synthetic test set w.r.t. equalized odds disparity (EO Disp.) and demographic parity disparity (DP Disp.). Disp....) We compare our algorithm with four types of baselines: (1) vanilla training: LR: (2) robust training: ITLM (Shen and Sanghavi 2019]; (3) fair training: FB [Roh et al.) 202T); and (4) fair and robust training:
FILM-FFB, ITLM Penalty (Zafar al. 2017a b).
and FR-Train (Roh et al.) 2020). We flip 10% of labels in the training data. Experiments are repeated
5 times. We highlight the best and second-best performances among the fair and robust algorithms.
Label Flipping
Group-Targes Label Flipping
Method
Acc.
EO Disp.
Acc.
DP Disp.
Acc.
EO Disp.
Acc.
DP Disp.
LR
.665..003
_557=.015
.665..003 400=.0000
.600=.002 .405=.008
.600=.002 .300=.006
ITLM
.716+.001 .424+.003
.716+.001 .380=.002
.719=0001 .315=.017
.719=.001
.287=.009
FB
.509+0017 .051+0018
.683+0002
.063+.0019
.544 .005 .072+-010
.526+.003
.082+0002
ITLM-
.718:1.003 .199:t.020
.725+.002
.089=0032
.77==0001
.108=1.000
.704:1003
.067=.027
-Penall
.5511.051 .172:1.046
.774..012
..68:.014
.706=.001
.0801.004
668:1.004
.044:.004
Ours
.727+-002 .064+.005
.720+-001
.006=.001
.726 .040+-002
,720+-001
.039-.0007
FR- Train
.722:1005 .0781.010
.7111.003 .057=.027
.715=.003 .0461.011 .7061.011 .079=.030
Table 2: Performances on the COMPAS test set w.r.t. equalized odds disparity (EO Disp.) and
demographic parity disparity (DP Disp.). Other experimental settings are identical to Table
Label Flipping
Label Flipping
Method
Acc. EO Disp.
Acc.
DP Disp.
Acc.
EO Disp.
Acc.
DP Disp.
LR
503:.002 .23::.021
5031.002 .093=.020
.513=000 .668:..000
513:1000 .648=.000
ITLM
53661000 .145+.003
536:.000 .944=.033
.539=0022
.593:.019
.32910002 .547=.015
.5091.001 .0832.013
503=.002
.046=.010
.525=.004
.093=.003
.507=.004
.059=.010
ITLM-FB
.5201.001
.0641.013
5231.003
.0399.000
538=,008
.315:10074
.5201.013
.130=.105
.523+0003
.077
.25+.0011
.059=.088
.517 014 432
.514+.000 .00
.094=-022
Ours
5211.000 .044:..00
.544
.025=.000
.545=009
.084:.012
.521=.000
.024=.000
FR- Train
6201.0099 .0541.015
.6071.016 .51
.611-029 .02
.5971.039 .045=.014
use cross-validation choose the hyperparameters that result in the best fairness while having an accuracy that best aligns with other results. More hyperparameters are described in Sec. B.1.
5.1 Accuracy and Fairness
We first compare the accuracy and fairness results of our algorithm with other methods on the synthetic dataset in Table We inject noise into the synthetic dataset either using label flipping or targeted label flipping. For both cases, our algorithm achieves the highest accuracy and fairness results compared to the baselines. Compared to LR, our algorithm has higher accuracy and lower EO and DP disparities, which indicates better fairness. Compared to ITLM and FB, our algorithm shows much better fairness and accuracy, respectively. FB's accuracy is noticeably low a result of worse accuracy-fairres tradeoff in the presence of noise. We now compare with the fair and robust training methods. Both ITLM- and ITLM- usually show worse accuracy and fairness compared to our algorithm, which shows that noise and bias cannot easily be mitigated in separate steps.
Compared to FR-Train, our algorithm surprisingly has similar accuracy and better fairness, which suggests that sample selection techniques like ours can outperform nn-processing techniques like FR-Train that also rely on clean validation sets. We also perform an accuracy- feurres-fairness trade-off comparison between our algorithm and FR- Train in Sec. B.4, where the trends are consistent with the results in Table
We now make the same comparisons using real datasets. We show the COMPAS dataset results here in Table 2 and the AdultCensus dataset results in Sec. B.5 as they are similar. The results for LR, ITLM, and FB are similar to those for the synthetic dataset. Compared the fair and robust algorithms ITLM->FB, ITLM->Penalty, and FR-Train, our algorithm usually has the best or second-best accuracy and fairness values as highlighted in Table Unlike the synthetic dataset, FR-Train obtains very high accuracy values with competitive fairness. We suspect that FR-Train is benefiting from its clean validation set, which motivates us to perform a more detailed comparison with our algorithm in the next section.
7

---

Table Detailed comparison with FR- Train on the COMPAS test set using 10% label flipping. We use the following three different validation sets for FR- Train::: (1) noisy; (2) noisy, but cleaned with ITLM; and (3) clean.
Method
Acc.
EO Disp.
Acc.
DP Disp.
FR-Train with clean val. set
.220=.009 0744.005
.0771.016 .050..013
FR-Train with noisy val. set cleaned with ITLM FR-Train with noisy val. set
.531=.033 008720088
.304.0224 .0591.018
.542+.034
.073=.021
5131.005 .411.0024
Ours
55011000
.0442.000
.544.000 .255..000
0.8
0.4
Accuracy
ITLM
DP Disp.
0.G
FR-Train (9.. Clran Val.
0.2
0.4
0.0
10%
15% Noise Rate
20%
10%
15%
20%
Noise Rate
(a) Accuracy.
(b) Demographic parity disparity.
Figure 3: Performances of LR, ITLM, FR-Train, and our algorithm (Ours) on the synthetic data while varying the noise rate using label flipping Pauuiice et al. 2018
5.2 Detailed Comparison with FR-Train
We perform a more detailed comparison between our algorithm and FR-Train. In Sec. 5.1 we observed that FR-Train takes advantage of its clean validation set to obtain very high accuracy values on the COMPAS dataset. We would like to see how that result changes when using validation sets that are less clean. In T we compare with two scenarios: (1) FR- TRaTrain has no clean data, so the validation set is just as noisy as the training set with 10% of its labels flipped and (2) the validation set starts as noisy, but is then cleaned using ITLM, which is the best we can do if there no clean data. As a result, our algorithm is comparable to the realistic scenario (2) for FR-Train.
We also compare runtimes (i.e., wall clock times in seconds) of our algorithm and FR-Train on the syn- thetic and COMPAS datasets w.r.t. equalized odds. As a result, the synthetic dataset, but a bit slower on the COM- PAS dataset. Our algorithm's 's runtime bottleneck is the greedy algorithm (Algorithm[I [ for batch selection. FR-Train trains slowly because it trains three models (one classifier and two discriminators) together. Hence, there is no clear winner.
Table 4: Runtime comparison (in seconds) of our algorithm and FR-Train on the syn- thetic and COMPAS test sets w.r.t. equal- ized odds disparity. Other experimental set- tings are identical to Table [
that our algorithm is faster on
Method
Synthetic
COMPAS
FR-Train Ours
90.986 44.746
230.304
241.291
5.3 Varying the Noise Rate
We compare the algorithm performances when varying the noise rate (i.e., portion of labels flipped) in the data. Figure shows the accuracy and fairness (w..t. demographic parity disparity) of LR. ITLM, FR-Train, and our algorithm when varying the noise rate in the synthetic data. Even the noise rate increases, the relative performances among the four methods do not change significantly. Our algorithm still outperforms LR and ITTLM in terms of accuracy and fairness and comparable to FR-Train having worse accuracy, but better fairness. The results w.r.t. equalized odds disparity similar and can be found in Sec. B.6.
5.4 Ablation Study
we conduct an ablation study to investigate the effect of each component in our algorithm. We consider three ablation scenarios: (1) remove the fairness constraints in Eq. 2 which reduce discrimination in sample selection, (2) remove the weights Eq. 3 which ensure fair model training, and (3) remove both functionalities (i.e., same as ITLM). As result, each ablation scenario leads
In Table 5
8

---

Table 5: Ablation study on the synthetic and COMPAS test sets using 10% label flipping. We consider the following three ablation scenarios: (1) w/o faimess constraints; (2) w/o ERM weights; and (3) w/o both.
Synthetic
COMPAS
Method
Acc.
EO Disp.
Acc.
DP Disp.
Acc.
EO Disp.
Acc.
DP Disp.
W/o both
.716:20011 .224.003
.716:.009
,380:1.002
.5362.000
_145=.003
.5362.000 .094:.003
W/o fairness const.
.718:+002 .070+.004
.723=-001 .036+.002
.520=1002 .110=.005
.520=.002 .073:+0002
W/o ERM weights
.727+.001
.233+.014
.727=000 .201=0009
.531=.004
.152=0018 .529:+.007 .058+.020
Ours
.727+-002 .044+0005
.720+0001 .006++0001
.521+.000 ,044=0000 .544+.000 .025+.000
to either worse accuracy and fairness or slightly-better accuracy, but much worse faireess. We thus conclude that both functionalities are necessary.
Related Work
Fair & Robust Training We cover the literature for fair training, robust training, and fair and robust training, in that order.
For model faimess, many definitions have been proposed to address legal and social issues Narayanan 2018). Among the definitions, we focus on group fairness: equalized odds [Hardi et al. 2016) and demographic parity [Feldman et al. 2015). The algorithms for obtaining group fairness can be
categorized as follows: (1) pre- pre-processing techniques that minimize bias in data [Kamiran and
Calders 2011 Zemel et al. 2013 Feldman et al., 20155 du Pin Calmon et al., @ al.
2020 Jiang and Nachum 2020]. (2) in in-processing techniques that revise the training process to
prevent the model from learning the bias [Kamishima et al. 2012 Zatar et al. 2017a Agarwal
eet al. 2018 Zhang et al. 2018 Cotter et al.] and (3) post-processing techniques that modify
the outputs of the trained model to ensure fairess [Kamiran al. 2012 Hardt et al.] 20166] Pleiss
jet al. 2017 Chzzhen et al. 2019). However, these fairness-only approaches do not address robust
training as we do. Beyond group fairness, there are other important fairness measures including
individual fairness [Dwork et al. 2012) and causality-based fairness (Kilbertus et al. 2017 Kusner
al. 2017 Zhang and Bareinboim) 2018 Nabi and Shpitser 2018 Khademi et al. 2019] Extending
our al gorithm to support these measures is an interesting future work.
Robust training focuses training accurate models against noisy adversarial data. While data can be problematic for many reasons [Xiao et al., 2013). most of the robust training literature assumes label noise (Song et al. 2020 and mitigates it by using (1) model architectures that are resilient
against the noise [Chen and Gupta 2015] Jiddal et al. 2016 Bekker and Goldberger, [206)] Han et al.
2018a], (2) loss regularization techniques Goodfellow al. 2015 Pereyra al. [2017] Tanno et al.
2019 Hendrycks et al. 2019 Menon et al. 2020]. , and (3)
loss correction techniques Patrini et al.
2017 Chang al. 2017 7 Ma al. 2018 Arazo et al. 2019 However, these works typically do not
focus on improving fairness.
A recent trend is to improve both fairness and robustness holistically [Lee et al. 202 The closest work to our algorithm is FR- Train [Roh et al.) 2020), which performs adversarial training among a classifier, fairness discriminator, and robustness discriminator. In addition, FR-Train relics clean validation set for robust training. In comparison, our algorithm is a sample selection method that does not require modifying the internals of model training and does not rely on a validation set either. There are other techniques that solve different problems involving faimess and robustness. A recent study observes that feature selection for robustness may lead to unfaireess (Khani and Liang] [021] and another study shows the limits of fair learning under data corruption [Konstantinov and Lampert 202T]. In addition, there techniques for (1) fair training against noisy or missing sensitive group
information [Hashimoto et al. 2018 Lamy et al. 2019 Lahoti et al. 2020 Wang et al., 2020 Awasthi
al. 2020 Celis et al. [002]] ]. (2) fair training with distributional robustness [Mandal et al. 2020
Rezaci et al. 2021). (3) robust fair training under sample selection bias [Du and Wu 2021]. and (4)
fairness-reducing attacks [Chang et al. 2020 Solans et al. 2020). In comparison, we treat fairness
and robustness as equals and improve them together.
Sample Selection While sample selection is traditionally used for robust training, it is recently being used for fair training as well. The techniques are quite different where robust training focuses

---

on selecting or fixing samples based on their loss values (Han et al. 20186 Jiang et al. 2018 Chen al. 2019 Shen and Sanghavi 2019), while fair training more about balancing the sample sizes among sensitive groups to avoid discrimination [Roh et al.) 2021]. In comparison, we combine the two approaches by selecting samples while keeping balance among sensitive groups.
7 Conclusion
We proposed to our knowledge the first sample selection- algorithm for fair and robust training. A key formulation is the combinatorial optimization problem of unbiased sampling in the presence of data corruption. We showed that this problem is strongly NP-hard and proposed an efficient and effective algorithm. In the experiments, fairness compared to other baselines. In comparison to the state-of-the-an FR-Train, our algorithm is competitive and casier to deploy without having modify the model training or rely a clean validation set.
algorithm shows better performances in accuracy and
Societal Impact & Limitation Although we anticipate our research have positive societal impact by considering fairness in model training, it may also have some negative impacts. First, our faimess-aware training may result in worse accuracy. Although we believe that faimesss will be indispensable in future machine learning systems, we should not unnecessarily sacrifice accuracy. Second, choosing the right fairness measure challenging. and poor choice may lead to undesirable results. In applications like criminal justice and finance where fairness issues have been thoroughly discussed, choosing the right measure may be straightforward. On the other hand, there are new applications like marketing where fairness has only recently become an issue. Here one needs to carefully consider the social context to understand what it means to fair.
We also discuss limitations. Fairness is a broad concept, and our contributions are currently limited prominent group fairness measures. There other important notions of fairesss like individual fairness, and we believe that extending our work to address them is an interesting future work. In addition, our robust training is currently limited to addressing label noise. However, ITLM is also known to handle ad adversarial perturbation on the features, so we believe our algorithm can be extended other types of attacks as well. Another avenue of research is to use other robust machine learning algorithms Ren et al. (2018) in addition to ITLM.
Acknowledgements
Yuji Roh and Steven E. Whang were supported by a Google AI Focused Research Award and by the Engineering Research Center Program by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. NRF-201 8R ASA and NRF- 2021R 1021RICICI005999) Kangwook Lee was supported by NSF/Intel Partnership on Machine Learning for Wireless Networking Program under Grant No. CNS-20031 29 and by the Understanding and Re- ducing Inequalities Initiative of the University of Siiconsin-Madisone Office of the Vice Chancellor for Research and Graduate Education with funding from the Wisconsin Alumni Research Foundation. Changho Suh was supported by Institute for Information communications Technology Planning & Evaluation (ITTP) grant funded by the Korea government (MSIT) (No. 2019-0-01396, Development of framework for analyzing, detecting, mitigating of bias in Al model and training data).
References
A. Agarwal, A. Beygelzimer, M. Dudfk, J. Langford, and H. M. Wallach. A reductions approach to fair classification. In ICML, pages 60 69, 2018
Y. Akçay, H. Li, and S. H. Xu. Greedy algorithm for the general multidimensional knapsack problem. Annals of Operations Research, 150(1): 2007.
Angwin, J. Larson, S. Mattu, and L. Kirchner. Machine bias: There's software used across the country to predict future criminals, And its biased against blacks. ProPublica, 2016.
E. Arazo, D. Ortego, P. Albert, N. O'Connor, and K. McGuinness. Unsupervised label noise modeling and loss correction. In /CML, pages 312-321, 2019.
10

---

P. Awasthi, M. Kleindessner, and J. Morgenstern. Equalized odds postprocessing under imperfect group information. In AISTATS, pages 1770-1780, 2020.
A. J. Bekker and J. Goldberger. Training deep pages 2682 2686, 2016.
based unreliable labels. In IEEE ICASSP.
E. Bellamy, K. Dey, Hind, S. Hoffiman, S. Houde, Kannan, Lohia, J. Martino, S. Mehta, A. Mojsilovic, S. Nagar, K. Ramamurthy, J. T. Richards, D. Saha, P. Sattigeri, M. Singh, K. R. Varshney, and Y. Zhang. Al fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias. IBM J. Res. Dev., (3(4/5)::1-4:111 2019.
A. Caprara, H. Kellerer, U. Pferschy, and D. Pisinger. Approximation algorithms for knapsack problems with cardinality constraints. EJOR, 123(2):3333345, 2000.
E. Celis, L. Huang, V. Keswani, and N. K. Vishnoi. Fair classification with noisy protected attributes: framework with provable guarantees. In JCML, 2021.
Chang, T. D. Nguyen, S. K. Murakonda, E, Kazemi, and Shokri. On adversarial the robustness of fair machine learning. ArXiv, 2020.
H.-S. Chang, E. Learned-Miller, and A. McCallum, Active bias: Training more accurate neural networks by emphasizing high variance samples. In NeurlPS, 2017.
P. Chen, B. B. Liao, G. Chen, and S. Zhang. Understanding and utilizing deep neural networks trained with noisy labels. In ICML, pages 1062-107,, 2019.
X. Chen and A. Gupta. Webly supervised learning of convolutional networks. In ICCV, pages 1431-1439, 2015.
K. Choi, Grover, T. Singh, Shu, and Ermon. Fair generative modeling via weak supervision. In JCML, 2020.
Chzhen, C. Denis. M. Hebiri, Oneto, and Pontil. Leveraging labeled unlabeled data consistent fair binary classification. In NewrlPS, pages 12760-12770. 2019.
A. Cotter, H. Jiang, and K. Sridharan. Two-player games for efficient non-convex constrained optimization. In ALT, pages 300-332, 2019.
W. Du and X. Wu. Robust fairnes--wware learning under sample selection bias. arXiv preprint wow.::::ii....I... 2021.
F. du Pin Calmon, D. Wei, B. Vinzamuri, K. N. Ramamurthy, and K. R. Varshney. Optimized pre- pre-processing for discrimination prevention. In NeurlPS, pages 3995-4004, 2017.
C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. Zemel. Fairness through awareness. In ITCS, pages 214-226, 2012.
M. Feldman, S. Friedler, J. Moeller, C. Scheidegger, and Venkatasubramannian. Certifying and removing disparate impact. In SIGKDD, pages 259-268, 2015.
M. R. Garey and D. S. Johnson. Computers and Intractabiliry: A Guide to the Theory of NP-completeness. Mathematical Sciences Series. W. H. Freeman, 1979.
I. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.
Google.
Responsible ai practices. sesponsibee-ai-pratiice 2020.
httpp://ai..goggarrrpooobbbbbbis
B. Han, J. Yao, N. Gang, M. Zhou, I. Tsang, Y. Zhang, and M. Sugiyama. Masking: perspective of noisy supervision. In NeurlPS, pages 5839-5849, 2018a.
Han. Q. Yao, Yu. G. Niu, M. Xu, W. Tsang, and Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In NeurIPS, pages 8535-8545, 2018b.
M. Hardt, E. Price, and N. Srebro. Equality of opportunity in supervised learning. In NeurlPS, pages 3315-3323, 2016.
T. Hashimoto, M. Srivastava, H. Namkoong, and P. Liang. Fairness without demographics in repeated loss minimization. In ICML, pages 1929- 1938,1988,1100 2018.

---

D. Hendrycks, K. Lee. and M. Mazeika. Using pre-training can improve model robustness and uncertainty. In ICML, pages 2712-2721, 2019.
IBM. Trusting ai. https://www.researr wtt..mmm.mmmmmmm......m.......tt wtt../c/maaticiiii.
http://iiicaainttlllleeeee//eees htt.aa/aaaaaaattttaaaaaaaaaaaaaa aii/ 2020.
Jiang O. Nachum, Identifying correcting label bias in machine learning. A/STATS, pages 702-712, 2020.
Jiang, Zhou, T. Leung. L.-J. Li. Fei-Fei. Mentorne:: Learning data-driven curriculum for very deep neural networks on corrupted labels. In ICML, pages 2304-2313, 2018.
L. Jindal, M. Nokleby, and X. Chen. Learning deep networks from noisy labels with dropout regularization. In IEEE ICDM, pages 967-972, 2016.
E. Kamiran and T. Calders. Data preprocessing techniques for classification without discrimination. Knowl. Inf. Syst. 33(1):1 33, 2011.
F. Kamiran, A. Karim, and X. Zhang. Decision theory for discrimination aware classification. In IEEE ICDM, pages 224-929, 2012.
T. Kamishima, S. Akaho, H. Asoh, and J. Sakuma. Fairness-aware classifier with prejudice remover regularizer. In ECML PKDD, pages 35-50, 2012.
H. Kellerer, U. Pferschy, and D. Pisinger. Knapsack Problems. Springer Nature Book Archives Millennium. 2004.
A. Khademi, S. Lee. D. Foley, and V. Honavar. Fairness in algorithmic decision making: An excursion through the lens of causality. In WWW, pages 2907-2914, 2019.
Khani and P. Liang. Removing spurious features can hurt accuracy and affect groups disproportionately In ACM PAceT, pages 196-205, 2021.
N. Kilbertus, Rojas-Carulla, Parascandolo, M. Hardt, D. Janzing, and B. Schölkopf. Avoiding discrimina- tion through causal reasoning. In NewrlPS, pages 656-666, 2017.
Kohavi. Scaling up accuracy of naive-bayes classifiers: decision-tree hybrid. In SIGKDD, pages 202 207, 1996.
N. Konstantinov and C. H. Lampert. Faimess-aware learning from corrupted data. ArXiv, 2021.
M. Kusner, J. Loftus, C. Russell, and R. Silva. Counterfactual fairness. In NeurlPS, pages 4066-4076. 2017.
Lahoti, A. Beutel, J. Chen, K. Lee, F. Prost, N. Thain, X. Wang, and E, Chi. Fairness without demographics through adversarially reweighted learning. In NeurIPS, pages 740, 2020.
A. Lamy, 7. Zhong, A. K. Menon, and Verma. Noise-tolerant fair classification. In NewrlPS, 2019.
J.-G. Lee, Y. Roh, H. Song, and S. E. Whang. Machine learning robustness, fairesss, and their convergence. In SIGKDD, 2021.
T. Liu and D. Tao. Classification with noisy labels by importance reweighting. IEEE TPAMI, 38(3)444-611 2015.
X. Ma, Y. Wang, M. E. Houle, S. Zhou, S. Erfani, S. Xia, S. Wijewickrema, and J. Bailey. Dimensionnllty-ddiien learning with noisy labels. In ICML, pages 3355-3366 2018.
D. Mandal, S. Deng, S. Jana, J. Wing, and D. J. Hsu. Ensuring faimess beyond the training data. In NeurIPS, pages 10445-8456, 2020.
A. K. Menon, A. S. Rawat, Reddi, and S. Kumar. Can gradient clipping mitigate label noise? In ICLR, 2020.
Microsoft. Responsible ai principles from microsoft. responsible-aii 2021.
www.:mmccoooooooo httpss//ww..mcroofff.cooo. us/ us/ai/
Nabi and I. Shpitser. Fair inference on outcomes. In AAAI, pages 1931-1940, 2018
A. Narayanan. Translation tutorial: 21 fairness definitions and their politics. In ACM FAceT, volume 1170, 2018.
G. Patrini, A. Rozza, A. Krishna Menon, R. Nock, and L. Qu. Making deep neural networks robust to label noise: loss correction approach. In CVPR, pages
12

---

A. Paudice, L. Munioz-Gnnzälez, and E. C. Lupu. Label sanitization against label flipping poisoning attacks. In ECML PKDD, pages 5-15, 2018.
G. Pereyra, G. Tucker, J. Chorowski. F.. Kaiser, and G. Hinton. Regularizing neural networks by penalizing confident output distributions. 2017.
G. Pleiss, M. Raghavan, F. Wu, J. M. Kleinberg, and K. Q. Weinberger. On fairness and calibration. In NeurlPS, pages 5684-5693, 2017.
M. Ren, W. 7eng. B. Yang, and R. Urtasun. Learning to reweight examples for robust deep learning. In ICMI., pages 4331-4340, 2018.
A. Rezaci, A. Liu, O. Memarrast, and B. D. Ziebart. Robust faimesss under covariate shift. AAAI, pages 9419-9427, 2021.
Y. Roh, K. Lec, S. E. Whang, and C. Sub. FR-Train: A mutual informatton-basses approach fair and robust training. In ICML, pages 8147-8157, 2020.
Y. Roh, K. Lee, S. E. Whang, and C. Suh. FairBatch: Batch selection for model faimesss. In ICLR, 2021.
P. J. Roussecuw, Least median of squares regression. Journal of the American Statistical Association, 79(388): 371-880, 1984.
Y. Shen and S. Sanghavi. Learning with had training data via iterative trimmed loss minimization. In ICMI., pages 5739-5748, 2019.
D. Solans, B. Biggio, and C. Castillo. Poisoning attacks on algorithmic fairness. In ECML PKDD, pages 162 2020.
H. Song, M. Kim, and J.-G. Lec. Selfie: Refurbishing unclcan samples for robust deep learning. In ICML, pages 5907-5915, 2019.
H. Song, M. Kim, D. Park, and J.-G. Lee. Learning from noisy labels with deep neural networks: A survey. ArXiv, 2020.
R. Tanno, A. Sacedi, S. Sankaranarayanan, D. C. Alexander, and N. Silberman, Learning from noisy labels by regularized estimation of annotator confusion. In CVPR, pages 11244-11253, 2019.
S. Wang. W. Guo, H. Narasimhan, A. Cotter, M. Gupta, and M. Jordan. Robust optimization for fairness with noisy protected groups. In NeurlPS, pages 5190-5203, 2020.
T. Xiao, T. Xia, Y. Yang. C. Huang. and X. Wang. Learning from massive noisy labeled data for image classification. In CVPR, pages 2691-2699 2015.
X. Yu, T. Liu, M. Gong, K. Batmanghelich, and D. Tao. An efficient and provable approach for mixture proportion estimation using linear independence assumption. In CVPR, pages 4480 -480, 2018.
M. B. Zafar, I. Valera, M. Gomez-Rodrigun and K. P. Gummadi. Fairness constraints: Mechanisms for fair classification. In AISTATS, pages 962-99 270.970.222 2017a.
M. B. Zafar, I. Valera, M. Gomez-Rodriguez, and K. P. Gummadi. Faimess beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In wwW, pages 1171-1180, 2017b.
R. S. Zemel, Y. Wu, K. Swersky, T. Pitassi, and C. Dwork. Learning fair representations. In JCML, pages 325-333, 2013.
B. H. Zhang, B. Lemoine, and M. Mitchell. Mitigating unwanted biases with adversarial learning. In AIES, pages 335-340, 2018.
J. Zhang and E. Bareinboim. Fairness in decision-making the causal explanation formula. In AAAI, pages 2037-2045, 2018.
13


Reviews:
[{"review_id": "jG61QDrT8Fl", "reviewer": "Reviewer_ERAJ", "summary": "This paper proposes an approach for training machine learning models which are simultaneously fair (with respect to conventional group fairness definitions) and robust to a fraction of the training data being corrupted. The idea is to, over the course of training, select a subset of instances to focus on which simultaneously trim off instances with high loss (likely outliers) while also up-weighting selected sensitive groups. Experimental results are shown for two standard datasets.\n", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Combining fairness and robustness is a worthwhile goal -- the example in Fig 2 is a nice illustration of how naively applying methods aimed at robustness could worsen fairness. A strength of the method proposed in this paper is that it is simple and efficient to implement, and can be applied to almost any ML model. The experimental results also show some promise. \n\nThere are two main weaknesses:\n\nFirst, the method considered is also somewhat incremental, more or less combining an existing approach for robustness and an existing approach for fairness. It's not clear if there is a big conceptual lesson learned from this.\n\nSecond, the experimental results are also somewhat mixed compared to previous work. Compared to the existing FR-Train method, the proposed method seems to sometimes improve on fairness while sometimes worsening accuracy a bit. A couple of suggestions: (a) showing a complete tradeoff curve between accuracy and fairness for both methods, to get a more complete idea of the Pareto frontier and when each method might be more appropriate (b) it could be helpful to provide some context for the magnitude of the relative differences in each measure. E.g., what does it mean for accuracy to change by 0.02 but EO to worsen by 0.03? Is this typically a good tradeoff?  \n\nAll together, given that the method is relatively incremental, it would be better to see a cleaner experimental evaluation to suggest that it will have a big impact in practice. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1.5"}, "extra_text": {}}, {"review_id": "NtVXJ4z6G_1", "reviewer": "Reviewer_ifBb", "summary": "This paper designs a classifier that is fair and robust. The authors choose sample selection as a means to ensure robustness. Such sample selection strategy eliminates points that have large loss. Fairness constraints are enforced by explicitly adding these constraints during training. \n\nThe overall optimization problem is setup as a bi-level optimization problem where the outer optimization problem selects the hypothesis and the inner optimization problem selects a subset of samples. Additionally, Lagrangian multipliers are introduced for fairness constraints and are updated according to a heuristic. Therefore, in my view, the entire optimization problem looks like a min-max optimization problem ($\\min_{\\theta, S} \\max_{\\lambda} \\ell(\\theta,S) + \\lambda \\textrm{Fairness}(\\theta, S)$) although the authors didn't explicitly mention this formulation.", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: There have been several works on designing fair and robust classifiers (1. Ensuring fairness beyond the training data, Neurips-2020, 2. Robust fairness under covariate shift, AAAI-2021). Unfortunately, they are neither mentioned in the related work and nor compared against in the experiment section. \n\nThe authors are mainly concerned about robustness against label flipping in the experimental section. This is a very restricted form of robustness compared to distributional or adversarial robustness. Moreover, if one only cares about label noise, then probably a better approach is to consider better loss functions.\n\nClarity: I could follow the overall algorithm, and the various choices made in the design. However, the presentation needs to improve in several places. \n1. The authors should be upfront about what kind of robustness they are targeting and why the proposed strategy is the best fit for that objective. \n2. The authors made a choice in directly encoding the fairness constraints during training. This is an in-processing approach and why should we prefer such an approach rather than learning a fair representation of the selected subset $S$ of samples.\n3. Please clarify how ILTM is run for figure 2. Do you run a full iterative method or just discard samples based on an already trained model according to the optimization problem defined on page 3 (first equation)?\n4. Finally, the authors should discuss why the proposed algorithm converge or why is it supposed to work. From the overall algorithm, it seems that the $\\lambda$-variables are updated through gradient ascent and the $(\\theta,S)$ parameters are just best responses. There is no reason to believe that last iterate of this procedure converges. However, the average prediction over all the iterates is the right choice and have been used in reduction-based approaches to fair classification.\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Although the design of fair and robust classifier is an interesting problem, there have been already several work in this area. Moreover, the paper has several limitations.\n1. The choice of sample selection strategy is not well-motivated.\n2. The kind of robustness guarantees provided is weak and there are already several work in designing fair classifier with adversarial and distributional type of robustness.\n3. There is no formal guarantee of the proposed algorithm and moreover this need not be the right approach (see my earlier comments).\n4. Several related work on fair and robust classifiers are missing and the authors haven't compared the proposed classifier with these existing methods.", "review_text": "", "extra_scores": {"ethical_concerns": "No.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "Five Hours"}, "extra_text": {}}, {"review_id": "I5ciN11kzri", "reviewer": "Reviewer_1QFb", "summary": "This paper presents an approach that does fair learning while ensuring it is also robust to label noise. The authors do so by proposing a combinatorial optimization approach that can be solved using a greedy algorithm. They compare their approach to a few baselines and show the superiority of their technique.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I will divide my review into a list of pros and cons.\n\nPros of this paper:\n- The paper proposes a sample selection framework that is aimed at both robustness and fairness. From the empirical results, it is shown that the proposed method outperforms all the baselines considered both in terms of fairness and accuracy measures.\n- It is an interesting approach to integrate robustness criteria with fairness constraints and solving it by posing it as a multi-dimensional knapsack problem.\n- The authors also provided a detailed empirical analysis of the impact of various factors like noise ratio, fairness constraints, and ERM weights on the model’s performance.\n- The authors show the NP-hardness of the optimization problem.\n\nCons of this work:\n- From the novelty perspective, the approach proposed in this paper is not very novel and is highly derived from previous works like ITLM and FB. Apart from this, there are no significant theoretical contributions from the work. \n- This paper seems like combining two well-known ideas, i.e. ITLM for robustness and FairBatch from fairness. This seems a little incremental in my opinion. \n- The experiments were done on binary label datasets like COMPAS, AdultCENSUS using a logistic regression model as classifier. It would be interesting to see the performance and the intricacies of the proposed approach using multi-label classification datasets.\n- Furthermore, in order to understand the scalability and effectiveness of the proposed approach in a deep learning setting, it would be interesting to see some experiments on datasets like UTKFace, FairFace using deep models.\n- Also, how does the algorithm work when applied to deep models as classifiers? The authors can consider CELEBA task (e.g. smile detection) with gender as a protected attribute, or age/gender recognition in UTKFace with race as a protected attribute. \n- The most important issue in my mind with this paper is why do a joint optimization (like the way proposed here) in the first place. There are literary \"N\" robust learning algorithms and \"M\" fair learning approaches (either pre-processing, in-processing and post-processing based), where N and M are large numbers growing by the year. Obviously, it is not possible to compare to all NxM combinations, but why will a joint learning approach perform better than an approach, that first, say filters out noisy examples and then does fair learning? \n- Related to the above and also the second con, the paper basically combines two well-known ideas, i.e. ITLM for robustness and FairBatch from fairness. Why only these two? Even if one could do a joint optimization (like the way proposed), we could easily formulate an approach that can \"combine\" these in a principled manner. Again, it is not possible to explore every possible combination, but I would have liked the authors to discuss why these two approaches would work well together and why the proposed idea would qualitatively be more principled compared to another such approach. Sorry, this comment is a little vague but I would have just liked some more elaboration on this.\n- For example, the authors mention \"we are not aware of a general data cleaning method that is designed to explicitly improve model fairness and robustness together\". I do not think one needs to do this together. One can clean the data first (there seem to be many stable and off-the-shelf approaches to do so) and then do an in-processing or post-processing approach. Though this would change the \"dataset\", it is still fine because in the end it is applying fairness only on the useful dataset.\n- Also the authors can check out a recent paper (https://arxiv.org/pdf/2105.11570.pdf). There seems to be some similarities in the approach (again, I don't think this is published work so it might be ok to not compare against them but it would be good to comment on the differences).\n\nOverall, I like the idea of doing fairness and robustness jointly. ML papers are often focused on only one desiderata along with accuracy (i.e. either only robustness and fairness), so this paper is solving the \"right\" problem and is headed in the right direction. However, I do think there are issues with this paper. One of the major one is why does it not work to just blindly (and sequentially) combine robustness and fairness together (e.g. noise filtration followed by either an in-processing or post-processing fairness approach). Why joint learning is important and why is the proposed combination the only way? See also some of my other comments.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Overall, fairness is an important problem as is robust learning. I do not really see as many \"negative societal impacts\" of this work.", "review_text": "", "extra_scores": {"ethical_concerns": "None in my opinion.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5 hours"}, "extra_text": {}}, {"review_id": "9G4hAjhuN4", "reviewer": "Reviewer_Dqdb", "summary": "This paper proposes a new framework of fair sample selection. The paper introduces fairness constraints into clean selection and formalize the optimization problem in the same format as the knapsack problem. Then the authors propose an algorithm iteratively uses FairBatch to decide the fraction of the sample size of each group, uses a Greedy algorithm approximate algorithm (for the knapsack problem) to select samples, and update model parameters. The experiments show that the proposed algorithm achieves better fairness performance and sometimes also achieves higher accuracy.", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: Improving fairness and robustness by an adaptive sample selection approach is new. The method proposed in this paper combines some well-established methods, including ITLM, FairBatch and a greedy algorithm for the knapsack problem. \n\nQuality: The submission is technically sound. The results of fairness are well supported. For the robustness, it is unclear to me what type of attacks (e.g., adversarial perturbation, data poisoning) can the proposed method deal with. In the experiments, the authors consider injecting noise into a fraction of training data (or a specific group). Is this the only type of attack that the algorithm is robust to?\n\nClarity: The submission is well written and organized. As mentioned above, more information on robustness should be addressed.\n\nSignificance: The proposed method performs competitive to the SOTA w.r.t. accuracy and outperforms the SOTA w.r.t. fairness.\n\nOther comments:\ntypo: in Eq. (6), the absolute value symbol should be removed.\n\n=====================================================\nAfter reading the authors' response and others' comments, the contribution w.r.t. robustness is not convincing to me. Thus, I changed my score.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3.5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Reviewers broadly appreciated the focus of the paper, and agree that the intersection of fairness and robust is a rich and relatively underexplored space.  Yet, reviewers found the work lacking in scope (e.g., with respect to the specific types of robustness considered and the characterization of the methods/approach) and novelty (e.g., again with respect to robustness, reviewers found that this was a very straightforward application of a known method).  Our private discussion largely followed the public commentary that occurred between Reviewer 1QFb and the authors, so I would encourage the authors to take that sentiment and those comments into account moving forward.  Broadly speaking, all reviewers (and the AC) feel that this work is promising but, at present, underdeveloped.
Decision: Accept (Poster)