Fair Infinitesimal Jackknife: Mitigating the Influence of Biased Training Data Points Without Refitting
Authors: Prasanna Sattigeri
Soumya Ghosh
Inkit Padhi
Pierre Dognin
Kush R. Varshney
Venue: NeurIPS 2022 Accept (2022)
Keywords: Fairness
Influence Functions
Infinitesimal Jackknife

Abstract: In consequential decision-making applications, mitigating unwanted biases in machine learning models that yield systematic disadvantage to members of groups delineated by sensitive attributes such as race and gender is one key intervention to strive for equity. Focusing on demographic parity and equality of opportunity, in this paper we propose an algorithm that improves the fairness of a pre-trained classifier by simply dropping carefully selected training data points. We select instances based on their influence on the fairness metric of interest, computed using an infinitesimal jackknife-based approach. The dropping of training points is done in principle, but in practice does not require the model to be refit. Crucially, we find that such an intervention does not substantially reduce the predictive performance of the model but drastically improves the fairness metric. Through careful experiments, we evaluate the effectiveness of the proposed approach on diverse tasks and find that it consistently improves upon existing alternatives. 

Fair Infinitesimal Jackknife: Mitigating the Influence of Biased Training Data Points Without Refitting
Prasanna Sattigeri IBM Research
Soumya Ghosh
Inkit Padhi
MIT-IBM Watson Al Lab, IBM Research
IBM Research Yorktown Heights, NY
Yorktown Heights, NY
Cambridge, MA
psattigdus. imm.mmmo....... com
ghoshso@us imm. com
inkit. padttipaahiii @ibm. com
Pierre Dognin
Kush R. Varshney
IBM Research Yorktown Heights, NY pdognin@us. ibm. com
IBM Research Yorktown Heights, NY
krvarshn@us 1imm com
Abstract
In consequential decision-making applications, mitigating unwanted biases in machine learning models that yield systematic disadvantage to members of groups delineated by sensitive attributes such race and gender is one key intervention to strive for equity. Focusing on demographic parity and equality of opportunity, in this paper we propose an algorithm that improves the fairness of a pre-trained classifier by simply dropping carefully selected training data points. We select instances based on their influence on the fairness metric of interest, computed using an infinitesimal jackknife-based approach. The dropping of training points is done principle, but in practice does not require the model to be refit. Crucially, we find that such intervention does not substantially reduce the predictive performance of the model but drastically improves the fairness metric. Through careful experiments, evaluate the effectiveness of the proposed approach on diverse tasks and find that it consistently improves upon existing alternatives.
1
Introduction
Among the many possible interventions to improve equity in society (most of them involve structural policy change), bias mitigation algorithms constitute one narrow sliver that has emerged in the machine learning literature to address distributive justice in high-stakes automated decision mak- ing. These algorithms may be categorized into pre-processing, in-processing, and post-processing approaches 37]. In the case of in-processing algorithms 200) the bias mitigation intervention occurs at the model training stage. This is usually achieved by minimizing the empirical risk regularized by a fairness metric surrogate that captures the dependence the prediction and the sensitive attribute. Pre-pre tre-processing methods typically learn transformations of the data distribution such that they do not contain information about the sensitive attributes 4R] 26 Task specific models are then learned from scratch these debiased representations. Retraining model from scratch is intractable in many real-world situations for a variety of reasons including policy, cost, and technical feasibility: post-processing approaches are the only viable option in such cases. For example, consider trying to refit large foundation models. Limiting ourselves to notions of group fairness such as demographic parity and equality of opportunity, existing post-processing bias mitigation algorithms tend to either randomly or deterministically alter the hard or soft predicted label of individual test data points that have been scored by a model 10
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Adult
Coverage
CivilComments
3.5
15.0
3.0
10
10.0
30
7.5-
20
5.0
10-
2.5
0.0
10000 12500 15000 17500 20000 Demographic Parity Influence
20000 25300 300n0 3500c 40000 45000 Demographic Parity Influence
240000
250000
260000 270600
Demographic Parity Influence
Figure I: Fairness influence. We plot the sorted influence scores of training instances on the average demographic parity of a held-out validation set for different datasets considered this paper. Only training instances with positive influence on demographic parity are plotted. The points to the right of the black line are the 500 most influential training instances. Models adjusted to mitigate the influence of these instances are substantially more fair. On held-out test sets of the Adult, Coverage, CivilComments datasets, demographic parity improves from 0.18 to 0.01, from 0.22 to 0.01, and 0.17 to 0.03.
In this paper, we propose a more "global" bias mitigation algorithm. Our procedure alters the entire model without focus individual test points. Similar to post-processing approaches, our method mitigates pre-trained models without requiring any additional refitting the model. Unlike standard post-processing approaches, however, our method does require access to the training data. In exchange for this additional requirement, we find that our approach typically substantially outperforms other post-processing techniques and can augment in- processsin approaches for better fairness/accuracy trade-off.
Our contributions. Our first contribution is methodological. We use the notion of influence functions to estimate the "influence" of training instances on various group fairness metrics of interest. We then perform post-hoc unfairness mitigation by approximately removing training instances that have a disproportional impact on group (un)fairness. We theoretically analyze the proposed approach and establish conditions under which it provably improves group fairness.
Next, we observe that influence calculations require the inversion of Hessian matrix, pro- hibitively expensive operation for models with a large number of parameters. Existing approximations 11 34 can either be expensive, inaccurate, or unstable 22 35
We develop IHVP- WoodFisher, WoodFisher [35] based Inverse-Hessiat Vector Product (IHVP) scheme for computing the fair- ness influence score of the training instances that is stable, easy to compute, and does not require constraints, such as restricted Figenspectrum of the loss curvature, that are hard to satisfy in practice.
Our final contribution is empirical. First, through careful experiments tabular data, show that our approach is effective at reducing group unfairness, is competitive with existing methods, and can even augment the latter to achieve better fairress/accuracy trade-off. Then, demonstrate how our approach can be easily adapted to more complex modalities such as natural language and be used for bias mitigation of large pre-traine tre-trained language models through prompt-tuning tuuiigg a use-case that is likely to become increasingly common with the proliferation of large language models.
2 Background and Related Work
2.1 Empirical and Weighted Risk Minimization
We begin by considering the standard supervised learning setup. Given a dataset = [z,, = (Xn, yn)) ).W of N features (x,, RR",, response pairs (yn, E )),, model (x(x) parameterized by set of parameters E θ C RD, and loss function é y - R, we minimize the empirical risk,
N (ynn he (xx)),
0 argmin dce
1=1

---

arrive at trained predictor hz(x). We will denote L( convenience. Next, consider weighted risk minimization problem,
Σ
((yn, for notational
N
1 N
0 (w) = argmin EE
Σ
Wn Eyn, ho (x.)),
(2)
1=1
that weights the loss at each training instance by a scalar weight wn and w denotes the column vector
[W1, W2, .... wNl I E RR, Setting all the weights to one, 1 def [w 1, I...., WN = 1J² and minimizing the right hand side of Equation 2 recovers the standard empirical risk minimization problem. On the other hand, setting the 1tth coordinate to recovers the solution to empirical risk minimization problem after dropping the n²h training instance. As is clear from Equation 2 and emphasized by our notation, 0 is function of the weights w. Although typically do not have closed form expression for this function, we can form a Taylor approximation to it:
0(w (w) = 0 ((w
(w 1) ((w-1))²)
(3)
w=1
where Vw E RDN is the Jacobian matrix. This first order Taylor approximation often referred to as the infinitesimal jackknife approximation [ [4]. The coordinate- -eree gradient do(w) die,, 11c measures the effect of perturbing the weight of the nh data point on i and is popularly referred to as the influence function [21], since it measures the "influence" of the nhth training instance on the model's parameters. When we are at a stationary point of (.),,, i.e., when Vol.(0) = 0, (.)))
is twice differentiable in 0, then,
d0 (w) dwn
HH gn:
(4)
w=1
where H def VoT(())e -= and gn above expression approximates the gradient well in the vicinity of stationary point with the accuracy of the approximation deteriorating smoothly with increasing distance from the stationary point. This result justifies the use of influence functions even when stochastic optimization is used for minimizing Equation D Finally, to measure the influence of training instance on a differentiable functional, M, of ((w)( apply chain rule to arrive at.
def Vol(yn, (x,)) e -- Recent work [5] has shown that the
IM. def d.)(O(w,, w)
T
veM((((w), w)
H
gn.
(5)
dwn,
w=
w=1,0=6
where our notation makes explicit the dependence of M w. Recent work has leveraged this
machinery to approximate cross-validation [16 36 [5], to interpret black-box machine learning
models (1I, and to assess the sensitivity of statistical analyses to training data perturbations [6]. among others. Differently from these, we show how this machinery can be leveraged for reducing disparities of pre-trained models across groups
2.2 Fair Classification
We further assume that for each data instance we have access to a sensitive attribute S, E [k], i.e.. D = [z., (Xn, Sn, yn))
1==1 ) that encodes the protected group membership of the nh data instance and that we are interested in binary classification, ) = {1,0}, In fair classification, we want to learn accurate classifiers that minimize disparities in predictions across groups.
To quantify disparities across groups. we primarily focus on two common fairness metrics demo- graphic (or statistical) parity (DP) [3] and equality of odds (EO) [[7]] DP requires the classifier's 's predictions to be statistically independent of the sensitive attribute, (x) JL S, where X and S are random variables representing the features and the sensitive attribute. For a binary sensitive attribute, DP implies, (X) = S 1) = P(ho(X) = S 0). EO, on the other hand, requires the classifier's 's predictions to be statistically independent of the sensitive attribute conditioned on the true outcome, (x)
Y. For a binary sensitive attribute, EO implies Y y) P(ho = 1 S = 0, = y) for both y = 0 and y = 1.
P(ho(X)= = Equality opportunity (EQOPP) 17 a special case of equality of odds where the predictions are conditionally independent of the sensitive attribute given the true outcome is positive. A common
3

---

strategy for learning fair classifiers is to then require the absolute difference in demographic parity (DP),
(D = P( ()= = | S (x)
1 =0);
or the absolute difference in equality of odds,
A =
1 ==,==11 = - P()h((X) = - =, =y>1
= =
==0
to be close to zero while minimizing the empirical risk (Equation I and AEO that are estimated from an empirical distribution are commonly used in practice 42
Smooth surrogates to ADP
MD (0) = x00xx=
(6)
M
A (0) = Σ
-x- (x [ho(x]]
9=0
where (0) denotes the surrogate for the faimess metric b estimated from dataset a. When 0 is itself function of we will use the notation (1(O(w), w).
2.3 Other Related Work
Many pre-processing based bias mitigation algorithms, learn low dimensional representations of the Others aim to learn fairness promoting Pre-processing methods that transform the data points can often run the risk of losing the semantics of the original data points. Often, they can be expensive, especially for high-dimensional data and large datasets. Furthermore, they must be performed before training any task-specific models and thus are not applicable when the goal is improve a model already trained with an expensive procedure. obtain a counterfactual feature distribution by identifying the test instances, which when dropped the pre-trained model predictions are fair on the remaining test instances. They then learn a optimal transport based randomized pre-processor that maps the transforms the new test samples from the unprivileged group to fair counterfactual distribution. In contrast, our goal to compute the influence scores for the training instances, which is more challenging. Additionally, KS requires the sensitive attributes known at test time as the pre-procesoor is specific to the unprivileged group. Instead, we aim to directly edit the trained model and eliminate the need of sensitive attribute labels at test time.
data that are independent of the sensitive attribute 43
26
transformations in the ambient space of the data 22 40
[8), the authors first
In-processing algorithmic fairness methodologies X are applicable when we can train models along with fairness constraints. Mary et al. 227 enforce independence through relaxation of the Hirschfeld-Geeelein-- Rényi Maximum Correlation Coefficient (HGR) dependency measure. Similarly, Rebias 2 41 uses the Tillbert-Schmiie Independence Criterion (HSIC) to reduce the dependence of the representations on the sensitive attibutes. FairMixup (9] is a data augmentation strategy to improve the generalization properties of in-processing algorithms. These methods can be sensitive to the regularization strength and can sacrifice too much accuracy. In contrast, our approach applicable when the base model to trained unconstrained on the main task, which can then be updated to remove the influence of the harmful instances and improve fairness.
Existing post-processing methods trained model to satisfy a measure of fairness. These can often be limiting as they do not provide control over the fairness accuracy trade-off, may require that predicted scores to be cell-calibrated, or may lead excessive reduction in performance. In contrast, our method exploits the training data and model gradients efficiently to generate stronger, yet computationally inexpensive post-hoc interventions at minimal loss of predictive performance.
19
25
learn to transform the predictions of a
3 Fair Classification through Post-Hoc Interventions
We now develop and analyze a post-process processocssng faimess algorithm that given (i) a pre-trained model, (ii) access to the training data and optionally validation set, (iii) a twice differentiable loss function
Nearly smooth. The absolute value is not differentiable at zero, but this is not a concern since we rarely encounter exact zeros in practice.

---

and once differentiable surrogate to the fairness metric, and (iv) an invertible Hessian at local optimum of the loss, improves the fairness characteristics of pre-trained model without requiring to be refit.
3.1 Influence Functions for Group Fairness
Nat Assuming that we use a held-out validation set Dval {xn, Sn, J== to estimate M M. (0) and (O)) can leverage the result in Equation instance on ADP,
to compute the influence of the teth training
IADP,
Va DeM (0) O' ga.
Vo (X= [hox)) -
(aapl//(xxl__pxxxxx/aaaaa Epo(X=x(S==)) /ha [x(x))] ا= - H gm.
and ABO,
IABO,
V AFO (0) (0) Dial
gn:
(8)
vel -
Σ
(xoxxx - -ie--E-pa Exp [o(
H
gns
y=0
0=0
where we have used MADP (0) and MA (0) to denote M (f w) w=1,0= and
(O w=1,0=0 We highlight that computing the influence of training instances on
group fairness metrics requires solving a single empirical risk minimization problem to recover The fairness metrics could also be estimated on the training data if no validation is available. However, empirically we find that a validation set improves results.
3.2 Post-Hoc Mitigation
note that the first order Taylor approximation about is function of w. This opens up the possibility post-hoc fairness improvement of pre-trained by searching for weight vector Wiarr such that M rair) where,
Revisiting Equation 3
(f
fair
def
0 ((Waii))
+
N
d0 d0(w)
(w)
fair (wp
- 1),
dwn
n=1
wwm1
0
H
Fair
1);
==1
and Wfair [uefair fair optimizing desired (0()), w) with respect requires ON D operations and is prohibitively expensive for large models. Instead, iterative procedures involving repeated Hessian-vector products are often used in practice [21] A gradient- based procedure would need to either perform this iterative procedure after every gradient step or pre-compute Σn, H "n. rendering the procedure computationally intractable for most cases of interest. Moreover, solely optimizing (O(w), w) will likely result fair but inaccurate classifiers, and the optimized weights will typically not be interpretable.
W T T
E N... We could use gradient- based methods to learn Wiair by However, computing and inverting the Hessian
We circumvent these issues by constraining the elements of w be binary. In Proposition 3.1 we show that we can construct Wain by simply zeroing out coordinates of Waarr that correspond to training instances with positive influence on the fairness metric of interest. This construction inherently interpretable. Setting element to zero implies training without the corresponding training instance. Zeroing out instances with positive influence equates to refitting the model after dropping training instances that increase disparity across groups.
We now establish conditions under which Wiair constructed above leads to classifiers with lower group disparities. Let 1 e V denote an N-dimensional vector of all ones, b denote a fairness metric, M (((w). (w), w) denote a linearized approximation Mb (0(w)), w) about 1, and 1 > ,3] denote indicator function that takes the value one if a 3 is true and zero otherwise.
5

---

Proposition 3.1. Let Wiair E {0.1)* V be N dimensional binary vector such that its th coordinate wp =1 1 To > 0). then,
Wiair argmin M D.n. (0(w), w) -M (0(1), wE(0,1)
and M Dena (0( (Wttir)), Wfair) M Dval (0(1), 1) S.
Proof.
Denote M (0) == MD (f 1). From a first order Taylor approximation about 1, we
have,
dM (Ō(w), w)
M Dan (0(w), w) -M (0)
Dual
(wn 1),
dwn
(10)
MM (0)
Io (wn 1).
Rearranging terms,
N
M M (f(w), w)M)) (0)
Σ ==1
In -1)
(11)
N
N
Σ
1 > Of - 1) + Σ n=1
1 S 3) 1).
n=1
Finally, the result follows from observing that w/ E f0. 1} and noting that the first term can be either
zero (when W,, or negative (when w,, = 0 and 1 [ 0)) and the second term can be either zero (when w., = 1) or positive (when w., = 0 and 1 [Z < 0)). Wfair drives the second term to zero and sets the first term to the smallest value attainable by a binary w.
follows that M3 (air),,) Wiate) RS M (0(1). 1). with the inequality holding when the linearization is accurate. Finally, defining D_ {z,, Zn E and IM 0}. we arrive at the post-hoc mitigated classifier by plugging in Wfair from Proposition 3.1 in Equation y
i fair θ
Σ mED
H H'g... gm.
(12)
In Appendix A we consider an alternate Wiar that is guaranteed decrease both the loss and the fairness metric on the validation set D In early experiments, we did not see consistent benefits from using this alternate version and do not consider it further this paper.
3.3 Practical Considerations
Hessian computation and inversion. The influence function computation involves computing and inverting the Hessian of the loss function the training data. This requires ONN D2 operations. Both computing and storing the Hessian becomes prohibitively expensive for large models. While diagonal approximations to the Hessian are possible, they tend to be inaccurate. Instead, iterative methods based on the (truncated) Neumann expansion have been proposed in the past 21. However, more recent work has found the Neumann approximation to be inaccurate, cf. B3. Appendix CJ and prone to numerical issues when the eigenvalues of the Hessian fall outside the [0, 1] interval. Motivated by these shortcomings. here we develop an alternative iterative procedure based on the recently proposed WoodFisher approximation 35
The WoodFisher approximation provides us with the following recurrence relation for estimating the inverse of the Hessian:
A. (o
A 1 A
(13)
N + Vol(yn+1. H, VoL(Un+1, ho hx(xx+1))
with A,' - Lp. and A a small positive scalar.

---

For computing influence functions we only need to store the product of the inverse Hessian with vector v, i.e. Ho'H1y 'v,, which should only require ((D) storage. However, if we first compute the inverse Hessian and then compute the Hessian-vector vector product (HVP), we would need O(D)) storage. To sidestep this issue, we develop the following coupled recurrences that only use ((D) storage. We call these coupled recurrences IHVP-Woode WoodFisher,
On+1 On On Vol(zn+1) On On
On V Vol(zn+1) Tnn
+ Tol(zn+1))o kn-1 knp
(14)
N V Fal(zan (z(+1) On
where, we use ((zn+1) ) as shorthand for hoott,,oo(xx+)))
Vollyr, (xi))) and k = v.
Proposition 3.2. Let 01 Vol(zı). k1 v, and N denote the number of training instances. The Hessian-vector product H is approximated by iterating through the CEVP-WoodFssher WoodFishherrecureeec recurrence in Equation 14
and computing ky.
We prove Proposition 3.2 in Appendix In practice, observe that even using B < N iterations produces useful approximations. In Appendix C we compare the approximation accuracy of the the WVV-WWodFFisher and the iterative Neumann approach cases where it is tractable to exactly compute the IHVP. Algorithm 2
(see Appendix) summarizes our vanilla approach.
Computational speedups: Although Algorithm 2 suggests running the HyP-WWodFiishe iterations for each training instance for clarity of exposition, in practice, we use the following trick to run the
iterations only once for the entire training dataset. First, for any p * 2 symmetric matrix and p-dimensional vectors x and y. x" A, y y Ax. From Equation 5 the influence calculation involves computing V (0, H 1 gn for all n in the training dataset. Since is symmetric, we can equivalently compute g H ( 1). We can then run the /HVP- WoodFisher iterations to approximate H Vo 1). Crucially, we need to do this only once. With the approximation in hand, computing the per data influence requires single dot product per data instance between Ga and the /HVP- WoodFisher approximated H Vo. 1). contrast to other approaches to scaling up influence functions
33 our approach only requires the storage of a single p-dimensional vector. We call this more efficient version Fair-IJ and is summarized in Algorithm
Most influential instances. Our development and analysis depends on first order linear approxima- tions of non-lineat functions about 1. We expect the quality of these approximations to deteriorate further away from 1, i.e., with increasing number of instances dropped. See Theorem in 6 for additional discussion the quality of approximation, We find that instead of dropping all instances with positive influence, dropping the & most influential instances yields better bias mitigation. We select the hyperparameter k that results in the lowest (best) fairness score on the validation set. Additionally, (35) observed that the WoodFisher Hessian estimate Ā differs from the true Hessian by scaling factor, i.e H x H. We select, from pre-specified set, the scaling factor that minimizes the fairness score on the validation set. We then scale the IHVP- WoodFisher estimates using the selected scaling factor. See Appendix D
Experiments
We first study our method on tabular datasets including the well-known Adult dataset I and the recently released ACSPublicCoveree datasets aimed be larger alternatives to previously available fairness datasets. We then investigate our method on the text modality and larger pre-trained models using the CivilComments dataset [5].
[1]] dataset. ACSPublieCoverage is one among a suite of
4.1 Tabular Datasets
Setup. The task in the Adult dataset to predict if person has an income above a threshold. We use gender as the sensitive attribute. This dataset comes with a fixed test set. A random 33% of the training data is used as the validation set for each trial of the experiments, We follow the pre-processing steps from 27 The task in the ACSPublicCoverage dataset is to predict if person has public health insurance coverage. For our experiments, we only consider instances from the year 2014, from the state of California, and belonging to the white or black race. We consider race the
7

---

Algorithm Fair-IJ
1: Input: Pre-trained model parameters training set D. loss function L, a validation set Dval and smooth surrogate to the fairness metric b {ADP, ΔEO), M.
2: Calculate: Vo Vo 1) using Equation or Equation
3: Calculate: H ' ( 1) by setting k1 = V M (Ô, 1) and iterating through Equation 14
for iterations.
4: Calculate: the fairness influence Ia,n of each training instance on vat by computing dot product between Yn, and r.
5: Construct: the set D_ and denote its cardinality, |D-| K.
6: Initialize: 00 = θ
7: for [1..... K do
8: Construct: D = (zn D_ Tb.n > [b,(K- 1)}, where Is.(K- denotes the (K k)th order statistic the influence scores [I,,I,..., I..K]
9:
Calculate: fair by replacing D_ with using D in Equation 12
If bD... (0 < bD... (0 set Ótair =- Oh else set O 0 and break of the for
loop. end for
11:
12:
Return: fair model parameters raie-
adult DP
adult: EO
adult -OOUt-EOOPP
0.24
FairMbxup
ERM
Fairiixpp
ERM
FairMbxup
ERM
HGR
HGR
HGR
FST
Fairhiiup-]]
FairMoup-i
HPS
HGR-I
HGR-j
FarMMioup-))
Enorr rate
Fiirs))
0.20
0.20
HGR-
0.20
Fai-ij
PST-B
0.18
0.18
HPS
0.18
Fair-])
0.16
0.14
0.00
0.05
0.15
0.14
0.00
0.05
0.15
0.20
0.25
AFOOPP
coverage-DP
0.8
coverage-EO
EQOPP
ERM
FairMixup
FarNiccu
0.255
HGG
PST
0.25
HGR
rate
0.250
TwitMiuu--)) HGR-I
FairMocup-l] HGR-1)
HPS
Fruir cana
0.50
HER-I
Fair-
0.245
PST:I]
0.245
Fair:)
HPS-1
0.240
Faii-]]
0.240
0.4
0.355
0.230
0.00
0.15
0.23
0.0
0.1
0.3
0.4
0.230
0.00
0.30
0:15
0.25
ADP
AECOFP
Figure 2: Accuracy and faimess (DP, EO, and EQOPP) Pareto frontier for the Adult and the Coverage datasets averaged over 10 runs. Points closer to the bottom-left achieve the best fairness/accuracy trade-off.
sensitive attribute. In the rest of the paper, we refer to this subset simply as Coverage dataset. We randomly split the dataset into rrain/vliiattionttst with partition 50% / 20% 30%, respectively, for each trial. Additionally, for both datasets we standardize the features before training our methods and the baselines.
first train 1-iiden layer fully connected neural network with SeLu activation function and 100 hidden units. This initial model is trained using standard ERM loss with batch size set to 256. We use the Adam optimizer with learning rate set 10-4 We train the ERM and baselines for 100 epochs and pick the checkpoint with best accuracy the validation set. We then employ Algorithm

---

arrive at the Fair-IJ solution. We select i and the IHVP scaling term based on the same validation set used compute the influence scores.
Compared algorithms. We compare our method to several in-proeessing and post-processing bias mitigation algorithms that are applicable wide range of model classes including deep neural networks. We omit the comparisons with pre-processing methods as our goal is to improve a given pre-trained model. Among in-processing algorithms, we compare with: FairMixup [ and HGR 127.. FairMixup achieves fairness through a mixup based regularization employed during training. The HGR approach proposes a surrogate to HGR dependence measure and promotes faimesss during training by enforcing conditional independence implied by the fairness metrics. On both of these methods, the faimess accuracy trade-off is achieved through the strength of the regularizer.
Among post-processing methods, we compare with FST 33 and HPS 17 FST optimally transforms the pre-trained model's prediction scores to satisfy specified fairness constraint and supports DP and EO metrics. To improve the performance of FST we re-calibrate the prediction scores using isotonic Regression. HPS is designed to enforce EO and requires knowledge of the sensitive attribute at test time. We use fixed pre-trained model, architecture and training procedure for all the baselines. We also run the baselines on the edited model obtained from the output of our Fair-IJ IJJ algorithm, Quir Specifically, we fine-tune the Fair-IJ solution using the in-orocessing algorithms. In the case of post-processing algorithms, we directly apply these to the Fair-IJ edited model.
Results. Figure shows the accuracy and fairness Pareto frontier for the Adult and the Coverage datasets averaged over 10 runs. It can be seen that Fair-IJ consistently produces lower disparities across datasets and metrics. Moreover, we observe that baselines operating on O FairMixup- HGR-IJ, FST-IJ, and HPS-IJ often achieve substantially better accuracy/fairness trade-off over their counterparts. In Figure [ demographic parity of held-out validation set for different datasets considered in this paper.
we plot the sorted influence scores of training instances on the average
4.2 CivilComments Dataset
Setup. The CivilComments dataset S consists of human- aanaan-ad attributes on hate comments posted on the internet. The task here is to predict whether particular comment is toxic. Prior work has shown that automatic toxicity classifiers can achieve sub-optimal performance on certain subpopulations 23 12 31) The goal is to apply our approach to mitigate bias in pre-trained toxicity classifiers. In our experiments we consider Muslim as the sensitive attribute. Similar to [22]. we assign an instance to the unprivileged group whenever it is annotated with that attribute and assign the rest to the privileged group.
To show the adaptability of our method large neural networks, we consider three different variants of the pre-trained frozen BERT [1O, where features are augmented with: a) BERT,c: a linear classifier head, b) BERT ninelr classifier head and c) BERTTT-a: with trigger-tokens tokens in the embedding layer. The last variant is an extension of prompt-tuning [23] or prefix-tuning [4 methods, which more powerful ways of fine-tune tnn-ttning langulan models than only updating classifier heads. It is worth noting that the adaptation of trigger-tokens scale fittingly in optimizing weights in Equation g We compare our results to the simple yet effective method of Gap Regularization (GapReg) from 9 where a model optimization is regularized by a fairness measure added to the training loss while scaled by À factor control the regularizer magnitude, defined in Equation of [9]
Results. We present our results in Table ! comparison to the baseline methods ERM and GapReg, Fair-IJ consistently performs better mitigating the group disparities. Additionally, Fair-IJ manages to have a better task performance (balanced accuracy for toxicity classification) trade-off while attempting to achieve a lower disparity. In Table 0 we also present the results on virtual trigger-tokens, which we notice to be performing equally well in lowering the disparity. This is significant observation as it shows how Fair-IJ can be efficiently integrated with large neural network through the scalable influence calculations of relatively few trigger parameters. Further training details, observations and baselines presented in the Appendix.
5 Conclusion
In this work, we proposed Fair-IJ, an infinitesimal jackknife-based approach to mitigate the influence of biased training data points without refitting the model. Our approach is limited to

---

Model
BA
BA
ADP
Model
BA
ΔEО
BA
ADP
BERTU--ERM
57.3
0.314
58.4
0.246
BERT ERM
57.5
0.360
57.5
0.268
SERTue-GapReg
57.9
0.133
57.5
0.185
Fair-IJ
56.0
0.102
55.2
0.042
BERTu--Fair- IJ
58.6
0.125
57.1
0.011
BERTT----ERE
58.2
0.317
58.2
0.254
BERT
59.3
0.326
59.3
0.261
BERTm---Fair--
56.5
0.113
56.4
0.071
BapRe
59.1
0.144
58.2
0.054
BERT7-110-RMM
58.3
0.348
58.3
0.234
BERTr--aair-IJ 59.8
0.126
58.6
0.008
BERTT--O-Fair-IJ
57.2
0.110
56.6
0.089
Table I: Comparison between ERM, Gap Regularization (for 1), and Fair-IJ for CivilCom-
ments on the sensitive attribute MUSLIM when we use pre-trained BERT model. We report the difference in equality of odds (AEO), difference in demographic parity (ADP), along with the task balanced accuracy (BA).
settings where the assumptions listed in Section hold. Also, care must be taken to choose appropriate fairness criterion and its differentiable surrogate for any given application to avoid unwanted consequences. We restricted the most common setup in the fairness literature, but our approach applies to any metric M that is once differentiable in the model parameters and any training loss that is twice differentiable in the parameters. This includes standard approaches to multiclass classification and regression.
analysis binary classification, since this is by far
Future work includes extending approach to black-box models where the gradients are inaccessible and incorporating higher-order Taylor approximations to improve the accuracy of the influence functions. We hope that method further encourages researchers and practitioners in studying and applying bias mitigation to diverse and complex models and datasets.
References
[1] Naman Agarwal, Brian Bullins, and Elad Hazan. Second-order stochastic optimization for machine learning in linear time. Journal of Machine Learning Research, 18(1):414841877 2017.
[2] Hyojin Bahng, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Seong Joon Oh. Learning de- biabi representations with biased representations. In International Conference on Machine Learning, pages 528-539, 2020.
[3] Solon Barocas, Moritz Hardt, and Arvind Narayanan. Fairness and Machine Learning. faimml- book.org, 2019. http://www.faiii book. org
[4] Samyadeep Basu, Philip Pope, and Soheil Feizi. Influence functions in deep learning are fragile. arXiv preprint arXiv:2006. 14651, 2020.
[5] Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Nuanced metrics for measuring unintended bias with real data for text classification. In Companion Proceedings of the World Wide Web Conference, pages 491-500, 2019.
[6] Tamara Broderick, Ryan Giordano, and Rachael Meager. An automatic finite-sample robustness metrie: When can dropping little data make a big difference? arXiv preprint arXiv:2011. 144999 2020.
[7] Flavio Calmon, Dennis Wei, Bhanukiran Vinzamuri, Karthikeyan Natesan Ramamurthy, and Kush R. Varshney. Optimized pre-procesd processing for discrimination prevention. Advances in Neural Information Processing Systems, 30, 2017.
[8] Ran Canetti, Aloni Cohen, Nishanth Dikkala, Govind Ramnarayan, Sarah Scheffler, and Adam Smith. From soft classifiers hard decisions: How fair we be? In Conference on Fairness, Accountability, and Transparency, pages 309-318, 2019.
[9] Ching Yao Chuang and Youssef Mroueh. Fair mixup: Fairness via interpolation. arXiv preprint arXiv:2103.06503, 202..
10

---

[10] Jacob Devlin, Ming- Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint 1ai.:::88100443 2018.
[11] Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt, Retiring adult: New datasets for fair machine learning. Advances in Neural Information Processing Systems, 34, 2021.
[12] Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Measuring and mitigating unintended bias in text classification. In AAAI/ACM Conference on Al, Ethics, and Society, pages 67-73, 2018.
[13] Dheeru Dua and Casey Graff. UCI machine learning repository, 2017.
[14] Bradley Efron. Nonparametric estimates of standard error: The jackknife, the bootstrap and other methods. Biometrika, 68(3):589-599 1981.
[15] Soumya Ghosh, Will Stephenson. Tin D. Nguyen, Sameer Deshpande, and Tamara Broderick. Approximate cross-validation for structured models. Advances in Neural Information Processing Systems, 33:8741-8752, 2020.
[16] Ryan Giordano, Michael I. Jordan, and Tamara Broderick. A higher-order Swiss Army infinites- imal jackknife. arXiv preprint 19t.:1907.111.. 2019.
[17] Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. In Advances Neural Information Processing Systems, pages 3315-3323, 2016.
[18] L. Jaeckel. The infinitesimal jackknife, memorandum. Technical report, MM 72-1215-11 Bell Lab. Murray Hill, NJ, 1972.
[19] Faisal Kamiran, Asim Karim, and Xiangliang Zhang. Decision theory for discrimination-awer aware classification. In IEEE International Conference Data Mining, pages 924-929, 2012.
[20] Toshihiro Kamishima, Shotaro Akaho, and Jun Sakuma. Fairness-aware learning through regularization approach. In IEEE International Conference on Data Mining Workshops, pages 643-650, 2011.
[21] Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International Conference on Machine Learning, pages 1885-1894, 2017.
[22] Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga. Richard Lanas Phillips, Irena Gao, Tony Lee, Etienne David, Ian Stavness, Wei Guo, Berton A. Earnshaw, Imran S. Haque, Sara Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. WILDS: A benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning, 2021.
[23] Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficicnt prompt tuning, arXiv preprint arXiv:2104.0880 08691, 2021.
[24] Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv: 101000/90, 2021.
[25] Pranay K. Lohia, Karthikeyan Natesan Ramamurthy, Manish Bhide, Diptikalyan Saha, Kush R. Varshney, and Ruchir Puri. Bias mitigation post-processing for individual and group fairness. In IEEE International Conference on Acoustics, Speech and Signal Processing, pages 2847-2851, 2019.
[26] Christos Louizos, Kevin Swersky, Yujia Li, Max Welling, and Richard Zemel. The variational fair autoencoder. arXiv preprint orXiv::51/ 1.00830, 2015.
[27] Jérémie Mary, Clément Calauzenes, and Noureddine El Karoui. Fairness-aware learning for continuous attributes and treatments. In International Conference on Muchine Learning, pages 2019.
11

---

[28] Ji Ho Park, Jamin Shin, and Pascale Fung. Reducing gender bias in abusive language detection. arXiv preprint 1808. 07231, 2018.
[29] Pouya Pezeshkpour, Sarthak Jain, Byron C. Wallace, and Sameer Singh. An empirical compari- son of instance attribution methods for NLP. arXiv preprint arXiv:2104.044 /28, 2021.
[30] Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q. Weinberger. On fairness and calibration. Advances in Neural Information Processing Systems, 30, 2017.
[31] Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks. In International Conference on Learning Representations, 2019.
[32] Prasanna Sattigeri, Samuel C. Hoffman, Vijil Chenthamarakshan, and Kush R. Varshney. Fairness GAN. arXiv preprint /00.://000009900, 2018.
[33] Andrea Schioppa, Polina Zablotskaia, David Vilar, and Artem Sokolov. Scaling up influence functions. In AAAI Conference on Artificial Intelligence, volume 36, pages 8179-8186, 2022.
[34] Jonathan Richard Shewchuk. An introduction to the conjugate gradient method without the agonizing pain. Technical report, Carnegie Mellon University, 1994.
[35] Sidak Pal Singh and Dan Alistarh. Woodfisher: Efficient second-order approximation for neural network compression. Advances in Neural Information Processing Systems, 33: 13:18998-18109, 2020.
[36] William T. Stephenson and Tamara Broderick. Approximate cross-validation in high dimensions with guarantees. In International Conference on Artificial Intelligence and Statistics, 2020.
[37] Kush R. Varshney. Trustworthy Machine Learning. Independently Published, Chappaqua, NY, USA, 2022.
[38] Hao Wang, Berk Ustun, and Flavio Calmon. Repairing without retraining: Avoiding disparate impact with counterfactual distributions. In International Conference on Machine Learning, pages 6618 1118-6627,210 2019.
[39] Dennis Wei, Karthikeyan Natesan Ramamurthy, and Flavio Calmon. Optimized score trans- formation for fair classification. In International Conference on Machine Learning, pages 1673-1683, 2020.
[40] Depeng Xu, Shuhan Yuan, Lu Zhang, and Xintao Wu. FairGAN: Fairness-aware generative adversarial networks. In IEEE International Conference on Big Data, pages 570-575, 2018.
[41] Yilun Xu, Hao He, Tianxiao Shen, and Tommi S. Jaakkola. Controlling directions orthogonal to classifier. International Conference on Learning Representations, 2021.
[42] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P. Gummadi. Fairness beyond disparate treatment & disparate impact: Learning classification without dis- parate mistreatment. In International Conference on World Wide Web, pages 1171-1180, 2017.
[43] Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork, Learning fair representa- tions. In International Conference on Machine Learning, pages 325-333, 2013.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See the Conclusion section.
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See the Conclusion section.
12

---

(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results... (a) Did you state the full set of assumptions of all theoretical results? [Yes] See the Section
(b) Did you include complete proofs of all theoretical results? [Yes] See the Sections 3.2,
3.3 and the Appendix.
3. If you experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material or as URL)? [No] The code is proprietary.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See the Appendix.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes] See the Appendix.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See the Appendix.
4. If you are using existing assets (c.g., code, data, models) or curating/relcasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes] See the Appendix.
(c) Did you include any new assets either in the supplemental material or as URL? [No]
(d) Did you discuss whether and how consent obtained from people whose data you'n
using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "vyel-EFJ_hv", "reviewer": "Reviewer_T4TM", "summary": "The draft proposes to simulate removal, i.e. without actually retraining the model, of training points highly influential on a fairness surrogate metric. The work builds upon the concept of influence functions and reuses them beyond loss functions to estimate the effect on other functions of parameters. The paper also presents a method to avoid the O(d^2) complexity of running the Wood-Fisher inverse Hessian approximation procedure (however there seem to exist better alternatives, see below). The results improve on top of the existing baselines by achieving smaller violations of the fairness constraints, while not much reducing the base task accuracy.\n\n", "questions": "The draft lacks a reference to a faster alternative to Wood-Fisher method for efficient influence functions calculation and to increase their accuracy: https://arxiv.org/abs/2112.03052. They compute an approximate Hessian, which is storable in the basis of few dozens of largest eigenvalues of the true Hessian, so that getting $H^{-1}$ is extremely cheap in that basis. With this, there would be no need to rerun iterations of Wood-Fisher for every point $v$, and influence calculation boils down to a few matrix multiplications. \n\nCould you explain in more detail the modifications to BERT? Do BERT_LC and BERT_NC sit on top of the CLS token embeddings?\n\nMinor:\n- inconsistent use of FairIJ vs Fair-IJ in text and in figures, and also in different fonts\n- 206: Does 'IJ' stand for infinitesimal jackknife? Please introduce it explicitly.\n- 227: Please state preferably earlier that you dubbed your approach FairIJ (the algorithm caption is easy to miss)\n", "limitations": "It's unclear how this work would scale to truly large models given the need of recomputing the iterative procedure for every validation example and over all training points.\n", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- a nice repurposing of the influence function approach to other functions than just the loss\n- strong results in reducing disparity without hurting the accuracy too much\n\nWeakness:\n- Potentially slow influence calculations (see the Questions section)\n- While the draft seem to use BERT as a stand-in for large (language) models, in reality it's very small compared to them.  \n- Writing could be improved by carefully introducing notation and abbreviations, and spending more time on model descriptions (e.g. BERT)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "ZjQS59UTsV", "reviewer": "Reviewer_38zQ", "summary": "The paper proposes an infinitesimal jackknife-based approach to mitigate the influence of biased training data points without refitting the model. At a high level, the approach firstly computes the “influence” of training data points on the group fairness metrics of interest (e.g., demographic parity and equality of opportunity considered in this paper) using the coordinate-wise gradient of the first order Taylor approximation to the weighted risk-minimized model parameter, then drop training data points that have a disproportional impact on group (un)fairness. The paper provides theoretical analysis and empirical experiments for the performance of the proposed approach. ", "questions": "Line 66, I may miss something, to compute the empirical risk minimization with dropping certain data points, the denominator in eq 2 should be the number of remaining data points?\n\nLine 152, the authors say that the weights for each data instance are restricted to be binary. Any motivations behind this restriction? \n\nLine 2 in Algorithm 1: what does B stand for?  \n\nLine 3 in Algorithm 1: the algorithm requires computing the fairness influence of each training data. But the definition of the fairness influence is from the true underlying data distribution and is computed in an expected manner. How do the authors compute these quantities given only access to the training data? \n\n\nOther comments: \nUsually, you can write the notations outside of your theorem/proposition environment so that the theorem/proposition statements can be more succinct.\n\n", "limitations": "See above", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The problem -- mitigating biases in machine learning models --  studied in this paper is definitely well-motivated. The proposed approach is (to my best knowledge) novel to me. The approach proposed in the paper that “mitigates pre-trained models without requiring any additional refitting of the model”, I think, should be useful when the pre-trained model is large and incur too much computation cost to train the model again. The authors also provide light theoretical analysis on why their proposed approach might work and also comprehensive experiments to validate the approach. \n\nThe proposed approach seems to only work for the binary classifier (at least the theoretical analysis is limited and only serves for binary classification). Adding results for multi-label classification to this paper would make it stronger. The theoretical result only establishes conditions under which the proposed approach could lead to classifiers with lower group disparities, it is hard to tell, from the experiment perspective, whether the proposed approach has a uniform higher accuracy compared with other approaches. It may greatly improve the paper if there exist theoretical guarantees for the achieved accuracy of the proposed method.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "R6bnzFA_Y7A", "reviewer": "Reviewer_Xt7c", "summary": "The paper proposes a post-processing debiasing algorithm for fairness using an infinitesimal jackknife approach. Their approach examines the influence function of a model's parameters with respect to a fairness metric (demographic parity / equality of odds) to provide an update on the parameters. This comes in the form of a weighted gradient update (newton step), where the weights are chosen to improve fairness. This update requires the computation of an inverse Hessian vector calculation, where the paper proposes the use of an iterative calculation based on the WoodFisher approximation of Hessians. Experiments are also conducted, in which their algorithm is shown to improve fairness at the cost of some accuracy. Experiments also show that the infinitesimal jackknife can be used in conjunction with other in-processing / post-processing methods to potentially improve fairness and accuracy trade-off.\n\nA few additional notes about the paper's approach:\n  - Not only is the training data required, access to the parameters of the pre-trained model is needed\n  - For Proposition 3.1 to allow for a reduction in the fairness penalty, the first order approximation of the criteria needs to be accurate at $ \\mathbf{w}_{\\rm fair} $\n  - Only $k$ coordinate updates (wrt training data) are used in practice and not all given by Proposition 3.1.", "questions": "Questions / Comments / Suggestions\n1. Although (from the narrative) it seems that training data is typically used to calculate the influence function and updates, what happens if one only updates the parameters with a validation dataset? Perhaps this would not be problematic if the pre-trained model / algorithm has strong generalization guarantees?\n2. On Line 199, the trade-off between zeroing high influence coordinates and \"accuracy / approximation quality\". In experiments, the \"best $k$\" (and \"IVHP scaling\") is selected. What is \"best\" here? (validation loss or fairness?) Furthermore, what does the trade-off look like here (between $k$ and wrt the definition of \"best\")?\n3. For the baseline approaches in the experiments, what data is given to these algorithms? Since the IJ update requires the training data + validation data, are both of these (concatenated?) passed to the baseline approaches?\n4. Do the baseline post-processing approaches require access to the pre-trained model parameters (/ training data)? If I recall correctly RST and HPS do not. It would be useful for clarification on whether or not the requirements of the IJ algorithm are shared amongst the baselines.", "limitations": "The limitations / requirements of the paper's proposed approach are documented clearly. However, I think that comments on whether or not the compared baselines share these requirements should be made explicitly. (Reiterated from above)\n\n---\n\nEdit: The limitations were regarding the requirements of baselines was answered in the authors' response.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n  - Experimentally the IJ algorithm has shows cases of strong performance. Particularly in combination with baseline fairness algorithms, which in some settings seem to be a strict improvement from running the baseline fairness algorithms alone.\n  - The update on the parameters is intuitive, as described before Proposition 3.1. \n\nWeaknesses\n  - The access to the training data (and the model parameters) would not be applicable to all scenarios.\n  - Some aspects of the experimental section are not clear, ie, training of baselines.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept