Original: Fair Infinitesimal Jackknife: Mitigating the Influence of Biased Training Data Points Without Refitting
Revised: Fair Inﬁnitesimal Jackknife: Mitigating the Inﬂuence of Biased Training Data Points Without Reﬁtting

Original: Abstract
Revised: Abstract

Original: In consequential decision-making applications, mitigating unwanted biases in machine learning models that yield systematic disadvantage to members of groups delineated by sensitive attributes such as race and gender is one key intervention to strive for equity.
Revised: In consequential decision-making applications, mitigating unwanted biases in machine learning models that yield systematic disadvantage to members of groups delineated by sensitive attributes such as race and gender is one key intervention to strive for equity.

Original: Focusing on demographic parity and equality of opportunity, in this paper we propose an algorithm that improves the fairness of a pre-trained classifier by simply dropping carefully selected training data points.
Revised: Focusing on demographic parity and equality of opportunity, in this paper we propose an algorithm that improves the fairness of a pre-trained classiﬁer by simply dropping carefully selected training data points.

Original: We select instances based on their influence on the fairness metric of interest, computed using an infinitesimal jackknife-based approach.
Revised: We select instances based on their inﬂuence on the fairness metric of interest, computed using an inﬁnitesimal jackknife-based approach.

Original: The dropping of training points is done in principle, but in practice does not require the model to be refit.
Revised: The dropping of training points is done in principle, but in practice does not require the model to be reﬁt.

Original: Crucially, we find that such an intervention does not substantially reduce the predictive performance of the model but drastically improves the fairness metric.
Revised: Crucially, we ﬁnd that such an intervention does not substantially reduce the predictive performance of the model but drastically improves the fairness metric.

Original: Through careful experiments, we evaluate the effectiveness of the proposed approach on diverse tasks and find that it consistently improves upon existing alternatives.
Revised: Through careful experiments, we evaluate the effectiveness of the proposed approach on diverse tasks and ﬁnd that it consistently improves upon existing alternatives.

Original:  Introduction
Revised:  Introduction

Original: Among the many possible interventions to improve equity in society (most of them involve structural policy change), bias mitigation algorithms constitute one narrow sliver that has emerged in the machine learning literature to address distributive justice in high-stakes automated decision making.
Revised: Among the many possible interventions to improve equity in society (most of them involve structural policy change), bias mitigation algorithms constitute one narrow sliver that has emerged in the machine learning literature to address distributive justice in high-stakes automated decision making.

Original: These algorithms may be categorized into pre-processing, in-processing and post-processing approaches [37].
Revised: These algorithms may be categorized into pre-processing, in-processing, and post-processing approaches [37].

Original: In the case of in-processing algorithms [20], the bias mitigation intervention occurs at the model training stage.
Revised: In the case of in-processing algorithms [20], the bias mitigation intervention occurs at the model training stage.

Original: This is usually achieved by minimizing the empirical risk regularized by a fairness metric surrogate that captures the dependence of the prediction and the sensitive attribute.
Revised: This is usually achieved by minimizing the empirical risk regularized by a fairness metric surrogate that captures the dependence of the prediction and the sensitive attribute.

Original: Pre-processing methods typically learn transformations of the data distribution such that they do not contain information about the sensitive attributes [43, 26].
Revised: Pre-processing methods typically learn transformations of the data distribution such that they do not contain information about the sensitive attributes [43, 26].

Original: Task specific models are then learned from scratch on these debiased representations.
Revised: Task speciﬁc models are then learned from scratch on these debiased representations.

Original: Retraining a model from scratch is intractable in many real-world situations for a variety of reasons including policy, cost, and technical feasibility; post-processing approaches are the only viable option in such cases.
Revised: Retraining a model from scratch is intractable in many real-world situations for a variety of reasons including policy, cost, and technical feasibility; post-processing approaches are the only viable option in such cases.

Original: For example, consider trying to refit large foundation models.
Revised: For example, consider trying to reﬁt large foundation models.

Original: Limiting ourselves to notions of group fairness such as demographic parity and equality of opportunity, existing post-processing bias mitigation algorithms tend to either randomly or deterministically alter the hard or soft predicted label of individual test data points that have been scored by a model [19, 17, 30, 8, 25, 39].
Revised: Limiting ourselves to notions of group fairness such as demographic parity and equality of opportunity, existing post-processing bias mitigation algorithms tend to either randomly or deterministically alter the hard or soft predicted label of individual test data points that have been scored by a model [19, 17, 30, 8, 25, 39].

Original: In this paper, we propose a more “global” bias mitigation algorithm.
Revised: In this paper, we propose a more “global” bias mitigation algorithm.

Original: Our procedure alters the entire model without a focus on individual test points.
Revised: Our procedure alters the entire model without a focus on individual test points.

Original: Similar to post-processing approaches, our method mitigates pre-trained models without requiring any additional refitting of the model.
Revised: Similar to post-processing approaches, our method mitigates pre-trained models without requiring any additional reﬁtting of the model.

Original: Unlike standard post-processing approaches, however, our method does require access to the training data.
Revised: Unlike standard post-processing approaches, however, our method does require access to the training data.

Original: In exchange for this additional requirement, we find that our approach typically substantially outperforms other post-processing techniques and can even augment in-processing approaches for a better fairness/accuracy trade-off.
Revised: In exchange for this additional requirement, we ﬁnd that our approach typically substantially outperforms other post-processing techniques and can even augment in-processing approaches for a better fairness/accuracy trade-off.

Original: Our contributions.
Revised: Our contributions.

Original: Our first contribution is methodological.
Revised: Our ﬁrst contribution is methodological.

Original: We use the notion of influence functions to estimate the “influence” of training instances on various group fairness metrics of interest.
Revised: We use the notion of inﬂuence functions to estimate the “inﬂuence” of training instances on various group fairness metrics of interest.

Original: We then perform post-hoc unfairness mitigation by approximately removing training instances that have a disproportional impact on group (un)fairness.
Revised: We then perform post-hoc unfairness mitigation by approximately removing training instances that have a disproportional impact on group (un)fairness.

Original: We theoretically analyze the proposed approach and establish conditions under which it provably improves group fairness.
Revised: We theoretically analyze the proposed approach and establish conditions under which it provably improves group fairness.

Original: Next, we observe that influence calculations require the inversion of a Hessian matrix, a prohibitively expensive operation for models with a large number of parameters.
Revised: Next, we observe that inﬂuence calculations require the inversion of a Hessian matrix, a prohibitively expensive operation for models with a large number of parameters.

Original: Existing approximations [21, 1, 34] can either be expensive, inaccurate, or unstable [29, 4, 35].
Revised: Existing approximations [21, 1, 34] can either be expensive, inaccurate, or unstable [29, 4, 35].

Original: We develop IHVP-WoodFisher , a WoodFisher [35] based Inverse-Hessian Vector Product (IHVP) scheme for computing the fairness influence score of the training instances that is stable, easy to compute, and does not require constraints, such as restricted Eigenspectrum of the loss curvature, that are hard to satisfy in practice.
Revised: We develop IHVP-WoodFisher , a WoodFisher [35] based Inverse-Hessian Vector Product (IHVP) scheme for computing the fairness inﬂuence score of the training instances that is stable, easy to compute, and does not require constraints, such as restricted Eigenspectrum of the loss curvature, that are hard to satisfy in practice.

Original: Our final contribution is empirical.
Revised: Our ﬁnal contribution is empirical.

Original: First, through careful experiments on tabular data, we show that our approach is effective at reducing group unfairness, is competitive with existing methods, and can even augment the latter to achieve a better fairness/accuracy trade-off.
Revised: First, through careful experiments on tabular data, we show that our approach is effective at reducing group unfairness, is competitive with existing methods, and can even augment the latter to achieve a better fairness/accuracy trade-off.

Original: Then, we demonstrate how our approach can be easily adapted to more complex modalities such as natural language and be used for bias mitigation of large pre-trained language models through prompt-tuning , a use-case that is likely to become increasingly common with the proliferation of large language models.
Revised: Then, we demonstrate how our approach can be easily adapted to more complex modalities such as natural language and be used for bias mitigation of large pre-trained language models through prompt-tuning , a use-case that is likely to become increasingly common with the proliferation of large language models.

Original:  Background and Related Work 2.1 Empirical and Weighted Risk Minimization
Revised:  Background and Related Work 2.1 Empirical and Weighted Risk Minimization

Original: We begin by considering the standard supervised learning setup.
Revised: We begin by considering the standard supervised learning setup.

Original: Given a dataset D = { z n = ( x n , y n ) } Nn =1 of N features ( x n ∈ R p ), response pairs ( y n ∈ Y ), a model h θ ( x ) parameterized by a set of parameters θ ∈ Θ ⊆ R D , and a loss function ℓ : Θ × Y →
Revised: Given a dataset D = { z n = ( x n , y n ) } N n =1 of N features ( x n 2 R p ), response pairs ( y n 2 Y ), a model h ✓ ( x ) parameterized by a set of parameters ✓ 2 ⇥ ✓ R D , and a loss function ` : ⇥ ⇥ Y !

Original: R , we minimize the empirical risk,
Revised: R , we minimize the empirical risk,

Original: [Equation]
Revised: [Equation]

Original: to arrive at a trained predictor h ˆ θ ( x ) .
Revised: to arrive at a trained predictor h ˆ ✓ ( x ) .

Revised: We will denote L ( ✓ ) def = 1 N

Revised: P

Original: We will denote L ( θ ) def = 1 N (cid:80) Nn =1 ℓ ( y n , h θ ( x n )) for notational convenience.
Revised: Nn [Equation] ( y n , h ✓ ( x n )) for notational convenience.

Original: Next, consider a weighted risk minimization problem,
Revised: Next, consider a weighted risk minimization problem,

Original: [Equation]
Revised: [Equation]

Original: that weights the loss at each training instance by a scalar weight w n and w denotes the column vector [ w 1 , w 2 , . .
Revised: that weights the loss at each training instance by a scalar weight w n and w denotes the column vector [ w 1 , w 2 , . .

Original: . , w N ] T ∈ R N .
Revised: . , w N ] T 2 R N .

Original: Setting all the weights to one, 1 def = [ w 1 = 1 , w 2 = 1 , . . .
Revised: Setting all the weights to one, 1 def = [ w 1 = 1 , w 2 = 1 , . . .

Original: , w N = 1] T , and minimizing the right hand side of Equation 2 recovers the standard empirical risk minimization problem.
Revised: , w N = 1] T , and minimizing the right hand side of Equation 2 recovers the standard empirical risk minimization problem.

Original: On the other hand, setting the n th coordinate to zero recovers the solution to an empirical risk minimization problem after dropping the n th training instance.
Revised: On the other hand, setting the n th coordinate to zero recovers the solution to an empirical risk minimization problem after dropping the n th training instance.

Original: As is clear from Equation 2 and emphasized by our notation, θ is a function of the weights w .
Revised: As is clear from Equation 2 and emphasized by our notation, ✓ is a function of the weights w .

Original: Although we typically do not have a closed form expression for this function, we can form a Taylor approximation to it:
Revised: Although we typically do not have a closed form expression for this function, we can form a Taylor approximation to it:

Original: [Equation]
Revised: [Equation]

Original: where ∇ w θ ( w ) ∈ R D × N is the Jacobian matrix.
Revised: where r w ✓ ( w ) 2 R D ⇥ N is the Jacobian matrix.

Original: This first order Taylor approximation is often referred to as the infinitesimal jackknife approximation [18, 14].
Revised: This ﬁrst order Taylor approximation is often referred to as the inﬁnitesimal jackknife approximation [18, 14].

Original: The coordinate-wise gradient dθ ( w ) dw n | w n =1 measures the effect of perturbing the weight of the n th data point on ˆ θ and is popularly referred to as the influence function [21], since it measures the “influence” of the n th training instance on the model’s parameters.
Revised: The coordinate-wise gradient d ✓ ( w ) dw n | w n =1 measures the effect of perturbing the weight of the n th data point on ˆ ✓ and is popularly referred to as the inﬂuence function [21], since it measures the “inﬂuence” of the n th training instance on the model’s parameters.

Original: When we are at a stationary point of L ( θ ) , i.e., when ∇ θ L ( θ ) = 0 , L ( θ ) is twice differentiable in θ , then,
Revised: When we are at a stationary point of L ( ✓ ) , i.e., when r ✓ L ( ✓ ) = 0 , L ( ✓ ) is twice differentiable in ✓ , then,

Original: [Equation]
Revised: [Equation]

Original: where H def = ∇ 2 θ L ( θ ) | θ =ˆ θ , and g n def = ∇ θ ℓ ( y n , h θ ( x n ))
Revised: where H def = r 2 ✓ L ( ✓ ) | ✓ =ˆ ✓ , and g n def = r ✓ ` ( y n , h ✓ ( x n ))

Original: | θ =ˆ θ .
Revised: | ✓ =ˆ ✓ .

Original: Recent work [15] has shown that the above expression approximates the gradient well in the vicinity of a stationary point with the accuracy of the approximation deteriorating smoothly with increasing distance from the stationary point.
Revised: Recent work [15] has shown that the above expression approximates the gradient well in the vicinity of a stationary point with the accuracy of the approximation deteriorating smoothly with increasing distance from the stationary point.

Original: This result justifies the use of influence functions even when stochastic optimization is used for minimizing
Revised: This result justiﬁes the use of inﬂuence functions even when stochastic optimization is used for minimizing

Original: Equation 1.
Revised: Equation 1.

Original: Finally, to measure the influence of a training instance on a differentiable functional, M , of θ ( w ) , we apply chain rule to arrive at,
Revised: Finally, to measure the inﬂuence of a training instance on a differentiable functional, M , of ✓ ( w ) , we apply chain rule to arrive at,

Original: [Equation]
Revised: [Equation]

Original: where our notation makes explicit the dependence of M on w .
Revised: where our notation makes explicit the dependence of M on w .

Original: Recent work has leveraged this machinery to approximate cross-validation [16, 36, 15], to interpret black-box machine learning models [21], and to assess the sensitivity of statistical analyses to training data perturbations [6], among others.
Revised: Recent work has leveraged this machinery to approximate cross-validation [16, 36, 15], to interpret black-box machine learning models [21], and to assess the sensitivity of statistical analyses to training data perturbations [6], among others.

Original: Differently from these, we show how this machinery can be leveraged for reducing disparities of pre-trained models across groups.
Revised: Differently from these, we show how this machinery can be leveraged for reducing disparities of pre-trained models across groups.

Original: 2.2 Fair Classification
Revised: 2.2 Fair Classiﬁcation

Original: We further assume that for each data instance we have access to a sensitive attribute s n ∈ [ k ] , i.e.,
Revised: We further assume that for each data instance we have access to a sensitive attribute s n 2 [ k ] , i.e.,

Original: D = { z n = ( x n , s n , y n ) } Nn =1 , that encodes the protected group membership of the n th data instance and that we are interested in binary classification, Y = { 1 , 0 } .
Revised: D = { z n = ( x n , s n , y n ) } Nn =1 , that encodes the protected group membership of the n th data instance and that we are interested in binary classiﬁcation, Y = { 1 , 0 } .

Original: In fair classification, we want to learn accurate classifiers that minimize disparities in predictions across groups.
Revised: In fair classiﬁcation, we want to learn accurate classiﬁers that minimize disparities in predictions across groups.

Original: To quantify disparities across groups, we primarily focus on two common fairness metrics — demographic (or statistical) parity (DP) [3] and equality of odds (EO) [17].
Revised: To quantify disparities across groups, we primarily focus on two common fairness metrics — demographic (or statistical) parity (DP) [3] and equality of odds (EO) [17].

Original: DP requires the classifier’s predictions to be statistically independent of the sensitive attribute, h θ ( X ) ⊥⊥ S , where X and S are random variables representing the features and the sensitive attribute.
Revised: DP requires the classiﬁer’s predictions to be statistically independent of the sensitive attribute, h ✓ ( X ) ?? S , where X and S are random variables representing the features and the sensitive attribute.

Original: For a binary sensitive attribute, DP implies, P ( h θ ( X ) = 

Revised: For a binary sensitive attribute, DP implies, P ( h ✓ ( X ) =

Original: | S = 1) =
Revised: | S = 1) =

Original: P ( h θ ( X ) = 1 | S = 0) .
Revised: P ( h ✓ ( X ) = 1 | S = 0) .

Original: EO, on the other hand, requires the classifier’s predictions to be statistically independent of the sensitive attribute conditioned on the true outcome, h θ ( X ) ⊥⊥ S | Y .
Revised: EO, on the other hand, requires the classiﬁer’s predictions to be statistically independent of the sensitive attribute conditioned on the true outcome, h ✓ ( X ) ?? S | Y .

Original: For a binary sensitive attribute, EO implies P ( h θ ( X ) = 1 | S = 1 , Y = y )=
Revised: For a binary sensitive attribute, EO implies P ( h ✓ ( X ) = 1 | S = 1 , Y = y )

Original:  P ( h θ ( X ) = [-
-] | S = 0 , Y = y ) for both y = 0 and y = 1 .
Revised: = P ( h ✓ ( X ) = [-
-]1 | S = 0 , Y = y ) for both y = 0 and y = 1 .

Original: Equality of opportunity (EQOPP) [17] is a special case of equality of odds where the predictions are conditionally independent of the sensitive attribute given the true outcome is positive.
Revised: Equality of opportunity (EQOPP) [17] is a special case of equality of odds where the predictions are conditionally independent of the sensitive attribute given the true outcome is positive.

Original: A common strategy for learning fair classifiers is to then require the absolute difference in demographic parity (DP),
Revised: A common strategy for learning fair classiﬁers is to then require the absolute difference in demographic parity (DP),

Original: [Equation]
Revised: [Equation]

Original: or the absolute difference in equality of odds, 
Revised: or the absolute difference in equality of odds,

Original: [Equation]
Revised: [Equation]

Original:  to be close to zero while minimizing the empirical risk (Equation 1).
Revised: ) | , to be close to zero while minimizing the empirical risk (Equation 1).

Original: Smooth 1 surrogates to ∆ DP and ∆ EO that are estimated from an empirical distribution are commonly used in practice [42],
Revised: Smooth 1 surrogates to  DP and  EO that are estimated from an empirical distribution are commonly used in practice [42],

Original: [Equation]
Revised: [Equation]

Original: where M ba ( θ ) denotes the surrogate for the fairness metric b estimated from dataset a .
Revised: where M ba ( ✓ ) denotes the surrogate for the fairness metric b estimated from dataset a .

Original: When θ is itself a function of w , we will use the notation M ba ( θ ( w ) , w ) .
Revised: When ✓ is itself a function of w , we will use the notation M ba ( ✓ ( w ) , w ) .

Original: 2.3 Other Related Work
Revised: 2.3 Other Related Work

Original: Many pre-processing based bias mitigation algorithms, learn low dimensional representations of the data that are independent of the sensitive attribute [43, 26].
Revised: Many pre-processing based bias mitigation algorithms, learn low dimensional representations of the data that are independent of the sensitive attribute [43, 26].

Original: Others aim to learn fairness promoting transformations in the ambient space of the data [7, 32, 40].
Revised: Others aim to learn fairness promoting transformations in the ambient space of the data [7, 32, 40].

Original: Pre-processing methods that transform the data points can often run the risk of losing the semantics of the original data points.
Revised: Pre-processing methods that transform the data points can often run the risk of losing the semantics of the original data points.

Original: Often, they can be expensive, especially for high-dimensional data and large datasets.
Revised: Often, they can be expensive, especially for high-dimensional data and large datasets.

Original: Furthermore, they must be performed before training any task-specific models and thus are not applicable when the goal is to improve a model already trained with an expensive procedure.
Revised: Furthermore, they must be performed before training any task-speciﬁc models and thus are not applicable when the goal is to improve a model already trained with an expensive procedure.

Original: In [38], the authors first obtain a counterfactual feature distribution by identifying the test instances, which when dropped the pre-trained model predictions are fair on the remaining test instances.
Revised: In [38], the authors ﬁrst obtain a counterfactual feature distribution by identifying the test instances, which when dropped the pre-trained model predictions are fair on the remaining test instances.

Original: They then learn a optimal transport based randomized pre-processor that maps the transforms the new test samples from the unprivileged group to fair counterfactual distribution.
Revised: They then learn a optimal transport based randomized pre-processor that maps the transforms the new test samples from the unprivileged group to fair counterfactual distribution.

Original: In contrast, our goal is to compute the influence scores for the training instances, which is more challenging.
Revised: In contrast, our goal is to compute the inﬂuence scores for the training instances, which is more challenging.

Original: Additionally, [38] requires the sensitive attributes be known at test time as the pre-processor is specific to the unprivileged group.
Revised: Additionally, [38] requires the sensitive attributes be known at test time as the pre-processor is speciﬁc to the unprivileged group.

Original: Instead, we aim to directly edit the trained model and eliminate the need of sensitive attribute labels at test time.
Revised: Instead, we aim to directly edit the trained model and eliminate the need of sensitive attribute labels at test time.

Original: In-processing algorithmic fairness methodologies [7, 20] are applicable when we can train models along with fairness constraints.
Revised: In-processing algorithmic fairness methodologies [7, 20] are applicable when we can train models along with fairness constraints.

Original: Mary et al.
Revised: Mary et al.

Original: [27] enforce independence through a relaxation of the
Revised: [27] enforce independence through a relaxation of the

Original: Hirschfeld-Gebelein-Rényi Maximum Correlation Coefficient (HGR) dependency measure.
Revised: Hirschfeld-Gebelein-Rényi Maximum Correlation Coefﬁcient (HGR) dependency measure.

Original: Similarly,
Revised: Similarly,

Original: Rebias [2, 41] uses the Hilbert-Schmidt Independence Criterion (HSIC) to reduce the dependence of the representations on the sensitive attibutes.
Revised: Rebias [2, 41] uses the Hilbert-Schmidt Independence Criterion (HSIC) to reduce the dependence of the representations on the sensitive attibutes.

Original: FairMixup [9] is a data augmentation strategy to improve the generalization properties of in-processing algorithms.
Revised: FairMixup [9] is a data augmentation strategy to improve the generalization properties of in-processing algorithms.

Original: These methods can be sensitive to the regularization strength and can sacrifice too much accuracy.
Revised: These methods can be sensitive to the regularization strength and can sacriﬁce too much accuracy.

Original: In contrast, our approach is applicable when the base model to trained unconstrained on the main task, which can then be updated to remove the influence of the harmful instances and improve fairness.
Revised: In contrast, our approach is applicable when the base model to trained unconstrained on the main task, which can then be updated to remove the inﬂuence of the harmful instances and improve fairness.

Original: Existing post-processing methods [19, 17, 30, 25, 39, 41] learn to transform the predictions of a trained model to satisfy a measure of fairness.
Revised: Existing post-processing methods [19, 17, 30, 25, 39, 41] learn to transform the predictions of a trained model to satisfy a measure of fairness.

Original: These can often be limiting as they do not provide control over the fairness accuracy trade-off, may require that predicted scores to be well-calibrated, or may lead to excessive reduction in performance.
Revised: These can often be limiting as they do not provide control over the fairness accuracy trade-off, may require that predicted scores to be well-calibrated, or may lead to excessive reduction in performance.

Original: In contrast, our method exploits the training data and model gradients efficiently to generate stronger, yet computationally inexpensive post-hoc interventions at minimal loss of predictive performance.
Revised: In contrast, our method exploits the training data and model gradients efﬁciently to generate stronger, yet computationally inexpensive post-hoc interventions at minimal loss of predictive performance.

Original:  Fair Classification through Post-Hoc Interventions
Revised:  Fair Classiﬁcation through Post-Hoc Interventions

Original: We now develop and analyze a post-processing fairness algorithm that given (i) a pre-trained model, (ii) access to the trainingdata and optionally a validation set, (iii) a twice differentiable loss function and a once differentiable surrogate to the fairness metric, and (iv) an invertible Hessian at a local optimum of the loss, improves the fairness characteristics of a pre-trained model without requiring it to be refit.
Revised: We now develop and analyze a post-processing fairness algorithm that given (i) a pre-trained model, (ii) access to the training and a once differentiable surrogate to the fairness metric, and (iv) an invertible Hessian at a local optimum of the loss, improves the fairness characteristics of a pre-trained model without requiring it to be reﬁt.

Original: 3.1 Influence Functions for Group Fairness
Revised: 3.1 Inﬂuence Functions for Group Fairness

Original: Assuming that we use a held-out validation set D val = { x n , s n , y n } N val n =1 to estimate M ∆ DP D val ( θ ) and
Revised: Assuming that we use a held-out validation set D val = { x n , s n , y n } N val n =1 to estimate M  DP D val ( ✓ ) and

Original: M ∆ EO D val ( θ ) , we can leverage the result in Equation 5 to compute the influence of the n th training instance on ∆ DP,
Revised: M  EO D val ( ✓ ) , we can leverage the result in Equation 5 to compute the inﬂuence of the n th training instance on  DP,

Original: [Equation]
Revised: [Equation]

Original: and on ∆ EO,
Revised: and on  EO,

Original: I ∆ EO ,n=
Revised: I  EO ,n

Original: [Equation]
Revised: [Equation]

Original:  where we have used M ∆ DP D val (ˆ θ ) and M ∆ EO D val (ˆ θ ) to denote M ∆ DP D val ( θ ( w ) , w ) (cid:12)(cid:12) w = 1 , θ =ˆ θ and
Revised:  [Equation] where we have used M  DP D val (ˆ ✓ ) and M  EO D val (ˆ ✓ ) to denote M  DP D val ( ✓ ( w ) , w )  w = 1 , ✓ =ˆ ✓ and

Original: M ∆ EO D val ( θ ( w ) , w ) (cid:12)(cid:12) w = 1 , θ =ˆ θ .
Revised: M  EO D val ( ✓ ( w ) , w )  w = 1 , ✓ =ˆ ✓ .

Original: We highlight that computing the influence of training instances on group fairness metrics requires solving a single empirical risk minimization problem to recover ˆ θ .
Revised: We highlight that computing the inﬂuence of training instances on group fairness metrics requires solving a single empirical risk minimization problem to recover ˆ ✓ .

Original: The fairness metrics could also be estimated on the training data if no validation set is available.
Revised: The fairness metrics could also be estimated on the training data if no validation set is available.

Original: However, empirically we find that a validation set improves results.
Revised: However, empirically we ﬁnd that a validation set improves results.

Original: 3.2 Post-Hoc Mitigation
Revised: 3.2 Post-Hoc Mitigation

Original: Revisiting Equation 3, we note that the first order Taylor approximation about 1 is a function of w .
Revised: Revisiting Equation 3, we note that the ﬁrst order Taylor approximation about 1 is a function of w .

Original: This opens up the possibility of post-hoc fairness improvement of a pre-trained ˆ θ by searching for a weight vector w fair such that M ba (ˆ θ fair ) ≈ 0 , where,
Revised: This opens up the possibility of post-hoc fairness improvement of a pre-trained ˆ ✓ by searching for a weight vector w fair such that M ba (ˆ ✓ fair ) ⇡ 0 , where,

Original: [Equation]
Revised: [Equation]

Original: and w fair =
Revised: and w fair =

Original: [ w fair 1 , w fair 2 , . . .
Revised: [ w fair 1 , w fair 2 , . . .

Original: , w fair N ] T ∈ R N .
Revised: , w fair N ] T 2 R N .

Original: We could use gradient-based methods to learn w fair by optimizing a desired M ( θ ( w ) , w ) with respect to w .
Revised: We could use gradient-based methods to learn w fair by optimizing a desired M ( ✓ ( w ) , w ) with respect to w .

Original: However, computing and inverting the Hessian requires O ( ND 2 + D 3 ) operations and is prohibitively expensive for large models.
Revised: However, computing and inverting the Hessian requires O ( ND 2 + D 3 ) operations and is prohibitively expensive for large models.

Original: Instead, iterative procedures involving repeated Hessian-vector products are often used in practice [21].
Revised: Instead, iterative procedures involving repeated Hessian-vector products are often used in practice [21].

Original: A gradientbased procedure would need to either perform this iterative procedure after every gradient step or pre-compute (cid:80) n H − 1 g n , rendering the procedure computationally intractable for most cases of interest.
Revised: A gradientbased procedure would need to either perform this iterative procedure after every gradient step or pre-compute P n H  1 g n , rendering the procedure computationally intractable for most cases of interest.

Original: Moreover, solely optimizing M ( θ ( w ) , w ) will likely result in fair but inaccurate classifiers, and the optimized weights will typically not be interpretable.
Revised: Moreover, solely optimizing M ( ✓ ( w ) , w ) will likely result in fair but inaccurate classiﬁers, and the optimized weights will typically not be interpretable.

Original: We circumvent these issues by constraining the elements of w to be binary.
Revised: We circumvent these issues by constraining the elements of w to be binary.

Original: In Proposition 3.1, we show that we can construct w fair by simply zeroing out coordinates of w fair that correspond to training instances with a positive influence on the fairness metric of interest.
Revised: In Proposition 3.1, we show that we can construct w fair by simply zeroing out coordinates of w fair that correspond to training instances with a positive inﬂuence on the fairness metric of interest.

Original: This construction is inherently interpretable.
Revised: This construction is inherently interpretable.

Original: Setting an element to zero implies training without the corresponding training instance.
Revised: Setting an element to zero implies training without the corresponding training instance.

Original: Zeroing out instances with positive influence equates to refitting the model after dropping training instances that increase disparity across groups.
Revised: Zeroing out instances with positive inﬂuence equates to reﬁtting the model after dropping training instances that increase disparity across groups.

Original: We now establish conditions under which w fair as constructed above leads to classifiers with lower group disparities.
Revised: We now establish conditions under which w fair as constructed above leads to classiﬁers with lower group disparities.

Original: Let 1 ∈ R N denote an N -dimensional vector of all ones, b denote a fairness metric,
Revised: Let 1 2 R N denote an N -dimensional vector of all ones, b denote a fairness metric,

Original: M b D val ( θ ( w ) , w ) denote a linearized approximation to M b D val ( θ ( w)) , w ) about 1 , and 1 [ α > β ] denote an indicator function that takes the value one if α > β is true and zero otherwise.
Revised: M [Equation] [Figure] ✓ ( w

Original: Proposition 3.1.
Revised: Proposition 3.1.

Original: Let w fair ∈ { 0 , 1 } N be a N dimensional binary vector such that its n th coordinate is w fair n = [Equation] [ I b,n > 0] , then,
Revised: Let w fair 2 { 0 , 1 } N be a N dimensional binary vector such that its n th coordinate is w fair n = 1  [ I b,n > 0] , then,

Original: [Equation]
Revised: [Equation]

Original: and M b D val ( θ ( w fair ) , w fair ) − M b D val ( θ ( 1 ) , 1 ) ≤ 0 .
Revised: and M b D val ( ✓ ( w fair ) , w fair )  M b D val ( ✓ ( 1 ) , 1 )  0 .

Original: Proof.
Revised: Proof.

Original: Denote M b D val (ˆ θ ) :=
Revised: Denote M b D val (ˆ ✓ ) :=

Original: M D val ( θ ( 1 ) , 1 ) .
Revised: M D val ( ✓ ( 1 ) , 1 ) .

Original: From a first order Taylor approximation about 1 , we have,
Revised: From a ﬁrst order Taylor approximation about 1 , we have,

Original: [Equation]
Revised: [Equation]

Original: Rearranging terms,
Revised: Rearranging terms,

Original: [Equation]
Revised: [Equation]

Original: Finally, the result follows from observing that w n ∈ { 0 , 1 } and noting that the first term can be either zero (when w n = 1 ) or negative (when w n = 0 and1 [ I b,n > 0] ) and the second term can be either zero (when w n = 1 ) or positive (when w n = 0 and1 [ I b,n ≤ 0] ).
Revised: Finally, the result follows from observing that w n 2 { 0 , 1 } and noting that the ﬁrst term can be either zero (when w n = 1 ) or negative (when w n = 0 and [ I b,n > 0] ) and the second term can be either zero (when w n = 1 ) or positive (when w n = 0 and [ I b,n  0] ).

Original: w fair drives the second term to zero and sets the first term to the smallest value attainable by a binary w .
Revised: w fair drives the second term to zero and sets the ﬁrst term to the smallest value attainable by a binary w .

Original: It follows that M b D val ( θ ( w fair ) , w fair ) ≈≤ M b D val (ˆ θ ( 1 ) , 1 ) , with the inequality holding when the linearization is accurate.
Revised: It follows that M b D [Equation] [Equation] val (ˆ ✓ ( 1 ) , 1 ) , with the inequality holding when the linearization is accurate.

Original: Finally, defining D − = { z n | z n ∈ D
Revised: Finally, deﬁning D  = { z n | z n 2 D

Original: and I M,n > 0 } , we arrive at the post-hoc mitigated classifier by plugging in w fair from Proposition 3.1 in Equation 9,
Revised: and I M,n > 0 } , we arrive at the post-hoc mitigated classiﬁer by plugging in w fair from Proposition 3.1 in Equation 9,

Original: [Equation]
Revised: [Equation]

Original: In Appendix A we consider an alternate w fair that is guaranteed to decrease both the loss ℓ and the fairness metric on the validation set D val .
Revised: In Appendix A we consider an alternate w fair that is guaranteed to decrease both the loss ` and the fairness metric on the validation set D val .

Original: In early experiments, we did not see consistent benefits from using this alternate version and do not consider it further in this paper.
Revised: In early experiments, we did not see consistent beneﬁts from using this alternate version and do not consider it further in this paper.

Original: 3.3 Practical Considerations
Revised: 3.3 Practical Considerations

Original: Hessian computation and inversion.
Revised: Hessian computation and inversion.

Original: The influence function computation involves computing and inverting the Hessian of the loss function on the training data.
Revised: The inﬂuence function computation involves computing and inverting the Hessian of the loss function on the training data.

Original: This requires O ( ND 2 + D 3 ) operations.
Revised: This requires O ( ND 2 + D 3 ) operations.

Original: Both computing and storing the Hessian becomes prohibitively expensive for large models.
Revised: Both computing and storing the Hessian becomes prohibitively expensive for large models.

Original: While diagonal approximations to the Hessian are possible, they tend to be inaccurate.
Revised: While diagonal approximations to the Hessian are possible, they tend to be inaccurate.

Original: Instead, iterative methods based on the (truncated) Neumann expansion have been proposed in the past [21].
Revised: Instead, iterative methods based on the (truncated) Neumann expansion have been proposed in the past [21].

Original: However, more recent work has found the Neumann approximation to be inaccurate, cf.
Revised: However, more recent work has found the Neumann approximation to be inaccurate, cf.

Original: [36, Appendix C] and prone to numerical issues when the eigenvalues of the Hessian fall outside the [0 , 1] interval.
Revised: [36, Appendix C] and prone to numerical issues when the eigenvalues of the Hessian fall outside the [0 , 1] interval.

Original: Motivated by these shortcomings, here we develop an alternative iterative procedure based on the recently proposed WoodFisher approximation [35].
Revised: Motivated by these shortcomings, here we develop an alternative iterative procedure based on the recently proposed WoodFisher approximation [35].

Original: The WoodFisher approximation provides us with the following recurrence relation for estimating the inverse of the Hessian:
Revised: The WoodFisher approximation provides us with the following recurrence relation for estimating the inverse of the Hessian:

Original: [Equation]
Revised: [Equation]

Original: , (13) with ˆ H − 1= λ −{+ 1 
 =  +} 1 I D , and λ a small positive scalar.
Revised: (13) with ˆ H {+ 1 
 =  +} 1 I D , and  a small positive scalar.

Original: For computing influence functions we only need to store the product of the inverse Hessian with a vector v , i.e., H − 1 v , which should only require O ( D ) storage.
Revised: For computing inﬂuence functions we only need to store the product of the inverse Hessian with a vector v , i.e., H  1 v , which should only require O ( D ) storage.

Original: However, if we first compute the inverse Hessian and then compute the Hessian-vector product (HVP), we would need O ( D 2 ) storage.
Revised: However, if we ﬁrst compute the inverse Hessian and then compute the Hessian-vector product (HVP), we would need O ( D 2 ) storage.

Original: To sidestep this issue, we develop the following coupled recurrences that only use O ( D ) storage.
Revised: To sidestep this issue, we develop the following coupled recurrences that only use O ( D ) storage.

Original: We call these coupled recurrences IHVP-WoodFisher ,
Revised: We call these coupled recurrences IHVP-WoodFisher ,

Original: [Equation]
Revised: [Equation]

Original: where, we use ℓ ( z n +1 ) as shorthand for ℓ ( y n +1 , h θ ( x n +1 )) , o 1 = ∇ θ ℓ ( y 1 , h θ ( x 1 )) , and k 1 = v .
Revised: where, we use ` ( z n +1 ) as shorthand for ` ( y n +1 , h ✓ ( x n +1 )) , o 1 = r ✓ ` ( y 1 , h ✓ ( x 1 )) , and k 1 = v .

Original: Proposition 3.2.
Revised: Proposition 3.2.

Original: Let o 1 = ∇ θ ℓ ( z 1 ) , k 1 = v , and N denote the number of training instances.
Revised: Let o 1 = r ✓ ` ( z 1 ) , k 1 = v , and N denote the number of training instances.

Original: The
Revised: The

Original: Hessian-vector product H − 1 v is approximated by iterating through the IHVP-WoodFisher recurrence in Equation 14 and computing k N .
Revised: Hessian-vector product H  1 v is approximated by iterating through the IHVP-WoodFisher recurrence in Equation 14 and computing k N .

Original: We prove Proposition 3.2 in Appendix B.
Revised: We prove Proposition 3.2 in Appendix B.

Original: In practice, we observe that even using B ≪ N iterations produces useful approximations.
Revised: In practice, we observe that even using B ⌧ N iterations produces useful approximations.

Original: In Appendix C, we compare the approximation accuracy of the the IHVP-WoodFisher and the iterative Neumann approach on cases where it is tractable to exactly compute the IHVP.
Revised: In Appendix C, we compare the approximation accuracy of the the IHVP-WoodFisher and the iterative Neumann approach on cases where it is tractable to exactly compute the IHVP.

Original: Algorithm 2 (see Appendix) summarizes our vanilla approach.
Revised: Algorithm 2 (see Appendix) summarizes our vanilla approach.

Original: Computational speedups: Although Algorithm 2 suggests running the IHVP-WoodFisher iterations for each training instance for clarity of exposition, in practice, we use the following trick to run the IHVP-WoodFisher iterations only once for the entire training dataset.
Revised: Computational speedups: Although Algorithm 2 suggests running the IHVP-WoodFisher iterations for each training instance for clarity of exposition, in practice, we use the following trick to run the IHVP-WoodFisher iterations only once for the entire training dataset.

Original: First, for any p ∗ p symmetric matrix A and p -dimensional vectors x and y , x T A , y = y T Ax .
Revised: First, for any p ⇤ p symmetric matrix A and p -dimensional vectors x and y , x T A , y = y T Ax .

Original: From Equation 5, the influence calculation involves computing ∇ θ M ( ˆ θ , 1 ) T H − 1 g n for all n in the training dataset.
Revised: From Equation 5, the inﬂuence calculation involves computing r ✓ M ( ˆ ✓ , 1 ) T H  1 g n for all n in the training dataset.

Original: Since is symmetric, we can equivalently compute g Tn H − 1 ∇ θ M ( ˆ θ , 1 ) .
Revised: Since is symmetric, we can equivalently compute g Tn H  1 r ✓ M ( ˆ ✓ , 1 ) .

Original: We can then run the IHVPWoodFisher iterations to approximate H − 1 ∇ θ M ( ˆ θ , 1 ) .
Revised: We can then run the IHVPWoodFisher iterations to approximate H  1 r ✓ M ( ˆ ✓ , 1 ) .

Original: Crucially, we need to do this only once.
Revised: Crucially, we need to do this only once.

Original: With the approximation in hand, computing the per data influence requires a single dot product per data instance between g n and the IHVP-WoodFisher approximated H − 1 ∇ θ M ( ˆ θ , 1 ) .
Revised: With the approximation in hand, computing the per data inﬂuence requires a single dot product per data instance between g n and the IHVP-WoodFisher approximated H  1 r ✓ M ( ˆ ✓ , 1 ) .

Original: In contrast to other approaches to scaling up influence functions [33] our approach only requires the storage of a single p -dimensional vector.
Revised: In contrast to other approaches to scaling up inﬂuence functions [33] our approach only requires the storage of a single p -dimensional vector.

Original: We call this more efficient version Fair-IJ and is summarized in
Revised: We call this more efﬁcient version Fair-IJ and is summarized in

Original: Algorithm 1.
Revised: Algorithm 1.

Original: Most influential instances.
Revised: Most inﬂuential instances.

Original: Our development and analysis depends on first order linear approximations of non-linear functions about 1 .
Revised: Our development and analysis depends on ﬁrst order linear approximations of non-linear functions about 1 .

Original: We expect the quality of these approximations to deteriorate further away from 1 , i.e., with increasing number of instances dropped.
Revised: We expect the quality of these approximations to deteriorate further away from 1 , i.e., with increasing number of instances dropped.

Original: See Theorem 1 in [6] for additional discussion on the quality of approximation.
Revised: See Theorem 1 in [6] for additional discussion on the quality of approximation.

Original: We find that instead of dropping all instances with positive influence, dropping the k most influential instances yields better bias mitigation.
Revised: We ﬁnd that instead of dropping all instances with positive inﬂuence, dropping the k most inﬂuential instances yields better bias mitigation.

Original: We select the hyperparameter k that results in the lowest (best) fairness score on the validation set.
Revised: We select the hyperparameter k that results in the lowest (best) fairness score on the validation set.

Original: Additionally, [35] observed that the WoodFisher Hessian estimate ˆ H differs from the true Hessian by a scaling factor, i.e ˆ H ∝ H .
Revised: Additionally, [35] observed that the WoodFisher Hessian estimate ˆH differs from the true Hessian by a scaling factor, i.e ˆH / H .

Original: We select, from a pre-specified set, the scaling factor that minimizes the fairness score on the validation set.
Revised: We select, from a pre-speciﬁed set, the scaling factor that minimizes the fairness score on the validation set.

Original: We then scale the IHVP-WoodFisher estimates using the selected scaling factor.
Revised: We then scale the IHVP-WoodFisher estimates using the selected scaling factor.

Original: See Appendix D.
Revised: See Appendix D.

Original:  Experiments
Revised:  Experiments

Original: We first study our method on tabular datasets including the well-known Adult dataset [13] and the recently released ACSPublicCoverage [11] dataset.
Revised: We ﬁrst study our method on tabular datasets including the well-known Adult dataset [13] and the recently released ACSPublicCoverage [11] dataset.

Original: ACSPublicCoverage is one among a suite of datasets aimed to be larger alternatives to previously available fairness datasets.
Revised: ACSPublicCoverage is one among a suite of datasets aimed to be larger alternatives to previously available fairness datasets.

Original: We then investigate our method on the text modality and larger pre-trained models using the CivilComments dataset [5].
Revised: We then investigate our method on the text modality and larger pre-trained models using the CivilComments dataset [5].

Original: 4.1 Tabular Datasets
Revised: 4.1 Tabular Datasets

Original: Setup.
Revised: Setup.

Original: The task in the Adult dataset is to predict if a person has an income above a threshold.
Revised: The task in the Adult dataset is to predict if a person has an income above a threshold.

Original: We use gender as the sensitive attribute.
Revised: We use gender as the sensitive attribute.

Original: This dataset comes with a fixed test set.
Revised: This dataset comes with a ﬁxed test set.

Original: A random 33% of the training data is used as the validation set for each trial of the experiments.
Revised: A random 33% of the training data is used as the validation set for each trial of the experiments.

Original: We follow the pre-processing steps from [27].
Revised: We follow the pre-processing steps from [27].

Original: The task in the ACSPublicCoverage dataset is to predict if a person has public health insurance coverage.
Revised: The task in the ACSPublicCoverage dataset is to predict if a person has public health insurance coverage.

Original: For our experiments, we only consider instances from the year 2014 , from the state of California, and belonging to the white or black race.
Revised: For our experiments, we only consider instances from the year 2014 , from the state of California, and belonging to the white or black race.

Original: We consider race as the sensitive attribute.
Revised: We consider race as the sensitive attribute.

Original: In the rest of the paper, we refer to this subset simply as Coverage dataset.
Revised: In the rest of the paper, we refer to this subset simply as Coverage dataset.

Original: We randomly split the dataset into train/validation/test with partition 50% / 20% / 30% , respectively, for each trial.
Revised: We randomly split the dataset into train/validation/test with partition 50% / 20% / 30% , respectively, for each trial.

Original: Additionally, for both datasets we standardize the features before training our methods and the baselines.
Revised: Additionally, for both datasets we standardize the features before training our methods and the baselines.

Original: We first train a 1 -hidden layer fully connected neural network with SeLu activation function andhidden units.
Revised: We ﬁrst train a 1 -hidden layer fully connected neural network with SeLu activation function andhidden units.

Original: This initial model is trained using standard ERM loss with batch size set to 256 .
Revised: This initial model is trained using standard ERM loss with batch size set to 256 .

Original: We use the Adam optimizer with learning rate set to 10 − 4 .
Revised: We use the Adam optimizer with learning rate set to 10  4 .

Original: We train the ERM and baselines for 100 epochs and pick the checkpoint with best accuracy on the validation set.
Revised: We train the ERM and baselines for 100 epochs and pick the checkpoint with best accuracy on the validation set.

Original: We then employ Algorithm 1 to arrive at the Fair-IJ solution.
Revised: We then employ Algorithm 1 to arrive at the Fair-IJ solution.

Original: We select k and the IHVP scaling term based on the same validation set used to compute the influence scores.
Revised: We select k and the IHVP scaling term based on the same validation set used to compute the inﬂuence scores.

Original: Compared algorithms.
Revised: Compared algorithms.

Original: We compare our method to several in-processing and post-processing bias mitigation algorithms that are applicable to a wide range of model classes including deep neural networks.
Revised: We compare our method to several in-processing and post-processing bias mitigation algorithms that are applicable to a wide range of model classes including deep neural networks.

Original: We omit the comparisons with pre-processing methods as our goal is to improve a given pre-trained model.
Revised: We omit the comparisons with pre-processing methods as our goal is to improve a given pre-trained model.

Original: Among in-processing algorithms, we compare with: FairMixup [9] and HGR [27].
Revised: Among in-processing algorithms, we compare with: FairMixup [9] and HGR [27].

Original: FairMixup achieves fairness through a mixup based regularization employed during training.
Revised: FairMixup achieves fairness through a mixup based regularization employed during training.

Original: The HGR approach proposes a surrogate to HGR dependence measure and promotes fairness during training by enforcing conditional independence implied by the fairness metrics.
Revised: The HGR approach proposes a surrogate to HGR dependence measure and promotes fairness during training by enforcing conditional independence implied by the fairness metrics.

Original: On both of these methods, the fairness accuracy trade-off is achieved through the strength of the regularizer.
Revised: On both of these methods, the fairness accuracy trade-off is achieved through the strength of the regularizer.

Original: Among post-processing methods, we compare with FST [39] and HPS [17].
Revised: Among post-processing methods, we compare with FST [39] and HPS [17].

Original: FST optimally transforms the pre-trained model’s prediction scores to satisfy a specified fairness constraint and supports DP and EO metrics.
Revised: FST optimally transforms the pre-trained model’s prediction scores to satisfy a speciﬁed fairness constraint and supports DP and EO metrics.

Original: To improve the performance of FST we re-calibrate the prediction scores using isotonic Regression.
Revised: To improve the performance of FST we re-calibrate the prediction scores using isotonic Regression.

Original: HPS is designed to enforce EO and requires knowledge of the sensitive attribute at test time.
Revised: HPS is designed to enforce EO and requires knowledge of the sensitive attribute at test time.

Original: We use a fixed pre-trained model, architecture and training procedure for all the baselines.
Revised: We use a ﬁxed pre-trained model, architecture and training procedure for all the baselines.

Original: We also run the baselines on the edited model obtained from the output of our Fair-IJ algorithm, ˆ θ fair .
Revised: We also run the baselines on the edited model obtained from the output of our Fair-IJ algorithm, ˆ ✓ fair .

Original: Specifically, we fine-tune the Fair-IJ solution using the in-processing algorithms.
Revised: Speciﬁcally, we ﬁne-tune the Fair-IJ solution using the in-processing algorithms.

Original: In the case of post-processing algorithms, we directly apply these to the Fair-IJ edited model.
Revised: In the case of post-processing algorithms, we directly apply these to the Fair-IJ edited model.

Original: Results.
Revised: Results.

Original: Figure 2 shows the accuracy and fairness Pareto frontier for the Adult and the Coverage datasets averaged over 10 runs.
Revised: Figure 2 shows the accuracy and fairness Pareto frontier for the Adult and the Coverage datasets averaged over 10 runs.

Original: It can be seen that Fair-IJ consistently produces lower disparities across datasets and metrics.
Revised: It can be seen that Fair-IJ consistently produces lower disparities across datasets and metrics.

Original: Moreover, we observe that baselines operating on ˆ θ fair , FairMixup-IJ,
Revised: Moreover, we observe that baselines operating on ˆ ✓ fair , FairMixup-IJ,

Original: HGR-IJ, FST-IJ, and HPS-IJ often achieve substantially better accuracy/fairness trade-off over their counterparts.
Revised: HGR-IJ, FST-IJ, and HPS-IJ often achieve substantially better accuracy/fairness trade-off over their counterparts.

Original: In Figure 1, we plot the sorted influence scores of training instances on the average demographic parity of a held-out validation set for different datasets considered in this paper.
Revised: In Figure 1, we plot the sorted inﬂuence scores of training instances on the average demographic parity of a held-out validation set for different datasets considered in this paper.

Original: 4.2 CivilComments Dataset
Revised: 4.2 CivilComments Dataset

Original: Setup.
Revised: Setup.

Original: The CivilComments dataset [5] consists of human-annotated attributes on hate comments posted on the internet.
Revised: The CivilComments dataset [5] consists of human-annotated attributes on hate comments posted on the internet.

Original: The task here is to predict whether a particular comment is toxic.
Revised: The task here is to predict whether a particular comment is toxic.

Original: Prior work has shown that automatic toxicity classifiers can achieve sub-optimal performance on certain subpopulations [28, 12, 31].
Revised: Prior work has shown that automatic toxicity classiﬁers can achieve sub-optimal performance on certain subpopulations [28, 12, 31].

Original: The goal is to apply our approach to mitigate bias in pre-trained toxicity classifiers.
Revised: The goal is to apply our approach to mitigate bias in pre-trained toxicity classiﬁers.

Original: In our experiments we consider Muslim as the sensitive attribute.
Revised: In our experiments we consider Muslim as the sensitive attribute.

Original: Similar to [22], we assign an instance to the unprivileged group whenever it is annotated with that attribute and assign the rest to the privileged group.
Revised: Similar to [22], we assign an instance to the unprivileged group whenever it is annotated with that attribute and assign the rest to the privileged group.

Original: To show the adaptability of our method on large neural networks, we consider three different variants of the pre-trained frozen BERT [10], where features are augmented with: a) BERT LC : a linear classifier head, b) BERT NC : a non-linear classifier head and c) BERT TT=n : with n trigger-tokens in the embedding layer.
Revised: To show the adaptability of our method on large neural networks, we consider three different variants of the pre-trained frozen BERT [10], where features are augmented with: a) BERT LC : a linear classiﬁer head, b) BERT NC : a non-linear classiﬁer head and c) BERT TT=n : with n trigger-tokens in the embedding layer.

Original: The last variant is an extension of prompt-tuning [23] or prefix-tuning [24] methods, which are more powerful ways of fine-tuning large-language models than only updating classifier heads.
Revised: The last variant is an extension of prompt-tuning [23] or preﬁx-tuning [24] methods, which are more powerful ways of ﬁne-tuning large-language models than only updating classiﬁer heads.

Original: It is worth noting that the adaptation of trigger-tokens scale fittingly in optimizing weights in Equation 9.
Revised: It is worth noting that the adaptation of trigger-tokens scale ﬁttingly in optimizing weights in Equation 9.

Original: We compare our results to the simple yet effective method of Gap Regularization (GapReg) from [9] where a model optimization is regularized by a fairness measure added to the training loss while scaled by λ factor to control the regularizer magnitude, as defined in Equation 1 of [9].
Revised: We compare our results to the simple yet effective method of Gap Regularization (GapReg) from [9] where a model optimization is regularized by a fairness measure added to the training loss while scaled by  factor to control the regularizer magnitude, as deﬁned in Equation 1 of [9].

Original: Results.
Revised: Results.

Original: We present our results in Table 1.
Revised: We present our results in Table 1.

Original: In comparison to the baseline methods ERM and GapReg,
Revised: In comparison to the baseline methods ERM and GapReg,

Original: Fair-IJ consistently performs better in mitigating the group disparities.
Revised: Fair-IJ consistently performs better in mitigating the group disparities.

Original: Additionally, Fair-IJ manages to have a better task performance (balanced accuracy for toxicity classification) trade-off while attempting to achieve a lower disparity.
Revised: Additionally, Fair-IJ manages to have a better task performance (balanced accuracy for toxicity classiﬁcation) trade-off while attempting to achieve a lower disparity.

Original: In Table 1, we also present the results on virtual trigger-tokens, which we notice to be performing equally well in lowering the disparity.
Revised: In Table 1, we also present the results on virtual trigger-tokens, which we notice to be performing equally well in lowering the disparity.

Original: This is a significant observation as it shows how Fair-IJ can be efficiently integrated with large neural network through the scalable influence calculations of relatively few trigger parameters.
Revised: This is a signiﬁcant observation as it shows how Fair-IJ can be efﬁciently integrated with large neural network through the scalable inﬂuence calculations of relatively few trigger parameters.

Original: Further training details, observations and baselines are presented in the Appendix.
Revised: Further training details, observations and baselines are presented in the Appendix.

Original:  Conclusion
Revised:  Conclusion

Original: In this work, we proposed Fair-IJ , an infinitesimal jackknife-based approach to mitigate the influence of biased training data points without refitting the model.
Revised: In this work, we proposed Fair-IJ , an inﬁnitesimal jackknife-based approach to mitigate the inﬂuence of biased training data points without reﬁtting the model.

Original: Our approach is limited to
Revised: Our approach is limited to

Original: [Table]
Revised: [Table]

Original: settings where the assumptions listed in Section 3 hold.
Revised: settings where the assumptions listed in Section 3 hold.

Original: Also, care must be taken to choose an appropriate fairness criterion and its differentiable surrogate for any given application to avoid unwanted consequences.
Revised: Also, care must be taken to choose an appropriate fairness criterion and its differentiable surrogate for any given application to avoid unwanted consequences.

Original: We restricted our analysis to binary classification, since this is by far the most common setup in the fairness literature, but our approach applies to any metric M that is once differentiable in the model parameters and any training loss that is twice differentiable in the parameters.
Revised: We restricted our analysis to binary classiﬁcation, since this is by far the most common setup in the fairness literature, but our approach applies to any metric M that is once differentiable in the model parameters and any training loss that is twice differentiable in the parameters.

Original: This includes standard approaches to multiclass classification and regression.
Revised: This includes standard approaches to multiclass classiﬁcation and regression.

Original: Future work includes extending our approach to black-box models where the gradients are inaccessible and incorporating higher-order Taylor approximations to improve the accuracy of the influence functions.
Revised: Future work includes extending our approach to black-box models where the gradients are inaccessible and incorporating higher-order Taylor approximations to improve the accuracy of the inﬂuence functions.

Original: We hope that our method further encourages researchers and practitioners in studying and applying bias mitigation to diverse and complex models and datasets.
Revised: We hope that our method further encourages researchers and practitioners in studying and applying bias mitigation to diverse and complex models and datasets.

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] See the Conclusion section.
Revised: [Yes] See the Conclusion section.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] See the Conclusion section.
Revised: [Yes] See the Conclusion section.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] See the Section 3.
Revised: [Yes] See the Section 3.

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes] See the Sections 3.2, 3.3 and the Appendix. 3.
Revised: [Yes] See the Sections 3.2, 3.3 and the Appendix. 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [No] The code is proprietary.
Revised: [No] The code is proprietary.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] See the Appendix.
Revised: [Yes] See the Appendix.

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] See the Appendix.
Revised: [Yes] See the Appendix.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] See the Appendix. 4.
Revised: [Yes] See the Appendix. 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [Yes] See the Appendix.
Revised: [Yes] See the Appendix.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No] (d) Did you discuss whether
Revised: [No] (d) Did you discuss whether

Original: and how consent was obtained from people whose data you’re using/curating?
Revised: and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?

Original: [N/A] 5.
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: ] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?[N/A] Appendix A Loss aware w fair
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: While this construction of w fair reduces the fairness metric it pays no heed to the original loss, ℓ , and may lead to classifiers that are less accurate than ˆ θ .

Original: To account for ℓ we additionally compute I ℓ,n the influence of each training instance on the validation loss, L D val ( θ ) =

Original: 1 N val

Original: [Equation]

Original: N

Original: [Equation]

Original: n

Original: [Equation]

Original: ) and set,

Original: [Equation]

Original: [Equation]

Original: i.e., we zero out those coordinates of w fair that correspond to training instances with a positive influence on both the fairness metric and the loss ℓ .

Original: Finally, defining D − = { z n | z n ∈ D

Original: and I M,n > 0 & I ℓ,n > 0 } , we arrive at the post-hoc mitigated classifier by plugging in the zeroed-out w fair in

Original: Equation 9,

Original: [Equation]

Original: B IHVP-WoodFisher

Original: In this section, we show that the coupled recurrences, that we refer to as IHVP-WoodFisher , computes the Inverse-Hessian Vector Product (IHVP).

Original: We begin by restating Proposition 3.2,

Original: Proposition B.1.

Original: Let o 1 = ∇ θ ℓ ( z 1 ) , k 1 = v , and N denote the number of training instances.

Original: The

Original: Hessian-vector product H − 1 v is approximated by iterating through the IHVP-WoodFisher recurrence in Equation 14 and computing k N .

Original: Proof.

Original: The WoodFisher approximation provides us with the following recurrence relation for estimating the inverse of the Hessian,

Original: [Equation]

Original: 17) with, H − 1= λ − 1 I D , and λ is small positive scalar.

Original: Multiplying, both sides by ∇ θ ℓ ( z n +1 ) , we get,

Original: [Equation]

Original: By substituting H − 1 n ∇ θ ℓ ( z n +1 ) with o n and assuming ∇ θ ℓ ( z n +1 ) and ∇ θ ℓ ( z n +2 ) are close, we construct the following recurrence relation

Original: [Equation]

Original: Now, multiplying both sides of Equation 17 by v , gives us the recurrence relation for the IHVP,

Original: [Equation]

Original: [-By substituting H − 1 n +1 v with k n +
-]
Revised: [-By substituting H − 1 n +1 v with k n +
-]

Original: and H − 1 n ∇ θ ℓ ( z n +1 ) with o n , we get

Original: [Equation]

Original: [Equation]

Original: Thus, under our assumptions, when H − 1 n converges to H − 1 , k n converges to the IHVP H − 1 v .

Original: C IHVP-WoodFisher Approximation Accuracy

Original: In this section, we discuss the approximation accuracy of IHVP-WoodFisher and compare it with

Original: IHVP-Neumann .

Original: IHVP-WoodFisher relies on the assumption that the empirical Fisher matrix is a good approximation to the Hessian of the loss.

Original: The Hessian of the loss is known to converge to the true Fisher matrix, when: a) the loss of the model used during training can be expressed as negative log likelihood; and b) the model likelihood has converged to the true data likelihood [35].

Original: The empirical Fisher matrix does not have convergence guarantees as the true Fisher but, it is computationally cheap and works well in practice as an approximation to the Hessian matrix.

Original: This is also seen in our experimental results.

Original: To study the approximation accuracy of IHVP-WoodFisher and compare with the Exact approach of computing IHVP, we consider a setting where the number of parameters is small; specifically, we generate a linearly separable variant of the two moon dataset consisting of 10000 points, where each point has 2 input features and can belong to one of the two classes.

Original: We create at 80 − 20 train-test split and train models with depth 1 , 2 , and 3 to observe the effect of depth.

Original: Hidden layers have a fixed width of 5 units.

Original: We use Adam optimizer with learning rate 0 .

Original: After training, we pick a random point from the test set and compute the influence score of the training instances using

Original: IHVP-WoodFisher and IHVP-Neumann approximations as well as exactly computing the IHVPs.

Original: For both approximations we use 1000 iterations and average over 10 runs.

Original: The IHVP-Neumann approximation has an additional hyper-parameter - scale .

Original: This is to ensure that the Eigenvalues of the Hessian are between [0 , 1] .

Original: For IHVP-Neumann ’s convergence, scale has to be greater than the largest Eigenvalue of the Hessian.

Original: [-In these experiments, we set this hyperparameter to 25 . 
-]
Revised: [-In these experiments, we set this hyperparameter to 25 . 
-]

Original: which is larger than the largest Eigenvalue we observed we

Original: In Figure 3 we compare the influence scores and report the Median Absolute Deviation (MAD) and Rscores.

Original: For each depth value, when plotting and computing the metrics we rescale the influence scores from both approximations to match the mean of the IHVP-Exact .

Original: It can be observed that both IHVPNeumann and IHVP-WoodFisher approaches match well with the influence scores obtained from Exact computation.

Original: The main benefit of IHVP-WoodFisher is that it does not require higher-order gradients.

Original: Additionally, unlike IHVP-Neumann , this method does not require expensive hyperparameter search to rescale the Eigenvalues of Hessian to ensure IHVP computation convergence.

Original: D Additional Dataset, Training Details and Results

Original: In Table 2, we provide additional information information about the three datasets – Adult 2 , Coverage 3 , and CivilComments 4 .

Original: We trained our models on NVIDIA A100 Tensor Core GPUs.

Original: In the case of the tabular datasets, 10 runs with a particular fairness metric took less than 2 hours.

Original: This includes training the base model using ERM followed by the application of our Fair-IJ algorithm.

Original: Within the algorithm, we search for the best k among 40 values spread uniformly in the range in the ranges 0 − 2000 .

Original: Similarly, for the IHVP scaling we select the best value among (0 . 01 , 0 . 1 , 1 . 0 , 2 . 0 , 3 . 0 , 5 . 0 , 10 . 0) .

Original: This search only requires inference over the validation and hence is relatively inexpensive.

Original: The post-processing baselines ( FST and HPS ), assume access to a pretrained model.

Original: Similar to our approach they use the validation data to mitigate bias in the pre-trained models.

Original: For the in-processing baselines ( HGR and FairMixup ), following standard practice, we train the models on the training set and use the validation set to select the hyper-parameter that determines the strength of the fairness regularizer employed by these methods.

Original: In Figure 4, we reproduce the

Original: Figure 2 with error bars for both the accuracy and fairness metrics.

Original: We now provide additional details regarding the experiments on CivilComments datasets.

Original: Tablepresents additional

Original: [Table]
Revised: [N/A]

Original: Maximum Correlation Coefficient (HGR) dependency measure [27], and Fair-IJ .

Original: Reported results are for the sensitive attribute MUSLIM .

Original: They include Equality of odds ( ∆ EO), demographic parity ( ∆ DP), and balanced accuracy (BA) and are the means and standard deviations over 5 training runs, each with a different random seed (i.e. 5 different seeds for each configuration).

Original: Each model was built with 100 epochs of SGD over the training data for a total of 24h of computation time (using a

Original: NVIDIA A100 GPU).

Original: All reported results were computed on the test dataset for models with the best validation loss over the 100 epochs of training (models being validated at the end of each epoch).

Original: For loss-regularized models using GapReg and HGR, values of λ = 1 . 0 and λ = 0 . 2 were used to add the regularizer term to the training loss; both sets of results are given in Table 3.

Original: Inference on the test dataset is quite fast (3 minutes) on NVIDIA A100 GPUs.

Original: Results for both BERT LC (Linear Classifier) and BERT NC (Non-linear Classifier) are provided for ERM, GapReg, HGR, and Fair-IJ .

Original: Both our BERT LC and BERT NC architectures are defined based on different classification layer(s) on top of a pre-trained BERT model.

Original: In all our experiments, we use BERT base model.

Original: While BERT LC uses just a single dense layer on top of the pooled vector from BERT representations, BERT NC has multiple dense layers on top of the pooled output.

Original: For the latter, we use two dense layers with hidden sizes of 768 and 128 intertwined with ReLU non-linearities.

Original: As described in 4.2, we also provide quantitative results on a variant of BERT LC , referred as BERT TT=N , which uses virtual tokens, where N refers to number of trigger tokens.

Original: In this setup, we introduce a new parameterized embedding, called trigger embeddings, which are learned during the training.

Original: Similar to methods introduced in [23] and [24], we add trigger tokens to each sequence during the training.

Original: In our quantitative analysis we use

Original: [Table]

Original: variants with 4,

Original: [Table]

Original: and 10 trigger tokens which are referred as BERT TT=4 , BERT TT=8 , and BERT TT= respectively in Table 3 and Table 1.

Original: We fine-tune these models with a maximum epochs of 100 and choose the best model based on the validation loss over the validation set.

Original: Similar to the case of tabular datasets, we apply Fair-IJ algorithm with same range of k and IHVP scaling.