Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper proposes an infinitesimal jackknife-based approach to mitigate the influence of biased training data points without refitting the model. At a high level, the approach firstly computes the “influence” of training data points on the group fairness metrics of interest (e.g., demographic parity and equality of opportunity considered in this paper) using the coordinate-wise gradient of the first order Taylor approximation to the weighted risk-minimized model parameter, then drop training data points that have a disproportional impact on group (un)fairness. The paper provides theoretical analysis and empirical experiments for the performance of the proposed approach. 
strengths_and_weaknesses: The problem -- mitigating biases in machine learning models --  studied in this paper is definitely well-motivated. The proposed approach is (to my best knowledge) novel to me. The approach proposed in the paper that “mitigates pre-trained models without requiring any additional refitting of the model”, I think, should be useful when the pre-trained model is large and incur too much computation cost to train the model again. The authors also provide light theoretical analysis on why their proposed approach might work and also comprehensive experiments to validate the approach. 

The proposed approach seems to only work for the binary classifier (at least the theoretical analysis is limited and only serves for binary classification). Adding results for multi-label classification to this paper would make it stronger. The theoretical result only establishes conditions under which the proposed approach could lead to classifiers with lower group disparities, it is hard to tell, from the experiment perspective, whether the proposed approach has a uniform higher accuracy compared with other approaches. It may greatly improve the paper if there exist theoretical guarantees for the achieved accuracy of the proposed method.


questions: Line 66, I may miss something, to compute the empirical risk minimization with dropping certain data points, the denominator in eq 2 should be the number of remaining data points?

Line 152, the authors say that the weights for each data instance are restricted to be binary. Any motivations behind this restriction? 

Line 2 in Algorithm 1: what does B stand for?  

Line 3 in Algorithm 1: the algorithm requires computing the fairness influence of each training data. But the definition of the fairness influence is from the true underlying data distribution and is computed in an expected manner. How do the authors compute these quantities given only access to the training data? 


Other comments: 
Usually, you can write the notations outside of your theorem/proposition environment so that the theorem/proposition statements can be more succinct.


limitations: See above
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper proposes a post-processing debiasing algorithm for fairness using an infinitesimal jackknife approach. Their approach examines the influence function of a model's parameters with respect to a fairness metric (demographic parity / equality of odds) to provide an update on the parameters. This comes in the form of a weighted gradient update (newton step), where the weights are chosen to improve fairness. This update requires the computation of an inverse Hessian vector calculation, where the paper proposes the use of an iterative calculation based on the WoodFisher approximation of Hessians. Experiments are also conducted, in which their algorithm is shown to improve fairness at the cost of some accuracy. Experiments also show that the infinitesimal jackknife can be used in conjunction with other in-processing / post-processing methods to potentially improve fairness and accuracy trade-off.

A few additional notes about the paper's approach:
  - Not only is the training data required, access to the parameters of the pre-trained model is needed
  - For Proposition 3.1 to allow for a reduction in the fairness penalty, the first order approximation of the criteria needs to be accurate at $ \mathbf{w}_{\rm fair} $
  - Only $k$ coordinate updates (wrt training data) are used in practice and not all given by Proposition 3.1.
strengths_and_weaknesses: Strengths
  - Experimentally the IJ algorithm has shows cases of strong performance. Particularly in combination with baseline fairness algorithms, which in some settings seem to be a strict improvement from running the baseline fairness algorithms alone.
  - The update on the parameters is intuitive, as described before Proposition 3.1. 

Weaknesses
  - The access to the training data (and the model parameters) would not be applicable to all scenarios.
  - Some aspects of the experimental section are not clear, ie, training of baselines.
questions: Questions / Comments / Suggestions
1. Although (from the narrative) it seems that training data is typically used to calculate the influence function and updates, what happens if one only updates the parameters with a validation dataset? Perhaps this would not be problematic if the pre-trained model / algorithm has strong generalization guarantees?
2. On Line 199, the trade-off between zeroing high influence coordinates and "accuracy / approximation quality". In experiments, the "best $k$" (and "IVHP scaling") is selected. What is "best" here? (validation loss or fairness?) Furthermore, what does the trade-off look like here (between $k$ and wrt the definition of "best")?
3. For the baseline approaches in the experiments, what data is given to these algorithms? Since the IJ update requires the training data + validation data, are both of these (concatenated?) passed to the baseline approaches?
4. Do the baseline post-processing approaches require access to the pre-trained model parameters (/ training data)? If I recall correctly RST and HPS do not. It would be useful for clarification on whether or not the requirements of the IJ algorithm are shared amongst the baselines.
limitations: The limitations / requirements of the paper's proposed approach are documented clearly. However, I think that comments on whether or not the compared baselines share these requirements should be made explicitly. (Reiterated from above)

---

Edit: The limitations were regarding the requirements of baselines was answered in the authors' response.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The draft proposes to simulate removal, i.e. without actually retraining the model, of training points highly influential on a fairness surrogate metric. The work builds upon the concept of influence functions and reuses them beyond loss functions to estimate the effect on other functions of parameters. The paper also presents a method to avoid the O(d^2) complexity of running the Wood-Fisher inverse Hessian approximation procedure (however there seem to exist better alternatives, see below). The results improve on top of the existing baselines by achieving smaller violations of the fairness constraints, while not much reducing the base task accuracy.


strengths_and_weaknesses: Strengths:
- a nice repurposing of the influence function approach to other functions than just the loss
- strong results in reducing disparity without hurting the accuracy too much

Weakness:
- Potentially slow influence calculations (see the Questions section)
- While the draft seem to use BERT as a stand-in for large (language) models, in reality it's very small compared to them.  
- Writing could be improved by carefully introducing notation and abbreviations, and spending more time on model descriptions (e.g. BERT)
questions: The draft lacks a reference to a faster alternative to Wood-Fisher method for efficient influence functions calculation and to increase their accuracy: https://arxiv.org/abs/2112.03052. They compute an approximate Hessian, which is storable in the basis of few dozens of largest eigenvalues of the true Hessian, so that getting $H^{-1}$ is extremely cheap in that basis. With this, there would be no need to rerun iterations of Wood-Fisher for every point $v$, and influence calculation boils down to a few matrix multiplications. 

Could you explain in more detail the modifications to BERT? Do BERT_LC and BERT_NC sit on top of the CLS token embeddings?

Minor:
- inconsistent use of FairIJ vs Fair-IJ in text and in figures, and also in different fonts
- 206: Does 'IJ' stand for infinitesimal jackknife? Please introduce it explicitly.
- 227: Please state preferably earlier that you dubbed your approach FairIJ (the algorithm caption is easy to miss)

limitations: It's unclear how this work would scale to truly large models given the need of recomputing the iterative procedure for every validation example and over all training points.
review_title: null

Meta-review: metareview: This paper presents a tool for mitigating the influence of biased training data without retraining a model. The identified shortcomings of the paper include the restriction on binary classification setting. The meta reviewer recognizes this caveat but feels the focus aligns with the existing fairness definitions and the literature broadly. But the authors are encouraged to make it explicit about the paper’s limitations regarding the above restriction on binary classes and should clarify what and how the analysis and approaches can generalize to the multi-class case. 

Prior to rebuttal, there was raised concern about the calculation of Hessian inverse and the generalizability to larger models. The authors addressed both during the rebuttal. Please add the new results to the final version. 

confidence: Less certain
Decision: Accept