Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: The paper proposes a post-processing approach to achieve different notions of fairness. To do this, the paper first approximates the probability distribution of the output model. Group-specific confusion matrices can be derived by thresholding this distribution. The entries of such a matrix can be used to compute accuracy and various fairness metrics. The authors experimentally evaluate their approach on several datasets.
main_review: There are several concerns I have about this paper.

--First of all, while it is known that to achieve fairness optimal thresholding of the risk scores/output probabilities of the model should be group-specific, there are many instances in practice where the use of the sensitive attribute is forbidden. So it is not clear to me that the model can be used in practical applications.

--Connecting to the previous point, it is not clear where the benefit of the current approach compared to previous approaches is coming from. Is it because of a more clear technical approach or is it due to allowing group-specific thresholds (which would obviously perform better than approaches that utilize a single threshold). So in that respect, in the response, I would like to see the same curves being generated with only one threshold for all groups.

--The approach relies on utilizing models that output probabilistic decisions as opposed to deterministic ones. This certainly does not fall into the category of utilizing black-box models. So I found the use of the term "black-box" misleading. 

--The approach also relies on good estimates of the distribution over probabilistic decisions of the model. According to the authors, the approach won't work when this distribution is mainly concentrated on the extremes. Hence, the approach is not suitable for truly black-box models. Moreover, the approach relies on a good estimation of this distribution. The authors propose several parametric families of distributions and choose the parameter which maximizes the likelihood. Is there a justification for why these families of distributions are sufficient? How does the error in estimating the probability distribution propagate to the result?

--By looking at the experimental results, it is not clear that the approach provides a better Pareto frontier than FACT. The more interesting regime in practice is where the fairness violation is at most 1%. In this regime, it seems like FACT consistently outperforms the current approach. Though for fairness violations that are much smaller (like 0.1% and lower), it seems the proposed approach is doing better. But how often in practice do we care about such small values of fairness violations?

--The optimization problem is not solved exactly. Are there any approximation guarantees?

--Many details of the experiments are missing. For example, what lambda values are used to derive the Pareto frontiers? Why there is only one accuracy/balanced accuracy are reported for different notions of fairness in Figure 3? Is this averaged over different notions of fairness?

Minor comments:

--The use of the log scale for the x-axis is a bit misleading in the figures. Is it because the approach only outperforms FACT in really small fairness violation values?

--The figures are very crowded. I have to zoom in to read the legend. 

--It seems a bit suspicious that adding multiple notions of fairness, can improve the total fairness violation compared to using only one fairness metric in Figure 3. Is there any justification for this observation?

-------------------------------
Post-rebuttal: please see my official comment


limitations_and_societal_impact: I found the discussion on the limitation and societal impact to be satisfactory.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper introduces a method for finding fair and optimal thresholds for binary classification. It considers the mean squared error as the objective function. Instead of solving a constrained optimization problem, they add a  fairness loss to the objective function. 

Then using Taylor approximation,  the optimization problem is reduced to an optimization function with a quadratic utility. The proposed method has low complexity ($\mathcal{O}(n+T)$). Moreover, it improves the fairness accuracy tradeoff as compared to existing works. 
main_review: While Algorithm 1 is pretty clear, the experiment results are not clear at all. 

For instance, in Figure 2, LAFTR is an adversarial method for fair learning. LAFTR works for different fairness notions (e.g., equal opportunity, statistical party (SP), etc.). However, in figure 2, is not clear which fairness notions have been considered for LAFTR.  Also, there are different points for a single algorithm in each plot. What does that mean? 
The x-axis in figure 2 is EOd. What does this mean? EOd imposes two constraints. how do you measure it in one dimension?

The same issue exists in Figure 3.  Eq.Odds [7] can be used to satisfy EOd or EOp.  LAFTR can be used for EOD EOp and SP. Which constraint are you using when training LAFTR or Eq.Odds. If LAFTR is trained for satisfying EOp, and GSTAR is designed to satisfy EOp, then we cannot compare the fairness accuracy tradeoff. 

You should clarify in detail that what each element in figure 2 and 3 mean. Without clarification, I am not able to compare your method with other methods.  It is very likely that I change my score after the authors' response. 
limitations_and_societal_impact: Yes, They have.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The authors propose a new post-processing group-aware threshold search procedure for fair classification. The procedure is efficient, amenable to different types of underlying classification models, and is empirically shown to be working with multiple fairness criteria. The authors also derive  Pareto frontier of their model for the fairness-accuracy trade-off. Lastly, the authors conclude with experiments on four datasets showing the efficacy of their method.
main_review: Strengths:
1. Relevant problem to the NeurIPS community.
2. Rigorous treatment to an important problem.
3. Experimental evaluation is good that includes multiple datasets along with relevant baselines.

Weaknesses:
1. Novelty: The idea of using multiple thresholds for different demographic groups seems to be an obvious thing to do. Prior work may not have done analysis of the type that is provided in this paper, but my guess is it could have been used in the previous papers (maybe as baselines). I would let other reviewers who are more aware of the fairness literature to decide on the novelty of the paper. 
2. The authors have not provided any convergence analysis of the proposed procedure. Is it guaranteed to converge to a good solution (good thresholds)?
2. Minor point: The entire theoretical analysis has been pushed to appendix including the results on Pareto frontier. If the paper is accepted, I would encourage authors to bring some results in the main paper. 

Detailed Comments:
Overall, I liked the paper and do not have any major concern other than novelty of the idea. The following points may further help to clarify/improve the paper.

1. In related works, especially, lines 74-79, can you please specify how your work is different from prior work?

2. Why is there a square in equation 3? Is it because it helps in derivation? Can we also use weighted measures for L_per and L_fair as done in the following papers:
Hiranandani, Gaurush, et al. "Fair Performance Metric Elicitation." NeurIPS (2020). -- Definition 1, Equation (6)
Hiranandani, Gaurush, et al. "Quadratic Metric Elicitation with Application to Fairness." arXiv preprint arXiv:2011.01516 (2020). -- Definition 4, equation (13)

3. Equation (8) does not contain any subscripts; whereas, the terms below line 180 do. Can you please clarify?

4. As mentioned before, the authors have not provided any convergence analysis of the proposed procedure. Is it guaranteed to converge to a good solution (good thresholds)?
Also, what happens to their procedure in case of multiple sensitive groups? Does a coordinate wise optimization for thresholds work in this case?

5. The graphs are not readable. Authors can increase the legend size, if possible, remove some trivial baselines to make the graphs more readable.

6. There are good insights in lines 272-275.

7. As the authors mention in their limitations that their method may not work if the underlying models gives over-confident probability distributions. One suggestion to the authors is that they can use calibration or label-smoothening in that case. 

Minor:
The authors can remove bold  text while explaining baselines. 
limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper proposes a threshold modification method to post-process the trained classifiers. The threshold learning is formulated as an optimization task and an efficient technique is proposed to learn the thresholds for groups. The proposed adaptive threshold learning is able to work with multiple fairness constraints.
main_review: originality: fair
quality: good
clarity: good
significance: good

Strength:

1. The proposed method learns adaptive thresholds for various groups to achieve fairness. The learning technique is efficient. by using Taylor expansion. The efficient algorithm works for multiple fairness constraints.
2. There theoretical results in Appendix are very interesting. 

Weakness:

1. The proposed method requires a pre-defined parametric probability distribution. The requirement is kind of difficult for many real-world applications.  It is unclear whether the proposed method is sensitive to the selection of distribution.
2. The alternative optimization method is fast but there is a lack of guarantee for the optimum. It is unclear how close the approximation to the optimal solution.

Overall, the contribution of the proposed idea (threshold adjusting) is interesting. The attempt to develop an efficient learning algorithm is also interesting.  My major concern is the efficient algorithm incurs approximation errors due to the Taylor expansion. The learning algorithm requires the density function. It is unclear how the proposed alternative optimization is sensitive to the density estimation.

## Post response

I truly appreciate the response. It is informative and helpful for better understanding the manuscript.

I had two major concerns on the density assumption and convergence. The convergence analysis is more like a practical solution to an optimization task. It shows how to approximate a solution but raises some new concerns such as how sensitive the solution is w.r.t. the relaxation and Taylor expansion. The empirical results show the sensitivity of accuracy and fairness w.r.t. the density estimation. I understand the theoretical challenge behind but I think it deserves further investigation as the optimization later requires a good estimation. In summary, my opinions and rating are unchanged. 
limitations_and_societal_impact: None
review_title: null

Meta-review: The paper proposes a post-processing technique to tune group-specific thresholds on a classifier to satisfy fairness constraints. The consensus from a majority of the reviewers is that the paper needs more work and is not ready for publication in its current form.

One of the main concerns is the assumption that the "estimated probability density function" follows a parametric form, and that it's unclear how practical this assumption is, and what happens when it fails or only holds approximately. A second concern is that the proposed threshold-tuning procedure does not come with a convergence guarantee. In response, the authors provide two separate results: (i) an asymptotic convergence guarantee for their optimization algorithm under some initial conditions, (ii) an adaptation of a result from Hardt et al. (2016) to show that the fairness constraint is approximately satisfied under some conditions on the estimated probability density. 

Both these are however substantial additions to the paper, and would require another round of review. Hence we strongly encourage the authors to use the feedback provided to improve their manuscript and submit it to a future venue. 

P.S. I appreciate the authors clarification on the similarities between the requirements in their paper and that in Hardt et al. (and the potential advantage that their approach offers in providing a tighter bound), but would have to give it some more thought. Bringing out some of these differences/similarities with Hardt et al. would make the paper more compelling.
Decision: Reject