Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: The paper introduces a post-processing method to enforce demographic parity. The authors provide theoretical guarantees for their approach as well as empirically evaluate its performance on various datasets and against several baselines.
main_review: Post-processing fairness-enforcing approaches are convenient as they can interact with black-box models as well as enjoy faster running times compared to in-processing methods. The paper is a nice addition to the growing literature on post-processing. It is known that post-processing can be suboptimal so it is important to understand the limits of such approaches. 

However, I do have some concerns about the paper:

--The bound on Theorem 2 depends on both $\gamma$ and $1/\gamma$. How should this parameter be selected in practice?  In particular, how should the bound in Theorem 2 be interpreted as constant $\gamma$ is used in the experiments? Why a fresh set of data points are needed in the statement of Theorem 2? Moreover, for the theory results, it would be great to add a discussion of the applicability and limitations. When should your approach be used? Are there any properties in the dataset or the black-box that make your approach more suitable compared to prior work?

--Some of the details in the experiments are not clear and some justifications are needed. I have not checked the appendix to see whether these details are provided. For example, why additional bias is introduced in DCCC? Why the performance is not reported for the reduction-based approach on CelebA? The details of averaging and normalization for the results in the scaling section are not described clearly. More particularly, I am curious about the following points:

        --How does the running time compare to Hardt et. al.? Both approaches seem to achieve the same level of fairness violation while the approach of Hardt et al is much simpler.

        --Why is there no comparison with Wei et. al, 2020 as they are also proposing a post-processing approach?

        --Imposing post-processing seems to improve accuracy for DCCC. This is confusing. As mentioned above, why additional bias is introduced in DCCC, and how does the accuracy is measured? i.e. for the original data or the biased data?

        --The results in Table 2 for DCCC are a bit confusing. Why does the error get larger as more fairness violation is allowed? Also, there seems to be no real trade-off in this dataset. So what is the takeaway?

        --I cannot follow the results in the scaling section. The details of the averaging and normalization are missing. There is no mention of the bias in this section and all the reported performances are based on normalized error. What is the takeaway here? Why there is no comparison with baselines?
 
--The choice of using CelebA dataset to predict attractiveness as a question regarding fairness is not appropriate. Please see my comments below.

Minor comments and questions:

--I would prefer to call this fairness notion demographic parity. Statistical parity refers to a more broad category of fairness metrics where parity on some statistical notion is enforced for different subgroups.

--Can the result in Example 1 be extended to approximate statistical parity instead of exact? I am curious whether randomization is truly necessary there.

--What is $n$ in Theorem 1? Is it supposed to be $N$?

This is a truly borderline paper. It can get accepted or rejected with an equal chance though I am leaning a bit towards rejection at the moment.
limitations_and_societal_impact: There is not much discussion on the limitation of the work. On the societal impact question 1(c), the authors answer: "We propose a debiasing algorithm for machine learning. We do not foresee any potential negative societal impacts of our work." I think at the least there should be references to the literature that highlights where demographic parity is not a reasonable notion of fairness. Moreover, I find the task of predicting the attractiveness of people from their images a bit inappropriate at the least. While the authors do not endorse using AI for such tasks, it would have been much better if other datasets are used (e.g., there are datasets for facial recognition algorithms where the prediction bias happens because of the skin color).
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: In this paper, the authors propose a post-processing method for debiasing trained models. The authors defined statistical parity as a 'bias'. The paper proposes a constrained optimization problem that takes the input data, sensitive attribute, partitioning and a trained model to yield a probabilistic decision rule. The authors proved theoretical properties and empirically validate its advantages on standard benchmark datasets across both classical algorithms.
main_review: Fairness is an essential topic in the machine learning community. The proposed method suggests a post-processing algorithm for reducing bias from trained models according to statistical parity. Although 'bias' is a societal concept that cannot be adequately measured by statistical parity, the reviewer thinks the proposed method suggests a reasonably acceptable solution.

For a minor comment, the reviewer thinks some explanations are relatively complicated (for instance, Example 1) due to the page limit. Some revision would be necessary for readability.

**Conclusion**
The reviewer thinks the paper is well-written and generally acceptable, which will be an excellent contribution to the NeurIPS conference. 
limitations_and_societal_impact: The authors did not address the limitations and potential societal impact of their work in the paper.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes a near-optimal post-processing algorithm (via a regularized optimization objective) that is theoretically guaranteed to debias trained machine learning models for the problem of statistical parity, which is both scalable and performs favorably relative to several existing post-processing and in-processing techniques on 2 datasets from the UCI Machine Learning repo, CelebA, and COCO. The focus of the paper is on binary classification, and the key is to output randomized predictions around the classifier’s threshold.
main_review: Originality:
- The method builds upon the existing work in fair ML, where the proposed approach allows for a randomized thresholding rule that mitigates the downsides of learning a deterministic thresholding rule. This randomization is key for the paper’s nice theoretical properties. The authors adequately cited related work in Section 2 and clearly outlined how their approach differs from existing work. They also provided an overview of baselines they compared to in Section 5, which I appreciated. 

Quality:
- The paper is technically sound and claims well-supported.

Clarity:
- The paper was clearly written and easy to follow, barring a few minor points of confusion (listed at the bottom of this text box). Example 1 was a nice illustration of why randomization is necessary, and the notation used in the main text was clear.
- I did have some trouble parsing the experimental results. For example, where exactly are the results for COCO? I saw some in Figures 3-5 but from the main text I expected the results to be included with all the other tables (e.g. results for CelebA, DCCC, etc.). Also how should I be interpreting Figure 2? It’s a bit cramped and the caption doesn’t really tell me what the takeaway is.

Significance:
- The results are significant, as debiasing machine learning models is an important research area, and most existing works do not provide theoretical guarantees. Since this work explores fairness in the statistical parity setting, and mentions that the algorithm generalizes to non-binary attributes, I could see how others would build upon these ideas.

------
UPDATE: I have read the other reviews as well as the authors' response. Thanks to the authors for clearing up some points of confusion, and I'll keep my score as is.
limitations_and_societal_impact: The authors stated that their results are only applicable to the setting of statistical parity, and made it clear that their method is not intended to address all issues in fair machine learning. They did not discuss potential negative societal impacts of their work, as they are developing an algorithm to mitigate bias in trained machine learning models.
review_title: null

Meta-review: The balance of the reviews is in favor of accepting the paper. I want to follow this consensus. While some of the reviewers described significant issues with the paper, I do not view these issues as disqualifying. I expect the camera-ready version of the paper will be able to address most of them as per the author response.
Decision: Accept (Poster)