Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: ---------------------After reading the author response and the feedback from the others ------------------------

Thank the authors for their response.  

I still think the assumptions made in the paper are strong.  For example, the assumption in equation 2.3 and 3.3 is hardly true. Even in the revised example 2.6, the missing rates might also depend on other factors. It is hard to identify every related attribute in general. 

I am still not convinced that "machine-learning practitioners are unwilling to sacrifice the performance of their ML model for fairness". First, there might be many other possible reasons that some machine learning applications do not incorporate these fairness considerations. For example, there is no consensus on what fairness constraints we should achieve. In fact, existing works show that considering fairness might has bad effect on protected groups (e.g. Liu et al Delayed impact of fair machine learning). Machine-learning practitioners might just be confused what to do rather than unwilling to do. Second, I have seem many applications that have fairness incorporated. (e.g. Search Google image CEO, we now see many female CEOs on the top). From my perspective, responsible businesses would like to pay extra money to obtain a fair ML solution for the benefit of the society and the reputation of the business as long as they know what to do and how to achieve fairness. 

So, I would like to keep my score. 

------------------------------------Original Review---------------------------------------


Summary

The paper proposed risk parity and conditional risk parity as fairness measures that ensure the risk of machine learning models have the same risk for each subgroup. The paper discusses the conditions that minimizing the risk under a biased data distribution with these fairness constraints will lead to Bayes optimal policy in the target test distribution. Empirical evaluations on simulation data and rea-world recidivism prediction data validate the theoretical analysis. 


Strengths:

1. The paper provides insight on when the optimal fair policy and the optimal utility policy coincide from a biased data point of view, i.e. ensuring some fairness on biased data might learn the optimal policy for the unbiased data. The discussion is very interesting. 

2. The paper is clearly written and easy to follow.

3. The paper characterizes the necessary and sufficient for a policy that minimizes the risk under the proposed fairness constraints on the biased data distribution is optimal in the target test distribution. 

Weakness:

1. Many assumptions are strong and not likely to be true in practice.

    1.1 For example, assumption in equation (2.3) or assumption (3.3) assumes that the risk of any policy on a subgroup is the same in both the biased data distribution and the true target test data distribution. In example 2.5, positive examples of disadvantaged group should be missing at random to satisfy the assumption. Missing at random is hardly true. For example, they might discard positive training examples based on some features, say the qualification of a female job candidate in the job screening example. Example 2.6 in general does not satisfy this assumption. I do not understand why the authors present this example 2.6. 

    1.2 Assumption 3.1 assumes that the unconstrained risk minimizer on unbiased data distribution is algorithmically fair. This is hardly true. 


2. If these assumptions do not hold, then the fairness definition sometimes does not make sense to me. For example, when assumption 3.1 is violated, the optimal classification accuracy for group A is 90%, the optimal classification accuracy fro group B is 85% and there is a policy \pi that achieves the optimal accuracy for both groups. But risk parity constraint says that, we cannot choose this optimal policy, we should select a policy that achieves 85% accuracy for both groups. This does not help group B but only harms group A. 

3. It would be great to compare with methods that directly model the bias in the data. For example, using importance sampling to correct the distribution difference between the training data distribution and the target test data distribution. 



Additional feedback

Contribution 1 "which which "-> "which"

The end of page 4 "that that" -> "that there"

It would be great to provide evidence that "machine-learning practitioners are unwilling to sacrifice the performance of their ML model for fairness" in the abstract and in the conclusion. 



review_title: There is no trade-off: enforcing fairness can improve accuracy

final_score: rating: 6: Marginally above acceptance threshold
review_content: 
In this paper, the authors show that under some assumptions, they can use the fairness constraint to understand some parts of the distribution shift between train and test, and as a result they can improve accuracy at the test time. The main assumptions are (i) the optimal classifier satisfies the fairness constraint and (ii) the risk profiles of all the classifiers in their hypothesis class are the same between biased and unbiased distribution.

The paper was generally well-written; however, Theorem 3.4 and the assumption 3.3 (eq 2.3) were not sufficiently explained. For Assumption 3.3 the only apparent scenario is that if we group data points according to A and V then the weight of each group can vary between the train and test distribution. What other important distribution shift is supported under this assumption, and why are such distributions important/plausible? Also, the relation to the covariate shift requires some clarification.

As I explained, one of the supported distribution shifts is when groups (according to A, V) have different weights in train and test data. In this case, this work is similar to work on group DRO where they optimize the worst group risk instead of the average risk and show they can increase the accuracy in the test data. The authors need to compare their work with this line of work. 

The intuition behind Theorem 3.1 (or at least what I understand) is that due to 3.2 we only need to consider the risk over groups according to A and V (RHS of 4.2). Depending on the weights of these groups, the optimal classifier is different.  The authors then assume in the test data the optimal classifier is fair; thus, this assumption leaks some information about the distribution of the test data (weights of these groups at the test data). By projecting the optimal classifier to the fair subspace, we can recover the optimal classifier under some assumption. Again, the only apparent scenario to me is when the distribution in test data has the same weights for the groups with different A but the same V. What other distributions can the test data have, in particular, for what distributions the fair classifier is also the optimal classifier? 

I think Theorem 3.2 needs a better explanation. What is the intuition of the normal cone, in what conditions we cannot recover the optimal classifier, etc.? For example, although the paper said that showing 4 dimensions in figure 1 is hard, one can have 4 points with different risks on different groups and show which point is chosen by what algorithm and why. 

If the optimal classifier satisfies some fairness constraint, then enforcing that fairness constraint on biased data should always increase the accuracy, right? So what is the surprising result here? Is it characterizing exactly when the optimal classifier is recovered? 

Overall I think this was a good paper, and I enjoyed reading it. However, I believe more intuition and maybe more results will make the paper stronger. 

review_title: Nice result, more intuion/explaination is needed.

final_score: rating: 4: Ok but not good enough - rejection
review_content: Although the paper studies a current and very important issue in algorithmic fairness, I have couple of concerns about the paper in general as follows:

1. The assumptions made in the paper makes the stated conclusion almost trivial making the technical contribution of the work not standing out. For instance, assumption 3.1 is not ever guaranteed to hold. Another major assumption is P* and the fact that the test data is coming from this distribution. This makes the problem trivial as it is almost saying that if I satisfy the fairness objective and make sure that my test data is satisfying the very same objective then the performance of the model will boost. This issue can also be observed in the experimental setting as described in my point #2 detailed below.

2. In experiments, the paper claims that we are forcing/incorporating the EO objective in our classification task but also making sure that the test data is satisfying the equal conditional factor. In this case yes the performance of the fair model on P* will increase, but as shown in the results, the performance of baseline model is better on P~ making the results not convincing to me.

3. I think there need to be more controlled and convincing experimental settings to show the significance of the claim. From the COMPAS results specifically, the baseline is clearly outperforming the fair classifier on P~. Yes sure the fair classifier will outperform on P* but this is only because P* is following and satisfying the objective of the fair classifier. This makes me wonder what the paper is trying to prove in this case.

4. Finally more datasets need to be tested specially to show the case. Having COMPAS only is not convincing in addition to the synthetic data which I noticed similar patter in it. The experimental setting is so brief specially on the real world benchmark fairness dataset, COMPAS.

5. A minor comment: The resume screening example brought up in the paper seems to me more like historical bias; however, authors refer to it as sampling bias.

6. Finally, the title makes a very strong claim which is not well supported in the paper considering both the experimental results and assumptions made in the paper.

Overall, the paper needs more experiments with clear and convincing setup with more benchmark datasets utilized in them. The paper can also improve its writing, clarity and accuracy of some minor points such as the sampling bias issue brought up in point 5. 
review_title: Overly assumptions; no strong justifiable experimental results.

Meta-review: The problem as formalized in this paper is essentially a domain adaptation problem. There is a training distrinution P and a test distribution P*. the learner gets training data generated by P and aims to minimize the loss of its hypothesis w.r.t. P*. How is it relate dto fairness? The authors add the assumption "we assume the unbiased Bayes decision rule is algorithmically fair in some sense and hope that enforcing the correct notion of fairness allows us to recover h∗ from P". Under such an assumption, almost by definition "enforcing fairness may improve accuracy". By a similar logic, if we assume the unbiased byes decision rule is biased against a certain group, then enforcing bias against that group will imporve accuracy ....
Decision: Reject