Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This work establishes a formal framework for exploring impossibility questions related to fair data representations. Impossibility results are proven within this framework with an emphasis on challenging prior work in this area.
main_review: The paper takes an interesting and important stance, critiquing a popular approach to fair classification (fair representations) and exploring its limitations. However, the claims seem a bit exaggerated with respect to prior work in relation to the perspective studied in the paper. The results here are narrowly tailored to a specific setting that assumes objective access to ground truth and appears opposed to the notion of demographic parity. E.g., the paper claims to resolve two open questions from [4], but it only resolves them under certain assumptions. This ground-truth-dependent perspective is useful for theoretical analysis and insights, but is unrealistic and fails to capture the complexity of fairness problems. These issues aside, I still feel the paper makes relevant contributions, including reining in overblown claims in prior work such as the promise of task independence.

As the paper notes, many of the results are straightforward and not necessarily technically complicated or surprising. Nevertheless, a lot of work has gone into defining and understanding a new perspective on an area of research. So the perspective, framework, and results shown are novel and have significance.

The clarity is okay, but could be improved and I found the paper hard to follow at times. The introduction could be condensed and restructured to decrease length and reduce redundancy. This would improve readability/clarity and give more room for exposition elsewhere or more discussion of related work as currently appears in the appendix.

Overall, I would like to see the claims toned down and the limitations of the ground-truth-dependent perspective acknowledged more. The paper makes valuable contributions that are hampered by the writing and limitations of the framework. It is a borderline accept in my opinion.


Additional comments:

The focus on ground truth leads to some questionable implications, which may be due in part to unfortunate phasing. On page 2, for example, assuming correlation between group membership and ground truth as stated in the paper is different from assuming correlation between group membership and labels in a training set. The latter allows for sample bias, label bias, etc. The former is a little too close to assuming inherent differences between the groups.

Likewise, the section on “Viewing perfectly accurate decisions as fair” says, “It makes a lot of sense in tasks like conviction in a crime - if you convict all criminals and no one else, you should not be accused on unfairness.” 
Criminal justice is a problematic example here given the high-profile issues with inaccuracy and discrimination in this area that many fairness studies attempt to address. Further, the goals of criminal justice are often one-sided rather than aiming for the impossible goal of perfect accuracy (either minimizing the risk of false conviction at the expense of releasing criminals or alternatively, minimizing the risk of releasing criminals at the expense of false conviction).

Grammar: 
“A representation is fairness-driven fair if there exists a loss minimizing (or an approximate minimizer) classifier based on that representation is fair”
“we restrict our discussion the case of one binary protected attribute”
“…this requirement is might not sufficient for guaranteeing that an accuracy232
driven decision maker arrives at a fair decision”
“…since f is not a constant over X there is are points in the other group on which f has the opposite value.”

limitations_and_societal_impact: The paper gives somewhat of a non-answer to the question of potential negative social impact. I’d like to see more discussion of limitations and potential negative impact such as how this work could potentially be misconstrued in a harmful way.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper demonstrates two impossibilities raised in a fair machine learning context: (i) the impossibility of achieving a generic fair data representation; (ii) the impossibility of achieving fairness notions (except disparate parity) in the case of manipulating a single feature. The paper also answers the two questions raised in Creager et. al. [3] via theoretical analysis under binary classification setting w.r.t. varying fairness notions. 
main_review: The paper raises important and practically relevant questions concerning multiple fairness notions. Nevertheless, several points need to be clarified. First, as demonstrated in 1.1, what is the difference between malicious and fairness-driven agents? The clear distinguishment between the three groups of agents should be stated through rich prior works; this also holds for the criteria of group fairness notions.  There are also significant problems with the presentation of this paper. For example, it is not clear why the authors looked into adversarial fairness (malicious decision-maker) and then multi-task setting afterward. Lastly, while the proof of Claim 1 does not have any significant technical contributions, Claim 2 and Collorary 1 are rather straightforward. Additionally, Theorem 3 only demonstrates a feature set that can lead to the worsening of unfairness; there can be certain cases in which adding or subtracting a feature improves fairness and performance. To conclude, the overall presentation is difficult to follow. Organizations and descriptions of the proofs can be improved. In addition, the authors used the same lines from Zemel et. al. [15]. 
limitations_and_societal_impact: A clear discussion of societal impacts is provided in the paper. In spite of the numerous questions raised, however, this paper does not adequately address limitations and previous works, especially regarding multi-task fairness problems. See the 1st point in “Main Review”.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: In this work authors analyzed the transferability of fair representations and showed that it is impossible to train a classifier using fair representations that can achieve the fairness goal for all the cases.
main_review: The paper targets an interesting concept that is also timely and new; however, there are some concerns:
1. As authors also point out, having fair representation can mean different things. I do not think that the taxonomy that the authors gave here are accurate or general enough to be able to get to a general conclusion. As authors ground their work based on these definitions, I find it difficult to draw any significant conclusions based on the results.

2. In addition, the paper is written and organized very poorly. Some major proofs are not placed in the main text which make it hard to follow the paper. The paper also has some writing issues as well some pointed below, but the organization is something that needs more work. Some grammar issues:
line 231: is might not sufficient -> is not sufficient. line 382: that claim to learned -> that claim to learn.

3. The paper also jumps from one definition to another without clear reason. I would suggest authors to maybe focus on DP as many work in fair representation learning focuses on DP and put all the major discussions along with the proofs for DP in the main text and put discussion about equalized odds in the appendix. As authors also point out work in DP is more so it would make more sense to put the focus on DP instead of Eqodds.

4. I am also curious to see what authors think about information theoretic work in this area. Maybe some brief discussion should be provided on different types of techniques people use and where they fall under according to their taxonomy. I again generally find their taxonomy narrow and have difficulties about figuring out which work may fall under which section, so maybe some clarity or even changing the taxonomy can be beneficial.


limitations_and_societal_impact: Authors mentioned that they do not see negative societal impacts related to their work and also did not provide sufficient discussion about the limitations of their work. Authors can maybe talk about limitations associated to their narrowed taxonomy.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper formalizes impossibility results for fair representation learning. The authors present a series of settings where various desirable properties cannot be achieved.
main_review: I think this paper makes a valuable contribution. Some of the results are fairly intuitive, but it's helpful to see them formally specified.

There's a sense in which the results, particularly in Section 4, aren't really about representations per se. It would've been helpful to see a general version of the step from Lemma 1 to Theorem 1 -- something like, if $h_1$ and $h_2$ don't satisfy some property w.r.t. both $P_1$ and $P_2$, there is no representation that can in the adversarial case.

I found the results on "fairness of a feature" to be fairly interesting.

I'd also appreciate a bit of caution in stating these results. They're in some sense worst-case, since they're about the existence of an alternate marginal under which a learned representation would perform poorly. But this doesn't say anything about real-world cases, which may be better than the worst case.

- l. 289: "incompetability" -> "incompatibility"
- l. 350: $\eta$ and $\epsilon$ are reversed

limitations_and_societal_impact: Yes
review_title: null

Meta-review: Reviewers thought that the paper addresses an important question in Fair-ML (that is, conditions under which fair data representation leads to fair predictions), and offers potentially significant insights. That said, reviewers found the paper challenging to follow and suggested several concrete steps to improve readability and crystalize the core message. In particular, they suggested authors:
(1) elaborate on the conditions under which their results can be applied to real-world datasets (for example, if the so-called ground-truth label in the training data is itself the result of biased decisions and practices, are the results still applicable? 
(2) Avoid making blanket statements about highly nuanced and controversial topics, such as fairness of automated decision-support tools in the criminal justice context---as expressed in the passage “if you convict all criminals and no one else, you should not be accused of unfairness.”. 
(3) Strengthen the connection between different parts of the paper (e.g., between sections 4 and 5, definitions of adversarial vs. fairness-aware vs. accuracy-aware agents, various fair representation formulations).
(4) acknowledge the worst-case nature of some of the results and their implications (or lack thereof) for real-world settings.
Considering the authors' responses to their suggestions, several reviewers expressed doubt that authors will revise their paper appropriately if accepted. The area chair re-assessed several key points of disagreement between authors and reviewers and concurred with the reviewers that while the paper has the potential for impact, authors need to enhance clarity and accessibility before their work is ready for publication in top-tier venues.
Decision: Reject