Attainability and Optimality: The Equalized-Odds Fairness Revisited

Abstract: Fairness of machine learning algorithms has been of increasing interest. In order to suppress or eliminate discrimination in prediction, various notions as well as approaches to impose fairness have been proposed. However, in different scenarios, whether or not the chosen notion of fairness can always be attained, even if with unlimited amount of data, is not well addressed. In this paper, focusing on the Equalized Odds notion of fairness, we consider the attainability of this criterion, and furthermore, if attainable, the optimality of the prediction performance under various settings. In particular, for classification with a deterministic prediction function of the input, we give the condition under which Equalized Odds can hold true; if randomized prediction is acceptable, we show that under mild assumptions, fair classifiers can always be derived. Moreover, we prove that compared to enforcing fairness by post-processing, one can always benefit from exploiting all available features during training and get better prediction performance while remaining fair. However, for regression tasks, Equalized Odds is not always attainable if certain conditions on the joint distribution of the features and the target variable are not met. This indicates the inherent difficulty in achieving fairness in certain cases and suggests a broader class of prediction methods might be needed for fairness.

Reviews and author responses:

The paper studies the attainability of the equalized-odd fairness criteria introduced by Hardt et al'16 in classification and regression tasks. In particular, the paper claims that under certain conditions EQ is not even attainable. They proved the claim for the regression task but I could not find exactly where they discuss the classification attainability.  In fact, the (non)attainability claim about EQ is confusing to me since by definition, a *perfect* predictor (which is non-trivial) satisfies EQ. Intuitively, their result is meaningful for the task of linear regression as a perfect predictor may not exist. But as we consider classification or non-linear regression it becomes less believable. For instance as the authors mentioned, in a previous work, Woodworth et al.'17 showed that even checking a predictor is fair w.r.t. EO notion is not possible when we use *finite* many samples. 

The results on classification with deterministic prediction seems restrictive as condition (i) seems very strong. Assuming condition (i), Theorem 4.1 seems straightforward to prove. Also, what is the difference between Theorem 4.2 and the previous results on the existence of fair predictor w.r.t. EQ in Hardt et al.'16 (e.g., Proposition 4.4.). Please elaborate on this. Lastly, Theorem 4.3 seems very interesting.

Minor comments:
- The figure 1 which has been referred to several times of the paper is missing.
- Hardt et al. entry in References has typo.
- page 2: outputing -> outputting, utiziling ->utlizing
- page 3: Eqaulized -> Equalized
- page 4: Eqalized -> Equalized
- page 6: unconstrianed -> unconstrained, seperately -> separately
- page 8: convariance -> covariance, appximated -> approximated, numerial -> numerical
- page 13: quadractic -> quadratic
- page 15: insection -> intersection

=====POST-REBUTTAL COMMENTS========
I would like to thank authors for their clarifications. Accordingly, I have increased my score to 6.

Thank you for the encouraging, insightful, and detailed feedback! Below, we address the concern and summarize the changes in the revised paper.

### 1. Q: "How is the level of fairness violation selected? Are they the best attainable level? ... the reductions approach returns a randomized classifier"

In the experiment presented in Figure 3, following Agarwal et al. (2018) (their [implementation](https://fairlearn.github.io/v0.5.0/user_guide/mitigation.html#fairness-constraints-for-binary-classification)), we pick 0.01 as the default violation bound that the EOdds violation does not exceed (if practically achievable for the method) during training.

In the revised version of the paper, we also clearly stated the fact that the approach proposed by Agarwal et al. (2018) outputs randomized predictions and that the randomization can come in two folds (by picking a classifier from its distribution and performing probabilistic prediction).

### 2. Comment: Related work by Cummings et al. (2019)

Thanks a lot for pointing out this related work. In the revised version of the paper, we included this in the discussion about the related works.

### 3. Q: "... in Theorem 3.1 how strong the assumption is that $f_2$ and $f_3$ are third-order differentiable"

This is a good question on the technical side. Here, $f_2$ and $f_3$ are log densities of the noise term. For most distributions we are familiar with, the third-order differentiable assumption holds true. In fact, this seems to be a standard assumption in the identifiability studies (e.g., Kagan, Abram M., Calyampudi Radhakrishna Rao, and Yurij Vladimirovich Linnik. "Characterization problems in mathematical statistics." (1973)).

### 4. Comment: "Conditions given in Theorem 4.1 seems to be the necessary and sufficient condition. It would be better if authors can clearly state it in the theorem."

Thank you for pointing this out. In the revised version of the paper we explicitly stated the equivalence between Equalized Odds criterion and the two conditions a deterministic classifier should satisfy.

### Summary of other changes in the revised paper

- Before presenting Theorem 4.2, we added a paragraph to further elaborate the difference between the result presented in Hardt et al. (2016) and this paper.

- We revised the statement of Theorem 4.3 and added more detail in the proof to further improve the clarity.

- The typos are fixed.

Thank you for the detailed feedback! Below, we respond to the comments.

### 1. Comment: "... make clear which statements are novel and which are already known in the literature. For example, the result in Theorem 4.2 seems to be contained in Hardt et al. (2016)"

Thank you for pointing out the connection between the algorithm by Hardt et al. (2016) (Proposition 4.4) and our result (Theorem 4.2). However, please note that we aim to establish the condition under which Equalized Odds is attainable in a non-trivial way -- this is not guaranteed by the algorithm. While the technique itself (to derive binary predictors via the ROC plane) is credited to Hardt et al. (2016), our contribution (w.r.t. fairness in classification) lies primarily in the formal establishment of the existence of non-trivial Equalized-Odds predictors (Theorem 4.2), as well as the relation between in-processing and post-processing fair predictors (Theorem 4.3).

NOTE: In the revised version of the paper, before presenting Theorem 4.2, we added a paragraph to further elaborate the difference between the result presented in Hardt et al. (2016) and this paper.

### 2. Comment: "... only in- and post-processing are compared. It would be interesting to include also the pre-processing to the discussion."

Thank you for pointing this out. To make it complete, we now have incorporated the reason why pre-processing approaches are not included in the comparison in the revised paper (Section 4.2.3).

More specifically, the pre-processing approaches to impose Equalized Odds (e.g., Madras et al. 2018; Zhao et al. 2020) are not included in the discussion since the objective is different. They approach the algorithmic fairness problem from a representation learning perspective, where the main focus is to learn fair yet informative representations from the original data. Therefore, when discussing the performance of fair predictors, we only draw comparison between the post-processing and in-processing approaches.

### 3. Comment: "The authors compare deterministic algorithms with a probabilistic algorithm from Hardt et al. (2016), which does not seem to be a fair comparison."

Thank you for pointing this out. It is indeed not a fair comparison as indicated by the results in the paper. However, we want to use this comparison to illustrate our point that Equalized Odds is not guaranteed to be attainable for arbitrary classifiers.

In terms of fairness or accracy, the deterministic and randomized algorithms actually have their own advantage. As we have shown in the paper, Equalized Odds is not always attainable with a deterministic prediction function (Theorem 4.1). On the other hand, the randomization that is introduced to eliminate the violation of fairness (e.g., the post-processing approach by Hardt et al. (2016)) comes at the cost of accuracy. We should mention that it is not our intention to promote a preference of one algorithm over another. Instead, we would like to present the fact that fairness may not be attainable, and we should always be cautious when deploying "fair" algorithms in practice.

### 4. Comment: "Lemma A.1 looks like a known fact."

To the best of our knowledge this is not presented before. We would appreciate it and directly refer to the contribution if you can kindly provide us with a pointer. We are aware of the result in the unconditional independence case (the joint density can be factored up if and only if variables are independent) but not the conditional independence case.

### 5. Comment: "... make clear transitions steps (in the proof for Theorem 3.1) by justifying what properties of random vectors are used."

Actually when we present Equation 4, we assume that the noise terms are mutually independent as is commonly assumed in the structural equation models. In the revised version of the paper, we made it explicit that noise terms are independent after presenting Equation 4.

### 6. Comment: "Theorem 4.3: to me, both the statement and the proofs are hard to parse."

In the updated version of the paper, we revised the statement of the theorem and added more detail to the proof. More specifically, we made it more explicit what ROC feasible areas we are comparing so that the equivalence between sets of constraints is clearly established.


Thank you for the feedback! Please notice that there is some misunderstanding. Below, we respond to the comments and hope that we can resolve the misunderstanding.

### 1. Comment: "... the (non)attainability of EQ is confusing to me since by definition, a perfect predictor satisfies EQ"

In this paper, we consider the (strict) attainability of Equalized Odds notion of fairness. In particular, we are considering the following question: for a given task (regression or classification), is it possible to find a non-trivial predictor that strictly satisfies Equalized Odds condition (i.e., there is **no** violation of fairness at all) in the large sample limit. 

### 2. Comment: "... could not find exactly where they discuss the classification attainability"

We show that for regression (Theorem 3.1) and classification (Theorem 4.1) tasks with deterministic prediction functions, Equalized Odds is not always attainable. However, under mild assumptions one can always find a non-trivial Equalized Odds binary classifier (i.e., Equalized Odds is attainable) if randomized prediction is taken into consideration (Theorem 4.2). We also discuss the optimality of performance under the fairness constraint between in-processing and post-processing predictors (Theorem 4.3).

### 3. Comment: "The results on classification with deterministic prediction seems restrictive as condition (i) seems very strong. Assuming condition (i), Theorem 4.1 seems straightforward to prove."

We believe there is some misunderstanding. Here we actually aim to find the conditions under which Equalized Odds can possibly hold true.

More specifically, instead of assuming condition (i), we start from the Equalized Odds constraint, and take into consideration the fact that the classification function is a deterministic mapping from input features to the output; then we derive the conditions under which Equalized Odds can hold true, which are the conditions (i) and (ii) of the theorem. We find that those two conditions are strong and restrictive (as we stated in Section 4.1), and this implies the fact that in general those conditions would be violated, i.e., Equalized Odds is not attainable in this case.

NOTE: In the revised version of the paper, we clearly stated that these conditions are actually both necessary and sufficient, i.e., under the setting specified by the theorem, Equalized Odds holds true if and only if conditions (i) and (ii) are satisfied.

### 4. Comment: "... the difference between Theorem 4.2 and the previous results ... in Hardt et al.'16 (e.g., Proposition 4.4)"

Thank you for pointing out the connection between the algorithm by Hardt et al. (2016) (Proposition 4.4) and our result (Theorem 4.2). However, please note that we aim to establish the condition under which Equalized Odds is attainable in a non-trivial way -- this is not guaranteed by the algorithm. While the technique itself (to derive binary predictors via the ROC plane) is credited to Hardt et al. (2016), our contribution (w.r.t. fairness in classification) lies primarily in the formal establishment of the existence of non-trivial Equalized-Odds predictors (Theorem 4.2), as well as the relation between in-processing and post-processing fair predictors (Theorem 4.3).

NOTE: In the revised version of the paper, before presenting Theorem 4.2, we added a paragraph to further elaborate the difference between the result presented in Hardt et al. (2016) and this paper.

### 5. Misc (Figure 1 missing; typos)

The Figure 1 seems to be showing correctly from our end, if there is still an issue viewing the graph, please kindly let us know. The typos in the reference and the text are corrected. Thank you for catching them.

We thank the reviewer for the encouraging feedback! Below, we respond to the specific comments and questions.

### 1. Q: "The result only proves weak inclusion of the Omega() sets. Is there any interpretable conditions when the inclusion is strict and in-processing is strictly better than post-processing?"

We believe there is some misunderstanding. In Theorem 4.3 we establish the equivalence between in-processing and post-processing approaches in terms of their Omega() sets. As is shown in the paper, apart from the constraints in (i), ROC feasible area should satisfy an additional set of "pseudo" constraints (ii), which are in general not vacuous. Compared to the post-processing approach, the in-processing approach is free from such "pseudo" constraints. Therefore the inclusion of the Omega() sets is in general a strict inclusion.

### 2. Q: "... does a similar result can be proven for other fairness constraints, perhaps extending the proof technique presented here?"

Similar result can be established for some of the other fairness notions. For example, the proof technique in the paper can be readily adapted to analyze the attainability of _Equal Opportunity_ (Hardt et al. 2016), which is a strictly weaker notion of Equalized Odds and imposes only equal TPR among groups. For _Demographic Parity_ (Calders et al. 2009), one can show that it is always possible to find a predictor that is independent from the protected feature by leveraging the proof of the existence of solutions to nonlinear ICA (c.f. Theorem 1 of [Hyvärinen and Pajunen (1999)](https://www.cs.helsinki.fi/u/ahyvarin/papers/NN99.pdf)), i.e., Demographic Parity is always attainable.

However, considering the difference in nature of various fairness notions that have been proposed in the literature (e.g., individual fairness (Dwork et al. 2012), counterfactual fairness (Kusner et al. 2017), etc.), it is not obvious to us at the current stage how to extend the current proof and develop a uniform framework that is able to analyze the attainability of these different fairness notions. Nevertheless, (as we have stated in the conclusion section) the attainability of more fine-grained fairness notions (e.g., individual fairness) as well as _procedure fairness_ would be an interesting direction for future works.

### 3. Comment: "... wondering if the authors have more intuition regarding this (the unattainability of Equalized Odds for non-linear regression with deterministic functions), at least, I don't see how the proof for the linear case extends for the general non-linear setting."

We made a lot of effort but haven't found a straightforward solution. As a consequence we just gave empirical illustrations. We agree that this study is important but clearly non-trivial. Therefore we consider this as our future work but at the current stage we have no clear clue regarding how to achieve it.

### 4. Comment: "... the techniques are not novel. The proof (for Theorem 4.2) relies on the ROC plane characterization of post-processing classifiers established in earlier papers."

Thank you for pointing out the connection between the algorithm by Hardt et al. (2016) (Proposition 4.4) and our result (Theorem 4.2). While the technique itself (to derive binary predictors via the ROC plane) is credited to Hardt et al. (2016), our contribution (w.r.t. fairness in classification) lies primarily in the formal establishment of the existence of non-trivial Equalized-Odds predictors (Theorem 4.2), as well as the relation between in-processing and post-processing fair predictors (Theorem 4.3).

NOTE: In the revised version of the paper, before presenting Theorem 4.2, we added a paragraph to further elaborate the difference between the result presented in Hardt et al. (2016) and this paper.

### 5. Comment: "Additionally, I think a more interesting question to consider would have been the trade-off between EO fairness and accuracy."

In the literature we have seen the trade-off between Equalized Odds fairness and the prediction accuracy (e.g., Agarwal et al. (2018), Baharlouei et al. (2020)) with various levels of fairness violation in practice. However, this does not entail a guarantee nor the impossibility of attaining Equalized Odds (with 0-violaition of fairness) in the large sample limit. Therefore in this paper we consider the strict attainability of Equalized Odds in various settings. We think it is important and necessary to recognize the phenomenon that Equalized Odds is not always attainable and be more cautious when utilizing "fair" predictors in reality.


Thank you for the feedback! Below please see our response to the specific comments or questions.

### 1. Comment: "Overall, I don't really see the motivation for this paper."

Although various approaches have been proposed to impose Equalized Odds, it is not clear whether or not this fairness criterion can actually be always attained (in the sense that there is no violation of fairness in the large sample limit). As shown in the paper, Equalized Odds is not always attainable for regression and even classification tasks, even if there is an unlimited amount of data. It is therefore essential and necessary to recognize this phenomenon and be more cautious when utilizing "fair" predictors in reality.

### 2. Comment: "Equalized odds isn't typically considered in linear regression tasks"

When introducing the Equalized Odds notion of fairness, Hardt et al. (2016) stated that "Equalized Odds applies to targets and protected attributes taking values in any space, including binary, multi-class, continuous or structured settings". Therefore, we include both the classification and the regression tasks when considering the attainability of Equalized Odds.

### 3. Comment: The paper doesn't answer "if the conditions (for exact Equalized Odds) ... are not met, is it still possible to get an epsilon-approximation of Equalized Odds?"

In the literature, violation of fairness is usually considered on finite samples. We admit that in practice we generally cannot have 0-violation of fairness because of finite samples. However, we are considering the asymptotic property of the fairness criterion, which examines the violation of fairness from a completely different way. If asymptotically speaking Equalized Odds is violated, in order to get an epsilon-approximation of fairness, we need to relax the fairness criterion itself, instead of allowing for an error bound of fairness violation "introduced by" the finite sample size.

### 4. Comment: "The techniques used to derive the positive result ... appear to just be a formal statement of the algorithm given by Hardt et al. (2016)."

Thank you for pointing out the connection between the algorithm by Hardt et al. (2016) (Proposition 4.4) and our result (Theorem 4.2). However, please note that we aim to establish the condition under which Equalized Odds is attainable in a non-trivial way -- this is not guaranteed by the algorithm. While the technique itself (to derive binary predictors via the ROC plane) is credited to Hardt et al. (2016), our contribution (w.r.t. fairness in classification) lies primarily in the formal establishment of the existence of non-trivial Equalized-Odds predictors (Theorem 4.2), as well as the relation between in-processing and post-processing fair predictors (Theorem 4.3).

NOTE: In the revised version of the paper, before presenting Theorem 4.2, we added a paragraph to further elaborate the difference between the result presented in Hardt et al. (2016) and this paper.

### 5. Comment: "Section 2.1 ... what the purpose is."

The Equalized Odds notion of fairness requires conditioning on the value of the target variable. If there is bias hidden in the data, this practice (of conditioning on $Y$) itself might be problematic. Therefore, in Section 2.1 we would like to state explicitly that in the setting of this paper, only prediction fairness (assuming the data generating procedure is not biased in any sense) is considered.

Summary: 

This paper studies the attainability of Equalized Odds fairness criterion in both the classification and regression problems. When the prediction function is deterministic, it shows that Equalized Odds may not be attainable under certain conditions. In contrast, if the prediction function is randomized, then Equalized Odds classifiers can always be achieved under some conditions. Moreover, it shows that the performance attained using the in-processing approach after exploiting all available features is always better than attained using the post-processing approach. 

Strengths:

1. The conditions under which perfect Equalized Odds can be attained are identified under various settings, including both regression and classification, and cases when prediction mapping is deterministic and randomized. 

2. Utilizing ROC feasible area, the paper builds a connection between two approaches used to achieve fairness: in-processsing approach and post-processing approach. Specifically, the constraint enforced by post-processing consists of the constraint enforced by in-processing and an additional constraint.  

Comments: 

1. In the experiments (Figure 3), how is the level of fairness violation selected? Are they the best attainable level? Because of the tradeoff between accuracy and fairness, it is possible to improve accuracy by adjusting the fairness guarantee. For example, according to Figure 1 in Agarwal et al. 2018, for many datasets, the reduction approach (in-processing) can achieve any desired fairness level between fairness of post-processing (Hardt et al. 2016) and fairness of unconstrained classifier. It means that it is possible to achieve perfect Equalized Odds using Agarwal et al. 2018. I hope the authors can explain this point. Moreover, the reduction approach returns a randomized classifier, rather than the deterministic function of features as mentioned in the paper. It is necessary to make clarifications in the paper. 

2. Related work: the attainability of perfect fairness (in addition to Equalized Odds) has been studied in the literature, which is highly related to this work. I suggest authors including them in related work. For example, "Cummings, R.; Gupta, V.; Kimpara, D.; and Morgenstern, J. On the compatibility of privacy and fairness, 2019". It shows that it’s impossible to train a differentially private classifier (a randomized classifier) that satisfies exact (perfect) Equal opportunity fairness and achieves a higher accuracy than a constant classifier.

3. It would be better if authors can add some explanations/intuitions of theorems, e.g., in Thm 3.1, how strong the assumption is that $f_2$ and $f_3$ are third-order differentiable.  

4. Conditions given in Theorem 4.1 seems to be the necessary and sufficient condition. It would be better if authors can clearly state it in the theorem (e.g., Equalized Odds holds if and only if conditions are satisfied).

5. Typos: in Eqn (5), $P_{\hat{Y}|AY}(1|a,y) —> P_{\hat{Y}|AY}(u|a,y)$ 




# Main claims and contributions

The authors study if Equalized Odds (EO) definition of fairness is attainable for regression and classification problems. In their paper, the authors make several contributions:
1. They provide an example of linear regression problem, for which EO can never be satisfied (Theorem 3.1)
2. For classification problem, the authors prove in Theorem 4.1 that if the predictor is deterministic, then EO is satisfied only under strict assumption on data distribution. In contrast, if the predictor is randomized, the authors show in Theorem 4.2 that EO is attainable under mild assumptions.  
3. The authors compare different techniques for obtaining EO-satisfying predictors. Namely, they compare in- and post-processing techniques. They prove in Theorem 4.3 that in-processing allows for better classification accuracy compare to the post-processing technique from Hardt et al. (2016).

The authors, perform numerical experiments to support their claims. For regression task, they learn linear and non-linear predictors on data with gaussian and non-gaussian noise and observe that EO is essentially impossible to satisfy in non-linear regression even if exogenous terms are gaussian. For classification problem, they compare different in-processing techniques with a post-processing technique from Hardt et al. (2016).

# Novelty

To the best of my knowledge, the question of attainability of EO for regression/classification for deterministic clarifier is novel in the literature.

On the other hand, the same question for randomized predictor (Section 4.2) seems to be already answered in the literature by Hardt et al. In particular, Theorem 4.2 seems already known and I did not understood clearly what is the contribution of 4.2.3.

# Clarity and soundness

The paper is relatively easy to follow, apart from Section 4.2.3 (and in general the discussion about the ROC curve, and the role of Figure 1). The assumption of infinite data allows the authors to simplify the exposition. Some details:

1. Lemma A.1. looks like a known fact. 
2. Theorem 3.1: make clear transitions steps by justifying what properties of random vectors are used.
3. Theorem 4.1: proof seems to be fine, no comments
4. Theorem 4.2: in Hardt et al. (2016), they show that the resampling probabilities can be calculated by solving an LP, need to justify the novelty of the result / emphasize the increment.
5. Theorem 4.3: to me, both the statement and the proofs are hard to parse.

# Reasons to accept

The authors ask an important question of whether the standard definition of fairness (EO) is attainable for any instance of problems even with infinite data. They provide a counterexample for regression problem. The authors give a condition under which EO is attainable for classification. The authors compare in- and post-processing techniques in terms of their efficiencies.

# Reasons to reject

The authors are encouraged to make clear which statements are novel and which were already known in the literature. For example, the result in Theorem 4.2 seems to be contained in Hardt et al. (2016).

The authors mention pre-, in- and post-processing techniques for fair classification/regression. However, only in- and post-processing are compared. It would be interesting to include also the pre-processing to the discussion.

The authors compare deterministic algorithms (Zafar et al. (2017a) , Rezaei et al. (2020)…) with a probabilistic algorithm from Hardt et al. (2016), which does not seem to be a fair comparison. 

## minor Typos:

P.4 Equalized Odds
P.5  Equalized Odds
P.6 separately
P.7 covariance, approximated


This paper studies under what conditions a classifier can satisfy the condition of equalized odds. The authors first prove an impossibility result which shows that (under linear non-gaussian case) any deterministic classifier	cannot achieve equalized odds across the protected groups. This leads to the question of randomized classifier. We can always satisfy equalized odds with randomized classifier  with a trivial classifier. However, the authors show two interesting results — (1) using the post-processing framework developed in Hard et. al. (2016) it is possible to obtain a non-trivial randomized classifier that satisfies EO, and (2) in-processing based classifiers are better than the post-processing based fair classifiers if we compare them in terms of accuracy.

I didn’t find the result about deterministic classifier to be surprising. However, it is interesting that the authors think that a similar result holds for non-linear regression. The experiments definitely suggests that but I am wondering if the authors have more intuition regarding this direction. At least, I don’t see how the proof for the linear case extends for the general non-linear setting.

The results for the randomized classifiers are interesting, however I think they are expected. Several prior papers have observed that in-processing based classifiers do perform better than the post-processing classifiers in practice (e.g. see experimental section in Agarwal et. al. 2018). But it is nice to see a formal proof of this fact. I have two questions regarding this result —

1. The result only proves weak inclusion of the Omega() sets. Is there any interpretable conditions when the inclusion is strict and in-processing is strictly better than post-processing?
2. I understand the focus of this paper is EO constraint, but does a similar result can be proven for other fairness constraints, perhaps extending the proof technique presented here?

Weaknesses:

Even though the results in this paper are interesting, the main weakness of the paper is that  the techniques are not novel. In fact, the proof relies on the ROC plane characterization of post-processing classifiers established in earlier papers. Additionally, I think a more interesting question to consider would have been the trade-off between EO fairness and accuracy. In fact, the plots in figure 3 does show such trade-offs. 

In summary, I think the paper makes interesting contributions in understanding the attainability of the EO fairness constraints. However, the results are not complete in its current setup and also the techniques are limited to the particular notion of fairness considered in this paper.


This paper examines the conditions under which equalized odds can be achieved from a theoretical perspective. The authors show that in general, equalized odds cannot be achieved in linear regression and clasisication tasks. However, when randomized prediction is allowed, they show that equalized odds can be achieved in binary settings.

Overall, I don't really see the motivation for this paper. Equalized odds isn't typically considered in linear regression tasks, and moreover, the generalization given in this paper is particularly restrictive -- it requires matching distributions across sensitive attributes conditioned on the target variable. One could imagine a less restrictive generalization like matching expectations, i.e., $E[\tilde Y | Y, A = a_1] = E[\tilde Y | Y, A = a_2]$.

In general, equalized odds can always be achieved with trivial, constant classifiers, so the authors specify that they're only interested in non-trivial classifiers. This paper only considers the question of whether classifiers can exactly satisfy equalized odds -- presumably, given that trivial classifiers satisfy equalized odds, something epsilon-close to a trivial classifier will satisfy an epsilon-approximation of equalized odds. What this paper doesn't answer, and what would in my mind be a far more interesting result, is the converse: if the conditions specified in the paper are not met, is it still possible to get an epsilon-approximation of equalized odds? If so, then this would imply that the conditions derived here are in a sense qualitatively necessary. If not, then these results would seem more brittle.

The techniques used to derive the positive result (that equalized odds can be satisfied with randomized classification) appear to just be a formal statement of the algorithm given by Hardt et al. (2016).

The paper itself is a little difficult to parse at times. Section 2.1 appears to be an attempt to specify a qualitative relationship between ideas about fairness, but I find it difficult to understand what the purpose is.


Meta-review: The paper study under which condition a classifier can respect the condition of equalized odds. The reviewers find the paper interesting but they also raise some important concerns about it.

First, multiple reviewers pointed out that the results are not particularly novel or surprising and, even after discussing the rebuttal, they consider the result a bit incremental.

Second, the motivation of the paper are also questioned by multiple reviewers that suggested to study the tradeoff between trade-off between EO fairness and accuracy.

Overall, the paper contains some interesting ideas but it is below the high acceptance bar of ICLR.

Decision: withdrawn-rejected-submissions