Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This work proposes a notion of long-term impact called equal improvability (EI) that equalizes the "effort required to improve" of rejected individuals belonging to different sensitive groups. This metric is theoretically compared to three other long-term metrics: Bounded Effort and Equal Recourse---conditions for meeting each of these notions of fairness (in relation to each other) are derived. Three fairness-based regularization methods are proposed to find policies that mitigate unfairness wrt EI, each with a unique differentiable penalty term. These methods are empirically tested against synthetic and real-world data. In the first set of experiments, the three mitigation strategies are tested against a fairness-unaware algorithm over two real-world datasets (non-dynamic). In the second set of experiments, results for a simple dynamical setting are shown.  
strength_and_weaknesses: Strengths / comments: 
- I enjoyed reading the introduction and the (first) related work section, where the authors pinpoint the importance of their metric and show how it is distinct from current known metrics in the field. 
- Concepts are elaborated on in the appendix or figures are provided to help readers better understand material. The example of $\mu(\Delta \mathbf{x_1})$ in the para underneath Defn 2.1 was very helpful. 
- The authors did not over-state the results of their fairness-based regularization algorithms, e.g., acknowledged that these methods can mitigate EI unfairness and do not provide further guarantees on solutions found. 
- The organization of section 3 was straightforward and well-written. 

Weaknesses / comments: 

- Why don't the authors compare the other long-term fairness metrics to the long-term fairness-unaware metrics in the first set of experiments? ERM is a very simple baseline, and arguably the three datasets provided are three more datasets over which other long-term fairness baselines can be used as a comparison. We could consider this a single-step episode, which is a subset of an n-step episode. The paper that presents ER also presents a fairness-based regularization method to find solutions that mitigate ER. 

- How easy is it to find policies that mitigate unfairness when there are multiple groups, as is the case in many real-world applications? (For example, in applications where one might consider race, gender, or relevant proxies.)  One of the issues with just adding a (fairness-based) penalty to the objective is that the alg is only given incentive to satisfy fairness, but has the freedom to violate fairness if that results in a more improvement to the original objective. Qualitatively, it seems that scaling the importance of each of the additional pair-wise penalties wrt the primary objective would not be simple or insightful for users. 

- Adding on to the previous point, the Zafar paper that inspired the covariance-based EI penalty creates proxies to the original fairness constraint, but does not show (theoretically) that solutions found by the new proxy-objective solve the original objective. The covariance-based EI method also has no guarantees on the solution found by the algorithms (perhaps the KDE-based penalty as well though I am unfamiliar with the work which inspired it, and did not take the time to read through the work). Ultimately this is a weakness of using fairness-based regularization to mitigate unfairness. I suggest that the users address this as a weakness in the main body. 
 
clarity,_quality,_novelty_and_reproducibility: Clarity: Overall the paper is straightforward to read. The proofs I skimmed (app A) seem clear. 
- Page 6 in experimental setting: Where are the "statistics for five trials"? Are the errors standard error or standard deviation? 

Reproducibility: Information needed for reproducibility is located in the appendix. I did not look at the code but the authors also assert that hyperparameters used in the experiments are located there as well. I feel confident that I could re-construct the experiments from the paper.

Quality/Novelty: The authors show that the notion of fairness considered is similar to current long-term metrics but distinct, and provide three algorithms, two of which are small changes from previous works that use soft constraints, i.e., create a fairness-based regularization penalty. These methods do not come with guarantees on the solution found by the optimization process, but can be considered stepping stones for future work.  

Small typos / issues and suggested improvements: 
- Appendix B first sentence: "we explain what each term means in EI definition means" -> "we explain what each term in the EI definition means" 
- I did not realize until the end of the paper that there was a second related work section---I suggest mentioning this in the first related work section! 
- "First-order" is never truly defined, and the reader is left to infer from the text. Does first-order imply that these metrics "take potential follow-up inequity risk" one "time-step" into the future, i.e., one decision into the future? The experimental section shows a one-step scenario (the first set of experiments), and a multi-step scenario (three rounds). Are the long-term metrics mentioned actually n-step fairness notions? 
- EI disparity can be inferred from the main text but I suggest that to reduce confusion it is defined in the main work. ATM the authors refer to appendix B.3 and then use this term later numerous times in the main body. 
summary_of_the_review: I make a recommendation to accept. 

- The approach seems well-motivated, and the authors take care to show that their metric is distinct from other current methods in the longterm fairness space. 
- The paper also does not over-state its claims, e.g., the three optimization algorithms introduced offer no guarantees on the solutions found will actually mitigate EI unfairness. This limitation is well-known in regards to soft constraints but is still important to address in the main body (see weaknesses for an elaboration). 
- There are also a few clarifying points from the experiments I would like addressed, such as the choice of baselines for the non-dynamic experiments, as well as the statistics being presented in the experiments. 

--- post rebuttal --- 
I thank the authors for the detailed response! The authors agreed to address many of my concerns in their next iteration, which I think will make the work more straightforward. After discussion with other reviewers, I still lean towards acceptance.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a new notion of group-level fairness, i.e., Equal Improvability (EI). The intuition behind the notion is that, by considering those who are rejected but are not too far away from the decision border, the effort taken by those to recourse should not vary among groups. The paper provides three empirical ways to (approximately) enforce EI in classification tasks. Empirical results in the simulated dynamics are also provided.
strength_and_weaknesses: ## Strength

The proposed EI notion combines advantages of previous notions of a similar flavor (as summarized in Table 1), and makes an effort to avoid (some) problems of previous group-level fairness notions. Three different ways to (approximately) enforce EI are also provided. The authors demonstrate the benefit of considering EI in the dynamic setting in the experiments.

## Weakness

### 1. w.r.t. the distinction between zero-order vs. first-order fairness

I am wondering what is the "order" over here. Roughly, I think the paper is pointing out the fact that EI (among others, e.g., Equal Recourse, Bounded Effort) cares about the ability to change the status quo. However, since it is a shared characteristics among some previous notions, I am not sure what specific benefit is unique to EI. Also, this distinction is only made in the beginning of the paper, and was not mentioned after Section 2. It would be very helpful if authors can share some insight over here, especially for the purpose of long-term fairness, as claimed in the paper.

### 2. the unmentioned connection between individual-level recourse fairness and EI

If I understand it correctly, the central claim w.r.t. the advantage of EI compared to other notions is the potential to "equally" improve decision outcome for those who are currently rejected. This intuition is shared by, as pointed out in the paper, effort-based fairness notions. In my opinion, the intuition is actually more aligned with individual-level recourse fairness, e.g., von Kügelgen et al. (2022) "On the Fairness of Causal Algorithmic Recourse". EI claims to be more focused on those who are potentially able to recourse. However, the fact that EI remains a group-level fairness notion indicates that individual-level recourse fairness is more fine-grained and can better serve the intended purpose laid out by the authors. Additional discussion on the connection and difference would be very helpful.

### 3. w.r.t Definition 2.1 EI

I am wondering why we should limit our consideration to $\Delta x$ such that $\Delta_M x = 0$. According to the notion, by definition mutable features do not contribute to the cost of effort but can still alter the final prediction outcome. Then, I think it is more reasonable to encode such detail in the measurement of cost instead of forcing $\Delta_M x = 0$, since this will implicitly limit the possibility to recourse for certain individuals and in turn affect the estimation of recourse probability.

clarity,_quality,_novelty_and_reproducibility: Overall, the paper is not hard to follow. The results are relatively clearly presented. The paper can benefit from some additional illustrations/discussions of the claimed advantages of EI. The implementation details are provided in the paper for reproducibility.
summary_of_the_review: The paper proposes a new group-level fairness notion and three ways to empirically enforce it in classification. There are some worries about the claimed advantages of EI, as well as the formulation of the definition of EI itself (as detailed in "Strength and Weakness"). It would be greatly appreciated if authors can kindly provide additional clarifications.

====== Post rebuttal =========

Thank authors for the response. I have updated my evaluation.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper studies the delayed impact of fairness. The authors introduce a notion of fairness called \emph{equal improvability}, which, subject to a bounded amount of improvement, equalizes the probability of acceptance for the rejected members across all populations. The authors aim to solve the typical loss minimization problem of classification subject to the equal improbability constraint. Since the constraint is non-convex, the paper proposes 3 different approaches to solve the constrained optimization. The paper concludes with empirical analysis both in the one-shot game as well as the repeated setting.
strength_and_weaknesses: ------------------------------------------
Strengths:
------------------------------------------
-- The delayed impact of fairness in machine learning has not received much attention in the community (mainly due to the hardness of modeling this impact) so in that sense, it is great to see papers on this topic.

--The paper is pretty well written and it is clear to follow the high-level ideas of the paper. The delayed impact notion of fairness is natural and sensible.

------------------------------------------
Weaknesses:
------------------------------------------
-- The main technical contribution of the paper is the definition. Hence, the technical novelty (when enforcing the equal improvability constraint) is limited. 

-- Section 4.2 contains the most interesting part of the paper, though some of the details are hard to follow/justify. It would be great to justify the choice of $\epsilon(x)$ as well as describe in more detail how the updates to the classifier are done.

------------------------------------------
Minor Comments:
------------------------------------------
-- Theorem 2.5 and Corollary 2.6 show that equal improvability and bounded effort are mathematically different. It would be nice to 
describe this in more detail as it is not clear to me how to set the reward function in Heidari et al., 2019 to connect the two works.

-- Are there deeper connections between equal recourse and equal improvability that are overlooked? If not, is not it surprising that these two methods work exactly the same in the experiments in section 4.2?

-- Please stop using the Adult dataset: https://proceedings.neurips.cc/paper/2021/hash/32e54441e6382a7fbacbbbaf3c450059-Abstract.html

-- The Y value in Figure 4 should be bounded by 1. Why is it above one in the left two sub-figures?

clarity,_quality,_novelty_and_reproducibility: The paper is clear and it is easy to follow the high-level ideas of the paper. The paper is relevant to the community. The definitions are novel though the connections to prior work should be described more clearly (see above). The code is provided.
summary_of_the_review: Overall, I enjoyed reading the paper. I think it is a nice addition to the growing literature on the delayed impact of fairness. Though the technical contributions are not strong, it can be a welcome addition to the ICLR's program.

------------------------
Post rebuttal:
------------------------
I want to sincerely thank the authors for answering my questions thoroughly. I have also read the other reviews and author responses. I think we all feel that this work can go either way. I am still (slightly) in favor of acceptance though.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a new notion of fairness called Equal Improvability (EI) that requires a binary classifier to be equally difficult to change predictions for each sensitive attribution group. To achieve this property, three different penalizations are discussed: covariance-based EI penalty, KDE-based EI penalty, and loss-based EI Penalty. The authors empirically showed that the three penalization methods achieve EI disparity. In addition, they claimed that EI promotes long-term fairness in dynamic scenarios using synthetic data.
strength_and_weaknesses: - Strength
  - The paper considers a timely important problem: training a fair machine learning model.

- Weaknesses
  - **Novelty:** It seems the novelty is weak. Specifically, the author proposed EI  
$$
\mathbb{P}\left( \max_{  \mu( \Delta \mathbf{x}_1 ) \leq \delta} f(\mathbf{x} + \Delta\mathbf{x}) > 0.5 \mid f(\mathbf{x}) < 0.5, \mathbf{z}=z \right) = \mathbb{P}\left( \max_{  \mu( \Delta \mathbf{x}_1 ) \leq \delta} f(\mathbf{x} + \Delta\mathbf{x}) > 0.5 \mid f(\mathbf{x}) < 0.5 \right) 
$$
which is equivalent to
$$ \mathbb{E} \left[  \mathbf{1} \set{ \max_{  \mu( \Delta \mathbf{x}_1 ) \leq \delta} f(\mathbf{x} + \Delta\mathbf{x}) > 0.5 }  \mid f(\mathbf{x}) < 0.5, \mathbf{z}=z \right] = \mathbb{E} \left[  \mathbf{1} \set{ \max_{  \mu( \Delta \mathbf{x}_1 ) \leq \delta} f(\mathbf{x} + \Delta\mathbf{x}) > 0.5 }  \mid f(\mathbf{x}) < 0.5 \right]. $$
Given that the same conditional distributions are considered in Equal Resource (ER) by Gupta et al. (2019), it seems the current idea might be a naive variant of the previous idea:
$$
\mathbb{E} \left[ \min_{f(\mathbf{x}^{\prime})>0.5} \mu( \mathbf{x}^\prime - \mathbf{x})  \mid f(\mathbf{x}) < 0.5, \mathbf{z}=z \right] = \mathbb{E} \left[  \min_{f(\mathbf{x}^{\prime})>0.5} \mu( \mathbf{x}^\prime - \mathbf{x})  \mid f(\mathbf{x}) < 0.5 \right].
$$
Also, using a fairness regularization function is not a new idea as the authors mentioned in section 3. To be fair, I believe the work can still have strengths even with these similarities. However, it is not clear why the proposed idea is powerful without comprehensive comparisons with ER. The authors explained the limitation of ER in section 2, but the behavior of EI on outliers is not explicitly examined. 

  - **Mismatch:** Mismatch between the motivation and the proposed method: the authors discussed the long-term fairness in Abstract and Introduction, but the proposed method is not directly solving the dynamics of distribution shifts. In section 4.1, the dynamic setting is considered, but it is not clear how the objective function in section 4.1.1 is related to the equation (1) in section 3.

  - **Experiments:** Most of experiments consider fairly simple settings ($\mathbf{x}_t \in \mathbb{R}$, $\mathcal{Z}=\{0,1\}$, and $T$ is small) that are not realistic. The real problems should be higher dimensionality on $\mathbf{x}$ and have more sensitive classes. Moreover, it is not clear why achieving EI helps to improve long-term fairness.


clarity,_quality,_novelty_and_reproducibility: The paper is fairly well written. 
 - The proposed methods are well explained with clear notations
 - Related works are thoroughly reviewed.

I haven't checked the details, but it seems there is no issue with reproducibility. 


summary_of_the_review: I think the main issue of the current submission is the lack of novelty and comprehensive experiments. I am happy to revise my reviews if there is anything trivial I miss something.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper proposed a new notion of fairness at a group level that captures a dynamic system. In particular, the motivating goal is to model a world where each group can spend "effort" to change their features to maximize for some desired outcome. Based on this new notion of fairness, which the authors term "equal improvability" (EI), several optimization objectives are proposed. In their empirical work, the authors show how enforcing EI can be advantageous for long term fairness. The initial empirical results were somewhat lacking in the dynamic setting and high dimensional setting. The other key weaknesses identified by the reviewers are summarized  in the AC-reviewer meeting notes below.
justification_for_why_not_higher_score: While the paper studies an important problem and proposes a new notion of fairness, the existence of Kügelgen et al. paper diminishes the novelty and contributions to the literature. In particular, the current submission can be seen as a group version of the fairness notions proposed in Kügelgen et al. Further, according to the reviewers, the proposed solutions to optimizing for EI are also no new and closely match those existing in previous work. 
justification_for_why_not_lower_score: After posting the meeting summary on openreview, the authors responded to each of the points proposing concrete edits to the paper. They also added additional experiments, which should strengthen the paper further. Overall, the paper is studying an interesting and important problem.
Decision: Accept: poster