Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper looks at how to guarantee fairness when the impact of the decisions we make today is only seen in the long term. The paper provides a counter-factual approach to predict future impact from historical data that may be coming from a different policy than the one we aim to deploy, and aims to find a policy that does not have an unfair long-term impact across populations. 
strength_and_weaknesses: Strengths:
- the topic of the paper is important. I.e., how do we guarantee fairness across different individuals or groups, especially when we cannot observe labels about the individuals we make decisions on and update our model immediately?
- the approach is technically interesting: it is counterfactual in flavor and relies on learning from historical data. This data comes from previously deployed policies that may have been unfair, and the observed outcomes are informed by the previously deployed policy itself. 
- the idea of moving away from having to model a specific analytical relationship between a classifier's prediction and its observed impact is nice

Weaknesses:

I have the following concerns about the paper:

i) Exposition and scope-wise, I am not sure whether there is anything in the paper that is specific to delayed impact and but not to counterfactual learning. Here, unless I missed something about the paper, it seems that the only way the delayed impact really comes in is that compared to a standard ML/fairness model, the observed label is defined as being, say, 2 years after the decision has been made. But the time dimension here does not actually play a role in the model:
- the dataset itself is static, coming from a single previously deployed policy. 
- The problem is, in fact, formulated as a static problem of counterfactual learning given historical i.i.d. data. There isn't a real time dimension/aspect in which the model is updated over time and more data is collected over time. 

In turn, it seems to be more of a traditional counterfactual learning problem where the delayed impact part feels "gimmicky" and does not impact the paper beyond its motivation. 

ii) Assumption 1 seems to essentially say that we look at models whose predictions do not depend on group identity. This is a "sensitive attribute-blind" approach which contradicts much of what is known in the space of fairness. I.e., the same features may mean different things for different populations, and not taking the sensitive attribute into account is a surefire way to promote unfairness. This seems like a bit of a naive assumption which says that we can take the lessons from one population and apply it to another directly.

iii) I think the assumptions that the historical data is well-behaved (i.i.d and having observations of the full support) may be a bit hard to believe. What I think may be hard and interesting about time-dynamics in the space of fairness is that the deployed rule influences the observed data, and the observed data influences the deployed rule, often leading to feedback loop with a long-term negative impact. 

As an example: if we only historically gave loans to wealthy white people, a naive model will learn to still only give loans to wealthy white people, and we will only collect repayment data from wealthy white people. In this case, we do not have access to i.i.d., full support data that is representative of the entire population, and have a missing data problem. It is then impossible (without collecting more data) to predict the outcomes on a different population that we have observed (almost) nothing about in the past. 



clarity,_quality,_novelty_and_reproducibility: The writing of the paper is clear, the results are proved, and experiments are provided to back up the theory. 
summary_of_the_review: I think the topic of the paper is interesting. However, I think I feel that much of what makes delayed-impact fairness hard in practice is in the details and specific assumptions made about the problem (non i.i.d. data and data from a single population that cannot be interpolated to a different population), and I think the assumptions of the current work are a bit naive. As such, I think the paper is a bit under the bar in its current form
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper investigates the fair classification problem in the long term. It introduces a new algorithm, ELF, to solve the fair classification problem as both a classification and a reinforcement learning problem — classification for optimizing the main objective (a measure of classification loss) and reinforcement learning when taking DI into account. 
strength_and_weaknesses: + The setting is interesting where the relationships between the delayed impact and prediction are unknown. A reinforcement learning solution is developed to infer the delayed impact from the observation using importance sampling.
+ Given the inferred delayed impact, a classification is built to ensure DI.
+ The theoretical analysis seems convincing.

- The variable of delayed impact observation and the formulation of long-term fairness is unclear. Eq 1 defines g(theta) for DI but it is not clear why a fair algorithm must ensure Eq 1. Usually, fairness means the absence of disparity between two groups. But variable I is an observation, e.g., the rate. It is unclear why the larger I corresponds to better DI. In brief, DI and DI fairness is not well defined. The confusion makes Sec 2 hard to understand.
- The setting of no analytic relationship between DI and prediction is controversial. Assumption 2 assumes the relationship. Otherwise, the off-policy learning cannot infer the delayed impact. In addition, it is unclear how the performance of importance sampling affects the downstream classification.
clarity,_quality,_novelty_and_reproducibility: The problem formulation is not very clear.

summary_of_the_review: This paper presents a solution to classification with long-term fairness constraints. A new setting is considered where the relationship between the delayed impact and prediction is unknown. To handle this challenge, the paper introduces a few assumptions and infers the delayed impact from prediction using a reinforcement learning method. Then the inferred DI is used for classification.

The method seems meaningful but the formulation is not clear. The statements about DI and fairness are confusing. It is not clear why Eq 2<0 is necessary for model fairness. 

Assumption 2 is contrary to the setting of the analytic relationship between DI and prediction.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper studies the delayed impact of fairness in machine learning and provides a new model that models this effect as opposed to the commonly studied static notions of fairness. The paper introduces an algorithm called ELF (Enforcing Long-term Fairness) to provide high confidence delayed impact fairness guarantees (when possible). The authors demonstrate the effectiveness of ELF empirically.

strength_and_weaknesses: ------------------------------
Strengths
------------------------------
-- The delayed impact of fairness in machine learning has not received much attention in the community (mainly due to the hardness of modeling this impact) so in that sense, it is great to see papers on this topic.

--The paper is pretty well written and it is clear to follow the high-level ideas of the paper.

------------------------------
Weaknesses
------------------------------
--Assumption 3 is strong. While it is common in the RL community to have this assumption, this is less realistic in the algorithmic fairness literature and a possible source of bias. How can this assumption be justified?

-- The theoretical analysis of the paper is rather straightforward. 

-- The experimental analysis is not convincing. 
(a) None of the baselines considers the delayed impact into consideration. Is it surprising to see that ELF will have a much lower failure rate compared to the baselines? 
(b) How did you come up with the form of delayed impact in Equation (4)? In general, having access to such data is a major challenge in the practicality of this work.
(c) The number of data points needed for ELF so that it outputs a solution is very large. Does this problem become less severe when looking at lower-confidence solutions? What is the dependency on the confidence parameter? 
(d) The accuracy of the ELF is significantly lower than the baselines. 


clarity,_quality,_novelty_and_reproducibility: The paper is clearly written. The setting is novel and I think at least the theoretical results of the paper should be easily reproducible. 
summary_of_the_review: While I think the paper tackles an interesting problem, it is hard to justify the proposed modeling and practicality of the results.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The delayed impact of fairness is an important and understudied problem.
However, the assumptions, empirical evaluation, and some technical details have been brought into question. No reviewer was championing the paper.
The author response is very detailed and thorough. However, given the importance of this type of work, the meta-reviewer believes it is necessary to incorporate these various clarifications into a revision of the paper for another round of reviewing.
justification_for_why_not_higher_score: Unanimously slightly below the bar, no champion.
Decision: Reject