Paper: Designing Long-term Group Fair Policies in Dynamical Systems (ICLR-2024)

Reviewer comment (Novelty):
The main weakness of the paper is in the contribution. As claimed in the introduction, the drawback of using RL based methods for unknown dynamics is the requirement of large amount of training data. In this paper, however, it is assumed that the model is already learned. So the problem is changed to an optimization problem with known parameters. Therefore, it does not seem to be a fair comparison that this is an improvement over RL based methods.

Author response:
Dear Reviewer NkS5, 

Firstly, we would like to comment on the differences between our approach and reinforcement learning (RL), similar to our comment on Reviewer LjFL. While there are shared elements with RL, our methodology differs significantly in key aspects. As RL, our policy (agent) makes decisions within an environment, where the environment responds to these decisions (actions). However, in RL, the response of the environment is quantified by feedback in the form of rewards that accumulate over time and are used to iteratively retrain the policy during deployment. Instead, our approach involves solving an optimization problem (OP) to determine a single long-term policy prior to deployment that we keep fixed during deployment. Notably, we do not retrain the policy based on feedback from its actions. Instead, in solving the optimization problem before deployment, we leverage the Markov Convergence properties and incorporate the expected feedback from the environment to determine the optimal policy. Therefore our policy does not aim to optimize for cumulative reward over time as in RL; rather, it focuses solely on the reward obtained at the stationary distribution. This distinction sets our approach apart from traditional RL methodologies.

Second, we believe our approach complements prior work on RL. As opposed to prior work (e.g., in RL), our paper takes a structured approach by separating the estimation problem (of the Markov kernel i.e., the dynamics) from the policy learning process. We believe that this is a strength of our approach, as it allows us to find a single time-invariant policy. In the Appendix we show results, where we estimate label distribution and dynamics from temporal data, when labels are only partially observed (as common in the lending scenario), and solve our OP. For these experiments, we used (comparably to RL) a small number of samples to estimate the distributions. We compare our results of policies based on estimated dynamics and true dynamics. We do recognize that the dynamics estimation problem itself is a significant challenge and requires careful attention and, as commented in Section 8, is the subject of a different line of active research and thus outside the scope of this paper.

We see our main contribution in providing one method for finding long-term fair policies that differ from previously proposed RL methods in the aspects detailed above.

Reviewer comment (Novelty):
As cited in section 6, there are many existing works with different long-term fairness constraints. It is not clear what is the advantage of having a unified framework. For example, can this new framework solve the problem faster especially for more complex dynamics with large state space? Otherwise, the new model framework seems like a reformulation of existing models.

Author response:
We appreciate your expression of concern that our framework is a reformulation of existing models and would like to ask you to provide us with references that provide a similar contribution. 

Our simulations - similar to the ones from prior work [1, 2, 3, 4, 5] - provide a straightforward example as a proof of concept. We acknowledge that the application of our framework to address problems involving complex dynamics and large state spaces is beyond the current scope of our paper. Nevertheless, we recognize this as a crucial avenue for future research.

Reviewer comment (Experiments):
The description of the simulation is not very clear. For example, from the results, what is the target fairness state? Why would $P(Y=1|S=0)$ be different from $P(Y=1|S=1)$?

Author response:
Your second question is not entirely clear to us. Could you please provide additional clarification? We welcome suggestions on improving the description of the simulation. One of the policies we address is an EOP-fair utility maximization policy. In this context, we aim to find a policy that converges to a fair stationary state that is EOP-fair. From the set of policies converging to an EOP-fair state, our objective is to identify the one maximizing utility. The results presented in Figure 2(b) demonstrate that our long-EOP policy indeed converges to a fair state, with EOP close to zero, and its utility is comparable to that of a short-term unfair policy solely focused on maximizing utility. This indicates that our policy is effectively converging to the targeted fair state.

Reviewer comment (Novelty):
In terms of novelty, this paper appears to be closely related to (Zhang et al., 2020), as acknowledged in the paper. However, the differences between the two have not been adequately discussed. From my perspective, both the models and techniques applied in this paper seem quite similar to those in (Zhang et al., 2020).

Author response:
Dear Reviewer nac1, 

Thank you for your review. We are happy to provide you with additional clarifications on your raised questions.

Question 1) We are happy to elaborate on the conceptual and technical novelty of this paper, compared to (Zhang et al. 2020). There are several differences, mainly i) in the modeling assumptions, ii) in the goal of the paper, and iii) in the results shown.

Firstly, there is a disparity in their assumptions regarding the generative approach, and this divergence significantly affects their analytical capabilities. Zhang et al. (2020) operate under the assumption of a Markov Kernel over a binary target variable $Y$. Their model posits an anti-causal relationship, where the target variable $Y$ influences the features $X$ ($Y \to X$). In contrast, our approach assumes a Markov Kernel over a categorical feature space $X$ and we assume a causal relationship where features $X$ influence the target variable $Y$ ($X \to Y$).

We base our modeling assumptions on several considerations. First, when working with the FICO loans dataset, we argue that it is more plausible for features like income and financial assets, which are aggregated into a risk score $X$, to be the causal factors influencing an individual's repayment behavior $Y. This is in contrast to the notion that individuals possess hidden information about their ability to repay $Y$, causing them to have a specific risk score $X$ that summarizes their financial information. This directional causal modeling aligns with prior research [1, 2, 3]. Secondly, we posit that it is more plausible for feedback to induce changes in the feature $X$ rather than altering the ability to repay $Y$, thus we assume that the Markov Chain is defined over $X$.

To the best of our knowledge, the analysis conducted by Zhang et al. (2020) is not easily adaptable to a causal data generative scenario $X \to Y$ with a Markov Chain defined over $X$. If one were to examine short-term policy updates for a Markov kernel defined over $X$ while assuming a causal relationship $X \to Y$, this would result in a time-varying Markov kernel. This introduces significant challenges in assessing long-term effects, as the conventional Markov Convergence theorem is formulated for time-invariant Markov Chains.

Second, the goals of our papers are fundamentally different. Zhang et al. (2020) focus on assessing the influence of fairness constraints on the disparity of qualification rates within the context of short-term policies. Instead, we propose a framework to learn a policy a long-term fair policy that takes the underlying dynamics into account. The goal of our paper is thus fundamentally different.

Third, Zhang et al. (2020) do have a short section 6, where they explore alternative interventions that can effectively improve qualification rates at the equilibrium and promote equality across different groups. This section appears closest to us. Zhang et al. (2020) show that sub-optimal short-term fair policies instead of the optimal ones can improve overall qualification in the long run, that under certain conditions on transitions, there always exist threshold policies leading to equitable equilibriums, and that instead of performing a policy intervention, one could also alter the value of transitions.

We propose a framework to explicitly learn a policy long-term fair policy that induces a time-invariant Markov Kernel and consequently imposes the necessary conditions from the Markov Convergence theorem on the policy.

Reviewer comment (Experiments):
This paper only considers one dataset, which raises questions about the generalizability of the framework to other datasets. For instance, (Zhang et al., 2020) considered two real-world datasets: the FICO dataset (same as in this paper) and the COMPAS dataset (which is not addressed in this paper). It would be much more beneficial to explore the performance of the model on multiple datasets to assess its applicability more comprehensively.

Author response:
Question 2) Regarding the choice of the dataset, we acknowledge that our current experiments focus on a single simulation setup, specifically centered around loan repayment. While conducting experiments across a broader range of environments may be interesting, focusing on a single generative model like Zhang et al. (2020) and a single guiding example is in line with prior published work [2, 3, 4] with the loan example used widely by previous work on long-term fairness [1, 2, 3, 4, 5]. Similarly, in our experiments, we vary dynamics and initial distributions, essentially simulating different datasets of the same generative model. Note also that we provide an example of how the framework can be applied to a different generative model in the Appendix. 
We hope that our responses have adequately addressed your questions. We appreciate you raising your score, if our response has alleviated your concerns. Kindly let us know if there is anything else we can clarify. 

Best regards

[1] D’Amour et al. Fairness is not static.

[2]  Liu et al. Delayed impact of fair machine learning. 

[3] Creager et al. Causal modeling for fairness in dynamical systems. 

[4] Wen et al. Algorithms for fairness in sequential decision making.

[5] Yu et al. Policy Optimization with Advantage Regularization for Long-Term Fairness in Decision Systems.

Reviewer comment (Novelty):
What is the conceptual and technical novelty of this paper, compared to (Zhang et al. 2020)?

Author response:
Dear Reviewer nac1, 

Thank you for your review. We are happy to provide you with additional clarifications on your raised questions.

Question 1) We are happy to elaborate on the conceptual and technical novelty of this paper, compared to (Zhang et al. 2020). There are several differences, mainly i) in the modeling assumptions, ii) in the goal of the paper, and iii) in the results shown.

Firstly, there is a disparity in their assumptions regarding the generative approach, and this divergence significantly affects their analytical capabilities. Zhang et al. (2020) operate under the assumption of a Markov Kernel over a binary target variable $Y$. Their model posits an anti-causal relationship, where the target variable $Y$ influences the features $X$ ($Y \to X$). In contrast, our approach assumes a Markov Kernel over a categorical feature space $X$ and we assume a causal relationship where features $X$ influence the target variable $Y$ ($X \to Y$).

We base our modeling assumptions on several considerations. First, when working with the FICO loans dataset, we argue that it is more plausible for features like income and financial assets, which are aggregated into a risk score $X$, to be the causal factors influencing an individual's repayment behavior $Y. This is in contrast to the notion that individuals possess hidden information about their ability to repay $Y$, causing them to have a specific risk score $X$ that summarizes their financial information. This directional causal modeling aligns with prior research [1, 2, 3]. Secondly, we posit that it is more plausible for feedback to induce changes in the feature $X$ rather than altering the ability to repay $Y$, thus we assume that the Markov Chain is defined over $X$.

To the best of our knowledge, the analysis conducted by Zhang et al. (2020) is not easily adaptable to a causal data generative scenario $X \to Y$ with a Markov Chain defined over $X$. If one were to examine short-term policy updates for a Markov kernel defined over $X$ while assuming a causal relationship $X \to Y$, this would result in a time-varying Markov kernel. This introduces significant challenges in assessing long-term effects, as the conventional Markov Convergence theorem is formulated for time-invariant Markov Chains.

Second, the goals of our papers are fundamentally different. Zhang et al. (2020) focus on assessing the influence of fairness constraints on the disparity of qualification rates within the context of short-term policies. Instead, we propose a framework to learn a policy a long-term fair policy that takes the underlying dynamics into account. The goal of our paper is thus fundamentally different.

Third, Zhang et al. (2020) do have a short section 6, where they explore alternative interventions that can effectively improve qualification rates at the equilibrium and promote equality across different groups. This section appears closest to us. Zhang et al. (2020) show that sub-optimal short-term fair policies instead of the optimal ones can improve overall qualification in the long run, that under certain conditions on transitions, there always exist threshold policies leading to equitable equilibriums, and that instead of performing a policy intervention, one could also alter the value of transitions.

We propose a framework to explicitly learn a policy long-term fair policy that induces a time-invariant Markov Kernel and consequently imposes the necessary conditions from the Markov Convergence theorem on the policy.

Reviewer comment (Experiments):
What is the reason for considering only one dataset?

Author response:
Question 2) Regarding the choice of the dataset, we acknowledge that our current experiments focus on a single simulation setup, specifically centered around loan repayment. While conducting experiments across a broader range of environments may be interesting, focusing on a single generative model like Zhang et al. (2020) and a single guiding example is in line with prior published work [2, 3, 4] with the loan example used widely by previous work on long-term fairness [1, 2, 3, 4, 5]. Similarly, in our experiments, we vary dynamics and initial distributions, essentially simulating different datasets of the same generative model. Note also that we provide an example of how the framework can be applied to a different generative model in the Appendix. 
We hope that our responses have adequately addressed your questions. We appreciate you raising your score, if our response has alleviated your concerns. Kindly let us know if there is anything else we can clarify. 

Best regards

[1] D’Amour et al. Fairness is not static.

[2]  Liu et al. Delayed impact of fair machine learning. 

[3] Creager et al. Causal modeling for fairness in dynamical systems. 

[4] Wen et al. Algorithms for fairness in sequential decision making.

[5] Yu et al. Policy Optimization with Advantage Regularization for Long-Term Fairness in Decision Systems.