Paper: Equal Long-term Benefit Rate: Adapting Static Fairness Notions to Sequential Decision Making (ICLR-2024)

Reviewer comment (Novelty):
The major concern is that the contribution of the paper feels relatively marginal besides the major contribution being the newly proposed long-term fairness notion.

Author response:
It seems that our technical contributions of the bias mitigation method, ELBERT-PO, are not fully clear to the reviewer. We clarify below.

1. Our technical framework relies on Reinforcement Learning (RL). However, the mathematical form of our proposed bias metric was not studied by RL. This is because RL generally studies how to optimize the cumulative reward, which is very distinct from our proposed bias metric.
2. In order to decrease bias, we need to compute the policy gradient of the bias, which was not obvious from RL literature. We have to mathematically derive the policy gradient of bias and figure out how to integrate it with popular RL methods. This should be regarded as one of our major contributions.

Reviewer comment (Theory):
The proposed method involves a lot of relaxations/approximations, without enough rigorous theoretical justification, as evident from the questions raised below.

Author response:
It is incorrect that our ELBERT-PO method involves `"a lot of"` approximations without `"enough rigorous theoretical justification"`. 

Our method only has two approximations:
1. It uses a regularization-based objective (Equation 2) to solve a hard-constrained objective (Equation 1). This is indeed an approximation and is also a very common practice, which we clarify in Q1.
2. In multi-group settings, we propose to use the "soft bias" instead of bias to avoid non-differentiability issue, which is theoretically justified by Proposition 3.3. This is also common and important in machine learning, when the evaluation metrics in non-differentiable but you need a differentiable training objective. For example, in tradition classification problem we use accuracy for evaluation and use cross-entropy, the differentiable surrogate of accuracy, for training.

Reviewer comment (Novelty):
Consequently, the work appears to lean more towards a heuristic application of conventional MDP techniques rather than introducing a genuinely innovative concept.

Author response:
It is incorrect that our ELBERT-PO method involves `"a lot of"` approximations without `"enough rigorous theoretical justification"`. 

Our method only has two approximations:
1. It uses a regularization-based objective (Equation 2) to solve a hard-constrained objective (Equation 1). This is indeed an approximation and is also a very common practice, which we clarify in Q1.
2. In multi-group settings, we propose to use the "soft bias" instead of bias to avoid non-differentiability issue, which is theoretically justified by Proposition 3.3. This is also common and important in machine learning, when the evaluation metrics in non-differentiable but you need a differentiable training objective. For example, in tradition classification problem we use accuracy for evaluation and use cross-entropy, the differentiable surrogate of accuracy, for training.

Reviewer comment (Theory):
How does solving (2) equivalent to solving (1)? I assume it's related to the Lagrangian form? Since equation (1) is the primary objective function of the paper, the authors should consider adding a detailed justification of the equivalency between the two equations.

Author response:
Here we adopt the penalty method [R1], using a regularization-based objective Eq (2) to address the hard-constrained problem Eq (1). This is because in practice the hard-constrained problem can be too restrictive and might lead to infeasibility in the solution space (e.g. no policy is under the bias threshold $\epsilon$). Also, using a unconstrained regularization-based objective facilitates the gradient-based optimization.

In the regularization-based objective Eq (2), the relative importance of reward and bias is controlled by the bias coefficient $\alpha$. Indeed, we observe in the experiment that increasing $\alpha$ typically reduces the bias. Our method achieves the lowest bias among all baseline and is thus very effective.

We also would like to mention that although unconstrained, the objective in Eq (2) is not easy to solve: its gradient is not directly given by traditional Reinforcement Learning. It is our contribution to compute its gradient in Eq (4).


Ref:
[R1] Nocedal, Jorge, and Stephen J. Wright, eds. Numerical optimization. New York, NY: Springer New York, 1999.

Reviewer comment (Writing):
What is the "Bellman Equation"? It might be unclear to a non-expert in MDP, but it seems like an important concept.

Author response:
Bellman Equation is a foundational equation for Reinforcement Learning. We have added appropriate reference in Section 3.1. The following is a brief summary of Bellman Equation and we refer the reviewer to [R2] for further details.

In Reinforcement Learning (RL) problems, it is important to consider the value functions. For example, the state value function in the following characterizes the cumulative reward of a certain state following a policy $\pi$.
$$V^\pi(s)=\mathbb{E}\_\pi\left[\sum\_{t=0}^\infty\gamma^t R(s\_t,a\_t)\Big|\pi,s\_0=s\right].$$

It turns out value functions satisfy the Bellman Equation 

$$V^\pi(s)=\sum_a\pi(a|s)\sum_{s'}T(s'|s,a)[R(s,a)+\gamma V^\pi(s')]$$

Commonly used advantaged-based policy optimization methods (e.g. PPO [R3]) rely on Bellman Equation in order to compute the policy gradient of the cumulative reward.

Ref
[R2] Sutton, Richard S., and Andrew G. Barto. Reinforcement learning: An introduction. MIT press, 2018.
[R3] Schulman, John, et al. "Proximal policy optimization algorithms." arXiv preprint arXiv:1707.06347 (2017).

Reviewer comment (Theory):
Continuing the above question, it's unclear how proposition 3.1 helps overcome the difficulty of being unable to compute $b(\pi)$ directly.

Author response:
It was previously unclear how to compute the gradient of $b(\pi)$ from RL literature. Proposition 3.1 shows how to compute the gradient of $b(\pi)$ (and $J(\pi)$) using $\nabla_{\pi}\eta_{g}^D$ and $\nabla_{\pi}\eta_{g}^S$, which we do know how to compute from RL literature. We provide more details below.

1. It was unclear how to directly compute the gradient of $b(\pi)=\left(\frac{\eta_1^S(\pi)}{\eta_1^D(\pi)}-\frac{\eta_2^S(\pi)}{\eta_2^D(\pi)}\right)^2$ since it is not in the form of cumulative sum (e.g. $\sum_{t=0}^\infty\gamma^t R(s_t,a_t)$). We only know how to compute gradient of cumulative sum from RL literature.
2. $\eta_{g}^D$ and $\eta_{g}^S$ are indeed in the form of cumulative sum (see Definition 2.2) so we know how to compute $\nabla_{\pi}\eta_{g}^D$ and $\nabla_{\pi}\eta_{g}^S$.
3. Proposition 3.1 computes the previously unknown $b(\pi)$ using $\nabla_{\pi}\eta_{g}^D$ and $\nabla_{\pi}\eta_{g}^S$ that are known in RL literature.

Reviewer comment (Writing):
I have several difficulties to parse the example provided in the introduction. First, why is the proposed medical treatment a fair decision-making problem? Given the framing with individuals seeking medical care as "demand" and a medical facility providing healthcare as "supply", this appears to me a fair allocation/ranking problem, which is - to my undestanding - different from the general decision making, in having an upper bound on the (usually) positive decision. Second, what is the motivation for a decision-maker to aim for demographic parity (equal acceptance rates) in a medical treatment scenario, i.e., why is it the goal to provide the same proportion of service to blue and red patients? The motivation of the proposed fairness notion lives from the observation that it would be “unfair” to provide nobody of the group of 100 red people with a treatment at time step t, just because we do not provide the 1 blue person at time step t with treatment. I am struggeling to think of a real-world scenario, where a decision-maker would take such decisions, thus I have difficulties to follow the motivation for the example and thus the proposed fairness notion. As indnicated in [S2], it would be easier for me to see an application of this in a hiring scenario.

Author response:
> W1.1: ... the example provided in the introduction. First, why is the proposed medical treatment a fair decision-making problem? Given the framing with individuals seeking medical care as "demand" and a medical facility providing healthcare as "supply", this appears to me a fair allocation/ranking problem, which is - to my undestanding - different from the general decision making, in having an upper bound on the (usually) positive decision.

The medical treatment allocation problem is a fair decision-making problem, where the decision maker maximizes the reward while ensuring fairness. 

1. This example features as one of the experimental environments, detailed in Case 2, Section 5.1.
2. Excluding fairness considerations, this problem represents a typical sequential decision-making scenario where the objective is to maximize the reward, defined as the overall percentage of a healthy population over time.
3. With fairness in mind, the decision-maker aims to minimize bias, adhering to the principle of demographic parity. The notions of "group demand" and "group supply" are instrumental in bias measurement, viewed from individual perspectives, as fairness is evaluated at this level.
4. In our initial writing, we prioritized brevity and focused solely on fairness in the introduction, inadvertently omitting the reward aspect. This omission led to some confusion.

To eliminate this ambiguity and reaffirm that our sequential decision-making framework encompasses both reward maximization and bias minimization, we have revised the introduction. It now features a Bank Lending example, which may be more relatable. The revisions include: (1) Defining the problem as a sequential decision-making process where the bank seeks to maximize long-term profits (the reward), and (2) acknowledging the bank's consideration of bias. (3) Additionally, the main figure in the introduction now explicitly represents both the reward and fairness-related elements.


> W1.2: Second, what is the motivation for a decision-maker to aim for demographic parity (equal acceptance rates) in a medical treatment scenario, i.e., why is it the goal to provide the same proportion of service to blue and red patients? 

It is incorrect that the only goal of the decision-maker is to provide the same proportion of service. As clarified above, the decision-maker has its own reward to maximize (such as the percentage of healthy individuals over time), while trying to achieve demographic parity. 

> W1.3: The motivation of the proposed fairness notion lives from the observation that it would be “unfair” to provide nobody of the group of 100 red people with a treatment at time step t, just because we do not provide the 1 blue person at time step t with treatment. 

This is not our motivation. Our motivation is: in order to **equally** treat any red person at $t$ (with 100 red people) and the red person at $t+1$ (with 1 red people), we should not treat both timesteps equally. 

We refer the reviewer to our response to W3 for more detailed explanations.

> W1.4: I am struggeling to think of a real-world scenario, where a decision-maker would take such decisions, thus I have difficulties to follow the motivation for the example and thus the proposed fairness notion. As indnicated in [S2], it would be easier for me to see an application of this in a hiring scenario.

All the scenarios (lending, medical treatment allocation, attention allocation) that we considered in the paper, as well as the hiring example mentioned by the reviewer, are in fact the examples where our proposed notion are more preferable than prior notions. This is because in all these scenarios, individuals from the same group arriving at different timesteps should have the same impact on the bias computation (just like in static fairness notions every individual from the same group has the same impact on static fairnss metrics). **Our proposed metric satisfies this whereas prior notions do not**. We refer the reviewer to our response to W3 for more detailed explanations.

Reviewer comment (Novelty):
If I understood the example and fairness notion correctly, the demographic parity notion from the introduction is only expected to yield a different result than static one-step fairness notions, when i) demographic group sizes vary across time steps and ii) if a) decisions are trivial (i.e., either “accept all” or “accept nobody”) or b) we have one group with a group size 1. Here is a scenario, where i) is fulfilled, but ii) not: When we have 20 non-binary individuals and 100 binary individuals at t, then 80 non-binary individuals and 10 binary individuals at t+1 and fulfill DP with an acceptance rate p \in (0, 1) at each time step (such that DP-unfairness is zero), then the proposed long term notion yields the same as the single time step notion, e.g., p_t = 0.5, p_{t+1} = 0.25, then |(10+40)/(20+80) – (50+5) / (100+10)| = |50/100 – 55/110| = |0|. To my understanding from the example, the notion is only useful in a very specific case, which limits its practicality. I am happy to be corrected by the authors, if I misunderstood.

Author response:
> W2: If I understood the example and fairness notion correctly, the demographic parity notion from the introduction is only expected to yield a different result than static one-step fairness notions, when i) demographic group sizes vary across time steps and ii) if a) decisions are trivial (i.e., either “accept all” or “accept nobody”) or b) we have one group with a group size 1. Here is a scenario, where i) is fulfilled, but ii) not: When we have 20 non-binary individuals and 100 binary individuals at t, then 80 non-binary individuals and 10 binary individuals at t+1 and fulfill DP with an acceptance rate p \in (0, 1) at each time step (such that DP-unfairness is zero), then the proposed long term notion yields the same as the single time step notion, e.g., p_t = 0.5, p_{t+1} = 0.25, then |(10+40)/(20+80) – (50+5) / (100+10)| = |50/100 – 55/110| = |0|. To my understanding from the example, the notion is only useful in a very specific case, which limits its practicality. I am happy to be corrected by the authors, if I misunderstood.

The computation provided by the reviewer is not correct. The provided example in fact shows that our proposed metric yields **different** results from prior notions most of the time, contrary to the reviewer's understanding.

First, It seems that there is a mistake in your example: $p_{t+1} = 0.5$, not 0.25. This is because $\frac{40}{80}$ = $\frac{5}{10}$ = 0.5. 

Second, this is actually a good example so let's do some calculation using $p_t$ and $p_{t+1}$. 
1. In the provided example, the step-wise biases are zero for both time steps. So the question is whether our proposed notion also yields zero bias.
2. In our proposed notion: $|(20p_t+80p_{t+1})/(20+80) – (100p_t+10p_{t+1}) / (100+10)| = \frac{78}{110}|p_t - p_{t+1}|$. Therefore, our notion computes zero bias if and only if $p_t = p_{t+1}$ in your example. For example, it will compute zero bias when $p_t = p_{t+1} = 0.5$. However, in general $p_t$ and $p_{t+1}$ are different and thus our notion will yield different results than prior notions.

Therefore, the correct conclusion is: the proposed notion is expected to yield a different result from static one-step notions in general. They yield the same results only in very specific cases (e.g. $p_t = p_{t+1}$ in the provided example).

Reviewer comment (Theory):
I find calling some time steps more “important” than others misleading. The authors call a time step “more important” for a demographic group, if there exist more of its members that are seeking a decision at that time step. This means, larger groups are paid more importance to than smaller groups. First, this is an ethical understanding that should be at least mentioned, if not discussed. Second, I find this understanding of fairness especially concerning, if the variance in the group sizes of individuals that arrive at each time step is biased or confounded by another factor. For a given demographic group (e.g., blue ones), individuals who tend to arrive in smaller groups may be treated unfairly in comparison to individuals, who tend to arrive in larger groups. This is because the fairness notion aggregates the blue individuals over all time steps to one large blue group and demands the cummulative decisions across all time steps to be fair to the cummulated blue groups. So this may yield to a type of inter-group unfairness.

Author response:
This is a **fundamental misinterpretation** of our proposed notion. The reviewer claimed that in our proposed notion, `"larger groups are paid more importance to than smaller groups"` and `"individuals who tend to arrive in smaller groups may be treated unfairly in comparison to individuals, who tend to arrive in larger groups"`, which is incorrect.

To clarify
1. Under our notion (first aggregating supply and demand over time, then computing the ratio), within a group, individuals from different timesteps have the **same** impact on bias computation. To see this, consider the red group in the motivating example: at time $t$ the group size is 100, and at $t+1$ the group size is 1. For any individual from either time $t$ or time $t+1$, the effect of the decision, changing from "rejection" to "acceptance" will change the Long-term Benefit Rate by $\frac{1}{101}$, which is the same for both $t$ and $t+1$. 
2. Under prior notions (first computing step-wise ratio, then aggregating), within a group, individuals from timesteps with larger group size have smaller impact on bias computation, causing discrimination. To see this, consider the red group in the motivating example again: At time $t$, this group has 100 members, and at $t+1$, it reduces to just one member. Under these conditions, a decision shift from "rejection" to "acceptance" for an individual at time $t$ alters the step-wise ratio marginally, by $\frac{1}{100}$. In stark contrast, the same decision change for an individual at $t+1$ significantly affects the step-wise ratio, by $\frac{1}{1}$, which is considerably more impactful than the change at $t$. Consequently, when aiming to enhance the overall well-being of the red group, the individual at the smaller group size of 1 at $t+1$ becomes disproportionately influential, as decisions regarding this individual have a greater effect. This leads to a discriminatory outcome where individuals at time $t$, due to their minimal impact on fairness measures, are effectively marginalized.

Therefore, prior notions do not equally treat individuals within the same group from different timesteps (with different number of people). The reviewer finds this concerning by saying `" I find this understanding of fairness especially concerning, if the variance in the group sizes of individuals that arrive at each time step is biased or confounded by another factor."` We are concerned too. **But this downside belongs to prior notions, not our proposed ELBERT**.

Finally, it seems that there is confusion between the following two concepts: within a group:
1. When defining the bias, a timestep with higher group demand (number of individuals from that group) should not be treated the same as a timestep with lower group demand. This is indeed our claim. If one treats all timesteps the same, then individuals from different timesteps will not be treated equally.
2. An individual arriving at a timestep with larger number of people, will be treated as differently than the one from a timestep with lower number of people. As explained above, this is NOT true for our notion and is a downside of prior notions.

Reviewer comment (Presentation):
The authors claim on page 2: “Without considering the variation in temporal importance within a group, these prior metrics lead to a false sense of fairness.” I strongly object the attempt to claim that prior fairness notions are “false”, which the authors do several times on page 2. Different fairness notions capture different understandings of fairness and which one to pick is very dependent on the context and scenario, and importantly should be at the heart of different research fields, such as philosophy, ethics, sociology etc. I advocate to remove these strong claims, and additionally merge the related work section with the introduction or put it right after it to contextualize the claims made as well as the motivate the need for a "new" fairness notion better.

Author response:
> W4: The authors claim on page 2: “Without considering the variation in temporal importance within a group, these prior metrics lead to a false sense of fairness.” I strongly object the attempt to claim that prior fairness notions are “false”, which the authors do several times on page 2. Different fairness notions capture different understandings of fairness and which one to pick is very dependent on the context and scenario, and importantly should be at the heart of different research fields, such as philosophy, ethics, sociology etc. I advocate to remove these strong claims....

First we encourage the reviewer to carefully read our response to W3.

It is a misunderstanding that our paper `"attempts to claim that prior fairness notions are “false”"`. We agree that different fairness notions capture different understandings of fairness. However we did identify how prior notions can lead to a false sense of fairness: they can cause discrimination since individuals arriving at different timesteps have distinct impact on the bias metrics. This is concerning in many real-world applications (e.g. lending, medical treatment allocation, attention allocation and hiring example provided by the reviewer).

Reviewer comment (Writing):
In the introduction, the authors keep writing that they measure “bias”, but to my understanding, what they are measuring is not a (statistical, sampling, cofounding etc.) bias, but their moral/ethical understanding of “unfairness”.

Author response:
> W5: In the introduction, the authors keep writing that they measure “bias”, but to my understanding, what they are measuring is not a (statistical, sampling, cofounding etc.) bias, but their moral/ethical understanding of “unfairness”. 

This is a misunderstanding of our terminology. The "bias" in our introduction is in fact a statistical concept (difference in the long-term benefit rate). 

Also, it is not true that our definition just come from our `moral/ethical understanding of “unfairness”`. In fact, long-term fairness metrics of all environments used in our experimental section were proposed and have been used in prior works, e.g. [1,2,3]. These metrics actually fall under ELBERT's general framework instead of prior notions (e.g. naively summing of step-wise biases). This should serve as a very strong motivation for using our proposed ELBERT.



Ref:
[1] D’Amour, Alexander, et al. “Fairness is not static: deeper understanding of long term fairness via simulation studies.” Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 2020.
[2] Yu, Eric Yang, et al. “Policy Optimization with Advantage Regularization for Long-Term Fairness in Decision Systems.” Neurips 2022.
[3] Atwood, James, et al. "Fair treatment allocations in social networks." arXiv preprint arXiv:1911.05489 (2019).

Reviewer comment (Theory):
In the conclusion the authors write “However, in real world applications, extra constraints on demand might be needed. For example, the demand should not be too large (e.g. when demand is the number of infected individuals) or too small (e.g. when demand is the number of qualified applicants).” I am failing to follow this argumentation. “Demand” in how the users understand it, is the number of individuals a decision-maker is deciding upon at each time step (e.g., loan applicants). I am failing to see, how and why these extra constraints might be needed.

Author response:
> W6: In the conclusion the authors write “...extra constraints on demand might be needed. For example, the demand should not be too large (e.g. when demand is the number of infected individuals) or too small (e.g. when demand is the number of qualified applicants).” ...“Demand” in how the users understand it, is the number of individuals a decision-maker is deciding upon at each time step (e.g., loan applicants). I am failing to see, how and why these extra constraints might be needed.

In the conclusion we said that extra constraints on **group demand** might be needed. Here is an example:

In medical treatment allocation, where the demand is defined as the number of individuals arriving at hospital. If the demand is very low, it could mean people are healthy so they do not go to hospital. Or it could mean people are not willing to go to hospital, which is potentially a bad thing. Depending on the definition of demand and the real world situation, extra constraints on group demand might be needed.

Reviewer comment (Presentation):
In times of increasing research on large language models, I would suggest the authors to pick a different name for their fairness notion than “ELBERT”, given the similarity to all the variations of BERT models.

Author response:
> (Minor) In times of increasing research on large language models, I would suggest the authors to pick a different name for their fairness notion than “ELBERT”, given the similarity to all the variations of BERT models.

Thanks for pointing out the similarity to BERT models! We will consider changing a different name for the notion.

Reviewer comment (Theory):
The authors write on page 3 “In many static fairness notions, the formulation of the group well-being can be unified as the ratio between supply and demand.“ I have difficulties to understand, where this understanding of demand and supply in fairness notions comes from. There is no citation given. To my understanding, at a classical sequential decision making scenario, there is at each time step a set of individuals that are required to be made a decision upon, and the decision maker takes the features of the individuals as input and decides usually aiming to maximize their utility while potentially adhering to some fairness notions. The concept of supply and demand comes from economics, where the general standard assumption is that goods demanded/supplied are uniform, i.e., every individual item is the same. This is not the case in fair decision making; every individual is different. There is not a demand and supply of credits and as long as a decision maker can supply credits they will. Instead, they will aim to give the credit to the individuals that they expect to have a high probability of paying back. Could the authors please clarify this?

Author response:
> Question: The authors write on page 3 “...the formulation of the group well-being can be unified as the ratio between supply and demand.“ I have difficulties to understand, where this understanding of demand and supply in fairness notions comes from. There is no citation given. To my understanding, at a classical sequential decision making scenario... and the decision maker takes the features of the individuals as input and decides usually aiming to maximize their utility while potentially adhering to some fairness notions. The concept of supply and demand comes from economics, where the general standard assumption is that goods demanded/supplied are uniform, i.e., every individual item is the same. This is not the case in fair decision making; every individual is different. There is not a demand and supply of credits and as long as a decision maker can supply credits they will. Instead, they will aim to give the credit to the individuals that they expect to have a high probability of paying back. Could the authors please clarify this?

1. (Ratio, supply & demand) Since many popular static fairness notions (such as demographic parity, equal opportunity, etc) involve comparing ratios between groups (such as acceptance rate), we adopt the same mathemtical form of the ratio. We did cite these static notions in our paper. We name "group demand" and "group supply" ourselves for lack of better terms. In static/sequential fair decision making, although every individual is different, they are treated equally when computing the fairness metrics. Therefore, this terminology is still consistent with the ones in economics.
2. `Instead, they will aim to give the credit to the individuals that they expect to have a high probability of paying back.`

    That is right! In the lending example, the bank aims to maximize profits so they will try to give the credit to the individuals that they expect to have a high probability of paying back. At the same time, the bank also needs to ensure fairness. Our proposed ELBERT-PO method helps maximize reward while reducing the bias.

Reviewer comment (Theory):
It is unclear why the supplies are modeled in a group-specific way from the user (red/blue group) perspective, instead of from the supplier (hospital) perspective.

Author response:
Your understanding is correct and is consistent with our full descriptions of this medical treatment allocation problem in Section 5.1, Case 2. Specifically,

* Decision-maker: the hospital.
* State space of the MDP includes (1) the health condition of the whole population (which determines the group **demand**); (2) The condition of the hospital (for example, the number of medicine available at that time step).
* Action space of the MDP includes whether to allocate medicine to each individuals, which determines group **supply**. 

The supply is not included in the state space and is determined by actions. Note that technically supply is not the same as action, but is determined by action: action is whether the hospital allocates treatment to each individual, while supply is the sum of all allocated treatments. 

Now that we have defined the MDP (we are done with modeling), the long-term benefit rate (for fairness evaluation) is defined to be the ratio between cumulative supply and demand, which is related to both state and actions.

Reviewer comment (Novelty):
While the reviewer understands the authors' emphasis on the fact that the ratio is calculated after aggregation, the reviewer does not think the proposed modeling is more general than aggregating the supply-demand ratios along the temporal axis. The reviewer believes those two modeling choices are complimentary, each suitable for specific practical scenarios.

Author response:
It is incorrect that prior notions are suitable for `"pratical scenarios"` considered in our paper (including lending, medical treatment allocation and attention allocation). In fact, prior notions have a concerning downside that: by first computing the ratio and ignoring temporal importance difference, they discriminate individuals from timesteps with higher group demand, which we clarify below. 

Under prior notions (first computing step-wise ratio, then aggregating), within a group, individuals from timesteps with larger group size have smaller impact on bias computation, causing discrimination. To see this, consider the red group in the motivating example again: at time $t$ the group size is 100, and at $t+1$ the group size is 1. For any individual from time $t$, the effect of the decision, changing from "rejection" to "acceptance" will change the step-wise ratio by only $\frac{1}{100}$. However, the effect of decision on step-wise ratio for any individual from $t+1$ is $\frac{1}{1}$, much larger than $\frac{1}{100}$. Therefore, when improving the red group's overall well-being under prior notions, the individual from smaller group size of 1 at $t+1$ is more favorable since the decision on this individual has larger impact. Individuals from $t$ are discriminated due to smaller impact when measuring fairness. 

On the other hand, under our notion (first aggregating over time, then computing the ratio), within a group, individuals from different timesteps have the **same** impact on bias computation. To see this, consider the red group in the motivating example: at time $t$ the group size is 100, and at $t+1$ the group size is 1. For any individual from either time $t$ or time $t+1$, the effect of the decision, changing from "rejection" to "acceptance" will change the Long-term Benefit Rate by $\frac{1}{101}$, which is the same for both $t$ and $t+1$. 

Therefore, our proposed notion is more suitable than prior notions in practical scenarios.

Reviewer comment (Evaluation):
The proposed long-term fairness notion is not robust towards group size imbalance. If the group size of red is significantly larger than that of blue, to the extent that even for the most unimportant states of the red group the demand is much larger than the entire population of the blue group (and thus larger than the demand of the blue group even in its most important state), then in the calculated ratio, the supply/demand of the blue group may not alter the result because they are out-numbered. As a result, the proposed fairness evaluation may not provide sensible distinction between numerical variation and suffering of a minority group.

Author response:
It is incorrect that `"the proposed long-term fairness notion is not robust towards group size imbalance"`and `"the supply/demand of blue group does not alter the result simply because they are out-numbered"`. This is because computing the long-term benefit rate for a group, only involves taking the ratio between the cumulative group supply and demand **for this group**. The size of another group do not affect the computation of long-term benefit rate for this group. The bias is then computed as the differences of long-term benefit rate between different groups.

As an example, say the demand of red group is 100 at $t$ and 200 at $t+1$, where as the demand of blue group is 2 at $t$ and 1 at $t+1$. The supply of red group is 10 at $t$ and 20 at $t+1$, where as the supply of blue group is 1 at $t$ and 1 at $t+1$. Overall the blue group is much smaller. 

The group benefit rate of blue group is $\frac{1+1}{2+1}=\frac{2}{3}$, which is not affected by the fact that the group size (demand) of the red group is much larger than blue group.

In this example, the long-term benefit rate of red group is $\frac{10+20}{100+200}=\frac{1}{10}$. The bias is computed as the difference in benefit rates, which is $\frac{2}{3}-\frac{1}{10}$. In this case, the blue group with smaller population size is actually the advantaged group.

Reviewer comment (Novelty):
The proposed notion is claimed to address potential issues of previous utilization of static fairness in dynamic settings, specifically regarding the sequence of applying aggregation and ratio calculation. However, the discussion should include more than one (somewhat exaggerating) example to establish the (potential) advantage of ratio-after-aggregation compared to ratio-before-aggregation. The reviewer feels the two ways are complimentary, and the proposed notion is not well motivated enough.

Author response:
It is incorrect that `"these two ways are complimentary"` since prior notions have some serious downsides. We encourage the reviewer to carefully read our previous response to Q1.2.

Although we use a `"somewhat exaggerating example"` in our paper, the following fact is general: prior notions can lead to a false sense of fairness/unfairness when different timesteps have different group demand, which is true in sequential decision making. 

Also, we would like to mention that the long-term fairness metrics of all environments used in our experimental section were proposed and have been used in prior works, e.g. [1,2,3]. These metrics actually fall under ELBERT's general framework instead of prior notions (e.g. naively summing of step-wise biases). This should serve as a very strong motivation for using our proposed ELBERT.



Ref:
[1] D’Amour, Alexander, et al. “Fairness is not static: deeper understanding of long term fairness via simulation studies.” Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 2020.
[2] Yu, Eric Yang, et al. “Policy Optimization with Advantage Regularization for Long-Term Fairness in Decision Systems.” Neurips 2022.
[3] Atwood, James, et al. "Fair treatment allocations in social networks." arXiv preprint arXiv:1911.05489 (2019).

Reviewer comment (Theory):
If the importance corresponds closely to the absolute size of demand at a given time, it does not seem natural to model group-specific importance of states for the purpose of achieving long-term fairness. If the focus is on fairness of resource distribution, the implication of state importance on long-term fairness is unclear when the importance is only an intrinsic property of a certain state (e.g., when the base population is large).

Author response:
1. In the paper we demonstrate that, for a certain group, it is crucial to  consider the **relative** importance (demand) among timesteps, not the absolute size of demand. For example, even if `"the base population is large"`, the group demand still varies over time.
2. In sequential decision making, the current action has future effects. In particular, different actions will lead to different change in demand across timesteps. For example, some actions might result in increase in future demands. 
3. While it is true that ```importance is a property of a certain state```, it is also important to note that in sequential settings, different actions will lead to different state transition in the future.
4. The proposed fairness metrics depend on supply and demand at all timesteps. Therefore, in order to achieve long-term fairness, one needs to deal with time-changing supply and demand when choosing the actions.