Tier Balancing: Towards Dynamic Fairness over Underlying Causal Factors
Authors: Zeyu Tang
Yatong Chen
Yang Liu
Kun Zhang
Venue: ICLR 2023 poster (2023)
Keywords: Algorithmic Fairness
Causality
Dynamic Modeling
Long-term Fairness

Abstract: The pursuit of long-term fairness involves the interplay between decision-making and the underlying data generating process. In this paper, through causal modeling with a directed acyclic graph (DAG) on the decision-distribution interplay, we investigate the possibility of achieving long-term fairness from a dynamic perspective. We propose Tier Balancing, a technically more challenging but more natural notion to achieve in the context of long-term, dynamic fairness analysis. Different from previous fairness notions that are defined purely on observed variables, our notion goes one step further, capturing behind-the-scenes situation changes on the unobserved latent causal factors that directly carry out the influence from the current decision to the future data distribution. Under the specified dynamics, we prove that in general one cannot achieve the long-term fairness goal only through one-step interventions. Furthermore, in the effort of approaching long-term fairness, we consider the mission of "getting closer to" the long-term fairness goal and present possibility and impossibility results accordingly.

Published as a conference paper at ICLR 2023
TIER BALANCING: TOWARDS DYNAMIC FAIRNESS OVER UNDERLYING CAUSAL FACTORS
Zeyu Tang', Yatong Chen', Yang Liu", and Kun Zhang!.
Department of Philosophy. Carnegie Mellon University
2 :Comptter Science and Engineering Department, University of California, Santa Cruz
$ Maahhnee Learning Department, Mohamed bin Zayed University of Artificial Intelligence
ze y ange cmu. edu, ychen5 5echen9920uunn ucsc.. edu, yangliuuucse edu, kunz 10z21cmm @em0eeemm.mmmm edu
ABSTRACT
The pursuit of long-term fairness involves the interplay between decision-making and the underlying data generating process. In this paper, through causal modeling with directed acyclic graph (DAG) on the Decision-distribution interplay, we investigate the possibility of achieving long-term fairness from dynamic perspec- tive. We propose Tier Balancing. a technically more challenging but more natural notion achieve in the context of long-term, dynamic fairness analysis. Different from previous fairness notions that are defined purely on observed variables, our notion goes one step further, capturing behind-the-scene situation changes on the unobserved latent causal factors that directly carry out the influence from the current decision to the future data distribution. Under the specified dynamics, we prove that in general one cannot achieve the long-term fairness goal only through one-step interventions. Furthermore, in the effort of approaching long-term fair- ness, consider the mission of "getting closer to" the long-term fairness goal and present possibility and impossibility results accordingly.
INTRODUCTION
The long-term fairness endeavor inevitably involves the interplay between decision policies and the underlying data generating process: when deriving a decision-making system, one usually makes use of data at hand; when we deploy such a system, the decision would impact how data will look in the future (Perdomo et al., 2020; Liu et al., 2021). 1). To understand why and how a data distribution responds to decision-making strategies, the investigation has resort to causal modeling. The pursuit of long-term fairness, in turn, should also consider the changes in the underlying causal factors.
Various fairness notions with different flavors have been proposed in the literature: associative fairness notions that capture the correlation or dependence between variables, c.g., Demographic Parity (Calders al., 2009), Equalized Odds (Hardt et al., 2016); causal faimess notions that involve modeling causal relations between variables, e.g., Counterfactual Fairness (Kusner et al., 2017; Russell al., 2017), Path-Specific Counterfactual Fairness (Chiappa, 2019; Wu et al., 2019), Causal Multi-Level Fairness (Mhasawade & Chunara, 2021). The previously proposed fairness notions with respect a snapshot of the static reality, and do not have built-in capacity model the distribution-decisionn interplay in the long-term fairness pursuit.
In the effort of enforcing fairness in the dynamic setting, rescarchers have approached the problem from different angles: they provide causal modeling for fairness notions (Creager et al., 2020), analyze the delayed impact or downstream effect on utilities (Liu al., 2018: Heidari et al., 2019; Kannan et al., 2019; Nilforoshan et al., 2022), enforce fairness in sequential or online decision-making (Joseph et al., 2016; Liu et al., 2017: Hashimoto et al., 2018; Heidari & Krause, 2018; Bechavod et al., 2019), investigate the relation between the long-term population qualification and fair decisions (Zhang et al., 2020), take into consideration the user behavior/action when deriving a decision policy (Zhang et al., 2019; Ustun al., 2019; Miller et al., 2020; von Kügelgen et al., 2022), provide fairness transfcrability guarantee across domains (Schumann et al., 2019; Singh et al., 2021), or derive robust fair predictors (Coston et al., 2019; Rezaei et al., 2021). The proposed dynamic faimess enforcing

---

Published as a conference paper at ICLR 2023
procedures usually limit their scope of consideration to only observed variables, and the fairness audit is performed directly on the decision or statistics defined on observed data.
In order to have built-in capacity to capture the influence from the current decision to future data distributions, and more importantly, to induce a fair future in the long run, in this paper, we propose Tier Balancing, long-term fairness notion that characterizes the interplay between mocksion-making and data dynamics through a detailed causal modeling with a directed acyelic graph (DAG). For example, the latent socio-economic status (whose estimation can be the output of FICO credit score model), although not directly measurable, plays an important role in credit applications. We are motivated by the goal of inducing fair future by actually balancing the inherent aacio-economic status, i.e., the "tier", of agents from different groups. We summarize our contributions as follows:
We formulate Tier Balancing, a fairness notion from the dynamic and long-term perspective that characterizes the pecision-distributie interplay with a detailed causal modeling over both observed variables and latent causal factors.
Under the specified data dynamics, we prove that in general, one cannot directly achieve the long-term fairness goal only through a one-step intervention, i.e., static decision-making
We consider the possibility of getting closer to the long-term faimess goal through sequence of algorithmic interventions, and present possibility and impossibility results derived from the one-step analysis of the decision-distribution interplay.
2 PROBLEM SETUP
In this section, we present the formulation of the problem of interest. We first demonstrate in Section 2.1 detailed causal modeling of the interplay between decision-making and data dynamics. Then in Section 2.2, we formulate Tier Balancing, a long-term fairness notion that captures the decision-distributin interplay with the presented causal modeling.
2.1 CAUSAL MODELING OF DECISION-DISTRIRIU
INTERPLAY ON DAG
Let us denote the time step T with domain of value N+. At time step T. let denote the protected feature as Ar with domain of value A = {0, 1}. additional feature(s) as XT,, with domain of value X, the (unmeasured) underlying causal factor Hy (we call it "tier") with domain of value H (0, 1]. the (unobserved) ground truth label y, and the observed label Yz with domain of value y = {0, 1}. and the decision Dr with domain of value D {0, 1}. Figure shows the causal modeling of the interplay between decision-making and underlying data generating processes, which involves multiple dynamics (from T = t to T = + 1).
Underlying data dynamics (stationary components) Considering the fact that the underlying data dynamics are relatively stable with respect to the timescale of decision-making (e.g., the societal changes happen at a much larger time scale compared to a particular credit application decision), we assume that processes governing how (Y .x,,) are generated from (He, Ad) for each individual in the population are stationary and do not change over different T = f. We also assume that the underlying data generating process that governs how He+1 is updated from (Hr, y( Dt) across time steps is stationary, and so are the process governing the observation of y/lobs) given (D,, y(41)) and the process governing the update of A4+1 from A,,
2+1
The tier H, fully captures the individual's key property that is directly relevant to the scenario of interest, and therefore the cause of y!(em) and X,, 's instead of the other way around. For example, the improvement in the socio-economic status can be reflected through an increase in income, while manipulating one's income only by changing the recorded number does not affect the actual ability to repay the loan. The determination of causal direction aligns with causal modelings in previous literature (see, e.g., Zhang et al. 2020).
Decision-making dynamics (non-stationary components) The institution (decision maker) as- signs decision D, to each individual according to the observed features (A,, Xx..) and the outcome
Due the space limit, provide additional discussions on decision-distributiion interplay in Appendix B.I.
2

---

Published as a conference paper at ICLR 2023
T=t-1
T
T=t+1
A-1
H
H4yy
A411
yrlors 2-1
(ari)
X1,1
X1.2
X1.3
(orky ++1
Bc-1
(obe)
Figure 1: The causal modeling of the that the corresponding variable (underlying factor) is unobservable.
interplay. The circle (diamond) indicates
record Y/abs) The interpretation of the aforementioned variables depends the problem hand. For instance, in the credit application scenario where Dr denotes the application decision (approval or denial), we can interpret the latent tier H, the underlying socio-cconomic status of an individual. Since the decision- mrriiinnnmmkingggg strategy can vary across different time steps, we explicitly introduce the underlying factor 0, (e.g.. a hyperparameter or an auxiliary variable) to indicate such (possible) non-stationary property of decision-makinn
The causal path from 6, to 6z+1 indicates the similarity between decision- decision-making making strategies as time goes by (e.g., the continuing interest on utility), although strategies themselves are not necessarily identical across different time steps.
We interpret the variable y(rr)) as "whether or not one would repay the loan were he/she approved the
credit at T t- 1 (which might not be the case in reality)". The variable y/(oha) is observed only this individual actually got approved at T = t - i.e., D = 1. We distinguish between the underlying ground truth y: and the observed record (((obs) because of their different roles in the decision-distribution interplay. On the one hand, only y((hu) is observed and therefore accessible to the decision maker (e.g., for training and evaluation). On the other, since the potential outcome Y reflects individual's 's inherent characteristic, which is not directly relevant to whether it is observable, yd(ar) is utilized when the underlying data generating process specifies the update from H; to H4+1-
2.2 THE NOTION OF TIER BALANCING
Definition 2.1 (K-Step Tier Balancing). Under the specified dynamics, starting from any time step T and a given K 2 0, let us denote a sequence of K decision- decision-making strategies as Dr.T+K- (Dr.... DT+ (an empty set if K = 0), and the latest hidden tier after K K-step decision-makin as llTTK. We say Dr.T*K-I satisfies K K-Step Tier Balancing, if at T + K the following condition holds true (where "Il" denotes statistical independence):
( HTtK 1L Ar+k, where H/T is updated from (Hr,, XTT+ Dr.+K-1).
(I)
Equation captures the statistical consequence in the future (in the form of an associative relationship) induced by the interplay between the underlying data dynamics and decision-male maaknng policies along the way. The causal modeling is essential in capturing our long-term fairness goal, since the attainment of Tier Balancing is induced outcome of sequence of K decision-making strategies Dr.r+k-1 (which are indispensable although the fairness notion itself is not explicitly defined on decisions).
Our Tier Balancing notion of algorithmic fairesss is distinguished from previously proposed faimess notions in several important ways.2 2 To begin with, Tier Balancing has a built-in dynamic flavor, whose definition involves variables that span across multiple time steps. Therefore the audit of Tier Balancing inevitably requires long-term and dynamic analysis, which is very different from previously proposed (both associative and causal) faimess notions defined with respect to a static snapshot of reality (e.g., Calders et al., 2009; Hardt et al., 2016; Kusner et al., 2017: Chiappa, 2019).
Besides, considering the fact that the in the hidden causal factors. Tier Balancing extends the scope of faimess consideration beyond only observed variables to hidden causal factors, which makes our notion a technically more challenging but more natural long-term faimess goal to achieve. The endeavor to explore the possibility of defining fairness in terms of latent causal factors is not an unrealistic fantasy. Recent advances in causal discovery literature have established identifiability results (under certain assumptions) on Due the space limit, we provide detailed discussions related works in Appendix
interplay often involves situation changes
3

---

Published as a conference paper at ICLR 2023
causal structures among latent variables (Xie al., 2020; Adams et al., 2021; Kivva et al., 2021; Xie et al., 2022), which provide not only a theoretical justification, but also an indication of the potential, for our effort in exploring long-term fairness endeavor through modeling latent causal factors.
Furthermore, although Tier Balancing is not directly defined in terms the decisions themselves, Teer Balancing is characterized with detailed causal modeling that involves both recisionnmaaing and data dynamics. The explicit causal modeling of the secision-distribution interplay offers both challenges and opportunities for more principled fairness inquiries in the long-term, dynamic context (Hu & Chen, 2018; 8; Liu et al., 2018; Mouzannar et al., 2019; Heidari et al., 2019; Zhang et al.. 2020).
In Definition 2.1, we specify the step K which Tier Balancing is evaluated. If K Hr IL Ar happens to be attained initially (although in general it may not be the case), Tier Balancing attained the beginning." When HT Ar and K 2 I, KSSpp Tier Balancing achieved with respect to the underlying causal factor HT+K-
i.e.,
3 CHARACTERIZING TIER BALANCING
In Section 2, we propose a detailed causal modeling of the interplay between decision-making and underlying data generating processes, based on which we formulate novel long-term fairness notion Tier Balancing. Our model is applicable to wide range of resource allocation scenarios, e.g., hiring practice (Hu & Chen, 2018; Kannan al., 2019), credit application (Liu et al., 2018), predictive policing (Ensign et al., 2018; Elzayn et al., 2019).
For the clarity of discussion, in this section we consider running example of credit application where agents in fixed population repeatedly apply for credit. We first demonstrate how one can apply the proposed causal modeling in the credit application scenario in Section 3.1. Then in Section 3.2, we characterize the Single-step Tier Imbalance Reduction (STIR) term for the purpose of conducting one-step analysis on the Tier Balancing notion long-term fairness.
3.1 MODELING DETAIL OF DECISION-DISTRIUTTIN DISTRIBUTION INTERPLAY
As shown in Figure the unmeasured latent causal factor He (the hidden socio-economic status) is the actual root cause of the ground truth label y! as well the (possibly) observed label y.(oh)) (the repayment record). For any given tier H, = h4 let us assume that the unobserved ground truth y(on)) is sampled from Bernoulli distribution with h, as the success probability, and that the observed repayment record y((abs) depends on both the ground truth label y( and the previously received decision Dz-1:
Dr =, where y/(0n) ~ Bernoulli(hh). is undefined if D, 0,
Y(or)
(((bb))
(2)
From Equation we can see that ( is a masked copy of y( (masked by Dz-1), and we have the following proposition capturing the property of the marginal distribution of y(obs);
Proposition 3.1. At time step T = for any H. h4 (0, 1]. under the specified dynamics, among the population where ground truth actually observable, i.e.. Y/(abs) is not undefined, we have:
ylobs) Bernoulli(hh).
Proposition 3.1 captures the fact that among the population where repayment record Y((obs) is observed,
the marginal distributions of y((os) and y((om) are actually identical. This property indicates that although one does not have access the unobserved tier, i.e., the socio-cconomic status H, one can still use the observed (obos) as a bridge to infer its behavior.
Fact 3.2. Let be the domain of value for the protected feature Ae, and be the domain of value for all other exogenous noise terms Fz. For each time step T we can represent D,, y((r)) and H, via functions o gb yrum) and ft respectively, where o A - {0, 1), ob youm AA x E + 1), and ft Ax (0, 11. i..., D, of (A,, E.), ((a)) ot (A,, E.), and H₃ fe E.).
In Appendix B.2, we analyze the scenario in which Tier Balancing initially satisfied.
4

---

Published as a conference paper at ICLR 2023
Notice that Fact 3.2 represents variables D,, Y! and H, with functions of all root causes (including the protected feature A, and exogenous noise terms Eg) in the system without explicitly specifying the respective functional forms, which may depend on further assumptions the joint distribution and the time step 7 = 1.* Fact 3.2 a direct result of representing causal relations with a functional causal model (FCM) (Spirtes et al., 1993; Pearl, 2009), and is for the purpose of notational convenience in later analysis."
Assumption 3.3 (Multiplicatiiv update of underlying tier). Let a'D ay E [0, 2 be the parameters that capture the influences from current decision D, and ground truth y, to next step:
H++1 min (1, H2 an -) + ay (2Y, -
-2 1)]}-
Assumption 3.3 states that Hz+1 treats H; as a baseline, with increase or decrease in a multiplicative form based agent's received decision and repayment information. We inspired by the evolution theory where multiplicative updates have been a common modeling choice to capture updates in relevant statistics (Friedman & Sinervo, 2016; Dawkins Davis, 2017). The explicit dependency on the update parameters, aD and ay. related D₁ and Y/(r)) respectively, characterizes the two important aspects of our model: the update in individuals' tier potentially depends on the received decision D,, as well as the ground truth y/ari) (even if unobserved). 6 The condition ap, oy [0, 2) makes sure that H1+1 > 0, and the min f- operation makes sure that H&+1 is upper-capped by
In practical scenarios where agents repetitively apply for resource (e.g., in our running example of credit application) at each time step T I, with the entire group remains unchanged, we have:
Assumption 3.4. The protected feature time step T + is an identical copy of that at T
=T=t: A4+1 = A
44)
3.2 ONE-STEP ANALYSIS TOWARDS TIER BALANCING
In Definition 1 we established the long-term faimess goal. Considering the dynamic property of this fairness notion, apart from defining "what exactly is fair in the long run", in order bridge the cognitive gap also need to clarify the meaning of "getting closer to the long-term fairness goal", i.e., achieving the long-term fairness goal through sequence of algorithmic interventions. this section, we present the stepstep theoretical analysis framework and characterize the Single-step Tier Imbalance Reduction (STIR) term ASTIR [++1 for the purpose of investigating Tier Balancing.
3.2.1 SINGLE- STEP TIER IMBALANCE REDUCTION (STIR)
Recall that the long-term fairness objective is the independence between the protected feature Ar and the hidden tier Hy = [T(AT, Er) (at a time step T). Equivalently, we would like Hy to not be function of Ar. We view (0, ET) and fr,1,
two dependent random variables, and quantify the amount "getting closer to the long-term fairness goal" by comparing the absolute difference between (O, Er) and (1, Er) before (when T = t) and after (when T + 1) one-step update, and see the gap decreases. Since the individual-level exogenous noise term Ey is the input, this comparison of absolute differences is on the individual level. Therefore in order to quantitatively characterize the overall amount of "getting closer to the long-term fairness goal", we need to take into account different possible combinations of decision D, and outcome y((at) (when = f) for each individual, and aggregate the individual-level comparisons over the population.
Given combinations of D, and y-(or)) when E, e, we denote the conditional joint probability density of (0, Ex), f((1, E,)) as qu (fr e), fc(1,e) e) d, d', 3/) = qe (fe(0, fe(1, 9."
d, (1,e) = d',g" gt um)
(0, E) y. (m) (1,e) = 3/). and calculate the Single-step Tier Imbalance
Reduction (STIR) term from 7' = t to = t ++++ 1, denoted by ASTIRI [++ as following:"
*We implicitly adopt the assumption that the protected feature itself is not caused by other variables.
In Appendix B.3, we discuss the role exogenous terms E, Fact 3.2.
'In Assumption 3.3 we explicitly specify that we are considering y/ instead of yraos) At T t, every individual has binary ground truth Yr However, it might not be the case that everyone has an observable yllbs) (since y/abs) undefined for individual if its D--1 0).
5

---

Published as a conference paper at ICLR 2023
ASTIRk+* = BB[1 Bat1) - Se+1 Ex+1)1 Beit)) - B.) E+)]
JCEE { 2e ((( ddd',y..
Σ
JEEE
(5)
d,d' ddd.,."'(00.11
Ge(e, p. " :d,d', o'd,,, y. v/ an, av) adede,
where (p(e) = f((0,e) - fe(1, e). G. is a function whose value only relies on the information
available at time step T = t, and (. d', y,yy y²) is the joint distribution of (d, d', y/ 3²): Pr d' v'
(1. Ex) = y²). We can then characterize "getting closer to long-term fairness goal" via the inequality ASTIRl+** 0.
(o. (0. E.) d, oe (1, Ea) d' g: y (0, Ex) y. g. y
3.2.2 SIMPLIFICATION ASSUMPTIONS
From Equation 5 we can see that the calculation of ASTIR comparison for each individual 1 dynamics q: (fc(0,, e), fc(1,e) e) d, y, v.y'), and the joint probability for different combinations of decision and ground truth before one-step dynamics Ps d', V. v'). In order to quantitatively analyze the property of ASTIRI? 1++ it is essential that we have access to all three aforementioned quantities.
I requires knowledge about the gap (ex),, the conditional joint density before one-step
To begin with, we need know the instantiations of (p-1 (er41) given (e(()) under the specified dynamics. Luckily, as we have illustrated in Table 4, Table 5, and Table 6 of Appendix, under the specified dynamics we can list all possible cases of the term (e++1(e++1) given (p((e)).
Besides, we need additional knowledge on the conditional joint density (), | d, d', y, )²). For the purpose of better elaboration, we present two assumptions on the behavior of this conditional joint density qualitative assumption and a quantitative assumption:
Assumption 3.5 (Qualitative assumption). For any time step T t and any exogenous term Eg = c E E, let us denote y = g.-" (0, c) and y' = oo (1, c). The following inequalities hold: ()()(0,,)) > f((1,) y > v > (f) < ((( y v'): Pt ef) < f((1,e)
(6)
3¹) > (f(( f(( /
Assumption 3.6 (Quantitative assumption). On top of Assumption 3.5, let further assume that the conditional joint density (J((0, e), e,(1,)) d, d'' y, y.vl) satisfies the following condition:
yad
jup)
if fc < f((),)
ge (f+(00, (((1,)) d, v.,v
(lw)
if (0) > s((1,c)
Yad' us
where " + add' = 2,
(up)
(low) Vad' yn!
(up) < <ddd'n uy! when y < y'. and (law) (up) 'dd' >
"ad' my when y > v
Assumption 3.5 is rather mild, stating that for any given exogenous noise term (of an individual) E, = ct, whenever the ground truth (! favors certain demographic group, it is more likely that the underlying tier also favors the same group. Assumption 3.6 is just special case of Assumption 3.5, with quantitative characteristies built-in for technical purposes.
Lastly, we need to know the (behavior of) joint probability density (i d', .''..''... v/) for all combina- tions of (d, d', y. g¹). In fact, we shall see in Section 4, when taking into consideration certain characteristics of the predictor, the joint probability Pt d', v') would follow some patterns that can simplify the analysis.
4 ONE-STEP ANALYSIS TOWARDS THE LONG-TERM FAIRNESS GOAL
In this section, we consider the possibility of attaining the long-term fairness goal with (a sequence of) one-step interventions. We first present a negative result that in general one cannot hope to achieve the long-term fairness goal through a single one-step intervention, i.e., static decision-making In light of this result, we further investigate the possibility getting closer to the long-term fairness goal with a sequence of one-step interventions, and present possibility and impossibility results accordingly.
"The details of the derivation can he found Appendix B.4.
In Appendix B.5, we present illustrative figures demonstrate the connection between qualitative and quantitative assumptions.

---

Published as a conference paper at ICLR 2023
4.1 THE GENERAL IMPOSSIBILITY OF ACHIEVING 1 1-STEP TIER BALANCING
In the following theorem, we prove that it is in general impossible to achieve Hr+1 JL A when initially H₂ M. At, i.e., in general we cannot achieve Tier Balancing with a single one-step intervention. Theorem 4.1. Let us consider the general situation where both Dj and Y are dependent with As Le., Dr V. A, Y(oni Ar- Then under Fact 3.2, Assumption 3.3, and Assumption 3.4, well the specified dynamics, when H₁ H As, only if at least one of the following conditions holds true for all e, E & can we possibly attain H441 JL At+1:
fel1.ex) (1) The ratio f.(0.c.) has a specific domain of value fe(0.ct) = itanday'
liepday.
fr(1.es)
(2) Positive (negative) label is exclusive to the advantaged (disadvantaged) group, and everyone receives a positive decision (if an > oyy);
(3) Negative (positive) labels only appear the advantaged (disadvantaged) group, and every- one receives a positive decision (if a > oty);
(4) Everyone has a positive label, but the positive decision is exclusive to the advantaged group
(if ap < ayy);
(5) Everyone has positive label, but the positive decision is exclusive to the disadvantaged group (if o& < o.y).
Theorem 4.1 lists all possible ways to directly achieve the long-term fairness goal (under the specified
condition). As we can that all of these conditions are rather restrictive: Condition (1) imposes strong conditions on the functional form of f()). In particular, when fo et), fa(1, e.) are both continuous random variables with non-zero density everywhere a continuous random variable (because the density is simply an integral over positive multiplications). We can see that the event specified in Condition (1) is a zero-measure one. Conditions (2-5) all require trivial decision- mecisinn-making policies. Therefore, in general one cannot directly achieve the long-term fairness goal. We need to consider the possibility of approaching the goal step-by-step.
the support (0, 1]. the ratio is still
4.2 POSSIBILITY OF GETTING CLOSER TO TIER BALANCING VIA ONE-STEP INTERVENTIONS
In Section 4.1. 1. we have seen that under the specified dynamics, single one-step intervention is in general not enough order to achieve long-term fairness. this section, we investigate if certain strategy can get closer to the long-term fairness goal through a sequence of algorithmic interventions. we follow the same principle derive the decision-making policy from the data at each time step, one-step analysis suffices for the purpose of studying the interplay between decision and distribution.
4.2.1 ONE-STEP ANALYSIS ON PERFECT PREDICTOR
In this section, we consider the perfect predictor, where the predicted output equals to the underlying ground truth, i.e., Dj = y<(or) The output of the perfect predictor D, is fully specified by the value of ground truth y and therefore is conditionally independent from the protected feature A, given y((a)) Based on the definition of Equalized Odds (Hardt et al., 2016), the prefect predictor is also the best possible Equalized Odds predictor (at time step t) with an accuracy of 100%.
Furthermore, the joint probability Po d', y, y') for perfect predictor is not always positive for any possible combination of (d, d', y, s/) We have the following sufficient condition for this joint probability to be zero:
d d y, or d' ty' = Pt(d, d', v.y/) With the help of this additional knowledge on the joint probability Pr d', y. v/) we can quanti- tatively analyze the Single-step Tier Imbalance Reduction (STIR) term ASTink [+1 and present the following impossibility result for the perfect predictor:
(8)
Theorem 4.2. Let us consider the general situation where both D, and y( are dependent with Ao. ie., D, V. A., Y(or)) H. Ar. Under Fact 3.2, Assumption 3.3, and Assumption 3.4, and Assumption 3.6. as well as the specified dynamics, when He J.. At, the perfect predictor does not have the potential to get closer to the long-term fairness goul after one-step intervention, i.e.,
D2 Y(orr)
(Perfact Predictor) (++1 0.
ASTIR
Compared to Theorem 4.1 that one in general cannot directly attain the long-term fairness goal (balanced tier) through one-step intervention, Theorem 4.2 provides further insights regarding
7

---

Published as a conference paper at ICLR 2023
0.5
7-t
T-
T=1
P-1+
H
(a) the latent causal factor H is sampled from Sancated-Gaussing distributions
(b) the latent causal factor H is sampled from Uniform distributions
Figure 2: Illustration of the interplay between decision with perfect predictors and data dynamics (20 steps) on simulated data, with different initialization of tier H,
what mission is possible through repetitive one-step interventions. In particular, under relatively mild assumptions, Theorem 4.2 establishes the impossibility even getting closer to the long-term fairness goal through one-step interventions with the perfect predictor.
4.2.2 ONE-STEP ANALYSIS ON COUNTERFACTUAL FAIR PREDICTOR
In this section, we consider the Counterfactual Fair (Kusner et al., 2017) predictor. Similar to the one-step analysis on perfect predictors, need make of the characteristic of Counterfactual Fair predictors to simplify the quantitative analysis on the term ASTIR [t+]
The definition of Counterfactual Fairness requires the predictor satisfy P E2) (1, within each time step T = t. Therefore, we have the following sufficient condition for the joint probability (d d', y: 3/) to zero: d' Pild,''y.. y.
(1, E,)
(10)
Theorem 4.3, Let us consider the general situation where both Dr and Y( are dependent with A.. Le., D. A,, Y/ori Ar. Let us further assume that the data dynamics satisfies an E (0, 3), ay = Then under Fact 3.2, Assumption 3.3, Assumption 3.4, and Assumption 3.6, as well as the specified dynamics, when He J. Ab, it is possible for the Counterfactual Fair predictor to get closer to the long-term fairness goal after one-step intervention, if certain properties of the data dynamics and the predictor behavior are satisfied simultaneously, i.e.,
9t (0, = g. (1, P. (1.1,)))
27
E (( P(111.)+PPLL11)) P(000.001))+(1000.1.. aD aDE
P,(0.0,0.1 P.
- 1, ay 0
A (Counterfactial Fair) [+1
STIR
< 0. (11)
Theorem 4.3 demonstrates the possibility (not guarantee) of getting closer to the long-term fairness goal through one-step interventions (under certain conditions) with Counterfactual Fair predictors. Compared the general impossibility results for perfect predictors (Theorem 4.2), there are additional requirements (on both data dynamics and (, d', y, 3²)) accompanying the possibility result for Counterfactual Fair predictors. That being said, Theorem 4.3 clearly illustrates that the understanding of data dynamics through a detailed causal modeling, combined with suitable decision strategy, can provide us with a promising way to approach the long-term dynamic fairness goal (K-Step Step Tier Balancing with K 1), step step.
5 EXPERIMENTS
In this section, we present experimental results on both simulated and real-world FICO data set from Board of Governors of the Federal Reserve System (US) (2007).9 In the sequence of decision- distribution interplay, the latent causal factor H.r is updated according to the specified dynamics (Equation 3) each time step. The output of the decision policy (at each time step) depends on the specific scenario. In particular, we consider perfect predictors and Counterfactual Fair (Kusner et al., 2017, Level implementation) predictors.
5.1 DECISION WITH PERFECT PREDICTORS ON SIMULATED DATA
In Figure 2 we present the 20-step interplay between decision and the underlying data generating process on the simulated data. The distributions of H, for different groups are initialized with
OOu code repository available Githuh: https: /tpp::////iie httpt/tttbbb.../o com/ wwwm://ytttgg///eeeee.e laneii ancing.
8

---

Published as a conference paper at ICLR 2023
ACC
0.75
T=
7 T- -+
0.50
2
0
10
20
Density
0.4
A
1.0
Black
0.25
PAID
0.5
0.2
Asian
Ispanic
0.0
White
10
20
10
0.0
0.5
1.0
0.0
0.5
1.0
(a) the trend of the changing tier I/ at different step
(b) the uccuracy (top) and upproval rute (bottom) for different groups
(c) the distribution of the underlying hidden tier Hf among groups at time step 7 - (left) and 7 r 20 (right)
Figure 3: Illustration of the interplay between decision with Counterfactual Fair predictors and the data dynamies (20 steps) on the credit score data set. Panel (a) and (b) present the step-by- step tracks of update in tier, accuracy, and approval rates for different groups; panel (c) presents group-conditioned distributions of tier before (left) and after (right) 20 steps of interventions. The legend is shared across panel (a), (b), and (c).
truncated Gaussian distributions and Uniform distributions, respectively. During each time step T = t we generate ground truth labels y according to data dynamics specified in Section 3.1 and set the decision Dr to be equal to the ground truth y! (perfect predictor as the decision-making policy); then the pair of (Y/(o) D2) are utilized by the data dynamics determine the tier HG+1 for next step. As we can see from the left-hand-side figures in panels (a) and (b), the gap between tier for different groups is enlarged the time goes by. This indicates that interventions through decision with perfect predictors did not get closer to the long-term fairness goal.
5.2 DECISION WITH COUNTERFACTUAL FAIR PREDICTORS ON CREDIT SCORE DATA
The FICO credit score data set contains 301,53 1,36 records of TransUnion credit score from 2003 (Board of Governors of the Federal Reserve System (US), 2007). In the preprocessed credit score data set (Hardt et al., 2016), we convert the cumulative distribution function (CDF) of TransRisk score among different groups into group-wise density distributions the credit score, and use them as the initial tier distributions for different groups.
In Figure 3
we present the summary of a 20-step interplay between decision with Counterfactual Fair predictors and the underlying data generating process the credit score data set. The Counterfactual Fair decision-making strategy retrained after each one-step data dynamics. From Figure 3(a) we can observe that the gap between step-by-step tracks of tiers for different groups actually decreases before increasing again (around step 12). This indicates that decision with Counterfactual Fair predictors does have the potential to get closer the long-term faimesss goal, if the data dynamics and the initial condition (for each one-step analysis at different time step) satisfy certain properties. Figure 3(a) also highlight the importance of our dynamic fairness analyzing framework: if one does model the interplay between decision and data dynamics, one may well deviate from long-term fairness goal even after making some progress by getting closer to the fairness objective.
6 CONCLUSION AND FUTURE WORK
In this paper, we propose Tier Balancing, dynamic fairness notion that characterizes the decision- distribution interplay through a detailed causal model over both observed variables and underlying causal factors. We characterize Tier Bulancing in terms of one-step analysis framework on Single- step Tier Imbalance Reductioo (STIR). We show that in general one cannot directly achieve the long-term faimess goal only through one-step intervention, i.c., static decision-mans makinn. We further show that under certain conditions it is possible (but not guaranteed) for one to get closer to the long-term faimess goal with (a sequence of) Counterfactual Fair decisions.
Our results highlight the challenges and opportunities of enforcing a faimess notion that has built- interplay over underlying causal factors. Future works naturally include developing algorithms to effectively and efficiently enforce the Tier Balancing notion of long-term, dynamic fairnsss for various practical scenarios.
in capacity to model

---

Published as a conference paper at ICLR 2023
ETHICS STATEMENT
The motivation of our work is to pursue long-term fairness. The conduct of research under full awareness of, and with adherence to, ICLR Code of Ethics. We focus on the decision-distribtiion interplay and present detailed causal modeling on both observed variables and latent causal factors. We present challenges and opportunities offered by this new type of fairness endeavor, and hope our work can inspire further research to promote fairness in the long run.
ACKNOWLEDGEMENS
This project was partially supported by the National Institutes of Health (NIH) under Contract R01HL by the NSF-Convergence Accelerator Track-D award #2134901, by a grant from Apple Inc., grant from KDDI Research Inc., and generous gifts from Salesforce Inc., Microsoft Research, and Amazon Research. YC and YL are partially supported by the National Science Foundation (NSF) under grants IIS-2143895 and IIS-2040800, and CCF-2023495.
REFERENCES
Jeffrey Adams, Niels Hansen, and Kun Zhang. Identification of partially observed linear causal models: Graphical conditions for the non- gaussian and heterogeneous cases. Advances in Neural Information Processing Systems, 34, 2021.
Yahav Bechavod, Katrina Ligett, Aaron Roth, Bo Waggoner, and Steven Z Wu. Equal opportunity in online classification with partial feedback. Advances in Neural Information Processing Systems, 32, 2019.
Board of Governors of the Federal Reserve System (US). Report to the congress on credit scoring and its effects on the availability and affordability of credit. Board of Governors the Federal Reserve System, 2007.
Toon Calders, Faisal Kamiran, and Mykola Pechenizkiy. Building classifiers with independency constraints. In 2009 IEEE International Conference on Data Mining Workshops, pp. 13-18. IERE, 2009.
Silvia Chiappa. Path-specific counterfactual fairness. In Proceedings of the AAAI Conference Artificial Intelligence, volume 33, pp. 7801-7808, 2019.
Amanda Coston, Karthikeyan Natesan Ramamurthy, Dennis Wei, Kush R. Varshney, Skyler Speak- man, Zairah Mustahsan, and Supriyo Chakraborty. Fair transfer learning with missing protected attributes. In Proceedings of the 2019 AAAI/ACM Conference on Al, Ethics, and Society, pp. 91-98, 2019.
Elliot Creager, David Madras, Toniann Pitassi, and Richard Zemel. Causal modeling for fairness in dynamical systems. In International Conference on Machine Learning, pp. 2185-2195, PMLR, 2020.
Richard Dawkins and Nicola Davis. The selfish gene. Macat Library, 2017.
Hadi Elzayn, Shahin Jabbari, Christopher Jung, Michael Kearns, Seth Necl, Aaron Roth, and Zachary Schutzman. Fair algorithms for learning in allocation problems. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pp. 170-179, 2019.
Danielle Ensign. Sorelle Friedler, Scott Neville, Carlos Scheidegger, and Suresh Venkatasubrama- nian. Runaway feedback loops in predictive policing. In Conference on Fairness, Accountability and Transparency, pp. 160-171. PMLR, 2018.
Daniel Friedman and Barry Sinervo. Evolutionary games in natural, social, and virtual worlds. Oxford University Press, 2016.
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. In Advances in Neural Information Processing Systems, pp. 3315-3323, 2016.
10

---

Published as a conference paper at ICLR 2023
Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang, Faimesss without demographics in repeated loss minimization. In International Conference on Machine Learning. pp. 1929-1938. PMLR, 2018.
Hoda Heidari and Andreas Krause. Preventing disparate treatment in sequential decision making. In IJCAI, pp. 2248-2254, 2018.
Hoda Heidari, Vedant Nanda, and Krishma Gummadi. On the long-term impact of algorithmic decision policies: Effort unfairness and feature segregation through social learning. 36th International Conference on Machine Learning, pp. 2692-2701, 2019.
Lily Hu and Yiling Chen. A short-term intervention for long-term faimess in the labor market. In Proceedings of the 2018 World Wide Web Conference, pp. 1389-1398, 2018.
Matthew Joseph. Michael Keamn,, Jamie H Morgenstemn and Aaron Roth. Faimess in learning:
Classic and contextual bandits. Advances in Neural Information Processing Systems, 29, 2016.
Sampath Kannan, Aaron Roth, and Juba Ziani. Downstream effects of affirmative action. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pp. 240-248, 2019.
Bohdan Kivva, Goutham Rajendran, Pradeep Ravikumar, and Bryon Aragam. Learning latent causal graphs via mixture oracles. Advances in Neural Information Processing Systems, 34, 2021.
Matt J Kusner, Joshua Loftus Chris Russell, and Ricardo Silva. Counterfactual fairness. In Advances in Neural Information Processing Systems, pp. 4066-4076, 2017.
Lydia Liu, Sarah Dean, Esther Rolf, Max Simehowitz, and Moritz Hardt. Delayed impact of fair machine learning. In International Conference Machine Learning, pp. 3150-3158. PMLR, 2018.
Yang Liu, Goran Radanovic, Christos Dimitrakakis, Debmalya Mandal, and David C Parkes. Cali- brated fairness in bandits. arXiv preprint arXiv: 2017.
Yang Liu, Yatong Chen, Zeyu Tang, and Kun Zhang. Model transferability with responsive decision subjects. arXiv preprint
10.000900011..00111100 2021.
Vishwali Mhasawade and Rumi Chunara. Causal multi-level fairness. Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pp. 784-794, 2021.
John Miller, Smitha Milli, and Moritz Hardt, Strategic classification is causal modeling in disguise.
In International Conference on Machine Learning, pp. 6917-6926. PMLR, 2020.
Hussein Mouzannar, Mesrob Ohannessian, and Nathan Srebro. From fair decision making to social equality. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pp. 359-368, 2019.
Hamed Nilforoshan, Johann D Gaebler, Ravi Shroff, and Sharad Goel, Causal conceptions of faimness and their consequences. In International Conference on Machine Learning, pp. 16848-16888 PMLR, 2022.
Judea Pearl. Causality. Cambridge University Press, 2009.
Juan Perdomo, Tijana Zmncc, Celestine Mendler-Düinner, and Moritz Hardt. Performative prediction. In International Conference on Machine Learning, pp. 7599-7609. PMLR, 2020.
Ashkan Rezaei, Anqi Liu, Omid Memarrast, and Brian D Ziebart. Robust fairness under covariate shift. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 9419-9427, 2021.
Chris Russell, Matt J Kusner, Joshua Loftus, and Ricardo Silva. When worlds collide: integrating different counterfactual assumptions in fairness. In Advances in Neural Information Processing Systems, pp. 6414-6423, 2017.
Candice Schumann, Xuezhi Wang, Alex Beutel, Jilin Chen, Hai Qian, and Ed H Chi. Transfer of machine learning faimess across domains. arXiv preprint arXiv: 100.1900.9999 09688, 2019.
11

---

Published as a conference paper at ICLR 2023
Harvineet Singh, Rina Singh, Vishwali Mhasawade, and Rumi Chunara. Faimess violations and mitigation under covariate shift. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pp. 3-13, 2021.
Peter Spirtes, Clark N Glymour, and Richard Scheines. Causation, prediction, and search. Springer New York, 1993.
Berk Ustun, Alexander Spangher, and Yang Liu. Actionable recourse in linear classification. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pp. 10-19, 2019.
Julius von Kügelgen, Schölkopf. On the fairesss of causal algorithmic recourse. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 3584-9594, 2022.
Karimi, Umang Bhatt, Isabel Valera, Adrian Weller, and Bernhard
Yongkai Wu, Lu Zhang, Xintao Wu, and Hanghang Tong. Pe-fairness: A unified framework for measuring causality-basee fairness. In Advances Neural Information Processing Systems, pp. 3399-3409, 2019.
Feng Xie, Ruichu Cai, Biwei Huang, Clark Glymour, Zhifeng Hao, and Kun Zhang. Generalized independent noise condition for estimating latent variable causal graphs. Advances in Neural Information Processing Systems, 33: 13841914499,, 2020.
Feng Xie, Biwei Huang, Zhengming Chen, Yangbo He, Zhi Geng, and Kun Zhang. Identification of linear non-gaussian latent hierarchical structure. In International Conference on Machine Learning, pp. 24370-24387 PMLR, 2022.
Xueru Zhang, Mohammadmahdi Khaliligarekami, Cem Tekin, and Mingyan Liu. Group retention when using machine learning in sequential decision making: the interplay between user dynamics and fairness. Advances Neural Information Processing Systems, 32:15269-152788 2019.
Xueru Zhang, Ruibo Tu, Yang Liu, Mingyan Liu, Hedvig Kjellstrom, Kun Zhang, and Cheng Zhang. How do fair decisions fare in long-term qualification? Advances in Neural Information Processing Systems, 33:18457-1844 18469, 2020.
12

---

Published as a conference paper at ICLR 2023
SUPPLEMENT TO
"TieR BALANCING: TOWARDS DYNAMIC FAIRNESS OVER UNDERLYING CAUSAL FACTORS"
Zeyu Tang', Yatong Chen², Yang Liu², and Kun Zhang!.3
I DDpaartment of Philosophy, Carnegie Mellon University
:Computer Science and Engineering Department, University of California, Santa Cruz Machine Learning Department, Mohamed bin Zayed University of Artificial Intelligence
zeyutang cmu edu, 5 2 ucsc. edu, yangliuuucsc. edu, kunz cmu. edu
TABLE OF CONTENTS: APPENDIX
A Detailed Discussions Related Works
14
A.1 Causal Notions of Fairness
14
A.2 Fairness Inquires in Dynamic Settings
14
Additional Results, Technical Details, and Discussions
15
B.1 Discussions on the Causal Modeling of Decision-Distribution Interplay
15
B.2 When Tier Balancing is Initially Satisfied
16
B.3 A Remark Fact 3.2
17
B.4 Detailed Derivation of ASTIRI |++1 (Section 3.2.1)
18
B.5 Further Illustration Assumption 3.5 and Assumption 3.6
20
B.6 Additional Experimental Results
20
B.7 Potential Limitations of Our Work
21
Proof of Results
C.1 Proof for Proposition 3.1
21
C.2 Proof for Theorem 4.1
22
C.3 Proof for Theorem 4.2
25
C.4 Proof for Theorem 4.3
LIST OF TABLES
When ap ay, compare cases of H4+1 with different values of A
37
2 When ap ay, compare cases of H++1 with different values of A..
38
3 When 0D ay = ax compare cases of H441 with different values of A,-
4. When a& ay list possible instantiations of (P+ (e+4()
40
5 When aD < ay , list possible instantiations of p (c++1).
42
6 When op ay a, list possible instantiations of (
44
13

---

Published as a conference paper at ICLR 2023
A DETAILED DISCUSSIONS ON RELATED WORKS
In this section, we provide detailed discussions related works. In particular, considering our focus on providing a novel long-term fairness notion with the help of the detailed causal modeling of involved dynamics, compare work with previous literature causal notions of faimess, as well as fairness inquiries in dynamic settings.
A.1 CAUSAL NOTIONS OF FAIRNESS
Various causal notions of algorithmic fairness have been proposed in the literature, for instance, fairness notions defined in terms of the woon-eexiteence of certain causal paths in the graph (Kamiran al., 2013; Kilbertus et al., 2017: Zhang et al., 2017), fairness notions defined through estimating or bounding causal effects (Kusner et al., 2017: Chiappa, 2019; Wu et al., 2019; Mhasawade & Chunara, 2021), faimess notions defined with respect to statistics on certain factual/counterfactual groups (Imai & Jiang, 2020; Coston et al., 2020; Mishler et al., 2021). The proposed causal notions audit fairness in an instantaneous manner, i.e., the fairness inquires are with respect to snapshot of reality, and the scope of consideration is limited to observed variables only. Our Tier Balancing notion has built-in capacity to inquire fairness in the long-term and dynamic setting, which is very different from instantaneous causal fairness notions (beyond the fact that our notion encompasses latent causal factors).
While we detect and measure discrimination based previous (instantaneous) causal notions of fairness (e.g.. the existence of certain causal paths or causal effects), eliminating such existence of causal paths or causal effects is valid goal achieve but might not be the means one should opt for. To begin with, there is no guarantee that eliminating a causal path or effect results in non-existence of such causal path or effect in the future under the interplay between decision-making and data dynamics. Furthermore, the data generating processes represented in the causal model might not be casily manipulable under the same timescale of decision-making (e.g., the ones governed by nature and/or the mode and structure of a society). One cannot expect that the manipulation on the causal model (for the purpose of enforcing fairness notions) directly translate real-world changes in the underlying data generating processes.
Different from previous causal fairness notions, instead of directly "going against" the underlying data generating process (e.g., by eliminating certain causal path or causal effect), our Tier Balancing notion encourages "working with" the underlying data generating processes. With a detailed causal modeling of the
interplay, Tier Balancing emphases on the possibility of inducing future data distribution that is fair in the long run.
A.2 FAIRNESS INQUIRES IN DYNAMIC SETTINGS
Previous literature have considered dynamic fairness in specific practical scenarios, for instance, opportunity allocation in labor market (Hu & Chen, 2018), a pipeline consisting of college admission followed by hiring (Kannan et al., 2019), opportunity allocation in credit application (Liu et al., 2018), and resource allocation in predictive policing (Ensign et al., 2018). Different from previous literature, we present a detailed causal modeling of the decision-distributio interplay that is general enough to be applicable in various resource allocation problems (c.g., loan applications, hiring practices) while also being specific enough to encompass nuances in data dynamics for the particular practical scenario of interest.
In terms of the analyzing framework, closely related works have considered the one-step amalysis (Liu et al., 2018; Kannan et al., 2019; Mouzannar et al., 2019; Zhang et al., 2019). However, previous works focus on the long-term effect of imposing certain fairness notions that are readily available, for example, Demographic Parity (Calders et al., 2009; Liu et al., 2018; Mouzannar et al., 2019) and Equal Opportunity (Hardt et al., 2016; Liu et al., 2018). In our work, we formulate a novel notion of long-term fairesss, namely, Tier Balancing, and explore the possibility of providing fairness notion that characterizes the dynamic nature of decision- distrribbtin interplay through detailed causal modeling on both observed variables and latent causal factors.
In terms of the modeling choice for data dynamics, most closely related works model data dynamics using variants of Markov Decision Processes (MDPs) (Jabbari et al., 2017; Siddique et al., 2020;
14

---

Published as a conference paper at ICLR 2023
Zhang et al., 2020; D' Amour et al., 2020; Wen al., 2021; Zimmer al., 2021: Ge al., 2021). For example, Zhang et al. (2020) consider the partially observed Markov decision process (POMDP) model, and conduct evolution and equilibrium analysis with respect Demographic Parity and Equaa Opportunity notions of fairness. The dynamics are modeled through transition matrices on group-level qualification rates. Compared the modeling of transition matrices in MDPs, model is more fine-grained and on the individual level, answering the call for "richer and more complex modelings [of involved dynamics]" in previous literature (Hu & Chen, 2018).
Another closely related work is the one-step analysis on the impact of causal fairness notions on downstream utilities conducted by Nilforoshan et al. (2022). They consider a detailed causal modeling on the college admission running example and analyze previously proposed (instantaneous) causal fairness notions, namely, Counterfactual Predictive Parity (Coston al., 2020), Counterfactual Equalized Odds (Coston et al.. 2020), and Conditional Principal Fairness (Imai & Jiang, 2020). Our work is different in several ways: instead of utilizing a static graph, we focus on the decision- distribution interplay and explicitly capture both observed and latent variables along the temporal axis; different from analyzing one-step downstream consequence in terms of utility, we formulate a long-term fairesss goal and investigate the challenges and opportunities revealed by the notion.
B ADDITIONAL RESULTS, TECHNICAL DETAILS, AND DISCUSSIONS
In this section, we provide additional results, technical details, and discussions of our work. In Appendix B.I, we provide additional discussions on our causal modeling the decision-distributiin interplay; in Appendix B.2, analyze the situation where Tier Balancing is initially attained; in Appendix B.3, we discuss the role of exogenous terms and provide a remark on Fact 3.2; in Appendix B.4, we present the detailed derivation of Single-step Tier Imbalance Reduction (STIR) term ASTIR [++, in Appendix B.5, we illustrate the connection between Assumption 3.5 and Assumption 3.6; in Appendix B.6, we present additional experimental results; in Appendix B.7, we discuss potential limitations of our work.
B.1 DISCUSSIONS ON THE CAUSAL MODELING OF DISERIU-DITTIIIUIOON INTERPLAY
In Appendix B.I.I, provide additional details of the involved dynamics in the causal modeling of the decision-distributtion interplay. In Appendix 1.1.2, we discuss the relation between the practical scenarios and the modeled dynamics.
B.1.1 ADDITIONAL. MODELING DETAILS OF THE INVOLVED DYNAMICS
We use Xt,''s 's to represent three different patterns (instead of the number of count) of variables with respect to how observed features caused by the protected feature At and the latent causal factor H₁. There are three types of observed features: (1) features that only have the latent causal factor He the case, e.g., Xt,1, (2) features that have both the latent causal factor H; and the protected feature A, as cause, e.g., Xe.2. and (3) features that only have the protected feature A, as the cause, e.g.. Xe.3- For conciseness, we omit features that are relevant the practical scenario of interest, i.e., variables that are not causally relevant to (H,, A.). One can replace X 's with the actual number of additional features together with the causal relations among them specific practical scenarios.
At every time step T = t, the decision maaiin strategy D, is trained on the joint distribution (Ar, Xt,t, (obb))). However, when making the decision, D, only takes (Ar, Xt.;) as input. Since we are modeling causal relations in data generating processes, we only include a directed edge in the DAG if there is causal relation between variables. Therefore, the data generating process of D.
does not involve an edge between y- ((bb)) and D.-
1.1 THE PRACTICAL SCENARIOS OF INTEREST
As from previous literature (discussed in Appendix A.2), the modeling choices are closely related to the practical scenarios of interest, and therefore, can be very different in terms of modeling details of the involved dynamics in long-term and dynamic settings.
15

---

Published as a conference paper at ICLR 2023
Our causal modeling of repetitive resource application and allocation keeps track of individual- level situation changes, and enables informative and principled analysis on the Deccision-distrbbnn interplay in different practical scenarios. For example, in credit application (e.g.. Liu et al. 2018), the agents are clients and the latent causal factor (tier) can be individual's socio-economic status or creditworthiness; in predictive policing (e.g., Ensign et al. 2018), the agents are neighborhoods and the latent tier can be neighborhood's 's safety ratings; in the dual market pipeline (c.g., temporary labor markets followed by the permanent labor market considered in Hu & Chen 2018) or the admission- followed-by soredd--b-iiiggg pipeline (e.g., Kannan et al. 2019), the agents are applicants who subject to sequence of decisions and the latent tier can be the relevant qualification for the school program and the job.
However, when the decision received by the individual once in a lifetime (or at least very long time compared to the timescale of the decision-making! repeated application and allocation of resource may not be a suitable modeling choice. For example, college admission decisions are made on a yearly basis but individual does not repeatedly apply for college every single year (Mouzannar et al., 2019). In this case, if we focus on the decision made by a specific college, it is more natural to study changes in the population in terms of the group-level qualification profiles (Mouzannar et al., 2019). As another example, in the context of health care (e.g.. Mhasawade Chunara 2021), 1), when the resource takes the form of the medical treatment for the purpose of improving health outcome, not all treatment requires regular doses and therefore, repeated allocation modeled on the individual-level may not be an optimal choice. One can, for instance, resort to the modeling the level of subgroups as an alternative (Mhasawade & Chunara, 2021).
Considering the difference in semantics of fairness various practical scenarios, previous literature has pointed out that there is in general no one-size-fite fitzz-litt--l solution for algorithmic fairness (e.g.. interplay.
Kearns & Roth 2019). By presenting a detailed causal modeling for the we do not intend to provide general framework to encompass long-term faimess considerations in all practical scenarios. Instead, we would like to demonstrate the opportunities and challenges and hope our work can inspire further research.
B.2 WHEN TIER BALANCING IS INITIALLY SATISFIED
In the paper we have presented possibility and impossibility results to achieve, get closer to, the long-term fairness goal when Tier Balancing is not initially satisfied. It is natural to wonder what we should do if we find out that Tier Balancing happen to be satisfied during fairness audit. In fact, as we shall see in Proposition B. if Tier Balancing is satisfied as the initial condition, under the specified dynamics, one can use Demographic Parity (Calders al., 2009) decision-making strategy to maintain the status of satisfying Tier Balancing. This indicates that when Tier Balancing satisfied (as a lucky initial condition, or as a result of K Ksstep interventions), we have at least one way to maintain the fair state of affairs.
Proposition B.1. When Tier Balancing is initially satisfied, i.e., H2 J. Ao under Fact 3.2, Assumption 3.3, and Assumption 3.4, as well as the specified dynamics, the Demographic Parity decision- making strategy, i.e., Dt 11. A,, can ensure Tier Balancing still holds true for the next time step, i.e. H4t 1L A4t1.
Proof. To begin with, since H, IL Ar, by Fact 3.2, He is not a function of A,. As direct result, y(ar) is also not a function of A, (since the distribution of y(ar)) is fully determined by the value of H, h.). Besides, since D, satisfies Demographic Parity, D, Il As, and therefore by Fact 3.2, D, is not a function of A,.
According to Assumption 3.3, under the specified dynamics, H++1 is fully determined by (He, Dr, yl(or)), among which none of them function of A,. Then, we have H4t1 is not function of At-
Recall that in the specified dynamics, the same group of agents repetitively apply for credit with the
entire group unchanged. According to Assumption 3.4, A4+1 is an identical copy of A,. Therefore we have H++1 not a function of A i.e. He+1 IL A.41-
16

---

Published as a conference paper at ICLR 2023
T=t-1
T=
T=1+1
A++1
(ort)
X X
X
y-torj
D
y((obe)
0g
D
(obob)
1+1
Bc41
T
Exi.
H
xx
xxa
Ex,a
A
CD.
y((on)
X1,1
X,,2
X1,
Cyloboo
Y((aaa)
8
10:
Figure 4: The causal modeling of the decision-distribution interplay. The circle indicates that the corresponding variable is unobserved. We use diamond to denote the underlying causal factor and explicitly indicate the (potential) non-stationary nature of the decision-making strategies across time.
B.3 A REMARK ON FACT 3.2
Let us first present the definition of a functional causal model (Spirtes et al., 1993; Pearl, 2009): Definition B.2 (Functional Causal Model). We can represent a causal model with a tuple (E, V. such that:
(1) V a set of observed variables involved in the system of interest;
(2) E is set of exogenous variables that we cannot directly observe but contains the background information representing all other causes of V and jointly follows a distribution P(E);
(3) F is a set of functions (also known as structural equations) fe..... where each fa corresponds one variable V, V and is mapping E U UVV (V: V,
The triplet (E, V, F) is known the functional causal model (FCM). We can also capture causal relations among variables via a directed acyclic graph (DAG), where nodes (vertices) represent variables and edges represent functional relations between variables and the corresponding direct causes (i.e., observed parents and unobserved exogenous terms).
For the purpose of illustration, in Figure 4 we present the DAG (at time step T = t) with the exogenous terms E, explicitly modeled, where E, is the concatenation of individual exogenous terms:
E, = (EHt,) Eygobo Eyror), Ex.,,
(B.1)
We the symbol certain exogenous noise terms, e.g., EH, and (y denote the fact that the corresponding variables are affected by previous time step (T= t - 1), and such influence are encapsulated into exogenous terms from the standpoint of current time step (T l). For example, the influence from the randomness in D.-1 (when T = 1) on current Y, ((bb)) encapsulated into exogenous term (obo when T = t.
As we can see from Figure 4, (A, are root causes of all other variables (He, Xx,,, ((or)) y/(os)) Dz). Applying Definition B.2, we can utilize the functional causal model and represent each variable with a function (the structural equation) of its direct causes (including observed parents and unobserved exogenous terms). Then, we can iteratively replace variables with its corresponding structural equation and eventually represent variables in (H,,X,,s, y y/ooi) y, (obs). D.) with functions of only root causes (At, Fx), summarized in Fact 3.2.
17

---

Published as a conference paper at ICLR 2023
The noise terms E, are the unobserved exogenous terms that signify the unique characteristics of an individual. The utilization of such uniqueness of individual can be found in the estimation of counterfactual causal effect by making use of the posterior distribution of exogenous noise terms conditioning on the observed features, e.g., Kusner et al. (2017).
B.4 DETAILED DERIVATION OF ASTIRE [++1 (SECTION 3.2.1)
In this section, we provide the derivation detail of the Single-step Tier Imbalance Reduction (STIR) term:
ASTIR 1+ E (0, E++1) Se+1 E++1 [I[ E+)1
(B.2)
Firstly, in Appendix B.4.1, we characterize the conditional joint density of (fr(0, ET), fr(1, ET)) Then, in Appendix B.4.2, we focus on the situation changes of each individual from T = to T = ++ 1 induced by the specified dynamics. Finally, in Appendix B.4.3, we can calculate the expectation in Equation B.2 by aggregating situation changes for each individual from T to T -
B.4.1 CHARACTERIZING CONDITIONAL JOINT DENSITY
We can view fr(0, Ex) and fr(1, Ex) as two dependent random variables. Given combinations of De and Y,, we can define their conditional joint probability density when Eg = e as qi (f( f((,cc) d, d', v') and calculate it as following:
scffi(0.),, g.3)).8))
(f((,,)), fl,, 19 (0.c) = d. d',g. yola (0,c) V. yiua (1,c)= =1))
= -/ (( - f((0,),, f.l(0.))=(00....f((((... f((1,c)
(B.3)
JEEE
yiom df (1, d' "..""²²) (0,c) () y. y.g. y (1,e)= (- 1
where 1 (.) } is the indicator function, and the subscript i of the conditional probability densities (e.g.. ge(+) and p.(-)) indicates that they (might) change over time with different time step T = t. The functional form of f, can be convoluted and it is not necessarily the case that f. -) and fe( injective mappings& & + (0, 1]. Therefore, for the purpose of generality, in Equation B.3 we explicitly introduce the identity function (0, fc(0, e), fr(1, fx(1,e) when characterizing the conditional joint density qt-
B.4.2 CAPTURING SITUATION CHANGES FOR AN INDIVIDUAL
(1) let us For specific individual (j), given the value of individual's exogenous terms E(3) = Et
denote the difference between [0 e (() and e1,, as (e((e))) fr ((0,e())) Sc(1, e and the sum of fo, (3)) and fr(1, (t) as (e(e)))) = fe el
(1) + el ())) We introduce (-(()) and
(e()) for the conciseness of notation, and we can always map (f.(-), nc))) back to (fe(0, :), fe(1, via coordinate transformation:
fe(0.e (1)) f((1.e 2
V2
cos _
sin cos 4 (n((l())
pc(e(())
(B.4)
2
sin
Let us consider the connection between Pe+ (e()) (0+1 (0, f1, 22,) in the time step T + 1 and (p((e(")) f=(0, (1)) St(1, el in the time step T L. We use different time step subscripts for the exogenous terms, e.g., e41 in pe+1 and (1) in p((e))) since not necessarily the case that ( = et even if we are focusing on the same individual from T= 7 t 1. Nevertheless, for the given functional forms of ft, g. .g. y the combination of (d)) ()), (()) the value of exogenous term ( the initial situation of the current one-step analysis (when T t), and the hyperparameters (an, ay))
uniquely derive the value of
18

---

Published as a conference paper at ICLR 2023
1.0
1.0
1.0
1.0
fe Fe.00 exal
Ja aai
0.5
0.5
fi
0.5
0.5
0
0.0 0.5
0.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
1.0
0.0
0.5
1.0
f/0, cz)
f..10.e.40
f c))
(--)
(a) the example of distribution change when the initial condition satisfies the quantitative assumption
(b) the example of distribution change when the initial condition satisfies the qualitative assumption
Figure 5: An illustration of the connection between qualitative and quantitative assumptions in terms of the one-step update of the conditional joint distribution gr (r(rr(,,er),f7((.e))
d, d' v') (when y < v', and from 7 to T = tt 1).
(1,e)+1) et1)), and list all possible instantiations of (pt+ (e in Table 4 (if ap > ay). Table 5 (if ap < ay), and Table 6 (if ap = ay).
(1+1(e = fi+1 G f1,1
Let us denote such mapping from )) to (e with the function G.. For the purpose of simplifying notations, we can omit the superscript (7) if without ambiguity, since the value of exogenous terms ey signify the unique characteristics of an individual:
()44())+1) fe+ (0, et+1) (1, e1+ Gr "i ,d.d, cr, av) (B.5)
Notice that the value of the function G, only relies the information available time step T
B.4.3 AGGREGATING INDIVIDUAL-LEVER SITUATION CHANGES
We can calculate Single-step Tier Imbalance Reduction, i.e., the term ASTIR as following:
ASTIlE [++ = E [ (0, FE+1)) Fe+)(1,Fe+1) E (0, F.,) (1,1
|p+1 (E++1) - (E,))
(
E (()+1(E)) (e)(
E++1=5;
The value of exogenous of individual take value
E
= value of exogenous terms an individual take value
Pe+1 (5) = Gelfe, g. of dd d' v' aaa
}H
This make that iPe+ (E) when T t + 1 indeed valid instantiation from (e(()) when T t. 1 (-) not valid instantiation p.(.), contribution expectation
keeping track
individual in the that,
(184)
Pt(d, v.
[
I de (IJr(O,)) d'd', d',v)'))
d,d' E
(1) -165)-1ww (p((e))) ((P++1(E) = TelfergP.oe ".
" ;d, u. s.
(B.6)
where the equality (i) is based the definition of (p((-) and 1+);((); the equality (ii) derived from the Law of Iterated Expectation, keeping track of individual-led situation changes in the inner conditional expectation; the equality (iii) is the aggregation of individual-leve situation changes by plugging in the conditional joint density qe calculated in Appendix B.4.1, joint probability P., and the individual-levee situation changes discussed Appendix B.4.2.
The indicator function 1 (() = Ge(fi, oi gf y'm d, d', y, y e, ap. )y makes sure that we keeping track of the same indívidual (whose exogenous noise term equals to time t) before and after the one-step dynamic, even if his/her exogenous noise term equals to & at time f + 1, and that might not be equal to
19

---

Published as a conference paper at ICLR 2023
ACC
888
T-t
T- +
0.6
22 0.50
10
Densry
Blak
1.0
0.25
PxDD=
0.2
Asian
0.5
Hispano
0.00
0.0
White
10
10
20
0.0
0.5
1.0
0.5
1.0
7
И
(a) the trend of the changing tier H different step
(b) the accuracy (top) and upproval rute (bottom) for different groups
(c) the distribution of the underlying hiddem tier If among groups at time step I (left) and 7= 20 (right)
Figure 6: Illustration of the interplay between decision with accuracy-oriented predictors and the data dynamics (20 steps) on the credit score data set. Panel (a) and (b) present the step-by- step tracks of update in tier, accuracy, and approval rates for different groups; panel (c) presents group-conditioned distributions of tier before (left) and after (right) 20 steps of interventions. The legend is shared across panel (a), (b), and (c).
The fact that we are keeping track of the same individual also justifies the practice of only inte- grating over (conditional) densities with subscript i, e.g., (t(-) and (-).). instead of both t and
t ++ 1. To see this from a different angle, keeping track of situation changes of each individual (when comparing P (et+1) with (ex)) also alleviates us from the trouble of estimating (condi- tional) densities that involve future information. At the time step t, we do not know the densities 4t+1 (fi+1 (0, Es+1), fi-1 (1, E++1) d, d', y, y') and Pt+1 (d, d', y. yf) since they involve future in- formation De+1 and V4+1 at the standpoint of time step t.
B.5 FURTHER ILLUSTRATION ON ASSUMPTION 3.5 AND ASSUMPTION 3.6
In this subsection, we provide further illustrations of the connection between Assumption 3.5 and Assumption 3.6. In Figure we present the one- step update of the conditional joint distribution (Jr(00 eT), fr eT) d, d', y. 2/) from T = t T = t (we present the case when v/ as an example). For panel (a) and (b). the joint distribution of (0r Er). (1, Er)) is plotted before and after one-step dynamics, with quantitative and qualitative assumptions respectively. The distributions are color-coded, the deeper the color, the larger the value of the joint density.
Compared to the qualitative assumption (Assumption 3.5, illustrated in Figure 5b), the quantitative assumption (Assumption 3.6, illustrated in Figure 5a) is just special case, with quantitative char- acteristics built-in for technical purposes (we will make use Assumption 3.6 in the proofs for Theorem 4.2 and Theorem 4.3 in Appendix C). From the illustrations in Figure 5, we can also see that the behaviors of the one-step update of conditional joint density under qualitative and quantitative assumptions similar, with deeper color patterns occurring on the upper-left corner, indicating sim- ilar changes in the corresponding conditional joint densities @r (Jr(0, eT) (r( er) | d', v.y') (when y < y', and from T to T 1).
B.6 ADDITIONAL EXPERIMENTAL RESULTS
In this section, we present additional experimental results on the preprocessed FICO credit score data set (Board of Governors of the Federal Reserve System (US), 2007: Hardt et al., 2016). Similar to the experiment summarized in Figure 3, we convert the cumulative distribution function (CDF) of group-wise TransRisk scores into group-wise density distributions of the credit score, and use them as the initial tier distributions for different groups.
We consider quilit--mxximing curacy oriented and there is no explicit faimess consideration. In Figure 6 we present the summary of a 20-step interplay between decision with accuracy-oriented predictors and the underlying data generating process on the credit score data set. The accuracy-oriente decision-makinn is retrained after each one-step data dynamics. From Figure 6(a), there is no obvious evidence that the gap between step-by-step tracks of tiers for different groups decreasing over time. This observation aligns with our theoretical analysis (Thcorem 4.2) and simulation results (Figure 2) for perfect predictors.
decision-makin strategies, i.e., the decision-making policy is ac-
strategy
20

---

Published as a conference paper at ICLR 2023
B.7 POTENTIAL LIMITATIONS OF OUR WORK
In this subsection, we discuss potential limitations of our work.
B.7.1 SPECIFIED DYNAMICS VS. CAUSAL DISCOVERY
In this paper, we present a detailed causal modeling of decision-distribution interplay on DAG (Section 2.1) and formulate the dynamic fairness notion, Tier Balancing, that captures the long-term nen-term fairness goal over the underlying causal factor.
The research of causal discovery, where the goal is to discover the causal relations among variables (Spirtes et al., 1993; Shimizu al., 2006; Zhang Hyvärinen, 2009; Zhang et al., 2011), is a highly relevant area but out of the scope of paper. Our Tier Balancing notion of dynamic fairness, as well as our analyzing framework, does not rely on a causal model derived from causal discovery. As discussed in the comparison with previous literature in dynamic fairness studies (Appendix A), our causal model is richer and more complex, which provides the potential of a more principled reasoning of the essential cecisionndisttibution interplay in the pursuit of long-term fairness. We acknowledge the fact that it is nice to have the ability to discover the underlying causal model of the involved dynamics, which would provide further refinmments of our dynamic modeling based on the specific practical scenario of interest. Causal discovery can act as the icing on the cake, but not necessary component, of our analysis.
B.7.2 THE NUMBER AND DIMENSION OF LATENT CAUSAL FACTORS
In the causal modeling of decision-distrie distributiin interplay we present in the paper, we consider one latent causal factor that carries on the influence of current decision to future distributions. Recent developments in the identification of causal structures that involve (more than one) latent factors (Xie et al., 2020; Adams et al., 2021; Kivva et al., 2021; Xie et al., 2022) provide not only theoretical justification, but also an indication of the potential, for our effort in exploring long-term fairness inquires over latent causal factors. We believe that our detailed causal modeling of decision- distribution interplay (on both observed variables and latent causal factors) and our formulation of Tier Balancing notion of long-term fairness act as important first step.
C PROOF OF RESULTS
In this section, we provide proofs for results presented in the paper. For better readability, we provide an additional Proef (sketch) before proving Theorem 4.1 (proof Appendix C.2), Theorem 4.2 (proof in Appendix C.3), and Theorem 4.3 (proof in Appendix C.4), respectively.
C.1 PROOF FOR PROPOSITION 3.1
Proposition. At time step T = t. for any H, h4 E (0, 1). under the specified dynamics, among the population where ground truth is actually observable, i.e., Y(ob)) is not undefined, we have:
y(obs) Bernoulli(hh).
Proof. To begin with, according the d-separation relation among Dr-1, He, and y((mn) Figure we notice that ((r) IL D.-1 11, Therefore we have:
y( Bernoulli(hh),
P(y = H; = h1) h4r
P(D,-1=1 H, he) d(hr),
where d(-) is function d: (0, [].
Notice that there is no claim that D,-1 can be uniquely determined by a function of only h, We only represent the conditional probability mass P(D,-1 1 H, he) with function of ht without specifying the exact functional form. In fact, as we shall see in the later part of this proof, the exact functional form of d(-) does not affect the validity of the result.
21

---

Published as a conference paper at ICLR 2023
Since Y/ubs) is in fact y masked by Dz-1, i.e. Y(obs) is observable only when Dz-1 1 and is undefined when Di-1 = O, we have:
P(D-11 = 0, y y((os) - 0 H1, = h+) - P(D-11 0, Y( = 1 11, h₂) = 0.
This indicates that among the population where Y( is not undefined (the population itself may change at different time step), vy E {0,
P(Y:!on)) = y H, h+)
P(D = 0, y(obs) y 11, h4) + P(D,-1 1, L. y HI, hd) = = 1, 1, = y H. hh)
P = 1, Y(ott) y H, he).
Then, when d(h;)) (0, we can calculate the following probability:
P(Y H2 = he)
P(Y =1 1 H. = he)
P(Y = 1 H1 = h₂) + P(X, =0 0 Hr he)
P(D, = 1, 1, =1 H, = h4)
P(D-1 = 1, (, = He = h4) + P(Dx-1 = 1, y(ori) = He = he)
d(hr)dic
d(hi)he + d(hr)(1 h4)
-he -P(Y/(rn) H, he):
when d(ha) 1, this indicates that if He h2s know for that this individual received positive decision the previous time step (when definition; when d(h,)) 0, this indicates that if H, ht, we know for sure that this individual did receive a positive decision in the previous time step (when T t - 1), and in this case yr(obs) is undefined.
1), and we have y((rn) y-(obs) by
Therefore, among the population where ground truth is actually observable, i.e., y,(bb) is not undefined, we have:
y(shs) Bernoulli(ha).
C.2 PROOF FOR THEOREM 4.1
Theorem, Let us consider the general situation where both Dj and Y!!ri are dependent with Au- i.e. Di KL A,, Ylrii I. A₁. Then under Fact 3.2, Assumption 3.3, and Assumption 3.4, as well as the specified dynamics, when H, + Au, only least one of the following conditions holds true for all E E can we possibly attain H411 JL A
(1) The ratio f.(0,e) has a specific domain of value: f((1.n)
fc(0, 1 ap tay
f((1,et)
1 1 ap tay
(2) Positive (negative) labels only appear in the advantaged (disadvantaged) group, and the decision for everyone is positive (if ap ay
Itan-ay ay 11,
f((1 E 1+a0+tor 11.
(0, e.) -0
vion
(1,ez) 1,
(0,,)))) = of (1, et) = =1;
22

---

Published as a conference paper at ICLR 2023
(3) Negative (positive) labels only appear in the advantaged (disadvantaged) group, and the decision for everyone is positive (if a > ay
f() E liaapaa 1|,
fr(1 E I+ap--o0 -ay 11,
9 h (0,c) = 1.9 (1,ct) = =0, g. (0,0,e)) = sl (1,e) =;
(4) Everyone has a positive label, but the positive decision is exclusive to the advantaged group (if a < ay)):
fz) E -aap 11,
e1(1, E ltap 11
2 (0,
pimi
(1,0)) = =1
d6 (0) 0. (10) =1;
(5) Everyone has positive label, but the positive decision is exclusive to the disadvantaged group (if &D )y):
f((0, ex) E
litaptay' 1],
[((, E aaa 11, pinn
of or (0, 91 (1, =1, o = 1, (L, =.
Proof (sketch). In order to see the exact condition under which is possible to achieve H441 IL Ast1, we consider the necessary and sufficient condition such that Hz+1 fe+ (Ar+1, E(+1) not function of Ac+1- This, together with Fact 3.2, Assumption 3.3, and Assumption 3.4, indicates that we need consider the condition under which H(+1 min { ft Ee) aD 1) ay (2 1)] is not function of At.
Since both D, and y.(")) binary, we can exhaustively consider all value combinations of Dr and y((r)) and list every possible value H(41 can take in each case in Table (if ap > ay), Table (if < ary), Table 3 (if an av). By exhaustively going through possible cases, we can have a full picture of the update of H411 based on (H,, y! Dz). and then derive conditions under which H&H1 is not a function of Ar, i.c. we have the conditions under which possible to attain H4t1 IL A++1.
Proof (full). In order to see the exact condition under which it is possible to achieve H(+1 IL At+1, consider the necessary and sufficient condition such that H4+1 = f++1 (At+1. E is not a function of Ar+1. By Fact 3.2, Assumption 3.3, and Assumption 3.4, it is necessary and sufficient consider the condition under which H1+1 min {1, fc Ed) [1 + apD2 1) + (2Y, not a function of A,,
1)]} is
Considering the fact that both D, and (((ri) binary, we can compare the values of H4+1 when A, = 0 and A, = for possible combinations of D, and yr(on) For any fixed e, E E, we can list all the cases in Table (if > av), Table 2 (if ap ay), Table 3 (if ap ay), and see if for all e, E E, there is no difference in the value of He+1 between the cases when A, = 0 and A, = 1.
From Table 1, Table 2, and Table 3, can see that if and only the following hold true can achieve He+1 IL A441: for every 6t E E, whenever the joint probability P(g (0, ex) d, g. (0,e;) y. v.
af (1, ex) = d', .tt yian (1, ez)) y/) is nonzero, the last two columns of the corresponding row(s) the table, i.e., the exact values of H need match. For example, when op > ay, if
know P(g (0, ex) 0, 0, y!m") (0, ex) = 0, 0.g (1) = 0, ot y!")) (1, = 0) + 0, we need the last
two columns of Case (i) of Table to equal to each other, i... we need fr ex) = f((1, e,) to hold true.
23

---

Published as a conference paper at ICLR 2023
Without further assumptions on the joint distribution of the data, we do not know which combination of (d. y, d', 3/) will result in a nonzero joint probability:
P(oP(0, ex) d, bt (0, v6 (1, et) dd yuui) (1, =0) +0.
However, considering the fact that a P(g (0, ez) = d. Di / (0, e)) y, y. (1,e))
d', St " (1, = = 1 holds for all ex E E, we do know that for any fixed ea E E, there is at least one possible instantiation of (d', y,, d',, y²") such that:
P(o. d", d" you (0, ()) V. ge. ol,(111 (1) =d" d. yuun (1,e))
5/") +
(C.1)
Let us first consider situations where ap > ay and focus on Table 1. The analysis on situations where ap < ay (i.e., Table 2) ap ay (i.e, Table 3), is of the same flavor and therefore omit the detail in the proof.
To begin with, we can observe that not every entry of the last two columns explicitly keeps the min(,, } operator. On the one hand, since ap > ay (aDD ay E [0, 2). as of Assumption 3.3), we have (1 ap ay E (0, 1) and fi(a), ea) ap tay) (0, 1) (since H, fa E (0, 1]); therefore, we do not need to keep the min(-, } operator explicit, for instance, in the second to last column of Case (v- viii). On the other hand, when the coefficients (1 + an + av) >
we not sure if fe(ae, (1 + aD = ay) ) exceed 1; therefore, we need to keep the min 1} operator explicit, for instance, in the last column of Case (v viii..
Besides, if only one entry of the last two columns explicitly has the min 1} operator, it is equivalent to require that the terms themselves (before applying the operator) are equal (since the one without the min(.,1 1} operator is known to be within the (0, 1) interval). For instance, Case (ix) requires that fi,
f1, e1) an ay), which is equivalent to requiring fr et) + an ay) = fi(1, an ay)).
+ an ay),
Furthermore, if both entries of the last two columns explicitly has the minf;; 1} operator, the exact condition of matching the last two columns depends on the actual value of f:(0, 6)) and f)) For instance, Case (xv) requires that min (f,, (1+ + +,y), 1] min (1) (1+aD- - 1]. which could be equivalent to one of the following conditions (recall that 1 + aD aay > 1):
if we have fc et) E lItabtay 1] and fe(,, E which trivially holds true;
[ Ii-a--ay 11, we require 1,
if we have fe(0,ez) E (0, (ooD+ay) and Si(1, E lIFaD-oY 1}. we require fe(0, (1) + aa + ax) 1, which cannot hold true;
if we have f(( E |I 1] and fe(1,e)) E (0, (4op-ay) ) we require = et(1, + an - av)) which cannot hold true;
if we have fc(0, ex) E (0, 1+aD+ay) and f((,,e)) (0, T+ap-oy) we require (100ct) A.lle.)
1tap-ey I+aptor
Recall that without further assumptions on the data distribution, we do not know which row(s) of the table correspond to nonzero probability P (0)((0, ex) = of ex) = y, (0.g((1,e)) = d', of (1, = v²). ). As a result, in general, we do not know which set of requirements we should enforce for each e, E E. Therefore,
cannot derive necessary and sufficient condition for attaining H411 II A441 in general cases. Nevertheless, we can summarize the previous analysis and derive the necessary condition of attaining Hi+1 IL A441, i.c. only if at least one of the following conditions holds true for all e, E & can we possibly attain H4+1 JL A
(1) The ratio f(00.e.) has specific domain of value: [e(1.s)]
f((0,e) 1 ap + ay
f((1,
= 1 ap + ay
24

---

Published as a conference paper at ICLR 2023
(2) Positive (negative) labels only appear in the advantaged (disadvantaged) group, and the decision for everyone is positive (if ap > ay):
ex) E l1tap-ay 11.
fi(l, et) E 1lob
1/.
of (0,ez) = 0. (1, =1 (0,0))
vioi
(1,et) 1;
(3) Negative (positive) labels only appear in the advantaged (disadvantaged) group, and the decision for everyone positive (if ap > ay):
fe(0, et) E Flooptaa 1): fill E Itap -ay 11.
gi gi (0,ez) = 1 (1, 0, s = gt (1, =
(4) Everyone has a positive label, but the positive decision exclusive to the advantaged group (if aD ayy):
fe(0, et) E [ 1 fe(l, li-aptay 1]. gt ex) = yi (1,6)) 11, op e0) 0, (l, 1;
++o 1],
(5) Everyone has positive label, but the positive decision is exclusive to the disadvantaged group (if ap < ax):
((0 E E litaptay f(ll,e) t-an 1].
oe e+) 0i pan
(1, =1,
g. (0, 1, 1.g. ((,e)) =0
C.3 PROOF FOR THEOREM 4.2
Theorem. Let us consider the general situation where both D, and Y/(r) are dependent with As iee. Di M. At, y-lor) y Ar Under Fact 3.2, Assumption 3.3, Assumption 3.4, and Assumption 3.6, as well the specified dynamics, when H, H Ar the perfect predictor does not have the potential get closer to the long-term fairness goal after one-step intervention, i.e.,
D. Y((or))
(Perfect Predictor) ASTIR
[++1
0.
Proof (sketch). The goal is to calculate if it is possible for Single-step Tier Imbalance Reduction ASTIRIi i++1 to be smaller than when using perfect predictors. As defined in Equation ASTIR [++1 is a weighted aggregation (integration followed by summation) of [p((e+1)] [p(e=)] The quantitative analysis involves three key components: instantiations of (p+ (e++1), the knowledge/asumppionn on q: (ffe(0,)) fe(1,, d, d', y.y'). 9²). and characteristies of P.(d, d', v/yy").
For the first component, can list all possible instantiations of (P++1 (e++1) in Table 4 (if a ay Table 5 (if ap < a,), and Table 6 (if ap = ay), respectively. For the second component, we can introduce quantitative assumption
qi (0, e). (1, d, d' y, s²) (Assumption 3.6). For the third component, we need to exploit the characteristic the predictor of interest to gain further insight into the joint distribution (dd, d', y/ v'). For perfect predictors, have (t d' y, v/) satisfies Equation 8 (as we have discussed in Section 4.2.
For the purpose calculating the value of ASTIR [++1 the proof contains two steps: (1) exhaustively derive the value of (p(e++))] (p(e))] after one-step dynamics in all possible cases, and (2) aggre- gate the difference ((((e+1)) ip(et) with the help of the additional knowledge/asuumpiions on q. (()()) d, d' v.v') and P. d' y, v').
25

---

Published as a conference paper at ICLR 2023
Proof (full). For the perfect predictor Dj Y among all possible instantiations of se+1 (et+1) as listed in Table 4, Table 5, and Table 6, not every case corresponds to a nonzero Pt d', y. y and therefore may not contribute to the computation of ASTIR|: |++1 as detailed in Equation 5, By applying Equation 8 we only need to consider Case (i), Case (vi), Case (xi), and Case (xvi) in Table (if ap > at), Table 5 (if op < a.y,, ) and Table 6 (if ap = ay ). respectively. We list all possible values of [p((++1]] lyp(e)) for each of the aforementioned cases (the result applies to scenarios where op ay, (D < oy,, or aD at))
When (d, v.4/ (0, 0, i.e., for Case (vi):
(vi.1.1) (p(e+1)) [p(e -aa + aar f((,, <0
(0, E f((,, E (0, Itaptay
if we have
(1((1,))) s tan |arctan aptor 4
(vi.1.2) - (aD tay e), + (aD aa + 2)f((1, <
(((0,c)) E 1], f((1, et) E (0, Happtay
if we have
f((,,e)) < tan |arctan
antay 4
2
f((1,e,) tan (arctan aD+ay
(vi.1.3) (p(e(+1)) - lp(e))) (aD tay 2),( + (aD + ay + 2)f((1,ee) > 0
(+(0,e))) E (0, 1], [,,,,, f((1,e))) ex) E (0, Itab+ay f((1,e)) < f)) f((1, 2 tan (arctan
if we have
2
antar
(vi.1.4 (ote:))llooa)))=oo0oo/ooooo/// (ar
+ f((1, 0
(0.1,, E (0, Hta (((1,e))2(00..))
if we have
(vi.2.1) (
(2- =
ey)) f((1,e)) >
(i((0,et) E (),,), ex) E litabtay:ll (1((1,0) (100,e)
if we have
(vi.2.2) (((e)) = (aD (0 et) ez), 00
(1( e)) E 10. ex) (1+abtay:1 ((() 2 fc(
if we have
When (d.d' = (1, 1, 0), i.e., for Case (xi):
(xi.1.1) (p(()+)) [o(ee) -(an + (Jr + fa(1, <0
ff((0,e)) E (0, 1 fo fi, ex) E (0, (i((,) 2 tan + aretan abtay
if we have
(xi.1.2) (p((e+)) [ = (an tay + (0,s(( e + (an tay - 2)fe(1, <0
f((0,e)) E (0, itoppay (11,e)) E (0,1) f((1,e)) < tan I arctan optoy (1(1,e)) 2 tan arctan aD+aY
if we have
3n
26

---

Published as a conference paper at ICLR 2023
50,e))
f(0..s))
no
(.:2.22
(x6.14)
2 3)
(oi113)
(oi.1.)
(06.14)
(H222)
ez(
e.(0,e))
(a) Case (vi), (d, d', v. = 1,,1,
(b) Case (xi), (d, ,', v. 1,00
Figure 7: Illustration of the sliced squares on the (fe(0, et), e() plane. Depending on the initial situation, i.c., the slice that the (fe(0, fi,, ex)) pair falls upon, the term [ę((t++)] - [x((e)] takes different values. The shaded slices indicate that if the initial situation satisfies the corresponding condition, the calculated [ş((t++)) - [o(e)) < O.
(xi.1.3) (p((e+)) - (x(e)) (an + ar + (0,,((0,ee) Ex) + (an aa 2)5e(1, ex) > 0
(t(00, E (0, Ha f((1, E (0,
if we have
f((1, < tan (3T - arctan aptay f((1,e)) 2 f((0,e))
2
(xi.1.4) (p((e+)) ((p(e)))) (an + aar) + f((1,e)) >0
if we have (f((0,c)) E (0, IFaptay ), fe(1,e)) E (0.1)
(t((),e)) < e()
(xi.2.1) (p(e(+1)) - lp(e))) + f((0,e:) - a2-ap- aD -y-
(f((0,et) ex) E | el,, (0.1) (1((1, 2 f1(0,e))
if we have
(xi.2.2) ((p()(+)) (p(e))
= = f((0,ez) + +(ap + ay fe(,)))) >0
[f((0, E lItabtay f),,((1,e)) e (0,1) (((1,)) < f.(0,e))
if we have
When (d, d', 3/) (0, 0, 0, 0), i..., for Case (i), or (d, "d.d....y)) = 1, i... for Case (xvi), (p((r+1) (p((e: = 0.
Now we proceed to the second step and aggregate (sp(e++1)) - (p((z) terms. According to Equation
5 and Equation B.5, for the perfect predictor we have:
(Perfect Predictor) [++
= 1,0,,
/ { (1p(+1(E)) - -(p1))
STIR
It
((P))1(5)= = G(ft, oe yum :0,1,0 (,,,0,,,,,a,,ax)) (e(f,())e),,((1.))) 0, 1,
(C.2)
P,(L,,,,,,) [
JEEF (1vm- - (pp(
1(P(41(E) = G(fr.g ge. y'm! C.tt:oPPPp".LLLLLLLL....... 1,0,1,0,,,ap.aa)) (((.(0))),,(11)))) | 1, dede.
27

---

Published as a conference paper at ICLR 2023
As from Equation C.2, need to perform two-dimensional integrations the (fr(0, f((1, plane, calculating the expectation of the term (4(e++1) - (p(())] over the conditional densities (0, e), ez(1,e) e) 0, 1, 1, and g: ft),) 1,0,1,0) 0). Since these conditional joint densities could be convoluted in general cases, the calculation of conditional ex- pectations in Equation C.2 could be rather complicated. Therefore, we propose to take advantage of Assumption 3.6 to quantitatively simplify the calculation yet remain consistent with the rather mild qualitative assumption (Assumption 3.5), and derive result that is numerically clear and infor- mative. For the purpose of better illustrating the connection berween (qualitative and quantitative) assumptions qe (0, e), fi e) d, d', y.y') and the computation of ASTIRIE [++11 also provide illustrative figures as shown in Figure
With the help of Assumption 3.6, convert the conditional expectations in Equation C.2 into calculations of multiple integrals on slices within a x 1 square on the 2-D plane, where Φy and Φ1 axes correspond to the value of f0, E1) and fr(1, Es) respectively:
SEES JEC.ee (E)L+1(E)I - [px(e))) (5) G(fe:g S.....g.... gt. yuu
;0..
-9 (fr(0,, c), f((1,)) f((,,)) | 0, 1, 1)ddde
(arctau apiay do
tan
70101
(low)
-aan + ay) + )1) doldon
14y
+
(aD + ay - 2) + (an + ay + 2) dōi
tan arctan optay -7 -0
Tap
+
+ay (aD + ay 2200 + (aD + +ay - 2)) dΦ déo
I1ap luy
tan
arctan aptay
00
1
-2-aan- - + Φ1 dø d déo
14ap +ay
ttaptay
+
70111
(up)
1aap-ay
1taptoy
(aD + (x) + Φ1) doi ddo
Itaptoy
+
1 (ap + ax) Φn dΦy déo
140D-ay
1 (an + ay) Φi de dōld )
+
1tap+oy
00
28

---

Published as a conference paper at ICLR 2023
JEE / (1) - lo.(e))) (x)+ G(r. si
(0((f((0, (),
tan
[ arctan antay
V1010
(up) (up)
(ap ado Φ1) dóudΦr
Ltaptay
(ap + ay + 2)00 (aD + ay 2) døodøi
0
tan 1
-arttan aptay
Toaptay
(aD + ay + 2)00 + (aD + aa - 2) ddpde 8n areetnn aptny
Toop+oy
11
1 (2 1-aap ap ayy døndΦr
Itaptay Aaapeta
Toaptay
Tooptay
+ 71010
(on+ + ayb + Φ1) denden
Ilupley lay
1 do + (ap + av 1r dóndó
do + (aD + ay )oi d j.
14aptay 61
Since 10100
+ +0101
(up)
and
71010 + 71010 = 2, we can derive the form
STIR
(Perfect Predictur) [++
A
(Perfoct Predictor)
6+1
(P,
(low) 10101
+ P,
(up) 71010)
{
STIR
+ ap +ay
tan? 2 (arclan +
π
3
2(1 ap
.x) tan |arctan
1
3
&D + ay
4
1-ap ay
0D Qy
6
2(1 + aap + ay)
3(aD + gy) 6(ap + a2) 19(ap + +ay ) 10
+
6(1 + ap 3
+ (P,(0,100,)) + + P,
aD + ay - ap + ay 1 an Fay
3(ap +
+ ay) +
3
3(1 + an + Fay
where To 71010 (0, 1) (according to Assumption 3.6), and ap, ay E 00 (according to Assumption 3.3).
Let us denote ((DD ay) tan (arctan an-ay - :t) to simplify the notation. Without loss of
(up) generality let assume that P:(0, 1, 0, 1) (llow P, V1010
29

---

Published as a conference paper at ICLR 2023
We can further compute the partial derivatives and find out that:
a
PPeffcct Predictor) [++1
STIR
1
a (O, 1, 1)
0010
(low)
P((10,,1,0)
(up)
1+ap + ay
((an.ar)
3 2(1 - -ap ay)
+
B(aD, ay)
3
ap ay
3 aD ay
2(1 ap + ay)
3(aD + ay )3
6(ap + ay 19(ap + ay) 10
6D + aa)
0, V Yan,ay E ().
and that:
0A
STIR
(Perfect Predictor)
1+1
(P(0,1.)))+((((01111))))
2 3(1 + aD + 33
BlaD ax))
3
+ (P,(0,1,0,1) 10101
(low
(up) P((1,,,1,0)
T100o
2(1 +ap + +ay)
08(aD,ay)
B(aD, ay)
3
Man ay)
2(1 aD - Qr) 0B(ap.
+
3
Ōap + +ay)
1
15(an + av) +11
+
6
(1+ ap + )
6(1 aa + ay)
[ +
= 1,,))) P,(1,0,1,0)
3
3(1 + ap + a
(low) + (P.(0,,,,01)
P.,(1,0,1,0 (up)
Vo101
10000
2(1 + (an,
+ op + +ay -1]
31++a++ay* (an + +ay 33 3 +
6(1 6(1+ap= + +ay)
10101 21010 E a0,1).ap,ay [0. 1
0, V (low)
(up)
n
where we utilize the fact that 08(p.oy)
(1 (LD, Htappay
Maptay
30

---

Published as a conference paper at ICLR 2023
Therefore, conclude that
A STIR
(Perfect Predictor)
lim
(P, -0100 + P,(1,,,1,0) 71010
(low)
10101 ++1
T1610 -1
+ay
+0
1 aD + oy
tan? [arctan
4
3
aD + ay
2(1 ap -ay)
1
π 4
3 tan arctan
uD + ax
-- aD ay
3 0D ay
(1++a + at)
3(aD + ay)
6(aD + ay) 19(aD ax) 10
6(1+aD +ay
PP(.0...
aD + ay
aD ap+ay ay
3(aD + a + ax) +
3
1 ap + aay
3(1 aap + +ar)²
ie., under the specified assumptions and dynamics, have
A flow) (up) 71010 E (0,1),ap,ar [0, 2 A
STIR
(Porfect Prodictor)
/ > 0.
(C.3)
C.4 PROOF FOR THEOREM 4.3
Theorem. Let us consider the general situation where both De and Y!!r) are dependent with A,, iee. Dr H. A,, Y-(ar)) y. Ar Let us further assume that the data dynamics satisfies E (0, 2),ay O. Then under Fact 3.2, Assumption 3.3, and Assumption 3.4, and Assumption 3.6, as well as the specified dynamics, when H, J._ Ar. it is possible for the Counterfactual Fair predictor to get closer to the long-term fairness faimmes goal after one-step intervention, if certain properties of the data dynamics and the predictor behavior satisfied simultaneously, i.e..
g. Ex) = g
ROLLD+ +10.0..1)+P...0.. <
27 &
OD P.L.L..DD+.....111.00 $ 1,
P10..00.1)++(00.11.000 ax =0 (Counterfactual Fair) t+1
O.
STIR
Proof (sketch). Similar to proving Theorem 4.2 (proof in Appendix C.3), the goal is to calculate if possible for the Single-step Tier Imbalance Reduction ASTIR Counterfactual Fair predictors.
[++1 to smaller than 0 when using
weighted aggregation of |p(e++1) (p(e)) (as defined in Equation 5), the quantitative analysis involves three key components: instantiations of (c41)) the knowl- edge/assumptiins on qe (Je(O, <), fe(1,)) d,, d', n.,y v')"). and characteristics P. d', v/.
Since ASTIR| 1++1
For the first component, since ay 0 is special case of scenarios where ap > ay, all possible instantiations of P (c++1) ) in Table 4 (when ap > aty). For the second component, we can introduce quantitative assumption on qe (0, c). f((,,)) <) d, d', y.y') (Assumption 3.6). For the third component, further insight into the joint distribution (, d', y, s'). For Counterfactual Fair predictors, we have (a(d, d' y. y') satisfies Equation 10 (as have discussed in Section 4.2.2).
list
need to exploit the characteristic of the predictor of interest gain
31

---

Published as a conference paper at ICLR 2023
For the purpose of calculating the value of ASTIRl" [ the proof contains two steps: (1) exhaustively
derive the value of [ - lp((e)) after one-step dynamics (finished in Appendix C.3 when proving Theorem 4.2), and (2) aggregate the difference (p((e+1)]] Ig(ee)] with the help of the additional knowledgeeaassupiions
qi (fr(0,,), fe(1,e) d' ,, v.y") and Pr(d, ".dd,',y.y)) y.y").
O
Proof(full) Based on the definition of ASTIR [ the proof calculates the aggregation (integra- tion followed by summation) of the difference (p(et+1)] lp(e))] with the help of the additional knowledgeeasssumpiions on (0, f((1,) d, d', v.v") and Pr(d, d', v.y').
Since we assume a E (0, } ay O, focus possible instantiations of Pr+ Or+1(et+1) as listed in Table 4 (ap > ay). For the Counterfactual Fair predictor that satisfies (0, E,) g. (1, Ee). not every case in Table 4 corresponds to a nonzero P. (dl d' y. v/) and therefore may not contribute to the computation of ASTIREE 1 as detailed in Equation 5. By applying Equation 10 we need to consider Case (i), Case (ii), Case (iii), Case (iv), Case (xiii), Case (xiv), Case (xv), and Case (xvi) in Table 4.
When (d. d' ys 3/) satisfies y y', ie, for Case (i), Case (iv), Case (xiii), and Case (xvi), we have (p((++1) - |p(ee)] = O. Therefore we only need to calculate A (Counterfactual Fair) [++1 for Case (ii), Case (iii), Case (xiv), and Case (xv) (although ay
explicitly keep the hyperparameter oy in lt
STTRR
the proof for the purpose of notation consistency).
According to Equation 5 and Equation B.5, for the Counterfactual Fair predictor have:
(Counterfactual Fair) ++1
P.(0,0,0.1)
STIR
t
((F(+1 = G(fe, 9h ylum 00,0,0.1,,,,0.. ()(fi(0,e), 1,0,0,1)dEdd
+ P,000,11,). /
LEE (19+21(9) - le(())))
(fp = G(fr, 96 you
((f(()))),,(11.)) 0,0,1,0)df
(C.4)
+ P,11,,,0,1 / JrEE (19++1 - (ex(e)))
-19(+1(E) = -C(f)( 99 you 1,,1,0,,,,,,,,,o0 ((f(((,)),((11.) 1,1,0,1)
P,(1,1,1,0). f
- (1++2((1)-
(1( G(f 9e yoa (,1,1,0,c,,,,..
(e(f((0,e),f((14) 1,1,
Similar to the proof of the result for perfect predictors presented in Appendix C.3, with the help of Assumption 3.6, convert the conditional expectations in Equation C.4 into calculations of multiple integrals on slices within a 1 square on the 2-D plane, where @o and Φi axes correspond to the value of fO, E.) and f1, E.) respectively:
32

---

Published as a conference paper at ICLR 2023
JEe. JEe (19)+1(5)) - lpc(e))) (1)+1(()) = G(S. ".. "0 ;0,0,0 1, (,,
((1,,) 0, 0,1,ddddd
1- 40 Taptoy
(low)
-aa + ay0 + (aD ar)o dóidóo
do
-(2 up arr (2 aD or ar) )ah den dépo
D -ay do
I 1/ (aD + +ay )00 - (aD arr)) d
(up)
70001
/ JEES JEEE (1))+1(9)- - (p((e))) (1) P. "h (0.00..0...aa..0r))
(() 0,00
0,0, 1, 0) dEde
1-40 01
70010
(up)
(aD ar) (aD + ar) dōpdói
(2 ap + aay) (2 - -aa- ar ) dōndōr
1-ap-ay d aD by
(low)
(ap ar) + (aD ar) doo.
30010
33

---

Published as a conference paper at ICLR 2023
SEEE Jsee (196+1(E)1 - [pc(e))) - y
,1,1,0,1,,,0,.00
(ge(fe(O, f(1,c) 1,,,,,1))d5 1ddgde
1tep-ox d2 Itap
Hantta
(low) 7110
ap + +ay)á )oi doodØi -22 aD ay + (2+
1tap-ay 1aaptay
1tap-o
(ap-ay)er (aD ay )o d dóo
1top oy
900
+
1- (2 aD ayyoo + Φ1 dø dôo
1taD -y
Itaptuy
(up)
Ilap by
91
-aa (y)ou (aD + ay don døpdΦ1
71101
Ilup lay
+
1- (an -r on deb deo
0
Happta
14ap-ay
1 - 1-(ap-- -ay)oo Φ1 d$ doo }
+
-aap-av
1
1 (1P)+1(E) - (<pe())) (1 = G(J, " "x ;,,,,1,0,,,,0,,,
JEEE
-c E), f((
1, 1,1,
Hlaplay.
T-ap+ay
TeaD-oy
71100
(2+ + ar - -
(2+ ap
- ar de déo
1-ap-ay di 1tap+ay
1+0D-ay
-(an + ar) + (an )x dóndór
0
0
1tap-ay
+
1 00 - (ap + ay d d
1aap-ay
1oaptay
040
+ 1000
(OD + axr)) - (ap - )0 den doo
34

---

Published as a conference paper at ICLR 2023
Since 70001 + 70001
(up)
2, (Counterfactual Fair) [+]
70010 + 70010 2, flow
(low)
(up)
11001 + 71101
(up)
2, and
7111 + 71110 2, we can
(low)
(up)
derive the form of the term A STIR
(Counterfactual Fuir)
[++1 (P,(0,0,0,1) 70001 + +,) -0010)
STIR
(nn - ay - uD - dr)2
(an + ay ay) -aap- - -aa)
6(1 -D + )xy)2
3(1 -1aap-- +ay
20y
- -2-aD - +xy+ (2 -2- -2 + ay ay) aD)
aD ay
+ 3(1 - aD + ay)
+
1- a-aptay
2
+ P,,,, 7110
20y
a2+ a2+ uD - 11+0--ayy ay + +ap)
3(1 + ap ay (1 + ap ay
1 1
+ 3(1 + an + +ay) an - )3
(an + -ay) + an )
(aD - arr ay)
2(1 aa + ar)
QD ay
2/1
(1 + aD tay)
6
2
2
(1 aD - aa)
(1+ ap - aa
(1 + ap + aay
3+ 2an
1+an QY
1
+
+
2D +
2
(1 (1+ap ap - oo)²
(1 + ap + ar
3+ 20p + 20y
1
(ap+ + ayjar
2(1+ ap + ay )
2
1 aD ay
1+aD ay
(1+ap + a3
(up) + 1,1,1,110)
71110
20y
(2+an+ar) -
ay ap)
3(1 aap ay (1 ta + + 0y)3
1 1+ap ar
+ 3(1 + ap + a1 an - 33
(aD - ay) +aal+a--00 - ay) (an + ay an ay) 21
a1 + aay
ap-ay ay
(ap + ay
(a))(((()))) (ap - ay)
ap + (I+an+ar)(1 a1 ar)
2(1 +ap + ay )(( [1| + aD ar)
ap ay
2(1 + oy)
(1 - )
6
2
(1 + ap + +0y)
+ (P:(0,0,00,)) + P,)),,,1,0) (- aD
+ay)
aD + ay)
+ 1,(,,,,0,11)
(I+an + )3
3
2(ap + +ay)ay
1
+
(1 + ap + +ay)3
1+aD ay
14ap aD +ay
2ap
2ay
1
1
+ - +
3
3
3
(1+ap- + aD - 1
33
(1+ aD + ax
1
1
(1+ap - ay).
(1 ay
+ 2
OD +ay)
P,1,
(1 ap + ay
35

---

Published as a conference paper at ICLR 2023
where Tooo 71010 (0, (according Assumption 3.6), and ap, ay [0, 2) (according to Assumption 3.3).
Now let consider the data dynamics where ay 0 and simplify the form of A
(Counteriictual Fair) [++
STIR
A (Conterfactual Fair) [++1
STIR
ODD
3 (P(0.0.0.1)+ P((0,0,1,0) +(1+a
oD +
As see, long have R(1111++((111
27 and at the same time the parameter
P(0000.1))+(00000000
satisfies ap E ( ((0.1100))++.( $ - 1, })) is possible for the counterfactual fair predictor to achieve a negative value for ASTIR [++1 after one-step intervention:
P.L.L..)) P.0.0.0.1) 8
E ( (001100+0001111)))) 5 1, 1)
A (Couterfaactul Fair) 1+1
STIR
0.
0
36

---

Table 1: When ap > ay, compare cases of H4+1 with different values of Ar.
Published as conference paper at ICLR 2023
Dr
(ar))
H4+1
Case
if A,
if A,
if A, = 1 if A, = 0
if A, = 0
if f4 =1
(i)
0
0
0
fe(0,
f((1,))
(ii)
0
0
fi(0, aD aY)
f((1, aD + aay
(iii)
f((0,e)(L - -ap aay)
f((,,e)(1 - -a
- -a ar)
(iv)
0
0
f((0,e)
f((1,e)
(v)
0
f((0, aD aa
m + ap - ay),
(vi)
1((0, -ap - aay
[1
vvii)
f((0,e:)(1 aD + +ay)
-
(vii))
fr(0,e:)(1 aD +ay)
(ix)
min(f((0,e)(111 ap ay),
a(),a)(1-a-aay)
(x)
0
min[/[(0,e][1+aap ann - av),
[((,,))(1- ann + +ay
(xi)
0
0
min(:(0==)((oo+arrr\]
f((1,))(( -an -av
(xii)
min(:(0ea)(Lap
f((1, -aa + +ay) f((,,))
(xiii)
f((0,
(xiv)
=mm min(f((1,e)( ap av), +n (in([((),e)(1+aapp - ay)
(xv)
(xvi)
f((0,e))
f((1,))

---

Table 2: When ap ay, compare cases of H4+1 with different values of Ar.
Published as conference paper at ICLR 2023
Dr
(ar))
H4+1
Case
if A,
if A,
if A, = 0
if A, = 1
if A, = 0
if =
=1
(i)
0
0
0
fe(
ft(1,e)
(ii)
0
0
fi(0, aD ax)
min(fi(1,e))(1 - + ay),
(iii)
min{si(0,ee)((1 aD + ar)),
a(l)e))(-aD-an -n ay)
(iv)
0
0
f((0,e)
f((1,e)
(v)
0
f((0,e)(11 - aD aa)
f((1,e)(1 ap aa)aaa))
(vi)
5(00, -ap - -ay)
min(J((1,a) +1+ + ay),
vvii)
min(fi(0,e:)(1 + ay),
+()+))](
(vii))
min(ff(0,e::(1 aD + ar),
an + ay),
(ix)
f100,a)(++a>> - -ay)
f(),a)(1ap-aa/
(x)
0
[i(0,e)(1+an -ya)
min(Si(1,e:)11 an + +ay,
(xi)
0
0
min( +
an an
av),
[((,,a)11 av)
(xii)
an ap min(f((1,e)] ap + +a
(xiii)
f((0,
f((1,))
(xiv)
+ -ap
(1 av),
(xv)
min) an +a)) (
(xvi)
f((
f((1,))

---

Table 3: When ap = = compare cases of Hz+1 with different values of Aç.
Published as conference paper at ICLR 2023
Dr
(ari)
H4+1
Case
if A,
if A,
A = 0
if A.
if A, =0
if A4 =
(i)
0
0
f((0,
f((1,e)
(ii)
0
0
f((0, 2a)
f((1,e))
(iii)
0
f((0,e+)
ei)), -
(iv)
0
0
f((0,e:
fi(1,e))
0
f((0, 2a)
f((1,e)
(vi)
f((0, 20)
min(J)(1, + 2a),
vii)
f((0,e))
fi(1,))
(viii)
fi(0,e:)
min{ + 2a), 1}
39
(ix)
f((0,e))
f((1,e)(1-2)
(x)
0
f((0,
fi(1,e))
(xi)
0
2a),1) (1) min{
+1 2a),
f((1, -20x)
(xii)
(i,
f((1,))
(xiii)
f((0,e))
f((1,))
(xiv)
1
f((0,
min(f((1,e))(1 + 2a).
(xv)
f((1,e))
(xvi)
f((),e))
f((1

---

Table 4: When op > ay, list possible instantiations of Pi+1 (e++1).
Published as conference paper at ICLR 2023
Dr
Case
if A,
0
Ar
1
if A
0
if At
=
0
(P((t)
(ii)
0
0
1
(((e)(1-ap) aym((e)
(iii)
0
1
0
p((e)(( ap) ayn(ee)
(iv)
pr(et)
(v)
0
0
(v.1)
pr(et)(1 - )x) af f((1,e) E (0, I+an
(v.2)
f((0,e)(L aD ay) otherwise
(vi)
(vi.1)
+ P((e)) af)
E (0, Hopptay
(vi.2)
f((0,ei(1 - -a - -y) 1,,hheevss
40
(Vi))
(vii.))
(i(e)) - ar (f f((,, E (0, ItaD-ay
(vii.2)
f(0,e)(l-aa-- ay otherwise
(vill)
(viii.
arr))[ if) f((, E (0, 1taptay
(viii.2)
f((0,e)(11 ap - otherwise
(ix)
(ix.1)
(((e)([-aa) if E (0, I+ap-ay
(ix.2)
(i(, - ap - ary)
(x)
(x.1)
po(et) + (ap ay if E (0, Itap-ay
(x.2)
1 f(1, an ay
(xi)
(xi.1)
p:(et) + (aD + of
E (0, Itantay
(xi.2)
1 1-
ap - ay
(xii)
1
(xii.))
ay) 1 f-), ap + aay), otherwise
f(0, ex) E (0, 1taptay
(xii.2)

---

Table 4 (continued from the previous page)
Published as a conference paper at ICLR 2023
Y-(on)
Case
(t+1(e = (0,
ex+1) - Se+1 (1,441)
A,
At
if A,
if Ar
(xiii)
0
0
(P(e))
(xiv)
0
1
(xiv.1)
a
E (0, I+ap-ay and fi(1, E (0, ItaDtar
(xiv.2)
((0,e)((+ap)) - 1, ((0, E (0, I+ap-ay and ft(1, E lltaptay 1]
(xiv.3)
1-
E litan 1] and f((1,e)) E (0, Itantay
(xiv.4)
0. otherwise
(Xv)
(xv.1)
f(00, et) E (0, 1taptay and f((1, E (0, 1oap aay
(xv.2)
[(0,ee E (0, Itap+ay and fr(1, E l 1]
(xv.3) - an f(0,, ea) E litantay 1] and f(1,,) E (0, Itap-ay
(xv.4)
0. otherwise
(xvi)
(p(())

---

Table 5:
When op < ay, list possible instantiations of Pi+1 (e++1).
Published as conference paper at ICLR 2023
Dr
Y/(r))
Case
if A,
0
Ar 1
if A
0
if At 1
pr((t)
(ii)
0
0
1
(ii.1)
(r(e)) - -y E (0, T-aptay f((0,e)(1 ap ay) 1, otherwise
(ii.2)
(iii
(iii.1)
pr(et)(1 - ap) ay))(e)), if f.(0, E (0, T-ap+ay
(iii.2)
1 ((1, - ap otherwise
(iv)
1
pp(et)
(v)
0
pc(e)( - σx) aDn((e)
(vi)
(vi.1)
p((e) - (ay + ay))) f((,,)) E (0, 1+antay
(vi.2)
[((0,e:)(1 ann ay) otterwee an
(vi))
(vii.))
p((ee) - (ap - ar (f es) (0, I-aptay
(vii.2)
ap - - -amp:l(++t))-)((())))))a)
(viii)
(viii.)
E (0, 1-aptay and ft(1, E (0, 1iaDtay
(vii..2
- -.0 E (0, T-antay and fi(1, E lifabtay 1]
(viii.3)
E <l-abbay 1] and f((1,e)) (0, Ttaptay
(viii.4)
0, otherwise
(ix)
pr(et) av) aap((e)
(x)
(x.1)
pr(et) + (aD - ay (f) ff E (0, I-antay
(x.2)
f((0,e)(1 ap a aa otherwise
(xi)
(xi.1)
(((e)) + (aD + ay (e)))) f( et) (0, 1
(xi.2)
1 f((1 ap ay), otherwise

---

Table 5 (continued from the previous page)
Published as conference paper at ICLR 2023
D2
Y-(on)
Case
(t+1(ee41) - (0+0 (1,1(1,
A,
A
A;
if At
(xii)
0
(xii.))
+ameleh (( E (0, 1taptay and fi(1,e)) E (0, i-aptay
(xii.2)
- f,i E (0, itan+ay and f((1, E Eli-ataar"
(xii.3)
aa f(0,,e) E litoptay 1] and f((1, E (0, T-aptor
(xii.4) 0, otherwise
(xiii)
0
0
(P((t))
(xiv)
0
(xiv.1)
aym E (0, Itaptay
(xiv.2)
(100 + aan -ayaaay)) - 1, otherwise
(xv)
(xv.1)
if fx(0, E (0, I+aptay
(xv.2)
1 f- ap ay), otherwise
(xvi)
p((e)

---

Table
When an - dy a, list possible instantiations (
Published as conference paper at ICLR 2023
Dr
Y(an)
Case
if A
if
At
if A.
if A
(i)
0
0
0
0
pr(es)
(ii)
0
0
0
pr(ee) a) -at)(
0
0
1
pr(et)(1 a) ann((et)
(iv)
0
0
1
1
vi(et)
0
vr(ee)(1 - a)) [p)) (et)(et)
(vi)
0
1
0
1
(vi.1)
pi(es) 2am if (1,, (0, 1420
(vi.2)
f1(0,))(1 - -2a) - 1, otherwise
Fr(et)
(viii)
(viii.1)
sr(e:)(1 + +a)
E (0, 1+2a
(viii.2
f1(0,et) - 1, otherwise
(ix)
pr(es) a) am. (c)(
(x)
p((et)
(xi)
(xi.1)
pr(et) 20m if f( ez) E (0, 1+2a 1-f e,), 2a). otherwise
(xi.2)
(Xii)
(xii.)
pr(e:)(1 + +a) (m,) (0, E (0, T+20
(xii.2)
1- otherwise
(xiii)
<r(et)
(xiv)
(xiv.1)
fr(et)(1 a.) - an if f. (1, ee) E (0, 1420
(xiv.2)
f((0,t;) - otherwise
(Xv)
(xv.1)
pr(e:)(1 + a) + an) if f( ex) E (0, 1+2a
(xv.2)
f(,, otherwise
(xvi)
pi(())

---

Published as a conference paper at ICLR 2023
REFERENCES
Jeffrey Adams, Niels Hansen, and Kun Zhang. Identification of partially observed linear causal models: Graphical conditions for the non-gaussian and heterogeneous cases. Advances Neural Information Processing Systems, 34, 2021.
Board of Governors of the Federal Reserve System (US). Report to the congress on credit scoring and its effects on the availability and affordability of credit. Board of Governors the Federal Reserve System, 2007.
Toon Calders, Faisal Kamiran, and Mykola Pechenizkiy. Building classifiers with independency constraints. In 2009 IEEE International Conference on Data Mining Workshops, pp. 13-18. IEEE, 2009.
Silvia Chiappa. Path-specific counterfactual fairness. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 7801-7808, 2019.
Amanda Coston, Alan Mishler, Edward H Kennedy, and Alexandra Chouldechova. Counterfactual risk assessments, evaluation, and fairness. In Pruceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pp. 582-593, 2020.
Alexander D' Amour, Hansa Srinivasan, James Atwood, Pallavi Baljekar, D Sculley, and Yoni Halpern. Fairness is not static: deeper understanding of long term fairness via simulation studies. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pp. 525-534, 2020.
Danielle Ensign, Sorelle A Friedler, Scott Neville, Carlos Scheidegger, and Suresh Venkatasubrama- nian. Runaway feedback loops predictive policing. In Conference on Fairness, Accountability and Transparency, pp. 160-171. PMLR, 2018.
Yingqiang Ge, Shuchang Liu, Ruoyuan Gao, Yikun Xian, Yunqi Li, Xiangyu Zhao, Changhua Pei, Fei Sun, Junfeng Ge, Wenwu Ou, and Yongfeng Zhang. Towards long-term fairness in recommendation. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, pp. 445-453, 2021.
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning, In Advances in Neural Information Processing Systems, pp. 3315-3323, 2016.
Lily Hu and Yiling Chen. short-term intervention for long-term fairess in the labor market. In Proceedings of the 2018 World Wide Web Conference, pp. 1389-1398, 2018.
Kosuke Imai and Zhichao Jiang. Principal faimess for human and algorithmic decision-making arXiv preprint arXiv:2005. 10400, 2020.
Shahin Jabbari, Matthew Joseph, Michael Kearns, Jamie Morgenstern, and Aaron Roth. Fairness in reinforcement learning. International Conference on Machine Learning, pp. 1617-1626. PMLR, 2017.
Faisal Kamiran, Indré Žliobaité, and Toon Calders. Quantifying explainable discrimination and removing illegal discrimination in automated decision making. Knowledge and Information Systems, 35(3):613 144461444444.1 2013.
Sampath Kannan, Aaron Roth, and Juba Ziani. Downstream effects of affirmative action. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pp. 240-248, 2019.
Michael Kearns and Aaron Roth. The ethical algoritium: The science of socially aware algorithm design. Oxford University Press, 2019.
Niki Kilbertus, Mateo Rojas-Carulla, Giambattista Parascandolo, Moritz Hardt, Dominik Janzing, and Bernhard Schölkopf. Avoiding discrimination through causal reasoning. In Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 656-666, 2017.
Bohdan Kivva, Goutham Rajendran, Pradeep Ravikumar, and Bryon Aragam. Learning latent causal graphs via mixture oracles. Advances in Neural Information Processing Systems, 34, 2021.
45

---

Published as a conference paper at ICLR 2023
Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. In Advances in Neural Information Processing Systems, pp. 4076, 2017.
Lydia Liu, Sarah Dean, Esther Rolf, Max Simchowitz, and Moritz Hardt. Delayed impact of fair machine learning. In International Conference on Machine Learning, pp. 3150-3158. PMLR, 2018.
Vishwali Mhasawade and Rumi Chunara. Causal multi-level fairesss. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pp. 784-794, 2021.
Alan Mishler, Edward H Kennedy, and Alexandra Chouldechova. Fairesss in risk assessment instruments: Post-processing to achieve counterfactual equalized odds. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pp. 386-400, 2021.
Hussein Mouzannar, Mesrob Ohannessian, and Nathan Srebro. From fair decision making to social equality. In Proceedings the Conference on Fairness, Accountability, and Transparency, pp- 359-368, 2019.
Hamed Nilforoshan, Johann D Gaebler, Ravi Shroff, and Sharad Goel. Causal conceptions of faimess and their consequences. In International Conference on Machine Learning, pp. 16848-16887. PMLR, 2022.
Judea Pearl. Causality. Cambridge University Press, 2009.
Shohei Shimizu, Patrik O Hoyer, Aapo Hyvärinen, Antti Kerminen, and Michael Jordan. A linear non-gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7(10), 2006.
Umer Siddique, Paul Weng, and Matthieu Zimmer. Learning fair policies in multi-objective (deep) reinforcement learning with average and discounted rewards. In International Conference on Machine Learning, pp. 8905-8915, PMLR, 2020.
Peter Spirtes, Clark Glymour, and Richard Scheines. Causation, prediction, and search. Springer New York, 1993.
Min Wen, Osbert Bastani, and Ufuk Topcu. Algorithms for fairness in sequential decision making. In International Conference on Artificial Intelligence and Statistics, pp. 1144-1152. PMLR, 2021.
Yongkai Wu, Lu Zhang, Xintao Wu, and Hanghang Tong. Pe-fairness: A unified framework for measuring causality-basse fairness. In Advances Neural Information Processing Systems, pp. 3399-3409, 2019.
Feng Xie, Ruichu Cai, Biwei Huang, Clark Glymour, Zhifeng Hao, and Kun Zhang, Generalized independent noise condition for estimating latent variable causal graphs. Advances in Neural Information Processing Systems, 33:14891-14900 2020.
Feng Xie, Biwei Huang, Zhengming Chen, Yangbo He, Zhi Geng, and Kun Zhang, Identification of linear non-gaussian latent hierarchical structure. In International Conference on Machine Learning, pp. 24370-22444 243872 PMLR, 2022.
Kun Zhang and Aapo Hyvärinen. On the identifiability of the post-nonlinear causal model. In 25th Conference on Uncertainty in Artificial Intelligence (UAI 2009), pp. 647-655. AUAI Press, 2009.
Kun Zhang, Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Kernel-based conditional independence test and application in causal discovery. In 27th Conference on Uncertainty in Artificial Intelligence (UAl 2011), pp. 804-813. AUAI Press, 2011.
Lu Zhang, Yongkai Wu, and Xintao Wu. A causal framework for discovering and removing direct and indirect discrimination. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, pp. 0929939335. 2017.
Xueru Zhang, Mohammadmahdi Khaliligarekani, Cem Tekin, and Mingyan Liu. Group retention when using machine learning in sequential decision making: the interplay between user dynamics and fairesss. Advances in Neural Information P Processing Systems, 12:15269-1527 2019.
46

---

Published as a conference paper at ICLR 2023
Xueru Zhang, Ruibo Tu, Yang Liu, Mingyan Liu, Hedvig Kjellstrom, Kun Zhang, and Cheng Zhang. How do fair decisions fare in long-term qualification? Advances in Neural Information Processing Systems, 33:18457-18469, 2020.
Matthieu Zimmer, Claire Glanois, Umer Siddique, and Paul Weng. Learning fair policies in decen- tralized cooperative multi-agent reinforcement learning. In International Conference on Machine Learning, pp. 12967-12978. PMLR, 2021.
47


Reviews:
[{"review_id": "ymyIqKs6wXJ", "reviewer": "Reviewer_qVYP", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposed a new concept for long-term fairness, called Tier Balancing (TB), under decision-making dynamics. The proposed TB characterizes the decision-distribution interplay with latent causal factors. Given some specific data dynamics, the authors proved that one cannot directly achieve the long-term fairness goal through a single-step decision. This further motivates them to develop a sequence of algorithmic interventions to get closer to the long-term fairness goal. Their method has been examined by both synthetic and real data.", "summary_of_the_review": "Overall, this paper considers an interesting problem of long-term fairness and is well-written. The authors gave a systematical analysis of achieving long-term fairness through single-step intervention v.s. dynamic interventions. Yet, I have a few concerns regarding their connection to the literature, the motivation, and the required strong assumptions.", "strength_and_weaknesses": "*Strength*:\n\n1. This paper considers an interesting problem of long-term fairness and provides a corresponding new definition.\n\n2. The authors gave a systematical analysis of achieving long-term fairness through single-step intervention and dynamic interventions.\n\n3. Numerical studies show their promising prospect.\n\n4. The paper is well written.\n\n*Weaknesses*:\n\n1. The connection to existing fairness works can be further improved. For instance, how about linking the proposed TB to existing fairness metrics mathematically? It would be nice to see it under the non-stationary dynamic that boils down or is close to classical ones.\n\n2. I would expect more motivation of proposing long-term fairness from real examples. The authors may consider expanding the FICO data and elaborating on it in the introduction, instead of simply stating 'in our running example of credit application' without any content.\n\n3. It is not clear if the assumptions stated are reasonable. For instance, in Assumption 3.3, is number 2 in Equation 3 and $\\alpha \\in [0,1/2)$ necessary? Can we change it to other combos, say 3 and 1/3? More justification is also needed for Assumption 3.4 besides using the credit example. The qualitative assumptions in 3.5 and 3.6 should be justified in comparison to the literature.\n\n4. Sensitivity analysis may be considered on assumption violation to enrich the numerical results. \n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "n2Dr5iAo8K", "reviewer": "Reviewer_N3M4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "\nThe paper aims to address long term fairness in decision making systems by using a dynamical approach to model the underlying changes over time. The paper proposes a causal DAG that models the connection between the data (background variable H, sensitive attribute A, observed features X) and a decision D at each time step. Given an unobserved variable H_t (for example: socio-economic status) which is a direct cause of the potential outcome Y^ori and X, the goal is to make H_ and A_t independent. A main contribution of the paper is showing that a one step intervention is insufficient to achieve this goal and you need multiple interventions over time and specific conditions to achieve the long-term fairness goal. \n\n", "summary_of_the_review": "I have updated my score to 6 after the rebuttal below. The authors have addressed my concerns. \n\n-----------------\n\nOriginal summary: This is an interesting paper which addresses an important problem in long term fairness. While the proposal to model the problem as a dynamical problem with a causal DAG and show impossibility results is interesting and useful, the clarity of the paper could be improved. Given the reliance on the DAG, it is hard to evaluate the contribution without a clear justification of the correctness of the graph. \n\n", "strength_and_weaknesses": "Strengths: \n- The addresses an important and under-explored problem of long term fairness. The modelling of long term fairness as a dynamical systems via a causal DAG seems novel and can potentially lead to further interesting work in this direction. \n- Despite the theoretical nature of the paper, the authors make a good effort in using a running example of credit application to illustrate the approach. \n\nWeaknesses:\n- Figure 1: the paper relies on the DAG in figure 1 which shows the causal relations between the variables of interest at each time step: H (hidden socio economic status), D (decision), A (sensitive attribute), Y^ori (\"groundtruth outcome\"), Y^obs (observed outcome), and X (observed features). However, it's not clearly explained what justifies the assumed causal relationship between these variables. For example, why is H_t+1 only updated from ((H_t, Y^(ori)_t, D_t)). If H is socio-economic status, wouldn't it also be dependant on some observed features such as income? Additionally, in paragraph \"Decision-making dynamics\", it says that \"The institution (decision maker) assigns decision D_t to each individual according to the observed features (A_t, X_t,i)  and the outcome record Y^obs\". However this is not reflected in the graph. There is no arrow between Y^obs_t and D. The causal relationship between H_t, A_t and X_t are also not clarified. Overall, the assumptions made in section 2.1 and the graph are not clearly justified.\n- Y^ori vs Y^obs: Y^ori is defined as the \"(unobserved) ground truth label\" and Y^obs label. In the graph, Y^ori is a direct cause of Y^obs. I am not sure how to interpret Y^ori. If Y^ori is the potential outcome, then what does the causal arrow between Y^ori and Y_obs mean? \n- Fact 3.2: it is unclear to me why why the structural equations for D_t, Y_^ori_t and H_t are only dependent on A and a noise term when this is not reflected in the graph. The authors mention that Fact3.2 is a direct result of representing causal relations with a functional causal model but it would be good to explain more how the structural equations relate to the graph.   ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "WZ2qdDJCou", "reviewer": "Reviewer_pPYU", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper is about defining long-term fairness in dynamic settings and characterizing whether it can be achieved with myopic decisions. A specific dynamic model is proposed, where tiers (representing social status) are a root cause of labels (representing social outcomes), and which in turn can be affected by decisions (representing social interventions). Another root cause influencing decisions are the protected attributes (representing historically underserved groups). Long-term fairness is defined as independence of tiers from protected attributes (representing a form of social equality), achieved over some time horizon. This is dubbed tier-balancing. Under several assumptions, the paper determines that (1) such fairness is not generally achievable over a single time step horizon, that (2) a perfect decision (mirroring the true label) worsens fairness, and that (3) counterfactually fair decisions could improve fairness.", "summary_of_the_review": "The long term fairness consideration of this paper remains a timely topic. However, the specific notion adopted here is not novel. More importantly, the paper makes very specific modeling choices, ranging from outright violating the reality of prejudice to making social mobility not reflect the stochasticity in real life.Since the paper hinges on these modeling choices, changing them is not a simple exercise. Furthermore, experiments do not strongly support the analysis. As such, the relevance of this work to the community at large is limited. The paper is further hampered by unclear technical exposition.\n\n[Post discussions: Thanks to the improved mathematical notation in the main text and other clarification, the score is raised. Some key concerns remain, detailed in a remark responding to authors.]", "strength_and_weaknesses": "### Strengths\n\n* Long-term fairness is a very strongly motivated problem and various perspectives on characterizing and achieving it are welcome.\n* Connecting causal and dynamic models in this framework is natural, and the paper’s model has some compelling features that could reflect many practical instances.\n\n### Weaknesses\n\n* The paper claims that the concept of tier balancing as the implicit long-term fairness objective is novel. However, many of the initial long-term fairness papers encoded similar social equality notions, e.g., the Mouzannar et al 2019 reference, where qualifications are equalized across protected groups.\n* The dynamic model that is adopted has some key social implications which are not discussed. Namely, social mobility (tiers shifting) is influenced only by social outcomes and interventions. This is markedly false in real life, where prejudices and privileges dictated by protected groups affect mobility. Unfortunately, this is not a simple fix for this paper, since the introduction of any such dependence would severely change its analysis and conclusions.\n* The dynamic model of the evolution of tiers, Eq. (3), is also too specific. In particular, I put in question the choice of a deterministic evolution, given labels and decisions. The entirety of the analysis hinges on this, as it determines exactly relationships between consecutive time steps, as enumerated case-by-case in the appendix tables. It is not clear how well the conclusions drawn from this determinism would apply to a more realistic stochastic model of social mobility. (A more specific nitpick I have is why mobility is influenced by non-instantiated labels, that is, why depend on $Y^{\\textrm{(ori)}}$ and not on $Y^{\\textrm{(obs)}}$? What I did not exhibit should not affect me.)\n* The technical exposition is generally disorganized and may have some flaws. I had to go back and forth between the main text and appendices to piece together the mathematical formalisms. Here are some issues I encountered:\n   * Most of the analysis is presented in terms of the functional causal model of the dynamic model. Here, at any time $t$, we characterize the tier $H_t$, the label $Y_t$ and the decision $D_t$ as respective functions of the attribute $A_t$ and exogenous randomness $E_t$. This is okay, except that the exogenous randomness must incorporate all the past evolution of society, namely because the current tier depends on them. As such, I believe it is not true that $E_t$ are independent for different $t$, as is mentioned in the appendix (p. 16). \n   * $\\Delta_\\textrm{STIR}$ is not defined before it is characterized. I believe that, at time $t$, the definition is $\\mathbf{E}[|f_{t+1}(0,E_{t+1})-f_{t+1}(1,E_{t+1})|]- \\mathbf{E}[|f_t(0,E_t)-f_t(1,E_t)|]$. This should be stated first and foremost and the dependence on $t$ should be explicit.\n   * $q_t$ is introduced as a conditional density on the pair of tiers (for each attribute). This might lead to understand it as a density with respect to the Lebesgue measure in $\\mathbf{R}^2$, since tiers are real valued. However, this is not the case. This density is rather relative to the measure of the exogenous variables $E_t$, and should indeed always be written as a function of the value the latter takes ($\\epsilon$), as follows: $q_t(f_t(0,\\epsilon),f_t(1,\\epsilon)|d,d’,y,y’                                                                                                                                                                                                               )$. It is worth clarifying this, otherwise the integration in Eq. (5) or the definition on p. 16 don’t make sense.\n* Some of the main results are presented in a way that hides the intuition of why they hold:\n   * Theorem 4.1’s limitations of why we cannot establish tier balance is due to the fact that the only interventions we have to control continuous random variables is a pair of discrete random variables.\n   * Theorem 4.2’s perfect estimator making tier balance worse is due to the fact that the dynamics in Assumption 3.6 are polarizing based on labels only.\n   * As for Theorem 4.3, there seems to be an issue in the proof. When simplifying $\\Delta_\\textrm{STIR}$ by assuming that $\\alpha_Y=0$, it seems that the values of $\\gamma$ (which determine $q_t$) are also set to something specific. If this is true, then it should also be part of the Theorem’s hypothesis. \n* The experiments do not seem to strongly support the analysis. While we see the polarization caused by the perfect predictor in Fig.2, it is not clear whether the counterfactually fair one is any better, as the tiers do not appear balanced in Fig. 3(c).\n\n\n### Question to Authors:\n* I tried to understand Eq. (5). From what I gather, you wanted to not have to outright characterize the time $t+1$ quantities. Instead, you wanted to couple calculating $\\mathbf{E}[|f_{t+1}(0,E_{t+1})-f_{t+1}(1,E_{t+1})|]$ at the same time as $\\mathbf{E}[|f_t(0,E_t)-f_t(1,E_t)|]$, by using the fact that the tier dynamics allow you to know exactly what happens at time  $t+1$, conditionally on the decisions and labels. However, I’m not sure the coupling is done right. Instead of pushing the $E_t$ distribution forward to get $E_{t+1}$, you are forcing them to be the same, and have this secondary integral in $\\xi$, which I am unable to make sense of. Instead, we know that conditionally on the labels, decisions, and $E_t=\\epsilon$ (or equivalently the past tiers), exactly what the future tiers will be, so why not just do:\n$$ \\Delta_\\textrm{STIR}(t) = \\sum_{d,d’,y,y’} P_t(d,d’,y,y’) \\int q_t(f_t(0,\\epsilon),f_t(1,\\epsilon)|d,d’,y,y’) \\cdot (|\\varphi_(t+1)(\\epsilon,d,d’,y,y’)-\\varphi_t(\\epsilon)|) \\mathrm{d} \\epsilon $$\nWhere $\\varphi_(t+1)(\\epsilon,d,d’,y,y’)$ is precisely the update given in the tables in the appendix, which depend only on the past $\\epsilon$ and the decisions and labels $d,d’,y,y’$.\n* In Appendix B.2, you talk about tracking individuals. This is related to the previous point, but I am not sure how you are defining individuals, based on what properties or partitioning of the population? Part of it seems to suggest a kind of probability transport, through which you couple the integrations at both times, but ultimately you are using the fact that the next time step is determined by the first given the decisions and labels. Please clarify.\n\n### Typos/Suggestions:\n- (p.3) since their different roles > because of their different roles\n- (p.5) inquiring > investigating\n- (p.6) 3.2.2 Simplification Assumptions\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "2tyvlMGTWZ4", "reviewer": "Reviewer_WEUm", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a technique to achieve long-term dynamic fairness by incorporating latent factors that can affect the outcomes and also related to the sensitive attributes. The paper also shows that in general it is not possible to achieve long-term fairness only through a one-step intervention, i.e., static decision-making but rather requires sequential decisions in certain situations. The paper also shows that under these conditions it is possible to get closer to the long-term fairness goal through Counterfactual Fair decisions. The empirical results support the theoretical claims.", "summary_of_the_review": "The paper tries to solve the important issue of achieving long-term fairness. I vote for marginally accepting with the hope that clarity is improved.\n\n", "strength_and_weaknesses": "[Strength]\n\nS1: The paper tries to demonstrate how to achieve long-term dynamic fairness\n\n[Weakness]\n\nW1: Writing could be improved; it is difficult to parse through sections 3 and 4 at first glance. I would encourage condensing the notations and adding more motivations for the said approach. \n\nW2: It seems that the conditional independence between the latent factors and the sensitive attributes could fail, for example, in health care or public health setting [1].\n\nW3: Adding more intuition about masking the observed outcomes based on the decisions would help. Moreover, the setup of repeated applications, although relevant in the credit setting, is not very general. When the treatment is designed for improving the outcome, as in healthcare settings, repeated treatments could, in turn, be a negative factor. Adding more settings where this framework applies would be helpful. \n\n\n1. Mhasawade, Vishwali, and Rumi Chunara. \"Causal multi-level fairness.\" Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. 2021.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster