A Local Method for Satisfying Interventional Fairness with Partially Known Causal Graphs

Authors: haoxuan  li
yue  liu
zhi  geng
kun  zhang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 16:26:42

Abstract: Developing fair automated machine learning algorithms is critical in making safe and trustworthy decisions. Many causality-based fairness notions have been proposed to address the above issues by quantifying the causal connections between sensitive attributes and decisions, and when the true causal graph is fully known, certain algorithms that achieve interventional fairness have been proposed. However, when the true causal graph is unknown, it is still challenging to effectively and efficiently exploit partially directed acyclic graphs (PDAGs) to achieve interventional fairness. To exploit the PDAGs for achieving interventional fairness, previous methods have been built on variable selection or causal effect identification, but limited to reduced prediction accuracy or strong assumptions. In this paper, we propose a general min-max optimization framework that can achieve interventional fairness with promising prediction accuracy and can be extended to maximally oriented PDAGs (MPDAGs) with added background knowledge. Specifically, we first estimate all possible treatment effects of sensitive attributes on a given prediction model from all possible adjustment sets of sensitive attributes via an efficient local approach. Next, we propose to alternatively update the prediction model and possible estimated causal effects, where the prediction model is trained via a min-max loss to control the worst-case fairness violations. Extensive experiments on synthetic and real-world datasets verify the superiority of our methods. To benefit the research community, we have released our project at https://github.com/haoxuanli-pku/NeurIPS24-Interventional-Fairness-with-PDAGs.

Main paper: 
 introduction: 
 Making automated machine learning algorithms fair is critical to producing safe and trustworthy decisions with different sensitive attributes [Brennan et al., 2009 , Dieterich et al., 2016 , Agarwal et al., 2018 , Chen et al., 2018 , Chouldechova et al., 2018 , Hoffman et al., 2018 , Yurochkin et al., 2019 , Li et al., 2023] . To achieve fair predictions, association-based and causality-based fairness notions have been proposed. Specifically, the former requires statistical independence between the sensitive attribute and predicted outcome [Dwork et al., 2012 , Hardt et al., 2016 , Chouldechova, 2017 , Jin et al., 2024a,b] , whereas the later investigates causal effect of the sensitive attribute on the predicted outcome, requiring that the predicted outcome be the same across the real-world without intervention and the counterfactual world with intervention on sensitive attribute [Zhang and Bareinboim, 2018 , Zhang et al., 2017a ,b, 2018a ,b, Khademi et al., 2019 , Galhotra et al., 2022] . Despite many algorithms have been developed to achieve causality-based fairness, most of them require the true causal directed acyclic graph (DAG) is fully known [Kusner et al., 2017 , Nabi and Shpitser, 2018 , Chiappa, 2019 , Chikahara et al., 2021] . Nevertheless, true causal DAGs and structural equations are usually not directly available in practice [Colombo et al., 2014] . Moreover, without strong assumptions, e.g., linearity [Shimizu et al., 2006] and additive noise [Hoyer et al., 2008 , Peters et al., 2014] , the true causal DAG may not be recoverable from only the observed data, which raises a great challenge to achieve causality-based fairness based on partially DAGs (PDAGs).To achieve causality-based fairness under partially known causal graphs, recent approaches can be broadly classified into two categories: variable selection methods [Zuo et al., 2022] and causal effect identification methods [Perkovic, 2020 , Zuo et al., 2024] . Specifically, the variable selection method first adopts causal discovery algorithms to obtain a Markov equivalence class of DAGs that encode the same set of conditional independencies from the data, and then classifies the covariate variables into three categories: definite non-descendants, possible descendants, and definite descendants of the sensitive attributes. By noting that a prediction model would be counterfactually fair if the prediction model is a function of the non-descendants of sensitive attributes [Kusner et al., 2017] , these methods proposed to use only the definite non-descendants or further incorporate possible descendants to make fair predictions. Despite can theoretically guarantee interventional fairness, disregarding the descendants results in a notable decline in performance. Another category of methods proposed to identify causal effects of sensitive attributes on the outcome variable directly from the maximally oriented PDAGs (MPDAGs), but relies on strong assumptions for identification.In this paper, we aim to effectively and efficiently achieve interventional fairness with partially known causal graphs. Different from the previous variable selection and causal effect identification methods, we exploit all variables to ensure relatively high prediction accuracy, as well as does not need to rely on additional strong assumptions for identification. Specifically, we propose a novel local method to partially identify the causal effects of sensitive attributes on the predictor for satisfying the interventional fairness, which does not require a global search of all possible DAGs, but can estimate all possible causal effects using the obtained CPDAG. Inspired by the IDA framework [Maathuis et al., 2009] , we first propose a local algorithm to obtain possible parental sets of the sensitive attributes on the PDAGs, from which we estimate all possible propensities for various cases. We then calculate all possible violations of intervention fairness using all possible propensities. Next, we propose to alternatively update the prediction model and the corresponding estimation of the possible causal effects, where the prediction model is trained via a min-max loss to control the worst-case fairness violations. The validity of our method also holds for MPDAGs with added background knowledge.The contributions of this paper are summarized as follows:• We propose a general min-max optimization framework to achieve interventional fairness, which enables to use all variables to achieve relatively high prediction accuracy, and can be extended to MPDAGs with added background knowledge.• Based on the proposed framework, we provide an efficient algorithm to estimate all possible causal effects of sensitive attribute on predictions for MPDAGs.• We further provide a joint learning approach that alternatively updates the prediction model and the corresponding estimation of the possible causal effects, where the prediction model is trained via a min-max loss to control the worst-case fairness violations.• We conduct extensive experiments on synthetic and real-world datasets to demonstrate the effectiveness of our methods in achieving interventional fairness with promising accuracy. 
dags, pdags, cpdags, and mpdags: 
 In a graph G = (V, E), where V and E represent the node set and edge set in G, we say G is directed, undirected, or partially directed if all edges in the graph are directed, undirected, or a mixture of directed and undirected edges, respectively. The skeleton of G is an undirected graph obtained by removing all arrowheads from G. Given a graph G, an X i is called a parent of X j andX j is called a child of X i if X i → X j in G. Also, X i is a sibling of X j if X i -X j in G. If X iand X j are connected by an edge, they are adjacent. The notation pa(X i , G), ch(X i , G), sib(X i , G), and adj(X i , G) respectively represent sets of parents, children, siblings, and adjacent vertices of X i in G. A graph is termed complete if all distinct vertices are adjacent. A path is a sequence of distinct vertices (X k1 , • • • , X kj ) where any two consecutive vertices are adjacent. A path is called partially directed from X k1 to X kj if X ki ← X ki+1 does not occur in G for any i = 1, . . . , j -1. A partially directed path is directed (undirected) if all edges on the path are directed (undirected). A vertex X i is an ancestor of X j and X j is a descendant of X i if there is a directed path from X i to X j or X i = X j . A directed (undirected) cycle is a directed (undirected) path from a vertex to itself. Particularly, a cycle with the number of edges equal to three is called a triangle.In a directed acyclic graph (DAG), all edges are directed and there is no directed cycle. A partially directed acyclic graph (PDAG) may contain both directed and undirected edges without directed cycles. Two DAGs are Markov equivalent if they induce the same set of conditional independence relations [Pearl, 1988] . A Markov equivalence class, denoted by [G] , contains all DAGs equivalent to G. A Markov equivalence class can be uniquely represented by a partially directed graph called completely partially directed acyclic graph (CPDAG) G * , in which two vertices are adjacent if and only if they are adjacent in G, and a directed edge occurs if and only if it appears in all DAGs in [G] [Andersson et al., 1997 , Chickering, 2002a] . A CPDAG G * can be refined to a maximally oriented partially directed acyclic graph (maximal PDAG or MPDAG) H by giving background knowledge B d consisting of some directed causal relationships between variables in the form X i → X j [Hauser and Bühlmann, 2012 , Eigenmann et al., 2017 , Wang et al., 2017 , Rothenhäusler et al., 2018] . Meek [1995] proved that, with a series of orientation rules called Meek's rules, some undirected edges may be further directed (see Algorithm 4 in Appendix for details), and the resulting graph is an MPDAG. Both a DAG and a CPDAG can be viewed as special cases of an MPDAG, where the background knowledge is fully known and unknown, respectively.We follow Pearl [2009] to define the structural causal model (SCM) as a triplet (V, U, F ) to describe the causal relationships between variables. Specifically, V is a set of observable endogenous variables, U is a set of latent independent background variables that cannot be caused by any variable in V , andF is a set of functions {f 1 , . . . , f |V | }, one for each V i ∈ V , such that V i = f i (pa i , U i ), where pa i ⊆ V \ {V i } and U i ∈ U .Notably, the set of equations F induces a directed graph over the variables, here assumed to be a DAG, where the directed causes of V i represents its parent set. A causal DAG model consists of a DAG G and a joint distribution P over V such that the distribution can be factorized as [Perković et al., 2017] .P (v 1 , ..., v |V | ) = vi∈V P (v i |pa(v i , G))Given a DAG G and two distinct variables X and Y , the causal effect of X on Y can be interpreted by the post-intervention distribution of Y intervening on X via do operator [Pearl, 1995 [Pearl, , 2009]] . Formally, given a distribution P (U ) over the background variables U , an intervention do(X = x) that force variable X to take certain value x is defined as the substitution of the structural equation X = f x (pa x , U x ) with X = x, and the post-interventional density of Y is denoted as f (Y = y | do(X = x)). However, if we only know a CPDAG G * or an MPDAG H, the causal effect of X on Y may not be identifiable from observational data [Perković et al., 2015 , 2017 , Wu et al., 2019a,b] .Build on the do operator, interventional fairness criterion [Kilbertus et al., 2017 , Salimi et al., 2019] requires that given the covariates, intervening the value of the sensitive attribute does not affect the output distribution of the output predictor. Formally, let A, Y , and X denote sensitive attributes, outcomes of interest, and other covariates, and Ŷ be a predictor produced by a learning algorithm as a prediction of Y . Without loss of generality2 , we say the predictor Ŷ is interventionally fair with respect to the sensitive attributes A if it satisfies the following condition: Definition 2.1 (Interventional fairness [Kilbertus et al., 2017] ). We say the prediction Ŷ is interventionally fair with respect to the sensitive attributes A if the following holds:P ( Ŷ = y | do(A = a)) = P ( Ŷ = y | do(A = a ′ )),for all possible values of y and any value that A can take. 
motivation and method overview: 
 Given only observational data, the underlying true causal DAG may not be recoverable without strong assumptions such as linearity [Shimizu et al., 2006] or additive noise [Hoyer et al., 2008 , Peters et al., 2014] . Instead, we can use causal discovery algorithms [Spirtes and Glymour, 1991 , Shimizu et al., 2006 , Zhang and Hyvärinen, 2009 , Peters et al., 2014] to obtain a CPDAG that contains that true causal DAG. To exploit the obtained CPDAG for achieving interventional fairness, previous methods have been built on variable selection [Zuo et al., 2022] or causal effect identification [Perkovic, 2020 , Zuo et al., 2024] , but limited to reduced prediction accuracy or strong assumption for identification.Differing from the above work, we propose a novel local method to partially identify the causal effects of sensitive attributes on the predictor for satisfying the interventional fairness. Interestingly, our approach does not require a global search of all possible DAGs, but can estimate all possible causal effects using the obtained CPDAG. In the following, we first theoretically state the necessary and sufficient condition for discriminating a set to be a possible parent set of the sensitive attribute for CPDAG and MPDAG, respectively, from which we propose a local method for finding all possible parent sets of sensitive attributes and estimating the corresponding propensities (Sec. 3.2). We then calculate the all possible degrees of intervention fairness being violated using all possible estimated propensities (Sec. 3.3). Finally, we further propose a min-max joint learning approach to make the predictor satisfy intervention fairness by controlling for worst-case fairness violation (Sec. 3.4).Given a CPDAG obtained from observational data, since enumerating all DAGs is infeasible when the size of the Markov equivalence class is large [He et al., 2015 , Zuo et al., 2022] , we propose to adopt a novel framework called IDA [Maathuis et al., 2009, Fang and He, 2020] to only enumerate possible parental sets of the sensitive attribute. This provides a more efficient solution since enumerating possible parental sets only requires the local structure around the sensitive attribute. We further extend the above theoretical results to MPDAGs with background knowledge added, and estimate possible propensities by regressing sensitive attribute on each possible parental set. Definition 3.1 (v-structure). For three distinct vertices X i , X j and X k , if X i → X j ← X k and X i is not adjacent to X k in G, then the triplet (X i , X j , X k ) is called a v-structure collided on X j . Pearl [2009] have shown that two DAGs are equivalent if and only if they have the same skeleton and the same v-structures. Given a CPDAG G * contains all DAGs equivalent to G and a sensitive attribute A, the local structure around A can be divided into three cases: parents pa(A, G * ) → A, children pa(A, G * ) ← A, and siblings sib(A, G * ) -A with undirected edges. Let S(A) be a subset of sib(A, G * ), we denote G * S(A)→A as a DAG that is obtained from CPDAG G * by changing all undirected edges {Z -A, ∀Z ∈ S(A)} into the directed edges {Z → A, ∀Z ∈ S(A)} as parents, and all of other undirected edges {Z -A, ∀Z ̸ ∈ S(A)} into the directed edges with opposite direction {Z ← A, ∀Z ̸ ∈ S(A)} as children. We say S(A) → A is a possible parental set of the sensitive attribute A for G * , if there exists a DAG G in the equivalence class G * with the same directed edges adjacent to A as G * S(A)→A . Then a sufficient and necessary condition for determining whether a setS(A) ⊂ sib(A, G * ) is a possible parent set of the sensitive attribute A is shown in below. Lemma 3.2 (Maathuis et al. [2009]). Given a CPDAG G * , a set S(A) ⊂ sib(A, G * ) is a possible parent set of the sensitive attribute A, if and only if there is no more v-structure in G * S(A)→A than G * .From the above lemma, given any S(A) ⊆ sib(A, G * ), we can determine whether S(A) is a possible parental set from a local way. In particular, let theinduced subgraph of G = (V, E) over V ′ ⊆ V be the subgraph G ′ = (V ′ , E ′) by restricting the edges E on the set of vertices V ′ , where the edge set E ′ contains all edges with both endpoints in V ′ . Then Lemma 3.2 is equivalent to check whether the induced subgraph of G * over S(A) is complete, i.e., all vertices in the induced subgraph of G * over S(A) are adjacent. This is because if there are two vertices X i and X j in S(A) that are not adjacent, then by Definition 3.1, a v-structure is formed asX i → A ← X j .For MPDAG, a key difference compared with CPDAG is the possible generation of a directed triangular cycle (e.g., A → X i → X j → A) [Fang and He, 2020] , when incorporating the background Update φ(m) by descending along the gradient ∇ φ(m) ℓ( φ(m) ; S (m) );end m) ) and m ← m + 1; end Output: A set S A of possible adjustment sets S (m) and propensity model parameters φ(m) . knowledge and using Meek's rule for orienting undirected edges adjacent to the sensitive attribute A. Motivated by such difference, we entend the theoretical results on CPDAGs to MPDAGs for determining possible parental sets of A, which can also be implemented via a local way. Definition 3.3 (Direct triangle structure). For three distinct vertices X i , X j and X k , if (Fang and He [2020] ). Given an MPDAG H, a set S(A) ⊂ sib(A, H) is a possible parent set of A, if and only if there is no more direct triangle structure and v-structure in H S(A)→A than H.S A ← S A ∪ (pa(A, H) ∪ S (X i → X j → X k → X i , then the triplet (X i , X j , X k ) is called a direct triangle structure. Lemma 3.4From Lemma 3.4, we can conclude that for a given MPDAG H, it is equivalent to checking whether the induced subgraph of H over S(A) is complete, as well as there does not exist S ∈ S(A) and C ∈ adj(A, H)\ (pa(A, H) ∪ S(A)) such that C → S, otherwise a direct triangle structrue is formed as A → C → S → A. This provides a efficient way to locally find the possible parental sets of A. Without loss of generality, we denote the set of possible parental sets of A with a total number M asS A = pa(A, H) ∪ S (1) (A), pa(A, H) ∪ S (2) (A), . . . , pa(A, H) ∪ S (M ) (A) .Next, to estimate P ( Ŷ = y|do(A = a)) in the interventional fairness notion, for each possible parental set pa(A, H) ∪ S (m) (A) with m = 1, . . . , M , we propose to first estimate the corresponding propensity P (A | pa(A, H) ∪ S (m) (A)). Specifically, we use the covariates X restricted on pa(A, H) ∪ S (m) (A), denoted as X| pa(A,H)∪S (m) (A) , and train the propensity modelg(X| pa(A,H)∪S (m) (A) ; φ(m) ) for estimating propensity P (A | pa(A, H) ∪ S (m) (A)) by minimizing ℓ( φ(m) ) = - 1 N N i=1 A i log g(X| pa(A,H)∪S (m) (A) ) + (1 -A i ) log 1 -g(X| pa(A,H)∪S (m) (A) ) ,where φ(m) is the propensity model parameter and ê(m) i = g(x i | pa(A,H)∪S (m) (A) ; φ(m) )is the estimated propensity of unit i corresponding to the possible parent set pa(A, H) ∪ S (m) (A) for i = 1, . . . , N and m = 1, . . . , M . Since CPDAGs can be viewed as special cases of MPDAGs, without loss of generality, we summarized the proposed local algorithm on MPDAGs in Alg. 1 (see Alg. 3 in Appendix for implementing the proposed local algorithm on CPDAGs).We then aim to estimate and bound all possible causal effects of sensitive attribute A on the predictor Ŷ . Note that each parental set pa(A, H) ∪ S (m) (A) is a valid back-door adjustment set in the back-door adjustment formula [Pearl, 1995 [Pearl, , 2009]] , we have the following identification results. Lemma 3.5. With observational data, for m ∈ {1, . . . , M }, if Ŷ / ∈ pa(A, H) ∪ S (m) (A)3 , then the post-intervention distribution can be calculated from the observational data by:P ( Ŷ = y|do(A = a)) = P Ŷ = y|A = a, pa(A, H) ∪ S (m) (A) dP pa(A, H) ∪ S (m) (A) = P Ŷ = y, A = a|pa(A, H) ∪ S (m) (A) P A = a|pa(A, H) ∪ S (m) (A) dP pa(A, H) ∪ S (m) (A) ,whereP A = a|pa(A, H) ∪ S (m) (A) is estimated via g(X| pa(A,H)∪S (m) (A) ; φ(m) ) in Sec. 3.2.Proof of Lemma 3.5.P ( Ŷ = y|do(A = a)) = P Ŷ = y|do(A = a), pa(A, H) ∪ S (m) (A) dP pa(A, H) ∪ S (m) (A) = P Ŷ = y|A = a, pa(A, H) ∪ S (m) (A) dP pa(A, H) ∪ S (m) (A) = P Ŷ = y, A = a|pa(A, H) ∪ S (m) (A) P A = a|pa(A, H) ∪ S (m) (A) dP pa(A, H) ∪ S (m) (A) ,where the first and the third equations are from the conditional probability formula. ℓ( ψ(m) ) = 1 N N i=1 A i f (x i ; θ) ê(m) i - (1 -A i )f (x i ; θ) 1 - ê(m) i -h(x i ; ψ(m) ) 2 ,where f (x i ; θ) = Ŷi is an outcome predictor parameterized by θ, and h(x i ; ψ(m) ) = τ (m) i aims to evaluate the interventional fairness violation of that learned predictor.We now aim to train a predictor to satisfy interventional fairness. Since the parental set of the sensitive attribute in the true DAG is unknown, we propose a min-max learning approach to control for the worst-case interventional fairness violations of the predictor. Specifically, given all possible causal effects τ (m) i of the sensitive attribute A on the predictor Ŷ in Section 3.3, the prediction model Ŷ = f (x; θ) is trained by minimizing the average prediction error with the worst-case violations of interventional fairness as a penalty term Update θ by descending along the gradient of min-max loss ∇ θ ℓ(θ; ψ(1) , . . . , ψ(M) );min θ ℓ(θ; ψ(1) , . . . , ψ(M) ) = 1 N N i=1 (Y i -f (x i ; θ)) 2 + γ • max m 1 N N i=1 ξ (m) i , s.t. τ (m) i ≤ C + ξ (m) i , i = 1, . . . , N, m = 1, . . . , M, τ (m) i ≥ -C -ξ (m) i , i = 1, . . . , N, m = 1, . . . , M, ξ (m) i ≥ 0, i = 1, . . . , N, m = 1, . . . , M,end end which is a convex optimization problem when τ (m) i = h(x i ; ψ(m) ) is linear. It is equivalent to min θ l(θ) = 1 N N i=1 (Y i -f (x i ; θ)) 2 + λ • max m 1 N N i=1 (-C - τ (m) i ) + + (τ (m) i -C) + ,where γ and λ are hyper-parameters for trade-off between prediction accuracy and interventional fairness. Since achieving strict interventional fairness for all individuals, i.e., having zero causal effects of sensitive attribute on the predictor, is usually unrealistic and would come at the cost of much prediction accuracy, we introduce a slack variable ξ (m) ifor each individual and a pre-specified threshold C, which penalizes the loss when the estimated causal effect |τ (m) i | > C. Note that when implementing the proposed min-max optimization approach, the treatment effect estimation models for evaluating the fairness violations in Section 3.3 and the prediction model controlling for worse-case fairness violations in Section 3.4 should be updated alternatively, which can be viewed as an iterative process of interventional fairness evaluation and policy improvement of the prediction model. We summarized the whole min-max optimization algorithm in Alg. 2. 
experiments: 
 In this section, both synthetic and real-world experiments are conducted to evaluate the prediction accuracy and fairness of our approach. The root mean squared error (RMSE) between Y and Ŷ is used to measure the prediction performance, and the RMSE between Ŷ |do(A = a) and Ŷ |do(A = a ′ ) is used to measure the violation of the interventional fairness, named "unfairness".Baselines. We consider six baseline prediction models: (1) Full uses all attributes, (2) Unaware uses all attributes except the sensitive attribute, (3) Oracle uses all attributes that are non-descendants of the sensitive attribute given the ground-truth DAG, (4) FairRelax uses all definite non-descendants and possible descendants of the sensitive attribute in a CPDAG (or an MPDAG), ( 5) Fair uses all definite non-descendants of the sensitive attribute in a CPDAG (or an MPDAG), and (6) ϵ-IFair uses all attributes and implement the constrained optimization problem.Synthetic Study. Synthetic data are generated from a linear structural equation model based on a ground-truth DAG. Specifically, we first randomly generate a DAG with d nodes and 2d directed edges according to the Erdős-Rényi (ER) model with d ∈ {10, 20, 30, 40} in our experiment. Following the previous studies [Zuo et al., 2022 [Zuo et al., , 2024]] , the path coefficients β jk of directed edges X j → X k are sampled from a uniform distribution U([-2, -0.5] ∪ [0.5, 2]). The data are generated using X k = Xj ∈pa(X k ) β jk X j + ϵ i , i = 1, . . . , n, where pa (X k ) represents the parent nodes of X k , noise ϵ i ∼ N (0, γ) with γ ∈ {1.5, 2.5}, and n is the sample size, which is set to 1,000 in our experiment. Next, we use the PC algorithm in the causal-learn package to learn a CPDAG. Then we randomly select two nodes as the outcome Y and the sensitive attribute A, respectively. We sample A from a Bernoulli distribution with probability σ( Xj ∈pa(A) β jA X j + ϵ i ), where σ(•) denotes the sigmoid function. The proportion of training data and test data are set to 0.8 and 0.2, respectively.Performance Comparison. Table 1 shows the results of baselines and our approach. First, Full and Unaware perform better on RMSE, while Fair, FairRelax, ϵ-IFair, and our approach have a significant advantage on unfairness. Note that our approach outperforms Fair, FairRelax, and ϵ-IFair in all scenarios on both RMSE and unfairness metrics, because the proposed method makes predictions with all attributes and controls unfairness by the adjustment sets, whereas Fair and FairRelax can hardly find the true descendants of the sensitive attribute and ϵ-IFair can hardly find the true causal effects when the learned CPDAG is not accurate in practice. In addition, Figure 1 shows the change in RMSE and unfairness as C and λ increase. When C is increasing, RMSE is decreasing significantly, while unfairness is increasing. Because the larger C is, the looser the control of causal effects, which is beneficial for prediction performance but hurts fairness. Similar arguments hold for λ, where a larger λ will increase the cost of fairness violations in the optimization problem, thus benefiting fairness but hurting prediction accuracy.MPDAG with Background Knowledge. Given the CPDAG, we randomly select a certain percentage of the directed edges in the true DAG as background knowledge and impose it on the already learned CPDAG. For example, if A → B is selected from the true DAG, we add this directed edge to the learned CPDAG regardless of the original relationship between A and B in the CPDAG to obtain an MPDAG and then adjust the MPDAG according to the Meek's rule. Figure 2 shows the effect of background knowledge on performance. As the background knowledge increases, the RMSE of Fair and FairRelax increases and the unfairness decreases significantly because more background knowledge forces Fair and FairRelax to have fewer nodes to make predictions. For our approach and ϵ-IFair, both prediction and unfairness performance become better as the background knowledge ratio increases, which is attributed to the more accurate identification of the possible adjustment sets and the causal effect. Note that our approach stably outperforms ϵ-IFair under varying background knowledge ratios. In addition, Table 2 reports the change of precision and recall for finding adjustment sets with increasing background knowledge ratio.Effect of Different Number of Parent Node. We evaluate the RMSE and unfairness performance with varying numbers p of parent nodes of the sensitive attribute. The results are shown in Table 3 . As p increases, due to the presence of more backdoor paths, the RMSE performance and unfairness performance of all methods decreases, however, our method still outperforms the baseline methods.The Performance on the Classification Problem. We conduct more experiments to examine if the proposed method can be applied to classification problems. Specifically, we modify the data generation process (DGP) to clip the outcome variable Y to 1 if Y > 0, and to 0 if Y ≤ 0, and the rest DGP remains the same. In this scenario, we adopt AUC as the evaluation metric instead of RMSE. The experiment results are shown in Table 4 . We find that both Full and Unaware perform better on AUC, while Fair, FairRelax, ϵ-IFair, and our approach perform better on unfairness. Note that our approach outperforms Fair, FairRelax, and ϵ-IFair in all scenarios for both AUC and unfairness.Case Study. The sensitive attributes contained in many widely used datasets for fair machine learning have no parent nodes, such as sex in the Adult dataset4 [Kohavi, 1996] and race in the COMPAS dataset5 [Angwin et al., 2022] . Because sex, race, and age cannot be affected by other collected features, we further consider the Open University Learning Analytics Dataset (OULAD) dataset6 [Kuzilek et al., 2017] , in which disability is treated as the sensitive attribute and final grade is treated as the outcome. The COMPAS dataset contains 6,172 units with 9 attributes such as gender and number of priors, the Adult dataset contains 48,842 units with 14 attributes such as age, education, and race, and the OULAD dataset contains 32,593 units with 11 attributes including demographic information such as gender, age, education level, etc. For this case study, we first learn a CPDAG from the raw data using the PC algorithm in the causal-learn package and obtain an MPDAG with the background knowledge such as sex can not be caused by other attributes. Second, we randomly generate a DAG as the ground-truth from the learned MPDAG. After obtaining the DAG, we then divide the data into 100 random batches, and for each batch, we learn an MPDAG from the observed data and background knowledge. The path coefficients are determined based on linear regression and regard the residual of the regression as noise. The subsequent steps are the same as in the synthetic study. The experiment results are shown in Table 5 , with density plots in Figure 3 . First, both ϵ-IFair and our method demonstrate the superiority of our approach in both prediction performance and fairness compared to other baselines. In addition, our method stably outperforms ϵ-IFair, further validating the effectiveness of the proposed min-max joint learning approach. 
conclusion: 
 This paper aims to achieve interventional fairness from observational data when the causal graph is unknown or partially known. Interestingly, we show it is actually sufficient to enumerate all possible parental sets of the sensitive attributes via a local approach, instead of enumerating all DAGs at high computational cost. We then propose a general min-max optimization framework to achieve interventional fairness that is easy applicable to CPDAGs and maximally oriented PDAGs (MPDAGs) with the added background knowledge. One limitation of our approach is due to the proposed approach relying on a CPDAG given by the causal discovery algorithm and estimations of the propensities, which may lead to mild violations of interventional fairness when the CPDAG or estimates are inaccurate. Another possible limitation, which also serves as a future research direction, is to achieve interventional fairness in the presence of hidden variables with partially known DAGs. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: The experimental details are presented in the experiment section.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: The experimental results are reported with error bars.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments? 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: We present the broader impact in Appendix.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? 
 
 Appendix: This paper proposes a general min-max optimization framework that can effectively achieve interventional fairness when the true causal graph is unknown or partially known. In contrast to statistical fairness, interventional fairness considers possible counterfactual decision-makings, not just based on the observed data. The implications of achieving interventional fairness in algorithms where the true causal graph is unknown or partially known are mainly in the following aspects. First, reducing bias: machine learning models may learn and reflect bias in the training data and unconsciously apply this bias to individuals in their predictions. interventional fairness helps reduce this possible bias. Second, improve fairness: if we can achieve interventional fairness on partially known causal graphs, our models will be prevented from treating unfairly because of the sensitive attributes. Third, enhancing trust: as our algorithms are able to process data in a fairer way, people's trust in those algorithms increases. This is critical in many areas, such as healthcare, finance, and justice. Fourth, promote policy making: understanding and addressing interventional fairness issues in algorithms can help policy makers better understand and regulate these technologies to ensure their fairness and transparency in practice. In a nutshell, studying how to effectively achieve interventional fairness in scenarios such as unknown causal graphs or the presence of hidden variables is both challenging and socially significant, and deserves more effort.To tackle the above problem, a recent work [Zuo et al., 2022] proposes to use observed data to first classify variables into three categories: definite non-descendants, possible descendants, and definite descendants of the sensitive attributes. Next, by noting that a prediction model would be counterfactually fair if the prediction model is a function of the non-descendants of sensitive attributes [Kusner et al., 2017] , as shown in Table 6 , FAIR method is proposed to use only the definite non-descendants, and FAIRRELAX method further incorporates possible descendants.Table 6 : Comparison of methods to achieve interventional fairness from PDAGs. Both FAIR and FAIRRELAX employ a two-stage approach: they first learn a CPDAG from observed data, and then make prediction with the definite non-descendants (and possible descendants) of the sensitive attribute.Our method alternatively updates the predictions using all variables and possible counterfactual treatment effects via a min-max optimization.Variables (example in Figure 4 However, both FAIR and FAIRRELAX forbid the use of definite descendants for model prediction, which greatly compromises the prediction accuracy. In particular, the sensitive attribute is usually an inherent nature of data, making most of the attributes are its descendants [Wu et al., 2019a] . class of DAGs that encode the same set of conditional independencies from the data, also known as a completely partially directed acyclic graph (CPDAG), as shown in Figure 4 (b). One on hand, the definite non-descendants of sensitive attribute A is a empty set, thus FAIR is unable to return valid prediction model. On the other hand, the possible descendants of the sensitive attribute A are {X 1 , X 2 }, thus FAIRRELAX uses both X 1 and X 2 to predict Y by minimizing the empirical risk without imposing any further fairness constraint. However, such relaxation would lead to a serious violation of interventional fairness, due to the nodes used (X 1 and X 2 in this example) for outcome regression might be descendants of the sensitive attribute A in the true DAG, as in Figure 4(a) .When the true causal graph is unknown, to the best of our knowledge, Zuo et al. [2022] performed the first work to obtain an interventional fairness predictor on an MPDAG, which focuses on utilizing the properties of the causal graph (Level 1 in Kusner et al. [2017] ) -to make predictions with the definite non-descendants (and possible descendants) of the sensitive attribute, as shown in Table 6 . However, further incorporating the descendants of sensitive attribute into the predictor may also achieve interventional fairness by "cancelling out" the counterfactual treatment effects, which utilizes the observed variables more sufficiently and further improves the accuracy of the prediction.We provide an intuition for the rationality and the advantages of using all variables by adopting the toy example in Figure 4 . Suppose the structural equations in Figure 4 (a) are: A = U A , X 1 = A + U 1 , X 2 = A + U 2 , and Y = 2X 1 + X 2 + U Y , which satisfies the faithfulness assumption [Uhler et al., 2013] . In such a case, as discussed before, the FAIR algorithm proposed in Zuo et al. [2022] prevents all variables from predicting Y , while the FAIRRELAX algorithm uses both X 1 and X 2 to predict Y without imposing any fairness constraints, and therefore cannot achieve counterfactual unfairness, since X 1 and X 2 are descendants of A. To achieve more accurate predictions with interventional Answer: [Yes] Justification: The codes are provided in the supplemental material and the used dataset is public available.• The answer NA means that paper does not include experiments requiring code.• Please see the NeurIPS code and data submission guidelines (https://nips.cc/ public/guides/CodeSubmissionPolicy) for more details. • While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.Justification: There is no requirement for the experiments, and we only use the laptop CPU to run all experiments.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).Justification: There is no ethical problem with our code.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).Justification: There is no such risks about our paper.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We have included CC-BY 4.0 license.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.Note that our proposed method can be naturally extend to interventional fairness with admissible attributes X ad ⊆ X[Salimi et al., 2019], defined as P ( Ŷ = y | do(A = a), do(X ad = x ad )) = P ( Ŷ = y | do(A = a ′ ), do(X ad = x ad )), by observing that do(A) and do(X ad ) are symmetric and including X ad into A.It is worth noting that this assumption always holds, since the training of the predictor Ŷ cannot affect the origin sensitive attribute A.https://archive.ics.uci.edu/dataset/2/adulthttps://www.kaggle.com/datasets/danofer/compasshttps://www.archive.ics.uci.edu/dataset/349/open+university+learning+analytics+dataset 


Decision: The authors proposed a method for achieving interventional fairness when the causal graph is unknown.
The proposed method enumerates only those equivalent PDAGs that affect interventional fairness among the equivalent causal DAGs, and then trains a model to minimize the worst-case fairness within these equivalent PDAGs.
Through discussions with the reviewers, the authors clarified the effectiveness of the proposed method and its broad applicability.
Although there were comments from the reviewers regarding the differences in assumptions between the proposed method and existing methods, as well as their performance comparison, the authors responded to these points appropriately. Incorporating the content of these discussions into the manuscript would further improve the quality of the paper.

Reviews: review: paper_summary: paper_summary: This paper aims for interventional fairness with sufficient prediction accuracy by employing a min-max optimization framework. The proposed approach aims for partially directed acyclic graphs (PDAGs) and extends itself to maximally oriented PDAGs (MPDAGs). Finally, the approach is evaluated on synthetic and real-world datasets.
strength_weakness: strength_weakness: Below I share some weaknesses.

* Above line 188: It is unclear what $S^{(i)}(A); 1<=i<=M$ refers to. The authors should explain the superscripts.
* Equations should be numbered. The Equation after line 192 should be explained in more detail.
* Line 201: The authors should provide some intuition/explanation about the backdoor adjustment formula.
* It seems that there are no baselines based on prior works. Is there no work that solves the fairness problem with causal effect estimation?
questions: questions: Below I provide my questions:
* Line 178: why can it not be $X_i \leftarrow A \rightarrow X_j$?
* What is the computational complexity of the proposed approach?
* How can the proposed approach be connected with the min-max approach proposed by Xia et al [1] and the invariant prediction problem proposed by Subbaswamy et al [2]


[1] Xia, Kevin, et al. "The causal-neural connection: Expressiveness, learnability, and inference." Advances in Neural Information Processing Systems 34 (2021): 10823-10836.\
[2] Subbaswamy, Adarsh, Peter Schulam, and Suchi Saria. "Preventing failures due to dataset shift: Learning predictive models that transport." The 22nd International Conference on Artificial Intelligence and Statistics. PMLR, 2019.
limitations: limitations: The authors properly discussed their limitations.
paper_summary: paper_summary: This work investigates interventional fairness given partially known causal graphs. Compared to existing methods, it employs all variables and does not need to rely on additional strong assumptions for identification. Specifically, it offers a min-max optimization framework which produces counterfactual fairness in maximally oriented PDAGs. Experiments on synthetic and real-world datasets showcase the efficacy of the proposed method.
strength_weakness: strength_weakness: Overall, the paper is well-composed, substantiated by robust experimental outcomes and comprehensive theoretical analysis. However, some sections could benefit from further clarification to enhance understandability. Specific points of ambiguity are addressed below:

•	The statement “One may notice that a function of X1 − X2 can be used to predict Y” appears somewhat abrupt and disrupts the flow of reading. A detailed explanation or a graphical illustration might clarify the independence between U1 − U2 and A.

•	At line 165, the rationale behind transforming certain undirected edges into directed edges in the opposite direction remains unclear. Elaboration on this would be beneficial.

•	The term ‘direct causal information set’ warrants a clear, explicit definition within the main text to avoid potential misunderstandings.


---

Thank you very much for providing thorough answers to my queries, which have satisfactorily addressed my concerns. I am pleased to adjust my score.
questions: questions: See the weakness above.
limitations: limitations: The limitation has been well discussed in the conclusion part.
paper_summary: paper_summary: In this paper, the author(s) talk about a better way to estimate interventional fairness in the case where we have partially known Directed Acyclic Graphs (DAG). It lists out how most previous works have only done this for fully known DAG's and recent attempts to address the case of unknown graphs have utilized strong assumptions and or reduced prediction accuracy. 
The authors introduce  a min-max optimization framework to achieve interventional fairness and boost accuracy and test it on both generated data and real-world data.
strength_weakness: strength_weakness: The paper's approach relies heavily on the accuracy of the CPDAG and on the generated propensity scores. The paper could address measures to address interventional fairness in the case of inaccuracies coming from the CPDAG
questions: questions: CPDAG was used on line 57 but was but the full meaning was not given until line 96.
limitations: limitations: The author brings it up clearly in their conclusion that their method relies on Completely Partially Directed Acyclic Graph (CPDAG) and on the propensity scores returned and therefore any errors in these can affect their method.
Also, the case where we have latent variables or confounders was not addressed.
paper_summary: paper_summary: This paper develops a method for satisfying interventional fairness when the causal graph is partially known. The paper first develops a method for checking possible parental sets as well as a method for estimating propensities given a possible parental set. Then, the interventional distribution is computed using the parental set as a back-door adjustment set. Finally, a min-max joint learning approach is proposed to learn a fair model based on the worst-case violation of interventional fairness.
strength_weakness: strength_weakness: This paper presents a solid work. The main reason I am inclined towards rejection is the applicability of this work. The proposed method relies on the existence of parent nodes of the sensitive attribute. However, as also mentioned in the paper, in most datasets the sensitive attributes have no parent nodes. To be more specific, most sensitive attributes have no parents in nature. For example, common sensitive attributes like sex/gender, race, nationality, etc., are determined at birth. As a result, the proposed method is applicable to very limited sensitive attributes like disability, which significantly limits the importance of this work. I recommend that the authors may apply the proposed method to more general causal inference scenarios where the treatment node may generally have parent nodes.
questions: questions: In the experiments, the accuracy is measured by RMSE. Can the proposed method be applied to classification problems?

===============post-rebuttal comments============

The authors partially addressed my concerns, and I value the contributions made by the paper. I have raised my score.
limitations: limitations: N/A

Comments: title: We would like to summarize our efforts and changes during rebuttal as follows.
We are happy that your concerns have been addressed. Thanks for raising your score!
Thanks for raising your score to "Strong Accept"!
Thank you for agreeing with our rebuttal and for the clarification on misclicking the ethical review.
comment: Dear Reviewer GJNz,

Since the discussion period will end in a few hours, we will be online waiting for your feedback on our rebuttal, which we believe has fully addressed your concerns.

We understand that you are very busy, but we would highly appreciate it if you could take into account our response when updating the rating and having a discussion with AC and other reviewers.

Thanks for your time,

Submission18019 Authors
- **In the degenerated case, we conclude that Fair [1] and FairRelax [1] perform poorly in both accuracy and fairness.**
  - On one hand, they only use partial variables, not all, which reduces their accuracy performance.
  - On the other hand, after selecting the definite descendants (and possible descendants) of the sensitive attribute, they does not impose any causal fairness constraint, which reduces their fairness performance.
  - If the sensitive attribute is a root node in the causal DAG, most of the variables would be identified as the definite descendants (and possible descendants) via the causal discovery algorithms, which increases the availability of more variables and makes the unfair problem be more serious.
  - We also validate such claims on our added experiments on the Adult and the COMPAS datasets during rebuttal.
- **In the degenerated case, both of the $\epsilon$-IFair [2] and our approaches can work well, but the time complexity for implementing $\epsilon$-IFair [2] in much larger than that of our approach.**
  - This is because $\epsilon$-IFair [2] enumerates all subclasses of DAGs where the causal effect is identifiable, and when the sensitive attribute is the root node, the causal effect is naturally identifiable, so $\epsilon$-IFair [2] enumerates almost all possible DAGs.
  - For our local method, we only enumerate all possible parental sets of the sensitive attribute, whose enumerating space is much smaller than that of $\epsilon$-IFair [2], especially when we have plenty nodes in the real-world datasets.

***

**We sincerely hope that the reviewer can carefully go over our discussion above -- we really spent a lot of time trying to make it clearer!** We are also very grateful for the multiple rounds of communication with you, as this level of engagement has significantly contributed to improving the quality and positioning of our work (in the extensive fairness literature).

As the discussion deadline approaches, please let us know if you have any further feedback. We truly appreciate your beneficial comments as well as your consideration to upgrade your score -- thank you so much!!

***

**References**

[1] Counterfactual fairness with partially known causal graph. NeurIPS, 2022.

[2] Interventional fairness on partially known causal graphs: A constrained optimization approach. ICLR, 2024.
First, we clarify that **methods for achieving fairness notions can be divided into the following three categories.**

|Methods| Input(s) | Fairness notions | Limitation |
|:--:|:--:|:--:|:--:|
|achieve correlation-based fairness | observational data | demographic parity, risk ratio, equal opportunity, etc. | **cannot achieve causal fairness** |
|achieve causal fairness **with** a known causal DAG | observational data, **a known causal DAG** |interventional fairness, counterfactual fairness, path-specific counterfactual fairness, etc. | **need strong prior knowledge for a known causal DAG** |
|achieve causal fairness **without** a known causal DAG (ours) | observational data | interventional fairness, counterfactual fairness, path-specific counterfactual fairness, etc. | **with observational data, we can only know the Markov equivalence class, not the unique DAG** |
||

**1. After careful thinking, we respectfully believe that it is not fair to compare our approach with methods for achieving correlation-based fairness in our degenerated case (in which the causal is the same as correlation), due to the following reasons.**

  - For methods achieving correlation-based fairness, **their applicability usually fails to achieve causal fairness.** However, **our approach can achieve causal fairness with general applicability,** where it degenerates to correlation-based fairness only in a specific case (in which the sensitive attribute is a root node in the causal DAG).
  - The most important point, from the table above, is **how do we determine if the sensitive attribute is the root node in the true causal DAG when we only have the observational data?**
- We kindly invite the reviewer to refer to a toy example in Figure 4 on page 13 of our manuscript. Specifically, Figure 4(a) is the true causal DAG, in which the sensitive attribute is a root node. However, as shown in Figure 4(b), **with only observation data, we cannot distinguish from the following causal relations: $X_1 \leftarrow A \to X_2$, or $X_1 \to A \to X_2$, or $X_1 \leftarrow A \leftarrow X_2$,** because they are all in the same Markov equivalence class! But **only the first causal relation** $X_1 \leftarrow A \to X_2$ corresponds to the case where **the sensitive attribute is a root node!**

**2. Meanwhile, we agree with the reviewer that it is fair to compare our approach with methods for achieving causal fairness _without_ a known causal DAG, in terms of factors like effectiveness, applicability, etc., especially at the degenerated case.**

**To the best of our knowledge, Fair [1], FairRelax [1], and $\epsilon$-IFair [2] are all the existing prior works focusing on the same problem** (causal fairness with partially known causal graph). To clarify the **core idea, key assumptions, limitations, sensitivity to wrong CPDAG, and how they perform in the degenerated case in which the sensitive attribute as the root node**, we summarize our findings as follows.

|Methods| Core idea | Key assumptions | Limitations | Sensitivity to wrong CPDAG | Sensitive attribute as the root node |
|:--:|:--:|:--:|:--:|:--:|:--:|
|Fair [1] and FairRelax [1] | two-phase approach: first obtains a CPDAG from causal discovery methods, then uses definite descendants (and possible descendants) of the sensitive attribute to train a fair model | a correct CPDAG, and **very strong assumption that the sensitive attribute is a root node for identification** | only uses partial variables, instead of all variables, may **significantly reduce the prediction accuracy** | **very high**, the effectiveness of such approaches completely rely on the correctness of the causal discovery algorithm to find the correct CPDAG | **poor fairness performance**, because the second phase does not impose any causal fairness constraint | 
|$\epsilon$-IFair [2] | a **global approach** by enumerating **all subclasses of DAGs where the causal effect is identifiable** | a correct CPDAG, and **strong assumptions that there is no undirect edge between sensitive features and non-sensitive features** | **high time complexity** | **weaker** than that of Fair [1] and FairRelax [1] (see Figure 1 in supplementary PDF) | causal effect is always identifiable in such a case, and **enumerating all DAGs globally needs much time** |
| Ours | a **local approac**h by enumerating **all possible parental sets of the sensitive attribute** | a correct CPDAG **(with no other strong assumptions, because we intend to do partially identification, not fully identification as in [1] and [2])** | **need a extra propensity model** for fair training | **weaker** than that of Fair [1] and FairRelax [1] (see Figure 1 in supplementary PDF) | causal effect is always identifiable in such a case, and **enumerating locally needs less time** |
||
I apologize for the late response. 

I agree with the authors' statement: if the sensitive attribute $A$ is a root node in the causal DAG, then there will be no backdoor path from the sensitive attribute $A$ to the outcome variable $Y$. In this case, the interventional fairness will degenerate to the correlation estimands.

Could the authors elaborate a bit more on the advantages of the proposed method in the degenerated case mentioned above? As we know, common correlation-based fairness notions include demographic parity, risk ratio, equal opportunity, etc., and much research has been conducted to achieve these fairness notions. I wonder how the proposed method compares to these existing approaches in terms of factors like effectiveness, applicability, etc.
Dear Reviewer GJNz,

Once again, we are grateful for your time and effort for reviewing our paper. Since the discussion period will end in around a day, we are very eager to get your feedback on our response. We understand that you are very busy, but we would highly appreciate it if you could take into account our response when updating the rating and having a discussion with AC and other reviewers.

We are encouraged by your kind words supporting the **Originality, Quality, Clarity, and Significance** of our work. Meanwhile, it seems that your only concern currently is around the **Applicability** of our method. To facilitate checking, we are happy to summarize our efforts during rebuttal as follows.

- **We theoretically showed that our proposed method can also works well in scenarios where sensitive attributes are as root nodes.**

- **We added experiments on two widely used real-world datasets: the Adult and the COMPAS datasets, where the sensitive attributes are "sex" and "race" as the root nodes, respectively.**

- **We claimed that our proposed method can also be applied to the classification problems, and highlighted that the current real-world experiment is conducted on the OULAD dataset, where the outcome variable is binary.**

- **As suggested by the reviewer, we conduct more synthetic experiments using AUC as the evaluation metric instead of RMSE.**

- **Benefiting from the reviewer, we have changed all _Lemmas_ to _Propositions_ to improve the presentation clarity.**

- (Reviewer BRmE) **We added more simulation experiments to explore the effect of the inaccuracy of CPDAG on the performance of our method, as shown in Fig. 1 in the Supplementary PDF.**

***

As the discussion deadline approaches, we are wondering whether our responses have properly addressed your concerns? Your feedback would be extremely helpful to us. If you have further comments or questions, we hope for the opportunity to respond to them.

Many thanks,

Submission18019 Authors
We're glad our rebuttal addressed your concerns. Thank you for your helpful comments -- it helps make our manuscripts easier to follow. May I know if our clarifications can improve your ```Confidence: 2``` in your review? We will definitely put our discussions into our final version -- thank you so much!!
Thank you for the rebuttal. The authors have addressed my concerns. Also, after reading other reviewers' responses, I have decides to increase my score.
We thank the reviewer for helping us to improve the presentation clarity of our manuscript -- we will definitely incorporate all the clarifications into our final version. We are happy that you are willing to improve your score from 7 to 8. Thanks!!
We are thankful that the reviewers took the time to fully read our rebuttal and found our rebuttal meaningful. We appreciate your very positive evaluation of our work. Benefiting from your comments, the added experiments about the performance under inaccurate CPDAG can further enhance the quality of our manuscript -- thank you so much!!
I have fully read your rebuttal and I agree to them and yes I misclicked the flag for ethics review