Wasserstein Distributionally Robust Optimization through the Lens of Structural Causal Models and Individual Fairness

Authors: ahmad-reza  ehyaei
golnoosh  farnadi
samira  samadi
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 13:00:10

Abstract: In recent years, Wasserstein Distributionally Robust Optimization (DRO) has garnered substantial interest for its efficacy in data-driven decision-making under distributional uncertainty. However, limited research has explored the application of DRO to address individual fairness concerns, particularly when considering causal structures and discrete sensitive attributes in learning problems.
To address this gap, we first formulate the DRO problem from the perspectives of causality and individual fairness. We then present the DRO dual formulation as an efficient tool to convert the main problem into a more tractable and computationally efficient form. Next, we characterize the closed form of the approximate worst-case loss quantity as a regularizer, eliminating the max-step in the Min-Max DRO problem. We further estimate the regularizer in more general cases and explore the relationship between DRO and classical robust optimization. Finally, by removing the assumption of a known structural causal model, we provide finite sample error bounds when designing DRO with empirical distributions and estimated causal structures to ensure efficiency and robust learning.

Main paper: 
 introduction: 
 Machine learning models must address discrimination because they often reflect and amplify biases present in their training datasets [31] . These biases can significantly influence decisions in domains such as healthcare [30] , education [3] , recruitment [18] , and lending services [6] . Consequently, these decisions disproportionately affect individuals based on sensitive attributes like race or gender, perpetuating systemic discrimination.To address and quantify unfairness, researchers have developed concepts like group fairness and individual fairness [45, 4] . Group fairness aims to achieve equitable outcomes across demographic groups, while individual fairness ensures that similar individuals receive similar treatment. Formally, with V as the feature space and Y as the label space, a model h : V → Y ensures individual fairness if it satisfies the condition in [20] :EQUATIONwhere d V and d Y are dissimilarity functions, often referred to as fair metrics on the input and output spaces. These functions capture the proximity of individuals and L ∈ R + is a Lipschitz constant. The metric d V reflects the intuition about which instances should be considered similar by the model.Due to challenges in defining such metrics, group fairness is often prioritized in fairness literature because it more straightforwardly addresses observable disparities among distinct groups, making measurement and implementation easier in practice [8] . Therefore, it is crucial to study and formulate individual fairness under different assumptions in machine learning.Individual fairness can be achieved through robust optimization methods such as Wasserstein DRO, which has gained significant attention for its applications in learning and decision-making [55, 43] . DRO incorporates a regularization term to mitigate overfitting [17, 26, 59] . By using a fair metric as the transportation cost function in computing the Wasserstein distance, models are designed to deliver consistent performance across varied data distributions, ensuring similar individuals receive comparable outcomes, thus satisfying individual fairness.Incorporating causal structures and sensitive attributes into data models complicates using an individual fair metric as a cost function within the DRO framework. The fair metric must account for perturbations in sensitive attributes based on counterfactuals to ensure counterfactual fairness [22] . This can violate the positive-definite property, where d(v, v ′ ) = 0 implies v = v ′ , a key assumption in many DRO theorems [55, 43] .Although previous works [42, 67, 70, 69, 57] have attempted to apply DRO to address individual fairness, they often do not explore the implications when causal structures and sensitive attributes are present in the learning problem. These studies are typically limited to linear Structural Causal Model (SCM) with specific metrics and do not discuss the form of the regularizer for other classical DRO theorems when using a fair metric. To accurately compare our work with related studies, we will postpone this discussion until after presenting our results in Section 4.1.In this work, we adopt the definition of a fair metric from [22] to define a Causally Fair Dissimilarity Function (CFDF), which delineates how to establish a fair metric through causality and sensitive attributes. Using CFDF, we introduce Causally Fair DRO and present a strong duality theorem for our approach. Under mild assumptions about CFDF and causal structure, we demonstrate that the DRO regularizer can be estimated, or in some cases can be explicitly solved. This estimation often leads to being more practical and computationally efficient than solving the min-max problem in (4), as supported by advancements in algorithms from previous research such as [14, 15] . Finally, Our numerical analysis of both real and synthetic data demonstrates the practicality of our theoretical framework in real-world applications. ( § 5). In summary, the main contributions of this work are:• Define a causally fair dissimilarity function, an individual fair metric incorporating causal structures and sensitive attributes (Def. 1), along with its representation form (Prop. 1). • Define a causally fair DRO problem with a causally fair dissimilarity function cost ( § 4).• Present the strong duality theorem for causally fair DRO (Thm. 1).• Provide the exact regularizer for linear SCM under mild conditions for the loss function in regression and classification problems (Thm. 2 and Thm. 3). • Estimate the first-order causally fair DRO regularizer for non-linear SCM (Thm. 4).• Provide the relation between classical robust optimization and causally fair DRO (Prop. 2).• Demonstrate that under unknown SCM assumptions, by estimating the SCM or cost function, we have finite sample guarantees for convergence of empirical DRO problems (Thm. 5). 
preliminaries & notations: 
 Data Model. Let V ∈ V denote a vector of feature space (predictor variables) and let Y ∈ Y represent the response variable, such that Z = (V, Y) comprises the observation variables with an underlying probability P * . Furthermore, assume that the feature vector V = (A, X) comprises both sensitive attributes A ∈ A and non-sensitive attributes X ∈ X . Let {z i = (v i , y i )} N i=1 represent the observations used to construct the empirical distribution P N , defined as P N := 1 N N i=1 δ z i , where δ z is the Dirac delta function. Given a loss function ℓ : Z × Θ → R, the risk function for a parameter θ ∈ Θ and a probability measure P is R(P, θ) = E P [ℓ(Z, θ)]. This leads to the common empirical risk minimization approach. This method seeks to find the minimizer θ erm N within the set θ erm N ∈ arg min θ∈Θ R(P N , θ), as an empirical way to obtaining the optimal solution θ * , which is given by θ * = inf θ∈Θ R(P * , θ).Assume the feature space is represented by a structural causal model (SCM) M = ⟨G, V, U, P U ⟩ [51] . This model includes structural equations {V i := f i (V Pa(i) , U i )} n i=1 , which delineate the causal relations among an endogenous variable V i , its causal predecessors V Pa(i) , and an exogenous variable U i representing unobservable factors. The model's structure is encapsulated in a directed acyclic graph G. Exogenous variables are posited as mutually independent, enabling P U to be expressed as n i=1 P Ui , assuming causal sufficiency and excluding hidden confounders [53] .Counterfactuals. In causal structures, data perturbation is achieved through counterfactuals, which are derived from interventions in SCMs. These interventions, conducted using do-calculus, include both hard and soft types [51] . Hard interventions fix a subset I ⊆ {1, . . . , n} of features V I to a constant τ , modifying their causal connections within the causal graph while maintaining the structural equations of other features [51] . This type of intervention is denoted as M do(V I :=τ ) and its structural equations are obtained by:{V i := τ i , ∀i ∈ I; V i := f i (V Pa(i) , U i ), ∀i / ∈ I}.Soft interventions, on the other hand, adjust the functions in the structural equations, such as through additive interventions, without disrupting existing causal links [53] . In an additive (or shift) intervention, a value ∆ ∈ R n is added to each feature within the SCM to enact manipulation:{V i := f i (V Pa(i) , U i ) + ∆ i } n i=1 .In SCMs, counterfactuals are computed by modifying structural equations to reflect hard interventions on specific variables, thus exploring what would occur if the intervention was applied. Under the assumption of acyclicity, a unique function F : U → V exists such that F (u) = v. Acyclicity remains unchanged by either hard or shift interventions, allowing for the existence of modified functions F do(V I :=τ ) and F do(V I +=∆) corresponding to these interventions, respectively. The counterfactual outcome for a hard intervention can thus be calculated using CF(v, τ ) = F do(V I :=τ ) (F -1 (v)), and similarly, for a shift intervention, it is defined as CF(v, ∆). These interventions are frequently applied in this analysis.Counterfactuals involving the modification of sensitive attributes (termed twins) are essential for addressing individual-level fairness [40, 64] . Twins are generated by altering the sensitive attribute from a to a ′ across its domain A. For any instance, v ∈ V, a set of counterfactual twins is produced as {v a = CF(v, a) : a ∈ A}, facilitating the analysis of fairness by comparing outcomes under different sensitive attribute values.Counterfactual Identifiability. To estimate the effects of interventions from observational data, counterfactuals must be identifiable within a causal framework. A notable example of such identifiable SCMs is the additive noise models (ANMs), which suggest that structural equations can be represented as:EQUATIONleading to a bijective mapping between U i and V i , ensuring no loss of information from exogenous to endogenous variables [50] . This relationship implies that V can be derived from U through a bijective reduced-form mapping F = (I -f ) -1 , where I(x) = x is the identity function. Besides ANM, there are other counterfactually identifiable models such as LSNM [34] and PNL [71] . However, for the sake of simplicity, our focus remains on ANM. Linear SCMs is a specific instance of ANMs, characterized by linear functions f i .Individual Fairness Through Robustness. In machine learning, individual fairness [20] is achieved through robustness by ensuring that similar individuals receive similar outcomes, regardless of variations in their inputs. This concept aligns with the notion of Lipschitz continuity in decision functions (Eq. 1), where small changes in input should not lead to excessively large changes in output.Depending on how the uncertainty set is defined, various types of robust optimization can be employed.In adversarially robust optimization [44, 7] , the uncertainty set is defined by introducing a slight perturbation δ based on the metric d to the input data. The goal is to find the optimal θ that minimizes risk even under the worst-case perturbation quantity:EQUATIONwhere p ∈ [0, ∞]. This formulation ensures that the optimization considers the maximum potential loss within the defined perturbation bounds.In counterfactually robust optimization [40, 37, 64, 23, 24] , the uncertainty set is generated by twins, which are obtained by creating counterfactuals concerning all levels of the sensitive attribute.In this scenario, the worst-case loss quantity is obtained by calculating the maximum loss over the twins of the input data:R cf δ (P, θ) = E v∼P sup a∈A ℓ(v a , y, θ) .Distributionally Robust Optimization [43, 55] is a data-driven approach designed to minimize the discrepancies between in-sample and out-of-sample expected losses, using ambiguity sets based on Wasserstein distances. Consider a lower semi-continuous cost function c(•, •) : Z × Z → [0, ∞] that satisfies c(z, z) = 0 for all z ∈ Z, serving as a fair metric. The optimal transport cost between two distributions P, Q ∈ P(Z), is represented by:W c,p (P, Q) ≜ min π∈P(Z×Z) E (z,z ′ )∼π [c p (z, z ′ )] 1 p : π 1 = P, π 2 = Q ,Here, π ∈ P(Z × Z) denotes the set of all joint probability distributions, and π 1 and π 2 are the marginals of π under first and second coordinates [54, 63] . When c(z, z ′ ) acts as a metric (in mathematics term) on Z, W c,p is called the Wasserstein distance [63] .An important ingredient in the DRO formulation is the description of the distributional uncertainty region B δ (P) that is defined by optimal transport cost:B δ (P) := {Q ∈ P(V) : W c,p (Q, P) ≤ δ}. DRO problem minimizes worst-case loss quantity:EQUATIONand obtained the θ dro N ∈ arg min θ∈Θ R δ (P N , θ). The main tool in DRO is the strong duality theorem [26, 46] , which converts an infinite-dimensional problem into a finite optimization problem. The theorem states that:EQUATIONwhereψ λ (v) is defined as ψ λ (v) := sup v ′ ∈V {ψ(v ′ ) -λd p (v, v ′ )}. 
causally fair dissimilarity function: 
 The key to robust optimization and individual fairness is the metric that measures individual similarity. This section outlines the properties of such a metric in a causal framework to protect sensitive attributes. We begin with an illustrative example.Example 1 Let M 1 and M 2 represent two SCMs describing the relationships among the variables gender (G), education (E), and income (I). M 1 models these variables as independent, whereas M 2 specifies a linear causal relationship:M 1 =    G := U G , U G ∼ B(0.5) E := U E , U E ∼ N (0, 1) I := U I , U I ∼ N (0, 1) , M 2 =    G := U G , U G ∼ B(0.5) E := G + U E , U E ∼ N (0, 1) I := G + 2E + U I , U I ∼ N (0, 1)Where U G represents the population distribution of gender, modeled by a Bernoulli distribution, while U E and U I are intrinsic talents for academic and income achievements, respectively, modeled by normal distributions. To compare individuals, let's consider the L 1 norm on non-sensitive attributes(d(v, v ′ ) = |e -e ′ | + |i -i ′ |).If two individuals have less than a 0.1 unit difference, they are deemed similar. Now, consider an individual with data v = (M, 1, 1). Based on experience, we expect that a perturbation in educational talent by .05 units will not significantly alter this individual's status. We model this perturbation with a shift intervention ∆ = (0, .05, 0). In Model 1, the result CF(v, ∆) = (M, 1.05, 1) is considered similar to v. However, in Model 2, CF(v, ∆) = (M, 1.05, 1.1) results in a distance of d(v, CF(v, ∆)) = 0.15, indicating dissimilarity. In the presence of causality, one attribute can be amplified multiple times in the final feature space. Therefore, we need to control our intuition of dissimilarity between the exogenous variables and the feature space.To protect against gender bias, we need to ensure that people with the same intrinsic characteristics but different genders behave similarly. This is modeled by a counterfactual change in gender. In Model 1, CF(v, F ) = (F, 1, 1) shows no difference (d(v, vF ) = 0). However, in Model 2, CF(v, F ) = (F, 0, -2) results d(v, vF ) = 4, which means that they are not similar.The example 1 demonstrates that in the presence of causality and protected variables, the standard l p -norm or any metric fails to accurately capture the intuition of similarity. In these scenarios, a dissimilarity function should incorporate counterfactuals and uniformly control for non-sensitive perturbations to effectively capture proximity. This approach is further elaborated in the following definition. Before proceeding, we introduce some notation. For a vector v or u, we define P A (•) and P X (•) as the projections onto the sensitive and non-sensitive parts, respectively. Definition 1 (Causally Fair Dissimilarity Function) Let d : V × V → [0, ∞] be a dissimilarity function defined on the feature space V, generated by a SCM M. Let A denote a set of sensitive attributes, and I represent their corresponding index within {1, . . . , n}. The metric is called a causally fair dissimilarity function or CFDF if it adheres to the following properties:• Zero Dissimilarity for Twin Pairs: For any v ∈ V and a ∈ A, the dissimilarity d (v, va ) between an instance and its twins is zero.• Guaranteed Similarity for Minor Perturbations: For every v ∈ V and any δ > 0, there exists an ϵ such that for any sufficiently small intervention (∥∆∥ ≤ ϵ) on the non-sensitive attributes (P A (∆) = 0), the distance d(v, CF(v, ∆)) remains less than δ.To understand the shape of d under the assumptions of Def. 1, we must first recognize that the CFDF needs to be defined on a larger space than Range(M) [22] . This is because, generally, when M is intervened upon by some sensitive attribute level a, we have Range(M) ⊆ a∈A Range(M do(A:=a) ).The complete space encompassing all counterfactual values can be defined as follows.Definition 2 (Parent-Free Sensitive Attribute SCM) Consider M with sensitive attributes indexed by I. The parent-free sensitive attribute SCM denoted as M 0 , is derived from M by removing the causal effects of parents of sensitive attributes and replacing their exogenous variables with indigenous ones. The structural equations for M 0 are as follows:V 0 i := U i U i := V i ∼ P Vi , i ∈ I f i (V 0 pa(i) ) + U i U i ∼ P Ui , i / ∈ IThe exogenous space corresponding to M 0 , denoted by U 0 , includes the sensitive attributes and the non-sensitive parts of the exogenous variables of M. This space called the semi-latent space, is constructed as U 0 = A × U X , where U X is the non-sensitive part of the exogenous space in M.If we know the structural equations of M, we can first map the CFDF to the exogenous space. In this space, the exogenous variables are assumed to be independent. Therefore, we can design a dissimilarity function for each variable separately and then combine them using product topology ( §.2 [48] ). Following this intuition, we introduce the bijective map g : V → U 0 from the feature space to the semi-latent space, along with its inverse, defined as follows:EQUATIONIf all sensitive attributes have no parents, the semi-latent space is equivalent to the exogenous space, and g = F -1 . The counterfactual with respect to M 0 is denoted by CF 0 (v, ∆). We can now present the following proposition to determine the shape of d.Proposition 1 Let M be an ANM, with g as its corresponding map to the semi-latent space 6 , and P X (u) the projection of vector u to the non-sensitive part U X . Then:(i) If d X is a continuous dissimilarity function on diagonal U X × U X , then the function d defined as:EQUATIONsatisfies the definitions of a CFDF.(ii) If d : V × V → [0, ∞] satisfies the CFDF definition and the triangle inequality property, then d can be represented as a dissimilarity function d X dependent solely on the non-sensitive componentsU X i.e., d(v, v ′ ) = d X (P X (g(v)), P X (g(v ′ ))).Since d X is defined on independent coordinates, its relation to the components is less complex than the CFDF d. We assume the dissimilarity function d X (x, x ′ ) is translation-invariant. Therefore, for simplicity, we assumed X (x ′ , x) = ∥x ′ -x∥. The dual of ∥ • ∥ is defined as ∥x∥ * = sup x ′ {x T x ′ | ∥x ′ ∥ ≤ 1}. Now we establish our assumptions about the SCM and its CFDF.Assumption 1 (i) M is an ANM with known structural equations and a semi-latent map g.(ii) The CFDF is defined asd(v, v ′ ) = ∥P X (g(v)) -P X (g(v ′ ))∥, where ∥.∥ is a some norm. (iii) Cost function over Z has form c((v, y), (v ′ , y ′ )) = d(v, v ′ ) + ∞ • |y -y ′ |.(iv) The ambiguity set is defined as:B δ (P) = {Q ∈ P(V) : W c,p (P, Q) ≤ δ}, for p ∈ [1, ∞).Remark 1 All results of this work apply to the homogeneous dissimilarity function (Def. 6), which includes a broad family of dissimilarity functions, such as norms. 
causally fair distributionally robust optimization: 
 To find out the impact of the CFDF in DRO problems, we first consider the dual form of the worst-case loss quantity, which simplifies the infinite-dimensional primal problem into a more tractable and computationally manageable form.Theorem 1 (Causally Fair Strong Duality) If Assumption 1 is satisfied, then for any reference probability distribution P and any function ψ : V → R that is both upper semi-continuous and L 1 -integrable, the following duality holds:EQUATIONwhere ψ λ (v) is defined asEQUATIONand CF 0 is counterfactual regarding parent-free SCM M 0 .The intuition behind the above formula is as follows: In the case where all features are independent, let v = (a, x). The CFDF should exhibit no difference between (a, x) and (a ′ , x) for each a, a ′ ∈ A. Consequently, the distance metric satisfies d((a, x), (a ′ , x ′ )) = d X (x, x ′ ). Under this condition, the classical strong duality theorem (Eq. 5) provides the following relationship:ψ λ (v) = sup (a ′ ,x ′ )∈V {ψ((a ′ , x ′ )) -λd p X (x, x ′ )} = sup a∈A sup ∆∈X ψ((a, x + ∆)) -λd p X (x, x + ∆)When we incorporate causal structure instead of coordinating a and x, the two dimensions va and CF 0 (v, ∆) are replaced accordingly.In the DRO formulation, the worst-case loss is expressed in a dual form and can act as a regularizer for parameter learning. Explicitly solving the dual problem eliminates the need to compute the worst-case distribution, resulting in faster, more efficient learning algorithms [14, 16, 62, 73] . Before presenting the general theorem, the next two theorems show that, under mild conditions, the dual formula for specific loss functions in classification and regression problems can be explicitly solved.Theorem 2 (Higher Order Linear Loss) Given Assumptions 1, let M be a linear SCM and the loss function ℓ(z, θ) p , where ℓ(z, θ) is of the form h(y -⟨θ, v⟩) or h(y • ⟨θ, v⟩) for functions h(t) such as |t|, max(0, t), |t -τ |, or max(0, t -τ ) for some τ ≥ 0, and p ∈ [1, ∞). Then the DRO problem 4 can be reduced to:R δ (P N , θ) =        R cf δ (P N , θ) 1 p + δ P X (M T θ) * p , diam (A) < ∞ R(P N , θ) 1 p + δ P X (M T θ) * p , s.t. P A (M T θ) = 0; diam (A) = ∞where M is the corresponding matrix for the linear map g -1 (see Eq. 6).Remark 3 In real-world datasets, the sensitive part always satisfies diam (A) < ∞. According to the above theorem, R δ (P N , θ) ≥ R cf δ (P N , θ). For practical applications, if the worst-case loss must not exceed a certain value, we can replace ∞ with some constant in the above theorem.Example 2 Here are specific examples of the above theorem. We offer a framework to study the equivalence between the worst-case loss in the DRO problem, with the cost function derived from the CFDF, and the regularization scheme for classification and regression problems.Lower Partial MomentsE P [|Y -⟨θ, V⟩| p ], p ≥ 1 E P [(Y -⟨θ, V⟩ -τ ) p + ], p ≥ 1, τ ∈ R Ridge Linear Regression τ -Insensitive Regression E P [(Y + ⟨θ, V⟩) 2 ] E P [(|Y -⟨θ, V⟩| -τ ) p + ], p ≥ 1, τ ∈ R Hinge Loss Binary Classification Support Vector Machine Classification E P [(1 -Y • ⟨θ, V⟩) p + ], p ≥ 1 E P [|1 -Y • ⟨θ, V⟩| p ], p ≥ 1The Thm. 2 can be extended to the non-linear regression loss function.Theorem 3 (Nonlinear Loss) Let assumptions 1 be satisfied, with p = 1, M linear with matrix M corresponding to map g -1 , and a loss function ℓ(z, θ) of the form h(y -⟨θ, v⟩) for regression and h(y • ⟨θ, v⟩) for classification, where h has the following two properties:(i) h is Lipschitz on R with L h constant, i.e., |h(t 2 ) -h(t 1 )| ≤ L h |t 2 -t 1 |, ∀t 1 , t 2 ∈ R. (ii) There exists sequence of {t k } ∞ k=1 goes to ∞ such that for each t 0 ∈ R we have lim k→∞ |h(t 0 + t k ) -h(t 0 )| |t k | = L h .By the above assumption, DRO problem 4 can be reduced as:R δ (P N , θ) =      R cf δ (P N , θ) + δL h P X (M T θ) * , diam(A) < ∞ R(P N , θ) + δL h P X (M T θ) * , s.t. P A (M T θ) = 0; diam(A) = ∞ Example 3The following forms of the loss function satisfy the conditions of h in Thm. 3:Now, the first-order estimation of the regularizer for non-linear SCM and loss function is ready to be stated.Theorem 4 (First-Order Estimation of DRO Regularizer) Assume M has structural equation f , which f and loss function ℓ are both twice continuously differentiable respect to non-sensitiveHuber Lossh : t → log(cosh(t)) h : t → 1 2 t 2 if |t| ≤ 1, |t| -1 2 otherwise; Quantile Loss Log-exponential Loss h : t → γt if t ≥ 0, -t otherwise, with γ ∈ (0, 1); h : t → log(1 + exp(-t))Smooth Hinge Loss Truncated Pinball Lossh : t →    0 if t ≥ 1, 1 2 (1 -t) 2 if 0 < t < 1, 1 2 -t otherwise; h : t →    1 -t if t ≤ 1, τ 1 (t -1) if 1 < t < τ 2 + 1, τ 1 τ 2 otherwise, where τ 1 ∈ [0, 1], τ 2 ≥ 0 are two given constants.attributes, diam (U) < ∞ and c satisfies the assumption 1 with p ∈ [2, ∞]. The necessary condition for the existence of a finite DRO solution is that for each v ∈ V:sup a∈A {ℓ(v a , y, θ)} < ∞.By these conditions, the worst-case loss quantity is equal to:EQUATION)where the O(δ 2 ) term is uniform over all θ ∈ Θ, q is p's conjugate, and the gradient ∇ CF ℓ equals to:∇ CF ℓ(v, y, θ) = lim ∆→0 ℓ(CF 0 (v, ∆), y, θ) -ℓ(v, y, θ) ∥∆∥where CF 0 is counterfactual regarding parent-free SCM M 0 .By applying Prop. 2 from Gao's work [28] , the next proposition presents the relationship between classical adversarial optimization 3 and DRO for CFDF.Proposition 2 (Approximation by Robust Optimization) Suppose A is a finite set and let {(v i , y i )} N i=1 be observational data. Under Assumption 1, assume that for the loss function ℓ there exist constants L, M ≥ 0 such that|ℓ(v, y, θ) -ℓ(v ′ , y, θ)| < Ld p (v, v ′ ) + M for all v, v ′ ∈ V and p ∈ [1, ∞).For an arbitrary K ∈ N, consider the adversarial loss within the setting:Radv δ (P N ) := sup (w ik ) i,k ∈ Bδ 1 N K N i=1 K k=1 sup a∈A ℓ( ẅik a , y i , θ) ,where the uncertainty set Bδ is defined as:Bδ := (w ik ) i,k : 1 N N i=1 K k=1 d p (v i , w ik ) ≤ δ, w ik ∈ V .Then, the DRO can be approximated by adversarial optimization as follows:Radv δ (P N ) ≤ R δ (P N ) ≤ Radv δ (P N ) + LD + M N K ,where D is independent of K.One of the main challenges in designing DRO for SCMs is that the CFDF depends on the causal structure. When the functional structure is unknown, it must be estimated from data. This empirical estimation impacts the DRO learning process. Therefore, it is crucial to control the uniform convergence error of the DRO problem between the true metric and distribution and the DRO estimated from the data. The following theorem guarantees learning from sample data, but certain assumptions need to be established first.Assumption 2(i) M is an unknown ANM, diam (V) < ∞, and Θ is a compact subset of R d .(ii) The loss function ℓ is uniformly bounded: there exists a positive constant M such that 0 ≤ ℓ(z, θ) ≤ M for all θ ∈ Θ. Moreover, ℓ is Lipschitz with respect to the counterfactual in M 0 ; that is, there exists a constant L such that:|ℓ(v, y, θ) -ℓ(CF 0 (v, ∆), y, θ)| ≤ ∥ℓ∥ Lip ).(iii) d is an estimation of the CFDF such that, with probability 1 -ϵ, there exists M d such that, at a rate of N -η , the discrepancy is uniformly bounded by:∀v, v ′ ∈ V : |d(v, v ′ ) -d(v, v ′ )| ≤ M d N -η , for some η > 0.The following theorem states that the efforts to estimate the metric or causal structures and the parameter θ dro N ,θ dro N := inf θ∈Θ sup Q:W ĉ,p (Q,P N )≤δ E z∼Q [ℓ(z, θ)]Where ĉ is the d corresponding cost on Z, leading to the estimation of the true parameters of the DRO problem. To state our result, we need the Dudley entropy integral [61] , which measures the complexity of the loss function class.Theorem 5 (Learning Finite Sample Guarantee) With assumption 1 and 2, then for θ dro N we have:R δ (P * , θ dro N ) -inf θ∈Θ R δ (P * , θ) ≤ N -1/2 c 0 + c 1 δ 1-p + c 2 δ 1-p N -η+1/2 + c 3 log(2/ϵ) ,With probability at least 1 -2ϵ. With C(L) denoting the Dudley entropy integral for the function class {ℓ(•, θ) : θ ∈ Θ}, the constants c 0 , c 1 and c 2 are identified as follows:c 0 := 96C(L), c 1 := 96L • diam (V) p , c 2 := 2pL • diam (V) p-1 • M d , and c 3 := 2 √ 2 × M.The final theorem completes our framework, enabling us to perform DRO on real-world datasets without knowing the SCM structures while providing performance bounds.Causally Fair Dissimilarity Function. Various studies have addressed the specification and learning of individual fair metrics, such as [33, 68, 70, 47] , but their construction based on causal structure and sensitive attributes remains unclear. Our work adopts and extends the concept of a causal fair metric, as discussed in the works [23, 24] .DRO and Individual Fairness. Previous works, such as [68, 70, 47] , address the DRO problem with an individual fairness metric but are limited to linear SCMs and p = 2. These studies do not discuss the duality theorem or regularizers. Additionally, [42] studied DRO, but its connection to causality remains unclear.Strong Duality Theorem. Various versions of the strong duality theorem have been explored in prior works. For instance, in [59, 46, 9, 14, 27, 28, 66] , the cost function must be a metric or [74] has convex property. Additionally, in [72, 11, 58] , the distance function d must be positive-definite, meaning d(v, v ′ ) = 0 if and only if v = v ′ . However, these conditions are not met for CFDF, necessitating a new formulation of the duality theorem 1.DRO as Regularizer. Previous works on using DRO as a regularizer, explicitly solved [60, 14, 16, 29] or through k-order estimation [5, 9, 6, 66, 27] , only consider cases where the cost function is derived from a metric or a positive-definite dissimilarity function. Therefore, their theorems do not apply directly to our CFDF. We present new results in Theorems 2, 3, and 4 tailored for our cases.Finite Sample Guarantee. Various works provide bounds on the performance of DRO solutions with finite samples [41, 25, 10, 12] , but these do not apply to our CFDF due to previously mentioned reasons. The studies [68, 70, 47] offer performance bounds only for the case of linear SCMs with p = 2. Therefore, we present a general case in Theorem 5.Optimal Transport and Causality. Recent works [39, 35, 13, 32, 21, 1, 2] on causal optimal transport focus on the causal structure of the transport map or plan, which differs from our problem. In our case, causality pertains to the transportation cost derived from SCMs. 
numerical studies: 
 In our numerical studies, we evaluate the impact of using causally fair DRO to mitigate individual unfairness, henceforth referred to as CDRO. We compare CDRO's performance against Empirical Risk Minimization (ERM), non-causal Adversarial Learning (AL) [44] , and the Ross method [56] .Our experiments employ real-world datasets, namely the Adult [38] and COMPAS [65] datasets, pre-processed according to [19] . Additionally, we use a synthetic dataset for linear SCM (LIN) with formulations detailed in Appendix C.1. We first fit a linear structural equation model for both the Adult and COMPAS datasets. Logistic regression is employed for classification, and performance is evaluated based on accuracy. Fairness is assessed using the Unfair Area Index (UAI), which is defined by the following equation:Adult COMPAS Real-world Data A L C D R O E R M R O S S A L C D R O E R M R O S S 0% 10% 20% 30% Unfair Area (D = 0.05) LIN Synthetic Data A L C D R O E R M R O S S 0% 10% 20% 30% 40% Adult COMPAS Real-world Data A L C D R O E R M R O S S A L C D R O E R M R O S S 0% 20% 40% 60% 80% Accuracy LIN Synthetic Data A L C D R O E R M R O S S 0% 20% 40% 60%U ∆ := P {v ∈ V : ∃v ′ ∈ V s.t. d(v, v ′ ) ≤ ∆ ∧ h(v) ̸ = h(v ′ )} .We evaluate UAI across different ∆ values, specifically 0.05 and 0.01. Additionally, we calculate the UAI for scenarios where no sensitive attributes are considered, representing the percentage of non-robust data. Detailed computational experiment procedures are provided in Section C.1.Our experiments, conducted using 100 different seeds, are summarized in Table 1 . Figures 1, 2 and 3 illustrate that the CDRO method achieves a lower unfair area (U ∆ ) for ∆ = .05, and ∆ = 0.01 in all scenarios. Although CDRO shows slightly lower accuracy than ERM, this trade-off is a common observation in several studies [52] . Additional results can be found in § C.6. 
discussion and limitations: 
 Our study introduces a novel framework for causally fair DRO, integrating causal structures and sensitive attributes into the DRO paradigm. This framework is supported by several theoretical advancements, including a strong duality theorem, explicit regularizer formulation, first-order regularizer estimation, and finite sample guarantees with unknown SCMs, enhancing its efficiency and practicality for real-world applications. Our experimental results demonstrate its effectiveness in various settings, highlighting its potential for mitigating biases in machine learning models.Despite the promising results, our study has several limitations that warrant further investigation. Firstly, the assumption of an additive noise model may not capture the complexity of all realworld causal relationships, posing challenges in computing additive interventions in general SCMs. Secondly, while Theorem 2 and Theorem 3 could be extended to more general cases, we omitted these extensions to avoid complexity. Lastly, further work is needed to explore the relationship between our method and causal optimal transport [13, 32] .Notations. In this work random variables are denoted by bold letters (e.g., V), their corresponding probability spaces by calligraphic letters (e.g., V), and instances by normal letters (e.g., v). The space of probability measures on V is represented by P(V) and probability measures by blackboard bold letters (e.g., P).Non-Sensitive Part. Let F : U → V be the reduced-form map of M. The vector v decomposes into sensitive and non-sensitive parts, v = (a, x), and we have a corresponding decomposition in the exogenous space denoted by u = (u a , u x ) and U A , U X are corresponding spaces. Using the ANM model, we can assume that both V and U are equivalent, and therefore, the non-sensitive feature space is the same as the non-sensitive part of the exogenous space. If P is a probability measure in P(V), then (P) X refers to the marginal probability over the non-sensitive part. We also refer to Q) X for marginal probability over the non-sensitive part of the exogenous space.Definition 3 (Push-forward Measure) Let P ∈ P(V), Q ∈ P(U) be two probability measures and T : V → U is map, the measure Q is called the push-forward of P through T is denoted by T # P if:Q(B) = P(T -1 (B)), ∀B ⊂ U Definition 4 (Set of Couplings)The set Γ(P, Q) represents the couplings of probability distributions P ∈ P(V), Q ∈ P(U), comprising distributions over V × U with margins P and Q. A measure π belongs to Γ(P, Q) if and only ifπ(A × U) = P(A) and π(V × B) = Q(B) ∀A ⊂ V, B ⊂ UBy extension, a random pair (X, Y ) ∼ π, where π ∈ Γ(P, Q), will also be called a coupling of P and Q.Definition 5 (Diameter of a Set) Let A be a set in a metric space with a distance function d. The diameter of A, denoted diam(A), is defined as:diam(A) = sup{d(x, y) : x, y ∈ A}where sup represents the supremum of the set of distances d(x, y) for all pairs (x, y) in A.Definition 6 (Homogeneous dissimilarity function) Let Λ be an extended-valued function Λ : X → [0, ∞] on a real vector space X with absolutely homogeneous assumption i.e. Λ(tx) = |t| Λ(x) for any t ∈ R and z ∈ X . In addition, Λ is proper it means there exists x 0 ∈ X such thatΛ(x 0 ) = 1. The cost function d : X × X → [0, ∞] is called Homogeneous dissimilarity function if is defined as d(x ′ , x) := Λ(x ′ -x) for any x ′ , x ∈ X .Lemma 1 If M is an additive noise model with mutually independent exogenous variables, then the parent-free sensitive M 0 attribute model retains both of these properties. Moreover, the map-reduced form mapping of M 0 is equivalent to g -1 , where g represents the mapping to the semi-latent space.Proof. First, M 0 is an additive noise model because its structure is derived from the initial equations of M by removing those equations related to the sensitive attributes and replacing the exogenous variable U i by V i .Regarding the mutual independence of the exogenous variables, in the original model M, the variables V i for i ∈ I are not independent of U j for j / ∈ I if they have parents. However, assuming a hard intervention for each instance of V i -where a do-action is executed for this intervention -it implies that the intervened variable V i can be considered independent from the other variables. Therefore, since we apply hard interventions to all sensitive variables, we can assume that V i for i ∈ I are mutually independent, and also that V i are independent of U j for all j / ∈ I.Finally, by referencing equations 6, it is observable that the map-reduced form mapping of M 0 is equivalent to the inverse of the map to the semi-latent space.Lemma 2 Let M be an additive noise model with a mapping g to semi-latent space U 0 . Assume M includes the sensitive attributes A and other non-sensitive attributes X that belong to the vector space X . Consider v = (a, x) as an instance in M, and let ∆ ∈ X represent a shift intervention value. Then, the counterfactual corresponding to additive shit is obtained by:P X (CF(v, ∆)) = P X (g -1 (g(v) + (0, ∆))).Moreover, if a ′ ∈ A represents another level of sensitive attributes, then the hard intervention concerning A := a ′ is achieved by:va ′ = CF(v, do(A:=a ′ )) = g -1 ((a ′ , P X (g(v)))).Proof. In additive noise models, an additive intervention can be conceptualized as adding a value δ to the exogenous variables, while all structural equations remain unchanged. Consequently, during such an intervention, the reduced-form mapping F M ∆ of the intervened SCM remains unchanged.Therefore by definition of intervention, it follows that:CF(v, do(X+=∆)) = F M ∆ (F -1 (v) + (0, ∆)) = F (F -1 (v) + (0, ∆)).Since F and g -1 are coincide in non-sensitive coordinates then P X (ψ(F -1 (v) + (0, ∆))) = P X (g -1 (g(v) + (0, ∆))) and it completes the first part. In this case, we denote F as M , which is an invertible matrix. Consequently, the counterfactual CF(v, do(X+=∆)) can be expressed as v + M -1 (0, ∆).To prove the second part, when intervention is performed on sensitive attributes, M transforms into a parent-free sensitive attribute model where the sensitive attribute a is replaced by a ′ . Given that the map-reduced form of the parent-free sensitive attribute model aligns with g -1 , and since g and F -1 coincide on the non-sensitive parts, the counterfactual can be expressed as follows:va ′ = CF(v, do(A:=a ′ )) = g -1 ((a ′ , P X (g(v)))).Proof of Proposition 1.(i) If d adheres to Eq. 7, it means that for each v ∈ V, the mapping g(v) = (a, x). For its counterfactual va , we have g(v a ′ ) = (a ′ , x). Using Eq. 7, we can express:d(v, va ) = d X (P X (g(v)), P X (g(v a ))) = d X (x, x) = 0This demonstrates that d retains the first property of Def. 1.Additionally, since d X is continuous, for each x and any ϵ > 0, there exists a δ > 0 such that if ∥∆∥ < δ, then d X (x, x + ∆) < ϵ. Referencing Lemma 2 and the formulation of d, it follows that for ∥∆∥ < δ and for each a ∈ A:d(v, CF(v, ∆)) = d(P X (g(v)), P X (g(CF(v, ∆)))) = d X (x, x + ∆) < ϵThus, it satisfies property (ii) of the CFDF.(ii) Let's consider a CFDF denoted as d : V × V → R, with an embedding g : V → Q that maps from the feature space to a semi-latent space. We define d * as the pull-back of d onto Q, d * (q 1 , q 2 ) = d(g -1 (q 1 ), g -1 (q 2 )) where d * is a dissimilarity function, and we aim to clarify which properties it inherits from Def. 1. We utilize a decomposition of Q into A × X , where q = g -1 (v) and v ∈ V, denoting q as (a, x). Property (i) of the CFDF ensures:d(v, va ′ ) = d * ((a, x), (a ′ , x)) = 0 ∀a ′ ∈ AThis property confirms that d * is insensitive to changes in the sensitive part A. To demonstrate, consider any two points q 1 = (a 1 , x 1 ) and q 2 = (a 2 , x 2 ), with an arbitrary a 0 ∈ A. By triangle property of dissimilarity function d it can be seen:d * ((a 1 , x 1 ), (a 2 , x 2 )) ≤ d * ((a 1 , x 1 ), (a 0 , x 1 )) + d * ((a 0 , x 1 ), (a 2 , x 2 )) =⇒ d * ((a 1 , x 1 ), (a 2 , x 2 )) ≤ d * ((a 0 , x 1 ), (a 2 , x 2 ))Here, d * ((s 1 , x 1 ), (s 0 , x 1 )) is zero due to property (i). Similarly, we can argue:d * ((a 0 , x 1 ), (a 2 , x 2 )) ≤ d * ((a 0 , x 1 ), (a 1 , x 1 )) + d * ((a 1 , x 1 ), (a 2 , x 2 )) =⇒ d * ((a 0 , x 1 ), (a 2 , x 2 )) ≤ d * ((a 1 , x 1 ), (a 2 , x 2 ))This results in d * ((a 1 , x 1 ), (a 2 , x 2 )) = d * ((a 0 , x 1 ), (a 2 , x 2 )). With similar reasoning, we have:d * ((a 1 , x 1 ), (a 2 , x 2 )) = d * ((a 1 , x 1 ), (a 0 , x 2 )) =⇒ d * ((a 1 , x 1 ), (a 2 , x 2 )) = d * ((a 0 , x 1 ), (a 0 , x 2 ))Hence, d * is invariant to the sensitive subspace. If d X is the dissimilarity function induced by d * on X , then d * ((a 1 , x 1 ), (a 2 , x 2 )) = d X (x 1 , x 2 ). In accordance with Lemma 2, the second property of Def. 1 states that for each ϵ > 0, there exists a δ such that if |∆| < δ, then d X (x, x + ∆) < ϵ. This property demonstrates the continuity of d X along the diagonal.Finally, d can be embedded in semi-latent space and described by another dissimilarity function on it that only depends on the non-sensitive part of exogenous space:d(v, w) = d X (P X (g(v)), P X (g(w)))This concludes the proof. (v, v ′ ) = d(g(v), g(v ′ ))where d is a metric on the space U. For every P, Q ∈ P(V), the following equation holds:W c,p (P, Q) = W d,p (g # P, g # Q)where W c and W d represent the Wasserstein distances with respect to the metrics c and d, respectively.Proof. By the definition of the Wasserstein distance,W c,p (P, Q) = inf π∈Γ(P,Q) V×V c p (v, v ′ ) dπ(v, v ′ ). Substituting c(v, v ′ ) = d(g(v), g(v ′)) and u = g(v) gives:W c,p (P, Q) = inf π∈Γ(P,Q) V×V d p (g(v), g(v ′ )) dπ(v, v ′ ).Consider a coupling π of P and Q. Define a measure π on U ×U by π(A×B) = π(g -1 (A)×g -1 (B)). π is a coupling of g # P and g # Q because:π(A × U) = π(g -1 (A) × V) = g # P(A); π(U × B) = π(V × g -1 (B)) = g # Q(B).Therefore, the p-Wasserstein distance for the push-forward measures isinf π∈Γ(P,Q) V×V d p (g(v), g(v ′ )) dπ(v, v ′ ) = inf π∈Γ(g # P,g # Q) U ×U d p (u, u ′ ) dπ(u, u ′ ) = W d,p (g # P, g # Q).Since π arises from π via g, and g is invertible and measure-preserving in this context, the values in the integrals of the definitions of W c,p (P, Q) and W d,p (g # P, g # Q) match. Thus, we have shown that W c,p (P,Q) = W d,p (g # P, g # Q).Lemma 4 (Optimal Transportation Cost on Subspace) Let U ⊆ R n and suppose U is decomposed into two subspaces, U = (A, X ), where A corresponds to the subset of some coordinates and X to its complements. Let P X denote the projection function onto the X space, i.e., P X (u) projects u ∈ U onto X components. Define a cost function c(u, u ′ ) = d(P X (u), P X (u ′ )), where d is a cost function on the space X . Consider probability measures P, Q ∈ P(U), and define P X = P X # P and Q X = P X # Q as the pushforward measures of P and Q under the projection P X , respectively, placing them in P(X ). Let π * X be the optimal transport plan concerning the Wasserstein distance W d (P X , Q X ). Then, any transport plan π ∈ P(U × U), whose marginal distribution over X × X equals π * X , should also be an optimal solution for the Wasserstein distance W c (P, Q) concerning the cost function c.Proof. Given any coupling π ∈ Γ(P, Q), we consider elements u = (a, x) and u ′ = (a ′ , x ′ ) in U = A × X . The cost function c is defined by c((a, x), (a ′ , x ′ )) = d(x, x ′ ), where d is a metric on the space X . By definition of optimal transport cost W c (P, Q) we have:sup π∈Γ(P,Q) U ×U c((a, x), (a ′ , x ′ )) dπ = sup π∈Γ(P,Q) U ×U d(x, x ′ ) dπ = sup π∈Γ(P,Q) X ×X A×A d(x, x ′ ) dπ((a, a ′ )|X = x, X ′ = x ′ ) d(π) X ×X = sup π∈Γ(P,Q) X ×X d(x, x ′ )d(π) X ×X = sup π∈Γ((P) X ,(Q) X ) X ×X d(x, x ′ )πwhere (P) X and (π) X ×X is marginal distribution over X and X × X respectively. π(.|X = x, X ′ = x ′ ) is conditional distribution of π condition to the first and second X components equal to x and x ′ .We observe that this integral effectively only depends on the X component since the cost function c does not involve A. Hence, we reduce the expression to:EQUATIONThe Eq. 11 shows that the optima cost function of W c (P, Q) equals to W c ((P) X , (Q) X ). Therefore if π * X be the optimal transport plan for P X to Q X with respect to d on X , then any coupling π in U × U that its marginal distribution (π) X ×X equals π * X is the solution of optimal transport. It results that the conditional distribution π((., .)|X = x, X ′ = x ′ ) could be any distribution. This completes the proof. This demonstrates the Principle of the Iterated Suprema.We prove the assertion in two steps: first, we assume that none of the sensitive attributes have parents, and second, we address and prove the general case. When all sensitive attributes do not have parents, in this case by definition2 semi-latent space equivalent with exogenous space and therefore g = F -1 . First, we show that the worst-case loss quantity can be decomposed into sensitive and non-sensitive components like as below equation:EQUATION)By the assumption, the CFDF has a form d(v, v ′ ) = d X (P X (F -1 (v)), P X (F -1 (v ′ ))). By Def. 2 in a case that sensitive attributes have no parents then the semi-latent space coincides with exogenous space and the map between feature space and semi-latent space equals g = F -1 . Therefore in the following equations, we use g instead of F -1 . Moreover since g is invertible by Lemma 3, we can write:sup Q∈B δ (P) E v ′ ∼Q [ψ(v ′ )] = sup Q∈B δ (g # P) E u ′ ∼Q [ψ(F (u ′ ))] = sup π∈P(U ×U ) E u ′ ∼π2 [ψ(F (u ′ ))] u ∼ g # P, E (u,u ′ )∼π [ dX (u, u ′ )] ≤ δ = sup π∈P(U ×U ) E u ′ ∼π2 [ψ(F ((u ′ a , u ′ x )))] u ∼ g # P, E (u,u ′ )∼π [d X (u x , u ′ x )] ≤ δ = sup π∈P(U ×U ) U ψ(F ((u ′ a , u ′ x ))) dπ 2 (u ′ ) u ∼ g # P, E (ux,u ′ x )∼π X ×X [d X (u x , u ′ x )] ≤ δ = * ,where d be a cost function on U defined as d(u, u ′ ) = d X (P X (u), P X (u ′ )), π 2 denotes the marginal distribution on second part and B δ (g # P) = {Q ∈ P(U) : W d(Q, g # P) ≤ δ}. Using the disintegration theorem ( [36] Chapter 3), the joint distribution π 2 can be decomposed into the product of the conditional distribution of U A given U X and the marginal distribution on U X . Therefore we haveU ψ(F ((u ′ a , u ′ x ))) dπ 2 (u ′ ) = U X U A ψ(F ((u a , u x ))) dπ 2 (u ′ a |U X = u ′ x ) d X (π 2 ) X (u ′ x ),where (π 2 ) X is the marginal distribution of π 2 over the non-sensitive part and π 2 (u ′ a |U X = u ′ x ) is a conditional distribution of the sensitive part of exogenous space condition by U X = u ′x . By disintegration formula, (*) can be rewritten as:EQUATIONSince d X depends only on the non-sensitive components, it follows from Lemma 4 that π 2 (.|U X = u ′ x ) can achieve any distribution. Moreover, since it does not depend on the Wasserstein distance in each coupling, the marginal distribution of the sensitive attribute can be considered independent of the marginal distribution of the non-sensitive attributes. Therefore, the supremum over π 2 (.|U X = u ′ x ) of integral equals the supremum of ψ(F ((u ′ a , u ′ x ))) over all values of u ′ a . Furthermore, the distributionπ 2 (u ′ a |u ′ x )does not influence the value of the Wasserstein distance. Based on these points, the last equation can be rewritten as:sup π2∈P(U X ) U X sup u ′ a ∈U A ψ(F ((u ′ a , u ′ x )))dπ 2 π ∈ P(U X × U X ), π 1 = (g # P) X , E (ux,u ′ x )∼π [d(u x , u ′ x )] ≤ δ = sup π2∈P(U X ) E u ′ x ∼π2 sup u ′ a ∈U A ψ(F ((u ′ a , u ′ x ))) π ∈ P(U X × U X ), π 1 = (g # P) X , E (ux,u ′ x )∼π [d(u x , u ′ x )] ≤ δ = sup Q∈B δ ((g # P) X ) E u ′ x ∼Q sup u ′ a ∈U A {ψ (F ((u ′ a , u ′ x )))} .The last equation concludes the proof of Eq. 12. Similarly, by altering the order of integration in Eq. 13, we arrive at the following equation:EQUATION)To proceed with the proof, we utilize the strong duality theorem. There are various kinds of duality theorems for DRO, but we apply the one proposed by Blanchet et al. [11] .Strong duality [11] . Suppose the transportation cost c : Z × Z → [0, ∞] satisfies c(z, z) = 0 for all z ∈ Z and lower semi-continuous. Then for any reference probability distribution P and upper semi-continuous ψ :EQUATIONwhere ψ λ (z) := sup z ′ ∈Z {ψ(z ′ ) -λc(z, z ′ )}.Based on the assumption about the CFDF, where only d(v, va ′ ) = 0, it follows that d(x, x ′ ) = 0 only if x = x ′ . Therefore, we can apply the duality theorem to Eq. 12. According to the duality theorem, it can be expressed as follows:EQUATIONwhereη λ (u x ) = sup u ′ x ∈U X sup ua∈U A {ψ(F ((u a , u ′ x ))} -λd X (u x , u ′ x ) . By using lemma 5 sup u ′ x ∈U X sup ua∈U A {ψ(F ((u a , u ′ x ))} = sup ua∈U A sup u ′ x ∈U X {ψ(F ((u a , u ′ x ))} .Now, since sensitive attributes don't have parents then two spaces U A and A are equal. By applying Lemma 2, we can replace the above equation with hard and soft interventions as follows:EQUATIONWhere ψλ (v a ) := sup ∆∈U X ψ(CF(v a , ∆)) -λd(v a , CF(v a , ∆)). The equations F ((a, u x + ∆)) = CF(v, ∆) and F ((a ′ , u x +∆)) = CF(v a ′ , ∆) hold true according to Lemma 2. Finally, by substituting va = CF(v, a) into the equation, we prove the equation:sup Q∈B δ (P) E v∼Q [ψ(v)] = inf λ≥0 λδ p + E v∼P sup a∈A ψ λ (v a ) ,where ψ λ (v) is defined asψ λ (v) := sup ∆∈X {ψ(CF 0 (v, ∆)) -λ p d(v, CF 0 (v, ∆))} ,Since, in this case, CF is equivalent to CF 0 , this completes the proof for case one.Now consider the scenario where sensitive attributes have parents. Eq. 17 shows in strong duality computation it needs to compute function in intervened M concerning the sensitive attributes levels.In this case, instead of using the structural causal model M, it is sufficient to employ the parent-free sensitive attribute SCM (Def. 2), M 0 . M 0 aligns with the semi-latent space and is compatible with the representation form outlined in Proposition 1. By adopting this strategy, we transform M into a model where sensitive attributes do not have parents. The proof procedure for M 0 remains the same as for M and completes the proof.Lemma 6 Let (Z, c) be a space with cost function c, P N an empirical probability measure based on observations {z i } N i=1 , and define Q as:Q = P N - 1 N δ z1 + 1 N δ z ′ 1where δ z1 and δ z ′ 1 are Dirac measures at z 1 and z ′ 1 , respectively. Then, the p-Wasserstein distance between P N and Q is given by:W c,p (P N , Q) = ( 1 N ) 1 p c(z 1 , z ′ 1 ).Proof. The definition of the p-Wasserstein distance between two probability measures P N and Q is:W c,p (P N , Q) = inf π∈Γ(P N ,Q) Z×Z c(z, z ′ ) p dπ(z, z ′ ) 1 p, where Γ(P N , Q) represents the set of all couplings of P N and Q. Since Q is obtained by transferring a mass of 1 N from z 1 to z ′ 1 , the optimal transport plan under the constraint that P N and Q differ only at two points involves only moving the mass 1 N from z 1 to z ′ 1 . The cost of this transportation is c(z 1 , z ′ 1 ) p , and because the entire mass 1 N is being moved:Z×Z c(z, z ′ ) p dπ(z, z ′ ) = c(z 1 , z ′ 1 ) p • 1 N .Therefore, substituting this into the formula for W p , we obtain:W c,p (P N , Q) = c(z 1 , z ′ 1 ) p • 1 N 1 p = 1 N 1 p c(z 1 , z ′ 1 ),thus proving the lemma.Lemma 7 Assume that f (a, x) is convex in x for each fixed a and continuous in both a and x. Also, assume f is uniformly continuous in x across a. If A is compact, then the function defined byg(x) = sup a∈A f (a, x)is convex and continuous in x.Proof. To show that g(x) is convex, consider any x 1 , x 2 in the domain and λ ∈ [0, 1]. By the definition of supremum and the convexity of f (a, x) in x,f (a, λx 1 + (1 -λ)x 2 ) ≤ λf (a, x 1 ) + (1 -λ)f (a, x 2 ).Taking the supremum over a in A on both sides, we get:sup a∈A f (a, λx 1 + (1 -λ)x 2 ) ≤ sup a∈A (λf (a, x 1 ) + (1 -λ)f (a, x 2 )).Using the properties of supremum,sup a∈A f (a, λx 1 + (1 -λ)x 2 ) ≤ λ sup a∈A f (a, x 1 ) + (1 -λ) sup a∈A f (a, x 2 ).Thus,g(λx 1 + (1 -λ)x 2 ) ≤ λg(x 1 ) + (1 -λ)g(x 2 ),proving that g(x) is convex.To show continuity of g(x) at a point x 0 , consider any sequence {x n } converging to x 0 . Since f is uniformly continuous in x, given ϵ > 0, there exists δ > 0 such that for all x, y with |x -y| < δ,|f (a, x) -f (a, y)| < ϵ for all a ∈ A.Thus,f (a, x n ) < f (a, x 0 ) + ϵ and f (a, x 0 ) < f (a, x n ) + ϵ for all a ∈ A and |x n -x 0 | < δ.Taking the supremum over a in A,g(x n ) ≤ g(x 0 ) + ϵ and g(x 0 ) ≤ g(x n ) + ϵ. This implies |g(x n ) -g(x 0 )| ≤ ϵ,establishing the continuity of g(x) at x 0 .Hence, we conclude that sup a∈A f (a, x) is convex and continuous in x.Let's consider the ℓ(v, y, θ) = h(Y -⟨θ, V⟩) (or in abbreviation ℓ(y)) or h(Y • ⟨θ, V⟩) where h : R → R has one of the forms |t|, max(0, t), |t -τ |, or max(0, t -τ ) for some τ ≥ 0First consider the case diam (A) = ∞. By property of CFDF for each Since the distance of v by its twins va is zero.Let z ∈ {z i } If P N the empirical distribution by lemma 6 it can be seen for observation Z = (v, y) the distributionQ a = P N - 1 N δ z + 1 N δ (va,y1) =⇒ W c,p (Q a , P N ) = 0 =⇒ Q a ∈ B δ (P N ).This equation results thatR δ (P N , θ) ≥ sup a∈A R δ (Q a , θ) ≥ R(P N , θ) - 1 N ℓ(v 1 ) + 1 N sup a∈A ℓ(v a )| ≥ 1 N sup a∈A ℓ(v a )| Let u = (u A , u X ) such that v = M u.By definition of hard intervention va is obtained by the formulava = (M -M pa ) × (u -(0, . . . , :=α a -u A , . . . , 0) T ) = M × u - : M A × α M × (0, . . . , α, . . . , 0) - : C M pa × u + : 0 M pa × (0, . . . , α, . . . , 0) = v -M A × α -Cwhere M pa refers to the effect of parents of sensitive variables and M A is the columns of matrix M related to sensitive attributes that show the effects of sensitive attributes on non-sensitive variables. By substituting that last equation in the loss function we have:R δ (P N , θ) ≥ 1 N sup a∈A ℓ(v -M A × α -C) ≥ 1 N sup α→∞ O(θ T M A × α)where B is some constant value. with the assumptions about loss function, all of them by choosing proper z 1 , its behavior when α is large enough is linear so can be approximated by its input value. Now Since the diam (A) = ∞ therefore the value of α goes to the infinity. Therefore to prevent the value of R δ (P N , θ) it needs that the expression θ T M A × α = 0 in the other word the P A (M T θ) needs to be zero. This condition implies that for all a ∈ A we have ℓ(v) = ℓ(v a ). By using strong duality 8 we have:EQUATIONwhereEQUATIONIn the equation 19, M 0 is reduced-form mapping of the parent-free sensitive attribute M 0 . By the definition it is easy in both SCM, the effect of the sensitive attributes is equal therefore P A (M T 0 θ) = P A (M T θ) = 0. Moreover since in M 0 the structure of non-sensitive attribute has not changed then P X (M T 0 θ) = P X (M T θ). Finally, by substituting θ 0 = P X (M T θ), it can be seen that the problem of finding worst-case loss quantity converts to the regular problem in space X . This problem was solved previously in works of [14, 58, 25, 66] . By using Theorem 3.2 and 3.3 and Proposition 4.1 and 4.2, of work by Chu et al. [14] , we can writeinf λ≥0 λδ + E v∼P N [ℓ λ (v)] = inf λ≥0 λδ + E ux∼(g # P N ) X [h λ (u x )] = R((g # P N ) X , θ 0 ) 1 p + δ ∥θ 0 ∥ * p = R(P N , θ) 1 p + δ P X (M T θ) * pThe equality R((g # P N ) X , θ 0 ) = R(P N , θ) holds by definition and property P A (M T θ) = 0. The last equation completes the proof of the first case. Now let's consider the case that diam (A) < ∞.Case: p ∈ (1, ∞). Lets consider Eq. 16 it implies that:EQUATIONwhere, l(u x ) = sup ua∈U A {ℓ(M (u a , u ′ x )} and η λ (u x ) = sup ∆∈U X l(u x + ∆) -λd X (u x , u x + ∆)}. By assumption, the whole type of loss functions are form h(⟨θ, v⟩) and convex and continuous. h can be written by M in the form h(⟨θ, M (u a , u x )⟩). Then l(u x ) = sup ua∈U A h(P A (M T θ)u a + P X (M T θ)u x ). Since all forms of function are uniformly continuous concerning the u x then lemma 7 implies that the l(u x ) is still continuous and convex. Theorem 6 and 7 of work [66] states that:Theorem. [66] Let ℓ : R → R be a non-negative, Lipschitz continuous and convex function. For an integer p ∈ (1, ∞), suppose that for any P ∈ P(X ), and ϵ ≥ 0, we have:sup Q∈B δ (P) E x∼Q [ℓ p (θ T x)] = E x∼P [ℓ p (θ T x)] 1/p + δ ∥θ∥ * p .By applying above theorem in Eq. 20 it can be seen:R δ (P) = E ux∼(g # P) X [ lp (u x )] 1/p + δ P X (M T θ) * p = E v∼P [sup a∈A ℓ p (v a )] 1/p + δ P X (M T θ) * p Case: p = 1.To complete the proof we use Theorem 2 and Corollary 2 of Gao et al. work [27] .Theorem [27] If ℓ is Lipschitz |ℓ(x 1 ) -ℓ(x 2 )| ≤ L ∥x k -x 0 ∥and satisfies tightness at infinity, i.e. for every v 0 there exists sequence {v k } ∞ k=1 ∈ V such that ∥x k -x 0 ∥ → ∞ we have:EQUATIONthen we have R δ (P) = R(P) + δ.L.Now back to the Eq. 20. It is necessary to show that l satisfies the Gao's theorem. Let h(t) be one of the functions |t|, (t -τ ) + , (|t| -τ ) + . All of loss function can be written as form h(y -⟨θ, v⟩) or h(y. ⟨θ, v⟩). It is easy to check that h is Lipschitz with constant 1 and there exist t k such that for each t 0 we have lim k→∞ |h(t0 + t k ) -h(t 0 )| |t k | = 1.To use this theorem for l, we need to prove that l is Lipschitz and has a tightness condition at infinity. Since the h is Lipschitz we know that:∀u a ∈ U A : |h(P A (M T θ)u a + P X (M T θ)u x ) -h(P A (M T θ)u a + P X (M T θ)u ′ x )| ≤ |P X (M T θ)(u x -u ′ x )| ≤ P X (M T θ) * ∥u x -u ′ x ∥ ⇒ sup ua∈U A |h(P A (M T θ)u a + P X (M T θ)u x ) -h(P A (M T θ)u a + P X (M T θ)u ′ x )| sup ua∈U A |h(P A (M T θ)u a + P X (M T θ)u x ) -h(P A (M T θ)u a + P X (M T θ)u ′ x )| = | sup ua∈U A h(P A (M T θ)u a + P X (M T θ)u x ) -sup ua∈U A h(P A (M T θ)u a + P X (M T θ)u ′ x )| = | l(u x ) -l(u ′ x )| ≤ P X (M T θ) * ∥u x -u ′x ∥ ⇒ l is Lipbschitz. To satisfy the condition of Gao's theorem, it remains to show for u 0x there exists sequence {u k x } ∞ k=1 such that lim∥u k x -u 0 x ∥→∞ | l(u k x ) -l(u 0 x )| ∥u k x -u 0 x ∥= L To prove it we consider that for function h there exists sequence such that lim k→∞ |h(t0 + t k ) -h(t 0 )| |t k | = 1. consider the specific point u 0 x and u a ∈ U A . Lets define t 0 = P A (M T θ)u a + P X (M T θ)u 0x . Then there exists t k that satisfies infinity tightness. Therefore there exist ∆ k ∈ U X such that P X (M T θ)u kx -P X (M T θ)u 0 x = t k therefore it can be written:1 =lim k→∞ |h(t 0 + t k ) -h(t 0 )| |t k | = lim k→∞ |h(P A (M T θ)u a + P X (M T θ)u k x ) -h(P A (M T θ)u a + P X (M T θ)u 0 x )| |P X (M T θ)u k x | ⇒ lim k→∞ |h(M T θ(u a , u k x )) -h(M T θ(u a , u 0 x )| ∥u k x ∥ = P X (M T θ) * ⇒ sup ua∈U A lim k→∞ |h(M T θ(u a , u k x )) -h(M T θ(u a , u 0 x )| ∥u k x ∥ = P X (M T θ) * ⇒ lim k→∞ sup ua∈U A |h(M T θ(u a , u k x )) -h(M T θ(u a , u 0 x )| ∥u k x ∥ = P X (M T θ) * ⇒ lim k→∞ | l(u k x ) -l(u 0 x )| ∥u k x ∥ = P X (M T θ) *In the above equation, since we have uniform convergence, we can change the limit and supremum. The last equation shows that there exits sequence of {u k x } ∞ k=1 satisfies tightness condition in ∞. By applying Gao's theorem we have:R δ (P) =E ux∼(g # P) X [ l(u x )] + δ P X (M T θ) * = E v∼P [sup a∈A ℓ p (v a )] + δ P X (M T θ) *The last equation completes the proof.The case diam (A) = ∞ corresponds exactly to the first part of the proof of Theorem 2, with the only difference being that in that theorem we have ∥h∥ Lip = 1, while in this case, we have ∥h∥ Lip = L h . Therefore we have the below equation:R δ (P) = R(P) + L h P X (M T θ) *Now consider the case diam (A) < ∞. To prove our assertion, we use Eq. 16 and it implies that:R δ (P) = inf λ≥0 λδ + E ux∼(g # P) X lλ (u x )where, l(ux ) = sup ua∈U A {ℓ(M (u a , u ′ x )} and η λ (u x ) = sup ∆∈U X l(u x + ∆) -λd X (u x , u x + ∆)}.To compute the right side of the above equation, we use the theorem 3.2 of work [14] that states.Theorem [14] . Let Z N := {z 1 , . . . , z n } ⊂ Z be a given dataset and P N be the corresponding empirical distribution. In addition, let c(•, •) be a cost function on Z × Z and δ ∈ (0, ∞) be a scalar. Suppose the loss function ℓ : Z × Θ → R, where satisfies the following assumptions:(A1) ℓ is Lipschitz respect o the cost function d at set Z N with L Z N θ ∈ (0, ∞); (A2) for any ϵ ∈ (0, L Z N θ ) and each z i ∈ Z N , there exists zi ∈ Z such that δ ≤ d(z i , z i ) < ∞ and ℓ(z i ) -ℓ(z i ) ≥ (L Z N θ -ϵ)c(z i , z i ).Then we have:sup P : W d,1 (Q,P N )≤δ E Q [ℓ(Z, θ)] = E P N [ℓ(Z, θ)] + L Z N θ δ.To use the above theorem we need that l satisfies conditions (A1) and (A2).A1. To prove the Lipschitz condition it can be seen:∀u a ∈ U X : |h(y -θ T M (u a , u x )) -h(y -θ T M (u a , u ′ x ))| ≤ L h |P X (M T θ)(u a -u ′ a )| ≤ L h P X (M T θ) * ∥u a -u ′ a ∥ ⇒ sup ua∈U X |h(y -θ T M (u a , u x )) -h(y -θ T M (u a , u ′ x ))| = | l(u a ) -l(u ′ x )| ≤ L h P X (M T θ) * ∥u x -u ′ x ∥The case h(y. ⟨θ, y⟩) is similar so we omit it. A2. To check that l satisfies (A2) condition we show that there exists sequence {∆ k } such that the ∥∆ k ∥ → ∞ for every v ∈ V and sequence v k = CF(v, ∆ k ) and we have:EQUATIONBy assumption about h we have: For each t 0 ∈ R there exists sequence of{t k } ∞ k=1 goes to ∞ then lim k→∞ |h(t 0 + t k ) -h(t 0 )| |t k | = L h . By changing variable v = M (u a , u x ). Let t 0 = y -θ T M (u a , u x ) and ∆ k ∈ X such that P X (M T θ)∆ k = t k it is clear ∆ k exist. No if we define v k = CF(v, ∆ k ) we have: L h =lim k→∞ |h(t 0 + t k ) -h(t 0 )| |t k | = lim k→∞ |h(y -θ T M (u a , u x ) + P X (M T θ)∆ k ) -h(y -θ T M (u a , u x ))| |P X (M T θ)∆ k | = lim k→∞ |h(y -θ T M (u a , u x + ∆ k )) -h(y -θ T M (u a , u x ))| |P X (M T θ)∆ k | = lim k→∞ |h(y -θ T v k ) -h(y -θ T v)| ∥P X (M T θ)∥ * ∥∆ k ∥ = lim k→∞ |h(y -⟨θ, v k ⟩) -h(y -⟨θ, v⟩)| ∥P X (M T θ)∥ * d(v k , v) =⇒ lim k→∞ |h(y -⟨θ, v k ⟩) -h(y -⟨θ, v⟩)| d(v k , v) = P X (M T θ) * .L hThe Last equation is valid because, by Holder inequality, there exists ∆ such that for all λ∆ the Holder inequality converts to equality. Now it is sufficient that find proper λ such that λ k = t k / P X (M T θ)∆ * so by define ∆ k = λ k ∆ we find sequence that holds the assertion.The case of h(y. ⟨θ, v⟩ is similar. By discussion of the first part, we can find ∆ k . Now since we have binary classification, so y ∈ {-1, 1} we define ∆k = sign(y)∆ k therefore for such ∆ k , we have y.P X (M T θ) ∆k ) = t k . By assumption, it can be written as:L h =lim k→∞ |h(t 0 + t k ) -h(t 0 )| |t k | = lim k→∞ |h(y.θ T M (u a , u x ) + y.P X (M T θ) ∆k ) -h(y.θ T M (u a , u x ))| |P X (M T θ) ∆k | = lim k→∞ |h(y.θ T M (u a , u x ) + y.P X (M T θ)∆ k ) -h(y.θ T M (u a , u x ))| |P X (M T θ)∆ k | = lim k→∞ |h(y.θ T M (u a , u x + ∆ k )) -h(y.θ T M (u a , u x ))| |P X (M T θ)∆ k | = lim k→∞ |h(y.θ T v k ) -h(y.θ T v)| ∥P X (M T θ)∥ * ∥∆ k ∥ = lim k→∞ |h(y. ⟨θ, v k ⟩) -h(y -⟨θ, v⟩)| ∥P X (M T θ)∥ * d(v k , v) =⇒ lim k→∞ |h(y -⟨θ, v k ⟩) -h(y -⟨θ, v⟩)| d(v k , v) = P X (M T θ) * .L h Let ℓ(v) = h(y -P X (M T θ)u x -P A (M T θ)u a ). By assumption h is Lipschitz so it ℓ is Lipschitz concerning each u x and u a and U A is bounded so it is compact. Then these properties imply uniformly continuous so we have:sup ua∈U A lim k→∞ |h(y -⟨θ, v k ⟩) -h(y -⟨θ, v⟩)| d(v k , v) = lim k→∞ sup ua∈U A |h(y -⟨θ, v k ⟩) -h(y -⟨θ, v⟩)| d(v k , v) = lim k→∞ | l(u x ) -l(∆ k )| ∥u x -∆ k ∥ = P X (M T θ) * .L hThe last equation satisfies the (A 2 ) condition because sincelim k→∞ | l(u x ) -l(∆ k )| ∥u x -∆ k ∥ = P X (M T θ) * .L h in other hand we have | l(u a ) -l(u ′ x )| ≤ L h P X (M T θ) * ∥u x -u ′ x ∥, then for each ϵ there exist ∆ k such that | l(u x ) -l(∆ k )| > (L h P X (M T θ) * ∥u x -∆ k ∥ -ϵ) ∥u x -∆ k ∥Now we can use Chu's Theorem and it implies that: R δ (P) = R cf (P) + L h P X (M T θ) * . The classification case is the same and it completes the proof.Let's prove the necessary condition. By first part proof of theorem 2, we haveR δ (P N , θ) ≥ 1 N sup a∈A ℓ(v a , y, θ).Therefore, for a finite solution to exist for the DRO problem, it is necessary that: sup a∈A ℓ(v a , y, θ) < ∞.To prove Eq. 10, we use again some parts of the proof of strong duality theorem1 and idea of proof of theorem 9.1 of Garcia's work [29] . It states:R δ (P N ) = sup Q∈B δ (P N ) E v∼Q [ℓ(v ′ , y, θ)] = sup Qx∈B δ ((g # P N ) X ) Qa∈P(A)      E u ′ x ∼Qx u ′ a ∼Qa ℓ g -1 ((u ′ a , u ′ x )) , y, θ,     where by discussion in lemma 4, we can suppose that Q x and Q a are independent of each other.For simplicity, we define J(u x , u a , θ) = ℓ(g -1 ((u a , u x )), y, θ). By assumption, since the f is twice differentiable, then (I -f ) -1 is also twice differentiable. Because the function g is obtained by I -f by removing the functional structure of sensitive attributes and is a set identity function instead of them, there are two functions g and g -1 . These results show that combination ℓ(g -1 ) is also twice differentiable, so the gradient of J exists concerning the u x . By using Taylor's expansion theorem if f : R → R is the function that has gradient then the first order estimation of f equals:f (x + h) = f (x) + ∇f (x) ⊤ h + 1 0 (∇f (x + th) -∇f (x)) ⊤ h dt.Let u x ∼ (g # P) X by writing Taylor's expansion around u x we have:E [J(u ′ x , u ′ a , θ)] = E [J(u x , u ′ a , θ) + ∇ x J(u x , u ′ a , θ) • (u ′ x -u x ) + 1 0 {∇ x J(u x + λ(u ′ x -u x ), u ′ a , θ) -∇ x J(u x , u ′ a , θ)} • (u ′ x -u x )dλ .Since the diam (U) < ∞ we can suppose that the space U is compact. By assumption J is twice differentiable, so ∇ x J(., u a , θ) is Lipschitz with constant ∥∇ x J(., u a , θ)∥ Lip that there exist L < ∞ such that ∥∇ x J(., u a , θ)∥ Lip ≤ L. By these assumptions, it can be written:E 1 0 {∇ x J(u x + λ(u ′ x -u x ), u ′ a , θ) -∇ x J(u x , u ′ a , θ)} • (u ′ x -u x )dλ ≤ E 1 0 ∥∇ x J(u x + λ(u ′ x -u x ), u ′ a , θ) -∇ x J(u x , u ′ a , θ)∥∥u ′ x -u x ∥ 2 e dλ = E 1 0 ∥∇ x J(., u ′ a , θ)∥ Lip ∥u ′ x -u x ∥ 2 e dλ = E 1 2 ∥∇ x J(., u ′ a , θ)∥ Lip ∥u ′ x -u x ∥ 2 e ≤ C 2 ∥∇ x J(., u ′ a , θ)∥ Lip E ∥u ′ x -u x ∥ 2 ≤ CL 2 E ∥u ′ x -u x ∥ 2 ≤ O(δ 2 ),where C is a constant that arises from the equivalence of norms in R d , it means that there exists C ∥.∥ e ≤ C ∥.∥. The inequality E ∥u ′ x -u x ∥ 2 ≤ δ 2 is valid because by definition Q x ∈ B δ ((g # P N ) X ) and the cost function in the space U X is expressed by the ∥u ′x -u x ∥ so by definition of B δ ((g # P N ) X ), for p ≥ 2 by applying Jensen's inequality we haveQ x ∈ B δ ((g # P N ) X ) ⇒E Qx [∥u ′ x -u x ∥ p ] 1 p ≤ δ ⇒ E Qx ∥u ′ x -u x ∥ 2 ≤ E Qx [∥u ′ x -u x ∥ p ] 2 p ≤ δ 2 .Since by assumption ∇J x is uniformly Lipschitz for different value of θ and u a therefore we have:R δ (P N ) = sup Qx∈B δ ((g # P N ) X ) Qa∈P(A) E [J(u x , u ′ a , θ) + ∇ x J(u x , u ′ a , θ) • (u ′ x -u x )] + O(δ 2 ),for O(δ 2 ) independent of θ and u a . The first expression of the above equation has the simple form:sup Qa∈P(U A )      E ux∼(g # P N ) X u ′ a ∼Qa [J(u x , u ′ a , θ)]      = E ux∼(g # P N ) X sup Qa∈P(U A ) E u ′ a ∼Qa [J(u x , u ′ a , θ)] = E ux∼(g # P N ) X sup u ′ a ∈U A {J(u x , u ′ a , θ)} = E ux∼(g # P N ) X sup u ′ a ∈U A ℓ(g -1 ((u x , u ′ a )), y, θ) = E v∼P N sup a∈A ℓ(v a , y, θ)The only term in the above equation that still depends on the Q is that term∇ x J(u x , u ′ a , θ)•(u ′ x -u x ).To remove this term we use extended Hölder inequality with the expectation that can be expressed using the following formula:E[|XY |] ≤ (E[|X| p ]) 1 p (E[|Y | q ]) 1 q ,and equality holds if and only if there exist constants c ≥ 0 such that:|Y | = c|X| p qalmost surely, c ≥ 0.. By using Hölder inequality, with the same reasoning we have:sup Qx∈B δ ((g # P N ) X ) Qa∈P(A)      E u ′ x ∼Qx u ′ a ∼Qa [∇ x J(u x , u ′ a , θ) • (u ′ x -u x )]      = sup Qx∈B δ ((g # P N ) X ) E u ′ x ∼Qx [ sup u ′ a ∈U A {∇ x J(u x , u ′ a , θ) • (u ′ x -u x )}] = E ux∼(g # P N ) X [ sup u ′ a ∈U A {∥∇ x J(u x , u ′ a , θ)∥ q * }] 1 q sup Qx∈B δ ((g # P N ) X )           E u ′ x ∼Qx ux∼(g # P N ) X [∥u ′ x -u x ∥ p ]    1 p        = δ E ux∼(g # P N ) X [ sup u ′ a ∈U A {∥∇ x J(u x , u ′ a , θ)∥ q * }] 1 pwhere equality can be attained whenever 1 ≤ p ≤ ∞ for a proper choice of u ′x -u x with (E[∥u ′ x -u x ∥ p ]) 1/p = δ. Therefore, R δ (P N ) = E v∼P N sup a∈A ℓ(v a , y, θ) + δ E v∼P N [sup a∈A {∥∇ CF ℓ(v, y, θ)∥ q * }] 1/q + O(δ 2 )where∇ CF ℓ(v, y, θ) = lim ∆→0 ℓ(CF 0 (v, ∆)) -f (v) ∥∆∥the last equation completes the proofs.B.5 Proof of Proposition 2.By Eq. 16 we have:R δ (P) = inf λ≥0 λδ + E ux∼(g # P) X lλ (u x )where, l(u x ) = sup ua∈U A {ℓ(M (u a , u ′ x )} and η λ (u x ) = sup ∆∈U X l(u x + ∆) -λd X (u x , u x + ∆)}. To prove we use Corollary 2 [28] for the l. First, we need to show that l satisfies the condition of Corollary 2 [28] . By assumption for v ′ ∈ V, L, M ≥ 0 such that |ℓ(v, y, θ) -ℓ(v ′ , y, θ)| < Ld p (v, v ′ ) + M for all v ∈ V and p ∈ [1, ∞). By setting v = g -1 ((u a , u a )) and v ′ = g -1 ((u ′ a , u ′ a )), and for simplicity ℓ(v) = ℓ(v, y, θ) we have:|ℓ(g -1 ((u a , u x ))) -ℓ(g -1 ((u ′ a , u ′ x )))| < L ∥u x -u ′ x ∥ p + M, ∀u x ∈ U X , u a ∈ U A ⇒ | l(u x ) -l(u ′ x )| ≤ L ∥u x -u ′ x ∥ p + Mwhere l(u x ) = sup ua∈U A ℓ(g -1 ((u a , u x ))). This equation implies that l satisfies the condition of corollary 2. So in consequence of corollary 2 and the equation 16 if we define uncertainty set:Bδ = (ω ik x ) i,k : 1 N N i=1 K k=1 u i x -ω ik ≤ δ, ω ik ∈ U XSince the casual fair metric d and loss function l do not depend on the sensitive part then Bδ is equivalent to the below uncertainty set.B δ = (w ik ) i,k : 1 N N i=1 K k=1 d p (v i , w ik ) ≤ δ, w ik ∈ V .By applying the uncertainty Bδ for l, the robust optimization problem has a form:Radv δ (P N ) = sup (ω ik ) i,k ∈ Bδ 1 N K N i=1 K k=1 l(ω ik ) = sup (ω ik ) i,k ∈ Bδ 1 N K N i=1 K k=1 max ua∈U A ℓ(g -1 ((u a , ω ik ))) = sup (w ik ) i,k ∈B δ 1 N K N i=1 K k=1 max a∈A ℓ( ẅik a )))and finally for Radv δ (P N ) we have:Radv δ (P N ) ≤ R n δ (P N ) ≤ Radv δ (P N ) + LD + M N K ,and it completes the proof.Lemma 8 Assume that the cost functions c and ĉ satisfy |c(x, x ′ ) -ĉ(x, x ′ )| < α for all x, x ′ ∈ R n and for some α ≥ 0. Then, for any λ ≥ 0, the difference between the λ-conjugates of f with respect to c and ĉ is bounded by λα:|f λ (x) -fλ (x)| ≤ λα for all x ∈ R n .Proof. To prove the proposition, we consider any x ∈ R n and examine the definitions of f λ (x) and fλ (x). Begin by expressing the bounds on ĉ: ĉ(x, x ′ ) ≤ c(x, x ′ ) + α and ĉ(x, x ′ ) ≥ c(x, x ′ ) -α.From these inequalities, for anyx ′ ∈ R n , f (x ′ ) -λĉ(x, x ′ ) ≥ f (x ′ ) -λ(c(x, x ′ ) + α) = f (x ′ ) -λc(x, x ′ ) -λα, f (x ′ ) -λĉ(x, x ′ ) ≤ f (x ′ ) -λ(c(x, x ′ ) -α) = f (x ′ ) -λc(x, x ′ ) + λα.Taking the supremum over all x ′ in the above expressions, we obtain: fλ (x) ≥ f λ (x) -λα and fλ (x) ≤ f λ (x) + λα.These two bounds together imply:|f λ (x) -fλ (x)| ≤ λα.Thus, the proof is complete, showing that the difference between the λ-conjugates of f is indeed bounded by λα.Lemma 9 Let c, ĉ : R n × R n → R be two functions such that |c(x, y) -ĉ(x, y)| < α for all x, y ∈ R n and some α > 0. For any real number p ≥ 1, the following inequality holds:|c(x, y) p -ĉ(x, y) p | ≤ p • M p-1 • α,where M ≥ max{|c(x, y)|, |ĉ(x, y)|} for all x, y.Proof. Consider the functions c and ĉ and any x, y ∈ R n . By the hypothesis, we have |c(x, y)ĉ(x, y)| < α. To find a bound on the difference of their powers, apply the mean value theorem to the function f (t) = t p , which is differentiable over R (or over R + if p is not an integer). The derivative of f is f ′ (t) = pt p-1 .Since f is continuously differentiable, there exists some ξ between c(x, y) and ĉ(x, y) such thatf (c(x, y)) -f (ĉ(x, y)) = f ′ (ξ) • (c(x, y) -ĉ(x, y)). Therefore, |c(x, y) p -ĉ(x, y) p | = |pξ p-1 (c(x, y) -ĉ(x, y))|.Using the bound |c(x, y) -ĉ(x, y)| < α and noting that ξ must be within the range of values between c(x, y) and ĉ(x, y), we have ξ p-1 ≤ M p-1 . Thus,|c(x, y) p -ĉ(x, y) p | ≤ pM p-1 |c(x, y) -ĉ(x, y)| < pM p-1 α.Lemma 10 ( [41] ) Fix some P ∈ P(Z), θ ∈ Θ and λ * ≥ 0 viaλ * := argmin λ≥0 {λδ p + E P [ℓ λ (Z, θ)]} .Then under ℓ Lipschitz and diam (Z) < ∞ assumptions, we have λ * ≤ Lδ -(p-1) .Proof. First, note that:λ * δ p ≤ λ * δ p + E P sup z ′ ∈Z {ℓ(z ′ , θ) -ℓ(z, θ) -λ * c p (z, z ′ )} = * ,since the left-hand side, is greater than the case where z ′ = z so it is positive. By the optimality of λ * in ℓ, the right-hand side can be further upper-bounded as follows for any λ ≥ 0: * ≤ λδ p + E P supz ′ ∈Z {ℓ(z ′ , θ) -ℓ(z, θ) -λc p (z, z ′ )} ≤ λδ p + E P sup z ′ ∈Z {Lc(z, z ′ ) -λc p (z, z ′ )} ≤ λδ p + sup t≥0 {Lt -λt p },using the Lipschitz property for the second line and setting t = c(z, z ′ ) in the third line. If p = 1, by setting λ = L, we obtain:λ * δ ≤ Lδ + sup t≥0 {Lt -Lt} = Lδ,which implies λ * ≤ L. For p > 1, using the optimal value t = (L/pλ) 1/(p-1) , we derive:λ * δ p ≤ λδ p + L p p-1 p -p p-1 (p -1)λ -1 p-1 .Minimizing the right-hand side with λ = L/pδ p-1 yields:λ * δ p ≤ Lδ ⇒ λ * ≤ Lδ -(p-1) ,resulting in the stated bound on λ * .Lemma 11 If f : A × R n → R is Lipschitz with respect to x uniformly in a, i.e., there exists a constant L such that for all a ∈ A and for all x, y ∈ R n , |f (a, x) -f (a, y)| ≤ L∥x -y∥, then sup a∈A f (a, x) is also Lipschitz in x.Proof. Let F (x) = sup a∈A f (a, x). We aim to show that there exists a constant L ′ such that for allx, y ∈ R n , |F (x) -F (y)| ≤ L ′ ∥x -y∥.Since f is Lipschitz continuous with respect to x uniformly in a with Lipschitz constant L, it holds for each a ∈ A and any x, y ∈ R n that |f (a, x) -f (a, y)| ≤ L∥x -y∥.Consider F (x) and F (y). By definition,F (x) = sup a∈A f (a, x) and F (y) = sup a∈A f (a, y). For any a ∈ A, since |f (a, x) -f (a, y)| ≤ L∥x -y∥, we can infer that f (a, x) ≤ f (a, y) + L∥x -y∥.Taking the supremum over all a ∈ A on both sides, we obtainF (x) ≤ F (y) + L∥x -y∥.Similarly, F (y) ≤ F (x) + L∥x -y∥.Combining these two inequalities, we find|F (x) -F (y)| ≤ L∥x -y∥.Therefore, F (x) = sup a∈A f (a, x) is Lipschitz continuous with Lipschitz constant L. This completes the proof. By the general strong duality theorem [26] we have:R δ (P * , θ * ) -Rδ (P N , θ * ) = sup Q:Wc,p(Q,P * )≤δ E Q [ℓ(Z, θ * )] - sup Q:W ĉ,p (Q,P N )≤δ E Q [ℓ(Z, θ * )] = inf λ≥0 {λδ p + E P * ℓ c λ (Z, θ * ) -inf λ≥0 {λδ p + E P N ℓ ĉ λ (Z, θ * ) ≤ λ N δ p + E P * ℓ c λ N (Z, θ * ) -(λ N δ p + E P N ℓ ĉ λ N (Z, θ * ) ) = E P * ℓ c λ N (Z, θ * ) -E P N ℓ ĉ λ N (Z, θ * )where the λ N := arginf λ≥0 λδ p + E P N ℓ ĉ λ (Z, θ * ) By assumption 2 and lemma 9,|ℓ c λ N (z, θ) -ℓ ĉ λ N (z, θ)| = sup v ′ ∈V ℓ(z ′ , y, θ) -λ N d p (v ′ , v) -sup v ′ ∈V ℓ(v ′ , y, θ) -λ N dp (v ′ , v) ≤ sup v ′ ∈V λ N |d p (v ′ , v) -dp (v ′ , v)| ≤ λ N p diam (V) p-1 M d N -η . This implies R δ (P * , θ * ) -Rδ (P N , θ * ) ≤ E P * ℓ c λ N (Z, θ) -E P N ℓ c λ N (Z, θ) + λ N p diam (V) p-1 M d N -η . If define λ * := arginf λ≥0 λδ p + E P ℓ c λ (Z, θ * ) , similarly, Rδ (P N , θ * ) -R δ (P * , θ * ) ≤ E P N ℓ ĉ λ * (Z, θ) -E P * ℓ c λ * (Z, θ) ≤ E P N ℓ c λ * (Z, θ) -E P * ℓ c λ * (Z, θ) + λ * p diam (V) p-1 M d N -η .We need to estimate the λ N and λ * . By using strong duality theorem by Eq. 16 we have:EQUATIONSo instead the solve problem for ℓ is it sufficient to prove our result for l(u x ) = sup ua∈U A ℓ(g -1 ((u a , u x )). At first, we show that l is Lipschitz on the space U X concerning the norm ∥.∥. By assumption, for each a ∈ A the function ℓ(g -1 ((u a , u x )) is also Lipschitz:ℓ(g -1 ((u a , u x )), y, θ) -ℓ(g -1 ((u a , u x + ∆)), y, θ) = ∥ℓ(CF(v, a), y, θ) -ℓ(CF(v, a, ∆), y, θ)∥ ≤ Ld(CF(v, a), CF(v, a, ∆)) = L ∥∆∥Now by using lemma 11 it can be concluded that the function l(u x ) = sup ua∈U A ℓ(g -1 ((u a , u x )) also has Lipschitz property with constant L. By Applying lemma 10 for equation 24 and (g # P N ) X , it can be seen λ N , λ * ≤ Lδ -(p-1) . Therefore until now, we have two inequalities:R δ (P * , θ * ) -Rδ (P N , θ * ) ≤ E P * ℓ c λ N (Z, θ) -E P N ℓ c λ N (Z, θ) + λ N p diam (V) p-1 M d N -η , Rδ (P N , θ * ) -R δ (P * , θ * ) ≤ E P N ℓ c λ * (Z, θ) -E P * ℓ c λ * (Z, θ) + λ * p diam (V) p-1 M d N -η ⇒ |R δ (P * , θ * ) -Rδ (P N , θ * )| ≤ sup f ∈L c Z f (z)d(P N -P * )(z) + Lδ 1-p p diam (V) p-1 M d N -η , where L c = {ℓ c λ (•, θ) : λ ∈ [0, Lδ 1-p ],θ ∈ Θ} is the DR loss class. In the remaining part of the proof, we estimate sup f ∈L c | Z f (z)d(P * -P N )(z)| using conventional methods from statistical learning theory. According to assumption 2, the functions within F are limited as shown:0 ≤ ℓ c λ (v, θ) ≤ sup v ′ ∈V ℓ(v ′ , y, θ) -λd(v, v ′ ) ≤ sup v ′ ∈V ℓ(v ′ , y, θ) ≤ M.similar to the proof Theorem 3 [41] , by utilizing the bounded-differences inequality and symmetrization, we derive that:sup f ∈L c Z f (z)d(P N -P * )(z) ≤ 2R N (L c ) + M log 2 ϵ 2Nholds with a probability of at least 1 -ϵ, where R n (L c ) represents the Rademacher complexity ofL c : R N (L c ) = E sup f ∈L c 1 N N i=1 σ i f (Z i ) .In the proof of Theorem 2 [41] , the authors has proved:R N (L c ) ≤ 24C(L) √ N + 24L.diam (V) p √ N δ p-1 .where C(L), the entropy integral of of loss class. By applying this result it can be written:|R δ (P * , θ * )-Rδ (P N , θ * )| ≤ M log 2 ϵ 2N + 48C(L) √ n + 48L.diam (V) p √ nδ p-1 +Lδ 1-p p diam (V) p-1 M d N -ηSince the prove does depend on value of θ, then |R δ (P * , θ dro N ) -Rδ (P N , θ dro N )| also satisfies in the above inequality. By combining two terms with probability 1 -ϵ we have, • CDRO: Our approach, as described in this paper, is formulated as follows:inf θ∈Θ sup Q ∈ B δ (P) E z∼Q [ℓ(z, θ)]For our loss function ℓ, we use the binary cross-entropy loss.The majority of the experimental setup is based on the work of Ehyaei et al. [22] . For each dataset and its respective label, we use a generalized linear model (GLM). Each training objective is applied to four different datasets, using 100 different random seeds. The optimization process is performed using the Adam optimizer with a learning rate of 10 -3 and a batch size of 100. After optimizing the benchmark time and considering the training rate, we set the number of epochs to 10 to ensure comparability in benchmarking.To assess the performance of various training methods concerning accuracy, unfair area, counterfactual fairness, and adversarial robustness, we employ seven distinct metrics as outlined below:• Acc: The accuracy of the classifier, is represented as a percentage.• U δ : The proportion of data points within the unfair area with a radius of δ.U δ := P {v ∈ V : ∃v ′ ∈ V s.t. d(v, v ′ ) ≤ δ ∧ h(v) ̸ = h(v ′ )} .• R δ : The fraction of data points that are vulnerable to adversarial perturbations within a radius of δ. This metric coincides with the unfair area in cases where no sensitive attribute is considered.R δ := P {v ∈ V : ∃∆ ∈ V s.t. d(v, CF(v, ∆)) ≤ δ ∧ h(v) ̸ = h(CF(v, ∆))} .• CF : The percentage of data points that exhibit counterfactual unfairness. This metric aligns with the unfair area when the perturbation radius is zero.CF := P {v ∈ V : ∃a ∈ A s.t. h(v) ̸ = h(v a )} .In this section, we present additional simulation results. CDRO performs well across all datasets except for the R δ measure in the Adult dataset, likely because the linear model does not fit the SCM well. Nevertheless, CDRO demonstrates robustness and counterfactual fairness, as shown in Table 1 , making it the preferred model when balancing both accuracy and fairness. Our theoretical framework bridges adversarial robustness, distributional robustness, individual fairness, and causality, aligning with the core pillars of responsible AI. By demonstrating the connection between these areas, we aim to inspire further research at their intersection and contribute to the development of safer, more equitable AI models for society. This approach holds the promise of improving decision-making under uncertainty while ensuring fairness and mitigating the impact of adversarial perturbations.However, we acknowledge several limitations and ethical implications inherent in our approach. While our method produces fair and robust predictions under specific conditions, it is important to note that it fundamentally relies on a machine learning model, which may inherit the same vulnerabilities as the original model in areas not explicitly addressed in this work such as multiplicity, privacy breaches, lack of explainability, and safety/security concerns. Additionally, our work operates under simplifying assumptions regarding the fairness notion. In real-world applications, fairness is a complex and context-dependent concept. Therefore, it is essential to define fairness carefully and consider multiple dimensions of fairness when applying our approach. We emphasize that this work is a proof of concept, and we strongly recommend involving diverse stakeholders, including ethicists, domain experts, and affected communities, before applying our approach to high-risk application domains. It's important for users to be aware of these limitations and potential biases that might not be fully addressed by our framework. 
 
 Appendix:  


Decision: The paper introduces Causally Fair Distributionally Robust Optimization (CDRO) framework to address individual fairness in machine learning using Wasserstein Distributionally Robust Optimization (DRO). The framework integrates causal modeling with DRO to handle fairness concerns involving sensitive attributes. It presents a dual formulation of DRO that simplifies the problem into a more tractable form, estimates regularizers for linear and non-linear Structural Causal Models (SCMs), and provides finite sample error bounds. Standard fairness evaluation datasets are used in empirical evaluations to demonstrate the effectiveness of the proposed method.

The reviewers found the combination of causal modeling, DRO, and rigorous optimization frameworks to address individual fairness novel. They appreciate the novelty in theory and relevance in practice. In general, the paper is found beneficial to the community with sufficient contributions. As for feedback for improvement, the reviewer mentioned that the paper could benefit from clearer explanations and more examples to improve readability. They also noted that the literature review part could expand significantly, and the discussions with existing works can further improve. There were additional comments for improving the readability and presentation of the paper (please see individual reviews).

Reviews: review: paper_summary: paper_summary: This paper proposes a novel framework called Causally Fair Distributionally Robust Optimization (CDRO) to address individual fairness in machine learning. It combines causal modeling with distributionally robust optimization, using a causally fair dissimilarity function (CFDF) to measure individual similarity while considering sensitive attributes. The framework provides a strong duality theorem, enabling efficient computation of worst-case losses under distributional uncertainty. It offers explicit solutions for the regularizer in linear Structural Causal Models (SCMs) and estimates it for non-linear SCMs, mitigating overfitting and ensuring fairness. Additionally, the framework provides finite sample guarantees for convergence even with unknown SCMs, enhancing its practicality. Empirical evaluations on real-world and synthetic datasets demonstrate CDRO's effectiveness in reducing unfairness while maintaining accuracy compared to other methods.
strength_weakness: strength_weakness: - Please define the abbreviation SCM before its first use in line 49.

- The experimental setting description could be improved. Consider providing an algorithm for the proposed approach to enhance clarity.

- The paper appears to have considerable overlap with Ehyaei et al. (https://arxiv.org/pdf/2310.19391):
  - The first contribution claimed in Section 1.1 (line 64) seems to have been previously established by Ehyaei et al.
  - Definition 1 and Proposition 1 in Section 3 appear to closely resemble Definition 2 and Proposition 1 in Ehyaei et al. Please clarify the novelty of these elements.

- The framework's reliance on an additive noise model assumption may limit its applicability in complex real-world scenarios. Could you discuss potential impacts on CFDF accuracy and fairness guarantees, and any plans to address this limitation?
questions: questions: Given that the proposed approach demonstrates lower prediction accuracy compared to existing methods, could the authors provide insights into potential factors contributing to this outcome? Additionally, how might this trade-off between prediction accuracy and other performance metrics be justified in the context of the method's overall objectives?
limitations: limitations: NA
paper_summary: paper_summary: This submission studies the connection between Wasserstein Distributionally Robust Optimization (DRO) and individual fairness in certain Structural Causal Models (SCMs). Namely, it is first shown that, in the case that the SCM at hand is an Additive Noise Model (ANM) with known structural equations, one may define a Causally Fair Dissimilarity Function (CFDF) on the feature space in a canonical manner. 

With this, the remainder of the paper concerns the problem of DRO of the risk function (i.e. minimization of $\mathcal R_{\delta}(\mathbb P,\theta)$ the worst-case risk over all distributions which lie in a (Wasserstein) ball of radius $\delta$ from $\mathbb P$). Notably, a dual form for $\mathcal R_{\delta}$ is provided by extending known results; this effectively converts the infinite-dimensional maximization problem defining $\mathcal R_{\delta}$ to a finite dimensional minimization problem. In the case that the center $\mathbb P$ for the Wasserstein ball in $\mathcal R_{\delta}$ is an empirical measure from $N$ samples, $\mathbb P_N$, it is shown that, under certain assumptions on the loss function, $\mathcal R_{\delta}$ can be recast exactly in terms of the standard empirical risk or the objective from the counterfactually robust optimization problem depending on the size of the diameter of the set of sensitive attributes. Under weaker assumptions on the loss and that the SCM is linear, a first order (in $\delta$) expansion of $\mathcal R_{\delta}$ is also provided. Next, it is demonstrated that standard adversarial optimization can be used to approximate DRO. Finally, the rate of convergence of $\mathcal R_{\delta}(\mathbb P_{\star},\hat \theta_N^{\mathrm{dro}})$ to $\inf_{\theta\in\Theta}\mathcal R_{\delta}(\mathbb P_{\star},\theta)$ is characterized.   

The paper concludes with a numerical study of the described causally fair DRO (CDRO). Namely, a comparison between the CDRO and other common approaches is provided on real-world and synthetic datasets. It is shown empirically that the CDRO exhibits slightly lower accuracy than the other models, but yields a lower unfair area (this is especially evident in the COMPAS and LIN datasets).
strength_weakness: strength_weakness: 1. It is difficult to get a sense for how strong some of the assumptions made in this work are. Although most of the assumptions are coupled with some examples of cases where they apply, I believe it would be relevant to provide some rationale for why these assumptions are necessary or describe primitive classes of examples where these assumptions hold rather than just some specific examples. 

2. Assumption 2 (iii) requires estimation of the CFDF; it would be useful to expand a bit more on this assumption keeping in mind the above point or at least provide some heuristics for what rates one can expect in general. 

3. More generally, the derived results would benefit from some additional discussion regarding their implications.
questions: questions: 1. I believe there is a small mistake regarding assumptions (i)-(ii) and the given examples. Notably the quantile loss is 1-Lipschitz, but $|h(t_0+t_k)-h(t_0)|=|\gamma t_k|$ if $t_0\geq 0$ or $|(\gamma-1)t_0+\gamma t_k|$ otherwise. In either case the limit from assumption (ii) is $\gamma\neq 1$. Perhaps there is a sign mistake? 

2. In Theorem 4 it is stated that the necessary condition for the existence of an infinite DRO solution is that [...]. Should it not be a finite DRO solution?
limitations: limitations: The authors address the limitations of their work in the conclusion and the broader impact is addressed in the appendix.
paper_summary: paper_summary: This paper uses wasserstein distributionally robust optimization to address individual fairness concerns with causal structures and sensitive attributes.
strength_weakness: strength_weakness: It does not seem easy to scale up this method.

minor issue: the DRO objective in line 142 should be sup_{Q in B_delta(P)} En_Q [l(Z,theta)].
questions: questions: How are the causal relationships determined for the experiments? If one assumes no causal relationship (e.g. i.i.d. formulation), does that impair the performance significantly?
limitations: limitations: adequately addressed
paper_summary: paper_summary: The authors propose a novel framework to enhance individual fairness guarantees under a Wasserstein distributionally robust optimization strategy. For such purposes, they employ counterfactuals based on the underlying causal structure of the model at hand. They further propose an alternative with theoretical guarantees for when the causal structure is unknown.
strength_weakness: strength_weakness: W1 - The text lacks clarity in several areas. It becomes quite technical at times, and it would benefit from including more examples alongside the technical explanations, particularly in the introduction, to help readers develop a better intuition. Additionally, there are instances where the ideas presented lack cohesion and do not clearly relate to one another, making the overall narrative difficult to follow. Improving the flow and connection between ideas would enhance the readability and comprehension of the text.

W2 - The paper does not cite many significant works on individual fairness and algorithmic fairness in general. For instance, citing [1] would be relevant, as it shares many similar points with the presented work. It would be beneficial to outline both the similarities and differences between them. Additionally, the related works section overlooks a significant body of literature on algorithmic fairness that is based on causality. These works are numerous and, in my opinion, should be acknowledged.

W3 - It is not very clear which are the main advantages and benefits of the proposed approach with respect to existing works in the literature. 

W4 - As I understand it, the authors propose modifying only the sensitive attribute while leaving all other attributes unchanged (please correct me if I am wrong). However, this approach can create unrealistic twins. For example, consider a male instance with a height of 1.85m, which is a typical height in a European country. If we change the gender to female without adjusting the height, the resulting instance would be an outlier and not representative of a typical female, making the two instances not equivalent. In the presence of such unrealistic twins, the classifier's decisions regarding these instances may not be informative. This issue arises because bias exists not only in the sensitive attribute but also in the non-sensitive attributes; for example, gender and income can be highly correlated.  Therefore, when we change the sensitive attribute of an instance, in order to create an ‘equivalent’ instance of the other gender, non-sensitive attributes should as well be modified. That is, to ensure the relevance of their analysis, the authors should verify that the twins they consider are realistic and plausible instances. For more insight on this matter, see [1].

W5 - The empirical evaluation of the proposal is poor. It does not include popular benchmark fairness-enhancing interventions, and only a few classification tasks are considered, despite the availability of numerous widely-used datasets in the algorithmic fairness literature. Besides, there is no deep discussion regarding the results. 

W6 - The model lacks any discussion or insights regarding its computational complexity or cost.

W7 - The acronym SCM is used before it is defined: it is first used in line 49, but it is defined in line 89.

W8 - (minor) I suggest moving the related works section into a separate section, as it represents a distinct aspect of the work.

W9 - (typos), line 60 (Our → our), line 113 (variables[50] → variables [50]), line 204 (space 9 ??)

[1] De Lara, L., González-Sanz, A., Asher, N., Risser, L., & Loubes, J. M. (2024). Transport-based counterfactual models. Journal of Machine Learning Research, 25(136), 1-59.
questions: questions: Q1 - Which are the main benefits/advantages that are provided by this proposal with respect to existing works in the literature?

Q2 - Does the approach use the conventional Wassterstein-ball based uncertainty set or is the uncertainty set considered the one from equation (16)? Is there any equivalence between them? 

 Q3 - Recognizing that assuming knowledge of the underlying causal structure for a given classification task is generally unrealistic, the authors propose an alternative approach that operates under more realistic conditions, where only a set of samples/instances is available. They offer guarantees as long as the assumptions outlined in Assumption 2 are met. However, it is unclear how realistic these assumptions are. Are they typically satisfied in real-world applications? Can you provide real examples where these assumptions hold true?

Q4 - Could this method be employed in classification tasks beyond tabular data?

Q5 - What do you mean by ‘We further estimate the regularizer in more general cases and explore the relationship between DRO and classical robust optimization.’? Where is this claim validated in the main text?
limitations: limitations: The authors discuss the limitations of their method in Section 6.

Comments: title: Request for Reviewer Engagement During Rebuttal Process
Reply to rebuttal
Enhancing Causal Consistency in Optimal Transport Applications
comment: We respectfully wish to express our concern regarding the lack of response from one of the reviewers during the rebuttal process. We invested significant effort in thoroughly addressing the concerns raised, providing a detailed response of approximately 6,000 characters to ensure that each issue was adequately covered. We were hopeful that the reviewer would engage with our rebuttal, as this dialogue is crucial for ensuring that all points are fully understood and resolved. Given the importance of this interaction, we kindly ask whether the reviewer feels that their concerns have been satisfactorily addressed or if there are any remaining questions that we can clarify during this stage. We believe that this feedback loop is essential to the integrity of the review process and the fair evaluation of our work.
I thank the authors for their clarification. I would like to maintain my score.
Thank you for your insightful comments, which have improved the clarity and impact of our work.

Our primary goal was to establish a theoretical framework for optimal transport tools tailored to dissimilarity cost functions derived from causal models, especially when data originates from such models. We argue that traditional metrics like $l\_p$ norms may not preserve causal relationships in these scenarios.

Due to space constraints, we focused on a fair learning example to demonstrate our method's effectiveness, though many applications remain. A particularly promising application is in generative adversarial networks, where our approach, not only compares distributions but also preserves the original data's causal structure.
Thank you for your responses! My concerns have been partially addressed, and I am willing to raise my score to 5. However, as you mentioned, "we faced challenges in finding datasets and methods compatible with our problem," I still have reservations about the applicability of the proposed approach to a wide range of real-world data scenarios.
I have read the authors' rebuttal.
Their response has answered the questions I raised.