Wasserstein Distributionally Robust Optimization through the Lens of Structural Causal Models and Individual Fairness
Authors: Ahmad Reza Ehyaei
Golnoosh Farnadi
Samira Samadi
Venue: NeurIPS 2024 poster (2024)
Keywords: Wasserstein Distributionally Robust Optimization
Individual Fairness
Structural Causal Model
Regularized Optimization
Primary area: fairness

Abstract: In recent years, Wasserstein Distributionally Robust Optimization (DRO) has garnered substantial interest for its efficacy in data-driven decision-making under distributional uncertainty. However, limited research has explored the application of DRO to address individual fairness concerns, particularly when considering causal structures and discrete sensitive attributes in learning problems.
To address this gap, we first formulate the DRO problem from the perspectives of causality and individual fairness. We then present the DRO dual formulation as an efficient tool to convert the main problem into a more tractable and computationally efficient form. Next, we characterize the closed form of the approximate worst-case loss quantity as a regularizer, eliminating the max-step in the Min-Max DRO problem. We further estimate the regularizer in more general cases and explore the relationship between DRO and classical robust optimization. Finally, by removing the assumption of a known structural causal model, we provide finite sample error bounds when designing DRO with empirical distributions and estimated causal structures to ensure efficiency and robust learning.

Wasserstein Distributionally Robust Optimization Through the Lens of Structural Causal Models and Individual Fairness
Ahmad-Res Rezad-Reza Ehyaei
Max Planck Institute for Intelligent Systems, Tübingen Al Center, Germany
ahmad ehyaei weweeeeitebbiinnnnieee mpg.ee...mp..e......
Golnoosh Farnadi"
Mila Québec Al Institute McGill University, Montréal, Canada f farnadig@mila, quebec
Samira Samadi
Max Planck Institute for Intelligent Systems, Tübingen Al Center, Germany ssamadi wtw.eiiintttiinnnnnnnnnnmmme mpg. de
Abstract
In recent years, Wasserstein Distributionally Robust Optimization (DRO) has garnered substantial interest for its efficacy data-driven decision-making under distributional uncertainty. However, limited research has explored the application DRO to address individual fairness concerns, particularly when considering causal structures and sensitive attributes in learning problems. To address this gap, we first formulate the DRO problem from causality and individual fairness perspectives. We then present the DRO dual formulation as an efficient tool to convert the DRO problem into a more tractable and computationally efficient form. Next, characterize the closed form of the approximate worst-case loss quantity as a regularizer, eliminating the max-step in the min-max DRO problem. We further estimate the regularizer in more general cases and explore the relationship between DRO and classical robust optimization. Finally, by removing the assumption of known structural causal model, provide finite sample error bounds when designing DRO with empirical distributions and estimated causal structures to ensure efficiency and robust learning.
Introduction
Machine learning models must address discrimination because they often reflect and amplify biases present in their training datasets [3T]. These biases can significantly influence decisions in domains
such as healthcare 30,, education [ recruitment [8]. and lending services [6] Consequently,
these decisions disproportionately affect individuals based sensitive attributes like race gender, perpetuating systemic discrimination.
To address and quantify unfairness, researchers have developed concepts like group fairness and individual fairness |5] 4 Group fairness aims to achieve equitable outcomes across demographic groups, while individual fairness ensures that similar individuals receive similar treatment. Formally, with V as the feature space and y as the label space, a model h V + y ensures individual fairness
"Lead scientific advisor on the project
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

if it satisfies the condition in 200]:
((h(w):(ee)) S v') for all o' E EV,
(I)
where dy and dy are dissimilarity functions, often referred to fair metrics the imput and output spaces. These functions capture the proximity of individuals and I. E IR+ is Lipschitz constant. The metric dy reflects the intuition about which instances should be considered similar by the model.
Due to challenges in defining such metrics, group fairness is often prioritized fairness literature because it more straightforwardly addresses observable disparities among distinct groups, making measurement and implementation easier in practice [. Therefore, it is crucial to study and formulate individual fairness under different assumptions in machine learning.
Individual fairness can be achieved through robust optimization methods such as Wasserstein DRO, which has gained significant attention for its applications in learning and decision- -anking [55] 43 DRO incorporates a regularization term to mitigate overfitting 17 26 59
By using a fair metric as the transportation cost function in computing the Wasserstein distance, models are designed to deliver consistent performance across varied data distributions, ensuring similar individuals receive comparable outcomes, thus satisfying individual fairnsss.
Incorporating causal structures and sensitive attributes into data models complicates using an indi- vidual fair metric as a cost function within the DRO framework. The fair metric must account for perturbations in sensitive attributes based on counterfactuals to ensure counterfactual fairness [22] This can violate the positive-definite property, where d(v, , v') = 0 implies e = v', a key assumption in many DRO theorems [55
Although previous works fairness, they often do not explore the implications when causal structures and sensitive attributes are present the learning problem. These studies are typically limited to linear Structural Causal Model (SCM) with specific metrics and do discuss the form of the regularizer for other classical DRO theorems when using a fair metric. To accurately compare our work with related studies, we will postpone this discussion until after presenting our results in Section
42
70
69 57 have attempted to apply DRO to address individual
1.1 Our Contributions
In this work, we adopt the definition of fair metric from [22] to define a Causally Fair Dissimilarity Function (CFDF), which delineates how to establish a fair metric through causality and sensitive attributes. Using CFDF, we introduce Causally Fair DRO and present a strong duality theorem for our approach. Under mild assumptions about CFDF and causal structure, we demonstrate that the DRO regularizer can be estimated, or in some cases can be explicitly solved. This estimation often leads to being more practical and computationally efficient than solving the min-max problem in (4) as supported by advancements in algorithms from previous research such as [14] [5]. Finally, Our numerical analysis of both real and synthetic data demonstrates the practicality of theoretical framework in real-world applications. ($ 5) In summary, the main contributions of this work are:
Define causally fair dissimilarity function, an individual fair metric incorporating causal structures and sensitive attributes (Def. [ along with its representation form (Prop. [].
Define a causally fair DRO problem with causally fair dissimilarity function cost (5 4).
Present the strong duality theorem for causally fair DRO (Thm. D
Provide the exact regularizer for linear SCM under mild conditions for the loss function in regression and classification problems (Thm. and Thm. 3
Estimate the first-order causally fair DRO regularizer for non-linear SCM (Thm.
Provide the relation between classical robust optimization and causally fair DRO (Prop.
Demonstrate that under unknown SCM assumptions, by estimating the SCM cost function, we have finite sample guarantees for convergence of empirical DRO problems (Thm.
2 Preliminaries & Notations
Data Model. Let V E V denote a vector of feature space (predictor variables) and let Y y represent the response variable, such that Z = (V, Y) comprises the observation variables with an

---

underlying probability P... Furthermore, assume that the feature vector V = (A, X) comprises both sensitive attributes A E AA and non-sensitive attributes X X. Let {a (u', )= represent the observations used to construct the empirical distribution NN defined as PN := N Σ i-1 62t, where 6= is the Dirac delta function. Given loss function 8 Z
R. the risk function for parameter 0 E @ and probability measure IP is R(P,, 0) = Ez [e(Z, 0) This leads to the common empirical risk minimization approach. This method seeks to find the minimizer AN within the set N E arg mingge R(PN N. f),, as an empirical way to obtaining the optimal solution 8., which is given by 0- infaee R(P..)) 0).
Assume the feature space is represented by a structural causal model (SCM) M (G, V, U.) [51]. This model includes structural equations (V; := f:(V Pa(s), U,) which delineate the causal relations among an endogenous variable V.. its causal predecessors V Pa(i), and an exogenous variable U; representing unobservable factors. The model's structure is encapsulated in a directed acyclic graph G. Exogenous variables are posited as mutually independent, enabling Pv to be expressed II; FU., assuming causal sufficiency and excluding hidden confounders 53
Counterfactuals. In causal structures, data perturbation is achieved through counterfactuals, which are derived from interventions in SCMs. These interventions, conducted using do-calculus, include both hard and soft types [51]. Hard interventions fix subset I S {I...., n of features Vz to constant r, modifying their causal connections within the causal graph while maintaining the structural equations of other features [51] This type of intervention is denoted as Mdo(Vzz and its structural equations are obtained by:
(V,:= := To Vi V (V U.), Vi IJ.
Soft interventions, on the other hand, adjust the functions in the structural equations, such as through additive interventions, without disrupting existing causal links [53). In an additive (or shift) intervention, value A E IR" is added to each feature within the SCM to enact manipulation:
(V. == f V Pa(()) U.) + A.)
In SCMs, counterfactuals are computed by modifying structural equations to reflect hard interventions on specific variables, thus exploring what would occur if the intervention was applied. Under the assumption of acyclicity, a unique function F 2 + V exists such that F(u) v. Acyclicity remains unchanged by either hard or shift interventions, allowing for the existence of modified functions pdo(Vz==r) and ===a( corresponding to these interventions, respectively. The counterfactual outcome for a hard intervention can thus be calculated using CF(u, r) = (F (F-)( and similarly, for a shift intervention, it is defined CF(u, A). These interventions are frequently applied in this analysis.
Counterfactuals involving the modification of sensitive attributes (termed twins) are essential for addressing individual-leve fairness [40] (44.. Twins are generated by altering the sensitive attribute from to a' across its domain A. For any instance, E V. a set of counterfactual twins is produced as U = CF(u, a) : a E A). facilitating the analysis fairness by comparing outcomes under different sensitive attribute values.
Counterfactual Identifiability To estimate the effects of interventions from observational data, counterfactuals must be identifiable within causal framework. A notable example of such identifi- able SCMs is the additive noise models (ANMs), which suggest that structural equations can be represented as:
(Vo== = /((VPa()) + U. - U-
= v=(1-)-((V)
(2)
leading to bijective mapping between U, and V,, ensuring no loss of information from exogenous to endogenous variables 55). This relationship implies that V can be derived from U through bijective reduced-form mapping F (T f)-1, where T(x)) = x is the identity function. Besides ANM, there are other counterfactually identifiable models such LSNM (34] and PNL [71]. However, for the sake of simplicity, our focus remains on ANM. Linear SCMS is specific instance of ANMs, characterized by linear functions f.
Individual Fairness Through Robustness. In machine learning, individual fairness 20 achieved through robustness by ensuring that similar individuals receive similar outcomes, regardless of
3

---

variations in their inputs. This concept aligns with the notion of Lipschitz continuity in decision functions (Eq. D. where small changes imput should not lead to excessively large changes in output.
Depending on how the uncertainty set is defined, various types of robust optimization can be employed. In adversarially robust optimization 44 77, the uncertainty set is defined by introducing slight perturbation based on the metric d to the input data. The goal is to find the optimal 0 that minimizes risk even under the worst-case perturbation quantity:
Rg (T,
E
sup d((e,+A))
E(v + U.
unF
where p E oo]. This formulation ensures that the optimization considers the maximum potential loss within the defined perturbation bounds.
In counterfactually robust optimization [40] 37
64 23 24], the uncertainty set is generated by
twins, which are obtained by creating counterfactuals concerning all levels of the sensitive attribute. In this scenario, the worst-case loss quantity is obtained by calculating the maximum loss over the twins of the input data:
Rdd (PP, 0)
E
sup E(ia, y. 0) LEEA
Distributionally Robust Optimization 443 55 is a data-driven approach designed to minimize the discrepancies between in-sample and sample expected losses, using ambiguity sets based on Wasserstein distances. Consider lower semi-continuous cost function c(:,-): Z x 2 - [0, oo] that satisfies c(z, 2) = 0
z E Z, serving as a fair metric. The optimal transport cost between two distributions IP, Q E PP [), ), is represented by:
Wa, cp (P, Q)
min TEP(ZxZ)
: = P. P2 Q
Here, E P(Z X Z) denotes the set of all joint probability distributions, and ग and T2 are the marginals of under first and second coordinates [54 63). When c(z, 2') acts as metric (in mathematics term) on Z, Wep is called the Wasserstein distance [63]
An important ingredient in the DRO formulation is the description of the distributional uncertainty region B5 that is defined by optimal transport cost:
Bs (P) = (Q P(V) W c. (Q.P) <
DRO problem minimizes worst-case loss quantity:
Rs (1P 0) A
sup
(Z]
(4)
Q E Bx())
and obtained the f e arg minge R& N,, 0). theorem (26 [66,, which converts The theorem states that:
The main tool in DRO is the strong duality problem into a finite optimization problem.
sup (P),(P)
A20 [w(w)] = inf {
(AOP + E (Vx(1)) mat
(5)
where 4/x (v) is defined (1) == sup (10(1 (d)
3 Causally Fair Dissimilarity Function
The key to robust optimization and individual fairness is the metric that measures individual similarity. This section outlines the properties of such metric in causal framework protect sensitive attributes. We begin with an illustrative example.
Example 1 Let Mj and M2 represent two SCMs describing the relationships among the variables gender (G), education (E), and income (I). Mi models these variables as independent, whereas M2 specifies a linear causal relationship:
G = Ug,
UG B(0.5)
G Uc. E = U+U,.
Ue B(0.5)
M1
E = Ug.
Ug N00,1)
M₂
Ug ~
= U,
1(0,1
1==++E++U;
U U,,A((011)

---

Where Ug represents the population distribution of gender. modeled by Bernoulli distribution, while U and U, are intrinsic talents for academic and income achievements, respectively, modeled by normal distributions. To compare individuals, let's consider the L1 norm on non-sensitive attributes (d(u, v²) = le -' If two individuals have less than a O. unit difference, they are deemed similar. Now, consider an individual with data v = (M, 1) Based on experience, we expect that a perturbation in educattinnll talent by .05 units will not significantly alter this individual's s status. We model this perturbation with shift intervention A (0, 00, 0). In Model I, the result CF(U, -) = (M, 1.05, 1) is considered similar to u. However, in Model 2, CF(v, A) = (M, 1.05, 1.1) results in a distance of d(v. CF(u, A)) = 0.15, indicating dissimilarity. In the presence of causality, one attribute can be amplified multiple times in the final feature space. Therefore, we need to control our intuition of dissimilarity between the exogenous variables and the feature space.
To protect against gender bias, we need to ensure that people with the same intrinsic characteristics but different genders behave similarly. This is modeled by counterfactual change in gender. In Model 1, CF F) = 1F,1,1) shows no difference (d(r, üp) = 0). However, Model 2. CF(v, F) (F, 22) results d(v, üp) = 4, which means that they are not similar.
The example demonstratess that in the presence of causality and protected variables, the standard f--norm any metric fails to accurately capture the intuition of similarity. In these scenarios, dissimilarity function should incorporate counterfactuals and uniformly control for non-sensitive perturbations to effectively capture proximity. This approach is further elaborated in the following definition. Before proceeding, we introduce some notation. For a vector v or u, we define Pa(-) and (-) the projections onto the sensitive and non-sensitive parts, respectively.
Definition (Causally Fair Dissimilarity Function) Let : V x V- - [0, oo] be a dissimilarity function defined on the feature space V, generated by SCM M. Let A denote set of sensitive auributes, and I represent their corresponding index within (1.....n} The metric is called a causally fair dissimilarity function or CFDF if adheres to the following properties:
Zero Dissimilarity for Twin Pairs: For any E V and a E A, the dissimilarity d(v, ūa) between an instance and its twins is zero.
Guaranteed Similarity for Minor Perturbations: For every V and any 5 > 0, there exists an such that for any sufficiently small intervention ( [| S e) on the non-sensitive altributes (P^(A)= = 0), the distance d(v, CF(v, A)) remains less than 5.
To understand the shape of d under the assumptions of Def. 1 we must first recognize that the CFDF needs be defined on a larger space than Range(M) (22). This is because, generally, when M is in- tervened upon by some sensitive attribute level a, we have Range(M) S U.eA Range(M (M The complete space encompassing all counterfactual values can be defined as follows.
Definition (Parent-Free Sensitive Attribute SCM) Consider M with sensitive attributes in- dexed by I. The parent-free sensitive attribute SCM denoted as Mo. is derived from M by removing the causal effects of parents of sensitive attributes and replacing their exogenous variables with indigenous ones. The structural equations for Mu are as follows:
[U, Vi- = ( +U, U.~
U Vi=VViNVV:
i I i i& I
The exogenous space corresponding to Mo. denoted by Uo, includes the sensitive altributes and the non-sensitive parts of the exogenous variables of M. This space called the semi-latent space, is constructed as Uo A x Uxx where Ux is the non-sensitive part of the exogenous space in M.
If we know the structural equations of M, we can first map the CFDF to the exogenous space. In this space, the exogenous variables are assumed to be independent. Therefore, we can design dissimilarity function for each variable separately and then combine them using product topology ($.2 4. Following this intuition, we introduce the bijective map g V - Ud from the feature space to the semi-latent space, along with its inverse, defined as follows:
Sui iEI
iET
9.(0) = - [((()) i i
9, (u)(w)= =
(6)
(u)) + +u, i
5

---

and If all sensitive attributes have no parents, the semi-latent space is equivalent to the exogenous space, and g = F- The counterfactual with respect MM is denoted by CFo A). We can now present the following proposition to determine the shape of d.
Proposition 1 Let M be an ANM, with g as its corresponding map to the semi-latent space 6 and (u( the projection of vector to the non-sensitive part Ux. Then:
(i) If dx is a continuous dissimilarity function on diagonal Ux lxx, then the function defined as: d(v, vf) =x((e((oo)))Pxx(ddd))))
satisfies the definitions of a CFDF
(ii) if: + [0, oc] satisfies the CFDF definition and the triangle inequality property, then can be represented as dissimilarity function dx dependent solely on the non-sensitive components Ux ie.. d(u, 0') = dx((P(((t))
Since dx is defined on independent coordinates, its relation to the components less complex than the CFDF d. We assume the dissimilarity function dx(x, )' is translation-invaric simplicity, we assume (x'"", ,:) = || - z||- The dual of || is defined as ||~|l = sup, {x x'
Therefore, for
||d||| < 1}. Now we establish our assumptions about the SCM and its CFDF.
Assumption
(i) M is an ANM with known structural equations and semi-latent map g- (ii) The CFDF is defined as d(v, v') = 11=11((x((()))) - Px(((('))ll where |I-Il is a some norm.
(iii) Cost function over Z has form c((v, .), (v', ,y)) = d(u, r/) + 00 v'l.
(iv) The ambiguity set is defined as: (B((P) = (Q E P(V) (e Q) S 8). for p E [c)
Remark All results of this work apply to the homogeneous dissimilarity function (Def. o which includes a broad family of dissimilarity functions, such as norms.
4 Causally Fair Distributionally Robust Optimization
To find out the impact of the CFDF in DRO problems, first consider the dual form of the worst-case loss quantity, which simplifies the infinite-dimensional primal problem into a more tractable and computationally manageable form.
Theorem (Causally Fair Strong Duality) If Asssupttion 7 is satisfied, then for any reference probability distribution P and any function a/ V + R that is both upper L1 Li-integrable, the following duality holds:
sup QE)
E m
[600)1
inf X20
AO E
sup ers
(8)
LaEA
where v/x (1) is defined
(x(0) = sup (V(CFo(V. A)) Xd CFo(v,
(9)
DEX
and CFo is counterfactual regarding parent-free SCM Mo.
Remark 2 The intuition behind the above formula is as follows: In the case where all features are independent, let v (a, The CFDF should exhibit no difference between (a, x) and (a', (x) for each a, a' A. Consequently, the distance metric satisfies di(a, x). (a', x')x dx dx(x, x'). Under this condition, the classical strong duality theorem (Eq. 5) provides the following relationship:
(x = sup ( -xd = sup sup (b A)) - Xd, (''
asA
LEX
When we incorporate causal structure instead of coordinating a and x, the two dimensions in and CFo(v. (v, A) are replaced accordingly.

---

In the DRO formulation, the worst-case loss is expressed in dual form and can act as a regularizer for parameter learning. Explicitly solving the dual problem eliminates the need compute the worst-case distribution, resulting in faster, more efficient learning algorithms [4 16 62 73 Before presenting the general theorem, the next two theorems show that, under mild conditions, the dual formula for specific loss functions in classification and regression problems can be explicitly solved.
Theorem (Higher Order Linear Loss) Given Assumptions ! let M be a linear SCM and the loss function {(2, 0),. where {(2, 8) is of the form h(y (6, v) hly (0, ()) for functions h(t) such as Itl. max(0,t), t), It TI or max(0, T) for some т 2 O, and p E (1,00). Then the DRO problem can be reduced to:
R (). + +1IIP((MTT))))'
diam (A) < oo
Rs (F
(R(PPN 0) + 6 ( s.t. Pa(M =0; diam (AA = 00
where M is the corresponding matrix for the linear map g-1 (see Eq. 6
Remark 3 In real-world datasets, the sensitive part always satisfies diam (A) < oc. According to the above theorem, R. N: 0) > R. (IP N. 0)- For practical applications, if the worst-case loss must not exceed a certain value, we can replace oo with some constant in the above theorem.
Example 2 Here are specific examples of the above theorem. We offer a framework to study the equivalence between the worst-case loss in the DRO problem, with the cost function derived from the CFDF, and the regularization scheme for classification and regression problems.
Regression
Lower Partial Moments
E, Y
0, V) p p21
Ep (0, V) T) 2 1, T R
Ridge Linear Regression
7-Inenssitive Regression
Ep + (0, V))a
E - (0, V) - p 1, R
Hinge Loss Binary Classification Ep Y (0, V))
Support Vector Machine Classification
Ep| Y (0, V) "),
The Thm. can be extended to the non- non-linear regression loss function.
Theorem (Nonlinear Loss) Let assumptions 7 The satisfied, with p 1, M linear with matrix M corresponding to map 3 and loss function U2 0) of the form h(y (0, v)) for regression and
byy (0, v)) for classification, where has the following two properties:
(i) h is Lipschitz R with Ln, constant, i.e., (h(tz) - h(tt)l s Laltz tal. Vt,, E R
(ii) There exists sequence of {t goes to such that for each to E IR we have limp |1(4o + (x) h(6o)l
L.p-
|t
By the above assumption, DRO problem can be reduced as:
R2 (PNN + 8Ln | ||
diam(A) <
R₂ O)
R(FN. 0) + SLM Px
s.t. PA(M 0) = 0; diam(A) 00
Example The following forms of the loss function satisfy the conditions of Thm.
Now, the first-order estimation of the regularizer for non-linear SCM and loss function is ready be stated.
Theorem (First-Order Estimation DRO Regularizer) Assume M has structural equation f. which and loss function é are both twice continuously differentiable respect non con-sensieive
7

---

Log-cosh Loss
Huber Loss
h: 1
if 14 S otherwise; Log-exponential Loss
h: ++ (og(cosh(t))
Quantile Loss
76
i 2 0,
h:
with y E (0;
h: t ++ log(1 1 + exp(-t))
t
otherwise,
Smooth Hinge Loss
Truncated Pinball Loss if S 1, if < otherwise,
ift 2 1, 1 < 1 otherwise;
1-t
h: 1 +
1 1)2
h: l +
T1 1)
t
T172
where Ti E [0.1], T2 2 are two given constants.
attributes, diam (4) so and satisfies the assumption 7 with p [2, oo). The necessary condition for the existence of a finite DRO solution is that for each v E V:
sup (E(öa: y. 0,0)
aEA
By these conditions, the worst-case loss quantity is equal to:
sup (( (, 0011 ) + 0(8)) (10)
1/9
RA(PN. 0)
E sup ((ū, y, 0) + 5. vnPN
LaEA
(unPN
LaEA
where the (622) term is uniform over all O q p's conjugate, and the gradient 4 equals to:
E(CFolU, A). 0) N,
vellv,,,
lim A-
|
where CFo is counterfactual regarding parent-free SCM Mo-
By applying Prop. 2 from Gao's work (28) the next proposition presents the relationship between classical adversarial optimization 3 and DRO for CFDF.
Proposition 2 (Approximation by Robust Optimization) Suppose A is a finite set and let assume that for the loss function E 1
((V' '') be observational data. Under Assumption there exist constants L, MM 2 such that
|e(v. y, 0) É(o', y. 0)] Ldd(o, M for all 0,v' V and p [1,
For an arbitrary E N, consider the adversarial loss within the setting:
Rod (PN)::
sup (wi )
sup llütk Ni. 0)
NK
aEA
where the uncertainty set Bg is defined as:
Bg
(w
('' wdt s 5, with E Ev-
1=
Then, the DRO can be approximated by adversarial optimization as follows:
Radv S (IP < Rady (PN)+ + LD + M
NK
where D is independent of K.
One of the main challenges in designing DRO for SCMs that the CFDF depends on the causal structure. When the functional structure is unknown, it must be estimated from data. This empirical estimation impacts the DRO learning process. Therefore, it crucial to control the uniform conver- gence error of the DRO problem between the true metric and distribution and the DRO estimated from the data. The following theorem guarantees learning from sample data, but certain assumptions need to be established first.

---

Assumption 2
(i) M is an unknown ANM, diam (V) < cc, and è is compact subset of IR".
(i) The loss function is uniformly bounded: there exists a positive constant M such that 0 I ((z,0) I M for all O. Moreover, E is Lipschitz with respect the counterfactual in Mo; that is, there exists a constant L such that:
[e(e v... ((CFF(F A), 0)1 [1e Lip)
(iii) d is an estimation of the CFDF such that, with probability e, there exists Ma such that, at rate of N- the discrepancy is uniformly bounded by:
Vu, o' E E [d(e, v') d(v,v')) S MaN"", for some 0.
The following theorem states that the efforts estimate the metric causal structures and the parameter 0 N
0 inf sup (Q.P a
E
Eele(2, 01]
ege
Where è is the d corresponding cost Z, leading to the estimation of the true parameters of the DRO problem. To state our result, we need the Dudley entropy integral [6]]. which measures the complexity of the loss function class.
Theorem (Learning Finite Sample Guarantee) With assumption
Land then for 0 we have:
Rs (1',, ( - inf Rs(F., 0) I N CD - + 026" -1 c3 ca (og((//))) dee
With probability at least 1 - 2e. With C(C) denoting the Dudley entropy integral for the function class (E( 0 @), the constants co, ei and 02 identified as follows:
co (6)(C) c1 96L diam (V)², C2 2pL diam (V)=-1. -Mas and C3 2v2 x M.
The final theorem completes our framework, enabling us to perform DRO on real-world datasets without knowing the SCM structures while providing performance bounds.
4.1 Related Works
Causally Fair Dissimilarity Function. Various studies have addressed the specification and learning of individual fair metrics, such as 33
47), but their construction based on causal structure and sensitive attributes remains unclear. Our work adopts and extends the concept of causal fair
68
70
metric, as discussed in the works 23
24
address the DRO problem with an individual fairness metric but are limited linear SCMs and 2. These studies do not discuss the duality theorem or regularizers. Additionally, 42 studied DRO, but its connection to causality
DRO and Individual Fairness. Previous works, such as 68 70 47
remains unclear.
Strong Duality Theorem. Various versions of the strong duality theorem have been explored in
prior works. For instance, in 59 46
14
27 66), the cost function must metric or 74
11 58 the distance function d must be positive-definit. meaning d(u, o') 0 if and only if v' However, these conditions are not met for CFDF, necessitating new formulation of the duality theorem
has convex property. Additionally, in 72
DRO Regularizer. Previous works using DRO regularizer, explicitly solved 660 14
16
29
or through k-order estimation 5
6 66 27), only consider cases where the cost function is derived
from metric or positive-defintt dissimilarity function. Therefore. their theorems do not apply directly to our CFDF. We present new results in Theorems 2 and Sodd4taan tailored for our cases.
Finite Sample Guarantee. Various works provide bounds the performance of DRO solutions with finite samples [41
25
10 12 but these do not apply to our CFDF due to previously mentioned
reasons. The studies 68 70 47 offer performance bounds only for the case of linear SCMs with
p = Therefore, we present general case in Theorem 5
on causal optimal transport focus on the causal structure the transport map or plan, which differs from our problem. In our case, causality pertains to the transportation cost derived from SCMs.
Optimal Transport and Causality. Recent works
39

---

Numerical Studies
In our numerical studies, evaluate the impact of using causally fair DRO mitigate individual unfairness, henceforth referred to as CDRO. We compare CDRO's performance against Empirical Risk Minimization (ERM), non-causal Adversarial Learning (AL. 44 and the Ross method [56). Our experiments employ real-world datasets, namely the Adult [38 and COMPAS [65] datasets, pre-processed according [19]. Additionally, we use a synthetic dataset for linear SCM (LIN) with formulations detailed in Appendix C.T
We first fit linear structural equation model for both the
AdM
COMPAS
LO
Adki
COMPAS
OM
Unbor Arca ta 0.05)
50%
Peeewooe Date
Accurscy
00%
Syvtielt CNW
40%
40%
92%
20%
0%
0%
05
0%
Figure 1: Displays the findings from numerical experiment, assessing the performance of DRO across different models and datasets. (left) Bar plot showing the comparison of models based the unfair area percentage (lower values are better) for = .05 (right) Bar plot comparing methods by prediction accuracy performance (higher values are better).
Adult and COMPAS datasets. Logistic regression is employed for classification, and performance evaluated based on accuracy. Fairness is assessed using the Unfair Area Index (UAI), which is defined by the following equation:
UA == P(V EV: WVEV s.t. d(u,)) A A(0) + Mb')))
We evaluate UAI across different A values, specifically 0.05 and 0.01. Additionally, we calculate the UAI for scenarios where no sensitive attributes are considered, representing the percentage of non-robust data. Detailed computational experiment procedures are provided in Section C.I
Our experiments, conducted using 100 different seeds, are summarized Table T Figures illustrate that the CDRO method achieves lower unfair area (U^) for A .05, and A 0.01 in all scenarios. Although CDRO shows slightly lower accuracy than ERM, this trade-off a common observation in several studies [52]. Additional results can be found in C.6
and
6 Discussion and Limitations
Our study introduces a novel framework for causally fair DRO, integrating causal structures and sensitive attributes into the DRO paradigm. This framework is supported by several theoretical advancements, including a strong duality theorem, explicit regularizer formulation, first-order regu- larizer estimation, and finite sample guarantees with unknown SCMs, enhancing its efficiency and practicality for real-world applications. Our experimental results demonstrate its effectiveness in various settings, highlighting its potential for mitigating biases in machine learning models.
Despite the promising results, our study has several limitations that warrant further investigation. Firstly, the assumption of an additive noise model may not capture the complexity of all real- world causal relationships, posing challenges in computing additive interventions general SCMs. Secondly, while Theorem and Theorem extensions to avoid complexity. Lastly, further work needed to explore the relationship between method and causal optimal transport [3 32
(heorem3]could couuld be extended to more general cases, we omitted these
Acknowledgments
The authors thank the Max Planck Institute for Intelligent Systems, Tübingen Center, supporting this project. Partial funding support also provided by the Canada CIFAR AI Chair program.
10

---

References
[1] Beatrice Acciaio, Julio Backhoff- Backhoff-Veraguas, and Anastasiia Zalashko. Causal optimal transport and its links enlargement of filtrations and continuous-time stochastic optimization. Stochastic Processes and their Applications, 130(5):29182299 2020.
[2] Julio Backhoff, Mathias Beiglbock, Yiqing Lin, and Anastasiaa Zalashko. Causal transport in discrete time and applications. SIAM Journal on Optimization, 27(():2528-25 2562, 2017.
[3] Ryan S Baker and Aaron Hawn. Algorithmic bias in education. International Journal of Artificial Intelligence in Education, pages 1-41, 2022.
[4] Solon Barocas, Moritz Hardt, and Arvind Narayanan. Fairness and machine learning: Limita- tions and opportunities. MIT Press, 2023.
[5] Daniel Bartl, Mathias Beiglböck, and Gudmund Pammer. The wasserstein space stochastic processes. arXiv preprint arXiv:2 11101..44...4 2021.
[6] Robert Bartlett, Adair Morse, Richard Stanton, and Naney Wallace. Consumer-lending discrim- ination in the fintech era. Journal of Financial Economics, 143(1):30-505 2022.
[7] Dimitris Bertsimas, David Brown, and Constantine Caramanis. Theory and applications of robust optimization. SIAM review, 53(3):464-501, 2011.
[8] Reuben Binns. On the apparent conflict between individual and group faireess. In Proceedings of the 2020 conference on fairness, accountability, and transparency, pages 514-524, 2020.
[9] Jose Blanchet, Yang Kang, and Karthyek Murthy, Robust wasserstein profile inference and applications to machine learning. Journal of Applied Probability, 56(3):830-877, 2019.
[10] Jose Blanchet, Jiajin Li, Sirui Lin, and Xuhui Zhang. Distributionally robust optimization and robust statistics. arXiv preprint 2tt.:2400..145555 2024.
[11] Jose Blanchet and Karthyek Murthy. Quantifying distributional model risk via optimal transport. Mathematics of Operations Research, 44(2):565-660 2019.
[12] Jose Blanchet, Karthyek Murthy, and Nian Si. Confidence regions in wasserstein distributionally robust estimation. Biometrika, 109(2):95--315, 2022.
[13] Patrick Cheridito and Stephan Eckstein. Optimal transport and wasserstein distances for causal models. arXiv preprint arX3 2023.
[14] Hong Chu, Meixia Lin, and Kim-Chuan Toh. Wasserstein distributionally robust optimization and its tractable regularization formulations. arXiv preprint arXiv:2402.03944 2024.
[15] Hong TM Chu, Kim-Chuan Toh, and Yangjing Zhang. On regularized square-root regression problems: distributionally robust interpretation and fast computations. The Journal of Machine Learning Research, 23(1):13885-1399) 2022.
[16] Hong TM Chu, Kim-Chuan Toh, and Yangjing Zhang. On regularized square-rool regression problems: distributionally robust interpretation and fast computations. Journal of Machine Learning Research, 23(308):1-39, 2022.
[17) Zac Cranko, Zhan Shi, Xinhua Zhang, Richard Nock, and Simon Kornblith. Generalised Lips- chitz regularisation equals distributional robustness. In International Conference on Machine Learning, pages 2178-2188. PMLR, 2021.
[18] Jeffrey Dastin. Amazon scraps secret ai recruiting tool that showed bias against women. In Ethics of data and analytics, pages 296-299. Auerbach Publications, 2022.
[19] Ricardo Dominguez-OImeda Amir H Karimi, and Bernhard Schölkopf. On the adversarial robustness of causal algorithmic recourse. In International Conference on Machine Learning, pages 5324-5342. PMLR, 2022.

---

[20] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pages 214-226, 2012.
[21] Stephan Eckstein and Gudmund Pammer. Computational methods for adapted optimal transport. The Annals of Applied Probability, 34(1A):675-713, 2024.
[22] Ahmad-Rezi Ehyaei, Golnoosh Farnadi, and Samira Samadi. Causal fair metric: Bridging causality, individual faimess, and adversarial robustness. arXiv preprint arXiv:2310.19391, 2023.
[23] Ahmad-Reza Ehyaei, Amir-Hossein Karimi, Bernhard Schölkopf, and Setareh Maghsudi. Robustness implies faireess in causal algorithmic recourse. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pages 984-1001, 2023.
[24] Ahmad-Rezz Ehyaei, Kiarash Mohammadi, Amir- Karimi, Samira Samadi, and Gol- noosh Farnadi. Causal adversarial perturbations for individual fairness and robustness in heterogeneous data spaces. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 10, pages 11847-1 1855, 2024.
[25] Rui Gao. Finite-sample guarantees for wasserstein distributionally robust optimization: Break- ing the curse of dimensionality. Operations Research, 71(6):2291-233 -2300, 2023.
[26] Rui Gao, Xi Chen, and Anton Kleywegt. Wasserstein distributionally robust optimization and variation regularization. arXiv preprint arXiv:1712.06000 2017.
[27] Rui Gao, Xi Chen, and Anton Kleywegt. Wasserstein distributionally robust optimization and variation regularization. Operations Research, 2022.
[28] Rui Gao and Anton Kleywegt. Distributionally robust stochastic optimization with wasserstein distance. Mathematics of Operations Research, 48(2):603-655, 2023.
[29] Camilo Andrés Garefa Trillos and Nicolás Garefa Trillos. On the regularized risk of distribu- tionally robust learning over deep neural networks. Research in the Mathematical Sciences, 9(3):54, 2022.
[30] Milena Gianfrancesco, Suzanne Tamang. Jínoos Yazdany, and Gabriela Schmajuk. Potential biases in machine learning algorithms using electronic health record data. JAMA internal medicine, 178(11):1444-547, 2018
[31] Melissa Hall, Laurens van der Maaten, Laura Gustafson, Maxwell Jones, and Aaron Adcock. A systematic study of bias amplification. arXiv preprint arXiv:2201.11706, 2022.
[32] Bingyan Han. Distributionally robust risk evaluation with a causality constraint and structural information. arXiv preprint arXiv:2203. 2022.
[33] Christina Ilvento. Metric learning for individual fairness. In 1st Symposium on Foundations of Responsible Computing (FORC 2020). Schloss- Schloss-Dagstull-E Dagstuhl-Leibniz. Zentrum für Informatik. 2020.
[34] Alexander Immer, Christoph Schultheiss, Julia E Vogt, Bernhard Schölkopf, Peter Bühlmann, and Alexander Marx. On the identifiability and estimation of causal location-scale noise models. arXiv preprint arXiv:22 2022.
[35] Yifan Jiang, Duality of causal distributionally robust optimization: the discrete-time case. arXiv preprint arXiv:2401. 2024.
[36] Olav Kallenberg and Olav Kallenberg. Foundations of modern probability, volume 2. Springer, 1997.
[37] Amir-Hossein Karimi, Bernhard Schölkopf, and Isabel Valera. Algorithmic recourse: from counterfactual explanations interventions. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 353-362, 2021.
12

---

[38] Ronny Kohavi and Barry Becker. Uci adult data set. UCI Meachine Learning Repository, 5, 1996.
[39] Daniel Kršek and Gudmund Pammer. General duality and dual attainment for adapted transport. arXiv preprint arXiv:2401.11955 1198,, 2024.
[40] Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. In Advances in Neural Information Processing Systems, pages 4069-4079, 2017.
[41] Jaeho Lee and Maxim Raginsky. Minimax statistical learning with wasserstein distances. Advances in Neural Information Processing Systems, 31, 2018.
[42] Peizhao Li, Ethan Xia, and Hongfu Liu. Learning antidote data to individual unfairness. In International Conference Machine Learning, pages
PMLR, 2023.
[43] Fengming Lin, Xiaolei Fang, and Zheming Gao. Distributionally robust optimization: A review on theory and applications. Numerical Algebra, Control and Optimization, 12(1):159-212, 2022.
[44] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint MXXXx:706.0000 2017.
[45] Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. survey on bias and faimess in machine learning. ACM computing surveys (CSUR), 54(6):1-35, 2021.
[46] Peyman Mohajerin Esfahani and Daniel Kuhn. Data-driven distributionally robust optimization using the wasserstein metrie: Performance guarantees and tractable reformulations. Mathemati- cal Programming. (71(1):115 2018
[47] Debarghya Mukherjee, Mikhail Yurochkin, Moulinath Banerjee, and Yuekai Sun. Two simple ways to learn individual fairness metrics from data. In International Conference on Machine Learning, pages 7097-7107. PMLR. 2020.
[48] James R Munkres. Topology. 2nd edn of [mr0464128), 2000.
[49] Razieh Nabi and Ilya Shpitser. Fair inference outcomes. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 1, 2018.
[50] Arash Nast-Esfahany, Mohammad Alizadeh, and Devavrat Shah. Counterfactual identifiability of bijective causal models. In International Conference on Machine Learning, pages 25733- 25754. PMLR, 2023.
[51] Judea Pearl. Causality: Models, Reasoning, and Inference. Cambridge University Press, 2009.
[52] Dana Pessach and Erez Shmueli. A review on faimess in machine learning. ACM Computing Surveys (CSUR), 55(3):1 44, 2022.
[53] Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference: founda- tions and learning algorithms. The MIT Press, 2017.
[54] Gabriel Peyré, Marco Cuturi, et al. Computational optimal transport. Center for Research in Economics and Statistics Working Papers, 2017-86, 2017.
[55] Hamed Rahimian and Sanjay Mehrotra. Frameworks and results in distributionally robust optimization. Open Journal of Mathematical Optimization, 3:1-85, 2022.
[56] Alexis Ross, Himabindu Lakkaraju, and Osbert Bastani. Learning models for actionable recourse. Advances in Neural Information Processing Systems, 34:18734-18746, 2021.
[57] Anian Ruoss, Mislav Balunovic, Mare Fischer, and Martin Vechev. Learning certified individu- ally fair representations. In Advances in Neural Information Processing Systems, 2020.
13

---

[58] Soroosh Shafieezadeh- Abadeh, Liviu Aolaritei. Florian Dörfter, and Daniel Kuhn. New per- spectives on regularization and computation in optimal transport-bucd based distributionally robust optimization. arXiv preprini arXiv:2303.033000 2023.
[59] Soroosh Shafieezadeh-Abadeh Daniel Kuhn, and Peyman Mohajerin Esfahani. Regularization via mass tramsportation. Journal of Machine Learning Research, 20(103):1-68 --8, 2019.
[60] Soroosh Shafieezadeh Abadeh, Peyman M Mohajerin Esfahani, and Daniel Kuhn. Distribu- tionally robust logistic regression. Advances in Neural Information Processing Systems, 28, 2015.
[61] Michel Talagrand. Upper and lower bounds for stochastic processes, volume 60. Springer, 2014.
[62] Peipei Tang, Chengjing Wang. Defeng Sun, and Kim-Chun Toh. sparse semismooth newton based proximal majorizationmmnniizziio algorithm for nonconvex square-root-los: regression problem. The Journal of Machine Learning Research, 21(1):9253-952 9290, 2020.
[63]
Cédric Villani et al. Optimal transport: old and new, volume 338. Springer, 2009.
[64] Julius Von Kügelgen, Amir- Hossein Karimi, Umang Bhatt, Isabel Valera, Adrian Weller, and Bernhard Schölkopf. On the fairness of causal algorithmic recourse. In Proceedings of the AAAI conference on artificial intelligence, volume 36, 9, pages 9584-9594, 2022.
[65] Anne L Washington. How to argue with an algorithm: Lessons from the compas-propublica debate. Colo. Tech. LJ, 17:131, 2018.
[66] Qinyu Wu, Jonathan Yu-Meng Li, and Tiantian Mao. On generalization and regularization via wasserstein distributionally robust optimization. arXiv preprint arXiv:22/2.057/6 2022.
[67] Samuel Yeom and Matt Fredrikson. Individual faimess revisited:
transferring techniques
from adversarial robustness. Proceedings of the Twenty-Ninth International Conference on International Joint Conferences Artificial Intelligence, 2021.
[68] Mikhail Yurochkin, Amanda Bower, and Yuekai Sun. Training individually fair ml models with sensitive subspace robustness. arXiv preprint arXiv:19077 0000000 2019.
[69] Mikhail Yurochkin, Amanda Bower, and Yuekai Sun. Training individually fair ml models with sensitive subspace robustness. In International Conference on Learning Representations, 2020.
[70] Mikhail Yurochkin and Yuekai Sun. Sensei: Sensitive set invariance for enforcing individual fairness. In International Conference on Learning Representations, 2021.
[71] K Zhang and A Hyvärinen. On the identifiability of the post-nonlincar causal model. In 25th Conference on Uncertainty Artificial Intelligence (UAl 2009), pages 647-655, AUAI Press, 2009.
[72] Luhao Zhang, Jincheng Yang, and Rui Gao. A simple and general duality proof for wasserstein distributionally robust optimization. arXiv preprint arXiv:2205.0033 2022.
[73] Yangjing Zhang, Ning Zhang, Defeng Sun, and algorithm for solving large-scale sparse group lasso problems. Mathematical Programming. 179:223-28 26322 2020.
Toh. An efficient hessian based
[74] Jianzhe Zhen, Daniel Kuhn, and Wolfram Wiesemann. A unified theory of robust and dis- tributionally robust optimization via the Research, 2023.
principle. Operations
14

---

A Supplementary Theoretical Details
Notations. In this work random variables are denoted by bold letters (e.g.. V). their corresponding probability spaces by calligraphic letters (e.g.. V), and instances by normal letters (e.g.. v). The space of probability measures on V is represented by P(V) and probability measures by blackboard bold letters (e.g. P).
Non-Sensitive Part. Let F : U1 + V be the reduced-form map of M. The vector v decomposes (a, and we have a corresponding decomposition in the exogenous space denoted by (ua, Ugx) and un, MA are corresponding spaces. Using the ANM model, we can assume that both V and Z1 are equivalent, and therefore, the non-sensitive feature space is the same as the non-sensitive part of the exogenous space. If IP is probability measure in P(V), then (I))x refers the marginal probability over the non-sensitive part. We also refer to Q)x for marginal probability over the non-sensitive part of the exogenous space.
into sensitive and non-sensitive parts,
Definition 3 (Push-forward Measure) Let Per P(V), Q E P(U) be two probability measures and T: T:V- U is map, the measure called the push-forward of through II' is denoted by TnP P if:
Q(B) = P(T (B)), YB cu
Definition 4 (Set of Couplings) The set (IP, Q) represents the couplings of probability distributions
P E P(V), O E P(U), comprising distributions over V x U with margins TP and Q A measure belongs ((P, Q) if and only if
u( P(A) and +(V B) Q(B) VACV,BCu
By extension, random pair (X,Y) ~ where E [[P, Q), will also be called a coupling of P and Q
Definition (Diameter Set) Let A be a set a metric space with a distance function d. The diameter of A, denoted diam( is defined as:
diam(A) sup(d(x, v) x, E A}
where sup represents the supremum of the set of distances d(x, y) for all pairs (x. v) A.
Definition 6 (Homogeneous dissimilarity function) Let be an extended-valued function
A: X + [0, col on a real vector space X with absolutely homogeneous assumption i.e. A(tx) [t] (xx) for any E R and X. addition,
proper it means there exists Fy E X such that A(zo) 1. The cost function d: X x X + [0, oo) is called Homogeneous dissimilarity function if is defined as d(x', = A(x' x)) for any x' x X.
Lemma If M an additive noise model with mutually independent exogenous variables, then the farent-freee sensitive Mo attribute model retains both of these properties. Moreover, the map-reduced form mapping of Mo is equivalent to g- where g represents the mapping to the semi-latent space.
Proof. First, My is an additive noise model because its structure is derived from the initial equations of M by removing those equations related the sensitive attributes and replacing the exogenous variable U, by V..
Regarding the mutual independence of the exogenous variables, in the original model M, the variables V; for E I are not independent of U, for j & I if they have parents. However, assuming a hard intervention for each instance of V, where do-action is executed for this intervention that the intervened variable V; can be considered independent from the other variables. Therefore, since we apply hard interventions all sensitive variables, we can assume that V; for i E I mutually independent, and also that V2 are independent of Uj for all j & I.
implies
Finally, by referencing equations 6 it is observable that the map-reduced form mapping of Mo is equivalent to the inverse of the map to the semi-latent space.
Lemma Let M an additive noise model with a mapping g to semi-latent space Un- Assume M includes the sensitive attributes A and other non-sensitive attributes X. that belong the vector
15

---

space X. Consider (a,z) as an instance in M. and let A E X represent a shift intervention value. Then, the counterfactual corresponding to additive shit is obtained by:
Px(CF(U,A)) = Px(g (e<gg'(((v)) + (0,))))-
Moreover, if d' E A represents another level of sensitive attributes, then the hard intervention
concerning A == a' is achieved by:
0e = CF(c 9- (((''))(((()))))
Proof. In additive noise models, an additive intervention can be conceptualized adding value d to the exogenous variables, while all structural equations remain unchanged. Consequently, during such an intervention, the reduced-form mapping FMa of the intervened SCM remains unchanged. Therefore by definition of intervention, follows that:
CF( do( (++=)) = FMM (F (v) + (0, A)) F(F (w) (0,
Since and g-1 are coincide in non-sensitive coordinates then Px(xb(F1(( + ())) = (x (g(() + (0, A))) and it completes the first part. In this case, we denote F M, which is an invertible matrix. Consequently, the counterfactual CF(u, do(X+=A)) can be expressed as + (o, A).
To prove the second part, when intervention is performed on sensitive attributes, M transforms into a parent-free sensitive attribute model where the sensitive attribute is replaced by a'. Given that the map-reduced form of the parent-free sensitive attribute model aligns with g and since and F- coincide the non-sensitive parts, the counterfactual can be expressed follows:
V = CF(,, do(A:=al)) = g ((a', (x((())))).
B Proof Section
Proof of Proposition
!
(i) If d adheres to Eq. 7, it means that for each v V. the mapping g(v) (a,z). For its we can express:
counterfactual Va we have g(üar) ) = (a),z). Using Eq. 7
d(v, (a)) dx (P (x(o((a)) = (x, x) 0
This demonstrates that d retains the first property of Def. [
Additionally, since dx is continuous, for each & and any > O, there exists a such that
|||| < 8, then dx (I, + A) < Referencing Lemma and the formulation of d, it follows that for
A < and for each a E A:
d(v, CF A)) =
A))) =
Thus, it satisfies property (ii) of the CFDF.
(ii) Let's consider a CFDF denoted as d V x + R, with an embedding g Q that maps from the feature space to semi-latent space. We define d' as the pull-back of d onto Q d' (g1, 42)
d(g "q1)), - where d" is dissimilarity function, and we aim to clarify which properties it inherits from Def. We utilize decomposition of Q into A X, where and v E V. denoting q as (a, x). Property (i) of the CFDF ensures:
(v)
d(., (.)) d" ((a, x), (d', = = Val E
This property confirms that d" is insensitive changes in the sensitive part A. To demonstrate, (a1,2) and = (22,22) with an arbitrary E A. By triangle
consider any two points property of dissimilarity function d can be seen:
d' ((a1, 21), (2); (az, x2)) s d" ((a1, 11), (an, + d" ((as, (1), (22, - d" d'((a1, 21), x2)) S d" ((00, æ1), x2))
Here, d' (($1, 21), (Sv, x1)) is zero due to property (i). Similarly, we can argue: d' ((a0, 21), (az, x2)) d' ((as: æ1), x1)) d" ((a1 21) (az, x2)) d' ((ag, x (az, x2)) s d" ((01, (2))
16

---

This results in d" ((01, [1), (02, x2)) = d' ((av, 11), (2)) With similar reasoning. we have: d' ((a1, (a2, 22)) d' ((a1, x1), (ao: x2)) d ((a1, 21), x2)) d' ((ao, x1), (ao, 22))
Hence, d' is invariant to the sensitive subspace. If dx is the dissimilarity function induced by d' on = dx (#1 x2)) In accordance with Lemma the second property of Def. 1 that for each 0, there exists 5 such that if | 6, then (x(x, + A) This property demonstrates the continuity of dx along the diagonal.
X. then d² ((01, (a2.x2) Ti),
Finally, d can be embedded in semi-latent space and described by another dissimilarity function on it that only depends on the non-sensitive part of exogenous space:
d(v, w) = dx Px
This concludes the proof.
Lemma 3 (Transformation by Bijective Map) Let getg:V: v -4 be an invertible function and let the transportation cost function be constructed by c(u, ) = d(g(v), g(v)) where is metric on the space U. For every P, Q P(V), the following equation holds:
W Wep (P, Q) Wa.pl 9
where We and Wa represent the Wasserstein distances with respect the metrics and d, respectively.
Proof. By the definition of the Wasserstein distance.
inf S ((o,v')d(uuçe
W (P, Q)
=Ef(F.0) vxv
Substituting v(u, = d(g(v), g(v')) and u = g(0) gives:
/ Vxv d' (((g(t),(())) dT(,,)) =EI(P.0)
We,p (P, = inf
Consider a coupling т of I and Q. Define measure ă on # (AxB) = (B)). # is a coupling of I IP and g# Q because:
((A x U) = (g (A) x V) = P(A); 7(U x B) = xV 9# QBB).
Therefore, the p- Wasserstein distance for the push-forward measures is
inf I TER(F.0) vxv ((d'))))('')dTT((,
inf #ET(ON 1) I du, dū,
WWa.lg C...PPP.... Q).
Since arises from æ via g. and is invertible and measure-preserring in this context, the values in the integrals of the definitions of We,p(PP Q) and Wa g# Q) match. Thus, have shown that Wap, Q = Wap(g P. P. Q).
Lemma 4 (Optimal Transportation Cost on Subspace) Let U C. Rn and suppose U is decom- posed into two subspaces, = (A, X), where A corresponds to the subset of some coordinates and X to its complements. Let Px denote the projection function onto the X space, i.e., Px(u) projects u E U onto components. Define a cost function c(u, u') = Px (u), Px(x')), where d is cost function on the spuce X. Consider probability measures P. Q P(U), and define Px Px P and Qx = Px as the pushforward measures P and Q under the projection Px, respectively, placing them in ((X). Let T'X be the optimal transport plan concerning the Wasserstein distance W Wa(Px: x )x Then, any transport plan π E P(u x U), whose marginal distribution over X' x X equals x'x, should also be optimal solution for the Wasserstein distance W(IP, Q) concerning the cost function c.
Proof. Given any coupling и [(P, Q), we consider elements u (a, x) and u' =aa, x') in U A x X. The cost function is defined by c((a, æ), (a', x')) d(z, x'), where d is metric
17

---

the space X' By definition of optimal transport cost W.(,, Q) we have:
Wenn cl(a,z), (a', dr d d(x,)') di di((a, dt = x, x,''= =,, =xd))
U d(ar, x') dr dmy)
sup =
sup
TET(F.0)
(i1))(((()))) d(T))xx))
sup
U d(,,)))(())xx =ET(())x,((a))
U Uxxx d(s
sup TEL(P.Q)
sup
where (P)xx and (x)xxx is marginal distribution over A' and X' x X' respectively. = x, X' r') is conditional distribution of и condition to the first and second X components equal to and x².
We observe that this integral effectively only depends on the X' component since the cost function c does not involve A. Hence, reduce the expression
U d(z, 2(x,,2)m
sup REr(())
(II)
Xxx
The Eq. 11 shows that the optima cost function of W.(P Q) equals to W W.((P)xx (Q)x). Therefore if T be the optimal transport plan for Px Qx with respect to d on X, then any coupling ग in U x UI that its marginal distribution xT)xxx equals x is the solution of optimal transport. It results that the conditional distribution TT(...)) I, X x') could be any distribution. This completes the proof.
Lemma Let X and be sets, and let X x A be a function. Then
sup sup f(z, a) sup sup f(z, a)) =eX dEA
aEA
Proof. Define:
L sup sup f(z, a) and R = sup sup f(z a). aEx
zEX aEA
To show that L R, we need to prove that < R and R S L. Consider any x X and A. By definition, f(x, a) S supaeA f(z a) for each fixed æ. Therefore,
f(x,a) S sup (x, a) S sup sup f(x, a) R. =ex =E aEA
aEA
Since f(x, a) was arbitrary, we have:
sup f(z,a) SR for all x X, aeA
and thus,
L= sup sup f(x a) S R. zEX aEd
Similarly, for any fixed a E A, f(x a) S sup f(z, a). Hence,
[(x,a) S sup f(x a) S sup sup [(x, a) = L. aεΛzeX
=eX
As before, since f(x, a) was arbitrary, we conclude:
sup f(z, 5 for all A. zex
and thus,
R == sup sup f(x, a) < L.
Since S. R and R S I,, it follows that T, = R. Therefore, we have proven that:
sup sup f(x a) sup sup (x, zEX aEA
A
This demonstrates the Principle of the Iterated Suprema.
18

---

B.1 Proof of Theorem
We prove the assertion in two steps: first, we assume that none of the sensitive attributes have parents, and second, we address and prove the general case. When all sensitive attributes do not have parents, in this case by definitionsssemin semi-latent space equivalent with exogenous space and therefore g = F- First, we show that the worst-case loss quantity can be decomposed into sensitive and non-sensitive components like as below equation:
(F ((ua (ell) }
sup (E)
E 1
sup GEB P)x)
E
sup
(12)
Live Ella
By the assumption, the CFDF has form d(u, 2²) dx (o)), Px a case that sensitive attributes have no parents then the semi-latent space coincides with exogenous space and the map between feature space and semi-latent space equals g F-1 Therefore in the following equations, we use instead of F-1, Moreover since is invertible by Lemma we can
(x) (v")). By Def. 2 in
write:
{ wwnQ
(F(((())))
sup (FBs(()
sup (E
(dx(u,')) s =
(b(((()))] u S#IP,
sup xEP(Uxt))
(u,
[dx u'ell S =0]=
sup xl)(
E. (NFF((( [(21))] (W'NT2
B#P.
E (uar))o
( ((F((1)) ) dT2 d7z(u')
(, [dx(ux: u/)) s 6
sup xlP(
u ~ P.
E (M
where d be a cost function on U defined as u²) (xx(
denotes the marginal
distribution on second part and Bg(g = E Q. S 8}. Using the disintegra- tion theorem (36] Chapter 3), the joint distribution can be decomposed into the product of the conditional distribution of UA given Ux and the marginal distribution on Uxx Therefore we have
Jn (F(((u)) (()))) d=2(w/) d=z(w)=- - - (F[((ua) (z))) di Ux = =a()) dx (x2)xxx((())
where (F2)x is the marginal distribution of T2 over the non-sensitive part and ma (u'a Ux = ulr) is conditional distribution of the sensitive part of exogenous space condition by Ux w/z By disintegration formula, (*) can be rewritten as:
TEP {T, vs ((F((1)) at (()))dz2( E P(U x U), T1 = I4
sup
E (M (()))x)
(d(u u/²1 S 8
{ Jex (Lan ((((uaa Un)) d ( Ux u'u) dx((T))(((b))
sup TEP(())
P(u U). TI I41P,
E (M (x)
[d(uas 2/:11 S 8 _d]
Jun I sup T=(l)
(f
(/))) ( Ux = w/:) ( d(m2))(())))
sup (F))
E P(U x U), T2X = (0+1)xx (u (()
E (dx(ux, 26y1 8
(13)
Since dx depends only on the non-sensitive components, it follows from Lemma 4 that (x = u(,) can achieve any distribution. Moreover, since it does not depend on the Wasserstein distance in each coupling, the marginal distribution of the sensitive attribute can be considered independent of the marginal distribution of the non-sensitive attributes. Therefore, the supremum over 22(- Uxx = v')) of integral equals the supremum of (FF((,, ()) over all values of u'. Furthermore, the distribution
19

---

Pe | does not influence the value of the Wasserstein distance. Based on these points, the last equation can be rewritten as:
{ sup (FF((uss ((F((H)::()))) P(Ux x Uxx) (((x)
sup
us ElA
T1 P)+
R
[d(uex u'w)] S {ō]
sup E2EP(Mx)
E
sup ur ELLA
(b(((((() 2221))
x E P(ux x Ux),
T1 = (g+P)x
E (ugg
[d(uz: u'a)l S
(F ((Va: (2)))
sup (F))
sup
Q
"CEMA
The last equation concludes the proof of Eq. Eq. 1 we arrive at the following equation:
12
Similarly, by altering the order of integration in
[+b (F ((ua: un) 1
sup (E3,(IF)
E e
sup u. ella
sup (EBA
E (12~
(14)
To proceed with the proof, we utilize the strong duality theorem. There are various kinds of duality theorems for DRO, but apply the proposed by Blanchet al. [11].
Strong duality 11].
Suppose the transportation cost e : Z x [0, oo| satisfies c(2, 2) 0 for all E Z and lower semi-continuone Then for any reference probability distribution P and upper semi-continuous &
IR satisfying [Z)] oc, we have
sup [v> = inf Ad (Z))
(15)
QEBx(P)
X20
where xx(2) = supzrez {x(2²) x')
Based the assumption about the CFDF, where only d(u, V. 0, it follows that d(<,,=') 0 only x = x'_ Therefore, we can apply the duality theorem to Eq. it can be expressed as follows:
12 According to the duality theorem,
sup us Ella (EB.((guP)))) where nx(uz) = sup Elix {sup {Fb((((ua, 2()))} - Adx (uz, u.)). By using lemma sup, {supa, E (b(F((ua: v(4))}} supa,EuA (supus (v((((u,, (())) {
sup
{
E Na
I (F (F ((ua) ux)))) wimll|
= inf { A20
A6 Ad
E
In [nx("x)]
(16)
u ())()=
Now, since sensitive attributes don't have parents then two spaces Uy and A are equal. By applying Lemma 2 can replace the above equation with hard and soft interventions follows:
(ux) = sup aEA
sup Eix
{EF(F v()))) - (dx(um v/33
{vb(F((a un + A])) - Adx Us + ]
sup
sup Aellx
A
(FF(((, U2 + A))) Adx (dx Px((a, + +1]1] Aellx
sup
sup
a
((F((a, u2 + +))) Xdx ( ux))),, Px "a((a, ur + Am1]=
sup
sup Aellx
a
sup (CF("" - Xo(ū@,C((U ا)) = sup (15)
sup
(17)
A
Aelix
a
20

---

Where (ū*) = supaeux ل (CF("" A)) - Ad(i", AF("",CF(ii CF("". The equations F((a, Ur + A))
CF(U, A) and F((a', uz uz++)) = CF( A) hold true according Lemma Finally, by substituting 8 = CF(U, a) into the equation, we prove the equation:
sup (E)a(()
E m
[
inf
Ad
E umd
sup (s(())
A20
LEEd
where (u( is defined as
(x) = sup (M(CC((v, A)) ) Xd CF)(
AEX
Since, in this case, CF is equivalent CFo, this completes the proof for case one.
Now consider the scenario where sensitive attributes have parents. Eq. T7 shows in strong duality computation it needs to compute function in intervened M concerning the sensitive attributes levels. In this case, instead of using the structural causal model M, it sufficient to employ the parent-free sensitive attribute SCM (Def. 2 -Mp. Mo aligns with the semi-latent space and is compatible with the representation form outlined in Proposition [ By adopting this strategy, we transform M into model where sensitive attributes do not have parents. The proof procedure for Mo remains the same as for M and completes the proof.
Lemma Let (Z.c) be a space with cost function c, Pv an empirical probability measure based on observations {2iJi=1, and define Q as:
1 Nb N N +
P
where 82, and 82g are Dirac measures at and z/,, respectively. Then, the --ssssesttiin distance between P and Q is given by:
W (P,, (PN, = (
Proof. The definition of the D-Waaserstein distance between two probability measures IP and Q is:
2(2,2'Pp dm dz, (2,22))
W (PP
inf =FE((PN.) zxz
where [[P Q) represents the set of all couplings of Px and Q. Since Q is obtained by transferring mass of 1 from 21 to 2/, the optimal transport plan under the constraint that F and Q differ only at two points involves only moving the mass from 21 to z1. The cost of this transportation is c(21, P, and because the entire mass I is being moved:
V
J 2 d=(2, 21) N
Therefore, substituting this into the formula for Wps we obtain:
1 ) N
1 1
W (PPNN Q (c(2112)))
c(21,21)
N
thus proving the lemma.
Lemma' Assume that f(a, is convex in x for each fixed a and continuous in both and x. Also, assume f uniformly continuous in x across a. If A is compact, then the function defined by
g(z) = sup f(a, x))
is convex and continuous in z.
21

---

To show that g(z) is convex. consider any 21,22 in the domain and X E [0, 1]. By the definition of supremum and the convexity of f(a, in x,
Proof.
fa, A + +1 A)r2) S xf(a, + (1 2))(a,
Taking the supremum over in on both sides, we get:
sup [a, A + (1 A)æ2) s sup(AS(a, + (1 (),
Using the properties of supremum.
sup faa Ar + (1 - A)02) & Asup f(a, 01) + (1 - A) sup f(a, aEA
aEA
aEA
Thus.
g(hx + (1- - A)a2) & Ag(z1) + +1 A)g(22),
proving that g(x) is convex.
To show continuity of g(x) point æn, consider any sequence (xn) converging æn. Since is uniformly continuous in æ, given > 0, there exists >> such that for all x, y with lar - yl < ô,
If(a,)) f(a, v) <e for all a E A.
Thus.
f(a, < 20) te and f(a < xn) +e for all E A and [T, xol < 6.
Taking the supremum over in A,
g(xn)) S g(zo) te and g(0o) S g(øn) +e.
This implies
lg(xn)) g(xo)l Se.
establishing the continuity of g(x) at 20.
Hence, conclude that suPae (a, x)) is convex and continuous in
B.2 Proof of Theorem
Let's consider the f(v,y,0 0) M(Y (0, V)) (or in abbreviation f(y)) h(Y (0, where h: IR has one of the forms |4], max L). T|. or max(0,4-17) for some 2 0
First consider the case diam (JA) co. By property of CFDF for each Since the distance of by its twins üg is Zero.Let E } If IPN the empirical distribution by lemma 6 it can be seen for observation Z = (v, y) the distribution
Qa = PN 6= +
Nz=
N (o..))) = Wap PN) = 0 - Qa E Bs(N))
This equation results that
1 sup {(Wa) aEA
R&(PN, 0) 2 sup Rs (Q R(PN. N ((r1) + sup ((ta)) N
N
aEA
aEA
Let (n,, such that Mu. By definition of hard intervention ö is obtained by the formula
82 = =M Mpa) (u (O....,. a U.A...., 0)²)
MA C
0
= = x u M x (0
,0) .)- -Map * # Mp x (0....;0,
.....
- Mo--MXa--C
22

---

where Mpa refers to the effect of parents of sensitive variables and MA is the columns of matrix M related sensitive attributes that show the effects of sensitive attributes on non-sensitive variables. By substituting that last equation in the loss function we have:
R (P 0) Z
sup E(v - N 0
x
sup 0 MA x a)
N
aEA
where B is some constant value. with the assumptions about loss function, all of them by choosing proper Ri, its behavior when large enough is linear so can be approximated by its input value. Now Since the diam (A) = oc therefore the value of a goes to the infinity. Therefore to prevent the value of R& (FN, . 0) it needs that the expression P MA x a in the other word the PA(M needs to be zero. This condition implies that for all E A we have {(v) = E(ta). By using strong duality g we have:
}
sup (E)
[((0)]] = inf uno
E
E uni
sup (x(a))
= inf { 1 + [Ba(w)]]
(18)
A20
LOEA
X20
and
where
(x(1)
sup (E(CF(u, A)) - CF(U,))(( DEX
sup 1(00" (o((Ma ur + A))) - Adx(Ux Ur + A)
DEX
h(Px OM) + A)) Adx( Uhe + +A)=
sup
AEX
h((0o, Uh + A)) - A |||| = (1x) AEX
sup
(19)
In the equation 19 Mo is reduced- form mapping of the parent-free sensitive attribute Ma. By the definition it is easy both SCM, the effect of the sensitive attributes is equal therefore (M 0) Pa(MT8) 8) 0. Moreover since in MMu the structure of non-sensitive attribute has not changed then 0) = Px (M) Finally, by substituting 0g = (M(*0) 0), it can be seen that the problem of finding worst-case loss quantity converts to the regular problem in space X. This problem was solved previously in works of [14
58
25 66]. By using Theorem 3.2 and 3.3 and Proposition 4.1
and 4.2, of work by Chu et al. [4]
write
[ha(un)))
inf X5
E PNNN
inf
18
E us PN)
A20
(R(PM. + ( Px
([(g*PN)an 00) + + 1100ll..)
The equality R((g 0o) R(PPN N. 0) holds by definition and property PM 0. The last equation completes the proof of the first case.
Now let's consider the case that diam (A) < oc.
Case: p E (1, 2o). Lets consider Eq. 16 implies that:
[Es(uz)]
R&(PP = inf X
A5 +
E (gg
(20)
where, E(uz)) = sup,, EM/ {E(M (ta, } and m) (ux) supAgly E(u, + A) Adx u2 A)) By assumption, the whole type of loss functions form h((6, v)) and convex and continuous. can be written by M in the form h((0, M (ua, uz))). Then E((u,) = sup (AP" Px (M-)))),) Since all forms of function are uniformly continuous concerning the u, then lemma implies that the f(uz) is still continuous and convex. Theorem and 7 of work (66) states that:
Theorem. 666] Let e. R + IR be neg-tega,ive Lipschitz continuous and convex function. For an integer p E (1, o0), suppose that for any IP E P(X'), and 2 O, we have:
sup E x)] = (Benr|e (00"=))) 1/p + 11011.)"
(F)
23

---

By applying above theorem in Eq. 20 can be seen:
1/8
Rs(PP)
[
(
+ 8 (x
1/8
P
(Eu-plsup E" (ūa))
+ |X
aEA
Case: p = 1. To complete the proof we use Theorem 2 and Corollary 2 of Gao et al. work (27
Theorem [27] If É is Lipschitz (((z1) f(x2)) I L || zoll and satisfies tightness at infinity, i... for every un there exists sequence {8t [0 E V such that || - zoll + oo we have:
[((xx)) - ((xo)] L.
lim I zo +00
(21)
1 - zoll
then we have (P( = (PP) + S.L.
Now back to the Eq. 20 It is necessary to show that i satisfies the Gao's 's theorem. Let h(t) be one of the functions [t], (t +) (It) - ()+ All of loss function be written as form h(y (0, u)) or hly. (8, v)). is easy to check that h is Lipschitz with constant and there exist ty such that for
Ih(to tx) h(to)] 1.
each to we have lim4+0o
itll
To use this theorem for É, we need to prove that ē is Lipschitz and has a tightness condition at infinity.
Since the h is Lipschitz we know that:
Vua E EMA: IMPA(MTT))u + Px(M Px(M h(P (PPA + O)(
(Px(MTT)(e () w/11 Px(M 1l4 v',ll
sup IMPA(MP + Px - MMP () + Px(MM=))u')
".EldA
sup IM(P + Px(MFO)uL) M(PA(M Px(M)
u.euA
sup M(PA(MFF))u + Px O)))) a, E,AA
sup (PPAM + Px 0)((2)]
miEifa
(ē(u.) - [((')) & |M 1Iue - u''l= +
i is Lipbschitz.
To satisfy the condition of Gao's theorem, it remains to show for uh there exists sequence 1 l((uCt) E(ug))
such that
lim 1106- 11 +00
L To prove it consider that for function h there exists
u2ll
1h(to + tx) h(to)l 1. consider the specific point up and u, E MMA- [tyl
sequence such that limg-
Lets define to -PAlMTT)ua + Px Px(MMTTju Then there exists lg that satisfies infinity tightness. Therefore there exist D E Ux such that (x Px(M(")) tx therefore it can be written:
Ih(to + tx) h(tool
-
[ea|
1h(PA Oyu, + Px (M - + Px |Px(MTT)uu)
limg-oo
Ii(M wtt) |
M(M 0(un, nd)l
limg-
1| +
IMM ((uaa ()
h(M ((UT u29l
limp+000
= -||Px(M0*00)1
sup u. Elia
|
[h(MTO (( xt
h(M M( ub!ll
limg-oo
sup EoEll
||4/||
- -|P)((r
(((()t - {(w0))
limg-000
||
24

---

In the above equation, since we have uniform convergence, we can change the limit and supremum. The last equation shows that there exits sequence of {u k=1 satisfies tightness condition oc. By applying Gao's theorem we have:
no
R5 =B., F)x (.)) + + Px
(8.)) + Px
acA
The last equation completes the proof.
B.3 Proof of Theorem
The case diam (A) oo corresponds exactly to the first part of the proof of Theorem difference being that that theorem we have |pp] 1, while this case, we have | Lb- Therefore we have the below equation:
2 with the only
RF( R(P) Ln P
Now consider the case diam (A) < xc. To prove our assertion, we use Eq. 16 and it implies that:
E [sttun)]
Rs = inf
Aδ A +
X20
(ag p)x
where, E(ux)) supa, Ella (E(M ) and y/ (ux) supaEux A(,, + A) - Xdx Uy + A)).
To compute the right side the above equation, we use the theorem 3.2 of work [4 that states.
Theorem [4]. Let NN = f2I..... In Z be given dataset and Px be the corresponding empirical distribution. In addition, let e(.," .) cost function on Z x and (0, xc) be scalar,
Suppose the loss function é 2 x θ > I,, where satisfies the following assumptions:
(A1) L is Lipschitz respect the cost function d at set EN with L." E (0)
(A2) for any 6 (0, v and each 2; 2n, there exists E such that 5 S d(E,, 2;) < 00 and
((2,) {(z,) 2 (L c)c(2,,
Then we have:
Eg (2( = Ez [e(Z,0) 2
sup P. (Q.WW)5
To use the above theorem we need that & satisfies conditions (A1) and (A2).
A1. To prove the Lipschitz condition it can be seen:
Vua E Ux::
|M(y - M(ua ux)) hly - 0= M (ua: v'y))
S T. P (M)T))()))) - v/21l < Ln Px (M2 0)l.. || -1/1
sup [h(y 8T M (Ua, uz)) hly TT M (ua: u(())l
u. Elxx
(((ua) U((, S Ln Px(M [|. || - -
The case hly. (0, u)) is similar so we omit it.
A2. To check that ? satisfies (A2) condition we show that there exists sequence {A*} such that the | || + oo for every v E V and sequence ii CF (u, A&) and we have:
[h(y (0, vx)) ) h(y x))] Ln. PP
limg
(22)
d(vx, v)
By assumption about h we have: For each 4o E RR there exists sequence of (1 10c goes to [h(to + (x) h((l))
oo then limg-220
Ly. By changing variable e MM MMau Uz). Let to
14
25

---

8 (Maa Uz) and A& E A' such that Px(M')))s = ty it is clear A exist. No if we define 2/4 CF(U, Dx) we have:
l1(to + (x) h(ttall
Ln = =limp
|txl
1yy 0z M(aas Uz))
(x) hly OT (uayuu))
limg->00
(xx())))A
1h(y 8T M (ua. Ux + A&)) hly 8T OTM uz))]l
1im
[Px(MT0)
1h(y 8T 0k)) h(y (Tvo)) -iimk->00
h1y - (0, uk)) - hly (0. v))1 (Px(MTO)I. d(uk, v)
lim
|| 1xx ())1. |1_ A
1h(y - (g)))) h1y - (0, w))1 = -|Px(MMT))] .nn.
= >limp-o
d(uR, v)
The Last equation is valid because, by Holder inequality, there exists such that for all AA the Holder inequality converts to equality. Now it is sufficient that find proper À such that A tw/ ( A) l. so by define A₂ A we find sequence that holds the assertion.
The case of h(y- (0, u) is similar. By discussion of the first part, we can find 4g. Now since we have binary classification, so y 1] we define A sign(y) therefore for such Ag, have y. PxP (x(PP = 4.- By assumption, it can be written as:
[n(to + (x) -h(to)
Ln = =iim
[tx
|h M(ua, uzz)) + ()) My.0T M M(uas uzz))l
limg-+50
11(y.0= M M(ua Uar) y. Px(M A) hly.0* M M(uay Uz))l
limg-100
1h(y.0T M(ua, Uz + Ax)) hly.0" M(ua; ux)))]
limg-+50
1yy vx) h(y.0T w)1
Inyy. (0, vk)) hly (0,
limp-+00
1xxxMx || A limp+00
[Px( d(vR, v)
[h(y (0, vx)) )- dl ((Vk,") v)
h1y 0))) ||xx [()l Ln
= -immx+20
Let ((v) = h1y Px(M (A(M O )..)). By assumption I is Lipschitz so it l is Lipschitz concerning each u, and un and U,4 is bounded so it is compact. Then these properties imply uniformly continuous so we have:
hLy vx)) hly - (0. v))l
sup limp u. Ell
d(ux; v))
h(y (8, uk)) hly (0, v))l lim d(vk. v)
(((ux) T(Ax))
limg-000
sup Ell
= PPx
| A&ll
(((ux) {(Ax))
The last equation satisfies the (Az) condition because since lime-
|1u Axll
Px LA in other hand we have (((ua) ((u(2) < La || (M |l. ||u - -u' ||. then for
each there exist A such that (ē(uz) ?(Db)) > (Ln Px(M A)]|_ ||u, Axll -c) l|ur A
Now we can use Chu's Theorem and it implies that: R₂ (TP) = R. (P) + T., | (M/ 0)],.- The classification case is the same and it completes the proof.
26

---

B.4 Proof of Theorem 4
Let's prove the necessary condition. By first part proof of theorem we have
PN,, 8) z sup E(Va: V. 0).
N aaA Therefore, for a finite solution to exist for the DRO problem, is necessary that:
sup
E(ia, y. < co.
.E.A
To prove Eq. 10 we use again some parts of the proof of strong duality theoren and idea of proof of theorem 9.1 of Garcia's work [29]. It states:
RF(PN)= = sup (Px)
[e(v', v. 10(0' m~Q
E
sup a. (x) Q.EP(A)
E (('a v.. uf Mr
where by discussion in lemma
4 we suppose that Qu and Qa are independent of each other. For simplicity, we define J(ug. Un 0) = E(g ((u,: ),)). 0). By assumption. since the f is twice differentiable, then (T f)- is also twice differentiable. Because the function g is obtained by - f by removing the functional structure of sensitive attributes and a set identity function instead of them, there are two functions g and gndg- These results show that combination {(g-1) is also twice differentiable, so the gradient of J exists concerning the ug. By using Taylor's 's expansion theorem f: IR the function that has gradient then the first order estimation of f equals:
L. (Vf(z + th) - f2())) dtt
f(x + h) = f(x) + Vf(x)
Let Ug ~ (o*P)x by writing Taylor's expansion around uy we have:
B [[ u/a 0)] = TE [J(x: v'a, 0) + V u/a 0) (u'g uz)
L V Vu + M(u'. uz), ula 0) V Vn (uzi u'a 0)3 (1/2 un Mnddd Since the diam (2)) < differentiable, so VzJ(., (,, ua, 0) is Lipschitz with constant J[,, ua, 0) I1lp that there exist L < oo
suppose that the space UI is compact. By assumption J is twice
such that I. uay lt S L.. By these assumptions, can be written:
E
( 1uu + A(ul ux), uaa 0) V V v'a 0)} (u'a - uz)dA 5
E
Vnr A(1ls - uz), 1/, 0)
V J (Uz₁ v/ 0) Illlul ux 112c2
V (., 2/2, 0) 11u 1126 - uu 11
= E
V V [., w/a 8) Lip 11ly
Uu 112 5
2
C
CL
V (., u/, 1) [11// u 11 v
[|14 u 13 O(8)).
2
where C' is a constant that arises from the equivalence of norms in Rd, it means that there exists C |I-1l < CII.Il- The inequality [[|u|] u [| I 82 is valid because by definition Qu PP)() and the cost function in the space Ux is expressed by the l|u1 u. so by definition of IBs (Bs( x))), for p 2 2 by applying Jensen's 's inequality we have
Qr E (Bx PN)x) =Eg. [ll uzz |
S <
ER, [|lu// u. 12 S Ec. Illu1/ ux [P] <².
Since by assumption J. is uniformly Lipschitz for different value of and ua therefore we have:
R₂ (P)
sup G. C. px)x) (AA)
{ [[] u', 0) + V u'a 0) (w'_ u.)) } + O(82),
27

---

for O independent of 0 and Ug. The first expression of the above equation has the simple form:
(a )
(J(uz ula 011
sup QuEP(ULa
E Ma (g) V/a 2u
[J(u, u
01)
E
sup QUEP(UA))
E [1, uaa
(gw PNlx w
E (a Px )x
sup ElLA
{J(uz, u'a, 0)}
E (oulFNa
sup 2l,E
{elg u())...00)
sup LaEAA
((ia: y, 0)
P--P
The only term in the above equation that still depends on the Q is that term Ju, uha 0)-(u/ -2z). To remove this term use extended Hölder inequality with the expectation that be expressed using the following formula:
B|J s [BI (B)(()))
and equality holds if and only if there exist constants 2 0 such that:
Y c| almost surely, c O.
By using Hölder inequality, with the same reasoning we have:
}-
[azi u'a 0) (u's - uz)) m
sup Q. Px)((())))) (AEP(A)
E u Cu By ~C
sup VVz Wao 0) (ulz - un))]]
sup Q (eoo(ggPP)a)
Go Ella
E
sup
V (u u/a 1)H
sup Q (.
E
[|]xLs - u 11']
uo PN) af, Ella
(9 Nx)x
sup (I[VzJ(uz, v/a 0111331
E My (o w EUA
where equality can be attained whenever S p S oo for a proper choice of v', U2 with (E(lln' uz ()) = 8. Therefore,
Rs(Px)= = E
sup E(ū,, v, LaEA
+ E
(sup pwPN aEA
v, - 0
VNNN
where
((CFolv, A)) S(v)
v-l(v.u.0) = lim 4-0
|
the last equation completes the proofs.
B.5
Proof of Proposition 2
By Eq. 16 we have:
[3r(un)]
RF inf X8 E
X20 us ()>
where, i(ux)) sup,, { (u,, u(r)} and mx(ux) supseur Eluz A) - )dx(uz, Us
A)}. To prove we use Corollary 28 for the É. First, we need to show that ? satisfies the condition of Corollary 2 (28). By assumption for v' V. L, M > 0 such that | v)
28

---

e(1', v.0) < (u r') + M for all VE V and p E o0). By setting v g ((ua: Ua)) and
o' =9 9 (1'g) u/a), and for simplicity {(v) (vv y. we have:
1k49 '(Ma; Uz))) { "("aa 22))i < I. 1| - "' + M, Vus E Uxx ua E EUA 7 ((uw)) [(ugg) - L 11ue u1,1 11
+ M
where ((uz) = supu, El4, lgg (uua. uz))), This equation implies that é satisfies the condition of
corollary 2. So in consequence corollary and the equation 16lif we define uncertainty set:
N K
Bs = (w V.
w*l|| s 6, with E Ux
Since the casual fair metric d and loss function i not depend on the sensitive part then Bg is equivalent to the below uncertainty set.
N
K d' wh) S with v)
Bg (
By applying the uncertainty Bs for é, the robust optimization problem has form: R
(PN)- =
N
K
K
i(wttty
max ((g (( ((May w Ella
sup (
sup De
NK
NK
i=1
Km
1
(with }))
sup E
max aEA
NK
and finally for Rgdv (Pd)(())) we have:
LD + M
Rede & Rgd
NK
and it completes the proof.
Lemma Assume that the cost functions e and satisfy lo(x, x/) - č(xx x²)l < a for all x. TR
and for some 2 O. Then, for any X 2 0, the difference between the X-conjugates of f with respect to and is bounded by Xoe:
[x(x) fx(x)) s Xa for all E R".
Proof. To prove the proposition, we consider any x E IR" and examine the definitions of fa fx(x) and fx(r). Begin by expressing the bounds on è:
ε(I,x') S c(æ, x²) a and c(x,x') 2 c(e, a. From these inequalities, for any x' E IR"
f(x') A8(x, x') 2 f(x') X(c(z, a - Ac(2, x') da
f(x') A6(x, x') S f(x') x(((x, a) f(x') - Ac(2, x') + Aa.
Taking the supremum over all x' in the above expressions, we obtain:
fx(x) 2 fx(x) λa and fx(x) S fx(x) + Aa.
These two bounds together imply:
- - fa(x) to
Thus, the proof is complete, showing that the difference between the X-conjugates of f is indeed bounded by Xo.
Lemma Let C : R x R. + be two funnctions such that le(x, v) Ö(x, y) < a for all J.y T and some O. For any real number p 2 1, the following inequality holds:
le(z, v) č(z, y) p1 < p -1-1, a.
where M 2 max{/e(a, v) Jê(z, w)1} for all x, y.

---

Proof. Consider the functions and and any 2, y E IR.. By the hypothesis, we have |e(x, y) 0(z, y)l a. To find a bound on the difference of their powers, apply the mean value theorem the function f(t) = IP, which is differentiable over IR (or over R= if p is not an integer). The derivative of f is (t) = ptP-1.
Since f is continuously differentiable, there exists some & between e(2, v) and é(z, y) such that f(c(z, 9) f(((z, y)) (() (c(x, v) v(z.))).
Therefore.
1c(a, y) 6(x y) 1pEp= y) Ö(z, 8))l
Using the bound (e(x, y) c(2, y) and noting that must be within the range of values between c(2, y) and 6(x, y). we have g-- < M Thus,
1e(2, v)" - ö(2, y)*1 S pMM "I(p, u) - ((x,u) < pM a.
Lemma 10 (141) Fix some IP P(2), 0 E θ and X" 0 via
X = ==
argmin (A0) + Er,
Then under Lipschitz and diam 2 assumptions, have X* s L5 (5-(p-1),
Proof.
First, note that:
8 S A-8 + EE
sup () ((2,0) - L2'EZ
since the left-hand side, is greater than the case where z' = z so it is positive. By the optimality of X in f, the right-hand side can be further upper-bounded as follows for any 0:
v Ad + ES
sup (f(z',0) - ((2,0) (x,
JEZ
< X8 + E3
sup (Le(z,2") Ae (2,22))
E'
$ X8 + sup At"}.
120
using the Lipschitz property for the second line and setting t c(2, 2/) in the third line. If 1, by setting À = L, we obtain:
X S L8 sup Lt} LS,
which implies X < L. For p > 1, using the optimal value t = (L/pA) we derive:
< 8
X8 L p p- (p 1)1 p-.
Minimizing the right-hand side with À L/p5" yields:
8 < Ld + X < L8
L8-(0-1)
resulting in the stated bound on X
Lemma 11 I x R is Lipsehitz with respect to x uniformly in a, i.e. there exists constant L, such that for all E A and all x, IR..
(fa, - f(a v)1 S Ll|x vll.
then suPaeA f(a, is also Lipschitz in
30

---

Proof. Let F(z) = s f(a,x). We aim to show that there exists a constant L' such that for all y, R",
F(z) F(v)l S 1/ vll-
Since is Lipschitz continuous with respect to z uniformly in with Lipschitz constant holds for each E A and any æ, IR" that
If(a,z) - f(a, y)1 S Li[a yll.
Consider F(z) and F(y). By definition,
F(x) sup xa, and Fly) sup I sa, v). aEA
aeA
For any a E A, since [a x)) [(a, v) S L|| yll. we can infer that f(a,z) S fa, y) + L| vll- Taking the supremum over all E A on both sides, we obtain
F(x) F(y) L|| y||-
Similarly,
Fly) Fz)) L|| | y||- Combining these two inequalities, we find
[F(x) - F(y) - L| yll.
Therefore, F(x) = suPa5A (a, x) is Lipschitz continuous with Lipschitz constant L. This completes the proof.
B.6 Proof of Theorem 5
Let define define (P, SUPo (Q.P)ss (z( the worst-case loss quantity over esti-
mation of the metric d and 0, = infoce (aP. (c((, [1]3 Therefore by definition,
we can write:
(s(PP.00r)) Ro S R5(P., 0 - (IP -R, (R((P,,O.) (23)
because we have RP (r)) - RP 0.).
We estimate two expression RR, (IP )r)) - R& N, , 0 ) and (P.,P.,).) RI general strong duality theorem 26 we have:
- 0.)l. By the
Rs 0.) Rs(PN
sup (QP.))
EglE(Z,0.))
sup (GFN)))
Eg 8.)1
inf {A8"- Ez. (Z, (Z, 0.)]] inf {28" EeN (2, 0.Z. AS0
A30
ANd" + EE, |C (Z, 0.)) } - (AN80 +Ep 1 ( (Z,0.)) Ep. [CS (Z, 0.)]} RR [C,, (Z.
where the xy := arginf, X { AA6 + Ep (Z, 8.)]}
By assumption 2 and lemma 9
1xx (2, - xxx (2, = sup {(2' y.. 0) - (nd sup E(v v. ANd vey S sup AN (v', d' v) S An p diam (V) M4 N vev
r'EV
This implies
Rs(P.,0.) R. 0.) S Er. [65 [2,
Ery [C 0) AN diam (V)P-1 Ma N-
31

---

If define A, :- -arginf, X20 {16" + [Be[<{(2, similarly,
R8( 0.) - Rs(P. 0.) S Ex [ (Z, 0) -Er. [ (Z, 0)]
EN 1e. (Z, 0)] Ee. (2. + X p diam (V) Ma N N
We need to estimate the AN and A.. By using strong duality theorem by Eq. 16 we have:
(P) inf { A20
Ad "a P)xx
E [ ((((ua))
(24)
So instead the solve problem for e is it sufficient to prove our result for E(ux)
sup,, EMA e(g-(((us Uz)). At first, we show that é is Lipschitz on the space Ux concerning the norm ||-1|- By assumption, for each a E A the function E(g- un)) is also Lipsehitz: ||e(y ((u uz)), y, 0) "( Ux + A)), y, 0)
[je(CF(D, a), y. f) ((CF(V, a, A), v, 0)1 S Ld(cF(U, a), CF(r, A)) L ||
Now by using lemma 11 it can be concluded that the function ((u,)) sup,, un ELL (l10-(((1) ((ua Uz)) also has Lipschitz property with constant L. By Applying lemma 10 for equation 24 and (o+ PN)x, it can be seen AN, A_ I L5 15-(p= (p-1), Therefore until now, we have two inequalities:
(.) RA(PN. N. 0.) S R.. [ (2,0) 0)] EPN [ (Z, 0]] AN p diam (V) Ma N-
RT( N. 0.) Ro(P., 00) S Er [ex. (Z. 0] Er. [4S. (Z, 0)] ^ diam (2))-1 M2 N
RR (P., (2) (P 0.)1 sup 1.ʃz (2) - (=)) + L8--P p diam (v)=-1 Ma N-",
SEC
where C- = (.)
[0, Lr" E θ} is the DR loss class.
In the remaining part of the proof. we estimate suprect Is f(z)d(P. [x)) using conventional methods from statistical learning theory. According assumption 2 limited as shown:
the functions within F are
0 S (v, < sup E(o', 0. ad(c,v') I sup l(v, v. 0) M.
41 by utilizing the V'EV
w'EV
similar to the proof Theorem tion, we derive that:
inequality and symmetriza-
log 2N
sup 1sz [())d(Px P..)())) < 29x M
SEC=
holds with probability of at least e, where Ri, (C9) represents the Rademacher complexity of 2
N Rxx(C9) = E sup Σ o2.
SEee N
1=1
In the proof of Theorem 2 41
the authors has proved:
Rv(C < 24C(C) 24L.diam (V)
VN
VN
where C(C), the entropy integral of of loss class. By applying this result it can be written:
log 2 2N
4SC(C)
48L.diam (V)²
RR, 0.. 0.)1 S MV
R.s
Vnde-1 + p diam (V)p-1 Ma -
Vn
Since the prove does depend on value 0, then |Ra ( Ro N. ( also satisfies in the above inequality. By combining two terms with probability 1 - we have,
2 N
9GC(L) vn
96I.diam (V)) +2L8'-P diam (Vyp-1 Ma N
RR (T.., ( - s My
Vnd"-1
Since by assumption, with probability - we have inequality Zz E |c(z, - 2(2, < M&N-7,
>
therefore by probability 1 - 2c the main inequality is true and it completes the proof.
32

---

C Numerical Analysis Supplementary
C.1
Synthetic Data Models
The structural equations used to generate the SCMs in $ 5 are listed below. For the LIN SCM, generate the protected feature A and variables X, according to the following structural equations:
linear SCM (LIN)):
A UA,
U^ B(0.5)
X = 2A + U1,
U ~ N00,1
X2 = A - X1 U2
U2 ~ ~20,1
Y Y~ (((1 + exp( (X + x²))=1)
Here, B(p) represents Bernoulli random variables with probability p. and (ju, σ²) represents normal random variables with mean μ and variance o², To generate the ground truth h(A, X2), we use linear model for the LIN method. In all the synthetic models considered, we treat A as benary-sensitive attribute.
C.2 Real-World Data
In our research, we have utilized the Adult dataset [38 and the COMPAS dataset 665 for our experimental analysis. To employ these datasets, we initially constructed an SCM based on the causal graph proposed by Nabi et al. [49]. For the Adult dataset, we incorporate features such as sex, age, native-country, marital-status, education-num, hours-per-week and consider gender as sensitive attribute. In the case the COMPAS dataset, the utilized features comprise age, race, sex, and priors count, which function as variables. Additionally, sex is considered sensitive attribute.
For classification purposes, we apply data standardization before the learning process.
A U
Sex
X₂ U2
Age
X3 U3
Country
Adult
X, -U4
Marital Status
X5 8
+ 8,2X, + Bsa + 8,4 U,
Education Level
XS X
+ 862X2 + 83 + 3, + 96s X5 + Us
Hours per Week
X1 U1
Sex
X2 = U2
Age
COMPAS
X3 U3
Race
X4 = B11X1 + B42X22 + B13X3 + U4
Priors Count
In these above equations, 3yy are the coefficients for the linear combinations of the variables, and U/ are the exogenous variables.
C.3
Training Methods
In study, utilize various training objectives train decision-making classifiers, with loss function ((v). The training objectives are as follows:
Empirical Risk Minimization (ERM): This approach minimizes the expected risk con- cerning the classifier parameters y), represented by
inf Ez-p [[,p]((2, 0)]
33

---

Adversarial Learning (AL): This method trains the model to withstand or defend against adversarial perturbations, represented by
sup ((= A 0 |} || 5ó
inf DE
Eg-7
Based on the work of Ross et al. [56), this method minimizes the expected risk along with an adversarial perturbation term, represented by
ROSS:
Eureree ((v. (vv.v.0)+ + inf E(v + -, 1, 1 |
inf EEB
CDRO: Our approach, as described in this paper, formulated follows:
inf
sup E
E E
For our loss function É, we use the binary cross-entropy loss.
C.4 Hyperparameter Tuning
The majority the experimental setup is based on the work of Ehyaei et al. 22 For each dataset and its respective label, to four different datasets, using 100 different random seeds. The optiiization process is performed using the Adam optimizer with learning rate of 10-3 and a batch size of 100. After optimizing the benchmark time and considering the training rate, we set the number of epochs to 10 to ensure comparability in benchmarking.
generalized linear model (GLM). Each training objective is applied
C.5 Metrics
To assess the performance of various training methods concerning accuracy, unfair area, counterfactual fairness, and adversarial robustness, we employ seven distinct metrics as outlined below:
Ace: The accuracy of the classifier, is represented a percentage.
Ug: The proportion of data points within the unfair area with radius of 8.
Ug = P(frev Eo' Ev s.t. d(v, 8 A h(u) + h())))
Rs: The fraction of data points that are vulnerable adversarial perturbations within radius of 5. This metric coincides with the unfair area in cases where no sensitive attribute is considered.
Rs P([ EV: FAEE s.t. d(v, CF A)) < 5 A M(v) # MCFF(1, A))))
CF: The percentage of data points that exhibit counterfactual unfairness. This metric aligns with the unfair area when the perturbation radius zero.
CF P((EE)
Ea E
M()) + M(ūa)))
C.6 Additional Results
In this section, we present additional simulation results. CDRO performs well across datasets except for the Rs measure in the Adult dataset, likely because the linear model does not fit the SCM well. Nevertheless, CDRO demonstrates robustness and counterfactual fairness, as shown Table T making it the preferred model when balancing both accuracy and fairness.
34

---

Countortacuall Unbox Area
Non-mbuut Aree 0.055
4%
2076
3%
Syetlk Daaa
20%
2%
10%
10%
176
9%
640
Figure 2: Displays the findings from our numerical experiment. assessing the performance of DRO across different models and datasets. (left) Counterfactual unfair area percentage (lower values are better). (right) Non-robust area performance of classifier (higher values are better) for A .05.
Unior (A (0.01)
35%
Nop-robuut (A. -0.01)
0.78%
20%
10%
20%
92%
0.22%
9%
6.0%%
0%
Figure 3: Displays the findings from our numerical experiment, assessing the performance of DRO across different models and datasets. (Left) Bar plot showing the comparison of models based on the unfair area percentage (l)owe values are better) at A = ,01. (Right) Bar plot showing the comparison of models based on the robustness percentage R(6) (lower values are better) A .0L.
Broader Impact Statement
Our theoretical framework bridges adversarial robustness, distributional robustness, individual fair- ness, and causality, aligning with the core pillars of responsible AI. By demonstrating the connection between these areas, we aim to inspire further research at their intersection and contribute to the development of safer, more equitable AI models for society. This approach holds the promise of improving decision-making under uncertainty while ensuring faimess and mitigating the impact of adversarial perturbations.
However, we acknowledge several limitations and ethical implications inherent in our approach. While our method produces fair and robust predictions under specific conditions, to note that it fundamentally relies on a machine learning model, which may inherit the same vulnerabilities as the original model in areas not explicitly addressed in this work such as multiplicity,
important
Real- World Data
Synthetic Data
Adult
COMPAS
LIN
Trainer
Ace Vo.s
Una
CF
R.
R.o1
Acc
Up.s
Uo.1
CF
R.
R.
Acc
Uo.a
Uo.1
CF
R.os
R.01
AL
0.79
0.06
0.04
0.04
0.03
0.01
0.67
0.2
0.17
0.16
0.04
0.01
0.67
0.2
0.19
0.19
0.11
0.1
CDRO
0.73
0.04
0.01
0
0.04
0.01
0.66
0.05
0.02
0.01
0.04
0.01
0.66
0.04
0.03
0.03
0.02
0.02
ERM
0.79
0.05
0.02
0.03
0.68
0.26
0.23
0.22
0.04
0.01
0.69
0.39
0.38
0.15
0.13
ROSS
0.78
0.06
0.04
0.03
0.03
0.01
0.67
0.33
0.3
0.3
0.04
0.01
0.69
0.44
0.43
0.43
0.18
0.17
Table I: The table presents the results of our numerical experiment, comparing various trainers based on their input sets in terms of accuracy (Acc, higher values are better), unfairness areas (U lower values are better), unfairness areas (U.02 lower values are better), Counterfactual Unfair area (CF, lower values are better), the non-robust percentage concerning adversarial perturbation with radii 0.05 (R os. lower values are better), and the non-robust percentage concerning adversarial perturbation with radii 0.01 (R. lower values are hetter). The perforroggnng techniques for each trainer, dataset, and metric are highlighted in bold. The findings demonstrate that CDRO excels in reducing unfair areas. The average standard deviation for CDRO is .029, while for the other methods, it is .031.
35

---

privacy breaches, lack of explainability, and safety/securrtt concern.. Additionally, our work operates under simplifying assumptions regarding the fairness notion. In real-world upplications, fairness is complex and context-dependent concept. Therefore, it is essential to define fairness carefully and consider multiple dimensions of fairness when applying our approach. We emphasize that this work is a proof of concept, and we strongly recommend involving diverse stakeholders, including ethicists, domain experts, and affected communities, before applying our approach to high-risk application domains. It's important for users to be aware of these limitations and potential biases that might not be fully addressed by our framework.
36

---

D NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our claims in the abstract and the introduction match the body of the text, the provided proofs, and the reported experimental results.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discuss the main limitations of our work in the conclusion.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: We provide general assumptions in the beginning, and all the lemmas and theorems have either full proof or general intuition in the main body of the paper, with the remainder of proofs in the appendix.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We mention the used datasets and models, the random seeds, and details about the data splits in the main body of the paper and the appendix. Information about libraries and used algorithms is also provided.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We mention the used datasets and models, the random seeds, and details about the data splits in the main body of the paper and the appendix. Information about libraries and used algorithms is also provided.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
We give detailed results in the form of tables (with mean and std), while we only plot the mean values not overerowd the figures and just report the standard deviation in the table caption.
Justification:
8. Experiments Compute Resources
37

---

Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [No]
Justification:
The experiments are hardware agnostic and use fairly small models and
datasets.
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics httpae///aarrg....ppppppl//aaaaa
Answer: [Yes]
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We include broader impact statement in the appendix.
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper does not create new data models.
12. Licenses for existing assets
Question:
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We properly credit the used datasets.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We documented the code.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/teview based on the requirements of your country or institution) were obtained?
Answer: [NA]
38


Reviews:
[{"review_id": "mw8YgX3NEz", "reviewer": "Reviewer_vsYr", "summary": "This paper proposes a novel framework called Causally Fair Distributionally Robust Optimization (CDRO) to address individual fairness in machine learning. It combines causal modeling with distributionally robust optimization, using a causally fair dissimilarity function (CFDF) to measure individual similarity while considering sensitive attributes. The framework provides a strong duality theorem, enabling efficient computation of worst-case losses under distributional uncertainty. It offers explicit solutions for the regularizer in linear Structural Causal Models (SCMs) and estimates it for non-linear SCMs, mitigating overfitting and ensuring fairness. Additionally, the framework provides finite sample guarantees for convergence even with unknown SCMs, enhancing its practicality. Empirical evaluations on real-world and synthetic datasets demonstrate CDRO's effectiveness in reducing unfairness while maintaining accuracy compared to other methods.", "questions": "Given that the proposed approach demonstrates lower prediction accuracy compared to existing methods, could the authors provide insights into potential factors contributing to this outcome? Additionally, how might this trade-off between prediction accuracy and other performance metrics be justified in the context of the method's overall objectives?", "limitations": "NA", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Introduces a new framework that integrates causality, individual fairness, and adversarial robustness into DRO, providing a comprehensive approach to address fairness concerns in machine learning.\n\n- Offers several theoretical advancements, including a strong duality theorem, explicit regularizer formulations, and finite sample guarantees, contributing to the theoretical foundation of fair and robust machine learning.\n\n- The framework is designed to be practical for real-world applications, even when the underlying causal structure is unknown, making it a valuable tool for addressing fairness in various domains.", "weaknesses": "- Please define the abbreviation SCM before its first use in line 49.\n\n- The experimental setting description could be improved. Consider providing an algorithm for the proposed approach to enhance clarity.\n\n- The paper appears to have considerable overlap with Ehyaei et al. (https://arxiv.org/pdf/2310.19391):\n  - The first contribution claimed in Section 1.1 (line 64) seems to have been previously established by Ehyaei et al.\n  - Definition 1 and Proposition 1 in Section 3 appear to closely resemble Definition 2 and Proposition 1 in Ehyaei et al. Please clarify the novelty of these elements.\n\n- The framework's reliance on an additive noise model assumption may limit its applicability in complex real-world scenarios. Could you discuss potential impacts on CFDF accuracy and fairness guarantees, and any plans to address this limitation?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your helpful comments. We will address each point of weakness and questions, labeled **Wi** and **Qi** respectively, in order.\n\n**W1.** We will add the Structural Causal Model (SCM) in line 49.\n\n**W2.** We appreciate your comment and agree that the numerical section could be improved. As mentioned in our global response, our method is not specific to fair learning. In the numerical section, we aim to showcase just one of the many applications of this framework. However, we faced challenges in finding datasets and methods compatible with our problem. \n\nThe algorithms in our work are similar to those used in regular optimal transport, such as the Sinkhorn algorithm [1]. In these algorithms, we simply replace the matrix of the cost function with the one calculated based on the CFDF function.\n\n**W3.** As mentioned in line 54, we adopt the definition of a fair metric from Ehyaei et al. to define a causally fair dissimilarity function (CFDF). However, their notion of a fair metric is not compatible with our assumptions. We require a more general definition, namely a dissimilarity function  because our cost function does not satisfy some of the fundamental properties of a metric, such as:\n\n- **Identity of Indiscernibles**: $d(x, y) = 0$ if and only if $x = y$.\n- **Triangle Inequality**: $d(x, z) \\leq d(x, y) + d(y, z)$.\n\nTherefore, to avoid misleading use of the term \"metric\" and to have broader applicability, we define a general notion of a dissimilarity function instead of a metric, which has weaker assumptions.\n\nTo obtain a representation theorem of the CFDF, we need Proposition 1. Unfortunately, Proposition 1 in the work of Ehyaei et al. does not work in the general case and is limited to metrics.\n\n**W4.** As mentioned in the global response, since our work uses counterfactuals from structural causal models, our model must be counterfactually identifiable. Therefore, we should base our method on counterfactually identifiable SCMs, such as the bijective generative mechanism (BGM). To avoid additional complexity, we focus on the additive noise model, a specific BGM instance. However, our results are valid for general BGM.\n\nIn our framework, the primary concern is the counterfactual identifiability of the SCM model. Once the SCM model or corresponding cost function is estimated, Theorem 5 guarantees that the learning problem has a convergent solution for finite samples.\n\n**Q1.** In fair learning, it is well known that there is a trade-off between individual fairness and accuracy. Achieving individual fairness necessitates adjusting the model to account for variations across individuals, which increases complexity and the potential for overfitting. This adjustment may reduce accuracy, as the model must balance fitting the overall data distribution with adhering to fairness constraints, potentially limiting its predictive precision. Therefore, increasing fairness often requires sacrificing some degree of accuracy.\n\nThis trade-off is justified by the method's primary objectives: reducing disparate impact and ensuring equitable treatment across individuals. While accuracy is important, the approach prioritizes mitigating biases to promote fairness and inclusion in algorithmic decision-making. Thus, the intentional trade-off of reduced accuracy aims to achieve a more equitable and socially responsible model.\n\n[1] Cuturi, M. (2013). Sinkhorn Distances: Lightspeed Computation of Optimal Transport. Advances in Neural Information Processing Systems, 26, 2292–2300.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Mn0P45jIwr", "reviewer": "Reviewer_rNAj", "summary": "This submission studies the connection between Wasserstein Distributionally Robust Optimization (DRO) and individual fairness in certain Structural Causal Models (SCMs). Namely, it is first shown that, in the case that the SCM at hand is an Additive Noise Model (ANM) with known structural equations, one may define a Causally Fair Dissimilarity Function (CFDF) on the feature space in a canonical manner. \n\nWith this, the remainder of the paper concerns the problem of DRO of the risk function (i.e. minimization of $\\mathcal R_{\\delta}(\\mathbb P,\\theta)$ the worst-case risk over all distributions which lie in a (Wasserstein) ball of radius $\\delta$ from $\\mathbb P$). Notably, a dual form for $\\mathcal R_{\\delta}$ is provided by extending known results; this effectively converts the infinite-dimensional maximization problem defining $\\mathcal R_{\\delta}$ to a finite dimensional minimization problem. In the case that the center $\\mathbb P$ for the Wasserstein ball in $\\mathcal R_{\\delta}$ is an empirical measure from $N$ samples, $\\mathbb P_N$, it is shown that, under certain assumptions on the loss function, $\\mathcal R_{\\delta}$ can be recast exactly in terms of the standard empirical risk or the objective from the counterfactually robust optimization problem depending on the size of the diameter of the set of sensitive attributes. Under weaker assumptions on the loss and that the SCM is linear, a first order (in $\\delta$) expansion of $\\mathcal R_{\\delta}$ is also provided. Next, it is demonstrated that standard adversarial optimization can be used to approximate DRO. Finally, the rate of convergence of $\\mathcal R_{\\delta}(\\mathbb P_{\\star},\\hat \\theta_N^{\\mathrm{dro}})$ to $\\inf_{\\theta\\in\\Theta}\\mathcal R_{\\delta}(\\mathbb P_{\\star},\\theta)$ is characterized.   \n\nThe paper concludes with a numerical study of the described causally fair DRO (CDRO). Namely, a comparison between the CDRO and other common approaches is provided on real-world and synthetic datasets. It is shown empirically that the CDRO exhibits slightly lower accuracy than the other models, but yields a lower unfair area (this is especially evident in the COMPAS and LIN datasets).", "questions": "1. I believe there is a small mistake regarding assumptions (i)-(ii) and the given examples. Notably the quantile loss is 1-Lipschitz, but $|h(t_0+t_k)-h(t_0)|=|\\gamma t_k|$ if $t_0\\geq 0$ or $|(\\gamma-1)t_0+\\gamma t_k|$ otherwise. In either case the limit from assumption (ii) is $\\gamma\\neq 1$. Perhaps there is a sign mistake? \n\n2. In Theorem 4 it is stated that the necessary condition for the existence of an infinite DRO solution is that [...]. Should it not be a finite DRO solution?", "limitations": "The authors address the limitations of their work in the conclusion and the broader impact is addressed in the appendix.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well-written and its contributions are clearly identified relative to the existing body of work. \n\nAlthough the connection between DRO and individual fairness has been considered before (in the linear SCM case), I believe the extension to the ANM case is of interest. Furthermore, the section on duality and corresponding representations of $\\mathcal R_{\\delta}$ provide a nice interpretation nice interpretation for this approach.", "weaknesses": "1. It is difficult to get a sense for how strong some of the assumptions made in this work are. Although most of the assumptions are coupled with some examples of cases where they apply, I believe it would be relevant to provide some rationale for why these assumptions are necessary or describe primitive classes of examples where these assumptions hold rather than just some specific examples. \n\n2. Assumption 2 (iii) requires estimation of the CFDF; it would be useful to expand a bit more on this assumption keeping in mind the above point or at least provide some heuristics for what rates one can expect in general. \n\n3. More generally, the derived results would benefit from some additional discussion regarding their implications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "First of all, I would like to thank you for your insightful comments. We appreciate the attention given to our work. In the following, I respond to the mentioned weaknesses and questions in detail.\n\n**W1.** In the global response, we explained the intuition behind our assumptions. \nAssumption 2 seems new, but it is common in finite sample guarantee theorems, such as Theorem 6 of [1]. The first part of Assumption 2 is natural, assuming the feature space is bounded and the parameter space is bounded and closed.\n\nThe third part is also reasonable, relating to the statistical properties of the estimator for the cost function or causal model. For example, in a linear SCM, we can estimate its functional structure with a convergence rate of $O(N^{-\\frac{1}{2}})$, allowing us to design a metric that satisfies the third assumption.\n\nThe second part of Assumption 2 is the Lipschitz condition on perturbing non-sensitive features, derived from counterfactuals. For example, consider a linear SCM with reduced-form mapping $M$, where $X = MU$ and all sensitive attributes are parents. Here, $CF_0(v, \\Delta) = v + M\\Delta$. Given a Lipschitz loss function $\\ell(v, y, \\theta) = h(\\theta^T v - y)$ and an $\\ell_p$ norm in the exogenous space, we get $d(v, CF_0(v, \\Delta)) = \\\\|\\Delta\\\\|_q$. This makes the Lipschitz condition:\n\n$$\n\\vert \\ell(v, y, \\theta) - \\ell(CF_0(v, \\Delta), y, \\theta) \\vert = \\vert h(\\theta^T v - y) - h(\\theta^T v - y + \\theta^T M \\Delta) \\vert \\leq L \\|\\theta^T M\\|_p \\\\|\\Delta\\\\|_q\n$$\n\nIf you need further clarification, we would be happy to engage during the discussion period.\n\n**W2.** Thank you for highlighting this point. Assumption 2(iii), related to the estimation of ANM, depends on the functional structure and smoothness of the model, resulting in different convergence rate orders. For example, for a linear ANM, the convergence rate is $O(N^{-1/2})$. We omitted a detailed discussion on this rate as it pertains to the estimation of SCMs more broadly. \nEach estimator with convergent rate $O(N^{-\\alpha})$ where $\\alpha>0$ works in Theorem 5.\nIn the revised version, we will add explanations to clarify this topic further.\n\n**W3.** To present a minimal and complete version of our proposed method, we focused heavily on the theoretical section. To address its applications and implications, we detailed its relation to previous research and included numerous references.\n\n**Q1.** We appreciate your attention. The words **for each** and **exists** were used incorrectly, and we have fixed them in the revised version. The correct assumption is:\n\n- For each $t_0 \\in \\mathbb{R}$, there exists a sequence  $\\\\{t_k\\\\}$ that goes to $\\infty$ such that we have  \n$$\n\\lim_{k \\rightarrow \\infty} \\dfrac{|h(t_0 + t_k) - h(t_0)|}{|t_k|} = L_h\n$$\n\nTo provide intuition behind this assumption, for each $t_0$ and each $\\epsilon >0$, we need to find $t_\\ast$ such that $|h(t_\\ast)-h(t_0)|>(L_h -\\epsilon) |t_\\ast-t_0|$ with $|t_\\ast-t_0| > \\delta$. Assumption (ii) addresses this without considering the magnitude of $\\delta$.\n\nAs you mentioned, the quantile loss is $1$-Lipschitz. Let $t_0$ be given. If we choose $\\\\{t_k\\\\}$ that goes to $-\\infty$, then  \n$$\n\\lim_{k \\rightarrow \\infty} \\dfrac{|h(t_0 + t_k) - h(t_0)|}{|t_k|} = 1\n$$\nTherefore, our assumption is satisfied.\n\n\n**Q2.** We apologize for the oversight; in fact, this should be finite. We corrected it.\n\n[1] Blanchet, Jose, Karthyek Murthy, and Viet Anh Nguyen. \"Statistical analysis of Wasserstein distributionally robust estimators.\"", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VTjSqn2kzl", "reviewer": "Reviewer_rhFX", "summary": "This paper uses wasserstein distributionally robust optimization to address individual fairness concerns with causal structures and sensitive attributes.", "questions": "How are the causal relationships determined for the experiments? If one assumes no causal relationship (e.g. i.i.d. formulation), does that impair the performance significantly?", "limitations": "adequately addressed", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The problem is well-motivated and novel to my knowledge. The formulation is clear. The solution is novel.", "weaknesses": "It does not seem easy to scale up this method.\n\nminor issue: the DRO objective in line 142 should be sup_{Q in B_delta(P)} En_Q [l(Z,theta)].", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1.** Thank you for pointing out this issue. We would like to address your question from two different perspectives:\n\n1. **Regularization and Scalability**: In our work, we demonstrate the strong duality theorem (Theorem 1), which shows that the DRO learning problem can be transformed into an empirical risk minimization (ERM) problem with an added regularizer. This transformation removes the need to compute the worst-case loss quantity directly, making the computation **more scalable**. Previous works, such as [1] and [2], support the use of algorithms that incorporate regularizers to improve learning efficiency.\n\n2. **Curse of Dimensionality**: The curse of dimensionality is a significant concern in traditional DRO, where the convergence rate of the distance between the empirical measure and the underlying distribution is affected by the feature space dimension:  \n$$W(\\mathbf{P}\\_{N}, \\mathbf{P}_*) = O(N^{-\\frac{1}{d}})$$\n\nHowever, if we assume that the distribution $\\mathbf{P}_\\ast$ is derived from a SCM, this rate improves to $O(N^{-\\frac{1}{2}})$, effectively breaking the curse of dimensionality. We plan to explore and demonstrate this point in detail in our following work.\n\n**Q1.** In our experiment, we assume that the data is generated by an additive noise model. We first learn the functional structure, and then design our causally fair dissimilarity function. Using the theorems, we can subsequently calculate the worst-case loss quantity.\n\nIf we do not make assumptions about causality (e.g. i.i.d. case) and do not have a protected variable, our results are equivalent to traditional Wasserstein DRO, ensuring that everything functions as expected.\n\n\n[1] Hong TM Chu, Kim-Chuan Toh, and Yangjing Zhang. On regularized square-root regression 391 problems: distributionally robust interpretation and fast computations. Journal of Machine 392 Learning Research, 23(308):1–39, 2022.\n\n[2] Yangjing Zhang, Ning Zhang, Defeng Sun, and Kim-Chuan Toh. An efficient hessian based 530 algorithm for solving large-scale sparse group lasso problems. Mathematical Programming, 531 179:223–263, 2020.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8t271LquSt", "reviewer": "Reviewer_zg8Z", "summary": "The authors propose a novel framework to enhance individual fairness guarantees under a Wasserstein distributionally robust optimization strategy. For such purposes, they employ counterfactuals based on the underlying causal structure of the model at hand. They further propose an alternative with theoretical guarantees for when the causal structure is unknown.", "questions": "Q1 - Which are the main benefits/advantages that are provided by this proposal with respect to existing works in the literature?\n\nQ2 - Does the approach use the conventional Wassterstein-ball based uncertainty set or is the uncertainty set considered the one from equation (16)? Is there any equivalence between them? \n\n Q3 - Recognizing that assuming knowledge of the underlying causal structure for a given classification task is generally unrealistic, the authors propose an alternative approach that operates under more realistic conditions, where only a set of samples/instances is available. They offer guarantees as long as the assumptions outlined in Assumption 2 are met. However, it is unclear how realistic these assumptions are. Are they typically satisfied in real-world applications? Can you provide real examples where these assumptions hold true?\n\nQ4 - Could this method be employed in classification tasks beyond tabular data?\n\nQ5 - What do you mean by ‘We further estimate the regularizer in more general cases and explore the relationship between DRO and classical robust optimization.’? Where is this claim validated in the main text?", "limitations": "The authors discuss the limitations of their method in Section 6.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "S1 - The issue being investigated is of significant importance. \n\nS2 - The introduction and abstract effectively substantiate all the claims made, including the contributions put forth by the authors. These assertions find validation through a thorough description of the methodology employed and the theoretical results provided.\n\nS3 - The paper demonstrates a strong mathematical foundation, supported by numerous theoretical results.\n\nS4 - The paper demonstrates commendable attention to reproducibility by providing detailed information regarding the experimental setup.", "weaknesses": "W1 - The text lacks clarity in several areas. It becomes quite technical at times, and it would benefit from including more examples alongside the technical explanations, particularly in the introduction, to help readers develop a better intuition. Additionally, there are instances where the ideas presented lack cohesion and do not clearly relate to one another, making the overall narrative difficult to follow. Improving the flow and connection between ideas would enhance the readability and comprehension of the text.\n\nW2 - The paper does not cite many significant works on individual fairness and algorithmic fairness in general. For instance, citing [1] would be relevant, as it shares many similar points with the presented work. It would be beneficial to outline both the similarities and differences between them. Additionally, the related works section overlooks a significant body of literature on algorithmic fairness that is based on causality. These works are numerous and, in my opinion, should be acknowledged.\n\nW3 - It is not very clear which are the main advantages and benefits of the proposed approach with respect to existing works in the literature. \n\nW4 - As I understand it, the authors propose modifying only the sensitive attribute while leaving all other attributes unchanged (please correct me if I am wrong). However, this approach can create unrealistic twins. For example, consider a male instance with a height of 1.85m, which is a typical height in a European country. If we change the gender to female without adjusting the height, the resulting instance would be an outlier and not representative of a typical female, making the two instances not equivalent. In the presence of such unrealistic twins, the classifier's decisions regarding these instances may not be informative. This issue arises because bias exists not only in the sensitive attribute but also in the non-sensitive attributes; for example, gender and income can be highly correlated.  Therefore, when we change the sensitive attribute of an instance, in order to create an ‘equivalent’ instance of the other gender, non-sensitive attributes should as well be modified. That is, to ensure the relevance of their analysis, the authors should verify that the twins they consider are realistic and plausible instances. For more insight on this matter, see [1].\n\nW5 - The empirical evaluation of the proposal is poor. It does not include popular benchmark fairness-enhancing interventions, and only a few classification tasks are considered, despite the availability of numerous widely-used datasets in the algorithmic fairness literature. Besides, there is no deep discussion regarding the results. \n\nW6 - The model lacks any discussion or insights regarding its computational complexity or cost.\n\nW7 - The acronym SCM is used before it is defined: it is first used in line 49, but it is defined in line 89.\n\nW8 - (minor) I suggest moving the related works section into a separate section, as it represents a distinct aspect of the work.\n\nW9 - (typos), line 60 (Our → our), line 113 (variables[50] → variables [50]), line 204 (space 9 ??)\n\n[1] De Lara, L., González-Sanz, A., Asher, N., Risser, L., & Loubes, J. M. (2024). Transport-based counterfactual models. Journal of Machine Learning Research, 25(136), 1-59.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "First of all, thank you for your detailed comments. We will address each of them in detail.\n\n**W1.** In the global response, we explained that introducing a new DRO framework requires considering key theorems such as strong duality, closed-form worst-case loss, regularizer estimation, and finite sample guarantees to ensure practical applicability. We aimed to present a minimal and complete framework, making our theoretical results the main part of our work. \n\n**W2.** The paper you mentioned shares only keywords with my work and differs significantly in scope. They aim to provide a new method for computing counterfactual instances by bypassing the conventional three steps (Abduction, Action, and Prediction). They propose using an optimal transport map, suggesting that if $(X, S=s) \\sim \\mathbb{P}\\_s$ and $(X, S=s' \\mid X=x, S=s) \\sim \\mathbb{P}\\_{s'}$ are probability distributions, then under strict conditions, the optimal transport theory can find a map $T: \\mathcal{X} \\to \\mathcal{X}$ such that $T\\_{\\\\#}\\mathbb{P}\\_s = \\mathbb{P\\}\\_{s'}$. This means they use $T(x)$ instead of causally computing counterfactuals. However, since they rely on Euclidean distance to find the optimal map, this approach only works in very special cases, as the authors mentioned in the paper [1]. Therefore, this work does not align with the scope of our work.\n\nAs mentioned in the global response, our work is not limited to algorithmic fairness and might omit some works in that field, similar to how we excluded papers in reinforcement learning and other fields. Our focus is on introducing new distributionally ambiguity set, and we have cited relevant references as comprehensively as possible.\n\n**W3.** In the global response, we discussed the philosophy and advantages of our work compared to existing methods. In addition, Chapter 4.1 outlines the benefits of our approach through comparisons with other works.\n\n**W4.** As mentioned in the background section, when we refer to counterfactuals, we mean computing them through the three steps: Abduction, Action, and Prediction. In line 100, we provide the counterfactual formulation. To clarify further, we detailed Example 1, demonstrating that the counterfactual of the instance $(M,1,1)$ is $(F,0,-2)$. \n\n\n**W5.** As mentioned in the global response, our framework is a general approach to applying DRO with causality and protected variables, applicable in fairness, adverse learning, robust learning, reinforcement learning, transfer learning, GAN, NLP, and more. We used fair learning as an example to demonstrate one application and compared our method only with those having similar assumptions, not all algorithmic fairness methods.\n\n**W6.** In lines 236-240, we discuss the computational aspect of the proposed DRO and reference papers that guarantee fast algorithms for computing our methods. We provide Theorems 2, 3, and 4 to demonstrate the computational efficiency of our approach by solving or estimating the worst-case quantity and incorporating it into the loss function.\n\n**W7-9.** We thank you for your suggestions; we have incorporated the changes and addressed these minor comments in the new version, and the related works may form a distinct chapter.\n\n**Q1.** We provided a detailed response to this in our global response.\n\n**Q2.** The uncertainty set in our method is defined using the Wasserstein distance, incorporating the causally fair dissimilarity function, effectively creating a Wasserstein ball equipped with CFDF. Proposition 2 shows that we can estimate the worst-case loss quantity using the robust optimization method defined in Equation 16, establishing their equivalence.\n\n**Q3.** The first part of Assumption 2 is natural in real applications, as it assumes that the feature space is bounded and the parameter space is bounded and closed.\n\nThe third part is reasonable, as it relates to the statistical properties of the estimator for the cost function or causal model structure. For example, in a linear SCM, we can estimate its functional structure with a convergence rate of $O(N^{-\\frac{1}{2}})$, allowing us to design a metric that satisfies the third assumption.\n\nThe second part of Assumption 2 might seem new but is essentially a Lipschitz condition on perturbing non-sensitive features. Since perturbations in the causal structure are derived from counterfactuals, this is quite natural. For example, consider a linear SCM with reduced-form mapping $M$, where $X = MU$ and all sensitive attributes are parents. Here, $CF_0(v, \\Delta) = v + M\\Delta$. Given a loss function $\\ell(v, y, \\theta) = h(\\theta^T v - y)$ where $h$ is Lipschitz, assuming a norm $\\ell_p$ in the exogenous space leads to $d(v, CF_0(v, \\Delta)) = \\|\\Delta\\|_q$. This makes the Lipschitz condition:\n\n$$\n\\vert \\ell(v, y, \\theta) - \\ell(CF_0(v, \\Delta), y, \\theta) \\vert = \\vert h(\\theta^T v - y) - h(\\theta^T v - y + \\theta^T M \\Delta) \\vert \\leq L \\|\\theta^T M\\|_p \\|\\Delta\\|_q\n$$\n\nThis naturally satisfies condition 2.\n\n**Q4.** Our method can be applied in fields where the Wasserstein distance or optimal transport is used, allowing for the incorporation of causality and protection of feature variables. In neural networks, especially GAN models, it enhances sensivity of model respct to causal structure. In signal processing, reinforcement learning, and NLP, it introduces causality and fairness into the Wasserstein distance.\n\n**Q5.** This statement refers to Theorem 4, which provides a first-order estimation of the worst-case loss quantity, described as a regularizer in our DRO framework. Proposition 2 further establishes the relationship between our proposed DRO method and conventional robust optimization techniques that typically involve adversarial perturbations.\n\nIn conclusion, we hope our responses are comprehensive enough to capture your interest in this work.\n\n[1] De Lara et al. (2024). Transport-based counterfactual models.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank the reviewers for their valuable feedback and constructive comments. We are honored to have received your attention.

**Motivation:** Distributionally Robust Optimization (DRO) is a data-driven framework addressing out-of-sample challenges, such as distribution overfitting or shifts, using an adversarial approach. It defines a distributional ambiguity set (DAS) around the estimated true probability based on empirical measures, ensuring the true distribution lies within this set.

In conventional Wasserstein DRO, the DAS includes all probability distributions within a certain Wasserstein distance, constructed by a metric on space (e.g., $\ell_p$ norm) from the empirical distribution, $B^W_{\delta}(\mathbb{P}) = \\{\mathbb{Q} \in \mathcal{P}(\mathcal{X}): W(\mathbb{P}, \mathbb{Q}) \leq \delta \\}$. This works well when data lacks specific structures, but when data or distribution shifts follow structures like temporal patterns or causal relationships, the Wasserstein DAS must be pruned to exclude unrealistic scenarios; otherwise, models become overly conservative and lose accuracy.

For example, consider a model predicting income based on gender, age, and education. In the sample data, the average age is 25, and the educational level is 3 (Upper is highly educated). Using a simple $\ell_p$ cost in Wasserstein distance, a population with an average age of 20 and educational level of 3.5 is treated the same as another with an age of 30 and level of 2.5 by the conventional Wasserstein DAS. However, education depends on age, making the first scenario unrealistic.

**Contribution:** To address this, we derive the transportation cost function from the estimated  causal structure instead of conventional norms, preventing impossible scenarios. Figure 1 (uploaded PDF) shows that a causal cost function results in a DAS that includes the true underlying probability with fewer unrealistic scenarios.

If there is a protected variable, the cost function must capture the proximity between an instance $v$ and its counterfactual $v_a$ from the causal structure, not simply by changing the labels of the instances. In Section 3, especially in Example 1, we demonstrate that $\ell_p$ norms cannot capture counterfactual proximity. We propose the causally fair dissimilarity function (CFDF) to address this.

Since the CFDF needs to capture the similarity between instances and their counterfactuals, counterfactuals must be identifiable from sample data, which requires the SCM to be counterfactually identifiable. We chose the additive noise model (ANM) to avoid added mathematical complexity. While our results apply to bijective generative mechanisms [1], a broad class of SCMs that are counterfactually identifiable. ANM is often preferred over general SCMs due to its simplicity, interpretability, and effective handling of noise. This makes it ideal for fields like statistics, causal inference, signal processing, image processing, economics, and social science, where additive noise is prevalent. 

The assumptions behind CFDF are intuitive. Since the variables in the exogenous space are mutually independent (by assumption), assume each has its own cost function, which can be combined through product topology. This allows the pushforward of the CFDF to have a simpler form in the exogenous space.
 Unfortunately, capturing causality in CFDF means it is not a true metric because it lacks the positivity property, which means $d(v,v_a) = 0 \nRightarrow v = v_a$. Most optimal transport facts assume a metric-based cost function. Using CFDF instead of a metric posed significant challenges and added theoretical 
 complexity to our work.

After introducing CFDF, we prove the Strong Duality Theorem to demonstrate our proposed DAS's real-world applicability. This theorem shows DRO problem converts into a tractable, computational efficient form. Theorems 2 and 3 showcases the effectiveness of our approach. For linear SCMs, many popular models (Examples 2 and 3) provide closed-form solutions for worst-case loss, which enables fast learning algorithms by eliminating worst-case step computation.

To ensure compatibility with complex nonlinear SCMs and neural networks, we present a first-order regularizer estimation in Theorem 4. We address the relationship between DRO and classical robust optimization in Proposition 2.

A key challenge is that CFDF is based on SCM's functional structure, which must be estimated in real applications. Theorem 5 shows how causally fair DRO performs well with an estimated structure, that demonstrates a finite sample theorem. Our results rely on Assumption 2, which is common in such theorems. Part (ii) of Assumption 2 is new, requiring the loss function to have a Lipschitz property for non-sensitive attribute perturbations. Since in SCM, perturbation is obtained by counterfactual, so this property is expressed like this.

Regarding the numerical experiment, we agree that this section could include more examples and datasets. Our framework is a general approach for DRO with causality and protected variables, applicable in areas like Fairness, Adversarial Learning, Reinforcement Learning, and NLP. In this paper, we applied it to fair learning to demonstrate one application given the limited space and availability of appropriate dataset/use cases.

We believe that our method efficiently captures the true underlying probability without unrealistic DAS scenarios, as shown in our results. Future work will need to demonstrate additional properties, such as breaking the curse of dimensionality in traditional DRO problems.

We appreciate your comments on notation and typesetting and have incorporated them into the revised version. If you need further clarification, we would be happy to discuss this during the review period. Next, we will address each reviewer's weaknesses and questions in more detail.

[1] Nasr-Esfahany et al. [2023], Counterfactual identifiability of bijective causal models.

Meta-review: The paper introduces Causally Fair Distributionally Robust Optimization (CDRO) framework to address individual fairness in machine learning using Wasserstein Distributionally Robust Optimization (DRO). The framework integrates causal modeling with DRO to handle fairness concerns involving sensitive attributes. It presents a dual formulation of DRO that simplifies the problem into a more tractable form, estimates regularizers for linear and non-linear Structural Causal Models (SCMs), and provides finite sample error bounds. Standard fairness evaluation datasets are used in empirical evaluations to demonstrate the effectiveness of the proposed method.

The reviewers found the combination of causal modeling, DRO, and rigorous optimization frameworks to address individual fairness novel. They appreciate the novelty in theory and relevance in practice. In general, the paper is found beneficial to the community with sufficient contributions. As for feedback for improvement, the reviewer mentioned that the paper could benefit from clearer explanations and more examples to improve readability. They also noted that the literature review part could expand significantly, and the discussions with existing works can further improve. There were additional comments for improving the readability and presentation of the paper (please see individual reviews).
Decision: Accept (poster)