Bridging Causality, Individual Fairness, and Adversarial Robustness in the Absence of Structural Causal Model
Authors: Ahmad Reza Ehyaei
Golnoosh Farnadi
Samira Samadi
Venue: Accepted by TMLR ()

Abstract: Despite the essential need for comprehensive considerations in responsible AI, factors such as robustness, fairness, and causality are often studied in isolation. Adversarial perturbation, used to identify vulnerabilities in models, and individual fairness, aiming for equitable treatment of similar individuals, despite initial differences, both depend on metrics to generate comparable input data instances. Previous attempts to define such joint metrics often lack general assumptions about data and were unable to reflect counterfactual proximity. To address this, our paper introduces a \emph{causal fair metric} formulated based on causal structures encompassing sensitive attributes and protected causal perturbation. To enhance the practicality of our metric, we propose metric learning as a method for metric estimation and deployment in real-world problems in the absence of structural causal models. We also demonstrate the applications of the causal fair metric in classifiers. Empirical evaluation of real-world and synthetic datasets illustrates the effectiveness of our proposed metric in achieving an accurate classifier with fairness, resilience to adversarial perturbations, and a nuanced understanding of causal relationships.

Published in Transactions on Machine Learning Research (01/2024)
Bridging Causality, Individual Fairness, and Adversarial Ro- bustness in the Absence of Structural Causal Model
Ahmad-Reza Ehyaei
ahmad shaeddddhheeSttcchngge...p..
Mar Planck Institute for Intelligent Systems Tübingen, AT Center, Tübingen, Germany
Golnoosh Farnadi
farnadig quecbec
Mila Québec AI Institule, Université de Montréal McGill University, Montréal, Conada
Samira Samadi
ssamadi@@uubingge..mpp.
Max Planck Institute for Intelligent Systems Tübingen Al Center, Tübhnngn, Germany
Reviewed on OpenReview: net/ forum? ? id= 2nRcWy3RLM
https: mas lhost. tuebangen. mpg. BeddSeeeeeett//BBBABBB.. http:d/rretrebbBBB///aeeeereeeee openreview.
Abstract
Despite the essent ial need for comprehensive considerations in responsible AI, factors such as robustness, fairness, and causality are often studied in isolation. Adversarial perturba- tion, used identify vulnerabilities models, and individual fairness, aiming for equitable treatment of similar individuals, despite initial differences, both depend on metries to gen- erate comparable input data instances. Previous attempts to define such joint metrics often lack general assumptions about data and were unable to reflect counterfactual proximity. To address this, our paper introduces a causal fair metric formulated based on causal structures encompassing sensilive altribules and protected causal perturbation. To enhance the appli- cability of our metric, we introduce metric learning as a method for estimating and deploying the metric in real-world scenarios, even when structural causal models are unavailable or when causal structures cannot be identified. We also demonstrate the applications of the causal fair metric in classifiers. Empirical evaluation of real-world and synthetic datasets illustrates the effectiveness of our proposed metric in achieving an accurate classifier with fairness, resilience to adversarial perturbations, and nuanced understanding of causal re- lationships.
1 Introduction
While fairness, robustness, and causality are central to responsible AI, they are often studied in isolation despite the need for systems to address them all comprehensively. In this work, however, we demonstrate that individual fairness and adversarial robustness, both of which rely on metrics to produce comparable data instances, are interconneeted and can be learned simultancously. On one hand, the concept of individual fairness, defined by [Dwork et. al.] 2012 focuses on the fair treatment of similar individuals to prevent discrimination based on individual characteristics. The definition of individual fairness, whether through the Lipschitz formulation (Dwork et al.] 2012) or the c c- 8 method (John et al., 2020), requires the creation and assessment of fair metric. These metrics are essential quantitative tools for evaluating whether algorithms adhere to the principles of individual fairness.
On the other hand, Adversarial perturbation, as outlined by [Goodfellow et al. 2014) and (Madry et al. 2017), involves the purposeful manipulation input data to uncover machine learning model vulnerabilities assess robustness. This concept is closely related to metrics that measure the impact of changes in input

---

Published in Transactions on Machine Learning Research (01/2024)
data on model performance. Often, it involves using distance metrics to quantify the differences between original and perturbed inputs.
In this work, we show that metrics for both individual fairness and adversarial robustness can be simul- tancously defined through the lens of causality to better reflects the true characteristics of the underlying data. When dealing with a causal structure underlying data, traditional metrics like the Fuclidean norm, fail to account for causal relationships, as noted by Kilbertus et al. (Kilbertus et al. 2017). This limitation becomes especially evident when aiming for fair treatment for sensitive attributes. In such scenarios, the most suitable metric would be one that generates minimal values for counterfactual instances associated with each data point.
Previous research frequently simplified counterfactual calculations by only modifying levels of sensitive fea- tures. In the work by Dominguez et al. (Dominguez- Olmeeo et al., 2022), adversarial perturbations were integrated into structural causal models (SCM), with a primary focus on continuous foatures, which may have neglected aspects of fairness. On the other hand, Ehyaei fair metric based on functional causal structure, tailored to safeguard sensitive attributes. However, their approach is limited to specific examples and lacks support from well-established theoretical foundation.
al. (Ehyaei et al.] 2023bb developed a
In this study, we aim to bridge the gap between fair metrics in cansal structures and sensitive attributes. We begin by identifying suitable properties that align with our objectives and subsequently derive this metric from observational data without knowing the causal structure assumption. In Section 4 we first introduce a definition for a causal fair metric, effectively addressing both causality and the protection of sensitive at- tributes. Next, in Section 5 we use the causal fair metric to create a protected causal perturbation, enhancing adversarial perturbation with causality and sensitivity considerations. We also examine its geometric prop- erties and attributes. Constructing a causal fair metric lypically requires knowledge of SCM, which is often unavailable in many real-world applications. To overcome this limitation, we propose to derive the metric from data. In Section D we illustrate that relying solely on observational or interventional data is insufficient for learning the causal fair metrie. To address the absence SCMs, an alternative approach involves metric learning using tagged distance data. These tags indicate proximily values or labels indicating data point closeness. By discussing the requirements of other methods, we focus on deep metric learning due to its compatibility with the structure of causal fair metrics. To enhance practicality, triplet deep metric learning scenarios. This approach demonstrates that the causal fair metric is learnable even in the absence of known causal structure or when causal detection is impossible in certain models. Finally, in Section [ through experiments on both synthetic and real-world datasets, we empirically verify our theoretical findings. Our results illustrate that knowing the structure of the causal fair metric amplifies learning performance within deep metric learning scenarios. Furthermore, our empirical analysis reveals that basel-based metric approaches strike a practical balance between applicability and accuracy and are more aligned with the concept of protected causal perturbation. To demonstrate the effectiveness of our empirical causal fair metric into fairness learning method for classifiers.
employ contrastive and
framework, we incorporate Unlike existing approaches that require knowledge of the causal structure, ours operates without SCMs, yet yields notable enhancements in fairness while preserving accuracy. In summary, our main contributions are:
Causal Fair Metric (44 : We present a causal fair metric that incorporates both causal consid- crations and the protection of sensitive attributes. In addition, we demonstrate how our proposod causal fair metric can be embedded in exogenous space.
We use our proposed causal fair metric to generate ad- versarial perturbation within causal structures while addressing fairness concerns.
Protected Causal Perturbation (55):
Causal Fair Metric Learning (66): Theoretically, we show that, without SCM assumptions,
learning a fair metric from observational or interventional distributions is not guaranteed. We introduce a learning am designed to extract causal fair metric from empirical data, with a focus on both causality and fairness considerations.
Fairness, Robustness and Causality Classifier (S 7 To demonstrate the effectiveness of our approach, we apply to a classification problem and introduce ECAPIFY, which combines metric learning and fair learning without relying on a known SCM.
2

---

Published in Transactions on Machine Learning Research (01/2024)
2
Related Work
In this section, we explore previous research defining metric learning, whether for adversarial perturbation or individual fairness. The most relevant study to ours is conducted by Ehyaci et al. (Ehyaci et al.) 2023b which worked on constructing fair metric the presence of causal structures and sensitive altribules. However, unlike ours, their metric is limited to specific family of dissimilarity functions and lacks a comprehensive charactorization of its properties. In Mukherjee (Mukherjee et al. 2020), the authors attempted fair metric learning, but their method didn't heavily rely on causal structure. They assumed embedding into space where sensitive attributes form linear subspace but didn't clarify its connection to SCM. Moreover, they assumed knowledge of the embedding map during metric learning. In llvento (llvento) 2020), submetrics were developed for learning metrics for individual fairness using human judgments. Under specific assump- tions about point distribution and representative point selection, these submetrics maintained accuracy relative to the truc metric. However, this work didn't address the impact of sensitive attributes, which often compromise metric properties. Spectral-based metric estimation methods, akin to those in Zhang (Zhang) et al. 2016) and Olson (Olson 2022), often require specific embedding kernel forms observations of all pairwisc distances d(u;, v;) for guaranteed metric convergence. Fair representation learning (Zemel et al. 2013 McNamaaa et al. 2017 Ruoss et al.) 2020 aims to map indiviuals to prototypes. Their primary aim frequently involves eliminating protected attributes while preserving performance-relevant information during the training phase. Another non-linea metric estimator is the tree-based approach, proposed by Demirovic (Demirović & Stuckey 2021). They introduced a novel algorithm using bi-objective optimization to compute decision trees that provably optimal for non-linear metries. Online learning algorithms, in (Bechavod et al., 2020 Gillen et al.) 2018), enstree finite number of fairness constraint violations and bounded regret, relying some metric- metric-based assumptions. Various spectral, probabilistie, and deep metric
learning methods are discussed in (Ghojogh et al., 2022 2023 Suárez et al. 2021 Francis & Raimond 2021).
To the best of our knowledge, none of the existing algorithms address the integration of causal structure and sensitive attributes in metric learning.
3 Background
Structural Causal Model. tuple M = (G.V, V. U.F.F.F... Pu) Pearl 2009 where:
A SCM for a set of n random variables V [V+) represented by the
The set FF = [V, == (VPP())) U contains structural equations, with each equation f. denoting the causal connection between the endogenous variable V,, its direct causal parents Vpact) from 9, and an exogenous variable U {U.)__ signifying unobservable background influences. In this work, we suppose G is directed acyelic graph.
The distribution Pu of exogenous noise variables factorizes, Pu IT= Pu,, due to the assumption of causal sufficiency.
Under acyclicity, each instance U of the exogenous space U uniquely determined by E V with the reduced-form mapping g u - V. where g is obtained by iteratively substituting the structural equations F following the causal graph's topological order G. The SCM entails a unique joint distribution Px over the endogenous variables through the reduced-form mapping, Pv(V = =) = = Pu = ((((UU--()))) g''(t)) where g is the preimage of g-
Causal Identifiability. Discovering true causal connections among variables solely from observational data typically necessitales additional assumptions about. the structural functions F. One identifiable family of SCMs is the additive noise model (ANM) (Hoyer et al. 2009), represented by V f(V) + U. In
ANMs, obtaining the relationship from to v is straightforward when considering I as the identity function (I(u) v), then is obtained by g (I f)-1. Post-nonlinear models (Zhang & Hyvarinen 2012 and location-scale noise models [Immer et al. 2023 are other identifiable SCM families.
Interventions. SCMs facilitate modeling and assessing the impact of external manipulation on the system represented by the intervention (Peters al. 2017). Two main intervention types are hard interventions
3

---

Published in Transactions on Machine Learning Research (01/2024)
and soft interventions. In hard interventions (expressed () subset I S (1,...,,) ,.) of features Vz is forcibly fixed to constant 0 E [R!2] by excluding relevant parts of the structural equations:
pdo(Vz:==) V 0: Vi = (((P)))) U.) otherwise
ife
Hard interventions disrupt the causal connections between affected variables and their ancestral com- ponents in the causal graph, whereas soft interventions maintain all causal relationships while adjust- ing the structural equation functions. For example, addilive (shift) intervention (Eberhardt & Scheines 2007), denoted as Mdo(Vz+=)), modify features V using perturbation vector d IR" with equations (Vi fi (VPa(i), U.) + )
Counterfactuals. Counterfactual is hypothetical scenario that represents what would have happened if certain interventions or changes were applied to the variables in the SCM. The counterfactual outcome CF 0) for a specific variable Vz under the hard intervention do = 0) can be computed using the mod- ified structural equations as g² 0 (w)), where gd represents the altered reduced-form mapping Mdo(V2:=9) after the intervention.
Sensitive Attribute. A sensitive attribute, like race, holds ethical or legal significance in recision-makingg such in hiring, lending, or criminal justice, determining equitable treatment outcomes for individuals groups. Let S E (V1...., Vn represent sensitive altribule with domain 8 (discrete or continuous). For each instance v V, the set of counterfactual twins regarding the sensitive feature S is obtained by v= (Ü, = s) s S}.
Individual Fairness. Individual fairness, as introduced by Dwork et al. 2012 ensures equitable treatment for individuals with comparable predefined metric similarities. Two formulations, including the Lipschitz mapping-base based formulation (Dwork et al. 2012
(y(h(v),h(m)) S L dy(u,v') Wo.
and the e-ô formulation (John et al. 2020
Ve,u' EV dy(v,u)) 58
dy(b(o), Se
have been proposed. Where, dx and dy are metries for the input and output spaces, respectively, with h the classifier and L E Rt. The essence of the definition is centered around the foir metric dx, which measures individual similarity based on relevant attributes.
Counterfactual fairness, as introduced by Kusner et al. (Kusner et al. 2017), defines fairness using causal models. This approach compares an individual's actual outcomes with hypothetical outcomes a scenario where sensitive fcatures differ. classifier is doemed counterfactually fair if satisfics the following condition, h(ö,) h(üz) Vs, s' S.
4 Fair Metric
The fair metric, often used in previous studies, becomes ambiguous when applied to problems involving causal structures and sensitive attributes (Ghojogh al.) 2022). To clarify the necessary properties of fair metric in these contexts, consider the following example.
Example 4.1 Consider two SCMs, M and M', descrübing gender (G), income (T), and education (E). M models these variables as independent, while M' specifies a linear causal relationship:
G = Uc.
G Uc.
Ue B(0.5)
M
E = Ug
M'
E = G U
, u
Ug~ N(O,1)
I U,
I:= 2GG222+1
U N(0,1

---

Published in Transactions on Machine Learning Research (01/2024)
V3
Ba(o)
Va B (x)
L. a (é. a 5)
Sae V3 (g)
se) i1 Waa - Telel A
woae entel a
Va
V.
V1 (a)
(b) V1
EV le- - s
(V/ W: (((()) p(0) S A}
(c) EVEV: [[ex(0) - px( S A)
Figure illustrates the progression from a basic perturbation to a protected causal perturbation ball. Consider a simple linear SCM with Euclidean norm both exogenous and endogenous spaces. (a) a perturhation hall that does not account for causality or the protection of sensitive features, (b) a perturbation ball that includes causality but assumes the absence of sensitive features, and (c) The counterfactual perturbation space, created using a counterfactual space based on a senaitive attribute, can be visualized as the axis L of a cylinder. Surrounding ellipses represent a causal perturbation ball B encompassing perturbations of non-sensitive with radii A.
Here, UG represents the gender distribulion, while Ug and U, are intrinsic talents for education and in- come, respectively- Consider the d(V, V') = -E | I-I' Iı-norm on non-sensitive altribules to compare individuals. If two individuals have less than 0.2 wnit difference, they are deemed similar.. For indi- vidual with data e = 1M)) (M = 1 for Male), a perturbation in education by 0.1 units (A = (0..1.0). in M resulls in A) = (M, 1,1,), which is similar to v. In M', CF( A) = (M, 2.2) gives a distance of 0.3, indicating dissimilarity. To protect against gender bias, individuals with the same intrinsic characteristics but different genders should behave similarly. This is modeled by a counterfactual change in gender. In M,, CF(v, F) 1.,1,)), so v and its twin are similar because d(u,üp) = 0. However, in M', CF(U, F)= (F,0, 11,, resulting in d(u,, ūr) 4, which indicates dissimälarity.
The above example shows that combining a causal structure with sensitive attributes requires a dissimilarity function that remains zero for counterfactull twins and is stable against small changes in non-sensitive attributes. Creating counterfactual twins is straightforward, but defining small changes for non-sensitive features nceds further exploration. Following Dominguez-Olmeda et al., 2022 and Ehyaci et al.] 2023a additive interventions are used as perturbations in additive noise models. Now we are ready for a proper definition a causal fair metric.
Definition 4.2 (Causal Fair Metric) Let : VXV + Rzo represent metric defined on the feature space V, generated by SCM M. Let S denote a sensitive attribute, and I represent the index set of sensitive features within the SCM. The metric is called a cousal fair metric if it adheres to the following properties:
(1) For all u E V and s E S, the metric is zero only for twin pairs, i.e., d(v, ((). =0 -.
(ii) For every V and any 8 O, there exists e such that for any sufficiently small intervention (|A|I S e) on the non-sensitive attributes, the distance d(v, CF(e, A)) remains less than 8.
The first property highlights that a fair metric maintains counterfactual fairness, meaning the distance between an instance and its counterfactual zero. The second property ensures that. the intuition of similarity in the exogenous space inherited by the fcature space, allowing us to set thresholds to define similarity effectively.
Constructing metric on M is challenging because it must account for causal relationships in its dissimilarity function. By applying the causal sufficiency principle, which ensures feature independence in the exogenous space, we can define individual similarity functions for each feature's noise. These individual metrics enable us to construct a holistic metric in the noise space, which is then extended to the feature space through We assume g invertible, as required for identifiability and counterfactual identifiability. Bijective generation mechanisms (BGMs), including additive noise models, satisfy these conditions. However, this approach is inadequate because d must be defined for
a fush-forward metric, i.c., dy(w, r²) = (v).
5

---

Published in Transactions on Machine Learning Research (01/2024)
every counterfactual of v. Generally, we have Range(M) Uses (ange() Therefore, need to consider space that encompasses all counterfactual values.
Definition 4.3 (Semi-latent Space (Ehyaei et al,, 2023b)) Consider M with sensitive attributes in- dered by I. The converted SCM denoted as M is derived from M by removing the causal effects of porents of sensilive altribules and replacing their erogenous variables with endogenous ones. The structural equalions for MS are as follows:
V;
iET fa(Vp)))) + +U; i
V
The erogenous space corresponding to M denoted by 2, includes the sensitive attributes and the non- sensitive parts of the erogenous variables of M. This space called the semi-latent space, is construcled as == x Uxx where Ux is the non-sensitive port of the exogenous space. There map bijective map Q from feature space to the semi-latent space by the below formulation:
(p(() = (6 i I
02
iEi
97
= [ui ( (u)) ui ig
(1)
where is the reduced-form mapping of M. The metric construction in the semi-latent space is simpler compared to the feature space due to the independence its components. This independence arises from the sufficiency assumption for U, and the intervention assumption for Vz in the SCM. Let (Q,, de) represent the metric space for the semi-latent space. We can define the dissimilarity function for all Q: using a product metric, similar to the Euclidean example i.e., d(x, y) = V/=, (x;. We aim to ascertain the specific formulations of the causal fair metric, delineated in Definition 4.2
Proposition 4.4 Let d: x - IR be a causal faär metrie, then d can be wrillen as form:
=/v dx)x=))(x())))xxxx
(2)
where px(()= = Px((p(v)), yp is the mapping from feature space to semi-lalent space, Px is projection the non-sensitive subspace of erogenous space, and dx represents the metric defined on the non-sensítive subspace Ux, which exhibits continuity along its diagonal with each of its components.
By aiding the proposition, when the semi-latent space metric defined by an inner product, the metric takes the well-known form of a kernelized Mahalanobis distance:
d(u, d') ((p(v) -0')))(()).))))))))))>))) E(p(v) p(v')))
(3)
where Σ the projection matrix on non-sensitive exogenous space.
5 Protected Causal Perturbation
An adversarial perturbation ball, a fundamental concept in machine learning robustness, defines a region in the input space within which data variations do not alter the model's predicted category.
This concept evaluates the model's sensitivity to input alterations, especially under adversarial attacks designed to mislead it. Metries are crucial in quantifying perturbations by gauging the distance between original and altered data. In this section, we extend this concept by applying fair causal metrics to define causal perturbations.
Definition 5.1 (Protected Causal Perturbation) Consider an SCM M that includes sensitive at- tributes, and let represent its causal fair metric. We define the protected causal perturbation (PCP) ball with radius A for instance as follows:
BBTF())= (v' E EV: d(),
(4)
where A is non-negalive real reumber.

---

Published in Transactions on Machine Learning Research (01/2024)
We will examine how the shape of the perturbation ball changes when we add causal structures and protect sensitive features. Fig. [ shows how the counterfactual ball evolves with these aspects. We define closed ball BA in space X' as (x) = fa' E E xx)) I A}. Equation 4 gives a simple formula: (v) = px (Bg (f(())). But since 9x is not bijective (because projection function Px is not bijective) when sensitive Teatures are present, B and B are not isomorphic. Define B as the part BACP that only includes the causal structure, leaving out the sensitive protected attributes, i.c., Bg (v) {o' E VV: Px(((()) Pt(((d') px((')) E (xx(())) We see that 0 isomorphic to B Thus, BReP is formed by combining causal balls around each counterfactual instances of
Proposition 5.2 Let (v) represent the PCP ball around the instance with radius of A. It can be decomposed as:
U (T.),,
(5)
SES
where S represents the level set of sensitive features (which may be continuous or discrete). B ezhibils invariance under twins, meaning that for all s E S, we have
The PCP definition, along with the causal fair metric property, captures the counterfactual proximity defi- nition. The subsequent lemma demonstrates that a PCP with a diameter of 0 represents the set of twins.
Proposition 5.3 Let S denole the protected features, and let d be the causal fair metric. The set of coun- terfactual twins corresponds the PCP with a zero radius i.e,, V lima- (p))(v).
6 Causal Fair Metric Learning
From Eq. 2 ereate fair metric using structural equations and metric for non-sensitive exogenous variables. This involves deriving the metric from data and dealing with unknowns such as sensitive features, the embedding function sp, and the metric for non-sensitive exogenous features. Assuming we know the sensitive features and have dissimilarity functions for each exogenous component from domain experts. With these assumptions, understanding the functional structures allows us to construct o and, in turn, develop a causal fair metric. This metric is fundamentally linked to counterfactuals, raising the critical question of whether it's possible to estimate counterfactuals from observational data. The below example that is adapted from Peters et al. 2017 $ 6.19 investigates the possibility of this idea.
Example 6.1 Let MA and MB be two SCM with below structural equations respectively:
V1 U1,
V1 U1,
1/2 == Vi(1 U2),
U2),
V/2 = Vi - U2),
MA =
MB
V/3 == Ivixvs (UUs>20V+++
V/3 U1 (Liis>0V/ VV++
Vz)= + Iv:
TVg=oV2) Tvg=v, (N - -U).
where, U1 and U2 have a Bernoulli distribution with a 0.5 probability, and Uz has a uniform distribution spanning from to a constant value N. Consider the instance v = 01, with V1 denoted as the sensitive feature. The counterfactuals for with respect MA and MB are (0, 0, 0) and (0,,,N), respectively.
Both SCMs have identical causal graphs, observational distributions, and intervention distributions for all possible interventions. Thus, randomized trials or observational data can distinguish between MA and MB. Therefore, for counterfactual statements, additional assumptions are essential. Example 6.1 establishes the following proposition.
Proposition 6.2 (Metric Estimation Not Guaranteed) If the set of descendants of intervened vari- ables is non-empty, estimating a causal fair metric, from observational data or with a causal graph, necessi- tates knowledge of the true structural equations, irrespective of data quantity or type.
Proposition 6.2 states that, in the general case where the SCM is unknown, data-driven metric learning becomes infeasible if we first attempt to estimate counterfactuals from the available sample data even
7

---

Proposition 6.3 ensures that deep metric learning, the estimator converges to the true underlying met- ric with a convergence rate of ( forming the core theoretical foundation of each proposed learning algorithm. Moreover, the proposition establishes that deep metric learning can effectively distinguish em- Published in Transactions on Machine Learning Research (01/2024)
when the causal graph can be accurately inferred. In other words, this implies that for certain SCMs, such as linear SCMs with Gaussian noise, learning the causal graph solely from sample data is impossible. Consequently, deriving a causally fair metric from the data is also unattainable.
As SCM knowledge is often clusive in practice, an alternative estimating the causal-fair metric directly from data labeled with distances. Metric learning methods vary, including spectral, probabilistic, and deep learning. Spectral techniques use eigenvalue decomposition to represent data in lower-dimensional space, while probabilistic methods infer a low-dimensional latent variable underlying the high-dimensional data. Both spectral and probabilistic metric learning techniques employ the generalized Mahalanobis distance, denoted as Eq. B. with predetermined kernel such as the Gaussian kernel or a kernel that is learned, aiming to optimize the dissimilarity matrix (Ghojogh et al., 2022
Conversely, deep metric learning utilizes neural networks to determine the embedding function. The network aims to roduce distances between similar points while increasing distances between dissimilar ones. This approach aligns with Proposition 4.4 which asserts the existence of an embedding fx R" R The causal fair metric is defined as di( w) dx(px(()) (x)()) with & indicating the dimension of the non- sensitive exogenous space. This leads to three main insights: the dimensionality of the embedding space, the guarantee of independence from coordinates in this space, and understandings about dx. These insights are crucial for creating specific deep-learning techniques for causal fair metrie learning.
In designing a neural network, we focus on ford-forward neural networks with a depth of L 1. These networks are characterized by their layer widths, denoted as d1....,d4 where do represents the input size, and dp k represents the output size. Each layer has an element-wise activation function 0,, which operates Rd:-1 and maps to Rd: The transformation process of the network is expressed as follows:
(w) = (WL x O L-1 x
(W)x)))....
(6)
Consequently, the causal fair metric can be expressed as d(v, w) = dx (v), Pw (w)), where W = (W....., Wz) denotes a tuple of matrices. Each matrix W, E R4xx, 1, and dx is a known metric. This matrix tuple family is symbolized as W, thus allowing the representation of the family of non-linear functions as & (P W W).
To assess how the causal fair metric affects deep learning, we need specific measures to track progress in various scenarios. Kozdoba et al. (Kozdoba & Mannor 2021 p adopted the approach from Bartlett et al. (Barllttt et al. 2017) by metric learning principles. Their results are based on the following norm definitions: The spectral norm of a matrix W E Raxt is denoted as ||W]] Additionally, ||W]]2, is introduced as the sum of the lg norms of each column in matrix W, where W., represents the i-th column of the matrix.
Proposition 6.3 (Kozdoba (Kozdoba & Mannor, 2021)) Consider a feed-forward network with L lay- ers described in Eq. 0 Assurving that aclivation functions Pt are Xi- --iápschilz, and the feature space V is bounded with ||o||2 I B for all v E V, the Rademacher complexity of db for family of matrix tuples W is bounded as follows:
2
n
)
W
R(D) 5
B² Vn
II
Σ
21
|
Here, [|W|] represents the supremum norm over W in W for W,, and [|Wall2,1 is the supremum over W in W for W, with respect to the f2,1 norm.
beddings regardless of dimensionality or the specific metric used, as dimensions are not explicitly considered in Eq. 7 However, numerical analysis ($S) shows that incorporating causal fair metric assumptions improves estimation accuracy compared to general metric learning methods.
8

---

Published in Transactions on Machine Learning Research (01/2024)
We conclude this chapter by highlighting that our approach to metric learning not only provides frame- work for learning causally fair metrics but also enables metric learning even when causal graph inference is infeasible.
7 Causality-aware Fair Adversarial Learning
Fair adversarial learning seeks to predict the target variable accurately while maintaining fairness concerning sensitive attributes. Using the set of observations D = ((),)))) it entails a min-max optimization problem where the model minimizes classification error and maximizes adversarial loss around each instance Vz.
A key insight from Prop. 6.2 is that fair adversarial learning using just observational data D is unattainable in causal structures, as inferring suitable metric for assessing counterfactuals is not possible. Practically, A is set by varying values within a perturbation ball to include samples deemed similar. Fssentially, people are learning the metric based on their experience. At best, this method estimates the upper bound of the appropriate A, highlighting the significance of metric learning.
To initiate fair adversarial learning, the first step is to metric estimation for constructing BReP, demonstrated in the subsequent min-max adversarial learning framework:
min E (na))Pol max E(hu(w),)), WEB (v)
where PD empirical distribution corresponding observation set and hy is parametric model with parameter vb In Ehyaei et al. (Ehyaei al. (003b)), the limitations of gradient descent in adversarial learning discussed, and superior, locally lincar method named CAPIFY is proposed for contexts with causal structures and sensitive attributes. This approach includes integrating regularizers into the loss function, as detailed below:
Ra = max ((h(G.)))) #2 (VV,(((F(,,))))))) (( + As * (2(2,))
#ES
where μ4 regularize coefficients and the term (v. y) is obtained by:
max (E(CF(), () - g( - 8'V (v,
(EBPCF(A)
The first part of the regularizer ensures counterfactual fairness by measuring the maximum loss of the instance label and its twins. The second and third parts assess the adversarial robustness of classifier h concerning continuous features around each twin w.r.t. causal structure. Calculating these two terms requires knowledge of the causal functional structure. Without knowing the SCM, twins of an instance can be estimated using the causal fair metric introduced in Section 6 As the causal structure unknown, by using twins, the second and third terms are estimated by max.es IAA V. v)1,, + fi (i,, where (v = (8(v A, y) (v.y) - A V. v).
By using the last equations, we introduce the ECAPIFY method, which operates without SCM knowledge and relies solely on metric learning. To train a classifier with ECAPIFY, the following regularizer added to the learning loss function.
RA (v, v) = max ((h(ū.).)) + * V.l... v)1 + #ES
(8.
8 Numerical Experiments
In this section, we empirically validate the metric learning method presented Section We compare our method, which incorporates causal structure and sensitive information, standard deep metric learning. We use deep learning to estimate the embedding function, and Siamese metric learning CCiicoo 2021
9

---

Published in Transactions on Machine Learning Research (01/2024)
the bascline. Siamese networks use neural network to map inputs Vi and va into embedding space, minimizing ((((v)), for similar points and maximizing it for dissimilar ones. Contrastive loss is used:
L y + -= max m -0 d)²,
where d = [v((1 - - y indicates similarity, and m is the margin. Simulations are divided into three sccnarios distance-based, label-based, and triplct-based (further details in 8 3
For designing the embedding network, we use a foed-forward network with 100-node layers and PReLU activation. We consider two embedding layer dimensions: a known dimension and half the input size for an unknown network. We test the network's depth with cither 5 or 14 hidden layers and evaluate the impact of known metric in the exogenous space by comparing scenarios with both known and unknown metrics. We investigate the impact of assuming coordinate independence by including decorrelation loss function (Patil& & Purcell 2022), which uses the Frobenius norm of the difference between the identity matrix and XIcor (Chatterjee 2021 non-parametric correlation mcasure, on training performance.
In our mumerical experiments, a major challenge finding well-known datasets in causal inference and metric learning. Collecting tagged data aligned with specific metrics is becoming increasingly prevalent. For example, in new Large Language Model (LLM) methodologies, tagged data plays critical role in providing explicit feedback on which responses better meet user needs.
To address the lack of appropriate data, for our real-world datasets, Adult (Kohavi & Becker 1996) and COMPAS [Washington 2018), we first establish a causal structure as in Nabi et al. (Nabi & Shpitser 2018)) We also use synthetic datasets for Linear (LIN) and Non-linear (NLM) SCMs. For each SCM, we create three data scenarios using its structure. We employ the PCP ball with radii A = 0.1, and 0.2 for contrastive label creation, generating 10,000 samples. We then assess deep metric learning across 100 iterations with varying random seeds. Note that in our study, we did not study the specifics of the embedding network architecturc. Instead, we employed a straightforward feod-forward network, which is better suited for our tabular data. This choice aligns with Prop. 6.2 assisting in discerning the impacts of various assumptions.
To assess learning performance, we employ classifier metrics such as accuracy (Ace), Matthews correlation coofficient (MCC), falso-negative (FN), and falso-positive (FP) rates for label outputs. For embedding kernel learning, we use root mean square error (RMSE) and mean absolute error (MAE). Continuous metrics are used both label and triplet-base based kernel learning. In sestance-baseyy scenarios, label predictions are made by generating labels within the Bger for uniform performance evaluation across different settings.
To evaluate the ECAPIFY approach, compare it with traditional empirical risk minimization (ERM). Adversarial Learning (AL) as delineated by Madry et al. (Madry et al.] 2017), and CAP/FY, which is recognized for its superior effectiveness in mitigating unfairness, as detailed in Ehyaei et al. (Ehyaei et al. 2023b). Our simulation settings and performance metrics mirror those in Ehyaei et al. (Ehyaci et al. 2023b To measure unfairness we use the unfair area quantity that is defined as the below
Unfair Area == IP(u E VV Do' such that d(c,v') S A and h(v) + h(o)).
Similarly, we define counterfactually unfair area.
Cunterfactual Unfair Area = Pl Ba e such that h(c) # h(ta)).
We set the perturbation radius at A = 0.01 and report the percentages of non-robust, non-counterfactual instances, and their combination, along with accuracy. Additional simulation details are in the appendix, and our numerical analysis codes are available on GitHub
Results of Metric Learning As demonstrated in Fig. and in details in Tab. I our simulation confirms that knowing the metric and dimensions of the embedding space improves accuracy in metric learning, Although Prop. asscrts that deep learning can converge with various layer sizes without embedding space knowledge, we show that additional information significantly enhances results, particularly in triplet- scenarios.
10

---

Published in Transactions on Machine Learning Research (01/2024)
Real-World World Data
Synthetic Data
Adult
COMPAS
Lin
NLM
A
Loes Punction
Acct
FN1
MAE
RMSE
Aoc
FN
MAE RMSE
Acc
FN
MAE
RMSE
Acc
FN
MAE
RMSE
Distance-based
0.10
sabell-based
0.822
0.003
0.088
0.134
0.842
0.000
0.098
0.130
0.829
0.000
0.096
0.129
0.843
0.000
0.085
0.126
Itiplet-busee based
0.619
0,202
0.173
0.228
0.614
0.206
0.177
0.233
0.637
0.219
0.177
0.233
0.738
0.200
0.174
0.230
Distanre-based
0.010
0.20 basel-based 0.825
0.002
0.192
0.266
0.850
0.000
0.193
0.257
0.838
0,000
0.195
0.259
0.852
0.000
0.189
0.252
Iriplet-based bused 0.805
0,187
0.346
0.457
0.841
0.073
0.353
0.486
0.663
0.210
0.354
0.467
0.743
0.150
0.348
0.459
Table 1: The table shows results of a mumerical experiment comparing different learning scenarios, evaluated by accuracy (Acc higher is better), false negative error (FN lower is better), root mean square error (RMSE- lower is better), and mean average error (MAE lower is better). The best scenario for each dataset and perturbation radius in bold. XTcor correlation loss function and 5-layer embedding network are used. To show variation, we included error bars in Figures and detailed measures in Tables 2 4 in the appendix. This table shows our framework effectively estimates causal fair metrics, achieving high confidence across datasets, perturbation radii, and tagged data scenarios.
100%
90%
Distance
80%
80%
70%
70%
00%
60%
80%
AddI
COMPAS
Ln
NUN
Addt
COMPAS
N.N
Addt
COMPAS
Ln
N-M
Known Embedding Space
Unknown Embedding Space
80%
Label baaobbbree
Tripler Trolee-boraa Sorsed
70%
70%
60%
30%
Addi
COMPAS
COMPAR Un
COMPAS
XICOR
Figure This figure demonstrates the effect of causal metric ussumptions on the accuracy of deep metric models: (up) Accuracy performance comparison based on embedding layer sizes and embedding space metric knowledge shows improved prediction accuracy. (down) In simpler models, the network efficiently learus embedding space properties. However, with less precise metric data, in Triplet-based scenarios, adding decorrelation ion methods boosts accuracy.
Fig. 2 shows that embedding learning works well in distance-based and label-based scenarios, where adding the decorrelation loss function does not make big difference. But in the triplet scenario, where only metric relations are known, this loss function improves results.
To find the optimal configurations for embedding network layers, we ran experiments with various depths of networks. We find that five layers network is ideal for label-based and distance-based scenarios, whereas triplet-based scenarios perform better with deeper network structures. Further results are available in the Appendix (see Tab. 4 To summarize simulation results, analysis of various learning methodologies shows that distance-based metric learning is most effective when precise distance- based data is available. However, in practical situations, this idcal may not be achicvable. In such cases, the label-based method becomes viable alternative for metric approxination. This method's 's accuracy improves when euneedding space dimensions and metric information is combined, as Tab. [ supports. The label- based approach also has a lower false negative rate compared to other methods, making it effective in approximating the true metric. This is particularly useful in scenarios requiring fair metrics, like robust learning, as it helps maintain robustness criteria and builds a stronger model. When label data unavailable, the triplet method, enhanced with a decorrelation loss function and deeper networks, effectively deduces the embedding function.
Results of ECAPIFY. Fig. our simulation results, utilizing both real-world and synthetic datasets, show that. ECAPIFY, which is equipped with metric learning and not necessitating knowledge of SCM,
11

---

Published in Transactions on Machine Learning Research (01/2024)
Adult
Counterfactune Unfeir Aroca
Unfair Area (A- -0.0111
20%
20%
15%
75%
KVKC Sunijuug
18%
Sewiietti Crate
50%
10%
50%
25%
25%
BOAPPY
A
Campy Blarey
carpy
Capppy
Carter
Ai
Ganay Blawe
caney maNnY
Ai
Daneyr BRANDY
Adul
COMPAS
LWV
NCM
Asudt
COMPAS
LIV
ALM
MCC Score
40%
40%
Symbbar Chal
Accuracy
60%
90%
30%
10%
20%
20%
0%
0%
0%
0%
Cappl
eoargy
carpy
cappy
acarery
mu
capgy Elargpy
ecarer
Eier
carer detare eod
Figure 3: Presents the results of our numerical experiment, evaluating ECAPIPY's performance across varions models and datasets. (Top left) Bar plot comparing madels using unfair area percentage (lower better) at A = .01. (Top right) Counterfactual unfair area percentage (lower better).. (Bottom left) Matthews correlation coefficient illustrating classifier performance (higher is better). (Bottom right) Bar plot contrasting methods by prediction performance (higher is better). CAPIFY performs better because it assumes knowledge of the causal metric, whereas ECAPIFY first learns the metric before fitting the model.
yields results akin to CAPIFY (regarded as an oracle), achieving similar effectiveness in diminishing unfair areas with A = 0.01. Additionally, there is a notable performance improvement compared to both ERM and AL, which concentrate on adversarial training, while still preserving high prodiction accuracy. A key aspect of ECAPIFY is that, despite Prop. 6.2 highlighting the impracticality of adversarial training with only observational data, ECAPIFY offers feasible approach using empirical data. Essentially, without knowledge of the SCM, estimating causal fair metric is crucial for adversarial learning.
9 Discussion and Future Work
In this study, we introduce the concept of a causal fair metric and propose a protected causal perturbation to integrate individual fairness, adversarial robustness, and causality. By leveraging causal structures and protecting sensitive attributes, our approach enables the training a causally aware, fair, and robust classifier without relying on a structural causal model (SCM). Instead, we utilize metric learning to balance individual fairness and adversarial robustness.
Our approach prioritizes fairness, robustness, and causality, aligning with responsible Al principles. However, it inherits vulnerabilities from underlying models, such as issues with privacy, explainability, and safety, requiring careful consideration in high-stakes applications. We emphasize that this work serves as a proof of concept, encouraging further exploration and collaboration. Although our method demonstrates promising results, it also highlights key limitations and avenues for future exploration.
Our methodology assumes an additive noise model, which may not fully encapsulate the complexity of real-world causal relationships, complicating additive interventions in general SCMs. Additionally, despite similarities to existing metric learning methods, our approach lacks theoretical guarantees for estimator performance and faces challenges such as local minima. Future work aims to address these issucs by impos- ing constraints on the causal fair metric structure and developing explicit convergence theorems. Another significant challenge is the scarcity real-world datasets suited for metric learning with causal structures. Furthermore, our proposed notion of protected causal perturbation extends to other domains within cansal machine learning, such as algorithmic recourse, causal bandits, and reinforcement learning, offering a broader framework for analyzing fairness and robustness.
12

---

Published in Transactions on Machine Learning Research (01/2024)
Acknowledgments
The authors thank the Max Planck Institute for Intelligent Systems, Tübingen Al Center, for supporting this project. Partial funding support was also provided by the Canada CIFAR AI Chair program.
References
Peter I. Bartlett, Dylan J Foster, and Matus J Telgarsky. Spectrally-normalizzed margin bounds for neural networks. Advances in neural information processing systems, 30, 2017.
Yahav Bechavod, Christopher Jung, and Steven Z Wu. Metric-frec individual fairness in online learning. Advances in neural information processing systems, 33:11214 11225, 2020.
Sourav Chatterjec. A new coefficient of correlation. Journal of the American Statistical Association, 116 (536):2009-2022 2021.
Davide Chicco. Siamese neural networks: An overview. Artificial neural networks, pp. 73 73-94, 2021.
Emir Demirović and Peter Stuckey, Optimal decision trees for nonlinear metrics. In Proceedings of the AAAI conference on artificial intelligence, 2021. Volume 35, Number 5, Pages 3733-3741.
Ricardo Dominguez-Olmedoo Amir H Karimi, and Bernhard Schölkopf. On the adversarial robustness of causal algorithmic recourse. In International Conference on Machine Learning, pp. 5324 5342. PMLR, 2022.
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pp. 214-20142 226, 2012.
Frederick Eberhardt and Richard Scheines. Interventions and causal inference. Philosophy of Science, 74(5): 981 995, 2007.
Ahmad-Reza Ehyaci, Amir-Hossels Karimi, Bernhard Schölkopf, and Setarch Maghsudi. Robustness im- plies fairness in causal algorithmic recourse. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pp. 984-1001, 2023a.
Ahmad-Reza Ehyaei, Kiarash Mohammadi,
Karimi, Samira Samadi, and Golnoosh Farnadi. Causal adversarial perturbations for individual fairness and robustness in heterogeneous data spaces. arXiv preprinl or 23v: 000388888, 2023b.
Deena P Francis and Kumudha Raimond. Major advancements in kernel function approximation. Artificial Intelligence Review, 54:843 876, 2021.
Benyamin Ghojogh, Ali Ghodsi, Fakhri Karray, and Mark Crowley. Spectral, probabilistic, and deep metric learning: Tutorial and survey. arXio preprint 2rXi0,2201.09267, 2022.
Benyamin Ghojogh, Mark Crowley, Fakhri Karray, and Ali Ghodsi. Elements of dimensionality reduction and manifold learning. Springer Nature, 2023.
Stephen Gillen, Christopher Jung, Michael Kearns, and Aaron Roth. Online learning with an unknown fairness metric. Advances in neural information processing systems, 31, 2018.
Tan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXie:1412.6577, 2014.
Patrik O Hoyer, Dominik Janzing, Joris M Mooij, Jonas Peters, and Bernhard Schölkopf. Nonlinear causal discovery with additive noise models. In Advances in neural information processing systems, pp. 689-688-88 689-696, 2009.
13

---

Published in Transactions on Machine Learning Research (01/2024)
Christina Ilvento. Metric learning for individual fairness. In 1st Symposium on Foundations of Responsible Compating, 2020.
Alexander Immer, Christoph Schultheiss, Julia E Vogt, Bernhard Schölkopf, Peter Bühlmann, and Alexan- der Marx. On the identifiability and estimation of causal location-scale noise models. In International Conference on Machine Learning, pp. 14316-14332. PMLR, 2023.
Philips George John, Deepak Vijaykeerthy, and Diptikalyan Saha. Verifying individual fairness in machine learning models. In Conference on Uncertainly in Artāficial Intelligence, pp. 749-758. PMLR, 2020.
Niki Kilbertus, Mateo Rojas Carulla, Giambattista Parascandolo, Moritz Hardt, Dominik Janzing, and Bernhard Schölkopf. Avoiding discrimination through causal reasoning. Advances in neural information processing systems, 30, 2017.
Ronny Kohavi and Barry Becker. Uci adult data set. UCI Meachine Learning Repository, 5, 1996.
Mark Kozdoba and Shie Mannor. Two regimes of generalization for non-linear metric learning. OpenReview, http://wweeeeeee//ooooom////////
ICLR 2022,
2021.
Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. Tn Advances Neurul Information Processing Systems, pp. 4069 4079, 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial atlacks. arXiv preprinl arXic: 1rXx:170660008,, 2017.
Daniel McNamara, Cheng Soon Ong, and Robert Williamson. Provably fair representations. arXiv preprint SEKI::1710.043999 2017.
Debarghya Mukherjee, Mikhail Yurochkin, Moulinath Banerjee, and Yuekai Sun. Two simple ways learn individual fairness metrics from data. In International Conference Machine Learning, pp. 7097-7107. PMLR, 2020.
Razieh Nabi and Ilya Shpitser. Fair inference on outcomes. In Proceedinggs of the AAAI Conference on Artificial Intelligence, 2018. Volume 32, Number 1.
Conlan Olson. Algorithmic Fairness, Metric Embedding, and Metric Learning. PhD thesis, Harvard Univer- sity, 2022.
Pranita. Patil and Kevin Purcell. Decorrelation-basel deep learning for bias mitigation. Future Internet, 14 (4):110, 2022.
Judea Pearl. Causalily: Models, Reasoning, and Inference. Cambridge University Press, 2009.
Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference: foundations and learning algorithms. The MIT Press, 2017.
Chongli Qin, James Martens, Sven Gowal, Dilip Krishnan, Krishnamurthy Dvijotham, Alhussein Fawzi, Soham De, Robert Stanforth, and Pushmect Kohli. Adversarial robustness through local linearization. Advances in Neural Information Processing Systems, 32, 2019.
Anian Ruoss, Mislav Balunovic, Marc Fischer, and Martin Vechev. Learning certified individually fair representations. Advances in neural information processing systems, 33:7584-7596, 2020.
Juan Luis Suárez, Salvador García, and Francisco Herrera. A tutorial on distance metric learning: Math- ematical foundations, algorithms, experimental analysis, prospects and challenges. Neurocomputing, 425: 300-322, 2021.
Anne Washington. How argue with an algorithm: Lessons from the compas-propublica debate. Colo. Tech. LJJ, 17:131, 2018.
14

---

Published in Transactions on Machine Learning Research (01/2024)
Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning fair representations. In International conference on machine learning, pp. 325 333. PMTRR 2013.
Kun Zhang and Aapo Hyvarinen. On the identifiability of the post-nonlinear causal model. arXiv preprint 12X5:: 2205,209,, 2012.
Luwan Zhang, Grace Wahba, and Ming Yuan. Distance shrinkage and euclidean embedding via regularized kernel estimation. Journal of the Royal Statistical Society Series B: Statistical Methodology, 78(4):849 867, 2016.
15

---

Published in Transactions on Machine Learning Research (01/2024)
Symbol
Notion
Symbol Px Px((2)
Notion
SCM
Structural causal model
Combination of embedding and projection on X Protected causal perturba- tion
d(v, uy)
Fair distance metric between and Bl
PCP
M
Structural causal model
V
Feature or endogenous space
B (v)
PCP ball for instance u with radii
includes n random variables
{Vi)==
B
simple closed ball with radii A in X
U
noise exogenous space in- cludes random variables {U:)
Non-sensitive part of B ball
g
Causal graph
activation function on layer i th Neural net embedding func- tion
Set structural equations f; Exogenous probability distri-
PU
Pw
bution
g:u +V
reduced-form mapping from U to V
W
Matrix parameters of neural net
Pv
Feature probability distribu- tion Additive noise model
W
Parameter space for tuples W Space of all embedding func- tions
$
ANM
Mdo(Vz:=0)
Hard intervention respect to subset of feature
||]
Spectral norm on Matrix Sum of the & norms of cach column in matrix W Rademacher complexity of + {V; yi w))= set of observa- tional data
[
Mdd(V=+=d)
Additive intervention
(F(2,0)
counterfactual instance re-
R(Φ)
spect hard intervention Altered reduced-form map- ping w.r.t. Mdo(V{;=0) Sensitive attribute
D
PD
Observational probability Learning loss function
s
s
The level sets of sensitive at- tribute Counterfactual twin for Set of all twins Metric on feature space Metric label space Classifier function counterfactual for additive noise interventions Cansal fair metric Causal perturbation over non-sensitive part Beruoulli distribution Semi-latent space embedding map from to the semi-latent space Inverse of embedding map metric space for each semi- latent space component
RA
CAPIFY regularizer
RA
ECAPIFY regularizer
0. = CF((,)))
L8
Huber loss function
4
[[--
Standard Hinge loss function
dx
Prop.
Proposition
dy
Fig.
Figure
h: + x->+>
Eq.
Equation
CF(e,))
d:VxV +
=0=0
B(p)
Q
P:V-0 Q
p-1
(Q,,,) d.)
x
Non-sensitive part of semi- latent space Metric that is define on X Projects semi-late semi--atent spate to X'
dx
Px
16

---

Published in Transactions on Machine Learning Research (01/2024)
.1 Additional Background
Definition .1 ((Pseudo-) Metric Space) A metric space (X,d) is defoned as set X accomponied by non-negative real-valued function d: X x X - Rzo, which is referred to metric. This metric function d adheres to the subsequent properties for any &, J. E X:
Non-negativity: d(x,y) 2 0, and d(x,y)
and only if y,.
Symmetry: d(z, d(y,
Triangle inequality: d(r, z) S d(x, v) d),,z).
When the posilivity condition, i.e,, d(x, v) 0 if and only if = y is relaved, the d is called pseudomelric (or semi-metric).
Definition .2 (The pull-back & push-forward metric) let f uu + be a mapping between the met- ric spaces (U, du) and (V,dy). The push-forward metric d induced by the funcction I is defoned as:
d(u1, f(22)) u1 E
Similarly, the pull-back metric the space U is defined as:
d(vi, v2) du (v)( (va)); U1, EV
These definitions allow us to relate distances U and V via the mapping f and its inverse f-1 Definition .3 (Huber Loss) For given predicted value and true largel value the Huber loss function is defined as:
(0 y) 102
f 1 1p-51sa
Lo(0. v)
010
olherwise
where I the predicled value, y is the true larget value, and ê is positive constand that determines the threshold for switching from quadratic loss (L2) linear loss (L1).
.2 Proofs
Proposition 4.4
Let's consider a causal fair motric denoted as d: V V + R, with an associated embodding ş V + Q, mapping from the feature space to semi-latent space. We define d' as the pull-back metric of onto Q:
d' =
d' possesses metric properties, and we aim elucidate which properties inherits from Definition 4.2 We consider a decomposition of into S X, and let q x (v), where V. Utilizing this decomposition, we express as (s, Property (i) of the causal fair metric implies:
d(v.üx) d' ((s, z), (s' x))
AP' S
This property implies that d' is invariant to the sensitive part S. To demonstrate this, assert that for any two points qı = (81,44) and g2 (22, along with an arbitrary so E S, the following equality holds:
d' (1), (82, x2)) d' ((50, (80)
17

---

Published in Transactions on Machine Learning Research (01/2024)
By utilizing the triangle property of d', can establish:
0
d' (1), x2)) d" ((so, x1). (82, x2)) + d' ((11,22)) 21)) d' (((1. 2 x2) S d'
21) (1)) (22,
The distance d' (($1, 11), (8)) zcro due to the first property. Similarly, it can be shown that:
d' (1), x2)) S d' ((51, 21) (s2, x2)) + + (() = d' ((50, (2)) S d' ((51. (1)
it concludes that:
d' ((50. 11,, (8), d' ((51. 21), (1),
similarly, can show:
d' ((50 (82, = d' ((50, 21), (80,
This last equation implies that d' invariant to the sensitive subspace. If we consider dx as the inducod metric of d' the sensitive subspace X, then we can express:
d' ((5), 21), (82, = dx(x1,
The second property of Def. 4.2 can be expressed in simplified form based dx- It implics that for every E X, the distance (x + +8), where E Rdim(X),, is continuous with respect to 6. This continuity implies that dx is continuous along each component on its diagonal, i.e., (z,z).
Finally, we replace x with Px(((()),, where Px the projection operator onto the subspace X' within Q. we obtain:
d(v,w) dx P
This equation completes the proof.
Proposition 5.2
The proof is straightforward when we write out the definitions. Let pp(v) (s, represent the embedding of the variable v in the semi-latent space. To begin, we can demonstrate how the semi--latent space enables us to describe the counterfactual of instance v concerning the hard action do(S:=s') as follows:
4 (v(() @r s') ( 0r 01 4 ( x)) CF do(S:=s')) ü..
In the above Equation, we use the symbol u Gr to represent a masking operator that modifies the values of the entries corresponding to set in vector v by replacing them with 0. The validity of the last line Equation 2 based on the definition of the semi-latent space embedding.
By the definition
5.1 the Brer(e) is equal to:
(v) BP
EV: d(v, 5 =) (.: s SA} (e' EV
(V' VVEV:dr(ar {} = U (s.d) S SA}= =
#ES
P Uf P+(((()) = & A dx(px((s) مA}= {A}}
RES
utde VVV::: (o((()) = P+((((')) A px(6') E (x(xx())))
EES
U D (i.)
EES
The last equation completes the proof.
18

---

Published in Transactions on Machine Learning Research (01/2024)
Proposition 5.3
To present the result, we must first prove the following lemma:
Lemma A Let d be causal metric, and let dx be the corresponding embedding metric on the non-sensitive part of the exogenous space. For the closed ball B we have:
lim B (x))==x A
We establish the aforementioned lemma through a proof by contradiction. Let us assume that there erists another point, denoted as x' + z, within the set lim ( (z). Consequently, we have dx dx(z,x") O. If we consider v' = ( ()),,,)),, then for v', have d(v,v 0, since o' & {ūa)- However, this contradicts the property inherent to one of the cousal fair metrics.
Proof
By utilizing the above lemma and Prop. 5.2 we can represent the result as follows:
BB (v) lim BPOP (v) A00
lim
B (i)) U
lim (8.)
2-0
SES A-0
U
lim lim e p(o'))= = (s', 2²) A = A o' (x)}
SS A-
U x') ^ s = x' E lim (x))
2-0
#ES
Uf VeVeV:lle (s, U 82 SES
#ES
.3 Simulation Scenarios
Utilizing distance-tagged triplets (v, of, d,), where d, indicates the non- negative real mumber d(v,, vf) as a distance. We also apply the Huber function E(v,, iy d.) = Ls(d; d((((1)), ((x())) for learning loss.
Distanco-basod:
Label-based: Utilizing a Siamese network (Chicco 2021 with a contrastive loss for triplets (v,, y1), where Yi {0, 1} indicates proximity between points, and the loss function f(v,, 1/ w) equals to (1 (;)d((p(ss) (p((()) + ([)) + m]+, here m is the marginal and Hl = max((. 0) is standard Hinge loss.
Triplet-based: In this approach, tuples (o), of uft ud) are considered, where yı denotes the closeness of ot to of compared to of and vf Embedding is trained using a Siamese network with the triplet loss function f(v), vf uf s)) = (((p())), (((()) + +m].
.4 Synthetic Data Models
In the $ S we detail the structural equations employed formulate the SCMs for both LIN and NLM models. The protected feature, denoted as S, and the non-sensitive variables represented by X, derived based the subsequent structural equations:
linear SCM (LIN):
S = Us.
Us ~ B(0.5)
X1 = 28 + U1,
U1 ~ N(0,
X2 S Xi U2,
U2 ~ (0,
Non--linar Model (NLM)
S Us:
Us B(0.5)
F
X1 = 252 + U1,
~ 1(0,1)
Xx:= : S X U2,
12~,((011
19

---

Published in Transactions on Machine Learning Research (01/2024)
Where B(p) represents Bernoulli random variables characterized by a probability p, and Mi, o?) denotes normal random variables, which are defined by mean of js and a variance of o².
.5 Real-World Data
In our study, we employed the Adult (Khhaii & Becker 1996 and COMPAS /asshingto 2018 datasets, constructing an SCM from the causal graph by Nabi (Nabi & Shpitser 2018). For the Adult dataset, we considcred features like sex, age, and education-num. with sex as a sensitive attribute. For COMPAS, features included age, race, and priors count, with sex as the sensitive attribute.
.6 Hyperparameter Tuning
In our experimental setup, we generated 10,000 samples for each SCM model. The data was divided into batches 1,000, and the learning process spanned 100 epochs. The coefficient of the decorrelation regularizer set 0.1. Furthermore, in the contrastive label-based scenario, the margin was set equal to the radius of the experiment, while in the trsslet-based scenario, the margin was set to zero to have more sensitivity for metric learning.
.7 Training Methods
In our study, we train decision-makee classifiers, denoted as h(x), using various training objectives:
Empirical Risk Minimization (ERM): Minimizes expected risk for classifier parameters sb defined as:
min E ((aa))Pp(((hs()))) y)l
Adversarial Learning (AL): Trains the model against adversarial perturbation:
min E (o [ max f(h + 5),8) oEBa(e)
CAPIFY:
Combines locally linear QQin et al. 2019 method principles with known CAP as perturbation attack:
min (v.g)) ((hh)) (z), 3) μn max ((h(U.).))))))))))))))(()))))( v) + n2 7(A, μa || f(o)].) sES
ECAPIFY: Combines LLR method principles with CAP as a perturbation attack:
min ((hm)) y) + max E(h(G.).)) + #2 IA V. v)1 + Aa (a...
eES
We utilize binary cross-entropy loss as our loss function
.8 Metrics
We use different metrics to evaluate trainers' performance in terms of accuracy, CAPI fairness (Ehyaei al. 2023b), counterfactual fairness, and adversarial robustness:
Acc: Classifier accuracy, expressed percentage.
M: The Matthews Correlation Coefficient (MCC) for binary classification quality. It ranges from
-1 (perfect inverse prediction) to +1 (perfect prediction), with 0 indicating random prediction. Formula:
(TPx TN - FP x FN)
(TP++P)(TP-4 FN)(TN + FP)(TN FN)
Where TP, TN, FP, and FN are True Positives, True Negatives, False Positives, and False Negatives, respectively.
20

---

Published in Transactions on Machine Learning Research (01/2024)
Falte Nogathee
Disiance basod
30%
9.4%
10%
MAR
Adal
COMPAS
AA3
COMPAS
Known Emibedding Space
Unknown Embedding Space
Falta Pooddve
Dielere boceed
Labes Laber-based based
20%
10%
3/6
5%
0%
0%
Adult
COMPAS
Adut
COMPAS
MM
Adui
COMPAS
1
Known Embediing
Unknown Embedding
Moin Abeehut Ereer
0.125
0.20
0.100
0.15
Lobbey So000
0.2
Triplo- Aoood
0.050
0.026
0.05
0.00
0.0
Adalt
COMPAS
61
NLM
AdMi
COMPAS
Ln
NLM
Adalt
COMPAS
L
NLW
Known Embbeding Space
Unkeowwn Embactiing Space
Root Meen Sqaaro EINe
0.3
0.00
CONPAS
Embadding Space
Watthees Conelation
1.0
0.5
0.5
0.4
0.4
0.2
02
0.2
Known Embedsing Space
Unknown Embedding Space
Figure 4: The performance metric of the learning scenario with varying knowledge regarding the embedding layer size.
MAE: Mean Absolute Error calculated as MAE 1/2 Σi ly4 Dcl-
RMSE: Root Mean Square Error with formulas: E (yi 2.)
Unfair Area: Proportion of data points within the unfair area of radius A as defined in Ehyaei (Ehyaei et al. 2023b
Non RobustArea: Fraction of non-robust data points to adversarial perturbation within radius A, equivalent to the unfair area in the absence of sensitive altribute.
CounterfactualnnaaiiArnaa Percentage of data points showing counterfactual unfairness, analo- gous the unfair area when perturbation radius zero.
.9 Additional Numerical Results
In the subsequent tables and figures, additional numerical analysis results are presented to support the assertions of this study. Their explanations can be found in g
21

---

Published in Transactions on Machine Learning Research (01/2024)
Embedding Layer Dimension
Loss Function
Performance Metric
Known Embedding Space
Unknown Embedding Space
Accuracy
0.983 + 0.011
0.882 + 0.038
False Negative
0.013
0.043 0.005
False Positive
0.005 0.001
0.075 0.039
Distance-based
Matthews Correlation t
0.965 + 0.027
0.766 1 0.073
Mean Absolute Error +
0.016 + 0.017
0.105 + 0.023
Root Mean Square Error +
0.025 0.027
0.17 + 0.041
Accuracy 1
0.845 + 0.012
0.812 + 0.021
False Negative
0 + 0.001
0.003 + 0.003
False Positive +
0.155 1: 0.012
0.186 0.019
Label-based
Matthews Correlation
0.725 0.019
0.67 +
Mean Absolute Frror +
0.192 + 0.003
0.19 + 0.003
Root Mean Square Error
0.257 + 0.006
0,266 + 0.009
Accuracy
0.763 = 0.08
0.708 0.088
False Negative
0,226 + 0.154
0,292 + 0.133
False Positive +
0.109 +:2 0.076
0.107 4: 0.063
Triplet-based Matthews Correlation f
0.529 0.157
0.415 0.176
Mean Absolute Frror
0.351 0.004
0.35 +:
Root Mean Square Error L
0.463 0.004
0.462 0.004
Table 2: The table displays the average performance metrics for comparing scenarios with knowledge of embedding dimensions and their corresponding metrics against scenarios with no knowledge of the embedding space. Green cell highlights denote superior performance, while smaller values indicate the standard deviation of the cstimations.
Decorrelation
Regularizer Function
Loss Function
Performance Metric
XICOR
Accuracy t
0.984 0.012
0.983 0.014
False Negative J
0.012 = 0.008
0.013 + 0.01
False Positive $
0.004 = 0.004
0.005 = 0.004
Distance-based
Matthews Correlation
0.969 0.023
0.965 1 0.027
Mean Absolute Error
0.016
0.016
Root Mean Square Error +
0.021 1: 0.028
0.025 1: 0.027
Accuracy f
0.843
0.017
0.845 + 0.012
False Negative J
+ 0.001
0 0.001
False Positive $
0.156 = 0.017
0.155 + 0.012
Label-based
Matthews Correlation f
0.721 1 0.028
0.725 1 0.019
Mean Absolute Error 4
0.19 1 0.008
0.192 1: 0.003
Root Mean Square Error
0.256
0.257
Accuracy 1
0.669 = 0.032
0.763 + 0.08
False Negative J
0.318 0.146
0.226 + 0.154
False Positive
0.117 + 0.09
0.109 + 0.076
Triplet-based based
Matthews Correlation 41
0.339 1 0.063
0.529 1 0.157
Mean Absolute Error 4
0.351 J. 0.002
0.351 1: 0.004
Root Mean Square Error
0.464 +: 0.003
0.463 1: 0.004
Table 3: The table displays average performance metrics for various scenarios, considering the presence of different decorrelation policies. Green cells highlight the best performance, while smaller values represent standard deviations of the estimates.
22

---

Published in Transactions on Machine Learning Research (01/2024)
Network Layer
Loss Function
Performance Metric
CIFNet Layers
CIFNet Layers
Accuracy +
0.846 + 0,066
0.924 + 0.091
False Negative 4
0,078
0.047 + 0.053
False Posilive 4
0,076 + 0.035
0.029 1: 0.039
Distance-based
Matthews Correlation
0.693 = 0.131
0.849 = 0. 08182
Mean Absolute Error +
0.07 + 0.049
0.029 + 0.038
Root Mean Square Error
0.104 0.07
0.044 + 0.056
Accuracy f
0.799 + 0.028
0.819 + 0.032
False Negative $
0.008 + 0,009
0.005 + 0.012
False Posilive
0.192 0.027
0.176 1: 0.024
Label-based
Matthews Correlation
0.644 = 0.049
0.68 = 0.06
Mean Absolute Error +
0.108 + 0.065
0.113 - 0.06
Root Mean Square Error
0.154 = 0.078
0.153 = 0.081
Accuracy +
0.543 0.032
0.642 + 0.065
False Negative 4
0.306 + 0.102
0.181 + 0.038
False Positive $
0.151 + 0.081
0.177 + 0.029
Triplet-based
Matthews Correlation
0.091 = 0.0G3
0.284 0.13
Mean Absolute Error
0.204
0.206 0.104
Root Mean Square Error
0.269 1 0.144
0.271 0.138
Table 4: To determine the optimal number of layers required for the best estimation of the embedding function, comparison conducted between two networks containing 5 and 14 layers, respectively.
23


Reviews:
[{"review_id": "10Kis3ukMt", "reviewer": "Reviewer_YuZs", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n- The paper is generally well-written and well-organized. \n    \n- The causal fair metric is intuitive and well-motivated with examples. \n    \n- The idea of learning the metric in causal latent space without directly using the structural causal models is novel and intuitive. \n    \n- The experiments are thorough and well-conducted.\n    \n- The paper clearly states the limitations of the proposed method and provides a detailed Broader Impact Statement.  \n    \n\nWeaknesses: \n\n- Some parts of the paper must be revised for clarity. Please see the requested changes below.\n    \n- The experiment did not clearly show how the proposed method improved individual fairness. See the requested changes for details.  \n    \n- Definition 4.2 shows that the causal fair metric still relies on structural causal models, even when defined using the Semi-latent Space. How does the proposed method completely alleviate the need for structural causal models? \n    \n- Since the proposed methods ensure boat individual fairness and robustness, does it also alleviate the disparate robustness across groups, or does it exacerbate it [1]? \n    \n\n[1] Xu, Han, et al. \"To be robust or to be fair: Towards fairness in adversarial training.\" International conference on machine learning. PMLR, 2021.**", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a framework for jointly learning a model that exhibits three important trustworthy AI properties: individual fairness, adversarial robustness, and causality. The proposed framework introduces a causal fair metric that encodes the causal structure and protected causal perturbation. This metric ensures that the similarity between individuals is both causally informed and sensitive to protected attributes. The authors then use metric learning to train classifiers that are both causally fair and robust. Experiments on real-world and synthetic datasets demonstrated the effectiveness of the method.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Citations in the paper must be improved. In some parts of the paper, the authors’ names are duplicated in the citations. For example, in the introduction, first paragraph, line 5, the reference name Dwork et al. is duplicated. The author should use only bib \\cite and \\citep properly without handwriting the author's name.  \n\n- The notations in the paper should be consistent to ease the reading. For example, in Example 4.1, the authors declare two structural causal models, $M$ and $M’$, and later state $M_1$ models the variables as independent, $M_2$ specifies a linear causal relationship. The authors should replace $M_1$ by $M$ and $M_2$ by $M’$. \nIn the same example, the authors should clarify the values of the gender distribution (e,g., {M, F}) and the corresponding numerical values. It was not clear to me how counterfactual change in gender in $M’$ yield $CF(v, F) = (F, 0, -1)$. How does the variable Income become -1 in under $M’$? And why is the $d(v, v_F)=3$ and not 4? Providing these clarifications can ease the comprehension of the example.  \n- In example 6.1, \"_The counterfactuals for v with respect to $M_A$ and $M_A$ are (0, 0, 0) and (0, 0, N ), respectively_\" it should be $M_A$ and $M_B$  \n\n- The authors should thoroughly define the fairness metrics used in the experiments, e.g., `UnfairArea`; \n\n- Discuss how the learned robust model affects adversarial robustness across demographic subgroups.  \n\n- Since there is a lack of real-world datasets for metric learning causal structures, can the authors briefly discuss the importance of having such datasets in the real world and how they can be collected in practice?  \n\n- Having the Broader Impact Statement at the beginning of the Appendix or before the references would be better.", "broader_impact_concerns": "The authors provided a Broader Impact Statement", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "nCxfVVNBqD", "reviewer": "Reviewer_fCeB", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\n- The paper addresses the topic of fairness, robustness, and causality integration, which is a growing focus in the AI research community.\n\n- The proposed causal fair metric and associated methods are thoroughly explained and well-supported by theoretical justifications and experimental evaluation.\n\n- The paper is well-written and well-organized and discusses the method's limitations.\n\n\n## Weaknesses\n\n- Results reported in Table 1 are incomplete and require additional details (see Requested Changes - 1).\n\t\t\n- The authors do not provide the source code.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose a causal fair metric to integrate fairness, robustness, and causality in AI, focusing on individual fairness and resilience to adversarial perturbations. This metric leverages causal structures involving sensitive attributes and creates protected causal perturbations without relying on structural causal models (SCMs), using metric learning instead for adaptability in real-world applications. Through experiments on synthetic and real datasets, their approach, ECAPIFY, demonstrates improvements in fairness and robustness within classifiers while preserving accuracy. Key contributions include (1) the causal fair metric, (2) protected causal perturbation, and (3) a practical SCM-independent metric learning framework.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Experimental Evaluation\n\n- Results in Table 1 compare different learning scenarios; however, to fully support the claim that accuracy improves when embedding space dimensions and metric information are combined, the results should include a measure of variation to validate this improvement.\n\n2. Improve Readability\n\n- To enhance readability, remove duplicate citations (e.g., \"Dwork et al. Dwork et al. (2012)\" in Section 1).\n- Proofread the manuscript to ensure consistent notation (e.g., Example 4.1 uses M and M' but also M_1 and M_2).\n- Relocate the Broader Impact Statement to follow the Discussion and Future Work section for better flow.", "broader_impact_concerns": "Nothing to add to the existing one.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "q4JvIwFgti", "reviewer": "Reviewer_P5Zm", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nI think that the paper tackles an interesting idea, in that there are some similarities in how we think about fairness versus adversarial robustness, and the idea of computing similarities in the context of a structural causal model very much strengthens the fairness framework. The background section is generally well-written.\n\nWeaknesses:\n\nUnfortunately the paper contains many weaknesses that renders it unsuitable for publication in its current form. The details of these are also outlined in the requested changes section; however, they are also summarized here.\n\n* The paper contains many errors, making it very difficult to read.\n* Sections of the paper use unclear language (e.g., this reviewer could not always understand what was meant by “embedding”)\n* The motivation for the definition of the metric is not clear. Specifically, note that the metric definition does not place any restrictions in the case $d\\big((s, r), (CF(s + \\Delta), r)\\big)$\n* The proofs are not always clear\n* The motivation for using a neural network for estimating the causal distance is unclear. Note that to learn this distance, it is required to have labelled data, for which a causal graph is required. If a causal graph is available, then why can the metric not be computed directly?\n* The experimental section is very hard to understand, both in terms of what experiments are actually performed, and why. The adversarial robustness of the classifiers doesn’t seem to be evaluated\n\nThe errors, in particular, make the paper very difficult to follow and evaluate. Addressing these issues will make it far easier to evaluate the paper’s contribution.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a novel distance metric that combines the ideas of individual fairness and adversarially-robust learning and embeds both in a structural causal model. Briefly, this metric fulfills the requirements that for a feature space consisting of sensitive attributes $s:={s_1, s_2, … , s_k} \\in S^{k}$ and non-sensitive attributes $r:={r_1, r_2, … , r_m} \\in S^{m}$,\n\n1.  $d((s, r) = d((s’, r)))$ for all $s, s^{‘} \\in S, r \\in R$\n2. $\\forall$  $s \\in S, r \\in R,  \\exists \\epsilon, \\delta$ s.t.  $\\forall \\Delta \\leq \\delta, d(s, CF(r + \\Delta)) \\leq \\epsilon$.\n\nWhere CF stands for the counterfactual \"do\" operartion.\n\nIn the paper, the authors motivate the need for a causal structure,  and describe how to compute this metric, by introducing the semi-latent spaces (in which the causal parents of sensitive attributes are removed). The authors then propose using deep learning to estimate the distance metric from labelled data, and then using this metric to perform fair learning.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The below proposed adjustments must be addressed for securing this reviewer’s recommendation to acceptance.\n\nWhen appropriate (most of the time), cite references using the Latex command \\citep, which correctly encloses the reference in parentheses.\n\nDefinition 4.2. Why place the restriction that the metric be 0 only for twin pairs? Also, note that the metric definition does not place any restrictions in the case $d(s, r), (CF(s + \\Delta), r)$. This seems undesirable.\n\nDefinition 4.3: “indigenous” -> “endogenous”. Also, “replacing the causal effects of parents of sensitive attributes and replacing their exogenous variables with endogenous ones” is unclear - possibly should be the reverse?   In the definition of $\\varphi_i$, what is $F_i$? Also, please demonstrate why this mapping is actually a bijection. Also a typo: “There map bijective map”. Also, “the semi-latent space is simpler due to the independence of its components” - the components are not independent, since changing a sensitive attribute can have causal effects on the non-sensitive attributes. Also, in the same paragraph, what is the “intervention assumption”?\n\nSection 5, 2nd line - what is meant by “data changes still fall the same category for the model”?\n\nIn the paragraph between Def. 5.1 and Prop. 5.2 - Note that two balls are not ‘isomorphic’ - two spaces with metrics may be isomorphic.\n\nSection 6 - please be explicit (perhaps with an example) on how $\\varphi$ is constructed.\n\nThe paragraph after Proposition 6.2 is not clear, but also seems circular - if we have a way to data with distances (i.e., a causal structural map), why is it necessary to use NNs or another method to approximate the metric? \n\nThe last paragraph on page 7: This leads to three main insights: the dimensionality of the embedding space, the guarantee of independence from coordinates in this space, and understandings about $d_{\\chi}$ - what is meant by “the guarantee of independence”, and “the understandings”, and how does it follow from the rest of the paper?\n\nTop of page 8: the activation functions $\\sigma$ do not map from $\\mathcal{R}^{d-1}$ to $\\mathcal{R}^{d}$.\n\nProposition 6.3 - This proposition doesn’t seem relevant to the paper, and I don’t agree with the interpretation. First, note that the dimensions of the weight matrices are implicitly included in the norm vectors (more dimensions -> higher norms). Second, the Rademacher complexity deals only with the expressivity of the functions, and has no implication on learning dynamics.\nPlease explain the first formula in section 7 in words (e.g., what is $h_{\\psi})? What is $\\mathcal{P}_{\\mathcal{D}}$ - earlier, $\\mathcal{D}$ was defined as a set of observations - what does it mean for variables to be distributed according to a set of observations?\n\nLikewise, please explain the formulas on the bottom of page 8/top of page 9 (as a side suggestion, it would have been helpful to number the equations). For example: What is $\\dot{y}$? Does the star symbol stand for multiplication? What are $\\mu_{1}$, $\\mu_{3}$, and $\\mu_{3}$?\n\nFirst paragraph of section 8 - please define Siamese metric learning. Also please justify why it’s relevant to try three different types of data annotation, and propose a scenario in which such labeled data would be available, but the actual causal structure would not be known. If there is no such scenario (i.e., the causal structure is known), then what is the utility of estimating this distance with a neural network?\n\nSecond paragraph of section 8 - please explain what is meant by “designing the embedding network. What is meant by “half the input size for an unknown network?  Why is that a reasonable baseline? It seems far too large.\n\nFirst paragraph of page 10 - “Although Prop 6.2 asserts that deep learning can converge with various layer sizes…” - it seems that the authors meant Prop. 6.3, but as detailed above, that proposition also does not assert this.\n\nPlease explain the experiments and results in Table 1.\n\nWhen discussing Figure 3, please address the fact that in many cases CAPIFY performs much better than ESCAPIFY, especially on the Adult dataset.\n\nPlease clearly indicate how the adversarial robustness is measured, and what the results are.\n\nIn the Appendix proof of Proposition 5.2, please explain the statement that $CF(v, do(S:=s’)) = \\ddot{v}_{s’}$. This seems incorrect, as the do-intervention can also impact other variables in the causal graph.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a framework for jointly learning a model that exhibits three important trustworthy properties: individual fairness, adversarial robustness, and causality. The proposed framework introduces a causal fair metric that encodes the causal structure and protected causal perturbation. This metric ensures that the similarity between individuals is both causally informed and sensitive to protected attributes. The authors then employ metric learning to train classifiers that are both causally fair and robust.

The majority of the reviews are favorable toward acceptance, though one reviewer raised several significant concerns. Below, I discuss the major issues raised.


> Concern #1. ‘The following claim (in the abstract) needs to be amended or removed: "To enhance the practicality of our metric, we propose metric learning as a method for metric estimation and deployment in real-world problems in the absence of structural causal models." The reviewer commented that while the SCM is not used in the metric learning framework, it is required to create distance-labeled training data for metric learning. In light of this, this claim is, at a minimum, highly misleading as written. 

I agree with this concern. However, I believe it comes more from a writing issue than an overstatement of the actual contribution. In Proposition 6.2, the authors formally prove that, without the ground-truth SCM, data-driven metric learning is infeasible. This proposition adequately highlights the framework’s inherent limitations. Additional evidence supporting this point is also presented in the paper.

At the same time, I agree that revisions are needed to address this gap clearly. I suggest the following revisions:

- In the abstract and introduction, clarify the gap, the impossibility proposition, and the proposed approximation.
- In Section 6, re-organize the discussion to emphasize these gaps explicitly.

> Q#2. In Section 6, the use of Proposition 6.3 is not justified or explained. In particular, the authors do not explain the relationship between Rademacher complexity and "discern[ing] embeddings regardless of dimension" and it is not at all clear what that relationship might be, or what exactly the authors mean by "discerning". Further, the claim that the Rademacher complexity is independent of dimension is not justified without additional evidence, as the norms in Equation 7 scale with dimension.

I agree with this concern. The scope of Proposition 6.3, derived from other papers, feels somewhat awkward in this context. As noted by the reviewer, this section requires revisions to better explain the relationships mentioned and provide the necessary justifications.


**Decision** 

After reviewing the concerns raised by the reviewer and considering feedback from other reviewers, as well as my own reading, I believe these issues, while valid, do not constitute substantial over-claiming. Most concerns arise from wording or narrative ambiguities rather than fundamental errors in the methodology or contributions. Therefore, I recommend minor revisions, focusing on improving the clarity and organization of key sections.
Decision: Accept (poster)