Path-specific effects for pulse-oximetry guided decisions in critical care
Authors: Kevin Zhang
Yonghan Jung
Divyat Mahajan
Karthikeyan Shanmugam
Shalmali Joshi
Venue: NeurIPS 2025 poster ()
Keywords: Causal inference
Computational healthcare
Algorithmic fairness
Primary area: social_and_economic_aspects_of_machine_learning

Abstract: Identifying and measuring biases associated with sensitive attributes is a crucial consideration in healthcare to prevent treatment disparities. One prominent issue is inaccurate pulse oximeter readings, which tend to overestimate oxygen saturation for dark-skinned patients and misrepresent supplemental oxygen needs. Most existing research has revealed *statistical disparities* linking device measurement errors to patient outcomes in intensive care units (ICUs) without causal formalization. This study *causally* investigates how racial discrepancies in oximetry measurements affect invasive ventilation in ICU settings. We employ a causal inference-based approach using *path-specific effects* to isolate the impact of bias by race on clinical decision-making. To estimate these effects, we leverage a doubly robust estimator, propose its self-normalized variant for improved sample efficiency, and provide novel finite-sample guarantees. Our methodology is validated on semi-synthetic data and applied to two large real-world health datasets: MIMIC-IV and eICU. Contrary to prior work, our analysis reveals minimal impact of racial discrepancies on invasive ventilation rates. However, path-specific effects mediated by oxygen saturation disparity are more pronounced on ventilation duration, and the severity differs across datasets. Our work provides a novel pipeline for investigating potential disparities in clinical decision-making and, more importantly, highlights the necessity of causal methods to robustly assess fairness in healthcare.

Path-specific effects for pulse-oximetry guided decisions in critical care
Kevin Zhang +-f Columbia University New York, USA
Yonghan Jung'
University of Illinois Trbana-Champian Champaign, USA
Divyat Mahajan Mila & Université de Montréal Montreal, CA
Karthikeyan Shanmugam Google DeepMind Bengaluru, IN
Shalmali Joshi Columbia University New York, USA
Abstract
Identifying and measuring biases associated with sensitive attributes is crucial consideration in healthcare to prevent treatment disparities. One prominent issue is inaccurate pulse oximeter readings, which tend overestimate oxygen saturation for dark-skinned patients and misrepresent supplemental oxygen needs. Most exist- ing research has revealed statistical disparities linking device measurement errors to patient outcomes in intensive care units (ICUs) without causal formalization. This study causally investigates how racial discrepancies in oximetry measurements affect invasive ventilation in ICU settings. We employ causal inference-basee approach using path-specific effects to isolate the impact bias by race on clinical decision-maie cecision-making.... To estimate these effects, we leverage doubly robust estimator, propose its self-normalized variant for improved sample efficiency, and provide novel finite-sample guarantees. Our methodology is validated on semi-synthetic data and applied to two large real-world health datasets: MIMIC-IV and eICU. Contrary to prior work, our analysis reveals minimal impact of racial discrepancies on invasive ventilation rates. However, path-specific effects mediated by oxygen saturation disparity differs across datasets. Our work provides a novel pipeline for investigating poten- tial disparities in clinical decision decision-making and, more importantly, highlights the necessity of causal methods to robustly assess faimess in healthcare.
pronounced ventilation duration, and the severity
Introduction
Bias in medical devices can perpetuate disparities by impacting healthcare decisions. For example, pulse oximeters tend to overestimate blood oxygen saturation for dark-skinned patients [45]
50 43
21 [5]. Such discrepancies can lead to "hidden hypoxemia," where a patient's 's true arterial oxygen saturation (SaO,) is dangerously low, despite reassuringly higher peripheral SpO,, reading from the oximeter. Figure demonstrates this using eICU data [4)] where red dots ('H.H') highlight instances where low SaO, (< 88%) is masked by higher SpO, (> 88%). Such hidden hypoxemia can result underestimated supplemental oxygen needs and delayed clinical interventions (0
Prior studies have advanced our understanding of pulse oximeter inaccuracies and associated patient outcomes. For instance, Sjoding et al. 455 and Wong hidden hypoxemia, and adverse outcomes like increased organ dysfunction and mortality in large
(50] established links between race,
Currently MIT. Contributions made when affiliated with Columbia University.
"oomesponding authors: (kzhang02@mit. edu, shalmali. www.aaai.oohshooooliie.o edu). 1 'Contributions made when affiliated with Purdue University.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Table 1. Comparison of relevant methods for path-specific causal analysis in ICU settings. (v: Primary focus; Part.: Some elements addressed; x: Not primary focus.)
22.30
Neern 1.H..
Causal Analysis
Path-Spec.
Multi- ICU Finite
Study
Analysis
Med. Data
Guar.
90
Se02 (%)
45
Assoc.
x
X
50
Assoc.
X
14
Part.
Part.
X
X
73
30
V
X
70
70
85
(%)
90
Density
47
V
V
X
X
X
Figure Distribution of SpO, and SaO₂ in eICU data. Samples with hidden hypoxemia are colored red, following Matos et al. (28
49
V
V
X
Ours
Similarly, Gottlieb et al. [4] examined racial differences in oxygen delivery rates using regression and simple mediation analysis in MIMIC-IV, while Henry et al. 18 and Fawzy et al. [12 reported delays in hypoxemia detection and treatment eligibility. These studies rely statistical associations or canonical mediation analysis to highlight measurement bias. However, they fail to isolate causal pathways of disparities mediated only through oximeter discrep- ancies in the presence of other mediators, which requires more advanced causal mediation analysis.
ICU datasets (e.g., MIMIC-IIIIV (2
Measuring fairness using path sath-specific causal analysis an active research area These methods isolate the influence of sensitive attributes, such as race, mediated through intermediate factors. In our context (Figure 2), we aim to quantify the causal effect of race on invasive ventilation specifically mediated by pulse oximetry discrepancies (V), which is distinct from the effect via other mediators (W). complex systems with multiple mediators, techniques for identifying and robustly estimating effects along specific pathways have been developed [4] 30
49
However, Table I lillustrates several key gaps. While sophisticated path-specific methods exist, their applications to analyze racial bias in ICU settings are limited, with most applications in population health survey data [64] [3.. Furthermore, there no finite-sample guarantees for path-specific effect estimators, which are crucial for generating robust evidence with potentially constrained sample sizes, like in critical care. Thus, a framework applying path-specific causal analysis for ICU problems with robust estimation strategies and theoretical backing is necessary.
To address these identified gaps, this paper makes the following primary contributions.
1. We propose a causal pipeline to detect and quantify racial disparities in the ICU. Adapting a standard fairness model (Figure 2), use path-specific analysis to isolate the impact of race mediated through oximeter discrepancies (V), called the V-xpecific Direct Effect (VDE).
2. We present robust estimators for the VDE, including a doubly robust estimator, and propose a self- normalized variant for improved sample variance, with novel finite-sumpll learning guarantees.
3. We apply our framework to analyze race-bat base- disparities mediated by oximeter discrepancies invasive ventilation (rate and duration) in MIMIC-IV [22] and eICU 440
To the best of our knowledge, this is the first formal path- specispecific causal analysis of pulse oximeter- mediated bias invasive ventilation using multiple mediators.
2 Preliminaries
We introduce preliminary notation and background in observational causal inference and fairness.
Notation. We use (X, X, I.) to denote random vector, variable, and their realized values, respectively. For function f(z), we use Σ, fx) to denote the summation//nteeration over dis- crete/continuous Z. For discrete vector X, we denote 1[X x] as indicator function such that

---

Table 2. Causal effects considered in this work. In addition to the Total Effect (TE) and Natural Direct/Indirect Effect (NDE/NIE), our applica- tion motivates the need for path-spectee effect through V (VDE).
Causal effect
Causal query
W
W
TE 39
52
E[ E|Y
(a)
(b)
Figure 2. Causal diagrams for the modified standard fairness model with two mediators W and V. For any other associations between the mediators, the VDE is not identifiable.
NDE 33 52
B W Von! E((20)
NIE 33 52
B|Y,, - L[ [Ve,,, W V=0
VDE (Ours)
E E|2,, - TE Wa.
1[X = 1f = x and 0 otherwise. P(V) denotes a distribution over V and P(v) a probability at V We let [[](V) and V[f(V)] denote the mean and variance of f(V) relative to P(V), and ||l| a ( as the Ln-norm of f with P. We use f f = (r.)) if f is consistent Or(rn) ) if f - f bounded in probability at rate rn-
estimator of f having rate Tn. and f We will say f is Lz-consistent if [I f|] P = (1(1) and f- Op(1) if ʃ is bounded in probability. Let D A (V::i= 1,...,,,, ,n) denote a set of n samples. The empirical average of f(V) over D is denoted Ep Ep]f(V)] & (1/ID)) Env.eD f(V.).
Structural Causal Models. We use structural causal models (SCMs) 36 as our framework. An SCM M is quadruple M (U, V, P( where U is set of exogenous (latent) variables following a joint distribution P(U), and V is a of endogenous (observable) variables whose values determined by functions F {fv; \v,ev such that V. fv; (pay,, uv;) where Pav; S V and Uv, S U. Each SCM M induces a distribution P(V) and a causal graph g = G(M) over in which directed edges exist from every variable in Pay; to V; and dashed-bidirecter arrows encode common latent variables. An intervention is represented using the do-operator, do(X = x), which encodes the operation of replacing the original equations of X (i.e., fx (pax x ux) )) by the constant for all X X and induces an interventional distribution P(V do(x)). For any Y S V, the potential response {u) is defined as the solution of Y in the submodel MM given U u. which induces a counterfactual variable Yx.
2.1 Related work
33 operationalizes computational method- ologies to identify, measure, and address disparate behavior related to (automated or other) decision- making processes. One prominent line of work evaluates differential performance of machine learning models across subpopulations defined by any sensitive attribute, and proposes interventions to equalize performance or enforce specific conditional independence assumptions, such as equalized odds or opportunity [16), demographic parity (7]. multicalibration [17], individual fairness [10], etc. Causal fairness is a complementary framework that assumes an underlying causal data-generating
Algorithmic Fairness.
Algorithmic fairness [16
Principal fairness uses causal inference enforce conditional independence of the sensitive attribute given counterfactual outcomes [20) and has been applied to coronary artery bypass grafting treatment allocation [53]
mechanism to operationalize fairness for predictive models 19 26
52 39
While algorithmic fairnsss frameworks have been used in healtheare [ 29 38), their downstream
utility has been low due to the systematic nature of statistical biases present in the data 4 51 46 32
broad class of causal effects that measure the influence of a treatment X on an outcome Y through specific causal pathways 37
Path-specific Effects. Path-specific effects
44 The
Total Effect (TE), defined as TE(I1, e EE|Y do(x;)] E|y | do(zo)), captures the influence transmitted through all causal paths connecting the treatment and the outcome. The total effect is decomposed into the direct effect and the indirect effect, where the latter is an effect mediated through intermediate variables. For example, in the scenario depicted by Fig. 2a the Natural Direct Effect E|y.,l. which captures variation in Y
(NDE) (36 of X on Y is NDE(21, mo) A E[Y,,,w.,, Wey v= if X changes from 2g 21, while the mediators W and V hypothetically remain the same. The Natural Indirect Effect (NIE) is NIE(21, 20) & E[Y,,1 E[Y,, W.,, V. J. This quantity represents the change in outcome due to the mediators shifting in response to the change in X from 2o to 21-
3

---

Much of the literature has focused on estimating NDE and NIE under single-mediate settings (e.g., 47 IT). Motivated by the presence of multiple mediators, contemporary research on path- specific analysis has broadened its scope to investigate effects transmitted along particular, often complex, paths within multi-mediator systems. For instance, effects acting through the colored path in Fig. 2a might be characterized by the nested counterfactual quantity EE a..o...e... Veo. in statistics and algorithmic faimess [49 30 522 6 g) Specifically, [30] developed robust estimator for measuring path-specific effects transmitted through a particular mediator in the presence of other mediators.
3 Pulse-oximetry bias using path-specific analysis
We define path-specific effects within the standard fairness model (SFM), illustrated by the DAG in Figure 2 which represents broad class of clinical problems. In our setup, X is a binary indicator outcomes, and Z denotes the pre-admission statisties. The mediator W represents post-admission measurements, while the mediator V includes variables related to oxygen saturation, such as the peripheral oxygen saturation reading from pulse oximeter SpOz- the more accurate arterial blood gas measurement (SaO2), and the discrepancy A SpO, SaO We examine two primary outcomes: (1) the rate of invasive ventilation, defined as the proportion of patients who receive any invasive ventilation procedure during their ICU stay, and (2) the duration of invasive ventilation, measured as the length of the first invasive ventilation event.
of race, Y represents ventilation-relate
We study the direct effect of X on V which then affects Y, termed the V-specific Direct Effect (VDE);
VDE(21, E EY,,, E|Y,,,
(1)
This path-specifie effect, which isolates the influence mediated through V given Wa, (the mediators W as they would be under exposure T1), corresponds to estimands discussed in prior work (e.g.. Miles et al. 30) Within our SFM, we specifically term this as the VDE. As an example, suppose 25 = White, . Black, and Y is the rate of invasive ventilation. The VDE captures the difference in expected invasive ventilation rate (Y) if post-admission measurements (W) and the ventilation decision were made if the patient were Black (x1), while oxygen saturation readings (SpO2- SaO,, A) were measured as if the patient were White (zo). This effect isolates treatment rate heterogeneity due to discrepancies in oxygen saturation measurements. The SFM in Figure captures capturess plausible associations between the mediators W and V in which the VDE is identifiable (see proof in Appendix A). Table 2 contextualizes and compares all causal effects of interest within our framework.
3.1
Estimating path-specific effects
This section presents identifiability conditions for the VDE (Eq. (I) and then develops its doubly robust estimator. Our approach integrates key insights from Miles et al. (30), who addressed similar
path-specific effects often under parametric assumptions, and Jung et al. (25), who provided a
general -arammemettic framework for complex estimands involving conditional expectations. By
synthesizing these, we develop a doubly robust VDE estimator that, crucially, does not require parametric modeling. This allows for the flexible use of machine learning methods to estimate these functions, which improves robustness to model misspecification. Under the SFM in Figs. (2a 2b). the VDE is identifiable as follows.
Proposition (Identifiability (3)). Under the SFMs in Fig. 2 the VDE E(Y E.Y.I. Vop.
identifiable and is given as follows: E[y2, = Σ. E
31, 2 P(z) and
E|Y Way
Σ W.O.3
B|y 11, w. v, 2) Po| ro, w. P(w x1, 2()))
(2)
We focus on estimating the nested counterfactual term of the VDE, E(Y,, Ven. represented in Eq. (2) since estimating E[YZ,) using the back-door adjustment (35 is well-known.
To construct the estimator, we will parameterize Eq. using the following:
Definition 1 (Nuisance Parameters (0)). Two sets of nuisance parameters ph 2 [Hd. Hē: mD) and Th [mpp _2 oo: T3) are defined as shown in the following table, where jig (V, W, AB à P.( zo, 2).
II, Z) and

---

Regression Parameters 140
Importance Sampling Parameters To
VV,V W,W, 2)
- y V, W, X,
(V W, W,
P.V]zo.W.2) [[mai P(V P(X
A Eip
W, X
m X, 2)
P(W]z 1X=mo P(WIX,2) P(X[2)
1b(X, ((,2)) 2)
X, /
(X, Z)
[[==ı]
P(X 2)
Then, the VDE in Eq. Lemma (Parametrization)
2 can be parametrized as follows.
Eq. 2 E|y के (V, W, W,X, = Ely (21, Z)]
Blu mb). for 1,2,3.
(3)
Among all parametrizations of Eq.
we use the one with the double robustness property:
(( V, W, X, To)
nv W, x, Z) rV(V, W, X)
(4)
XX (W, () W, 21,,, 2) Po(W, ,X, 2)}
(5)
+ (X () :0: Z) - -m0(X, 2)} + (11, 2).
(6)
The functional Ip is valid representation since (F VV, W X, Z); Po: TO)] Eq. 2 Moreover,
exhibits the following robustness property:
Lemma 2 (Double Robustness Property). Let V = (Y, W, X, X,Z). Z). For any arbitrary vectors μt, ग, the functional satisfies the following double robustness properly:
Ely(VV po, no) Bip(Vi Σ Or PbllellT -
Eq- 7 shows debiasedness because whenever fi', # converges to pb. má with rate n-144 then the error in yp converges at much faster n 11/2 rate. We construct the following doubly robust estimator.
Definition 2 (Doubly Robust VDE Estimator). Given a sample 'D S.i.d. P, the doubly robust VDE estimator is constructed as follows:
-Sample-Slliting) Take any L--olld random partition for the dataset D & (V1,"" Va); ie. D = =D where the size of the partitioned dataset De is equal to n/ L..
2. (Learning by Partitions) For each E = 1,2,.... ... I,, construct the estimator iul a using D \ Dr for 1,2,3 3 and compute Th 2 [xY, V, W, X, x, Z); R) #]
3. (Aggregation) The one-step estimator 3 is average of ( 3 i.e. & e + +... (Z).
Following the analysis from [Theorem 25 we provide novel finite- sample learning guarantees: Theorem 2 (Fimite-Sample Analysis). Suppose iits pb < oo and O < # mi < oo almost surely for i = 1,2, Suppose the third moment of & exists. Let R₁ & (1/L) El (Ex, [&d] Erlpo)). Let
A A V[po]. Let Ni & E|lip! ). Let (2x) denote the standard normal CDF. Then,
L 3
サー = RA + I
ElAt
MoHa #HHl.
(8)
(=1
im1
where, with probability greater than c,
Lp
L
L L poll2
R1 <
+
and
(9)
VL Vnpo
1
L Re
Poll
0.4748k
Pr
Ri
((x)
S
(10)
2
Pi vn
1=1
5

---

Theorem 2 demonstrates that the error can be decomposed into two terms. The term R1 closely approximates standard normal distribution variable, and the remaining term exhibits the doubly- robustness behavior. This is formalized in the corresponding asymptotic analysis.
Corollary 2 (Asymptotic error). Assume pà: P2 oo and 0 < Tे= #े < oo almost surely. Suppose jig and Rig are lzz-consistent. Then,
- R1
Op(ilit mbll 11 -221),,
L
1=
and VnRi converges in distribution to Normal(O, (k, Pk.0)
To estimate E ft, we can rewrite the nuisance expressions using Bayes' rule to avoid computing densities in high dimensions. However, estimation can still be challenging because propensities in the denominator may cause the variance to explode. Since E ETT]] = 1, we can improve the estimation stability using seff-normaiization (SN) by setting #5N f- #$ E J. This technique is known to reduce the variance [19]. The self-normalized estimator for the NDE/NIE is outlined in Appendix B Theoretical results follow analogously for the self-normalized doubly-robon robust estimator.
4
Experiments
We evaluate our proposed canonical and self-normalized doubly robust estimators for computing the fairness effects Table across synthetic, semi-synthetic, and real-world settings. Like most causal inference literature, we validate methodology using synthetic and semi-synthetie data, which provide access to the true data-geneg mechanisms and counterfactual outcomes.
We then apply our methodology to two real-world ICU datasets to estimate the path-specific effect of race (specifically mediated by discrepancies in pulse oximetry measurements) on the rate and duration of invasive treatment. Note that unlike previous studies which examine patient outcomes, we study decision recision-making regarding invasive ventilation procedures in the ICU.
For all experiments, we perform sample-splitting with L = 5 folds and clip propensities within the range j,, e]. where 10-1 All propensity and outcome models are trained using XGBoost. Details on hyperparameter selection are provided in Appendix D and code credits are in Appendix L
4.1 Synthetic data
In this experiment, we demonstrate the finite-sample behavior of the canonical and self celf-normalized doubly robust estimator. We validate convergence of causal estimands using a linear synthetic setting, where Z and W are
continuous vectors.
We generate samples from a linear SCM with randomly initialized weights, where Z, are dimensional, W is 10-dimensional, and X, Y are single-dimensional. These dimensionalities are chosen to be similar to the real-world data. We estimate all causal queries for computing fairness effects in T namely: [Y.... E| [E[Y and E[Y Via,,.Va, J. To show the convergence, we vary the sample size from 1,000 to 32,000, which roughly matches the number of eICU samples.
In Figures 3a and [3b] report the mean and 95% confidence interval of the relative error for each causal query across 100 bootstraps using the standard doubly robust (non-SN) and self-normalized (SN) estimator. While both estimators converge to the true causal quantities, the errors without self-normalization exhibit significantly higher variance. In Appendix D we demonstrate that this difference primarily occurs because the empirical mean of the nuisance parameters deviates from one.
4.2 ICU cohort selection
To conduct cemi-syntheticrreal-weeee world experiments, we use two large, publicly available critical care datasets: the eICU Collaborative Research Database (eICU) with ICU admissions from hospitals across the continental U.S. [40], and Medical Information Mart for Intensive Care (MIMIC-IV), with ICU data from the Beth Israel Deaconess Medical Center in Boston 22

---

Synthetic continuous queries (non-SN)
Synthetic continuous queries (SN)
50
Relative error (%)
Relative error (%)
23
-25
-50
-75
-100
K
2000
1000 Sample Size
10000
1000
2000
8000 Sample Size
3000
16000
(a)
(b)
Semi-synthetic eICU queries (non-SN)
Semi-synthetic eICU queries (SN)
300
Relative error (%)
250
B|G
M))
Relative error (%%
200
BI
150
100
-10
-20
K[Ya E[YY,M
-50
-100
-30
RXY
1000
2000
900M
2000
16000 32000
1000
2000
4000
16000
32000
Sample Size
Sample Size
(d)
Figure Convergence the relative error of causal queries using the canonical (left column) and self-normalized (right column) doubly robust estimator on a continuous synthetic setting (top row) and semi-syuthetic eICU data (bottom row). The plots show the mean 95% confidence interval over bootstrap iterations. The estimates using nofmaaamalzatoon exhibit significantly lower variance.
We interested in investigating the causal effect of race on invasive ventilation rate and duration mediated by oxygen saturation discrepancy. If a patient undergoes multiple intubations over the course of a single ICU admission, only the first considered. We treat gender, age, and comorbidity scores as confounders, representing patient characteristics available before ICU admission and prior to any clinical procedures or measurements. Comorbidity is quantified using the Charlson Comorbidity Index and the pre-ICU OASIS score, both of which summarize baseline health status.
The mediators W consist of post-admission measurements: arterial blood gas (ABG) results excluding SpO, and SaO,, laboratory values such as creatinine and lactate, and periodic vital signs (e.g., temperature, respiratory rate). The V mediators include any SpO₂ and SaO, values within the range (70,, 100%). as well as the measurement discrepancy A = SpO, SaO,. For computing A, we match each SpO₂ reading with an SaO; measurement taken within the next five minutes if available.
Note that our mediators include not only the discrepancy, but also SaO₂ and SpO because race influences the discrepancy through oxygen saturation measurements. Although V includes patient variables that are not intrinsically unfair, incorporating all oximetry-related variables captures the mechanism by which racial bias in pulse oximetry mediates clinical decisions more comprehensively.
To summarize measurement trajectories over time, we compute an exponentially weighted average from ICU admission up to the time of first ventilation (or discharge if the patient was not intubated). For sequence of observations Iı, En taken .1,...,4 minutes before the first ventilation or discharge, the aggregated value is 7 Σ, w.sts/ Σ, Wi where w, exp(-2(4) is an exponentially decaying weight and 7 is a smoothing parameter. We restrict the analysis to stays lasting at least 24 hours and with least recorded measurement of SpO,, SaO,, and a matched discrepancy value. The final cohort consists of 37,222 admissions from eICU and 4,897 admissions from MIMIC-IV. Figure 4 presents the distribution of ICU stays by race, ventilation status, and treatment duration. Details on the distribution of pre-admission severity scores are included in Appendix
4.3 Semi-synthetic eTCU experiments
We create a semi-synthetic cohort by leveraging real-data patterns to construct synthetic SCM. Semi-synthetic data allows us to evaluate our methodology more realistic setting while retaining access to ground-trun truth effects for error analysis.
7

---

1.0
Vent
eICU With Vent
eICU Density
MIMIC No Vent
0.5
Number of Patients
MIMIC With Vent
0.0
10a.
White
MIMIC-IV Density
Black
0.5
Hispanic
Asian
10a
0.0
White
Black Hispanic Asian Race
10a
10a
10
Duration of invasive ventilation (hrs)
(b)
Figure 4. Distribution of samples in eICU and MIMIC-IV datasets, showing (a) number of patients by race and ventilation status and (b) invasive ventilation duration, stratified by race. The baseline duration of ventilation higher for eICU patients, and is associated with greater pre-admission severity (see Appendix(C)
To generate the cohort, we use the real-world eICU dataset to train separate XGBoost models to predict each variable in the set {X, W, V, Y) given its observed parents. In the semi-symmen semi-synthettc setting, is binary variable indicating whether the patient received invasive ventilation procedure during the stay. These trained models serve as the true causal mechanisms when generating samples. We use the eICU dataset due its larger sample size compared to MIMIC-IV.
Following the setup in the synthetic experiment, we compute the relative error for E|Y2, E[Y,, M.,,,,M,,J. ). and E[ across varying sample sizes in Figures 3e and 3d Similar to the synthetic experiments, our proposed self-normalized variant exhibits sigmificanlly lower variance in a healthcare- -egobccare-grounded setting with complex relationships between variables. This behavior is consistent with results using MIMIC-basee semi-synthetic data, which we include in Appendix G Additionally, we compare our estimators with state-of-the of-the-art effect estimation baselines in Appendix] and demonstrate their robustness imbalance between Zy and 21 subpopulations Appendix
Results on semi-synthetic data support our approach over other methods in Table Most prior work only allow for single mediator, which could be applicable in other ICU settings, but is limited for our pulse oximetry application because relevant patient variables beyond oxygen discrepancy must be taken into consideration for valid and unbiased estimates of fairness. While Miles et [30] use multiple mediators, their estimator is conceptually analogous to our con-sef-noormalieee variant, which exhibits high variance in small-sample settings like ours. Based on these considerations, we adopt the normaliedd estimator for quantifying path-specific effects in real-world data.
4.4 Real-world eICU and MIMIC-IV experiments
We estimate the effects in Table 2 using demographic of the cohort) and I1 Black. Additionally, we compute the NIE if we entirely ignored W and denote this effect as NIE This alternative effect allows for comparison with existing research, as prior studies have quantified the effect of race mediated by measurement discrepancy without accounting for other post-admission mediators (denoted as W in our setup). For example, Gottlicb et al. [44] used NIE in the context of pulse oximetry to quantify the effect of the discrepancy oxygen supplementation levels, but did not account for the impact of other potential mediators on the outcome. In contrast, our methodology explicitly models these mediators using W.
White as the baseline (reflecting the majority
We compute all causal effects over 500 bootstrap iterations and report the mean and 95% confidence intervals. Figure the effects for the rate and duration of ventilation. A positive-values valued effect indicates that the corresponding pathway contributes to more frequent or longer invasive ventilation treatments for Black patients relative to White patients. We include the results using the canonical doubly robust estimator in Appendix H All of our analysis and conclusions are consistent between the neff-normalized and the canonical doubly robust estimators, with some slight differences in the effect magnitudes.

---

Duration of ventilation (hrs)
20
Rate of ventilation (%)
MIMIC-IV
2.41 4.21
1120
-085
.0.21
0.46
15
-2
10
-3
3
-3
eICU
-1.1
MIMIC-IV
13.6 19.0
6.2
74
21.0
0.4
11.5
2.3
7.4
-6
TE
NDE
NIE
NIE
VDE
TE
NDE
NIE
NIE
VDE
(b)
Figure 5. Average causal fairness measures across 500 bootstraps using the self-normalized estimator for the (a) rate and (b) duration of invasive ventilation on eICU and MIMIC-IV data. Colors represent different effects and the numerical label for each bar indicates the mean across all bootstraps. Error bars show 95% confidence intervals. Positive values indicate higher rate or longer duration of invasive ventilation for Black patients relative to White patients.
Analysis of ventilation rates. Figure suggests that the overall total effect varies across the two datasets, which indicates baseline practice differences. While the negative TE indicates baseline ventilation rates are higher for White patients compared to Black patients, these differences do not indicate unfairness without studying the decomposed effects. A negative NDE in eICU versus positive NDE in MIMIC-IV suggests direct discrepancy adjusted only for baseline patient confounders, but not ICU-specific patient condition. Much of the total variation in both eICU and MIMIC- IV is dominated by the NIE, suggesting potential differences in White vs. Black patient health conditions influencing ventilation rates. The VDE for invasive ventilation rates for both eICU (-0.21 percentage points, 95% CI [-.23 to -0.19]) and MIMIC-IV (-0.46 percentage points, 95% CI [-0.6 to -0.36] datasets is relatively small, indicating low unfairness mediated by oxygen saturation discrepancy. In addition, the NIE* shows that ignoring W mediators for this problem can exacerbate the estimated scale of unfaimess and potentially flip the direction of disparity.
Analysis of ventilation duration. Figure shows significant differences the duration of ventila- tion between eICU and MIMIC-IV, which is largely indicative of baseline distribution shifts in both ICU datasets (see Appendix C). The flipped signs of NDE specify that across multicenter eICU data, Black patients are ventilated for longer, while in MIMIC-IV, White patients are ventilated for longer, when adjusting for pre-admission covariates. The NIE is the primary pathway influencing the total effect, however the effect severity drastically shifts when considering only subset of mediators (i.c. NIE*), which suggests the need decompose NIE further to assess VDE. We observe slightly longer durations of invasive ventilation among Black over White patients in eICU (2.3 hrs, 95% CI [2.2 to 2.5]) and significantly longer durations in MIMIC-IV (7.4 hrs, 95% CI [6.9 to 7.9]), mediated by pulse oximetry discrepancies. The MIMIC-IV bias in duration is broadly in line with
reported differences levels of end-of-life care by race 24 233 but requires further investigation to attribute observed VDE differences meaningful causes.
To better understand how oxygen discrepancy influences the VDE, we present the effect conditioned on both the discrepancy A and the SpO2 value. Training a regression model for the VDE proved challenging due to small sample sizes in certain regions; therefore, we compute averages over discrete binned intervals of A and SpOz. The conditional effects are shown in Figure 6
While MIMIC-IV samples are sparse, the eICU conditional VDE suggests that under hypoxemia (SpO, < 85% and A 2 0%), higher rates of ventilation tend to occur for Black compared to White patients. However, the effect flips when 85% < SpO, S 90% and 10% S A < 20%. The conditional VDE for ventilation duration is relatively small and uniform when there is little oxygen discrepancy (AA) < 10%). Meanwhile, Black patients experience significantly longer ventilation times when 90% < SpO, < 95% and 10% < A < 20% (hidden hypoxemia) in both datasets. This pattern also persists when 80% < SpO, S 90% and 20% S A S -10% (moderate hypoxemia), but the effect flips in the region 90% < SpO, < 95% for eICU samples.

---

Rate of ventilation (%)
Duration of ventilation (hrs)
A Sp02 Sa02 (%)
30
20
10
eICU
0.5
A SpO2 Sa02 (%)
20
30
10
0
eICU
10
-10
0.0
-10
-20
-20
-30
-30
30
20
10
MIMIC-IV
-1.0
MIMIC-IV
0
-10
-10
-20
-13
-20
-30
-30
80
100
70
75
85 90 100
SpO2 (%)
SpO2 (%)
(a)
(b)
Figure 6. VDE conditioned on the discrepancy A and SpO: the (a) rate and (b) duration of invasive ventilation on eICU and MIMIC-IV data. Due to small sample sizes certain regions, we take the average VDE over binned intervals of (A, SpOz). Regions with fewer than 20 units are shown in gray.
5 Discussion
In this work, we introduce the V-specific Direct Effect (VDE) to quantify racial biases in critical care decision-making that mediated by measurement discrepancies from pulse oximeter devices. While previous studies have documented disparate outcomes for Black patients, our approach is the first to apply path-specific causal framework to examine heterogeneity by race in clinically actionable healthcare process. We develop a doubly robust estimator and self-normalized variant for the VDE, provide favorable finite-sample guarantees, and demonstrate strong empirical performance on both synthetic and semi-synthetie health data.
In two publicly available ICU datasets, we find negligible disparities in invasive ventilation rates indicating slightly less frequent treatment for Black patients relative to White patients mediated by oxygen saturation discrepancy. Both datasets also show longer treatment durations for Black compared to White patients with different effect severities. We consistently observe that the canonical NIE exacerbates the magnitude of disparity while the NIE may potentially flip the direction. This highlights the necessity of our path-specific causal framework to accurately examine fairness.
For assessing clinical recision-making. our findings indicate that bias arising from pulse oximetry measurements primarily affects the duration of invasive ventilation rather than initiation of treatment. This pattern suggests that clinicians may effectively integrate additional patient information beyond oxygen saturation when deciding whether to ventilate. Alternatively, the bias may manifest through pathways not captured within our current framework, such as through unmeasured variables or outcomes. Understanding the mechanisms underlying such bias remains an active area of research.
Limitations. As with most causal inference methods, our framework relies on a specified causal graph and thus depends on assumptions about the data-generating process that can be challenging to verify in practice. Consequently, it shares common limitations related to potential misspecification of the causal structure. For example, if there is strong reason to believe that W and V have latent common cause in given application, any estimated path-specific effect for the VDE would be inaccurate because the true effect is not identifiable.
Our causal graph reflects generally plausible mechanisms by ensuring clinically relevant quantities reported in the datasets are modeled, while making the minimal assumptions required for identifiability of the causal effects of interest. The graph explicitly captures these underlying assumptions, which is an important advantage highlighted in critical care literature (27].
We only consider the first invasive ventilation and do not explicitly model temporality in our analysis. Additionally, sparse SaO, measurements limit the frequency of matched oxygen saturation discrep- ancy measurements, further affecting sample size. However, we emphasize that this a limitation method.
with the dataset and not inherent
Broader Impact. Several clinical problems can be framed using the multiple-mediation analyses. Our contributions are intended to provide practical algorithm for computing particular path-specific effects more broadly. The methods provide more nuanced understanding of heterogeneity current healthcare practices.
10

---

Acknowledgements
KZ. and SJ acknowledge partial support from Google Inc. DM acknowledges support via FRQNT doctoral scholarship (https://doi org/10. 69777/354785 for his graduate studies. The authors thank Jack Gallifant and Leo Celi for preliminary discussions, which helped inform the problem setup. Any opinions, findings, conclusions, or recommendations in this manuscript are those of the authors and do not reflect the views, policies, endorsements, expressed or implied, of any aforementioned funding agencies/instiutions.
References
[1] Chen Avin, Ilya Shpitser, and Judea Pearl. Identifiability of path-specific effects. In Proceedings of the 19th International Joint Conference on Artificial Intelligence, pages 357-363, 2005.
[2] Solon Barocas, Moritz Hardt, and Arvind Narayanan. Fairness and machine learning: Limita- lions and opportunities. MIT Press, 2023.
[3] Jay Chandra, Miguel Ángel Armengol de la Hoz, Gwendolyn Lee, Alexandra Hope Lee, Patrick J Thoral, Paul W. G. Elbers, Hyung- Chul Lee, John Munger. Leo Anthony Celi, and David A Kaufman. A novel vascular leak index identifies sepsis patients with higher risk for in-hospital death and fluid accumulation. Critical Care, 26, 2021.
[4] Irene Y Chen, Emma Pierson, Sherri Rose, Shalmali Joshi, Kadija Ferryman, and Marzyeh Ghassemi. Ethical machine learning in healthcare. Annual Review of Biomedical Data Science, 4(1):123-14,, 2021.
[5] Richard J Chen, Judy J Wang. Drew FK Williamson, Tiffany Y Chen, Jana Lipkova, Ming Y Lu, Sharifa Sahai, and Faisal Mahmood. Algorithmic fairness in artificial intelligence for medicine and healthcare. Nature Biomedical Engineering, 7(6):719-742, 2023.
[6] Silvia Chiappa. Path-specific counterfactual faimess. In Proceedings the AAAI conference on artificial intelligence, volume 33, pages 7801-7808, 2019.
[7] Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data, 5(2):153-163, 2017.
[8] Juan Correa, Sanghack Lee, and Elias Bareinboim. Nested counterfactual identification from arbitrary surrogate experiments. In Advances in Neural Information Processing Systems, volume 34, pages 6856-6867, 2021.
[9] Amanda Coston, Alan Mishler, Edward H Kennedy, and Alexandra Chouldechova. Counter- factual risk assessments, evaluation, and fairness. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pages 582-599 593, 2020.
[10] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, pages 214-226, 2012.
[11] Helmut Farbmacher, Martin Huber, Henrika Langen, and Martin Spindler. Causal mediation analysis with double machine learning. The Econometrics Journal, 2020.
[12] Ashraf Fawzy, Tianshi David Wu, Kunbo Wang. Matthew L Robinson, Jad Farha. Amanda Bradke, Sherita H Golden, Yanxun Xu, and Brian T Garibaldi. Racial and ethnic discrepancy in pulse oximetry and delayed identification of treatment eligibility among patients with covid-19 19. JAMA Internal Medicine, 182(7):730-338 2022.
[13] Thomas A Glass, Steven N Goodman, Miguel A Hernán, and Jonathan M Samet. Causal inference in public health. Annual Review of Public Health, 34(1) 115-755 2013.
[14] Eric Raphael Gottlieb, Jennifer Ziegler, Katharine Morley, Barret Rush, and Leo Anthony Celi. Assessment of racial and ethnic differences in oxygen supplementation among patients the intensive care unit. JAMA Internal Medicine, 182(8))849-88) 2022.

---

[15] Sicheng Hao, Katelyn Dempsey, João Matos, Christopher E Cox, Veronica Rotemberg, Judy W Gichoya, Warren Kibbe, Chuan Hong, and An-Kwok Ian Wong. Utility of skin tone on pulse oximetry in critically ill patients: prospective cohort study. Critical Care Explorations, 6(9): el 133, 2024.
[16] Moritz Hardt, Eric Price, and Nathan Srebro. Equality of opportunity in supervised learning. In Advances in Neural Information Processing Systems, volume 29, 2016.
[17] Ursula Hébert-Johnson, Michael Kim, Omer Reingold, and Guy Rothblum. Multicalibration: Calibration for the (compulationall--ddeniiiia masses. In The 35th International Conference on Machine Learning, pages 1939-1948. PMLR, 2018.
[18) Nicole R Henry, Andrew C Hanson, Phillip J Schulte, Nafisseh S Warner, Megan N Manento, Timothy J Weister, and Matthew A Warner. Disparities in hypoxemia detection by pulse oximetry across self-identified racial groups and associations with clinical outcomes. Critical Care Medicine, 50(2):204-211, 2022.
[19] Tim Hesterberg. Weighted average importance sampling and defensive mixture distributions. Technometries,
1995.
[20] Kosuke Imai and Zhichao Jiang. Principal fairness for human and algorithmic decision-making. Statistical Science, 38(2):317-328, 2023.
[21] Haya Jamali, Lauren T Castillo, Chelsea Cosby Morgan, Jason Coult, Janice I. Muhhammad, Oyinkansola O Osobamiro, Elizabeth C Parsons, and Rosemary Adamson. Racial disparity in oxygen saturation measurements by pulse oximetry: evidence and implications. Annals of the American Thoracic Society, 19(12):151--164 2022.
[22] Alistair Johnson, Lucas Bulgarelli, Tom Pollard, Steven Horng, Leo Anthony Celi, and Roger Mark. Mimic-iv. PhysioNet, 2020.
[23] Kimberly S Johnson. Racial and ethnic disparities in palliative care. Journal of Palliative Medicine, 16(1 1):1:13: 1334, 2013.
[24] Rehecca W Johnson, L. Kristin Newby, Christopher B Granger, Wendy A Cook, Eric D Peterson, Melvin Echols, Wanda Bride, and Bradi B Granger. Differences in level of care at the end of life according to race. American Journal of Critical Care, 19(4):335-343, 2010.
[25] Yonghan Jung, Jin Tian, and Elias Bareinboim. Unified covariate adjustment for causal inference. In Advances in Neural Information Processing Systems, volume 37, pages 6448-6499, 2024.
[26] Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. In Advances in Neural Information Processing Systems, volume 30, 2017.
[27] David Lederer, Scott Cameron Bell, Richard D) Branson, James D) Chalmers, Rachel Marshall, David M Maslove, David E Ost, Naresh M Punjabi, Michael Schatz, Alan Robert Smyth, Paul W Stewart, Samy Suissa, Alex Adjei, Cezmi A Akdis, Élie Azoulay, Jan Bakker, Zuhair K Ballas, Philip G Bardin, Esther Barreiro, Rinaldo Bellomo, Jonathan A Bernstein, Vito Brusasco, Timothy Buchman, Sudhansu Chokroverty, Nancy A Collop, James D Crapo, Dominic Fitzgerald, Lauren Hale, Nicholas Hart, Felix J. F. Herth, Theodore J Iwashyna, Gisli Jenkins, Martin R. J. Kolb, Guy B Marks, Peter J Mazzone, Randall Moorman, Thomas M Murphy, Terry L Noah, Paul R Reynolds, Dieter Riemann, Richard E. K. Russell, Aziz Sheikh, Giovanni Sotgiu, Erik R Swenson, Rhonda D Szezesniak, Ronald Szymusiak, Jean-Louis Teboul, and Jean Louis Vincent. Control of confounding and reporting of results in causal inference studies. guidance for authors from editors of respiratory, sleep, and critical care journals. Annals of the American Thoracic Society, 16:22-28 2019.
[28] João Matos, Tristan Mirko Struja, Jack Gallifant, Luis Filipe Nakayama, Maric Charpignon, Manomou--Zavlanee Jaime S Cardoso, Kimberly S Johnson, Nrupen A
Xiaoli Liu, Nicoleta Bhavsar, Judy Wawira Gichoya, Leo Anthony Celi, and An-Kwok Ian Wong. Bold: Blood-gas and oximetry linked dataset. Scientific Data, 11, 2024.
[29] Vishwali Mhasawade, Yuan Zhao, and Rumi Chunara. Machine learning and algorithmic fairness in public and population health. Nature Machine Intelligence, (8)):59-666) 2021.
12

---

[30] Caleb H Miles, Ilya Shpitser, Phyllis Kanki, Seema Meloni, and Eric Tchetgen Tchetgen. On semiparametric estimation of aath-specific effect in the presence of mediator-outcome confounding. Biometrika, 107:159- 2017.
[31] Shira Mitchell, Eric Potash, Solon Barocas, Alexander D' Amour, and Kristian Lum. Algorithmic fairness: Choices, assumptions, and definitions. Annual Review of Statistics and its Application, 8(1):141-163, 2021.
[32] Hamed Nilforoshan, Johann D Gaebler, Ravi Shroff, and Sharad Goel. Causal conceptions of fairmess and their consequences. In The 39th International Conference Machine Learning, pages 16848-16887, 2022.
[33] Luca Oneto and Silvia Chiappa. Fairness in machine learning, pages 155-196. Springer, 2020.
[34] Xiaxian Ou, Xinwei He, David Benkeser, and Razieh Nabi. Assessing racial disparities in healthcare expenditures using causal path-specific effects. CoRR, abs/2504.21688, 2025.
[35] Judea Pearl. Causal diagrams for empirical research. Biometrika, 82(4):669-711 1995.
[36] Judea Pearl. Causality: Models, Reasoning, and Inference. Cambridge University Press, 2000.
[37] Judea Pearl. Causal inference. Causality: Objectives and Assessment, pages 39-58, 2010.
[38] Stephen R Pfohl, Agata Foryciarz, and Nigam H Shah. An empirical characterization of fair machine learning for clinical risk prediction. Journal of Biomedical Informatics, 113:103621, 2021.
[39] Drago Plečko and Elias Bareinboim. Causal fairness analysis: causal toolkit for fair machine learning. Foundations and Trends in Machine Learning, 17(3):304-58,, 2024.
[40] Tom J Pollard, Alistair W. Johnson, Jesse Raffa, Leo Celi, Roger Mark, and Omar Badawi. The eicu collaborative research database, a freely available multi-center database for critical care research. Scientific Data, 5(1):1-13, 2018.
[41] Alvin Rajkomar, Michaela Hardt, Michael D Howell, Greg Corrado, and Marshall H Chin. Ensuring fairness in machine learning to advance health equity. Annals Internal Medicine, 169(12))666-87, 2018.
[42] Halley Ruppel, Spandana Makeneni, Jennifer A Faerber, Meghan B Lane-Fall Fall, Elizabeth E Foglia, Michael L O' Byrne, and Christopher P Bonafide. Evaluating the accuracy of pulse oximetry in children according to race. JAMA Pediatrics, 177(5):540-55)5 5430, 2023.
[43] Chunhu Shi, Mark Goodall, Jo Dumville, James Hill, Gill Norman, Oliver Hamer, Andrew Clegg, Caroline Leigh Watkins, George Georgiou, Alexander Hodkinson, Catherine Lightbody Elizabeth, Paul Dark, and Nicky Cullum. The accuracy of pulse oximetry in measuring oxygen saturation by levels of skin pigmentation: systematic review and meta-analysis. BMC Medicine, 20(1):267, 2022.
[44] Ilya Shpitser and Eli Sherman. Identification of personalized effects associated with causal pathways. In Conference on Uncertainty Artificial Intelligence, volume 2018, page 198, 2018.
[45] Michael W Sjoding, Robert P Dickson, Theodore J Iwashyna, Steven E Gay, and Thomas Valley. Racial bias in pulse oximetry measurement. The New England Journal of Medicine, 383 (25):2477-2478. 2020.
[46] Harini Suresh and John Guttag. A framework for understanding sources harm throughout the machine learning life cycle. In Proceedings of the 1st ACM Conference Equity and Access in Algorithms, Mechanisms, and Optimization, pages 1-9, 2021.
[47] Eric Tchetgen Tchetgen and Ilya Shpitser. Semiparametric theory for causal mediation analysis: efficiency bounds, multiple robustness, and sensitivity analysis. Annals of Statistics, 40: 1816 1845, 2012.
13

---

[48] Willem van den Boom, Michael Hoy, Jagadish Sankaran, Mengru Liu, Haroun Chahed, Mengling Feng, and Kay Choong See. The search for optimal oxygen saturation targets in critically ill patients: Observational data from large icu databases. Chest, 2020.
[49] Tyler WanderWeele and Stijn Vansteelandt. Mediation analysis with multiple mediators. Epi- demiologic Methods, 2(1):95-115, 2014.
[50] An-Kwok lan Wong. Marie Charpignon, Han Kim, Christopher S Josef, Anne A. H. de Hond, Jhalique Jane R Fojas, Azade Tabaie, Xiaoli Liu, Eduardo MiireeesCaaboder Leandro Car- valho, Rishikesan Kamaleswaran, R. W. M. A. Madushani, Lasith Adhikari, Andre L Holder, Ewout Willem Steyerberg. Timothy G Buchman, Mary Lough, and Leo Anthony Celi. Analy- sis of discrepancies between pulse oximetry and arterial oxygen saturation measurements by race and ethnicity and association with organ dysfunction and mortality. JAMA Network Open,
4. 2021.
[51] Haoran Zhang, Harvineet Singh, Marzyeh Ghassemi. and Shalmali Joshi. Why did the model fail?: Attributing model performance changes to distribution shifts. In Proceedings of the 40th International Conference Machine Learning, pages 41 41550-41578, 2023.
[52] Junzhe Zhang and Elias Bareinboim. Fairness in decision-making the causal explanation formula. In AAAI Conference on Artificial Intelligence, 2018.
[53] Linying Zhang, Lauren R Richter, Yixin Wang, Anna Ostropolets, Noémie Elhadad, David M Blei, and George Hripesak. Causal fairness assessment of treatment allocation with electronic health records. Journal of Biomedical Informatics, 155:104656, 2024.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We propose to apply path-specific analysis assess the fairness of treatment decisions in the ICU and propose a self-normalized estimator motivated by healthcare- specific challenges.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss potential limitations of our analysis in Section
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer: [Yes]
Justification: All assumptions and proof' are included in Appendix AA] including conditions for con-identifiabilityy of causal effects. Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: All results are obtained using publicly available health data. All code will be made publicly available GitHub. The code has been submitted supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
16

---

The authors should provide scripts to reproduce all experimental results for the new reproducible, they Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: All datasets are publicly available for academic research. The code is submitted as supplementary material with a README and environment file, and will be made publicly available upon publication.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/mtpss.cc/pullc/giidde htapo//ii/gguudee///oeieeeeeeeee
sssssssssiiyyyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data etc.
proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permirted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All experimental design details outlined in Section 4 and the hyperparame- ter selection described in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Statistical significance of all results demonstrated using bootstrapping with confidence intervals presented for all results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
17

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CL, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: All compute resources are outlined Appendix K
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer:
[Yes]
Justification:
The research adheres the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Broader impacts of this work have been summarized in Section 5
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
18

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This work does not pose high risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All data, prior code, and models have been credited citations and/or GitHub links (where applicable) and summarized in Appendix L
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
19

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/dataseta
latasets.paperssit
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The only new asset is code, which has been submitted as supplementary material and will be open-sourced upon publication.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
This work does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This research is a retrospective analysis of health data and does not constitute research with human subjects or involve crowdsoureing.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
20

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
This method does not involve LLMs in any component.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should not be described.
21

---

A Proofs
A.1 Identifiability of VDE in Figures 2a 2b
Figures
2a
2b imply the following conditional independences between counterfactual variables:
Veo.w -L Wo, Z,
(11)
W2 X | Z
(12)
Vi,w IL W, X Z
(13)
Yay00 Vo Z, Wz,
(14)
These conditional independences lead the following derivation:
Vao.
Σ [E Wa Vao. (x))(z)
E| Voo.w U, Wa, w. =W, = =, W, = =
E| VaO.w U, Wa, w, 20 PVV v Wz, P(w) w 2())
E[Yar
Voo.w = Way = w. :| zo, w; P) 21, P()))
E|Y U1 U. w. P(u æo. w. P(w Φ1, 21. P()),
2,00,
where
1. Since Wa 1L X Z, we have P(W,, 2) P(w] [1, 2).
2. Since Va 11. W Z, have P(V W =1 w1,2) P(V.o.u 2).
3. Since Vz IL W, X Z, have P(V ((Vz,.w 2) P( To, 2)
4. Since Y Y,,,0 I1 Vz
Z,, W Way have
E VaN.w U, W w,
E EY,,,. Y Y
W., w,
do v), w, = E x1 U, w. 2].
A.2 Non-identifiability of VDE in Figures
aa 7b
Z
(a)
(b)
Figure 7. Causal diagrams for the modified standard fairness model with two mediators W and V with latent common causes that render the green path-specific effect unidentifiable.
Whenever there exist unmeasured confounders between W and V, the VDE is not identifiable. Specifically, we can write the counterfactual nested term P(Y,, Viggwa, y) as follows using the
22

---

identification algorithm proposed by g
P(Y = y) = Σ P(Y.N = y, Voo =v)
(15)
Σ
P(Y. = y, Van = U, Wa1 = w)
(16)
0,30
Σ
P(Y = y, Vioiw, =; Wz1,= = =. = 2, Z = =)
(17)
1,
Σ
= P(Y = =) = U, W = = = 2).
=
(18)
This reduces the problem to identifying P(Veg,u, = U. W Wer.== w). By [Theorem 3 in 8), this term
is not identifiable because of inconsistency (where X = 0 in the counterfactual w., while X = 1 in W ).
A.3
Proof of Lemma 1
We note that
,,(,, 2) e El W. 21, Z) W,X, Z]
(19)
Σ 1e (v. W x1, P) | W,X,2)
(20)
Σ
2P W,
(21)
and
16(X, Z) - Elug W 3o 2) 1 Z]
(22)
= H.ww, Fo. Zw, P) (( 22
(23)
Σ
B|Y U. 21, 2) w, 2o. P)P(W) X,
(24)
Therefore,
(1,, 2) = Eq.
(25)
Furthermore,
E|-o(V, W, x, 2) = Bl W, X, (V W, W,X,20 XX, 2)]
(26)
Σ
1 (v. 11, 2) P(v zo, w. 2) P(w P(s)
(27)
Eq.
(28)
Also,
Σ r xr 2) P(w æι x), 2o)
(W, X, ( xx, (0,,,,
P(w,,,,2
(29)
P(w x, 2) P(x 2)
H (w, xo, 2) P(w ( P())
(30)
w,z
E u, w, x1, 2) P(v No, P(w 21, 2)
(31)
Eq.
(32)
Finally,
( (X, 2)] = Blng(11, 2)] = Eq. 2
(33)
This completes the proof.
23

---

A.4 Proof of Lemma 2
We will first use the following helper lemma:
Lemma 3 (Helper Lemma). For any functional (W), and (a(W) b(W), for any
WC V, the following holds:
(bo (W) - b(W) + (oW))((W)))))) 0o (W))
(34)
[oaao - al a(W)) - (o (W))]-
(35)
Proof. Define F(W) as function satisfying the following equation:
Fla(W))((W) (lla((Wbb) (W)] F(W) + (a (W) al ((W))(b(W) bo (W))) (36)
We will omit W for notational convenience for now. The above equation shows that
F = Ejab agdo (au aa bu)]
(37)
Elab agb + agbo + ab abo agdol
(38)
E[2ab - agb abo].
(39)
Then,
Blab- F| = Blagb + abs - abj = E[a((W b) + aob].
(40)
By the definition of F, we have
Elab- F] Elagbo] E[(ao a (WW))(b(W)) bo (W))
(41)
This completes the proof.
Based on this helper lemma, for i = 1,2,3, 3, we have
EiT fug n²) + Thp² TomOl Blimg yaa -m]]..
(42)
Then, consider
[ ود² + TUp To
(43)
=F(T (Y 11² + -003 t2
(44)
= =El( (8) (=) 1
(45)
where the equation holds since E[TT/a] = vho _ Eq. (2) and by the law of the total expectation.
Next, define 2 - p6(r3] & Elux(( W,,1, W X, Z] for any fixed y². Then,
(1² - ,²) + - T
(46)
(1 W. ,1, -)--²² + m0147 E(T
(47)
Ellp² - p2] m²]].
(48)
Next, define n A pblu?]] A Elu² (W, 20, Z) | X, Z| for any fixed μ²² Then,
Eii [ut 1t } + Tbp -
(49)
Elm (1² (V, W. 31 7) - - + mbn²) Bimdrt]]
(50)
Ellp' - (m] - -].]
(51)
Combining,
(V, -V (V, 2),, W, 2) v/o
(52)
+ Bia (W,x,
X, Z)} -/,,),
(W,
E(Tr3
E
(53)
+ E(T (X, (1 (W, In, 2)
n² (X, (x) - E|Tbu!]
(54)
3
Σ Ellu' H67TT x *
(55)
24

---

Then.
B[ W. X, (V, W, X, 2) = B(m X, Z) (W. X, x,2)],
(56)
since
B T (W x, 2W 2]]
(57)
x, 2) (V. W,
( [
(58)
=E 2e,, w,z, 2) m x. 2) 2) 2(,,2,2)
(59)
( w. 21, 2) P(u w, 2) P(w| I1, 2)1 xo) P(w,,,=)
Σ
(60)
P(w x 2 2)
Σ P(U w, up 2) P(w x1,2))(22)
(61)
0,00
Σ (e, w. x, 2) 1(r x1) P(u P( 2) P(u 2,
w. 2o, 2)
P(e | w. 2, P(W 2(x) 2))
(62)
2,
Blu (V X, 2) m0 (V, W, 2)]
(63)
Therefore, E(THu²) in the first term and -B[T in the second term can be canceled out.
Furthermore,
WW = (x Z)},
(64)
since
E 2x, 2)] Blu Z)]
(65)
= E[EB/ (W, zo. 2)] x1, 2
(66)
ΣΣ (w, No, 2) P(w x1, 2((2)
(67)
Σ (w, x, 1[x xol P(w x1,2) P(w, P(w x. P(x
(68)
w,
El (W, X, 2)F² 1² (W X, 2)]-
(69)
Therefore, [[] in the second term and BEiT uc] in the third term can be canceled out. Therefore, we can conclude that
E (V, X, (V, WVW,,,,))
(70)
+ B(n x, (V W,,, 2) -) (W,X. X, Z)}
(71)
+ EiT (x, 2), (W,,,,)) - (x. + Z) do
(72)
(73)
Σ Bl - Hbb - =
(74)
This completes the proof.
A.5 Proof of Theorem
2
A.5.1 Proof of Eq. 8 in Theorem
Let V (Y, V, W, X, Z}. We note that the doubly robust estimator given as
き T, 1=1
1
Σ ED, (s( (V; pae 7.)]-
(75)
25

---

Then.
L v0
(76)
ED, (e tz)l EPlp(V No:
(77)
1
[D. (p)
(78)
T.
L
+
(Ep; - ()) <(V: mb)
(79)
L
+
(V; - To)]
(80)
t=1
Define
L EE. - Er)lp(V p. R)).
R
Eq.
78 +
Eq.
79
(81)
1=1
Then, with the proof of Lemma 3 we observe that Eq.
8
holds.
A.5.2
Proof of Eq. (9) in Theorem
Let
VV (4: To),
(82)
2c(V) & @VV; Fúg #t).
(83)
We first study the term (ED Ep))oo(( By Chebyshev's inequality,
P((IBD-P)) BP) (V)]] 4
(84)
where n 2 |D]. Equivalently,
1 p
[x) > x1)=
(85)
17 n
or equivalently,
1 0/1
P([ - [p) 54) >
(86)
77
By [Lemma D.3 in 25 we have
1 L (V) Φol V)I
P([(ED, (V) -oo))]]]])] 12) <
(87)
t₂2
By [Lemma D.4 25 we have
T. (V) (V)
P
(Ex, Ex)lo oo(V]] -V) 5 21 Σ
(88)
t2
E=1
Choose
Pd
4
(89)
2
LIl do (V)
f2
(90)
n
26

---

Then, with probability greater than 1 - e,
R1 S 1
(91)
E 2
L
LIlo, (V) do(V)
V 2
+
(92)
2
1=1
4 LI (V) - 6 (V)13
Pd
V 2
+
(93)
A.5.3 Proof of Eq. 10 in Theorem 2
From the previous proof, we have
(RD - BV) - (o
21 56)21-
Σ
L
V)
6o (V)
(94)
12"
71
1=1
By choosing
L (V) - 6 (V)
t2
(95)
we have
7 Σ
1 (ED (V) [V)
with probubility 1 -
1 L (Ve
2 (V)l
(96)
72
t=1
Define
A (ED Er]( (V)],
(97)
B 1
(ED. Ey) Φe (V) VV(V)
(98)
L
C
(ED, EV) 500 (V)Il-
(99)
L ( - 60 (V)
A
(100)
"
1=1
Here,
Rí B.
(101)
Then,
wpl-c
- P(A
A).
(102)
Then,
-
(103)
=|P(A< A) 4
A) + (x A) 4(z)
(104)
P(A<< A) 6(x
A)
8 (z + A) +(x))
(105)
0.4748
+ A) (<))
[Prop. D.1 in 25
(106)
Pivn
0.4748
1 (a |
(mean-value theorem)
(107)
Piva
0.4748k
1
(108)
pn
V2T
This completes the proof.
27

---

B Self-normalized estimators for the NDE and NIE
We focus on estimating the counterfactual term for the NDE and NIE, E[Y,, Wap Vag. since estimating B| and E[ using the back-door adjustment 35 is well-known. Let M {W, V be the combined set of mediators. The counterfactual quantity identifiable in Figure 2 using the formula:
E| Mo
Σ m,
21, m, =m zo. P(z).
=
Following the recipe in (25], the expression for vo B[Y Maol can be parameterized using the following nuisance parameters:
Regression Parameters 140
Importance Sampling Parameters
(M(M,
Y|Y
M, X, Z]
m X, Z)
P(M 2) 1[x=1] P[ P(X
16(X,2) 2)
Elpg
X, Z]
(X Z)
=|== P(XIZ)
where 1ng M6(M, 21, Z).
The doubly robust estimator for tho
+ E| 1331 L PbJ E[ag].
We estimate the nuisance parameters using the same sample-spliting procedure as for the VDE. To estimate #' E A, rewrite the expressions using Bayes' rule avoid computing high- dimensional densities. Since [[[ 1, the self-normalized variant uses AsN + # #'/En[#").
C Additional data analysis
All analysis is restricted adult patients. better understand the distribution of patients in our datasets, we analyze baseline characteristics and pre-admission severity scores across datasets and racial groups. F
the distribution of age and sex, stratified by race. To assess pre-ICU severity, we use two measures: the OASIS score and the Charlson Comorbidity Index. Both are integer-based scores, with higher values indicating greater clinical severity. These distributions are shown in Figure
We observe that the distribution patient severity remains consistent across different racial groups. Moreover, baseline patient conditions tend to be more severe in the eICU dataset, which explains the longer ventilation times in Figure 4b
Projortion
0.10
9.65
0.0
MIMIC
0.6
0.10
MISIC-IV
ProporUuu
0.4
0.00
620
31.23
26-30
31-23
M6-40
11-43
46-30
51-33
56-40
01-43
66-78
71-73
76-80
82-03
30
Female
Male
Figure 8. Distribution of patient baseline characteristics, (a) age and (b) sex, the eICU and MIMIC-IV datasets, stratified hy race.
28

---

0,15
White
0.4
Black
Proper Mooe
Hllppanncc
ettu
0.05
0.00
0.0
0.15
MHARC-I Progeettin Eoon
0.10
0.05
0.00
5
10
0.0
Charbon Comorbidiity Indlxx
pre-iUu OASIS Score
(a)
Figure 9. Distribution of pre ade-ssmmssonn severity scores, (a) Charlson Comorbidity Index and (b) pre- ICU OASIS Score, in the eICU and MIMIC-IV datasets, stratified by race. The eICU cohort exhibits a slightly higher pre-admission severity score.
D Hyperparameter selection
In this section, we outline our approach for selecting the hyperparameters of the XGBoost models compute E[Y,,, 21,Wa, and E[ E[,,,,V=,,,,, For both the synthetic and semi-synthetic settings, we generate random dataset consisting of 32,000 samples and perform a separate grid search for each propensity model and each outcome model. For the real-world setting, we perform the hyperparameter search using the eICU and MIMIC-IV datasets.
The grid search optimizes three hyperparameters: the number of estimators from the set {20,50, 50, 100, 200). the maximum tree depth from the set {3, 5, 6), and the fg-regularization penalty from the set {0.5, 1, 5]. using 5-fold cross-validation For propensity models, we select the hyperparameter configuration according to the Brier score, which evaluates prediction calibra- tion. For outcome models, we select the configuration using the mean squared error. The chosen hyperparameters remain fixed across all bootstraps and sample sizes throughout the corresponding experimental setting.
Table 3. Selected hyperparameters for each model used in the real-world ICU experiments. Each tuple represents the best (tree depth, number of cstimators, fg-reuularization) respectivoly. For the nested regression models in the bottom four rows, we specify the computed effect in parenthesis to distinguish between regression parameters for different estimates.
Vent, Rate (eICU)
Vent. Rate
Vent. Dur.
Vent. Dur.
Model
(MIMIC-IV)
(cICU)
(MIMIC-IV)
p(X
2)
(3, 20, 5)
(3,20, 5)
(3, 20, 5)
(3, 20, 1)
P(X
W,
(3,20,
(3,20,
(3,20,5) 5)
(3, 20, 1)
P(X
V, Z)
(3,20,5)
(3,20,
(3,20,2
(3, 20, 2)
P(X
W,W,W, 2)
(3, 20,5)
(3,20,5) 5)
(3, 20, 5)
(3, 20, 5)
E| 20. V, V.
(3,50, 5)
(3,20,5) 5)
(3, 20, 2)
(3, 20, 5)
EY
1T, V,
(3,20,
(3,20,
(3, 20,50,
(3, 20,5)
20,
W, V, V
(3,20,
(3,20
(3, (3,20,5) 5)
(4, 5)
B
ø1, W, V,
(3,20, 5)
(3, 20,
(3, 20,2)
(6, 20, 5)
E L
L0, W, 2] (VDE)
(4, 200, 5)
(4, 200, 2)
(3, 200, 1)
(3, 200, 5)
1;; 2] (VDE)
(3, 20,
(3,20,
(3, (3,20,5)
(3, 20, 5)
2o. Z] (NDE/NIE)
(4, 20,
(3, (3,20,5
(3, (3,20,2)
(3, 20, 5)
1
2o. Z| (NIE*)
(4, 20,
(3,20,5)
(3, 20,
(3, 20, 5)
29

---

E VDE nuisance parameter analysis
We demonstrate that the improved variance of our proposed self-normalized estimator, shown in Figures can be attributed to better nuisance estimates when the sample size small. In Figure TO we present the mean and 95% confidence interval across 100 bootstraps for the empirical average of the VDE nuisance parameters #², A², and A!.
As the number of samples increases, the empirical mean converges to the true value of onc. For smaller sample sizes, the nuisance parameters tend to be large, which causes greater variance in the standard doubly robust estimator. In contrast, parameters so that the empirical average is one, which reduces the variance the estimation. In the semi-synthetic setting, the nuisance parameters exhibit slow convergence small bias due to propensity clipping.
self-normalized variant scales the nuisance
Synthetic continuous nulsance (VDE)
Seml-synthetlc eICU nulsance (VDE)
90
Empirical mean
Empirical
E
B
1000
2000
4000
8000
16000 32000
1000
2000
4000
2000
16000
32000
Sample Size
Sample Size
(b)
Figure 10. Convergence the empirical mean of the VDE nuisance parameters in the (a) synthetic setting with continuous variables and the (b) semi-synthetic experiment based on eICU data. The dashed horizontal line indicates the theoretical mean for the nuisance parameters.
Synthetic binary setting
In addition to the synthetic experiment with continuous variables, also analyze synthetic setting with singleton binary variables. This experiment aims to demonstrate the convergence of causal effects and nuisance parameter estimates. By choosing simplified setting, we illustrate that when the nuisance parameters are accurate, the standard and normaaordl estimators produce approximately identical results.
We generate data according to the following model, where all observed variables take singleton binary values and the unobserved variables are Ux, Uxz, Uw, Uv, Uy ~ N/ 1):
Z = 1[xxz > 0.2),
X 1[Z + Ux +xz > 0.2].
W = =1[X 7 U- > 0.8),
1[X Z W Uv > 008),
Y = 1 7 2(W - -V) Uy 0.2].
We estimate the causal queries necessary to compute fairness effects: E|Y,,), E|Y,, E,Y,, M. J. Like the synthetic setting with continuous variables, we vary the sample size
and [E[Y,,, ,, V., w., V.o, from 1,000 to 32,000.
We report the mean and 95% confidence intervals across 100 bootstraps the relative enror on each causal query and the empirical mean of the nuisance parameters in Figure II As the sample size increases, our estimates converge to the true causal quantities. Due to the simplicity of the singleton binary variables in this experiment, the empirical mean of the nuisance parameters is always close to one. Consequently, the estimates obtained with the self-normalized estimator are nearly identical to those shown in Figure Ila
30

---

8000 Synthetic binary queries (non-SN)
Synthetic binary nuisance (VDE)
Relative error (%)
1.00
1.00
Empirical
B|Xa]
1.02
-2
1.00
Sample
Sample Size
(a)
Figure 11. Convergence of (a) the relative error of causal queries using the canonical estimator and (b) the empirical mean of the nuisance parameters. Results for the self-normalized estimator are omitted as they nearly identical to (a), because the empirical mean of the nuisance parameters is close one.
G Semi-synthetic MIMIC-IV results
We show the experimental results for our estimators semi-synthetic data based on patterns in the MIMIC-IV dataset. Following the setup in the semi-synthetic eICU setting, we train XGBoost model to predict each variable in the set (X, W, V, given its observed parents in the real-world data. We fix Y to be a binary variable indicating whether the patient received an invasive ventilation procedure during the stay.
We report the mean and 95% confidence interval across 100 bootstraps of the relative error for each causal query across varying sample sizes in Figure [2] Like for the semi-synthetic eICU setting, we observe that our estimands approximately converge to the value, however, the convergence is n1 Veoway using the canonical doubly robust estimator.
slow for (Y
Semi Remt-synthetie MIMIC-IV querles (non-SN)
Semi-synthetic MIMIC-IV queries (SN)
Relative error (%)
15000
MYY
60
B|YJJ
[Ym
10000
E[
Relative error
40
BI
3000
20
-3000
-10000
-13000
1000
2000
4000
16000
32000
1000
2000
4000
8000
16000
32000
Sample Size
Sample Size
(a)
(b)
Figure 12. Convergence of the relative of causal queries using the canonical and (b) self- normalized estimator on the semi-synthetic MIMIC-IV data.
H Canonical doubly robust estimator results on real-world data
We compute the causal effects in Table 2 juiing the canonical doubly robust estimator. We report the mean and 95% confidence intervals for the rate and duration of invasive ventilation in Figure 13 The VDE for invasive ventilation rates for eICU (-0.21 percentage points, 95% CI [-0.23 to 191) and MIMIC-IV (-0.47 percentage points, 95% CI [-0.7 to -0.36]) datasets is relatively small. Moreover, the VDE indicates slightly longer ventilation durations for Black relative to White patients in eICU data (2.4 hrs, 95% CI [2.2 to 2.5)) and significantly longer durations MIMIC-IV data (8.1 hrs, 95% CI [7.5 to 8.7]).
The direction and statistical significance of all causal fairness measures obtained using the self- normalized estimator (Figure 5) and canonical estimator are in agreement. Furthermore, we observe that the canonical estimator typically predicts wider confidence intervals across all effects, which consistent with our experimental results in the synthetic and semi-synthetic settings.
31

---

Duration of ventilation (hrs)
eCCU
Rate of ventilation (%)
MIMIC-IV
MIMIC-IV
22.4 4.21
3.553
-8.86
00.21 -.47
-2
-3
-4
-1.
13.0
20.5
0.5
22.4
12.3
8.3
-6
TE NDE
NIE
NIE
VDE
TE
NDE
NIE
NIE
VDE
(a)
Figure 13. Average causal fairness measures across 500 bootstraps using the canonical doubly robust estimator for the (a) rate and (b) duration of invasive ventilation on eICU and MIMIC-IV data. Colors represent different effects and the numerical label for each bar indicates the mean across all bootstraps. Error bars show 95% confidence intervals. Positive values indicate higher rate or duration of invasive ventilation for Black patients relative to White patients.
I State-of-the-art comparison
We compare our proposed canonical doubly robust estimator and its self-normalized variant against several state-of-the-arr and baseline methods. To the best of our knowledge, prior work directly targets the VDE beyond the canonical estimator. Therefore, we include the following (somewhat misspecified) baselines: (1) the X-learner, which estimates the ATE without accounting for mediation; (2) single-mediator estimator, which treats W and V as a single merged mediator (i.e., NIE" from Section 4.4 and (3) the nested regression estimator, corresponding μo in Equation 6
Table 4 presents the comparison VDE prediction errors. Our results show that the self-normalized estimator consistently achieves the lowest error while maintaining valid 95% confidence intervals. Although the nested regression estimator performs similarly on semi-synthetic data, its confidence intervals are overly narrow. State-of-the-arr methods for similar effect estimation tasks, such as the X-learne, have large error in all settings since they do not account for the correct mediation structure.
Table 4. Comparison our canonical doubly robust estimator (Figures Bal Be and self nelf-normalized variant (Figures 36 3d] with state-of-the-art and baseline methods. We report the mean and standard error over 100 bootstraps, where each iteration samples datasct size 32,000. Relative errors are shown in parentheses, except for the semi-synthetic settings, where the true VDE is very close to zero and relative стror is not informative. Entries in bold denote the best performing model for each datasct.
Synthetic binary
Synthetic continuous
Semi-synthetic eICU
Semi-synthetic MIMIC
Ground truth VDE
-0.125
-0.163
0.000
0.001
X X-learer
0.364 (292%) + 0.001
0.352 (217%) + 0.001
-0.007 0.000
0.029 = 0.000
Single-mediator doubly robust
0.033 (26%) + 0.001
0.311 (192%) 0.008
0.024
0..027 = 0.004
estimator 47 49
Nested- regression
0.001 (0.5%) +: 0.001
0.024 (15%) + 0.002
0.004 0.001
00.001 + 0.000
estimator (Eq.))
Canonical doubly robust estimator 30
00001 (0.5%) 0.001
0.006 (4%) +: 0.008
0.018:10.00
0.025 + 0.176
Self-normalized doubly robust estimator (ours)
0.001 (0.5%) + 0.001
(40) 0.006
0.003 0.007
0.004 + 0.004
32

---

Group imbalance experiments
In the real-world data, the eICU cohort comprises approximately 31,000 White and 4,000 Black patients, while MIMIC-IV includes around 4,000 White and 400 Black patients. These distributions reveal a substantial group imbalance between White and Black patients, which approximately preserved in our semi-synthetic experiments.
Although our self-normalized estimator demonstrates strong convergence properties despite this imbalance, we further examine its performance under varying degrees of group imbalance. Using the eICU-based and MIMIC-based semi-synthetic datasets, we control the imbalance by thresholding the predictions of X at different values to induce varying proportions of zo and æ1 subgroups. To # iz samples
We assess performance using the relative error quantify the imbalance, we define 1) of total samples
for each causal query in Figure 14
As expected, both the relative between the 2 and rj subgroups grows. Nevertheless, the overall variation in estimator performance across different values of 17 remains modest, with relative errors increasing by only few percentage points. These findings suggest that our estimator is moderately robust to the group imbalance present the real-world experiments.
and variance of the estimators tend increase the imbalance
elCU-based semi-synthetic queries
MIMIC-based semi-synthetic queries
B|Ga]
M[
Relative error (%)
B|
Relative error
-10
-10
EY EY,
EDY EDG.M
EJV E
-20
-20
1
(a)
Figure 14. Causal faimess estimates for (a) cICU-based and (b) MIMIC-based semi-synthetic datascts, computed using self-normalized estimator across varying levels of group imbalance E [0.1, 0.2, 0.33, 0.5)). Each point represents the estimate with 95% confidence interval computed over 100 bootstrap iterations, each sampling dataset of size 32,000.
K Compute resources
All models were trained on single NVIDIA RTX A6000 GPU, 32 CPU cores, 256GB of system RAM, running Ubuntu 22.04 with kernel 5.15. The synthetic and cemt-synthetic experiments compute 100 bootstraps for all sample sizes in two hours, while the real-world experiments require two hours to compute 500 bootstraps for each dataset and ventilation-base outcome.
Experiment assets
In experiments, available under the PhysioNet Credentialed Health Data License 1.5.0. Data processing is performed using the eICU-CRD Code Repository and the MIMIC Code Repository both under the MIT License. For eICU ventilation outcomes, we follow the data extraction procedure described in [48] For Charlson Comorbidity Scores in the eICU dataset, follow the process in 3 provided in the code implementation under the MIT License ? All the code and instructions for reproducing our experiments are available at
the eICU 44 v2.0 and MIMIC-IV 22 v3.1 datasets, both publicly
www.wwwwwwoo.......o..o.ww.ww.ee Lab/PSE Pulse-uxueeeee -xiisee-ximmtry
33


Reviews:
[{"review_id": "6rMFYdiWx2", "reviewer": "Reviewer_1gtf", "summary": "The manuscript describes a methodology to estimate path-specific causal effects in the presence of multiple mediators, with application to a fairness analysis in ICU data. In terms of the methodology, the authors develop a doubly robust estimator for the path-specific effect of a treatment on the outcome via one particular mediator. They provide a theoretical analysis of the properties of this estimator (identifiability, double robustness property, finite-sample behavior, asymptotic behavior) and verify its behavior in synthetic as well as semi-synthetic experiments. They then proceed to apply this estimator to the task of assessing whether measurement biases in pulse oximeters (which are known to produce mistakenly low measurements for Black patients at a higher rate than for white patients) have a specific effect on race-specific ventilation-related outcomes. This study is conducted in two ICU datasets, eICU and MIMIC-IV. The results indicate that following their methodology, the effect of these race-specific measurement discrepancies on ventilation outcomes appears to be small.", "questions": "- One thing left me a bit puzzled, but maybe I just missed a discussion of this somewhere? The authors estimate the V-specific effect, but V includes SpO2, SaO2, and the discrepancy between the two. In terms of the fairness question the authors are posing, it seems to me to make a crucial difference whether the authors are estimating an effect race -> SaO2 -> ventilation, or rather race -> (SpO2-SaO2) -> ventilation. The latter path would be unfair (and what the authors are really after, I believe) while the former path might be considered acceptable. Could the authors clarify this? E.g., which of these effects are the \"VDEs\" in Fig. 5?\n- Possibly related to my previous question: in the same figure (5b), it appears highly unlikely that the effect of the measurement discrepancy would be positive in one study and negative in another. Is this because - as indicated in my previous question - the VDE here is actually *not* just the effect of the measurement discrepancy? In any case, the result as presented here would seem to indicate that the uncertainty estimation probably yields highly optimistic bounds, no? (Assuming that the true effect of measurement discrepancies should have the same directionality in both studies, the confidence intervals should at least cover the null in both studies.)\n- In Fig. 3 top right panel, the variance actually does not appear to converge to 0, or at least *very* slowly. Is there a reason for this? Could the efficiency be improved somehow?\n- In the causal graph, why is race (X) affected by pre-admission statistics (Z)? That seems at least very curious.\n- The authors write that \"Algorithmic fairness [15, 28, 2, 30] operationalizes evaluation of differential performance of machine learning models across subpopulations\" - this is a very reductive description of the AF field. Not all AF conceptions are about subpopulations, and even of those not all are about differential performance. Some would say that AF also encompasses ethical questions beyond those that can be answered by looking at csv files with model predictions and \"ground truths\". (For one: what the authors are doing in this manuscript does not fall under the above description but would surely be classified as \"algorithmic fairness\".)", "limitations": "I would like to see the limitations of this approach presented more clearly, specifically the dependence on a rather large range of complex assumptions about the data-generating process that may be difficult to understand in detail and even more difficult to verify for a given application. One obvious risk is that people might misapply this method and conclude that something is \"fair\", when really it is not.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "The manuscript is very well-written, thorough, and dense. The development of a novel, doubly robust path-specific effect estimator in the presence of multiple mediators is an essential contribution and presented very thoroughly. The provided theoretical analysis and empirical experiments (confirming the validity of the theoretical results) further strengthen this contribution. Finally, the case study alone is already very interesting and well-conducted, and moves far beyond the prototypical standard bias case studies presented in many papers. The topic of the case study is also very timely; I believe the manuscript really adds something to the ongoing debate about these measurement discrepancies.\n\nI essentially don't see any major weaknesses - I just have some questions or very minor suggestions which I am leaving below.", "was_revised": true, "final_justification": "This is a technically solid and very well-written manuscript that includes a non-trivial novel problem framing, theoretical developments in terms of path-specific effect estimators, and a complex, timely, and relevant case study. The original manuscript was already strong, and additionally, the authors have thoroughly engaged with all the points raised by the reviewers, including conducting additional experiments. Hence, I am (still) recommending acceptance.", "rebuttal": "Thank you for your thoughtful review and for raising constructive points, which we address below. \n\n**Questions**\n1. In our analysis, $V$ includes $\\text{SpO}_2$, $\\text{SaO}_2$, and their discrepancy, which is inherently defined by the joint behavior of $\\text{SpO}_2$ and $\\text{SaO}_2$. Isolating a path solely through the discrepancy is not as meaningful in our setting because race influences $\\text{SpO}_2$ and $\\text{SaO}_2$ directly, and the discrepancy arises from their measurements, not from how it is mathematically computed. While the VDE may, as the reviewer noted, include components like $\\text{SaO}_2$ that are not inherently unfair, we include the full set of oximetry-related variables in $V$ to more comprehensively capture the mechanism by which racial bias in pulse oximetry mediates clinical decisions. We will clarify this modeling choice explicitly in the revised manuscript.\n\n###\n\n2. We have previously identified an issue in the hyperparameter search that incorrectly selected the number of estimators for the XGBoost model. After conducting a proper hyperparameter search across 64 settings for each propensity and regression model, both the eICU and MIMIC datasets show a **positive** VDE. Specifically, eICU has an effect of $0.8$ hrs ($95 \\\\%$ CI [$0.7$ to $0.9$]), while MIMIC shows an effect of $7.4$ hrs ($95 \\\\%$ CI [$7.2$ to $7.7$]). The results related to invasive ventilation rates are not meaningfully affected. We will update Figure 5b and the corresponding text in the revised manuscript to reflect the corrected estimates and clarify what the VDE measures.\n\n###\n\n3. Although the variance still decreases, the convergence rate may be slow because of the propensity score clipping. Nonetheless, we emphasize that the purpose of the self-normalized estimator is to mitigate variance compared to the canonical doubly robust version. Empirically, the behavior is better than one without self-normalization (see Figure 3a vs. 3b). Improving these properties further is certainly an active area of open research more broadly.\n\n###\n\n4. We agree that the variable $X$, representing race, should not be modeled as receiving a directed arrow from $Z$ alone. The more appropriate structure includes a hidden confounder between $X$ and $Z$, as demographic variables such as age may influence the distribution of racial groups. Importantly, introducing a bidirected arrow between $X$ and $Z$ does not affect our estimators or the theoretical analysis presented in Appendix A. We will revise the causal graph accordingly in the updated manuscript.\n\n###\n\n5. We agree and will expand the definition to the following: \"Algorithmic fairness operationalizes computational methodologies to identify, measure, and address disparate behavior related to (automated or other) interventions such as decision-making processes. One prominent line of work evaluates differential performance of machine learning models across subpopulations…\"\n\n**Limitations**\n1. We agree on the importance of clearly stating the assumptions underlying our approach. As with most causal inference methods, our framework relies on a specified causal graph and thus depends on assumptions about the data-generating process that can be challenging to verify in practice. Consequently, it shares common limitations related to potential misspecification of the causal structure. For example, if there is strong reason to believe that $W$ and $V$ have a latent common cause in a given application, the path-specific effect is unidentifiable. While our setup is more general than has been considered in typical causal fairness frameworks and more appropriate for studying bias in pulse oximetry applications, we will clarify these assumptions more explicitly in the \"Limitations\" section of the manuscript. This includes both assumptions specific to our ICU application in Section 4, and more generally, the limitations of potential conclusions drawn from our framework.\n\nWe hope this addresses the points you raised in your review. We are happy to clarify any additional questions you may have."}, {"review_id": "WMXl8Oqmra", "reviewer": "Reviewer_UBhm", "summary": "The authors employ a causal inference framework to isolate the specific impact of pulse oximeter measurement discrepancies on invasive ventilation rates and duration. As key contributions, the authors propose a method using path-specific effects to quantify racial disparities in ICU decision-making. They define a specific causal query, the V-specific Direct Effect (VDE), to isolate the influence of race that is mediated specifically through oxygen saturation measurement discrepancies. They also present a doubly robust estimator for the VDE and introduces a self-normalized variant to improve sample efficiency and reduce variance, providing novel finite-sample guarantees for the estimator.\n\nThe methodology is validated on semi-synthetic data and then applied to two large ICU datasets, MIMIC-IV and eICU. Contrary to prior work suggesting significant disparities, this causal analysis finds a minimal impact of the measurement bias on invasive ventilation rates. However, the study does reveal a more pronounced and varied effect on the duration of ventilation, highlighting different outcomes between the two datasets.", "questions": "The paper finds a negligible effect of oximetry bias on the  rate of ventilation, which seems counterintuitive. Does this finding imply that for the critical decision of initiating ventilation, clinicians are successfully integrating a wide array of data beyond the pulse oximeter, effectively mitigating the device's known racial bias? Or could the bias be manifesting in ways not captured by this outcome, such as delays in receiving non-invasive oxygen support before a decision to intubate is made?", "limitations": "Yes.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\nNovel causal inference framework applied to real-world dataset with insightful analysis. \n\nWeakness\nThe authors acknowledge as a limitation that they only consider the first invasive ventilation and do not explicitly model temporality. They aggregate time-series data using an exponentially weighted average.", "was_revised": true, "final_justification": "The authors promise expanded discussion. Maintain original assessment.", "rebuttal": "Thank you for acknowledging the novelty of the method, and for your constructive comments and review.\n\n**Questions**\n1. Thank you for the interesting question. We focus on the specific research question of path-specific effects of rate and duration of ventilation. We agree that we cannot rule out either of the two possibilities, that either the clinicians successfully integrate other information, reducing the overall effect of pulse oximeter discrepancy, or that other outcomes need to be measured, which is an active area of research. Our study is designed in line with the outcomes chosen in prior studies. We will add these details to our discussion.\n\nPlease let us know if you have any additional questions and we are happy to clarify further."}, {"review_id": "ud4Fq2OiHE", "reviewer": "Reviewer_ScmX", "summary": "In this paper, the authors focus on tackling the problem of racial bias on ICU settings, where medical device bias can lead to delaying clinical interventions for certain sensitive groups. To this end, the authors introduce the V-specific direct effect to estimate the racial biases in the ICU. Subsequently, the authors propose a doubly robust estimator and a self-normalized variant for the V-specific direct effect with theoretical guarantees. Experimental results show the effectiveness of the proposed method in analyzing the causal effects of pulse oximeter mediated bias on invasive ventilation.", "questions": "Please refer to the weaknesses.", "limitations": "Yes, the authors have discussed the limitations of the proposed method.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The first causal analysis method to investigate the racial biases in ICU settings.\n2. The authors propose a theoretically sound estimator to quantify the path-specific causal effects of racial bias on invasive ventilation.\n3. The writing is clear and easy to follow.\n\nWeaknesses:\n1. The unique challenges of addressing fairness issues in ICU settings are not sufficiently discussed. In addition, it is unclear why existing causal analysis methods, such as [27], [44], and [46] shown in Table 1, are not applicable to ICU data.\n2. The proposed method relies on several implicit assumptions to ensure the identifiability of the V-specific direct effect, e.g., the absence of hidden confounders. However, the plausibility of these assumptions in real ICU settings is not sufficiently discussed.\n3. The numerical experiments lack comparisons with state-of-the-art methods, including those based on statistical correlations, to demonstrate the advantages of the proposed approach.\n4. How does the proposed method performs under imbalanced sensitive group distributions, as is often the case in practice where black patients are underrepresented compared to white patients.", "was_revised": true, "final_justification": "The authors have addressed most of my concerns, and I maintain my rating.", "rebuttal": "Thank you for your thoughtful and constructive questions. We address all your concerns in the following response.\n\n**Weaknesses**\n1. In Table 1, [27], [44], and [46] investigate path-specific effects, but only [27] focus on effects isolated through a single mediator (as shown in the fourth column). [27] is conceptually similar to our non-self-normalized estimator which results in high variance (see Figures 3a and 3c), especially in small sample regimes like our application. [44] and [46] allow only a single mediator, which could be applicable in other ICU settings, but is limited in the pulse oximetry setting because all relevant patient variables  beyond the supplemental oxygenation discrepancy should be considered for valid and unbiased fairness estimation. Nonetheless, we compare with this \"misspecified\" setting (i.e. assume only one mediator), called the NIE$^*$, in Figure 5. We will add a discussion to address the research gap in the revisions.\n\n###\n\n2. In Appendix A, we explicitly outline the identifiability conditions, which assume no unmeasured confounding between W and V in order to identify the VDE. This is reasonable in the context of our ICU application, and we will expand the appendix to include a discussion of the plausibility of this assumption based on the available covariates and the clinical context of our ICU application.\n\n###\n\n3. We thank the reviewer for identifying this point. To address the concern, we compare our proposed estimators, the canonical doubly robust estimator for the VDE and its self-normalized variant, with three additional baselines across synthetic (binary and continuous) and semi-synthetic (eICU and MIMIC-based) datasets. Since, to the best of our knowledge, no prior work directly targets the VDE beyond the canonical estimator, we include the following (somewhat misspecified) baselines:\n    - **X-learner**, which estimates the ATE without accounting for the mediation structure (ignoring $W$ and $V$), and cannot isolate the target effect we want.\n    - **Single-mediator doubly robust estimator**, which computes the effect through the mediators as if $W$ and $V$ were merged into a single variable (this is equivalent to the NIE$^*$ used in our real-world experiments).\n    - **Nested regression estimator**, which corresponds to $\\mu_0^1$ in Equation 6. \n\n   We compare all five estimators in the table below, reporting the mean and standard error over 100 bootstraps, where each iteration samples a dataset of size 32,000. Relative errors are shown in parentheses, except for the semi-synthetic eICU setting, where the true VDE is very close to zero and relative error is not informative. Entries in bold denote the best performing model for each dataset.\n\n    |                                  | Synthetic binary                          | Synthetic continuous                       | Semi-synthetic eICU                | Semi-synthetic MIMIC                   |\n    |:---------------------------------|:-----------------------------------------:|:------------------------------------------:|:----------------------------------:|:-------------------------------------------:|\n    | **Ground truth**                | $-0.125\\phantom{+}$                                  | $-0.163\\phantom{+}$                                   | $0.000$                            | $0.001$                                     |\n    | **X-learner**                   | $\\phantom{+}0.364\\ (292\\\\\\%) \\pm 0.001$              | $-0.352\\ (217\\\\\\%) \\pm 0.001$              | $-0.007 \\pm 0.000$                | $-0.029\\ (4819\\\\\\%) \\pm 0.000$              |\n    | **Single-mediator doubly robust estimator [44, 46]** | $-0.033\\ (26\\\\\\%) \\pm 0.001$ | $-0.312\\ (192\\\\\\%) \\pm 0.008$             | $\\phantom{+}0.024 \\pm 0.014$                 | $-0.027\\ (4354\\\\\\%) \\pm 0.004$              |\n    | **Nested-regression estimator (Eq. 6, $\\mu_0^1(x_1, Z)$ term)** | $\\mathbf{\\phantom{+}0.001\\ (0.5\\\\\\%) \\pm 0.001}$ | $-0.024\\ (15\\\\\\%) \\pm 0.002$         | $\\phantom{+}0.004 \\pm 0.001$                 | $\\mathbf{-0.001\\ (117\\\\\\%) \\pm 0.000}$           |\n    | **Canonical doubly robust estimator ([27], Fig. 3a&c)** | $\\mathbf{-0.001\\ (0.5\\\\\\%) \\pm 0.001}$ | $\\mathbf{\\phantom{+}0.006\\ (4\\\\\\%) \\pm 0.008}$      | $\\phantom{+}0.018 \\pm 0.020$                 | $\\phantom{+}0.025\\ (4039\\\\\\%) \\pm 0.176$              |\n    | **Self-normalized doubly robust estimator (Proposed, Fig. 3b&d)** | $\\mathbf{-0.001\\ (0.5\\\\\\%) \\pm 0.001}$ | $\\mathbf{\\phantom{+}0.006\\ (4\\\\\\%) \\pm 0.006}$      | $\\mathbf{\\phantom{+}0.003 \\pm 0.007}$             | $\\phantom{+}0.004\\ (622\\\\\\%) \\pm 0.004$  \n\n   The findings show that the self-normalized estimator typically yields the lowest error and maintains a $95\\\\%$ confidence interval that includes the true value. While the nested regression estimator performs comparably on semi-synthetic data, its confidence intervals are overly narrow. State-of-the-art methods for similar effect estimation tasks (i.e. the X-learner and single-mediator estimator) exhibit high error across all settings since they do not account for the correct mediation structure.\n\n###\n\n4. We have roughly 31,000 White and 4,000 Black patients in eICU, and around 4,000 White and 400 Black patients in MIMIC, thus capturing the imbalance you have correctly highlighted. This mainly reflects in the variance of the estimator, which we propose to mitigate using the proposed self-normalized variant. We also approximately preserve the imbalance in our semi-synthetic setup (the ratio of White to Black patients is 2:1). Figures 3c and 3d demonstrate that the self-normalized doubly robust estimator demonstrates good convergence behavior despite the imbalance.\n\n   ###\n\n   We also include an experiment demonstrating how varying levels of imbalance affect the performance of our estimator. Using the two semi-synthetic datasets, we threshold the predictions of $X$ at different values to induce varying proportions of $x_0$​ and $x_1$ subgroups. To quantify this imbalance, we define $\\eta = \\frac{\\\\# \\text{ of $x_1$ samples}}{\\\\# \\text{ of total samples}}$. Following the approach in Figure 3, we evaluate performance using the relative error for each causal query. The mean and standard error are reported over 100 bootstrap iterations, each sampling a dataset of size 32,000. Results are presented in the two tables below.\n\n   | **eICU-based semi-synthetic data**      | $\\quad\\quad \\eta \\approx 0.5$               | $\\quad\\\\:\\\\:\\\\: \\eta \\approx 0.33$              | $\\quad\\quad \\eta \\approx 0.2$               | $\\quad\\quad \\eta \\approx 0.1$               |\n   |:----------------------------------------|:--------------------------------:|:--------------------------------:|:--------------------------------:|:--------------------------------:|\n   | $\\mathbb{E}[Y_{x_0}]$                   | $\\phantom{+}0.1\\\\% \\pm 0.1\\\\%$   | $-0.1\\\\% \\pm 0.1\\\\%$             | $\\phantom{+}0.1\\\\% \\pm 0.1\\\\%$   | $\\phantom{+}0.0\\\\% \\pm 0.1\\\\%$   |\n   | $\\mathbb{E}[Y_{x_1}]$                   | $-0.1\\\\% \\pm 0.1\\\\%$             | $\\phantom{+}0.0\\\\% \\pm 0.2\\\\%$   | $-0.5\\\\% \\pm 0.2\\\\%$                | $-0.5\\\\% \\pm 0.3\\\\%$             |\n   | $\\mathbb{E}[Y_{x_1, M_{x_0}}]$          | $-2.8\\\\% \\pm 6.6\\\\%$             | $-2.7\\\\% \\pm 3.5\\\\%$             | $\\phantom{+}7.2\\\\% \\pm 6.7\\\\%$   | $-9.1\\\\% \\pm 6.6\\\\%$             |\n   | $\\mathbb{E}[Y_{x_1, V_{x_0}, W_{x_1}}]$ | $-1.2\\\\% \\pm 1.6\\\\%$             | $\\phantom{+}1.2\\\\% \\pm 1.7\\\\%$   | $-0.8\\\\% \\pm 2.8\\\\%$             | $-3.9\\\\% \\pm 2.8\\\\%$             |\n\n   | **MIMIC-based semi-synthetic data** | $\\quad\\quad \\eta \\approx 0.5$             | $\\quad\\\\:\\\\:\\\\: \\eta \\approx 0.33$            | $\\quad\\quad\\eta \\approx 0.2$             | $\\quad\\quad\\eta \\approx 0.1$             |\n   |:----------------------------------------|:------------------------------:|:------------------------------:|:------------------------------:|:------------------------------:|\n   | $\\mathbb{E}[Y_{x_0}]$                   | $-0.6\\\\% \\pm 0.2\\\\%$           | $-0.6\\\\% \\pm 0.2\\\\%$           | $-0.6\\\\% \\pm 0.2\\\\%$           | $-0.6\\\\% \\pm 0.2\\\\%$           |\n   | $\\mathbb{E}[Y_{x_1}]$                   | $-0.5\\\\% \\pm 0.2\\\\%$           | $\\phantom{+}0.3\\\\% \\pm 0.4\\\\%$ | $-0.9\\\\% \\pm 0.4\\\\%$           | $-0.6\\\\% \\pm 0.6\\\\%$           |\n   | $\\mathbb{E}[Y_{x_1, M_{x_0}}]$          | $\\phantom{+}2.0\\\\% \\pm 4.3\\\\%$ | $-3.8\\\\% \\pm 2.2\\\\%$           | $\\phantom{+}0.2\\\\% \\pm 3.9\\\\%$ | $-6.7\\\\% \\pm 3.7\\\\%$           |\n   | $\\mathbb{E}[Y_{x_1, V_{x_0}, W_{x_1}}]$ | $-6.8\\\\% \\pm 2.6\\\\%$           | $\\phantom{+}6.0\\\\% \\pm 5.1\\\\%$ | $-5.7\\\\% \\pm 6.6\\\\%$           | $-1.1\\\\% \\pm 7.7\\\\%$           |\n\n   As expected, the relative error and variance of the estimator generally increase with greater imbalance between the $x_0$ and $x_1$ subgroups (with the exception of $\\mathbb{E}[Y_{x_1, V_{x_0, W_{x_1}}}]$ on MIMIC-based data). Nonetheless, the overall differences in estimator performance across levels of $\\eta$ remain small, with relative error increasing by at most a few percentage points.\n\nWe hope that our response addresses your concerns. We are happy to clarify any other questions you may have."}, {"review_id": "dUvjEQqSiG", "reviewer": "Reviewer_vpLv", "summary": "This paper considers the fairness issues in Healthcare scenarios and aims for investigating the influence of racial bias in oximetry measurements on invasive ventilation in Intensive Care Units (ICU) settings from a causal perspective. Specifically, it leverages path-specific analysis to detect the causal effects of sensitive attribute (race) on invasive ventilation mediated by pulse oximetry discrepancies. To quantify such path-specific effects, it proposes a doubly robust estimator and a self-normalized variant, with fine-sample guarantees. Experiments on synthetic and real-world datasets showcase the effectiveness of the proposed method.", "questions": "1. What are the unique challenges of the ICU settings compared to other healthcare applications scenarios?\n2. Please refer to the weaknesses.", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The authors address a nice and importance fairness problem in this work.\n2. The authors propose a novel doubly robust estimator and self-normalized variant for quantifying V-specific direct effects.\n3. Theoretically sound. The proposed method is built upon existing theoretical results while also providing rigorous theoretical analysis.\n4. The experiments results demonstrate that the proposed method can effectively assess V-specific direct effects.\n\nWeaknesses:\n1. The presentation in articulating the research gap is not so clear. More explanations about how existing path-specific methods can be adapted or applied to ICU settings would help in understanding the flexibility and integration potential of them, e.g., Reference [27].\n2. In Table 1, Reference [46] also focuses on multiple mediators settings.\n3. The proposed V-specific direct effect is a special case of path-specific effects.\n4. The validity of the proposed method relies on the assumptions of the causal models illustrated in Figure 2. However, the rationale behind data generation mechanism aligning with these causal models is not clearly presented. For example, X, representing race as an inherent characteristic of individuals, raises questions regarding why it is caused by Z. The authors do not provide any explanations where these causal models may hold, thus making it difficult to justify the assumptions.\n5. There is a lack of comparison with SOTA methods in the experiments. More SOTA methods should be considered.", "was_revised": true, "final_justification": "The authors have addressed most of my questions, and I‘ll keep my rating.", "rebuttal": "Thank you for your detailed review and valuable suggestions. We address each of your comments below.\n\n**Weaknesses**\n\n1. We will address weaknesses 1 and 2 together.\n\n   ###\n\n   In Table 1, [27], [44], and [46] investigate path-specific effects, but only [27] focus on effects isolated through a single mediator (as shown in the fourth column). [27] is conceptually similar to our non-self-normalized estimator which results in high variance (see Figures 3a and 3c), especially in small sample regimes like our application. [44] and [46] assume a single mediator, which could be applicable in other ICU settings, but is limiting in the pulse oximetry setting because many variables beyond the supplemental oxygenation discrepancy should be considered for valid fairness estimation. Nonetheless, we compare with this \"misspecified\" setting (i.e. one mediator), called the NIE$^*$, in Figure 5. We will add a discussion to address the research gap in the revisions.\n\n### \n\n4. As with most causal inference methods, our framework relies on a specified causal graph and thus makes assumptions about the data-generating process that can be challenging to verify in practice. To support these assumptions, we will expand the appendix to include a detailed discussion of their plausibility in the context of our ICU application. We will also emphasize that, although the validity of our methodology does indeed depend on the specified causal diagrams, using DAGs is advantageous because they clearly and explicitly represent the assumptions being made, and its importance has been highlighted in critical care literature as well [1]. These assumptions can be more easily verified or corrected compared to those in more opaque methods, making causal models relevant in medical (and other) domains.\n\n   ###\n\n   We agree that the variable $X$, representing race, should not be modeled as receiving a directed arrow from $Z$ alone. The more appropriate structure includes a hidden confounder between $X$ and $Z$, as demographic variables such as age may influence the distribution of racial groups. Importantly, introducing a bidirected arrow between $X$ and $Z$ does not affect our estimators or the theoretical analysis presented in Appendix A. We will revise the causal graph accordingly in the updated manuscript.\n\n###\n\n5. We agree with the reviewer and have included comparisons with three additional baseline methods in both the synthetic and semi-synthetic experiments. To the best of our knowledge, no existing work directly estimates the VDE beyond the canonical doubly robust estimator, which is challenging to interpret due to high variance. Therefore, we evaluate several state-of-the-art methods for similar effect estimation tasks. For details on the experimental setup and evaluation, please see our response 3 to Reviewer ScmX.\n\n**Questions**\n1. The ICU is a particularly complex clinical setting because of its longitudinal nature and the high dimensionality of patient data. These characteristics make it difficult to apply standard algorithmic fairness frameworks directly. This motivates us to incorporate domain knowledge and a clinically grounded data-generating process to more appropriately assess fairness in this setting.\n\nWe hope our response resolves your concerns and welcome any further questions you may have.\n\n[1] - David J. Lederer, Scott C. Bell, Richard D. Branson, et al. Control of confounding and reporting of results in causal inference studies. Guidance for authors from editors of respiratory, sleep, and critical care journals. *Annals of the American Thoracic Society*, 16, 2019."}]

Author rebuttal:

Meta-review: The paper proposes mediating multiple confounders with path-specific effects to measure bias due to specific confounders. Reviewers generally found the paper interesting and a good contribution over existing work. Some concerns were raised over the assumptions made by the method, and the lack of incorporation of time-series data (e.g. changing code status) is a significant limitation. Otherwise it an interesting approach to modelling racial bias.
Decision: Accept (poster)