Sharp Bounds for Generalized Causal Sensitivity Analysis
Authors: Dennis Frauen
Valentyn Melnychuk
Stefan Feuerriegel
Venue: NeurIPS 2023 poster (2023)
Keywords: Causal machine learning
treatment effect estimation
sensitivity analysis
unobserved confounding
uncertainty estimation
TL;DR: We propose a novel causal sensitivity model that generalizes various sensitivity models from the literature and allows us to perform sensitivity analysis in a variety of causal inference settings.

Abstract: Causal inference from observational data is crucial for many disciplines such as medicine and economics. However, sharp bounds for causal effects under relaxations of the unconfoundedness assumption (causal sensitivity analysis) are subject to ongoing research. So far, works with sharp bounds are restricted to fairly simple settings (e.g., a single binary treatment). In this paper, we propose a unified framework for causal sensitivity analysis under unobserved confounding in various settings. For this, we propose a flexible generalization of the marginal sensitivity model (MSM) and then derive sharp bounds for a large class of causal effects. This includes (conditional) average treatment effects, effects for mediation analysis and path analysis, and distributional effects. Furthermore, our sensitivity model is applicable to discrete, continuous, and time-varying treatments. It allows us to interpret the partial identification problem under unobserved confounding as a distribution shift in the latent confounders while evaluating the causal effect of interest. In the special case of a single binary treatment, our bounds for (conditional) average treatment effects coincide with recent optimality results for causal sensitivity analysis. Finally, we propose a scalable algorithm to estimate our sharp bounds from observational data.

Sharp Bounds for Generalized Causal Sensitivity Analysis
Dennis Frauen, Valentyn Melnychuk & Stefan Feuerriegel
LMU Munich
Munich Center for Machine Learning l de
Abstract
Causal inference from observational data is crucial for many disciplines such as medicine and economies. However, sharp bounds for causal effects under relax- ations of the unconfoundedness assumption (causal sensitivity analysis) are subject ongoing research. So far, works with sharp bounds restricted fairly sim- ple settings (e.g., a single binary treatment). In this paper, we propose a unified framework for causal sensitivity analysis under unobserved confounding various settings. For this, we propose flexible generalization of the marginal sensitivity model (MSM) and then derive sharp bounds for a large class of causal effects. This includes (conditional) average treatment effects, effects for mediation analysis and path analysis, and distributional effects. Furthermore, our sensitivity model is applicable to discrete, continuous, and time-varying treatments. It allows us to interpret the partial identification problem under unobserved confounding as a distribution shift in the latent confounders while evaluating the causal effect of interest. In the special case single binary treatment, our bounds for (condi- tional) average treatment effects coincide with recent optimality results for causal sensitivity analysis. Finally, we propose scalable algorithm to estimate our sharp bounds from observational data.
Introduction
Causal effects crucial for decision-making in many disciplines, such as marketing (71), medicine (77), and economics (T. For example, physicians need to know the treatment effects to personalize medical care, and governments
interested in the causal effects of policies on infection rates during a pandemic. In many such applications, randomized experiments are costly or infeasible, because of which causal effects must be estimated from observational data (59)
Estimating causal effects from observational data may lead to bias due to the existence confounders, ie., variables that affect both treatment and outcome (55) A remedy is to observe all confounders and thus to assume unconfoundedness (e.g., as [6 63 64)) D. However, in many practical applications, the assumption of unconfoundedness is violated. For example, electronic health records do not capture a patient's 's ethnic background, which is a known confounder medicine (52). such cases, the causal effect is not identified from observational data, and unbiased estimation is thus impossible 277]
A popular way to perform causal inference in the presence of unobserved confounders is causal sensitivity analysis. Causal sensitivity analysis aims to derive bounds on the causal effect of interest under relaxations of the unconfoundedness assumption. Here, the strength of unobserved confounding typically controlled by some sensitivity parameter I', which also determines the tightness of the bounds. In practice, one chooses I' through domain knowledge [14] 33 or data-driven heuristics 29 Then, one derives that the causal effect of interest lies some informative region. For example,
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

suitable I' may enable despite unobserved confounding to infer the sign of the treatment effect, which is often sufficient for consequential decision- maakin (35].
common model for causal sensitivity analysis is the marginal sensitivity model (MSM) [31 35 36
67]. A benefit of the MSM is that it does not impose any kind of parametric assumptions the data-
generating process. However, the standard MSM is only applicable in settings where the treatment is single binary variable. Different extensions have been proposed for continuous treatments [10 32
45] and for time-varying treatments and confounders [TO]. However, these works are restricted to specific settings, while a unified framework for causal sensitivity analysis is still missing.
Contributions!] In this paper, we pro- pose a generalized marginal sensitiv- ity model (GMSM). Our GMSM pro- vides a unified framework for causal sensitivity analysis under unobserved confounding in various settings with multiple discrete, continuous, and time-varying treatments. Crucially, our GMSM includes existing models, such as those [67] and (32) as spe- cial cases. As result, our GMSM enables a unified approach to deriv- ing sharp bounds for large class of causal effects. To do so, we bound a distribution shift in the unobserved confounders while performing an intervention on the treatments. As result, we obtain sharp bounds for various causal effects, including (conditional) average treatment effects but also effects for mediation analysis and path analysis (see Fig. D and for distributional effects, which have not been studied under an MSM-type sensitivity analysis previously. We also show that, for binary treatments, our bounds coincide with recent optimality results for (conditional) average treatment effects under the MSM. Finally, we propose a scalable algorithm to estimate our sharp bounds from observational data and perform extensive computational experiments to show the validity of our bounds empirically.
M
M
Ms
Figure 1: Three examples of causal inference settings where our GMSM and bounds are applicable. Mi, M2 are medi- ators, Y is the outcome, and X and A are (multiple) dis- crete, continuous, or time-varying covariates and treatments. Variables Uw are unobserved confounders between A and W E {M,, M2, Y
2 Related work
In the following, we review related work on causal sensitivity analysis. For a more general review of partial identification and treatment effect estimation under unconfoundedness, refer to Appendix A
Causal sensitivity analysis: Causal sensitivity analysis dates back to a study from 1959 showing that unobserved confounders cannot explain away the causal effect of smoking on cancer (14). This was formalized by introducing sensitivity models that yield bounds on the causal effect of interest under some restriction on the amount of confounding. Previous works introduced a variety of sensitivity models that make different assumptions about the data-generating process and the confounding mechanism. Examples include sensitivity models based parametric assumptions [30 62], difference between potential outcome distributions [60 70). and Rosenbaum's 's sensitivity model that uses randomization tests [61].
Marginal sensitivity model (MSM): The marginal sensitivity model (MSM) 677 is common model for sensitivity analysis aimed at settings with binary treatments. Many methods were proposed to estimate bounds from observational data under the MSM. Examples of such methods include linear fractional programs
[82) and machine learning such as including kernel-based methods (35) and deep
neural networks [3T]. Recently, Dorn et al. (88) and Jin et al. (33 showed that estimates from the
previous methods are too conservative bounds and, as a remedy, derived closed-fome solutions for
sharp bounds under the MSM. This was also extended to semiparametric inference [19 53). However,
all previous methods are limited to binary treatments and (conditional) average treatment effects.
Different extensions for the MSM have been proposed. Jesson et al. [32]. Bonvini al. [TO), and Marmarelis et al. [45) developed extensions for continuous treatments. Furthermore, Bonvini et al. [TO) also extended the MSM time- mane-varynng treatments and confounders. However, both works
are not applicable to settings beyond the estimation of (conditional) average treatment effects, such as
'Code is available http:///ghhhhoooo///ms/////aunun

---

mediation analysis and path analysis or distributional effects. In contrast, we propose a generalized marginal sensitivity model that compatible with binary, continuous, and time-varying treatments for a variety of causal effects.
Research gap: To the best of our knowledge, existing causal sensitivity analysis based on the MSM provides a unified framework for deriving bounds for binary, continuous, and time-varying treatments. Furthermore, MSM-based causal sensitivity analysis is restricted (conditional) average treatment effects and is thus not applicable for complex settings such as effects for mediation analysis and path analysis or distributional effects.
3 Generalized marginal sensitivity model (GMSM)
We first formally define general setting for causal sensitivity analysis that includes mediation and path analysis (Sec. 3.1 and Sec. 3.2) We then propose our generalized marginal sensitivity model (GMSM) Sec. 3.3 and compare it with existing sensitivity models from the literature.
Notation: We write random variables as capital letters X and their realizations in lowercase x. Bold letters X represent (random) vectors. If X (X1,- X,) is sequence of random variables of length f, we denote X (XI,..,X Xs)) for 1 S k < L. We denote probability distributions over X as Ip-X where required. The probability mass function for discrete X denoted as P PP = ær). If X is continuous, P(z) is the probability density function. We denote P(-) as the corresponding probability mass/density function not evaluated at a specific x. Similarly, we write conditional probability mass functions/density functions as Py x) P(Y = X x) and conditional expectations as E[Y I 2] E[Y X z] = Py x) dy. We denote Ply do(X æ)) as the probability mass function/density function of Y after performing the do-intervention do( (X x) (55). Finally, we define Xy pY( n X, where pY( denotes the parents of Y in given causal graph.
3.1 Problem setup for generalized causal sensitivity analysis
We formalize causal sensitivity analysis based on Pearl's structural causal model framework 55 Definition 1. A structural causal model (SCM) M is a tuple (V, UV, where V (Vi..... V&) are observed endogenous variables, U are unobserved exogenous variables determined outside of the model, FF = {fvi.... fv,} is a set of functions so that each fv; maps a set of parents pa(V)) S VVU to V,, and pd is probability distribution on U.
Every SCM M induces unique directed grapt GM on VOU U by drawing directed edge from Vi to V₂ if V12 E pa(V2). In this paper, assume that Gy is acyelic, i.e., does not contain any directed cycle. Then, M induces a unique joint probability distribution PVUt on V U U. Furthermore. M induces unique interventional distributions PV-U when performing the do-intervention do(A a) do]A=aj
for some observed treatments A C V 55
We consider settings where we observe four distinct types of endogenous variables V (X, M Y): observed confounders X E X' Rdz, treatments A E A C R2d, discrete mediators M, = (M,...., M1) with M, E N for i E, and an outcome Y E R. In medical setting, X might be patient characteristics (gender, age, medical history, etc.), A medical treatment, M, a change in diet, and Y a variable indicating a health outcome. Possible causal graphs are shown in Fig. 1 We assume w.L.og. that (M1,... Mc) are ordered causally, i.e., M,-1 are parents of M, for each E (2....,f) f}. Given treatment interventions ä = (ai...., as+1), we are interested in causal effects of the form
my do(A = ac+1)) P(m+ x. 1m Σ (. MIENE
doí A = a,)),
where D is some functional that maps the density Tp x, m, do = a(+1)) to scalar value and sum over all possible realizations m, of M,. We are also interested in averaged causal effects
Note that we consider graphs over both observed and unobserved variables. In the causal inference literature. these are also called augmented causal graphs 9
3

---

fx Q(x, M)) P(x) dx or differences (Appendix D). The effect Q(x, a1x.1 generalizes many common effects across different causal inference settings.
Example 1 (CATE. L = 0). If D = E||1 is the expectation functional, Eq. 1 reduces to
Q(x, a, = Y do a)). When A is continuous, this is known as the conditional dose- response function 7 32 For binary treatments A E {0. 1.. the query Q(x, M, - Q(x 0, MM) is known as the conditional average treatment effect (CATE) 16 78). and its averaged version as the average treatment effect (ATE) 664 69].
Example 2 (Mediation analysis. L 1). ID = E and M M is single mediator, Eq. ① reduces to Q(x MM) = Σm || x, m, do(A A = a2) P(m x, do(A = a))). If E {0, 1) is binary and the M-Y relationship is unconfounded, we obtain the following causal effects studied
in mediation analysis [56]: Q(x, (a, = O, az = M). - Q(x, (a1 = 0, 0, = M) is the
(conditional) natural direct effect (NDE), and Q(x, (a1 1, a2 M) Q (a1 0, a2 M) is the (conditional) natural indirect effect (NIE).
In general, Eq. (I) includes so-called path-specific effects if the relationship between mediators and outcome is unconfounded [2] [5]. For example, by setting aı and a = 0 for all 2 < k < ++ 1, we obtain the indirect effect that is passed through the mediator sequence (M, M2). Path- specific effects are important in various applications, including algorithmic fairness, where the aim is to mitigate effects through paths that are considered unfair (50] (1]. Furthermore, we can set 'D to a quantile instead of using the mean in all the examples above. This results in distributional versions of CATE, NDE, NIE, and path-specific effects. For example. if the outcome distribution is skewed or contains outliers, practitioners might prefer the median or other quantiles over the mean due their robustness properties [1]].
If all confounders between treatment and mediators and outcome were observed, we could (under additional assumptions) identify the causal effect Q(x, MM) from the observational distribution P.V. V by replacing all do-operations with conditional probabilities according to the backdoor criterion [55]. However, under unobserved confounding, identifiability is not possible because SCMs M and M' exist, which induce the same observational distribution PVV but which result in different
causal effects Q(x, Mc + Q(x, Mc) 54 74..
3.2 Causal sensitivity analysis
In the following, we formalize causal sensitivity analysis as maximizing//mnimizzng the causal effect
Q(x, āc+1. MM) over all SCMs M that are compatible with a predefined sensitivity model. Similar approaches have been used for partial identification with instrumental variables [40] [54] and testing
identifiability of counterfactuals (74 75), where all SCMs are considered that are compatible with the observational data. However, a sensitivity model must additionally restrict the joint distribution of both observed and unobserved variables in order to allow for informative bounds on the causal effect.
Definition 2 (Sensitivity model). Let PV denote the distribution of the observed variables V = (X, A, Me, Y). A sensitivity model S is a tuple (U, P), where U (Uw)w are unobserved confounders between A and W E {M1...., Mf, Y), }. respectively (see Fig. and a family P of joint probability distributions on V J U, such that, for all P E P. it holds that J PP(v, u) du = PpV (v). We denote the set of all SCMs M compatible with S (i.c., that respect the causal graph, induce a distribution IP E P, and do not contain additional confounders) C(S) (see Appendix E
Our definition of sensitivity models excludes unobserved confounding between mediators and the effect
outcome. This ensures that the causal effect from Eq. (T) can be interpreted as 56). We refer to Sec. 6 a detailed discussion.
Using the definitions above, we can define causal sensitivity analysis as the following partial identifi- cation problem: we aim to obtain bounds Q (x, 5) v Q S so that
(x,
sup Q(x, aM). and Q (x, = inf Q Mc. MEC(S)
.ME(S))
Q*(xxā(41, ā,41, S) is the maximal causal effect that can be achieved by any SCM that is compat- ible with the sensitivity mode S (and vice versa for Qx S)) Hence, if the sensitiv- ity model valid, i.e., contains the ground-truth distribution over observed variables and unob- served confounders, we know that the ground-truth causal effect must be contained in the interval [Q,(x, ā,+1, Q+ (x, ā 1.+1. S)]. Bounds for average causal effects and effect differences follow immediately (see Appendix D
4

---

3.3 Generalized marginal sensitivity model (GMSM)
We introduce now the GMSM. We begin by providing the general definition and then show that this extends various marginal sensitivity models from existing literature.
Definition 3 (GMSM), The generalized marginal sensitivity model (GMSM) is sensitivity model (U,P), where P contains all P that satisfy the following sensitivity constraint: For each W e {M Mj, } there exist bounds sw (a, 1 1 I (a,xx, so that, for all uw, x, and a P(Uw uw x.
(a, S P(Uw = uw x, do A = a]) S (a,
The GMSM bounds the distribution shift in the unobserved confounders Uw when performing the intervention do(A A = a) instead of conditioning on A = a. This restriction on the strength of the effect the unobserved confounders Uw can have on the treatment A. If Uw has no effect on A, Eq. (3) holds with sw (a, x) = st (a. x) = 1. Hence, the further siy (a, x) and si (a, x) deviate from 1, the larger is the effect from Uw on A that the GMSM allows for. We will often use weighted GMSM, which expresses the bounds in terms of sensitivity parameter.
Definition 4 (Weighted GMSM). A weighted GMSM is GMSM where Syr (a, x) and sty x) can be written as sw (a, x) (1 (1-Tw (o. and sw (a, x) (1-rw )aw (a,))+rr for a sensitivity parameter T 2 1 and a weight function qw (a, x) E [0, 1] for all x and a.
In weighted GMSM, the sensitivity parameter I Fw captures the overall restriction on the unobserved confounding strength across individuals. Tw = 1, no unohserved confounding is allowed, and unconfoundedness holds. For I'w + co, the restriction is relaxed completely, and arbitrary confounding strength is allowed. The weight function qw (w(a, x) E [0, 1] offers several advantages. It allows to further restrict confounding for individuals with treatments a and covariates
As qu (a. x) - 1, confounding is more strongly restricted until unconfoundedness is reached. This is helpful in applications where prior knowledge about the confounding structure is available. As an example, consider a medical setting where we want to estimate the effect of new drug treatments on the risk of developing certain disease, but suspect that the data is confounded by the individual's genetic risk for the disease, which is not measured in the observational data. However, we also might know that genetic risk for the disease is not relevant for specific combination of age and gender.
Comparison with the MSM and its extensions: In the following, we show that the weighted GMSM extends popular sensitivity models from the literature. Proofs are in Appendix C Note that existing sensitivity models do not consider settings with mediators (i.e., E = 0), we thus can write I = Ty for the sensitivity parameter, ga,, qy (a, x) for the weight function, and U Uy for the unobserved confounders. For binary treatments A = E {0, the marginal sensitivity model (MSM) (67] is defined via
x) 7(x) 1 (xx(x,u) < P, where Txx) = IPA
I-+(x} <(xuu
denotes the observed propensity score and ग (x, u) P(A
x, u) denotes the full propensity score. For continuous treatments A, the continuous marginal sensitivity model (CMSM) (32) is
r. For longitudinal settings with time-varying observed confounders X (Xi,... Xr), unobserved confounders = (UI..... Ur), treatments A = (AI..... Ar), we define the longitudinal marginal sensitivity model (LMSM) via } S <T
defined via + P(ajx.u)
F(a
P(a: (.. "(aa, (r1)
< T.
Lemma 1.
The MSM, CMSM and LMSM are special cases of the weighted GMSM by choosing the weight functions q(a, x) = P(a| x) (MSM) and q( x) = 0 (CMSM, LMSM).
Lemma provides a new interpretation of the MSM: If the probability P(a x) is large, the MSM restricts the confounding strength because most of the randomness in the treatment is already explained by the observed confounders x. We can extend this approach to arbitrary discrete treatments within the weighted GMSM framework. The CMSM and LMSM apply the same confounding restriction for all a and x.
Bounding causal effects under the GMSM
In this section, we derive sharp bounds under our GMSM. We first quantify the maximal shift in conditional distributions (Sec. 4.I. This allows us to derive an algorithm to compute explicit bounds (Sec. 4.2 Finally, we show how these bounds can be estimated from finite data (Sec. 4.3
5

---

4.1 Shifting interventional distributions
In the following. we provide some intuition before stating the main result. Let us consider simple setting with treatment A, single unobserved confounder Uy = U E R, outcome Y. and GMSM bounds (a) and (a) (see Fig. 2 left). Any SCM M that compatible with S describes the relationship between A, U, and Y via a functional assignment Y [v U.) For fixed treatment a, we denote fy (a...) as fa- We are interested in the interventional density P (v) do(A = a)) =
Pú [y] a, U) P(u) du = fag Pu Pu (v), where Pu denotes the push-forward distribution induced by fa pV, However, we only have access to the observational density P a) = P (y/ a, u) P(u a) du (agP((()) Henee, for fixed functional assignment fa. we can quantify the discrepancy between IP (y do(A a)) and P (y) a) via the distribution shift between pe and pula.
Fig. 2 shows a toy example where ptla is the uniform distribution on [0. 1]. and the func- tional assignment fa is the inverse standard normal CDF o- so that IP (y a) is the standard normal probability density. We now want to "right-shift" the interventional den- sity P ( do(A = a)) as much as possible, so that F(y a) > Fly do(A a)) for the CDFs. To achieve this, the distribution p must put more probability mass
on the right-hand side of the unit interval [0, 1] as compared pMla (see Fig. 2 left). Then, the functional assignment will also push more probability mass to the right of PP(y do(A A a)) as com-
Poll
pared to P (yy a) (see Fig. 2 right).
However, the GMSM bounds the dis-
tribution shift between pp" and ppla
50
57
via (a)((a) S P(u) I (/s)(a) (Eq. (3) using that P(u) = P(u do(A a)). Intuitively, the max- imal possible right shift under the
f00
Figure 2: Intuition for bounding interventional distributions
under our GMSM.
GMSM should occur by choosing P(u) that the bounds attained, i.e., IP(u) 1(u S c+)(1/))) + 1(u > (1/ss) for some ot that can be obtained via the normalization constraint P du = 1. The corresponding "right-shifted" interventional density F+, (v | do(A A = a)) is defined via the push-forward (see Fig. (2).
It turns out that the density P+ (y | do(A = a)) is the maximally right-shifted interventional distribu- tion that can be obtained under any SCM that is compatible with the GMSM. Furthermore, the above arguments can be generalized beyond the toy example from Fig. 2
Theorem 1. Let S be a GMSM with bounds sw = (a, x) and sw = $ x) for W e (1- (1- )aw s -w
(M...., V.. We define cit
W E R is continuous, we define the probability
density function
(1/s P(w x, mw, a), F(w) S cir
P+(w| x, mwm a)
(4)
(1/ sw)) P( x, mw,a), a), if F(w) cw.
where F(.) is the CDF corresponding to PP( x, m.) a). I N discrete, we define the probability mass function
(1/s P(w| x, mwm a),
if < city F - 11) CW:
P+(w|] x, mw, a) = (1/sw) P(w x, mwmmaa a),
(1/sW) ) (ow - F(w- -1) 1)) + (1/5W) (F(w) - -wr) else.
(5)
Let F+(-) denote the conditional CDF corresponding to P+ x, mww a) and let F- (-) be the conditional CDF of PfP_ (-1 k.w,,ar a), which is defined by swapping signs (in ew and sw). For any SCM M, denote the CDF corresponding to P (: x, mw, do(A A a)) as F. Then, for all w,
(w) S inf MEC(S)
(u) and F_(w) sup FM(w).
(6)
MEC(S)
Assume now that P(uw x, do(A a)) = P(uw x). Then, the bounds are sharp (i.e., equality holds in Eq. (6) whenever A is discrete and it holds that 1/sty P(a x), A is continuous.

---

Proof. See Appendix
The result in Theorem I does not depend on the distribution or dimensionality of unobserved confounders, and it does not depend on any specific SCM. As such, our sharp bounds applicable to wide class of causal effects, without restricting assumptions on the confounding structure beyond the sensitivity constraint of the GMSM.
In case is a weighted GMSM with sensitivity parameters Iww for all W E {MI..... My, }), the quantiles oj and 'w are of the particular simple form cw = I Tww + + and cw = 1/(1 + T'w. Furthermore, the discrete sharpness condition simplifies to (1 - T )gw (a. x) + 2 P(a x). In particular, the bounds are sharp whenever P(a x). The assumption PP(uw | x, do(A = a)) = (u x) excludes the time-varying case (LMSMM). Deriving sharp bounds for the LMSM is interesting direction for future research.
choose weighting function that satisfies (w
4.2 Bounding causal effects
We now leverage Theorem to obtain explicit solutions for the partial identification problem from Eq. (2) with monotone 'D (see Appendix [.. This includes expectation and distributional effects.
Corollary (Bounds without mediators). If O and D is monotone. obtain sharp bounds
Q* S (PI x, and Q (x, D ( x. x))
and sharpness holds under the same conditions in Theorem
Proof. See Appendix B
If is the expectation functional, A is bi- nary, and is continuous, this coincides with the optimality result from Dorn and Guo (18). Hence, Corollary [ general- izes the result from Dorn and Guo [18 to distributional effeets and arbitrary treat- ments (c.g., categorical, continuous, or time- varyin.
Algorithm 1: Causal sensitivity analysis with mediators
Input Causal query Q(x, MM, GMSM S with sw and sw-
Output:: Upper hound Q (x, S)
Outcone bound
o (t-"i) for W E (M...... Mr,
27 (me) D D (
my ac+1)) m. aupp(Me)
their paper, Dorn and Guo proved the sharpness of the bounds by using the Neyman- Peanson Lemma. In con- trast, we take the more principled approach outlined in Sec. [. which is applicable to more general settings.
// Adjusting for confounding in nediators
(f....,1 1)
for m supp(M..1) do
Permotation map ascending order of
(m. (m.)))
m,exupp(M.))
F(m,) Sm 7(M; m m; 1:::)
(1/534, P(m: m.n))
In the following, we consider settings with mediators, i.c., aim to derive bounds for the causal effect in Eq. (I). The idea is to first obtain outcome bounds D (-) x, m,, a(+1)) conditioned on all possible values m, E (upp(M,) in the support of the mediators Mr. Without observed confounding between treatments and mediators, we can obtain the upper bound Q* (x, S) from Eq. (I by re- placing D (P (- x, m,g do(A = a)+1)) with D (P (:| x, my a)+1)). With unobserved confounding, we need to additionally take the distribution shift in the mediators into account. To maximize the causal effect, the shifted mediator distribution should put more probability mass on values m, for which D (PY large. Hence, we can apply the maximal right-shift from Theorem to the mediator distributions in Eq. (T. but where the values m, are permuted to order D (P x, m, ac+1)) in ascending order. We provide the details our iterative procedure to compute the upper bound Q+ (x, A,+1, S) in Algorithm [ The lower bound Q (x, ag-1, S) can be computed analogously by swapping signs in ew and sw.
it ((((m)))) < <oige
(1/e P(mm | a.).
(m,)) +
if F(x(m))))) - cte.
(2/0h4) > (cl, - F(x [ 1)) +(1/85g) (F(m(m))) )
clsc.
Q (m:-1) 1)+ Em, (m. P.
end
end
(x, āc-1, 01
me ac+1)) is
7

---

Corollary 2. Under the Assumptions of Theorem 7 Algorithm 7 Eq.
returns the sharp bounds from
Proof. See Appendix B
4.3 Empirical bounds via importance sampling
In practice, we only have access to an empirical distribution PY of sample size n instead of the full observational distribution PV. We thus obtain estimates P(w x, mw, a) of the conditional density probability mass functions P(w x, mw, a) for all W E (M1...., Mr. Y7. If W is discrete, this reduces to a standard (multi-class) classification problem, and the estimated class probabilities can be plugged into Algorithm 1 If W = Y is continuous,
arbitrary conditional density estimators obtain P(y X, my, a). We propose an importance sampling approach to estimate D (IP' (- x, m,,a) a)) assuming that we can sample from from our estimated density PP (- x, m,, a) (see Appendix F for details and derivations). D is the expectation functional, our estimator is
D (PP m.
E + Eliueii Vi.
where
(w.2 ~ (- x, 1)) sorted.
(8)
We also provide importance sampling estimators for distributional effects in Appendix F We also obtain empirical confidence intervals by using the same bootstrap procedure as described in 32)
Implementation: We use feed-forward neural networks with softmax activation function to estimate discrete probability mass functions. For densities, we use conditional normalizing flows (73) (neural spline flows (2T)). which are universal density approximators and allow for sampling. We perform training using the Adam optimizer [41]. We also perform extensive hyperparameter tuning in our experiments. Implementation and hyperparameter tuning details are in Appendix G
5
Experiments
Baselines:
Most existing methods focus on sensitivity analysis for binary treatments under the MSM
[18 [8. 19 31 35 53]. In this setting, our bounds coincide with existing optimality results (18). For mediation analysis, we are to the best of our knowledge the first to propose bounds for MSM-based sensitivity analysis. This is why we refrain from benchmarking against baselines in our experiments and only show the validity of our bounds.
Synthetic data:
We perform extensive experiments using synthetic data from various causal inference settings to evaluate the validity of our bounds. Synthetic data are commonly used to evaluate causal
inference methods as they ensure that the causal ground truth is available 66 16 76). Here, generate six different synthetic datasets with n = 50, 000 samples in the following manner: We first construct three SCMs with binary treatments for the causal graphs in Fig. [ that is. (i) no mediators. (ii) a single mediator M, and (iii) two mediators Mj and M2. We then construct three more SCMs for settings (i)-(ii) with continuous treatments. Details are Appendix H
To demonstrate the validity of our bounds, aim to show that they contain the oracle causal effect when- ever the sensitivity con- straints are satisfied. We evaluate two versions of our GMSM: the MSM for binary treatments and the CMSM for continuous treat- ments (see Sec. 3.3). Using oracle knowledge from the SCMs, we estimate the re- spective density ratio from Eq. 3 and obtain the oracle sensitivity parameter I+ (details are in Appendix H
Stenec p009
coppeniity
coppumaty
Figure 3: Results for the binary treatment setting. Settings (ii-i are ordered from left right. The top row shows the oracle sensitivity parameter i (depending on r),, and the bottom row shows the bounds.

---

We thus demonstrate that our bounds contain the oracle causal effect whenever Tw 2 Tip, i... whenever we choose sensitivity parameters w least large as the oracle Twy.
The results for binary treatments are shown Fig. 3 and for continuous treatments in Fig. 4
For binary treatments, we evaluate the causal effect Q(x, ae+1,MM for over É [-1, 11 with three (arbitrary) treatment combinations corresponding to the three different settings: (i) àj and (iii) å3 = (1,0,0..
1, (ii) 82 = (1,
For continuous treatments, we also use three (arbi- trary) treatment combina-
tions: (i) å
0.6,
56
14
(ii) åg (0.9, and (iii) as (0.2, 0.5.). Results for additional treat- ment combinations are in Appendix J Evidently, the oracle causal effect is con- tained within our bounds. Hence, the results confirm the validity of our bounds.
Figure 4: Results for continuous treatment setting. Settings (i)(iii) ordered from left to right. The top row shows the oracle sensitivity parameter I' (depending on .r), and the bottom row shows the bounds.
We also compare the bounds from Jesson et al. [2] with our sharp bounds under the CMSM and examine whether we can improve on these by using a different weight function (Def. 4 For this purpose, we modify the SCM from setting (i) in the continuous treatment setting that there is no unobserved confounding for individuals with > 0 (i.e., r 0). Details are in Appendix H. Table reports the bounds from Jesson et al., our bounds under the CMSM, and our bounds under a weighted CMSM with weight function qy (x) 1 (x > 0). The two main findings are: (i) Using the weight function, we can leverage prior knowledge about the confounding structure obtain even tighter bounds. (ii) Our bounds much faster to compute (using 5, 000 samples). The latter is because we derived closed-form solutions for our bounds, while the method proposed by Jesson al. is an approximation that uses grid search.
Table 1: Experiment with weighted CMSM
Ty
Interval length
Coverage
Time (MR))
1.2
Josom et (CMSM)
0.00
0.74
0.01
bounds MMM
0.00
0.01
1
0.30
Our bounds (weighted CMSM)
0.17
0.01
0.37
0.02
0.63
0.03
0.6
0,62
Reported: Average sundard deviation random (nest beld).
We demonstrate our bounds using example with real-world data. We consider setting from the COVID-19 pandemic where mobility (captured through telephone movement) was monitored to obtain leading predictor of case growth (57). Details regarding the data (publicly available) and our analysis are in Appendix []] Here. mobility is the mediator. case growth is the outcome, and stay-home order (ban of gatherings with more than people) the treatment. We are interested in the natural directed effect (NDE, see Example 2 of stay-home order on the case growth. We suspect that unobserved confounders between treatment and mediator might exist (e.g.. adherence, etc. of the population). Hence we perform a causal sensitivity analysis where we vary I'M and plot the bounds Fig. 5 We observe that the estimated effect under unconfoundedness (T's = 1) is negative, i.e., the stay-home order decreases case growth. For I'M > 1, we obtain bounds around this estimand. For Ar < the bounds are negative, which means that under moderate unobserved confounding, it seems likely that the NDE is nonzero, in line with prior evidence [57]
Real-world data:
6 Discussion
Assumptions: As common in the causal inference literature, our results rely on assumptions on the data-generating process that must be justified by domain knowledge. Our main assumption that we exclude confounders between mediators and the outcome in our analysis (see Def. 2). The reason

---

for this assumption is that it allows us to interpret the causal query from Eq. (I) as a path-specific effect. Path-specific effects are defined as so-called nested counterfactuals that lie the third layer of Pearl's causal hierarchy [55) Using the assumption from Theorem 1, they can be reduced to the query from Eq. ①.. which lies in layer's of Pearl's hierarchy, i.e., only depends on interventions [15). Our sensitivity analysis then bridges the gap from layer 2 to layer I (observational data). Approaches for relaxing this assumption, e.g., by combining our results with sensitivity analysis from layer 3 to layer 2, are left for future work.
Sensitivity parameter and weighting function: Both the sensitivity parameter I' and the weighting function g(a, x) incorporate domain knowledge about the unob- served confounding and need to be chosen by the practi- tioner accordingly.
0.5
-05
-1.5
The sensitivity parameter Iw controls the strength un- observed confounding. In practice, one typically chooses I' by domain knowledge or data-driven heuristics [35] 28]
-20
25
30
One approach is obtain the smallest I' that the cor- responding partially identified interval includes 0. Then, I can be interpreted a level of "causal uncertainty". quantifying the smallest violation of unconfoundedness that would explain away the causal effect [1] 33).
Figure 5: Estimated upper/lower bound for the NDE on real-world data. Re- ported: mean and standard deviation over 10 runs.
The weight function q(a, x) offers additional opportunities to incorporate domain knowledge about the confounding
structure to obtain tighter bounds. Consider an observational study on the effect of smoking on cancer risk, confounded by certain unobserved genes. For example, it may be known that genes that act as unobserved confounders do not affect the cancer risk for a certain population with certain covariates x, which allows us set g(a, x) 1. Note that we can always set q(a, x) P(a x) (discrete treatments) or q(a, x) = 0 (continuous treatments) if no domain knowledge is available, leading to established sensitivity models from the literature. For example. practitioners may use our bounds for the MSM in mediation analysis setting without ever explicitly using the GMSM formulation via
weighting function.
Other sensitivity models: In this paper, we provide bounds for MSM-type sensitivity models. Recently, other types of sensitivity models have been proposed in the literature, which may provide less conservative bounds in situations where the data-generating process does not follow an MSM. Examples include f-sensitivity models (34), Lz-sensitivity models [81), curvature sensitivity models (48) and the 5-MSM [45]. Extending our results to these sensitivity models may be another possible direction for future work.
Our main results (Theorem [ Corollary 2 are identifiability results, i.c., hold in the limit of infinite data. We did not provide results on efficient estimation. Therefore, future work may consider extending our approach to incorporate semiparametric efficiency theory (37]
Efficient estimation:
We proposed flexible generalization of the MSM and derived sharp bounds for variety of different causal inference settings and data types (e.g., continuous, categorical). Our work provides practitioners with unified framework for causal sensitivity analysis. This enables reliable causal inferences from observational data the presence of unobserved confounders, thus promoting safe decision-male makinn.
Conclusion:
10

---

Acknowledgments
SF acknowledges funding from the Swiss National Science Foundation (SNSF) via Grant 186932.
References
[1] Joshua D. Angrist. "Lifetime earnings and the vietnam draft lotter: Evidence from social security administrative records". In: The American Economic Review 80.3 (1990), pp. 313-336. [2] Chen Avin, Ilya Shpitser, and Judea Pearl. "Identifiability of path-specific effects". In: IJCAI. 2005.
[3] Vahid Balazadeh, Vasilis Syrgkanis, and Rahul G. Krishnan. "Partial identification treatment effects with implicit generative models". In: NeurlPS. 2022.
[4] Elias Bareinboim et al. "On Pearl's hierarchy and the foundations of causal inference". In: Probabilistic and Causal Inference: The Works of Judea Pearl (2022), pp. 507-556.
[5] Andrew Bennett, Nathan Kallus, and Tobias Schnabel. "Deep generalized method of moments for instrumental variable analysis". In: NeurIPS, 2019.
[6] Jeroen Berrevoets et al. "To Impute or not to Impute? Missing data treatment effect estima- tion". In: AISTATS. 2023.
[7] Ioana Bica, James Jordon, and Mihaela van der Schaar. "Estimating the effects of continuous- valued interventions using generative adversarial networks". In: NeurIPS. 2020.
[8] Ioana Bica et al. "Estimating counterfactual treatment outcomes over time through adversarially balanced representations". In: ICLR. 2020.
[9] Stephan Bongers et al. "Foundations of structural causal models with cycles and latent vari-
ables". In: The Annals of Statistics 49.5 (2021), pp. 2885-2915.
[10] Matteo Bonvini et al. "Sensitivity analysis for marginal structural models". arXiv preprint arXiv:2210.0466 (2022).
[11] Kan Chen, Bingkai Wang, and Dylan S. Small. "A differential effect approach partial identification of treatment effects". In: arXiv preprint arXiv:2303.06333 (2023).
[12] Victor Chernozhukov, Ivan Fernández- Val, and Blaise Melly. "Inference on counterfactual
distributions". In: Econometrica 81.6 (2013), pp. 2205-2268.
[13] Victor Chernozhukov et al. "Double/debiased machine learning for treatment and structural parameters" In: The Econometrics Journal 21.1 (2018), 8), pp. CI-C68.
[14] James Cornfield et al. "Smoking and lung cancer: Recent evidence and discussion of some
questions". In: Journal of the National Cancer Institute 22.1 (1959), pp. 173-203.
[15] Juan D. Correa, Sanghack Lee, and Elias Bareinboim. "Nested counterfactual identification
from arbitrary surrogate experiments". In: NeurlPS. 2021.
[16] Alicia Curth and Mihacla van der Schaar. "Nonparametric estimation of heterogencous treat- ment effects: From theory learning algorithms". In: AISTATS. 2021.
[17] David L. Donoho and Peter Huber. "The notion of breakdown point". In: A Festschrifi for Erich L. Lehmann (1983), pp. 157-184.
[18] Jacob Dorn and Kevin Guo. "Sharp sensitivity analysis for inverse propensity weighting via quantile balancing". In: Journal of the American Statistical Association (2022). [19] Jacob Dorn, Kevin Guo, and Nathan Kallus. "Doubly-valid/ doubly-sharp sensitivity analysis for causal inference with unmeasured confounding". In: arXiv preprint arXiv:21 12t.:211.11111 1449 (2022).
[20] Guilherme Duarte et al. "An automated approach to causal inference in discrete settings". In:
Journal of the American Statistical Association (2023).
[21] Conor Durkan al. "Neural spline flows". In: NewrIPS. 2019.
[22] Helmut Farbmacher et al. "Causal mediation analysis with double machine learning". In: The Econometrics Journal 25.2 (2022), pp. 277-300.
[23] Dennis Frauen and Stefan Feuerriegel. "Estimating individual treatment effects under unob-
served confounding using binary instruments". In: ICLR. 2023.
[24] Dennis Frauen et al. "Estimating average causal effects from patient trajectories". In: AAAI.
2023.

---

[25] Florian Gunsilius. "A mathssnpling method to partially identify causal effects in instrumental variable models". In: arXiv preprint arXiv:1910.0950% (2020).
[26] Wenshuo Guo et al. "Partial identification with noisy covariates: A robust optimization ap- proach". In: CLeaR. 2022.
[27] Jason Hartford et al. "Deep IV: A flexible approach for counterfactual prediction". In: JCMI. 2017.
[28] Tobias Hatt, Daniel Tschernutter, and Stefan Feuerriegel. "Generalizing off- off-po learning under sample selection bias". In: UAI. 2022.
[29] Jesse Y. Hsu and Dylan S. Small. "Calibrating sensitivity analyses to observed covariates in observational studies". In: Biometrics 69.4 (2013), pp. 803-811.
[30] Guido W. Imbens. "SSessitiitty to exogeneity assumptions in program evaluation". In: American Economic Review 93.2 (2003), pp. 128-132.
[31] Andrew Jesson et al. "Quantifying ignorance in individual-Ievel causal-effect estimates under hidden confounding", In: ICML. 2021.
[32] Andrew Jesson al. "Scalable sensitivity and uncertainty analysis for causal-effect estimates interventions". In: NeurIPS. 2022.
of continuous-valie
[33] Ying Jin, Zhimei Ren, and Emmanuel J. Candès. "Sensitivity analysis of individual treatment effects: A robust conformal inference approach". In: Proceedings of the National Academy of Sciences (PNAS) 120.6 (2023).
[34] Ying Jin, Zhimei Ren, and Zhengyuan Zhou. "Sensitivity analysis under the f-sensitivity models: A distributional robustness perspective". In: arXiv preprint arXiv:2203.0437 (2022).
[35] Nathan Kallus, Xiaojie Mao, and Angela Zhou. "Interval estimation of individual-level causal effects under unobserved confounding". In: AISTATS. 2019.
[36] Nathan Kallus and Angela Zhou. "Confounding-robust policy improvement". In: NeurlPS.
2018.
[37] Edward H. Kennedy. "Semiparametric doubly robust targeted double machine learning: review". In: arXiv preprint (2022).
[38] Edward H. Kennedy. "Towards optimal doubly robust estimation of heterogeneous causal effects". In: arXiv preprint (2022).
[39] Edward H. Kennedy, Sivaraman Balakrishnan, and Larry Wasserman. "Semiparametric coun- terfactual density estimation". In: Biometrika (2023).
[40] Niki Kilbertus, Matt Kusner, and Ricardo Silva. "A class of algorithms for general instru- mental variable models". In: NeurIPS. 2020.
[41] Diederik P. Kingma and Jimmy Ba. "Adam: A method for stochastic optimization". In: ICLR.
2015.
[42] Sören R. Künzel et al. "Metalearners for estimating heterogeneous treatment effects using machine learning". In: Proceedings of the National Academy of Sciences (PNAS) 116.10 (2019), pp. 4156-4165.
[43] Bryan Lim, Ahmed M. Alaa, and Mihaela van der Schaar. "Forecasting treatment responses over time using recurrent marginal structural networks". In: NeurIPS. 2018
[44] Charles F. Manski. "Nonparametric bounds on treatment effects". In: The American Economic
Review 80.2 (1990), pp. 319-323.
[45] Myrl G. Marmarelis et al. "Partial identification of dose responses with hidden confounders".
In: UAI. 2023.
[46] Valentyn Melnychuk, Dennis Frauen, and Stefan Feuerriegel. "Causal transformer for estimat-
ing counterfactual outcomes". In: ICML. 2022.
[47] Valentyn Melnychuk, Dennis Frauen, and Stefan Feuerriegel. "Normalizing flows for interven-
tional density estimation". In: ICML. 2023.
[48] Valentyn Melnychuk, Dennis Frauen, and Stefan Feuerriegel. "Partial counterfactual identifica-
tion of continuous outcomes with a curvature sensitivity model". In: NeurIPS. 2023.
[49] Krikamol Muandet et al. "Counterfactual mean embeddings". In: Journal of Machine Learning Research 22 (2021), pp. 1-71.
[50] Razieh Nabi, Daniel Malinsky, and Ilya Shpitser. "Learning optimal fair policies". In: ICML.
2019.
[51] Razich Nabi and Ilya Shpitser. "Fair inference on outcomes". In: AAAI. 2018
12

---

[52] Ziad Obermeyer et al. "Dissecting racial bias in an algorithm used to manage the health of populations". In: Science 366.6404 (2019), pp. 447-453.
[53] Miruna Oprescu et al. "B-learner: Quasi-oracle bounds on heterogeneous causal effects under hidden confounding". In: ICML. 2023.
[54]
Kirtan Padh et al. "Stochastic causal programming for bounding treatment effects".. In: CleaR. 2023.
[55]
Judea Pearl. Causality. New York City: Cambridge University Press, 2009.
[56]
Judea Pearl. "Interpretation and identification of causal mediation". In: Psychological methods 19.4 (2014), pp. 459-481.
[57]
Joel Persson, Jurriaan F. Parie, and Stefan Feuerriegel. "Monitoring the COVID-19 epidemic with nationwide telecommunication data". In: Proceedings of the National Academy of Sciences of the United States of America 1 118.26 (2021).
[58]
Danilo Jimenez Rezende and Shakir Mohamed. "Variational inference with normalizing flows".
In: ICML. 2015.
[59] James M. Robins, Miguel A. Hernán, and Babette Brumback. "Marginal structural models and causal inference in epidemiology". In: Epidemiology 11.5 (2000), pp- 550-560.
[60] James M. Robins, Andrea Rotnitkky, and Daniel o. Scharfstein. "Sensitivity analysis for selection bias an unmeasured confounding in missing data and causal inference models". In: Statistical Models in Epidemiology, the Environment, and Clinical Trials 116 (2000), pp. 1-94.
[61] Paul R. Rosenbaum. "Sensitivity analysis for certain permutation inferences in matched observational studies". In: Biometrika 74.1 (1987), pp. 13-26.
[62] Paul R. Rosenbaum and Donald B. Rubin. "Assessing sensitivity unobserved binary covariate in an observational Study with binary outcome". In: Journal of the Royal Statistical Society: Series B 45.2 (1983), pp. 212-218.
[63] Uri Shalit, Fredrik D. Johansson, and David Sontag. "Estimating individual treatment effect: Generalization bounds and algorithms". In: ICML. 2017.
[64] Claudia Shi, David M. Blei, and Victor Veitch. "Adapting neural networks for the estimation of treatment effects". In: NeurIPS. 2019.
[65]
Rahul Singh, Maneesh Sahani, and Arthur Gretton. "Kernel instrumental variable regression". In: NeurlPS. 2019.
[66]
Vasilis Syrgkanis et "Machine learning estimation of heterogeneous treatment effects with instruments" In: NeurIPS. 2019.
[67] Zhiqiang Tan. "A distributional approach for causal inference using propensity scores". In: Journal of the American Statistical Association 101 476.47 (2006), pp. 1619-1637.
[68] Eric J. Tchetgen Tchetgen and Ilya Shpitser. "Semiparametric theory for causal mediation analysis: Efficiency bounds, multiple robustness, and sensitivity analysis". In: Annals of Statistics 40.3 (2012), Pp. 8316-8455
[69] Mark J. van der Laan and Donald B. Rubin. "Targeted maximum likelihood learning". In: The International Journal of Biostatistics 2.1 (2006).
[70] Stijn Vansteelandt et al. "Ignorance and uncertainty regions as inferential tools in a sensitivity analysis". In: Statistica Sinica 16 (2006), pp. 953-979.
[71] Hal R. Varian. "Causal inference in economics and marketing". In: Proceedings of the National Academy of Sciences (PNAS) 1 113.27 (2016), pp. 7310-7315.
[72] Stefan Wager and Susan Athey. "Estimation and inference of heterogeneous treatment effects using random forests". In: Journal of the American Statistical Association 113.523 (2018), pp. 1228-12222 1242.
[73] Christina Winkler et al. "Learning likelihoods with conditional normalizing flows", In: arXiv preprint arXiv:1912.00042 (2019).
[74] Kevin Xia, Yushu Pan, and Elias Bareinboim. "Neural causal models for counterfactual identification and estimation". In: ICLR. 2023.
[75] Kevin Xia et al. "The -eeural connection: Expressiveness, learnability, and inference".
In: NeurlPS. 2021.
[76] Liyuan Xu al. "Learning deep features in instrumental variable regression".. In: ICLR. 2021.
[77] Azam M. Yazdani and Eric Boerwinkle, "Causal inference in the age of decision medicine".
In: Journal of Data Mining in Genomics & Proteomics 6.1 (2015).
13

---

[78] Jinsung Yoon, James Jordon, and Mihaela van der Schaar. "GANITE: Estimation of individu- alized treatment effects using generative adversarial nets". In: ICLR. 2018.
[79] Marco Zaffalon, Alessandro Antonucci, and Rafael Cabañas. "Structural causal models are (solvable by) credal networks". In: International Conference Probabilistic Graphical
Models 2020. 2020.
[80] Junzhe Zhang. Jin Tian, and Elias Bareinboim. "Partial counterfactual identificatiton from observational and experimental data". In: ICML. 2022.
[81] Yao Zhang and Qingyuan Zhao. "Bounds and semiparametric inference L and L sensi- tivity analysis for observational studies". In: arXiv preprint Ttt::2111004697 (2022).
[82] Qingyuan Zhao, Dylan S. Small, and Bhaswar B. Bhattacharya. "Sensitivity analysis for inverse probability weighting estimators via the percentile bootstrap". In: Journal of the Royal Statistical Society: Series 81.4 (2019), pp. 735-761.
14

---

A Extended related work
A.1 Partial identification under unobserved confounding
There are various works for partial identification (i.e., bounding causal effects) under unobserved confounding that do not impose sensitivity models. In the following, we discuss the difference between this literature stream, which we call causal partial identification (CPA), to causal sensitivity analysis (CSA).
The main difference between CSA and CPA is that CSA imposes sensitivity models, that is, as- sumptions on the strength of unobserved confounding, which is controlled by sensitivity parameter T. Methods for CPA do not impose sensitivity constraints but instead impose other assumptions. In practice, CSA can be used test the robustness of causal effect estimates violations of the unconfoundedness assumption (by varying I'), while CPA may be applicable in situations where no domain knowledge about the confounding strength is available.
Approaches for CPA can be roughly described by two categories that describe the type of assumptions that are imposed to derive informative bounds.
1. Additional variables: In order to achieve informative bounds without restricting the strength of unobserved confounding, some works impose assumptions on the data-generating process by postulating the existence of additional variables. One example is instrumental variables (IVs), i.e., variables, which only have direct effect on treatment variables but not on outcomes. Under certain assumptions, IVs render bounds for causal effects informative
without assumptions on the underlying confounding structure [3 25 40 54 Other examples
include leaky mediation [3 54), differential effects (11), noisy proxy settings (26).
2. Discrete SCMs: Another stream of literature derives informative bounds in discrete SCMs. Examples include 20
75
79
80
Comparison with our bounds: None of the methods above is applicable in the causal inference settings we consider (e.g., continuous treatments or outcomes, no IVs available, etc.). In contrast, a well-known CPA result in our setting is the so-called no-assumptions bound (44) that leverages discrete treatments. In the following, we consider the standard setting for CATE estimation without mediators (e = 0) and a binary treatment A E {0, 1}. We show that we obtain the no-assumptions
bound with our sensitivity analysis for the query EE x. do(A a)] for - oc, i.e. when lifting the sensitivity constraint. Let (pt, P2] denote the support of the conditional density P(y x, a) with c.d.f. F. Using the MSM we obtain
((FT)
axxx
y ((1-r~))P( x) +r-1) Ply x, dy
(9)
P1
P2
y ((1 - (PPPa x) + 1') Ply x, a) dy
(10)
F(a
EY
x, a| + Pa x)) p2:
(11)
T-000
which corresponds to the result in [4 The result for the lower bound Q (x, a, S) follows analo- gously.
A.2 Estimation of causal effects under unconfoundedness
Under certain additional assumptions, unconfoundedness makes it possible to point-identify causal effects from the observational data, so that the causal inference problem reduces purely statistical estimation problem. Various methods for estimating point-identified causal effects under unconfound- edness have been proposed that make use of machine learning and/or (semiparametric) statistical theory. Examples include methods for conditional average treatment effects [33
16
38
42
63 72
78), average treatment effects (24 64 69). instrumental variables (5] 23 27
66 76,, varyyng
data [8] 43 46). mediation analysis [68] [2]]
22), and distributional effects 12 39 47 49 Note that all
the methods above are biased if the unconfoundedness assumption is violated, which outlines the
need for our causal sensitivity analysis.
15

---

B Proofs of the GMSM bounds
B.1 Proof of Theorem 1
Proof. We give the proof for continuous W E R (see Eq. (4) in the main paper). The derivation for discrete W E N (see Eq.
(5) in the main paper) follows with the same arguments and the X, mw, a) = 1. We first show the inequality
normalization constraint En m (w
F+(w) inf FM(()
(12)
MMC(IS)
and then provide sharpness results by showing
(w))
inf FM(w)
(13)
MEC(S)
under the sharpness conditions of Theorem
The result for F- (w) follows analogously.
Validity of the bounds (=): We provide proof by contradiction. To do so, we assume that there exists an SCM M E C(S) and w E R so that
F+w) > FM(w).
(14)
By the definition of F+ (w), there must exist a set Wi S (cF()")))) so that
P+ (w01 x, mw a) > P(w1 x, m., do(A = a)) for all wn E W1.
(15)
or a set W2 C B- so that
P++ (wo2 x, mw, a) P(2z m, do(A 10o0A-0) =)) for all w2 E W2.
(16)
otherwise P(w x, mw do(A a)) would not integrate to 1. Let W = f(X, Mw, A, Uw) be
the functional assignment of M and let U1 fx, (W1) S R and U2 = fx.m (W2) S Rd
denote the preimages of W1 and W/2 under fx,mw.a in the confounding space.
We can again write (w x, mw, a) as a push forward
Ip+ (w x, mw, a) = fx.mw= a pp pUwlxx (w)
(17)
for some density P+ (uw and Eq. (28) we obtain
x, a) on the confounding space. By the definition of F+(w x, mw,a) a)
1 P(u x,a)
P (u x)
P(u, x)
and (u x.
(18)
SW
for all u1 E U4 and u2 E U2. Due to the definition of 2f and U2 it follows that there exist u1 E U4 and ug U2z so that
P(u, x, a)
Pl, xx
(19)
P(u: x, do(A A = =]]
P+ (x,a)
and
P(u
x,a) a)
Pu xx
P+ xx sw-
(20)
P(u do A = a))
Both Eq. 119 and Eq. (0) are contradictions the GMSM constraint Eq. 13 of the main paper.
Hence, M M C(S).
We show that under the sharpness conditions from Theorem I there exists an SCM M E C(S) with induced interventional density P+ x, mww a) for all w. The construction of M is similar to that of our motivational toy example in Sec. 4. of the main paper. We first define an (interventional) probability density for the unobserved confounder Uw E Rd given X via
Sharpness of the bounds (2):
P(uw x, do(A a)) 1(0 < u S (1/ 1(1 2 w > (1/sw),
(21)
(1) where un denotes the first coordinate of uw. P(uw x, mw) is a properly normalized density with support [0, 1] because
/ P(uw x, xodo(A A duw
CW
G.
1 1 Sw
SW 1
1, (22)
SW
Sw
st
Sw
Sw
Sw
16

---

where we used the definition cw
(1- (1-8w
Bir -
We now define the probability density for the unobserved confounder Uw E IR given X, Mw, and treatments A the uniform density [0, 1]6, i.e.,
PP(uw x. a) 1(0 s uw 1);
(23)
where the indicator function is defined coordinate-wist
We now need to show that there always exists an SCM M which induces the densities in Eq. 21
and Eq. (22) and that satisfies the sensitivity constraints. For this purpose, we use the assumptions to write the interventional density for discrete A
Puu x, do = a))) Puw x) = P(a x) + (1 [P(a] x) P)) x, *
(24)
Hence, our definitions for PPuw
x, do( and P(uw xx a) are valid whenever
PP(uw x, do A a)) P(a x)
S P(uw
x, A + a)
(25)
1 Paaa x)
which follows from the sharpness condition 1/8 2 because we can approximate with a sequence of discrete treatments (A,), and it holds that (as long P(a x) non-degenerate). Finally, holds that
Pa x). Sharpness for continuous A follows
P(an. x) n-+00
P(Uw = uw x, a)
x, do a)) S sw for all uw E 1J,
(26)
W
P(Uw
uw
so that M respects the sensitivity constraint of the GMSM S.
Let now TP(w mw: a) denote the observational density of W given X, Mw, and A with corresponding cumulative distribution function (CDF) given by F(w). To complete our construction of M, we define functional assignment W fw (X,
(w(X, Mw, A, Uw) via the inverse CDF
[w((, mw, a, uw) = F-1
WW
(1)
(27)
By denoting (x, mw, a, as fx.m as we can write the observational distribution under M as the push forward
fx.mw a pu bx. (w)) = PFW x, ww a)
(28)
due Eq. 22 and Eq. 27 Note that we used here that Uw is not parent of Mw (see Fig. I Hence, M C(S) is compatible with the sensitivity model S. Furthermore, the induced interven-
tional distribution can be written the push-forward
fx.mw a p pU (o(A (w) (w X, mw,aj a)
(29)
because of Eq. (21)
B.2 Proof of Corollary
1
Here, we formally restate Corollary 1 for monotone functionals. For two probability densities PP()) and (y)) we denote < IP' if holds almost surely for the corresponding CDFs.
Definition 5. A functional D called monotone D(P(.)) S D(P)(.)) whenever P < P
Intuitively, monotone functional increases if applied on distribution that is further right-shifted. Note that both the expectation functional D(P(()) = yP((y) dy and the quantile functionals D(P(-)) = (a) for a E [0, 1] are monotone.
Corollary (Restatement). If M 0 and monotone, we obtain sharp bounds
Q (x = D (F) x, a)) and Q a = (P x, a)).
(30)
Proof. Follows directly from Theorem for W = Y.
17

---

B.3 Proof of Corollary 2
Proof. We derive Algorithm for Q* (x, ae+1, S). The case for Q (x, ac+1, follows analogously.
Recall that we want to maximize the causal effect
Q(x,ā, DPX (- m, do(A ac+1)) II P(m, xx 1, =a)). (31)
over all possible SCMs M E C(S) that are compatible with the GMSM S. By using the assump- tion (no unobserved confounding between mediators and outcome), we can write Q(x, a, M) in terms of functional assignments fx.mw.a a defined via W fW (X, Mw, A, Uw) and induced (interventional) distributions pUw|x |x in the following way:
Q aM+1. (x.Fmmma p II
M x
pUM, (m.)).))...
(32)
m
Hence, the optimization problem reduces to maximizing Eq. (3) over all functional assignments FW fxmmw.a and distributions TUw/> xx that are compatible with S. Note that the terms in the product do not depend each other the term in the sum. Thus, by rearranging the suprema and products, can equivalently perform the following iterative procedure: First, we initialize
2t (x, my S)+1, = sup D (f puv))())
(33)
MEC(S)
and then define
O (x, m 5) = sup MEC(S)
E (x m, ae+1)) (Iam, i purlx (34)
for all i E {l...., .....,,,,, 1}, which results in the sharp upper bound
Q+ (x, 1. S) = Qf (x, S)+
(35)
For Eq. 33 and monotone D, we can directly apply Theorem and obtain
Q m. (e+1, = (P x, m, a(+1))
(36)
pUr Mj that is compatible with S and puts probability mass m, where Q (x, m; āe+1)) is large. Hence, apply the discrete version of Theorem I with W (M,),, where supp(M,) supp(M, is permutation map that (Q+ (x, m,-1, π (m:)), az+1) mcExupp(M.) is ordered in ascending order.
For Eq.
34
need find an induced distribution fx #
The corresponding update step is shown in Algorithm !
18

---

C Special cases of the GMSM
In this section, we prove Lemma] | i.e., we show that all sensitivity models introduced in Sec. 3.3 of the main paper are special cases of our (weighted) GMSM. Recall that we consider settings without mediators (i.e., weight function, and U Uy for the unobserved confounders. In this case, the weighted GMSM is defined via the confounding restriction
0), and write I' I' for the sensitivity parameter, q(a, x) qy (a, x) for the
PI(U a)
P(U = x, do = S ((a -T--
(37)
(1 - ()
C.1
Marginal sensitivity model (MSM):
The MSM 67 for binary treatment A A {0, 1} is defined via
1
π (x)
x u)
ST,
(38)
r
(x)
π u
where π (x) (AA 1| x) denotes the observed propensity score and 7(x, u) PP(A x,u) denotes the full propensity score. By rearranging the terms, we obtain
P(a
x, u) x)
1
(39)
P) x) +
P(a
(1-1-1)P(a
(x) +---
for a E {0, 1}. Furthermore, by Bayes' theorem, it follows that
P(a x, u) [P(u x, a) x) Puu x,a) Puu a) P(a x) P(u x)P x) Puu x)
(40)
Puu x, do (A a))
which implies that the MSM is weighted GMSM with weight function g(a, (P x).
Comparison the MSM definition using potential outcomes: Some papers define the MSM in terms of potential outcomes (Y1, Yo) instead of unobserved confounders U [31 67 Here, Y, denotes the potential outcome under the treatment intervention do(AA a). In the following, we show that this is equivalent to the definition we use in our paper.
Lemma 2. The following two statements are equivalent:
1. There exists an unobserved confounder U that satisfies Y1. 1 A X, U so that it holds that s < F(ulx.a) P(u[x]
< ss
2. It holds that s- P(Y Yo x.a) Yo|x)
Proof. The second statement follows directly from the first one by defining U (Y1, Yo). For the other direction, we proceed via proof by contradiction. Assume there exists a pair (Y1, Yo) that violates the second statement, say w.l.o.g. P(Y, Yolx.a) > st. We the independence condition from the first statement write
IP(Y, Yo x. = P(Y,, Yo x, a,u x, du / P(Y,, Yo x, u)) x, a) du (41)
Furthermore, we have that
P(Y,, Yo x) = Plyl, / yo
x. u) x) du.
(42)
follows that
P(Y,, Yo x, u) x, a) du
s Hence, there exists a such that * F..... the sensitivity constraint of the first statement.
(43)
P(Y1, Yo
u x) du
P(uxx.a)
which is a contradiction to P(ui))
19

---

C.2
Continuous marginal sensitivity model (CMSM):
For continuous treatments R the continuous marginal sensitivity model (CMSM) 32 is defined via
1
ST
(44)
T P(a x)
With the same arguments as in Eq. (40), it follows that the CMSM is a weighted GMSM with weight function (a, 0.
C.3 Longitudinal marginal sensitivity model (LMSM):
For longitudinal settings with time-varying observed confounders unobserved confounders U = U- (UI, Ur), treatments A AT (Ai,..., Ar), we define the longitudinal marginal sensitivity model (LMSM) via
XT (XI,...,Tr)
ar XT, Ut a 1) Ny. a
ST
(45)
P(a:
holds that
P(
XT, ac)
T
P(ū;
XT, āz)
P(ür
87, aT)
(46)
1=11
Pū
XT, āt)
t=1
[
XT, ar
P(ay
XT,, Ue ax-1)
P(ū
8xT, ag az-1)
(47)
(Paa:
XT, ac-1)
P(ū,1 1
XT, a
<==1
T
P(ay
Xy, Ug a
P(u XT, a,-1, u1-
(48)
P(a: XT, ar-1)
T P(ay Xy Ü. a
P(ur XT, do (Ar är)),
(49)
P(a: XT,
where (+) follows by applying Bayes' theorem on P(ūú Xr. az). Hence, the LMSM is a weighted GMSM with weight function q(ār, xr) Note that we can also define LMSMs with different weight functions via
P(a. Xy, ū. āt-1)
I -1-1"-1)gaa7 1' XT + 1-11
(50)
(a) XT) +1
P a, XT a:-1)
20

---

D Bounds for average causal effects and differences
Here, we show that we can use our sharp bounds to obtain sharp bounds for causal effect averages and differences. We state the results for the upper bound
Q (x. 1. =
sup MEC(S)
Q(x, a,+1,
(51)
All definitions and bounds for the lower bound Q (x, S) can be obtained by swapping the signs.
We are interested in the sharp upper bound for the average causal effect
/ ((xxat+1,M)dx MEC(S)
Q S)
sup
(52)
and the sharp upper bound for the causal effect difference
(1) (2) Qti (x, a a. a
(1) (x Q a
(xx,,a/t)_))) (H
sup MEC(S)
(53)
Lemma can compute Qave (ag-1, S) and Q a ;4 from our sharp bounds Q+ (x, S) and Q (,,1, via
Q / (x+,,xx+,)
(54)
and
Q (x,a (1) (2)
8,4 1441,0841,0 s) Q+ (x, a 8) - (x, (x.
(55)
Proof. The result for Q (5)+1, follows directly from interchanging the supremum and integral. (1) (2)
For Qaift
x, a a S) we note that
Qia a a () S sup
Q(x, a (1) M,)
inf M2E((S)
Q(x, a M2)
(56)
MiEC(S)
Q+ 8 Q -x
(1)
(57)
To show the equality in Eq. (56 show that, for each pair of SCMs Mi, M2 E C(S), find an SCM M E C(S) such that
Q(x, a M1) - Qxx, a a Q(x, M) M) Q(xx MM)
(58)
We assume w.l.o.g. that all M C(S) induce the same distributions pUwlx on the confound- ing space. We denote the functional assignments of Mi and M2 as fM, (x, mw a, uw) and fM (x, mw. a, ) We can now define a functional assignment fM (x, mw. a, uw) for M so that
MM (,,)) SM (y ( (),)) and (2),)) = (2))(
(59)
which implies Eq. (58)
21

---

E Compatibility of SCMs with sensitivity models
Here, we provide details regarding our class C(S) of SCMs that are compatible with sensitivity model S from Def. 2
Definition (Compatibility). Let PV denote the distribution of the observed variables (X, A, Mr, Y) and let 8 be sensitivity model with unobserved confounders U and family of joint distributions P. Let M be an SCM with observed variables VM = V and unobserved variables UM 2 that contain the unobserved confounders U from the sensitivity model but also potential exogenous noise. Then, M is compatible with S, the following three conditions are satisfied:
1. Graph compatibility: the induced causal graph GM by M on UUU must coincide with our assumed causal structure (see Sec. 3.1
2. Latent unconfoundedness: P(w x, mw, do(A a)) = P(w x, uw, mw. a))((uw)
x, mw ) duw for all x E X, mw supp Mw) ). A, and W (M...... Mr, Y).
3. Sensitivity constraints: .M E P, where P denotes the distribution on VUU induced by M.
In Definition 6, the latent unconfoundedness condition ensures that all interventional densities P(w X, mw; do A a)) are point-identified under access the unobserved confounders. In particular, this ensures that the SCM M can not contain additional unobserved confounders (e.g.. between mediators and outcome). Similar assumptions are standard in the causal sensitivity literature 18
22

---

F Importance sampling estimators for finite sample bounds
In this section, we derive estimators for the outcome bound D (P (: x, meg a)) for continuous
Y E R. We assume that we have already obtained an estimator PY (- x, mz, a) of the observational distribution (- x, m, a), and that we are able to sample (3/ )
P x, me, a) (see Appendix G foor implementation details). Note that the outcome bound D (P x, m, a)) depends on the shifted distribution P (- X, Mi a) and not on the observational distribution P (- x. M. a). Hence, we use an importance sampling approach derive our estimators, which we outline in the following for the expectation functional and distributional effects. We denote the CDFs corresponding P. (- x, my, a) and PY x, me; a) by F p px..map. and Fprix.m... respectively.
Expectation functional: For the expectation functional, D (IP) (.1 a)) E y
rewrite the outcome bound
[Y]
(60)
P. (Y x, m a) pr (Y
By
(61)
m a) pr mx (att)
Eyaprixmaa
Y
1 (y S
Y + = 1 (r
F
F
p (c+)
mi
By
by
(62)
to obtain the consistent estimator
|ke
(PX(- me, a))
1
Vi
yi
(63)
SY
k
i=lke||+
++1
where (wi) p (. mg, a) are sampled from the estimated observational distribution. This corresponds to Eq. (8 in the main paper.
Computational complexity: Given trained models and a sample (44)= ~ P- x, my a).
estimator from Eq. (63) has complexity of Ok) as it only involves summing and quantile computation. To set this in relation to existing work, Algorithm 1 of Jesson et al. (32) has a complexity of O(k)), where is number of grid search points. This is demonstrated in Table 1 where choose k = n 5000.
Distributional effects: We now derive estimators for distributional effects, i.e. for quantile function- als D of the form
D (P m. a)) - " (a)
(64)
with a E (0, 1). We again use importance sampling approach and rewrite
F Fpyix.m,. (v) = R YNpp i.. (1 -
(65)
x, my a) 1(Y < 1 pr (Y m a)
P. (Y
(66)
[(Y < minty,
v, F f (ct)3) 1 (Fpttmm,a (ct) < Y S 8)
8y
(67)
Hence, we can sample (y: 15 1p (. x, my a) and obtain the consistent estimator
D (P x, mr, a))
min F.x.m (8))aa
yi
(68)
F
where
|ka/
F x.z (y)
Σ
(1/i S
1 1 S v)
(69)
4 i=1
k
ket J+1
23

---

G Implementation and hyperparameter tuning details
All our experimental settings feature continuous outcome Y R and (optionally) discrete mediators M; E N. Hence, we need to estimate the conditional outcome density IP' (- x, m, a) and conditional probability mass functions P.M (- x, m, a) in order to estimate our bounds with Eq. 663 Eq. (68
and Algorithm !
from the main paper.
Conditional outcome density: We use conditional normalizing flows (CNFs) 73 for estimating the conditional density IP) (. x, m,, a). Normalizing flows (NFs) model a distribution p of target variable Y by transforming simple base distribution pe (e.g., standard normal) of latent variable U through an invertible transformation Y = fe(U), where & denotes learnable parameters [58). In order estimate the conditional density _ (- x, M,,, we leverage CNFs, that is, we define the parameters 0 as an output of a hyper network 0 = 9n (x, M,, a) with learnable parameters Given 1 V- we learn y) by maximizing the log-likelihood
sample {x,, me,s, a,, yt
{(n) Σ log (eenn)) Pu (wi)
(70)
log (pe/ (v)))))en,:(iii)) + log (
dy (
(71)
where (x,(x,,m.. m.))..))) #
pll (v:) denotes the (push-forward) density induced by /9 I and (x) follows from the change-of- ofrvibbll theorem for invertible transformations.
In our implementation, via a spline flow as described in [21]. We use a feed-forward neural network for the hypernetwork (x, my, a) with 2 hidden layers, ReLU activation functions, and linear output. We set the latent distribution PP standard normal distribution (0, 1). For training, we the Adam optimizer
neural spline flows. That is, model the invertible transformation fo
41..
Conditional probability mass functions: The estimation of the conditional probability mass function p. x, a) is a standard (multi-class) classification problem. We use feed-forward neural networks with 3 hidden layers, ReLU activation functions, and softmax output. For training, we minimize the standard cross-entropy loss by using the Adam optimizer (4T). We use the same approach to estimate the propensity scores pd x) for discrete treatments A.
Hyperparameter tuning:
We perform hyperparameter tuning for our experiments on synthetic data using grid search on a validation set. The tunable parameters and search ranges are shown in Table 2 For reproducibility purposes, we report the selected hyperparameters as .aml files
Table 2: Hyperparameter tuning details.
MODEL
TUNABLE PARAMETERS
SEARCH RANGE
CNFs
Epochs Batch size
32, 64, 128
Learning rate
0.0005, 0.001, 0.005
Hidden layer size (hyper network)
5, 10, 20, 30
Number of spline bins
4. 30
Feed forward neural networks
Epochs Batch size
32, 64, 128
Learning rate
0,0005, 0.001, 0.005
Hidden layer size
5. 10. 20, 30
Dropout probability
0, 0.1
*Code is available in the supplementary materials and
24

---

H
Experiments using synthetic data
Here we provide details regarding our experiments using synthetic data. This includes data generation, obtaining oracle sensitivity parameters, and details regarding experimental evaluation.
Overall dat-ggeneg process: We first describe the overall data-generating process which use as basis to generate data for all settings (i)-(iii) and binary/continuous treatments. We construct an SCM following the causal graph in Fig. (right) from the main paper. We have an observed confounder X E R. a (binary or continuous) treatment A, two binary mediators M1 and M2, and continuous outcome
E I. Furthermore, we consider three unobserved confounders: (i) UM. confounding the A-M1 relationship, (ii) UMa confounding the A-M2 relationship, and (iii) Uy confounding the A-Y relationship. Our Lata-generating process is inspired by synthetic experiments from previous works on causal sensitivity analysis [3] 35). We start the data data-generating process by sampling
(1.idd) ~
X ~ Uniform(-1,1], and UM UMg Uy
Bernoulli 0.5)
(72)
Depending on the setting, we either generate binary treatments A E {0, 1} via
A ~ Berroull(ssigmoi((.. 7M, uM, + YM, UM, + Tyuuy)
(73)
or continuous treatments A E (0, 1) via
Beta(a, 3) with a = 3 = 2 YM, (um, - 0.5) + TM, (UM, - 0.5) + (uy -00yy0... 0.5), (74)
where "M TM2 and "y are parameters controlling the strength of unobserved confounding. We then generate the mediators and outcome via functional assignments
Mi fas, (X, A, U,U,,,
M2 fM, (X, A, M1. UM, (M2)
(75)
and
Y == fx, M1, M2Uy, where EDd, Ey ~ (O, 1) are standard normal distributed noise variables. The functional assignments are defined
(76)
SM (x, eM) 1 sin(() + (1 a) sin(4x) - PMi (um 0.5) + +EM, > 0} (77)
for M1,
[Ma a, minm,, =1 {a m. sin(x) a) mi sin(4x)
(78)
a (1 m.) sin(x) - (1 - a) - (1- sin(4x)
(79)
Pad, ((uM, 0.5) + <M,) 0)
(80)
for M2, and
mi, -a mi m2 sis()) (1 - a) mi min(4()))
(81)
+ a amm (1 - sin + (1 a) mi (1 m2) sin(x)
(82)
a (1 m1)
(1 -1 a) (1 m1) m2 sin a mi) (1 10) sin
(83)
(84)
a) (1 m1) m2) sin(
(85)
+ pY ((uy - 0.5) + cy)
(86)
for Y, where PM,, PMr,, and py are parameters that control the noise level.
Settings (i)-(iii): We define the settings (i)-(ii) in Sec. 5 via specific values of the confounding parameters 7M 7Mg, and Ty, and the noise parameters PM, PM, and py (see Table Note that the settings are defined to mimic the causal graphs in Fig. [ from the main paper. For example, the only unobserved confounder in setting (i) Uy which means that we can ignore the mediators and our data to evaluate our bounds for settings without mediators.
25

---

Table 3: Definition of settings (i)-(ii)..
7/1
MM2
TY
PM1
PM2
Setting (i). binary A
0
0
1.5
0.2
0.2
2
Setting (i), continuous A
0
0
1.5
0.2
0.2
Setting (ii)
1.5
0
1.5
0.2
Setting (iii)
1.5
1.5
1.5
0.2
0.2
Obtaining rip: We provide details regarding our approach to obtain oracle sensitivity parameters Ti for all W E (M1, M2, ). By sampling from our previously defined SCM, we can obtain Monte Carlo estimates of the GMSM density ratio
P(uw x, a)
(+) P(a
x, uw)
r(uw,ar a) = Pllww a) P(a
(87)
x)
for all uw E (0, 10, and x, where (*) follows from Bayes' theorem. We then define
Ww (1, a)
max r(w,, I a) uw E(0.)
and (2, min r("w, 2. Ew
(88)
For binary treatment settings, we define parameters the density ratio bounds in the MSM from Eq. 33 i.e.
I I (a, a) and Tw -w (x, a) that attain
Ww (x) a)
and
Tw (x, a) - (1-Tw w P((a x) I
(89)
(1 - rw P((a x) +
For continuous treatment settings, we define w and w as the sensitivity parameters that attain the density ratio bounds the CMSM from Eq.
i..
w (z, a) = T andr w (2,
(90)
Fw
Finally, we define I' the parameter corresponding to the maximum possible violation of uncon- foundedness, i.c.,
Fw max
(1))
By definition of Fiv, our bounds should contain the oracle causal effect whenever we choose sensitivity parameters Tw Thw for all W (M1 M2, Y).
Weighted GMSM experiment (Table []: For our experiment in Table I we modify the treatment assignment from Eq. (74) in setting (i) to
A ~ Beta(a, 3)
(92)
with
a=3=2 2 2++++11 < <0 (1M, (uM, - 0.5) + YMs (uM, - 0.5) + (uy
- 0.5)).
(93)
Hence, unobserved confounding only affects individuals with x < We then compare our bounds under the CMSM with our bounds under a weighted CMSM (Def. with weight function (z)
1(x 0).
We also provide results for the bounds from Jesson et al. 32 under the CMSM. We implemented the grid search algorithm from Jesson al. (32 and used 5, 000 samples for the search space. For fair comparison, we also used 5, 000 samples for our importance sampling estimators. Note that the method from Jesson al. (2) requires estimation both the conditional outcome density Ip' x, a)
and the conditional expectation E x, a). For F2 (- x, a), we use the same (normalizing flow- based) estimator for our bounds. For E[ x, a), we train separate feed-forward neural network with linear output activation for continuous outcomes. Implementation and hyperparameter tuning done the same way as described Appendix G for the feed-forward neural networks.
26

---

I Experiment using real-world data
Data: We consider setting from the COVID-19 pandemic where mobility in Switzerland (captured through telephone movement) was monitored to obtain leading predictor of case growth. In total, 1,5 billion trips were monitored from 10 February through 26 April 2020. All data are recorded across 26 different states (cantons). For our analysis, we use an aggregated, de-identified, and pre-processed version of the data provided by Persson, Parie, and Feuerriegel (57]. The preprocessed The www.e/oooeeeeeeraaaaieeeaa Franeeennnnsaaannnaaaaanaeaa Saa.eeannaaarrreeeeeaarrasaaaaaa ChansaaaallSnsssiiiiii
htwp:///ooooiiiomooomiiiilillolo data is publically available at https:// hawp://iiwb...ooeeeeeeeaaeeeeaae code for our analysis is available httpas///ggabhbboo///ennnnnnnii
We consider a binary treatment A in the form of stay-at-home order, which bans gatherings with more than people. We encode mobility of trips on specific day is larger than the median number of trips during the entire time horizon, and 0 otherwise. Our outcome the 10-day-ahead case growth. We include the following observed variables as confounders X: the canton code (swiss member state at a subnational level), the canton population, and whether the weekday a Monday or not. After removing the first 10 recorded days for each canton (due to spillover effects from other countries) and rows with missing values, we obtain a dataset with n = 3276 observations.
single binary mediator M, which is if the total number
Analysis: We perform a causal sensitivity analysis for the natural directed effect (NDE) of the stay-at-home order A on the case growth Y. That is, we are interested in the part of the causal effect of A on Y that is not explained by the path via M (i.e., through the change in mobility). The NDE in an SCM M defined as
NDE(M) - - ((x Q
02 M) Q(x (a1 0, 02 0) dx.
(94)
Fig. 5 (main paper) shows causal sensitivity analysis for violations of the unconfoundedness between treatment A and mediator M. Hence, consider GMSM for binary treatments with sensitivity parameters M and y = O. For each M, we estimate our bounds for the expectation functional and the treatment combinations described in Appendix D
(0, and a (0, 0). We then obtain bounds for the NDE as
27

---

J Additional experimental results
Here, we provide additional experimental results on synthetic data that extend the results from Sec. in the main paper. We provide (i) results for additional treatment combinations and (ii) results for distributional effects. We follow the same experimental setup described in Sec. (main paper) and Appendix.
H
J.1 Additional treatment combinations
Results for additional treatment combinations are shown in Fig. 6 (binary treatment settings) and Fig. (continuous treatment settings). The results are similar to those in Sec. 5 the main paper and empirically confirm the validity of our bounds. Hence, our results remain valid independently of the choice of treatment combination.
2:00
20
1.25
15
1.30
propersdy
possendly
poppesaly
12
[2
12
-
1.00
0.75
0.50
Oods
Dojet nwot
175
0.4
3.00
Fay 17
-1.00
Fg-17 Cx
Figure 6: Results for additional treatments in the binary treatment setting. From left to right is shown:
setting (ii) with a (0, 1), setting (iii) with à = (0, 1, 0), and setting (iii) with a (0. 0, 1). The top row shows the oracle sensitivity parameter fy (depending on æ)), and the bottom row shows the bounds.
J.2 Distributional effects
also provide results for distributional effects, that is, we choose the a-quantile functional Here, we consider three quantiles with 0.7. 0.5
D (P x, mi a)) =F
(median), and a = 0.3. We use our importance sampling estimator derived in Appendix F (Eq. 668 estimate our bounds. The results are shown in Fig [ (binary treatment) and Fig. 9 continuous treatment) for settings (i)-(ii)) from Fig. in the main paper. Again, our bounds cover the underlying oracle effect in regions where the chosen sensitivity parameters Iw are larger than the oracle sensi- tivity parameters rip- This also confirms empirically the validity of our bounds for distributional effects.
J.3 Semi-synthetic data
We provide additional results for the semi- semi-synthetic IHDP data with unobserved confounding from Jesson et al. (3I). Here, we demonstrate the effectiveness of our bounds for distributional effects for Docision-making. We follow Jesson et al. [31] and evaluate our bounds by measuring the error rate of associated deferral policies, which defer a prespecified fraction of test samples with the largest uncertainty to an expert. We mimic a high-stakes decision-making problem where a wrong decision to prescribe treatment much worse than wrong decision not to prescribe treatment. To do measure performance with weighted error rate, that penalizes wrong
28

---

14
-05
0.
-05
0.5
Grade La 1i- Tama Gantaren
0740
(weld (y10
+100 -375
-256
-025
0.0
0
6.00
635
1.58
-100 -035 -028
-035
000
625
656
875
1.00
+500 A175 4.50 +225
000
025
6d0
675
1.0
Figure 7: Results for additional treatments in the continuous treatment setting. From left to right shown: setting (i) with à = 0.9, setting (ii) with a (0.2, 0.4), and setting (ii) with a = (0.4, 0.5, 0.3). The top shows the oracle sensitivity parameter Fw (depending x),, and the bottom row shows the bounds.
treatment decisions twenty times heavier than wrong decisions to treat. We then compare three different deferral policies: (i) the (standard) policy based on the expectation which treats whenever E x, = > E[y|X x, 0], (ii) a more conservative quantile policy that treats Qi-g[Y = x, A = 0] with
whenever 1 x, A= of TP = x, = 1)), and (iii) the same quantile policy with 4 = 0.2. Note that our bounds for policy (i) coincide with sharp bounds for binary CATE from the literature [T8). As done in [3], we report means and standard deviations of 400 random initializations of the THDP data. The results (Fig. 10 show that the conservative quantile policies improve over the expectation policy.
0.4 (Q, denotes the q-quantile
29

---

240
15
Observe procersly sore
Clearved procerndy scre
125
Clearved properaly scre
1.0
1.50
05
0.50
-1.0
-05
0.
0.5
10
-1.0
-05
0.5
-1.0
-05
0.
1.25
0.66
17--1a
6:0
21v/
-310
4025
025
116
455
E2E 656
+100
Oneck
Orade w Ta
2w03
fa helt
TA Tow
-0.25
85
-350 4025
1.6
435
626
656
-555
650
Orece 2003
1.0
1.0
Onede swop
Ta Fo
meett tato
0.5
TA tyrls
De Sesteset Ga Swet
-18
-250
-035
Figure 8: Results for distributional effects in the binary treatment setting using the same treatments as in Fig. (main paper). Settings (i)-iii) ordered from left right. The top row shows the oracle sensitivity parameter F (depending on 2). Rows 2, 3, and show the bounds for the a-quantiles of the interventional distribution with 0.7, a 0.5, and 0.3.
30

---

1.6
1.8
-0.5
1.0
10
Drank 2-13
Dookk
-8.5
0.00
1.25
0.75
8.79
0.00
8.00
0.25
Dralk wod5
0.00
5/00
+0.00
-0.0
-8.00
-0.0
Oryde g
0.79
Duato 23
Ohde 2
3.50
5.00
1.55
Dreak
8.00
Fynt0
4.5
-0.0
+8.0
-1.
+000
Figure Results for distributional effects in the continuous treatment setting using the same treatments as in Fig. 4 (main paper). Settings (i)-iii) are ordered from left to right. The top row shows the oracle sensitivity parameter iw (depending on x)). Rows 2, 3, and show the bounds for the a-quantiles of the interventional distribution with = 0.7, = 0.5, and 0.3.
Expactation policy
2.5
Quantile policy o 0.4
policy o 02
Weightd errTr rale
2.0
1.5
1.0
0.0
0.0
0.2
0.4
0.6
0.8
1.0
Defermal
Figure 10: results for the semi-synthetic IHDP data with unobserved confounding from Jesson et al. 31
31


Reviews:
[{"review_id": "ZZF8LBwxnJ", "reviewer": "Reviewer_9HdS", "summary": "The authors study the problem of bounding a given causal effect. To this end, they propose a generalized marginal sensitivity model (GMSM) that is applicable to multiple discrete, continuous , and time-varying treatments. They also present a new interpretation of the partial identification.", "questions": "See above.", "limitations": " The authors addressed the limitations.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The proposed GMSM model generalizes the previous models and leads to sharp bound for certain causal effects with certain causal graphs. \n\nThe bounds depend on observed variables and thus can be estimated. \n\nExcept some minor ambiguities (see below), the paper is well-written.", "weaknesses": "Although the proposed model generalized the previous models but the presented theoretical results hold under specific  graphical constraints e.g., no confounder between M and outcome, no hidden confounders between X and $\\{A,M,Y\\}$. \n\nThe other weakness about the results is its generalizability to arbitrary causal graphs. Based on the presented proofs in the appendix, it is not clear how this results can be generalized by relaxing the assumptions. \n\nThere is ambiguity about the notation of U. Does $U_Y$ denote the unobserved exogenous variable for Y in the definition of SCM or is it a hidden confounder between Y and A? Is it possible to have hidden confounders among the mediators (e.g., $M_1$ and $M_2$)?\n\nThe explanation below (3) says  “If U_w has no effect on A, Eq. (3) holds with $s^-_W (a, x) = S^+_W (a, x) = 1$”. But it seems that (3) encodes the effect of A on $U_W$.!\n\nIn (35) in the Appendix, what is U exactly?  Does the setting imply that u and x are independent, i.e., $p(u|x)=p(u)$?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to reviewer 9HdS\n\nThank you for your positive review and your helpful comments. We improved our paper in the following ways:\n\n### Response to “Weaknesses”\n\n* Thank you for giving us the opportunity to clarify the assumptions in our paper.\n    * The assumption that no unobserved confounders exist between mediators $M$ and outcome $Y$ is necessary to provide results for **nested counterfactuals** (see point three below for a more detailed discussion). We would like to emphasize that we need this assumption only for settings with mediators (i.e., Corollary 2) and **not** for our results without mediators (Corollary 1). Furthermore, this assumption is **weaker** than assumptions usually taken in the mediation literature (see point three).\n    * We do **not** assume that there is no unobserved confounding between $X$ and $A, M, Y$.  For example, we allow for correlation between $X$ and $U$, so that $U$ could also be an unobserved confounder between $X$ and $Y$. Technically speaking, we only assume that $\\{X, U_W\\}$ is a sufficient adjustment set for the $A-W$ relationship ($W \\in \\{M, Y\\}$). This is equivalent to ignorability assumptions in the potential outcomes framework like $Y_1, Y_0 \\perp A \\mid X, U$ (for binary treatments), which are **standard in the literature** on unobserved confounding and imply that $U$ captures all unobserved confounders (Dorn and Guo 2022).\n    * **Action:** We will add a detailed discussion of our assumptions to our paper.\n\n* We agree that our results cannot be generalized for arbitrary causal graphs in a straightforward way. However, the aim of our paper is not to provide results for arbitrary graphs, but rather for (conditional) treatment effects and mediation/ path analysis. We argue that the setting we study in our paper is already **very general compared to current state-of-the-art** work on causal sensitivity analysis. In particular, we are the first to study MSM-type sensitivity analysis for mediation and path analysis. Furthermore, our setting includes the whole literature on (conditional) average treatment effect estimation as a special case. We believe that extending our results to other causal graphs (e.g., with instrumental variables) is an interesting direction of possible future research, but out of the scope of this paper.  \\\n**Action:** We will expand our section on future work and limitations, and discuss possible causal inference settings which may be of interest for future work on sensitivity analysis.\n\n* Thank you for pointing out the ambiguity in our notation $U_W$. All variables $U_W$ we consider explicitly are unobserved confounders between treatment $A$ and mediator/ outcome $W$ and not exogenous noise. The latter is implicitly considered as part of the SCM definition.  In Theorem 1, we assume that there exist no unobserved confounders between mediators, e.g., between $M_1$ and $M_2$. \n    * The main reason for this assumption is that it allows us to interpret the causal query $Q(\\mathbf{x}, \\overline{\\mathbf{a}}, \\mathcal{M})$ from Equation (1) as a path-specific effect. Path-specific effects are defined as so-called nested counterfactuals that lie in the third layer of Pearl’s causal hierarchy. Using the assumption from Theorem 1, they can be reduced to the query $Q$, which lies in layer 2 of Pearl’s hierarchy, i.e., only depends on interventions (Correa et al. 2021). Our sensitivity analysis then bridges the gap from layer 2 to layer 1 (observational data). We believe that, in principle, relaxing the assumption should be possible. For example, one could consider combining our results with a sensitivity analysis from layer 3 to layer 2, e.g., by imposing a sensitivity model on the level of confounding between mediators $M_1$ and $M_2$. While this goes beyond the scope of our paper, it seems like an interesting direction for future work. \n    * Note that **we rely on this assumption only in mediation settings**. A large part of our contribution is the derivation of bounds for distributional effects for different treatment types in settings without mediators (Corollary 1), where we do not impose such an assumption. \n    * We would like to emphasize that our assumptions are **weaker** than most state-of-the-art work for mediation/path analysis (e.g., Shpister and Tchetgen Tchetgen 2016). Most work assumes unconfoundedness between all variables, while we allow for unobserved confounding between treatment and mediators/ outcome and only assume unconfoundedness between the mediators and outcome.\n    * **Action:** We expand our discussion of the assumption from Theorem 1, clarifying that we do not allow for unobserved confounding between mediators. We also expand our section on future work and clarify the definition of $U_W$.\n\n* In Eq. (3), $U_W$ is by definition a parent of $A$ in the causal graph. Eq. (3) only limits the level of dependence (in the language of probability theory) between the random variables $U_W$ and $A$. Here, the order of the variables in $\\mathbb{P}(u_W \\mid x, a)$ does not reflect the causal order between $U_W$ and $A$. Note also that for the MSM we can simply rewrite the fraction from Eq. (3) as $\\frac{\\mathbb{P}(a \\mid x, u_W)}{\\mathbb{P}(a \\mid x)}$ (see Appendix C), which reverses the order of the variables in the probabilities.  \\\n**Action:** We will add a clarification to the paper.\n\n* Eq. (35) is part of Appendix C, where we consider the three important sensitivity models MSM, CMSM, and LMSM. These sensitivity models are defined without mediators, which means that we only have one unobserved confounder $U = U_Y$ between treatment $A$ and outcome $Y$. We do **not** assume that U and X are independent, as this would be a very strong assumption. For example, if we are interested in the effect of smoking ($A$) on cancer risk ($Y$), $U$ may be specific gene expressions correlated with observed confounders $X$ such as medical history. \\\n**Action:** We will add a clarification to Appendix C.\n"}, {"review_id": "n9TRESsITV", "reviewer": "Reviewer_taFV", "summary": "The authors propose a unified framework for causal sensiitivity analysis under unobserved confounding that generalizes the Marginal Sensitivity Model (MSM). They derive sharp bounds for a diverse set of causal effects such as the CATE, mediation and path analysis effects, and distributional effects. The framework is applicable to discrete, continuous, and time-varying interventions. They offer, to my knowledge, a novel interpretation of the marginal sensitivity model via SCM. They show that in the case of binary treatments, their derived bounds  coincide with the optimality result of Dorn and Guo 2023. They provide a closed form solution to and algorithm to estimate the bounds and show empircally that it improves over line search methods (a formal complexity analysis would be nice).", "questions": "How does one show that  $P(a \\mid x, y_t) = P(a \\mid x, u)$?", "limitations": "Yes", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "This paper makes several novel contributions to an active area of study in causal machine learning, which are listed above in the summary. They provide theoretical and experimental evidence supporting their claims. They do a good job presenting and comparing to the related work. An exceptionally well writen and organized paper given the complexity of the subject matter.", "weaknesses": "## I have one primary comment. \n\nI think there may be a step missing in the special cases proofs of Appendix C. I appreciate how the authors have utilized SCM to define the GMSM. But, in defining the  MSM, CMSM, and LMSM in terms of hidden confounders $u$, it seems that there is a step missing from how these models are originally defined. Namely, they are defined with respect to potential-outcomes / counterfactuals $Y_{t}$ and the conditional independence relation: $Y_t \\perp T \\mid X$. For example, it's not obvious to me how you move from $P(a \\mid x, y_t)$, to $P(a \\mid x, u)$. Given that you show equivalence in terms of $P(a \\mid x, u)$, I think it is important to be explicit here. If this is resolved, and I admit that this could be completely trivial and I just don't see it, I would happily increase my score. If it cannot be resolved, I would suggest removing these claims and my score would remain the same.\n\n## I have a few minor comments.\n\nFirst, the last two paragraphs of the introduction can be streamlined as the contributions paragraph essentially reiterates the points of the paragraph starting on line 45. I like both styles, with slight preference for the contribution format.\n\nThere is a recent paper proposing a marginal sensitivity analysis for continuous treatments that could be added to the related works.\n\nLine 70 \"... when while ...\" seems to be a typo", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to reviewer taFV\n\nThank you for your positive evaluation of our paper! We took all your comments at heart and improved our paper accordingly.\n\n### Response to “A formal complexity analysis would be nice”\n\n* Thank you for pointing this out. In the following, we assume that all models are trained and we have a sample $(y_i)_{i=1}^k \\sim \\hat{\\mathbb{P}}^Y(\\cdot \\mid \\mathbf{x}, \\mathbf{m}, \\mathbf{a})$ (necessary for both estimators) available. Then, our estimator from Eq. (8) has a complexity of $\\mathcal{O}(k)$ because it only involves summing and quantile computation. Algorithm 1 of Jesson et al. (2022) has a complexity of  $\\mathcal{O}(k n)$, where $n$ is the number of grid search points. For Table 1, we choose $k = n = 5000$.  \\\n**Action:** We will add this to our appendix and thereby highlight the benefits our work.\n\n### Response to “Primary comment”\n\n* This is an excellent question. First of all, while we are the first to use Pearl’s SCM framework for MSM-type sensitivity analysis, we are **not** the first to define the MSM in terms of an unobserved confounder $U$ instead of the potential outcomes $Y_t$. Note that the SCM and potential outcome framework are logically equivalent and we decided to use SCMs because they allowed us to think in a new way about the bounding problem for sensitivity analysis (via probability mass transport, see Fig. 2). The following state-of-the-art papers all use the unobserved confounder $U$ in their definitions within the potential outcomes framework:\n    * [1] Dorn and Guo 2022, “Sharp sensitivity analysis for inverse propensity weighting via quantile balancing”, JASA, Equation (2)\n    * [2] Dorn et al. 2022, “Doubly-valid/ doubly-sharp sensitivity analysis for causal inference with unmeasured confounding”, arXiv:2112.11449, Definition 1\n    * [3] Oprescu et al. 2023, “B-learner: Quasi-oracle bounds on heterogeneous causal effects under hidden confounding”, ICML, Assumption 1\n    * [4] Bonvini and Kennedy 2022, “Sensitivity analysis for marginal structural models”, arXiv:2210.04681, Equation (5)\n* We agree that the equivalence of the two MSM definitions is not immediately clear and that the papers above do not seem to provide formal results on this. In [1], the authors only write “However, as pointed out by a referee, these assumptions are equivalent”. In the following, we provide a formal result using the GMSM formulation (note that this is equivalent to the original MSM definition via log-odds ratio).\n* Lemma: The following two statements are equivalent: \n    1) There exist a $U$ with $Y_1, Y_0 \\perp  A \\mid X, U$ so that $s^- \\leq \\frac{\\mathbb{P}(u \\mid x, a)}{\\mathbb{P}(u \\mid x)} \\leq s^+$\n    2) It holds that $s^- \\leq \\frac{\\mathbb{P}(y_1, y_0 \\mid x, a)}{\\mathbb{P}(y_1, y_0 \\mid x)} \\leq s^+$\n* Proof (sketch):\n    * Direction 2. $\\to$ 1.: Define $U = (Y_1, Y_0)$.\n    * Direction 1. $\\to$ 2.: We proceed via proof by contradiction. Assume there exists a pair $(y_1, y_0)$ that violates 2), say w.l.o.g. $\\frac{\\mathbb{P}(y_1, y_0 \\mid x, a)}{\\mathbb{P}(y_1, y_0 \\mid x)} > s^+$. We can use the ignorability condition from 1. to write $\\mathbb{P}(y_1, y_0 \\mid x, a) = \\int \\mathbb{P}(y_1, y_0 \\mid x, u, a) \\mathbb{P}(u \\mid x, a) du = \\int \\mathbb{P}(y_1, y_0 \\mid x, u) \\mathbb{P}(u \\mid x, a) du $. Furthermore, $\\mathbb{P}(y_1, y_0 \\mid x) = \\int \\mathbb{P}(y_1, y_0 \\mid x, u) \\mathbb{P}(u \\mid x) du$. It follows that $s^+ &lt; \\frac{\\int \\mathbb{P}(y_1, y_0 \\mid x, u) \\mathbb{P}(u \\mid x, a) du}{\\int \\mathbb{P}(y_1, y_0 \\mid x, u) \\mathbb{P}(u \\mid x) du} $. Hence, there exists a $u$ such that $s^+ &lt; \\frac{\\mathbb{P}(y_1, y_0 \\mid x, u) \\mathbb{P}(u \\mid x, a)}{\\mathbb{P}(y_1, y_0 \\mid x, u) \\mathbb{P}(u \\mid x)} = \\frac{\\mathbb{P}(u \\mid x, a)}{\\mathbb{P}(u \\mid x)}$, which is a contradiction to the sensitivity constraint from 1.\n* **Action**: We will add a section to the appendix where we discuss the two definitions and prove their equivalence with the arguments provided above. \n\n### Response to “Minor comments”\n\n* Thank you for pointing out the redundancy. **Action:** We will streamline the last two paragraphs of the introduction into a larger “Contribution” paragraph, as proposed.\n* We assume that you are referring to the following paper: “Partial identification of dose responses with hidden confounders”, Marmarelis et al., UAI 2023. Thanks for pointing this out! The paper was published after the NeurIPS deadline, which is why we did not include it in the related work at the time of submission. We would like to emphasize that the paper derives bounds for a different sensitivity model that is only applicable to continuous treatments and **not** to binary or time-varying treatments, and mediation or distributional effects. **Action:** We will add the paper to the related work and also add a discussion to the appendix, in which we compare the different sensitivity models for continuous treatments.\n* Thanks for pointing out the typo! **Action:** We will carefully proofread our paper."}, {"review_id": "bdtA0LJQ1V", "reviewer": "Reviewer_H542", "summary": "This paper is about sensitivity analysis (SA) of causal queries in SCMs. In practice, given a causal query and a set of models, the goal is to compute a query's lower and upper bounds. The authors first derive a class of models to be used for SA and show how this extends existing models. An algorithm to obtain the bounds is such cases is derived.\n\n(After the rebuttal, I decided to raise the rating of the paper from 4 to 6)\n", "questions": "Would it be possible to compare the present method against algorithms for the bounding of non-identifiable queries? E.g. Zhang and Bareinboim, Duarte et al., and Zaffalon et al. worked in this direction in the last two years. I think the sensitivity analysis the authors consider is a heuristic approach to the same problem. If this is true, not having a comparison against these methods is a serious issue.\n\nI also believe that the authors would take advantage of the literature about so-called \"imprecise probabilities\" and \"credal networks\" as these models implement the kind of sensitivity analysis of interest for the authors. In particular, the paper \"Structural Causal Models Are (Solvable by) Credal Networks\" might be a helpful reading.\n", "limitations": "I don't see specific issues in this direction.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The technical results are sound and non-trivial.\nThe experiments show good bounds obtained in this way.\n", "weaknesses": "The literature on partially identifiable queries is ignored. In particular existing techniques for bounding such queries are not considered.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to reviewer H542\n\nThank you for your review and your helpful comments!\n\n### Response to “Questions”\n\n* Thank you for giving us the opportunity to clarify the difference between two related approaches for bounding causal effects: **Causal sensitivity analysis (CSA)** and **causal partial identification (CPA) without sensitivity models**. In the following, we outline why CSA is **not** a heuristic approach to CPA and why the two approaches should not be benchmarked against each other. \n    * The main difference between CSA and CPA is that CSA imposes **sensitivity models**, that is, assumptions on the “strength” of unobserved confounding, which is controlled by a sensitivity parameter $\\Gamma$. Methods for CPA do not make sensitivity assumptions but instead impose other strong assumptions (see points below). In practice, CSA can be used to test the robustness of causal effect estimates to violations of the unconfoundedness assumption (by varying $\\Gamma$), while CPA may be applicable in situations where no domain knowledge about the confounding strength is available.\n    * **CSA is not a heuristic approach to CPA.** There is a large stream of literature on CSA in both the machine learning and statistics community (see our related work), with most papers deriving rigorous mathematical results on validity and optimality (“sharpness”) of bounds. In our paper, we provide **formal proofs** for all our results in addition to an extensive empirical evaluation.\n    * In our paper, we study the standard setting for (conditional) average treatment effects with continuous outcomes and optional mediators. Here, CPA is a **special case** of CSA when setting the sensitivity parameter to $\\Gamma = \\infty$. Hence, CSA provides **provably tighter bounds** than CPA. In fact, CPA bounds are **well characterized** for our setting: For binary treatments, so-called “no assumptions bounds” have been derived in [1]. Indeed, we obtain the same results with our bounds in the limit $\\Gamma \\to \\infty$. For continuous treatments, informative CPA is impossible, that is, the CSA bounds converge to the boundary of the support of the outcome distribution for $\\Gamma \\to \\infty$ [2].\n    * There is no free lunch in causal inference. Because approaches for CPA do not restrict the strength of unobserved confounding, they often impose **other (strong) assumptions** in order to derive informative bounds. For example, all three mentioned papers (Zhang and Bareinboim 2022, Duarte et al. 2021, Zaffalon et al. 2022) are only applicable to settings with **discrete observed variables**. In contrast, these papers are **not** applicable to continuous outcomes, which is the main focus of our paper. Other papers on CPA derive informative bounds by assuming the existence of valid instrumental variables (e.g., [3, 4]).\n    * For the reasons above, there is **no point in benchmarking** our bounds against CPA approaches. In our setting, this would reduce to setting $\\Gamma = \\infty$. Likewise, methods for only discrete variables or instruments are not applicable, and can thus **not** be used as baselines.\n    * **Summary**: CSA derives bounds under assumptions on the strength of unobserved confounding, and CPA derives bounds under other assumptions and often for more complex/ arbitrary causal graphs (e.g., using valid instrumental variables, or assuming only discrete variables). Both approaches are complementary to each other, and practical usage depends on the assumptions one is willing to impose on the underlying data-generating process.\n    * **Action:** We will expand our related work on CPA (see Appendix A.1) and include the mentioned references. Furthermore, we will add a detailed discussion comparing CSA and CPA, using the points provided above.\n* Thank you for pointing out the literature on imprecise probabilities. It seems to be closely related to the CPA literature as the paper assumes **discrete** **variables** and does **not** impose any **sensitivity models**. We would like to reiterate that, in our setting, CPA is solved and corresponds to setting the sensitivity parameter to $\\Gamma = \\infty$. However, combining ideas from the imprecise probability literature with CSA could potentially be an interesting direction for future research.  \\\n**Action:** We will expand our related work to include the literature on imprecise probabilities, in particular the paper mentioned. We will also add a discussion on potential future work.\n\n\n### References\n\n[1] Manski 1990, “Nonparametric bounds on treatment effects”, The American Economic Review\n\n[2] Jesson et al. 2022, “Scalable sensitivity and uncertainty analysis for causal-effect estimates of continuous-valued interventions” NeurIPS\n\n[3] Kilbertus et al. 2020, “A Class of Algorithms for General Instrumental Variable Models”, NeurIPS\n\n[4] Padth et al. 2023, “Stochastic Causal Programming for Bounding Treatment Effects”, CLeaR 2023\n"}, {"review_id": "NLd3ARYNAh", "reviewer": "Reviewer_GZoa", "summary": "The authors generalize a class of causal sensitivity models that includes the traditional MSM, the continuous-treatment CMSM, and the longitudinal (time-varying treatment) LMSM. They show how to compute sharp bounds for the causal estimands by taking inspiration from recent work. Their general framework also allows mediation analysis. They provide an algorithm for computing these bounds.", "questions": "Could you support your reasoning for why this weighting function is a natural interpretation of the more specific sensitivity models? How is it helpful and how can I use it in newer settings? Why should it be set to zero in some cases?", "limitations": "Limitations are discussed a bit but they do not address potential societal impacts. It is debatable if that is necessary for this kind of work, but I think pitfalls of these kinds of sensitivity analyses should be discussed.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The method is solid and explained well. Figure 2 is nice. Mediation analysis is an important contribution to causal sensitivity models. A broader understanding of sensitivity models is always valuable and the effort to generalize is a good one.", "weaknesses": "My main concern is that this generalization that unifies the MSM, CMSM, and LMSM is not very useful. The weighting function seems a bit contrived. It is necessary because the CMSM and LMSM do not use the nominal propensity at all, but the MSM does. Isn't it strange to ignore the nominal propensity and give the same bounds to all potential outcome distributions? Shouldn't the observed confounding inform the unobserved confounding?\n\nA recent alternative to the CMSM is the $\\delta$MSM [arXiv:2204.11206] that takes a different approach and appears to perform better. The authors could discuss alternative models like this one or at least keep them in mind when considering general classes of sensitivity models.\n\nThe clever approach to sharp partial identification is not novel [see for instance arXiv:2304.10577]. The benefit of the mediation analysis is not really made clear in the results of this submission.\n\n\nIn terms of results, the authors employ a purely synthetic benchmark. The real-world data are interesting but they lack a ground truth. The authors could include some well-known semi-synthetic benchmarks like IHDP and induce hidden confounding by hiding some of the observed confounders.\n\nThe authors do not compare with previous methods in their benchmark except for Table 1, where they use a custom weighting function to beat the older CMSM algorithm. That is not very convincing. Also Table 1 should at least be discussed more. The results for the weighted CMSM seem conflicting: tighter bounds but worse coverage?\n\nOn a lesser note, the technical details are a bit dense.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your helpful review!\n\n### Response to “Weaknesses”\n\n* Thank you for your comments. The main motivation behind the GMSM definition (and weighting function) is indeed to unify the three important sensitivity models from the literature.\n    * We argue that our **main contribution** is the derivation of **novel bounds** that are valid for all **sensitivity models**, **distributional effects**, and **mediation/path analysis**. For example, practitioners may use our bounds for the well-established MSM in a mediation analysis setting **without ever explicitly using the GMSM formulation via a weighting function**. Hence, you may simply think about the weighting function as a **notational tool**.\n    * However, we believe that the weighting function has **additional advantages** beyond providing a unified notation when **domain knowledge about the confounding structure is available**. Consider an observational study on the effect of smoking on cancer risk, confounded by certain unobserved genes. It may be known that these genes do not affect the cancer risk for a certain population with covariates $\\mathbf{x} \\in \\mathcal{X}_0$, e.g., female individuals. Hence we can set the weight function to $q(\\mathbf{a}, \\mathbf{x}) = 1$ for all $\\mathbf{x} \\in \\mathcal{X}_0$ to obtain sharper bounds for the average treatment effect.\n    * **The CMSM sets the weighting function to zero because observed confounding is not necessarily informative for unobserved confounding**. For example, $X$ and $U$ could be independent, or at least nothing about the dependence may be known. The binary MSM uses the propensity score $\\mathbb{P}(a \\mid x)$ as a weighting function because, for large or small propensities, most of the “randomness” in $A$ is explained by $X$. This implies that $U$ can not have a large effect on $A$ for such $x$. However, this kind of **reasoning breaks down for continuous treatments**. Note that the CMSM has been **proposed in several previous papers** and is not our invention. Furthermore, our paper makes **major contributions in settings beyond continuous treatments**.\n    * **Action:** We will add a detailed discussion about the weighting function to our paper, summarizing the points above.\n* The delta-MSM was **published after the NeurIPS deadline** (UAI 2023), which is why we did not include it in the related work at the time of submission. \n    * We would like to emphasize that it is **not** applicable to **other treatment types**, **mediation/path analysis** or **distributional effects**. Extending our bounds to the delta-MSM may be an interesting direction for future research.\n    * Benchmarking different sensitivity models is difficult due to different assumptions on the data-generating process. We follow **established literature** on sensitivity analysis and study the optimality of bounds under specific sensitivity models, **not the optimality of the sensitivity models** themselves.\n    * **Action:** We will add a discussion to the appendix.\n* The paper mentioned (arXiv:2304.10577) is one of multiple papers that deals with **CATEs for binary treatments**, which we all cite in our related work section. In our paper, **we never claim that our bounds for binary CATE are novel** (e.g., see our abstract). However, our approach and intuition for deriving the bounds **is novel** (via SCM-based probability mass transport, Fig. 2). In contrast to previous work, it can be easily **generalized to more complex causal inference settings** beyond binary CATE. **Action**: We will clarify this in our section on related work.\n* **Mediation analysis** from observational data is crucial in many disciplines such as epidemiology, economics, and algorithmic fairness. It aims to answer questions like “What is the direct causal effect of a medical treatment on health, that is not mediated through a change in diet?”. Our results show that we obtain valid bounds for direct, indirect, and path-specific effects under unobserved confounding. **Action:** We will add a discussion to the appendix.\n* We do not compare against more previous works because: (1) For mediation/path analysis and distributional effects, there **are no baselines** for MSM-based sensitivity analysis. (2) For binary CATE, we obtain the same sharp bounds (in population) as previous work. **Action:** We will add a clarification to the experiment section.\n* With Table 1, we try to convey two messages: (1) Under the CMSM, we get almost the same bounds as Jesson et al. (2022). This is because their algorithm is an **approximation** of our bounds. In contrast, we have **closed-form solutions**, and we thus achieve better **computational speed**. (2) Under a weighted CMSM, we can obtain **tighter bounds** for the average treatment effect, that is, smaller overall interval lengths and lower coverage for small $\\Gamma$. Note that we do **not** claim that the weighted CMSM is superior to the unweighted CMSM. As mentioned earlier, we **refrain from benchmarking sensitivity models**. Instead, we outline the possible use of the weight function for incorporating domain knowledge. **Action:** We will clarify this in the paper.\n* **Additional experiments**: Note that we **provided rigorous mathematical proofs for all our results**. However, we agree that adding experiments using IHDP is an excellent idea to improve the experimental section of our paper. \\\n**Action:** We performed additional experiments using the IHDP data with hidden confounding from Jesson et al. (2021). Note that IHDP is a dataset with binary treatments and no mediators. Here, our contributions are bounds for **distributional effects**. Details and results are provided in the uploaded PDF file. We also updated our anonymized repository.\n* Thank you for pointing out the density of technical details. **Action:** We will move certain technical details to the appendix.\n\n### Response to \"Limitations\"\n* **Action:** As recommended, we will add a discussion on potential societal impacts."}]

Author rebuttal:
## Response to all reviewers

Thank you very much for the constructive evaluation of our paper and your helpful comments! We addressed all of them in the comments below and uploaded additional empirical results as a PDF file. Our main improvements are the following:



* We provided **clarifications** regarding our assumptions. Therein, we explain that our assumptions are weaker than in previous literature.
* We discussed the **connection to related literature** (e.g., partial identification without sensitivity models) and spell out explicitly how our work is different and novel. 
* We obtained **new empirical and theoretical results**, including a new experiment on semi-synthetic data.

We will incorporate all changes (labeled with **Action)** into the camera-ready version of our paper. Given these improvements, we are confident that our paper will be a valuable contribution to the causal machine learning literature and a good fit for NeurIPS 2023.

Meta-review: All reviewers agreed about the novelty and relevance of this work to the NeurIPS community. The authors managed to iron out a few initial misunderstandings and unclear points during the rebuttal and we strongly encourage them to include the promised action points (also new theoretical results developed during the rebuttal, etc.) in the final version.
Decision: Accept (poster)