Title: Sharp Bounds for Generalized Causal Sensitivity Analysis
Venue: NeurIPS 2023
Accepted: yes
Keywords: Causal machine learning

treatment effect estimation

sensitivity analysis

unobserved confounding

uncertainty estimation
Abstract: Causal inference from observational data is crucial for many disciplines such as medicine and economics. However, sharp bounds for causal effects under relaxations of the unconfoundedness assumption (causal sensitivity analysis) are subject to ongoing research. So far, works with sharp bounds are restricted to fairly simple settings (e.g., a single binary treatment). In this paper, we propose a unified framework for causal sensitivity analysis under unobserved confounding in various settings. For this, we propose a flexible generalization of the marginal sensitivity model (MSM) and then derive sharp bounds for a large class of causal effects. This includes (conditional) average treatment effects, effects for mediation analysis and path analysis, and distributional effects. Furthermore, our sensitivity model is applicable to discrete, continuous, and time-varying treatments. It allows us to interpret the partial identification problem under unobserved confounding as a distribution shift in the latent confounders while evaluating the causal effect of interest. In the special case of a single binary treatment, our bounds for (conditional) average treatment effects coincide with recent optimality results for causal sensitivity analysis. Finally, we propose a scalable algorithm to estimate our sharp bounds from observational data.

Sharp Bounds for Generalized Causal Sensitivity Analysis

Abstract

Causal inference from observational data is crucial for many disciplines such as medicine and economics. However, sharp bounds for causal effects under relaxations of the unconfoundedness assumption (causal sensitivity analysis) are subject to ongoing research. So far, works with sharp bounds are restricted to fairly simple settings (e.g., a single binary treatment). In this paper, we propose a unified framework for causal sensitivity analysis under unobserved confounding in various settings. For this, we propose a flexible generalization of the marginal sensitivity model (MSM) and then derive sharp bounds for a large class of causal effects. This includes (conditional) average treatment effects, effects for mediation analysis and path analysis, and distributional effects. Furthermore, our sensitivity model is applicable to discrete, continuous, and time-varying treatments. It allows us to interpret the partial identification problem under unobserved confounding as a distribution shift in the latent confounders while evaluating the causal effect of interest. In the special case of a single binary treatment, our bounds for (conditional) average treatment effects coincide with recent optimality results for causal sensitivity analysis. Finally, we propose a scalable algorithm to estimate our sharp bounds from observational data.

1 Introduction

Causal effects are crucial for decision-making in many disciplines, such as marketing [71], medicine [77], and economics [1]. For example, physicians need to know the treatment effects to personalize medical care, and governments are interested in the causal effects of policies on infection rates during a pandemic. In many such applications, randomized experiments are costly or infeasible, because of which causal effects must be estimated from observational data [59].

Estimating causal effects from observational data may lead to bias due to the existence of confounders, i.e., variables that affect both treatment and outcome [55]. A remedy is to observe all confounders and thus to assume unconfoundedness (e.g., as in [16, 63, 64]). However, in many practical applications, the assumption of unconfoundedness is violated. For example, electronic health records do not capture a patient's ethnic background, which is a known confounder in medicine [52]. In such cases, the causal effect is not identified from observational data, and unbiased estimation is thus impossible [27].

A popular way to perform causal inference in the presence of unobserved confounders is causal sensitivity analysis . Causal sensitivity analysis aims to derive bounds on the causal effect of interest under relaxations of the unconfoundedness assumption. Here, the strength of unobserved confounding is typically controlled by some sensitivity parameter Γ , which also determines the tightness of the bounds. In practice, one chooses Γ through domain knowledge [14, 33, 82] or data-driven heuristics [29]. Then, one derives that the causal effect of interest lies in some informative region. For example,

a suitable Γ may enable - despite unobserved confounding - to infer the sign of the treatment effect, which is often sufficient for consequential decision-making [35].

A common model for causal sensitivity analysis is the marginal sensitivity model (MSM) [31, 35, 36, 67]. A benefit of the MSM is that it does not impose any kind of parametric assumptions on the datagenerating process. However, the standard MSM is only applicable in settings where the treatment is a single binary variable. Different extensions have been proposed for continuous treatments [10, 32, 45] and for time-varying treatments and confounders [10]. However, these works are restricted to specific settings, while a unified framework for causal sensitivity analysis is still missing.

Contributions: 1 In this paper, we propose a generalized marginal sensitivity model (GMSM) . Our GMSM provides a unified framework for causal sensitivity analysis under unobserved confounding in various settings with multiple discrete, continuous, and time-varying treatments. Crucially, our GMSM includes existing models, such as those in [67] and [32], as special cases. As a result, our GMSM enables a unified approach to deriving sharp bounds for a large class of

causal effects. To do so, we bound a distribution shift in the unobserved confounders while performing an intervention on the treatments. As a result, we obtain sharp bounds for various causal effects, including (conditional) average treatment effects but also effects for mediation analysis and path analysis (see Fig. 1) and for distributional effects, which have not been studied under an MSM-type sensitivity analysis previously. We also show that, for binary treatments, our bounds coincide with recent optimality results for (conditional) average treatment effects under the MSM. Finally, we propose a scalable algorithm to estimate our sharp bounds from observational data and perform extensive computational experiments to show the validity of our bounds empirically.

2 Related work

In the following, we review related work on causal sensitivity analysis. For a more general review of partial identification and treatment effect estimation under unconfoundedness, we refer to Appendix A.

Causal sensitivity analysis: Causal sensitivity analysis dates back to a study from 1959 showing that unobserved confounders cannot explain away the causal effect of smoking on cancer [14]. This was formalized by introducing sensitivity models that yield bounds on the causal effect of interest under some restriction on the amount of confounding. Previous works introduced a variety of sensitivity models that make different assumptions about the data-generating process and the confounding mechanism. Examples include sensitivity models based on parametric assumptions [30, 62], difference between potential outcome distributions [60, 70], and Rosenbaum's sensitivity model that uses randomization tests [61].

Marginal sensitivity model (MSM): The marginal sensitivity model (MSM) [67] is a common model for sensitivity analysis aimed at settings with binary treatments. Many methods were proposed to estimate bounds from observational data under the MSM. Examples of such methods include linear fractional programs [82] and machine learning such as including kernel-based methods [35] and deep neural networks [31]. Recently, Dorn et al. [18] and Jin et al. [33] showed that estimates from the previous methods are too conservative bounds and, as a remedy, derived closed-form solutions for sharp bounds under the MSM. This was also extended to semiparametric inference [19, 53]. However, all previous methods are limited to binary treatments and (conditional) average treatment effects.

Different extensions for the MSM have been proposed. Jesson et al. [32], Bonvini et al. [10], and Marmarelis et al. [45] developed extensions for continuous treatments. Furthermore, Bonvini et al. [10] also extended the MSM to time-varying treatments and confounders. However, both works are not applicable to settings beyond the estimation of (conditional) average treatment effects, such as

mediation analysis and path analysis or distributional effects. In contrast, we propose a generalized marginal sensitivity model that is compatible with binary, continuous, and time-varying treatments for a variety of causal effects.

Research gap: To the best of our knowledge, no existing causal sensitivity analysis based on the MSM provides a unified framework for deriving bounds for binary, continuous, and time-varying treatments. Furthermore, MSM-based causal sensitivity analysis is restricted to (conditional) average treatment effects and is thus not applicable for complex settings such as effects for mediation analysis and path analysis or distributional effects.

3 Generalized marginal sensitivity model (GMSM)

We first formally define a general setting for causal sensitivity analysis that includes mediation and path analysis (Sec. 3.1 and Sec. 3.2). We then propose our generalized marginal sensitivity model (GMSM) in Sec. 3.3 and compare it with existing sensitivity models from the literature.

Notation: We write random variables as capital letters X and their realizations in lowercase x . Bold letters X or x represent (random) vectors. If X = ( X 1 , . . . , X ℓ ) is a sequence of random variables of length ℓ , we denote ¯ X k = ( X 1 , . . . , X k ) for 1 ≤ k ≤ ℓ . We denote probability distributions over X as P X where required. The probability mass function for a discrete X is denoted as P ( x ) = P ( X = x ) . If X is continuous, P ( x ) is the probability density function. We denote P ( · ) as the corresponding probability mass/density function not evaluated at a specific x . Similarly, we write conditional probability mass functions/density functions as P ( y | x ) = P ( Y = y | X = x ) and conditional expectations as E [ Y | x ] = E [ Y | X = x ] = ∫ y P ( y | x ) d y . We denote P ( y | do ( X = x )) as the probability mass function/density function of Y after performing the do-intervention do ( X = x ) [55]. Finally, we define X Y = pa ( Y ) ∩ X , where pa ( Y ) denotes the parents of Y in a given causal graph.

3.1 Problem setup for generalized causal sensitivity analysis

We formalize causal sensitivity analysis based on Pearl's structural causal model framework [55].

Definition 1. A structural causal model (SCM) M is a tuple ( V , U , F , P U ) , where V = ( V 1 , . . . , V k ) are observed endogenous variables, U are unobserved exogenous variables determined outside of the model, F = { f V 1 , . . . , f V k } is a set of functions so that each f V i maps a set of parents pa ( V i ) ⊆ V ∪ U to V i , and P U is a probability distribution on U .

Every SCM M induces unique directed graph 2 G M on V ∪ U by drawing a directed edge from V 1 to V 2 if V 1 ∈ pa ( V 2 ) . In this paper, we assume that G M is acyclic, i.e., does not contain any directed cycle. Then, M induces a unique joint probability distribution P V ∪ U on V ∪ U . Furthermore, M induces unique interventional distributions P V ∪ U do ( A = a ) when performing the do-intervention do ( A = a ) for some observed treatments A ⊆ V [4, 55].

We consider settings where we observe four distinct types of endogenous variables V = { X , A , ¯ M ℓ , Y } : observed confounders X ∈ X ⊆ R d x , treatments A ∈ A ⊆ R d a , discrete mediators ¯ M ℓ = ( M 1 , . . . , M ℓ ) with M i ∈ N for 1 ≤ i ≤ ℓ , and an outcome Y ∈ R . In a medical setting, X might be patient characteristics (gender, age, medical history, etc.), A a medical treatment, ¯ M ℓ a change in diet, and Y a variable indicating a health outcome. Possible causal graphs are shown in Fig. 1. We assume w.l.o.g. that ( M 1 , . . . , M ℓ ) are ordered causally, i.e., ¯ M i -1 are parents of M i for each i ∈ { 2 , . . . , ℓ } . Given treatment interventions ¯ a ℓ +1 = ( a 1 , . . . , a ℓ +1 ) , we are interested in causal effects of the form

(1)

where D is some functional that maps the density P Y ( · | x , m , do ( A = a ℓ +1 )) to a scalar value and we sum over all possible realizations ¯ m ℓ of ¯ M ℓ . We are also interested in averaged causal effects

∫ X Q ( x , ¯ a ℓ +1 , M ) P ( x ) d x or differences (Appendix D). The effect Q ( x , ¯ a ℓ +1 , M ) generalizes many common effects across different causal inference settings.

Example 1 (CATE, ℓ = 0 ) . If D = E [ · ] is the expectation functional, Eq. (1) reduces to Q ( x , a , M ) = E [ Y | x , do ( A = a )] . When A is continuous, this is known as the conditional doseresponse function [7, 32]. For binary treatments A ∈ { 0 , 1 } , the query Q ( x , 1 , M ) -Q ( x , 0 , M ) is known as the conditional average treatment effect (CATE) [16, 78], and its averaged version as the average treatment effect (ATE) [64, 69].

Example 2 (Mediation analysis, ℓ = 1 ) . If D = E [ · ] and ¯ M 1 = M is a single mediator, Eq. (1) reduces to Q ( x , ¯ a 1 , M ) = ∑ m E [ Y | x , m, do ( A = a 2 )] P ( m | x , do ( A = a 1 )) . If A ∈ { 0 , 1 } is binary and the M -Y relationship is unconfounded, we obtain the following causal effects studied in mediation analysis [56]: Q ( x , ( a 1 = 0 , a 2 = 1) , M ) -Q ( x , ( a 1 = 0 , a 2 = 0) , M ) is the (conditional) natural direct effect (NDE), and Q ( x , ( a 1 = 1 , a 2 = 0) , M ) -Q ( x , ( a 1 = 0 , a 2 = 0) , M ) is the (conditional) natural indirect effect (NIE).

In general, Eq. (1) includes so-called path-specific effects if the relationship between mediators and outcome is unconfounded [2, 15]. For example, by setting a 1 = 1 and a k = 0 for all 2 ≤ k ≤ ℓ +1 , we obtain the indirect effect that is passed through the mediator sequence ( M 1 , . . . , M ℓ ) . Pathspecific effects are important in various applications, including algorithmic fairness, where the aim is to mitigate effects through paths that are considered unfair [50, 51]. Furthermore, we can set D to a quantile instead of using the mean in all the examples above. This results in distributional versions of CATE, NDE, NIE, and path-specific effects. For example, if the outcome distribution is skewed or contains outliers, practitioners might prefer the median or other quantiles over the mean due to their robustness properties [17].

̸

If all confounders between treatment and mediators and outcome were observed, we could (under additional assumptions) identify the causal effect Q ( x , ¯ a ℓ +1 , M ) from the observational distribution P V on V by replacing all do -operations with conditional probabilities according to the backdoor criterion [55]. However, under unobserved confounding, identifiability is not possible because SCMs M and M ′ exist, which induce the same observational distribution P V , but which result in different causal effects Q ( x , ¯ a ℓ +1 , M ) = Q ( x , ¯ a ℓ +1 , M ′ ) [54, 74].

3.2 Causal sensitivity analysis

In the following, we formalize causal sensitivity analysis as maximizing/minimizing the causal effect Q ( x , ¯ a ℓ +1 , M ) over all SCMs M that are compatible with a predefined sensitivity model . Similar approaches have been used for partial identification with instrumental variables [40, 54] and testing identifiability of counterfactuals [74, 75], where all SCMs are considered that are compatible with the observational data. However, a sensitivity model must additionally restrict the joint distribution of both observed and unobserved variables in order to allow for informative bounds on the causal effect.

Definition 2 (Sensitivity model) . Let P V denote the distribution of the observed variables V = ( X , A , ¯ M ℓ , Y ) . A sensitivity model S is a tuple ( U , P ) , where U = ( U W ) W are unobserved confounders between A and W ∈ { M 1 , . . . , M ℓ , Y } , respectively (see Fig. 1) and a family P of joint probability distributions on V ∪ U , such that, for all P ∈ P , it holds that ∫ P ( v , u ) d u = P V ( v ) . We denote the set of all SCMs M compatible with S (i.e., that respect the causal graph, induce a distribution P ∈ P , and do not contain additional confounders) as C ( S ) (see Appendix E).

Our definition of sensitivity models excludes unobserved confounding between mediators and the outcome. This ensures that the causal effect from Eq. (1) can be interpreted as a path-specific effect [2, 56]. We refer to Sec. 6 for a detailed discussion.

Using the definitions above, we can define causal sensitivity analysis as the following partial identification problem: we aim to obtain bounds Q -( x , ¯ a ℓ +1 , S ) ≤ Q + ( x , ¯ a ℓ +1 , S ) so that

Q + ( x , ¯ a ℓ +1 , S ) is the maximal causal effect that can be achieved by any SCM that is compatible with the sensitivity mode S (and vice versa for Q -( x , ¯ a ℓ +1 , S ) ). Hence, if the sensitivity model is valid, i.e., contains the ground-truth distribution over observed variables and unobserved confounders, we know that the ground-truth causal effect must be contained in the interval [ Q -( x , ¯ a ℓ +1 , S ) , Q + ( x , ¯ a ℓ +1 , S )] . Bounds for average causal effects and effect differences follow immediately (see Appendix D).

3.3 Generalized marginal sensitivity model (GMSM)

We introduce now the GMSM. We begin by providing the general definition and then show that this extends various marginal sensitivity models from existing literature.

Definition 3 (GMSM) . The generalized marginal sensitivity model (GMSM) is a sensitivity model ( U , P ) , where P contains all P that satisfy the following sensitivity constraint: For each W ∈ { M 1 , . . . , M ℓ , Y } , there exist bounds s -W ( a , x ) ≤ 1 ≤ s + W ( a , x ) , so that, for all u W , x , and a

The GMSM bounds the distribution shift in the unobserved confounders U W when performing the intervention do ( A = a ) instead of conditioning on A = a . This is a restriction on the strength of the effect the unobserved confounders U W can have on the treatment A . If U W has no effect on A , Eq. (3) holds with s -W ( a , x ) = s + W ( a , x ) = 1 . Hence, the further s -W ( a , x ) and s + W ( a , x ) deviate from 1 , the larger is the effect from U W on A that the GMSM allows for. We will often use a weighted GMSM, which expresses the bounds in terms of a sensitivity parameter.

Definition 4 (Weighted GMSM) . A weighted GMSM is a GMSM where s -W ( a , x ) and s + W ( a , x ) can be written as s -W ( a , x ) = 1 (1 -Γ W ) q W ( a , x )+Γ W and s + W ( a , x ) = 1 (1 -Γ -1 W ) q W ( a , x )+Γ -1 W for a sensitivity parameter Γ W ≥ 1 and a weight function q W ( a , x ) ∈ [0 , 1] for all x and a .

In a weighted GMSM, the sensitivity parameter Γ W captures the overall restriction on the unobserved confounding strength across individuals. If Γ W = 1 , no unobserved confounding is allowed, and unconfoundedness holds. For Γ W → ∞ , the restriction is relaxed completely, and arbitrary confounding strength is allowed. The weight function q W ( a , x ) ∈ [0 , 1] offers several advantages. It allows to further restrict confounding for individuals with treatments a and covariates x . As q W ( a , x ) → 1 , confounding is more strongly restricted until unconfoundedness is reached. This is helpful in applications where prior knowledge about the confounding structure is available. As an example, consider a medical setting where we want to estimate the effect of new drug treatments on the risk of developing a certain disease, but we suspect that the data is confounded by the individual's genetic risk for the disease, which is not measured in the observational data. However, we also might know that genetic risk for the disease is not relevant for a specific combination of age and gender.

Comparison with the MSM and its extensions: In the following, we show that the weighted GMSMextends popular sensitivity models from the literature. Proofs are in Appendix C. Note that existing sensitivity models do not consider settings with mediators (i.e., ℓ = 0 ), we thus can write Γ = Γ Y for the sensitivity parameter, q ( a , x ) = q Y ( a , x ) for the weight function, and U = U Y for the unobserved confounders. For binary treatments A = A ∈ { 0 , 1 } , the marginal sensitivity model (MSM) [67] is defined via 1 Γ ≤ π ( x ) 1 -π ( x ) 1 -π ( x , u ) π ( x , u ) ≤ Γ , where π ( x ) = P ( A = 1 | x ) denotes the observed propensity score and π ( x , u ) = P ( A = 1 | x , u ) denotes the full propensity score. For continuous treatments A , the continuous marginal sensitivity model (CMSM) [32] is defined via 1 Γ ≤ P ( a | x , u ) P ( a | x ) ≤ Γ . For longitudinal settings with time-varying observed confounders X = ( X 1 , . . . , X T ) , unobserved confounders U = ( U 1 , . . . , U T ) , treatments A = ( A 1 , . . . , A T ) , we define the longitudinal marginal sensitivity model (LMSM) via 1 Γ ≤ ∏ T t =1 P ( a t | ¯ x T , ¯ u t , ¯ a t -1 ) P ( a t | ¯ x T , ¯ a t -1 ) ≤ Γ .

Lemma 1. The MSM, CMSM and LMSM are special cases of the weighted GMSM by choosing the weight functions q ( a , x ) = P ( a | x ) (MSM) and q ( a , x ) = 0 (CMSM, LMSM).

Lemma 1 provides a new interpretation of the MSM : If the probability P ( a | x ) is large, the MSMrestricts the confounding strength because most of the randomness in the treatment is already explained by the observed confounders x . We can extend this approach to arbitrary discrete treatments within the weighted GMSM framework. The CMSM and LMSM apply the same confounding restriction for all a and x .

4 Bounding causal effects under the GMSM

In this section, we derive sharp bounds under our GMSM. We first quantify the maximal shift in conditional distributions (Sec. 4.1). This allows us to derive an algorithm to compute explicit bounds (Sec. 4.2). Finally, we show how these bounds can be estimated from finite data (Sec. 4.3).

4.1 Shifting interventional distributions

In the following, we provide some intuition before stating the main result. Let us consider a simple setting with treatment A , a single unobserved confounder U Y = U ∈ R , outcome Y , and GMSM bounds s -Y ( a ) and s + Y ( a ) (see Fig. 2, left). Any SCM M that is compatible with S describes the relationship between A , U , and Y via a functional assignment Y = f Y ( A , U ) . For a fixed treatment a , we denote f Y ( a , · ) as f a . We are interested in the interventional density P ( y | do ( A = a )) = ∫ P ( y | a , u ) P ( u ) d u = f a # P U ( y ) , where f a # P U denotes the push-forward distribution induced by f a on P U . However, we only have access to the observational density P ( y | a ) = ∫ P ( y | a , u ) P ( u | a ) d u = f a # P U | a ( y ) . Hence, for a fixed functional assignment f a , we can quantify the discrepancy between P ( y | do ( A = a )) and P ( y | a ) via the distribution shift between P U and P U | a .

Fig. 2 shows a toy example where P U | a is the uniform distribution on [0 , 1] , and the functional assignment f a is the inverse standard normal CDF Φ -1 , so that P ( y | a ) is the standard normal probability density. We now want to 'right-shift' the interventional density P ( y | do ( A = a )) as much as possible, so that F ( y | a ) ≫ F ( y | do ( A = a )) for the CDFs. To achieve this, the distribution P U must put more probability mass on the right-hand side of the unit interval [0 , 1] as compared to P U | a (see Fig. 2, left).

Then, the functional assignment will also push more probability mass to the right of P ( y | do ( A = a )) as compared to P ( y | a ) (see Fig. 2, right). However, the GMSM bounds the distribution shift between P U and P U | a via 1 /s + Y ( a ) ≤ P ( u ) ≤ 1 /s -Y ( a ) (Eq. (3) using that P ( u ) = P ( u | do ( A = a ) ). Intuitively, the max- imal

possible right

shift under

the

GMSM should occur by choosing P ( u ) so that the bounds are attained, i.e., P ( u ) = ✶ ( u ≤ c + Y )(1 /s + Y ) + ✶ ( u > c + Y )(1 /s -Y ) for some c + Y that can be obtained via the normalization constraint ∫ P ( u ) d u = 1 . The corresponding 'right-shifted' interventional density P + ( y | do ( A = a )) is defined via the push-forward (see Fig. 2).

It turns out that the density P + ( y | do ( A = a )) is the maximally right-shifted interventional distribution that can be obtained under any SCM that is compatible with the GMSM. Furthermore, the above arguments can be generalized beyond the toy example from Fig. 2.

Theorem 1. Let S be a GMSM with bounds s -W = s -W ( a , x ) and s + W = s + W ( a , x ) for W ∈ { M 1 , . . . , M ℓ , Y } . We define c + W = (1 -s -W ) s + W s + W -s -W . If W ∈ R is continuous, we define the probability density function

where F ( · ) is the CDF corresponding to P ( · | x , m W , a ) . If W ∈ N is discrete, we define the probability mass function

Let F + ( · ) denote the conditional CDF corresponding to P + ( · | x , m W , a ) and let F -( · ) be the conditional CDF of P -( · | x , m W , a ) , which is defined by swapping signs (in c W and s W ). For any SCM M , denote the CDF corresponding to P M ( · | x , m W , do ( A = a )) as F M ( · ) . Then, for all w ,

Assume now that P ( u W | x , do ( A = a )) = P ( u W | x ) . Then, the bounds are sharp (i.e., equality holds in Eq. (6) ) whenever A is discrete and it holds that 1 /s + W ≥ P ( a | x ) , or A is continuous.

The result in Theorem 1 does not depend on the distribution or dimensionality of unobserved confounders, and it does not depend on any specific SCM. As such, our sharp bounds are applicable to a wide class of causal effects, without restricting assumptions on the confounding structure beyond the sensitivity constraint of the GMSM.

In case S is a weighted GMSM with sensitivity parameters Γ W for all W ∈ { M 1 , . . . , M ℓ , Y } , the quantiles c + W and c -W are of the particular simple form c + W = Γ W / (1 + Γ W ) and c -W = 1 / (1 + Γ W ) . Furthermore, the discrete sharpness condition simplifies to (1 -Γ -1 W ) q W ( a , x ) + Γ -1 W ≥ P ( a | x ) . In particular, the bounds are sharp whenever we choose a weighting function that satisfies q W ( a , x ) ≥ P ( a | x ) . The assumption P ( u W | x , do ( A = a )) = P ( u W | x ) excludes the time-varying case (LMSM). Deriving sharp bounds for the LMSM is an interesting direction for future research.

4.2 Bounding causal effects

We now leverage Theorem 1 to obtain explicit solutions for the partial identification problem from Eq. (2) with monotone D (see Appendix B). This includes expectation and distributional effects.

Corollary 1 (Bounds without mediators) . If ℓ = 0 and D is monotone, we obtain sharp bounds

and sharpness holds under the same conditions as in Theorem 1.

If D is the expectation functional, A is binary, and Y is continuous, this coincides with the optimality result from Dorn and Guo [18]. Hence, Corollary 1 generalizes the result from Dorn and Guo [18] to distributional effects and arbitrary treatments (e.g., categorical, continuous, or time-varying). In their paper, Dorn and Guo proved the sharpness of the bounds by using the Neyman-Pearson Lemma. In contrast, we take the more principled approach outlined in Sec. 4.1, which is applicable to more general settings.

In the following, we consider settings with mediators, i.e., we aim to derive bounds for the causal effect in Eq. (1). The idea is to first obtain outcome bounds D ( P Y + ( · | x , ¯ m ℓ , a ℓ +1 ) ) conditioned on all possible values ¯ m ℓ ∈ supp ( ¯ M ℓ ) in the support of the mediators ¯ M ℓ . Without unobserved confounding between treatments and mediators, we can obtain the upper bound Q + ( x , ¯ a ℓ +1 , S ) from Eq. (1) by replacing D ( P Y ( · | x , ¯ m ℓ , do ( A = a ℓ +1 ) )

with D ( P Y + ( · | x , ¯ m ℓ , a ℓ +1 ) ) . With unobserved confounding, we need to additionally take the distribution shift in the mediators into account. To maximize the causal effect, the shifted mediator distribution should put more probability mass on values ¯ m ℓ for which D ( P Y + ( · | x , ¯ m ℓ , a ℓ +1 ) ) is large. Hence, we can apply the maximal right-shift from Theorem 1 to the mediator distributions in Eq. (1), but where the values ¯ m ℓ are permuted to order D ( P Y + ( · | x , ¯ m ℓ , a ℓ +1 ) ) in ascending order. We provide the details on our iterative procedure to compute the upper bound Q + ( x , ¯ a ℓ +1 , S ) in Algorithm 1. The lower bound Q -( x , ¯ a ℓ +1 , S ) can be computed analogously by swapping signs in c W and s W .

Corollary 2. Under the Assumptions of Theorem 1, Algorithm 1 returns the sharp bounds from Eq. (2) .

4.3 Empirical bounds via importance sampling

In practice, we only have access to an empirical distribution P V n of sample size n instead of the full observational distribution P V . We thus obtain estimates ˆ P ( w | x , m W , a ) of the conditional density or probability mass functions P ( w | x , m W , a ) for all W ∈ { M 1 , . . . , M ℓ , Y } . If W is discrete, this reduces to a standard (multi-class) classification problem, and the estimated class probabilities can be plugged into Algorithm 1. If W = Y is continuous, we can use arbitrary conditional density estimators to obtain ˆ P ( y | x , ¯ m ℓ , a ) . We propose an importance sampling approach to estimate D ( P Y + ( · | x , ¯ m ℓ , a ) ) assuming that we can sample from from our estimated density ˆ P Y ( · | x , ¯ m ℓ , a ) (see Appendix F for details and derivations). If D is the expectation functional, our estimator is

We also provide importance sampling estimators for distributional effects in Appendix F. We can also obtain empirical confidence intervals by using the same bootstrap procedure as described in [32].

Implementation: We use feed-forward neural networks with softmax activation function to estimate discrete probability mass functions. For densities, we use conditional normalizing flows [73] (neural spline flows [21]), which are universal density approximators and allow for sampling. We perform training using the Adam optimizer [41]. We also perform extensive hyperparameter tuning in our experiments. Implementation and hyperparameter tuning details are in Appendix G.

5 Experiments

Baselines: Most existing methods focus on sensitivity analysis for binary treatments under the MSM [18, 19, 31, 35, 53]. In this setting, our bounds coincide with existing optimality results [18]. For mediation analysis, we are to the best of our knowledge the first to propose bounds for MSM-based sensitivity analysis. This is why we refrain from benchmarking against baselines in our experiments and only show the validity of our bounds.

Synthetic data: We perform extensive experiments using synthetic data from various causal inference settings to evaluate the validity of our bounds. Synthetic data are commonly used to evaluate causal inference methods as they ensure that the causal ground truth is available [6, 16, 76]. Here, we generate six different synthetic datasets with n = 50 , 000 samples in the following manner: We first construct three SCMs with binary treatments for the causal graphs in Fig. 1, that is, (i) no mediators, (ii) a single mediator M , and (iii) two mediators M 1 and M 2 . We then construct three more SCMs for settings (i)-(iii) with continuous treatments. Details are in Appendix H.

























Y





















M

1

=

Y



























M

1

=

M

2

=

Y

































Y



Y



Y



Y









































M

1



Y



M

1



Y



M

1



Y



M

1



Y





































M

1



M

2



Y



M

1



M

2



Y



M

1



M

2



Y



M

1



M

2



Y









To demonstrate the validity of our bounds, we aim to show that they contain the oracle causal effect whenever the sensitivity constraints are satisfied. We evaluate two versions of our GMSM: the MSM for binary treatments and the CMSMfor continuous treatments (see Sec. 3.3). Using oracle knowledge from the SCMs, we estimate the respective density ratio from Eq. (3) and obtain the oracle sensitivity parameter Γ ∗ W (details are in Appendix H).

We thus demonstrate that our bounds contain the oracle causal effect whenever Γ W ≥ Γ ∗ W , i.e., whenever we choose sensitivity parameters Γ W at least as large as the oracle Γ ∗ W .

The results for binary treatments are shown in Fig. 3 and for continuous treatments in Fig. 4. For binary treatments, we evaluate the causal effect Q ( x , ¯ a ℓ +1 , M ) for over x ∈ [ -1 , 1] with three (arbitrary) treatment combinations corresponding to the three different settings:























Y























M

1

Y

























M

1

M

2

Y







































Y



Y



Y



Y







































M

1



Y



M

1



Y



M

1



Y



M

1



Y



































M

1



M

2



Y



M

1



M

2



Y



M

1



M

2



Y



M

1



M

2



Y









(i) ¯ a 1 = 1 , (ii) ¯ a 2 = (1 , 0) , and (iii) ¯ a 3 = (1 , 0 , 0) . For continuous treatments, we also use three (arbitrary) treatment combinations: (i) ¯ a 1 = 0 . 6 , (ii) ¯ a 2 = (0 . 9 , 0 . 5) , and (iii) ¯ a 3 = (0 . 2 , 0 . 4 , 0 . 5) . Results for additional treatment combinations are in Appendix J. Evidently, the oracle causal effect is contained within our bounds. Hence, the results confirm the validity of our bounds.

We also compare the bounds from Jesson et al.

[32] with our sharp bounds under the CMSM and examine whether we can improve on these by using a different weight function (Def. 4). For this purpose, we modify the SCM from setting (i) in the continuous treatment setting so that there is no unobserved confounding for individuals with x > 0 (i.e., Γ ∗ Y = 0 ). Details are in Appendix H. Table 1 reports the bounds from Jesson et al., our bounds under the CMSM, and our bounds under a weighted CMSM with weight function q Y ( x ) = ✶ ( x > 0) . The two main findings are: (i) Using the weight function, we can leverage prior knowledge about the confounding structure to obtain even tighter bounds. (ii) Our bounds are much faster to compute (using 5 , 000 samples). The latter is because we derived closed-form solutions for our bounds, while the method proposed by Jesson et al. is an approximation that uses grid search.

Real-world data: We demonstrate our bounds using an example with real-world data. We consider a setting from the COVID-19 pandemic where mobility (captured through telephone movement) was monitored to obtain a leading predictor of case growth [57]. Details regarding the data (publicly available) and our analysis are in Appendix I. Here, mobility is the mediator, case growth is the outcome, and stay-home order (ban of gatherings with more than 5 people) is the treatment. We are interested in the natural directed effect (NDE, see Example 2) of a stay-home order on the case growth. We suspect that unobserved confounders between treatment and mediator might exist (e.g., adherence, etc. of the population). Hence we perform a causal sensitivity analysis where we vary Γ M and plot the bounds in Fig. 5. We observe that the estimated effect under unconfoundedness ( Γ M = 1 ) is negative, i.e., the stay-home order decreases case growth. For Γ M > 1 , we obtain bounds around this estimand. For Γ M < 6 the bounds are negative, which means that under moderate unobserved confounding, it seems likely that the NDE is nonzero, in line with prior evidence [57].

6 Discussion

Assumptions: As common in the causal inference literature, our results rely on assumptions on the data-generating process that must be justified by domain knowledge. Our main assumption is that we exclude confounders between mediators and the outcome in our analysis (see Def. 2). The reason

for this assumption is that it allows us to interpret the causal query from Eq. (1) as a path-specific effect. Path-specific effects are defined as so-called nested counterfactuals that lie in the third layer of Pearl's causal hierarchy [55] Using the assumption from Theorem 1, they can be reduced to the query from Eq. (1), which lies in layer 2 of Pearl's hierarchy, i.e., only depends on interventions [15]. Our sensitivity analysis then bridges the gap from layer 2 to layer 1 (observational data). Approaches for relaxing this assumption, e.g., by combining our results with a sensitivity analysis from layer 3 to layer 2, are left for future work.











M



















Sensitivity parameter and weighting function: Both the sensitivity parameter Γ W and the weighting function q ( a , x ) incorporate domain knowledge about the unobserved confounding and need to be chosen by the practitioner accordingly.

The sensitivity parameter Γ W controls the strength of unobserved confounding. In practice, one typically chooses Γ by domain knowledge or data-driven heuristics [35, 28]. One approach is to obtain the smallest Γ so that the corresponding partially identified interval includes 0 . Then, Γ can be interpreted as a level of 'causal uncertainty', quantifying the smallest violation of unconfoundedness that would explain away the causal effect [31, 33].

The weight function q ( a , x ) offers additional opportunities to incorporate domain knowledge about the confounding structure to obtain tighter bounds. Consider an observational study on the effect of smoking on cancer risk, confounded by certain unobserved genes. For example, it may be known that genes that act as unobserved confounders do not affect the cancer risk for a certain population with certain covariates x , which allows us to set q ( a , x ) = 1 . Note that we can always set q ( a , x ) = P ( a | x ) (discrete treatments) or q ( a , x ) = 0 (continuous treatments) if no domain knowledge is available, leading to established sensitivity models from the literature. For example, practitioners may use our bounds for the MSM in a mediation analysis setting without ever explicitly using the GMSM formulation via a weighting function.

Other sensitivity models: In this paper, we provide bounds for MSM-type sensitivity models. Recently, other types of sensitivity models have been proposed in the literature, which may provide less conservative bounds in situations where the data-generating process does not follow an MSM. Examples include f -sensitivity models [34], L 2 -sensitivity models [81], curvature sensitivity models [48] and the δ -MSM [45]. Extending our results to these sensitivity models may be another possible direction for future work.

Efficient estimation: Our main results (Theorem 1, Corollary 1, 2) are identifiability results , i.e., hold in the limit of infinite data. We did not provide results on efficient estimation. Therefore, future work may consider extending our approach to incorporate semiparametric efficiency theory [37].

Conclusion: We proposed a flexible generalization of the MSM and derived sharp bounds for a variety of different causal inference settings and data types (e.g., continuous, categorical). Our work provides practitioners with a unified framework for causal sensitivity analysis. This enables reliable causal inferences from observational data in the presence of unobserved confounders, thus promoting safe decision-making.

Acknowledgments

SF acknowledges funding from the Swiss National Science Foundation (SNSF) via Grant 186932.

References

Joshua D. Angrist. 'Lifetime earnings and the vietnam era draft lotter: Evidence from social security administrative records'. In: The American Economic Review 80.3 (1990), pp. 313-336.

Chen Avin, Ilya Shpitser, and Judea Pearl. 'Identifiability of path-specific effects'. In: IJCAI . 2005.

Vahid Balazadeh, Vasilis Syrgkanis, and Rahul G. Krishnan. 'Partial identification of treatment effects with implicit generative models'. In: NeurIPS . 2022.

Elias Bareinboim et al. 'On Pearl's hierarchy and the foundations of causal inference'. In: Probabilistic and Causal Inference: The Works of Judea Pearl (2022), pp. 507-556.

Andrew Bennett, Nathan Kallus, and Tobias Schnabel. 'Deep generalized method of moments for instrumental variable analysis'. In: NeurIPS . 2019.

Jeroen Berrevoets et al. 'To Impute or not to Impute? Missing data in treatment effect estimation'. In: AISTATS . 2023.

Ioana Bica, James Jordon, and Mihaela van der Schaar. 'Estimating the effects of continuousvalued interventions using generative adversarial networks'. In: NeurIPS . 2020.

Ioana Bica et al. 'Estimating counterfactual treatment outcomes over time through adversarially balanced representations'. In: ICLR . 2020.

Stephan Bongers et al. 'Foundations of structural causal models with cycles and latent variables'. In: The Annals of Statistics 49.5 (2021), pp. 2885-2915.

Matteo Bonvini et al. 'Sensitivity analysis for marginal structural models'. In: arXiv preprint arXiv:2210.04681 (2022).

Kan Chen, Bingkai Wang, and Dylan S. Small. 'A differential effect approach to partial identification of treatment effects'. In: arXiv preprint arXiv:2303.06332 (2023).

Victor Chernozhukov, Ivan Fernández-Val, and Blaise Melly. 'Inference on counterfactual distributions'. In: Econometrica 81.6 (2013), pp. 2205-2268.

Victor Chernozhukov et al. 'Double/debiased machine learning for treatment and structural parameters'. In: The Econometrics Journal 21.1 (2018), pp. C1-C68.

James Cornfield et al. 'Smoking and lung cancer: Recent evidence and a discussion of some questions'. In: Journal of the National Cancer Institute 22.1 (1959), pp. 173-203.

Juan D. Correa, Sanghack Lee, and Elias Bareinboim. 'Nested counterfactual identification from arbitrary surrogate experiments'. In: NeurIPS . 2021.

Alicia Curth and Mihaela van der Schaar. 'Nonparametric estimation of heterogeneous treatment effects: From theory to learning algorithms'. In: AISTATS . 2021.

David L. Donoho and Peter J. Huber. 'The notion of breakdown point'. In: A Festschrift for Erich L. Lehmann (1983), pp. 157-184.

Jacob Dorn and Kevin Guo. 'Sharp sensitivity analysis for inverse propensity weighting via quantile balancing'. In: Journal of the American Statistical Association (2022).

Jacob Dorn, Kevin Guo, and Nathan Kallus. 'Doubly-valid/ doubly-sharp sensitivity analysis for causal inference with unmeasured confounding'. In: arXiv preprint arXiv:2112.11449 (2022).

Guilherme Duarte et al. 'An automated approach to causal inference in discrete settings'. In: Journal of the American Statistical Association (2023).

Conor Durkan et al. 'Neural spline flows'. In: NeurIPS . 2019.

Helmut Farbmacher et al. 'Causal mediation analysis with double machine learning'. In: The Econometrics Journal 25.2 (2022), pp. 277-300.

Dennis Frauen and Stefan Feuerriegel. 'Estimating individual treatment effects under unobserved confounding using binary instruments'. In: ICLR . 2023.

Dennis Frauen et al. 'Estimating average causal effects from patient trajectories'. In: AAAI . 2023.

Florian Gunsilius. 'A path-sampling method to partially identify causal effects in instrumental variable models'. In: arXiv preprint arXiv:1910.09502 (2020).

Wenshuo Guo et al. 'Partial identification with noisy covariates: A robust optimization approach'. In: CLeaR . 2022.

Jason Hartford et al. 'Deep IV: A flexible approach for counterfactual prediction'. In: ICML . 2017.

Tobias Hatt, Daniel Tschernutter, and Stefan Feuerriegel. 'Generalizing off-policy learning under sample selection bias'. In: UAI . 2022.

Jesse Y. Hsu and Dylan S. Small. 'Calibrating sensitivity analyses to observed covariates in observational studies'. In: Biometrics 69.4 (2013), pp. 803-811.

Guido W. Imbens. 'Sensitivity to exogeneity assumptions in program evaluation'. In: American Economic Review 93.2 (2003), pp. 128-132.

Andrew Jesson et al. 'Quantifying ignorance in individual-level causal-effect estimates under hidden confounding'. In: ICML . 2021.

Andrew Jesson et al. 'Scalable sensitivity and uncertainty analysis for causal-effect estimates of continuous-valued interventions'. In: NeurIPS . 2022.

Ying Jin, Zhimei Ren, and Emmanuel J. Candès. 'Sensitivity analysis of individual treatment effects: A robust conformal inference approach'. In: Proceedings of the National Academy of Sciences (PNAS) 120.6 (2023).

Ying Jin, Zhimei Ren, and Zhengyuan Zhou. 'Sensitivity analysis under the f -sensitivity models: A distributional robustness perspective'. In: arXiv preprint arXiv:2203.04373 (2022).

Nathan Kallus, Xiaojie Mao, and Angela Zhou. 'Interval estimation of individual-level causal effects under unobserved confounding'. In: AISTATS . 2019.

Nathan Kallus and Angela Zhou. 'Confounding-robust policy improvement'. In: NeurIPS . 2018.

Edward H. Kennedy. 'Semiparametric doubly robust targeted double machine learning: A review'. In: arXiv preprint (2022).

Edward H. Kennedy. 'Towards optimal doubly robust estimation of heterogeneous causal effects'. In: arXiv preprint (2022).

Edward H. Kennedy, Sivaraman Balakrishnan, and Larry Wasserman. 'Semiparametric counterfactual density estimation'. In: Biometrika (2023).

Niki Kilbertus, Matt J. Kusner, and Ricardo Silva. 'A class of algorithms for general instrumental variable models'. In: NeurIPS . 2020.

Diederik P. Kingma and Jimmy Ba. 'Adam: A method for stochastic optimization'. In: ICLR . 2015.

Sören R. Künzel et al. 'Metalearners for estimating heterogeneous treatment effects using machine learning'. In: Proceedings of the National Academy of Sciences (PNAS) 116.10 (2019), pp. 4156-4165.

Bryan Lim, Ahmed M. Alaa, and Mihaela van der Schaar. 'Forecasting treatment responses over time using recurrent marginal structural networks'. In: NeurIPS . 2018.

Charles F. Manski. 'Nonparametric bounds on treatment effects'. In: The American Economic Review 80.2 (1990), pp. 319-323.

Myrl G. Marmarelis et al. 'Partial identification of dose responses with hidden confounders'. In: UAI . 2023.

Valentyn Melnychuk, Dennis Frauen, and Stefan Feuerriegel. 'Causal transformer for estimating counterfactual outcomes'. In: ICML . 2022.

Valentyn Melnychuk, Dennis Frauen, and Stefan Feuerriegel. 'Normalizing flows for interventional density estimation'. In: ICML . 2023.

Valentyn Melnychuk, Dennis Frauen, and Stefan Feuerriegel. 'Partial counterfactual identification of continuous outcomes with a curvature sensitivity model'. In: NeurIPS . 2023.

Krikamol Muandet et al. 'Counterfactual mean embeddings'. In: Journal of Machine Learning Research 22 (2021), pp. 1-71.

Razieh Nabi, Daniel Malinsky, and Ilya Shpitser. 'Learning optimal fair policies'. In: ICML . 2019.

Razieh Nabi and Ilya Shpitser. 'Fair inference on outcomes'. In: AAAI . 2018.

Ziad Obermeyer et al. 'Dissecting racial bias in an algorithm used to manage the health of populations'. In: Science 366.6404 (2019), pp. 447-453.

Miruna Oprescu et al. 'B-learner: Quasi-oracle bounds on heterogeneous causal effects under hidden confounding'. In: ICML . 2023.

Kirtan Padh et al. 'Stochastic causal programming for bounding treatment effects'. In: CLeaR . 2023.

Judea Pearl. Causality . New York City: Cambridge University Press, 2009.

Judea Pearl. 'Interpretation and identification of causal mediation'. In: Psychological methods 19.4 (2014), pp. 459-481.

Joel Persson, Jurriaan F. Parie, and Stefan Feuerriegel. 'Monitoring the COVID-19 epidemic with nationwide telecommunication data'. In: Proceedings of the National Academy of Sciences of the United States of America 118.26 (2021).

Danilo Jimenez Rezende and Shakir Mohamed. 'Variational inference with normalizing flows'. In: ICML . 2015.

James M. Robins, Miguel A. Hernán, and Babette Brumback. 'Marginal structural models and causal inference in epidemiology'. In: Epidemiology 11.5 (2000), pp. 550-560.

James M. Robins, Andrea Rotnitzky, and Daniel o. Scharfstein. 'Sensitivity analysis for selection bias an unmeasured confounding in missing data and causal inference models'. In: Statistical Models in Epidemiology, the Environment, and Clinical Trials 116 (2000), pp. 1-94.

Paul R. Rosenbaum. 'Sensitivity analysis for certain permutation inferences in matched observational studies'. In: Biometrika 74.1 (1987), pp. 13-26.

Paul R. Rosenbaum and Donald B. Rubin. 'Assessing sensitivity to an unobserved binary covariate in an observational Study with binary outcome'. In: Journal of the Royal Statistical Society: Series B 45.2 (1983), pp. 212-218.

Uri Shalit, Fredrik D. Johansson, and David Sontag. 'Estimating individual treatment effect: Generalization bounds and algorithms'. In: ICML . 2017.

Claudia Shi, David M. Blei, and Victor Veitch. 'Adapting neural networks for the estimation of treatment effects'. In: NeurIPS . 2019.

Rahul Singh, Maneesh Sahani, and Arthur Gretton. 'Kernel instrumental variable regression'. In: NeurIPS . 2019.

Vasilis Syrgkanis et al. 'Machine learning estimation of heterogeneous treatment effects with instruments'. In: NeurIPS . 2019.

Zhiqiang Tan. 'A distributional approach for causal inference using propensity scores'. In: Journal of the American Statistical Association 101.476 (2006), pp. 1619-1637.

Eric J. Tchetgen Tchetgen and Ilya Shpitser. 'Semiparametric theory for causal mediation analysis: Efficiency bounds, multiple robustness, and sensitivity analysis'. In: Annals of Statistics 40.3 (2012), pp. 1816-1845.

Mark J. van der Laan and Donald B. Rubin. 'Targeted maximum likelihood learning'. In: The International Journal of Biostatistics 2.1 (2006).

Stijn Vansteelandt et al. 'Ignorance and uncertainty regions as inferential tools in a sensitivity analysis'. In: Statistica Sinica 16 (2006), pp. 953-979.

Hal R. Varian. 'Causal inference in economics and marketing'. In: Proceedings of the National Academy of Sciences (PNAS) 113.27 (2016), pp. 7310-7315.

Stefan Wager and Susan Athey. 'Estimation and inference of heterogeneous treatment effects using random forests'. In: Journal of the American Statistical Association 113.523 (2018), pp. 1228-1242.

Christina Winkler et al. 'Learning likelihoods with conditional normalizing flows'. In: arXiv preprint arXiv:1912.00042 (2019).

Kevin Xia, Yushu Pan, and Elias Bareinboim. 'Neural causal models for counterfactual identification and estimation'. In: ICLR . 2023.

Kevin Xia et al. 'The causal-neural connection: Expressiveness, learnability, and inference'. In: NeurIPS . 2021.

Liyuan Xu et al. 'Learning deep features in instrumental variable regression'. In: ICLR . 2021.

Azam M. Yazdani and Eric Boerwinkle. 'Causal inference in the age of decision medicine'. In: Journal of Data Mining in Genomics & Proteomics 6.1 (2015).

Jinsung Yoon, James Jordon, and Mihaela van der Schaar. 'GANITE: Estimation of individualized treatment effects using generative adversarial nets'. In: ICLR . 2018.

Marco Zaffalon, Alessandro Antonucci, and Rafael Cabañas. 'Structural causal models are (solvable by) credal networks'. In: International Conference on Probabilistic Graphical Models 2020 . 2020.

Junzhe Zhang, Jin Tian, and Elias Bareinboim. 'Partial counterfactual identificatiton from observational and experimental data'. In: ICML . 2022.

Yao Zhang and Qingyuan Zhao. 'Bounds and semiparametric inference in L ∞ and L 2 sensitivity analysis for observational studies'. In: arXiv preprint arXiv:2211.04697 (2022).

Qingyuan Zhao, Dylan S. Small, and Bhaswar B. Bhattacharya. 'Sensitivity analysis for inverse probability weighting estimators via the percentile bootstrap'. In: Journal of the Royal Statistical Society: Series B 81.4 (2019), pp. 735-761.

A Extended related work

A.1 Partial identification under unobserved confounding

There are various works for partial identification (i.e., bounding causal effects) under unobserved confounding that do not impose sensitivity models. In the following, we discuss the difference between this literature stream, which we call causal partial identification (CPA), to causal sensitivity analysis (CSA).

The main difference between CSA and CPA is that CSA imposes sensitivity models, that is, assumptions on the strength of unobserved confounding, which is controlled by a sensitivity parameter Γ . Methods for CPA do not impose sensitivity constraints but instead impose other assumptions. In practice, CSA can be used to test the robustness of causal effect estimates to violations of the unconfoundedness assumption (by varying Γ ), while CPA may be applicable in situations where no domain knowledge about the confounding strength is available.

Approaches for CPA can be roughly described by two categories that describe the type of assumptions that are imposed to derive informative bounds.

Additional variables: In order to achieve informative bounds without restricting the strength of unobserved confounding, some works impose assumptions on the data-generating process by postulating the existence of additional variables. One example is instrumental variables (IVs), i.e., variables, which only have a direct effect on treatment variables but not on outcomes. Under certain assumptions, IVs render bounds for causal effects informative without assumptions on the underlying confounding structure [3, 25, 40, 54]. Other examples include leaky mediation [3, 54], differential effects [11], noisy proxy settings [26].

Discrete SCMs: Another stream of literature derives informative bounds in discrete SCMs. Examples include [20, 75, 79, 80].

Comparison with our bounds : None of the methods above is applicable in the causal inference settings we consider (e.g., continuous treatments or outcomes, no IVs available, etc.). In contrast, a well-known CPA result in our setting is the so-called no-assumptions bound [44] that leverages discrete treatments. In the following, we consider the standard setting for CATE estimation without mediators ( ℓ = 0 ) and a binary treatment A ∈ { 0 , 1 } . We show that we obtain the no-assumptions bound with our sensitivity analysis for the query E [ Y | x, do ( A = a )] for Γ →∞ , i.e. when lifting the sensitivity constraint. Let [ p 1 , p 2 ] denote the support of the conditional density P ( y | x, a ) with c.d.f. F . Using the MSM we obtain

which corresponds to the result in [44]. The result for the lower bound Q -( x , a, S ) follows analogously.

A.2 Estimation of causal effects under unconfoundedness

Under certain additional assumptions, unconfoundedness makes it possible to point-identify causal effects from the observational data, so that the causal inference problem reduces to a purely statistical estimation problem. Various methods for estimating point-identified causal effects under unconfoundedness have been proposed that make use of machine learning and/or (semiparametric) statistical theory. Examples include methods for conditional average treatment effects [13, 16, 38, 42, 63, 72, 78], average treatment effects [24, 64, 69], instrumental variables [5, 23, 27, 65, 66, 76], time-varying data [8, 43, 46], mediation analysis [68, 22], and distributional effects [12, 39, 47, 49]. Note that all the methods above are biased if the unconfoundedness assumption is violated, which outlines the need for our causal sensitivity analysis.

B Proofs of the GMSM bounds

B.1 Proof of Theorem 1

Proof. We give the proof for continuous W ∈ R (see Eq. (4) in the main paper). The derivation for discrete W ∈ N (see Eq. (5) in the main paper) follows with the same arguments and the normalization constraint ∑ w P + ( w | x , m W , a ) = 1 . We first show the inequality

and then provide sharpness results by showing

under the sharpness conditions of Theorem 1. The result for F -( w ) follows analogously.

Validity of the bounds ( ≤ ): We provide a proof by contradiction. To do so, we assume that there exists an SCM M∈C ( S ) and w ∈ R so that

By the definition of F + ( w ) , there must exist a set W 1 ⊆ R ≤ F -1 ( c + W ) , so that

or a set W 2 ⊆ R >F -1 ( c + W ) , so that

as otherwise P ( w | x , m W , do ( A = a )) would not integrate to 1 . Let W = f ( X , M W , A , U W ) be the functional assignment of M and let U 1 = f -1 x , m W , a ( W 1 ) ⊆ R d and U 2 = f -1 x , m W , a ( W 2 ) ⊆ R d denote the preimages of W 1 and W 2 under f x , m W , a in the confounding space.

We can again write P + ( w | x , m W , a ) as a push forward

for some density P + ( u W | x , a ) on the confounding space. By the definition of P + ( w | x , m W , a ) and Eq. (28), we obtain

for all u 1 ∈ U 1 and u 2 ∈ U 2 . Due to the definition of U 1 and U 2 , it follows that there exist u 1 ∈ U 1 and u 2 ∈ U 2 , so that

and

Both Eq. (19) and Eq. (20) are contradictions to the GMSM constraint Eq. (3) of the main paper. Hence, M / ∈ C ( S ) .

Sharpness of the bounds ( ≥ ): We show that under the sharpness conditions from Theorem 1, there exists an SCM M∈C ( S ) with induced interventional density P + ( w | x , m W , a ) for all w . The construction of M is similar to that of our motivational toy example in Sec. 4.1 of the main paper. We first define an (interventional) probability density for the unobserved confounder U W ∈ R d given X via

✶ ✶ where u (1) W denotes the first coordinate of u W . P ( u W | x , m W ) is a properly normalized density with support [0 , 1] d because

where we used the definition of c + W = (1 -s -W ) s + W s + W -s -W .

We now define the probability density for the unobserved confounder U W ∈ R d given X , M W , and treatments A as the uniform density on [0 , 1] d , i.e., where the indicator function is defined coordinate-wise.

We now need to show that there always exists an SCM M which induces the densities in Eq. (21) and Eq. (22) and that satisfies the sensitivity constraints. For this purpose, we use the assumptions to write the interventional density for discrete A as

̸

Hence, our definitions for P ( u W | x , do ( A )) and P ( u W | x , a ) are valid whenever

̸

which follows from the sharpness condition 1 /s + W ≥ P ( a | x ) . Sharpness for continuous A follows because we can approximate A with a sequence of discrete treatments ( A n ) n and it holds that P ( a n | x ) - - - - → n →∞ 0 (as long as P ( a | x ) is non-degenerate). Finally, it holds that

so that M respects the sensitivity constraint of the GMSM S .

Let now P ( w | x , m W , a ) denote the observational density of W given X , M W , and A with corresponding cumulative distribution function (CDF) given by F ( w ) . To complete our construction of M , we define functional assignment W = f W ( X , M W , A , U W ) via the inverse CDF

By denoting f W ( x , m W , a , · ) as f x , m W , a , we can write the observational distribution under M as the push forward

due to Eq. (22) and Eq. (27). Note that we used here that U W is not a parent of M W (see Fig. 1). Hence, M∈C ( S ) is compatible with the sensitivity model S . Furthermore, the induced interventional distribution can be written as the push-forward

because of Eq. (21).

B.2 Proof of Corollary 1

Here, we formally restate Corollary 1 for monotone functionals. For two probability densities P ( y ) and P ′ ( y ) , we denote P ≤ P ′ if F ≥ F ′ holds almost surely for the corresponding CDFs.

Definition 5. A functional D is called monotone if D ( P ( · )) ≤ D ( P ′ ( · )) whenever P ≤ P ′ .

Intuitively, a monotone functional increases if applied on a distribution that is further right-shifted. Note that both the expectation functional D ( P ( · )) = ∫ y P ( y ) d y and the quantile functionals D ( P ( · )) = F -1 ( α ) for α ∈ [0 , 1] are monotone.

Corollary 3 (Restatement) . If M = ∅ and D is monotone, we obtain sharp bounds

Proof. Follows directly from Theorem 1 for W = Y .

B.3 Proof of Corollary 2

Proof. We derive Algorithm 1 for Q + ( x , ¯ a ℓ +1 , S ) . The case for Q -( x , ¯ a ℓ +1 , S ) follows analogously. Recall that we want to maximize the causal effect

over all possible SCMs M∈C ( S ) that are compatible with the GMSM S . By using the assumption (no unobserved confounding between mediators and outcome), we can write Q ( x , ¯ a , M ) in terms of functional assignments f W x , m W , a defined via W = f W ( X , M W , A , U W ) and induced (interventional) distributions P U W | x in the following way:

Hence, the optimization problem reduces to maximizing Eq. (32) over all functional assignments f W x , m W , a and distributions P U W | x that are compatible with S . Note that the terms in the product do not depend on each other or the term in the sum. Thus, by rearranging the suprema and products, we can equivalently perform the following iterative procedure: First, we initialize

and then define

for all i ∈ { ℓ, . . . , 1 } , which results in the sharp upper bound

For Eq. (33) and monotone D , we can directly apply Theorem 1 and obtain

For Eq. (34), we need to find an induced distribution f M i x , ¯ m i -1 , a i # P U Y | x on M i that is compatible with S and puts most probability mass on m i where Q + i +1 ( x , ¯ m i -1 , m i , ¯ a ℓ +1 )) is large. Hence, we can apply the discrete version of Theorem 1 with W = π ( M i ) , where π : supp ( M i ) → supp ( M i ) is a permutation map so that ( Q + i +1 ( x , ¯ m i -1 , π ( m i )) , ¯ a ℓ +1 ) m i ∈ supp ( M i ) is ordered in ascending order. The corresponding update step is shown in Algorithm 1.

C Special cases of the GMSM

In this section, we prove Lemma 1, i.e., we show that all sensitivity models introduced in Sec. 3.3 of the main paper are special cases of our (weighted) GMSM. Recall that we consider settings without mediators (i.e., ℓ = 0 ), and write Γ = Γ Y for the sensitivity parameter, q ( a , x ) = q Y ( a , x ) for the weight function, and U = U Y for the unobserved confounders. In this case, the weighted GMSM is defined via the confounding restriction

C.1 Marginal sensitivity model (MSM):

The MSM [67] for binary treatment A = A ∈ { 0 , 1 } is defined via

where π ( x ) = P ( A = 1 | x ) denotes the observed propensity score and π ( x , u ) = P ( A = 1 | x , u ) denotes the full propensity score. By rearranging the terms, we obtain

for a ∈ { 0 , 1 } . Furthermore, by Bayes' theorem, it follows that

which implies that the MSM is a weighted GMSM with weight function q ( a , x ) = P ( a | x ) .

Comparison to the MSM definition using potential outcomes: Some papers define the MSM in terms of potential outcomes ( Y 1 , Y 0 ) instead of unobserved confounders U [31, 67]. Here, Y a denotes the potential outcome under the treatment intervention do ( A = a ) . In the following, we show that this is equivalent to the definition we use in our paper.

Lemma 2. The following two statements are equivalent:

There exists an unobserved confounder U that satisfies Y 1 , Y 0 ⊥ A | X , U so that it holds that s -≤ P ( u | x ,a ) P ( u | x ) ≤ s + .

It holds that s -≤ P ( Y 1 ,Y 0 | x ,a ) P ( Y 1 ,Y 0 | x ) ≤ s + .

Proof. The second statement follows directly from the first one by defining U = ( Y 1 , Y 0 ) . For the other direction, we proceed via proof by contradiction. Assume there exists a pair ( Y 1 , Y 0 ) that violates the second statement, say w.l.o.g. P ( Y 1 ,Y 0 | x ,a ) P ( Y 1 ,Y 0 | x ) > s + . We can use the independence condition from the first statement to write

Furthermore, we have that

It follows that

Hence, there exists a u such that s + < P ( Y 1 ,Y 0 | x , u ) P ( u | x ,a ) P ( Y 1 ,Y 0 | x , u ) P ( u | x ) = P ( u | x ,a ) P ( u | x ) , which is a contradiction to the sensitivity constraint of the first statement.

C.2 Continuous marginal sensitivity model (CMSM):

For continuous treatments A ∈ R d , the continuous marginal sensitivity model (CMSM) [32] is defined via

With the same arguments as in Eq. (40), it follows that the CMSM is a weighted GMSM with weight function q ( a , x ) = 0 .

C.3 Longitudinal marginal sensitivity model (LMSM):

For longitudinal settings with time-varying observed confounders X = ¯ X T = ( X 1 , . . . , X T ) , unobserved confounders U = ¯ U T = ( U 1 , . . . , U T ) , treatments A = ¯ A T = ( A 1 , . . . , A T ) , we define the longitudinal marginal sensitivity model (LMSM) via

It holds that

where ( ∗ ) follows by applying Bayes' theorem on P (¯ u t | ¯ x T , ¯ a t ) . Hence, the LMSM is a weighted GMSM with weight function q (¯ a T , ¯ x T ) = 0 . Note that we can also define LMSMs with different weight functions via

D Bounds for average causal effects and differences

Here, we show that we can use our sharp bounds to obtain sharp bounds for causal effect averages and differences. We state the results for the upper bound

All definitions and bounds for the lower bound Q -( x , ¯ a ℓ +1 , S ) can be obtained by swapping the signs.

We are interested in the sharp upper bound for the average causal effect

and the sharp upper bound for the causal effect difference

Lemma 3. We can compute Q + avg (¯ a ℓ +1 , S ) and Q + diff ( x , ¯ a (1) ℓ +1 , ¯ a (2) ℓ +1 , S ) from our sharp bounds Q + ( x , ¯ a ℓ +1 , S ) and Q -( x , ¯ a ℓ +1 , S ) via

and

Proof. The result for Q + avg (¯ a ℓ +1 , S ) follows directly from interchanging the supremum and integral. For Q + diff ( x , ¯ a (1) ℓ +1 , ¯ a (2) ℓ +1 , S ) , we note that

To show the equality in Eq. (56), we show that, for each pair of SCMs M 1 , M 2 ∈ C ( S ) , we can find an SCM M∈C ( S ) such that

We can assume w.l.o.g. that all M∈C ( S ) induce the same distributions P U W | x on the confounding space. We denote the functional assignments of M 1 and M 2 as f W M 1 ( x , m W , a , u W ) and f W M 2 ( x , m W , a , u W ) . We can now define a functional assignment f W M ( x , m W , a , u W ) for M so that

which implies Eq. (58).

E Compatibility of SCMs with sensitivity models

Here, we provide details regarding our class C ( S ) of SCMs that are compatible with a sensitivity model S from Def. 2.

Definition 6 (Compatibility) . Let P V denote the distribution of the observed variables V = ( X , A , ¯ M ℓ , Y ) and let S be a sensitivity model with unobserved confounders U and a family of joint distributions P . Let M be an SCM with observed variables V M = V and unobserved variables U M ⊇ U that contain the unobserved confounders U from the sensitivity model but also potential exogenous noise. Then, M is compatible with S , if the following three conditions are satisfied:

Graph compatibility: the induced causal graph G M by M on V ∪ U must coincide with our assumed causal structure (see Sec. 3.1).

Latent unconfoundedness: P ( w | x , m W , do ( A = a )) = ∫ P ( w | x , u W , m W , a ) P ( u W | x , m W ) d u W for all x ∈ X , m W ∈ supp ( M W ) , a ∈ A , and W ∈ { M 1 , . . . , M ℓ , Y } .

Sensitivity constraints: P V ∪ U M ∈ P , where P V ∪ U M denotes the distribution on V ∪ U induced by M .

In Definition 6, the latent unconfoundedness condition ensures that all interventional densities P ( w | x , m W , do ( A = a )) are point-identified under access to the unobserved confounders. In particular, this ensures that the SCM M can not contain additional unobserved confounders (e.g., between mediators and outcome). Similar assumptions are standard in the causal sensitivity literature [18].

F Importance sampling estimators for finite sample bounds

In this section, we derive estimators for the outcome bound D ( P Y + ( · | x , ¯ m ℓ , a ) ) for continuous Y ∈ R . We assume that we have already obtained an estimator ˆ P Y ( · | x , ¯ m ℓ , a ) of the observational distribution P Y ( · | x , ¯ m ℓ , a ) , and that we are able to sample ( y i ) k i =1 ∼ ˆ P Y ( · | x , ¯ m ℓ , a ) (see Appendix G for implementation details). Note that the outcome bound D ( P Y + ( · | x , ¯ m ℓ , a ) ) depends on the shifted distribution P Y + ( · | x , ¯ m ℓ , a ) and not on the observational distribution P Y ( · | x , ¯ m ℓ , a ) . Hence, we use an importance sampling approach to derive our estimators, which we outline in the following for the expectation functional and distributional effects. We denote the CDFs corresponding to P Y + ( · | x , ¯ m ℓ , a ) and P Y ( · | x , ¯ m ℓ , a ) by F P Y | x , ¯ m ℓ , a + and F P Y | x , ¯ m ℓ , a , respectively.

Expectation functional: For the expectation functional, we can rewrite the outcome bound as

to obtain the consistent estimator

where ( y i ) k i =1 ∼ ˆ P Y ( · | x , ¯ m ℓ , a ) are sampled from the estimated observational distribution. This corresponds to Eq. (8) in the main paper.

Computational complexity: Given trained models and a sample ( y i ) k i =1 ∼ ˆ P Y ( · | x , ¯ m ℓ , a ) , our estimator from Eq. (63) has a complexity of O ( k ) as it only involves summing and quantile computation. To set this in relation to existing work, Algorithm 1 of Jesson et al. [32] has a complexity of O ( kn ) , where n is a number of grid search points. This is demonstrated in Table 1, where we choose k = n = 5000 .

Distributional effects: We now derive estimators for distributional effects, i.e., for quantile functionals D of the form

with α ∈ (0 , 1) . We again use an importance sampling approach and rewrite

Hence, we can sample ( y i ) k i =1 ∼ ˆ P Y ( · | x , ¯ m ℓ , a ) and obtain the consistent estimator

where

G Implementation and hyperparameter tuning details

All our experimental settings feature a continuous outcome Y ∈ R and (optionally) discrete mediators M i ∈ N . Hence, we need to estimate the conditional outcome density P Y ( · | x , m , a ) and conditional probability mass functions P M i ( · | x , ¯ m i -1 , a ) in order to estimate our bounds with Eq. (63), Eq. (68), and Algorithm 1 from the main paper.

Conditional outcome density: We use conditional normalizing flows (CNFs) [73] for estimating the conditional density P Y ( · | x , ¯ m ℓ , a ) . Normalizing flows (NFs) model a distribution P Y of a target variable Y by transforming a simple base distribution P U (e.g., standard normal) of a latent variable U through an invertible transformation Y = f θ ( U ) , where θ denotes learnable parameters [58]. In order to estimate the conditional density P Y ( · | x , ¯ m ℓ , a ) , we leverage CNFs, that is, we define the parameters θ as an output of a hyper network θ = g η ( x , ¯ m ℓ , a ) with learnable parameters η . Given a sample { x i , ¯ m ℓ,i , a i , y i } n i =1 , we learn η by maximizing the log-likelihood

where f g η ( x i , ¯ m ℓ,i , a i ) # P U ( y i ) denotes the (push-forward) density induced by f g η ( x i , ¯ m ℓ,i , a i ) on R and ( ∗ ) follows from the change-of-variables theorem for invertible transformations.

In our implementation, we use neural spline flows. That is, we model the invertible transformation f θ via a spline flow as described in [21]. We use a feed-forward neural network for the hypernetwork g η ( x , ¯ m ℓ , a ) with 2 hidden layers, ReLU activation functions, and linear output. We set the latent distribution P U to a standard normal distribution N (0 , 1) . For training, we use the Adam optimizer [41].

Conditional probability mass functions: The estimation of the conditional probability mass function P M i ( · | x , ¯ m i -1 , a ) is a standard (multi-class) classification problem. We use feed-forward neural networks with 3 hidden layers, ReLU activation functions, and softmax output. For training, we minimize the standard cross-entropy loss by using the Adam optimizer [41]. We use the same approach to estimate the propensity scores P A ( · | x ) for discrete treatments A .

Hyperparameter tuning:

We perform hyperparameter tuning for our experiments on synthetic data using grid search on a validation set. The tunable parameters and search ranges are shown in Table 2. For reproducibility purposes, we report the selected hyperparameters as .yaml files. 3

H Experiments using synthetic data

Here we provide details regarding our experiments using synthetic data. This includes data generation, obtaining oracle sensitivity parameters, and details regarding experimental evaluation.

Overall data-generating process: We first describe the overall data-generating process which we use as a basis to generate data for all settings (i)-(iii) and binary/continuous treatments. We construct an SCM following the causal graph in Fig. 1 (right) from the main paper. We have an observed confounder X ∈ R , a (binary or continuous) treatment A , two binary mediators M 1 and M 2 , and a continuous outcome Y ∈ R . Furthermore, we consider three unobserved confounders: (i) U M 1 confounding the A -M 1 relationship, (ii) U M 2 confounding the A -M 2 relationship, and (iii) U Y confounding the A -Y relationship. Our data-generating process is inspired by synthetic experiments from previous works on causal sensitivity analysis [31, 35]. We start the data-generating process by sampling

Depending on the setting, we either generate binary treatments A ∈ { 0 , 1 } via

or continuous treatments A ∈ (0 , 1) via

where γ M 1 , γ M 2 , and γ Y are parameters controlling the strength of unobserved confounding. We then generate the mediators and outcome via functional assignments

and

where ϵ M 1 , ϵ M 2 , ϵ Y ∼ N (0 , 1) are standard normal distributed noise variables. The functional assignments are defined as

for M 2 , and

for Y , where ρ M 1 , ρ M 2 , and ρ Y are parameters that control the noise level.

Settings (i)-(iii): We define the settings (i)-(iii) in Sec. 5 via specific values of the confounding parameters γ M 1 , γ M 2 , and γ Y , and the noise parameters ρ M 1 , ρ M 2 , and ρ Y (see Table 3). Note that the settings are defined to mimic the causal graphs in Fig. 1 from the main paper. For example, the only unobserved confounder in setting (i) is U Y , which means that we can ignore the mediators and use our data to evaluate our bounds for settings without mediators.

with

α = β = 2 + x + ✶ ( x < 0) ( γ M 1 ( u M 1 -0 . 5) + γ M 2 ( u M 2 -0 . 5) + γ Y ( u Y -0 . 5)) . (93) Hence, unobserved confounding only affects individuals with x < 0 . We then compare our bounds under the CMSM with our bounds under a weighted CMSM (Def. 4) with weight function q Y ( x ) = ( x > 0) .

✶ We also provide results for the bounds from Jesson et al. [32] under the CMSM. We implemented the grid search algorithm from Jesson et al. [32] and used 5 , 000 samples for the search space. For a fair comparison, we also used 5 , 000 samples for our importance sampling estimators. Note that the method from Jesson et al. [32] requires estimation of both the conditional outcome density P Y ( · | x , a ) and the conditional expectation E [ Y | x , a ] . For P Y ( · | x , a ) , we use the same (normalizing flowbased) estimator as for our bounds. For E [ Y | x , a ] , we train a separate feed-forward neural network with linear output activation for continuous outcomes. Implementation and hyperparameter tuning are done the same way as described in Appendix G for the feed-forward neural networks.

Obtaining Γ ∗ W : We provide details regarding our approach to obtain oracle sensitivity parameters Γ ∗ W for all W ∈ { M 1 , M 2 , Y } . By sampling from our previously defined SCM, we can obtain Monte Carlo estimates of the GMSM density ratio

for all u W ∈ { 0 , 1 } , a , and x , where ( ∗ ) follows from Bayes' theorem. We then define

For binary treatment settings, we define parameters Γ + W = Γ + W ( x, a ) and Γ -W = Γ -W ( x, a ) that attain the density ratio bounds in the MSM from Eq. (39), i.e.

For continuous treatment settings, we define Γ + W and Γ -W as the sensitivity parameters that attain the density ratio bounds in the CMSM from Eq. (44), i.e.

Finally, we define Γ ∗ W as the parameter corresponding to the maximum possible violation of unconfoundedness, i.e.,

By definition of Γ ∗ W , our bounds should contain the oracle causal effect whenever we choose sensitivity parameters Γ W ≥ Γ ∗ W for all W ∈ { M 1 , M 2 , Y } .

Weighted GMSM experiment (Table 1): For our experiment in Table 1, we modify the treatment assignment from Eq. (74) in setting (i) to

I Experiment using real-world data

Data: We consider a setting from the COVID-19 pandemic where mobility in Switzerland (captured through telephone movement) was monitored to obtain a leading predictor of case growth. In total, ∼ 1 , 5 billion trips were monitored from 10 February through 26 April 2020. All data are recorded across 26 different states (cantons). For our analysis, we use an aggregated, de-identified, and pre-processed version of the data provided by Persson, Parie, and Feuerriegel [57]. The preprocessed data is publically available at https://github.com/jopersson/covid19-mobility/blob/main/Data. The code for our analysis is available at https://github.com/DennisFrauen/SharpCausalSensitivity.

We consider a binary treatment A in the form of a stay-at-home order, which bans gatherings with more than 5 people. We encode mobility as a single binary mediator M , which is 1 if the total number of trips on a specific day is larger than the median number of trips during the entire time horizon, and 0 otherwise. Our outcome is the 10-day-ahead case growth. We include the following observed variables as confounders X : the canton code (swiss member state at a subnational level), the canton population, and whether the weekday is a Monday or not. After removing the first 10 recorded days for each canton (due to spillover effects from other countries) and rows with missing values, we obtain a dataset with n = 3276 observations.

Analysis: We perform a causal sensitivity analysis for the natural directed effect (NDE) of the stay-at-home order A on the case growth Y . That is, we are interested in the part of the causal effect of A on Y that is not explained by the path via M (i.e., through the change in mobility). The NDE in an SCM M is defined as

Fig. 5 (main paper) shows causal sensitivity analysis for violations of the unconfoundedness between treatment A and mediator M . Hence, we consider a GMSM for binary treatments with sensitivity parameters Γ M and Γ Y = 0 . For each Γ M , we estimate our bounds for the expectation functional and the treatment combinations ¯ a = (0 , 1) and ¯ a = (0 , 0) . We then obtain bounds for the NDE as described in Appendix D.

J Additional experimental results

Here, we provide additional experimental results on synthetic data that extend the results from Sec. 5 in the main paper. We provide (i) results for additional treatment combinations and (ii) results for distributional effects. We follow the same experimental setup described in Sec. 5 (main paper) and Appendix. H.

J.1 Additional treatment combinations

Results for additional treatment combinations are shown in Fig. 6 (binary treatment settings) and Fig. 7 (continuous treatment settings). The results are similar to those in Sec. 5 in the main paper and empirically confirm the validity of our bounds. Hence, our results remain valid independently of the choice of treatment combination.





















M

1

=

Y





















M

1

=

Y



























M

1

=

M

2

=

Y







































M

1



Y



M

1



Y



M

1



Y



M

1



Y











































M

1



M

2



Y



M

1



M

2



Y



M

1



M

2



Y



M

1



M

2



Y









































M

1



M

2



Y



M

1



M

2



Y



M

1



M

2



Y



M

1



M

2



Y



J.2 Distributional effects

We also provide results for distributional effects, that is, we choose the α -quantile functional D ( P Y + ( · | x , ¯ m ℓ , a ) ) = F -1 P Y | x , ¯ m ℓ , a + ( α ) . Here, we consider three quantiles with α = 0 . 7 , α = 0 . 5

(median), and α = 0 . 3 . We use our importance sampling estimator derived in Appendix. F (Eq. (68)) to estimate our bounds. The results are shown in Fig. 8 (binary treatment) and Fig. 9 (continuous treatment) for settings (i)-(iii) from Fig. 1 in the main paper. Again, our bounds cover the underlying oracle effect in regions where the chosen sensitivity parameters Γ W are larger than the oracle sensitivity parameters Γ ∗ W . This also confirms empirically the validity of our bounds for distributional effects.

J.3 Semi-synthetic data

We provide additional results for the semi-synthetic IHDP data with unobserved confounding from Jesson et al. [31]. Here, we demonstrate the effectiveness of our bounds for distributional effects for decision-making. We follow Jesson et al. [31] and evaluate our bounds by measuring the error rate of associated deferral policies, which defer a prespecified fraction of test samples with the largest uncertainty to an expert. We mimic a high-stakes decision-making problem where a wrong decision to prescribe treatment is much worse than a wrong decision not to prescribe treatment. To do so, we measure performance with a weighted error rate, that penalizes wrong























Y

























M

1

Y























M

1

M

2

Y









































Y



Y



Y



Y









































M

1



Y



M

1



Y



M

1



Y



M

1



Y









































M

1



M

2



Y



M

1



M

2



Y



M

1



M

2



Y



M

1



M

2



Y



treatment decisions twenty times heavier than wrong decisions not to treat. We then compare three different deferral policies: (i) the (standard) policy based on the expectation which treats whenever E [ Y | X = x , A = 1] > E [ Y | X = x , A = 0] , (ii) a more conservative quantile policy that treats whenever Q q [ Y | X = x , A = 1] > Q 1 -q [ Y | X = x , A = 0] with q = 0 . 4 ( Q q denotes the q-quantile of P ( Y | X = x , A = i )) , and (iii) the same quantile policy with q = 0 . 2 . Note that our bounds for policy (i) coincide with sharp bounds for binary CATE from the literature [18]. As done in [31], we report means and standard deviations of 400 random initializations of the IHDP data. The results (Fig. 10) show that the conservative quantile policies improve over the expectation policy.

























Y





















M

1

=

Y



























M

1

=

M

2

=

Y



































=0.3



=0.5



=0.7

Y



Y



Y



Y





































=0.3



=0.5



=0.7

M

1



Y



M

1



Y



M

1



Y



M

1



Y











































=0.3



=0.5



=0.7

M

1



M

2



Y



M

1



M

2



Y



M

1



M

2



Y



M

1



M

2



Y











































=0.3



=0.5



=0.7

Y



Y



Y



Y





































=0.3



=0.5



=0.7

M

1



Y



M

1



Y



M

1



Y



M

1



Y









































=0.3



=0.5



=0.7

M

1



M

2



Y



M

1



M

2



Y



M

1



M

2



Y



M

1



M

2



Y











































=0.3



=0.5



=0.7

Y



Y



Y



Y







































=0.3



=0.5



=0.7

M

1



Y



M

1



Y



M

1



Y



M

1



Y





































=0.3



=0.5



=0.7

M

1



M

2



Y



M

1



M

2



Y



M

1



M

2



Y



M

1



M

2



Y

























Y























M

1

Y

























M

1

M

2

Y



































=0.3



=0.5



=0.7

Y



Y



Y



Y





































=0.3



=0.5



=0.7

M

1



Y



M

1



Y



M

1



Y



M

1



Y













































=0.3



=0.5



=0.7

M

1



M

2



Y



M

1



M

2



Y



M

1



M

2



Y



M

1



M

2



Y











































=0.3



=0.5



=0.7

Y



Y



Y



Y











































=0.3



=0.5



=0.7

M

1



Y



M

1



Y



M

1



Y



M

1



Y











































=0.3



=0.5



=0.7

M

1



M

2



Y



M

1



M

2



Y



M

1



M

2



Y



M

1



M

2



Y





































=0.3



=0.5



=0.7

Y



Y



Y



Y













































=0.3



=0.5



=0.7

M

1



Y



M

1



Y



M

1



Y



M

1



Y











































=0.3



=0.5



=0.7

M

1



M

2



Y



M

1



M

2



Y



M

1



M

2



Y



M

1



M

2



Y





































q

=0.4



q

=0.2

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors generalize a class of causal sensitivity models that includes the traditional MSM, the continuous-treatment CMSM, and the longitudinal (time-varying treatment) LMSM. They show how to compute sharp bounds for the causal estimands by taking inspiration from recent work. Their general framework also allows mediation analysis. They provide an algorithm for computing these bounds.\n\nSTRENGTHS:\nThe method is solid and explained well. Figure 2 is nice. Mediation analysis is an important contribution to causal sensitivity models. A broader understanding of sensitivity models is always valuable and the effort to generalize is a good one.\n\nWEAKNESSES:\nMy main concern is that this generalization that unifies the MSM, CMSM, and LMSM is not very useful. The weighting function seems a bit contrived. It is necessary because the CMSM and LMSM do not use the nominal propensity at all, but the MSM does. Isn't it strange to ignore the nominal propensity and give the same bounds to all potential outcome distributions? Shouldn't the observed confounding inform the unobserved confounding?\n\nA recent alternative to the CMSM is the $\\delta$MSM [arXiv:2204.11206] that takes a different approach and appears to perform better. The authors could discuss alternative models like this one or at least keep them in mind when considering general classes of sensitivity models.\n\nThe clever approach to sharp partial identification is not novel [see for instance arXiv:2304.10577]. The benefit of the mediation analysis is not really made clear in the results of this submission.\n\n\nIn terms of results, the authors employ a purely synthetic benchmark. The real-world data are interesting but they lack a ground truth. The authors could include some well-known semi-synthetic benchmarks like IHDP and induce hidden confounding by hiding some of the observed confounders.\n\nThe authors do not compare with previous methods in their benchmark except for Table 1, where they use a custom weighting function to beat the older CMSM algorithm. That is not very convincing. Also Table 1 should at least be discussed more. The results for the weighted CMSM seem conflicting: tighter bounds but worse coverage?\n\nOn a lesser note, the technical details are a bit dense.\n\nQUESTIONS:\nCould you support your reasoning for why this weighting function is a natural interpretation of the more specific sensitivity models? How is it helpful and how can I use it in newer settings? Why should it be set to zero in some cases?\n\nLIMITATIONS:\nLimitations are discussed a bit but they do not address potential societal impacts. It is debatable if that is necessary for this kind of work, but I think pitfalls of these kinds of sensitivity analyses should be discussed.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-19"}

comments: SUMMARY:
This paper is about sensitivity analysis (SA) of causal queries in SCMs. In practice, given a causal query and a set of models, the goal is to compute a query's lower and upper bounds. The authors first derive a class of models to be used for SA and show how this extends existing models. An algorithm to obtain the bounds is such cases is derived.

(After the rebuttal, I decided to raise the rating of the paper from 4 to 6)

STRENGTHS:
The technical results are sound and non-trivial.
The experiments show good bounds obtained in this way.

WEAKNESSES:
The literature on partially identifiable queries is ignored. In particular existing techniques for bounding such queries are not considered.

QUESTIONS:
Would it be possible to compare the present method against algorithms for the bounding of non-identifiable queries? E.g. Zhang and Bareinboim, Duarte et al., and Zaffalon et al. worked in this direction in the last two years. I think the sensitivity analysis the authors consider is a heuristic approach to the same problem. If this is true, not having a comparison against these methods is a serious issue.

I also believe that the authors would take advantage of the literature about so-called "imprecise probabilities" and "credal networks" as these models implement the kind of sensitivity analysis of interest for the authors. In particular, the paper "Structural Causal Models Are (Solvable by) Credal Networks" might be a helpful reading.

LIMITATIONS:
I don't see specific issues in this direction.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-05

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a unified framework for causal sensiitivity analysis under unobserved confounding that generalizes the Marginal Sensitivity Model (MSM). They derive sharp bounds for a diverse set of causal effects such as the CATE, mediation and path analysis effects, and distributional effects. The framework is applicable to discrete, continuous, and time-varying interventions. They offer, to my knowledge, a novel interpretation of the marginal sensitivity model via SCM. They show that in the case of binary treatments, their derived bounds  coincide with the optimality result of Dorn and Guo 2023. They provide a closed form solution to and algorithm to estimate the bounds and show empircally that it improves over line search methods (a formal complexity analysis would be nice).\n\nSTRENGTHS:\nThis paper makes several novel contributions to an active area of study in causal machine learning, which are listed above in the summary. They provide theoretical and experimental evidence supporting their claims. They do a good job presenting and comparing to the related work. An exceptionally well writen and organized paper given the complexity of the subject matter.\n\nWEAKNESSES:\n## I have one primary comment. \n\nI think there may be a step missing in the special cases proofs of Appendix C. I appreciate how the authors have utilized SCM to define the GMSM. But, in defining the  MSM, CMSM, and LMSM in terms of hidden confounders $u$, it seems that there is a step missing from how these models are originally defined. Namely, they are defined with respect to potential-outcomes / counterfactuals $Y_{t}$ and the conditional independence relation: $Y_t \\perp T \\mid X$. For example, it's not obvious to me how you move from $P(a \\mid x, y_t)$, to $P(a \\mid x, u)$. Given that you show equivalence in terms of $P(a \\mid x, u)$, I think it is important to be explicit here. If this is resolved, and I admit that this could be completely trivial and I just don't see it, I would happily increase my score. If it cannot be resolved, I would suggest removing these claims and my score would remain the same.\n\n## I have a few minor comments.\n\nFirst, the last two paragraphs of the introduction can be streamlined as the contributions paragraph essentially reiterates the points of the paragraph starting on line 45. I like both styles, with slight preference for the contribution format.\n\nThere is a recent paper proposing a marginal sensitivity analysis for continuous treatments that could be added to the related works.\n\nLine 70 \"... when while ...\" seems to be a typo\n\nQUESTIONS:\nHow does one show that  $P(a \\mid x, y_t) = P(a \\mid x, u)$?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

comments: SUMMARY:
The authors study the problem of bounding a given causal effect. To this end, they propose a generalized marginal sensitivity model (GMSM) that is applicable to multiple discrete, continuous , and time-varying treatments. They also present a new interpretation of the partial identification.

STRENGTHS:
The proposed GMSM model generalizes the previous models and leads to sharp bound for certain causal effects with certain causal graphs. 

The bounds depend on observed variables and thus can be estimated. 

Except some minor ambiguities (see below), the paper is well-written.

WEAKNESSES:
Although the proposed model generalized the previous models but the presented theoretical results hold under specific  graphical constraints e.g., no confounder between M and outcome, no hidden confounders between X and $\{A,M,Y\}$. 

The other weakness about the results is its generalizability to arbitrary causal graphs. Based on the presented proofs in the appendix, it is not clear how this results can be generalized by relaxing the assumptions. 

There is ambiguity about the notation of U. Does $U_Y$ denote the unobserved exogenous variable for Y in the definition of SCM or is it a hidden confounder between Y and A? Is it possible to have hidden confounders among the mediators (e.g., $M_1$ and $M_2$)?

The explanation below (3) says  “If U_w has no effect on A, Eq. (3) holds with $s^-_W (a, x) = S^+_W (a, x) = 1$”. But it seems that (3) encodes the effect of A on $U_W$.!

In (35) in the Appendix, what is U exactly?  Does the setting imply that u and x are independent, i.e., $p(u|x)=p(u)$?

QUESTIONS:
See above.

LIMITATIONS:
The authors addressed the limitations.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-06