Title: Partial Counterfactual Identification of Continuous Outcomes with a Curvature Sensitivity Model
Venue: NeurIPS 2023
Accepted: yes
Keywords: causal inference

counterfactual inference

partial identification

sensitivity model

normalizing flows

causal machine learning
Abstract: Counterfactual inference aims to answer retrospective "what if" questions and thus belongs to the most fine-grained type of inference in Pearl's causality ladder. Existing methods for counterfactual inference with continuous outcomes aim at point identification and thus make strong and unnatural assumptions about the underlying structural causal model. In this paper, we relax these assumptions and aim at partial counterfactual identification of continuous outcomes, i.e., when the counterfactual query resides in an ignorance interval with informative bounds. We prove that, in general, the ignorance interval of the counterfactual queries has non-informative bounds, already when functions of structural causal models are continuously differentiable. As a remedy, we propose a novel sensitivity model called Curvature Sensitivity Model. This allows us to obtain informative bounds by bounding the curvature of level sets of the functions. We further show that existing point counterfactual identification methods are special cases of our Curvature Sensitivity Model when the bound of the curvature is set to zero. We then propose an implementation of our Curvature Sensitivity Model in the form of a novel deep generative model, which we call Augmented Pseudo-Invertible Decoder. Our implementation employs (i) residual normalizing flows with (ii) variational augmentations. We empirically demonstrate the effectiveness of our Augmented Pseudo-Invertible Decoder. To the best of our knowledge, ours is the first partial identification model for Markovian structural causal models with continuous outcomes.

Partial Counterfactual Identification of Continuous Outcomes with a Curvature Sensitivity Model

Abstract

Counterfactual inference aims to answer retrospective 'what if' questions and thus belongs to the most fine-grained type of inference in Pearl's causality ladder. Existing methods for counterfactual inference with continuous outcomes aim at point identification and thus make strong and unnatural assumptions about the underlying structural causal model. In this paper, we relax these assumptions and aim at partial counterfactual identification of continuous outcomes, i.e., when the counterfactual query resides in an ignorance interval with informative bounds. We prove that, in general, the ignorance interval of the counterfactual queries has non-informative bounds, already when functions of structural causal models are continuously differentiable. As a remedy, we propose a novel sensitivity model called Curvature Sensitivity Model . This allows us to obtain informative bounds by bounding the curvature of level sets of the functions. We further show that existing point counterfactual identification methods are special cases of our Curvature Sensitivity Model when the bound of the curvature is set to zero. We then propose an implementation of our Curvature Sensitivity Model in the form of a novel deep generative model, which we call Augmented PseudoInvertible Decoder . Our implementation employs (i) residual normalizing flows with (ii) variational augmentations. We empirically demonstrate the effectiveness of our Augmented Pseudo-Invertible Decoder . To the best of our knowledge, ours is the first partial identification model for Markovian structural causal models with continuous outcomes.

1 Introduction

Counterfactual inference aims to answer retrospective 'what if' questions. Examples are: Would a patient's recovery have been faster, had a doctor applied a different treatment? Would my salary be higher, had I studied at a different college? Counterfactual inference is widely used in datadriven decision-making, such as root cause analysis [18, 139], recommender systems [16, 36, 83], responsibility attribution [50, 73, 77], and personalized medicine [84, 138]. Counterfactual inference is also relevant for various machine learning tasks such as safe policy search [108], reinforcement learning [19, 38, 63, 86, 91], algorithmic fairness [76, 104, 146], and explainability [42, 44, 64, 65, 75].

Counterfactual queries are located at the top of Pearl's ladder of causation [8, 51, 101], i. e., at the third layer L 3 of causation [8] (see Fig. 1, right). Counterfactual queries are challenging as they do reasoning in both the actual world and a hypothetical one where variables are set to different values than they have in reality.

State-of-the-art methods for counterfactual inference typically aim at point identification . These works fall into two streams. (1) The first stream [21, 29, 31, 70, 84, 100, 113, 114, 117, 118,

Counterfactual

Observational

Interventional

Bayesian network

Parallel worlds network

Ladder of causation

d-separation in parallel

worlds network

Markovianity

conditional

expectation

expected

potential outcome

expected counterfactual

outcome of [un]treated

expected counterfactual

outcome

expected potential

outcome of [un]treated

Causal diagrams

̸

134, 135, 140] makes no explicit assumptions besides assuming a structural causal model (SCM) with Markovianity (i. e., independence of the latent noise) and thus gives estimates that can be invalid . However, additional assumptions are needed in counterfactual inference of layer L 3 to provide identifiability guarantees [94, 137]. (2) The second stream [4, 32, 59, 66, 93, 96, 123, 148] provides such identifiability guarantees but makes strong assumptions that are unnatural or unrealistic . Formally, the work by Nasr-Esfahany et al. [93] describes bijective generation mechanisms (BGMs), where, in addition to the original Markovianity of SCMs, the functions in the underlying SCMs must be monotonous (strictly increasing or decreasing) with respect to the latent noise. The latter assumption effectively sets the dimensionality of the latent noise to the same dimensionality as the observed (endogenous) variables. However, this is highly unrealistic in real-world settings and is often in violation of domain knowledge. For example, cancer is caused by multiple latent sources of noise (e.g., genes, nutrition, lifestyle, hazardous exposures, and other environmental factors).

In this paper, we depart from point identification for the sake of more general assumptions about both the functions in the SCMs and the latent noise. Instead, we aim at partial counterfactual identification of continuous outcomes. Rather than inferring a point estimation expression for the counterfactual query, we are interested in inferring a whole ignorance interval with informative bounds . Informative bounds mean that the ignorance interval is a strict subset of the support of the distribution. The ignorance interval thus contains all possible values of the counterfactual query for SCMs that are consistent with the assumptions and available data. Partial identification is still very useful for decision-making, e. g., when the ignorance interval for a treatment effect is fully below or above zero.

We focus on a Markovian SCM with two observed variables, namely, a binary treatment and a continuous outcome. We consider a causal diagram as in Fig. 1 (left). We then analyze the expected counterfactual outcome of [un]treated abbreviated by ECOU [ECOT]. This query is non-trivial in the sense that it can not be simplified to a L 1 / L 2 layer, as it requires knowledge about the functions in SCM, but can still be inferred by means of a 3-step procedure of abduction-action-prediction [102]. ECOU [ECOT] can be seen as a continuous version of counterfactual probabilities [6] and allows to answer a retrospective question about the necessity of interventions: what would have been the expected counterfactual outcome for some treatment, considering knowledge about both the factual treatment and the factual outcome?

In our paper, we leverage geometric measure theory and differential topology to prove that, in general, the ignorance interval of ECOU [ECOT] has non-informative bounds. We show theoretically that this happens immediately when we relax the assumptions of (i) that the latent noise and the outcome have the same dimensionality and (ii) that the functions in the SCMs are monotonous (and assume they are continuously differentiable). As a remedy, we propose a novel Curvature Sensitivity Model (CSM), in which we bound the curvature of the level sets of the functions and thus yield informative bounds. We further show that we obtain the BGMs from [93] as a special case when setting the curvature to zero. Likewise, we yield non-informative bounds when setting it to infinity. Therefore, our CSM provides a sufficient condition for the partial counterfactual identification of the continuous outcomes with informative bounds.

We develop an instantiation of CSM in the form of a novel deep generative model, which we call Augmented Pseudo-Invertible Decoder (APID). Our APID uses (i) residual normalizing flows with (ii) variational augmentations to perform the task of partial counterfactual inference. Specifically, our APID allows us to (1) fit the observational/interventional data, (2) perform abduction-actionprediction in a differentiable fashion, and (3) bound the curvature of the SCM functions, thus yielding informative bounds for the whole ignorance interval. Finally, we demonstrate its effectiveness across several numerical experiments.

Overall, our main contributions are following: 1

We prove that the expected counterfactual outcome of [un]treated has non-informative bounds in the class of continuously differentiable functions of SCMs.

We propose a novel Curvature Sensitivity Model (CSM) to obtain informative bounds. Our CSM is the first sensitivity model for the partial counterfactual identification of continuous outcomes in Markovian SCMs.

We introduce a novel deep generative model called Augmented Pseudo-Invertible Decoder (APID) to perform partial counterfactual inference under our CSM. We further validate it numerically.

2 Related Work

We briefly summarize prior works on (1) point and (2) partial counterfactual identification below but emphasize that none of them can be straightforwardly extended to our setting. We provide an extended literature overview in Appendix A.

(1) Point counterfactual identification has been recently addressed through neural methods [21, 29, 31, 70, 84, 100, 113, 114, 117, 118, 134, 135, 140] but without identifiability results.

To ensure identifiability, prior works usually make use of (i) symbolic identifiability methods or (ii) put restrictive assumptions on the model class, if (i) led to non-identifiability (see Fig. 2).

symbolic

identifiability?

Yes

No

Point identification with

probabilistic

expressions (e.g., ETT,

path-specific effects)

Assumption of

monotonisity?

Counterfactual query with continuous outcome

Yes

Point identification with

functional class restrictions

(e.g., BGMs, transport-based

counterfactuals)

No

Partial identification

(our

Curvature

Sensitivity Model

)

(i) Symbolic (non-parametric) identifiability methods [27, 121] aim to provide a symbolic probabilistic expression suitable for point identification if a counterfactual query can be expressed via lower-layer information only. Examples of the latter include the effect of treatment of the treated (ETT) [119] and path-specific effects [122, 146]. However, these are not suited for partial counterfactual identification.

Alternatively, identifiability can be achieved by (ii) making restrictive but unrealistic assumptions about the SCMs or the data generating mechanism [4, 32, 59, 66, 93, 96, 123, 148]. A notable example is the BGMs [93], which assumes a Markovian SCM where the functions in the SCMs must be monotonous (strictly increasing or decreasing) with respect to the latent noise. The latter assumption effectively sets the dimensionality of the latent noise to the same dimensionality as the observed (endogenous) variables, yet this is unrealistic in medicine. As a remedy, we depart from point identification and instead aim at partial counterfactual identification, which allows us to relax the assumptions of BGMs.

Kilbertus et al. [69] build sensitivity models for unobserved confounding in semi-Markovian SCMs, but still assume a restricted functional class of SCMs, namely, an additive noise model [66, 123]. We, on the other hand, build a sensitivity model around the extended class of functions in the SCMs, which is non-trivial even in the Markovian SCMs.

(2) Partial counterfactual identification has been studied previously, but only either for (i) discrete SCMs [97, 103, 129, 137, 141, 142, 143, 147], or for (ii) specific counterfactual queries with informative bounds [2, 4, 37]. Yet, these (i) do not generalize to the continuous setting; and (ii) are specifically tailored for certain queries with informative bounds , such as the variance of the treatment effects, and, thus, are not applicable to our (non-informative) ECOU [ECOT].

Likewise, it is also not possible to extend partial interventional identification of continuous outcomes [35, 49, 60, 61, 126] from the L 2 to L 3 , unless it explicitly assumes an underlying SCM.

Research gap. To the best of our knowledge, we are the first to propose a sensitivity model for partial counterfactual identification of continuous outcomes in Markovian SCMs.

3 Partial Counterfactual Identification of Continuous Outcomes

In the following, we derive one of our main results: the ignorance interval of the ECOU [ECOT] has non-informative bounds if we relax the assumptions that (i) both the outcome and the latent noise are of the same dimensionality and that (ii) the functions in the SCMs are monotonous.

3.1 Preliminaries

Notation. Capital letters A,Y,U , denote random variables and small letters a, y, u their realizations from corresponding domains A , Y , U . Bold capital letters such as U = { U 1 , . . . , U n } denote finite sets of random variables. Further, P ( Y ) is an (observational) distribution of Y ; P ( Y | a ) = P ( Y | A = a ) is a conditional (observational) distribution; P ( Y A = a ) = P ( Y a ) an interventional distribution; and P ( Y A = a | A ′ = a ′ , Y ′ = y ′ ) = P ( Y a | a ′ , y ′ ) a counterfactual distribution. We use a superscript such as in P M to indicate distributions that are induced by the SCM M . We denote the conditional cumulative distribution function (CDF) of P ( Y | a ) by F a ( y ) . We use P ( Y = · ) to denote a density or probability mass function of Y and E ( Y ) = ∫ y d P ( y ) to refer to its expected value. Interventional and counterfactual densities or probability mass functions are defined accordingly.

A function is said to be in class C k , k ≥ 0 , if its k -th derivative exists and is continuous. Let ∥·∥ 2 denote the L 2 -norm, ∇ x f ( x ) a gradient of f ( x ) , and Hess x f ( x ) a Hessian matrix of f ( x ) . The pushforward distribution or pushforward measure is defined as a transfer of a (probability) measure P with a measurable function f , which we denote as f ♯ P .

SCMs. We follow the standard notation of SCMs as in [8, 13, 101, 137]. An SCM M is defined as a tuple ⟨ U , V , P ( U ) , F⟩ with latent (exogenous) noise variables U , observed (endogenous) variables V = { V 1 , . . . , V n } , a distribution of latent noise variables P ( U ) , and a collection of functions F = { f V 1 , . . . , f V n } . Each function is a measurable map from the corresponding domains of U V i ∪ Pa V i to V i , where U V i ⊆ U and Pa V i ⊆ V \ V i are parents of the observed variable V i . Therefore, the functions F induce a pushforward distribution of observed variables, i.e., P ( V ) = F ♯ P ( U ) . Each V i is thus deterministic (non-random), conditionally on its parents, i. e., v i ← f V i ( pa V i , u V i ) . Each SCM M induces an (augmented) causal diagram G ( M ) , which is assumed to be acyclic.

We provide a background on geometric measure theory and differential geometry in Appendix B.

3.2 Counterfactual Non-Identifiability

In the following, we relax the main assumption of bijective generation mechanisms (BGMs) [93], i. e., that all the functions in Markovian SCMs are monotonous (strictly increasing) with respect to the latent noise variable. To this end, we let the latent noise variables have arbitrary dimensionality and further consider functions to be of class C k . We then show that counterfactual distributions under this relaxation are non-identifiable from L 1 or L 2 data.

Definition 1 (Bivariate Markovian SCMs of class C k and d -dimension latent noise) . Let B ( C k , d ) denote the class of SCMs M = ⟨ U , V , P ( U ) , F⟩ with the following endogenous and latent noise variables: U = { U A ∈ { 0 , 1 } , U Y ∈ [0 , 1] d } and V = { A ∈ { 0 , 1 } , Y ∈ R } , for d ≥ 1 . The latent noise variables have the following distributions P ( U ) : U A ∼ Bern( p A ) , 0 < p A < 1 , and U Y ∼ Unif(0 , 1) d and are all mutually independent 2 . The functions are F = { f A ( U A ) , f Y ( A,U Y ) } and f Y ( a, · ) ∈ C k for u Y ∈ (0 , 1) d ∀ a ∈ { 0 , 1 } .

All SCMs in B ( C k , d ) induce similar causal diagrams G ( M ) , where only the number of latent noise variables d differs. Further, it follows from the above definition that BGMs are a special case of B ( C 1 , 1) , where the functions f Y ( a, · ) are monotonous (strictly increasing).

̸

Task: counterfactual inference. Counterfactual queries are at layer L 3 of the causality ladder and are defined as probabilistic expressions with random variables belonging to several worlds, which are in logical contradiction with each other [8, 27]. For instance, P M ( Y a = y, Y a ′ = y ) , a = a ′ for an SCM M from B ( C k , d ) . In this paper, we focus on the expected counterfactual outcome of [un]treated ECOU [ECOT], which we denote by Q M a ′ → a ( y ′ ) = E M ( Y a | a ′ , y ′ ) .

(](⌈(∫(∑(∏(∨(∮(⊎(∮(〈(⊔(∨(∑(〈(⊗(](⊕(∏(∮(∮(⊎(∮(∏(⊔(∮(](]

(⌊()(]((⊎(∮(⊔(⊕(⊔(∨(⊕(∫(∏

([()(]

⌊(〉(⊔(∑(⊔(∨(⊕(∫(∏

(]()(]

⌊(⊔(∨(⊕(∫(∏

(]((〉(∧(∮(⊎(∏(〈(∨(⊕(∫(∏(〈(⊗(](⊕(∏(∮(∮(⊎(∮(∏(⊔(∮(](]

(](/(∏(∨(∮(⊎ ∏(∮(∏ ∨(⊕ ∫ ∏(〈(⊗(](⊕(∏(∮(∮(⊎(∮(∏(⊔(∮

(]

(]

(]

(]

(]

(∖(∮(⊙(∮(∏(⊔

(∖(〈(∨(∮(∏(∨(](∧(⋃(〈(⊔(∮(](∫(∮(]

(](〈(∏(⊔

(](∫(⊕ ∨(⊙(](⊗(∮(∏(∮(⊗

(∧(∮(∨(∧(]

(⌉(∮(∏(∧(⊕(∨(⋂(](∫(∮(]

(](∫ ⊕ ∨(⊙

(∮(⋂(∑(⊕(⊔(⊕(∧(∨(〈(∏(∨(](⊙(⊎(⊕(⊔(](∫(∮(](⋃(∫(⊕(∏(∨(∧(]

(∨(⊙(∮(](∮(∑(∏(⊔(∨(⊕(∫(∏(]

For an SCM M from B ( C k , d ) , ECOU [ECOT] can be inferred by the following three steps. (1) The abduction step infers a posterior distribution of the latent noise variables, conditioned on the evidence, i.e., P ( U Y | a ′ , y ′ ) . This posterior distribution is defined on the level set of the factual function given by { u Y : f Y ( a ′ , u Y ) = y ′ } , i. e., all points in the latent noise space mapped to y ′ . (2) The action step alters the function for Y to f Y ( a, U Y ) . (3) The prediction step is done by a pushforward of the posterior distribution with the altered function, i.e. f Y ( a, · ) ♯ P ( U Y | a ′ , y ′ ) . Afterwards, the expectation of it is then evaluated.

The existence of counterfactual queries, which are non-identifiable with L 1 or L 2 data in Markovian SCMs, was previously shown in [137] (Ex. 8, App. D3) for discrete outcomes and in [13, 30] (Ex. D.7) for continuous outcomes. Here, we construct an important example to (1) show nonidentifiability of counterfactual queries from L 1 or L 2 data under our relaxation from Def. 1, and to consequently (2) give some intuition on informativity of the bounds of the ignorance interval, which we will formalize later.

Example 1 (Counterfactual non-identifiability in Markovian SCMs) . Let M 1 and M 2 be two Markovian SCMs from B ( C 0 , 2) with the following functions for Y :

where F -1 (0 , U Y 1 , U Y 2 ) is the solution in Y of the implicitly defined function F ( Y, U Y 1 , U Y 2 ) = U Y 1 -U Y 2 -2 ( Y -1) |-U Y 1 -U Y 2 +1 | -1 + √ ( Y -2) 2 (8 ( Y -1) 2 +1) = 0 .

It turns out that the SCMs M 1 and M 2 are observationally and interventionally equivalent (relative to the outcome Y ). That is, they induce the same set of L 1 and L 2 queries. For both SCMs, it can be easily inferred that a pushforward of uniform latent noise variables U Y 1 and U Y 2 with f Y ( A,U Y 1 , U Y 2 ) is a symmetric triangular distribution with support Y 0 = [ -1 , 1] for A = 0 and Y 1 = [0 , 2] for A = 1 , respectively. We plot the level sets of f Y ( A,U Y 1 , U Y 2 ) for both SCMs in Fig. 3 (left). The pushforward of the latent noise variables preserves the transported mass; i. e., note the equality of (1) the area of each colored band between the two level sets in the latent noise space and (2) the area of the corresponding band under the density graph of Y Fig. 3 (left).

Despite the equivalence of L 1 and L 2 , the SCMs differ in their counterfactuals; see Fig. 3 (right). For example, the counterfactual outcome distribution of untreated, P M ( Y 1 = y | A ′ = 0 , Y ′ = 0) , has different densities for both SCMs M 1 and M 2 . Further, the ECOU, Q M 0 → 1 (0) = E M ( Y 1 | A ′ = 0 , Y ′ = 0) , is different for both SCMs, i. e., Q M 1 0 → 1 (0) = 1 and Q M 2 0 → 1 ≈ 1 . 114 . Further details for the example are in Appendix C.

The example provides an intuition that motivates how we generate informative bounds later. By 'bending' the bundle of counterfactual level sets (in blue in Fig. 3 left) around the factual level set

(in orange in Fig. 3, right), we can transform more and more mass to the bound of the support. We later extend this idea to the ignorance interval of the ECOU. Importantly, after 'bending' the bundle of level sets, we still must make sure that the original observational/interventional distribution is preserved.

3.3 Partial Counterfactual Identification and Non-Informative Bounds

We now formulate the task of partial counterfactual identification. To do so, we first present two lemmas that show how we can infer the densities of observational and counterfactual distributions from both the latent noise distributions and C 1 functions in SCMs of class B ( C 1 , d ) .

Lemma 1 (Observational distribution as a pushforward with f Y ) . Let M∈ B ( C 1 , d ) . Then, the density of the observational distribution, induced by M , is where E ( y, a ) is a level set (preimage) of y , i. e., E ( y, a ) = { u Y ∈ [0 , 1] d : f Y ( a, u Y ) = y } , and H d -1 ( u Y ) is the Hausdorff measure (see Appendix B for the definition).

We provide an example in Appendix C where we show the application of Lemma 1 (therein, we derive the standard normal distribution as a pushforward using the Box-Müller transformation). Lemma 1 is a generalization of the well-known change of variables formula. This is easy to see, when we set d = 1 , so that P M ( Y = y | a ) = ∑ u Y ∈ E ( y,a ) |∇ u Y f Y ( a, u Y ) | -1 . Furthermore, the function f Y ( a, u Y ) can be restored (up to a sign) from the observational distribution, if it is monotonous in u Y , such as in BGMs [93]. In this case, the function coincides (up to a sign) with the inverse CDF of the observed distribution, i. e., f Y ( a, u Y ) = F -1 a ( ± u Y ∓ 0 . 5+0 . 5) (see Corollary 1 in Appendix D).

Lemma 2. Let M∈ B ( C 1 , d ) . Then, the density of the counterfactual outcome distribution of the [un]treated is

where δ ( · ) is a Dirac delta function, and the expected counterfactual outcome of the [un]treated, i.e., ECOU [ECOT], is

̸

where E ( y ′ , a ′ ) is a (factual) level set of y ′ , i. e., E ( y ′ , a ′ ) = { u Y ∈ [0 , 1] d : f Y ( a ′ , u Y ) = y ′ } and a ′ = a .

Equations (2) and (3) implicitly combine all three steps of the abduction-action-prediction procedure: (1) abduction infers the level sets E ( y ′ , a ′ ) with the corresponding Hausdorff measure H d -1 ( u Y ) ; (2) action uses the counterfactual function f Y ( a, u Y ) ; and (3) prediction evaluates the overall integral. In the specific case of d = 1 and a monotonous function f Y ( a, u Y ) with respect to u Y , we obtain two deterministic counterfactuals, which are identifiable from observational distribution, i. e., Q M a ′ → a ( y ′ ) = F -1 a ( ± F a ′ ( y ′ ) ∓ 0 . 5+0 . 5) . For details, see Corollary 3 in Appendix D. For larger d , as already shown in Example 1, both the density and ECOU [ECOT] can take arbitrary values for the same observational (or interventional) distribution.

...

...

...

...

...

Definition 2 (Partial identification of ECOU (ECOT) in class B ( C k , d ) , k ≥ 1 ) . Given the continuous observational distribution P ( Y | a ) for some SCM of class B ( C k , d ) . Then, partial counterfactual identification aims to find bounds of the ignorance interval [ Q a ′ → a ( y ′ ) , Q a ′ → a ( y ′ )] given by

where P M ( Y | a ) is given by Eq. (1) and Q M a ′ → a ( y ′ ) by Eq. (3) .

CSM

Non-informative

Informative

CSM

CSM

Partial identification

Point identification

All bivariate SCMs with measurable functions

(Theorem 1)

(Example 8)

(Example 1)

(Example 2)

(Example 7)

(Example 6)

BGMs

(Example 5)

BGMs-EQTDs

identification gap

(Lemma 3)

Residual NF

Legend

Fully-connected

network

NF

Normalizing flow,

parametrized by

Inputs / outputs with

corresponding

distributions

Losses

Noise regularization

Connections

Gradient blocking

Hence, the task of partial counterfactual identification in class B ( C k , d ) can be reduced to a constrained variational problem, namely, a constrained optimization of ECOU [ECOT] with respect to f Y ( a, · ) ∈ C k . Using Lemma 2, we see that, e. g., ECOU [ECOT] can be made arbitrarily large (or small) for the same observational distribution in two ways. First, this can be done by changing the factual function, f Y ( a ′ , · ) , i. e., if we increase the proportion of the volume of the factual level set E ( y ′ , a ′ ) that intersects only a certain bundle of counterfactual level sets. Second, this can be done by modifying the counterfactual function, f Y ( a, · ) , by 'bending' the bundle of counterfactual level sets around the factual level set. The latter is schematically shown in Fig. 4. We formalize this important observation in the following theorem.

Theorem 1 (Non-informative bounds of ECOU (ECOT)) . Let the continuous observational distribution P ( Y | a ) be induced by some SCM of class B ( C ∞ , d ) . Let P ( Y | a ) have a compact support Y a = [ l a , u a ] and be of finite density P ( Y = y | a ) < + ∞ . Then, the ignorance interval for the partial identification of the ECOU [ECOT] of class B ( C ∞ , d ) , d ≥ 2 , has non-informative bounds: Q a a ( y ′ ) = l a and Q a a ( y ′ ) = u a .

Theorem 1 implies that, no matter how smooth the class of functions is, the partial identification of ECOU [ECOT] will have non-informative bounds. Hence, with the current set of assumptions, there is no utility in considering more general classes. This includes various functions, such as C 0 and the class of all measurable functions f Y ( a, · ) : R d → R , as the latter includes C ∞ functions. In the following, we introduce a sensitivity model which nevertheless allows us to obtain informative bounds.

4 Curvature Sensitivity Model

In the following, we develop our Curvature Sensitivity Model (CSM) to restrict the class B ( C k , d ) so that the ECOU [ECOT] obtains informative bounds. Our CSM uses the intuition from the proof of Theorem 1 in that, to construct the non-informative SCM M non-inf, we have to 'bend' the bundle of the counterfactual level sets. As a result, our CSM provides sufficient conditions for informative bounds in the class B ( C 2 , d ) by bounding the principal curvatures of level sets globally.

Assumption κ . Let M be of class B ( C 2 , d ) , d ≥ 2 . Let E ( y, a ) be the level sets of functions f Y ( a, u Y ) for a ∈ { 0 , 1 } , which are thus d -1 -dimensional smooth manifolds. Let us assume that principal curvatures i ∈ { 1 , . . . , d -1 } exist at every point u Y ∈ E ( y, a ) for every a ∈ { 0 , 1 } and y ∈ ( l a , u a ) ⊂ Y a , and let us denote them as κ i ( u Y ) . Then, we assume that κ ≥ 0 is the upper bound of the maximal absolute principal curvature for every y , a , and u Y ∈ E ( y, a ) :

Principal curvatures can be thought of as a measure of the non-linearity of the level sets, and, when they are all close to zero at some point, the level set manifold can be locally approximated by a flat hyperplane. In brevity, principal curvatures can be defined via the first- and second-order partial derivatives of f Y ( a, · ) , so that they describe the degrees of curvature of a manifold in different directions. We refer to Appendix B for a formal definition of the principal curvatures κ i . An example is in Appendix C.

Now, we state the main result of our paper that our CSM allows us to obtain informative bounds for ECOU [ECOT].

Theorem 2 (Informative bounds with our CSM) . Let the continuous observational distribution P ( Y | a ) be induced by some SCM of class B ( C 2 , d ) , d ≥ 2 , which satisfies Assumption κ . Let P ( Y | a ) have a compact support Y a = [ l a , u a ] . Then, the ignorance interval for the partial identification of ECOU [ECOT] of class B ( C 2 , d ) has informative bounds, dependent on κ and d , which are given by Q a ′ → a ( y ′ ) = l ( κ, d ) > l a and Q a ′ → a ( y ′ ) = u ( κ, d ) < u a .

Theorem 2 has several important implications. (1) Our CSM is applicable to a wide class of functions B ( C 2 , d ) , for which the principal curvature is well defined. (2) We show the relationships between different classes B ( C k , d ) and our CSM ( κ ) in terms of identifiability in Fig. 5. For example, by increasing κ , we cover a larger class of functions, and the bounds on ECOU [ECOT] expand (see Corollary 4 in Appendix D). For infinite curvature, our CSM almost coincides with the entire B ( C 2 , d ) . (3) Our CSM ( κ ) with κ ≥ 0 always contains both (i) identifiable SCMs and (ii) (informative) non-identifiable SCMs. Examples of (i) include SCMs for which the bundles of the level sets coincide for both treatments, i. e., { E ( y, a ) : y ∈ Y a } = { E ( y, a ′ ) : y ∈ Y a ′ } . In this case, it is always possible to find an equivalent BGM when the level sets are flat and thus κ = 0 (see M flat from Example 6 in the Appendix C) or curved and thus κ = 1 (see M curv from Example 7 in the Appendix C). For (ii), we see that, even when we set κ = 0 , we can obtain non-identifiability with informative bounds. An example is when we align the bundle of level sets perpendicularly to each other for both treatments, as in M perp from Example 8 in the Appendix C. We formalize this observation with the following Lemma.

Lemma 3 (BGMs-EQTDs identification gap of CSM( κ = 0 )) . Let the assumptions of Theorem 2 hold and let κ = 0 . Then the ignorance intervals for the partial identification of ECOU [ECOT] of class B ( C 2 , d ) are defined by min/max between BGMs bounds and expectations of quantile-truncated distributions (EQTDs):

where E ( Y | Y < · ) , E ( Y | Y > · ) are expectations of truncated distributions.

Lemma 3 shows how close we can get to the point identification after setting κ = 0 in our CSM. In particular, with CSM ( κ = 0 ) the ignorance interval still contains BGMs bounds and EQTDs, i. e., this is the identification gap of CSM. 3 Hence, to obtain full point identification with our CSM, additional assumptions or constraints are required. Notably, we can not assume monotonicity, as there is no conventional notion of monotonicity for functions from R d to R .

We make another observation regarding the choice of the latent noise dimensionality d . In practice, it is sufficient to choose d = 2 (without further assumptions on the latent noise space). This choice is practical as we only have to enforce a single principal curvature, which reduces the computational burden, i.e.,

Importantly, we do not lose generality with d = 2 , as we still cover the entire identifiability spectrum by varying κ (see Corollary 5 in Appendix D). We discuss potential extensions of our CSM in Appendix E.

Interpretation of κ . The sensitivity parameter κ lends to a natural interpretation. Specifically, it can be interpreted as a level of non-linearity between the outcome and its latent noises that interact with the treatment . For example, when (i) the treatment does not interact with any of the latent noise variables, then we can assume w.l.o.g. a BGM, which, in turn, yields deterministic counterfactual outcomes. There is no loss of generality, as all the other SCMs with d > 1 are equivalent to this

BGM(see Examples 6 and 7 in Appendix C). On the other hand, (ii) when the treatment interacts with some latent noise variables, then the counterfactual outcomes become random, and we cannot assume a BGM but we have to use our CSM. In this case, d corresponds to the number of latent noise variables which interact with the treatment. Hence, κ bounds the level of non-linearity between the outcome and the noise variables. More formally, κ , as a principal curvature, can be seen as the largest coefficient of the second-order term in the Taylor expansion of the level set (see Eq. (19) in Appendix B). This interpretation of the κ goes along with human intuition [20]: when we try to imagine counterfactual outcomes, we tend to 'traverse' all the possible scenarios which could lead to a certain value. If we allow for highly non-linear scenarios, which interact with treatment, we also allow for more extreme counterfactuals, e. g., interactions between treatment and rare genetic conditions.

5 Augmented Pseudo-Invertible Decoder

We now introduce an instantiation of our CSM: a novel deep generative model called Augmented Pseudo-Invertible Decoder (APID) to perform partial counterfactual identification under our CSM ( κ ) of class B ( C 2 , 2) .

Architecture: The two main components of our APID are (1) residual normalizing flows with (2) variational augmentations (see Fig. 6). The first component are two-dimensional normalizing flows [107, 125] to estimate the function f Y ( a, u Y ) , u Y ∈ [0 , 1] 2 , separately for each treatment a ∈ { 0 , 1 } . Specifically, we use residual normalizing flows [24] due to their ability to model free-form Jacobians (see the extended discussion about the choice of the normalizing flow in Appendix F). However, two-dimensional normalizing flows can only model invertible transformations, while the function ˆ f Y ( a, · ) : [0 , 1] 2 → Y a ⊂ R is non-invertible. To address this, we employ an approach of pseudo-invertible flows [10, 54], namely, the variational augmentations [22], as described in the following. The second component in our APID are variational augmentations [22]. Here, we augment the estimated outcome variable ˆ Y with the variationally sampled ˆ Y aug ∼ N ( g a ( ˆ Y ) , ε 2 ) , where g a ( · ) is a fully-connected neural network, and ε 2 > 0 is a hyperparameter. Using the variational augmentations, our APID then models ˆ f Y ( a, · ) through a two-dimensional transformation ˆ F a = ( ˆ f Y aug ( a, · ) , ˆ f Y ( a, · )) : [0 , 1] 2 → R ×Y a . We refer to the Appendix F for further details.

Inference: Our APID proceeds in first steps: (P1) it first fits the observational/interventional distribution P ( Y | a ) , given the observed samples; (P2) it then performs counterfactual inference of ECOU [ECOT] in a differentiable fashion, which, therefore, can be maximized/minimized jointly with other objectives; and (P3) it finally penalize functions with large principal curvatures of the level sets, by using automatic differentiation of the estimated functions of the SCMs.

We achieve (P1)-(P3) through the help of variational augmentations: · In (P1), we fit two twodimensional normalizing flows with a negative log-likelihood (for the task of maximizing or minimizing ECOU [ECOT], respectively). · In (P2), for each normalizing flow, we sample points from the level sets of ˆ f Y ( a, · ) . The latter is crucial as it follows an abduction-action-prediction procedure and thus generates estimates of the ECOU [ECOT] in a differential fashion. To evaluate ECOU [ECOT], we first perform the abduction step with the inverse transformation of the factual normalizing flow, i.e., ˆ F -1 a ′ . This transformation maps the variationally augmented evidence, i. e., ( y ′ , { Y ′ aug } b j =1 ∼ N ( g a ( y ′ ) , ε 2 )) , where b is the number of augmentations, to the latent noise space.

Then, the action step selects the counterfactual normalizing flow via the transformation ˆ F a . Finally, the prediction step performs a pushforward of the latent noise space, which was inferred during the abduction step. Technical details are in Appendix F. · In (P3), we enforce the curvature constraint of our CSM. Here, we use automatic differentiation as provided by deep learning libraries, which allows us to directly evaluate κ 1 ( u Y ) according to Eq. (11).

Training: Our APID is trained based on observational data D = { A i , Y i } n i =1 drawn i.i.d. from some SCM of class B ( C 2 , 2) . To fit our APID, we combine several losses in one optimization objective: (1) a negative log-likelihood loss L NLL with noise regularization [110], which aims to fit the data distribution; (2) a counterfactual query loss L Q with a coefficient λ Q , which aims to maximize/minimize the ECOU [ECOT]; and (3) a curvature loss L κ with coefficient λ κ , which penalizes the curvature of the level sets. Both coefficients are hyperparameters. We provide details about the losses, the training algorithm, and hyperparameters in Appendix G. Importantly, we

incorporated several improvements to stabilize and speed up the training. For example, in addition to the negative log-likelihood, we added the Wasserstein loss L W to prevent posterior collapse [26, 132]. To speed up the training, we enforce the curvature constraint only on the counterfactual function, ˆ f Y ( a, · ) , and only for the level set, which corresponds to the evaluated ECOU [ECOT], u Y ∈ E ( ˆ Q a ′ → a ( y ′ ) , a ) .

6 Experiments

Datasets. To show the effectiveness of our APID at partial counterfactual identification, we use two synthetic datasets. This enables us to access the ground truth CDFs, quantile functions, and sampling. Then, with the help of Lemma 3, we can then compare our APID with the BGMs-EQTDs identification gap (= a special case of CSM with κ = 0 ). Both synthetic datasets comprise samples from observational distributions P ( Y | a ) , which we assume to be induced by some (unknown) SCM of class B ( C 2 , 2) . In the first dataset, P ( Y | 0) = P ( Y | 1) is the standard normal distribution, and in second, P ( Y | 0) and P ( Y | 1) are different mixtures of normal distributions. We draw n a = 1 , 000 observations from P ( Y | a ) for each treatment a ∈ { 0 , 1 } , so that n = n 0 + n 1 = 2 , 000 . Although both distributions have infinite support, we consider the finite sample minimum and maximum as estimates of the support bounds [ ˆ l 1 , ˆ u 1 ] . Further details on our synthetic datasets are in Appendix H. In sum, the estimated bounds from our APID are consistent with the theoretical values, thus showing the effectiveness of our method.

Results. Fig. 7 shows the results of point/partial counterfactual identification of the ECOU, i.e., ˆ Q 0 → 1 ( y ′ ) , for different values of y ′ . Point identification with BGMs yields two curves, corresponding to strictly increasing and strictly decreasing functions. For partial identification with our APID, we set λ Q = 2 . 0 and vary λ κ ∈ { 0 . 5 , 1 . 0 , 5 . 0 , 10 . 0 } (higher values correspond to a higher curvature penalization). We observe that, as we increase λ κ , the bounds are moving closer to the BGMs, and, as we decrease it, the bounds are becoming non-informative, namely, getting closer to [ ˆ l 1 , ˆ u 1 ] . We report additional results in Appendix H (e. g., for APID with λ Q = 1 . 0 ).

Case study with real-world data. In Appendix I, we provide a real-world case study. Therein, we adopt our CSM to answer 'what if' questions related to whether the lockdown was effective during the COVID-19 pandemic.

7 Discussion

Limitations. Our CSM and its deep-learning implementation with APID have several limitations, which could be addressed in future work. First, given the value of κ , it is computationally infeasible to derive ground-truth bounds, even when the ground-truth observation density is known. The inference of

-

2

.

0

-

1

.

5

-

1

.

0

-

0

.

5

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

y

′

-

4

-

2

0

2

4

ˆ

Q

0

→

1

(

y

′

)

P

(

Y

|

0) =

P

(

Y

|

1) =

N

(0

,

1)

-

2

.

0

-

1

.

5

-

1

.

0

-

0

.

5

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

y

′

-

4

-

3

-

2

-

1

0

1

2

3

4

ˆ

Q

0

→

1

(

y

′

)

P

(

Y

|

0) = Mixture

0

(

Y

)

P

(

Y

|

1) = Mixture

1

(

Y

)

APID(

λ

κ

= 0

.

5)

APID(

λ

κ

= 1

.

0)

APID(

λ

κ

= 5

.

0)

APID(

λ

κ

= 10

.

0)

BGMs

EQTDs

[ ˆ

l

1

,

ˆ

u

1

]

these bounds would require solving a constrained optimization task including partial derivatives and Hausdorff integration. This is intractable, even for such simple distributions as standard normal. Second, the exact relationship between κ and corresponding λ κ is unknown in APID, but they are known to be inversely related. Third, our APID sometimes suffers from computational instability, e. g., the bounds for the multi-modal distribution (see Fig. 7), are inaccurate, i. e., too tight. This happens as the gradual 'bending' of the level sets is sometimes numerically unstable during training, and some runs omit 'bending' at all as it would require passing through the high loss value region.

Conclusion. Our work is the first to present a sensitivity model for partial counterfactual identification of continuous outcomes in Markovian SCMs. Our work rests on the assumption of the bounded curvature of the level sets, yet which should be sufficiently broad and realistic to cover many models from physics and medicine. As a broader impact, we expect our bounds to be highly relevant for decision-making in safety-critical settings.

Acknowledgments

SF acknowledges funding from the Swiss National Science Foundation (SNSF) via Grant 186932.

References

Giovanni Alberti, Stefano Bianchini, and Gianluca Crippa. 'Structure of level sets and Sardtype properties of Lipschitz maps'. In: Annali della Scuola Normale Superiore di Pisa-Classe di Scienze 12.4 (2013), pp. 863-902.

Peter M. Aronow, Donald P. Green, and Donald K.K. Lee. 'Sharp bounds on the variance in randomized experiments'. In: The Annals of Statistics 42.3 (2014), pp. 850-871.

Chen Avin, Ilya Shpitser, and Judea Pearl. 'Identifiability of path-specific effects'. In: International Joint Conference on Artificial Intelligence . 2005.

Sivaraman Balakrishnan, Edward Kennedy, and Larry Wasserman. 'Conservative inference for counterfactuals'. In: arXiv preprint arXiv:2310.12757 (2023).

Vahid Balazadeh Meresht, Vasilis Syrgkanis, and Rahul G. Krishnan. 'Partial identification of treatment effects with implicit generative models'. In: Advances in Neural Information Processing Systems . 2022.

Alexander Balke and Judea Pearl. 'Counterfactual probabilities: Computational methods, bounds and applications'. In: Uncertainty Proceedings . Elsevier, 1994, pp. 46-54.

Nicolas Banholzer et al. 'Estimating the effects of non-pharmaceutical interventions on the number of new infections with COVID-19 during the first epidemic wave'. In: PLoS one 16.6 (2021), e0252827.

Elias Bareinboim et al. 'On Pearl's hierarchy and the foundations of causal inference'. In: Probabilistic and Causal Inference: The Works of Judea Pearl . Association for Computing Machinery, 2022, pp. 507-556.

Jens Behrmann et al. 'Invertible residual networks'. In: International Conference on Machine Learning . 2019.

Jan Jetze Beitler, Ivan Sosnovik, and Arnold Smeulders. 'PIE: Pseudo-invertible encoder'. In: arXiv preprint arXiv:2111.00619 (2021).

Matthew Blackwell. 'A selection bias approach to sensitivity analysis for causal effects'. In: Political Analysis 22.2 (2014), pp. 169-182.

Vladimir Igorevich Bogachev and Maria Aparecida Soares Ruas. Measure theory . Vol. 2. Springer Berlin, 2007.

Stephan Bongers et al. 'Foundations of structural causal models with cycles and latent variables'. In: The Annals of Statistics 49.5 (2021), pp. 2885-2915.

Matteo Bonvini and Edward H. Kennedy. 'Fast convergence rates for dose-response estimation'. In: arXiv preprint arXiv:2207.11825 (2022).

Matteo Bonvini et al. 'Sensitivity analysis for marginal structural models'. In: arXiv preprint arXiv:2210.04681 (2022).

Léon Bottou et al. 'Counterfactual reasoning and learning systems: The example of computational advertising'. In: Journal of Machine Learning Research 14.11 (2013).

Babette A. Brumback et al. 'Sensitivity analyses for unmeasured confounding assuming a marginal structural model for repeated measures'. In: Statistics in Medicine 23.5 (2004), pp. 749-767.

Kailash Budhathoki et al. 'Causal structure-based root cause analysis of outliers'. In: International Conference on Machine Learning . 2022.

Lars Buesing et al. 'Woulda, coulda, shoulda: Counterfactually-guided policy search'. In: International Conference on Learning Representations . 2019.

Lenart Celar and Ruth M.J. Byrne. 'How people reason with counterfactual and causal explanations for artificial intelligence decisions in familiar and unfamiliar domains'. In: Memory & Cognition (2023), pp. 1-16.

Patrick Chao, Patrick Blöbaum, and Shiva Prasad Kasiviswanathan. 'Interventional and counterfactual inference with diffusion models'. In: arXiv preprint arXiv:2302.00860 (2023).

Jianfei Chen et al. 'Vflow: More expressive generative flows with variational data augmentation'. In: International Conference on Machine Learning . 2020.

Ricky T.Q. Chen et al. 'Neural ordinary differential equations'. In: Advances in Neural Information Processing Systems . 2018.

Ricky T.Q. Chen et al. 'Residual flows for invertible generative modeling'. In: Advances in Neural Information Processing Systems . 2019.

Scott Chen and Ramesh Gopinath. 'Gaussianization'. In: Advances in Neural Information Processing Systems . 2000.

Florentin Coeurdoux, Nicolas Dobigeon, and Pierre Chainais. 'Sliced-Wasserstein normalizing flows: Beyond maximum likelihood training'. In: European Symposium on Artificial Neural Networks . 2022.

Juan Correa, Sanghack Lee, and Elias Bareinboim. 'Nested counterfactual identification from arbitrary surrogate experiments'. In: Advances in Neural Information Processing Systems . 2021.

Alicia Curth and Mihaela van der Schaar. 'Nonparametric estimation of heterogeneous treatment effects: from theory to learning algorithms'. In: International Conference on Artificial Intelligence and Statistics . 2021.

Saloni Dash, Vineeth N. Balasubramanian, and Amit Sharma. 'Evaluating and mitigating bias in image classifiers: A causal perspective using counterfactuals'. In: IEEE/CVF Winter Conference on Applications of Computer Vision . 2022.

A. Philip Dawid. 'Influence diagrams for causal modelling and inference'. In: International Statistical Review 70.2 (2002), pp. 161-189.

Edward De Brouwer. 'Deep counterfactual estimation with categorical background variables'. In: Advances in Neural Information Processing Systems . 2022.

Lucas De Lara et al. 'Transport-based counterfactual models'. In: arXiv preprint arXiv:2108.13025 (2021).

Laurent Dinh, David Krueger, and Yoshua Bengio. 'NICE: Non-linear independent components estimation'. In: arXiv preprint arXiv:1410.8516 (2014).

Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. 'Density estimation using real NVP'. In: International Conference on Learning Representations . 2017.

Jacob Dorn and Kevin Guo. 'Sharp sensitivity analysis for inverse propensity weighting via quantile balancing'. In: Journal of the American Statistical Association (2022), pp. 1-13.

Tom Everitt et al. 'Agent incentives: A causal perspective'. In: AAAI Conference on Artificial Intelligence . 2021.

Yanqin Fan and Sang Soo Park. 'Sharp bounds on the distribution of treatment effects and their statistical inference'. In: Econometric Theory 26.3 (2010), pp. 931-951.

Jakob Foerster et al. 'Counterfactual multi-agent policy gradients'. In: AAAI Conference on Artificial Intelligence . 2018.

Dennis Frauen and Stefan Feuerriegel. 'Estimating individual treatment effects under unobserved confounding using binary instruments'. In: International Conference on Learning Representations . 2023.

Dennis Frauen, Valentyn Melnychuk, and Stefan Feuerriegel. 'Sharp bounds for generalized causal sensitivity analysis'. In: Advances in Neural Information Processing Systems . 2023.

Dennis Frauen et al. 'A neural framework for generalized causal sensitivity analysis'. In: arXiv preprint arXiv:2311.16026 (2023).

Sainyam Galhotra, Romila Pradhan, and Babak Salimi. 'Explaining black-box algorithms using probabilistic contrastive counterfactuals'. In: International Conference on Management of Data . 2021.

Ron Goldman. 'Curvature formulas for implicit curves and surfaces'. In: Computer Aided Geometric Design 22.7 (2005), pp. 632-658.

Yash Goyal et al. 'Counterfactual visual explanations'. In: International Conference on Machine Learning . 2019.

Will Grathwohl et al. 'Scalable reversible generative models with free-form continuous dynamics'. In: International Conference on Learning Representations . 2019.

Luigi Gresele et al. 'Causal inference through the structural causal marginal problem'. In: International Conference on Machine Learning . 2022.

Luigi Gresele et al. 'The incomplete rosetta stone problem: Identifiability results for multiview nonlinear ICA'. In: Conference on Uncertainty in Artificial Intelligence . 2020.

Florian Gunsilius. 'A path-sampling method to partially identify causal effects in instrumental variable models'. In: arXiv preprint arXiv:1910.09502 (2019).

Wenshuo Guo et al. 'Partial identification with noisy covariates: A robust optimization approach'. In: Conference on Causal Learning and Reasoning . 2022.

Joseph Halpern and Max Kleiman-Weiner. 'Towards formal definitions of blameworthiness, intention, and moral responsibility'. In: AAAI Conference on Artificial Intelligence . 2018.

Joseph Y. Halpern and Judea Pearl. 'Causes and explanations: A structural-model approach. part i: Causes'. In: The British Journal for the Philosophy of Science (2005).

Tobias Hatt and Stefan Feuerriegel. 'Estimating average treatment effects via orthogonal regularization'. In: International Conference on Information and Knowledge Management . 2021.

Lars Hörmander. Composition with Smooth Maps . Springer Berlin Heidelberg, 2003, pp. 133157.

Christian Horvat and Jean-Pascal Pfister. 'Denoising normalizing flow'. In: Advances in Neural Information Processing Systems . 2021.

Yaowei Hu et al. 'A generative adversarial framework for bounding confounded causal effects'. In: AAAI Conference on Artificial Intelligence . 2021.

Yimin Huang and Marco Valtorta. 'Identifiability in causal bayesian networks: A sound and complete algorithm'. In: National Conference on Artificial Intelligence (AAAI) . 2006.

Aapo Hyvarinen, Hiroaki Sasaki, and Richard Turner. 'Nonlinear ICA using auxiliary variables and generalized contrastive learning'. In: International Conference on Artificial Intelligence and Statistics . 2019.

Guido W. Imbens and Joshua D. Angrist. 'Identification and estimation of local average treatment effects'. In: Econometrica 62.2 (1994), pp. 467-475.

Alexander Immer et al. 'On the identifiability and estimation of causal location-scale noise models'. In: International Conference on Machine Learning . 2023.

Andrew Jesson et al. 'Quantifying ignorance in individual-level causal-effect estimates under hidden confounding'. In: International Conference on Machine Learning . 2021.

Andrew Jesson et al. 'Scalable sensitivity and uncertainty analyses for causal-effect estimates of continuous-valued interventions'. In: Advances in Neural Information Processing Systems . 2022.

Ying Jin, Zhimei Ren, and Zhengyuan Zhou. 'Sensitivity analysis under the f -sensitivity models: A distributional robustness perspective'. In: arXiv preprint arXiv:2203.04373 (2022).

Nathan Kallus and Angela Zhou. 'Confounding-robust policy evaluation in infinite-horizon reinforcement learning'. In: Advances in Neural Information Processing Systems . 2020.

Amir-Hossein Karimi, Bernhard Schölkopf, and Isabel Valera. 'Algorithmic recourse: From counterfactual explanations to interventions'. In: ACM Conference on Fairness, Accountability, and Transparency . 2021.

Amir-Hossein Karimi et al. 'A survey of algorithmic recourse: Definitions, formulations, solutions, and prospects'. In: arXiv preprint arXiv:2010.04050 (2020).

Ilyes Khemakhem et al. 'Causal autoregressive flows'. In: International Conference on Artificial Intelligence and Statistics . 2021.

Ilyes Khemakhem et al. 'Variational autoencoders and nonlinear ICA: A unifying framework'. In: International Conference on Artificial Intelligence and Statistics . 2020.

Niki Kilbertus, Matt J. Kusner, and Ricardo Silva. 'A class of algorithms for general instrumental variable models'. In: Advances in Neural Information Processing Systems . 2020.

Niki Kilbertus et al. 'The sensitivity of counterfactual fairness to unmeasured confounding'. In: Conference on Uncertainty in Artificial Intelligence . 2020.

Hyemi Kim et al. 'Counterfactual fairness with disentangled causal effect variational autoencoder'. In: AAAI Conference on Artificial Intelligence . 2021.

Diederik P. Kingma and Jimmy Ba. 'Adam: A method for stochastic optimization'. In: International Conference on Learning Representations . 2015.

Durk P. Kingma et al. 'Improved variational inference with inverse autoregressive flow'. In: Advances in Neural Information Processing Systems . 2016.

Max Kleiman-Weiner et al. 'Inference of intention and permissibility in moral decision making'. In: CogSci . 2015.

Murat Kocaoglu et al. 'CausalGAN: Learning causal implicit generative models with adversarial training'. In: International Conference on Learning Representations . 2018.

Gunnar König, Timo Freiesleben, and Moritz Grosse-Wentrup. 'Improvement-focused causal recourse (ICR)'. In: AAAI Conference on Artificial Intelligence . 2023.

Matt J. Kusner et al. 'Counterfactual fairness'. In: Advances in Neural Information Processing Systems . 2017.

David A. Lagnado, Tobias Gerstenberg, and Ro'i Zultan. 'Causal responsibility and counterfactuals'. In: Cognitive Science 37.6 (2013), pp. 1036-1073.

John M. Lee. Riemannian manifolds: An introduction to curvature . Vol. 176. Springer New York, 2006.

Sanghack Lee, Juan D. Correa, and Elias Bareinboim. 'General identifiability with arbitrary surrogate experiments'. In: Conference on Uncertainty in Artificial Intelligence . 2020.

Ang Li, Scott Mueller, and Judea Pearl. 'Epsilon-identifiability of causal quantities'. In: arXiv preprint arXiv:2301.12022 (2023).

Ang Li and Judea Pearl. 'Bounds on causal effects and application to high dimensional data'. In: AAAI Conference on Artificial Intelligence . 2022.

Ang Li and Judea Pearl. 'Probabilities of causation with nonbinary treatment and effect'. In: arXiv preprint arXiv:2208.09568 (2022).

Ang Li and Judea Pearl. 'Unit selection based on counterfactual logic'. In: International Joint Conference on Artificial Intelligence . 2019.

Mohammad Lotfollahi et al. 'Compositional perturbation autoencoder for single-cell response modeling'. In: BioRxiv (2021).

Christos Louizos et al. 'Causal effect inference with deep latent-variable models'. In: Advances in Neural Information Processing Systems . 2017.

Chaochao Lu et al. 'Sample-efficient reinforcement learning via counterfactual-based data augmentation'. In: arXiv preprint arXiv:2012.09092 (2020).

Charles F. Manski. 'Monotone treatment response'. In: Econometrica: Journal of the Econometric Society (1997), pp. 1311-1334.

Charles F. Manski. 'Nonparametric bounds on treatment effects'. In: The American Economic Review 80.2 (1990), pp. 319-323.

Myrl G. Marmarelis et al. 'Partial identification of dose responses with hidden confounders'. In: Conference on Uncertainty in Artificial Intelligence . 2023.

Valentyn Melnychuk, Dennis Frauen, and Stefan Feuerriegel. 'Normalizing flows for interventional density estimation'. In: International Conference on Machine Learning . 2023.

Thomas Mesnard et al. 'Counterfactual credit assignment in model-free reinforcement learning'. In: International Conference on Machine Learning . 2021.

Wang Miao, Zhi Geng, and Eric J. Tchetgen Tchetgen. 'Identifying causal effects with proxy variables of an unmeasured confounder'. In: Biometrika 105.4 (2018), pp. 987-993.

Arash Nasr-Esfahany, Mohammad Alizadeh, and Devavrat Shah. 'Counterfactual identifiability of bijective causal models'. In: International Conference on Machine Learning . 2023.

Arash Nasr-Esfahany and Emre Kiciman. 'Counterfactual (non-)identifiability of learned structural causal models'. In: arXiv preprint arXiv:2301.09031 (2023).

Luigi Negro. 'Sample distribution theory using coarea formula'. In: Communications in Statistics Theory and Methods (2022), pp. 1-26.

Marcel Nutz and Ruodu Wang. 'The directional optimal transport'. In: The Annals of Applied Probability 32.2 (2022), pp. 1400-1420.

Michael Oberst and David Sontag. 'Counterfactual off-policy evaluation with gumbel-max structural causal models'. In: International Conference on Machine Learning . 2019.

Miruna Oprescu et al. 'B-Learner: Quasi-oracle bounds on heterogeneous causal effects under hidden confounding'. In: International Conference on Machine Learning . 2023.

Kirtan Padh et al. 'Stochastic causal programming for bounding treatment effects'. In: Conference on Causal Learning and Reasoning . 2023.

Nick Pawlowski, Daniel Coelho de Castro, and Ben Glocker. 'Deep structural causal models for tractable counterfactual inference'. In: Advances in Neural Information Processing Systems . 2020.

Judea Pearl. Causality . Cambridge University Press, 2009.

Judea Pearl. Causality: Models, Reasoning and Inference . Cambridge University Press, 2000.

Judea Pearl. 'Probabilities of causation: Three counterfactual interpretations and their identification'. In: Synthese 121 (1999), pp. 93-149.

Drago Plecko and Elias Bareinboim. 'Causal fairness analysis'. In: arXiv preprint arXiv:2207.11385 (2022).

Boris T. Polyak and Anatoli B. Juditsky. 'Acceleration of stochastic approximation by averaging'. In: SIAM Journal on Control and Optimization 30.4 (1992), pp. 838-855.

Ashesh Rambachan, Amanda Coston, and Edward Kennedy. 'Counterfactual risk assessments under unmeasured confounding'. In: arXiv preprint arXiv:2212.09844 (2022).

Danilo Rezende and Shakir Mohamed. 'Variational inference with normalizing flows'. In: International Conference on Machine Learning . 2015.

Jonathan Richens, Rory Beard, and Daniel H. Thompson. 'Counterfactual harm'. In: Advances in Neural Information Processing Systems . 2022.

James M. Robins, Andrea Rotnitzky, and Daniel O. Scharfstein. 'Sensitivity analysis for selection bias and unmeasured confounding in missing data and causal inference models'. In: IMA Volumes in Mathematics and its Applications 116 (2000), pp. 1-94.

Jonas Rothfuss et al. 'Noise regularization for conditional density estimation'. In: arXiv preprint arXiv:1907.08982 (2019).

Donald B. Rubin. 'Estimating causal effects of treatments in randomized and nonrandomized studies.' In: Journal of Educational Psychology 66.5 (1974), p. 688.

Michael C. Sachs et al. 'A general method for deriving tight symbolic bounds on causal effects'. In: Journal of Computational and Graphical Statistics (2022), pp. 1-10.

Pedro Sanchez and Sotirios A. Tsaftaris. 'Diffusion causal models for counterfactual estimation'. In: Conference on Causal Learning and Reasoning . 2021.

Pablo Sánchez-Martin, Miriam Rateike, and Isabel Valera. 'VACA: Designing variational graph autoencoders for causal queries'. In: AAAI Conference on Artificial Intelligence . 2022.

Numair Sani, Atalanti A. Mastakouri, and Dominik Janzing. 'Bounding probabilities of causation through the causal marginal problem'. In: arXiv preprint arXiv:2304.02023 (2023).

George Saon, Satya Dharanipragada, and Daniel Povey. 'Feature space gaussianization'. In: International Conference on Acoustics, Speech, and Signal Processing . 2004.

Axel Sauer and Andreas Geiger. 'Counterfactual generative networks'. In: International Conference on Learning Representations . 2021.

Xinwei Shen et al. 'Weakly supervised disentangled generative causal representation learning'. In: Journal of Machine Learning Research 23 (2022), pp. 1-55.

Ilya Shpitser and Judea Pearl. 'Effects of treatment on the treated: Identification and generalization'. In: Conference on Uncertainty in Artificial Intelligence . 2009.

Ilya Shpitser and Judea Pearl. 'Identification of joint interventional distributions in recursive semi-Markovian causal models'. In: National Conference on Artificial Intelligence (AAAI) . 2006.

Ilya Shpitser and Judea Pearl. 'What counterfactuals can be tested'. In: Conference on Uncertainty in Artificial Intelligence . 2007.

Ilya Shpitser and Eli Sherman. 'Identification of personalized effects associated with causal pathways'. In: Conference on Uncertainty in Artificial Intelligence . 2018.

Peter Spirtes et al. Causation, prediction, and search . MIT Press, 2000.

Suraj Srinivas et al. 'Efficient training of low-curvature neural networks'. In: Advances in Neural Information Processing Systems . 2022.

Esteban G. Tabak and Eric Vanden-Eijnden. 'Density estimation by dual ascent of the log-likelihood'. In: Communications in Mathematical Sciences 8.1 (2010), pp. 217-233.

Zhiqiang Tan. 'A distributional approach for causal inference using propensity scores'. In: Journal of the American Statistical Association 101.476 (2006), pp. 1619-1637.

Jin Tian and Judea Pearl. 'Probabilities of causation: Bounds and identification'. In: Annals of Mathematics and Artificial Intelligence 28.1-4 (2000), pp. 287-313.

Rianne Van Den Berg et al. 'Sylvester normalizing flows for variational inference'. In: Conference on Uncertainty in Artificial Intelligence . 2018.

Athanasios Vlontzos, Bernhard Kainz, and Ciarán M. Gilligan-Lee. 'Estimating categorical counterfactuals via deep twin networks'. In: Nature Machine Intelligence 5.2 (2023), pp. 159168.

Matthew J. Vowels et al. 'A free lunch with influence functions? Improving neural network estimates with concepts from semiparametric statistics'. In: arXiv preprint arXiv:2202.09096 (2022).

Linbo Wang and Eric Tchetgen Tchetgen. 'Bounded, efficient and multiply robust estimation of average treatment effects using instrumental variables'. In: Journal of the Royal Statistical Society: Series B 80.3 (2018), pp. 531-550.

Prince Zizhuang Wang and William Yang Wang. 'Riemannian normalizing flow on variational Wasserstein autoencoder for text modeling'. In: Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies . 2019.

Pengzhou Abel Wu and Kenji Fukumizu. ' β -intact-V AE: Identifying and estimating causal effects under limited overlap'. In: International Conference on Learning Representations . 2022.

Yulun Wu et al. 'Predicting cellular responses with variational causal inference and refined relational information'. In: International Conference on Learning Representations . 2023.

Yulun Wu et al. 'Variational causal inference'. In: arXiv preprint arXiv:2209.05935 (2022).

Kevin Xia et al. 'The causal-neural connection: Expressiveness, learnability, and inference'. In: Advances in Neural Information Processing Systems . 2021.

Kevin Muyuan Xia, Yushu Pan, and Elias Bareinboim. 'Neural causal models for counterfactual identification and estimation'. In: International Conference on Learning Representations . 2023.

Kentaro Yamada, Manabu Kuroki, et al. 'Counterfactual-based prevented and preventable proportions'. In: Journal of Causal Inference 5.2 (2017), pp. 1-15.

Teppei Yamamoto. 'Understanding the past: Statistical analysis of causal attribution'. In: American Journal of Political Science 56.1 (2012), pp. 237-256.

Jinsung Yoon, James Jordon, and Mihaela van der Schaar. 'GANITE: Estimation of individualized treatment effects using generative adversarial nets'. In: International Conference on Learning Representations . 2018.

Marco Zaffalon, Alessandro Antonucci, and Rafael Cabañas. 'Structural causal models are (solvable by) credal networks'. In: International Conference on Probabilistic Graphical Models . 2020.

Marco Zaffalon et al. 'Bounding counterfactuals under selection bias'. In: International Conference on Probabilistic Graphical Models . 2022.

Marco Zaffalon et al. 'Learning to bound counterfactual inference in structural causal models from observational and randomised data'. In: arXiv preprint arXiv:2212.02932 (2022).

Matej Zeˇ cevi´ c et al. 'Relating graph neural networks to structural causal models'. In: arXiv preprint arXiv:2109.04173 (2021).

Junzhe Zhang and Elias Bareinboim. 'Bounding causal effects on continuous outcome'. In: AAAI Conference on Artificial Intelligence . 2021.

Junzhe Zhang and Elias Bareinboim. 'Fairness in decision-making: The causal explanation formula'. In: AAAI Conference on Artificial Intelligence . 2018.

Junzhe Zhang, Jin Tian, and Elias Bareinboim. 'Partial counterfactual identification from observational and experimental data'. In: International Conference on Machine Learning . 2022.

Kun Zhang and Aapo Hyvärinen. 'On the identifiability of the post-nonlinear causal model'. In: Conference on Uncertainty in Artificial Intelligence . 2009.

A Extended Related Work

A.1 Counterfactual inference

In the following, we explain why existing work does not straightforwardly generalize to the partial counterfactual identification of continuous outcomes. In particular, we do the following:

We pinpoint that symbolic (non-parametric) counterfactual identifiability methods only provide the probabilistic expression suitable for point identification, if a certain query can be expressed via lower layer information.

We note that existing point identification methods do not have any guidance towards partial identification, and often do not even provide identifiability guarantees. In this case, valid point identification can only be achieved via additional assumptions either (i) about the SCM or (ii) about the data generating mechanism.

We discuss why methods for partial identification of (i) discrete counterfactual queries, (ii) continuous counterfactual queries with informative bounds, and (iii) continuous interventional queries can not be extended to our setting.

Ultimately, we summarize related works on both point and partial identification for two layers of causality, namely interventional and counterfactual, in Table 1.

Legend:

1. Symbolic identifiability and point identification with probabilistic expressions. Complete and sound algorithms were proposed for symbolic (non-parametric) identifiability of the counterfactuals based on observational ( L 1 ) or interventional ( L 2 ) information, and causal diagram of a semiMarkovian SCM. For example, [121] adapted d-separation for the parallel-worlds networks [3] for simple counterfactual queries, and [27] extended it to nested counterfactuals with counterfactual unnesting theorem. These methods aim to provide a probabilistic expression, in the case the query is identifiable, which is then used in downstream point identification methods. Rather rare examples of identifiable queries include the treatment effect of the treated (ETT) [119] and path-specific effects [122, 146] for certain SCMs, e. g., when the treatment is binary. If the query is non-identifiable, like in the case with ECOU [ECOT] as in our paper, no guidance is provided on how to perform partial identification and methods can not be used.

2(i). Point counterfactual identification with SCMs. Another way to perform a point identification when the counterfactual query is symbolically non-identifiable is to restrict a functional class in SCMs. Examples of restricted Markovian SCMs include nonlinear additive noise models (ANMs) [66, 123], post nonlinear models (PNL) [148], location-scale noise models (LSNMs) [59], and bijective generation mechanisms (BGMs) [93, 94]. The latest, BGMs, also work in semi-Markovian settings with additional assumptions. Numerous deep generative models were also proposed for

point identification in both Markovian and semi-Markovian settings. Examples are normalizing flows [100]; diffusion models [21, 113]; variational inference [70, 100, 114, 134, 135]; adversarial learning [29, 84, 100, 117, 118, 140], and neural expectation-maximization [31]. As it was shown in [94], all those deep generative models above need to assume the BGM of the data to yield valid counterfactual inference. 4 The assumption of monotonicity of BGMs effectively sets the dimensionality of the latent noise to the same as the observed variables, which is rarely a realistic assumption. In our paper, we relax this assumption, and let the latent noise have arbitrary dimensionality.

2(ii). Point counterfactual identification with non-SCMs-based approaches. Alternative, nonSCM-based definitions of the counterfactuals exist, e. g., transport-based counterfactuals [32, 96], and conservative counterfactual effects [4]. Still, for the identifiability of ECOU [ECOT], they also rely on additional assumptions such as monotonicity. 5 These approaches were shown to coincide with standard SCM-based counterfactuals when the latent noise can be deterministically defined with observed data. Therefore, they coincide with BGMs.

3(i). Discrete partial counterfactual identification. Partial counterfactual identification was rigorously studied only for discrete SCMs or discrete outcomes. For example, bounds under no assumptions were derived for counterfactual probabilities [6, 80, 82, 103, 127]. More general counterfactual queries can also be tackled with response functions framework [6] and, more generally, canonical partitioning [112] in combination with deep neural networks [97, 137, 141, 142, 143, 147]. The same idea was used for causal marginal problem [46, 115] where the authors combined different experimental and observational data with overlapping observed variables. However, response functions framework and canonical partitioning can not be extended to the continuous setting in practice, as their computational complexity grows exponentially with respect to the cardinality of the observed variables. Hence, the aforementioned methods are not relevant baselines in our setting.

3(ii). Continuous partial counterfactual identification of queries with informative bounds. Certain counterfactual queries have ignorance intervals with informative bounds, which have a closed-form solution using Frechet-Hoeffding bounds for copulas. For example, informative bounds were derived, e. g., for the variance of the treatment effects [2], Var( Y a -Y a ′ ) ; for the CDF of the joint potential outcomes distribution [37], P ( Y a , Y a ′ ) , and other related queries [4]. Yet, these bounds cannot be applied to our setting, as our query of interest, namely, ECOU [ECOT], has non-informative bounds and, thus, requires a sensitivity model .

3(iii). Continuous partial interventional identification. The problem of partial interventional identification arises in semi-Markovian SCMs and usually aims at hidden confounding issues of treatment effect estimation [88]. For example, instrumental variable (IV) models always obtain informative bounds under the assumption of instrumental validity [48], so that partial identification is formulated as an optimization problem [55, 68, 99, 145]. In other cases, hidden confounding causes non-informative bounds and additional assumptions about its strength are needed, i. e., a sensitivity model . For example, the marginal sensitivity model (MSM) assumes the odds ratio between nominal and true propensity scores and derives informative bounds for average or conditional average treatment effects (ATE and CATE, respectively) [15, 35, 40, 41, 60, 61, 62, 89, 98, 126]. The outcome sensitivity models [15, 106] and confounding functions framework [17, 109] were also introduced for the sensitivity analysis of ATE and CATE. [49] develops a sensitivity model for ATE assuming a noise level of proxy variables. All the mentioned sensitivity models operate on conditional distributions and, therefore, do not extend to counterfactual partial identification, as the latter requires assumptions about the SCM. Some methods indeed restrict functional classes in SCMs to achieve informative bounds. For example, linear SCMs are assumed for partial identification of average treatment derivative [5]. By developing our CSM in our paper, we discuss what restrictions are required for SCMs to achieve informative bounds for partial counterfactual identification of continuous outcomes. Nevertheless, the methods for partial interventional identification are not aimed at counterfactual inference, and, thus, are not directly applicable in our setting.

A.2 Identifiability of latent variable models and disentanglement

The question of identifying latent noise variables was also studied from the perspective of nonlinear independent component analysis (ICA) [47, 57, 67]. Khemakhem et al. [67] showed that the joint distribution over observed and latent noise variables is in general unidentifiable. Although, nonlinear ICA was applied for interventional inference [133], these works did not consider SCMs or counterfactual queries.

B Background materials

Geometric measure theory. Let δ ( x ) : R → R be a Dirac delta function, defined as zero everywhere, except for x = 0 , where it has a point mass of 1. Dirac delta function induces a Dirac delta measure, so that (with the slight abuse of notation)

where f is a C 0 function with compact support. Dirac delta function satisfies the following important equality

̸

where f is a C 0 function with compact support, g is a C 1 function, g ′ ( x i ) = 0 , and x i are roots of the equation g ( x ) = 0 .

In addition, we define the s -dimensional Hausdorff measure H s , as in [95]. Let E ⊆ R n be a s -dimensional smooth manifold ( s ≤ n ) embedded into R n . Then, E is a Borel subset in R n , is Hausdorff-measurable, and s -dimensional Hausdorff measure H s ( E ) is the s -dimensional surface volume of E . For example, if s = 1 , the Hausdorff measure coincides with a line integral, and, if s = 2 , with surface integral:

where × is a vector product, x i ( t ) is a t -parametrization of the line, and x i ( s, t ) is a ( s, t ) -parametrization of the surface.

Also, H n ( E ) = vol n ( E ) for Lebesgue measurable subsets E ⊆ R n , where vol n is a standard n -dimensional volume (Lebesgue measure). In the special case of s = 0 , Hausdorff measure is a counting measure: ∫ E f ( x ) d H 0 ( x ) = ∑ x ∈ E f ( x ) .

Importantly, the Hausdorff measure is related to the high-dimensional Dirac delta measure via the coarea formula [53] (Theorem 6.1.5), [95] (Theorem 3.3):

where f is C 0 function with compact support, and g is a C 1 function with ∇ x g ( x ) > 0 for x ∈ E . Functions for which ∇ x g ( x ) > 0 for x ∈ R d holds are called regular .

We define a bundle of level sets of the function y = f ( x ) as a set of the level sets, indexed by y , i.e., { E ( y ); y ∈ Y ∈ R } , where E ( y ) = { y ∈ R : f ( x ) = y } and Y ⊆ R . A bundle of level sets of regular functions are closely studied in the Morse theory . In particular, we further rely on the fundamental Morse theorem , i. e., the level set bundles of a regular function are diffeomorphic to each other. This means that there exists a continuously differentiable bijection between every pair of the bundles.

Differential geometry of manifolds. In the following, we formally define the notion of the curvature for level sets. As the result of the implicit function theorem , level sets E ( y ) of a regular function f of class C 1 are Riemannian manifolds [78], namely smooth differentiable manifolds. Riemannian manifolds can be locally approximated via Euclidean spaces, i. e., they are equipped with a tangent space T x ( E ( y )) at a point x , and a dot product defined on the tangent space. The tangent space, T x ( E ( y )) , is orthogonal to the normal of the manifold, ∇ x f ( x ) .

For the level sets of regular functions of the class C 2 , we can define the so-called curvature [43, 78]. Informally, curvature defines the extent a Riemannian manifold bends in different directions.

Convex regions of the manifold correspond to a negative curvature (in all directions), and concave regions to positive curvature, respectively. Saddle points have curvatures of different signs in different directions. Formally, the curvature is defined via the rate of change of the unit normal of the manifold, which is parameterized with the orthogonal basis { ˜ x 1 , . . . , ˜ x n -1 } of the tangent space, T x ( E ( y )) . This rate of change, namely a differential, forms a shape operator (or second fundamental form) on the tangent space, T x ( E ( y )) . Then, eigenvalues of the shape operator are called principal curvatures , κ i ( x ) for i ∈ { 1 , . . . , n -1 } . Principal curvatures are a measure of the extrinsic curvature, i. e., the curvature of the manifold with respect to the embedding space, R n .

Principal curvatures for Riemannian manifolds, defined as the level sets of a regular C 2 function f , can be also expressed via the gradient and the Hessian of the following function [43]:

where root i are roots of the equation with respect to λ ∈ R . For x ∈ R 2 , the level sets E ( y ) are curves, and there is only one principal curvature

One of the important properties of the principal curvatures is that we can locally parameterize the manifold as the second-order hypersurface, i. e.,

where { ˜ x 1 , . . . , ˜ x n -1 } is the orthogonal basis of the tangent space T x ( E ( y )) .

C Examples

Example 1 (Counterfactual non-identifiability in Markovian-SCMs (continued)) . Here, we continue the example and provide the inference of observational (interventional) and counterfactual distributions for the SCMs M 1 and M 2 .

Let us consider M 1 first. It is easy to see that a pushforward of uniform distribution in a unit square with f Y ( A,U Y 1 , U Y 2 ) = A ( U Y 1 -U Y 2 + 1) + (1 -A ) ( U Y 1 + U Y 2 -1) induces triangular distributions. For example, for f Y (0 , U Y 1 , U Y 2 ) , a cumulative distribution function (CDF) of P M 1 ( Y | A = 0) = P M 1 ( Y a =0 ) will have the form

which is the CDF of a triangular distribution. Analogously, a CDF of P M 1 ( Y | A = 1) = P M 1 ( Y a =1 ) is

To infer the counterfactual outcome distribution of the untreated, P M 1 ( Y a =1 | A ′ = 0 , Y ′ = 0) , we make use of Lemma 2 and properties of the Dirac delta function (e. g., Eq. (13) ). We yield

where ( ∗ ) introduces a parametrization of the line { u Y 1 + u Y 2 -1 = 0 } with t , namely

Therefore, the ECOU for M 1 is

Now, let us consider M 2 . Hence, f Y (0 , U Y 1 , U Y 2 ) is the same for both M 1 and M 2 , so are the observational (interventional) distributions, i. e., P M 1 ( Y | A = 0) = P M 2 ( Y | A = 0) . The same is true for f Y (1 , U Y 1 , U Y 2 ) with (0 ≤ U Y 1 ≤ 1) ∧ ( U Y 1 ≤ U Y 2 ≤ 1) or, equivalently, P M 1 ( Y = y | A = 1) = P M 2 ( Y = y | A = 1) with y ∈ (0 , 1] .

Now, it is left to check whether P M 2 ( Y = y | A = 1) is the density of a triangular distribution for y ∈ (1 , 2] . For that, we define level sets of the function f Y (1 , U Y 1 , U Y 2 ) in the remaining part of the unit square in a specific way. Formally, they are a family of 'bent' lines in the transformed two-dimensional space ˜ u Y 1 , ˜ u Y 2 with

where b ( t ) is a bias, depending on t . The area under the line should change with quadratic speed, as it will further define the CDF of the induced observational distribution, F M 2 ( y | A = 1) , y ∈ (1 , 2] , i. e.

so that S (0) = 0 and S (1) = 1 . On the other hand, the area under the line from the family is

where ˜ u ∗ = 1 -b ( t ) 8 t 2 +1 . Therefore, we can find the dependence of b ( t ) on t , so that the area S ( t ) is preserved:

Let us reparametrize t with t = y -1 , so that the line from a family with t = 0 corresponds to y = 1 and t = 1 to y = 2 , respectively. We then yield

In order to obtain the function f Y (1 , U Y 1 , U Y 2 ) in the original coordinates u Y 1 , u Y 2 , we use a linear transformation T given by

Hence, the family of 'bent' lines can be represented as the following implicit equation:

Importantly, the determinant of the Jacobian of the transformation T is equal to 2; therefore, the area under the last line S (1) shrinks from 1 (in space ˜ u Y 1 , ˜ u Y 2 ) to 0.5 (in space u Y 1 , u Y 2 ). Thus, we can also easily verify with Eq. (31) that the CDF of the induced observational distribution coincides with the CDF of the triangular distribution for y ∈ (1 , 2] :

To infer the counterfactual outcome distribution of the untreated, i.e., P M 2 ( Y a =1 | A ′ = 0 , Y ′ = 0) , we again use the Lemma 2 and properties of Dirac delta function (e. g., Eq. (13) ). We yield

where ( ∗ ) introduces a parametrization of the line { u Y 1 + u Y 2 -1 = 0 } with t , namely

Finally, the ECOU for M 2 can be calculated numerically via

Example 2 (Box-Müller transformation) . Here, we demonstrate the application of the Lemma 1 to infer the standard normal distribution with the Box-Müller transformation. The Box-Müller transformation is a well-established approach to sample from the standard normal distribution, which omits the usage of the inverse CDF of the normal distribution. Formally, the Box-Müller transformation is described by an SCM M bm of class B ( C 1 , 2) 6 with the following function for Y :

We will now use the Lemma 1 to verify that P M bm ( Y | a ) = N (0 , 1) . We yield

where ( ∗ ) introduces a parametrization of the level set { √ -2 log( u Y 1 ) cos( πu Y 2 ) = y } , y > 0 with t , namely

This parametrization is also valid for y < 0 and t ∈ ( 1 2 , 1] . Due to the symmetry of the function f Y , we only consider one of the cases (see Fig. 8 (left) with the level sets).

We plot the level set for y = -0 . 5 in Fig. 9, namely E ( y ) = { u Y ∈ [0 , 1] 2 : f Y ( u Y 1 , u Y 2 ) = y } . Here, we see that the level sets consist of an infinite number of connected components with an infinite total length.

0

.

00

0

.

25

0

.

50

0

.

75

1

.

00

U

Y

1

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

U

Y

2

E

(

-

1

.

0)

0

.

00

0

.

25

0

.

50

0

.

75

1

.

00

U

Y

1

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

U

Y

2

E

(

-

0

.

5)

0

.

00

0

.

25

0

.

50

0

.

75

1

.

00

U

Y

1

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

U

Y

2

E

(0

.

5)

0

.

00

0

.

25

0

.

50

0

.

75

1

.

00

U

Y

1

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

U

Y

2

E

(1

.

0)

|

|

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

U

Y

1

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

U

Y

2

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

U

Y

1

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

U

Y

2

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

U

Y

1

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

U

Y

2

0

10

20

30

40

50

κ

1

(

u

Y

)

Example 5 ( M tri ) . There exist BGMs in class B ( C 1 , 1) , for which the curvature of level sets are not properly defined at certain points. For example, an SCM M tri , which induces triangular distributions, like in Example 1. For this example, M tri will have the following function for Y :

Example 6 ( M flat ) . Here, we provide examples of SCMs in the class B ( C ∞ , d ) , for which the bundles of the level sets coincide for both treatments, i. e., { E ( y, a ) : y ∈ Y a } = { E ( y, a ′ ) : y ∈ Y a ′ } , and all the level sets are flat hypersurfaces ( κ = 0 ). For example, SCMs with the following functions for Y :

where g ( a, · ) is an invertible function in class C ∞ and w 1 , . . . , w d are coefficients from the linear combination. After a reparametrization, it is always possible to find an equivalent BGM with the function:

where F -1 ˜ U Y ( · ) is an inverse CDF of ˜ U Y = w 1 U Y 1 + · · · + w d U Y d . The smoothness of the inverse CDF then defines the smoothness of the BGM.

Example 7 ( M curv ) . This example is the generalization of the Example 6. Here, we also use the same bundles of the level sets for both treatments, but allow for the curvature of κ > 0 :

where g ( a, · ) is an invertible function in class C ∞ , and h ( · ) : [0 , 1] d → R is a function in class C ∞ with bounded by κ curvature of the level sets. In this case, we also can reparametrize the latent space with a non-linear transformation and find an equivalent BGM:

where F -1 ˜ U Y ( · ) is an inverse CDF of ˜ U Y = h ( U Y ) . The smoothness of the inverse CDF then analogously defines the smoothness of the BGM.

Example 8 ( M perp ) . We can construct an SCM of class B ( C ∞ , 2) , so that the level sets are all straight lines ( κ = 0 ) and perpendicular to each other for different a ∈ { 0 , 1 } , e. g.,

where g 1 ( · ) , g 2 ( · ) are invertible functions in class C ∞ . In this case, the ECOU for y ′ ∈ Y a ′ always evaluates to the conditional expectation

Similarly, the ECOT is E M perp ( Y | A = 0) . This result is, in general, different from the result of BGMs, which, e. g., would yield Q M a ′ → a ( y ′ ) = F a (0 . 5) for y ′ = F -1 a ′ (0 . 5) . Thus, point identification is not guaranteed with κ = 0 .

D Proofs

Uniformity of the latent noise. Classes of SCMs B ( C k , d ) assume independent uniform latent noise variables, i. e., U Y ∼ Unif(0 , 1) d . This assumption does not restrict the distribution of the latent noise. Namely, for a bivariate SCM M non-unif with d -dimensional non-uniform continuous latent noise, it is always possible to find an equivalent 7 SCM M unif of class B ( C k , d ) . This follows from the change of variables formula and the inverse probability transformation [25, 116]:

where ˜ U Y = T ( U Y ) , and T ( · ) : (0 , 1) d → R d is a diffeomorphism (bijective C 1 transformation) and a solution to the following equation:

For further details on the existence and explicit construction of T ( · ) , see [25]. Thus, f Y ( a, · ) is of class C min(1 ,k ) , as the composition of T ( · ) and ˜ f Y ( a, · ) .

In our CSM we require the functions of the SCMs to be of class C 2 . In this case, B ( C 2 , d ) also includes all the SCMs with non-uniform continuous latent noise with densities of class C 1 . This also follows from the change of variables theorem, see Eq. (62).

More general classes of functions, e. g., all measurable functions, or more general classes of distributions, e. g., distributions with atoms, fall out of the scope of this paper, even though we can find bijective (but non-continuous) transformations between probability spaces of the same cardinality (see the isomorphism of Polish spaces, Theorem 9.2.2 in [12]).

Lemma 1 (Observational distribution as a pushforward with f Y ) . Let M∈ B ( C 1 , d ) . Then, the density of the observational distribution, induced by M , is

where E ( y, a ) is a level set (preimage) of y , i. e., E ( y, a ) = { u Y ∈ [0 , 1] d : f Y ( a, u Y ) = y } , and H d -1 ( u Y ) is the Hausdorff measure (see Appendix B for the definition).

Proof. Lemma is a result of the coarea formula (see Eq. (16)):

where ( ∗ ) holds, as U Y is independent of A . In a special case, when we set d = 1 , we obtain a change of variables formula

Corollary 1 (Identifiable functions (BGMs)[93]) . The function f Y ( a, u Y ) can be identified (up to a sign) given the observational distribution P ( Y | a ) , induced by some SCM M of class B ( C 1 , 1) if it is strictly monotonous in u Y . In this case, the function is

where F -1 a is an inverse CDF of the observational distribution, and the sign switch corresponds to the strictly monotonically increasing and decreasing functions.

Proof. As d = 1 , we can apply the change of variables formula from Eq. (66):

where f -1 Y ( a, y ) is an inverse function with respect to u Y (it is properly defined, as the function is strictly monotonous). Thus, to find f Y , we have to solve the following differential equation [116]:

By the properties of derivatives, this equation is equivalent to

Since the latter holds for every y , we can integrate it from -∞ to f Y ( a, u Y ) :

Therefore, F a ( f Y ( a, u Y )) = ± u Y ∓ 0 . 5 + 0 . 5 .

Corollary 2 (Critical points of f Y ( a, · ) ) . Lemma 1 also implies that critical points of f Y ( a, u Y ) , i.e., { u Y ∈ [0 , 1] d : ∥∇ u Y f Y ( a, u Y ) ∥ 2 = 0 } , are mapped onto points y with infinite density. Therefore, the assumption, that the function is regular, namely ∇ u Y f Y ( a, u Y ) > 0 , is equivalent to the assumption of the continuous observational density with the finite density.

Lemma 2. Let M∈ B ( C 1 , d ) . Then, the density of the counterfactual outcome distribution of the [un]treated is

where δ ( · ) is a Dirac delta function, and the expected counterfactual outcome of the [un]treated, i.e., ECOU [ECOT], is

̸

where E ( y ′ , a ′ ) is a (factual) level set of y ′ , i. e., E ( y ′ , a ′ ) = { u Y ∈ [0 , 1] d : f Y ( a ′ , u Y ) = y ′ } and a ′ = a .

Proof. Both counterfactual queries can be inferred with the abduction-action-prediction procedure.

(1) Abduction infers the posterior distribution of the latent noise variables, conditioned on the evidence:

where ( ∗ ) holds, as U Y is independent of A .

(2)-(3) Action and prediction are then pushing forward the posterior distribution with the counterfactual function. For example, the density of the counterfactual outcome distribution of the [un]treated is

where ( ∗ ) holds due to the independence of Y from Y ′ and A ′ , conditional on U Y (see the parallel worlds network in Fig. 1). The inference of the ECOU [ECOT] is analogous

where ( ∗ ) holds due to the independence of Y from Y ′ and A ′ , conditional on U Y (see the parallel worlds network in Fig. 1), and due to the law of the unconscious statistician.

Corollary 3 (Identifiable counterfactuals (BGMs bounds)[93]) . The ECOU [ECOT] can be identified (up to a sign) given the observational distribution P ( Y | a ) , induced by some SCM M of class B ( C 1 , 1) if it is strictly monotonous in u Y . In this case, the ECOU [ECOT] is

Proof. The corollary is a result of the the Lemma 2 and the Corollary 1:

Therefore, the ECOU [ECOT] is

Theorem 1 (Non-informative bounds of ECOU (ECOT)) . Let the continuous observational distribution P ( Y | a ) be induced by some SCM of class B ( C ∞ , d ) . Let P ( Y | a ) have a compact support Y a = [ l a , u a ] and be of finite density P ( Y = y | a ) < + ∞ . Then, the ignorance interval for the partial identification of the ECOU [ECOT] of class B ( C ∞ , d ) , d ≥ 2 , has non-informative bounds: Q a ′ → a ( y ′ ) = l a and Q a ′ → a ( y ′ ) = u a .

Proof. Without the loss of generality, let us consider the lower bound of the ECOU [ECOT], namely,

The proof then proceeds in two steps. First, we prove the statement of the theorem for d = 2 , i. e., when latent noise is two-dimensional. Then, we extend it to arbitrary dimensionality.

Step 1 ( d = 2 ) . Lemma 2 suggests that to minimize the ECOU [ECOT], we have to either increase the proportion of the length (one-dimensional volume) of the factual level set, which intersects the bundle of counterfactual level sets, or change the counterfactual functions, by 'bending' the bundle of counterfactual level sets around a factual level set. Here, we focus on the second case, which is sufficient to construct an SCM with non-informative bounds.

Formally, we can construct a sequence of SCMs {M ε ( y ) non-inf : y ∈ Y a , 0 < ε ( y ) < 1 } of class B ( C ∞ , 2) , for which Q M ε ( y ) non-inf a ′ → a ( y ′ ) gets arbitrarily close to l a as y → l a and ε ( y ) → 0 .

For all {M ε ( y ) non-inf : y ∈ Y a , 0 < ε ( y ) < 1 } , we choose the same factual function where F -1 a ′ ( · ) is the inverse CDF of the observational distribution P ( Y | a ′ ) . This is always possible as a result of the Corollaries 1 and 2. Hence, all the level sets of the factual function are horizontal straight lines of length 1 (see Fig. 4), due to E ( y ′ , a ′ ) = { F -1 a ′ ( u Y 2 ) = y ′ } .

ε

(

y

)

induced CDF coincides with the observational CDF at every point, namely, ∀ y ∈ Y a : F M non-inf a ( y ) = F a ( y ) . (3) All the level sets have the nested structure, namely, all the points of the interior of E ε ( y ) ( y, a ) are mapped to y < y , and points of the exterior, to y > y . Additionally, for some fixed y ∈ Y a , we set ε = ε ( y ) as the proportion of the factual level set, E ( y ′ , a ′ ) , fully contained in the exterior of the counterfactual level set, E ( y, a ) . Thus, the interior of the counterfactual level set covers 1 -ε ( y ) of the factual level set.

Now, we construct the counterfactual functions in the following way. For a fixed ε ( y ) , we choose the level sets of the function f Y ( a, · ) , { E ε ( y ) ( y, a ) , y ∈ Y a } , so that they satisfies the following properties. (1) Each E ε ( y ) ( y, a ) intersects the factual level set E ( y ′ , a ′ ) at a certain point once and only once and ends at the boundary of the boundaries of the unit square. (2) Each E ε ( y ) ( y, a ) splits the unit square on two parts, an interior with the area F a ( y ) and an exterior, with the area 1 -F a ( y ) , respectively (where F a ( · ) is the CDF of the counterfactual distribution). This property ensures that the

Therefore, the ECOU [ECOT] for M ε ( y ) non-inf for some fixed y ∈ Y a is

which follows from the mean value theorem for integrals, since, for every y ∈ Y a , we can choose ε ( y ) arbitrarily close to zero, and, thus

Step 2 ( d > 2 ). The construction of the factual and counterfactual level sets extends straightforwardly to the higher dimensional case. In this case, the factual level sets are straight hyperplanes and the counterfactual level sets are hypercylinders, 'bent' around the factual hyperplanes.

Theorem 2 (Informative bounds with our CSM) . Let the continuous observational distribution P ( Y | a ) be induced by some SCM of class B ( C 2 , d ) , d ≥ 2 , which satisfies Assumption κ . Let P ( Y | a ) have a compact support Y a = [ l a , u a ] . Then, the ignorance interval for the partial identification of ECOU [ECOT] of class B ( C 2 , d ) has informative bounds, dependent on κ and d , which are given by Q a ′ → a ( y ′ ) = l ( κ, d ) > l a and Q a ′ → a ( y ′ ) = u ( κ, d ) < u a .

Proof. We will now show that ECOU [ECOT] always have informative bounds for every possible SCM satisfying the assumptions of the theorem. Without the loss of generality, let us consider the lower bound, i. e.,

The proof contains three steps. First, we show that the level sets of C 2 functions with compact support consist of the countable number of connected components, each with the finite Hausdorff

measure. Each connected component of almost every level set is thus a d -1 -dimensional Riemannian manifold. Second, we demonstrate that, under Assumption κ , almost all the level set bundles have a nested structure, are diffeomorphic to each other, have boundaries, and their boundaries always coincide with the boundary of the unit hypercube [0 , 1] d . Third, we arrive at a contradiction in that, to obtain non-informative bounds, we have to fit a d -ball of arbitrarily small radius to the interior of the counterfactual level set (which is not possible with bounded absolute principal curvature). Therefore, CSM with Assumption κ has informative bounds.

Step 1. The structure of the level sets of C 2 functions with the compact support (or more generally, Lipschitz functions) were studied in [1]. We employ Theorem 2.5 from [1], which is a result of Sard's theorem. Specifically, the level sets of C 2 functions with the compact support consist of the countable number of connected components, each with the finite Hausdorff measure (surface volume), namely, H d -1 ( E ( y, a )) < ∞ for a ∈ { 0 , 1 } . In Example 3, we provide an SCM with the level sets with indefinitely many connected components. Furthermore, connected components of almost every level set are ( d -1) -dimensional Riemannian manifolds of class C 2 . Some points y have ( d -2) - and lower dimensional manifolds, but their probability measure is zero (e. g., the level sets of the bounds of the support, E ( l a , a ) and E ( u a , a ) ).

Step 2. The Assumption κ assumes the existence of the principal curvatures at every point of the space (except for the boundaries of the unit hypercube [0 , 1] d ), for both the factual and the counterfactual function. Thus, the functions of the SCMs are regular, as otherwise the principal curvatures would not be defined at the critical points (see Eq. (17)). As a result of the fundamental Morse theorem (see Appendix B), almost all the level set bundles are nested and diffeomorphic to each other. Specifically, the level set bundles { E ( y, a ) : y ∈ ( l a , u a ) } for a ∈ { 0 , 1 } . The latter exclude the level sets of the bounds of the Y a , i. e., E ( l a , a ) and E ( u a , a ) for a ∈ { 0 , 1 } , which in turn are laying completely in the boundary of the unit hypercube [0 , 1] d . Another important property is that all the level sets have a boundary (otherwise, due to the diffeomorphism, the critical point would exist in their interior) and this boundary lies at the boundary of the unit hypercube [0 , 1] d .

Step 3. Let us fix the factual level set, E ( y ′ , a ′ ) . By the observational distribution constraint, we know that the interior occupies F a ′ ( y ′ ) of the total volume of the unit hypercube [0 , 1] d , where F a ′ ( · ) is the CDF of the observational distribution. Also, its absolute principal curvatures are bounded by κ .

Then, let us assume there exists a counterfactual function, which obtains non-informative bounds of ECOU [ECOT]. For this, a counterfactual level set for some y close to l a has to contain exactly F a ( y ) volume in the interior. At the same time, this level set has to contain as much of the surface volume of the factual level set, H d -1 ( E ( y ′ , a ′ )) , as possible in its interior. This is only possible by 'bending' the counterfactual level set along one of the directions, so that the boundaries of the counterfactual level set lay at the boundaries of the unit hypercube, [0 , 1] d . Due to the bound on the maximal absolute principal curvature, we have to be able to fully contain a d -ball with radius 1 κ inside the interior of the counterfactual level set. This d -ball occupies the volume

where Γ( · ) is a Gamma function. Therefore, the volume of the interior of the counterfactual level set has to be at the same time arbitrarily close to zero, as F a ( y ) → 0 as y → l a , and at least of the volume of the d -ball with radius 1 κ . Thus, we arrived at the contradiction, which proves the theorem.

Corollary 4 (Monotonicity wrt. to κ ) . By construction, it is guaranteed that, with decreasing κ , the bounds for ECOU [ECOT] could only shrink, i. e., for κ 1 > κ 2 : l ( κ 1 , d ) ≤ l ( κ 2 , d ) and u ( κ 1 , d ) ≥ u ( κ 2 , d ) . This happens, as we decrease the class of functions in the constraints of the optimization problem from Definition 2.

Corollary 5 (Sufficiency of d = 2 ) . In general, upper and lower bounds under assumed CSM ( κ ) are different for different d 1 , d 2 ≥ 2 , e. g., l ( κ, d 1 ) ≤ l ( κ, d 2 ) and u ( κ, d 1 ) ≥ u ( κ, d 2 ) for d 1 > d 2 . This follows from the properties of the high-dimensional spaces, e. g., we one can fit more d -balls of the fixed radius to the hypercube [0 , 1] d with the increase of d . Nevertheless, without the loss of generality and in the absence of the information on the dimensionality, we can set d = 2 , as we still cover the entire identifiability spectrum by varying κ .

Lemma 3 (BGMs-EQTDs identification gap of CSM( κ = 0 )) . Let the assumptions of Theorem 2 hold and let κ = 0 . Then the ignorance intervals for the partial identification of ECOU [ECOT] of class B ( C 2 , d ) are defined by min/max between BGMs bounds and expectations of quantile-truncated distributions (EQTDs):

where E ( Y | Y < · ) , E ( Y | Y > · ) are expectations of truncated distributions.

Proof. We provide an informal proof for d = 2 based on geometric intuition. When κ = 0 , all the level sets are straight lines (hyperplanes for d > 2 ). It is possible to characterize all the possible relative arrangements of the factual level set and the bundle of counterfactual level sets, and the corresponding values of ECOU [ECOT]. For a particular arrangement, displayed in Fig. 11, we see that the area of the interior of the factual level set is vol 2 = F a ′ ( y ′ ) . At the same time, the factual level set crosses the counterfactual bundle corresponding to the interval [ l a , F -1 a (2 F a ′ ( y ′ ))] , thus the counterfactual distribution of [un]treated is simply a 2 F a ′ ( y ′ ) -quantile-truncated distribution, i. e., P ( Y | a, Y < F -1 a (2 F a ′ ( y ′ ))) . Therefore, the bounds of the ignorance interval of ECOU [ECOT] are defined by the min/max of both BGMs and expectations of quantile-truncated distributions (EQTDs).

0.0

2

F

a

′

(

y

′

)

1.0

0.0

1.0

U

Y

2

vol

2

=

F

a

′

(

y

′

)

U

Y

1

E Discussion of extensions and limitations

Sharp bounds. To obtain the sharp bounds under CSM ( κ ), one has to exactly solve the constrained variational problem task formulated in the Definition 2 in the space of the SCMs, which satisfy the Assumption κ . This is a very non-trivial task, the distributional constraint and the constraint of level sets of bounded curvature both include the partial differential equation with the Hausdorff integrals. For this task, in general, an explicit solution does not exist in a closed form (unlike, e. g., the solution of the marginal sensitivity model [126]).

Extension of CSM to discrete treatments and continuous covariates. Our CSM ( κ ) naturally scales to bivariate SCMs with categorical treatments, A ∈ { 0 , . . . , K } ; and multivariate Markovian SCMs with additional covariates, X ∈ R m , which are all predecessors of Y . These extensions, nevertheless, bring additional challenges from estimating P ( Y | A,X ) from the observational (interventional) data. Hence, additional smoothness assumptions are required during modeling.

Extension of CSM to semi-Markovian SCMs. CSM ( κ ) is limited to the Markovian SCMs. For semi-Markovian SCMs, e. g., a setting of the potential outcomes framework [111], additional assumptions are needed. Specifically, in semi-Markovian SCMs, the latent noise variables could be shared for X and Y , and this complicates counterfactual inference. Future work may consider incorporating other sensitivity models to our CSM ( κ ), e. g., marginal sensitivity model, MSM ( Γ ) [126], as described in Fig. 12.

CSM (

)

MSM (

)

F Details on architecture and inference of Augmented Pseudo-Invertible Decoder

Our APID provides an implementation of CSM ( κ ) for SCMs of class B ( C 2 , 2) . In the following, we list several important requirements, that the underlying probabilistic model has to satisfy. Then, we explain how deep normalizing flows [107, 125] meet these requirements.

A probabilistic model has to explicitly model an arbitrary function ˆ f Y ( a, u Y ) , u Y ∈ [0 , 1] 2 , of class C 2 . Normalizing flows with free-form Jacobians can implement arbitrary invertible C 2 transformations from ˆ F a : R 2 → R 2 . Then, by omitting one of the outputs, we can model functions f Y ( a, · ) : [0 , 1] 2 →Y a ⊂ R . We discuss normalizing flows with free-form Jacobians in Sec. F.1.

A model has to fit the observational (interventional) distribution, given a sample from it, D = { A i , Y i } n i =1 ∼ P ( A,Y ) . This is possible for the normalizing flows, as they are maximizing the log-likelihood of the data, ˆ P ( Y = Y i | A = A i ) , directly with the gradientbased methods. Importantly, we also employ variational augmentations [22] to evaluate the log-likelihood. We discuss this in detail in Sec. F.2.

A probabilistic model should be able to perform the estimation of ECOU [ECOT], through abduction-action-prediction steps. In normalizing flows, this can be achieved with the help of the variational augmentations [22]. Specifically, for evidence, y ′ and a ′ , our proposed APID can infer arbitrarily many points from the estimated factual level set ˆ E ( y ′ , a ′ ) , which follow the estimated posterior distribution of the latent noise, i. e., ˆ P ( U Y | y ′ , a ′ ) . We discuss this in detail in Sec. F.2.

All the level sets of the modeled function need to have a bounded curvature. This is possible for normalizing flows as a deep learning model. Namely, the curvature κ 1 ( u Y ) can be evaluated via automatic differentiation tools and added the loss of the model. We discuss this in detail in Sec. F.3.

F.1 Choice of a normalizing flow with free-form Jacobians

Normalizing flows (NFs) [107, 125] differ in how flexible the modeled transformations in high dimensions are. Some models, like planar NF [107] or Sylvester NF [128], only allow for transformations with low-rank Jacobians. Other models employ masked auto-regressive networks [72] or coupling blocks [33, 34] to construct transformations with lower-triangular (structured) Jacobians.

Recently, several models were proposed for modeling free-form Jacobian transformations, e. g., (i) continuous NFs [23, 45] or (ii) residual NFs [9, 24]. However, this flexibility comes at a computational cost. For (i) continuous NFs, we have to solve a system of ordinary differential equations (ODEs) for every forward and reverse transformation. As noted by [45], numerical ODE solvers only work well for the non-stiff differential equations (=for non-steep transformations). For (ii) residual NFs, the computational complexity stems from the evaluation of the determinant of the Jacobian, which is required for the log-likelihood, and, from the reverse transformation, where we have to employ fixed point iterations.

We experimented with both continuous NFs and residual NFs but we found the latter to be more stable. The drawbacks of residual NFs can be partially fixed under our CSM ( κ ). First, the determinant of the Jacobian can be evaluated exactly, as we set d = 2 . Second, the numerical complexity of the reverse transformation 8 can be lowered by adding more residual layers, so that each layer models less steep transformation and the total Lipschitz constant is larger for the whole transformation. Hence, in our work, we resorted to residual normalizing flows [9, 24].

F.2 Variational augmentations and pseudo-invertibility

Variational augmentations were proposed for increasing the expressiveness of normalizing flows [22]. They augment the input to a higher dimension and then employ the invertible transformation of

the flow. Hence, normalizing flows with variational augmentations can be seen as pseudo-invertible probabilistic models [10, 54].

We use variational augmentations to model the inverse function, ˆ f Y ( a, ˙ ) , i. e., sample points from the level sets. For this, we augment the (estimated) outcome Y ∈ Y with Y aug ∼ N ( g a ( Y ) , ε 2 ) ∈ R , where g a ( · ) is a fully-connected neural network with one hidden layer and parameters θ a , and ε 2 is a hyperparameter. As such, our proposed APID models ˆ f Y ( a, · ) through a two-dimensional transformation ˆ F a = ( ˆ f Y aug ( a, · ) , ˆ f Y ( a, · )) : [0 , 1] 2 → R ×Y a . Variational augmentations facilitate our task in two ways.

(i) They allow evaluating the log-likelihood of the data via

where β a are the parameters of the residual normalizing flow, and N ( · ; g a ( Y i ) , ε 2 ) is the density of the normal distribution. Here, we see that, by increasing the ε 2 , the variance of the sample estimate of the log-likelihood of the data increases. On the other hand, for ε 2 → 0 , the transformations of the residual flow become steeper, as we have to transport the point mass to the unit square, [0 , 1] 2 .

(ii) Variational augmentations enable the abduction-action-prediction to estimate ECOU [ECOT] in a differential fashion. At the abduction step, we infer the sample from the posterior distribution of the latent noise, defined at the estimated factual level set ˆ E ( y ′ , a ′ ) . For that, we variationally augment the evidence y ′ with b samples from ( y ′ , { Y ′ aug ,j } b j =1 ∼ N ( g a ′ ( y ′ ) , ε 2 )) , and, then, transform them to the latent noise space with the factual flow

Then, the action step selects the counterfactual flow, ˆ F a ( · ) , and the prediction step transform the abducted latent noise with it:

In the end, ECOU [ECOT] is estimated by averaging ˆ Y j :

In our APID, the parameters of the residual normalizing flow, β a , and the parameters of the variational augmentations, θ a , are always optimized jointly. We use the reparametrization trick, to back-propagate through sampling of the augmentations.

F.3 Penalizing curvatures of the level sets

Although there exist deep learning models which explicitly bound the curvature of the modeled function (e. g., [124]), none of the works (to the best of our knowledge) did enforce the curvature of the level sets of the modeled function.

In our APID, the curvature κ 1 ( u Y ) is evaluated using automatic differentiation exactly via Eq. (11) and then incorporated into the loss. As the calculation of the second derivatives is a costly operation, we heuristically evaluate the curvature only at the points of the counterfactual level set, which corresponds to the estimated ECOU [ECOT], namely ( u Y 1 , u Y 2 ) ∈ ˆ E ( ˆ Q a ′ → a ( y ′ ) , a ) . For this, we again use the variationally augmented sample of size b for ˆ y = ˆ Q a ′ → a ( y ′ ) : (ˆ y, { ˆ Y aug ,j } b j =1 ∼ N ( g a (ˆ y ) , ε 2 )) . Then, we use the counterfactual flow (such as in Eq. (94))

while the curvature has to be only evaluated at b points given by

G Details on training of Augmented Pseudo-Invertible Decoder

G.1 Training objective

Our APID satisfies all the requirements set by CSM ( κ ), by combining several losses with different coefficients (see the overview in Fig. 13). Given observational data D = { A i , Y i } n i =1 drawn i.i.d. and a counterfactual query, Q a ′ → a ( y ′ ) , for the partial identification, APID minimizes the following losses:

(1) Negative log-likelihood loss aims to fit the observational data distribution by minimizing the negative log-likelihood of the data, D a = { A i = a, Y i } , modeled by APID. To prevent the overfitting, we use noise regularization [110], which adds a normally distributed noise: ˜ Y i = Y i + ξ i ; ξ i ∼ N (0 , σ 2 ) , where σ 2 > 0 is a hyperparameter. Then, the negative log-likelihood loss for a ∈ { 0 , 1 } is

where the log-likelihood is evaluated according to the Eq. (93).

(2) Wasserstein loss prevents the posterior collapse [26, 132] of the APID, as the sample { Y i } is not guaranteed to cover the full latent noise space when mapped with the estimated inverse function. We thus sample b points from the latent noise space, { ( U Y 1 ,j , U Y 2 ,j ) } b j =1 ∼ Unif(0 , 1) 2 and map with the forward transformation F a ( · ) for a ∈ { 0 , 1 } via

Then, we evaluate the empirical Wasserstein distance

where ˆ F -1 ˆ Y ( · ) and ˆ F -1 Y ( · ) are empirical quantile functions, based on samples { A i = a, Y i } and { ˆ Y a,j } , respectively.

(3) Counterfactual query loss aims to maximize/minimize ECOU [ECOT]:

where ∓ changes for maximization/minimization correspondingly, ˆ Q a ′ → a ( y ′ ) is evaluated as described in the Eq. (96), and Softplus( x ) = log(1 + exp( x )) . We use the softplus transformation to scale the counterfactual query logarithmically so that it matches the scale of the negative log-likelihood. We also block the gradients and fit only the counterfactual flow, when maximizing/minimizing ECOU [ECOT], to speed up the training.

(4) Curvature loss penalizes the curvature of the level sets of the modeled counterfactual function ˆ f Y ( a, · ) :

where { κ 1 (ˆ u Y 1 ,j , ˆ u Y 2 ,j ) } b j =1 are defined in Sec. F.3.

All the losses are summed up to a single training objective

G.2 Training algorithm and hyperparameters

Training algorithm. The training of APID proceeds in three stages; see the pseudocode in Algorithm 1. At a burn-in stage ( n B = 500 training iterations), we only fit two residual normalizing flows (one for each treatment) with L NLL and L W . Then, we copy the counterfactual flow and variational

Residual NF

Residual NF

Residual NF

(3) Counterfactual query loss

(4) Curvature loss

Residual NF

(1) Negative log-

likelihood loss

Residual NF

loss

Fully-connected network

NF

Normalizing flow,

parametrized by

Inputs / outputs with

corresponding distributions

Losses

Noise regularization

Connections

Gradient blocking

Legend

augmentation parameters twice, for the task of maximization/minimization, respectively. Also, we freeze the factual flow parameters. At a query stage ( n Q = 100 training iterations), we, additionally to previous losses, enable the counterfactual query loss, λ Q L Q . During this stage, two counterfactual flows are able to realign and start 'bending' their level sets around the frozen factual level set. Ultimately, at the last curvature-query stage ( n CQ = 500 training iterations), all the parts of the loss in Eq. (104) are enabled. After training is over, we report the upper and lower bounds for the counterfactual query as an output of the APID with exponential moving average (EMA) of the model parameters [105]. EMA of the model parameters allows us to reduce the variance during training and is controlled via the smoothness hyperparameter γ = 0 . 99 .

Hyperparameters. We use the Adam optimizer [71] with a learning rate η = 0 . 01 and a minibatch size of b = 32 to fit our APID. This b = 32 is also used for all the sampling routines at other losses. For the residual normalizing flows, we use t = 15 residual transformations, each with h t = 5 units of the hidden layers. We set the relative and absolute tolerance of the fixed point iterations to 0 . 0001 and a maximal number of iterations to 200 . For variational augmentations, we set the number of units in the hidden layer to h g = 5 , and the variance of the augmentation to ε 2 = 0 . 5 2 . For the noise regularization, we chose σ 2 = 0 . 001 2 . This configuration resulted in a good fit (with respect to the test log-likelihood) for all the experiments, and, thus, we did not tune hyperparameters. We varied the coefficients for the losses, λ Q and λ κ , and report the results in Sec. 6 and Appendix H.

Algorithm 1 Training algorithm of APID

1: Input. Counterfactual query Q a ′ → a ( y ′ ) ; training dataset D = D 0 ∪ D 1 ; number of training iterations n B , n Q , n CQ ; minibatch size b ; learning rates η ; intensity of the noise regularization σ 2 ; the variance of the augmentation ε 2 ; EMA smoothing γ ; loss coefficients λ Q and λ κ . 2: 3: Init. Flows parameters: residual NFs ( β 0 , β 1 ) and variational augmentations ( θ 0 , θ 1 ). 4: for k = 0 to n B do ▷ Burn-in stage 5: for a ∈ { 0 , 1 } do 6: { A i = a, Y i } b i =1 ← minibatch of size b from D a 7: ξ i ∼ N (0 , σ 2 ); ˜ Y i ← Y i + ξ i ▷ Noise regularization 8: ˜ Y aug ,i ∼ N ( g a ( ˜ Y i ) , ε 2 ) ▷ Variational augmentations 9: L NLL ( β a , θ a ) ←-1 n ∑ b i =1 log ˆ P β a ,θ a ( Y = ˜ Y i | a ) ▷ (1) Negative log-likelihood loss 10: { ( U Y 1 ,j , U Y 2 ,j ) } b j =1 ∼ Unif(0 , 1) 2 11: { ( ˆ Y aug ,a,j , ˆ Y a,j ) } b j =1 ← ˆ F a ( { ( U Y 1 ,j , U Y 2 ,j ) } b j =1 ) 12: L W ( β a ) ← ∫ 1 0 | ˆ F -1 ˆ Y a ( q ) -ˆ F -1 Y a ( q ) | d q ▷ (2) Wasserstein loss 13: L ← L NLL ( β a , θ a ) + L W ( β a ) 14: optimization step for β a , θ a wrt. L with learning rate η 15: end for 16: end for 17: Output. Pretrained flows parameters: residual NFs ( β 0 , β 1 ) and variational augmentations ( θ 0 , θ 1 ) 18: 19: β a , β a ← β a ; θ a , θ a ← θ a ; ▷ Copying counterfactual flow parameters 20: [ ˆ l a , ˆ u a ] ← [min i ( Y i ); max i ( Y i )] 21: for k = 0 to n Q + n CQ do ▷ Query & curvature-query stages 22: for ( β a , θ a ) ∈ { ( β a , θ a ) , ( β a , θ a ) } do 23: { A i = a, Y i } b i =1 ← minibatch of size b from D a 24: same procedure as in lines 7-12 to evaluate L NLL ( β a , θ a ) and L W ( β a ) 25: L ← L NLL ( β a , θ a ) + L W ( β a ) 26: ( y ′ , { Y ′ aug ,j } b j =1 ∼ N ( g a ′ ( y ′ ) , ε 2 )) ▷ Variational augmentations 27: { (˜ u Y 1 ,j , ˜ u Y 2 ,j ) } b j =1 ← ˆ F -1 a ′ ( { ( y ′ , Y ′ aug ,j ) } b j =1 ) 28: { ( ˆ Y aug ,j , ˆ Y j ) } b j =1 ← ˆ F a ( { (˜ u Y 1 ,j , ˜ u Y 2 ,j ) } b j =1 ) 29: ˆ Q a ′ → a ( y ′ ) = 1 b ∑ b j =1 ˆ Y j ▷ Estimated ECOU [ECOT] 30: if ˆ Q a ′ → a ( y ′ ) / ∈ [ ˆ l a , ˆ u a ] and k < n Q then 31: continue ▷ Non-informative bounds 32: else 33: L Q ( β a ′ , θ a ′ , β a ) ← Softplus( ∓ ˆ Q a ′ → a ( y ′ )) ▷ (3) Counterfactual query loss 34: L ← L + λ Q L Q ( β a ′ , θ a ′ , β a ) 35: end if 36: if k ≥ n Q then 37: ˆ y ← ˆ Q a ′ → a ( y ′ ) ; (ˆ y, { ˆ Y aug ,j } b j =1 ∼ N ( g a (ˆ y ) , ε 2 )) ▷ Variational augmentations 38: { (ˆ u Y 1 ,j , ˆ u Y 2 ,j ) } b j =1 ← ˆ F -1 a ( { (ˆ y, ˆ Y aug ,j ) } b j =1 ) 39: L κ ( β a , θ a ) ← 1 b ∑ b j =1 κ 1 (ˆ u Y 1 ,j , ˆ u Y 2 ,j ) ▷ (4) Curvature loss 40: L ← L + λ κ L κ ( β a , θ a ) 41: end if 42: optimization step for β a , θ a wrt. L with learning rate η 43: EMA update of the flow parameters ( β a , θ a ) 44: end for 45: end for 46: Output. ˆ Q a ′ → a ( y ′ ) evaluated with smoothed with EMA ( β a , θ a ) and ( β a , θ a )

H Synthetic experiments

H.1 Datasets

We conduct the experiments with datasets, drawn from two synthetic datasets. In the first dataset, we use

and, in the second,

Importantly, the ground truth SCMs (and their κ ) for both datasets remain unknown, and we only have access to the observational distributions. This is consistent with the task of partial counterfactual identification.

H.2 Additional results

In Fig. 14, we provide additional results for the synthetic experiments. For our APID, we set λ Q = 1 . 0 and vary λ κ ∈ { 0 . 0 , 0 . 5 , 1 . 0 , 5 . 0 } . Here, we see that estimated by APID bounds are located closer to (i) the theoretic upper and lower bounds of BGMs and (ii) closer to each other.

-

2

.

0

-

1

.

5

-

1

.

0

-

0

.

5

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

y

′

-

4

-

3

-

2

-

1

0

1

2

3

4

ˆ

Q

0

→

1

(

y

′

)

P

(

Y

|

0) =

P

(

Y

|

1) =

N

(0

,

1)

-

2

.

0

-

1

.

5

-

1

.

0

-

0

.

5

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

y

′

-

4

-

3

-

2

-

1

0

1

2

3

4

ˆ

Q

0

→

1

(

y

′

)

P

(

Y

|

0) = Mixture

0

(

Y

)

P

(

Y

|

1) = Mixture

1

(

Y

)

APID(

λ

κ

= 0

.

0)

APID(

λ

κ

= 0

.

5)

APID(

λ

κ

= 1

.

0)

APID(

λ

κ

= 5

.

0)

BGMs

EQTDs

[ ˆ

l

1

,

ˆ

u

1

]

H.3 Runtime

Table 2 provides the duration of one training iteration of our APID for different training stages. Namely, we report the mean and std. across all the experiments (total of 720 runs) for the burn-in, the query, and the curvature-query stages. In our experiments, the long training times are attributed to the reverse transformations in the residual normalizing flows (as they require fixed point iterations) and to the computation of the Hessian to evaluate the curvatures.

I Case study with real-world data

In our case study, we want to analyze retrospective 'what if' questions from the COVID-19 pandemic. Specifically, we ask what the impact on COVID-19 cases in Sweden would have been, had Sweden implemented a stay-home order (which Sweden did not do but which is under scrutiny due to post-hoc legal controversies).

I.1 Dataset

We use multi-country data from [7]. 9 It contains weekly-averaged number of recorded COVID-19 for 20 Western countries. The outcome, Y , is the (relative) case growth per week (in log), namely, the ratio between new cases and cumulative cases. The treatment, A ∈ { 0 , 1 } , is a binary variable referring to either no lockdown or enforcement of a stringent lockdown, respectively. We assume the data is i.i.d. and comes from a randomized control trial, i. e., there is no confounding between Y and A . We filtered out the weeks where the cumulative number of cases is smaller than 50 , which resulted in n = n 0 + n 1 = 136 + 124 observations in total, for both treatments, respectively. We plot a histogram of case growth per week depending on the treatment in Fig. 15, i. e., empirical densities of distributions ˆ P ( Y | A = 0) and ˆ P ( Y | A = 1) .

-

7

-

6

-

5

-

4

-

3

-

2

-

1

0

Observed case growth,

y

0

.

00

0

.

05

0

.

10

0

.

15

0

.

20

0

.

25

0

.

30

0

.

35

0

.

40

A

= 0 (no lockdown)

A

= 1 (stringent lockdown)

Weassume that the observational distributions P ( Y | a ) are induced by some (unknown) SCM of class B ( C 2 , 2) . Then, we fit our APID the same way as in synthetic experiments (see Appendix G). We additionally report the BGMs-EQTDs identification gap based on empirical CDF of the observational distributions.

I.2 Results

Fig. 16 shows the results for (upper) partial counterfactual identification of the ECOU for our method. Hence, the ECOU reports for the expected counterfactual case growth for countries without stay-home orders, i. e., the weekly case growth (in log) if a stay-home order would have actually been enforced. We also highlight the upper bounds on the ECOU for Sweden during weeks 10-13 of 2020, where no lockdown was implemented. Evidently, the ECOU would lie below the decision boundary for BGMs-EQTDs and for certain levels of λ κ . This implies that Sweden could have had significantly lower case growth had it implemented a stay-home order, even when allowing for some level of non-linearity between outcome and independent latent noises that interact with the treatment, e.g., cultural distinctions, level of trust in government, etc. The results of the COVID-19 case study demonstrate that our CSM and its instantiation with APID could be applied for decision-making with real-world data.

-3.0

-2.5

-2.0

-1.5

-1.0

-0.5

week 10

week 11

week 12

week 13

Observed case growth,

y

′

-

4

.

5

-

4

.

0

-

3

.

5

-

3

.

0

-

2

.

5

-

2

.

0

-

1

.

5

-

1

.

0

-

0

.

5

Counterfactual case growth,

ˆ

Q

0

→

1

(

y

′

)

APID(

λ

κ

= 0

.

5)

APID(

λ

κ

= 1

.

0)

APID(

λ

κ

= 5

.

0)

APID(

λ

κ

= 10

.

0)

BGMs

EQTDs

[ ˆ

l

1

,

ˆ

u

1

]

Observed case growth for country = Sweden

Decision boundary ( ˆ

Q

0

→

1

(

y

′

) =

y

′

)

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the partial identification of counterfactual probabilities from observational data. More specifically, the authors consider the standard bandit model containing a treatment A and a reward Y; exogenous variables U exist affecting both A and Y. The treatment A is a binary variable; while the reward Y could take any arbitrary real value bounded in an interval [l, u]. The goal is to evaluate the counterfactual probability P(y_a | a’, y’) from the observational data drawn from P(Y, A). The author first shows that this learning setting is ill-posed without any additional assumptions. That is, the learner could only obtain non-informative bound [l, u] from the observational data. This observation suggests that one should explore additional parametric assumptions to obtain informative bounds. More specifically, the authors propose a sensitivity model called Curvature Sensitivity Model. They further propose an implementation of our Curvature Sensitivity Model in the form of a novel deep generative model, called Augmented Pseudo\u0002Invertible Decoder.\n\nSTRENGTHS:\n- This paper is well written and well organized. The authors provide extensive examples and graphical illustrations. This is much appreciated.\n\n- Bounding counterfactual probabilities over a continuous outcome from the observational data is a challenging problem in causal inference. The authors study partial identification in multi-armed bandit models, which are widely applied in reinforcement learning literature. The target query P(y_a | a’, y’) is called the expected counterfactual outcome of un]treated ECOU; its applications include fairness analysis in machine learning (Kushner et al., 2018). Since ECOU is generally non-identifiable and existing bounding algorithms focus on discrete domains, this paper could have an impact on the causal inference community. \n\n- The non-informative bounding result in Theorem 1 is interesting. As far as I am aware, this result is novel. The authors also provide a novel sensitivity model of Curvature Sensitivity Model (CSM), which permits one to obtain informative bounds by bounding the curvature of level sets of the functions. They further show that CSMs generalize existing parametric assumptions for point counterfactual identification methods when the bound of the curvature is set to zero.\n\nWEAKNESSES:\n- This paper studies the bounding of a specific counterfactual probability, called ECOU in canonical bandit models. However, some statements in the abstract sound more general than it is. For instance, the paper states \"We prove that, in general, the ignorance interval of the counterfactual queries has non-informative bounds, already when functions of structural causal models are continuously differentiable.\" Based on my reading, the non-informative bounds in Theorem 1 only apply to bounding ECOU in bandit models. However, if we consider bounding a causal effect P(y_a) in the bandit model, Manski's bound (1990) has shown to be informative and applicable to continuous reward signals. \n\n- The simulation plots could be improved. For instance, it is unclear what the ground truth ECOUs are in Figure 7. Without that, it is hard to evaluate the validity of the proposed bounding strategy.\n\nQUESTIONS:\n- What is the actual ECOU in Figure 7?\n\nLIMITATIONS:\nAssumptions behind the proposed algorithms are explicitly stated. This paper is theoretical and its long-term societal impact is not immediate to see.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors study the problem of counterfactual identification of continuous outcome variable with a specific causal graph. They show that the expected counterfactual outcome of (un)treated has non-informative bound when the function class is arbitrary smooth. Then, they introduce a new model called CSM that give informative bounds. Finally, they propose a deep generative based method for partial counterfactual inference in CSM.\n\nSTRENGTHS:\nThey study an important problem. The paper is well-written and the Appendix is educational. \n\nThe analysis is quite interesting as it combines differntial geometry and causal inference. This opens a new door to improve the current results in causal inference and likely to develop a general framework for counterfactual analysis and more.\n\nThe presented examples in both the main text and the appendix are informative. \n\nTheoretical results are quite interesting. In particular, Theorem 1 and 2.\n\nWEAKNESSES:\nThe main question is about the generalizability of this method to more complex causal grpahs. It is not hard to see that as the causal diagram becomes a bit more complex, then applying similar analysis might be quite challenging.\n\nIt is a bit confusing that CSMs with arbitrary $\\kappa$ lead to informative bounds, i.e., $l(\\kappa,d)>l_a$ while, for instance, $\\mathcal{B}(C^2,d)$ is not informative. Because, as it is also mentioned in the text, as $\\kappa\\rightarrow\\infty$, CSM$(\\kappa)\\rightarrow \\mathcal{B}(C^2,d)$. What is the interpretaion of this?\n\nThe proof of Theorem 2 is of existance type that is the form of the informative bounds and how they depend on $\\kappa$ and $d$ are not given. But is it still possible to say something about their dependencies, e.g., $l(\\kappa,d)$ is decreasing with respect to $\\kappa$?\n\nAlthough, the augmented Pseudo-Invertible decoder method is a fine application of deep generative methods but it is not clear how sensitive it is with respect to the model hyperparametersa and sample size.\n\nQUESTIONS:\nSee above\n\nLIMITATIONS:\nThe authors addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the partial identifiability of counterfactual queries and proposes an interesting proof of general non-identifiability resulting in non-informative bounds. The authors build upon the intuition used for this proof to propose a new class of structural causal model using principal curvatures. Under this assumption, the authors can achieve informative partial counterfactual identification. The authors then propose an architecture that leverages this assumption.\n\nSTRENGTHS:\nThis paper proposes a systematic and rigorous study of the partial counterfactual estimation problem. This is exemplified by the breadth and depth of literature search and by the organization of the related works. While general non-identifiabilty of counterfactuals is known, theorem 1 shows that constraints on the smoothness of repsonse function does not improve the identifiability, which I think is novel. \n\nThe proposed curvature sensitivity model, motivated by Theorem 1, is shown to benefit from the identifiability properties which are theoretically demonstrated. This causal model structure seems to be fairly general and appears to generalize concepts such as non-intersection of response surfaces (which is pointed at in the experiments). \n\nThe authors further show that the theoretical properties of the CSM can be leveraged empirically be proposing a new architecture.\n\nWEAKNESSES:\nWhile there is significant effort in bringing intuition regarding how to construct counterfactuals (bending of the counterfactual level sets), I believe more intuition could be given regarding the general applicability of the CSM in practice. It is unclear when it seems reasonable to assume this model would be valid, and hence the partial counterfactual estimation correct. I appreciate that this is a general problem in counterfactual estimation but more insight into the applicability would be welcome.\n\nWhile there is a strong theoretical work behind this paper, it is more frugal in terms of experiments. I would encourage the authors to provide more evaluations of their approach, with different types of repsonse surfaces and noise levels. The design of more experiments could potentially lead to more intuition in the assumptions of the method.\n\nQUESTIONS:\nCould the author motivate the CSM assumption from a more practical perspective ? \n\nExtending the experiments section would help strenghten the paper.\n\nLIMITATIONS:\nLimitations and assumptions are clearly stated.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a method for obtaining bounds on continuous counterfactual outcomes.  In general, level-3 counterfactual outcomes (what would have happened under action $a'$, given knowledge of what did happen under action $a$) are not identifiable from either interventional or observational data.  Prior work makes structural assumptions on the underlying structural causal model (e.g., monotonicity with respect to a single unobserved noise variable) to obtain identification, while this work instead makes an assumption on the principal curvature of level sets with respect to two unobserved noise variables.  A computational approach is provided to obtain bounds based on this principal curvature assumption, and is demonstrated on two simple synthetic cases.\n\nSTRENGTHS:\nIn terms of originality and clarity, this paper presents a novel approach to obtaining bounds on continuous counterfactual outcomes, and the contribution is clearly placed in the context of related work.  Having seen a few papers on the general topic of partial identification, I found Section A (Extended Related Work) in the Appendix to be a well-written, well-researched, and generally helpful overview of how the current work differs from prior approaches, primarily in the problem considered.\n\nMoreover, in terms of clarity and quality, I found Example 1 (Figure 3) to be a very helpful example of counterfactual non-identifiability, and a good explanation of the intuition for how \"bending\" the counterfactual level sets around the factual level set leads to a different distribution of counterfactual outcomes.\n\nWEAKNESSES:\n## Summary \nI see two major weaknesses in this paper, and look forward to discussion with the authors during the response period if I've misunderstood anything below.  Both weaknesses relate to the significance of the approach.\n\n1. How should domain experts choose the parameter $\\kappa$?  Choosing the right bound on the principal curvature of level sets doesn't seem like something a domain expert would plausibly be able to do.\n2. Even if we knew the right $\\kappa$, can we guarantee that the computational approach will give bounds that are at least conservative (e.g., outer bounds on the true bounds implied by $\\kappa$) if not tight?\n\nOverall, my current score incorporates my view that these two weaknesses are genuinely challenging problems, so I am somewhat sympathetic to the paper in its current state, but I would at least like to see more discussion of these challenges in the main paper.\n\nI'll discuss these weaknesses in more detail below.  I am basing my score primarily on these points, though I also had an important clarifying question regarding the ability of the approach to \"cover the entire identifiability spectrum\". \n\n## Details of major concerns \n\n(1) **Domain Knowledge for $\\kappa$?** One of the appealing properties of the given approach is that it can interpolate between the extremes of identifiability (though see my clarifying question (3) below).  However, there are many trivial ways to interpolate between e.g., point-identification and the non-informative bounds (e.g., by simply choosing a point between them). The value of sensitivity analysis approaches (in my view) lies in the ability to translate domain knowledge into informative bounds. \n\nIn other areas of sensitivity analysis (for e.g., interventional queries in the presence of unobserved confounding) that rely on domain knowledge to derive bounds, there is an emphasis on creating additional tools to help end-users calibrate their choices of these hyperparameters by e.g., benchmarking (see Figure 2 of Cinelli and Hazlett 2019, full citation below).  Informally, this type of benchmarking allows one to say \"an unobserved confounder would have to be 3x as strong as the variable 'age' in order to change our conclusions\", a statement that is a bit easier for domain experts to engage with versus reasoning about the value of an abstract hyperparameter.\n\nI'm aware that this example deals with a different type of causal query (i.e., counterfactuals vs interventional queries), and that the lack of identifiability in the current paper is fundamentally different from unobserved confounding - my point is that for any partial identification approach that derives informative bounds via \"domain knowledge\", like the current approach, a bit more effort is required to make it plausible for domain experts to convert their domain knowledge into the required (abstract) hyperparameters, like bounding the principal curvature of counterfactual level sets.\n\nI would be interested in any comments from the authors on this point.\n\nCitation: Carlos Cinelli , Chad Hazlett, Making Sense of Sensitivity: Extending Omitted Variable Bias, Journal of the Royal Statistical Society Series B: Statistical Methodology, Volume 82, Issue 1, February 2020, Pages 39–67, https://doi.org/10.1111/rssb.12348\n\n(2) **Corresponding between computational bounds and theoretical bounds**:  Supposing we know the right value of $\\kappa$, Definition 2 defines an upper and lower bound on the quantity of interest (e.g., the ECOU), with the additional constraint (e.g., in Theorem 2) that all SCMs satisfy Assumption $\\kappa$ and are in the class $\\mathcal{B}(C^2, d)$.  For the practical algorithm, we use $d = 2$.\n\nIn an ideal world, the computational method (i.e., the Augmented Pseudo-Invertible Decoder in Section 5) would take a given value of $\\kappa$ and return the bounds as defined in Definition 2.  However, somewhat understandably, returning the exact bounds is not necessarily feasible. \n\nInstead, we are left with heuristics - we optimize an objective over a certain set of SCMs where one of the four terms relates to maximizing / minimizing the counterfactual outcome, and is controlled by the hyperparameter $\\lambda_Q$.  It was not clear to me how this term trades off with the other terms, e.g., the extent to which we can view the optimization problem as choosing a fixed constraint on $\\kappa$ and then maximizing / minimizing the counterfactual subject to this constraint among all observationally-equivalent models.\n\n**The use of practical heuristics would be less concerning if this approach resulted in less informative bounds than those in Definition 2, but as I understand it, it may result in *more* informative bounds.**\n\nTo clarify, this point is briefly discussed in Appendix E (discussion of extensions and limitations) under \"Tight bounds\". However, my concern is not that the bounds are insufficiently tight (e.g., too wide and could be narrowed), but that they might be *too narrow* (e.g., the theoretical bounds are actually wider than the bounds produced by the algorithm).\n\nI believe there is some evidence to suggest that these heuristics change the interpretation of the results.  For instance, I would expect that as $\\lambda_{\\kappa}$ changes, the bound should monotonically increase/decrease. However, looking closely at Figure 7 suggests that in practice there is some crossover (see e.g., $y' = -0.5$, where the lower bound for $\\lambda_{\\kappa} = 1$ goes above the line for $\\lambda_{\\kappa} = 10$).  I believe my observation below relating to Figure 13 in the appendix (where $\\lambda_{\\kappa} = 0$ still gives informative bounds) is also related to this point.\n\nOverall, it would be good to see a comparison, perhaps for a very simple example, between the ground-truth bounds in Definition 2 and the bounds returned by the approach.\n\n(3) **Clarification regarding non-informative bounds with $\\kappa=\\infty$**: It is stated in Corollary 4 that by setting $d = 2$ and varying $\\kappa$, we can \"cover the entire identifiability spectrum\".  I took this statement to mean that if e.g., there is no restriction on $\\kappa$, then the resulting bounds should be non-informative (i.e., equal to $[l_a, u_a]$), and that for $\\kappa = 0$, all values of $d$ yield the same point-identified solution.  I'm deriving my understanding primarily from the explanation given in the introduction (see lines 72-73).  Is my understanding correct?\n\nIf my understanding is correct, then what is going on in Figure 13 in the appendix?  It appears that setting $\\lambda_{\\kappa} = 0$ (corresponding to no constraints on $\\kappa$) does not result in non-informative bounds.  Is this due to the (perhaps necessary) mismatch between the computational approach and perfectly solving the optimization in Definition 2?\n\n## Minor Points \n\nAs a minor point, it may be worth clarifying where we are interested in taking a maximum/minimum (e.g., over observationally-equivalent SCMs) and where we are interested in taking an average (e.g., over the counterfactual posterior for a single SCM).  E.g., when I first read the procedure for abduction on lines 341-344, I was concerned that the approach draws samples from the pre-image, as opposed to looking for $U_1, U_2$ which yield the maximum/minimum counterfactual outcomes, before I realized that this approach makes sense for a fixed SCM to compute a counterfactual expectation like ECOU / ECOT.\n\nI also found Figure 6 difficult to follow, especially the bi-directional arrows for \"connections\".\n\nQUESTIONS:\n1. How would you go about helping domain experts choose an appropriate value of $\\kappa$ or $\\lambda_{\\kappa}$ in practice?\n2. Is there any guarantee that the bounds derived from this approach will be less informative than the theoretical bounds?  Or, should we generally expect the bounds to be more informative, since we will inevitably fail to find the SCMs that yield the true maximum / minimum for a given $\\kappa$?\n3. Related to the previous question, is my understanding of \"cover the entire identifiability spectrum\" correct in Corollary 4?  If so, it may be worth revising the theoretical statement to make this clear.  If my understanding is correct, then what is going on in Figure 13, i.e., why does setting $\\lambda_{\\kappa} = 0$ still yield informative bounds?\n\nLIMITATIONS:\nIn the main paper, I would have liked to see more discussion of the limitations outlined in the \"weaknesses\" section above.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-27"}