Partial Counterfactual Identification of Continuous Outcomes with a Curvature Sensitivity Model
Authors: Valentyn Melnychuk
Dennis Frauen
Stefan Feuerriegel
Venue: NeurIPS 2023 spotlight (2023)
Keywords: causal inference
counterfactual inference
partial identification
sensitivity model
normalizing flows
causal machine learning

Abstract: Counterfactual inference aims to answer retrospective "what if" questions and thus belongs to the most fine-grained type of inference in Pearl's causality ladder. Existing methods for counterfactual inference with continuous outcomes aim at point identification and thus make strong and unnatural assumptions about the underlying structural causal model. In this paper, we relax these assumptions and aim at partial counterfactual identification of continuous outcomes, i.e., when the counterfactual query resides in an ignorance interval with informative bounds. We prove that, in general, the ignorance interval of the counterfactual queries has non-informative bounds, already when functions of structural causal models are continuously differentiable. As a remedy, we propose a novel sensitivity model called Curvature Sensitivity Model. This allows us to obtain informative bounds by bounding the curvature of level sets of the functions. We further show that existing point counterfactual identification methods are special cases of our Curvature Sensitivity Model when the bound of the curvature is set to zero. We then propose an implementation of our Curvature Sensitivity Model in the form of a novel deep generative model, which we call Augmented Pseudo-Invertible Decoder. Our implementation employs (i) residual normalizing flows with (ii) variational augmentations. We empirically demonstrate the effectiveness of our Augmented Pseudo-Invertible Decoder. To the best of our knowledge, ours is the first partial identification model for Markovian structural causal models with continuous outcomes.

Partial Counterfactual Identification of Continuous Outcomes with a Curvature Sensitivity Model
Valentyn Melnychuk, Dennis Frauen Stefan Feuerriegel LMU Munich & Munich Center for Machine Learning (MCML)
Munich, Germany
melnychuk@lmm.de
Abstract
Counterfactual inference aims to answer retrospective "what if"" questions and thus belongs the most uine-graneed type inference in Pearl's causality ladder. Existing methods for counterfactual inference with continuous outcomes aim at point identification and thus make strong and unnatural assumptions about the underlying structural causal model. In this paper, we relax these assumptions and aim at partial counterfactual identification of continuous outcomes, i.e., when the counterfactual query resides in an ignorance interval with informative bounds. We prove that, in general, the ignorance interval of the counterfactual queries has non-informative bounds, already when functions of structural causal models are continuously differentiable. As a remedy, we propose a novel sensitivity model called Curvature Sensitivity Model. This allows bounds by bounding the curvature of level sets of the functions. We further show that existing point counterfactual identification methods are special cases our Curvature Sensitivity Model when the bound of the curvature is set to zero. We then propose an implementation of our Curvature Sensitiviry Model in the form of a novel deep generative model, which we call Augmented Pseudo- Invertible Decoder. Our implementation employs (i) residual normalizing flows with (ii) variational augmentations. We empirically demonstrate the effectiveness Decoder. To the best of our knowledge, ours
obtain informative
of our Augmented is the first partial identification model for Markovian structural causal models with continuous outcomes.
Introduction
Counterfactual inference aims answer retrospective "what if" questions. Examples are: Would patient's 's recovery have been, faster, had doctor applied a different treatment? Would my salary be higher, had studied a different college? Counterfactual inference widely used in data- driven decision-makingg
such as root cause analysis [18] 139]. recommender systems [16] (6]36] 83].
infretteeeenneeeeeeeeeee inference is also relevant for various machine learning tasks such as safe policy search [108), reinforcement
responsibility attribution (50) 73 (7), and personalized medicine [88] Counterfactual
learning 38
86 9T. algorithmic fairness 776 104 146), and explainability
75
Counterfactual queries are located the top of Pearl's ladder of causation 8 51 101 i. e., the third layer L3 of causation [8] (see Fig. [ right). Counterfactual queries are challenging as they do reasoning in both the actual world and hypothetical one where variables are different values than they have in reality.
State-of-ue the-art methods for counterfactual inference typically aim at point identification. These works fall into two streams. (1) The first stream 21
70
100
118
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

g., when the ignorance interval for a treatment effect is fully below or Causal diagrams ((M)
Ladder of causation
(Y,, a'
(Y v
(M, a') mitet
Lg Counteffecual
Conperation pamtle nanvo
(M,))
Lg Interventional
L (Y GANPON
Beyasion network
Causel Bayesion network
Panitall worlds natvors
C Observationa
comparing observational, interventional, and counterfactual queries corresponding to the SCM M with two observed variables, i.e., binary treatment A E {0, }} and continuous outcome Y E R. We also plot three causal diagrams, G(M), corresponding to each layer of causation, namely, Bayesian network, causal Bayesian network, and parallel worlds network. Queries with gray background can be simplified, e.e be expressed via lower-layer distributions. The estimation of the queries with yellow background requires additional assumptions distributions from the same layer. In this paper, we focus on partial identification of the expected counterfactual outcome of [un]tretted, E(Y, | aa,''''' ,y)), a' j a, shown in orange.
Figure 1: Pearl's s ladder of causation 8 5T
134. 135 140] makes no explicit assumptions besides assuming structural causal model (SCM) with Markovianity (i. e. independence of the latent noise) and thus gives estimates that can be invalid. However, additional assumptions are needed in counterfactual inference of layer L3 to provide identifiability guarantees [94] 137]. (2) The second stream [4] 32 59 66 93 96 123 148) provides such identifiability guarantees but makes strong assumptions that are unnatural or unrealistic. Formally, the work by Nasr-Esfahany et al. (93) describes bijective generation mechanisms (BGMs), where, in addition to the original Markovianity of SCMs, the functions in the underlying SCMs must be monotonous (strictly increasing or decreasing) with respect to the latent noise. The latter assumption effectively sets the dimensionality of the latent noise to the same dimensionality as the observed (endogenous) variables. However, this is highly unrealistic in real-world settings and is often in violation of domain knowledge. For example, cancer is caused by multiple latent sources of noise (e.g., genes, nutrition, lifestyle, hazardous exposures, and other environmental factors).
In this paper, we depart from point identification for the sake of more general assumptions about both the functions in the SCMs and the latent noise. Instead, we aim at partial counterfactual identification of continuous outcomes. Rather than inferring point estimation expression for the counterfactual query, we are interested in inferring a whole ignorance interval with informative bounds. Informative bounds mean that the ignorance interval strict subset of the support of the distribution. The ignorance interval thus contains all possible values of the counterfactual query for SCMs that are consistent with the assumptions and available data. Partial identification is still very useful for decision-making above zero
We focus on a Markovian SCM with two observed variables, namely. a binary treatment and continuous outcome. We consider a causal diagram in Fig. (left). We then analyze the expected counterfactual outcome of [trettteed abbreviated by ECOU [ECOT]. This query is non-trivial in the sense that it can not simplified to a L L2 layer, as it requires knowledge about the functions in SCM, but can still be inferred by means of a 3-step procedure of abduction-action prreiintoniiiioon [102]. ECOU [ECOT] can be seen as a continuous version of counterfactual probabilities (6) and allows to answer retrospective question about the necessity of interventions: what would have been the expected counterfactual outcome for some treatment, considering knowledge about both the factual treatment and the factual outcome?
In our paper, we leverage geometric measure theory and differential topology to prove that, in general, the ignorance interval of ECOU [ECOT] has non-informative bounds. We show theoretically that this happens immediately when we relax the assumptions of (i) that the latent noise and the outcome have the same dimensionality and (ii) that the functions in the SCMs are monotonous (and assume they are continuously differentiable). As a remedy, we propose a novel Curvature Sensitivity Model (CSM), in which we bound the curvature of the level sets of the functions and thus yield informative bounds. We further show that we obtain the BGMs from 933 as a special case when setting the curvature to zero. Likewise, we yield non-informative bounds when setting it to infinity. Therefore, our CSM provides a sufficient condition for the partial counterfactual identification of the continuous outcomes with informative bounds.

---

We develop an instantiation of CSM in the form of a novel deep generative model, which we call Augmented Pseudo-Invertible Decoder (APID). Our APID uses (i) residual normalizing flows with (ii) variational augmentations to perform the task of partial counterfactual inference. Specifically, our APID allows us to (1) fit the observationalinteeeettoon data, (2) perform abduction- actioon-- prediction in a differentiable fashion, and (3) bound the curvature of the SCM functions, thus yielding informative bounds for the whole ignorance interval. Finally, we demonstrate its effectiveness across several numerical experiments.
Overall, our main contributions are following
1. We prove that the expected counterfactual outcome of [un] has non-informative bounds in the class of continuously differentiable functions of SCMs.
2. We propose a novel Curvature Sensitivity Model (CSM) to obtain informative bounds. Our CSM is the first sensitivity model for the partial counterfactual identification of continuous outcomes in Markovian SCMs.
3. We introduce a novel deep generative model called Augmented Pseudo-Imvertblle Decoder (APID) to perform partial counterfactual inference under our CSM. We further validate it numerically.
2 Related Work
Counterfactual query with continuuus outcome
We briefly summarize prior works on (1) point and (2) partial counterfactual identification below but emphasize that none of them can be straightforwardly extended to our setting. We pro- vide an extended literature overview in Appendix [A]
ide derirlatiity?
Point identiiicaion with probabilisti eegressions g. ETT, palh-spacifo ehocts)
Kcoumption of monsionisty?
Ves
No
(1) Point counterfactual identification has been recently ad- dressed through neural methods [21]
29
31 70
100 113
Point identifcatioo with BGM,s bxerspor-bassdd countee (
(our Curvature Sensitivity Model)
118 134
135
140) but without identifability results.
To ensure identifiability, prior works usually make use of (i) sym- bolic identifiability methods or (ii) put restrictive assumptions on the model class, if (i) led to non-identifiability (see Fig. 2p.
Figure 2:
Flow chart of identifia- bility for a counterfactual query.
(i) Symbolic parammetrrtr) identifiability methods 121 aim to provide a symbolic probabilistic expression suitable for point identification if counterfactual query can be expressed via lower-layer information only. Examples of the latter include the effect of treatment of the treated (ETT) [119) and
path-specific effects [12 146]. However, these not suited for partial counterfactual identification.
Alternatively, identifiability can be achieved by (ii) making restrictive but unrealistic assumptions about the SCMs or the data generating mechanism 4 32
93 96 123 148). A notable example the BGMs [93). which assumes Markovian SCM where the functions in the SCMs must be monotonous (strictly increasing decreasing) with respect to the latent noise. The latter assumption effectively sets the dimensionality of the latent noise to the same dimensionality as the observed (endogenous) variables, yet this is unrealistic in medicine. As remedy, we depart from point identification and instead aim at partial counterfactual identification, which allows us to relax the assumptions of BGMs.
59
66
Kilbertus et al. 669 build sensitivity models for unobserved confounding in sari-Markovian SCMs, but still assume restricted functional class of SCMs, namely, an additive noise model [66] 123 We, on the other hand, build a sensitivity model around the extended class of functions in the SCMs, which is non-trivial even in the Markovian SCMs.
(2) Partial counterfactual identification has been studied previously, but only either for (i) discrete 143 147). or for (ii) specific counterfactual queries with 37 Yet, these (1) do not generalize to the continuous setting; and (ii) are
SCMs 97 103 129
137
141
142
informative bounds 22
specifically tailored for certain queries with informative bounds, such as the variance of the treatment effects, and, thus, are not applicable to our (non- non-informative) ECOU [ECOT]]
Likewise, it is also not possible to extend partial interventional identification of continuous outcomes
35 49 60
61 126 from the L2 to L3, unless it explicitly assumes an underlying SCM.
Research gap. To the best of our knowledge, we are the first to propose a sensitivity model for partial counterfactual identification of continuous outcomes in Markovian SCMs.
'Code is available at
3

---

3 Partial Counterfactual Identification of Continuous Outcomes
In the following, we derive one of our main results: the ignorance interval of the BCOU [ECOT] has non-informative bounds if we relax the assumptions that (i) both the outcome and the latent noise are of the same dimensionality and that (ii) the functions in the SCMs are monotonous.
3.1 Preliminaries
Notation. Capital letters A. Y, denote random variables and small letters
their realizations
from corresponding domains A, V, U. Bold capital letters such as U {UI,.... ....., Un denote finite sets of random variables. Further, P.Y) is an (observational) distribution of Y; IP(Y a) PPY A a) is conditional (observational) distribution; P(Yx=a) P(Y.) an interventional distribution; and P(YA= A' = = Y = P(Y. a', y') counterfactual distribution. We use a superscript such in P.M to indicate distributions that are induced by the SCM M. We denote the conditional cumulative distribution function (CDF) of I(Y a) by F. (v). We use P(Y = -) to denote a density or probability mass function of Y and EY( = f dlP(() to refer to its expected value. Interventional and counterfactual densities probability mass functions are defined accordingly.
A function is said to be in class C*, k if its k-tth derivative exists and is continuous. Let |I-I12 denote the L2-norm, V fx) a gradient of f(x), and Hess, f fx) Hessian matrix of f(x). The pushforward distribution or pushforward measure is defined as a transfer of a (probability) measure with a measurable function f. which we denote as ftlP.
137]. An SCM M is defined as tuple (U. V, P(,,, F)) with latent (exogenous) noise variables U, observed (endogenous) variables V (V1 1.... Vn, ). a distribution of latent noise variables (P) and a collection of functions F (Jv,. (Jv..-..,vv Jv. Each function a measurable map from the corresponding domains of Uy; U Pay, to Vi, where Uv, C U and Pay, S V V; parents of the observed variable V,, Therefore, the functions FF induce a pushforward distribution of observed variables, i.e., P(V) = F.)( Each V, is thus deterministic (non-random), conditionally on its parents, c., Ui + fv; (pay, uv;). Each SCM M induces an (augmented) causal diagram (.M), which is assumed to be acyclic.
SCMs. We follow the standard notation of SCMs in 88
13 101
We provide background on geometric measure theory and differential geometry in Appendix B
3.2 Counterfactual Non-Identifiability
In the following, we relax the main assumption of bijective generation mechanisms (BGMs) [93]. e.. that all the functions in Markovian SCMs are monotonous (strictly increasing) with respect to the latent noise variable. To this end, we let the latent noise variables have arbitrary dimensionality and further consider functions to be of class C* We then show that counterfactual distributions under this relaxation are non-identifiable from L1 or Le data.
Definition 1 (Bivariate Markovian SCMs of class Ch and d-dimension latent noise). Let B(C* d) denote the class of SCMs M = (U, V(U),, F) with the following endogenous and latent noise variables: = {UA e Uy E [0, 1 and V = [A E {0, 1], E K}, for 2 1. The latent noise variables have the following distributions P(U): UA Beru(pA), 0 PA < L, and Uy ~ Un 1)d and are all mutually independent
2 The functions are F (fA(UA), (A,
and (a -) E C for uy
E (0, 1)a Va E 1).
All SCMs in B(C*, d) induce similar causal diagrams G(M), where only the number of latent noise variables d differs. Further, it follows from the above definition that BGMs are a special case of B(C", 1), where the functions fy (a. are monotonous (strictly increasing).
Task: counterfactual inference. Counterfactual queries are at layer L3 of the causality ladder and are defined as probabilistic expressions with random variables belonging to several worlds, which are in logical contradiction with each other {8] 27]. For instance, IPM(Y, = Y, y),a & a' for an SCM M from 2 (C* d). In this paper, we focus on the expected counterfactual outcome of [un Itreated ECOU [ECOT], which we denote by Q. (v') EM v²,
Uniformity of the latent noise does not restrict the definition, the discussion in Appendix D

---

4 Observatoaa inferncee = Eg Intervetional inferenee
Eg Countarfacuul inference
1. Adductio neguufe of ap
2. Artion
3. Prediction
4de
A-1 #
M1
my de Po
g
M2
Arll "gal
Legend
space of Uys Uye with ((Mr), Myr) fr(e, Marl, Myye)
Density with
Puskifarward noisa variabieaa with function fvlA, Uy1,
equidatam god points y
Figure 3: Inference of observational and interventional distributions (left) and counterfactual distribu- tions (right) for SCMs Mi and M2 from Example Left: the observational query PM = y a) coincides with the interventional query PYa = y) for each MM and M2. Right: the counterfactual queries can still differ substantially for M1 and M22, thus giving a vastly different counterfactual out- come distribution of the untreated, (YM A' 0, Y 0). Thus, L3 queries are non-identifiable from L1 or Le information.
For an SCM M from (CC*, d), ECOU [ECOT] can be inferred by the following three steps. (1) The abduction step infers posterior distribution of the latent noise variables, conditioned on the evidence, i.e. P(Uy a', a/). This posterior distribution is defined on the level set of the factual function given by {uy : fy (a' y' J. e.e., all points in the latent noise space mapped to y'. (2) The action step alters the function for Y to fy (a, Uy). (3) The prediction step is done by a pushforward of the posterior distribution with the altered function, i.e. fr ()P(() a', v'))"). Afterwards, the expectation is then evaluated.
The existence of counterfactual queries, which are non-identifiable with L1 or L2 data in Markovian SCMs, was previously shown [137) (Ex. App. D3) for discrete outcomes and in [13 30] (Ex. D.7) for continuous outcomes. Here, we construct an important example to (1) show non- identifiability of counterfactual queries from Lı or La data under our relaxation from Def. [ and to
consequently (2) give some intuition on informativity of the bounds of the ignorance interval, which we will formalize later.
Example 1 (Counterfactual non-identifiability in Markovian SCMs). Let Mı and Ma be two Markovian SCMs from (CCo, 2) with the following functions for Y:
M1: : (y Uy2) = A AUvx= - Uy2 + 1) + (1 - A) (Uy1 + (1y2-1))
Uy1 + Uy2 1,
A=
M2: fy (A, Uv>)
Uy Uv2 +1,
A= (05 S $1)A (Uv1 S Uy² $ 1)),
F-, Uys),
otherwise,
where F F- Uv=, Uya) ) is the solution in Y of the implicitly defined function functiinn F(Y,Uy1, Uv2) + + 1 v(Y 2y2 (8 1)2 +1
Uy Uyz 2 - 1) 1--yy--- Uy
turns out that the SCMs Mi and M2 are observationally and interventionally equivalent (relative to the outcome Y). That is, they induce the same set of Ln and L2 queries. For both SCMs, it can be easily inferred that a pushforward of uniform latent noise variables Uy1 and Uy2 with fy (A, Uy1, Uva) is a symmetric triangular distribution with support Vo = [], for A = 0 and J1 [0, for A ==1 1, respectively. We plot the level sets of fy Uy1, for both SCMs in Fig. (left). The pushforward of the latent noise variables preserves the transported mass; i. e., note the equality of (1 (I) the area of each colored band between the two level sets in the latent noise space and (2) the area of the corresponding band under the density graph Y
Y Fig. 3 (left).
Despite the equivalence of Ln and Lz. the SCMs differ in their counterfactuals; see Fig. (right). For example, the counterfactual outcome distribution of untreated, PY = A' = 0, Y' 0), has different densities for both SCMs M1 and M2. Further, the ECOU, 10 EE A'
0, Y' = 0), is different for both SCMs, i. e., (0))(0) = and Ma,a z 1.114. Further details for the example are in Appendix C
The example provides an intuition that motivates how we generate informative bounds later. By "bending" the bundle of counterfactual level sets (in blue in Fig, Befft around the factual level set
5

---

(in orange in Fig. right), we can transform more and more mass to the bound of the support. We later extend this idea to the ignorance interval of the ECOU. Importantly, after "bending" the bundle of level sets, we still must make sure that the original observational/ineeveniiooa distribution is preserved.
3.3 Partial Counterfactual Identification and Non-Informative Bounds
We now formulate the task of partial counterfactual identification. To do so, we first present two lemmas that show how we can infer the densities of observational and counterfactual distributions from both the latent noise distributions and C' functions in SCMs of class B(C d). Lemma 1 (Observational distribution pushforward with fv). Let BB(",, d). Then, the density of the observational distribution, induced by M, is
1
PM B(ura) Va fy(a, uy) dz(
(1)
where Ely, a) is level set (preinage) of e. Ely,a) a) fuy [0, (r( = y}, and Hdt (uy') ) is the Hausdorff measure (see Appendix B for the definition).
We provide an example in Appendix C where we show the application of Lemma (therein, we derive
the standard normal distribution as a pushforward using the Box- Müller transformation). Lemma ! is generalization of the well-known change of variables formula. This is easy to see, when
1y)] Furthermore, the function (r,a, uy) can be restored (up to sign) from the observational distribution, it is monotonous in wy, such as in BGMs [93]. In this case, the function coincides (up sign) with the inverse CDF of the observed distribution, i. e., Jy (a, uy) = Fo (+uy F 0.5 + 0.5) (see Corollary I Appendix D)
set d = so that IPM(Y- y a) E EE(y.) fy
Lemma 2. Let M B(C", d). Then, the density of the counterfactual outcome distribution of the [un]treated is
1
5(fy (a, uy) 3) aH dux),
I (Ya
d', ./v PM(Y = y' a') J(s''a') V fy (a', uy )ll
(2)
where 8(.) is a Dirac delta function, and the expected counterfactual outcome of the [un]treated, i.e. ECOU [ECOT], is
PM(Y = =v la') S SEIV..a) II fr fa uy 111
1
(a(a,
= EE ( dd.v)=
17(4-1(us)
(3)
where Elyl, ,a) is (factual) level set of ie, Fly', a') = (ur [0, fr 3 and a' # a
Equations and implicitly combine all three steps of the induuction-action catiio--aactooo---eeitioon procedure: (1) abduction infers the level sets Fly/, ,a)) with the corresponding Hausdorff measure Hd (uy): (2) action uses the counterfactual function fy (a, uy): and (3) predic- tion evaluates the overall integral. In the specific case of d = 1 and a monotonous function fy(a, uy) with respect to uy, we obtain two deterministic counterfactuals, which are identifiable from observa- tional distribution, i. c., Q,y_... (y/") " (1.5505++.0) For details, Corollary 3 in Appendix D For larger d, as already shown in Example [ both the density and ECOU [ECOT] can take arbitrary values for the same observational (or interventional) distribution.
po ml
1yla
Figure 4: "Bending" the bun- dle of counterfactual level sets [E(y, a) E [1, 2]} in blue around the factual level set Ely', a!) in orange.
Definition 2 (Partial identification of ECOU (ECOT) in class B B(Ckk, d), k 2 1). Given the continuous observational distribution P(Y a) for some SCM of class B(C* d). Then, partial counterfactual identification aims to find bounds of the ignorance interval [Qa (y'), Qa given by
Qy') inf C.Ma(('' S.I. Va [0,1] F(Y a) P-M(Y a),
(4)
MEBiCKKA)
Qa'- M)
sup
Q.M S.P. V. (0.1) P(Y a) -P-(Y (a),
(5)
where 1pM(( a)) is given by Eq. and and Q.M (V)) by Eq. 3

---

Non-iffomaaee
Informaivo
Port Idertiiatio
Residual NF
* Marry S-aape
Ow PITIA
ai 4
Alpp Scaspes
Legend
Corndd
Lossee
convoooding
S10 Jerme Jenve
M.M Porcen)
Mar Pumpo
pesorwatiedd
ng Rovu
Note replarctioon
Figure 5: Venn diagram of different SCM classes B(C*, d) arranged by partial vs. point point identification. See our CSM with different K. We further show the relationships between different classes of SCMs B(C*,, d) and statements about them, e. g., Lemmas and 2 The referenced examples are in the Appendix
Figure 6: Overview of our Augmented Pseudo- Invertible Decoder (APID). Our APID uses (i) two residual normalizing flows for each treat- ment a 10, respectively; and (ii) varia- tional augmentations, Youg. Together, both com- ponents enable the implementation of CSM un- der Assumption
Hence, the task of partial counterfactual identification in class B(C* d) can be reduced con- strained variational problem. namely, a constrained optimization of ECOU [ECOT] with respect to (a. E Ck Using Lemma 2 we see that, e. g., ECOU (ECOT) can be made arbitrarily large (or small) for the same observational distribution in two ways. First, this can be done by changing the factual function, fy (a',)) .). i.e. e.. if we increase the proportion of the volume of the factual level set Ely', .d)) that intersects only a certain bundle of counterfactual level sets, Second, this can be done by modifying the counterfactual function, (a, -), by "bending" the bundle of counterfactual level sets around the factual level set. The latter is schematically shown in Fig. 4 We formalize this important observation in the following theorem.
Theorem 1 (Non-informative bounds of ECOU (ECOT)). Let the continuous observational distribu- tion P(Y a) be induced by some SCM of class B (Cx,, d). Let P(Y a) have a compuct support )aa Ba, Ma) and be of finite density IP(Y partial identification of the ECOU [ECOT] of class B B(C ,d). d 2 2, has non informative bounds: Qu' (1') = La and Qa'
a) < +00. Then, the ignorance interval for the
Un.
Theorem I implies that, no matter how smooth the class of functions is, the partial identification of ECOU ] will have -on-informative bounds. Hence, with the current set of assumptions, there is no utility in considering more general classes. This includes various functions, such as C and the class of all measurable functions (a(a,.) Rd - IR, as the latter includes C functions. In the following, we introduce a sensitivity model which nevertheless allows us to obtain informative bounds.
4 Curvature Sensitivity Model
In the following, we develop our Curvature Sensitivity Model (CSM) to restrict the class B(C*, d) so that the ECOU [ECOT] obtains informative bounds. Our CSM uses the intuition from the proof of Theorem in that, to construct the non-informative SCM Mnon-infs we have to "bend" the bundle of the counterfactual level sets. As result, our CSM provides sufficient conditions for informative bounds in the class BCC d) by bounding the principal curvatures of level sets globally.
Assumption k. Let M be of class B B(C², d), d 2 2. Let Ely, a) be the level sets of functions fy (a, wy) for a E {0, which are thus d - 1-dimensional smooth manifolds. Let us assume that principal curvatures E {l,...,d d 1} exist at every point uy E E(y, a) for every a E {0, 1 and y E (la; u.) C Yn. and let us denote them as keluy). Then, we assume that K 2 0 is the upper bound of the maximal absolute principal curvature for every y, and uy E Ely,a), a):
K
max
max |nc(uy)]-
(6)
(E(1....d11)
Principal curvatures can be thought of as a measure of the non-linearity of the level sets, and, when they are all close to zero at some point, the level set manifold can be locally approximated by flat hyperplane. In brevity, principal curvatures can be defined via the first- and second-order partial derivatives of (a, -). so that they describe the degrees of curvature of manifold in different B for a formal definition of the principal curvatures Kg. An example directions. We refer to Appendix is in Appendix C
7

---

Now, we state the main result of our paper that our CSM allows us to obtain informative bounds for ECOU [ECOT].
Theorem 2 (Informative bounds with our CSM), Let the continuous observational distribution P(Y a) be induced by some SCM of class BCC², .d). .. 2 2, which satisfies Assumption Rk] Let P(Y a) have a compact support Vo = Vo, ua]. Then, the ignorance interval for the partial (ECOU(ECOT] identification of ECOU [ECOT] of class B(C², .d) has informative bounds, dependent on K and d,
which are given by Qa'+a = [(,, d) la and ao' - u(k, d)
Uha
Theorem 2 has several important implications. (1) Our CSM is applicable to wide class of functions B 2CC2, 2,, d), for which the principal curvature is well defined. (2) We show the relationships between different classes B(C*, d) and our CSM (k) in terms of identifiability in Fig. 5 For example, by increasing K, we cover larger class of functions, and the bounds on ECOU [ECOT]
expand (see Corollary 4 in Appendix D). For infinite curvature, our CSM almost coincides with the entire BCC², d). (3) Our CSM (R) with K. 2 0 always contains both (i) identifiable SCMs and (ii) (informative) non non-identifiable SCMs. Examples of (i) include SCMs for which the bundles of the level sets coincide for both treatments, i. e.. {E a) y E V.} {Ely, a') y V }. In this case, it is always possible to find an equivalent BGM when the level sets are flat and thus x = 0 (see Mia from Example 6 in the Appendix C or curved and thus
(see Manr from Example in the Appendix (). For (ii), we see that, even when we set k = 0. we can obtain non-identifiability with informative bounds. An example is when we align the bundle of level sets perpendicularly to each other for both treatments, as in Mperp from Example 8 in the Appendix C observation with the following Lemma.
We formalize this
Lemma 3 (BGMs-EQTDs identification gap of CSM(K 0)). Let the assumptions of Theorem hold and let K. = O. Then the ignorance intervals for the partial identification of ECOU (ECOT) of class ((C d) are defined by min/max between BGMs bounds and expectations of quantile-truncated distributions (EQTDs):
Qar->aly)//Q (v')/Qu = min / max (BGM+(V) BGM- (O))) BGM- (y') = -1 Exr(y/)
(7)
BGM (1/) = F. (F (B'))
(8)
EQTD((y') = E (Y Y < F. (1 2 | - (f)))
(9)
(QTD,(U)) E (Y a, Y > F (2 Far (v') 0.5l))
(10)
where EEY I < ), Y :) are expectations of truncated distributions.
Lemma 3 shows how close we can get to the point identification after setting K = 0 in our CSM.
In particular, with CSM (x 0) the ignorance interval still contains BGMs bounds and EQTDs, i. e.e. this is the identification gap of CSM Hence, to obtain full point identification with our CSM, additional assumptions or constraints are required. Notably, we can not assume monotonicity, as there is no conventional notion of monotonicity for functions from Hd to R.
We make another observation regarding the choice of the latent noise dimensionality d. In practice, is sufficient to choose d = 2 (without further assumptions on the latent noise space). This choice is practical as we only have to enforce a single principal curvature, which reduces the computational burden, i.e.,
Vuy Ir((a uy) (nn(Tr(a,u)) Ki (uy) = - by (IVmr wy (r, (r(a, )y)
uy E Ely,a).
(11)
Importantly, we do not lose generality with d = 2, as we still cover the entire identifiability spectrum by varying K. (see Corollary S in Appendix D). We discuss potential extensions of our CSM in
Appendix E
Interpretation of K. The sensitivity parameter lends to natural interpretation. Specifically, it can be interpreted as a level of non-linearity between the outcome and its latent noises that interact with the treatment. For example, when (i) the treatment does not interact with any of the latent noise variables, then we can assume w.l.o.g. a BGM, which, in turn, yields deterministic counterfactual outcomes. There is no loss of generality, as all the other SCMs with d > 1 are equivalent this
II general, BGMs bounds and EQTDs have different values, especially for skewed distributions. Neverthe- less, it can be shown, that they approximately coincide y' goes to lar or u.!.
8

---

BGM (see Examples and in Appendix C. On the other hand, (ii) when the treatment interacts with some latent noise variables, then the counterfactual outcomes become random, and we cannot assume a BGM but we have to use our CSM. In this case, d corresponds to the number of latent noise variables which interact with the treatment. Hence, k: bounds the level of non-linearity between the outcome and the noise variables. More formally, K, as principal curvature, can be seen as the largest coefficient of the second-order term the Taylor expansion of the level set (see Eq. (19) in Appendix B This interpretation of the goes along with human intuition [20): when we try to imagine counterfactual outcomes, we tend to "traverse" all the possible scenarios which could lead to certain value. If we allow for highly non-linear scenarios, which interact with treatment, also allow for more extreme counterfactuals, e. g., interactions between treatment and rare genetic conditions.
5 Augmented Pseudo-Invertible Decoder
We now introduce an instantiation of our CSM: novel deep generative model called Augmented Pseudo-Imvertibl Decoder (APID) to perform partial counterfactual identification under our CSM of class (CC², 2).
Architecture: The two main components of our APID are (1) residual normalizing flows with (2) variational augmentations (see Fig. 6). The first component are two-dimensional normalizing flows [107 125 to estimate the function fY (a, uy), uy E [0. 1J2, separately for each treatment E {0, IJ. Specifically, we residual normalizing flows (24) due their ability to model free-form Jacobians (see the extended discussion about the choice of the normalizing flow in Appendix (F). However, (wo-dimensional normalizing flows can only model invertible transformations, while the function fy (a.-) [o. 1]2 + Ya C R is inverrlbiible. To address this, we employ an approach of pseudo-invertible flows [10 54), namely, the variational augmentations (22). as described in the following. The second component in our APID are variational augmentations [22). Here, we augment the estimated outcome variable Ý with the variationally sampled Yong ~ N (Y) where gd (-) a fully-connected neural network, and g² > variational augmentations, our APID then models fy (a, :) through a two-dimensional transformation F. = -fr..g (a, -). fy (a. Jo, 1]2 + R x Va. We refer to the Appendix F for further details.
a hyperparameter. Using the
Inference: Our APID proceeds in first steps: (P1) it first fits the observatiiona/iitereenite distribution PPY|| a), given the observed samples; (P2) it then performs counterfactual inference of ECOU [ECOT] in differentiable fashion, which, therefore, with other objectives; and (P3) it finally penalize functions with large principal curvatures of the level sets, by using automatic differentiation of the estimated functions of the SCMs.
be maximized/minimized jointly
We achieve (P1)-(33 through the help of variational augmentations: + In (PI), we fit two two- dimensional normalizing flows with negative log-likelihood (for the task of maximizing min- imizing ECOU [ECOT], respectively).
In (P2), for each normalizing flow, we sample points from the level sets of fr )) The latter is crucial as it follows an abduecion-aaciionprediiiooo procedure and thus generates estimates of the ECOU [ECOT] in differential fashion. To evaluate ECOU [ECOT], we first perform the abduction step with the inverse transformation of the factual normalizing flow, i.e., F-l. This transformation maps the variationally augmented evidence, i. i..., N (gg()))) =)), where b is the number of augmentations, to the latent noise space. Then, the action step selects the counterfactual normalizing flow via the transformation P... Finally, the prediction step performs pushforward of the latent noise space, which was inferred during the abduction step. Technical details are in Appendix F In (P3), we enforce the curvature constraint of CSM. Here, we use automatic differentiation as provided by deep learning libraries, which allows us to directly evaluate Ki (wy) according to Eq. (II.
(v/ VVYYYig
Training: Our APID is trained based on observational data D YJ, drawn i.i.d. from some SCM of class B(C², 2). To our APID, we combine several losses one optimization objective: (1) a negative likelioed loss LNU with noise regularization [110), which aims to fit the data distribution; (2) a counterfactual query loss Lo with a coefficient Xg. which aims to maximize/mininize the ECOU [ECOT]; and (3) a curvature loss L. with coefficient X which penalizes the curvature of the level sets. Both coefficients are hyperparameters. We provide details
about the losses, the training algorithm, and hyperparameters in Appendix G Importantly,

---

incorporated several improvements to stabilize and speed up the training, For example, in addition to the negative log- likelihood, we added the Wasserstein loss Ly to prevent posterior collapse (26) 132]. To speed up the training, we enforce the curvature constraint only on the counterfactual function, (a -), and only for the level set, which corresponds to the evaluated ECOU [ECOT), wy aa))
Experiments
Datasets. To show the effectiveness of our APID at partial counterfactual identification, we use two synthetic datasets. This enables us access the ground truth CDFs, quantile functions, and sampling. Then, with the help of Lemma 3 we can then compare our APID with the BGMs-EQTDs identification gap (= a special case of CSM with A = 0). Both synthetic datasets comprise samples from observational distributions P(Y u), which we assume to be induced by some (unknown) SCM of class B (C,
(C2,2) 2). In the first dataset, IP(Y 0) = P(Y 1) is the standard normal distribution, and in second, PY 0) and P(Y 1) are different mixtures of normal distributions. We draw na 1, 000 observations from IP(Y a) for each treatment a E {0, 1]. so that n = Ro + n2 = 2, 000. Although both distributions have infinite support, we consider the finite sample minimum and maximum as
estimates of the support bounds [1, ú.]. Further details our synthetic datasets in Appendix H
In sum, the estimated bounds from our APID are consistent with the theoretical values, thus showing the effectiveness of our method.
P/Y - RLY N(0.1
Results. Fig. 7 shows the results of point/partial counterfac- tual identification of the ECOU, i.e., (v-) for different values of y Point identification with BGMs yields two curves, corresponding to strictly increasing and strictly decreasing functions. For partial identification with our APID, we set λo = 2.0 and vary X; E (0.5, 1.0, 5.0, 10.0) (higher values correspond to a higher curvature penalization). We observe that, as we increase Au. the bounds are moving closer to the BGMs, and, as we decrease it, the bounds are becom- ing non-informative, namely, getting closer to [ē, ú, We report additional results in Appendix (e. g., for APID with λo= = 1.0).
-10
0.5
P1y MictunglY))
PLY Mieture (V)
Case study with real-world data, In Appendix real-world case study. Therein, we adopt CSM to swer "what if" questions related to whether the lockdown was effective during the COVID-19 pandemic.
[ we provide
ATEENAL 025)
APIDIA, 1000
7 Discussion
APITA 1.0
APIDA 5.00
Limitations. Our CSM and its deep-learning implementation with APID have several limitations, which could be addressed in future work. First, given the value of k, it is computation- ally infeasible to derive ground-truth bounds, even when the ground-tran -tuth observation density is known. The inference of these bounds would require solving constrained optimization task including partial derivatives and Hausdorff integration. This is intractable, even for such simple distributions as standard normal. Second, the exact relationship between k and corresponding A,e unknown APID, but they are known to be inversely related. Third, our APID sometimes suffers from computational instability, e. g.g the bounds for the multi-modal distribution (see Fig. 7. are inaccurate, e., too tight. This happens as the gradual "bending" of the level sets is sometimes numerically unstable during training. and some runs omit "bending" at all as it would require passing through the high loss value region.
Figure 7: Results for partial counter- factual identification of the ECOU across two datasets. Reported: BGMs-EQTDs identification gap and mean bounds of APID over five runs.
Conclusion. Our work is the first to present sensitivity model for partial counterfactual identification of continuous outcomes in Markovian SCMs. Our work rests on the assumption of the bounded curvature of the level sets, yet which should be sufficiently broad and realistic to cover many models from physics and medicine. As broader impact, we expect our bounds to be highly relevant for decision-making in safety-critical settings.
10

---

Acknowledgments
SF acknowledges funding from the Swiss National Science Foundation (SNSF) via Grant 186932.
References
[1] Giovanni Alberti, Stefano Bianchini, and Gianluca Crippa. "Structure of level sets and Sard- type properties of Lipschitz maps". In: Annali della Scuola Normale Superiore di Pisa-Clas Pisa-Classe di Scienze 12.4 (2013), pp. 863-902.
[2] Peter M. Aronow, Donald P. Green, and Donald K.K. Lee. "Sharp bounds on the variance in randomized experiments". In: The Annals of Statistics 42.3 (2014), pp. 550-871. [3] Chen Avin, Ilya Shpitser, and Judea Pearl. "Identifiability of path-specific effects". In: Inter- national Joint Conference Artificial Intelligence. 2005.
[4] Sivaraman Balakrishnan, Edward Kennedy, and Larry Wasserman. "Conservative inference for counterfactuals". In: arXiv preprint arXiv:2310. 12757 (2023).
[5] Vahid Balazadeh Meresht, Vasilis Syrgkanis, and Rahul G. Krishnan. "Partial identification of treatment effects with implicit generative models". In: Advances in Neural Information Processing Systems. 2022.
[6] Alexander Balke and Judea Pearl. "Counterfactual probabilities: Computational methods, bounds and applications". In: Uncertainty Proceedings. Elsevier, 1994, pp. 46-54.
[7] Nicolas Banholzer al. "Estimating the effects of non-pharmaceutical interventions on the number of new infections with COVID-19 during the first epidemic wave". In: PLoS one 16.6 (2021), e0252827.
[8] Elias Bareinboim al. "On Pearl's hierarchy and the foundations of causal inference". In: Probabilistic and Causal Inference: The Works of Judea Pearl. Association for Computing Machinery, 2022, pp. 507-556.
[9] Jens Behrmann et al. "Invertible residual networks". In: International Conference on Machine
Learning. 2019.
[10] Jan Jetze Beitler, Ivan Sosnovik, and Arnold Smeulders. "PIE: Pseudo-invertible encoder".
In: arXiv preprint arXiv:2111.006 (2021).
[11] Matthew Blackwell. "A selection bias approach to sensitivity analysis for causal effects". In: Political Analysis 22.2 (2014), pp. 169-182.
[12] Vladimir Igorevich Bogachev and Maria Aparecida Soares Ruas. Measure theory. Vol.
Springer Berlin, 2007
[13] Stephan Bongers et al. "Foundations of structural causal models with cycles and latent variables". In: The Annals of Statistics 49.5 (2021), pp. 2885-2915.
[14] Matteo Bonvini and Edward H. Kennedy. "Fast convergence rates for dose-response estima- tion". In: arXiv preprint arXiv:2207. 1825 (2022).
[15] Matteo Bonvini al. "Sensitivity analysis for marginal structural models". In: arXiv preprint (2022).
2rXiv:2210.0445
[16] Léon Bottou et al. "Counterfactual reasoning and learning systems: The example of computa- tional advertising". In: Journal of Machine Learning Research 14.11 (2013).
[17] Babette A. Brumback et "Sensitivity analyses for unmeasured confounding assuming a marginal structural model for repeated measures". In: Statistics in Medicine 23.5 (2004), pp. 749-767.
[18] Kailash Budhathoki et al. "Causal structure-based root cause analysis outliers". In: Inter-
national Conference Machine Learning. 2022.
[19] Lars Buesing et al. "Woulda, coulda, shoulda: Counterfactually-guide policy search". In: International Conference on Learning Representations. 2019
[20] Lenart Celar and Ruth M.J. Byrne. "How people reason with counterfactual and causal explanations for artificial intelligence decisions in familiar and unfamiliar domains". In: Memory & Cognition (2023), pp. 1-16.
[21] Patrick Chao, Patrick Blöbaum, and Shiva Prasad Kasiviswanathan, "Interventional and counterfactual inference with diffusion models". In: arXiv preprint arXiv:2302.0m 00300000060 (2023).
[22] Jianfei Chen al. "Vflow: More expressive generative flows with variational data augmenta- tion". In: International Conference on Machine Learning. 2020.
11

---

[23] Ricky T.Q. Chen et al. "Neural ordinary differential equations". In: Advances in Neural
Information Processing Systems. 2018.
[24] Ricky T.Q. Chen et al. "Residual flows for invertible generative modeling". In: Advances in Neural Information Processing Systems. 2019.
[25] Scott Chen and Ramesh Gopinath. "Gaussianization" In: Advances in Neural Information Processing Systems. 2000.
[26] Florentin Cocurdoux, Nicolas Dobigeon, and Pierre Chainais. "Sliced-Wasserster Wasserstein normal- izing flows: Beyond maximum likelihood training". In: European Symposium on Artificial Neural Networks. 2022.
[27] Juan Correa, Sanghack Lee, and Elias Bareinboim. "Nested counterfactual identification from arbitrary surrogate experiments". In: Advances in Neural Information Processing Systems.
2021.
[28] Alicia Curth and Mihaela van der Schaar. "Nonparametric estimation of heterogeneous treatment effects: from theory to learning algorithms". In: International Conference on Artificial Intelligence and Statisties. 2021.
[29] Saloni Dash, Vineeth N. Balasubramanian, and Amit Sharma. "Evaluating and mitigating bias in image classifiers: A causal perspective using counterfactuals", In: JEEE/CVF Winter Conference on Applications of Computer Vision. 2022.
[30] A. Philip Dawid. "Influence diagrams for causal modelling and inference". In: International Statistical Review 70.2 (2002), pp. 161-189.
[31] Edward De Brouwer. "Deep counterfactual estimation with categorical background variables". In: Advances in Neural Information Processing Systems. 2022.
[32] Lucas De Lara al. "Transport-based counterfactual models". In: arXiv preprint
arXiv:2108. 13025 (2021).
[33] Laurent Dinh, David Krueger, and Yoshua Bengio. "NICE: Non-linear independent compo-
nents estimation". In: arXiv preprint 140. (2014).
[34] Laurent Dinh, Jascha Sohl- Dal--Dicsttein, and Samy Bengio. "Density estimation using real
NVP". In: International Conference Learning Representations: 2017.
[35]
Jacob Dorn and Kevin Guo. "Sharp sensitivity analysis for inverse propensity weighting via quantile balancing". In: Journal of the American Statistical Association (2022), pp. 1-13.
[36]
Tom Everitt et al. "Agent incentives: A causal perspective". In: AAAI Conference on Artificial Intelligence. 2021.
[37]
Yanqin Fan and Sang Soo Park. "Sharp bounds on the distribution of treatment effects and their statistical inference". In: Econometric Theory 26.3 (2010), pp. 931-951. Jakob Foerster et al. "Counterfactual multi-agent policy gradients". In: AAAI Conference on A Artificial Intelligence. 2018.
[38]
[39]
Dennis Frauen and Stefan Feuerriegel. "Estimating individual treatment effects under unob- served confounding using binary instruments". In: International Conference on Learning Representations. 2023.
[40] Dennis Frauen, Valentyn Melnychuk, and Stefan Feuerriegel. "Sharp bounds for generalized causal sensitivity analysis". In: Advances in Neural Information Processing Systems. 2023.
[41] Dennis Frauen et al. "A neural framework for generalized causal sensitivity analysis". In: arXiv preprint arXiv:231 1.6622 (2023).
[42] Sainyam Galhotra, Romila Pradhan, and Babak Salimi. "Explaining black- box algorithms using probabilistic contrastive counterfactuals". In: International Conference Management
of Data. 2021.
[43] Ron Goldman. "Curvature formulas for implicit curves and surfaces". In: Computer Aided Geometric Design 22.7 (2005). pp. 632-658 052-658.
[44] Yash Goyal et al. "Counterfactual visual explanations". In: International Conference on Machine Learning. 2019.
[45] Will Grathwohl et al. "Scalable reversible generative models with free-form continuous dynamics". In: International Conference on Learning Representations. 2019.
[46] Luigi Gresele et al. "Causal inference through the structural causal marginal problem".. In: International Conference on Machine Learning, 2022.
12

---

[47] Luigi Gresele et al. "The incomplete rosetta stone problem: Identifiability results for multi- view nonlinear ICA". In: Conference on Uncertainty Artificial Intelligence. 2020.
[48] Florian Gunsilius. "A path-sampling method to partially identity causal effects in instrumental
variable models". In: arXiv preprint arXiv: 0.0000090220 (2019).
[49] Wenshuo Guo et al. "Partial identification with noisy covariates: A robust optimization
approach". In: Conference on Causal Learning and Reasoning. 2022.
[50] Joseph Halpern and Max Kleiman- Weiner.. "Towards formal definitions of blameworthiness,
intention, and moral responsibility". In: AAAI Conference on Artificial Intelligence. 2018.
[51] Joseph Y. Halpern and Judea Pearl. "Causes and explanations: A structural-mode approach. part i: Causes". In: The British Journal for the Philosophy of Science (2005).
[52] Tobias Hatt and Stefan Feuerriegel. "Estimating average treatment effects via orthogonal regularization". In: International Conference on Information and Knowledge Management. 2021.
[53]
Lars Hörmander. Composition with Smooth Maps. Springer Berlin Heidelberg, 2003, pp. 133- 157.
[54]
Christian Horvat and Jean-Paseal Pfister. "Denoising normalizing flow". In: Advances in Neural Information Processing Systems. 2021.
[55]
Yaowei Hu et al. "A generative adversarial framework for bounding confounded causal effects". In: AAAI Conference on Artificial Intelligence. 2021.
[56]
Yimin Huang and Marco Valtorta. "Identifiability in causal bayesian networks: A sound and complete algorithm". In: National Conference Artificial Intelligence (AAAI). 2006.
[57] Aapo Hyvarinen, Hiroaki Sasaki, and Richard Turner. "Nonlinear ICA using auxiliary vari- ables and generalized contrastive learning". In: International Conference on Artificial Intelli- gence and Statistics. 2019.
[58] Guido W. Imbens and Joshua D. Angrist. "Identification and estimation of local average treatment effects". In: Econometrica 62.2 (1994), pp. 467-475.
[59] Alexander Immer al. "On the identifiability and estimation of causal location-scale noise models". In: International Conference on Machine Learning. 2023.
[60] Andrew Jesson et al. "Quantifying ignorance in individual-level causal-effect estimates under hidden confounding". In: International Conference on Machine Learning. 2021.
[61] Andrew Jesson et al. "Scalable sensitivity and uncertainty analyses for causal-effect estimates of continuous- interventions".
In: Advances in Neural Information Processing Systems.
2022.
[62] Ying Jin, Zhimei Ren, and Zhengyuan Zhou. "Sensitivity analysis under the f-sensitivity models: distributional robustness perspective". In: arXiv preprint arXiv:2203. 04373 (2022).
[63] Nathan Kallus and Angela Zhou. "Confounding-robus policy evaluation in infinite-horizon reinforcement learning". In: Advances in Neural Information Processing Systems. 2020.
[64]
Amir-Hossein Karimi, Bernhard Schölkopf, and Isabel Valera. "Algorithmic recourse: From counterfactual explanations to interventions". In: ACM Conference on Fairness, Accountabil- ity, and Transparency. 2021.
[65] Amir-Hossein Karimi et al. "A survey of algorithmic recourse: Definitions, formulations, solutions, and prospects". In: arXiv preprint arXi::2010.04055 (2020).
[66] Ilyes Khemakhem et al. "Causal autoregressive flows". In: International Conference on Artificial Intelligence and Statistics. 2021.
[67] Ilyes Khemakhem et al. "Variational autoencoders and nonlinear ICA: A unifying framework".
In: International Conference on Artificial Intelligence and Statisties. 2020.
[68] Niki Kilbertus, Matt Kusner, and Ricardo Silva. "A class of algorithms for general instru- mental variable models". In: Advances in Neural Information Processing Systems. 2020.
[69] Niki Kilbertus et al. "The sensitivity of counterfactual faimess unmeasured confounding".
In: Conference on Uncertainty in Artificial Intelligence. 2020.
[70] Hyemi Kim et al. "Counterfactual fairness with disentangled causal effect variational autoen-
coder". In: AAAI Conference on Artificial Intelligence. 2021.
[71] Diederik P. Kingma and Jimmy Ba. "Adam: A method for stochastic optimization". In:
International Conference Learning Representations. 2015.
13

---

[72] Durk P. Kingma et al. "Improved variational inference with inverse autoregressive flow". In: Advances in Neural Information Processing Systems. 2016.
[73] Max Kleiman- Weiner et al. "Inference of intention and permissibility in moral decision making". In: CogSci. 2015.
[74] Murat Kocaoglu al. "CausalGAN: Learning causal implicit generative models with adver-
sarial training". In: International Conference on Learning Representations. 2018.
[75] Gunnar König. Timo Freiesleben, and Moritz
peprrovement-focusee causal recourse (ICR)". In: AAAI Conference on Artificial Intelligence. 2023.
[76] Matt J. Kusner et al. "Counterfactual fairness". In: Advances in Neural Information Processing Systems. 2017.
[77] David A. Lagnado, Tobias Gerstenberg, and Ro'1 Zultan. "Causal responsibility and counter- factuals". In: Cognitive Science 37.6 (2013), pp. 1036-1073.
[78] John M. Lee. Riemannian manifolds: An introduction to curvature. Vol. 176. Springer New York, 2006.
[79] Sanghack Lee, Juan D. Correa, and Elias Bareinboim. "General identifiability with arbitrary surrogate experiments". In: Conference on Uncertainty in Artificial Intelligence. 2020.
[80] Ang Li, Scott Mueller, and Judea Pearl. arXiv preprint arXiv:2301. 12022 (2023).
of causal quantities". In:
[81] Ang Li and Judea Pearl. "Bounds on causal effects and application to high dimensional data".
In: AAAI Conference on Artificial Intelligence. 2022.
[82] Ang Li and Judea Pearl. "Probabilities of causation with nonbinary treatment and effect", In: arXiv preprini arXiv:2208. 095688 (2022).
[83] Ang Li and Judea Pearl. "Unit selection based counterfactual logic". In: International Joint Conference on Artificial Intelligence. 2019.
[84] Mohammad Lotfollahi et al. "Compositional perturbation autoencoder for single-cell response
modeling". In: BioRxiv (2021).
[85] Christos Louizos et "Causal effect inference with deep latent-variable models". In: Ad- vances in Neural Information Processing Systems. 2017.
[86] Chaochao Lu et al. "Sample-efficient reinforcement learning via satteerffcttal-baseee data augmentation". " In: arXiv preprint arXiv:2012.09092 (2020).
[87] Charles F. Manski. "Monotone treatment response", In: Econometrica: Journal of the Econo-
metric Society (1997). pp. 1311-1334.
[88] Charles F. Manski. "Nonparametric bounds on treatment effects" In: The American Economic
Review 80.2 (1990), pp. 319-323.
[89] Myrl G. Marmarelis et al. "Partial identification of dose responses with hidden confounders". In: Conference on Uncertainty in Artificial Intelligence. 2023.
[90]
Valentyn Melnychuk, Dennis Frauen, and Stefan Feuerriegel. "Normalizing flows for inter- ventional density estimation". In: International Conference on Machine Learning. 2023.
[91] Thomas Mesnard al. "Counterfactual credit assignment in model-free reinforcement learning". In: International Conference on Machine Learning. 2021.
[92] Wang Miao, Zhi Geng, and Eric J. Tchetgen Tchetgen. "Identifying causal effects with proxy variables of an unmeasured confounder". In: Biometrika 105.4 (2018), pp. 987-993.
[93] Arash Nasr-Esfahany, Mohammad Alizadeh, and Devavrat Shah. "Counterfactual identi- fiability of bijective causal models". In: International Conference on Machine Learning. 2023.
[94] Arash Nasr-Esfahand Esfahany and Emre Kiciman. "Counterfactual (non- moon)iddetiffabiiiity of learned structural causal models". In: arXiv preprint arXiv:2301.0900 00003 (2023).
[95] Luigi Negro. "Sample distribution theory using coarea formula". In: Communications in Statistics Theory and Methods (2022), pp. 1-26.
[96] Marcel Nutz and Ruodu Wang. "The directional optimal transport". In: The Annals of Applied Probability 32.2 (2022), pp. 1400-1420.
[97] Michael Oberst and David Sontag. "Counterfactual off-pplicy evaluation with gumbel-max structural causal models". In: International Conference on Machine Learning. 2019.
[98] Miruna Oprescu et al. "B-Leamner: Quasi-oracle bounds on heterogeneous causal effects under hidden confounding". In: International Conference on Machine Learning. 2023,
14

---

[99] Kirtan Padh et al. "Stochastic causal programming for bounding treatment effects". In Conference on Causal Learning and Reasoning. 2023.
[100] Nick Pawlowski, Daniel Coelho de Castro, and Ben Glocker. "Deep structural causal models
for tractable counterfactual inference". In: Advances in Neural Information Processing Systems. 2020.
[101] Judea Pearl. Causality. Cambridge University Press, 2009.
[102] Judea Pearl. Causality: Models, Reasoning and Inference. Cambridge University Press, 2000.
[103] Judea Pearl. "Probabilities of causation: Three counterfactual interpretations and their identi-
fication". In: Synthese 121 (1999), pp. 93-149.
[104] Drago Plecko and Elias Bareinboim. "Causal fairness analysis". In: arXiv preprint
arXiv:2207.1133
(2022).
[105] Boris T. Polyak and Anatoli B. Juditsky, "Acceleration of stochastic approximation by averaging". In: SIAM Journal on Control and Optimization 30.4 (1992), pp. 838-855.
[106] Ashesh Rambachan, Amanda Coston, and Edward Kennedy. "Counterfactual risk assessments under unmeasured confounding". In: arXiv preprint arXiv:22 2tt::22/2.008444 (2022). [107] Danilo Rezende and Shakir Mohamed. "Variational inference with normalizing flows". In: International Conference on Machine Learning. 2015.
[108] Jonathan Richens, Rory Beard, and Daniel H. Thompson. "Counterfactual harm". In: Ad- vances in Neural Information Processing Systems. 2022.
[109] James M. Robins, Andrea Rotnitzky, and Daniel O. Scharfstein. "Sensitivity analysis for selection bias and unmeasured confounding in missing data and causal inference models", In: IMA Volumes in Mathematics and its Applications 116 (2000), pp. 1-94.
[110] Jonas Rothfuss et al. "Noise regularization for conditional density estimation". In: arXiv preprint DeeXiv:1907.0999
(2019).
[111] Donald B. Rubin. "Fstimating causal effects of treatments in randomized and nonrandomized studies." " In: Journal of Educational Psychology 66.5 (1974), p. 688.
[112] Michael C. Sachs et al. "A general method for deriving tight symbolic bounds on causal
effects". In: Journal of Computational and Graphical Staristics (2022), pp. 1-10.
[113] Pedro Sanchez and Sotirios A. Tsaftaris. "Diffusion causal models for counterfactual estima-
tion". In: Conference Causal Learning and Reasoning. 2021.
[114] Pablo Sánchez-Martin. Miriam Rateike, and Isabel Valera. "VACA: Designing variational
graph autoencoders for causal queries". In: AAAI Conference on Artificial Intelligence. 2022.
[115] Numair Sani, Atalanti A. Mastakouri, and Dominik Janzing. "Bounding probabilities of causation through the causal marginal problem". In: arXiv preprint FrXXi:23044.0222. (2023).
[116] George Saon, Satya Dharanipragada, and Daniel Povey. "Feature space gaussianization". In: International Conference Acoustics, Speech, and Signal Processing. 2004.
[117] Axel Sauer and Andreas Geiger. "Counterfactual generative networks". In: International Conference on Learning Representations. 2021.
[118] Xinwei Shen al. "Weakly supervised disentangled generative causal representation learn- ing". In: Journal of Machine Learning Research 23 (2022), pp. 1-55.
[119] Ilya Shpitser and Judea Pearl. "Effects of treatment on the treated: Identification and general- ization".. In: Conference on Uncertainly in Artificial Intelligence. 2009.
[120] Ilya Shpitser and Judea Pearl. "Identification of joint interventional distributions in recursive
Semi-Markovial causal models". In: National Conference on Artificial Intelligence (AAAI). 2006.
[121] Ilya Shpitser and Judea Pearl. "What counterfactuals can be tested".. In: Conference on Uncertainty in Artificial Intelligence. 2007.
[122] Ilya Shpitser and Eli Sherman. "Identification of personalized effects associated with causal pathways". In: Conference on Uncertainty in Artificial Intelligence. 2018.
[123] Peter Spirtes et al. Causation, prediction, and search. MIT Press, 2000.
[124] Suraj Srinivas et al. "Efficient training of low-curvature neural networks". In: Advances in Neural Information Processing Systems. 2022.
[125] Esteban G. Tabak and Eric Vanden- Eijnden. "Density estimation by dual ascent of the log- liellikei.ood".. In: Communications in Mathematical Sciences 8.1 (2010), pp. 217-233.
15

---

[126] Zhiqiang Tan. "A distributional approach for causal inference using propensity scores". In: Journal of the American Statistical Association 101.476 (2006), pp. 1619-1637.
[127] Jin Tian and Judea Pearl. "Probabilities of causation: Bounds and identification". In: Annals of Mathematics and Artificial Intelligence 28.1-4 (2000), pp. 287-313.
[128] Rianne Van Den Berg al. "Sylvester normalizing flows for variational inference". In: Conference on Uncertainty in Artificial Intelligence. 2018.
[129] Athanasios Vlontzos, Bernhard Kainz, and Ciarán M. Gilligan-Lee. "Estimating categorical counterfactuals via deep twin networks". In: Nature Machine Intelligence 5.2 (2023), pp. 159-
168.
[130] Matthew J. Vowels et al. "A free lunch with influence functions? Improving neural network estimates with concepts from semiparametric statistics", In: arXiv preprint arXiv:2202.09090 (2022).
[131] Linbo Wang and Eric Tchetgen Tchetgen. "Bounded, efficient and multiply robust estimation of average treatment effects using instrumental variables". In: Journal of the Royal Statistcca Society: Series 80.3 (2018), pp. 531-550.
[132] Prince Zizhuang Wang and William Yang Wang. "Riemannian normalizing flow on variational Wasserstein autoencoder for text modeling". In: Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019.
[133] Pengzhou Abel Wu and Kenji Fukumizu. "B-intact-VAE: Identifying and estimating causal effects under limited overlap". In: International Conference on Learning Representations. 2022.
[134] Yulun Wu et al. "Predicting cellular responses with variational causal inference and refined relational information". In: International Conference on Learning Representations. 2023.
[135]
Yulun Wu et al. "Variational causal inference". In: arXiv preprint arXiv:2209. 00935 (2022). Kevin Xia et al. "The causal-neural connection: Expressiveness, learnability, and inference". In: Advances Neural Information Processing Systems. 2021.
[136]
[137] Kevin Muyuan Xia, Yushu Pan, and Elias Bareinboim. "Neural causal models for counterfac- tual identification and estimation". In: International Conference on Learning Representations. 2023.
[138] Kentaro Yamada, Manabu Kuroki, et al. "Counterfactuull-asse prevented and preventable proportions". In: Journal of Causal Inference 5.2 (2017), pp. 1-15.
[139] Teppei Yamamoto. "Understanding the past: Statistical analysis of causal attribution". In: American Journal of Political Science 56.1 (2012), pp. 237-256.
[140] Jinsung Yoon, James Jordon, and Mihaela van der Schaar. "GANITE: Estimation of individu- alized treatment effects using generative adversarial nets". In: International Conference on Learning Representations. 2018
[141] Marco Zaffalon, Alessandro Antonucci, and Rafael Cabañas. "Structural causal models are (solvable by) credal networks". In: International Conference on Probabilistic Graphical Models. 2020.
[142] Marco Zaffalon et al. "Bounding counterfactuals under selection bias". In: International Conference Probabilistic Graphical Models. 2022.
[143] Marco Zaffalon al. "Learning to bound counterfactual inference in structural causal models from observational and randomised data". In: arXiv preprint arXiv:2212.02932 (2022).
[144] Matej Zečević et al. "Relating graph neural networks to structural causal models", In: arXiv preprint arXiv:2219 009.04173 (2021).
[145] Junzhe Zhang and Elias Bareinboim. "Bounding causal effects on continuous outcome". In: AAAI Conference on Artificial Intelligence. 2021.
[146] Junzhe Zhang and Elias Bareinboim. "Fairness in decision makiing: The causal explanation
formula". In: AAAI Conference on Artificial Intelligence. 2018.
[147] Junzhe Zhang, Jin Tian, and Elias Bareinboim. "Partial counterfactual identification from observational and experimental data". In: International Conference on Machine Learning. 2022.
[148] Kun Zhang and Aapo Hyvärinen. "On the identifiability of the post-nonlinear causal model". In: Conference on Uncertainty in Artificial Intelligence. 2009.
16

---

A Extended Related Work
A.1 Counterfactual inference
In the following, we explain why existing work does not straightforwardly generalize to the partial counterfactual identification of continuous outcomes. In particular, we do the following:
1. We pinpoint that symbolic (non-paranee )oo-maramerrre) counterfactual identifiability methods only provide the probabilistic expression suitable for point identification, a certain query can be expressed via lower layer information.
2. We note that existing point identification methods do not have any guidance towards partial identification, and often do not even provide identifiability guarantees. In this case, valid point identification can only be achieved via additional assumptions either (i) about the SCM or (ii) about the data generating mechanism.
3. We discuss why methods for partial identification of (i) discrete counterfactual queries, (ii) contin- uous counterfactual queries with informative bounds, and (iii) continuous interventional queries can not be extended to our setting.
Ultimately, we summarize related works on both point and partial identification for two layers of causality, namely interventional and counterfactual, in Table
Table I: Overview of methods for point and partial identification, assuming a known causal diagram. Relevant methods are highlighted in yellow.
Layer
MIM
Symbolic idcntifubiiity
Point identifiaaions metheds
Diserete outcumes
Partial identiication methods
Continats outesmes
man-informaiive Bounds Infommalive boand
Always
genenaise model
critzion
Eg Inareenionn
Do-caleulas
reles 36
Potental nutcomes finmewerk 22
Partially observed back-
No-sscmmptin Saand i
MSM 40
41 89
ATD
peobubliity
502 [BU] binayy
Hiuus doee varables 11
MSM 153854 tog 11 Co
grony varaells
comoicat SCM 166
126 malesan
SM
maddl [10)
founding functions
[09]
IV
E. Counter
works [12
theorem
Parallel worlds net
Safiatiin unnesting
123
tefociaa
Deap seace mudkl 21.29
Marlovan
[E 114
Inasport based coun
McMs WW pn [1
72 12]
pullopecilie
117
[
linss tramework / canen
icai partiiooiig
PN. PS, PNS [
[4]
respanec lune-
[
44
97
tean 37 cosservative coun
Viriace of the trotiment of- Iects [ CDF of the siitt
theocpp terianmal effects [4]
potethi mutormes distribu-
discueiin in Appendic D
CSM (this paper)
paper)
:seni-
129
hidden confounding
BCMS
SCM
1. Symbolic identifiability and point identification with probabilistic expressions. Complete and sound algorithms were proposed for symbolic monnparrmettric identifiability of the counterfactuals based on observational (L1) or interventional (C2) information, and causal diagram of a semi- Markovian SCM. For example, [I2T] adapted d-separation for the parallel-worldh worlds networks (3] for simple counterfactual queries, and (27) extended it to nested counterfactuals with counterfactual unnesting theorem. These methods aim to provide a probabilistic expression, the case the query is identifiable, which is then used in downstream point identification methods. Rather rare examples of identifiable queries include the treatment effect of the treated (ETT) [119] and path-specific effeets
122
[46] for certain SCMs, g., when the treatment is binary. If the query is non-identifiable. like
in the case with ECOU [ECOT] as in our paper, no guidance is provided on how to perform partial identification and methods can not be used.
2(1). Point counterfactual identification with SCMs. Another way to perform a point identification when the counterfactual query is symbolically non-identifiable is to restrict functional class in SCMs. Examples of restricted Markovian SCMs include nonlinear additive noise models (ANMs)
[66] [23]. post nonlinear models (PNL) [148. location-scale noise models (LSNMs) (5]], and
bijective generation mechanisms (BGMs) 93 94 The latest, BGMs, also work in Meri-Mark
settings with additional assumptions. Numerous deep generative models were also proposed for
17

---

point identification in both Markovian and semi- Markovian settings. Examples are normalizing flows
1000;
diffusion models (2T [13]: ; variational inference (70, 100 [14 [35]: adversarial learning
[31]. As was shown in (94), all those deep generative models above need to assume the BGM of the data to yield valid counterfactual inference The assumption of monotonicity of BGMs effectively sets the dimensionality of the latent noise to the same as the observed variables, which is rarely a realistic assumption. In our paper, we relax this assumption, and let the latent noise have arbitrary dimensionality.
22 88
T00
117.
118 140] and neural pnaectatoo-maximizaae
2(ii). Point counterfactual identification with non-SCMs-based approaches. Altemative, non- SCM-based definitions of the counterfactuals exist, e. g.. transport-based counterfactuals [32 [6]. and conservative counterfactual effects [4). Still, for the identifiability of ECOU [ECOT], they also rely on additional assumptions such as monotonicity. [5] These approaches were shown to coincide with standard SCM-based counterfactuals when the latent noise can be deterministically defined with observed data. Therefore, they coincide with BGMs.
3().. Discrete partial counterfactual identification. Partial counterfactual identification was rigorously studied only for discrete SCMs or discrete outcomes. For example, bounds under no assumptions were derived for counterfactual probabilities 6 80
103
127 More general
counterfactual queries can also be tackled with response functions framework 6 and, more generally,
canonical partitioning [12] in combination with deep neural networks 197 137
141
142 143
147]. The same idea was used for causal marginal problem [46 where the authors combined different experimental and observational data with overlapping observed variables. However, response functions framework and canonical partitioning can not be extended to the continuous setting in practice, as their computational complexity grows exponentially with respect to the cardinality of the observed variables. Hence, the aforementioned methods are not relevant baselines in our setting.
3(i). Continuous partial counterfactual identification of queries with informative bounds. Certain counterfactual queries have ignorance intervals with informative bounds, which have a bounds for copulas. For example, informative bounds
closed-form solution using Frechet- were derived, e. g., for the variance of the treatment effects (2). Var(Y. - Yar); for the CDF of the joint potential outcomes distribution (37j. P(,.: Yar), and other related queries (4). Yet, these bounds cannot be applied to our setting, as our query of interest, namely, ECOU [ECOT], has non non-informattive bounds and, thus, requires a sensitivity model.
3(ii). Continuous partial interventional identification. The problem of partial interventional identification arises in semi-Markovian SCMs and usually aims at hidden confounding issues of treatment effect estimation (88). For example, instrumental variable (IV) models always obtain informative bounds under the assumption of instrumental validity (48), so that partial identification
is formulated as an optimization problem [55
68 99 [45]. In other cases, hidden confounding
causes non-informative bounds and additional assumptions about its strength are needed, e. a sensitivity model. For example, the marginal sensitivity model (MSM) assumes the odds ratio between nominal and true propensity scores and derives informative bounds for average or conditional average
treatment effects (ATE and CATE, respectively) [15] 35 40
41
60,
61
62
89
98
126). The
were also develops a sensitivity model for ATE assuming noise level of proxy variables. All the mentioned sensitivity models operate on conditional distributions and, therefore, do not extend to counterfactual partial identification, as the latter requires assumptions about the SCM. Some methods indeed restrict functional classes SCMs to achieve informative bounds. For example, linear SCMs are assumed for partial identification of average treatment derivative (5). By developing our CSM in our paper, we discuss what restrictions required for SCMs to achieve informative bounds for partial counterfactual identification of continuous outcomes. Nevertheless, the methods for partial interventional identification are not aimed counterfactual inference, and, thus, not directly applicable in our setting.
outcome sensitivity models [5] 106) and confounding functions framework
17
109
introduced for the sensitivity analysis of ATE and CATE. [49]
'In semi- semi-Markouee Markovian SCM additional assumptions are needed about the shared latent noise 31 "In the context of the non-S SCM- -baedd counterfactuals, the monotonicity assumption assumes stochastic dominance of one potential outcome over the other [87 103 i. e.., P(Y. 2 Y.:) = 1.
18

---

A.2 Identifiability of latent variable models and disentanglement
The question of identifying latent noise variables was also studied from the perspective of nonlinear independent component analysis (ICA) [7 57 67]. Khemakhem et al. [67] showed that the joint distribution over observed and latent noise variables is in general unidentifiable. Although, nonlinear ICA was applied for interventional inference [133), these works did not consider SCMs or counterfactual queries.
19

---

B Background materials
Geometric measure theory. Let ((x) R + IR be a Dirac delta function, defined as zero everywhere. except for z = O, where it has a point mass of 1. Dirac delta function induces a Dirac delta measure, so that (with the slight abuse of notation)
J [(x) [x)))())== J ((o) = I(0),
(12)
where f is a O function with compact support. Dirac delta function satisfies the following important equality
/ (())((())))
E f(z:)
(13)
lg' (x:4
where is C function with compact support, is a C function, g'(x,) # O, and Ty are roots of the equation g(x)
In addition, we define the s-dimensional Hausdorff measure H", as in 195 Let E S R" be a s-dimensional smooth manifold (s n) embedded into R Then, E is Borel subset in R", is and s-dimensional Hausdorff measure H (E) is the s-dimensional surface volume of E. For example, if the Hausdorff measure coincides with line integral, and, if 8 2, with surface integral:
Hausdorff-measure
xi(())
dx((t) dt dt,
1:
E=
= x(t)
(x)
(14)
(1)
(s)
[d((s,)) t) dx(s,t) t)
= 2:
E =
= x(
dTH(())
ds x
dt ds (15)
Ie (s,
where X is a vector product, 2x( is t-parametrization of the line, and (, is a (s,
parametrization of the surface.
Also, (F) vo)" for Lebesgue measurable subsets E S R", where vol" is standard n--imensional volume (Lebesgue measure). In the special case of s = 0, Hausdorff measure is a counting measure: Jrz (x) d (z) ΣeER f(x).
Importantly, the Hausdorff measure is related to the high-dimensional Dirac delta measure via the coarea formula 53 (Theorem 6.1.5), (95] (Theorem 3.3):
f gm f(x))(((x)) dx
f(x) [xg(1)]l dH "xx",
(16)
(=(x)))=
where f is Co function with compact support, and g a C* function with Vxg(x) > for x E E. Functions for which Vxg(x) > for x E IRd holds are called regular.
We define a bundle of level sets of the function y == f(x) as a set of the level sets, indexed by y. ie., {E(v): E Y' R), where E(y) {y E R : f(x) y} and C R. A bundle of level sets of regular functions are closely studied in the Morse theory. In particular, we further rely on the fundamental Morse theorem, e., the level set bundles of regular function diffeomorphic to each other. This means that there exists a continuously differentiable bijection between every pair of the bundles.
Differential geometry of manifolds. In the following, we formally define the notion of the curvature for level sets. As the result of the implicit function theorem, level sets E(y) of regular function of class C are Riemannian manifolds (78). namely smooth differentiable manifolds. Riemannian manifolds can be locally approximated via Euclidean spaces, i. e., they equipped with tangent space Tx(E Ey)) at point x, and a dot product defined on the tangent space. The tangent space, T&( Tx(E(v)) is orthogonal to the normal of the manifold, (x)
For the level sets regular functions of the class C2, we can define the so-called curvature 43 78 Informally, curvature defines the extent a Riemannian manifold bends in different directions.
20

---

Convex regions of the manifold correspond to a negative curvature (in all directions), and concave regions positive curvature, respectively. Saddle points have curvatures of different signs different directions. Formally, the curvature is defined via the rate of change of the unit normal of the manifold, which is parameterized with the orthogonal basis (#1,.... 1n of the tangent space, Tx (x))) This rate of change, namely differential, forms a shape operator (or second fundamental form) on the tangent space, xx( (w)).. Then, cigenvalues of the shape operator are called principal curvatures, 1}. Principal curvatures are measure of the extrinsic curvature, i. e., the
<x(x) for i {1, {1..... curvature of the manifold with respect to the embedding space, R
Principal curvatures for Riemannian manifolds, defined as the level sets of regular C² function f. can be also expressed via the gradient and the Hessian of the following function (43)
- X root, det. [Hessy f(x)-
Vxf(x))
(Vxf(x))
0
N; (x)
iE (L....,, n 13.
(17)
[x)(x)]l2
where root, are roots of the equation with respect to X E IR. For x E R2, the level sets Fly) are curves, and there is only one principal curvature
Ki (x - - Vx
Vx[(x)
(18)
One of the important properties of the principal curvatures is that can locally parameterize the manifold as the second-order hypersurface, i. e.,
(R|
K (11)))))
(19)
where (......, -n-11 } is the orthogonal basis of the tangent space TN (E()))
21

---

C Examples
Example I (Counterfactual non-identifiability in Markovian-SCMS (continued)). Here, we con- tinue the example and provide the inference of observational (interventional) and counterfactual distributions for the SCMs Mi and M2.
Let us consider MI first. It is easy to see that a pushforward of uniform distribution in a unit square with fy Uy,, ) A (Uy1 Uya +1 + (1 A) (Uy1 + Uya - 1) induces triangular distributions. For example, for fy Uy1, Uva), a cumulative distribution function (CDF) of (Y AA = =Y=o) will have the form
My A 0) = pMMi( (Y S y
A = 0) = P(Uy' + +yy+ - -
(20)
0. [(uyituys 1ss duy1 duya,
-1) (v 1),
(21)
otherwise
0,
(u S -1) (y>>),
[0+1)²
V (-1,0).
(22)
y E (0,
which is the CDF of triangular distribution. Analogously, a CDF of TY | A 1)
P.M: 1==1) is
A = 1) pMi (Y S y A= 1) P(Uy1 Uyz + 1 15 BMM (yl
(23)
0,
(v V (u > 2),
(24)
[uyy +1 duy duyz,
otherwise
O) (y > 2),
y (0,1),
(25)
(2 "
y E (1.
To infer the counterfactual outcome distribution of the untreated, PM (Ya=1 A' = 0, Y,Y' = 0), we
make use of Lemma and properties of the Dirac delta function (e. g., Eq. (3)). We yield
|
5(uy: uy + -1-1) y)
P.Ms (Ya-1 A'= Y, 0)
dll (uy)
[uyxtmys -1=0] | (uys uyy 1)1l2
(26)
(1)
2
(())
2
8(t y) dt 2..0 (- y( dt
1/ y [0.2].
2 +
(27)
2o
10, otherwise,
where (+) introduces a parametrization of the line {uyi + +uy - 1 0} with t, namely
(uyi) - 1t uya) (1=4))-
[022] = d7l = v (1) + 1
dt.
(28)
2
Therefore, the ECOU for Mi
1- 2
2441 (0) = 1
dy 11
(29)
Now, let us consider M2. Hence, Jy (0, Uy1, Uv2) is the same for both M2 and My. so are the
observational (interventional) distributions, i.e. IMMi (Y == 0) -MMa (Y A 0). The
same is true for (1, Uy2) with (0 < Uy I 1) (Uvi & Uya s 1) or, equivalently,
P.1 (Y y lA= = PaMa) = A= with y (0,1).
Now, it is left to check whether TP = A = 1) is the density of a triangular distribution for y E (1, For that, we define level sets of the function (1, Uy,, Uy2) in the remaining part of the unit square in a specific way. Formally, they are a family of "bent" lines in the transformed two-dimensional space u uyz with
uyz= = 8t2 |ūyz + b(t), E 10, uy. E 11-1,1]
uyz E 10, 1 --uy ll.
(30)
22

---

where b(t) is bias, depending on t. The area under the line should change with quadratic speed, as it will farther define the CDF of the induced observational distribution, EM (y A 1 / 21.
L.
S(1
21 c²
(31)
so that 5(0) 0 and S(1) = 1. On the other hand, the area under the line from the famity is
/
p.
S(1) = 2
uyr + b(t)) duys + (1 - m. = 8l?02 + +ūū + (1 - u. )
(32)
1-b(() 814 Therefore, we can find the dependence of b(t) on t, so that the area S(t) is
where u, preserved:
21 --ビ 12 = = + 2ū.b(t)) + +1-+0111111 (t- +1.
(33)
Let us reparametrize t with = y - 1, so that the line from a family with t = 0 corresponds to y = 1 and to y = 2, respectively. We then yield
uy= = 2 1) أرyا + (w- 2
(34)
In order to obtain the function fy (1, UY,, Uvz) in the original coordinates uy uyz, we use a linear transformation T given by
T:
ūy uyz
uy² - ) Uy -
-
(35)
uuy uy²
Hence, the family of "bent" lines can be represented as the following implicit equation:
Fly, uy1, uy2) = - 2 ty
uy2 + 1111111440 +1 /v 2)2 (8 1)2 + 0.
(36)
Importantly, the determinant of the Jacobian of the transformation T is equal to 2; therefore, the area under the last line S(1) shrinks from (in space ūy:; ūya) to 0.5 (in space uy uy2). ). Thus, we can also easily verify with Eq. Q3T) that the CDF of the induced observational distribution coincides with the CDF of the triangular distribution for y E (1,2]:
JMM2(y (v) | (-=1)= = 1 2 (1 -
1 1 S(y - 11= =1 (2 (2-
y)²
+ 2
(37)
2
To infer the counterfactual outcome distribution of the untreated, i.e., PY-2 A'' 0, Y = 0), we again use the Lemma
2 and properties of Dirac delta function (e. g., Eq. (13 We yield
1
P.Ma (Ya= = y A' = Y =0) = 2' y (0,1,,
=0)=
(38)
(Y = A' = Y. y 0) /
8(F uy1,
d7l (uy)
[uyittuy -1=0} Vuy (uy: + uyy - 1)l12
(39)
1 2
2
V2 Jo V 2
+
(1 11 V 2) di
(40)
1 - - (8 3 200-)
(5- 4y)4y)²(2 - y)
E (1,2),
(41)
2
WV - 1)² +1)
where (+) introducce a parametrization of the line {uyι + uuz - -1- 0} with l, namely
(uy:) uyz)
- (3-t))) I LE [0,0
dH'' = v (
2
1
dl.
(42)
2
Finally, the ECOU for M2 can be calculated numerically via
//') 1 4d I 2/11 - 2)2 +1
(5 (5-4y) (2)
2643 (0)
dy ~ 4
+ 0.864 R 1.114.
(43)
2
23

---

Example 2 (Box-Müller transformation). Here, we demonstrate the application of the Lemma 7 to infer the standard normal distribution with the Box-Müller transformation. The Box-Mülle transformation is well-established approach to sample from the standard normal distribution, which omits the usage of the inverse CDF of the normal distribution. Formally, the Box-Miller transformation is described by an SCM Mom of class B(C², , with the following function for Y:
Mom fy (A, Uya, = (Uy Uyz) = /-2 log(UUr)) cos(TUv=).
(44)
We will now use the Lemma to verify that MoM (Y a) = No, 1). We yield
P.Mi (Y = a) 1 /V- -llogguy))) cos(xuy2)=g)
dyy
Var (v - log(uy1) cOs(TTyy)))
(45)
d7l (ux)
(46)
(1y1) (oo(Nuy2)=))
ros (xN2))
2 nr² log(uy)) sin (inn))uunn((
2 u log(
my² sin cus' (T
1/2
+
exp
cos
1/2 lul exp (000
dt
dt
(47)
cOs2(T()
(ot)
1/2 000' 23
x² (nt)y" (x
exp
122
exp
0,1:0,1).
(48)
22T
2
where (+) introduces a parametrization of the level set (V (V-22og logg(uy) cos(TTyy)) -y, with I, namely
(- 20000(1133 )) LE (0.
uyi uyz
eep
(49)
(ux)
=
( TY² sin( 1)
(T
exp
2 cos2 t)
y22
+ 1 dt.
(50)
This parametrization is also valid for y < 0 and (5), 1]. Due to the symmetry of the function fy, we only consider one of the cases (see Fig. 8 (left) with the level sets).
N1
1r
Figure 8: Box-Müller transformation as an example of the SCM of class B(C 2) (see Example 2b
Here, (y; 0, 1) the density the standard normal distribution.
FFrmally, fy C. only for uy 1)
24

---

Example 3 (Connected components of the factual level sets). Here, we construct the function with the level sets consisting of multiple connected components. For that, we extend Example with the Box-Müller trunsformation. We define a so-called oscillating Box-Müller transformation
fy (Uy)
1-2log(Uyr cos(2-1log:)) ([looz(UU>2))))11 mUv=).
(51)
We plot the level set for y = 0.5 in Fig. _
namely Ely) = {uy E [0, 1]2 [v ( = y)
Here, we see that the level sety consist of an infinite number of connected components with an infinite total length.
B(- 1.00
B(-0.5)
E10.5)
Bi1
1.0
1.0
1.0
1.0
0.6
0.4
0.4
0.4
01.4
0:2
0.2
0.0
0.0
0.0
0.00
0.25
0..00
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
05e
0yy
Uy:
0y'
Figure 9: Level sets with multiple connected components Ely) fur E [0,1]3 fy fr(y,,,uy)) y) for oscillating Box-Müller transformation (see Example B).
Example (Curvature of level sets). Let us consider three SCMs of class B(C², 2), which satisfy the Assumption [ with function fv (see Eq. (TT). Fig. provides a heatmap with the absolute curvature, (Ra (uy) ) for the counterfactual level sets, (E a) E Va Here, all three SCMs are instances of our APID.
50. Then, the curvature of the level sets is properly defined for the
06-
05-
41-
07-
0.2
003
3.6
05-
Figure 10: Curvature of the counterfactual level sets [Ely. a) y E VoF in blue for three SCMs, satisfying the Assumption Factual level set for the quantile of the distribution, E(F (0.5). a') given for reference in orange. As can be seen, "bent" level sets have larger curvature (right) than "straight" (left). Here, = Maa KK (1(y)) amounts to 50.
Example 5 (Mu)). There exist BGMs in class B(C², 1), for which the curvature of level sets are not properly defined at certain points. For example, an SCM Mn, which induces triangular distributions, like in Example 7 For this example, Min will have the following function for Y:
( (2 + (A- 1) (V2Ux - 1), A - /2 1)) (4 (A-
UUv E (0.0,0.5] Uf (0.5, 1.0).
Mmi fr,
(2(Ux+1))
(52)
Example 6 (Maa). Here, we provide examples of SCMs in the class B for which the bundles of the level sets coincide for both treatments, i. e., {Ely, a) y E Va [E(y, a') y Ve and all the level sets are flat hypersurfaces (K 0). For example, SCMs with the following functions for Y:
Mn fy I(A, wi Uy: +... waUU..)
(53)
where g(a,,) :) is an invertible function in class O and Wi.... wa are coefficients from the linear combination. After a reparametrization, is always possible to find an equivalent BGM with the function:
g(A, w/UYr +... waUya) ((A,ūy) = g(A, F. Ey (Uv:)),
(54)
25

---

where F By (-) is an inverse CDF Uy = w2Uvi + +... WaUya. The smoothness of the inverse CDF then defines the smoothness of the BGM.
Example (Meurv). This example is the generalization of the Example 6 Here, we also use the same bundles of the level sets for both treatments, but allow for the curvature of k > 0:
Meurv (A
= ((A,)(Uy)),
(55)
where g(a, .) is an invertible function in class C and h(.)) [0, 10 + R is a function in class C with bounded by is curvature of the level sets. In this case, we also can reparametrize the latent space with non-linear transformation and find an equivalent BGM:
g(A,)(Uy)) = -(A,Uv)) g(A, By (Uv:))
(56)
where F F-1 () is inverse CDF of Uy M(U)). The smoothness of the inverse CDF then Uy analogously defines the smoothness of the BGM.
Example (Mparp.. We can construct an SCM of class B(Cx 2). so that the level sets are all straight lines (R. = 0) and perpendicular to each other for different a E {0,1), g.
Mperp : (A, Uy Uyz) A (UV=) ) + (A 92Uv=),
(57)
where g1 .2))))))))))))))))) are invertible functions in class C this case, the ECOU for E Va' always evaluates the conditional expectation
(i(uy1)
Q (3¹)
dll (ux)
(58)
MM (Y 3/ A - (1y2=97'(w)) (v/)) V (lya02((yz))
MM (Y =
A A=0) 0)
So (u(y))) duys = M.M (Y AA= 1). (59)
(uyu
P.Meo (Y 1/ A 0)
Similarly, the ECOT is (M A 0). This result is, in general, different from the result of
BGMs, which, g., would yield Q. (y') = (0.5) for s/ (0.)..)). Thus, point identification not guaranteed with k = O.
26

---

D Proofs
Uniformity the latent noise. Classes of SCMs (( .d) assume independent uniform latent noise variables, i.. Uy Unif(O, 1)d This assumption does restrict the distribution of the latent noise. Namely, for a bivariate SCM Mnon-unif with d-dimensional non-uniform continuous latent noise, it is always possible find an equivalen |CM Munit of class B B(C*, d). This follows SerralennsSaa
from the change of variables formula and the inverse probability transformation [25 116]:
Moon-unit Y = (A,
Uy P(Uy); (a C*,
(60)
Munit = (v(A, = (y(,UU,,, Uy ~ Unit(o, (v(,))) E Cmmin(I.k)
(61)
where Uy = T.UY). and T()) (0, Rd is diffeomorphissm (bijective C' transformation) and solution the following equation:
EC
(T
(ūv)
P(Uy = uy)
det
(62)
dìty
EC
EC
For further details the existence and explicit construction of T()), see [25 Thus, fr is of class (mmin(1.k), as the composition of T()) and (a,
In our CSM we require the functions of the SCMs to be of class C2, In this case, B d) also includes all the SCMs with non-uniform continuous latent noise with densities of class C This also follows from the change of variables theorem, see Eq. (62).
More general classes of functions, e. g.g., all measurable functions, more general classes of distri- butions, e. g.. distributions with atoms, fall out the scope of this paper, even though we can find bijective (but non-conninuous) transformations between probability spaces of the same cardinality (see the isomorphism of Polish spaces, Theorem 9.2.2 in (12)).
Lemma 1 (Observational distribution as a pushforward with fv). Let M B(C dd). Then, the density of the observational distribution, induced by M, is
PM y
| (BCy
dux))
(I)
fo (a. ul
where Fly, a) is a level set (preimage) of y, i. e., Ely, a) {uy E 1]4 fr(a, = y), and Hd Hd-)(uy) is the Hausdorff measure (see Appendix B for the definition).
Proof. Lemma is result of the coarea formula (see Eq. (16):
d. = y a) = Io.11 PM(y = y, Uv = -uy a) duy
(63)
pM = = v luy wuu,r (Ux = uy) duy
(64)
S V (a, uy
S(fr(a,ur) v) duy
dH (uy),
1.11
(65)
where (+) holds, as Uy independent of A. In a special case, when we set d we obtain a change of variables formula
pMiy- = = = /
dHL
Σ
Vur (v,
(66)
B(yya) V fy (a, uy
E(E.a)))
'Equivalence of the SCMs is defined as the almost surely equality of all the functions in two SCMs 13
27

---

Corollary 1 (Identifiable functions (BGMs) (93)) The function fy (a. uy) can be identified (up to sign) given the observational distribution [ a), induced by some SCM M of class B(C", 11 if is strictly monotonous in uy. In this case, the function is
fy fa, uy) = F (+uy F 0.5
(67)
where F~1 an inverse CDF of the observational distribution, and the sign switch corresponds to the strictly monotonically increasing and decreasing functions.
Proof. As d 1, we can apply the change of variables formula from Eq. (66)
P(Y a)
Σ
uuy fr uy) = V Jr(a uy))+ My fr (a, (68)
E(yia)
where (a, y) is an inverse function with respect my (it properly defined, the function is strictly monotonous). Thus, to find fy, we have to solve the following differential equation [166]
P(Y (a, uy) a) = Vay (a uy "
(69)
By the properties of derivatives, this equation is equivalent to
P(Y = y a) = (a.))
(70)
Since the latter holds for every y, we can integrate it from - xc to fr uy):
(y(a
P(Y = dy f
(
V Vyf (a. dy dy
(71)
By
(a) dt = -o dt = uy,
fy is strictly increasing,
(72)
1/4 (0.-)))) dt = Suy dt = 1 uy,
uy
fy is strictly decreasing.
Therefore, Fa (a ux)) = tuy F 0.5 + 0.5.
Corollary 2 (Critical points of fy(a, )). Lemma 7 also implies that critical points of JY (a, uy). i.e., {uy E 1] : [|V fy (a, uy 0}, are mapped onto points y with infinite density. Therefore, the assumption, that the function is regular, namely uuy fy (a, uy) > 0, is equivalent to the assumption of the continuous observational density with the finite density.
Lemma 2. Let M E B(C", d). Then, the density of the counterfactual outcome distribution of the [un]treated is
1 PMa - d',v'' I' = PM(Y = v a') E(y',a') V JY (a', uy)ll2
(0 uy) - 3) dHd d
(2)
where 5(.) is a Dirac delta function, and the expected counterfactual outcome of the [un]treated, i.e. ECOU [ECOT], is
Qai = MYa d', d'.v)= = P.McY = D la')
1
J Maoyiar Wyy uy)
fr(
dz (uy),
(3)
where E(yl,a') ,a) is (factual) level set of i.e. Ely', a') {uy E [0. 1] fr(a',ux) 3/3 and a' $ a
Proof. Both counterfactual queries can be inferred with the abduction-actions cetiriooonaatiionppreiititooonno procedure.
(1) Abduction infers the posterior distribution of the latent noise variables, conditioned on the evidence:
PM,y uy A = d' Y = v ( PM = y a uy IpM uy) I-M uy a', d'.y/)- P. =  = w/)
PM(Y = 1/ aa)
(73) (74)
8(fr(a', ( - v//)
PMIY 3/ |d)
28

---

where (+) holds, as Uy is independent of A.
(2)-(3) Action and prediction are then pushing forward the posterior distribution with the counterfac- tual function. For example, the density of the counterfactual outcome distribution of the [un]treated
is
PM(Y. = y v''.z')
10.1j4 pu (Va -uy | ld'."')e duy
(75)
10.11e
PYa y I '' 3', uy) pM pM = uy | ///;/) duy
(76)
punyY = 1f 10''') Ja.ja 8(fr (a, uy) - y) 6(fr (a', ux) - duy (77)
8[Sr(a, uy) y) a
(78)
PM = a') E(s',a") |V fo (a', uy
where (*) holds due to the independence of Y from Y' and A', conditional on Uy (see the parallel worlds network in Fig. D. The inference of the ECOU [ECOT] is analogous
(0) = B- (Va d', d'. = //2 Jo.11 PM(Y = y, Uy = uy 1 d'.v/) duy dy
(79)
= =1f 10.1" EM(Yo a' uy) ( (Uy- = uy I ld',v/ v/) duy
(80)
PMiyy v' d') Soaape (r(a, (r(a,, (a' -1 dur
(81)
PMiYY = 3/ a') S megy.e)) V Jy (a', ly
fr(a, (ux))
(82)
where (+) holds due to the independence of Y from Y' and A', conditional on Uy (see the parallel worlds network in Fig. ... and due to the law of the unconscious statistician.
Corollary 3 (Identifiable counterfactuals (BGMs bounds) 93).. The ECOU [ECOT] can be identified (up to a sign) given the observational distribution P(Y a). induced by some SCM M of class B(C², 1) if it is strictly monotonous in uy. In this case, the ECOU [ECOT] is
= R (v') = 0.5 + 0.5).
(83)
Proof. The corollary is a result of the the Lemma 2 and the Corollary
Q0')000) PM(Y aa')
1
J Bigrar May fy fa(aay uuy) dTx)
Sy(a, uy)
(84)
fy(a,
Fu V (a'
ff(a, uy fF ./).
(85)
Therefore, the ECOU [ECOT] is
( (- F 0.5 0.5) (+E F 0.5 + 0.5).
(86)
Theorem (Non-informative bounds of ECOU (ECOT)). Let the continuous observational distribu- tion P(Y a) be induced by some SCM of class B(C d). Let P(Y a) have compact support Va = Ma un] and be of finite density IP = a) < +00. Then, the ignorance interval for the partial identification of the ECOU (ECOT] of class B B(C .d), d 2 2, has non-informative bounds:
Qa (y') la and Qa -a (y')
Ua-
Proof. Without the loss of generality, let us consider the lower bound of the ECOU [ECOT], namely,
Qa- = inf
ax s.t. Va (0, 13 P(Y a) PM
| a). (87)
MMBB(CKK) d))
29

---

The proof then proceeds in two steps. First, we prove the statement of the theorem for d = 2, i.e., when latent noise is wwo-dimensional. Then, we extend it to arbitrary dimensionality.
Step I (d = 2). Lemma suggests that to minimize the ECOU [ECOT), we have to either increase the proportion of the length (one-dimensional volume) of the factual level set, which intersects the bundle of counterfactual level sets, or change the counterfactual functions, by "bending" the bundle of counterfactual level sets around a factual level set. Here, we focus on the second case, which is sufficient to construct an SCM with non-informative bounds.
Formally, we construct a sequence of SCMs {M y E Va.0 < <(v) < 1] of class Musii B(C ,2), for which Qa (v') gets arbitrarily close to la as + La and =(v) + 0.
For all (M nan-int E Va, < <(v) < 1}, we choose the same factual function fr Uydd F.
(88)
where F (+) is the inverse CDF of the observational distribution P(Y a'). This is always possible as a result of the Corollaries [ and 2 Hence, all the level sets of the factual function are horizontal straight lines of length I (see Fig. due to E(y', a') {I (uy²) = =
Now, we construct the counterfactual functions in the following way. For fixed e(y). we choose the level sets of the function (a.-), {E<l)) (y. y).y E Val. so that they satisfies the following properties. (1) Each ( (y. a) intersects the factual level set Ely', a') at a certain point once and only once and ends at the boundary of the boundaries of the unit square. (2) Each E a) splits the unit square on two parts, an interior with the area Fa (v) and an exterior, with the area - Fa (v). respectively (where (-) is the CDF of the counterfactual distribution). This property ensures that the
induced CDF coincides with the observational CDF at every point, namely, Vy Va Fa FMMai (v) (a(y)- (3) All the level sets have the nested structure, namely, all the points of the interior of Eetv)(,, a) are mapped to y < y. and points of the exterior, to T > y. Additionally, for some fixed y E Va. set E(y) the proportion of the factual level set, E(y', ('y',a') a'), fully contained in the exterior of the counterfactual level set, Evy, a). Thus, the interior of the counterfactual level set covers E(y) of the factual level set.
Therefore, the ECOU (ECOT] for M=(x) for some fixed E Vo is
a'
= (1- (1)
E Mas w). E [y, ua].
(89)
which follows from the mean value theorem for integrals, since, for every y E Va, we can choose E(y) arbitrarily close to zero, and, thus
inf Qu--aa
mosit (y')
inf
(1 e(v))y + e(y)y la-
(90)
yEV. (cc(o)
V vcX,,)ca())
Step (d > 2). The construction of the factual and counterfactual level sets extends straightforwardly to the higher dimensional case. In this case, the factual level sets are straight hyperplanes and the counterfactual level sets are hypercylinders, "bent" around the factual hyperplanes.
Theorem 2 (Informative bounds with our CSM). Let the continuous observational distribution P(Y' a) be induced by some SCM of class B (CC2, d), d > 2, which satisfies Assumption K Let P(Y a) have compact support Va = [a. ua]. Then, the ignorance interval for the partial identification of ECOU [ECOT] of class BCC .d) has informative bounds, dependent on K and d, which are given by Qo'+ [(K, d) la and Qo') u(k, d) < Va.
Proof. We will now show that ECOU [ECOT] always have informative bounds for every possible SCM satisfying the assumptions of the theorem. Without the loss of generality, let us consider the lower bound, i. e.,
Qa Qu/=a((/)) (EB((KK d)
inf
S.L. Va E (0,1) P(Y a) PM a).
(91)
The proof contains three steps. First, we show that the level sets of C2 functions with compact support consist of the countable number of connected components, each with the finite Hausdorff
30

---

measure. Each connected component of almost every level iet thus a d - 1-dimensional Riemannian manifold. Second, we demonstrate that, under Assumption K almost all the level set bundles have nested structure, are diffeomorphic to each other, have boundaries, and their boundaries always coincide with the boundary of the unit hypercube [0, 1j.. Third, we arrive at contradiction in that, to obtain non-informative bounds, we have to fit a d-ball of arbitrarily small radius to the interior of the counterfactual level set (which is not possible with bounded absolute principal curvature). Therefore, CSM with Assumption X has informative bounds.
Step I. The structure of the level sets of C2 functions with the compact support (or more generally, Lipschitz functions) were studied in (T]. We employ Theorem 2.5 from [ which is a result of Sard's theorem. Specifically, the level sets of C2 functions with the compact support consist of the countable number of connected components, each with the finite Hausdorff measure (surface volume), namely, (F a)) < oo for a E {0, 1]. In Example 3 we provide an SCM with the level sets with indefinitely many connected components. Furthermore, connected components of almost every level set are (d 1) di-mennniaa Riemannian manifolds of class C2. Some points y have (d - 2)- and lower dimensional manifolds, but their probability measure is zero (e. g., the level sets of the bounds of the support, Ela,, a) and E(ua, a)).
Step 2. The Assumption k asssmees the existence of the principal curvatures at every point of the space (except for the boundaries of the unit hypercube [0. 1] for both the factual and the counterfactual function. Thus, the functions of the SCMs are regular, as otherwise the principal curvatures would not be defined at the critical points (see Eq. (7p). As a result of the fundamental Morse theorem (see Appendix B). almost all the level set bundles are nested and diffeomorphic to each other. Specifically, the level set bundles (F(y, a) : y E (la, v²)} for a E {0, 1). The latter exclude the level sets of the bounds of the Va. i. e., F. a) and E(ua; a) for a E {0, 1), which in turn are laying completely in the boundary of the unit hypercube 1] Another important property is that all the level sets have a boundary (otherwise, due the diffeomorphism, the critical point would exist in their interior) and this boundary lies at the boundary of the unit hypercube [0, 1|
Let fix the factual level set, Flyy, ,'). By the observational distribution constraint, we know that the interior occupies F. (1y) of the total volume of the unit hypercube [0, 1Jd, where F. (-) is the CDF of the observational distribution. Also, its absolute principal curvatures are bounded by K.
Step 3.
Then, let us assume there exists a counterfactual function, which obtains non- non-informative bounds of ECOU [ECOT]. For this, counterfactual level set for some y close to la has contain exactly (v) volume in the interior. At the same time, this level set has to contain as much of the surface volume of the factual level set, HF''(F(y'' d')), as possible in its interior. This is only possible by "bending" the counterfactual level set along one of the directions, so that the boundaries of the counterfactual level set lay at the boundaries of the unit hypercube, [0, 1jd. Due to the bound on the maximal absolute principal curvature, we have to be able to fully contain d-ball with radius 1 inside x
the interior of the counterfactual level set. This d-ball occupies the volume
n1/2
vol% (d-ball) = T(2 + 1) Add
(92)
where r'(-) is a Gamma function. Therefore, the volume of the interior of the counterfactual level set has to be at the same time arbitrarily close to zero, as Fa(y) + 0 as y + lap and at least of the volume of the d-ball with radius k. Thus, we arrived at the contradiction, which proves the theorem.
Corollary 4 (Monotonicity wrt. to K). By construction, it is guaranteed that, with decreasing K, the bounds for ECOU [ECOT] could only shrink, i. e., for Ki K2 : [(K1,)) d) S 1182, d) and u(Ki, d) 2 u(K2, d). This happens, as we decrease the class of functions in the constraints of the optimization problem from Definition2
Corollary 5 (Sufficiency of 2). In general, upper and lower bounds under assumed CSM (R) are different for different d2, d2 2 2, e. g., 1R, d2) & U(K, d2) and u(R, d1) 2 u(n, d2) for d1 > d2- This follows from the properties of the high high-dimensional spaces, e, g., we one can fit more d-balls of the fixed radius to the hypercube |0, 1]2 with the increase of d. Nevertheless, without the loss of generality and in the absence of the information on the dimensionality, we can set d = 2, as we still cover the entire identifiability spectrum by varying
31

---

Lemma 3 (BGMs-EQTDs identification gap of CSM(K = 0)), Let the assumptions of Theorem hold and let 0. Then the ignorance intervals for the partial identification of ECOU [ECOT] of
class B( d) are defined by min/max between BGMs bounds and expectations of quantile-trnncated distributions (EQTDs):
Qa' (1) (aa = min / max {BGM, (10)+((),.0 BGM EQTD, EQTD,
(v))
(7)
HGM+(y') FF. (v')) BGM-(y) (y-) F Farly)),
(8)
(2/) -E a, Y F. (1 |0.5 (1))i)
(9)
RQTDu(y') = E (Y a, > FT (2 F (v') - 0.5)))
(10)
where E(Y IY < ) B(Y Y .) are expectations of truncated distributions.
Proof. We provide an informal proof for d 2 based on geometric intuition. When K 0, all the level sets are straight lines (hyperplanes for d > 2). It possible to characterize all the possible relative arrangements the factual level set and the bundle of counterfactual level sets, and the corresponding values of ECOU [ECOT]. For a particular arrangement, displayed Fig. we see that the area of the interior of the factual level set is vol2 Fr((y)) At the same time, the factual level set crosses the counterfactual bundle cor- responding to the interval [4, F.- (2F 1FF11.FF.... (2)))r thus the counterfactual distri- bution of [un]treated is simply a 2F 2F (1/" (gfa((g)-quaniile-tttu >-quantile-trumcatee distribution, i. e., P(Y a, Y < F (2F (v/)))//) Therefore, the bounds of the ignorance interval of ECOU [ECOT] are defined by the min/max of both BGMs and expectations of quantile-
0.0
3P//7)
Figure 11: Factual
level set (orange), counterfactual level sets (blue).
quantile-truncated truncated distributions (EQTDs).
32

---

E Discussion of extensions and limitations
Sharp bounds. To obtain the sharp bounds under CSM (k), one has exactly solve the constrained variational problem task formulated in the Definition 2 in the space of the SCMs, which satisfy the Assumption This is very non-trivial task, the distributional constraint and the constraint of level sets of bounded curvature both include the partial differential equation with the Hausdorff integrals. For this task, in general, an explicit solution does not exist in a closed form (unlike, e. g., solution of the marginal sensitivity model (126)).
Extension of CSM to discrete treatments and continuous covariates. Our CSM (vs) naturally scales } K) and multivariate Markovian SCMs with additional covariates, X IRm which are all predecessors of Y. These extensions, nevertheless, bring additional challenges from estimating P(Y A, XX) from the observational (interventional) data. Hence, additional smoothness assumptions are required during modeling.
to bivariate SCMs with categorical treatments, A E {O....,
Extension of CSM to semi-Markovian SCMs. CSM (R) limited to the Markovian SCMs. For Markovan SCMs, e. g.. setting of the potential outcomes framework (TIT), additional assumptions are needed. Specifically, in semi-Markovian SCMs, the latent noise variables could be shared for X and Y, and this complicates counterfactual inference. Future work may consider incorporating other sensitivity models to our CSM (k), e. g., marginal sensitivity model, MSM (T) [26]. as described in Fig. [2
MSM (T)
x
X
U
X
A
Y
Uy
CSM (x)
Figure 12: Possible combination of our CSM (k) with the marginal sensitivity model, MSM (T)
126). for the potential outcome framework 111
33

---

F Details on architecture and inference of Augmented Pseudo-Invertible Decoder
Our APID provides an implementation of CSM (k) for SCMs of class B(C², 2). In the following, we list several important requirements, that the underlying probabilistic model has to satisfy. Then, we explain how deep normalizing flows 107 T25 meet these requirements.
1. A probabilistic model has to explicitly model an arbitrary function (a uy): by E [0. 1J2 of class C2_ Normalizing flows with free-form Jacobians can implement arbitrary invertible Ca transformations from Pa : IR² + R2a Then, by omitting one of the outputs, we can model functions (a, [0, 1]a + Vo C R. We discuss normalizing flows with free-form Jacobians Sec. F.T
2. A model has to fit the observational (interventional) distribution, given a sample from it, D = {A,, Y. Y)) This is possible for the normalizing flows, they are maximizing the log-likelihood of the data, IP(Y Y; | A = A,). directly with the gradient- based methods. Importantly, we also employ variational augmentations [22) to evaluate the log-likelihood We discuss this in detail in Sec. F2
3. A probabilistic model should be able perform the estimation of ECOU [ECOT], through abduction abduction-action pereiinttreitiiinn steps. In normalizing flows, this can be achieved with the help of the variational augmentations (22). Specifically. for evidence, y' and a', our proposed APID can infer arbitrarily many points from the estimated factual level set Ely a'), which follow the estimated posterior distribution of the latent noise, i. P(Uy 3/, a'). We discuss this in detail in Sec. F2
4. All the level sets of the modeled function need to have bounded curvature. This is possible for normalizing flows as a deep learning model. Namely, the curvature H2 (uy) can be evaluated via automatic differentiation tools and added the loss of the model. We discuss this in detail in Sec.
F1 Choice of a normalizing flow with free-form Jacobians
Normalizing flows (NFs) [107 125] differ in how flexible the modeled transformations in high dimen- sions are. Some models, like planar NF [107] or Sylvester NF [128), only allow for transformations with low-rank Jacobians. Other models employ masked auto-regressive networks (72] or coupling to construct transformations with lower-triangular (structured) Jacobians.
blocks 33 34
Recently, several models were proposed for modeling free-form Jacobian transformations, g.. (i) continuous NFs [23] (55 or (ii) residual NFs (9] 24J. However, this flexibility comes a computational cost. For (i) continuous NFs, we have solve system of ordinary differential equations (ODEs) for every forward and reverse transformation. As noted by [45), numerical ODE solvers only work well for the non-stiff differential equations (=for non-steep transformattons). For (ii) residual NFs, the computational complexity stems from the evaluation of the determinant of the Jacobian, which is required for the log- likelihood, and, from the reverse transformation, where have to employ fixed point iterations.
We experimented with both continuous NFs and residual NFs but we found the latter to be more stable. The drawbacks of residual NFs can be partially fixed under our CSM (x). First, the determinant of the Jacobian can be evaluated exactly, as we set d 2. Second, the numerical complexity of the reverse transformatior can lowered by adding more residual layers, so that each layer models less steep transformation and the total Lipschitz constant is larger for the whole transformation. Hence, in our work, we resorted to residual normalizing flows 24
F.2 Variational augmentations and pseudo-invertibility
Variational augmentations were proposed for increasing the expressiveness of normalizing flows 22]. They augment the input to higher dimension and then employ the invertible transformation of
Note that, in APID, forward transformation corresponds to the inverse function, i.e., fy (a.
34

---

the flow. Hence, normalizing flows with variational augmentations can be seen as pseudo-invertible probabilistic models [TO 54]
We use variational augmentations to model the inverse function, fy (a. ), i.e., sample points from the level sets, For this, we augment the (estimated) outcome Y E 3r with Youg NYg"(Y), E R. where (-) is fully-connected neural network with one hidden layer and parameters 0,, and e² is a hyperparameter. As such, our proposed APID models (a through a two-dimensional transformation Fa = (fx.., (a. -), fy (a,-)) [0. 1]2 + R x Va. Variational augmentations facilitate our task in two ways.
(i) They allow evaluating the log-likelihood of the data via
log P, (Y = a) Ey,,, [log P.. (Yang = Yu =Y),) - log NY 9 (Vi), e2)
(93)
where 3a are the parameters of the residual normalizing flow, and the normal distribution. Here, we see that, by increasing the =², the variance of the sample estimate of the log-likelihood of the data increases. On the other hand, for =² - O, the transformations of the residual flow become steeper, we have transport the point mass the unit square, [0, 1]2.
is the density of
(ii) Variational augmentations enable the abduction- ceaeeiioonaation--o cittiionpreiitioo to estimate ECOU [ECOT] in a differential fashion. At the abduction step, infer the sample from the posterior distribution of the latent noise, defined the estimated factual level set F(y' a'). For that, variationally augment the evidence y' with samples from (y', Tm... V2 ((gg (v)),,?)), and, then, transform them to the latent noise space with the factual flow
{ uy² - (161 1
(94)
Then, the action step selects the counterfactual flow, Fa(t), and the prediction step transform the abducted latent noise with it:
(Yamg.j 8,)};=1= Fa (((iy) uyz (y (y)
(95)
In the end, ECOU [ECOT] is estimated by averaging Yy:
b Qu'- M Y
(96)
In our APID, the parameters of the residual normalizing flow, Pa. and the parameters of the variational augmentations, 9a2 are always optimized jointly. We use the reparametrization trick, to propapre
through sampling of the augmentations.
F.3 Penalizing curvatures of the level sets
Although there exist deep learning models which explicitly bound the curvature of the modeled function (e. g., [24)). none of the works (to the best of our knowledge) did enforce the curvature of the level sets of the modeled function.
In our APID, the curvature Ni (uy) ) is evaluated using automatic differentiation exactly via Eq. [I] and then incorporated into the loss. As the calculation of the second derivatives is a costly operation, we heuristically evaluate the curvature only at the points of the counterfactual level set, which corresponds to the estimated ECOU (ECOT), namely (uy1, , uy=) E E(Qur a). For this, we again use the variationally augmented sample of size b for 0 = +a (U, (Yag.s
N (I) e)). Then, we use the counterfactual flow (such as in Eq. 94
{ uy -
= = (1(6. Ymm
(97)
while the curvature has to be only evaluated at points given by
( uyz 3/
(98)
35

---

G Details on training of Augmented Pseudo-Invertible Decoder
G.1 Training objective
Our APID satisfies all the requirements set by CSM (k), by combining several losses with different coefficients (see the overview Fig. 13). Given observational data {A,, Yi)== drawn i.i.d. and counterfactual query, Qw' (y (3)), for the partial identification, APID minimizes the following losses:
(1) Negative log-likelihood loss aims to fit the observational data distribution by minimizing the negative log- lelliielihood of the data, D {A, a, Yi), modeled by APID. To prevent the overfitting, we use noise regularization [110), which adds normally distributed noise: Y, Y; E E
N (0, )²), where o² > 0 is a hyperparameter. Then, the negative log-likelihood loss for a E {0,1] is
(Na =- 72 Σ log P 9. (Y Y; a)
(99)
where the log-lik logeliiel is evaluated according to the Eq. 93
(2) Wasserstein loss prevents the posterior collapse 226 132 of the APID, as the sample (Y:) not guaranteed to cover the full latent noise space when mapped with the estimated inverse function. We thus sample points from the latent noise space, {(Uy Uys ~ Unif(O, 1)2 and map with the forward transformation Fa for a E {0,1} via
(Yangg Ya. )a = Fa (((UV) Uv (y2
(100)
Then, we evaluate empirical Wasserstein distance
Cw = 1. - (a)l dg.
(101)
where F72( and (-) are empirical quantile functions, based on samples {A, a, Yi} and (Ya, respectively.
(3) Counterfactual query loss aims maximize/minimizzz ECOU [ECOT]:
Ca(Bar, 02 Ba) = Softplus (Ftp:
(102)
where changes for maximizattio//miiiiizatooo correspondingly, (a') is evaluated de- scribed in the Eq. (96). and Softplus(x) = log(1 + exp(x)). We use the softplus transforma- that it matches the scale of the negative
tion to scale the counterfactual query logarithnically log-likelihool lielliikeiood. We also block the gradients and fit only the counterfactual flow, when maximiz- ing/minimizing ECOU [ECOT], to speed up the training.
(4) Curvature loss penalizes the curvature of level sets of the modeled counterfactual function (a.
Cn, 0m) =
Σ Ki (fyy),0 j=1
(103)
where (Ki(ūy1,) 1/=1 are defined in Sec. F33
All the losses are summed up to a single training objective
L(B Osr, 0a)
Σ
(An, 0n) + (w(( bo Lo(Ba,, Ou. Ba) + Are La 0n)-
aE (E(0..)
(104)
G.2 Training algorithm and hyperparameters
Training algorithm. The training of APID proceeds in three stages; see the pseudocode in Algo- rithm T At a burn-in stage (n 500 training iterations), we only two residual normalizing flows (one for each treatment) with LNLL and Lw. Then, copy the counterfactual flow and variational
36

---

(2) Wasserstein (1) Negative log loss
likellhood loss
Youg (M,
Unid(o,
(4) Curvature loss (3) Counterfactturs query loss
Ving Nic (Kogl///1 (/),
Residual NF
Residual NF
Unid(o, 11
P(V14)
4 ENLL.
Residual NF 8a
Undf(O, 1)
Residual NF
Cw
A
Legend
Fan (1)"
Residual NF
Normiizing parametrized by
Noise aagiriiitioonn
Da
Fully sonnected network
imputs outputs with cornsponndin distributions
Gradiant blockling
Figure 13: Overview of the training and inference our Augmented
Decoder.
augmentation parameters twice, for the task of maximization/niiiiiizzoonn respectively. Also, freeze the factual flow parameters. At a query stage (nQ = 100 training iterations), we, additionally to previous losses, enable the counterfactual query loss, Xo Lq- During this stage, two counterfactual flows are able to realign and start "bending" their level sets around the frozen factual level set. Ultimately, the last curvature-query stage (neQ 500 training iterations), all the parts of the loss in Eq. (104) are enabled. After training is over, we report the upper and lower bounds for the counterfactual query an output of the APID with exponential moving average (EMA) of the model parameters [105]. EMA of the model parameters allows us to reduce the variance during training and controlled via the smoothness hyperparameter ? 0.99.
Hyperparameters. We use the Adam optimizer [1 with a learning rate 77 = 0.01 and a minibatch size of b 32 to fit our APID. This 6 32 is also used for all the sampling routines at other losses. For the residual normalizing flows, we use t = residual transformations, each with h, units of the hidden layers. We set the relative and absolute tolerance of the fixed point iterations to 0.0001 and a maximal number of iterations to 200. For variational augmentations, we set the number of units in the hidden layer hy 5, and the variance of the augmentation to e² 0.5². For the noise regularization, we chose o² 0.0012. This configuration resulted in a good fit (with respect to the test log- ligeliioohhod) for all the experiments, and, thus, we did not tune hyperparameters. We varied the coefficients for the losses, Xo and X and report the results in Sec. and Appendix
37

---

Algorithm Training algorithm of APID
1: Input. Counterfactual query Qe (y') training dataset D = Do U D1; number of training iterations RB; ng, ncq: minibatch size learning rates 7,; intensity of the noise regularization
o²; the variance of the augmentation =²; EMA smoothing 7; loss coefficients Xo and AR.
2:
3: Init. Flows parameters: residual NFs (Su. 31) and variational augmentations (fg. 01). > Burn-in stage
4: for k == to ng do
5:
for E {0, 1] do
6:
(A; a, Yi) minibatch of size from Dr
7:
~ N(0,00 :):0,-))YY+-0 +
Noise regularization
8:
Yuuga N(gg(Y)))
D. Variational augmentations
9:
(9,, 0,) t - a. (Y = =Yi b (1) Negative log- oog-likelhhod loss
10:
{[Uy1,j. () 1n)
11:
((Sagga.j) [ag));=1 + (((Ur)) Uy 5
12:
Cw(3,) + So 1 (q) (f)
P (2) Wasserstein loss
13:
L+ LNII (3,, 0a) + Lw,,)
14:
optimization step for Ba, Og wrt. L with learning rate
15:
end for
16: end for
Output. Pretrained flows parameters: residual NFs (Bo, B1) and variational augmentations
(0o.01
18:
19: B B t Pa; Ba.l. + ou;
> Copying counterfactual flow parameters
20: [1a ual + [min; (Y;); max; (Y;)]
21: for k to no ncQ do
D Query & curvature-query stages
for (a) E [B. B.). (B) 0.)) do
23:
{A: = a, Y. + minibatch of size from Da same procedure as in lines 7-12 to evaluate CNLL (3a: 0g) and (w(Ba) L+ (N,L 0.) + (Bx)
25:
26:
(W.YYuug 16 Ngg (2/),=²))
P Variational augmentations
27:
[(Ay- uyz,)) ( + Fo (((1) Y Ymg.)) ))==))
((Yagg.) 8,)1 t- Pa (((ūy a
29:
Qu (30)aa(1)))) ク Y,
Estimated ECOU (ECOT]
30:
if Q a' (2/) [la, ūn) and is < na then
31:
continue
> Non-informative bounds
32:
else
33:
LolBa, 3a) t Softplus(7Qo L+ L + da (2 ,a) 3a)
(3) Counterfactual query loss
34:
35:
end
36:
if k 2 no then
37:
ÿ + Qa
(9, (Vmpy)) ((g)()))
D Variational augmentations
38:
((yy ")} - F-1 (10), Pmngs))
39:
(3. 0a) 1/j Ε (uyz uy2,z)
(4) Curvature loss
L L >, LB,
41:
end if
42:
optimization step for Bay Og wrt. C with learning rate EMA update of the flow parameters (Ba, 0o)
43:
44:
end for
45:
end for
46: Output. Qx/)a((/) evaluated with smoothed with EMA (B,, Ba) and (g,
38

---

H
Synthetic experiments
H.1
Datasets
We conduct the experiments with datasets, drawn from two synthetic datasets. In the first dataset, we use
Y 1 P(Y 0) N(0, 1), 1 1) (N,
(105)
and, in the second,
YY P(Y 10) Mixture (0.7 (- 1.52) 0.3 0.3.5.., 111~ 1) (0
(... 0.3.
(106)
Importantly, the ground truth SCMs (and their x) for both datasets remain unknown, and we only have access to the observational distributions. This is consistent with the task of partial counterfactual identification.
H.2 Additional results
In Fig. we provide additional results for the synthetic experiments. For our APID, we set λo = 1.0 and vary AN {0.0, 0.5, 1.0, 5.0). Here, that estimated by APID bounds are located closer to (i) the theoretic upper and lower bounds of BGMs and (ii) closer to each other.
FLY (1)
Misturn
P( Misture (Y)
APIDIA,
APIDIA, 1.0)
APHILA,
BGM
EQJES
-155
3.0
Figure 14: Additional results for partial counterfactual identification of the ECOU across two datasets. Reported: theoretical bounds of BGMs and mean bounds of APID over five runs. Here, Xq 1.0.
H.3 Runtime
Table providee the duration of one training iteration of our APID for different training stages. Namely, we report the mean and std. across all the experiments (total of 720 runs) for the burn-in, the query, and the curvature-quer, stages. In our experiments, the long training times are attributed to the reverse transformations the residual normalizing flows they require fixed point iterations) and to the computation of the Hessian to evaluate the curvatures.
Training stage
Number of training iterations
Evaluated losses
Duration per training iteration (in s)
Burn-in stage
500
EXLL Loy ENLL, Lw,, L.Q. CNTA Cxp, Co., Cs.
2.60 + 1.72
Query stage
n2
11.04 10.65
Curvature-query stage
#ii 500
39.27 27.26
Table 2: Total runtime (in seconds) for different stages of training for APID. Reported: mean +
standard deviation across all the experiments, which amount to a total of 720 runs (lower is better). Experiments carried out on 2 GPUs (NVIDIA A100-PCIE-40GB) with IntelXeon Silver 4316 CPUs 2.30GHz.
39

---

I Case study with real-world data
In our case study, we want analyze retrospective "what if" questions from the COVID-19 pandemic. Specifically, we ask what the impact on COVID-19 cases in Sweden would have been, had Sweden implemented stay-home order (which Sweden did not do but which under scrutiny due post-hoc legal controversies)
L.1 Dataset
We use multi-country data from 7 It contains weekly-averaged number recorded COVID-19
for 20 Western countries. The outcome, Y. is the (relative) case growth per week (in log), namely, the ratio between new cases and cumulative cases, The treatment, A E {0, 1), a binary variable referring to either no lockdown or enforcement of stringent lockdown, respectively. We assume the data i.i.d. and comes from a randomized control trial, i. e., there no confounding between Y and A. We filtered out the weeks where the cumulative number of cases is smaller than 50, which resulted n no + 72 136 + 124 observations in total, for both treatments, respectively. We plot a histogram of case growth per week depending on the treatment in Fig. [5] i.e., empirical densities of distributions IP(Y A 0) and IP A
0.40
(to lackdown)
A (dttingrat lociddow))
0.35
0.30
0.25
0.20
0.15
0.05
0.00
Obarrvd caso growth y
Figure 15:
Empirical distributions of case growth per week (in log) for both treatments. Larger values mean a larger incidence.
We assume that the observational distributions P(Y a) are induced by some (unknown) SCM of class B(C², 2). Then, we fit our APID the same way in synthetic experiments (see Appendix G). We additionally report the BGMs-EQTDs identification gap based on empirical CDF of the observational distributions.
1.2 Results
Fig. 16 shows the results for (upper) partial counterfactual identification of the ECOU for our method. Hence, the ECOU reports for the expected counterfactual case growth for countries without stay-home orders, i. e., the weekly case growth (in log) if stay-home order would have actually been enforced. We also highlight the upper bounds the ECOU for Sweden during weeks 10-13 of 2020, where no lockdown was implemented. Evidently, the ECOU would lie below the decision boundary for BGMs-EQTDs and for certain levels of Ay. This implies that Sweden could have had significantly lower case growth had it implemented a stay-home order, even when allowing for some level of non-linearity between outcome and independent latent noises that interact with the treatment, e.g.. cultural distinctions, level of trust in government, etc. The results of the COVID-19 case study demonstrate that our CSM and instantiation with APID could be applied for decision-makee with real-world data.
'The data is available at http:////gtthtccooooooooooooooor haster//ataa/ara_peeroooosss
40

---

-05
-15
Counectacual case goont, 26-165
APIDOA,
20
APIDIA 1.00
APIDIA 10.0)
-25
EQTTS
Obeerved gootth for Sweden
Deiston bomdary (O..155) v
-35
-15
330
255
mede 12
wedt 13
2.00
-55
Wode 11
worl 10
-.00
-0.3
Observed cree gooth,
Figure 16: Case study where we analyze retrospective" if" questions from the COVID-19 pandemic. Reported: BGMs-EQTDs identification gap and ECOU upper bounds of APID over five runs (in log). The gray-shaded area above denotes the decision boundary with the negative counterfactual effect, namely where stay-home order would have led an increase case growth, respectively. The red lines show the observed case growth for Sweden during weeks 10-13 of 2020, where no stay-home order was implemented.
41


Reviews:
[{"review_id": "fngLQptcVY", "reviewer": "Reviewer_8Zs8", "summary": "This paper presents a method for obtaining bounds on continuous counterfactual outcomes.  In general, level-3 counterfactual outcomes (what would have happened under action $a'$, given knowledge of what did happen under action $a$) are not identifiable from either interventional or observational data.  Prior work makes structural assumptions on the underlying structural causal model (e.g., monotonicity with respect to a single unobserved noise variable) to obtain identification, while this work instead makes an assumption on the principal curvature of level sets with respect to two unobserved noise variables.  A computational approach is provided to obtain bounds based on this principal curvature assumption, and is demonstrated on two simple synthetic cases.\n", "questions": "1. How would you go about helping domain experts choose an appropriate value of $\\kappa$ or $\\lambda_{\\kappa}$ in practice?\n2. Is there any guarantee that the bounds derived from this approach will be less informative than the theoretical bounds?  Or, should we generally expect the bounds to be more informative, since we will inevitably fail to find the SCMs that yield the true maximum / minimum for a given $\\kappa$?\n3. Related to the previous question, is my understanding of \"cover the entire identifiability spectrum\" correct in Corollary 4?  If so, it may be worth revising the theoretical statement to make this clear.  If my understanding is correct, then what is going on in Figure 13, i.e., why does setting $\\lambda_{\\kappa} = 0$ still yield informative bounds?\n", "limitations": "In the main paper, I would have liked to see more discussion of the limitations outlined in the \"weaknesses\" section above.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "In terms of originality and clarity, this paper presents a novel approach to obtaining bounds on continuous counterfactual outcomes, and the contribution is clearly placed in the context of related work.  Having seen a few papers on the general topic of partial identification, I found Section A (Extended Related Work) in the Appendix to be a well-written, well-researched, and generally helpful overview of how the current work differs from prior approaches, primarily in the problem considered.\n\nMoreover, in terms of clarity and quality, I found Example 1 (Figure 3) to be a very helpful example of counterfactual non-identifiability, and a good explanation of the intuition for how \"bending\" the counterfactual level sets around the factual level set leads to a different distribution of counterfactual outcomes.\n", "weaknesses": "## Summary \nI see two major weaknesses in this paper, and look forward to discussion with the authors during the response period if I've misunderstood anything below.  Both weaknesses relate to the significance of the approach.\n\n1. How should domain experts choose the parameter $\\kappa$?  Choosing the right bound on the principal curvature of level sets doesn't seem like something a domain expert would plausibly be able to do.\n2. Even if we knew the right $\\kappa$, can we guarantee that the computational approach will give bounds that are at least conservative (e.g., outer bounds on the true bounds implied by $\\kappa$) if not tight?\n\nOverall, my current score incorporates my view that these two weaknesses are genuinely challenging problems, so I am somewhat sympathetic to the paper in its current state, but I would at least like to see more discussion of these challenges in the main paper.\n\nI'll discuss these weaknesses in more detail below.  I am basing my score primarily on these points, though I also had an important clarifying question regarding the ability of the approach to \"cover the entire identifiability spectrum\". \n\n## Details of major concerns \n\n(1) **Domain Knowledge for $\\kappa$?** One of the appealing properties of the given approach is that it can interpolate between the extremes of identifiability (though see my clarifying question (3) below).  However, there are many trivial ways to interpolate between e.g., point-identification and the non-informative bounds (e.g., by simply choosing a point between them). The value of sensitivity analysis approaches (in my view) lies in the ability to translate domain knowledge into informative bounds. \n\nIn other areas of sensitivity analysis (for e.g., interventional queries in the presence of unobserved confounding) that rely on domain knowledge to derive bounds, there is an emphasis on creating additional tools to help end-users calibrate their choices of these hyperparameters by e.g., benchmarking (see Figure 2 of Cinelli and Hazlett 2019, full citation below).  Informally, this type of benchmarking allows one to say \"an unobserved confounder would have to be 3x as strong as the variable 'age' in order to change our conclusions\", a statement that is a bit easier for domain experts to engage with versus reasoning about the value of an abstract hyperparameter.\n\nI'm aware that this example deals with a different type of causal query (i.e., counterfactuals vs interventional queries), and that the lack of identifiability in the current paper is fundamentally different from unobserved confounding - my point is that for any partial identification approach that derives informative bounds via \"domain knowledge\", like the current approach, a bit more effort is required to make it plausible for domain experts to convert their domain knowledge into the required (abstract) hyperparameters, like bounding the principal curvature of counterfactual level sets.\n\nI would be interested in any comments from the authors on this point.\n\nCitation: Carlos Cinelli , Chad Hazlett, Making Sense of Sensitivity: Extending Omitted Variable Bias, Journal of the Royal Statistical Society Series B: Statistical Methodology, Volume 82, Issue 1, February 2020, Pages 39–67, https://doi.org/10.1111/rssb.12348\n\n(2) **Corresponding between computational bounds and theoretical bounds**:  Supposing we know the right value of $\\kappa$, Definition 2 defines an upper and lower bound on the quantity of interest (e.g., the ECOU), with the additional constraint (e.g., in Theorem 2) that all SCMs satisfy Assumption $\\kappa$ and are in the class $\\mathcal{B}(C^2, d)$.  For the practical algorithm, we use $d = 2$.\n\nIn an ideal world, the computational method (i.e., the Augmented Pseudo-Invertible Decoder in Section 5) would take a given value of $\\kappa$ and return the bounds as defined in Definition 2.  However, somewhat understandably, returning the exact bounds is not necessarily feasible. \n\nInstead, we are left with heuristics - we optimize an objective over a certain set of SCMs where one of the four terms relates to maximizing / minimizing the counterfactual outcome, and is controlled by the hyperparameter $\\lambda_Q$.  It was not clear to me how this term trades off with the other terms, e.g., the extent to which we can view the optimization problem as choosing a fixed constraint on $\\kappa$ and then maximizing / minimizing the counterfactual subject to this constraint among all observationally-equivalent models.\n\n**The use of practical heuristics would be less concerning if this approach resulted in less informative bounds than those in Definition 2, but as I understand it, it may result in *more* informative bounds.**\n\nTo clarify, this point is briefly discussed in Appendix E (discussion of extensions and limitations) under \"Tight bounds\". However, my concern is not that the bounds are insufficiently tight (e.g., too wide and could be narrowed), but that they might be *too narrow* (e.g., the theoretical bounds are actually wider than the bounds produced by the algorithm).\n\nI believe there is some evidence to suggest that these heuristics change the interpretation of the results.  For instance, I would expect that as $\\lambda_{\\kappa}$ changes, the bound should monotonically increase/decrease. However, looking closely at Figure 7 suggests that in practice there is some crossover (see e.g., $y' = -0.5$, where the lower bound for $\\lambda_{\\kappa} = 1$ goes above the line for $\\lambda_{\\kappa} = 10$).  I believe my observation below relating to Figure 13 in the appendix (where $\\lambda_{\\kappa} = 0$ still gives informative bounds) is also related to this point.\n\nOverall, it would be good to see a comparison, perhaps for a very simple example, between the ground-truth bounds in Definition 2 and the bounds returned by the approach.\n\n(3) **Clarification regarding non-informative bounds with $\\kappa=\\infty$**: It is stated in Corollary 4 that by setting $d = 2$ and varying $\\kappa$, we can \"cover the entire identifiability spectrum\".  I took this statement to mean that if e.g., there is no restriction on $\\kappa$, then the resulting bounds should be non-informative (i.e., equal to $[l_a, u_a]$), and that for $\\kappa = 0$, all values of $d$ yield the same point-identified solution.  I'm deriving my understanding primarily from the explanation given in the introduction (see lines 72-73).  Is my understanding correct?\n\nIf my understanding is correct, then what is going on in Figure 13 in the appendix?  It appears that setting $\\lambda_{\\kappa} = 0$ (corresponding to no constraints on $\\kappa$) does not result in non-informative bounds.  Is this due to the (perhaps necessary) mismatch between the computational approach and perfectly solving the optimization in Definition 2?\n\n## Minor Points \n\nAs a minor point, it may be worth clarifying where we are interested in taking a maximum/minimum (e.g., over observationally-equivalent SCMs) and where we are interested in taking an average (e.g., over the counterfactual posterior for a single SCM).  E.g., when I first read the procedure for abduction on lines 341-344, I was concerned that the approach draws samples from the pre-image, as opposed to looking for $U_1, U_2$ which yield the maximum/minimum counterfactual outcomes, before I realized that this approach makes sense for a fixed SCM to compute a counterfactual expectation like ECOU / ECOT.\n\nI also found Figure 6 difficult to follow, especially the bi-directional arrows for \"connections\".\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are very grateful for such a rigorous, helpful, and in-depth review. We also appreciate that the reviewer found our paper clearly written, our method novel, and our contribution well-placed in the related work. Here is our answer to the questions:\n - (1) We stress that counterfactual inference is a non-trivial problem in statistics and probability. Even in human cognition, counterfactuals are much more complicated than causal statements [1]. To the best of our knowledge, our paper is the first to address partial counterfactual inference for continuous outcomes. For this, we provided a measure-theoretic formulation and formulated the results of non-informative bounds for smooth functions. Our CSM is one viable solution (out of many) to build a sensitivity model. We agree that κ in our CSM may be perceived as an abstract sensitivity parameter. Yet, it can be interpreted as the *level of non-linearity between the outcome and its latent noises that interact with treatment*. To illustrate this, let us consider two following cases:\n   - (i) When the treatment does not interact with any of the latent noises, then we can assume w.l.o.g. a BGM. There is no loss of generality, as all the other SCMs with d > 1 are equivalent to this BGM (see Ex. 6).\n   - (ii) When the treatment interacts with some latent noise variables we cannot assume a BGM and have to use our CSM. In this case, d corresponds to the number of latent noise variables which interact with the treatment. Hence, κ bounds the level of non-linearity between the outcome and the noise variable. More formally, κ and principal curvatures can be seen as the largest coefficient of the second-order term in the Taylor expansion of the level set (see Eq. 15 in App. B). This interpretation of the κ goes along with human intuition (see [1]): when we try to imagine counterfactual outcomes, we tend to “traverse” all the possible scenarios which could lead to a certain value. If we allow for highly non-linear scenarios, we also allow for more extreme counterfactuals, e.g., interactions between treatment and rare genetic conditions.\n\n   **Action**: We will add the interpretation of κ and the intuition behind its choice to our revised paper. To strengthen the practical value of our CSM, we provided a new case study (see **PDF**). Therein, we adopt our CSM to answer “what if” questions related to whether the lockdown was effective during the COVID-19 pandemic.\n \n- (2) We agree that, for a given value of κ, it is computationally infeasible to derive ground-truth (GT) bounds, even when the GT observation density is known. This is different from, e.g., layer 2 models like MSM, where the GT solution could be expressed in terms of conditional quantiles. The GT bounds in Def. 2 require solving a constrained optimization task including partial derivatives. This is intractable, even for simple distributions as normal.   \n\n  Thus, we resort to using tractable alternatives and, hence, design our APID. The approach of turning constraints into parts of the loss is a common practice in deep learning for partial identification [2, 3]. In our case, this involves either adding the curvature loss controlled by λ_κ and the query loss controlled by λ_Q. The exact relationship between κ and corresponding λ_κ is unknown, but they are inversely connected. In general, this relationship is usually known only for very simple models, like ridge or lasso regressions. Regarding the choice of λ_Q, we provided additional experiments in our original supplements in App. H.2. Therein, the bounds are simply moved towards/away from BGMs-EQTDs bounds, but their relative location almost does not change. We acknowledge that our APID is a proof of concept rather than a fully-fledged method. Hence, for the multi-modal distribution in Fig. 7, APID sometimes yielded inaccurate bounds (e.g., too tight), mainly due to the computational instability (i.e., the gradual “bending” of the level sets is unstable during training, and some runs omit “bending” at all as it requires passing through the high loss value region).\n\n  **Action:** We will add this discussion to our Limitations section.\n\n- (3) Thank you for this correction. Indeed, by setting κ= 0 in our CSM, we do not achieve point identification. Generally, it is not clear how to generalize the point identification of BGMs, e.g., there is no natural notion of monotonicity for functions from R^d to R.\n\n  We summarized this result as a part of our *newly added theoretic result*, which we presented in Lemma 3 of the **PDF**. Therein, we introduce a so-called BGMs-EQTDs identification gap. This gap describes the closest we can get to the point identification by setting κ=0. Although our sensitivity model does not achieve full point identification for κ= 0, our CSM can still be useful for decision-making, as we show in our newly added case study (see **PDF**).\n\n  Regarding the computational experiment (Fig. 13):  when setting λ_κ=0 (same as κ= ∞), some runs omitted “bending” as that comes at the cost of the loss increase.\n\n  **Action:** We will add the new theoretical result from Lemma 3 (BGMs-EQTDs identification gap) to our revised paper.\n\nMinor Points\n* We will clarify throughout our paper that we work with ECOU, which are *expectations* of the counterfactual outcomes. Then, min/max are performed on top of the expectations.\n* Regarding Fig. 6, we have added an expanded version in the supplements of our original paper (see App. G, Fig. 12) with unidirectional edges, which explains the training and inference of APID.\n\nReferences:\n- [1] Celar, Lenart et al. \"How people reason with counterfactual and causal explanations for AI decisions in familiar and unfamiliar domains.\" Memory & Cognition (2023): 1-16.\n- [2] Kevin Xia et al. “Neural causal models for counterfactual identification and estimation”. In: ICLR. 2023.\n- [3] Kevin Xia et al. “The causal-neural connection: expressiveness, learnability, and inference”. In: Advances in NeurIPS. 2021"}, {"review_id": "ZX7LhGVqzk", "reviewer": "Reviewer_JMz8", "summary": "This paper studies the partial identifiability of counterfactual queries and proposes an interesting proof of general non-identifiability resulting in non-informative bounds. The authors build upon the intuition used for this proof to propose a new class of structural causal model using principal curvatures. Under this assumption, the authors can achieve informative partial counterfactual identification. The authors then propose an architecture that leverages this assumption.", "questions": "Could the author motivate the CSM assumption from a more practical perspective ? \n\nExtending the experiments section would help strenghten the paper.", "limitations": "Limitations and assumptions are clearly stated.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "This paper proposes a systematic and rigorous study of the partial counterfactual estimation problem. This is exemplified by the breadth and depth of literature search and by the organization of the related works. While general non-identifiabilty of counterfactuals is known, theorem 1 shows that constraints on the smoothness of repsonse function does not improve the identifiability, which I think is novel. \n\nThe proposed curvature sensitivity model, motivated by Theorem 1, is shown to benefit from the identifiability properties which are theoretically demonstrated. This causal model structure seems to be fairly general and appears to generalize concepts such as non-intersection of response surfaces (which is pointed at in the experiments). \n\nThe authors further show that the theoretical properties of the CSM can be leveraged empirically be proposing a new architecture.", "weaknesses": "While there is significant effort in bringing intuition regarding how to construct counterfactuals (bending of the counterfactual level sets), I believe more intuition could be given regarding the general applicability of the CSM in practice. It is unclear when it seems reasonable to assume this model would be valid, and hence the partial counterfactual estimation correct. I appreciate that this is a general problem in counterfactual estimation but more insight into the applicability would be welcome.\n\nWhile there is a strong theoretical work behind this paper, it is more frugal in terms of experiments. I would encourage the authors to provide more evaluations of their approach, with different types of repsonse surfaces and noise levels. The design of more experiments could potentially lead to more intuition in the assumptions of the method. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the positive evaluation of our work. We are happy that you consider our paper a novel, systematic, and rigorous study. We would like to discuss the mentioned weaknesses and questions in the following.\n\n- We agree that the $\\kappa$ in our CSM may be perceived — in parts — as an abstract sensitivity parameter. Nevertheless, the sensitivity parameter $\\kappa$ lends to a natural interpretation. It can be interpreted as the *level of non-linearity between the outcome and its latent noises that interact with treatment*. To illustrate this, let us consider two following scenarios: \n   - (i) When the treatment does not interact with any of the latent noise variables, then we can assume w.l.o.g. a BGM (thus: deterministic counterfactual outcome). There is no loss of generality, as all the other SCMs with $d > 1$ are equivalent to this BGM (i.e., their level set bundles coincide for both factual and counterfactual transformation). \n   - (ii) When the treatment interacts with some latent noise variables, then the counterfactual outcome becomes random, and we cannot assume a BGM but we have to use our CSM. In this case, $d$ corresponds to the number of latent noise variables which interact with the treatment. Hence, $\\kappa$ bounds the level of non-linearity between the outcome and the noise variables. More formally, $\\kappa$ and principal curvatures can be seen as the largest coefficient of the second-order term in the Taylor expansion of the level set (see Eq. 15 in the Appendix). This interpretation of the $\\kappa$ goes along with human intuition (see [1]): when we try to imagine counterfactual outcomes, we tend to “traverse” all the possible scenarios which could lead to a certain value. If we allow for highly non-linear scenarios, which interact with treatment, we also allow for more extreme counterfactuals, e.g., interactions between treatment and rare genetic conditions. \n\n  Importantly, in order to verify the particular value of $\\kappa$, we would have to measure all the latent noises which interact with treatment. Verifying counterfactual inference is a very complex task, and can be achieved by, e.g., combining several datasets (known as causal marginal problem [2], yet which has been done only for discrete variables and not for continuous variables as in our work). Still, we are confident that our paper provides a valuable theoretical foundation for future research and can help practical applications (see next bullet item).\n\n  **Action**: We will add the interpretation of $\\kappa$ and its intuition to our revised paper. \n\n- We experimented with different coefficients for the $\\lambda_\\kappa$ (which correspond to different bounds on the curvature of level sets). Our CSM considers *all the possible response surfaces* with a certain max curvature of the level sets, even when the dimensionality of the latent noise is set to $d=2$ (see Corollary 4 in Appendix D).  \n\n  To address the issue of applicability and to strengthen the intuition behind our CSM, we performed an additional experiment with a real-world case study (see **PDF**). Therein, we adopt our CSM to answer “what if” questions related to whether the lockdown was effective during the COVID-19 pandemic. Specifically, we ask what the impact on COVID-19 cases in Sweden would have been, had Sweden implemented a stay-home order (which Sweden did not do but which is under scrutiny due to post-hoc legal controversies). Our results imply that Sweden could have had significantly lower cases had it implemented a stay-home order, even when allowing for some level of non-linearity between outcome and independent latent noise variables that interact with the treatment, e.g., cultural differences, level of trust in government, etc.\n\n  **Action**: We will add our case study with real-world data to our revised paper.\n\nReferences: \n- [1] Celar, Lenart, and Ruth MJ Byrne. \"How people reason with counterfactual and causal explanations for Artificial Intelligence decisions in familiar and unfamiliar domains.\" Memory & Cognition (2023): 1-16.\n- [2] Luigi Gresele et al. “Causal inference through the structural causal marginal problem”. In: International Conference on Machine Learning. 2022.\n\n\n"}, {"review_id": "nYrHr5GewF", "reviewer": "Reviewer_53D4", "summary": "The authors study the problem of counterfactual identification of continuous outcome variable with a specific causal graph. They show that the expected counterfactual outcome of (un)treated has non-informative bound when the function class is arbitrary smooth. Then, they introduce a new model called CSM that give informative bounds. Finally, they propose a deep generative based method for partial counterfactual inference in CSM.", "questions": "See above", "limitations": " The authors addressed the limitations.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "They study an important problem. The paper is well-written and the Appendix is educational. \n\nThe analysis is quite interesting as it combines differntial geometry and causal inference. This opens a new door to improve the current results in causal inference and likely to develop a general framework for counterfactual analysis and more.\n\nThe presented examples in both the main text and the appendix are informative. \n\nTheoretical results are quite interesting. In particular, Theorem 1 and 2.", "weaknesses": "The main question is about the generalizability of this method to more complex causal grpahs. It is not hard to see that as the causal diagram becomes a bit more complex, then applying similar analysis might be quite challenging.\n\nIt is a bit confusing that CSMs with arbitrary $\\kappa$ lead to informative bounds, i.e., $l(\\kappa,d)>l_a$ while, for instance, $\\mathcal{B}(C^2,d)$ is not informative. Because, as it is also mentioned in the text, as $\\kappa\\rightarrow\\infty$, CSM$(\\kappa)\\rightarrow \\mathcal{B}(C^2,d)$. What is the interpretaion of this?\n\nThe proof of Theorem 2 is of existance type that is the form of the informative bounds and how they depend on $\\kappa$ and $d$ are not given. But is it still possible to say something about their dependencies, e.g., $l(\\kappa,d)$ is decreasing with respect to $\\kappa$?\n\nAlthough, the augmented Pseudo-Invertible decoder method is a fine application of deep generative methods but it is not clear how sensitive it is with respect to the model hyperparametersa and sample size.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review! We appreciate that you found our paper interesting, important, and well-written. We would like to respond to the mentioned weaknesses.\n\n- Regarding the generalizability of our method to the more complex graphs: Our CSM generalizes in theory to other Markovian-SCMs (see discussion in Appendix E). This applies to settings where we observe other parents of the outcome, independent of the latent noise. \n\n  We would like to stress that, to the best of our knowledge, our paper is the first to address partial counterfactual inference for continuous outcomes (therefore we considered a rather simple causal diagram). For this, we are first to provide a measure-theoretic formulation aimed at the partial identification task. Further, we are first to formulate the results of non-informative bounds for smooth functions. We think that our theoretical results are an essential foundation for future work aimed at studying more general causal graphs.\n\n- Our CSM with $\\kappa = \\infty$ (or equivalently $\\lambda_\\kappa = 0$) should lead to non-informative bounds. However, the heuristic implementation, i.e., APID, sometimes encounters computational and optimization issues that may lead to informative bounds (e.g., in Figure 13 in Appendix H). For example, the gradual “bending” of the level sets is sometimes numerically unstable during training, and some runs omit “bending” at all as it would require passing through the high loss value region. Therefore, the computed bounds for ECOU [ECOT] may be too tight. \nAs reviewer 8Zs8 nicely summarized, “In an ideal world, the computational method (i.e., the Augmented Pseudo-Invertible Decoder in Section 5) would take a given value of $\\kappa$ and return the [exact] bounds as defined in Definition 2. However, somewhat understandably, returning the exact bounds is not necessarily numerically feasible.”  Instead, our AIPD offers numerical estimates for them.\n\n  **Action**: We will expand the discussion of our experiments and the above limitations.\n\n- By construction, it is guaranteed that, with decreasing $\\kappa$, the bounds for ECOU [ECOT] will shrink, as we decrease the class of permissible functions in the constraints of the optimization problem (Definition 2).\n\n  **Action**: We will add a formal statement as a new Corollary after Theorem 2. \n\n- Our APID should be seen as a proof of concept rather than a fully-fledged out-of-the-box method. Nevertheless, we identified a specific set of hyperparameters (incl. hidden dimensionalities, regularization terms, etc.) to work well in all the experiments  (see Appendix G.2), including the newly added real-world study. In particular, we found that the goodness-of-fit on the held-out subset (e.g., test log-likelihood) was consistently favorable after the burn-in stage of training for all the experiments.\n\n  Regarding the sample size, our APID works well as soon as it can fit the univariate density of the outcome well. In our newly added case study, the total number of observations was only $n = n_0 + n_1 = 136 + 124$, for which still obtained robust results. "}, {"review_id": "DdhrFVJNZV", "reviewer": "Reviewer_23yB", "summary": "This paper studies the partial identification of counterfactual probabilities from observational data. More specifically, the authors consider the standard bandit model containing a treatment A and a reward Y; exogenous variables U exist affecting both A and Y. The treatment A is a binary variable; while the reward Y could take any arbitrary real value bounded in an interval [l, u]. The goal is to evaluate the counterfactual probability P(y_a | a’, y’) from the observational data drawn from P(Y, A). The author first shows that this learning setting is ill-posed without any additional assumptions. That is, the learner could only obtain non-informative bound [l, u] from the observational data. This observation suggests that one should explore additional parametric assumptions to obtain informative bounds. More specifically, the authors propose a sensitivity model called Curvature Sensitivity Model. They further propose an implementation of our Curvature Sensitivity Model in the form of a novel deep generative model, called Augmented Pseudo\u0002Invertible Decoder.", "questions": "- What is the actual ECOU in Figure 7?", "limitations": "Assumptions behind the proposed algorithms are explicitly stated. This paper is theoretical and its long-term societal impact is not immediate to see.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- This paper is well written and well organized. The authors provide extensive examples and graphical illustrations. This is much appreciated.\n\n- Bounding counterfactual probabilities over a continuous outcome from the observational data is a challenging problem in causal inference. The authors study partial identification in multi-armed bandit models, which are widely applied in reinforcement learning literature. The target query P(y_a | a’, y’) is called the expected counterfactual outcome of un]treated ECOU; its applications include fairness analysis in machine learning (Kushner et al., 2018). Since ECOU is generally non-identifiable and existing bounding algorithms focus on discrete domains, this paper could have an impact on the causal inference community. \n\n- The non-informative bounding result in Theorem 1 is interesting. As far as I am aware, this result is novel. The authors also provide a novel sensitivity model of Curvature Sensitivity Model (CSM), which permits one to obtain informative bounds by bounding the curvature of level sets of the functions. They further show that CSMs generalize existing parametric assumptions for point counterfactual identification methods when the bound of the curvature is set to zero.", "weaknesses": "- This paper studies the bounding of a specific counterfactual probability, called ECOU in canonical bandit models. However, some statements in the abstract sound more general than it is. For instance, the paper states \"We prove that, in general, the ignorance interval of the counterfactual queries has non-informative bounds, already when functions of structural causal models are continuously differentiable.\" Based on my reading, the non-informative bounds in Theorem 1 only apply to bounding ECOU in bandit models. However, if we consider bounding a causal effect P(y_a) in the bandit model, Manski's bound (1990) has shown to be informative and applicable to continuous reward signals. \n\n- The simulation plots could be improved. For instance, it is unclear what the ground truth ECOUs are in Figure 7. Without that, it is hard to evaluate the validity of the proposed bounding strategy.\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for the review. It is great to hear that you found our paper well-written and well-organized, and that you think that the paper will have an impact on the causal inference community. We would like to address mentioned weaknesses and open questions.\n\n### Response to weaknesses\nWe want to stress, that our setting is different from (causal) bandits in reinforcement learning, as we do **not** have access to the environment. As such, we do **not** have the ability to perform experiments and, therefore, do not aim to maximize some reward. Instead, we have a setting based on observational/experimental data (also called logged data) with treatment and outcome (e.g., from a randomized controlled trial), and we aim to infer the expectation of the counterfactual outcome.   \n\nThe Manski's bound (1990) for the causal effect P(y_a) is fundamentally **different** from our counterfactual query (see examples in [1]), namely ECOU[ECOT]. As such, it is **not** applicable to our setting. The query P(y_a) lies only at the interventional (second) layer of Pearl’s hierarchy of causation [2] (see Table 1 in Appendix A.1). Generally, the inference on different layers almost never coincides (this is known as the *Causal Hierarchy Theorem* [2]). \n\nIn our setting, ground-truth bounds would be desirable but are *intractable*. The reason is the following: For a given value of $\\kappa$, it is computationally infeasible to derive ground truth bounds for ECOU [ECOT], even when the ground truth observation density is known. The ground-truth bounds would require solving a constrained optimization task including partial derivatives and Hausdorff integration. This is intractable, even for such simple distributions as standard normal.  \n\n### Response to questions\nUnfortunately, while ground-truth bounds are desirable, they are *intractable*. As we stated above, the reason is the following. For a given value of $\\kappa$, it is computationally infeasible to derive ground truth bounds for ECOU [ECOT], even when the ground truth observation density is known. The ground-truth bounds would require solving a constrained optimization task including partial derivatives and Hausdorff integration. This is *intractable*, even for such simple distributions as standard normal.  \n\nReferences:\n- [1] Alexander Balke and Judea Pearl. “Counterfactual probabilities: computational methods, bounds and applications”. In: Uncertainty Proceedings. Elsevier, 1994, pp. 46–54.\n- [2] Elias Bareinboim et al. “On Pearl’s hierarchy and the foundations of causal inference”. In: Probabilistic and Causal Inference: The Works of Judea Pearl. Association for Computing Machinery, 2022, pp. 507–556.\n"}]

Author rebuttal:
We are very grateful for the positive and in-depth feedback from the reviewers. We have carefully addressed all of the questions in the individual responses below. We will incorporate all changes (labeled with **Action**) into the camera-ready version of our paper. 

We have additionally uploaded empirical results as a **PDF file**. Our main improvement is the following: we added a **new experiment with a real-world case study**. Therein, we adopt our curvature sensitivity model (CSM) to answer “what if” questions related to whether the lockdown was effective during the COVID-19 pandemic. We find that our CSM provides important and meaningful insights that are of practical value. 

We want to highlight that, to the best of our knowledge, our paper is the first to address partial counterfactual inference for continuous outcomes. In particular, we are first to provide a measure-theoretic formulation to the partial identification task, and we are first to formulate the results of non-informative bounds for smooth functions. In general, designing a sensitivity model for counterfactual inference is a non-trivial task. For example, it is not clear how to straightforwardly generalize the point identification results for this purpose (e.g., there is no conventional notion of monotonicity for function from R^d to R). Our curvature sensitivity model (CSM) is one viable solution to build such a sensitivity model, where the sensitivity parameter $\kappa$ can be interpreted as the *level of non-linearity between the outcome and its latent noise variables that interact with treatment*.

Meta-review: This paper contributes several new results for partial counterfactual identification of continuous outcomes. First, the authors show that if the class of SCMs is sufficiently smooth, then the ignorance interval of the expected counterfactual outcome of [un]treated becomes un-informative (Theorem 1). Second, the authors propose the Curvature Sensitivity Model which restricts the class of SCMs in order to obtain an informative bound. Finally, the authors propose a deep generative model to perform partial counterfactual identification under the proposed curvature sensitivity model.

All reviewers provide thoughtful and positive feedbacks regarding the novelty and the theoretical contributions of the paper. The authors also provide concrete action plans for the revision, notably to strengthen the numerical experiments of the paper. I strongly encourage the authors to implement thoroughly these action plans in the camera-ready version of the manuscript.
Decision: Accept (spotlight)