A Neural Framework for Generalized Causal Sensitivity Analysis
Authors: Dennis Frauen
Fergus Imrie
Alicia Curth
Valentyn Melnychuk
Stefan Feuerriegel
Mihaela van der Schaar
Venue: ICLR 2024 poster (2024)
Keywords: Causal machine learning
treatment effect estimation
sensitivity analysis
unobserved confounding
uncertainty estimation
Primary area: causal reasoning
TL;DR: We propose a neural framework for causal sensitivity analysis under unobserved confounding that is applicable to a variety of sensitivity models, treatment types, and causal queries.

Abstract: Unobserved confounding is common in many applications, making causal inference from observational data challenging. As a remedy, causal sensitivity analysis is an important tool to draw causal conclusions under unobserved confounding with mathematical guarantees. In this paper, we propose NeuralCSA, a neural framework for generalized causal sensitivity analysis. Unlike previous work, our framework is compatible with (i) a large class of sensitivity models, including the marginal sensitivity model, $f$-sensitivity models, and Rosenbaum's sensitivity model; (ii) different treatment types (i.e., binary and continuous); and (iii) different causal queries, including (conditional) average treatment effects and simultaneous effects on multiple outcomes. This generality is achieved by learning a latent distribution shift that corresponds to a treatment intervention using two conditional normalizing flows. We provide theoretical guarantees that NeuralCSA is able to infer valid bounds on the causal query of interest and also demonstrate this empirically using both simulated and real-world data.

Published as a conference paper at ICLR 2024
A NEURAL FRAMEWORK FOR GENERALIZED CAUSAL SENSITIVITY ANALYSIS
Dennis Frauen
Fergus Imrie
Alicia Curth
Valentyn Melnychuk" 1,2
Stefan Feuerriege!!.2
Mihaela van der Schaar
ABSTRACT
Unobserved confounding is common in many applications, making causal infer- ence from observational data challenging. As remedy, causal sensitivity analy- sis is important tool to draw causal conclusions under unobserved confounding with mathematical guarantees. In this paper, we propose NEURALCSA, a neu- ral framework for generalized causal sensitivity analysis. Unlike previous work, our framework is compatible with (i) a large class of sensitivity models, including the marginal sensitivity model, f-sensitivity models, and Rosenbaum' 's sensitivity model; (ii) different treatment types (i.e., binary and continuous); and (iii) differ- ent causal queries, including (conditional) average treatment effeets and simulta- neous effects multiple outcomes. The generality of NEURALCSA A is achieved by learning a latent distribution shift corresponding to a treatment intervention using two conditional normalizing flows. We provide theoretical guarantees that NEURALCSA can infer valid bounds on the causal query of interest and also demonstrate this empirically using both simulated and real-world data.
INTRODUCTION
Causal inference from observational data is central to many fields such medicine (Frauen al.
2023a Feuerriegel et al., 2024). economics (Imbens & Angrist 1994). or marketing (Varian (0166)
However, the presence of unobserved confounding often renders causal inference challenging (Pearl
2009). As an example, consider an observational study examining the effect of smoking on lung cancer risk, where potential confounders, such genetic factors influencing smoking behavior and
cancer risk (Erzurumluoglu & et al.) (020), are not observed. Then, the causal relationship is not identifiable, and point identification without additional assumptions is impossible (Pearl. 2009).
Causal sensitivity analysis offers a remedy by moving from point identification to partial identifi- cation. To do so, approaches for causal sensitivity analysis first impose assumptions on the strength of unobserved confounding through so-called sensitivity models (Rosenbaum) 1987 Imbens] (003) and then obtain bounds on the causal query of interest. Such bounds often provide insights that the causal quantities can not reasonably be explained away by unobserved confounding, which is sufficient for consequential decision- decision-making in many applications (Kallus et al. 2019).
Existing works causal sensitivity analysis can be loosely grouped by problem settings. These vary across (1) sensitivity models, such as the marginal sensitivity model (MSM) (Tan 2006). f- sensitivity model (Jin al. 2022), and Rosenbaum' sensitivity model (Rosenbaum 1987 )) (2) treat- ment type (i.e., binary and continuous); and (3) causal query of interest. Causal queries may include (conditional) average treatment effects (CATE), but also distributional effects simultaneous effects on multiple outcomes. Existing works typically focus on a specific sensitivity model, treatment type, and causal query (Table I.. However, none applicable all settings within (1)-(3).
LMU Munich Munich Center for Machine Learning UCLA University of Cambridge Alan Turing
Instiuute
Corresponding author (frauen de)

---

Published as a conference paper at ICLR 2024
To fill this gap, propose NEURALCSA, neural framework for causal sensitivity analysis that is applicable to numerous sensitivity models, treatment types, and causal queries, including multiple outcome settings. For this, we define a large class of sensitivity models, which we call generalized treatment sensitivity models (GTSMs). GTSMs include common sensitivity models such as the MSM, f-sensitivity models, and Rosenbaum's sensitivity model. The intuition behind GTSMs is as follows: when intervening on the treatment A, the U-A edge is removed in the corresponding causal graph, which leads distribution shift in the latent confounders (see Fig. D GTSMs then impose restrictions on this latent distribution shift, which corresponds to assumptions on the "strength" of unobserved confounding.
NEURALCSA is compatible with any sensitivity model that can be written as GTSM. This is crucial prac- tical applications, where sensitivity models correspond to different assumptions on the data-generating process and may lead to different results (Yin et al.) (022)). To achieve this, NEURALCSA learns the latent distribution shift in the unobserved confounders from Fig. [using two separately trained conditional normalizing flows (CNFs). This is different from previous works for causal sensi- tivity analysis, which do not provide a unified approach across numerous sensitivity models, treatment types, and causal queries. We provide theoretical guarantees that NEURALCSA learns valid bounds on the causal query of interest and demonstrate this empirically.
P(O
P(U (e)
20 (2,9)
((r()) (x)
Y
A
Y(u)
Figure 1: Idea behind NEURALCSA to learn the latent distribution shift due treatment intervention (A). Orange nodes denote observed (random) vari- ables. Blue nodes denote unobserved variables pre-intervention. Green nodes indicate unobserved variables post- intervention under GTSM M.. Ob- served confounders X are empty for simplicity.
Our contribution are: (1) We define general class of sensitivity models, called GTSMs. (2) We propose NEU- RALCSSA, neural framework for causal sensitivity anal- ysis under any GTSMs. NEURAL A is compatible with various sensitivity models, treatment types, and causal queries. In particular, NEURALCSA is ap- plicable in settings for which bounds are not analytically tractable and solutions exist yet. (3) We provide theoretical guarantees that NEURALCSA learns valid bounds on the causal query of interest and demonstrate the effectiveness of our framework empirically.
2 RELATED WORK
In the following, we provide an overview of related literature on partial identification and causal sensitivity analysis. A more detailed overview, including literature on point identification and esti- mation, can be found in Appendix
Partial identification: The aim of partial identification is compute bounds on causal queries whenever point identification is not possible, such as under unobserved confound- ing (Manskil 1990). There are several literature streams that impose different assumptions on the data-generating process in order to obtain informative bounds. One stream addresses partial identification for general causal graphs with discrete variables (Duarte et al.] 2023).
Another stream assumes the ex- istence of valid instrumental variables (Gunsilius) 2020 Kil- bertus et al.) (020). Recently, there has been growing inter- est in using neural networks for partial identification (Xia et al.) 2021 2023 Padh et al. (2023). However, none of these methods allow for incorporating sensitiv- ity models and sensitivity analy-
Overview of key settings for causal sensitivity analyses and whether covered by existing literature (V) not (X). Treat- ments are either binary or continuous. Details are in Appendix A Our NEURALCCSA framework is applicable in all settings.
Table I:
Sensitivity model
Causal quory
MSM
f s-sesitiiiity
Rosenhaum
Binary
Cont."
Binary
Cont.
Binary
Cont.
CATE:
Distributional effects
Interventional density
Multiplc outcomes
x
X
sis.
The MBM fo cosinoon uestunnt shoclllel continncns MSM (CMSM) Jown
b002
Code is available hhtps//hithaaccommDmmDmiiiiiuuun
2

---

Published as a conference paper at ICLR 2024
Causal sensitivity analysis: Causal sensitivity analysis addresses the partial identification of causal queries by imposing assumptions on the strength of unobserved confounding via sensitivity models. It dates back to Cornfield et al. (1959), who showed that unobserved confounding could not reason- ably explain away the observed effect of smoking on lung cancer risk.
Existing works can be grouped along three dimensions: (1) the sensitivity model, (2) the treatment type, and (3) the causal query of interest (see Table [T] details in Appendix [A). Popular sensitivity models include Rosenbaum's sensitivity model (Rosenbaum) 1987). the marginal sensitivity model
(MSM) (Tan, 2006), and f-sensitivity models (Jin et al., 2022). Here, most methods have been
proposed for binary treatments and conditional average treatment effects (Kallus al.) [2099] Zhao et al. 2019 Jesson et al. 2021 Dorn & Guo [022, Dorn et al. 2022/Oprrecrt et 2023). Exten-
sions under the MSM have been proposed for continuous treatments (Jesson et al.) 20.22002|] Marmarris
et al. 2023a) and individual treatment effects (Yin et al.
2022 Jin et al. [023] Marmarelis et al.,
2023b). However, approaches for many settings are still missing (shown by in Table [). In an attempt to generalize causal sensitivity analysis, Frauen et al. (2023b) provided bounds for different treatment types (i.e., binary, continuous) and causal queries (e.g., CATE, distributional effects but not multiple outcomes). Yet, the results are limited to MSM-type sensitivity models.
To the best of our knowledge, no previous work proposes a unified solution for obtaining bounds under various sensitivity models (c.g., MSM, f-sensitivity, Rosenbaum's 's), treatment types (i.e., bi- nary and continuous), and causal queries (e.g., CATE, distributional effects, interventional densities, and simultaneous effects multiple outcomes).
MATHEMATICAL BACKGROUND
Notation:
We denote random variables X capital letters and their realizations in lowercase. We further write PPr) for the probability mass function if X is discrete, and for the probability density function with respect the Lebesque measure if X is continuous. Conditional probability mass X = x)) are written as Py xx. Finally, we denote the conditional distribution of Y X x as P(Y z) and its expectation as E{Y x].
functions/ densities P(Y
3.1 PROBLEM SETUP
Data generating process: We consider the standard setting for (static) treatment effect estimation
under unobserved confounding (Dorn& & Guo 2022). That is, we have observed confounders X C Rd., unobserved confounders U E 2 C Rd.,, treatments A E A C Rd., and outcomes Y TEYE C dR Note that we allow for (multiple) discrete or continuous treatments and multiple outcomes, i.e., da, dy 2 1. The underlying causal graph is shown in Fig. g We have access to an observational dataset D = (Ii, as y: y;) 1 sampled i.i.d. from the observational distribution A, Pobs. The full distribution (X, U,U,A,Y)
P unknown.
We use the potential outcomes framework to formalize the causal inference problem (Rubin, 1974) and denote (a)) as the potential outcome when in- tervening on the treatment and setting it to A = a. We impose the following standard assumptions (Dorn & Guo 2022).
X
A
Y
Assumption 1. We assume that for all x E and a E A the following three conditions hold: (i) A = implies Y (a) Y (consistency); (ūi) P(a x)
Figure 2: Causal graph. Observed variables are col- ored orange and unobserved blue. We allow for arbi- trary dependence between X and U.
(positivity)) and (fii) (a) 11 A X, U (latent unconfoundedness).
Causal queries: We are interested in a wide range of general causal queries. We formalize them as functionals Q(x, a, IP)) (Y (a) x)), where a functional that maps the potential outcome distribution ((Y(a) x)) to real number (Frauen al., 2023b). Thereby, we cover various queries from the causal inference literature. For example, by setting J = [E[:], we obtain the conditional expected potential outcomes/ dose-respone also obtain distributional versions of these queries by setting F to quantile instead of the expecta- tion. Furthermore, our methodology will also apply to queries that can be obtained by averaging or taking differences. For binary treatments A (0. 1}. the query T r(x) E[Y(1) x) ( a]
curves Q(z, a, P) = (a) x]. We can
3

---

x),, and the full Published as a conference paper at ICLR 2024
is called the conditional average treatment effect (CATE), and its averaged version f Tx))P(z)) dr the average treatment effect (ATE).
Our formalization also covers simultaneous effects multiple outcomes (i.c., dy 2 2). Consider query Q(x, a, P) = P(Y E S z). which is the probability that the outcome Y (a) is contained in some set S C after intervening the treatment. For example, consider two potential outcomes Y (a) and (2(a) denoting blood pressure and heart rate, respectively. We then might be interested in (a) S (2(a) l2 x), where i and l2 are critical threshold values (see Sec. 6).
3.2 CAUSAL SENSITIVITY ANALYSIS
Causal sensitivity analysis builds upon sensitivity models that restrict the possible strength of unob- served confounding ge. [Rosenbaum Rubin 1983a). Formally, we define a sensitivity model as family of distributions of (X, U, U,,,A,,, A. Y) that induce the observational distribution Pobs.
Definition 1. A sensitivity model M is a family of probability distributions P defined X U x A x y for arbitrary
U so that fu P(x,, y) du = Poba(I, y) for all IP E M.
Task:
Given a sensitivity model M and an observational distribution Pobe the aim of causal sensi- tivity analysis is solve the partial identification problem
QM a) = sup Q(x, P) and (M( - inf aF.
(1)
PEM
PEM
By its definition, the interval (2, a), (x) is the tightest interval that is guaranteed to contain the ground-truth causal query Q(z, a, P) while satisfying the sensitivity constraints. We can also obtain bounds for averaged causal queries and differences via I Q (z, (a)) dx and Qi (Ir - (m, a2) (see Appendix D for details).
We now recap three types of prominent sensitivity models from the literature, namely, the MSM, f-sensitivity models, and Rosenbaum's sensitivity model. These are designed for binary treatments A E 10, To formalize them, we first define the odds
Sensitivity models from the literature:
ratio OR(a, b) (T-a propensity score (2 P(A 1 x, u)
(1-b) the observed propensity score π (α) = P(A
Then, the definitions are:
1. The marginal sensitivity model (MSM) (Tan 2006) is defined the family of all P that satisfy s OR(T(E),x(I; < I' for all æ E X and u E ll and a sensitivity parameter I' 7 1.
2. s-sensitivity models (Jin et al.] (022) build upon given convex function f: IR>0 - I with f(1 0 and are defined via max { Sy / (OR(T(x), (τ, u))) P(u x, A =
1) du, Suf (OR-I(T(e T() u))) P(u a = du < I for all x E X.
3. Rosenbaum's sensitivity model (Rosenbaum
1987
is defined
via 1
OR(T(Z, u1), u2)) & T for all x E and und u2 E...
Interpretation and choice 1': In the above sensitivity models, the sensitivity parameter I' controls the strength of unobserved confounding. Both MSM and Rosenbaum's sensitivity model bound on odds-ratio uniformly over all E U, while the f-sensitivity model bounds an integral over u. 1 in the above sensitivity models
We refer to Appendix C for further differences. Setting corresponds to unconfoundedness and thus point identification. For I >> 1, point identification is not possible, and we need to solve the partial identification problem from Eq. 4 instead.
In practice, one typically chooses I by domain knowledge data-driven heuristics (Kallus et al.) 2019 |Hatt et al. (2022). For example, a common approach in practice is to determine the smallest so that the partially identified interval [Q_ (a, a), QJ (zx a)] includes O. Then, I' can be interpreted as level of "causal uncertainty". quantifying the smallest violation of unconfoundedness that would
explain away the causal effect (Jesson al. 2021 Jin et al. 2023)
Corresponding sensitivity models for continuous treatments can be defined by replacing the odds ratio with the density ratio DR(a,b) F and the propensity scores with the densities P(a x) and P(a u) (Bonvini We refer to Appendix Clor details and further examples of sensitivity models.
2022 Jesson et al. 2022
4

---

Published as a conference paper at ICLR 2024
4 THE GENERALIZED TREATMENT SENSITIVITY MODEL (GTSM)
We now define our generalized treatment sensitivity model (GTSM). The GTSM subsumes large class of sensitivity models and includes MSM, (ou....smmmm f-sensitivity, and Rosenbaum's sensitivity model).
Intuitively, we define the GTSM so that it includes all sensitivity models that restrict the latent distribution shift in the confounding space due to the treatment intervention (see Fig. To formalize this, we can write the observational outcome density under Assumption as
Motivation:
Pobs (y (2,a) = P(y x; u, PP( 2, a) du.
(2)
When intervening the treatment, remove the U-A edge in the corresponding causal graph (Fig. D and thus artificially remove dependence between U and A. Formally, we can write the potential outcome density under Assumption as
(aY y = = ((Y = ( ur) xx)
du du = / P(y I. u, PM x)) du.
(3)
Eq. 22 and imply that Pobs (v a) and (a y x) only differ by the densities P(u and
under the integrals (colored red and orange). If the distributions IPU 2, and
P(U .)) would coincide, it would hold that (a) = x)) (y 2r, a) and the potential outcome distribution would be identified. This suggests that we should define sensitivity models by measuring deviations from unconfoundedness via the shift between IPU a) and P(U 2z).
Definition 2. A generalized treatment sensitivity model (GTSM) is a sensitivity model M that con- tains all probability distributions that satisfy Da,a (P(U x), P(U x, a)) S for a functional of distributions Dr a sensitivity parameter T E R2., and all E X and E A.
Lemma 1. The MSM, the f-sensitivity model, and Rosenbaun's 's sensitivity model are GTSMs.
The class of all GTSMs is still too large for meaningful sensitivity analysis. This is because the sensitivity constraint may not be invariant w.r.t. transformations (e.g., scaling) of the latent space U. Definition 3 Transformation-invarianee
A GTSM M is transformation invariant if it satisfies Da,a(P(U P( x, a)) 2 Dz, (P((UU) P(t(U) 2, æ,a)) for any measurable function t: t:u - U to another latent space U.
cnnnsformationninaaiannnc is necessary for meaningful sensitivity analysis because it implies that once we choose latent space U and a sensitivity parameter I, we cannot find transformation to another latent space U so that the induced distribution on i violates the sensitivity constraint. All sensitivity models we consider in this paper are transformation-inaaiinn
stated below.
Lemma 2. The MSM, f-sensitivity models, and Rosenbaum's 's sensitivity model are transformation- invariant.
NEURAL CAUSAL SENSITIVITY ANALYSIS
We now introduce our neural approach to causal sensitivity analysis as follows. First, we simplify the partial identification problem from Eq. (I) under GTSM and propose (model-agnostic) two- stage procedure (Sec. J.I.. Then, we provide theoretical guarantees for our two-stage procedure (Sec. 5.2b. Finally, we instantiate our neural framework called NEURALCSA (Sec. 5.3).
5.1 SENSITIVITY ANALYSIS UNDER GTSM
Recall that, by definition, GTSM imposes constraints on the distribution shift in the latent confounders due to treatment intervention (Fig. [}. Our idea is to propose a two-stage procedure, where Stage learns the observational distribution (Fig. left), while Stage learns the shifted distribution of U after intervening on the treatment under a GTSM (Fig D right). In Sec. [.2 will see that, under weak assumptions, learning this distribution shift in separate stages is guar- anteed to lead to the bounds Q (z, a) and Q (x. a). To formalize this, we start by simplifying the partial identification problem from Eq. (1 for GTSM M.
Motivation:
5

---

Published as a conference paper at ICLR 2024
We begin by rewriting Eq. using the GTSM definition. Without loss of generality, we consider the upper bound Q2 a). Recall that Eq. (T) seeks to maximize over all probability distributions that compatible both with the observational data and with the sensitivity model. However, note that any GTSM only restricts the U-A part of the distribution, not the U-Y part. Hence, we can use Eq. and Eq. (2) to write the upper bound
Simplifying Eq. D
(f x. u, a, |x) (x)du) (4)
@M(
sup {F( ( H oo st 2t. (PUU) a.))STT and (ulx)= (all
sup P(U],,a), (F(Y u.)))) bucu s.t. Eq. 2 holds
F
where we maximize over (families of) probability distributions (P(U) xx. a'))'''e (left supremum), and P(U | z, a), {P(Y) z, u, (,,a)) (right supremum). The coloring indicates the components that appear in the causal querylobjective. The constraint in the right supremum ensures that the respective components of the full distribution TP are compatible with the observational data, while the constraints in the left supremum ensure that the respective components are compatible with both observational data and the sensitivity model.
Pul Pi
The partial identification problem from Eq. (4 is still hard solve as it involves two nested con- strained optimization problems. However, we can further simplify Eq. 4 We will show Sec. 5.2 that we can replace the right supremum with fixed distributions Tpe (U x, a) and (* x, a, u) for all u E u C Rd so that Eq. holds. Then, Eq. (4) reduces to single constrained opti- mization problem (left supremum). Moreover, we will show that we can choose PY(Y a, u) delta-distribution induced by an invertible function fū,a: 2 - y. The con- Figure 3: Overview of the two-stage procedure.
E>
5(Y - f2.a (u))
straint in Eq. that ensures compatibility with the observational data then reduces to Pobe (Y
|
r, a) = P ( (U) x, a). This motivates the following two-stage procedure (see Fig. 3h.
Two-stage procedure: In Stage 1, we fix PU Ix, ax and fix an invertible function f2. U y so that Pobs (Y z, a) F (fi,a a) holds. That is, the induced push-forward distribution of PU x. a) under fü must coincide with the observational distribution (Y x, a). The existence such a function is always guaranteed (Chen & Gopinath 2000). In Stage 2, we then set P(U x, a) p (U x, a) and P(Y x, u) PY z, a, u) in Eq. and only optimize over the left supremum. That is, we write stage for discrete treatments as
sup s.. P(a]z)=P" (ul)))))"(ull,a))))) A(ule,AAa)) and D..a (P(U)o). P' (U(
F (U)
(5)
where we maximize over the distribution P(u x, A A a) for a fixed treatment intervention a. For continuous treatments, can directly take the supremum over PP(u z).
5.2 THEORETICAL GUARANTEES
We now provide a formal result that our two-stage procedure returns valid solutions to the partial
identification problem from Eq. (4. The following theorem states that Stage 2 of our procedure
able to attain the optimal upper bound Q+ (z, a) from Eq. 4 even after fixing the distributions
P PU F. a) and IP* (Y) xr, a, u) as done in Stage 1. proof provided in Appendix B
Theorem (Sufficiency of two-stage procedure). Let M be transformationninvare invariant GTSM. For fixed x E X and a E A, let TP" (U a) be a fixed distribution on U BRdy and fä U + Y fixed invertible function so that (Y (r,a)) = F (f2,a
x, a). Let P* denote the space of all full probability distributions Bx that induce PU x, a) and P (Y a) = 8(Y - fü (u)) and that satisfy Pe* M. Then, under Assumption and (u a) infr-ep- Q(x, P*).
i holds that Q (z, a) supr.ep- Q a.

---

Published as a conference paper at ICLR 2024
Intuition: Theorem I has two major implications: (i) It is sufficient to fix the distributions P r,a) and PY I, U, a), i.c. the components in the right supremum of Eq. (4) and only optimize over the left supremum; and (ii) is sufficient to choose 1 (Y z, u, a)
(u)) as delta- distribution induced by an invertible function Sz U -> V/ which satisfies the data-compatibillyy constraint Poba (Y z, a) = (f**aa()) | x; a). Intuition for (i): In Eq. 4 we optimize jointly over all components of the full distribution. This suggests that there are multiple solutions that differ only in the components of unobserved parts of IP (i.e., in U) but lead to the same potential outcome distribution and causal query. Theorem T states that may restrict the space of possible solutions by fixing the components IP xz, a. and P* 2, a, u), without loosing the ability to attain the optimal upper bound Q Q'n (2r, a) from Eq. (4). Intuition for (ii): We cannot pick any P I, a, u) that satisfies Eq. (2). For example, any distribution that induces Y TT U X, A would satisfy Eq. (2) but implies unconfoundedness and would thus not lead to a valid upper bound Qu (3, a). Intuitively, we have to choose a P(Y x, a, u) that induces "maximal dependence" (mutual information) between U and Y (conditioned on X and A), because the GTSM does not restrict this part of the full probability distribution P.. The maximal mutual information is achieved if we choose PPY a, = o(Y -Sa (u))
5.3 NEURAL INSTANTIATION: NEURALCSA
We now provide a neural instantiation called NEURALCSA for the above two-stage procedure using conditional normalizing flows (CNFs) (Winkler et al. 2019). The architecture of NEU- RALCSA is shown in Fig. NEURALCSA instantiates the two-step procedure as follows:
Stage 1:
We fix (U x, a) to the standard normal distribution on 4 = Rdd Our task is then to learn an invertible function f2.:: U + y that maps the standard Gaussian distribution on U to IP
Legend
8
CNF
CNF
NN
network
Gord. Normaiiring
NN
NN
CNF
Now pareratee
PPote (Y I, a). We model f² as CNF aa) where f" is a normalizing flow (Rezende & Mohamed| (015)) for which the parameters the output of fully connected
ars colbtt
Observed variable
Stage
x
Stage 1
Figure 4: Architecture of NEURALCSA.
neural network gy. which itself is parametrized by 0 WWinklrr et al. 2019). We obtain 0 by maximiz- ing the empirical Stage loss Li (0) >i= log [PT (x..2.) (t..e)) (U) y:). where U NN(O,, , Ia,,)
is standard normally distributed, The stage I loss can be computed analytically via the change-of- variable formula (see Appendix F).
Stage 2: In Stage 2, we need to maximize over distributions on U in the latent space U that maximize the causal query F(P(f)) gh.ge (U) | x)), where popt is solution from maximizing (0) in
We can do this by learning second CNF Fängr,aj. where F: ú U is a normalizing flow that maps a standard normally distributed auxiliary U ~ No0,, Ia,)) to the latent space U, and whose parameters are the output of a fully connected neural network F, parametrized by 1). The CNF
stage 1.
F (a.a) from Stage 2 induces a new distribution on U, which mimics the shift due to unobserved confounding when intervening instead of conditioning (i.c., going from Eq. (2) Eq. (3) We can compute the query under the shifted distribution by concatenating the Stage 2 CNF with the Stage CNF and applying F the shifted outcome distribution (see Fig. More precisely, we optimize n) by maximizing or minimizing the empirical Stage 2 loss
C2(n) = = (P( (Fic (Rs,as) (1 (0- Do (8) + Emla)
(6)
where Erg.a, (a:no(a: I;)), if A is discrete, and Em;.a: 0, if A is continuous.
Learning algorithm for stage 2: There are two remaining challenges we need to address in Stage 2: (i) optimizing Eq. (6) does not ensure that the sensitivity constraints imposed by the GTSM M hold; and (ii) computing the Stage 2 loss from Eq. (6) may not be analytically tractable. For (i), we propose to incorporate the sensitivity constraints by using the augmented Lagrangian method (Nocedal& & Wright 2006), which has already been successfully applied in the context of partial identification with neural networks (Padh et al. 20033 Schröder et al., 2024). For (ii), we propose L.d. Bernoulli(Po.... x))
to obtain samples ū
(w2)) (12,0)
iid. .V(Oa,) Ta,) and (4(2))
7

---

Published as a conference paper at ICLR 2024
together with Monte Carlo estimators (n () of the Stage loss C2(n) and Dz,a(n, 17, 11) of the sensitivity constraint ( z), 2; x,a)). We refer to Appendix for details, including instantiations of our framework for numerous sensitivity models and causal queries.
Implementation: We use autoregressive neural spline flows (Dukkan et al. 2019 Dolatabadi et al. 2020 For estimating propensity scores (a I:), we use fully connected neural networks with softmax activation. We perform training using the Adam optimizer (Kingma & Bal 2015). We choose the number of epochs such that NEURALCSA satisfies the sensitivity constraint for given sensitivity parameter. Details are in Appendix
EXPERIMENTS
We now demonstrate the effective- ness of NEURALCSA for causal sen- sitivity analysis empirically. common in the causal infer- ence literature, we use synthetic and semi-synthetic data with known causal ground truth to evaluate NEU- RALCSA (Kallus et al.] 2019 Jes- son et al. (2022). We proceed as follows: (i) We use synthetic data to show the validity of bounds from
Onf-2
0F 2
Oea [-4
-1
Gr=2
As
Figure 5: Validating the correctness of NEURALCSA (ours) by comparing with optimal closed-form solutions (CF) for the MSM on simulated data. Left: Dataset 1, binary treat- ment. Right: Dataset 2, continuous treatment. Reported: mean + standard deviation over runs.
NEURALCSA A under multiple sensitivity models, treatment types, and causal queries. We also show that for the MSM, the NEURALCSA bounds coincide with known optimal solutions. (ii) We show the validity of the NEURALCSA bounds using a semi-synthetic dataset. (iii) We show the appli- cability of NEURAL A in study using real-world dataset with multiple outcomes, which cannot be handled by previous approaches. We refer to Appendix D for details regarding datasets and experimental evaluation, and to Appendix H for additional experiments.
(i) Synthetic data: We consider two synthetic datasets of sample size n 10000 inspired from previous work on sensitivity analysis: Dataset is adapted from Kallus et al. (2019) and has a binary treatment A E (0. 1).
0.34
TV -020
The Hata-generating process follows an MSM with oracle sensitivity pa- rameter I* = 2. We are interested in the CATE r(x) = (1) Y z). Dataset is adapted from Jes- son et al. (2022) and has a continuous treatment A E [0. 1). Here, are in- terested in the dose-response function u(z, a) E (a) | x]. where we choose 0.5. We report results for further treatment values in Appendix H
45
-0.0
0.5 -10
-25
3.
05
Confirming the validity of our NEURALCSA bounds for various sensitivity models. Left: Dataset bi- nary treatment. Right: Dataset 2, continuous treatment. Re- ported: mean standard deviation over runs.
Figure 6:
We first compare our NEURALCSA A bounds with existing results closed-form bounds (CF) for the
MSM (Domn& & Guo, 2022 Frauen et al. 2023b), which have been proven be optimal. We plot both NEURALCSA and the CF for both datasets and three choices of sensitivity parameter I e {2. 4, 10} (Fig. 5). Our bounds almost coincide with the optimal CF solutions, which confirms that NEURALCSA learns optimal bounds under the MSM.
We also show the validity of our NEURAI.CSA bounds for Rosenbaum's 's sensitivity model and the following f-sensitivity models: Kullbach-Leibler (KL, fxx = log(z:)) Total Variation (TV, f(x) = 0.5 x - 1)) Hellinger (HE, f(x) (Vr x/ 1afx--133 1)) and Chi-squared (x2. fxx) = (æ 1)²). To do so, we choose the ground-truti sensitivity parameter I"* for each sensitivity model that satisfies the respective sensitivity constraint (see Appendix G for details). The results are in Fig. 6 We make the following observations: (i) all bounds cover the causal query both datasets, thus confirming the validity of NEURALCSA. (ii) For Dataset 1, the MSM returns the tightest bounds because our simulation follows MSM.
8

---

Published as a conference paper at ICLR 2024
We create semi-synthetic dataset using MIMIC-III (Johnson et al. 2016). which includes electronic health records from patients admitted to intensive care units. We extract 8 confounders and binary treatment (mechanical ventilation). Then, augment the data with a synthetic unobserved confounder and outcome. We obtain n = 14719 patients and split the data into train (80%). val (10%), and test (10%). For details, see Appendix G
(ii) Semi-synthetic data:
We verify the validity of our NEURALCSA A bounds for CATE in the following way: For each sen- sitivity model, we obtain the smallest oracle sensitivity parameter I' that guarantees coverage (i.e., satisfies the respective sensitivity constraint) for 50% of the test samples. Then, we plot the coverage and median interval length of the NEURALCSA bounds over the test set. The results are in Table
We observe that (i) all bounds achieve at least 50% coverage, thus con- firming the validity of the bounds, and (ii) some sensitivity models (e.g., the MSM) are conservative, i.e., achieve much higher coverage and in- terval length than needed. This because the sensitivity constraints of these models do not adapt well to the data-generating process, thus the need for choosing a large I to guarantee coverage. This highlights the importance of choosing sensitivity model that captures the data- generating process well. For further details, we refer to (Jin et al. (2022). We also provide further insights into the difference between two exem- plary sensitivity models: the MSM and the KI Kl--sensitivity model. To do so, we plot the observational distribution from stage I together with the shifted distributions from stage 2 that lead to the respective upper bound for a fixed test patient (Fig. 77. The distribution shift corresponding to the MSM a step function, which is consistent with results from established literature (Jin et al. 2023). This is in contrast to the smooth distribution shift obtained by the KL-sensitivity model. In addition, this example illustrates the possibility of using NEURALCSA for sensitivity analysis on the entire interventional density.
Zage
3.5
Figure 7: Analytic stage 2 densities for MSM and KL-sensitivity model (upper bounds).
(iii) Case study using real-world data: We now demon- strate an application of NEURAICCSA to perform causal sensitivity analysis for an interventional distribution on multiple outcomes. To do so, we use the same MIMIC- III data from our semi-synthetic experiments but add two outcomes: heart rate (Y1) and blood pressure (Y2). We consider the causal query PYY (1) 2 115, (1) 2 90
Table 2: Results for semi-synthetic data
Sensitivity model
Coverage
Interval length
MSM 1 = 5.48
0.91 0.03
0.77 : 0.03
KLT I 0.25
0.51= 0.07
0.31 = 0.01
TV I 0.38
0.86 0.09
0.83 : 0.14
HE I' = 0.18
0.83 0.06
0.63 = 0.03
X 0.68
0.67 0.07
0.41 +
RB 1 14.42
0.79 0.07
0.56 : 0.03
X = x). i.e., the joint probability of achieving heart rate higher than 115 and a blood pressure higher than 90 under treatment intervention ('danger area"). We consider an MSM and train NEURALCSA with sensitivity parameters I' E 42. Then, we plot the stage distribution together with both stage 2 distributions for fixed, untreated patient from the test set in Fig. g
Reported: mean + standard deviation runs).
As expected, increasing I leads to a distribution shift in the direction of the "danger area",, i.e., high heart rate and high blood pressure. For T = 2, there is only a moderate fraction of probability mass inside the danger area, while, for I' 4, this fraction is much larger. A practitioner may potentially decide against treatment if there are other unknown factors (e.g., undetected comorbidity) that could result in confounding strength of = 4.
La
120
Figure 8: Contour plots of 2D densities obtained by NEU- RALCSA under an MSM. Here, we aim to learn an upper bound of the causal query P(Y (1) 2 115, (2)) > 90 X ro) for a test patient zg. Left: Stage 1/ observational distribu- tion. Middle: Stage 2, r = 2. Right: Stage 2, r = 4.
Conclusion. From a methodological perspective, NEURALCSA offers new ideas causal sensi- tivity analysis and partial identification: In contrast to previous methods, NEURALCSA explicilly learns latent distribution shift due to treatment intervention. We refer to Appendix[] discussion on limitations and future work. From applied perspective, NEURALCSA enables practitioners to perform causal sensitivity analysis in numerous settings, including multiple outcomes. Furthermore, it allows for choosing from wide variety of sensitivity models, which may be crucial to effectively incorporate domain knowledge about the detergnngg process.

---

Published as a conference paper at ICLR 2024
Acknowledgements. S.F. acknowledges funding via Swiss National Science Foundation Grant 186932.
REFERENCES
Matteo Bonvini, Edward Kennedy, Valerie Ventura, and Larry Wasserman. Sensitivity analysis for marginal structural models. arXiv preprint, arXiv:2210006681 2022.
Scott Shaobing Chen and Ramesh A. Gopinath. Gaussianization. In NewrIPS, 2000.
Victor Chernozhukov, Ivan Fernández- Val, and Blaise Melly. Inference on counterfactual distribu- tions. Econometrica, 81(6):2205-220) 2013.
Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitey Newey, and James M. Robins. Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21(1):C1-C68 2018. ISSN 1368-4221.
James Cornfield, William Haenszel, E. Cuyler Hammond, Abraham M. Lilienfeld, Michael B. Shimkin, and Ernst L. Wynder. Smoking and lung cancer. Recent evidence and a discussion of some questions. Jourmal of the National Cancer Institute, 22(1):173-203, 1959.
Alicia Curth and Mihacla van der Schaar. Nonparametric estimation of heterogeneous treatment effects: From theory to learning algorithms. In AISTATS, 2021.
Haid M. Dolatabadi, Sarah Erfani, and Christopher Leckie. Invertible generartive modeling using linear rational splines. In AISTATS, 2020.
Jacob Dorn and Kevin Guo. Sharp sensitivity analysis for inverse propensity weighting via quantile balancing. Journal of the American Statistical Association, 2022.
Jacob Dorn, Kevin Guo, and Nathan Kallus. Doubly-valid/ doubly-sharp sensitivity analysis for causal inference with unmeasured confounding. arXiv preprint, urXiv:2112.1444 1449, 2022.
Guilherme Duarte, Noam Finkelstein, Dean Knox, Jonathan Mummolo, and Ilya Shpitser. An tomated approach to causal inference in discrete settings. Journal of the American Statistical Association, 2023.
Conor Durkan, Artur Bekasov, lain Murray, and George Papamakarios. Neural spline flows. In NeurlPS, 2019.
A. Mesut Erzurumluoglu and et al. Meta-analysis of up to 622,409 individuals identifies 40 novel smoking behaviour associated genetic loci. Molecular psychiatry, 25(10):2392-2409 2020.
Stefan Feuerriegel, Dennis Frauen, Valentyn Melnychuk, Jonas Schweisthal, Konstantin Hess, Ali- cia Curth, Stefan Bauer, Niki Kilbertus, Isaac S. Kohane, and Mihaela der Schaar. Causal machine learning for predicting treatment outcomes. Nature Medicine, 2024.
Dennis Frauen, Tobias Hatr, Valentyn Melnychuk, and Stefan Feuerriegel. Estimating average causal effects from patient trajectories. In AAAÍ, 2023a.
Dennis Frauen, Valentyn Melnychuk, and Stefan Feuerriegel. Sharp bounds for generalized causal sensitivity analysis. In NewriPS, 2023b.
Florian Gunsilius. A path-sampling method to partially identify causal effects in instrumental vari- able models. arXiv preprint, arXiv: 191.099900 2020.
Tobias Hatt, Daniel Tschernutte, and Stefan Feuerriegel. Generalizing off off-policy learning under sample selection bias. In UAI, 2022.
Siyu Heng and Dylan S. Small. Sharpening the rosenbaum sensitivity bounds adress concerns about interactions between observed and unobserved covariates. Statistica Sinica, (1(Onee spe- cial issue):2331-2353 2021.
10

---

Published as a conference paper at ICLR 2024
Jennifer L. Hill. Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphical Statistics, 20(1):2017-24444 2011.
Guido W. Imbens. Sensitivity to exogeneity assumptions in program evaluation. American Economic
Review, 93(2):128 132, 2003, ISSN 0002-8282.
Guido W. Imbens and Joshua D. Angrist. Identification and estimation of local average treatment
effects. Econometrica, 62(2):467-475, 1994.
Andrew Jesson, Sören Mindermann, Yarin Gal, and Uri Shalit. Quantifying ignorance in individual- level causal-effect estimates under hidden confounding. In ICML,, 2021.
Andrew Jesson, Alyson Douglas, Peter Manshausen, Nicolai Meinshausen, Philip Stier, Yarin Gal, and Uri Shalit. Scalable sensitivity and uncertainty analysis for causal-effect estimates of continuous-valued interventions. In NeurIPS, 2022.
Ying Jin, Zhimei Ren, and Zhengyuan Zhou. Sensitivity analysis under the f-sensitivity models: A distributional robustness perspective. arXiv preprint, arXiv:2203.04377, 2022.
Ying Jin, Zhimei Ren, and Emmanuel J. Candès. Sensitivity analysis of individual treatment ef- fects: A robust conformal inference approach. Proceedings of the National Academy of Sciences (PNAS), 120(6), 2023.
Fredrik D. Johansson, Uri Shalit, and David Sonntag. Learning representations for counterfactual inference. In ICML, 2016.
Alistair E. W. Johnson, Tom Pollard, Lu Shen, Li-wei H. Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G. Mark. MIMIC-III, a freely accessible critical care database. Scientific Data, 3(1):160035, 2016. ISSN 2052-4463.
Nathan Kallus and Angela Zhou. Confounding-robust policy improvement. In NewrIPS, 2018.
Nathan Kallus, Xiaojie Mao, and Angela Zhou. Interval estimation of individual-level causal effects under unobserved confounding. In AISTATS, 2019.
Edward H. Kennedy. Towards optimal doubly robust estimation of heterogeneous causal effects. Electronic Journal of Statisties, 17(2):3088-3044) 2023.
Edward H. Kennedy, Sivaraman Balakrishnan, and Larry Wasserman. Semiparametric counterfac- tual density estimation. Biometrika, 2023. ISSN 0006-3444.
Niki Kilbertus, Philip J. Ball, Matt Kusner, Adrian Weller, and Ricardo Silva. The sensitivity of counterfactual faimess to unmeasured confounding. In UAI, 2019.
Niki Kilbertus, Matt J. Kusner, and Ricardo Silva. A class of algorithms for general instrumental variable models. In NeurIPS, 2020.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
Sören R. Künzel, Jasject S. Sekhon, Peter J. Bickel, and Bin Yu. Metaleamers for estimating het- erogeneous treatment effects using machine learning. Proceedings of the National Academy of Sciences (PNAS), 116110)4156-4165 2019.
Charles F. Manski. Nonparametric bounds on treatment effects. The American Economic Review, 80(2):319 123333,,,, 1990.
Myrl G. Marmarelis, Elizabeth Haddad, Andrew Jesson, Jeda Jahanshad, Aram Galstyan, and Greg Ver Steeg. Partial identification of dose responses with hidden confounders. In UAI, 2023a.
Myrl G. Marmarelis, Greg Ver Steeg, Aram Galstyan, and Fred Morstatter. Ensembled prediction in- tervals for causal outcomes under hidden confounding. arXiv preprint, arXiv:2306.09520, 2023b.
Valentyn Melnychuk, Dennis Frauen, and Stefan Feuerriegel. Normalizing flows for interventional density estimation. In ICMI, 2023a.
11

---

Published as a conference paper at ICLR 2024
Valentyn Melnychuk, Dennis Frauen, and Stefan Feuerriegel. Partial counterfactual identification of continuous outcomes with a curvature sensitivity model. In NeurIPS, 2023b.
Krikamol Muandet, Montonobu Kanagawa, Sorawit Saengkyongam, and Sanparith Marukatat. Counterfactual mean embeddings. Journal of Machine Learning Research, 22:1-71, 2021.
Jorge Nocedal and Stephen J. Wright. Numerical optimization. Springer series in operations re-
search. Springer, New York, 2nd ed. edition, 2006. ISBN 0387303030.
Miruna Oprescu, Jacob Dorn, Marah Ghoummaid, Andrew Jesson, Nathan Kallus, and Uri Shalit. B-learner: Quasi-oracle bounds on heterogeneous causal effects under hidden confounding. In ICML, 2023.
Kirtan Padh, Jakob Zeitler, David Watson, Matt Kusner, Ricardo Silva, and Niki Kilbertus. Stochas- tic causal programming for bounding treatment effects. In CLeaR, 2023.
Judea Pearl. Causality. Cambridge University Press, New York City, 2009. ISBN 9780521895606
Yury Polyanskiy and Yihong Wu. Information theory: From coding to learning. Draft. Cambridge University Press, 2022.
Danilo Jimenez Rezende and Shakir Mohamed. Variational inference with normalizing flows. In ICML, 2015.
Paul R. Rosenbaum. Sensitivity analysis for certain permutation inferences in matched observational studies. Biometrika, 74(1):13-26, 1987. ISSN 0006-3444.
Paul R. Rosenbaum and Donald B. Rubin. The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1):41-55, 1983a. ISSN 0006-3444.
Paul R. Rosenbaum and Donald B. Rubin. Assessing sensitivity to an unobserved binary covariate in an observational study with binary outcome. Journal of the Royal Statistical Society: Series B, 45(2):212-216 1983b. ISSN 1467-9868.
Donald B. Rubin. Estimating causal effects of treatments in randomized and nonrandomized studie.. Journal of Educational Psychology, 66(5))688-701, 1974. ISSN 0022-0663.
Maresa Schröder, Dennis Frauen, and Stefan Feuerriegel. Causal fairness under unobserved con- founding: neural sensitivity framework. ICLR, 2024.
Jonas Schweisthal, Dennis Frauen, Valentyn Melnychuk, and Stefan Feuerriegel. Reliable off-policy learning for dosage combinations. In NeurlPS, 2023.
Uri Shalit, Fredrik D. Johansson, and David Sontag. Estimating individual treatment effect: Gener- alization bounds and algorithms. In /CML, 2017.
Claudia Shi, David M. Blei, and Victor Veitch. Adapting neural networks for the estimation of treatment effects. In NeurlPS, 2019.
Dan Soriano, Eli Ben-Michael, Peter J. Bickel, Avi Feller, and Samuel D. Pimentel. Interpretable sensitivity analysis for balancing weights. Journal of the Royal Statistical Society Series A: Statis- fics in Sociery, 93(1):113, 2023. ISSN 0964-1998.
Zhiqiang Tan. distributional approach for causal inference using propensity scores, Journal of the American Statistical Association, 101(47)::619-1637 2006.
Mark J. van der Laan and Donald B. Rubin. Targeted maximum likelihood learning. The Interna- tional Journal of Biostatistics, 2(1), 2006.
Hal R. Varian. Causal inference in economics and marketing. Proceedings of the National Academy of Sciences (PNAS), 1 113(27)7310-73115 2016.
Stefan Wager and Susan Athey. Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, 1228523):1228-1222 2018
12

---

Published as a conference paper at ICLR 2024
Shirly Wang, Matthew B.A. McDermott, Geeticka Chauhan, Marzyeh Ghassemi, Michael C. Hughes, and Tristan Naumann. MIMIC-extract: A data extraction, preprocessing, and repre- sentation pipeline for MIMIC-III In CHIL, 2020.
Christina Winkler, Daniel Worrall, Emiel Hoogeboom, and Max Welling. Learning likelihoods with conditional normalizing flows. arXiv preprint, 1rXiv:1912.0000 2019.
Kevin Xia, Kai-Zhan Lee, Yoshua Bengio, and Elias Bareinboim. The causal-neural connection:
Expressiveness, learnability, and inference. NeurIPS, 2021.
Kevin Xia, Yushu Pan, and Elias Bareinboim. Neural causal models for counterfactual identification and estimation. In ICLR, 2023.
Mingzhang Yin, Claudia Shi, Yixin Wang, and David M. Blei. Conformal sensitivity analysis for individual treatment effects. Journal of the American Statistical Association, pp. 1-14, 2022.
Jinsung Yoon, James Jordon, and Mihaela van der Schaar. Ganite: Estimation of individualized treatment effects using generative adversarial nets. In ICLR, 2018.
Qingyuan Zhao, Dylan S. Small, and Bhaswar B. Bhattacharya. Sensitivity analysis for inverse probability weighting estimators via the percentile bootstrap. Journal of the Royal Statistical Sociery: Series B, 81(4):735-76 761, 2019. ISSN 1467-9868.
13

---

Published as a conference paper at ICLR 2024
A EXTENDED RELATED WORK
In the following, we provide an extended related work. Specifically, we elaborate on (1) systematic overview causal sensitivity analysis, (2) its application in settings beyond partial identification of interventional causal queries, and (3) point identification and estimation of causal queries.
A.1 A SYSTEMATIC OVERVIEW ON CAUSAL SENSITIVITY ANALYSIS
In Table 3 group by the underlying sensitivity model, the treatment type, and the causal query. As such, Table 3 extends Table in that we follow the same categorization but now point to the references from the literature.
we provide systematic overview of existing works for causal sensitivity analysis, which
Table 3: Overview of key works for causal sensitivity analyses under the MSM, f-ensittivity models, Rosenbaum's sensitivity model. Settings with no existing literature indicated with red cross (X). Treatments are either binary or continuous. NEURALCSA framework is applicable in all settings.
Sensitivity model
MSM
f f-sensitivity
Rosenhsum
Causal query'
Binazy
Cont.
Binary
Cont.
Binasy
Cont.
CATE
2006
Bonini
20221
Rosenbaum
Rubin
1983b
Ziby 3
2019
PRaDER CE
20236
Hong & Smaill 202]
Jessn R m M17
A122
Optese
Soyano 30
Distributiona effects
Fraien al.
20236
Francn et al.
2023b
X
X
x
Interventional density
Jt 2013
20236
Jin
2022
Marinauh Q ml
2023b
Frilion cl
102.66
Multiple outcomes
x
X
x
The
105113340M
Evidently, many works have focused on sensitivity analysis for CATE binary treatment settings. For many settings, such f-sensitivvity and Rosenbaum's sensitivity model with continuous treat- ments or multiple outcomes, no previous work exists. Here, NEURALCSA the first work that allows for computing bounds in these settings.
A.2 SENSITIVITY ANALYSIS OTHER CAUSAL SETTINGS
Causal sensitivity analysis has found applicability not only in addressing the partial identification problem, as discussed in Eq. (I) but also in various domains machine learning and causal infer- ence. We briefly highlight some notable instances where ideas from causal sensitivity analysis have made substantial contributions.
One such stream of literature is off-policy learning, where sensitivity models have been leveraged to account for unobserved confounding or distribution shifts (Kallus & Zhou 2018 Hatt et al.) (022). Here, sensitivity analysis enables robust policy Icarning. Another example is algorithmic fairness, where sensitivity analysis has been used to study causal fairness notions (e.g., counterfactual fair- ness) under unobserved confounding (Kilbertus et al. 2019). Finally have been used to study the partial identification of counterfactual queries (Melnychuk et al. 2023b
A.3 POINT IDENTIFICATION AND ESTIMATION
we replace the latent unconfoundedness assumption in Assumtion Tith (non-latent) unconfound- edness, that is,
(a)) IL A A| for all a E A, we can idint-identify the distribution the potential outcomes via
(PY(a)) x)) (a) y xr, a) Pans x.
(8)
14

---

Published as a conference paper at ICLR 2024
Hence, inferring the causal query Q a, P) reduces to purely statistical inference problem, i.e., estimating obs (Y 22 a)) from finite data.
Various methods for estimating point-identified causal effects under unconfoundedness have been proposed. In recent years, a particular emphasis has been on methods for estimating (conditional) average treatment effects (CATEs) that make use of machine learning model flexible non-lineat
relationships within the data. Examples include forest- based methods (Wager Athey (2018) and deep learning (Johansson al. 2016 Shalit al. 2017 Yoon al. 2018 [hh al. 2019). An- other stream of literature incorporates theory from
statistics and provides robustness Chemozhukov al. 20188) Künzel al.
and efficiency guarantees (van der Laan Rubin 2006
2019 Curth & van der Schaar [2021] Kennedy 2023). Beyond CATE, methods have also been
proposed for estimating distributional effects or potential outcome densities (Chernozhukov et al, 2013 Muandet et al., 2021] Kennedy et al. 2023). In particular, Melnychuk et al. (2023a) proposed normalizing flows for potential outcome densities. Finally, [Schweisthal al.[ leveraged nor- malizing flows for estimating the generalized propensity score in setting with continuous treatment. We emphasize that all these methods focus on estimation of point-identified causal queries, while are interested in causal sensitivity analysis and thus partial identification under violations of the unconfoundedness assumption.
15

---

Published as a conference paper at ICLR 2024
B PROOFS
B. PROOF OF LEMMA 1
We provide proof for the following more detailed version of Lemma 1
Lemma 3. The MSM, the f-sensitivity model, and Rosenbaum's 's sensitivity model are GTSMs with
sensifivity parameter r. Let p(z, a,
P(a x)) and p(r, u1, u2, a) For the MSM, we have
[-F(a(z] [[ula,a]
-- P(ulz)
(owl 11a. P(ualzaa)Pu P(y Plwa
Dæ,a (P(U 2), x, 2,))) max sup p(z, u, a), sup p(z, u, a.)
(9)
wuel
well
For f-sensitiiiity models, we have
(P(U P(U a,))) max U1 (lllz, a)) x, a. du, / (,ox,,,aa)")))
x, du
(10)
For Rosenbaum's 's sensitivity model, we have
p(z,, u1, U2, a); sup rlz, u1, U2 a) }-) (11) wa.
Dr (PP(( P(U a., max sup
Proof. We show that all three sensitivity models (MSM, f-sensitivity models, and Rosenbaum's sensitivity model) are GTSMs. Recall that the odds ratio is defined as OR(a, b) (1-a) (1-b)
MSM: Using Bayes' theorem, we obtain P(u x, a)
((ala) and therefore P(a
PPuu x)
p(2, u, a) - P(a x))
Pa(a xx)
(12)
P(u x.a)
P(a |x)
IP(a x))
(13)
P(a x))
P(aa x;, u)
P(a xx)
(a xr, u)
(14)
1- PP(a ())
IP(a ar, u)
OR (P (x), P(a u))
(15)
Hence, max (sup p(z, u, a), sup p(z, u, a) - T is equivalent to
1
r (P(a P(a Ir u)) S r
(16)
for all u E U, which reduces the original MSM defintion for a 1. /-sensitivity models: Follows immediately from p(a, a, = OR (IP(a) x),, P(a x, u)).
Rosenbaum's sensitivity model: We can write
PP( xr, PP
(u2
xr, P(a ()) P(u: xr P(u2 x)
p(r, M1, U2, a)
(17)
P(u xr, P U22
2., a.a x) P(uz
x, a.a)P((1 x))
(a
ar, u1 P((n
x))
(P(a x, P2)2
x) - PPu2 | xx)
(18)
(a 2,, ug P())
22)
(Paa x, P(,s
x) - P(u1 (x)
(a x.
[(aa 22 -
(19)
IP(a 2, w2)
[P(a xr, (1
OR (IP(a xr, w1). P(a xr u2))
(20)
Hence, max {sup uusp use p(z, u1, U2 a), sup SuPay u.eu P(z, U1, M2 a) -1 < T is equivalent to OR (P( I 11, x, x2)) I
(21)
for all U1 U2 E U, which reduces to the original definition of Rosenbaum's sensitivity model for n
16

---

Published as a conference paper at ICLR 2024
B.2 PROOF OF LEMMA 2
Proof. We show transformation-invaaiic separately for all three sensitivity models (MSM, f- sensitivity models, and Rosenbaum's sensitivity model).
MSM: Let (TP(U r), PUU a)) = r, which implies that implies $ S p(x, a. S P for all u E U. By rearranging terms, we obtain
P(u 2)) s (T(1 (a(a x)) P(a x)) P(u x.
(22)
and
P(u 12) { - P(a (P)a| P(a P(a)))) P(u x,
(23)
Let L: U +u u be transformation of the unobserved confounder. By using Eq.
22 we can write
plx, (())((
1
P(t(u) x))
IP(a x))
(24)
1- P(a x)
P(((u) x.
1
[[ ((u))) (x) du'
P((a | xx)
(25)
1 1 1-a(a x:)
([(((u))((<())))) x, du
1
((((()))((v))F(( a,a) du'
r
(26)
P(a x)
((((()))(((())(()) x.a)
(27)
for u E EM Similarly, we can use Eq.
23 to obtain
1
[( ((u)))((()) x) du' [ F(( x, a) du
(x,t(u)),a) 1 1- P-a(a x)
P(a ((a|x) 1z)
(28)
2 1
x a) du'
(29)
P(a x)
((((()))(w())((C))) (,
1
(30)
r
for all u E U. Hence,
D Pz,a(P(tt( P(((U) x. a))
max
sup p(x; t(u), a), sup p(x, ((u) ST. (31)
well
well
f /sensitiivity models: This follows from the data compression theorem for f-divergences. We refer to Polyanskiy & Wu 2022) for details.
Rosenbaum's 's sensitivity model: We begin by rewriting
IP(u a) P(ua
xr P(a x)) - P(ui xr, (.. P(22 x))
p(x, u1, U2
(32)
P(u1 Ir, ar,a)P(wz
x a(a) x) P(uz
x, a.
1
P(u2 x))
P(a z²)
(33)
[(u]==)
P(a x))
P(u2 x,a)
Flu [.a]
as function of density ratios on 44 Let now Dz.a (P((U x)) P(U x., a)) = r. This implies
[
P(uz xx)
P(u 2)) <
[r'
-P(a| P(alz)) P(a x) Plus 2x,a)
(34)
P P(u2
2,0)
and
P(u2 x))
P((1 [x) 2
IP(a ()) + P(a at) P(un (x,a)
(35)
P (ug
17

---

Published as a conference paper at ICLR 2024
for all u1 U2 E U. Let t: u i7 be transformation. By using Eq. (34 and Eq.
(35
we obtain
(36)
(10 F(a| x))
.x) dul
P(a x))
(37)
J ((t((u)) - tt(s)))P(us a) dul
P(u1 (x)
r
-((a x))
(38)
P(uz (x) P(ug(z.a)
IP(a x))
F(u x a)
r
(39)
p(x, U1, 41, a)
for all U1, U2 E U. Hence,
inf p(z, ((u)), a) & T.
(40)
01,02
By using analogous arguments, we can also show that
sup p(a, (er)t((),,, a) S, u1:02
(41)
which implies
Da Da,a 20) x,a)) S.
(42)
B.3 PROOF OF THEOREM
!
Before stating the formal proof for Theorem | we provide sketch to give an overview of the main ideas and intuition.
Why is it sufficient to only consider invertible functions J2 :4 + 3, iee, model (P xr a, u) 5(Y (2) as a Dirac delta distribution? Here, it is helpful to have a closer look at Fig. Intervening on the treatment A causes a shift the latent distribution of U, which then leads to shifted interventional distribution (a) = z)) The question is now: How can we obtain an interventional distribution that results a maximal causal query (for the upper bound)? Let consider a structural equation of the form Y 9r. (U, where is some independent noise. Hence, the "randomness" (entropy) in Y comes from both and e, however, the distribution shift only arises through U. Intuitively, the interventional distribution should be maximally shifted if Y only depends on the unobserved confounder and not on independent noise, i.e., L (U, c) = fi.a(U) (U) for some invertible function fi... One may also think about this as achieving the maximal "dependence" (mutual information) between the random variables U and Y. Note that any GTSM only restricts the dependence between and A, but not between U and
Why fix PU Jr, a) and fa without losing the ability achieve the optimum in Eq. The basic idea is as follows: Let P(Ú x. and Ju, be optimal solutions to Eq. for po- tentially different latent variable U. Then, we can define a mapping
f²,a -1 Saaa between latent spaces that transforms IP( x, a) into fixed F (U Ir a) (because both fz.o and ft. respect the observational distribution). Furthermore, we can use to push the optimal
shifted distribution PF( x) (under treatment intervention) to the latent variable (see Eq. (46) We will show that this is sufficient to obtain a distribution IP* that induces IP | x, a) and T (Y I, a) = 5(Y - ffaa(u)) and which satisfies the sensitivity constraints. For the latter property, we require the sensitivity model to be "invariant" with respect to the transformation t, for which we require our transformationiinvann
assumption (Definition 3
We proceed now with our formal proof of Theorem
Proof. Without loss of generality, we provide a proof for the upper bound Q (2r a). Our arguments work analogously for the lower bound Q (2, Furthermore, we only show the incquality
(2, < sup Q(x, P*),
(43)
P.ep.
18

---

Published as a conference paper at ICLR 2024
because the other direction ("2") holds by definition of Q a). Hence, it is enough to show the existence of a sequence full distributions (PE) with Pg E p for all & E N that satisfies lime-+20 Q(2, a, 1,) = QM (z)
To do so, we proceed in three steps: In step 1, we construct a sequence (PP) of full distributions that induce P7 (U 2; a) = PU z, a) and (Y æ, u, a) = -Y x, u, a) = 5(Y - f (u)) for every & E N. In step 2, show compatibility with the sensitivity model, i.c, P M for all l N. Finally, in step 3, we show that lime-zoc Q( a, P2) (2(
Step 1: Let IP be a full distribution on X' x ú x y for some latent space i that the solution to Eq. (T. By definition, there exists sequence (Pe)eEN with P. E .M and limy--oo Q(x, a, Pe) = TM (ar,a), Let PP 2) and Py(Y I, u, a) be corresponding induced distributions (for fixed x. a). Without loss of generality, we can assume that iP is induced by a structural causal model (Pearl 2009). so that we can write the conditional outcome distribution with a (not necessarily invertible) functional assignment Y (U)) as a point distribution Pe(Y I;, u, a) 5(Y f...,((u)), Note that we do not explicitly consider exogenous noise because we can always consider this part of the latent space U. By Eq. (3) and Eq. we can write the observed conditional outcome distribution as
P Pohs Y a) Fe, I x, and the potential outcome distribution conditioned on x as
(44)
(a) x) = -Pe(fnae()))a
(45)
We now define the sequence (EE)) First we define a distribution on on S IRdv via
(U x) = P.(St. (Y(a)) x) Peltã. -1 (Faar(t)) x)).
(46)
We then define full probability distribution IP; for the fixed x and a and all u E UI y/ 3' as
(2, a, v) = 8 (L) y) TP* (u x. a) ( (z)
(47)
Finally, we can choose family of distributions (U (,,aa) (ad))aaa so that P2(UU x)= = f PU I, Pobeaaa x) da and define
(x, u, d'.)) = (F y) (M x. a aaaaa''PPaaa Pabs d').
(48)
By definition, IP induces the fixed components Pa (U xr, a) and P. (Y x, u, a) 5( -0 (H,aau)() as well as (U a:) from Eq. (46) and the observational data distribution Potx((X A, Y).
Step 2: We now show that P2 respects the sensitivity constraints, i.e., satisfies Py E M. holds that
(U x, = Pi (SGa(U)) a))a)
(49)
(1) Poralfe -1 (Y) -(Y) a)
(50)
Pe(St (Imas(U)) x, a),
(51)
where (1) holds due to the data-compatibility assumption on fūa and (2) holds due to Eq. 44).
We now define transformation L:: ii 21 via 822 Ja.at We obtain
Dr. (P PU x, x2)))
(1) (ra x)) (((()) (2,a))
(52)
(2)
D Dr (Pr( x). P x. ar))))
(53)
(3)
< r.
(54)
where (1) holds due to Eq. (46) and Eq. (49
(2) holds due to the tranfommation-invaaiance property
of M, and (3) holds because P2 M. Hence, P. E M.
19

---

Published as a conference paper at ICLR 2024
Step 3: We show now that lime-ro Q(x, (z, a), which completes proof. By
Eq. (46). holds that
(a) .)) = [
(55)
which means that potential outcome cnniiiibttionssscniniiiioii on coincide for IP; and Pr. follows that
Q a, P7) = {
= m)) 10000 + (2)
x)) =
(56)
20

---

Published as a conference paper at ICLR 2024
C FURTHER SENSITIVITY MODELS
In the following, we list additional sensitivity models that can be written as GTSMS and thus can be used with NEURALCSA.
Continuous marginal sensitivity model (CMSM): The CMSM has been proposed by Jesson et al. (2022) and Bonvini al. (2022). It defined via
1
P(a x
< T
(57)
P(a
L, u)
for all x E X' u E u, and a E A. The CMSM can be written as CMSM with sensitivity parameter I by defining
P(u x)) P(u x.a)
P(u x. P(u x) Plala.u)P(claa)
D2 (TP(( P(u xr a))))
sup well
sup well
(58)
This directly follows by applying Bayes' theorem PPu I ax,a)=
P(a 2)
Continuous /-sensitivvty models: Motivated by the CMSM, we can define f-sensitivitty models for continuous treatments via
max (1e
P(a P(ala) P(a (2 P(u I, a) [ P(a x,u) P(a (PP(Pa x)
P(u 2, a. sr
(59)
for all X and A. By using Bayes' theorem, model as GTSM by defining
write any continuous f-sensitivity
Dr.a (P(u 2)) a)) = max {
P(u x))
P(u xx a) du,
(60)
P(u x,
[
Pu xr, a)
f
P(u x d)
(61)
PPu x))
Weighted marginal sensitivity models: Frauen al. (203bb) proposed a weighted version of the MSM, defined via
P(u x. a)
<
(62)
(') +1
P(u
x) (1 ((z, +1-11
where q(2, a) is a weighting function that incorporates domain knowledge about the strength of unobserved confounding. By using similar arguments as in the proof of Lemma [ we can write the weighted MSM as GTSM by defining
Dz.a (PUU 12) x, a)) max sup p(z, u, a), sup p(z, u, a.a) (uell
(63)
well
where
P(u x)
g(x, g(2,a)) a))
p(x, u. = 1 1 g(z,a) a)
(64)
P(u x,a)
21

---

Published as a conference paper at ICLR 2024
D QUERY AVERAGES AND DIFFERENCES
Here, we show that we can use our bounds Qu a) and Q (z. a) to obtain sharp bounds for averages and differences of causal queries. We follow established literature on causal sensitivity analysis (Dorn & Guo 2022 Dorn et al. 2022 Frauen et al. 2023b).
Averages: We interested in the sharp upper bound for the average causal query
Qm(a, = / Q Puba (2)) dr.
(65)
An example is the average potential outcome E[Y(() which can be obtained by averaging condi- tional potential outcomes via ([Y(a)) E x)) Pots (x) dr. We can obtain upper bounds via
@ū((a= = sup / Q(x, a, Pon dx / a) Poba dæ, X
(66)
and lower bounds via
Qm(()= = inf FEM / P. / Qu( a) Poba(x) dz,
(67)
whenever we can interchange the supremum/ infimum and the integral. That is, bounding the aver- aged causal query Pa reduces to averaging the bounds Q (z, a) and (z,
Differences: For two different treatment values 01, 02 E A, we are interested in the difference of causal queries
Q a1, aP PP) - Q(x, 02, IP).
(68)
An example is the conditional average treatment effect B[Y(1) x] B[Y z/). We can obtain an upper bound via
Qx 02) sup (Q(x, 01, P) Q(x, 022 P))
(69)
PEM
sup Q( 01, P)
inf Q 02, P) PEM
(70)
PEM
Qu,(z, a1) - (x, az).
(71)
Similarly, a lower bound is given by
(x, 2 Q a1) Qun(2 a2).
(72)
It has been shown that these bounds are even sharp for some sensitivity models such as the MSM, ie., attain equality (Dorn & Guo 2022).
22

---

Published as a conference paper at ICLR 2024
E TRAINING DETAILS FOR NEURALCSA
In this section, we provide details regarding the training of NEURALCSA, in particular, the Monte Carlo estimates for Stage and the full learning algorithm.
E. MONTE CARLO ESTIMATES OF STAGE LOSSES AND SENSITIVITY CONSTRAINTS
In the following, we assume that we obtained samples (ub.a)
itd NN(Oa, Ia,) and & =
(S Bernoulli(PannoC x)).
E.1.1 STAGE 2 LOSSES
Here, we provide our estimators L2 ü, {) of the Stage loss L2(1). We consider three different causal queries: (i) expectations, (ii) set probabilities, and (iii) quantiles.
Expectations correspond to setting F(PP) [[ Then, we can estimate our Stage loss via the empirical mean, i.e.,
Expectations:
C2(n, T)
eli Taoge (u))a (11 Sur, )88 (Tb) + Szi,a, al))aa))
(73)
k
Set probabilities: Here we consider queries of the form F(P) PP(X S) for some set C
We first define the log-likelihood
l n1,
c(i)
P
(F (Ewaa)) ((1 (a (an (T) + Excar
(74)
f Sogg (a))) (11 e(j) Ed fa, (T() ( + El) a0) Sara ))
which corresponds to the log-likelihood the shifted distribution (under Stage 2) at the point that is x(1) obtained from plugging the Monte Carlo samples Varaa, and S2y.a into the CNFs. We then optimize Stage by maximizing this log-like likelihood only at points in S. That is, the corresponding Monte Carlo estimator of the Stage 2 loss is
cti
12 Ao
=
T.
(7,
im1j=l
(75)
1 { (er,s)
(1- (1- -2,00, 1- (o 22.0
(T() )
+
cQ)
T0) Sayaa P
E Es.
Here, only backpropagate through the ligeliidl to obtain informative (non-zero) gradients.
Quantiles: We consider quantiles of the form F(P) = F where Fx the c.d.f. corre- sponding to and (0, For this,
the same Stage loss in Eq. (7) by
defining the set S = = (1)( where 8.51 is empirical c.d.f. corresponding to
{ f ( (x),as) (11 ) (Tl) 6i) 70) Ua
)}
j=1
1.1 SENSITIVITY CONSTRAINTS
Here, we provide our estimators D.. (1), ū) sensitivity constraint Dr... (P(U P(U x a)). We consider the three sensitivity models from the main paper: (i) MSM, (ii) f-sensitivity models, and (iii) Rosenbaum's 's sensitivity model.
MSM: We define
P (11 Er. (U))))) (T.
=
f(I, u, a, = 1-PPaa P(a
P(a x)) (76)
PÚŪ= =
23

---

Published as a conference paper at ICLR 2024
Then, our estimator for the MSM constraint is
Da,a ū))
max
max p(z,, 1i,, max p(z, u, a,) NEn
(77)
f-sessitivity models: Our estimator for the f-sensitivity constraint is
(2,a(1,
Σ (é(r, al a, a,n)), Σ M 70
2,00 a, a, )
(78)
Rosenbaum's sensitivity model: We define
[P( u2) 2) -P Ex,a)
( fo, Se, u2)
P(Ù = u1)
A(r, u1, u2, a,
P(Ù = u2)
PP( un PP) x) P (1 En. En.a) ( (U) Er P(Ù u1)
(79)
Then, our estimator
Dr,a(1.0) = max
max p(x, u1, U2 a, a)), max p(a, u1 U2 a, 17) a1 Mi
(80)
01
E.2 FULL LEARNING ALGORITHM
Our full learning algorithm for Stage and Stage 2 is shown in Algorithm| I For Stage 2, we use our Monte-Carlo estimators described in the previous section in combination with the augmented lagrangian method to incorporate the sensitivity constraints. For details regarding the augmented lagrangian method, we refer to Nocedal & Wright (006)). chapter 17.
Reusability: Using CNFs instead of unconditional normalizing flows allows to compute bounds (x, a) and Q (x, a) without the need to retrain our model for different x X' and A. In particular, we can simultaneously compute bounds for averaged queries or differences without retraining (see Appendix [). Furthermore, Stage is independent of the sensitivity model, which means that we reuse our fitted Stage CNF for different sensitivity models and sensitivity parameters I, and only need to retrain the Stage 2 CNE.
Analytical potential outcome density: Once our model is trained, we can not only compute the bounds via sampling but also the analytical form (by using the density transformation formula) of the potential outcome density that gives rise to that bound. Fig, 8, shows example. This makes it possible to perform sensitivity analysis for the potential outcome density itself. i... analyzing the "distribution shift due intervention".
24

---

Published as a conference paper at ICLR 2024
Algorithm 1:
Full learning algorithm for NEURALCSA
Input : causal query Q(z, a), GTSM M, and obs. dataset Poss
epoch numbers Rg. Ri, n2i batch size nai learning rates "o. 7ı, and
Output: learned parameters Oups and Nappt of Stage and Stage
// Stage
P (U a. < fixed probability distribution on 24 Rd Initialize ((3) and U ~ (o,,,, Ja,) for E (1,...,7o)) do
(I, a. v) + batch of size
(1) - E( (a.a.z) log (P (f (z))) (U) = y)
g(++)) optimization step C1 (A(1)) w.r.t. g(3) with learning rate "o
end
opt + g(me)
// Stage
Initialize 11 (1), and (1))
for i E {1, ..,,.................... n do
for 8 (1...,,92) n2 do (0) 171))
(x, a) + batch of size 7b
ū + (ub))s f.dd
N(Od,, Ia,)
+ (e itd. Bernoulli(Pooer 2))
(1) + ---Dooon
[(1, M, c- L2(n, ū) E(e,)) )m.88z((1)) + 2 (1))
Mit1 optimization step of L((b') 200), (1)) w.r.t. ( with learning rate "
(1)
end
rj(it)) -- Tima
An.a (i+1) - max 00 110 a1, (1ttt))
((i+1) t (1)
end
Nopt f- (m()
E.3 FURTHER DISCUSSION OF OUR LEARNING ALGORITHM
Non-neural alternatives: Note that our two-stage procedure is agnostic to the estimators used, which, in principle, allows for non-neural instantiations. However, we believe that our neural in- stantiation (NEURALCSA)) offers several advantages over possible non-neural alternatives:
1. Solving Stage 1: Conditional normalizing flows (CNFs) are natural choice for Stage because they are designed to learn an invertible function fi..: U Y that satisfies Pots xx (ft (U) x a). In particular, CNFs allow for inverting fz ana- lytically, which enables tractable optimization of the log- ligeliied (see Appendix F). In principle, f² could also be obtained by estimating the conditional c.d.f. F(y a) us- ing some arbitrary estimator and then leveraging the inverse transform sampling theorem, which states that we can choose fe = f-1 I, x a) whenever we fix Pp (U xs a) to be uniform. However, this approach only works for me-dimensional Y and requires inverting the estimated F(y x; a) numerically.
2. Solving Stage 2: Stage requires to optimize the causal query F (P(ft, | x)) over latent distribution P(U æ)), where f is learned in Stage 1. In NEURALCSA, we achieve this by fitting a second CNF in the latent space U. which we then concatenate with the CNF from Stage to backpropagate through both CNFs. Standard density estimators not applicable in Stage because we fit a density in the latent space to optimize a causal query that is dependent Stage 1, and not standard likeliood.o While there may exist
25

---

Published as a conference paper at ICLR 2024
non-neural alternatives to solve the optimization in Stage 2, this goes beyond the scope of our paper, and we leave this for future work.
3. Analytical interventional density: Using NEs in both Stages and enables us to obtain analytical interventional once NeuralCSA is fitted. Hence, we can perform sensitivity analysis for the whole interventional density (see Fig. andd without the need for Monte Carlo approximations through sampling.
4. Universal density approximation: CNFs are universal density approximators, which means that we can account for complex (e.g., multi-modal, skewed) observational distributions.
Complexity of CSA compared closed-form solutions: Stage requires fitting CNF to estimate the observational distribution Poss x. This step is also necessary for closed- form bounds (e.g., for the MSM), where such closed-form bounds depend on the observational distribution (Frauen et al. 2023b). This renders the complexity in terms of implementation choices equivalent to Stage I.
In Stage 2, closed-form solutions under the MSM allow for computing bounds directly using the observational distribution, NEURALCSA fits an additional NF that is training using Algorithm Hence, additional hyperparameters are the hyperparameters of the second NF as well as the learning rates of the augmented Lagrangian method (used incorporate the sensitivity constraint). Hence, we recommend using NEURALCSA A method for causal sensitivity analysis whenever bounds are not analytically tractable. our experiments, we observed that the training of NeuralCSA was very stable, indicated by a low variance over different runs (see, e.g., Fig. 5
Existence of a global optimum: A sufficient condition for the existence of global solution in Eq. 44) the continuity of the objective/causal query well the compactness of the constraint set. Continuity holds for many common causal queries such as the expectation. The compactness of the constraint depends the properties functional Dz,a. i.e., the choice of the sensitivity model. The existence of global solutions has been shown for many sensitivity models from the literature, e.g., MSM (Dorn et 2022) and f-sensitivity models (Jin et al. 2022). Note that, in Theorem []] we do not assume the existence of global solution. In principle, our two-stage procedure is valid even a global solution to Eq. (5) does not exist. this case, we can apply our Stage 2 learning algorithm (Algorithm I until convergence and obtain an approximation of the desired bound, even it is not contained in the constraint set.
26

---

Published as a conference paper at ICLR 2024
F DETAILS ON IMPLEMENTATION AND HYPERPARAMETIE TUNING
Stage 1 CNF: We use a conditional normalizing flows (CNF) (Winkler et al. 2019) for stage of NEURALCSA. Normalizing flows (NFs) model a distribution P(Y) of target variable by transforming a simple base distribution P(U) (e.g., standard normal) of a latent variable U through an invertible transformation Y [U(),, where denotes parameters (Rezende & Mohamed, 2015
In order to estimate the conditional distributions P(Y Ir a), CNFs define the parameters 0 as an output of a hyper network = go a) with learnable parameters A. The conditional log-likelihood can be written analytically
( log (P(UU- ( (w)) log det (ay ( (81)
log (P( (U) v))
where (+) follows from the change-of-variables theorem for invertible transformations.
Stage CNF: As Stage I, we use CNF that transforms U = (Ū),, where nj = [, a) with learnable parameters y,. The conditional log-likelihood can be expressed analytically via
log (o = =)) = log (P - F ((ieao(()))) (P= (0(2,0) (w))) + log det du ((u
(82)
In our implementation, we use autoregressive neural spline flows. That is, we model the invertible transformation via spline flow deseribed in inDoaatabadi al. (2020). We autoregressive neural network for the hypernetwork (x, m, a) with 2 hidden layers, ReLU activation functions, and linear output. For training, we use the Adam optimizer (Kingma & Ba 2015).
Propensity scores: The estimation of the propensity scores P(a x) is a standard binary clas- sification problem. We use feed-forward neural networks with hidden layers, ReLU activation functions, and softmax output. For training, we minimize the standard cross-entropy loss by using the Adam optimizer (Kingma Ba 2015).
Hyperparameter tuning: We perform hyperparameter tuning for our propensity score models and Stage CNFs. The tunable parameters and search ranges are shown in Table Then, we use the same optimally trained propensity score models and Stage CNFs networks for all Stage 2 models and closed-form solutions (in Fig. 5 For the Stage CNFs, we choose hyperparameters that lead to stable convergence of Alg. while ensuring that the sensitivity constraints are satisfied. For reproducibility purposes, we report the selected hyperparameters as .yam files
Table 4: Hyperparameter tuning details.
MODEL
TUNABLE PARAMETERS
SEARCH RANGE
Stage 1 CNF
Epochs
50
Batch size
32, 64, 128
Learning rate
0.0005, 0.001, 0.005
Hidden layer size (hyper network)
5, 10, 20, 30
Number of spline bins
2, 4,
Propensity network
Epochs
30
Batch size
32, 64, 128
Learning rate
0.0005, 0.001, 0.005
Hidden layer size
5, 10, 20, 30
Dropout probability
0, 0.1
Code is available http://ghhhaaccmm/mmmem/aannaaaa
27

---

Published as a conference paper at ICLR 2024
G DETAILS REGARDING DATASETS AND EXPERIMENTS
We provide details regarding the datasets we use in our experimental evaluation in Sec.
G.1
SYNTHETIC DATA
Binary treatment: We simulate an observed confounder X Uniform(-1, and define the observed propensity score as (2) = (A x) = 0.25 + 0.5 a (3x), where o+)) denotes the sigmoid function. Then, simulate unobserved confounder
X x Bemoulli (p= (T (-1))(())))))))))))))())))))(( +11
(83)
r T+1 1
and binary treatment
A A=1|X = x. U Bemoulli (p (p= un uT(x)) (I, a) (1 u)T(2)s ())
(84)
where stt (z, a) [I-1 ()(())+++ r and (I. (1-1))(()+++*
Finally, simulate continuous outcome
Y 7(22 (2A-1)44 1)X + (2A 1) sin 1)X) 2(2U + + (85) where 1).
The data-generating process is constructed so that P(A=1(s..) us (z, a) + -1-11) u)s (z, a) or, =(x)
cquivalently, that OR(z, u) ul' (1 =)]=1. Hence, the full distribution follows an MSM with
sensitivity parameter Furthermore, by Eq. (83). we have
P(A = x; PU = 1 x) + P(4 = x, OU = 0 x) = xxx = Pots (4 = so that P induces Pabx (A = x.)
(86)
Continuous treatment: We simulate observed confounder X Uniform[-1, 1] and indepen- dently a binary unobserved confounder ~ Bernoullip = 0.5). Then, we simulate a continuous treatment via
A = Beta(a, 3) with a
7( 0.5),
(87)
where y is a parameter that controls the strength of unobserved confounding. In our experiments, we chose 7 = 2. Finally, we simulate an outcome via
Y A + exp( - AA) - 0.5(U - 0.5) + (0.5X + +
(88)
where 8 ~ ((,,)) 1). Note that the data-generating process does nol follow a (continuous) MSM.
Oracle sensitivity parameters r": We obtain Oracle sensitivity parameters [" a) for each sample with X = x and A = by simulating from our previously defined data-generating process estimate the densities P(u xr a) and IP(u x) and subsequently solve for (xx a) in the GTSM equations from Lemma 3 By definition, [' a) is the smallest sensitivity parameter such that the corresponding sensitivity model guaranteed produce bounds that cover the ground- -ruth causal query. For binary treatments, it holds (ir, a) = r* i.e., the oracle sensitivity parameter does not depend on æ and a. For continuous treatments, we choose = (a) = [ (x, ())P(r)))))))))))))))))))))))) da in Fig. 6
G.2 SEMI-SYNTHETIC DATA
We obtain covariates X and treatments from MIMIC-III (Johnson et al. 2016) as described the paragraph regarding real-world data below. Then we learn the observed propensity score *(x) P(A 1 x) using feed-forward neural network with three hidden layers and ReLU activation function. We simulate uniform unobserved confounder U ~ 2(00, 1]. Then, we define weight
w u(XUU)= = 22 #(X)
(Y 2U - -7))+ +))++
(89)
< *(X) / (2 *(X)
1)
2U #(X)
(90)
28

---

Published as a conference paper at ICLR 2024
and simulate synthetic binary treatments via
A = 1 = z, = u~ Bernoulli (p= w(x, u)+(x)).
(91)
Here, a parameter controlling the strength of unobserved confounding, which we set to 0.25.
The data-generating process is constructed in a way such that the full propensity P(A X x, U = u) induces the (estimated) observed propensity ø from the real-world data. We then simulate synthetic outcomes via
de Σ
+ +v) +
Y Y= (2A
X
(92)
dx +1
where E ~ (0,00.). In our experiments, we use 90% of the data for training and validation, and 10% of the data for evaluating test performance. From our test set, filter all samples that satisfy either P(A = 1 x) < 0.3 or TPA P x) > 0.7. This is because these samples associated with large empirical uncertainty (low sample sizes). In experiments, only demonstrate the validity of our NEURALCSA A bounds in settings with low empirical uncertainty.
Oracle sensitivity parameters -": Similar our fully synthetic experiments, first obtain the Oracle sensitivity parameters n (1c, a) for each test sample with confounders I and treatment a. We then take the median overall [' [z, a) of the test sample. By definition, NEURALCSA should then cover at least 50% of all test query values (see Table 2).
G.3 REAL-WORLD DATA
We use the MIMIC-III dataset Johnson et al. (2016), which includes electronic health records from patients admitted to intensive care units. We use a preprocessing pipeline (Wang et al.) (2200) to extract patient trajectories with 8 hourly recorded patient characteristics (heart rate, sodium blood pressure, glucose, hematocrit, respiratory rate, age, gender) and binary treatment indicator (me- chanical ventilation). We then sample random time points for each patient trajectory and define the covariates X E R as the past patient characteristics averaged over the previous 10 hours. Our treatment A E {0, }} is an indicator of whether mechanical ventilation was done in the subsequent 10-hour time. Finally, we consider the final heart rate and blood pressure averaged over hours as outcomes. After removing patients with missing values and outliers (defined by covariate values smaller than the corresponding 1th percentile or larger than the corresponding 99.9th percentile), we obtain a dataset of size 14719 patients. We split the data into train (80%), val (10%). and test (10%).
29

---

Published as a conference paper at ICLR 2024
H ADDITIONAL EXPERIMENTS
H.1 ADDITIONAL TREATMENT COMBINATIONS FOR SYNTHETIC DATA
Here, we provide results for additional treatment values a {0.1, 0.9) for the synthetic experiments with continuous treatment in Sec. O Fig. 9 shows the results for our experiment where we compare the NEURALCSA bounds under the MSM with (optimal) closed-form solutions. Fig. TO shows the results of our experiment where compare the bounds of different sensitivity models. The results are consistent with our observations from the main paper and show the validity the bounds obtained by NEURALCSA,
2
Orack
Orccle p(xa)
Ours -=
Cur r=2
CF C
CF =
=
CF =
CF CF
Ours 10
Our 10
1= 10
CF -= 10
-1.0
0.5
-1.0 -0.5
1.0
X
Figure 9: Validating the correctness of NEURALCSA (ours) by comparing with optimal closed-form solutions (CF) for the MSM in the synthetic continuous treatment setting. Left: 0.1. Right: 0.9. Reported: mean + standard deviation over 5 runs.
1.0
1.0
0.5
0.0
0.5
Oradi pix,
-05
Cracie (rx,a)
NSM r* =.15
MSM 3315
-10
-10
=
-1.5
TV 0.29
HE =0.12
HE r* 0.12
x 0.89
x r² 0.89
-20
-0.5
1.0
-1.0
-0.5
0.0
0.5
1.0
X
X
Figure 10: Conftrming the validity of NEURALCSA bounds for various sensitivity models in the synthetic continuous treatment setting. Left: 0.1. Right: a 0.9. Reported: mean + standard deviation over runs.
H.2 DENSITIES FOR LOWER BOUNDS ON REAL- DATA
Here, we provide the distribution shifts for our real-world case study (Sec. 6). but for the lower bounds instead of the upper. The results are shown in Fig. I1 contrast to the shifts for the upper bounds, increasing I' leads to a distribution shift away from the direction of the danger area, i.e., high heart rate and blood pressure.
H.3 ADDITIONAL SEMI-SYNTHETE EXPERIMENT
We provide additional experimental results using semi-synthetic dataset based on the IHPD data Hill (20IT). IHDP is a randomized dataset with information on premature infants. It was originally designed to estimate the effect of home visits from specialist doctors cognitive test scores. For
30

---

Published as a conference paper at ICLR 2024
003337
4.02087
5.02457
102
002223
200
5022300
100
2032500
002372
60
4.021/00
2002786
D01223
2002271
5020114
2030726
23
120
210
120
130
00
200
1.20
320
120
50
205
110
222
230
Figure 11: Contour plots of 2D densities obtained by NEURALCSA under an MSM. Here, we aim learn lower bound of the causal query P(Y 115, Y2(1) 90 X = zo) for a test patient Left: Stage 1/ observational distribution. Center: Stage 2, r = 2. Right: Stage 2, r = 4.
experiment, extract d, 7 covariates X (birthweight, child's head cireumference, number of weeks pre-term that the child was born, birth order, neo-natal health index, the mother's age, and the of the child) of n 985 infants. Then, and define an observational propensity score as (z) Poss (A x) 0.25 0.5 E X2). Then, similar as in Appendix G.2 introduce unobserved confounding by generating synthetic treatments via
x=1|x=z,
Bernoulli (p w(x, u))(x))
(93)
where w(r, u) is defined in Eq. (89) and ~ u10, 1]. We then generate synthetic outcomes via
== (2A 12A-1)
1 dr +1
Σ Xj +U
+
(94)
where E 1).
In our experiments. we split the data into train (80%) and test set (20%). We verify the validity of NEURALCSA bounds for CATE analogous our experiments using the MIMIC (Sec. 6 For each sensitivity model (MSM, TV, HE, RB). we obtain the smallest oracle sensitivity parameter I' that guarantees coverage (i.e., satisfies the respective sensitivity constraint) for 50% of the test samples. Then, we plot the coverage and median interval length of the NEURAICSA bounds over the set. The results are in Table 5 The results confirm the validity of NEURALCSA.
Table 5: Results for IHDP-based semi-synthetic data.
Sensitivity model
Coverage
Interval length
MSM 1* 3.25
0.92 0.03
1.63 + 0.05
TV P 0.31
0.71 + 0.28
1.23 + 0.54
HE [ 0..11
0.70 = 0.14
1.10 + 0.21
RB [* -9.62
0.56 + 0.15
0.95 + 0.29
Reported: mean + standard deviation (5 runs).
31

---

Published as a conference paper at ICLR 2024
I DISCUSSION ON LIMITATIONS AND FUTURE WORK
Limitations: NEURALCSA is a versatile framework that can approximate the bounds of causal effects in various settings. However, there are few settings where (optimal) closed-form bounds exist (e.g., CATE for binary treatments under the MSM), which should be preferred when available. Instead, NEURALCSA offers unified framework for causal sensitivity analysis under various sensi- tivity models, treatment types, and causal queries, and can be applied in settings where closed-form solutions have not been derived or do not exist (Table I
Our research hints at the broad applicability of NEURAL beyond the three sensi- tivity models that we discussed above (see also Appendix For future work, it would be interesting to conduct a comprehensive comparison of sensitivity models and provide practical recommenda- tions for their usage. Future work may further consider incorporating techniques from semiparamet- ric statistical theory to obtain estimation guarantees, robustness properties, and confidence intervals. Finally, we only provided identifiability results that hold in the limit of infinite data. would be interesting provide rigorous empirical uncertainty quantification for NEURALCSA, c.g., via Bayesian approach. While in principle the bootstrapping approach from (Jesson et al. 2022) could be applied in our setting, this could be computationally infeasible for large datasets.
Future work:
32


Reviews:
[{"review_id": "qYUBbQR2qS", "reviewer": "Reviewer_i97n", "summary": "The authors provide a framework for generalized causal sensitivity analysis, an approach that subsumes three previous methods and provides additional advantages.", "questions": "Are alternative (non-neural) implementations of the GTSM possible? What advantages does the neural implementation provides over alternatives? How does the complexity (e.g., number of hyper-parameters and other implementation choices) of NeuralCSA compare to MSM?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The key advantage of the authors' approach (according to the authors) is that it is much more widely applicable than any other single approach (which typically focus on a specific sensitivity model, treatment type, and causal query).\n\nThe paper is well-written, and it provides substantial background about existing methods for causal sensitivity analysis.\n\nThe experiments appear consistent with the existing literature, well-motivated, and useful.", "weaknesses": "The generality of the approach appears to come at a substantial cost in terms of complexity (with a corresponding potential for unexpected sources of error, bias, or misspecification). The single advantage over MSM appears to be allowing causal queries with multiple outcomes. \n\nIt is unclear the extent to which alternative (non-neural) implementations of the GTSM are possible. The paper would be improved by clearly describing what advantages the neural implementation provides over alternatives.\n\nIt seems somewhat odd to cite D'Amour 2019 (an excellent paper about a very specific topic) for the idea that \"unobserved confounding often renders causal inference challenging.\" That has been known for more 50 years, going back at least to Reichenbach's common cause principle.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wguZ6WjIAF", "reviewer": "Reviewer_9LA1", "summary": "The paper proposes NeuralCSA, a framework for performing causal sensitivity analysis, i.e., partially identification of a causal functional under assumptions on the unmeasured confounders. They propose a 2-stage training procedure, modeling the latent distributions using normalizing flows.", "questions": "See weaknesses section.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper overall is well-written and easy to understand. I found the motivation and setup to be clear. I also appreciated comparisons to existing sensitivity models.\n- The GTSM framework subsumes many of the existing sensitivity models and thus is more generally applicable. In principle, the framework also applies to arbitrary functionals (e.g., quantiles) of the interventional outcome distributions.\n- The clarity of the two-stage procedure can be improved (see Weaknesses section), but overall, the procedure is simple and easy to follow. It is also nicely motivated using Theorem 1.", "weaknesses": "- I found Sec 5.1 and 5.2 difficult to read and I think clarity can be improved. What confused me initially was that you suggest fixing $P^*(U|x, a)$ but then the $\\sup$ in Eq. 5 is also over the distributions $p(u|x, A)$. Reading it further, the sup is only for $A \\neq a$ but I think clarifying that you only fix for the treatment $a$ that enters into $Q$ would be useful. Maybe this is obvious, but it will still make it easier to understand what is being optimized over in the $\\sup$.\n- It would also be nice to have some intuition of the proof of Theorem 1. Also, the invertible function $f^*$ would depend on the fixed $P^*$. Does certain distributions $P^*$ make it easier to determine $f^*$. In practice, how should you determine which $P^*$ to fix?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "M7wJb8Fbk9", "reviewer": "Reviewer_MpUG", "summary": "This paper proposes a generalized sensitivity analysis framework that is compatible with many different sensitivity models, including the marginal sensitivity model, f-sensitivity model, and Rosenbaum's sensitivity model. The framework is suitable for different treatment types and different causal queries. The authors propose to learn the latent distribution shift with two separately trained conditional normalizing flows.", "questions": "- Regarding the color coding in equation (4), does it indicate the parameters for the optimization problem? Does the right supremum also maximize over $P(U|x,a)$?\n- I'm not fully understand the two-stage procedure. Could you to provide a more detailed explanation about replace the right supremum with \n fixed of $\\mathbb{P}^*(U|x,a)$ and $\\mathbb{P}^*(Y|x,u,a)$?\n- In relation to the optimization problem presented in equation (5), are there specific constraints placed on the functional $\\mathcal{D}_{x,a}$ to ensure that the global optimal can be achieved?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The concise summary of sensitivity models enhances the paper's readability and flow.\n- The authors introduce a novel learning strategy to model the latent distribution.\n- Experiments with both synthetic and real-world data are used to demonstrate the validity and effectiveness of the proposed method.", "weaknesses": "- It is not very obvious how does the bounds the proposed framework compares with some existing works such as GMSM. \n- The section 5.1 might be a little hard to follow. Please find some questions I have below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XWW9ODWetS", "reviewer": "Reviewer_ijNf", "summary": "This paper proposes a neural framework for generalized causal sensitivity analysis that aims to achieve lower and upper bounds for some causal queries when exact identification is not possible due to unobserved confounding. Specifically, the framework leverages two normalizing flows to encode the latent distributions that are compatible with the observed ones and optimize the causal quantities of interest and can be used to approximate previous sensitivity analysis models.", "questions": "I noticed that the authors apply the augmented Lagrangian method to incorporate the sensitivity constraints in the optimization process. I wonder to what extent could the constraints be satisfied since the constraint now becomes a soft one that may well be violated. It is possible that the effective constraint parameter $\\Gamma$ achieved by the optimization significantly deviates from the intended value, which may make the final bounds less useful. Additionally, I am concerned about the stability of training the normalizing flows, as instability in this aspect could further exacerbate the situation.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The writing is clear, and the paper is easy to follow.\n2. The proposed neural network-based causal sensitivity analysis method is effective and general, which is compatible with previous studies like MSM and f-sensitivity models and can be easily extended to other models by modifying the constraints that specify the strength of unobserved confounding.", "weaknesses": "The experiments, to some extent, appear to be lacking in comprehensiveness. The semi-synthetic datasets utilized in this study are exclusively derived from the MIMIC-III dataset. The findings derived from this single source may not offer an adequate illustration of the model's performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)