Disentangling Causal Effects from Sets of Interventions in the Presence of Unobserved Confounders
Authors: Olivier Jeunen
Ciarán M. Lee
Rishabh Mehrotra
Mounia Lalmas
Venue: NeurIPS 2022 Accept (2022)
Keywords: Causal Inference
Additive Noise Models
Joint Interventions

Abstract: The ability to answer causal questions is crucial in many domains, as causal inference allows one to understand the impact of interventions. In many applications, only a single intervention is possible at a given time. However, in some important areas, multiple interventions are concurrently applied. Disentangling the effects of single interventions from jointly applied interventions is a challenging task---especially as simultaneously applied interventions can interact. This problem is made harder still by unobserved confounders, which influence both treatments and outcome. We address this challenge by aiming to learn the  effect of a single-intervention  from  both observational data and sets of interventions. We prove that this is not generally possible, but provide identification proofs demonstrating that it can be achieved under non-linear continuous structural causal models with additive, multivariate Gaussian noise---even when unobserved confounders are present. Importantly, we show how to incorporate observed covariates and learn heterogeneous treatment effects. Based on the identifiability proofs, we provide an algorithm that learns the causal model parameters by pooling data from different regimes and jointly maximising the combined likelihood. The effectiveness of our method is empirically demonstrated on both synthetic and real-world data.

Disentangling Causal Effects from Sets of Interventions in the Presence of Unobserved
Confounders
Olivier Jeunen" Amazon Edinburgh, UK
Ciarán M. Gilligan-Lee Spotify & UCL London, UK
Rishabh Mehrotra" Sharechat London, UK
Mounia Lalmas Spotify London, UK
Abstract
The ability to answer causal questions is crucial in many domains, as causal infer- ence allows one to understand the impact of interventions. In many applications, only single intervention is possible at given time. However, in some important areas, multiple interventions are concurrently applied. Disentangling the effects of single interventions from jointly applied interventions is a challenging task- especially as simultaneously applied interventions can interact. This problem made harder still by unobserved confounders, which influence both treatments and outcome. We address this challenge by aiming to learn the effect of a single- intervention from both observational data and sets of interventions. We prove that this is not generally possible, but provide identification proofs demonstrating that it can be achieved under non-linear continuous structural causal models with additive, multivariate Gaussian noise even when unobserved confounders are present. Importantly,
show how to incorporate observed covariates and learn heterogeneous treatment effects. Based on the identifiability proofs, we provide an algorithm that learns the causal model parameters by pooling data from different regimes and jointly maximizing the combined likelihood. The effectiveness our method is empirically demonstrated both synthetic and real-world data.
Introduction
The ability to answer causal questions crucial in science, medicine, economics, and beyond, [Gilligan-Lee, 2020] for high-level overview. This because causal inference allows one to understand the impact of interventions. 2 In many applications, only single intervention is possible at given time, or interventions are applied one after another sequential manner. However, some important areas, multiple interventions are concurrently applied. For instance, in medicine, patients that possess many commodities may have be simultaneously treated with multiple prescriptions; in computational advertising, people may be targeted by multiple concurrent campaigns: and in dietetics, the nutritional content of meals can considered joint intervention from which we wish learn the effects of individual nutritional components.
Disentangling the effects of single interventions from jointly applied interventions is challenging task task-especially as simultaneously applied interventions can interact, leading to consequences not seen when considering single interventions separately. This problem is made harder still by the possi- ble presence of unobserved confounders, which influence both treatments and outcome. This paper addresses this challenge, by aiming learn the effect of single-intervention from both observational
"Work done while author Spotify. "Causal inference also allows one to ask and answer counterfactual questions, see [Perov et al., 2020] and [Vlontzos et al., 2022].
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

data and sets of interventions. We prove that this is not generally possible, but provide identifi- cation proofs demonstrating can be achieved in certain classes of non-linear continuous causal models with additive multivariate Gaussian noise-even in the presence of unobserved confounders. This reasonably weak additive noise assumption prevalent the causal inference and discovery literature [Rolland et al., 2022, Saengkyongam and Silva, 2020, Kilbertus et al., 2020]. Importantly, we show how to incorporate observed covariates, which can be high-dimensional, and hence learn heterogeneous treatment effects for single-interventions. Our main contributions are:
1. A proof that without restrictions on the causal model, intervettttinnnnoon effects cannot be identified from observations and
($3.1.3.2)
2. Proofs that single-interee interventions can be identified from observations and joint-interventions when the causal model belongs to certain (but not all) classes of non-linear continuous structural causal models with additive, multivariate Gaussian noise. ($3.2,3.3)
3. An algorithm that learns the parameters of the proposed causal model and disentangles single interventions from joint interventions. ($4)
4. An empirical validation of our method on both synthetic and real data.² 3 ($5)
2 Related Work
Disentangling multiple concurrent interventions: [Parbhoo et al., 2021] study the question of disentangling multiple, simultaneously applied interventions from observational data. They propose a specially designed neural network for the problem and show good empirical performance on some datasets. However, they do not address the formal identification problem, nor do they address possible presence of unobserved confounders. By contrast our work derives the conditions under which identi- fiability holds. We moreover propose an algorithm that can disentangle multiple interventions even in the presence of unobserved confounders as long as both observational and interventional data is available. Related work by [Parbhoo et al., 2020] investigated the intervention ceanttionnddieetnnlenenee problem from a reinforcement learning perspective, where each intervention combination constitutes different action that reinforcement learning agent can take. Unlike this approach, our work explicitly focuses on modelling the interactions between interventions to learn their individual ef- fects. Closer to our work is [Saengkyongam and Silva, 2020]. who investigate identifiability of joint effects from observations and angle-interventionn data. They prove this is not generally possible, but provide identification proof for non-linear causal models with additive Gaussian noise. Our work addresses a complementary question; we want to learn the effect of a ingelliittrvention from obser- vational data and sets of interventions. Additionally, another difference between our work and that of [Saengkyongam and Silva, 2020] is that they do not consider identification of individual-level causal effects given observed covariates. In precursor the work by [Saengkyongam and Silva, 2020), [Nandy et al., 2017] developed method to estimate the effect of joint interventions from obser- vational data when the causal structure is unknown. This approach assumed linear causal mod- els with Gaussian noise, and only proved identifiability in this case under a sparsity constraint. However, like [Saengkyongam and Silva, 2020], our result does not need the linearity assump- tion, and no sparsity constraints are required in our identification proof. Finally, others includ- ing [Schwab et al., 2020. Egami and Imai, 2018, Lopez and Gutman, 2017, Ghassami et al., 2021] explored how to estimate causal effects of single categorical-, or continuous-valuee treatment, where different intervention values can produce different outcomes. Unlike our work, they do not consider multiple concurrent interventions that can interact.
Combining observations and interventions:
[Bareinboim and Pearl, 2016] have investigated non- parametric identifiability of causal effects using both observational and interventional data, in a paradigm they call "data fusion." More general results were studied by [Lee al., 2020], who provided necessary and sufficient graphical conditions for identifying causal effects from arbitrary combinations of observations and interventions. Recent work in [Correa al., 2021] explored identification of counterfactual as opposed to interventional distributions from combinations of observational and interventional data. Finally, [[lse al., 2021] investigated the most efficient way to combine observational and interventional data to estimate causal effects. They demonstrated they could significantly reduce the number of interventional samples required to achieve a certain fit when
The code reproduce results is available at gittth..cooodiiieeeeennddettinnn 200r

---

adding sufficient observational training samples. However, they only prove their method theoretically in the Aheer-Gaassian case. In the non-linear case, they parameterise their model using normalising flows and demonstrate their method empirically. They only consider estimating single-interventions, and do not deal with multiple, interacting interventions.
Additive noise models: While certain causal quantities may not be generally identifiable from obser- vational and interventional data, by imposing restrictions on the structural functions underlying causal models, one can obtain semi-parametric identifiability results. One of the most common weak restric- tions used the causal inference community are additive noise models (ANMs), first studied in the context of causal discovery by [Hoyer et al., 2009]-am still widely used today [Rolland et al., 2022]. ANMs limit the form of the structural equations to be additive with respect to latent noise variables- but allow nonlinear interactions between causes. [Janzing et al., 2009] used ANMs to devise method for inferring latent confounder between two observed variables. This is otherwise not possible without additional assumptions on the underlying causal model. See [Lee and Spekkens, 2017], [Lee et al., 2019], and [Dhir and Lee, 2020] for an extension of this approach beyond ANMs. ANMs have also been employed by [Kilbertus al., 2020] to investigate the sensitivity of counterfactual notions of fairness to the presence of unobserved confounding. Our work proves that in certain classes of ANMs, the effect of a single-intervention can be identified from observational data and sets of interventions even in the presence of unobserved confounders. Moreover, we show how to incor- porate observed covariates in these ANMs to learn the heterogeneous effects of inglllintervenntons.
3 Model Identifiability
Identifiability is a fundamental concept in parametric statisties that relates to the quantities that can, or can not, be learned from data [Rothenberg, 1971]. An estimand is said to be identifiable from data if it is theoretically possible to learn this estimand, given infinite samples. If two causal models coincide on said data then they must coincide on the value of the estimand in question. Hence if one finds two causal models which agree on said data, but disagree on the estimand, then the estimand is not identifiable unless further restrictions are imposed. In this section, we provide identification proof's for single-variable variable interventional effects from observational data and joint interventions, for several model classes. Our theoretical analysis provides insights into the fundamental limitations of causal inference-aan the assumptions that are required for identification.
Problem Definition. We adopt the Structural Causal Model (SCM) framework as introduced by (Pearl, 2009]. An SCM M defined by (IC, X, Y), U, f, Pv), where (C, X.Y} are endogenous variables separated into covariates C, treatments X, and the outcome Y, U are exogenous variables (possibly confounders), f are structural equations, and Py defines a joint probability distribution over the exogenous variables.
The SCM M also induces a causal graph-where vertices represent endogenous variables, and edges represent structural equations. Vertices with outgoing edges to an endogenous variable X; are denoted as the parent set of this variable, or PA(X.). Typically, the observed covariates C' causally influence the treatments as well as the outcome, and are part of this set. Every endogenous variable X; (including Y' is then function of its parents in the graph PA(X;) and a latent noise term U, denoting the influence of factors external to the model:
X₁ = fo (PA( U.).
(1)
In Markovian SCMs, these latent noise terms are all mutually independent. However, in general, distinct noise terms can be correlated according to some global distribution Py. In this case, such correlation is due to the presence of unobserved confounders.
An intervention on variable X, is denoted by do(X, I;). and it corresponds replacing its structural equation with a constant, or removing all incoming edges in the causal graph. The core question we wish to answer in this work, is under which conditions the treatment effect of a single intervention can be disentangled from joint interventions and observational data. That is, given samples from the data regimes that induce
E|y|X; 2;, X, I, C cl, and E|y do(X, r;, X; 23)) C dl when can we learn conditional average causal effects:
E|y]do(X; = x;),x, = .j, C c), or | = do = 2z))
= C=
3

---

Table 2: Interventional distributions X1 under SCMs M and M'
X2 (d(()))())
Y = 0
Y =
(M,(Y, X2
Y=0
Y
do(X1 =
X2 0
1
0
do = 0)
X2 = 0
1- p
0
X2 =
0
X2 =
0
X2 =
1 1-
0
X2 =0
1-p
0
do do(X1 1)
do(X1= = 1) X2
X2 1
p
0
p
In what follows, we first show that this quantity is not identifiable without restrictions on the causal model-a proof by counterexample. We then go to prove, again by counterexample, that simply assuming ANMs without restrictions on the structure of the causal graph- the core assumption made by [Saengkyongam and Silva, 2020] is not enough for this complementary research question. Finally, we prove identifiability of the treatment effect for ANMs without causal interactions between treatments. Note that this latter case does not mean treatments independent: treatments can be influenced by observed covariates and unobserved confounders, and their interactions on the outcome defined by the unrestricted function fy-
3.1 Unidentifiability for general SCMs
Table 1: SCMs for 3.1
For simplicity, but without loss of generality, we con- sider two treatment variables {X1,X2] X2) and no covari-
M
M'
ates. We will show that two different SCMs M and
X1 = U1
X1 U1
M' can yield identical observational distributions, as well as joint interventional distributions. They will also agree on the singlevvrriblle interventional distribution
X₂ X1U2
X2 U2
Y = XiXoUX
Y X1X2Uy
for treatment X2, but disagree on the single-variable effect of treatment X1. As such, given data from sets of interventions, this example shows the treatment effect of single-varien variablee interventions is not
generally identifiable. Our SCMs are defined in Table I, where U1 = U22 = Uy ~ Bernoulli(p). The observational, joint, and single-variable interventional distributions M'; we defer them to the supplemental material. The interventional distribution on X₁ differs for M and M', shown in Table 2. This counterexample shows that single-variable interventional effects are not identifiable for general SCMs, given observational data and joint interventions.
X2 are identical under M and
3.2 Unidentifiability for general ANMs
An Additive Noise Model is an SCM where the influence of the latent noise variables is restricted to be additive the structural equations. That is, Eq. 1 restricted to the form:
X; = (PA((X)) + U,.
(2)
Following (Saengkyongam and Silva, 2020], we additionally assume the noise distribution to a zero-centered Gaussian with an arbitrary covariance matrix: Pú ~ (O E). Table 3 defines two SCMs that satisfy these assumptions. M and M' yicld identical observational, joint interventional, and inteeeeinterveniionnll distributions on X2. However-they disagree intervening on X1. An intuitive underlying reason for this, is that we can identify the expression X[X2|X, = x1 = f2(21) ) + EEUU[Xx = 21]. but have no tools to disentangle the effects coming from the structural equation, f2 (1). from those stemming from the additive noise, E [X, = x1]. xz)] ) fz(11) is not identifiable, proving that general ANMs with unrestricted causal structures are insufficient for disentangling treatment effects. In this example, and in general SCMs of this structure, joint interventions can be disentangled for the consequence treatment X2, but not for the causing treatment X₁:
the causal effect of
As result, E[X,, |[Xg||o([X;
Theorem 1 (Identifiability of disentangled conditional average treatment effects in additive noise models with a causal dependency between treatments).
Let M ((C, X, Yj.U U. U.F.P.P.. Pu) be an SCM, where
X; = (((C++U)
X, = (C, + U,.
Y
and Pt (O, E). The estimand (oo)) (X,): ;) C) is identifiable from the conjunction of two data regimes: (1) the observational distribution, and (2) the joint interventional distribution on (X,, X,).

---

Table 3: SCMs for 3.2, where (U,,U2,
U2 Uy)M (O,, and (U1 U2 Uy)M Mr ~ NN(,
M
M
[ 1 17
0
11
X1 = U1
X1 = U1
where Σ
1
and
0 0
X2 X1+U2
X2 -X + U2
1 1 1
1
0
1
X1122 Uy
Y = =X1X2 + Uy
We prove this by showing that the structural equations are identifiable from the joint interventional whereas the necessary (co-)variancee are identifiable from observational data. Formal proofs are deferred to the supplementary material.
3.3 Identifiability for Symmetric ANMs
The crux of the problem of non-identifiability in Section 3.2 comes from the fact that treatment X has direct causal effect treatment X2. In many realistic applications, this might never occur. Treatments will decidedly be correlated, but this can be encoded in the SCM via either the observed covariates the unobserved confounders. Then still, form of the structural equations with respect to the treatments (i.e. fy), treatments can still yield highly nonlinear interaction effects the outcome. As such, now focus the case where the SCM is devoid of causal links between treatments, but includes observed covariates as well as unobserved confounders. In this case, the structural equations take the following form:
have restrictions the functional
X; = f((C)) + U..
= fy
Uy
Identifiability in this setting requires additional assumptions about the correlation between the covariates and confounders. That is, without any assumptions on the nature of the relation between C' and U, our estimand is not identifiable. We include proof (by counterexample) of this result in the supplemental material. Intuitively, the problem is that allowing E|Uy CI be non-zero impacts the identifiability of fy under joint interventions as fy also depends on C whilst this is crucial allow for disentanglement. As we have shown additional restrictions on the class of SCMs are necessary, we now assume independence between C and U. Future work can explore whether this can be relaxed to allow for linear relationships, among others.
Fig. visualises the structure of the causal graph we assume, when we have K treatment variables. We call such causal graphs "symmetric". SCMs with this structure occur many application areas from economics to online advertising and beyond. In SCMs of this form, the causal effect of all single-variable interventions can be disentangled from observational and joint interventional data:
Theorem 2 (Identifiability of disentangled conditional average treatment effects in additive noise models with symmetric structure).
Let M ((C, X, YJ, U, f. be an SCM, where X; = f((C) + Us
X1
X₂
X
Vi = 1..1...,44 K,
Y= fc, +U,
NO E). The estimand E[Y)do(X.),, C) identifiable from the conjunction of two data regimes: (1) the observational distribution, and (2) any interventional distribution on a set of treatments X S X that holds X₃: X, Xin-
CL U, and Pu
Figure/ 1: DAG for a symm. SCM with covariates and unobserved confounders.
In this section, we have studied for several classes of SCMs whether causal effects can be disentangled. Our results provide insights into the fundamental limits of learning and inference, and help crystallise which assumptions are necessary and sufficient to make disentanglement of effects from sets of interventions feasible. Figure 2 visualises and summarises our key results denoting which single- variable causal effects can be disentangled in ANMs with zero-mean Gaussian noise. In what follows, we present learning methodology and validate our theoretical identifiability results with empirical observations on synthetic and real data.
5

---

do x2)
do x2)
X2
X1
X2
X3
do(X1 xi)
do(Xi x1)
(a) Only E|yCC = c, do =
(b) Only B| c, do
(c) All B|Y|C = c. do(X;
#2]] is generally identifiable.
Ja]] is generally identifiable.
24), are generally identifiable.
Figure 2: Causal DAGs illustrating under which conditions the single- variable causal effect on the outcome [ c, do(X, z;)] is identifiable from observational and joint interventional data.
4 Methodology
In this section, we introduce our methodology for estimating SCMs and providing estimates for treat- ment effects under any set of interventions. Estimating an SCM from combination of observational and interventional data boils down to (1) estimating the structural equations, and (2) estimating the noise distribution. We extend the Expectation- Expectaioo-Maximisttion Maximisation-styl iterative algorithm proposed by [Saengkyongam and Silva, 2020] to handle observed covariates, and to disentangle causal effects instead of combining them. The resulting method not limited to learning single-variable causal effects, as the results from [Saengkyongam and Silva, 2020) allow us extend these newly learned single-variable effects to sets of interventions. As a result, our method able to generalise to sets of interventions that were never observed in the training data, with the only restriction that every variable that makes up the set was part of some intervention set in the training data.
Say we intervene on a subset of treatments X int C X, and Xobs X decompose causal query with interventions on Xim as follows:
Xint- In general, we can
Y C: do Xim); Xahs (C; X) + R|Uy| Xam).
Samples consist of observed values for all endogenous variables. For convenience, we denote sample by æ. Suppose we have data from different data regimes, corresponding to d different sets of interventions (the empty set corresponds the observational regime. The full dataset consists of samples and their corresponding interventions D
{ inti x)).
Estimating the structural equations. We pa- rameterise the structural equations with 0, de- noted as (+; 0). The Gaussian likelihood with
Algorithm SCM Estimation for Symm. ANMs
1: Input: Dataset D
2: Output: Parameter estimates Ō, E
covariance matrix E is denoted (u E).
3: Initialise 0 and &
The likelihood for a single endogenous vari- able is defined L(x;; M) Pr(z; (p): )). The likelihood for a sample is defined the product of the likelihoods for every endogenous variable that was not inter- vened in that sample:
4: while not converged do
5: // Solve for 9 with fixed E
6: Optimise log-likelihood in Eq. 6
7: // Solve for Σ with fixed Ø
8: Estimate Σ from = x -f(x;
return 0, E
L(x; X ints i E)
II X
L(x,0,E), (5)
Naturally, the log-likelihood of the dataset is then:
{(D;0
Σ (X...)ED
log L(x; Xin. 0, E)
(6)
In principle, any iterative optimisation procedure can be used maximise Eq. when we fix the covariance E. Typically, appropriate method is chosen with respect to the model parameterisation.

---

Estimating the noise distribution. Following [Saengkyongam and Silva, 2020], we note that the maximum likelihood estimate for the covariance matrix of a multivariate normal can be directly com- puted from the sample. This allows for efficient closed-form computation of this step. Additionally, note that the assumption of zero-mean Gaussian noise simplifies the proof and yields an analytical solution for this step, but this is not a general limitation. Indeed- -what matters is that we can estimate the conditional noise B|Uy [obs]. which can in principle be estimated via a different model. Algorithm I shows the full procedure to estimate the parameters of symmetric ANM. At inference time, Eq. 4 allows us to estimate the outcome under any- any-even unseen-see of interventions.
5 Empirical Validation
We now empirically validate the effectiveness of our method in estimating SCMs from observational and coint-ntervvnntions data, and assess the accuracy of the inferred outcomes under varying sets of interventions. We do this both on synthetic data and semi- symi-sytthetic data based on real data from medical study. The research questions we wish to answer from empirical results, are the following:
RQ1: Are able accurately estimate the outcome under previously unseen sets of interventions? RQ2: Are we able to accurately estimate the structural equations and the noise distribution? RQ3: How does our method handle varying levels of unobserved confounding?
5.1 Synthetic experiment
First we adopt a simulation setup, which gives us the freedom to vary the true underlying SCM and observe performance differences among competing methods. The structural equations fi, J' in Eq. 3 are polynomials with second-order interactions to illustrate the effectiveness of the learning method even when treatment effects are highly non-linear, and the optimisation surface is highly non-convex. We use the well-known Adam optimiser in our experiments [Kingma and Ba, 2014]. As a baseline learning method we adopt a regression model that estimates the outcome from its direct treatments using pooled data from different regimes. Here, the presence of unobserved confounders severely hinders the method to provide accurate estimates under varying sets of interventions, further motivating the need for causal models.*
We let |C|==l = X = 4, yielding 0; E R Vf: and By E R for the parameterisation of fy. The pa- rameters for the structural equations are randomly sampled from a uniform distribution over 2-2, +2) The covariance matrix Σ E 5x5 is uniformly sampled from uniform distribution over [-1, and then ensured to be positive semi-definite through Ledoit Wolf shrinkage [Ledoit and Wolf, 2004]. We repeat this process 10 times with varying random seeds and report confidence intervals over obtained measurements.
We vary the size of the training sample [D] E (2% = 5,... ...,, 13), where the set of intervened treat- ments Xm is one of {0: (Xo. }1}: {²): Xx) }. As such, no single-variabee interventions and the majority of the possible sets of interventions never observed. From this data, we estimate an SCM using the procedure laid out in Algorithm
Then, for every possible (2)x1) set of interventions, we simulate 10 samples and estimate the outcome based on our estimated SCM using Eq. 4. We report the Mean Absolute Error (MAE) between our estimated outcome and the true outcome. Note that, as the true outcome Gaussian, the optimal estimate is the location of that Gaussian, which will have an expected deviation of / where a is the standard deviation on the noise parameter Uy.
Estimating outcomes (RQ1). Figure 3 visualises the results from the procedure laid out above, increasing the size of the training sample over the x-axis and reporting the MAE on the y-axis. Note that both axes are logarithmically scaled, and the shaded regions indicate 95% confidence intervals. The plots clearly indicate that our method is able to provide accurate and close to optimal estimates for the outcome, under any possible do Xo, X1), do{,, X2) and do(X: X2, Xx) show the accuracy of our method to estimate the outcome under interventions that occurred in the training data-but the remaining twelve plots relate to previously unseen data regimes. While our results allow us to disentangle joint interventions, the
of interventions. Indeed, the plots labelled observational,
'The code reproduce results is available at gltabb..ww/diieeeeeeeevviiinnnnn nwwr:pp--
7

---

Observational
do(XGo)
do(Xi)
do((s))
MAE
do(XX))
do(Xy, X.)
do(Xn, Xt.)
do(Xp, Xy)
MAR
do(X,, X;)
Lo(Xy,X.)
do(Xin X1.
MAB
10
do(Xp, XT)
do N2. XC)
do(X,, Nq. Xa)
dolXn, X1, N2. Xy)
MAR
Sample Size
Sample Size
Sample Size
Sample Size
Model
Regression
Symm. ANM
Data
Seen
Unseen
Figure 3: Mean Absolute Error for estimating outcome under varying sets of interventions, for varying sample sizes, comparing baseline regression with method. We empirically show estimator unbiased and consistent, exhibiting optimal predictions even at modest training sample sizes.
0 0
MAB
10-1
10-2
10a
10a
10
10
10
Sample Size Model
Sample Size
Regression
Symm ANM
Figure 4: Mean Absolute Error for estimating the parameters of the underlying SCM, for varying sample sizes, comparing baseline regression with our method. We empirically validate our estimator unbiased and consistent, giving accurate estimates even at modest training sample sizes.
results of [Saengkyongam and Silva, 2020] allow us to combine interventions. As such, our method incorporates (and subsumes) theirs, in order generalise to arbitrary sets of interventions. In contrast, the regression method that is oblivious to confounders fails to accurately estimate the outcome under any data regime- even those that are seen in the training data, those where confounders yield no influence on the outcome (i.e. all treatments are intervened on). As such, the reported results validate that the proposed method provides unbiased and consistent estimator for the disentangled causal effect, learned from sets of interventions the presence of unobserved confounders.
Estimating SCMs (RQ2). We visualise the results with respect to the accuracy of the parameter estimates in Figure 4. These measurements are obtained using the same procedure and runs as for RQI. Increasing the size of the training sample over the x-axis, the leftmost plot shows the accuracy of the estimated parameters for fy. The rightmost plot shows the accuracy of the estimated covariance matrix for the multivariate normal that defines the noise distribution. As the baseline regression method oblivious to the noise distribution, are logarithmically scaled, and the shaded regions indicate 95% confidence intervals. We observe that our method is able efficiently and effectively estimate the underlying causal model empircelpry validating the identifiability results presented in this work.
not included in the latter plot. Both axes

---

5.2 Semi-synthetic experiment based on real medical data
We now show our method can accurately estimate unseen sets of interventions (RQ1) and demonstrate it is consistent under varying levels of unobserved confounding (RQ3) using semi-synthetic setup on real-world data from the International Stroke Trial database [Carolei, 1997]. This was large, randomized trial of up to days antithrombotic therapy after stroke onset. There are two possible treatments: the aspirin allocation dosage aa and the heparin allocation dosage at+- The goal is to understand the effects of these treatments on a composite outcome, a continuous value in [o, predicting the likelihood of patients' recovery.
We partially follow the semi-synthetic setup laid out in Appendix 3 of [Zhang and Bareinboim, 2021]. Specif- ically, adopt their probability table for the joint observational distribution of the covariates: the gender, age and conscious state a patient. This table was computed from the dataset to reflect a real-world observational distribution. While [Zhang and Bareinboim, 2021] deal with single discrete treatments, we extend the continuous and multi-treatment case where treatments can be interpreted as varying dosages of aspirin and heparin. This procedure is described in detail in the Supplementary Material.
Predicting Under
Absollate
10
0.00
0.25
0.50
Maximal Covariance
Regression
Syzam, ANM
Optinal
Figure 5: Mean Absolute Error for estimating the impact of interventions on treatment Oa. learned only from observational and joint interventional data. We vary the magnitude the covariance between treatments and outcome to assess the effect of varying confounding.
We add zero-mean Gaussian noise to the treatments and the outcome, where we randomly generate positive semi-definit covariance matrices with bounded non-diagonal entries evariingarriing the limit the size the covariances in order to assess the effect of varying confounding on our method. We repeat this process times, and sample 512 observational and 512 joint-interventional samples (both Oa and or to learn the SCM from). We evaluate our learning methods on 5 000 evaluation samples to predict the outcome under virile-variable intervention the aspirin dose Oa-
Figure shows results with respect to the accuracy of the learnt model for making predictions on the outcome under varying single-variable interventions on the aspirin dosage Oa. Naturally, as the regression method has no way accounting for the confounding effect, it fails dramatically. Our proposed method is able to consistently approximate the optimal performance whiiommanneeehchh which is attained by predicting the expectation of the outcome distribution and limited by its variance. For increasing levels of confounding, the gap between the optimal performing oracle and our learnt model increases slightly. As such, this semi-synthetic experiment validates the effectiveness of our proposed learning method, which predict outcomes under unseen sets of interventions (RQ1) and is robust to varying levels confounding (RQ3), for a use-case based on real-world medical data.
Conclusions & Future Work
In this work, we motivated the need for methods that can disentangle the effects of single interventions from jointly applied interventions. multiple interventions can interact in possibly complex ways, this is challenging task; even more so in the presence of unobserved confounders. First, we proved that such disentanglement is not possible in the general setting, even when we restrict the influence of the unobserved confounders be additive in nature. By restricting the structure of the causal graph to symmetric we showed identifiability can be acquired. Additionally, we showed how to incorporate observed covariates, and have empirically demonstrated our method.
Future work will tackle the case where the noise distribution is not restricted a zero-mean mean multivari- ate Gaussian. As mentioned in Section 4, this assumption provides closed-form expression for the conditional expectation of the outcome noise given the observed treatments- but universal function approximators could be used to obtain similar guarantees in more general model classes. Investigating whether the assumption of causal independence between the unobserved confounders and covariates can be relaxed, to allow for linear relationships say, provides another avenue to explore.

---

References
[Bareinboim and Pearl, 2016]
Bareinboim, E. and Pearl, J. (2016). Causal inference and the data- fusion problem. Proceedings of the National Academy of Sciences, 113(27):745-7355.
[Carolei, 1997] Carolei, A. (1997). The international stroke trial (ist): randomised trial of aspirin, subcutaneous heparin. both, or neither among 19435 patients with acute ischaemic stroke. The Lancet, 349(9065):1569-13 1581.
[Correa al., 2021] Correa, J. D., Lee, S., and Bareinboim, E. (2021). Nested counterfactual identification from arbitrary surrogate experiments. arXiv preprint arXivv2107.00. 1090099900
[Dhir and Lee, 2020] Dhir, A. and Lee, C. M. (2020). Integrating overlapping datasets using bivariate causal discovery. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 3781-3790.
[Egami and Imai, 2018] Egami, N. and Imai, K. (2018). Causal interaction in factorial experiments: Application to conjoint analysis. Journal of the American Statistical Association.
[Ghassami et al., 2021] Ghassami, A., Sani, N., Xu, Y., and Shpitser, (2021). Multiply robust causal mediation analysis with continuous treatments. arXiv preprint arXiv:2
[Gilligan-Lee, 2020] Gilligan-Lee, C. (2020). Causing trouble. New Scientist, 246(3279).32-35.
[Hoyer et al., 2009] Hoyer, P., Janzing, D., Mooij, J., Peters, J., and Schölkopf, B. (2009). Nonlinear causal discovery with additive noise models. In Twenty-Second Annual Conference on Neural Information Processing Systems (NIPS 2008), pages 689-696. Curran.
[llse et al., 2021] llse, M., Forré, P., Welling, M., and Mooij. J. M. (2021). Efficient causal inference from combined observational and interventional data through causal reductions. arXiv preprint arXhv:2103.04788 047766
(Janzing al., 2009] Janzing, D., Peters, J., Mooij, J., and Schölkopf, B. (2009). Identifying con- founders using additive noise models. 25th Conference on Uncertainty in Artificial Intelligence (UAI 2009), pages 249-2571 AUAI Press.
[Kilbertus al., 2020] Kilbertus, N., Ball, P.J., Kusner, M. J., Weller, A., and Silva, R. (2020). The sensitivity of counterfactual fairness to unmeasured confounding. In Uncertainty in Artificial Intelligence, pages 616-626. PMLR.
[Kingma and Ba, 2014] Kingma, D. and Ba, J. (2014). Adam: A method for stochastic optimiza- tion. arXiv preprint 1412v914
[Ledoit and Wolf, 2004] Ledoit, O. and Wolf, M. (2004). A well-conditioned estimator for large- dimensional covariance matrices. Journal of Multivariate Analysis, 88(2):365-411
[Lee et al., 2019] Lee, C. M., Hart, C., Richens, J. G.,, and Johri, S. (2019). Leveraging directed causal discovery to detect latent common causes. arXiv preprint 1910v
[Lee and Spekkens, 2017] Lee, C. M. and Spekkens, R. W. (2017). Causal inference via algebraic geometry: fcasibility tests for functional causal structures with two binary observed variables. Journal of Causal Inference, 5(2).
[Lee al., 2020] Lee, S., Correa, J. D., and Bareinboim, H. (2020). General identifiability with arbitrary surrogate experiments. In Uncertainty in Artificial Intelligence, pages 389-398. PMLR.
[Lopez and Gutman, 2017] Lopez, M. J. and Gutman, R. (2017). Estimation of causal effects with multiple treatments: a review and new ideas. Statistical Science, pages 432-454.
[Nandy et al., 2017] Nandy, P., Maathuis, M. H., and Richardson, T. (2017). Estimating the effect of joint interventions from observational data in sparse high-dimensional settings. The Annals of Statistics, 45(2):6476674
[Parbhoo et al., 2021] Parbhoo, S., Bauer, S., and Schwab. P. (2021). Ncore: Neural counterfactual representation learning for combinations of treatments. arXiv preprint arXiv:2103.1117
[Parbhoo et al., 2020] Parbhoo, S., Wieser, M., Roth, V., and Doshi-Veles F. (2020). Transfer learning from well-curated to less-resourced populations with hiv. Machine Learning for Healthcare Conference, pages 589-609. PMLR
[Pearl, 2009] Pearl. J. (2009). Causality. Cambridge university press.
10

---

[Perov et al., 2020] Perov, Y., Graham, L., Gourgoulias, K., Richens. J., Lee, C., Baker, A., and Johri, S. (2020). Multiverse: causal reasoning using importance sampling in probabilistic programming. In Symposium on advances in approximate bayesian inference, pages 1-36. PMLR.
[Rolland et al., 2022] Rolland, P., Cevher, V., Kleindessner, M., Russel, C., Schölkopf, B., Janzing,
D., and Locatello, F. (2022). Score matching enables causal discovery of nonlinear additive noise models. arXiv preprint arXiv:2203,044: 044.1113..
[Rothenberg, 1971] Rothenberg, T. (1971). Identification in parametric models. Econometrica, 39(3):577-591.
[Saengkyongam and Silva, 2020] Saengkyongam, S. and Silva, R. (2020). Learning joint nonlinear effects from single-variable interventions in the presence of hidden confounders. In Peters, J. and Sontag, D., editors, Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI), volume 124 of Proceedings of Machine Learning Research, pages 300-309. PMLR.
[Schwab al., 2020] Schwab, P., Linhardt, L., Bauer, S., Buhmann, M., and Karlen, W. (2020). Learning counterfactual representations for estimating individual dose-response curves. In Pro- ceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 5612-5619.
[Vlontzos et al., 2022] Vlontzos, A., Kainz, B., and Gilligan-Lee, C. M. (2022). Estimating categor- ical counterfactuals via deep twin networks. In UAI 2022 Workshop on Causal Representation Learning.
[Zhang and Bareinboim, 2021] Zhang, J. and Bareinboim, E. (2021). Bounding causal effects continuous outcome, Proceedings of the 35nd AAAI Conference on Artificial Intelligence.

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See the discussion on the assumptions in Section 3.3
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes) See Appendix.
(b) Did you include complete proofs of all theoretical results? [Yes] See Appendix.
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See 5. Appendix and source code.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes] See Figures and 3.
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, cloud provider)? [Yes] See Appendix, the notebooks on laptops or Google Colaboratory.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes] We cite the Interan- tional Stroke Trial Database [Carolei, 1997]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any assets either in the supplemental material URL? [Yes] Our notebooks are made available.
(d) Did you discuss whether and how consent was obtained from people whose data you'l 're
using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
12


Reviews:
[{"review_id": "uLBEJGTncNI", "reviewer": "Reviewer_FYX4", "summary": "The paper considers estimating the causal effect of a single intervention when multiple interventions are applied at the same time.\nThe paper characterizes several cases where the single intervention's effect can be identified. \nIn particular, the paper considers the identifiability characterization under non-linear structural models, unobserved confounders, and interactions among interventions. The paper also designs an algorithm that learns the causal effect by combining observed data and interventional data.\n", "questions": "* It would be very helpful to discuss the necessity and sufficiency of the identifiability conditions. \n\n* It would be very helpful to provide some theoretical insights of the proposed algorithm. \n\n* It would be very helpful to discuss how the identifiability conditions could be verified/checked on a real dataset.\n", "limitations": "Yes.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n* The paper extends previous works of linear structural models to allow non-linear dependence. The paper also takes into account unobserved confounders---a fundamental challenge in causal inference.\n\n* The paper employs observational data to augment interventional data. This is impactful because there are usually more observational datasets available, and often in large scale.  \n\nWeaknesses:\n\n* The identifiability discussed in the paper relies on the multivariate normal assumption. This may not always be satisfied in practice. For example, some variables may be binary and do not follow Gaussian distributions.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "GXFXzLeHmNh", "reviewer": "Reviewer_P8j4", "summary": "In this paper, the authors aim to disentangle the single intervention's effect with the observational data and interventional data upon **many variables** simultaneously, in the presence of latent confounders. The authors suppose the additive noise model, and propose two cases when the effect can be disentangled, one is the case when all the treatment variables are in exact causal order and we want to identify the causal effect of the treatment variable at the last of the order on the outcome, in Theorem 1, the other is the case when there are no causal relations between the treatment variables, in Theorem 2. Further, the authors present the method to disentangle the effect by EM algorithm.\n", "questions": "I am not quite sure whether the examples in the first paragraph are convincing. For example, people are targeted by multiple concurrent campaigns in computational advertising, or patients that possess many commodities may have to be simultaneously treated with multiple prescriptions. In these cases, our goal is usually to know how to intervene to make the people buy our commodity, or how to do to make patients restored from sickness. Why do we identify the single intervention's effect?", "limitations": "The authors should make it clear that it is an assumption that covariates $C$ are not allowed to be caused by the treatment variable. ", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n\n(+) The problem of disentangling single intervention's effect is interesting.\n\n(+) In total, the paper is clearly written.\n\nWeaknesses:\n\n(-) According to my understanding, to the problem of disentangling single intervention's effect, the most interesting part is not to give some cases when the disentangling can be finished, but when the disentangling cannot be finished. A sufficient and necessary condition for the ANM model is interesting and novel. The current paper only presents two cases when the single intervention's effect can be identified. A deeper study is expected.\n\n\n(-) In this paper, it is not allowed that some covariate $C$ is caused by the treatment variable, as Line 121. I think the authors should make it clear that it is an assumption in this paper. In general SCM framework, there are not such assumptions that the causal directions between covariates and treatments are from covariates to treatments.\n\n(-) The formalization of the problem on Line 133 is not quite good. The intervention can be upon more than two variables, but in the format here there are only two intervention variables.\n\n(-) I think it will be better if the authors can further clarify the motivation of the disentangling from interventions upon many variables. In the view of theory, Lee et al. 2020 have given the sound and complete result for the non-parametric identifiability with arbitrary experiments. The manuscript can be seen as a special case when we (1) consider ANM; (2) assume covariates cause both treatment and outcome; (3) Gaussian noise. I am not saying it is not good, I can understand that by this way there are some advantages in computation. But considering that the theoretical results in the manuscript are not complete, it is vital to argue for the importance of their results or methods in reality.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "DTEs27u4ay", "reviewer": "Reviewer_sK28", "summary": "Assuming $C \\perp U$ and $X_i \\perp X_j \\mid C, U$ ($i \\not = j$) in Figure 2(c), this paper provides identification proofs under non-linear continuous structural causal models with additive, multivariate Gaussian noise. The estimation is identifiable from the conjunction of two data regimes: (1) the observational distribution, and (2) any intervention distribution on a set of treatments. Then, this paper extends the Expectation-Maximization-style iterative algorithm to disentangle causal effects.", "questions": "1. This paper claims that disentangling the effects of single interventions from jointly applied interventions is a challenging task. Thus, they assume $C \\perp U$ and $X_i \\perp X_j \\mid C, U$ ($i \\not = j$) in Figure 2(c). In this setting, is it still a challenge to disentangle the effects of single interventions?\n\n2. Under additive multivariate Gaussian noise assumption, can we use traditional methods for single treatment analysis? That is, we can focus on a single treatment variable $X_i$ and treat other treatments $\\{X_j\\}_{j \\not= i}$ as confounders. \n\n3. Does this paper require prior knowledge of the structural equations?  The reasons for uniform distribution over [-2,+2] and [-1,+1] are?\n\n4. The estimation is identifiable from the observational distribution. Why would this paper identifies causal effects from intervention distribution? To be honest, the acquisition of experimental data is also a problem.\n\n5. As the baseline for this paper, I think some causal algorithms for distribution modeling should also be considered, such as GANITE (Yoon et al., 2018), SCIGAN (Bica et al. 2020), CEVAE (Louizos et al., 2017), and Intact-VAE (Pengzhou et al., 2022)...\n\n6. In the presence of unmeasured confounders (unconfoundedness assumption is violated), can the causal effect of a single intervention variable be accurately estimated under non-linear continuous structural causal models with additive, multivariate Gaussian noise? ", "limitations": "I do not foresee any major limitations and/or societal impacts. ", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This paper claims that disentangling the effects of single interventions from jointly applied interventions is a challenging task. Thus, they assume $C \\perp U$ and $X_i \\perp X_j \\mid C, U$ ($i \\not = j$) in Figure 2(c). In this setting, is it still a challenge to disentangle the effects of single interventions? I will adjust my score according to the authors’ responses to my comments.\n\n**Strengths**\n- This paper considers a interesting problem in Multiple Treatment Effect and provide identification assumption. \n- Theoretically, this paper provide that learning the effect of a single-intervention from both observational data and sets of interventions is not generally possible, but provide identification proofs demonstrating that it can be achieved under non-linear continuous structural causal models with additive, multivariate Gaussian noise. \n\n**Weaknesses**\n- Additive multivariate Gaussian noise assumption is not a general condition, which is rarely satisfied in practice. \n- In Experiment Sectoin, the regression method is not a representative baseline in causal inference.\n- In the observed data, disentangling single-treatment effect does not seem to be a challenge. We can analyze the single-treatment effect one by one and treat all other variables as confounders.\n\n-----------------------\nI thank the authors for addressing my comments. I increase my score to 6.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "-Hibm3PqRP", "reviewer": "Reviewer_5byu", "summary": "This paper is on disentangling the effects of single interventions from jointly applied interventions, given observational data and the joint interventions. In particular, this paper shows that the causal effect of single interventions is generally unidentifiable, but it is identifiable for the so-called symmetric ANMs, where there is no direct causal connection among the treatment variables. Moreover, this paper proposes a practical method to estimate the causal effect of single interventions.", "questions": "If only some of the treatment variables are influenced by hidden confounders, then what will the identifiability conditions be? Can they become weaker?", "limitations": "The authors did not discuss the limitations and potential negative societal impact of their work.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThe motivation is clearly stated at the beginning of the paper, and the overall organization is good. The studied problem is new to me.\n\nThe paper gives both theoretical identifiability results and practical estimation methods.\n\nWeaknesses:\nThe authors give the identifiability in rather strong assumptions. One is that if there is no direct causal connection among the treatment variables for ANMs, then the causal effect of single interventions is identifiable. The other is that for general ANMs, the causal effect of the sink treatment variable is identifiable. It could be interesting to investigate weaker identifiability conditions. \n\nIn addition, in some cases, we may want to know the identifiability of a subset of the interventions, so it would be also interesting to see the identifiability conditions for the causal effect of a subset of interventions. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept