Interventional Sum-Product Networks: Causal Inference with Tractable Probabilistic Models
Authors: Matej Zecevic
Devendra Singh Dhami
Athresh Karanam
Sriraam Natarajan
Kristian Kersting
Venue: NeurIPS 2021 Poster (2021)
Keywords: Interventions
Causality
Tractablity
Probabilistic Models

Abstract: While probabilistic models are an important tool for studying causality, doing so suffers from the intractability of inference. As a step towards tractable causal models, we consider the problem of learning interventional distributions using sum-product networks (SPNs) that are over-parameterized by gate functions, e.g., neural networks. Providing an arbitrarily intervened causal graph as input, effectively subsuming Pearl's do-operator, the gate function predicts the parameters of the SPN. The resulting interventional SPNs are motivated and illustrated by a structural causal model themed around personal health. Our empirical evaluation against competing methods from both generative and causal modelling demonstrates that interventional SPNs indeed are both expressive and causally adequate.

Interventional Sum-Product Networks: Causal Inference with Tractable Probabilistic Models
Matej Zečević!
Devendra Singh Dhami'
Athresh Karanam²
Sriraam Natarajan
Kristian Kersting!
'Computer Science Department, TU Darmstadt Computer Science Department, The University of Texas at Dallas CCnntre for Cognitive Science, TU Darmstadt, and Hessian Center for Al (hessian. Al) {matej zecevic, devendra. aevendra.dhami, dhami, kerstingh tu-darmstadt. de {athresh. karanam, sriraam.... wwtemmaaaaaaaaaaaa jatarajan/@utdallll edu
Abstract
While probabilistic models are an important tool for studying causality, doing suffers from the intractability of inference. As step towards tractable causal models, we consider the problem of learning interventional distributions using sum- product networks (SPNs) that are over-parameterized by gate functions, e.g., neural networks. Providing an arbitrarily intervened causal graph as input, effectively subsuming Pearl's do-operator, the gate function predicts the parameters of the SPN. The resulting interventional SPNs are motivated and illustrated by structural causal model themed around personal health. Our empirical evaluation against competing methods from both generative and causal modelling demonstrates that interventional SPNs indeed are both expressive and causally adequate.
Introduction
Identifying causal relationships between variables in observational data is of the fundamental
and well-studied problem in machine learning. There have been several great strides in causality
[Granger] 1969 Pearl 2009 [areinboim and Pearl 2016 over the years specifically characterized by
efforts that focused on reasoning about interventions Hagmayer et al. 2007 Dasgupta et al.) 2019
and counterfactuals [Morgan and Winship 2015 Oberst and Sontag 2019).
The notion of causality has long been explored in the realm of probabilistic models Oaksford
and Chater 2017 Beckers and Halpern| 2019] with special focus graphical models, called
causal Bayesian networks (CBNs) [Heckerman et al.) 1995] Neapolitan 2004 Pearl [993] Acharya
et al. 2018). CBNs have widely been applied to infer causal relationships in high-impaet diverse
applications such as disease progression (Koch et al. 2017), ecological risk assessment [Carriger and Barron 2020 and more recently Covid-19 [Fenton al. 2020 Feroze 20200 to name few. Although successful, classical CBN models are difficult to scale and also sutter from the problem of intractable inference. Recently, tractable probabilistic models such as probabilistic sentential decision diagrams [Kisa et al.) (0144) and sum con-prrduct networks (Poon and Domingos/ 2011] have emerged, which guarantee that conditional marginals can be computed in time linear in the of the model. While weaving in the notion of interpretability, the computational view probabilistic models allows one to exploit ideas from deep learning and can thus be very useful modelling complex problems.
Recently, there has been an effort to take advantage of this tractability to reason for causality. Zhao al. [2015) showed how compile back and forth between sum-product networks (SPNs) and Bayesian networks (BNs). Although this opened up a whole range of possibilities for tractable
35th Conference on Neural Information Processing Systems (NcurIPS 2021), virtual.

---

O
B
seagss Mode
Ground Truhh
do O = B
Figure I: Capturing interventional distributions using iSPN. The interventional distributions for the ASIA data set using a causal Bayesian network (CBN, small-scale gold standard, gray bars) as well as an interventional SPN (iSPN) by intervening on lung. The iSPN is sensible to all the influences of the given intervention onto the system i.e., subsequent effects in the causal hierachy. (Best viewed color.)
causal models, Papantonis and Belle 2020 argued that such conversion leads to degenerated BNs thereby rendering it ineffective for causal reasoning. For the considered compilation of SPNs to BNs, this is indeed the case since a bipartite graph between the hidden and observed variables loses the relationships between the actual variables. Thus, either a new compilation method for transforming between tractable and causal model or alternatively method to condition the probabilistic models directly on the do-operator to obtain interventional distributions, P do(x)), is being required.
Here, we consider the latter strategy and extend the idea of conditionally parameterizing SPNs (Shao et al. 2019] by conditioning on the do-operator while predicting the complete set of observed variables therefore capturing the effect of intervention(s). The resulting interventional sum-product networks (iSPNs) take advantage of both the expressivity, due to the neural network, and the tractability, due the SPN in order capture the interventional distributions faithfully. This shows that the dream of tractable causal models is not insurmountable, since iSPNs are causal. Pearl [2019] defined three-level causal hierarchy that separates association (purely statistical level 1) from intervention (level 2) and counterfactuals (level 3), and argued that the latter two levels involve causal inference. iSPNs are a modelling scheme for arbitrary interventional distributions. They thus belong to level and, in turn, are causal. So, while SPNs are not universal function approximators and the use of gate functions turns them into universal approximators, we go one step ahead and make the first effort towards introducing causality to SPNs without the need for compilation to Bayesian networks, as the functional approximator subsumes the do-operator. Fig. shows an example of the effectiveness of iSPNs to capture interventional distributions on the ASIA data set. Our extensive experiments against strong baselines demonstrate that our method is able to capture ideal interventional distributions.
To summarize, we make the following contributions:
1. We introduce iSPNs, the first method that applies the idea of tractable probabilistic models to causality without the need for compilation and that accordingly generate interventional distributions w.r.t. the provided causal structure.
2. We formulate the inductive bias necessary for turning conditional SPN into iSPN while taking advantage of the neural network modelling capacities within the gating nodes for for modelling interventional distributions while capturing all influences within the given intervention (i.e., consequences propagated through the structural hierarchy).
3. We show that, by construction, iSPNs can identify any interventional distribution permitted by the underlying structural causal model due to the inducted bias on the interface modalities in junction with the universal function approximation of the underlying gated SPN.
We proceed as follows. We start by reviewing the basic concepts required and related work, namely the tractable model class of sum-product networks and key concepts from causality. Then we motivate using newly curated causal model themed around personal health. Subsequently, we introduce iSPNs formally and prove that by construction they are capable of approximating any do-query (given corresponding data). Before concluding, present experimental evaluation in which

---

challenge iSPN in its density estimation and causal inference capabilitiess against various baselines. We make our code publically available at: htttpe///gittub..cmm//ccoooo www.://woccoooo//.o.co//coocc.cc
2 Background and Related Work
Let us briefly review the background on tractable probabilistic models and causal models used in subsequent sections for developing our new model class based on CSPNs that allow for identifying causal quantities i.e., interventional distributions.
Notation. We denote indices by lower-case letters, functions by the general form g()), scalars or random variables interchangeably by upper-case letters, vectors, matrices and tensors with different boldface font v, V, V respectively, and probabilities of a set of random variables X as pXX).
Sum-Product Networks (SPNs). Introduced by Poon and Domingos 201 generalizing the notion of network polynomials based on indicator variables Xx== (x) E [0 for (finite-state) RVs X from Darwiche 2003]. Sum-Product Networks (SPNs) represent special type of probabilistic model that allows for a variety of exact and efficient inference routines. Generally, SPNs are considered as directed acyclic graphs (DAG) consisting of product, sum and leaf (or distribution) nodes whose structure and parameterization can be efficiently learned from data to allow for efficient modelling of joint probability distributions p(X). Formally SPN S (G, w) consists of non-negative parameters w and DAG G = (V, B) with indicator variable leaf nodes and exclusively internal sum and product nodes given by,
S(A)
Σ Ws,cC(A) P(A) CEch(S)
II C(A), CEch(S)
where the SPN output S is computed the root node (S(A) S(x)) and the probability density
for x is p(x) ee S(x''' They are members of the family of probabilistic circuits [Van den
S(x)
Broeck et al. 2019]. A special class, to be precise. that satisfies properties known completeness and decomposability. Let N denote a node in SPN S, then
{ {x} 10--UUe UcEchiN se(C)
i is (Ax=a)
sc(N)
(2)
else
is called the scope of N and
VS E S: (VC,, C2 E ch((): sc(Ci) = sc(C2)) vP E S: (VC1, C2 E ch(S): C, C2 sc(C)) = 0)
(3)
(4)
the completeness and decomposability properties respectively. Since their introduction, SPNs have been heavily studied such as by Trrppp et al. 2019] that present a way to learn SPNs in a
Bayesian realm whereas (Kalra et al. 2018] learn SPNs in online setting. Several different types of SPNs have also been studied such as Random SPN [Peharz et al.) 2020) Credal SPNs [Levray and Belle 2020) and ceae--ooout-uottiiie
Networks (Sharir and Shashua 2018) name a few. For more details readers are referred to the survey of París, Sánchez-Cauce, and Diez 2020)] On another note, Gated or Conditional SPNs (CSPNs)
deep tractable models for estimating multivariate, conditional probability distributions Y X) over mixed variables Y (Shao et al. 2019 They introduce functional gate nodes g (X) that act as functional parameterization of the SPN's information flow and leaf distributions given the provided evidence X.
Causal Models. A Structural Causal Model (SCM) as defined by Peters al. 2017 is specified as C = (S, PN)) where PN is product distribution over noise variables and S is defined to be a set of d structural equations
X, fe(pa(())) No), where i 1,...,d
with pa(X.) representing the parents of X; in graph G(C). An intervention SCM e as defined in (5) occurs when (multiple) structural equations are being replaced through new non-parametric functions (pa(XX)) N:) thus effectively creating an alternate SCM é. Interventions are referred to imperfect if pa(X.) pa(X.) and as atomic if f for R. An important property
3

---

of interventions often referred to as "modularity" or "autonomy") states that interventions are fundamentally of local nature, formally
U
p (x, pa((x))) = -x; pa(X;))
(6)
where the intervention of E occured on variable X5 opposed to X,. This suggests that mechanisms remain invariant to changes in other mechanisms which implies that only information about the effective changes induced by the intervention need to be compensated for. An important consequence of autonomy is the truncated factorization
PVV) = II
s p(X, pa(X.))
(7)
derived by Pearl [2009], which suggests that an intervention S introduces an independence of an intervened node X; to its causal parents. Another important assumption in causality is that causal mechanisms do not change through intervention suggesting notion of invariance to the cause-effect relations variables which further implies an invariance to the origin of the mechanism i.c., whether it occurs naturally or through means of intervention [Pearl et al. 2016].
SCM capable of emitting various mathematical objects such as graph structure, statistical and causal quantities placing it at the heart of causal inference, rendering it applicable to machine learning applications in marketing (Hair and Sarstedt 202T]), healthcare [Bica et al. (0200) and education [Hoiles and Schaar, 2016)_ A SCM induces a causal graph G, an observationaallssooiiiiii distribution p., can be intervened upon using the do-operator and thus generate interventional distributions p (iioo(..) and given some observations v can also be queried for interventions within system with fixed noise terms amounting to counterfactual distributions y \vvdooo)).... To query for samples of given SCM, the structural equations are being simulated sequentially following the underlying causal structure starting from independent, exogenous variables and then moving along the causal hierarchy of endogenous variables (i.c.. following the causal descendants).
The work closest to our work is by Brouillard et al. 2020 although it solves the different problem of causal discovery. Causality for machine learning has recently gained a lot of traction (Schölkopf) 2019] with the study of both interventions (Shanmugam et al. 2015] and counterfactuals Kusner et al. 2017] gaining speed. For more details the readers are referred to [Zhang et al.) 2018
3 Interventional SPNs
Now we are ready to develop interventional SPNs (iSPNs). To this end, we re-introduce the impor- tance of adaptability of models interventional queries and present a newly curated synthetic data set to both motivate and validate the formalism of iSPNs that, through over-parametric extension of SPNs, allows them to adhere causal quantities.
3.1 Adaptation to Causal Change
Peters et al. [0177] motivated the necessity of causality via the adequate generalizability of predictive models. Specifically, consider a simple regression problem f(a) = b with data vectors a, e I that are strongly positively correlated in some given region, e.g. (1
co, 1 < b co). Now query is posed outside the data support, e.g. (0, (0)). As argued by Peters et al. the underlying data
generating processes be ambiguous causal process i.e., the data hand can be explained by two different causal structures being either A + B or a common confounder with A + C B.
Assuming the wrong causal structure or ignoring it altogether could be fatal, therefore, any form of generalization out of data support requires assumptions to be made about the underlying causal structure. We adopt this point of view and further argue that ignoring causal change(s) in a system, i.e.. the change of structural equation(s) s) underlying the system, can lead to a significant performance decrease and safety hazards" Therefore, it is important to account for distributional changes present in the data due to experimental (thus interventional) settings.
See Section 6.6 [Peters et al. 2017
"This extended notion of performance degeneration through ignorance to the underlying causality is priori- tized in this paper.

---

(a.
(b)
Dronctinal Consal Mare!
lay ATOLL
F(0,0)
D geay
1-00000
s(D:))
0100000
c
xXO
XO. SIGHt - 1001)
(O0 HKP D1N 310030000
POK 160 010 (A1)
Age
Food mobes
Figure An Overview iSPN. (a) The hidden process underlying the observable reality is modelled via SCM that can be modified through interventions. The given SCM induces a causal graph and can generate data. An intervention can significantly alter the resulting data. (b) An over-parameterized density estimation framework using SPN is presented. The universal function approximator (here neural network) conditions on the mutilated causal graph and provides parameters to the SPN such that the given data's 's density be modelled accordingly. The FA subsumes the do-operator. (c) Different causal queries are being presented. Furthermore, iSPN adapts to mnarvention-consqquenee
Consequently, we consider the learning problem, where the given data samples have been generated by different interventions do(U, = u,) in a common SCM e while the induced mutilated causal graphs GCC, do( ) are assumed to be known, such that the trained model is capable of at least inferring all involved causal distributions pCV(C) do(U, = u,)) with V being the variables.
3.2 Data Generating Process
To both validate and demonstrate the expressivity of interventional SPNs in modelling arbitrary interventional distributions, we curate a new causal data set based on the SCM e presented in Fig. a(),, which subsequently refer as Causal Health data set. The SCM encompasses four different structural equations the form V = f {pa Vi), N₂). where pa(Vi) are the parents of variable V, and N; are the respective noise terms that form a factor distribution PNio i.c. the N; are jointly independent. Now, the SCM e describes the causal relations an individual's health and mobility attributes with respect their age and nutrition.
Note that the Causal Health data set does not impose assumptions over the type of random variables or functional domains of the structural equation which additionally constraints learned model to adapt flexibly. While we generally not restrict our method to any particular type of intervention, the following mainly considers perfect interventions as introduced in Sec. 2
Perfect interventions fully remove the causal mechanism of the parents of given node, which is consistent with the idea behind randomized controlled trials (RCTs) where the given intervention randomizes the given specific node, often referred gold standard in causality related literature. We consider the special case of uniform randomization, i.e., uniform across the domain of the given variable") An intervention performed on one node immediately changes the population and, thus, has major effect on the generating processes of subsequent causal mechanisms in the respective causal sequence of events.
To provide the reader with a concrete example of interventions within the causal health data set, consider the following: In concern of virus infection the individuals of the Causal Health study should be vaccinated. The vaccine is expected to have side-effect(s), however, it has been poorly designed and has reached the population with the capability of completely changing the individual's
Assumptions on the func. form structural eq. are crucial for identification (Tab. 7.1 Peters et al. 2017 Note that for binary variables this amounts to Bernoulli (
5

---

health state. The observed changes do not show any form of pattern and are therefore assumed to be random. A young fit person could thus become sick, while an old person might feel better health wise. This sudden change will have an effect on the individual's mobility and also be independent of their age and nutrition. Such scenario is mathematically being captured through do (H)) where U-) is the uniform distribution over given domain.
3.3 Introducing Interventional SPNs
After motivating both the importance and the occurrences of interventions within relevant systems, we now start introducing interventional SPNs (iSPNs).
Definition of iSPN. As motivated in Sec. I the usage of the compilation method from [hhao et al. 2015) for causal inference within SPN is arguably of degencrate nature given the properties of the compilation method [Papantonis and Belle, 2020]. While the results of Pappanoonis and Belle (202)) arguably negative, there exists yet no proof of non-existence of such compilation method and as the authors point out in their argument for future lines of research in this direction, a model class extension poses a viable candidate for overcoming the problems of using SPN for causal inference.
While agreeing on the latter aspect, we do not go the "compilation road" but extend the idea of conditional parameterization for SPN [Shao et al. 2019] by conditioning on a modified form of the do-operator introduced by Pearl (2009) while predicting the complete set of observed variables.
Mathematicallyy, we estimate p(V; do(U, = uy)) by learning non- parametric function approxima- tor f(; 0) (e.g, neural network), which takes as input the (mutilated) causal graph G E {0, 1 encoded as an adjacency matrix, to predict the parameters vp of SPN gDD Φb) that estimates the density of the given data matrix {V& = =D E RKN, With this, iSPNs are defined as follows:
Definition (Interventional is the joint model m D) = g(D; & = ((G; 0)), where g(-) is a SPN, J(.) a non-parametric function appreximator and ab (G) are shared parameters.
Network). An interventional sum- cum-product network (iSPN)
They are called interventional because consider it to be causal model given its capability of answering queries from the second level of the causal hierarchy (Pearl, 2019]. namely, that of interventions The shared parameters &b allow for information flow during learning between the conditions and the estimated densities. Setting the conditions such that they contain information about the interventions, in the form of the mutilated graphs G, effectively renders f to subsume a sort of do-calculus in the spirit of truncated factorization shown in Eq. i.e., the gate model acts as an estimand selector. Generally, we note that our formulation allows for different function and density estimators J.g. We choose I to be neural network for two reasons (1) their empirically established capability to act as causal sub-modules (e.g. Ke et al. [2099] use a cohort of neural nets to mimic a set of structural equations, thus, SCM) and (2) their model capacity being a universal function approximator, while we choose g to be a SPN for its tractability properties for inference.
We have argued the importance of adaptability to interventional changes within the causal system and intend now to prove that iSPN are capable of approximating these different causal quantities.
Proposition (Expressivity). Assuming autonomy and invariance, an iSPN mGG, D) is able to identify any interventional distribution (V =. do(U, = =u,)). permitted by a SCM E through interventions, with knowledge of the mutilated causal graph G and data generated from the intervened SCMs by modelling the conditional distribution Pa (V. -v; VV = Uz))
Proof. It follows directly from the definition of the do-calculus PParl 2009 that PG V, = vi
do = uz))= = P v; U,= = uz)) where & is the mutilated causal graph according to the intervention do, = uy) i.e., observations in the intervened system are akin to observations made when intervening on the system. Given the mutilated causal graph G (as adjacency matrix), the only remaining aspect to show is that the density estimating SPN can approximate a joint probability p( (X)
using D. This naturally follows from [Ponn and Domingos 2011].
5 A hipartite graph in which the actual variables of interest are not connected called degenerate.
'The first level of the causal hierarchy association is considered to be purely statistical.

---

The expressivity of iSPN stems from both the capacities of gate function and the knowledge of intervention as well as availability of respective data. As an important remark, causal inference is often interested in estimating interventional distributions, i.e, causal quantities from purely observational models. Therefore, an alternative formulation to Prop. [ would be to replace the knowledge of the intervened causal structure G with knowledge on valid adjustment set. In the following, we only consider the direct setting where actual interventional data from the system is assumed to be captured thereby freeing the investigation of iSPN from the independent research around hidden confounding.
Universal Function Approximation (UFA). The gating nodes of CSPNs extend SPNs in way that allows them to also induce functions which are universal approximators. For instance, using threshold gates z; I c E E. one can realize testing arithmetic circuits (Choi and Darwiche, 2018) which have been proven to be universal approximators rendering iSPN to be UFA by construction.
Learning of iSPN. An interventional sum-produet network is being learned using a set of mixed- distribution samples generated from simulating the Causal Health SCM for different interventions, where the observational case is considered to be equivalent to an intervention on the empty set. The parameters 0, 2 of the iSPN describe the weights of the gate nodes and the distributions at the leaf nodes. The full model m is differentiable if the provided gate function f and each of the leaf models of are differentiable. Therefore, to train an iSPN, as depicted in Fig. 2.b). it is sufficient to optimize the conditional log- -ikeeeildooo end-to-end using gradient based optimization techniques. We assess the performance of our learned model through inspection of the adaptation of the model to the different interventions manifesting in the resulting marginals pVV do(U,)).
As can be observed in Fig. 2 or alternatively in Fig. (top row), the learned iSPN successfully adapts both the interventions as well as its consequences. Considering for instance the intervention p(V: do(F" = {}))) which removes the edge A + F' and thus renders Age (A) and Food Habits (F) independent. Given the drastic population change in F and the fact that the Health (!) of an individual is causally dependent on both A and F significant change in H is being expected. Indeed, both H and Mobility (M), being a causal child of H, broaden distribution wise and also these subsequent changes are captured correctly.
graeriii-gg do-calculus [Pearl 2009 is complete in that it can find estimands for any identifiable query in finite amount of transformation. An SPN, and thereby also iSPN, is capable of providing estimates to said demands when given observational data (L1 on the PCH). Consider for instance the well-known non-Markovian "Napkin" graph G = (W - Z -
Causal Estimation. The
YYY 4 X, W.W& Y) where < denotes a confounded relation. The causal effect is identified as ply do(z)) = (E p(a p(r/ =)p(w))- '(E., my p(y, x 2//,,2))p =))(w)) using the do-calculus where each of the r.h.s. components can be modelled by (i)SPN respectively. However, iSPN also capable of directly expressing the causal quantity pvy do (z))) as proven in Prop [ similar to other neural-causa
models like CausalGAN Kocaoglu et al. 2019 MLP--baeed NCM Xia et al. 2021).
Tractability, Inference in BNs and Markov networks is at least an NP-problem and existing exact algorithms have worst-case exponential complexity [Cooper] 1990 Roth 1996]. while SPNs can do inference in time proportional to the number of links in the graph. L.e., SPNs in general are able to compute any marginalization and conditioning query in time linear of the model's representation size r, that (r). We deploy simple neural network architectures for which the runtime for the forward seraadd-pass pass is that of matrix- mntrrx-mulliplication i.e., the time complexity scales cubically in the size of the input n, that is (n²)). The gradient descent procedure that involves forward and backward passes, assuming m gradient descent iterations, scales to Omm)3) which is the overall complexity we achieve during training phase. For SPN we deploy a random SPN structure which circumvents structure learning such. Therefore, overall, training will generally be in the 0(mrer)) regime while any causal query (that is, both L1 observational and L2 interventional in our case) will be answered within On""r). However, assuming that the weights of the random SPN were already initialized by the neural parvneter-provider in a previous step, any causal query becomes answerable in ((r). Since asking for inferences/queries q within single distribution (q - d E La) more common than changes between distributions (d, d C Li), this linear complexity by our tractable model being leveraged fully for performing causal inference.
Discussion. To reconsider and answer the general question of why the modelling of a conditional distribution via an over parammmteeter architecture is a sensible idea consider the following. One can
For details on the remarked alternative formulation consider the Supplement.
7

---

Query
V11
V/2
V/3
1/4
Method
ISPN
0011 # 000
.007 = 01
.003 + 000
0133 .01
MADE
588 + .59
.108 = 16
.015 + 022
.105 + .12
MDN
118
.263
= 14
.184 + .12 .079 + .01
Table 1: Jensen-Shannon--Dinn Shannon-Divergence Evaluation of Estimated
Figure 3: Mean Running Times in sec. till convergence (Causal Health) for 50 full passes. More data sets results in supplementary.
Interventional Distributions. Numerical pendant to Fig 4 mean and standard deviation per p p(V do (V; = U(V))) where U
the uniform distribution across all data sets, Lower=better.
represent a conditional distribution p( pYY X) by applying Bayes Rule to joint distribution density model (e.g. regular SPN) p(Y X) = p(Yx))
p(X) However, this assumes non-empty support i.e., pXX) 0. Furthermore, the joint distribution p(Y, X) optimizes all possibly derivable distributions, diminishing single distr. expressivity. Therefore, our considered formulation of gate model allows for effectively subsuming the do-operator i.e., the gate model orchestrates the do queries such that the density estimator can easily switch between different interventional distributions. While not introducing specific limitations, general CSPN limitation regarding OOD generalization are inherited.
Experimental Results
The assumptions made in causality usually require control over the data generating process which almost never readily available the real world. This amounts to scarcity of the available public data sets and also their implications for transfers to the real world and even when available, they are usually artificially generated as some causal extension of a known, pre-existing data set (e.g. MorphoMNIST data set introduced by Castro et al. (2019]). While it is difficult to consider real- world-esque experimental settings for causal models, we do not restrict our investigations of iSPN to rather specific problem settings like certain noise decision variable instances (which is common in causal inference). The evaluation is performed on data sets with varying number of variables and in both continuous and discrete domains. For the introduced causal health data set, we even consider arbitrary underlying noise distributions. We have made our code repository publicly available
Data Sets. We evaluate iSPNs four data sets. Three benchmarks: ASIA (A) [Lauritzen and Spiegelhalter 1988) with variables, Earthquake (E) with variables and Cancer (C) Korb and Nicholson 2010)] with 5 variables. One newly curated synthetic causal health (H) data set with 4 variables. More information about the data sets is presented in the Supplement.
Baselines. For generative capacities, we compare our method against Mixture Density Networks (MDN) [Bishop] 1994) and Masked Autoencoder for Density Estimation (MADE) (Germain et al.. 2015]. Both methods are expressive, parametric neural network based approaches for density estimation. Generally, the causality for machine learning literature suggests strong favor for neural based function approximators for modelling causal mechanisms [Ke et al. [2019]. For causal capacities, we compare our method against the renown causal baselines from CausalML (Chen et al.
2020) and Why [Sharma and Kiciman 2020 for the modelling of average treatment effects (ATE) Pearl 2009 Peters et al. 2017] considered to be a gold standard task within causal inference.
Protocol and Parameters. To account for reproducibility and stability of the presented results, we used learned models for five different random seeds per configuration. For means of visual clarity, the competing baselines MDN and MADE only present the best performing seed while iSPN is being presented with a mean plot and the standard deviation. The CBN, which performs exact inference according to the do-calculus (while best performing) has to be considered as gold standard and is therefore not part of the visualization. Furthermore, as cannot compare in terms of feasibility. The considered interventions were of uniform nature. Each block of four columns represents variable being randomized uniformly. We deployed a RAT-SPN [Peharz et al. 2020) selecting the leaf node distributions be Gaussian distributions. For further experimental details consider the Supplement.
8

---

Vi
V/
V.
V4
Caosall Hoatth
ASI4
Patthuute
do 0V63
XV, do(V U(V)))
Figure 4: Generative Baselines. A comparison to the ground-truth (via underlying SCM) and competing estimated distributions. Each row represents a data set and each column represents a variable for given causal query. Best in color.)
Our empirical analysis investigates iSPN for the following questions: Q1: How is the estimation quality for interv. distributions? Q2: How important is the model's s capacity? Q3: How is the runtime performance? Q4/5: How is the performance relative to state-of of-the-art generative and causal models? Q6: How does the model adapt to different types of interventions?
(QL. Precise Estimation of do-influenced variables) The density functions learned by iSPN (see Fig. fit with high degree of precision as visualized by the difference in the peak of the modes of the learned distribution and that of the ground truth. While our visual analysis is arguably the superior method of evaluation as it conveys information about how the distributions of interest compare (which is feasible due to marginal inference and the locality of interventions within a SCM), we have also considered numerical evaluation criteria like the visually confirmed) iSPN outperforms the baselines (see T For more detailled observation and interpretation consider the Supplement.
on which (as
(Q2. Capacity Ablation Study) We test the robustness iSPN as the size of the associated SPN (b) is varied. We obtain 5 different iSPNs for each of the 4 data sets by using 5 different numbers of sum node weights, 600, 1200, 1800, 2400, 3200, effectively changing the capacity of the parameter-sharing neural network f(; We observe iSPNs be robust varying hyper- parameters that control the size of the SPN g(-) and effectively the complexity of the associated function approximator f(-). More details and figures are in the Supplement.
(Q3. Comparison of rumning times) SPNs are tractable by design as long the networks size is polynomial in the input size. The superiority in running time also becomes apparent during training on the same (Causal Health) data set where we observe the mean run times over 50 passes of the whole data set to be significantly faster than competing methods (see Fi
(Q4. Comparison to Generative models) We compare the performances terms of precision of fit of the learned distributions well the flexibility of the models. iSPN outperforms the baselines across all data sets both in precision of the of the learned density functions and flexibility of adaptation to the different interventions seen in figure 4 In experiments, MDN had the worst performance with estimated densities being consistently and significantly different from the ground truth, as the model settled for average distribution across all interventions. MADE is able to estimate to high precisions but showed to be generally inconsistent across the experimental settings.
(Q5. Comparison to Causal models) We compare the numerical results as seen in figure 5 and observe that iSPN matches the performance of the causal baselines. The simple regressor employed by Ca fails in the confounding case as it estimates wrongly the conditional, both Do and iSPN can handle even the more difficult Simpson's 's paradox [Simpson] 195T scenario. analytical derivation for the ATE is exampled in the Supplement.

---

Conforming
Conditioning
Groendl Truth
CanaaML
DoWhy
SPN
No
0.0374
0.C057 [6/4]]
0.0397
0.0397
0.0347
(asla bub)
No
0.9271
0 1.9337
0 9337
0.9337
0.9139
(BTgguggy Alarm)
(9 92348)
Yes
0.6706
0.6703
0.6703
0.6897
0.6551
(broo, dysp)
(1991099
Yes
-0.0457
0.0537
-0.0154
0.0538
0.0645
ATE (Suggry, Recovery)
(6.9g]
ATE(T, E) = E do(T = 1)]
E dT(T = O)]
Figure 5: Causal Baselines. Different causal structures and corresponding causal effect estimation methods (CausaIML, DoWhy) are being compared against iSPN. When confounding is present, then conditioning becomes different from intervening p(Y X) * p(Y do(X)) and iSPN correctly captures all evaluated cases. ( are analytical solutions, differences of means actual interventional distributions, Best viewed in color.)
Log-Likeihoodd
Epochs
(H)ealth
Jobooiiity
Figure 6: Adaptation to Different Interventions. Training results for different kinds of interventions on the continuous CH data set. Left, the respective mean objective curves (log-lik,,,hood), indicating consistent training and convergence for all three random seeds per configuration. Right, the (mean) density functions for two different interventions on H: Uniform U (a, b) and Gamma I(p. q) (other interventions shown in the supplementary). (Best viewed in color.)
(Q6. Different Types of Intervention) By construction, iSPNs are capable to handle arbitrary interventions and our empirical results corroborate this impression. Figló] shows the training of multiple models on different interventions alongside two example interventions and their appearance in the marginal distributions. Depending on the training setup, e.g. how many different interventional distributions need be learned, any single intervention might become more easily optimizable since the model can exploit similarities between distributions. For more detailled elaboration and also vi- sualizations other interventional distributions (including atomic and con-continuous interventions), direct the reader to our extensive supplementary material.
5 Conclusions
We presented way to connect causality with tractable probabilistic models by using sum uun-product networks parameterized by universal function approximators the form of neural networks. We show that our proposed method can adapt to the underlying causal changes in a given domain and generate near perfect interventional distributions irrespective of the data distribution and the intervention type thereby exhibiting flexibility. Our empirical evaluation shows that our method is able to precisely estimate the conditioned variables and outperform generative baselines.
Finding a different compilation method for SPNs such as by making use of tree CBNs is important for learning pure causal probabilistic models. Testing our method on larger real world causal data sets is an interesting direction. Finally, using rich expert domain knowledge in addition to observational data essential for causality and extending our method to incorporate such knowledge is essential.
10

---

Acknowledgments and Disclosure of Funding
The authors thank the anonymous reviewers for their valuable feedback. This work was supported by the ICT-48 Network of AI Research Excellence Center "TAILOR" (EU Horizon 2020, GA No 952215) and by the Federal Ministry of Education and Research (BMBF; project "PlexPlain", FKZ 011S19081 It benefited from the Hessian research priority programme LOEWE within the project WhiteBox, the HMWK cluster project "The Third Wave of AI," and the Collaboration Lab "Al in Construction" (AICO) of the TU Darmstadt and HOCHTIEF. Furthermore, the authors gratefully acknowledge the support RRO1 146 from NICHD and 191 from US Army Research Office (ARO).
References
Jayadev Acharya, Amab Bhattacharyya, Constantinos Daskalakis, and Saravanan Kandasamy. Learn- ing and testing causal models with interventions. NeurlPS, 2018.
Elias Bareinboim and Judea Pearl. Causal inference and the data-fusion problem. PNAS, 2016.
Sander Beckers and Joseph Y Halpem. Abstracting causal models. In AAAI, 2019.
Ioana Bica, Ahmed Alaa, and Mihaela Van Der Schaar. Time series deconfounder: Estimating treatment effects over time in the presence of hidden confounders. In ICML, 2020.
Christopher M Bishop. Mixture density networks. 1994.
Philippe Brouillard, Sébastien Lachapelle, Alexandre Lacoste, Simon Lacoste-Julien, and Alexandre Drouin. Differentiable causal discovery from interventional data. NeurlPS, 2020.
John F Carriger and Mace G Barron. A bayesian network approach refining ecological risk assessments: Mercury and the florida panther (puma concolor coryi). Ecological Modelling, 2020.
Daniel C Castro, Jeremy Tan, Bernhard Kainz, Ender Konukoglu, and Ben Glocker. Morpho-mnist: quantitative assessment and diagnostics for representation learning. Journal of Machine Learning Research, 20(178):1-29, 2019.
Huigang Chen, Totte Harinen, Jeong- Yoon Lee. Mike Yung, and Zhenyu Zhao. Causalml: Python package for causal machine learning. arXiv preprint 1rXiv:2002.11631, 2020.
Arthur Choi and Adnan Darwiche. On the relative expressiveness of bayesian and neural networks. In International Conference on Probabilistic Graphical Models, pages 157-168. PMLR, 2018.
Gregory F Cooper.
The computational complexity of probabilistic inference using bayesian belief networks. Artificial intelligence, 42(2-3):393 405, 1990.
Adnan Darwiche. A differential approach inference in bayesian networks. Journal of the ACM (JACM), 50(3)00000-0))) -055 2003.
Ishita Dasgupta, Jane Wang, Silvia Chiappa, Jovana Mitrovic, Pedro Ortega, David Raposo, Edward Hughes, Peter Battaglia, Matthew Botvinick, and Zeb Kurth-Nelson. Causal reasoning from meta-reinforcement learning. arXiv preprint arXiv: 110/1000088811 /62, 2019.
Norman E Fenton, Martin Neil, Magda Osman, and Scott McLachlan. Covid- 19 infection and death rates: the need incorporate causal explanations for the data and avoid bias in testing. Journal of Risk Research, 2020.
Navid Feroze. Forecasting the patterns of covid-19 19 and causal impacts of lockdown in top five affected countries using bayesian structural time series models. Chaos, Solitons Fractals, 2020.
Mathieu Germain, Karol Gregor, Iain Murray, and Hugo Larochelle. Made: Masked autoencoder for distribution estimation. In International Conference on Machine Learning, pages 881-889. PMLR,
2015.
Clive WJ Granger. Investigating causal relations by econometric models and cross-spectral methods. Econometrica: journal of the Econometric Society, 1969.

---

York Hagmayer, Steven A Sloman, David A Lagnado, and Michael R Waldmann. Causal reasoning through intervention. Causal learning: Psychology, philosophy, and computation, 2007.
Joseph Hair Jr and Marko Sarstedt. Data, measurement, and causal inferences in machine learning: opportunities and challenges for marketing. Journal of Marketing Theory and Practice, 2021.
David Heckerman, Dan Geiger, and David M Chickering. Learning bayesian networks: The combi- nation of knowledge and statistical data. Machine learning, 1995.
William Hoiles and Mihaela Schaar. Bounded off-policy evaluation with missing data for course recommendation and curriculum design. In ICML, 2016.
Agastya Kalra, Abdullah Rashwan, Wei-Shou Hsu, Pascal Poupart, Prashant Doshi, and Georgios Trimponias. Online structure learning for feed-forward and recurrent sum-product networks. NeurIPS, 2018.
Nan Rosemary Ke, Olexa Bilaniuk, Anirudh Goyal, Stefan Bauer, Hugo Larochelle, Bernhard Schölkopf, Michael C Mozer, Chris Pal, and Yoshua Bengio. Learning neural causal models from unknown interventions. arXiv preprint arXiv: 2019.
Doga Kisa, Guy Van den Broeck, Arthur Choi, and Adnan Darwiche. Probabilistic sentential decision diagrams. In KR, 2014.
Murat Kocaoglu, Chris Snyder, Alex Dimakis, and Sriram Vishwanath. Causalgan. 2019.
Daniel Koch, Robert S Eisinger, and Alexander Gebharter. A causal bayesian network model of disease progression mechanisms in chronic mycloid leukemia. Journal of theoretical biology, 2017.
Kevin B Korb and Ann E Nicholson. Bayesian artificial intelligence. CRC press, 2010.
Matt J Kusner, Joshua R Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. NeurIPS, 2017.
Steffen Lauritzen and David Spiegelhalter. Local computations with probabilities on graphical structures and their application expert systems. Journal of the Royal Statistical Society: Series (Methodological), 1988.
Amélie Levray and Vaishak Belle. Learning credal sum-product networks. AKBC, 2020.
Stephen L Morgan and Christopher Winship. Counterfactuals and causal inference. Cambridge University Press, 2015.
Richard E. Neapolitan. Learning buyesian nelworks. Pearson Prentice Hall Upper Saddle River, NJ, 2004.
Mike Oaksford and Nick Chater. Causal models and conditional reasoning. Oxford library of psychology. The Oxford handbook of causal reasoning. 2017.
Michael Oberst and David Sontag. Counterfactual off-policy evaluation with gumbel-max structural causal models. In ICML, 2019.
Ioannis Papantonis and Vaishak Belle. Interventions and counterfactuals in tractable probabilistic models: Limitations of contemporary transformations. arXiv preprint arXiv:2001. 2020.
Iago París, Raquel Sänchez-Cauce and Francisco Javier Diez. Sum Sum-product networks: A survey.
arXiv preprint arXiv:2004.01 2020.
Judea Pearl. From bayesian networks to causal networks. In Mathematical models for handling partial knowledge in artificial intelligence. 1995.
Judea Pearl. Causality. Cambridge university press, 2009.
Judea Pearl. The seven tools of causal inference, with reflections on machine learning. Communica- tions of the ACM, 62(3):54 60, 2019.
12

---

Judea Pearl, Madelyn Glymour, and Nicholas P Jewell. Causal inference in statistics: A primer. John Wiley & Sons, 2016.
Robert Peharz, Antonio Vergari, Karl Stelzner, Alejandro Molina, Xiaoting Shao, Martin Trapp, Kristian Kersting, and Zoubin Ghahramani. Random sum-product networks: A simple and effective approach to probabilistic deep learning. In UAI, 2020.
Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference. The MIT Press, 2017.
Hoifung Poon and Pedro Domingos. Sum-product networks: A new deep architecture. 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pages 689-690. IEEE, 2011.
Dan Roth. On the hardness of approximate reasoning. Artificial Intelligence, 82(1 (2(1-)):733-32, 1996.
Bernhard Schölkopf. Causality for machine learning, arXiv preprint arXiv: 10500, 2019.
Karthikeyan Shanmugam, Murat Kocaoglu, Alexandros G Dimakis, and Sriram Vishwanath. Learning causal graphs with small interventions. NeurlPS, 2015.
Xiaoting Shao, Alejandro Molina, Antonio Vergari, Karl Stelzner, Robert Peharz, Thomas Liebig, and Kristian Kersting, Conditional sum-product networks: Imposing structure on deep probabilistic architectures. arXiv preprint arXiv:1905.08550 2019.
Or Sharir and Amnon Shashua. Sum-product-quotien networks. In AISTATS, 2018.
Amit Sharma and Emre Kiciman. Dowhy: An end-to-end library for causal inference. arXiv preprint 2:Xi::0011.04216, 2020.
Edward H Simpson. The interpretation of interaction in contingency tables. Journal of the Royal Statistical Society: Series (Methodological), 13(2):238-22 2-1,, 1951.
Martin Trapp, Robert Peharz, Hong Ge, Franz. Pernkopf, and Zoubin Ghahramani. Bayesian learning of sum-product networks. NeurIPS, 2019.
Van den Broeck, Di Mauro, and A Vergari. Tractable probabilistic models: Representations, algorithms, learning, and applications. Tutorial at UAI, 2019.
Kevin Xia, Kai-Zhan Lee, Yoshua Bengio, and Elias Bareinboim. The causal-neural connection: Expressiveness, learnability, and inference. 2021.
Kun Zhang, Bernhard Schölkopf, Peter Spirtes, and Clark Glymour. Learning causality and causality- related learning: some recent progress. National science review, 2018.
Han Zhao, Mazen Melibari, and Pascal Poupart. On the relationship between sum-product networks and bayesian networks. In ICML, 2015.
13


Reviews:
[{"review_id": "vmohC5V4qdW", "reviewer": "Reviewer_WdJo", "summary": "The paper introduces iSPNs (a modification of existing SPNs) to model interventions. The authors claim it is the first method that applies the idea of tractable probabilistic models to causality without the need for compilation. ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Since SPNs and Bayesian Networks (BNs) can convert to each other, it is not surprising that SPNs can handle intervention, if the resulting modified network structure by the intervention is carried over to SPNs. This is exactly what iSPNs have done using the updated/modified adjacency matrix that reflects the structure change. Though it is not surprising, but it does have originality to some degree.\n\nWhat is missing in the paper is IF and WHY the proposed iSPNs are more powerful and/or useful than existing causal BNs and other causal models. The content of iSPNs is very small (the definition of iSPNs only starts to appear in page 6). The advantages over other causal models are unclear. As pointed out in [Zhao et al., 2015], with Algebraic Decision Diagrams (ADDs) to compactly represent the local conditional probability distributions at each node in the resulting BN, the advantage of SPNs and subsequently iSPNs is not clear and isn’t sufficiently discussed and analysed in this paper. The experiments are done in very limited small scale toy-like settings. \n\nOverall, the paper has originality, but lacks clarity, quality and significance in its current form. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Discussion for limitations provided. Societal impact is not mentioned. ", "review_text": "", "extra_scores": {"ethical_concerns": "I don't have ethical concerns for the work.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5 hours"}, "extra_text": {}}, {"review_id": "id-BROGU5v7", "reviewer": "Reviewer_4XqK", "summary": "The paper proposes to learn interventional distributions using a family of generative models known as sum-product networks (SPN). SPNs are models based on gate functions that allow for tractable inference. Specifically, the authors propose interventional SPNs (iSPN) which are conditional SPNs (they model conditional distributions) that, licensed by a causal graph and samples from the interventional distribution, are shown to be expressive and causally adequate.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper introduces the use of SPNs for learning interventional distributions from data in the context of Markovian (no hidden variables affecting more than one observable) and with knowledge of the corresponding causal graph. This approach is intended to overcome the shortcoming of related work on translating Bayesian networks into SPNs as it has been shown that this leads to causally inadequate models. \nThe authors show that based on the causal graph, the interventional distributions P(V | do(X=x)) generated for different values of x can be modeled as a conditional distribution in the context of the mutilated graph. Based on this observation, the authors bias the learning process of a conditional SPN to correctly model such interventional distribution for which data is given.\n\nThe definition of iSPNs and their ability to model interventional distributions is well supported by the discussions and results in the paper. The definition of structural causal model used in the paper corresponds to the class of Markovian SCMs where no hidden variable affects more than one observable. However, it is unclear to me why they authors do not consider non-Markovian SCMs since it seems to me that Proposition 1 still holds under this class of models.\n\nOverall, the paper is well organized, necessary concepts are gently introduced and the experimental section serves the purpose of evaluation the benefits of using this approach in terms of efficiency and accuracy, compared to other methods over different datasets.\n\nIn terms of the impact of the work, I wonder what are the current application it could prove significantly advantageous. In particular, since the method assumes that the input data is already interventional, except for some marginalization this input already contains a target of inference such as P(y | do(x)). It seem interesting, however, to consider the possibility of modeling the observational distribution P with an SPN and use it to produce estimates of an identifiable causal effect as it can be expressed as a function of conditionals and marginalizations over P. I understand from the discussion that modeling a joint distribution instead may diminish expressivity but could the authors touch on this in their response?\n\nOther small details/comments:\n- Line 58, 59: repeated \"for\"\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have provided some discussion related to the technical limitations of their method.\nAt this point I can't foresee any societal impact that needs to be addressed by the authors.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}, {"review_id": "cfAcM98zYXX", "reviewer": "Reviewer_UWyh", "summary": "The authors propose (conditional) sum-product networks and the knowledge of the causal graph to estimate the interventional distributions. ", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The idea of training a neural network using conditional distribution and using gates to mimic interventional samples reminds me of CausalGAN by Kocaoglu et al. Please cite and compare by highlighting the differences. Under the same assumptions of the manuscript, it demonstrates that one can learn NN-based causal generative models (Q3+Q4 in experiments) and also obtain interventional samples under the strict positivity assumption. What is the benefit of using SPNs or iSPNs instead? I believe the advantage here would be the estimation of the interventional distribution rather than obtaining samples from it. But nevertheless, a comparison is in order here since one can always estimate the interventional distribution from the samples obtained from CausalGAN. \n\nBayes Net to SPN transformation and relations should be explained. What is the space-complexity time-complexity tradeoff of using an SPN rather than a generative model for the data. \n\nIn general, no insight into SPNs or conditional SPNs are provided. The manuscript should be self contained with a detailed and formal description of these. \n\nMinor comments:\n\"thereby freeing the investigation of iSPN from the independent research around hidden confounding.\"\nHidden confounding is not independent but its non-existence is a rather strong assumption. Please consider paraphrasing this statement. \n\nPost-rebuttal note:\nI am increasing my score based on the engagement of authors and their commitment to include relevant literature and provide clarifications. Unfortunately, there are still several points I am still unclear about which prevent me from recommending acceptance: what kind of guarantee can we give using a random SPN structure? Many claims in the rebuttal seem to heavily rely on past work which are not fully explained in the submission. I recommend the authors clarify the contribution, explain the process of going from causal graph/query to iSPN through examples, and explain the existing work that this paper builds on in more detail. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No. I think the limitations should be more explicitly discussed than being presented as part of future work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "_vMnEupx_hN", "reviewer": "Reviewer_s1DW", "summary": "It considers the problem of learning interventional distributions using sum-product networks (SPNs) that are parameterized by neural networks. They propose interventional sum-product networks (iSPNs) that take advantage of both the expressivity of the neural network and the tractability of the SPN in order to learn the interventional distributions. ", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Learning interventional effect is an important problem that is considered in this paper and has a lot of applications in different fields. \n\nThe paper is well-written but it has room for improvements. For instance, it would improve its readability if there are more detailed information about SPN in the main text (or in the appendix) instead of its background. The importance of learning causal effects (i.e., computing an interventional distribution) is quite clear. Hence, I suggest to reduce such discussions and add more details. As an example, in the definition of iSPN, it is said that g(.,.) is SPN, what does it mean mathematically? How does it look like? Based on figure 2.(b), g(.,.) is a conditional interventional distribution! What is J in this figure?\n\nAs far as I understood, the setting of the paper is when there are no hidden confounders in the causal system. In this case, it is known that every interventional query is identifiable (can be computed from observational distribution). The more interesting setting is of course when there are hidden confounders in which the complete algorithm is proven. How does iSPN perform (if it is applicable) in causal networks with latent confounders? What would be the output when an interventional query is unidentifiable?\n\nHow would this method scale to larger networks?\nmore importantly, how would it generalise to settings with hidden confounders?\n\nIs it possible to compute interventional distributions using iSPN with atomic interventions? \n \nBased on Fig. 5, DoWhy performs better than the iSPN or am I missing something there?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Please see my comments. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers appreciated the use of SPN's for learning interventional distributions from data. Two common criticisms were: (i) the restriction to the Markovian setting (no hidden variables), and (ii) poor readability.

Despite the lack of full support from the reviewers' scores, I am recommending acceptance, because (i) the work addresses a fundamental problem in an original way, and (ii) the authors addressed all the concerns raised by the reviewers. For the next revision, I'd ask the authors to introduce and motivate SPN's more fully (perhaps reducing the text motivating learning of causal effects), and also to emphasize the fact that the work does not address hidden (latent) variables.
Decision: Accept (Poster)