Interventional Sum-Product Networks: Causal Inference with Tractable Probabilistic Models
Authors: Matej Zecevic
Devendra Singh Dhami
Athresh Karanam
Sriraam Natarajan
Kristian Kersting
Venue: NeurIPS 2021 Poster (2021)
Keywords: Interventions
Causality
Tractablity
Probabilistic Models

Abstract: While probabilistic models are an important tool for studying causality, doing so suffers from the intractability of inference. As a step towards tractable causal models, we consider the problem of learning interventional distributions using sum-product networks (SPNs) that are over-parameterized by gate functions, e.g., neural networks. Providing an arbitrarily intervened causal graph as input, effectively subsuming Pearl's do-operator, the gate function predicts the parameters of the SPN. The resulting interventional SPNs are motivated and illustrated by a structural causal model themed around personal health. Our empirical evaluation against competing methods from both generative and causal modelling demonstrates that interventional SPNs indeed are both expressive and causally adequate.

Interventional Sum-Product Networks: Causal Inference with Tractable Probabilistic Models
Matej Zečević!
Devendra Singh Dhami'
Athresh Karanam²
Sriraam Natarajan
Kristian Kersting!
'Computer Science Department, TU Darmstadt Computer Science Department, The University of Texas at Dallas CCnntre for Cognitive Science, TU Darmstadt, and Hessian Center for Al (hessian. Al) {matej zecevic, devendra. aevendra.dhami, dhami, kersting tu-darmstadt. de {athresh. karanam, sriraam.... wwtemmaaaaaaaaaaaa jatarajan/@utdallll edu
Abstract
While probabilistic models are an important tool for studying causality, doing suffers from the intractability of inference. As step towards tractable causal models, we consider the problem of learning interventional distributions using sum- product networks (SPNs) that are over-parameterized by gate functions, e.g., neural networks. Providing an arbitrarily intervened causal graph as input, effectively subsuming Pearl's do-operator, the gate function predicts the parameters of the SPN. The resulting interventional SPNs are motivated and illustrated by structural causal model themed around personal health. Our empirical evaluation against competing methods from both generative and causal modelling demonstrates that interventional SPNs indeed are both expressive and causally adequate.
Introduction
Identifying causal relationships between variables in observational data is of the fundamental
and well-studied problem in machine learning. There have been several great strides in causality
[Granger] 1969 Pearl 2009 [areinboim and Pearl 2016 over the years specifically characterized by
efforts that focused on reasoning about interventions Hagmayer et al. 2007 Dasgupta et al.) 2019
and counterfactuals [Morgan and Winship 2015 Oberst and Sontag 2019).
The notion of causality has long been explored in the realm of probabilistic models Oaksford
and Chater 2017 Beckers and Halpern| 2019] with special focus graphical models, called
causal Bayesian networks (CBNs) [Heckerman et al.) 1995] Neapolitan 2004 Pearl [993] Acharya
et al. 2018). CBNs have widely been applied to infer causal relationships in high-impaet diverse
applications such as disease progression (Koch et al. 2017), ecological risk assessment [Carriger and Barron 2020 and more recently Covid-19 [Fenton al. 2020 Feroze 20200 to name few. Although successful, classical CBN models are difficult to scale and also sutter from the problem of intractable inference. Recently, tractable probabilistic models such as probabilistic sentential decision diagrams [Kisa et al.) (0144) and sum con-prrduct networks (Poon and Domingos/ 2011] have emerged, which guarantee that conditional marginals can be computed in time linear in the of the model. While weaving in the notion of interpretability, the computational view probabilistic models allows one to exploit ideas from deep learning and can thus be very useful modelling complex problems.
Recently, there has been an effort to take advantage of this tractability to reason for causality. Zhao al. [2015) showed how compile back and forth between sum-product networks (SPNs) and Bayesian networks (BNs). Although this opened up a whole range of possibilities for tractable
35th Conference on Neural Information Processing Systems (NcurIPS 2021), virtual.

---

O
B
seagss Mode
Ground Truhh
do O = B
Figure I: Capturing interventional distributions using iSPN. The interventional distributions for the ASIA data set using a causal Bayesian network (CBN, small-scale gold standard, gray bars) as well as an interventional SPN (iSPN) by intervening on lung. The iSPN is sensible to all the influences of the given intervention onto the system i.e., subsequent effects in the causal hierachy. (Best viewed color.)
causal models, Papantonis and Belle 2020 argued that such conversion leads to degenerated BNs thereby rendering it ineffective for causal reasoning. For the considered compilation of SPNs to BNs, this is indeed the case since a bipartite graph between the hidden and observed variables loses the relationships between the actual variables. Thus, either a new compilation method for transforming between tractable and causal model or alternatively method to condition the probabilistic models directly on the do-operator to obtain interventional distributions, P do(x)), is being required.
Here, we consider the latter strategy and extend the idea of conditionally parameterizing SPNs (Shao et al. 2019] by conditioning on the do-operator while predicting the complete set of observed variables therefore capturing the effect of intervention(s). The resulting interventional sum-product networks (iSPNs) take advantage of both the expressivity, due to the neural network, and the tractability, due the SPN in order capture the interventional distributions faithfully. This shows that the dream of tractable causal models is not insurmountable, since iSPNs are causal. Pearl [2019] defined three-level causal hierarchy that separates association (purely statistical level 1) from intervention (level 2) and counterfactuals (level 3), and argued that the latter two levels involve causal inference. iSPNs are a modelling scheme for arbitrary interventional distributions. They thus belong to level and, in turn, are causal. So, while SPNs are not universal function approximators and the use of gate functions turns them into universal approximators, we go one step ahead and make the first effort towards introducing causality to SPNs without the need for compilation to Bayesian networks, as the functional approximator subsumes the do-operator. Fig. shows an example of the effectiveness of iSPNs to capture interventional distributions on the ASIA data set. Our extensive experiments against strong baselines demonstrate that our method is able to capture ideal interventional distributions.
To summarize, we make the following contributions:
1. We introduce iSPNs, the first method that applies the idea of tractable probabilistic models to causality without the need for compilation and that accordingly generate interventional distributions w.r.t. the provided causal structure.
2. We formulate the inductive bias necessary for turning conditional SPN into iSPN while taking advantage of the neural network modelling capacities within the gating nodes for for modelling interventional distributions while capturing all influences within the given intervention (i.e., consequences propagated through the structural hierarchy).
3. We show that, by construction, iSPNs can identify any interventional distribution permitted by the underlying structural causal model due to the inducted bias on the interface modalities in junction with the universal function approximation of the underlying gated SPN.
We proceed as follows. We start by reviewing the basic concepts required and related work, namely the tractable model class of sum-product networks and key concepts from causality. Then we motivate using newly curated causal model themed around personal health. Subsequently, we introduce iSPNs formally and prove that by construction they are capable of approximating any do-query (given corresponding data). Before concluding, present experimental evaluation in which

---

challenge iSPN in its density estimation and causal inference capabilitiess against various baselines. We make our code publically available at: htttpe///gittub..cmm///coooo www.://woccoooo//.o.coo/coocc.cc
2 Background and Related Work
Let us briefly review the background on tractable probabilistic models and causal models used in subsequent sections for developing our new model class based on CSPNs that allow for identifying causal quantities i.e., interventional distributions.
Notation. We denote indices by lower-case letters, functions by the general form g()), scalars or random variables interchangeably by upper-case letters, vectors, matrices and tensors with different boldface font v, V, V respectively, and probabilities of a set of random variables X as pXX).
Sum-Product Networks (SPNs). Introduced by Poon and Domingos 201 generalizing the notion of network polynomials based on indicator variables Xx== (x) E [0 for (finite-state) RVs X from Darwiche 2003]. Sum-Product Networks (SPNs) represent special type of probabilistic model that allows for a variety of exact and efficient inference routines. Generally, SPNs are considered as directed acyclic graphs (DAG) consisting of product, sum and leaf (or distribution) nodes whose structure and parameterization can be efficiently learned from data to allow for efficient modelling of joint probability distributions p(X). Formally SPN S (G, w) consists of non-negative parameters w and DAG G = (V, B) with indicator variable leaf nodes and exclusively internal sum and product nodes given by,
S(A)
Σ Ws,cC(A) P(A) CEch(S)
II C(A), CEch(S)
where the SPN output S is computed the root node (S(A) S(x)) and the probability density
for x is p(x) ee S(x''' They are members of the family of probabilistic circuits [Van den
S(x)
Broeck et al. 2019]. A special class, to be precise. that satisfies properties known completeness and decomposability. Let N denote a node in SPN S, then
{ {x} 10--UUe UcEchiN se(C)
i is (Ax=a)
sc(N)
(2)
else
is called the scope of N and
VS E S: (VC,, C2 E ch((): sc(Ci) = sc(C2)) vP E S: (VC1, C2 E ch(S): C, C2 sc(C)) = 0)
(3)
(4)
the completeness and decomposability properties respectively. Since their introduction, SPNs have been heavily studied such as by Trrppp et al. 2019] that present a way to learn SPNs in a
Bayesian realm whereas (Kalra et al. 2018] learn SPNs in online setting. Several different types of SPNs have also been studied such as Random SPN [Peharz et al.) 2020) Credal SPNs [Levray and Belle 2020) and ceae--ooout-uottiiie
Networks (Sharir and Shashua 2018) name a few. For more details readers are referred to the survey of París, Sánchez-Cauce, and Diez 2020)] On another note, Gated or Conditional SPNs (CSPNs)
deep tractable models for estimating multivariate, conditional probability distributions Y X) over mixed variables Y (Shao et al. 2019 They introduce functional gate nodes g (X) that act as functional parameterization of the SPN's information flow and leaf distributions given the provided evidence X.
Causal Models. A Structural Causal Model (SCM) as defined by Peters al. 2017 is specified as C = (S, PN)) where PN is product distribution over noise variables and S is defined to be a set of d structural equations
X, fe(pa(())) No), where i 1,...,d
with pa(X.) representing the parents of X; in graph G(C). An intervention SCM e as defined in (5) occurs when (multiple) structural equations are being replaced through new non-parametric functions (pa(XX)) N:) thus effectively creating an alternate SCM é. Interventions are referred to imperfect if pa(X.) pa(X.) and as atomic if f for R. An important property
3

---

of interventions often referred to as "modularity" or "autonomy") states that interventions are fundamentally of local nature, formally
U
p (x, pa((x))) = -x; pa(X;))
(6)
where the intervention of E occured on variable X5 opposed to X,. This suggests that mechanisms remain invariant to changes in other mechanisms which implies that only information about the effective changes induced by the intervention need to be compensated for. An important consequence of autonomy is the truncated factorization
PVV) = II
s p(X, pa(X.))
(7)
derived by Pearl [2009], which suggests that an intervention S introduces an independence of an intervened node X; to its causal parents. Another important assumption in causality is that causal mechanisms do not change through intervention suggesting notion of invariance to the cause-effect relations variables which further implies an invariance to the origin of the mechanism i.c., whether it occurs naturally or through means of intervention [Pearl et al. 2016].
SCM capable of emitting various mathematical objects such as graph structure, statistical and causal quantities placing it at the heart of causal inference, rendering it applicable to machine learning applications in marketing (Hair and Sarstedt 202T]), healthcare [Bica et al. (0200) and education [Hoiles and Schaar, 2016)_ A SCM induces a causal graph G, an observationaallssooiiiiii distribution p., can be intervened upon using the do-operator and thus generate interventional distributions p (iioo(..) and given some observations v can also be queried for interventions within system with fixed noise terms amounting to counterfactual distributions y \vvdooo)).... To query for samples of given SCM, the structural equations are being simulated sequentially following the underlying causal structure starting from independent, exogenous variables and then moving along the causal hierarchy of endogenous variables (i.c.. following the causal descendants).
The work closest to our work is by Brouillard et al. 2020 although it solves the different problem of causal discovery. Causality for machine learning has recently gained a lot of traction (Schölkopf) 2019] with the study of both interventions (Shanmugam et al. 2015] and counterfactuals Kusner et al. 2017] gaining speed. For more details the readers are referred to [Zhang et al.) 2018
3 Interventional SPNs
Now we are ready to develop interventional SPNs (iSPNs). To this end, we re-introduce the impor- tance of adaptability of models interventional queries and present a newly curated synthetic data set to both motivate and validate the formalism of iSPNs that, through over-parametric extension of SPNs, allows them to adhere causal quantities.
3.1 Adaptation to Causal Change
Peters et al. [0177] motivated the necessity of causality via the adequate generalizability of predictive models. Specifically, consider a simple regression problem f(a) = b with data vectors a, e I that are strongly positively correlated in some given region, e.g. (1
co, 1 < b co). Now query is posed outside the data support, e.g. (0, (0)). As argued by Peters et al. the underlying data
generating processes be ambiguous causal process i.e., the data hand can be explained by two different causal structures being either A + B or a common confounder with A + C B.
Assuming the wrong causal structure or ignoring it altogether could be fatal, therefore, any form of generalization out of data support requires assumptions to be made about the underlying causal structure. We adopt this point of view and further argue that ignoring causal change(s) in a system, i.e.. the change of structural equation(s) s) underlying the system, can lead to a significant performance decrease and safety hazards" Therefore, it is important to account for distributional changes present in the data due to experimental (thus interventional) settings.
See Section 6.6 [Peters et al. 2017
"This extended notion of performance degeneration through ignorance to the underlying causality is priori- tized in this paper.

---

(a.
(b)
Dronctinal Consal Mare!
lay ATOLL
F(0,0)
D geay
1-00000
0100000
c
xXO
XO. SIGHt - 1001)
(O0 HKP D1N 310030000
POK 160 010 (A1)
Age
Food mobes
Figure An Overview iSPN. (a) The hidden process underlying the observable reality is modelled via SCM that can be modified through interventions. The given SCM induces a causal graph and can generate data. An intervention can significantly alter the resulting data. (b) An over-parameterized density estimation framework using SPN is presented. The universal function approximator (here neural network) conditions on the mutilated causal graph and provides parameters to the SPN such that the given data's 's density be modelled accordingly. The FA subsumes the do-operator. (c) Different causal queries are being presented. Furthermore, iSPN adapts to mnarvention-consqquenee
Consequently, we consider the learning problem, where the given data samples have been generated by different interventions do(U, = u,) in a common SCM e while the induced mutilated causal graphs GCC, do( ) are assumed to be known, such that the trained model is capable of at least inferring all involved causal distributions pCV(C) do(U, = u,)) with V being the variables.
3.2 Data Generating Process
To both validate and demonstrate the expressivity of interventional SPNs in modelling arbitrary interventional distributions, we curate a new causal data set based on the SCM e presented in Fig. aa),, which subsequently refer as Causal Health data set. The SCM encompasses four different structural equations the form V = f {pa Vi), N₂). where pa(Vi) are the parents of variable V, and N; are the respective noise terms that form a factor distribution PNio i.c. the N; are jointly independent. Now, the SCM e describes the causal relations an individual's health and mobility attributes with respect their age and nutrition.
Note that the Causal Health data set does not impose assumptions over the type of random variables or functional domains of the structural equation which additionally constraints learned model to adapt flexibly. While we generally not restrict our method to any particular type of intervention, the following mainly considers perfect interventions as introduced in Sec. 2
Perfect interventions fully remove the causal mechanism of the parents of given node, which is consistent with the idea behind randomized controlled trials (RCTs) where the given intervention randomizes the given specific node, often referred gold standard in causality related literature. We consider the special case of uniform randomization, i.e., uniform across the domain of the given variable") An intervention performed on one node immediately changes the population and, thus, has major effect on the generating processes of subsequent causal mechanisms in the respective causal sequence of events.
To provide the reader with a concrete example of interventions within the causal health data set, consider the following: In concern of virus infection the individuals of the Causal Health study should be vaccinated. The vaccine is expected to have side-effect(s), however, it has been poorly designed and has reached the population with the capability of completely changing the individual's
Assumptions on the func. form structural eq. are crucial for identification (Tab. 7.1 Peters et al. 2017 Note that for binary variables this amounts to Bernoulli (
5

---

health state. The observed changes do not show any form of pattern and are therefore assumed to be random. A young fit person could thus become sick, while an old person might feel better health wise. This sudden change will have an effect on the individual's mobility and also be independent of their age and nutrition. Such scenario is mathematically being captured through do (H)) where U-) is the uniform distribution over given domain.
3.3 Introducing Interventional SPNs
After motivating both the importance and the occurrences of interventions within relevant systems, we now start introducing interventional SPNs (iSPNs).
Definition of iSPN. As motivated in Sec. I the usage of the compilation method from [hhao et al. 2015) for causal inference within SPN is arguably of degencrate nature given the properties of the compilation method [Papantonis and Belle, 2020]. While the results of Pappanonnis and Belle (202)) arguably negative, there exists yet no proof of non-existence of such compilation method and as the authors point out in their argument for future lines of research in this direction, a model class extension poses a viable candidate for overcoming the problems of using SPN for causal inference.
While agreeing on the latter aspect, we do not go the "compilation road" but extend the idea of conditional parameterization for SPN [Shao et al. 2019] by conditioning on a modified form of the do-operator introduced by Pearl (2009) while predicting the complete set of observed variables.
Mathematicallyy, we estimate p(V; do(U, = uy)) by learning non- parametric function approxima- tor f(; 0) (e.g, neural network), which takes as input the (mutilated) causal graph G E {0, 1 encoded as an adjacency matrix, to predict the parameters vp of SPN gDD Φb) that estimates the density of the given data matrix {V& = =D E RKN, With this, iSPNs are defined as follows:
Definition (Interventional is the joint model m D) = g(D; & = ((G; 0)), where g(-) is a SPN, J(.) a non-parametric function appreximator and ab (G) are shared parameters.
Network). An interventional sum- cum-product network (iSPN)
They are called interventional because consider it to be causal model given its capability of answering queries from the second level of the causal hierarchy (Pearl, 2019]. namely, that of interventions The shared parameters &b allow for information flow during learning between the conditions and the estimated densities. Setting the conditions such that they contain information about the interventions, in the form of the mutilated graphs G, effectively renders f to subsume a sort of do-calculus in the spirit of truncated factorization shown in Eq. i.e., the gate model acts as an estimand selector. Generally, we note that our formulation allows for different function and density estimators J.g. We choose I to be neural network for two reasons (1) their empirically established capability to act as causal sub-modules (e.g. Ke et al. [2099] use a cohort of neural nets to mimic a set of structural equations, thus, SCM) and (2) their model capacity being a universal function approximator, while we choose g to be a SPN for its tractability properties for inference.
We have argued the importance of adaptability to interventional changes within the causal system and intend now to prove that iSPN are capable of approximating these different causal quantities.
Proposition (Expressivity). Assuming autonomy and invariance, an iSPN mGG, D) is able to identify any interventional distribution (V =. do(U, = =u,)). permitted by a SCM E through interventions, with knowledge of the mutilated causal graph G and data generated from the intervened SCMs by modelling the conditional distribution Pa (V. -v; VV = Uz))
Proof. It follows directly from the definition of the do-calculus PParl 2009 that PG V, = vi
do = uz))= = P v; U,= = uz)) where & is the mutilated causal graph according to the intervention do, = uy) i.e., observations in the intervened system are akin to observations made when intervening on the system. Given the mutilated causal graph G (as adjacency matrix), the only remaining aspect to show is that the density estimating SPN can approximate a joint probability p( (X)
using D. This naturally follows from [Ponn and Domingos 2011].
5 A hipartite graph in which the actual variables of interest are not connected called degenerate.
'The first level of the causal hierarchy association is considered to be purely statistical.

---

The expressivity of iSPN stems from both the capacities of gate function and the knowledge of intervention as well as availability of respective data. As an important remark, causal inference is often interested in estimating interventional distributions, i.e, causal quantities from purely observational models. Therefore, an alternative formulation to Prop. [ would be to replace the knowledge of the intervened causal structure G with knowledge on valid adjustment set. In the following, we only consider the direct setting where actual interventional data from the system is assumed to be captured thereby freeing the investigation of iSPN from the independent research around hidden confounding.
Universal Function Approximation (UFA). The gating nodes of CSPNs extend SPNs in way that allows them to also induce functions which are universal approximators. For instance, using threshold gates z; I c E E. one can realize testing arithmetic circuits (Choi and Darwiche, 2018) which have been proven to be universal approximators rendering iSPN to be UFA by construction.
Learning of iSPN. An interventional sum-produet network is being learned using a set of mixed- distribution samples generated from simulating the Causal Health SCM for different interventions, where the observational case is considered to be equivalent to an intervention on the empty set. The parameters 0, 2 of the iSPN describe the weights of the gate nodes and the distributions at the leaf nodes. The full model m is differentiable if the provided gate function f and each of the leaf models of are differentiable. Therefore, to train an iSPN, as depicted in Fig. 2.b). it is sufficient to optimize the conditional log- -ikeeeildooo end-to-end using gradient based optimization techniques. We assess the performance of our learned model through inspection of the adaptation of the model to the different interventions manifesting in the resulting marginals pVV do(U,)).
As can be observed in Fig. 2 or alternatively in Fig. (top row), the learned iSPN successfully adapts both the interventions as well as its consequences. Considering for instance the intervention p(V: do(F" = {}))) which removes the edge A + F' and thus renders Age (A) and Food Habits (F) independent. Given the drastic population change in F and the fact that the Health (!) of an individual is causally dependent on both A and F significant change in H is being expected. Indeed, both H and Mobility (M), being a causal child of H, broaden distribution wise and also these subsequent changes are captured correctly.
graeriii-gg do-calculus [Pearl 2009 is complete in that it can find estimands for any identifiable query in finite amount of transformation. An SPN, and thereby also iSPN, is capable of providing estimates to said demands when given observational data (L1 on the PCH). Consider for instance the well-known non-Markovian "Napkin" graph G = (W - Z -
Causal Estimation. The
YYY 4 X, W.W& Y) where < denotes a confounded relation. The causal effect is identified as ply do(z)) = (E p(ar p(r/ =)p(w))- '(E.. my p(y, x 2//,,2))p =))(w)) using the do-calculus where each of the r.h.s. components can be modelled by (i)SPN respectively. However, iSPN also capable of directly expressing the causal quantity pvy do (z))) as proven in Prop [ similar to other neural-causa
models like CausalGAN Kocaoglu et al. 2019 MLP--baeed NCM Xia et al. 2021).
Tractability, Inference in BNs and Markov networks is at least an NP-problem and existing exact algorithms have worst-case exponential complexity [Cooper] 1990 Roth 1996]. while SPNs can do inference in time proportional to the number of links in the graph. L.e., SPNs in general are able to compute any marginalization and conditioning query in time linear of the model's representation size r, that (r). We deploy simple neural network architectures for which the runtime for the forward seraadd-pass pass is that of matrix- mntrrx-mulliplication i.e., the time complexity scales cubically in the size of the input n, that is (n²)). The gradient descent procedure that involves forward and backward passes, assuming m gradient descent iterations, scales to Omm)3) which is the overall complexity we achieve during training phase. For SPN we deploy a random SPN structure which circumvents structure learning such. Therefore, overall, training will generally be in the 0(mrer)) regime while any causal query (that is, both L1 observational and L2 interventional in our case) will be answered within On""r). However, assuming that the weights of the random SPN were already initialized by the neural parvneter-provider in a previous step, any causal query becomes answerable in ((r). Since asking for inferences/queries q within single distribution (q - d E La) more common than changes between distributions (d, d C Li), this linear complexity by our tractable model being leveraged fully for performing causal inference.
Discussion. To reconsider and answer the general question of why the modelling of a conditional distribution via an over parammmteeter architecture is a sensible idea consider the following. One can
For details on the remarked alternative formulation consider the Supplement.
7

---

Query
V11
V/2
V/3
1/4
Method
ISPN
0011 # 000
.007 = 01
.003 + 000
0133 .01
MADE
588 + .59
.108 = 16
.015 + 022
.105 + .12
MDN
118
.263
= 14
.184 + .12 .079 + .01
Table 1: Jensen-Shannon--Dinn Shannon-Divergence Evaluation of Estimated
Figure 3: Mean Running Times in sec. till convergence (Causal Health) for 50 full passes. More data sets results in supplementary.
Interventional Distributions. Numerical pendant to Fig 4 mean and standard deviation per p p(V do (V; = ((V))) where U
the uniform distribution across all data sets, Lower=better.
represent a conditional distribution p( pYY X) by applying Bayes Rule to joint distribution density model (e.g. regular SPN) p(Y X) = p(Yx))
p(X) However, this assumes non-empty support i.e., pXX) 0. Furthermore, the joint distribution p(Y, X) optimizes all possibly derivable distributions, diminishing single distr. expressivity. Therefore, our considered formulation of gate model allows for effectively subsuming the do-operator i.e., the gate model orchestrates the do queries such that the density estimator can easily switch between different interventional distributions. While not introducing specific limitations, general CSPN limitation regarding OOD generalization are inherited.
Experimental Results
The assumptions made in causality usually require control over the data generating process which almost never readily available the real world. This amounts to scarcity of the available public data sets and also their implications for transfers to the real world and even when available, they are usually artificially generated as some causal extension of a known, pre-existing data set (e.g. MorphoMNIST data set introduced by Castro et al. (2019]). While it is difficult to consider real- world-esque experimental settings for causal models, we do not restrict our investigations of iSPN to rather specific problem settings like certain noise decision variable instances (which is common in causal inference). The evaluation is performed on data sets with varying number of variables and in both continuous and discrete domains. For the introduced causal health data set, we even consider arbitrary underlying noise distributions. We have made our code repository publicly available
Data Sets. We evaluate iSPNs four data sets. Three benchmarks: ASIA (A) [Lauritzen and Spiegelhalter 1988) with variables, Earthquake (E) with variables and Cancer (C) Korb and Nicholson 2010)] with 5 variables. One newly curated synthetic causal health (H) data set with 4 variables. More information about the data sets is presented in the Supplement.
Baselines. For generative capacities, we compare our method against Mixture Density Networks (MDN) [Bishop] 1994) and Masked Autoencoder for Density Estimation (MADE) (Germain et al.. 2015]. Both methods are expressive, parametric neural network based approaches for density estimation. Generally, the causality for machine learning literature suggests strong favor for neural based function approximators for modelling causal mechanisms [Ke et al. [2019]. For causal capacities, we compare our method against the renown causal baselines from CausalML (Chen et al.
2020) and Why [Sharma and Kiciman 2020 for the modelling of average treatment effects (ATE) Pearl 2009 Peters et al. 2017] considered to be a gold standard task within causal inference.
Protocol and Parameters. To account for reprodueibility and stability of the presented results, we used learned models for five different random seeds per configuration. For means of visual clarity, the competing baselines MDN and MADE only present the best performing seed while iSPN is being presented with a mean plot and the standard deviation. The CBN, which performs exact inference according to the do-calculus (while best performing) has to be considered as gold standard and is therefore not part of the visualization. Furthermore, as cannot compare in terms of feasibility. The considered interventions were of uniform nature. Each block of four columns represents variable being randomized uniformly. We deployed a RAT-SPN [Peharz et al. 2020) selecting the leaf node distributions be Gaussian distributions. For further experimental details consider the Supplement.
8

---

Vi
V/
V.
V4
Caosall Hoatth
ASI4
Patthuute
do 0V63
XV, do(V U(V)))
Figure 4: Generative Baselines. A comparison to the ground-truth (via underlying SCM) and competing estimated distributions. Each row represents a data set and each column represents a variable for given causal query. Best in color.)
Our empirical analysis investigates iSPN for the following questions: Q1: How is the estimation quality for interv. distributions? Q2: How important is the model's s capacity? Q3: How is the runtime performance? Q4/5: How is the performance relative to state-of of-the-art generative and causal models? Q6: How does the model adapt to different types of interventions?
(QL. Precise Estimation of do-influenced variables) The density functions learned by iSPN (see Fig. fit with high degree of precision as visualized by the difference in the peak of the modes of the learned distribution and that of the ground truth. While our visual analysis is arguably the superior method of evaluation as it conveys information about how the distributions of interest compare (which is feasible due to marginal inference and the locality of interventions within a SCM), we have also considered numerical evaluation criteria like the visually confirmed) iSPN outperforms the baselines (see T For more detailled observation and interpretation consider the Supplement.
on which (as
(Q2. Capacity Ablation Study) We test the robustness iSPN as the size of the associated SPN (b) is varied. We obtain 5 different iSPNs for each of the 4 data sets by using 5 different numbers of sum node weights, 600, 1200, 1800, 2400, 3200, effectively changing the capacity of the parameter-sharing neural network f(; We observe iSPNs be robust varying hyper- parameters that control the size of the SPN g(-) and effectively the complexity of the associated function approximator f(-). More details and figures are in the Supplement.
(Q3. Comparison of rumning times) SPNs are tractable by design as long the networks size is polynomial in the input size. The superiority in running time also becomes apparent during training on the same (Causal Health) data set where we observe the mean run times over 50 passes of the whole data set to be significantly faster than competing methods (see Fi
(Q4. Comparison to Generative models) We compare the performances terms of precision of fit of the learned distributions well the flexibility of the models. iSPN outperforms the baselines across all data sets both in precision of the of the learned density functions and flexibility of adaptation to the different interventions seen in figure 4 In experiments, MDN had the worst performance with estimated densities being consistently and significantly different from the ground truth, as the model settled for average distribution across all interventions. MADE is able to estimate to high precisions but showed to be generally inconsistent across the experimental settings.
(Q5. Comparison to Causal models) We compare the numerical results as seen in figure 5 and observe that iSPN matches the performance of the causal baselines. The simple regressor employed by Ca fails in the confounding case as it estimates wrongly the conditional, both Do and iSPN can handle even the more difficult Simpson's 's paradox [Simpson] 195T scenario. analytical derivation for the ATE is exampled in the Supplement.

---

Conforming
Conditioning
Groendl Truth
CanaaML
DoWhy
SPN
No
0.0374
0.C057 [6/4]]
0.0397
0.0397
0.0347
(asla bub)
No
0.9271
0 1.9337
0 9337
0.9337
0.9139
(BTgguggy Alarm)
(9 92348)
Yes
0.6706
0.6703
0.6703
0.8897
0.6551
(broo, dysp)
(1991099
Yes
-0.0457
0.0537
-0.0154
0.0538
0.0645
ATE (Suggry, Recovery)
(6.9g]
ATE(T, E) = E do(T = 1)]
E dT(T = O)]
Figure 5: Causal Baselines. Different causal structures and corresponding causal effect estimation methods (CausaIML, DoWhy) are being compared against iSPN. When confounding is present, then conditioning becomes different from intervening p(Y X) * p(Y do(X)) and iSPN correctly captures all evaluated cases. ( are analytical solutions, differences of means actual interventional distributions, Best viewed in color.)
Log-Likeihoodd
Epochs
(H)ealth
Jobooiiity
Figure 6: Adaptation to Different Interventions. Training results for different kinds of interventions on the continuous CH data set. Left, the respective mean objective curves (log-lik,,,hood), indicating consistent training and convergence for all three random seeds per configuration. Right, the (mean) density functions for two different interventions on H: Uniform U (a, b) and Gamma I(p. q) (other interventions shown in the supplementary). (Best viewed in color.)
(Q6. Different Types of Intervention) By construction, iSPNs are capable to handle arbitrary interventions and our empirical results corroborate this impression. Figló] shows the training of multiple models on different interventions alongside two example interventions and their appearance in the marginal distributions. Depending on the training setup, e.g. how many different interventional distributions need be learned, any single intervention might become more easily optimizable since the model can exploit similarities between distributions. For more detailled elaboration and also vi- sualizations other interventional distributions (including atomic and con-continuous interventions), direct the reader to our extensive supplementary material.
5 Conclusions
We presented way to connect causality with tractable probabilistic models by using sum uun-product networks parameterized by universal function approximators the form of neural networks. We show that our proposed method can adapt to the underlying causal changes in a given domain and generate near perfect interventional distributions irrespective of the data distribution and the intervention type thereby exhibiting flexibility. Our empirical evaluation shows that our method is able to precisely estimate the conditioned variables and outperform generative baselines.
Finding a different compilation method for SPNs such as by making use of tree CBNs is important for learning pure causal probabilistic models. Testing our method on larger real world causal data sets is an interesting direction. Finally, using rich expert domain knowledge in addition to observational data essential for causality and extending our method to incorporate such knowledge is essential.
10

---

Acknowledgments and Disclosure of Funding
The authors thank the anonymous reviewers for their valuable feedback. This work was supported by the ICT-48 Network of AI Research Excellence Center "TAILOR" (EU Horizon 2020, GA No 952215) and by the Federal Ministry of Education and Research (BMBF; project "PlexPlain", FKZ 011S19081 It benefited from the Hessian research priority programme LOEWE within the project WhiteBox, the HMWK cluster project "The Third Wave of AI," and the Collaboration Lab "Al in Construction" (AICO) of the TU Darmstadt and HOCHTIEF. Furthermore, the authors gratefully acknowledge the support RRO1 246 from NICHD and 191 from US Army Research Office (ARO).
References
Jayadev Acharya, Amab Bhattacharyya, Constantinos Daskalakis, and Saravanan Kandasamy. Learn- ing and testing causal models with interventions. NeurlPS, 2018.
Elias Bareinboim and Judea Pearl. Causal inference and the data-fusion problem. PNAS, 2016.
Sander Beckers and Joseph Y Halpem. Abstracting causal models. In AAAI, 2019.
Ioana Bica, Ahmed Alaa, and Mihaela Van Der Schaar. Time series deconfounder: Estimating treatment effects over time in the presence of hidden confounders. In ICML, 2020.
Christopher M Bishop. Mixture density networks. 1994.
Philippe Brouillard, Sébastien Lachapelle, Alexandre Lacoste, Simon Lacoste-Julien, and Alexandre Drouin. Differentiable causal discovery from interventional data. NeurlPS, 2020.
John F Carriger and Mace G Barron. A bayesian network approach refining ecological risk assessments: Mercury and the florida panther (puma concolor coryi). Ecological Modelling, 2020.
Daniel C Castro, Jeremy Tan, Bernhard Kainz, Ender Konukoglu, and Ben Glocker. Morpho-mnist: quantitative assessment and diagnostics for representation learning. Journal of Machine Learning Research, 20(178):1-29, 2019.
Huigang Chen, Totte Harinen, Jeong- Yoon Lee. Mike Yung, and Zhenyu Zhao. Causalml: Python package for causal machine learning. arXiv preprint 1rXiv:2002.11631, 2020.
Arthur Choi and Adnan Darwiche. On the relative expressiveness of bayesian and neural networks. In International Conference on Probabilistic Graphical Models, pages 157-168. PMLR, 2018.
Gregory F Cooper.
The computational complexity of probabilistic inference using bayesian belief networks. Artificial intelligence, 42(2-3):393 405, 1990.
Adnan Darwiche. A differential approach inference in bayesian networks. Journal of the ACM (JACM), 50(3)00000-0))) -05, 2003.
Ishita Dasgupta, Jane Wang, Silvia Chiappa, Jovana Mitrovic, Pedro Ortega, David Raposo, Edward Hughes, Peter Battaglia, Matthew Botvinick, and Zeb Kurth-Nelson. Causal reasoning from meta-reinforcement learning. arXiv preprint arXiv: 110/1000088811 /62, 2019.
Norman E Fenton, Martin Neil, Magda Osman, and Scott McLachlan. Covid- 19 infection and death rates: the need incorporate causal explanations for the data and avoid bias in testing. Journal of Risk Research, 2020.
Navid Feroze. Forecasting the patterns of covid-19 19 and causal impacts of lockdown in top five affected countries using bayesian structural time series models. Chaos, Solitons Fractals, 2020.
Mathieu Germain, Karol Gregor, Iain Murray, and Hugo Larochelle. Made: Masked autoencoder for distribution estimation. In International Conference on Machine Learning, pages 881-889. PMLR,
2015.
Clive WJ Granger. Investigating causal relations by econometric models and cross-spectral methods. Econometrica: journal of the Econometric Society, 1969.

---

York Hagmayer, Steven A Sloman, David A Lagnado, and Michael R Waldmann. Causal reasoning through intervention. Causal learning: Psychology, philosophy, and computation, 2007.
Joseph Hair Jr and Marko Sarstedt. Data, measurement, and causal inferences in machine learning: opportunities and challenges for marketing. Journal of Marketing Theory and Practice, 2021.
David Heckerman, Dan Geiger, and David M Chickering. Learning bayesian networks: The combi- nation of knowledge and statistical data. Machine learning, 1995.
William Hoiles and Mihaela Schaar. Bounded off-policy evaluation with missing data for course recommendation and curriculum design. In ICML, 2016.
Agastya Kalra, Abdullah Rashwan, Wei-Shou Hsu, Pascal Poupart, Prashant Doshi, and Georgios Trimponias. Online structure learning for feed-forward and recurrent sum-product networks. NeurIPS, 2018.
Nan Rosemary Ke, Olexa Bilaniuk, Anirudh Goyal, Stefan Bauer, Hugo Larochelle, Bernhard Schölkopf, Michael C Mozer, Chris Pal, and Yoshua Bengio. Learning neural causal models from unknown interventions. arXiv preprint arXiv: 2019.
Doga Kisa, Guy Van den Broeck, Arthur Choi, and Adnan Darwiche. Probabilistic sentential decision diagrams. In KR, 2014.
Murat Kocaoglu, Chris Snyder, Alex Dimakis, and Sriram Vishwanath. Causalgan. 2019.
Daniel Koch, Robert S Eisinger, and Alexander Gebharter. A causal bayesian network model of disease progression mechanisms in chronic mycloid leukemia. Journal of theoretical biology, 2017.
Kevin B Korb and Ann E Nicholson. Bayesian artificial intelligence. CRC press, 2010.
Matt J Kusner, Joshua R Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. NeurIPS, 2017.
Steffen Lauritzen and David Spiegelhalter. Local computations with probabilities on graphical structures and their application expert systems. Journal of the Royal Statistical Society: Series (Methodological), 1988.
Amélie Levray and Vaishak Belle. Learning credal sum-product networks. AKBC, 2020.
Stephen L Morgan and Christopher Winship. Counterfactuals and causal inference. Cambridge University Press, 2015.
Richard E. Neapolitan. Learning buyesian nelworks. Pearson Prentice Hall Upper Saddle River, NJ, 2004.
Mike Oaksford and Nick Chater. Causal models and conditional reasoning. Oxford library of psychology. The Oxford handbook of causal reasoning. 2017.
Michael Oberst and David Sontag. Counterfactual off-policy evaluation with gumbel-max structural causal models. In ICML, 2019.
Ioannis Papantonis and Vaishak Belle. Interventions and counterfactuals in tractable probabilistic models: Limitations of contemporary transformations. arXiv preprint arXiv:2001. 2020.
Iago París, Raquel Sänchez-Cauce and Francisco Javier Diez. Sum Sum-product networks: A survey.
arXiv preprint arXiv:2004.01 2020.
Judea Pearl. From bayesian networks to causal networks. In Mathematical models for handling partial knowledge in artificial intelligence. 1995.
Judea Pearl. Causality. Cambridge university press, 2009.
Judea Pearl. The seven tools of causal inference, with reflections on machine learning. Communica- tions of the ACM, 62(3):54 60, 2019.
12

---

Judea Pearl, Madelyn Glymour, and Nicholas P Jewell. Causal inference in statistics: A primer. John Wiley & Sons, 2016.
Robert Peharz, Antonio Vergari, Karl Stelzner, Alejandro Molina, Xiaoting Shao, Martin Trapp, Kristian Kersting, and Zoubin Ghahramani. Random sum-product networks: A simple and effective approach to probabilistic deep learning. In UAI, 2020.
Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference. The MIT Press, 2017.
Hoifung Poon and Pedro Domingos. Sum-product networks: A new deep architecture. 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pages 689-690. IEEE, 2011.
Dan Roth. On the hardness of approximate reasoning. Artificial Intelligence, 82(1 (2(1-2):233-32, 1996.
Bernhard Schölkopf. Causality for machine learning, arXiv preprint arXiv: 10500, 2019.
Karthikeyan Shanmugam, Murat Kocaoglu, Alexandros G Dimakis, and Sriram Vishwanath. Learning causal graphs with small interventions. NeurlPS, 2015.
Xiaoting Shao, Alejandro Molina, Antonio Vergari, Karl Stelzner, Robert Peharz, Thomas Liebig, and Kristian Kersting, Conditional sum-product networks: Imposing structure on deep probabilistic architectures. arXiv preprint arXiv:1905.08550 2019.
Or Sharir and Amnon Shashua. Sum-product-quotien networks. In AISTATS, 2018.
Amit Sharma and Emre Kiciman. Dowhy: An end-to-end library for causal inference. arXiv preprint 2:Xi::0011.04216, 2020.
Edward H Simpson. The interpretation of interaction in contingency tables. Journal of the Royal Statistical Society: Series (Methodological), 13(2):238-22 2-1,, 1951.
Martin Trapp, Robert Peharz, Hong Ge, Franz. Pernkopf, and Zoubin Ghahramani. Bayesian learning of sum-product networks. NeurIPS, 2019.
Van den Broeck, Di Mauro, and A Vergari. Tractable probabilistic models: Representations, algorithms, learning, and applications. Tutorial at UAI, 2019.
Kevin Xia, Kai-Zhan Lee, Yoshua Bengio, and Elias Bareinboim. The causal-neural connection: Expressiveness, learnability, and inference. 2021.
Kun Zhang, Bernhard Schölkopf, Peter Spirtes, and Clark Glymour. Learning causality and causality- related learning: some recent progress. National science review, 2018.
Han Zhao, Mazen Melibari, and Pascal Poupart. On the relationship between sum-product networks and bayesian networks. In ICML, 2015.
13


Reviews:
[{"review_id": "whoPSgr_P2q", "reviewer": "Reviewer_aP4W", "summary": "This submission proposes interventional sum-product networks, a tractable method for estimating causal effects from interventional data. The authors prove that iSPNs are universal function approximators. They show that iSPNs produce accurate estimates of interventional queries from interventional data on collection of synthetic data generating processes.", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "While the authors begin to address an important general topic of tractable probabilistic causal inference, the current submission requires substantial revision before it is ready for publication. Specifically, it is unclear how the specific task the authors have focussed on is significant, the clarity of the writing could be improved, and the submission appears to be missing a technical description of the key methodology.\n\nSignificance of the Task:\nAs the authors note in the submission that \"observations in the intervened system are akin to observations made when intervening on the system\". In other words, inferring the effect of an intervention using data sampled from the same intervention reduces to standard probabilistic inference. In that vein, it appears that iSPNs are just SPNs using interventional data, and do not leverage causal assumptions or interventional semantics to relate observational data (or data from a different interventional setting) to inform estimates of interventional queries. \n\nIt is also not clear from the submission how the tractability of SPNs contribute to answering interventional queries from a learned model. As the submission only considers causal graphs without latent confounders, probabilistic inference in the (fully parameterized) intervened model reduces to forward sampling, which is already tractable for directed graphical models. \n\nClarity:\nGiven that the paper is assuming a known graph structure, it is unclear what relevance Section 3.1 has to the central claims and contributions. I suggest removing the discussion about how multiple causal structures induce the same observational distribution. In addition, the author appears to be conflating causal inference outside of the data support (i.e. violating positivity assumptions) with structural ambiguity.\n\nMy understanding of SPNs is that they result in tractable inference because they place restrictions on the space of joint densities that can be represented. However, this seems to contradict the submission's claims that iSPNs are universal function approximators. How do the authors reconcile this apparent contradiction?\n\nMissing Presentation:\nSection 3.3 describes what iSPNs are not, but does not thoroughly describe what iSPNS are. For example, how is g(*) constructed? This section could be substantially improved by including an algorithm block for each of the following; (i) how g(*) is constructed, (ii) how neural network components relate to underlying graph structure, and (iii) how iSPNs perform inference at query time. I understand that some of these will be very simple, but being explicit will help readers understand what is novel about the contribution, and what is a simple reduction of existing techniques.\n\nMinor suggestions:\n\nThe figures could be substantially improved by clearly labeling and captioning subfigures. Additionally, some of the figure axis labels, etc. are too small to be legible.\n\nGiven that U is often used to denote latent confounders or exogenous noise in SCMs, I recommend choosing a different notation than do(U=u) throughout.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors discuss some limitations (in the form of future work) in the conclusions section.", "review_text": "", "extra_scores": {"ethical_concerns": "None.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "eYfzTyAm3oG", "reviewer": "Reviewer_3eLf", "summary": "This paper proposes a modification of sum product networks termed iSPN to estimate interventional distributions from data. The proposed model performs a manipulation to the structure of the SPN in order to provide estimates from the mutilated distribution. The authors motivate the use of this model in terms of computational complexity and the ability to represent estimates of arbitrary causal quantities on the graph. Empirical results show the proposed model having impressive running time and comparable error results to modern software packages for causal effect estimation. ", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I am of two minds on this paper. On one hand, I think there is real novelty in a black box representation such as what the authors are presenting. The proposed framework is very general and could likely be of real utility for developing a wide range of downstream applications. On the other hand this paper is overly vague in many places making it difficult to propose acceptance. The authors make use of \"universal approximators\" but don't provide any theory which would define the criteria for consistent and unbiased estimation from the proposed model. This is crucial in causal inference applications, which are essentially unsupervised models. It's also not clear to me what the comparative advantage is of this model over existing estimation procedures which come with _very_ strong theoretical guarantees and robust empirical performance. The authors point to computational efficiency, however computational efficiency is rarely the bottleneck for causal estimation tasks (whereas statistical efficiency often is). It is also unclear to me how hyperparameter selection is to be carried out. Finally, the authors don't make plain what assumptions are being required of the data in order to make valid inferences.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "It would be very helpful if the authors more clearly define the assumptions, limitations, and expected failure points of the proposed method.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "Iyir3x3oAJH", "reviewer": "Reviewer_KcVJ", "summary": "The paper proposes a new causal inference framework based on so-called ‘interventional Sum-Product Networks’ to target the problem of a lack of tractability in causal inference. It is based on standard SPNs and employs deep learning techniques to capture complex nonparametric functions describing multivariate conditional probability distributions. It is evaluated on several small synthetic data sets.\n", "questions": "", "limitations": "", "rating": 2, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The problem is highly relevant, and the promise of having a means of reducing the computational complexity of causal (interventional) queries by obtaining sufficiently close but tractable approximations is very exciting.\n\nHowever unfortunately the paper is extremely vague and implicit about the actual proposed methodology. After spending 5 pages describing known background, theory and other approaches, all in great clarity with some nice examples, the actual description of the contribution itself (p6) is surprisingly vague and without sufficient details to properly assess what is actually proposed or how to reproduce the results (see detailed comments below). \nIn fact, I am unsure whether the goal suggested at the beginning of the paper (‘here is a method that allows to compute interventional distributions from observational data + causal graph using an approximation based on tractable models’) is actually what the paper is about, or whether the main purpose of the paper is  to show that in principle SPNs could be used as tractable approximators of interventional distributions. If the first then I did not find it in the paper, if the second then it is hardly a surprising conclusion. \n\nI am willing to entertain the notion I missed or misunderstood the main contribution of the paper (hence reduced confidence), but all things considered I doubt the paper is suitable for NeurIPS in its current form.\n\noriginality: novel suggestion, though unclear about the details \nquality: poor: there is simply not enough detail to properly assess what is actually proposed and to what extent it makes sense or solves the initial problem statement\nclarity: very clear on background, existing work and general examples (p1-5), but completely unclear when it gets to the actual contribution of the paper\nsignificance: due to the lack of details the paper is unlikely to have significant impact in its current form\n\ndetailed comments:\np1.6 ‘subsuming’ suggests you claim to do it better\np1.23: ‘difficult to scale’ => why would they be? there are CBNs containing thousands to even millions of variables\np1.27: ‘weaving in the notion of interpretability’ => poetic but too vague\np2.36: ‘.. since a bipartite graph ..’ => bit sudden if you are not familiar with that ref.\np2.fig1: explain what the intervention ‘setting lung cancer to B(1/2)’ means\np2.43: ‘dream of tractable causal models is not insurmountable’ => yes but at what cost ? in general it has to be at best an approximation of the true interventional quantities right?\np2.57: ‘the inductive bias’ => perhaps introduce this first\np2.61-63: point 3. this is far too cryptic at this stage\np3.69: define CSPN before using acronym\np3.105: ‘causal mechanisms do not change through intervention’ => this is not what is meant by the assumption of ‘invariant causal mechanisms’: for example there are interventions corresponding to mechanism change, e.g. adding a catalyst to stimulate/enable a particular chemical reaction\np4.117: SCMs also allow non-recursive interactions (feedback); in addition not al variables need to be observed\np4.section 3.1: this insight is laudable but is standard common knowledge in the field of causal inference dating back at least to Reichenbach (1931) and before\np4.146: ‘curating’ is perhaps a bit much for generating data from such a tiny toy model …\np5,Fig2: ‘.. conditions on the mutilated causal graph ..’ => please explain better\nalso: figure is a tad small for such a crucial example => try to maximise (space permitting)\n\np5.173: so far we spent 5 pages without introducing anything new …\np6.176: move prefixed bold sentence ‘Definition of iSPN’ to line 184: now this paragraph is very confusing when you think you are finally getting to the ‘meat’ of the paper but then still end up just discussing other work first again\n\np6.185: ‘.. as inout the (mutilated) causal graph..’ => which one is it? the causal graph or the mutilated graph or both? (to be clear: you need both, or at least the original causal graph + target node of intervention)\nidem: this definition (based on an adjacency matrix) does not seem to leave any room for possible unobserved confounders that were deemed so crucial to handle in p4.133?\np6.188, Def.1: this definition should be improved. at the moment there is nothing ‘interventional’ about this definition (even though you may intend to use it as such later). \nalso: ‘shared parameters’ => shared between what?\n\np6.191: being able to ‘answer’ a causal query is meaningless if you cannot provide guarantees about the outcome … and so far we have not seen anything that argues how or why it should be\np6.193: ‘The shared parameters allow for information flow during learning between the conditions and estimated densities’ => sounds great but I have no idea what it means as you have not explained any details yet\n\np6.294, Prop.1: ‘ .. with data D generated from the intervened SCM ..’ => ??? The problem is: we have data D from an unintervened system, and a corresponding causal graph … from this can we predict what will happen when we have intervene on Xi. If you assume you have data corresponding to this intervention the whole problem goes away. Or is this what you suggest to do in the paper: learn the structural equations from the data, set intervened equation to constant X_i = a, and sample new data from this modified system so you can then do standard inference on it given the intervened graph?\n\np6.217+: this is just so vague I have no idea what you are trying to say. Seriously, what does ‘freeing the investigation of iSPN from independent research around hidden confounding’ mean or why should we care?\n\np7.225: ok: here you are apparently learning the iSPN directly from the interventional data … if that is what the paper is about, then I’m afraid it has little new insights to offer … if not then please make sure to describe in detail exactly what you do.\n\np7-9: given that I still have no idea what you actually do or try to do I cannot assess the relevance or validity of the results presented here. This may be due to my lack of understanding (hence reduced confidence), but  tbh I don’t think that is the case here.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No negative societal impact.", "review_text": "", "extra_scores": {"ethical_concerns": "No ethical concerns", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "3VKc48q9AG", "reviewer": "Reviewer_mLY8", "summary": "This paper discusses a variant of sum-product networks (SPNs) called \"interventional sum-product networks\" which is a variant of SPNs that take as input a causal graph and use this to model interventional distributions. The claim is that this is the first paper that applies tractable probabilistic models to causality. Theoretical aspects are discussed and the method is evaluated and compared to existing methods.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The clarity of the paper could be improved. The key definition in the paper is the definition of an interventional SPN (Definition 1), which is defined very abstractly, i.e., the concepts SPN and gated SPN are not discussed in detail. It is also not clear to me why f(G) are shared parameters (shared between what?). There is also quite some discussion on SCMs, but the paper never really makes clear what kind of SCMs the method supports. For example, the paper does not say whether the SCM should be acyclic (and whether cyclic SCMs were evaluated).  Besides that, the paper is well-structured and quite easy to read. Improving the clarity slightly should be little work.\n\nAs far as I know, this is a novel approach of combining causal knowledge as part of a \"deep\" representation that can be learned from data, bridging a gap between two areas that usually do not have a lot of overlap. While in this paper the authors chose SPNs, but it is not hard to see that similar ideas could be applied in other areas. As a result, I think the paper could have significant impact, and could open new avenues for research.\n\nTechnically, the paper is straightforward and seems correct. The experiments are limited, but seem appropriate for this paper: I think it convincingly shows that iSPNs are capable for learning generative models and provides good causal effect estimations. In my opinion, this is sufficiently convincing.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The societal impact is not relevant for this paper. There is a brief discussion on limitations, in context of future work. This seems appropriate.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}]

Author rebuttal:

Meta-review: A few of the reviewers felt that the paper is in its current form too vague and would improve from a major revision.  For instance, they found the claim about how SPNs relate to causal inference not adequately developed. Unfortunately the rebuttal did not clarify the questions the reviewers raised. In light of this and the discussions and the reviews, I agree that the current version of the paper is not ready for publication.
Decision: Accept (Poster)