Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: The authors propose (conditional) sum-product networks and the knowledge of the causal graph to estimate the interventional distributions. 
main_review: The idea of training a neural network using conditional distribution and using gates to mimic interventional samples reminds me of CausalGAN by Kocaoglu et al. Please cite and compare by highlighting the differences. Under the same assumptions of the manuscript, it demonstrates that one can learn NN-based causal generative models (Q3+Q4 in experiments) and also obtain interventional samples under the strict positivity assumption. What is the benefit of using SPNs or iSPNs instead? I believe the advantage here would be the estimation of the interventional distribution rather than obtaining samples from it. But nevertheless, a comparison is in order here since one can always estimate the interventional distribution from the samples obtained from CausalGAN. 

Bayes Net to SPN transformation and relations should be explained. What is the space-complexity time-complexity tradeoff of using an SPN rather than a generative model for the data. 

In general, no insight into SPNs or conditional SPNs are provided. The manuscript should be self contained with a detailed and formal description of these. 

Minor comments:
"thereby freeing the investigation of iSPN from the independent research around hidden confounding."
Hidden confounding is not independent but its non-existence is a rather strong assumption. Please consider paraphrasing this statement. 

Post-rebuttal note:
I am increasing my score based on the engagement of authors and their commitment to include relevant literature and provide clarifications. Unfortunately, there are still several points I am still unclear about which prevent me from recommending acceptance: what kind of guarantee can we give using a random SPN structure? Many claims in the rebuttal seem to heavily rely on past work which are not fully explained in the submission. I recommend the authors clarify the contribution, explain the process of going from causal graph/query to iSPN through examples, and explain the existing work that this paper builds on in more detail. 
limitations_and_societal_impact: No. I think the limitations should be more explicitly discussed than being presented as part of future work.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposes to learn interventional distributions using a family of generative models known as sum-product networks (SPN). SPNs are models based on gate functions that allow for tractable inference. Specifically, the authors propose interventional SPNs (iSPN) which are conditional SPNs (they model conditional distributions) that, licensed by a causal graph and samples from the interventional distribution, are shown to be expressive and causally adequate.
main_review: The paper introduces the use of SPNs for learning interventional distributions from data in the context of Markovian (no hidden variables affecting more than one observable) and with knowledge of the corresponding causal graph. This approach is intended to overcome the shortcoming of related work on translating Bayesian networks into SPNs as it has been shown that this leads to causally inadequate models. 
The authors show that based on the causal graph, the interventional distributions P(V | do(X=x)) generated for different values of x can be modeled as a conditional distribution in the context of the mutilated graph. Based on this observation, the authors bias the learning process of a conditional SPN to correctly model such interventional distribution for which data is given.

The definition of iSPNs and their ability to model interventional distributions is well supported by the discussions and results in the paper. The definition of structural causal model used in the paper corresponds to the class of Markovian SCMs where no hidden variable affects more than one observable. However, it is unclear to me why they authors do not consider non-Markovian SCMs since it seems to me that Proposition 1 still holds under this class of models.

Overall, the paper is well organized, necessary concepts are gently introduced and the experimental section serves the purpose of evaluation the benefits of using this approach in terms of efficiency and accuracy, compared to other methods over different datasets.

In terms of the impact of the work, I wonder what are the current application it could prove significantly advantageous. In particular, since the method assumes that the input data is already interventional, except for some marginalization this input already contains a target of inference such as P(y | do(x)). It seem interesting, however, to consider the possibility of modeling the observational distribution P with an SPN and use it to produce estimates of an identifiable causal effect as it can be expressed as a function of conditionals and marginalizations over P. I understand from the discussion that modeling a joint distribution instead may diminish expressivity but could the authors touch on this in their response?

Other small details/comments:
- Line 58, 59: repeated "for"

limitations_and_societal_impact: The authors have provided some discussion related to the technical limitations of their method.
At this point I can't foresee any societal impact that needs to be addressed by the authors.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: It considers the problem of learning interventional distributions using sum-product networks (SPNs) that are parameterized by neural networks. They propose interventional sum-product networks (iSPNs) that take advantage of both the expressivity of the neural network and the tractability of the SPN in order to learn the interventional distributions. 
main_review: Learning interventional effect is an important problem that is considered in this paper and has a lot of applications in different fields. 

The paper is well-written but it has room for improvements. For instance, it would improve its readability if there are more detailed information about SPN in the main text (or in the appendix) instead of its background. The importance of learning causal effects (i.e., computing an interventional distribution) is quite clear. Hence, I suggest to reduce such discussions and add more details. As an example, in the definition of iSPN, it is said that g(.,.) is SPN, what does it mean mathematically? How does it look like? Based on figure 2.(b), g(.,.) is a conditional interventional distribution! What is J in this figure?

As far as I understood, the setting of the paper is when there are no hidden confounders in the causal system. In this case, it is known that every interventional query is identifiable (can be computed from observational distribution). The more interesting setting is of course when there are hidden confounders in which the complete algorithm is proven. How does iSPN perform (if it is applicable) in causal networks with latent confounders? What would be the output when an interventional query is unidentifiable?

How would this method scale to larger networks?
more importantly, how would it generalise to settings with hidden confounders?

Is it possible to compute interventional distributions using iSPN with atomic interventions? 
 
Based on Fig. 5, DoWhy performs better than the iSPN or am I missing something there?

limitations_and_societal_impact: Please see my comments.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper introduces iSPNs (a modification of existing SPNs) to model interventions. The authors claim it is the first method that applies the idea of tractable probabilistic models to causality without the need for compilation. 
main_review: Since SPNs and Bayesian Networks (BNs) can convert to each other, it is not surprising that SPNs can handle intervention, if the resulting modified network structure by the intervention is carried over to SPNs. This is exactly what iSPNs have done using the updated/modified adjacency matrix that reflects the structure change. Though it is not surprising, but it does have originality to some degree.

What is missing in the paper is IF and WHY the proposed iSPNs are more powerful and/or useful than existing causal BNs and other causal models. The content of iSPNs is very small (the definition of iSPNs only starts to appear in page 6). The advantages over other causal models are unclear. As pointed out in [Zhao et al., 2015], with Algebraic Decision Diagrams (ADDs) to compactly represent the local conditional probability distributions at each node in the resulting BN, the advantage of SPNs and subsequently iSPNs is not clear and isn’t sufficiently discussed and analysed in this paper. The experiments are done in very limited small scale toy-like settings. 

Overall, the paper has originality, but lacks clarity, quality and significance in its current form. 
limitations_and_societal_impact: Discussion for limitations provided. Societal impact is not mentioned.
review_title: null

Meta-review: The reviewers appreciated the use of SPN's for learning interventional distributions from data. Two common criticisms were: (i) the restriction to the Markovian setting (no hidden variables), and (ii) poor readability.

Despite the lack of full support from the reviewers' scores, I am recommending acceptance, because (i) the work addresses a fundamental problem in an original way, and (ii) the authors addressed all the concerns raised by the reviewers. For the next revision, I'd ask the authors to introduce and motivate SPN's more fully (perhaps reducing the text motivating learning of causal effects), and also to emphasize the fact that the work does not address hidden (latent) variables.
Decision: Accept (Poster)