Venue: CLeaR 2022 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary: The paper considers causal discovery via the post-nonlinear noise model and contributes a novel autoencoder-estimation technique for the multivariate case by applying the same causal order and pruning procedure as proposed for the ANM model.
Main review: The paper follows a straightforward approach: The authors extend the autoencoder-based estimation of the bivariate PNL model by Uemura and Shimizu (2020) to the multi-cause case and then apply a causal order and pruning procedure as proposed for the ANM case in Peters et al. (2014). To learn y = g(f(x) + e) they use an auto-encoder with loss terms for the independence between x and e, here max_i(HSIC(xi, e)) and the reconstruction error of y. The approach is applied to a single (!) DAG model and real-world datasets. Overall a weak accept.


# Pros:

* (Potentially) the first estimator for the multi-cause PNL
* Several steps to aid the stability of estimation are undertaken

# Cons:

* Numerical experiments extremely limited (owing to extreme computational demands)
* Few limitations / weaknesses discussed
* Omitted proofs despite Supplementary Material
* No code provided

# Originality

The authors state that even the publication of the PNL model was already in 2012 there hasn't been work on multi-cause estimators. I found another work ("Post-nonlinear Causal Model with Deep Neural Networks" by Chung etal, 2019) that also addresses a multi-cause PNL with neural networks. Hence, the authors would have to explain the theoretical difference and/or compare them numerically.

# Significance

The PNL model is the to-date most general among the class of restricted structural causal models and may in many cases be the only one applicable given complex datasets. A novel estimator is, hence, an important contribution.

# Quality

The introduction and preliminaries sufficiently cover the problem setting. The proposed method straightforwardly follows from the bivariate previous work and the causal order and pruning approach from the ANM. These are sufficiently described. For the only Theorem the authors state "The “if“ part is obvious from the definition. The converse can be easily verified in the same manner as the ANM case (see Peters et al., 2014, A. 15).". Since there are no space problems here, I would provide the proof for completeness. There are also no guarantees discussed for how well the autoencoder may fit in a general setting.

The authors mention the issue of stabilization in the Conclusions, but could they elaborate on this also in the Discussion?

My main concern is about the numerical comparison. In fact only one single DAG with a few SCMs is considered and the proposed approach compared to the RESIT estimator of ANM and DirectLinGAM. As mentioned above, there seems to be another relevant neural network estimator that could be considered here.

The main issue seems to be the extreme computational complexity of the method. Could the authors discuss how this can be alleviated? Also, except for computational complexity no weaknesses are discussed.

I would also suggest to provide code somewhere.


# Clarity

The style of writing suffers a bit from English grammar issues.


# Further comments

* "Causal discovery methods assume that causal structures follow structural equation models"
What about constraint/score-based methods. These do not assume one particular SEM, rather a very general class. I suggest to frame the problem a bit more general here.
review_title: null

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: Summary: This paper considers the problem of learning causal structure from observed data generated by the post-nonlinear model. The authors first study the estimation of the PNL model with multi-parents by autoencoder strategy of Neural network. Then, the authors propose a two-step method to estimate the post-nonlinear model, including identifying causal order and removing redundant edges.
Main review: The paper considers an essential problem of estimating causal structure from observation data. \
The paper builds on prior work in the field of post-nonlinear models. The authors put some effort into estimating the whole graph when the data follow the PNL model.\
The critical challenge is how to estimate PNL efficiently. The authors use neural networks to represent and optimize the two functions of PNL. \
The theoretical analysis or experimental results are presented in a logical way.

The way of literately finding a sink is my only concern. This way will affect the performance of your method when the number of variables is large. It would be nice to discuss this issue in the main paper.
review_title: null

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary: This paper extended the AbPNL method developed for bivariate causal discovery to process multivariated case. It first borrowed a widely used  causal order learning method, and combined it with multivariate AbPNL to learn a causal order. An edge pruning method is then used to get a DAG based on the causal order.
Main review: The paper is clearly written and organized. But I have the following concerns:
1. Novelty: the paper has some small innovation points, but can not give us valuable enlightenment. It is more a combination of some existing methods/tricks with tiny adaption.
2. Theoritical foundation: the paper lacks a systematic proof of model identifiability and inference consistency.
3. Experiments: For the experiments on both simulation and real data, the datasets are too simple with just several variables, how the performace will be if running on large datasets? The propsed method relies on learning many DNN models to evaluate data fitting, however, when the variable and sample scale are large, the time efficiency and hyperparameter adjusting will be a problem. The author claimed the proposed method is a general one which can process both linear and nonlinear relations, however, results in table I do not support such cliam. For the accuracy showed in table 2, how much is from the deep models used in the new method, and how much is from the causal order learning?
review_title: null

Meta-review: A Multivariate Causal Discovery based on Post-Nonlinear Model This paper modifies an algorithm for searching for additive noise models to searching for multivariate post-nonlinear models. The paper also tests the algorithm on synthetic (100 random DAGs and equations) and real data. The authors have agreed to make the code public. They use an autoencoder based estimator of Uemura and Shimizu to the multi-cause case. Simulations show the superiority of the new multivariate PNL algorithm over algorithms that make more restrictive assumptions in the non-linear case (although it performs worse than DirectLiNGAM in the linear case). THe algorithm was also tested on a well-known carefully constructed simulation of fMRI data. In response to the reviewers they have added simulations on larger numbers of variables (15) and analyzed the complexity of the algorithm.

The main argument against the paper is that it is a fairly simple adaption of already existing techniques, so its theoretical novely is quite low. On the other hand, its practical novelty is much higher, since it is the only described and implemented algorithm for searching for multi-variate PNL causal models.

Two of the reviewers voted a 6 and one voted an 8. I concur that it should be accepted, given the practical usefulness of the algorithm and the tests on the simulated and real data.
Decision: Accept (Poster)