Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper introduces a method of estimating ADMGs from observational data. The authors build on existing work on ADMG identifiability and on graph learning with deep neural networks. First, two assumptions (sufficient conditions) are proposed that guarantee identifiability of the underlying ADMG (inspired by Maeda and Shimizu, 2021). Subsequently, a variational scheme (N-ADMG) for learning the ADMG is introduced (based on Geffner et al., 2022). The paper concludes with a series of experiments showing causal discovery (ADMG recovery) and causal inference (average treatment effect estimation).
strength_and_weaknesses: STRENGTHS

While this paper builds heavily on existing work, both in deriving the sufficient conditions for ADMG identifiability and for specifying the variational model optimized with neural nets, the two parts fit together nicely which leads to a well-defined model for ADMG estimation.

The research area is also very relevant and has received a lot of attention recently (deep causal models, accounting for unobserved confounding).


WEAKNESSES

The novelty of the proposed approach is somewhat limited by the fact that the vital building blocks (ADMG identification criteria and variational causal graph structure learning) are already present in the literature.

The experiments appear inconclusive, especially given the fact that both in the causal discovery and causal inference (ATE estimation) experiment, the methods coming close to (or beating) N-ADMG are the ones corresponding to the "building blocks", i.e. N-DAG and CAM-UV.

As a minor issue, Figures 2 and 3 are difficult to read.

clarity,_quality,_novelty_and_reproducibility: The paper is well-written and easy to follow.
summary_of_the_review: This paper combines two existing ideas which leads to a somewhat novel approach to ADMG estimation from observable data.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper provides a gradient based approach to causal discovery in the presence of unobserved confounding. By making assumptions on the structure of the graph along with additive noise assumptions - identification results for non-linear SCMs are provides. Since the conditional independencies involves learning functions, a neural ADMG learning framework is proposed. Finally, empirical evaluation is performed where the method is shown to perform well. 
strength_and_weaknesses: Strengths: This paper establishes causal discovery results on a more flexible class of ADMGs than what exists in the literature. Proper identification conditions are established, and the inference framework has computational tractability advantages.

Weakness: The statistical framework for neural ADMG learning does require a lot of modelling decisions when it comes to things such as choice of variational approximation - and I do worry how how much flexibility these methods lose when such choices are made. However, I understand the necessity of these assumptions, and I consider this only a minor weakness. 
clarity,_quality,_novelty_and_reproducibility: This paper is well written, and a rigorous discussion of existing work and its limitations is provided. Additionally, the explanation of concepts flows well and allows for the reader to understand the points made in this paper very well. The work here is novel as it provides new identification results as well as a new numerical framework for causal discovery. 
summary_of_the_review: Overall, this paper presents a novel causal discovery algorithm under the assumption of additive noise. Identification results are first established, then the proposed neural ADMG learning framework is used to learn an ADMG. This paper is novel because it allows for learning ADMGs while relaxing assumptions made by previous works since, it allows for non-linear functions in the SCM. Next, instead of relying on traditional conditional independence tests - which can sometimes run into tractability issues in high dimensions, the neural ADMG framework is proposed that is more suitable for dealing with high dimensional data. High quality paper, I recommend for acceptance.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors modeled the data structure with acyclic directed mixed graphs (ADMGs). Under the condition where latent confounding is present, a new autoregressive  flow-based approach is proposed to learn the causal and functional relations of the given data. In the experiments. the authors validate competitive performance of their approach in comparison to the baselines.
strength_and_weaknesses: Strengths: 
1 I like the mathematical formulation discussed in Sec. 4. The introduction of ADMG identifiability seems to be quite clear. Though the additive noise assumption and confounding assumption are simplified, the authors achieve a good balance between practicality in real-world application and technicality in mathematical reasoning. The discussions from  lemma 1-3 also make sense to me and fairly easy to understand.

2. The autoregressive approach that is implemented with MLP is well-motivated. The authors have justified the method on both  a toy and real datasets.

Weakness:
1 I think the major issue is that the authors altered the layout of the ICLR template and significantly decrease the space between sections.
I am not sure if this is fair for other authors who follow the submission guidelines. 
clarity,_quality,_novelty_and_reproducibility: The clarity and quality of the paper is okay.
It is hard for the reviewer to evaluate the reproducibility of the paper. Because of the page limit, many model implementation details are not disclosed in the paper. 
summary_of_the_review: Considering the violation of ICLR template's layout and the length of the paper, I would kindly suggest submitting this paper to a journal instead.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors generalize the differentiable causal discovery method to the setting in the presence of latent variables. Compared to related work, the method can tackle non-linear data. They provide both theoretical and empirical results.
strength_and_weaknesses: Strength:
1. The paper is clearly written. 
2. The authors give a detailed review of the related studies. 
3. There are some new theoretical results in this paper.

Weaknesses:
I do not find some significant weaknesses except for some typos. 
clarity,_quality,_novelty_and_reproducibility: Overall, the quality is good. The authors take a further step towards the differentiable causal discovery when the data generating process is non-linear.


typo or minor aspects
1. They enable us to answer questions causal in nature on Page 1.
2. Lemma 1-Lemma 3 "if *and* only if "on Page 4.


questions:
1. I am not clear how the proposed method enforces that the ADMG learning satisfies
assumptions in Section 4, as said on Page 5. Shouldn't whether the assumptions are fulfilled related to the task only and not relevant to the method?

2. What does it mean by "Whilst the observed data distribution may still be identifiable"?
summary_of_the_review: Overall, the quality is good. The authors propose a differentiable method for learning ADMG under the current causal discovery framework, with theoretical guarantees and rational experiments. Hence, I give a positive score. The main reason for the borderline score is that I do not quite agree with the study regarding learning the ADMG (this discussion is possibly beyond the scope of this paper.). The assumptions necessary for ADMG learning are too strong. I am strongly skeptical whether such a target of learning ADMG can be achieved in practice, and whether this study can help the practical problems.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The reviewers were split about this paper and did not come to a consensus: on one hand they appreciated the importance of the problem addressed and the clarity of the writing, on the other they questioned whether the method was sufficiently novel. After going through the paper and the discussion I have decided to vote to accept for the following reasons: (a) the identifiability result presented in the paper is non-trivial and extremely useful for the field (adding a new identifiability scenario beyond linearity or fully decoupled effects), (b) the gradient-based optimization framework is highly-practical and would be easy to implement in any autodiff framework. The authors will release their code which will be a great contribution to the community. Authors: you've already indicated that you've updated the submission to respond to reviewer changes, if you could double check their comments for any recommendation you may have missed on accident that would be great! The paper will make a great contribution to the conference!
justification_for_why_not_higher_score: Even though I find it promising, it is likely only interesting to a specialized audience.
justification_for_why_not_lower_score: The paper makes useful contributions in causal identification and estimation, accepting it will improve the conference.
Decision: Accept: poster