Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes an algorithm to learn an ordered group decomposition of the causal graph over variables $\mathbf{X}$ which are not measured directly. In other words, each measured variable $\tilde{X}$ is a proxy of a target variables $X_i$ such that $X_i = \tilde{X}_i + E_i$, where $E_i$ is the measurement error. 

The approach assumes the LiNGAM’s setting, I.e.:
1) acyclic model
2) causal sufficiency (no unmeasured confounders between the measured and also between the target variables), 
3) linear functions, and 
4) non-Gaussian error terms.

Further, it assumes a random measurement error model, where $X_i = \tilde{X}_i + E_i$  and  $E_i$  are additive errors assumed to be mutually independent and independent of $X_i$. 

Specifically, the authors propose the Transformed Independent Noise (TIN) condition, which checks independence between a particular linear combination of some variables and others. This condition generalizes the current approaches IN and GIN and provides information about the graphical structure over the measured variables, even when there are measurement errors. The authors show that the ordered group decomposition of the causal model is identifiable in this setting.

The proposed techniques were evaluated both through simulations and an application to real data. 


strengths_and_weaknesses: The paper tackles an important and relevant problem. The considered assumptions are a bit restrictive. The parametric assumptions were well justified and motivated in the text and the theoretical contributions were illustrated through examples and seem solid. However, most of the proofs and intuitions were moved to the appendix, making the main part of the paper hard to read. 

A critical issue of the paper is that the assumption of causal sufficiency is not even mentioned in the paper.  When describing LinNGAM in line 49, the authors say that

“the generating process for $\mathbf{\tilde{X}}$ is linear, non-Gaussian, acyclic model (LiNGAM)”.

Then, in line 249, the authors say: 

“ Assumption 1 is the only one we make besides LiNGAM throughout  the paper, where violation of Assumption 1 is of Lebesgue measure 0, and LiNGAM is testable.” 

The causal sufficiency assumption is not mentioned neither for LiNGAN nor for the proposed method. This is the strongest assumption and is NOT testable. Although it may be okay to make this assumption in this context, an appropriate justification must be provided. Further, it would be really appreciated if some simple examples illustrating what would happen if the causal assumption is violated. 

Disclaimer:  Apart from the discussion on causal sufficiency, the text is well-written and the problem is well-motivated. Examples also helped to understand the contributions. However, I didn't check the proofs in the appendix and I am not familiar with the previous methods (IN and GIN), so I cannot judge the novelty and soundness of the methods.

questions: No further questions.
limitations: See sections above.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: In this paper, the authors study causal graph recovery among unobserved target variables from observations made with   measurement error. The authors propose  the Transformed Independent  Noise (TIN) condition, which checks for independence between a specific linear   transformation of some measured variables and certain other measured variables. By  utilizing TIN, the ordered group decomposition of the causal model is identifiable.  Experimental results on both synthetic and real-world data demonstrate the effectiveness and reliability of the method. 
strengths_and_weaknesses: Strengths: 

 1. The paper is well written with good structures. Detailed illustrations and annotation make it easy to read.  

  2. The methodology proposed by this paper and technical sections look solid and sound with technical details. 

Weaknesses:

1. Experiments on more real-world datasets with measurement errors could make the paper stronger. 

2. Experimental comparison with other methods such as [1] and [2] on ordered group decomposition should be added.

[1] Zheng, Xun, et al. "Dags with no tears: Continuous optimization for structure learning." Advances in Neural Information Processing Systems 31 (2018). 

[2] Rolland, Paul, et al. "Score matching enables causal discovery of nonlinear additive noise models." arXiv preprint arXiv:2203.04413 (2022).

questions: 
1. Is it possible to apply the method directly to causal structure recovery?
limitations: The paper can be strengthened with additional real-world examples and experimental results.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors propose a generalization of the existing causal discovery approaches for learning the structure between latent variables in a parametric linear additive non-Gaussian structural causal model - under the assumption that each latent node has exactly one measurement node that is a child. 
strengths_and_weaknesses: + The exposition is very clear. Authors closely follow several examples and do it well. 

+ An important problem with some recent results. This work seems to generalize these for another special case of measurement graphs. 

- Presentation gets overly succinct in Section 5.

- Some concerns with the theory. 

- Some parts of the algorithmic approach is unclear. 

- Emoji-based writing in the intro makes it hard to parse the text/the story.
questions: Thank you for your submission. I will be happy to increase my score after clarifications of the below. 

"necessarily being one, all results in this paper still hold"
I guess this depends on the downstreat objective. This might not always be a valid assumption. 

An example/insight for Prop. 1 would be useful.

"the GIN condition can be readily used to fully identify the structure of G, which is already a breakthrough over existing methods"
Can you elaborate on this? This was not identified by the previous authors? They only learned the partial graph? This statement is missing some context.

Could you elaborate a bit on the recursion idea that is being hinted at for the GIN paper and that I believe will be used here as well? If the source node is discovered, one can get rid of its effect by conditioning on it. But here we only have access to its descendant. So we cannot condition. How can one recurse after the source is found?

line172: "while this is impossible on \tilde{G}"
This and other impossibility claims seem hard to check. Could you give some intuition on which mathematical tool can be used to verify this impossibility claim?

Ancestors in Theorem 1 probably means exogenous ancestors.

Assumption 1 seems reasonable as an alternative formulation of faithfulness. 

The method requires precise measurement model, which is a disadvantage, correct? If an observed node is caused by two latent variables, this will not work. Please verify.

Could you please explain the proof of Theorem 2? I am a little concerned with the mapping to the max-flow min-cut formulation. Proof connects through B matrix, which is weighted - each edge has a weight potentially different from 1. Max-flow min-cut connection is valid for weighted or non-weighted. But using B matrix tells me that you are using the weighted version. But then we are making judgements about the rank of matrices which is typically not about the amount of max-flow determined by weighted edges. 

Specifically the statement:
"the maximum amount of non-intersecting paths from source to sink is equal to the size of the minimum vertex cut from source to sink."
Maybe I am not familiar with this version because if flow is on an edge weighted graph, I thought cut should also be counted as edge weighted.

Theorem 3 should require some sort of guarantee that the measurement does not lose information. Can you formalize this?

My main concern is about finding the correct value \omega for conducting the transformed independence tests. I also don't see how this statement follows from Theorem 4:
"This transforms the task of estimating the dimension of \Omega(Z;Y) to a simpler one: counting size of the subsets Y0."
One still needs to search for existence of a w in these subsets.

Section 5 in general is too short and is very hard to follow without any explanation. How do we find w_YY in (11)?

line 367:" then check whether all solution"
How can we efficiently check all solutions if there are uncountably many? Please explain.

limitations: More like future work than the limitations and critique. Could be improved.
review_title: null

Meta-review: metareview: The paper considers structure recovery when there is a causal DAG on variables X where causal mechanisms are linear but exogenous noise variables are non-Gaussian (similar setting to the one in the standard prior work LiNGAM). However, each variable is not directly observed but through measurement independent noise. Authors show that by using independence tests between transformations of variables, one can recover the order group decomposition of the graph. 

I think the identifiability result is novel. Reviewers are overall positive. 
Main concerns were:
a) Main concern of the reviewer with the lowest score is simply that causal sufficiency has not been spelled out
b) Another concerns is missing references to assumptions when deriving theoretical results clearly. 

These two are not very major and I suggest the authors to pay attention to these and comprehensively list all assumptions clearly upfront in their camera ready.


confidence: Certain
Decision: Accept