Recursive Causal Structure Learning in the Presence of Latent Variables and Selection Bias

Abstract: We consider the problem of learning the causal MAG of a system from observational data in the presence of latent variables and selection bias. Constraint-based methods are one of the main approaches for solving this problem, but the existing methods are either computationally impractical when dealing with large graphs or lacking completeness guarantees. We propose a novel computationally efficient recursive constraint-based method that is sound and complete. The key idea of our approach is that at each iteration a specific type of variable is identified and removed. This allows us to learn the structure efficiently and recursively, as this technique reduces both the number of required conditional independence (CI) tests and the size of the conditioning sets. The former substantially reduces the computational complexity, while the latter results in more reliable CI tests. We provide an upper bound on the number of required CI tests in the worst case. To the best of our knowledge, this is the tightest bound in the literature. We further provide a lower bound on the number of CI tests required by any constraint-based method.  The upper bound of our proposed approach and the lower bound at most differ by a factor equal to the number of variables in the worst case. We provide experimental results to compare the proposed approach with the state of the art on both synthetic and real-world structures.


Reviews and author responses:

 Sorry for the delay. After reading the reviews and author responses, I am happy with acceptance for this paper.

A causal discovery method is presented for MAG learning based on removing an appropriately chosen variable, then recursively learning the structure over the remaining variables. Also, a lower bound is given for the number of CI tests that any constraint-based method for this problem needs, and the number of CI tests by the proposed method compares favorably to this bound.  The algorithm is based on interesting ideas that are put together well (how to approach the structure learning problem recursively, and how to use Markov boundaries to pick the right node to recurse on). Additionally, the paper is well-written. I am not sufficiently familiar with constraint-based algorithms for MAG learning to evaluate the theoretical results directly, or to confirm their novelty in comparison with other recent work in this field (but as far as I can tell, this approach to MAG learning is novel). But the empirical validation looks properly performed, and its results are very positive. The reported improvements in running time as well as accuracy make this a significant result.

UPDATE: I have read the authors' response, and my score is unchanged. Yes

 Thanks for the responses. I appreciate the comparison of RLCD with FCI+ and can see the theoretical gains. However, given that FCI+ is (in some sense) the closest competitor, it would be helpful to include a comparison in the empirical results. Nonetheless, I keep my original rating.

 We thank the reviewer for their precise and insightful comments. We are glad that the reviewer finds our approach novel.
***
Regarding comparing with FCI+, that algorithm requires the maximum degree (delta) as an
input which cannot be estimated from data, unlike Markov boundaries required by our approach which can be learned from data.
Therefore, we have to resort to using an upper bound for delta to be able to compare FCI+ with our approach. Note that the bound on the number of conditional independence tests proposed in that work is polynomial in |V| only if the upper bound is a constant in the
number of variables. Therefore, if we were to use the maximum size of the Markov boundary as a bound as the reviewer suggests, since this value is usually not constant in the number of variables, the resulting bound would not even be polynomial in the number of variables. Further, we note that if we use the maximum size of the Markov boundary as a bound as the reviewer suggests, since Pa+ is always a
subset of the Markov boundary, RLCD will achieve a much better bound than FCI+.

***
Regarding the existence of a removable node in each iteration, as mentioned in Appendix D, there is only a very specific structure related to the edge induced subgraph of the MAG G over its undirected edges (i.e., those due to selection variables) which can result in G not having a removable variable: The requirement for having at least one removable variable is that the edge induced subgraph of G over its undirected edges is chordal. (We mentioned an excessive requirement in the appendix, which we later realised is not necessary.) Note that if this subgraph is chordal for the initial MAG, this condition holds for all the subsequent iterations. The violation of this requirement is very rare in the sense that it only happens if there is a cycle of length at least 4 consisting of only undirected edges, where each pair of
adjacent variables on this cycle have a common selection variable which is not a descendant of any of the other variables on the cycle (Figure 5.a of the Appendix.) A proof of this result is as follows. 

- **Case1:** If there exists at least one directed or bidirected edge in the graph, then
consider any node that has an arrowhead towards it on all of its incident edges (i.e, it
has no children or neighbors. Such a node exists due to acyclicity of the MAG.) This
node is removable due to Theorem 4 in the Appendix. 

- **Case 2:** If the graph consists only of undirected edges: By the chordal assumption, we
know the graph has a perfect elimination ordering [A,B]. Now consider the first node
in this ordering. The set of the neighbors of this node form a clique, and therefore it is
removable (following from Theorem 4).

[A] Fulkerson, Delbert, and Oliver Gross. &quot;Incidence matrices and interval graphs.&quot;
Pacific journal of mathematics 15.3 (1965): 835-855.

[B] Blair, J. R., and Peyton, B. 1993. An introduction to chordal graphs and clique
trees. In Graph theory and sparse matrix computation. Springer. 1–29.

***
Regarding Figure 1, we thank the reviewer for being so precise. The edge between W
and T in our example must have been from W to T (in figures 1.a and 1.b) and we will
correct it. We also appreciate the other minor comments pointed out and will apply
them.

 We are glad that the reviewer finds our results positive. We thank the reviewer for
their comments.

 We are glad that the reviewer finds our approach original and useful. We thank the
reviewer for their helpful comments.
***
Regarding our choice of coefficients in the experiments, the exclusion is done to ensure that there is a meaningful causal effect
between the parents and their children in the graph, as setting a value very close to
zero would result in variables that are connected in the DAG but have almost no
correlation in the generated data. Note that this is the exact same setting of
coefficients used in papers such as the recently well received paper DAGs with NOTEARS [A], etc.

[A] Zheng, Xun, et al. &quot;DAGs with NO TEARS: Continuous Optimization for Structure
Learning.&quot; NeurIPS2018.

***
Regarding table 1, as we focused mainly on the recovery of the adjacency and we used the
algorithm described in [29] to orient the edges, we considered the adjacencies in our
precision, F1 score, etc. 
In Table 1 we compared the output PAGs with the ground truth PAG, that is, we
compared two classes of graphs with each other, where there are 9 possibilities for each existing edge. Therefore we need a metric that treats any type of mistake differently. We are not aware of such an approach in the literature, but we will add a description of the performance that will discuss this to the revised paper.

***
We used the main authors’ cited implementation [B] for M3HC and the causal
graphs software package [C] for the other algorithms. We will include the
corresponding information in the final version of our paper.

[B] https://github.com/mensxmachina/M3HC

[C] https://github.com/striantafillou/causal_graphs



The authors propose a method for learning a MAG from observational data in the presence of latent variable and selection, a problem addressed previously by FCI, RFCI, etc. The procedure works by identifying a removable variable, removing it, and so on, which renders the problem simpler and simpler as the algorithm proceeds. One benefit of the procedure is that it reduces the number of conditional independence tests needed for the search.  I found the approach nice, of identifying removable variables and then removing them one by one, as a way of optimizing the problem. The theory was clear.

I found the paper to be original and useful. I've never seen this approach before. The quality of the theory was good. The writing was fine.

I did have an issue with the experimental section. Table 1. It seems, though I can’t find text to support it, that F1, precision, and recall in this table are for adjacency. Is this right? If so, one wonders about orientation--could those stats be included?

Also, for Table 1, what packages were used for the various algorithms? I couldn’t find this information, though maybe it was there.

Again, Table 1, what graph were the results being compared to, and was it the same in each case? The true DAG? The true PAG?

In line 298, an especially wide interval about zero was excluded for random selection of coefficients. Is this necessary?
 I don't see how this could have negative societal impact.

The authors consider the problem of causal structure learning when there are latent variables and selection bias (i.e., estimating a MAG from observational data). While this is a well studied problem, most existing procedures can be computationally expensive when the number of variables is large and perform a large number of conditional independence tests which can suffer from statistical inefficiency. The authors propose a procedure which only considers subsets of the observed variables by checking whether certain variables are "removable." This results in less computation and also increases statistical efficiency by generally requiring smaller conditioning sets when checking conditional independencies. The authors show that this improves estimation accuracy (as measured by F1 scores) and uses less computation.       I think the problem that the article addresses is interesting and the approach is novel. Overall, I think this is a nice contribution as the procedure retains the same theorettical guarantees as previous methods, but results in improved empirical performance. However, since the authors give the same theoretical guarantees as FCI+ (i.e., sound and complete), I think it is quite important to compare RLCD to FCI+.

Major comments:
- It's not immediately obvious to me that every $$V \subseteq \mathcal{O}$$ contains a removable node, but it seems that Line 257 requires this. Prop 2 requires X to be removable in order for the upper bound to hold. Therefore, if there are no removable nodes, line 257 may not hold.   
-Although they present simulations which show that the RLCD procedure is drastically reduces computation / the number of CI tests, I am surprised they did not consider FCI+ as a competitor in the simulations, despite citing the paper. FCI+ is also sound and complete and is polynomial in |V| when an upper bound on the sparsity is given (an exact number is not required as the current authors state). This is not too different from RLCD which requires an initial estimate of the Markov boundary (who's size could be used as an upper bound on the degree). Also, comparing the theoretical results could be helpful. In particular, FCI+ states the computational cost in terms of maximum degree k, which in some cases could be smaller than the size of Pa+

Minor comments:
- Perhaps I'm missing something in Figure 1 (c), but when T is removed, shouldn't this result in a bidirected edge between W and Z? For instance, in (a), conditioning on W, the path Z, T, W, U, Y d-connects Z and Y and implies that Y \not \perp Z | W. However in C, Y \perp Z | W. 
- Pg 1, line 74: it would be helpful to define n here since it is not defined until later
- Pg 5, line 181: "substantially decrease" should be "a substantial decrease"
- Pg 5, line 201: Should all the X's be X_1?    
- Pg 6, line 230: It seems that the LHS of the equation should be \not \perp Yes

Meta-review: The authors have proposed a recursive algorithm for learning the structure of a Bayesian network in the presence of latent variables, based on the concept of a removable variable. All of the reviewers agreed that the approach is novel and interesting, and there is a consensus to recommend acceptance.

Decision: accept