Paper:
Learning and Testing Causal Models with Interventions
We consider testing and learning problems on causal Bayesian networks as defined by Pearl [Pea09]. Given a causal Bayesian network M on a graph with n discrete variables and bounded in-degree and bounded "confounded components", we show that O(log n) interventions on an unknown causal Bayesian network X on the same graph, and O(n/ 2 ) samples per intervention, suffice to efficiently distinguish whether X = M or whether there exists some intervention under which X and M are farther than in total variation distance. We also obtain sample/time/intervention efficient algorithms for: (i) testing the identity of two unknown causal Bayesian networks on the same graph; and (ii) learning a causal Bayesian network on a given graph. Although our algorithms are non-adaptive, we show that adaptivity does not help in general: Ω(log n) interventions are necessary for testing the identity of two unknown causal Bayesian networks on the same graph, even adaptively. Our algorithms are enabled by a new subadditivity inequality for the squared Hellinger distance between two causal Bayesian networks.

Review:
This paper answers a series of important questions about hypothesis testing and learning of causal models that can contain both measured and unmeasured variables, and contain only discrete variables. (The causal models are represented by semi-Markov causal models that contain a directed edge from A to B representing that A is a direct cause of B, and a bidirected edge between A and B when there is an unmeasured common cause of A and B). It provides bounds on the number of interventions, samples and time steps required by the algorithms. The distance between two models is measured by the maximum difference between interventions on the models. For an unknown model it is assumed that an experiment can perform experiments on any subset of the variables, and then measure the results of the intervention on a sample. The problems include determining when a hypothesized causal model is within epsilon of the unknown true model, when two unknown models are within epsilon of each other, and how to output a model that is within epsilon of the true model (given the correct graph). The paper also shows that the number of interventions required by their algorithms are almost optimal for the latter two problems. The fundamental strategy is to find a small "covering set" of interventions that supports a locality result that shows that when two causal models are far apart, there exists a marginal distribution of some intervention in the covering set that is far apart.   I am familiar with the causal part of the arguments, but less familiar with the complexity part of the arguments. I don't understand the cases where d represents not only the maximum indegree of the model, but also the maximum degree of the model Lemma 2 states that outputting a covering set of interventions is O(K^(ld)*((3d)^l)(ld^2 log(K), where K is the number of different values the variables can take, l is the maximum size of a c-component, and d is the maximum degree of a vertex. However, there is nothing in this formula which mentions n, the number of vertices in the graph (unlike the case where d represents only the maximum indegree of the model). Similarly, Theorem 4 says that same formula is the complexity of determining when two unknown causal models are the same. But given a DAG that consists of a number of subgraphs that are disconnected from each other, and each subgraph consists of a variable and a fixed number of different parents, the number of disconnected subgraphs does not change K, l or d. Nevertheless all of the individual subgraphs would have to be experimented on, and the number of interventions needed would have to increase with n in the worst case. Also, the number of subtests performed by the algorithm is said to depend on n.   There is a subtlety about the models that they are testing and finding that is not mentioned in the article. Once there are latent variables that can't be intervened on, there are multiple causal models that can generate the results of all of the possible interventions. For example, if a latent variable L causes measured X1 and X2, there are parameter values which make both X1 and X2 dependent on L, but independent of each other. In addition, intervening on X1 or X2 will produce no change in the other measured variable. So the intervention distributions can be represented by a semi-Markovian causal model in which X1 and X2 have no edge between them in the causal graph (M1) , as well as a particular parameterization of a semi-Markovian causal model in which X1 and X2 are connected by a bi-connected edge (M2). No experiment on just X1 and X2 can distinguish M1 from M2. So while the "true" causal model (i.e. the one that generated the data) might be M2, the algorithm will state that a hypothesized properly parameterized M1 is correct. For many purposes that is correct (if understood as being about correct predictions about interventions) but it is not the model that generated the data, and while M2 might suggest looking for an unmeasured common cause to manipulate, M1 does not.   There were a couple of minor typos. Theorem 2 uses Sigma before it is defined. Lemma 2 refers to delta - is that the same delta as defined in Lemma 1?  This paper has theoretical significance, particularly with respect to lower bounds on causal inference by any algorithm, as in Theorem 2. It is not clear what the practical significance of the algorithms or the theoretical results are for several reasons. First, most of the algorithms assume that the causal graph is completely known. Second, because the number of interventions (and sample sizes and time) grows so quickly with the size of the number of values of variables, the maximum indegree, and the size of the largest c-component. Even with l = 3, d =  4, K =4, K^(ld) which appears in the algorithm for choosing the covering set of interventions, is more than a million.