Collaborative Causal Discovery with Atomic Interventions

We introduce a new Collaborative Causal Discovery problem , through which we model a common scenario in which we have multiple independent entities each with their own causal graph , and the goal is to simultaneously learn all these causal graphs . We study this problem without the causal sufficiency assumption , using Maximal Ancestral Graphs ( MAG ) to model the causal graphs , and assuming that we have the ability to actively perform independent single vertex ( or atomic ) interventions on the entities . If the M underlying ( unknown ) causal graphs of the entities satisfy a natural notion of clustering , we give algorithms that leverage this property , and recovers all the causal graphs using roughly logarithmic inM number of atomic interventions per entity . These are significantly fewer than n atomic interventions per entity required to learn each causal graph separately , where n is the number of observable nodes in the causal graph . We complement our results with a lower bound and discuss various extensions of our collaborative setting . 1 Introduction . In this paper , we introduce a new model for causal discovery , the problem of learning all the causal relations between variables in a system . Under certain assumptions , using just observational data , some ancestral relations as well as certain causal edges can be learned , however , many observationally equivalent structures can not be distinguished [ Zhang , 2008a ] . Given this issue , there has been a growing interest in learning causal structures using the notion of an intervention described in the Structural Causal Models ( SCM ) framework introduced by Pearl [ 2009 ] . As interventions are expensive ( require carefully controlled experiments ) and performing multiple interventions is time-consuming , an important goal in causal discovery is to design algorithms that utilize simple ( preferably , single variable ) and fewer interventions [ Shanmugam et al. , 2015 ] . However , when there are latents or unobserved variables in the system , in the worst-case , it is not possible to learn the exact causal DAG without intervening on every variable at least once . Furthermore , multivariable interventions are needed in presence of latents [ Addanki et al. , 2020 ] . ...... t h b d e x a s ht b d e x a s ht b d e x a s Figure 1 : Examples of M causal graphs constructed from Lung Cancer dataset [ Lauritzen and Spiegelhalter , 1988 ] . Here , the causal graphs differ only in the presence of latents ( nodes with dotted square box ) , but they could differ elsewhere too . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . On the other hand , in a variety of applications , there is no one true causal structure , different entities participating in the application might have different causal structures [ Gates and Molenaar , 2012 , Ramsey et al. , 2011 , Joffe et al. , 2012 ] . For example , see figure 1 . In these scenarios , generating a single causal graph by pooling data from these different entities might lead to flawed conclusions [ Saeed et al. , 2020 ] . Allowing for interventions , we propose a new model for tackling this problem , referred here as Collaborative Causal Discovery , which in its simplest form states that : given a collection of entities , each associated with an individual unknown causal graph and generating their own independent data samples , learn all the causal graphs while minimizing the number of atomic ( single variable ) interventions for every entity . An underlying assumption is that each entity on its own generates enough samples in both the observational and interventional settings so that conditional independence tests can be carried out accurately on each entity separately . To motivate this model of collaborative causal discovery , let us consider two different scenarios . ( a ) Consider a health organization interested in controlling incidence of a particular disease . The organization has a set of M individuals ( entities ) whose data it monitors and can advise interventions on . Each individual is an independent entity that generates its own set of separate data samples1 . In a realistic scenario , it is highly unlikely that all the M individuals share the same causal graph ( e.g. , see Figures 3a and 3b from Joffe et al . [ 2012 ] in Appendix A ) . It would be beneficial for the organization to collaboratively learn all the causal graphs together . The challenge is , a priori the organization does not know the set of possible causal graphs or which individual is associated with which graph from this set . ( b ) An alternate setting is where , we have M companies ( entities ) wanting to work together to improve their production process . Each company generates their own data ( e.g. , from their machines ) which they can observe and intervene on [ Nguyen et al. , 2016 ] . Again if we take the M causal graphs ( one associated with each company ) it is quite natural to expect some variation in their structure , more so because we do not assume causal sufficiency ( i.e. , we allow for latents ) . Since interventions might need expensive and careful experimental organization , each company would like to reduce their share of interventions . The collaborative aspect of learning can be utilized if we assume that there is some underlying ( unknown ) clustering/grouping of the causal graphs on the entities . Our Contributions . We formally introduce the collaborative causal discovery problem in Section 2 . We assume that we have a collection of M entities that can be partitioned into k clusters such that any pair of entities belonging to two different clusters are separated by large distance ( see Definition 2.1 ) in the causal graphs . Due to presence of latents variables , we use a family of mixed graphs known as maximal ancestral graphs ( MAGs ) to model the graphs on observed variables . Each entity is associated with a MAG . In this paper , we focus on designing algorithms that have worst-case guarantees on the number of atomic interventions needed to recover ( or approximately recover ) the MAG of each entity . We assume that there are M MAGs one for each entity over the same set of n nodes . Learning a MAG with atomic interventions , in worst case requires n interventions ( see Proposition 3.2 ) . We show that this bound can be substantially reduced if the M MAGs satisfy the property that every pair of MAGs from different clusters have at least αn nodes whose direct causal relationships are different . We further assume that entities belonging to same cluster have similar MAGs in that every pair of them have at most βn ( β < α ) nodes whose direct causal relationships are different . We refer to this clustering of entities as ( α , β ) -clustering ( Definition 2.2 ) . A special but important case is when β = 0 , in which case all the entities belonging to the same cluster have the same causal MAG ( referred to as α-clustering , Definition 2.3 ) . An important point to notice is that while we assume there is a underlying clustering on the entities , it is learnt by our algorithms . Similar assumptions are common for recovering the underlying clusters , in many areas , for e.g. , crowd-sourcing applications [ Ashtiani et al. , 2016 , Awasthi et al. , 2012 ] . We first start with the observation that under ( α , β ) -clustering , even entities belonging to the same cluster could have a different MAG , which makes exact recovery hard without making a significant number of interventions per entity . We present an algorithm that using at most O ( ∆ log ( M/δ ) / ( α− β ) 2 ) many interventions per entity , with probability at least 1− δ ( over only the randomness of the algorithm ) , can provably recover an approximate MAG for each entity . The approximation is such 1As is common in causal discovery , for the underlying conditional independence tests , the data is assumed to be i.i.d . samples from the interventional/observational distributions . that for each entity we generate a MAG that is at most βn node-distance from the true MAG of that entity ( see Section 3 ) . Here , ∆ is the maximum undirected degree of the causal MAGs . Our idea is to first recover the underlying clustering of entities by using a randomized set of interventions . Then , we distribute the interventions across the entities in each cluster , thereby , ensuring that the number of interventions per entity is small . By carefully combining the results learnt from these interventions we construct the approximate MAGs . For the number of interventions , the linear dependence on ∆ is not uncommon for learning causal graphs [ Kocaoglu et al. , 2017 ] . Moreover , most real-world causal bayesian networks are known to have small maximum degrees ( see section 5 ) . Under the slightly more restrictive α-clustering assumption , we present algorithms that can exactly recover all the MAGs using at most min { O ( ∆ log ( M/δ ) /α ) , O ( log ( M/δ ) /α+ k2 ) } many interventions per entity ( see Section 4 ) . Again , randomization plays an important role in our approach . Complementing these upper bounds , we give a lower bound using Yao ’ s minimax principle [ Yao , 1977 ] that shows for any ( randomized or deterministic ) algorithm Ω ( 1/α ) interventions per entity is required for this causal discovery problem . This implies the 1/α dependence in our upper bound in the α-clustering case is optimal . Finally , a note about parameters . The ( α , β ) -clustering is universal , in the sense that any collection of MAGs will satisfy the ( α , β ) -clustering property for some value of α , β ( with α > β ) . Ideally , we would like in our problem instance , α to be close to 1 and β to be close to 0 . In most real-world applications , we would also expect k to be relatively small and M n , k. In Section 5 , we show experiments on data generated from both real and synthetic networks with added latents and demonstrate the efficacy of our algorithms for learning the underlying clustering and the MAGs . Related Work . A number of algorithms , working under various assumptions , for learning causal graph ( or a causal DAG ) using interventions have been proposed in the literature , e.g. , [ Eberhardt , 2007 , Hyttinen et al. , 2013 , Hu et al. , 2014 , Shanmugam et al. , 2015 , Kocaoglu et al. , 2017 , Ghassami et al. , 2018 , Lindgren et al. , 2018 , Acharya et al. , 2018 , Bello and Honorio , 2018 , Kocaoglu et al. , 2019 , Greenewald et al. , 2019 , Jaber et al. , 2020 , Addanki et al. , 2020 , 2021 , Tadepalli and Russell , 2021 ] . Saeed et al . [ 2020 ] consider a model where the observational data is from a mixture of causal DAGs , and outline ideas that recover a union graph ( up to Markov equivalence ) of these DAGs , without any interventions . Our setting is not directly comparable to theirs , as we have entities generating data and doing conditional independence tests independently ( no pooling of data from entities ) , but show stronger guarantees for recovering causal graphs , assuming atomic interventions .

Summary: The paper proposes an approach to learn multiple causal graphs, in the presence of latents, collaboratively and simultaneously by means of clustering and by performing atomic interventions. First, the work proposes the property $(\alpha,\beta)$-clustering where graphs in different clusters are "different" by a factor of $\alpha$ at least, while graphs within the same cluster are "different" by a factor of $\beta$ at most. Using this property, an approximate algorithm is proposed to (1) learn the clusters such that the $(\alpha,\beta)$-clustering property is satisfied, and (2) learn one dominant graph in each cluster assuming the clusters are more homogeneous. Next, a special case of the clustering property is considered where all the graphs within the same cluster are identical, i.e. $\beta=0$, and a learning algorithm is proposed with a lower bound on the number of interventions required to learn the graphs than the general case. Finally, the proposed algorithms are evaluated using data generated from synthetic and real causal graphs.