Scalable Intervention Target Estimation in Linear Models

1 Introduction . Directed acyclic graphs ( DAG ) are commonly used for encoding the cause-effect relationships among random variables . Extensive research has been dedicated to learning the structure of DAGs from their associated observational data . Structure learning from the observational data relies on uncovering conditional independence ( CI ) among the random variables . Since structurally distinct DAGs can encode the same set of CI relations , a DAG is identifiable only up to its Markov equivalence class ( MEC ) from the observational data . Subsequently , interventional data can be used to further refine the MEC obtained from the observational data and learn specific causal effects . This paper is motivated by addressing two significant independent challenges in causal discovery . First , most of the existing approaches for learning with interventional datasets require the intervention target set to be known , which can be a strong assumption . For instance , gene-editing technologies are known to perform cleavage at off-target genome sites [ 1 ] . Therefore , identifying the intervened nodes alone is a critical problem in structure learning , and despite its significance , it remains uninvestigated . Secondly , besides learning the structures of single DAGs , there exist application domains in which the goal is learning the structural changes between two related networks and their associated DAGs , or learning the sites of interventions . For instance , structural differences between the gene regulatory 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . networks of different subtypes of cancers can help to identify the roles of specific genes [ 2 ] . In electroencephalography analysis , the objective is to detect different brain regions that have different interactions when the subject is performing various tasks [ 3 ] . These brain regions correspond to intervened nodes in a causal graph representation . Another application area is fault detection in large-scale Internet of things and cloud applications [ 4 ] . Faulty nodes in the system can be considered as intervened nodes , and they are localized through intervention target estimation . In practice , fixing the target variable at a specific value or removing its causal dependencies is often difficult , while disturbing the distribution of a target variable is easier [ 5 ] . The type of interventions that do not remove the causal effects are commonly observed in the real world . For example , elements of an advertising system can be modified without removing the causal effects [ 6 ] . In another example , consider molecular biology , in which the effects of infused chemicals to the cell are not set to specific values nor are they known precisely [ 7,8 ] . Therefore , we consider a soft intervention setting , in which we assume the conditional distributions of the target variables are changed , but no assumption is made on the causal effects . Finally , we assume that the topological ordering remains the same after the intervention procedure . Under the soft intervention model , we propose an algorithm for estimating the intervention targets given the data from two linear SEMs associated with the observational and interventional data . Motivated by the fact that the difference of the precision matrices associated with these two models is sparse , we focus on estimating the sparse difference between precision matrices to avoid extensive conditional independence testing . This leads to a significant improvement in the computational complexity compared to those of the alternative methods . This facilitates scaling up to high-dimensional settings . Furthermore , we show that this algorithm can be used in conjunction with an observational DAG learning procedure to refine the MEC to interventional-MEC ( I-MEC ) . Besides being consistent in the population setting , we provide the finite-sample guarantees for linear SEM with Gaussian noise when the soft interventional changes between the two models are sparse . Our main contributions are as follows : • We propose an algorithm that identifies intervention targets under the interventionfaithfulness assumption . We show that our algorithm identifies I-MEC given the observational MEC . • We provide the sample complexity of our algorithm under linear SEM with Gaussian noise . • We perform experiments on both real biological and synthetic datasets to illustrate the ability to work in the high-dimensional settings and the gains compared to the relevant methods . 2 Related work . Among the broad range of approaches to intervention recovery , there exist two methods that are closely related to the scope of this paper : ( i ) estimating the difference between two DAGs , and ( ii ) learning from a combination of observational and interventional datasets . Direct estimation of differences : Direct estimation of differences in linear SEMs has been studied recently . The study in [ 9 ] proposes a PC-style algorithm for learning changes in the edge weights by testing invariances of regression coefficients and noise variances . Even though the differences can be sparse , individual models can be dense , and estimating these variables through regression can be inaccurate . Furthermore , the number of hypothesis tests is exponential in the number of nodes that are affected by the changes , which can be prohibitive even under sparse changes in the hub nodes . Estimating the difference of two precision matrices is a relatively easier task and has received attention recently [ 10–13 ] , providing finite-sample guarantees in the high-dimensional regime when such a difference is sparse . An existing study closest to the scope of our work is [ 14 ] , which proposes re-estimating precision difference to progressively eliminate nodes and estimate the difference DAG . This approach critically hinges on the assumption that the noise variance is invariant , rendering limited applicability to intervention settings . In contrast , our algorithm builds on the changes in noise variances and estimates the intervention targets efficiently . We demonstrate the effect of this difference in Appendix B.1 . Learning interventional-MEC : There is a growing number of studies on causal structure learning from both observational and interventional data . Score-based greedy interventional equivalence search ( GIES ) [ 15 ] and hybrid interventional greedy sparsest permutation ( IGSP ) [ 16 ] algorithms are proposed for settings in which there are no latent confounders . Both of these algorithms assume that the intervention targets are known . However , knowing the intervention targets can be a strong assumption since even for controlled interventional experiments , off-target effects are common . For instance , noise variances of the off-target variables can change , resulting in the intervened nodes being unknown . For causal structure learning without the knowledge of the intervention targets , an existing study includes the dynamic programming approach in [ 8 ] . This approach has limited scalability due to its time complexity being exponential in the model size . Structural discovery from interventions proposed in [ 17 ] is a neural network-based method that can learn from interventional data without target knowledge . Nevertheless , it requires discrete data and can have at most one intervened node in a given setting , limiting its applicability . The differential causal discovery from interventional data ( DCDI ) algorithm proposed in [ 18 ] extends the approach of [ 17 ] without making as strong assumptions , e.g. , learning a distribution over all potential interventional families via continuous optimization . While this method is shown to converge to the real intervention targets , its runtime becomes prohibiting even in models with as few as 100 nodes . The Unknown-target IGSP ( UTIGSP ) algorithm proposed in [ 19 ] learns the intervention targets simultaneously while learning the causal structure . Even though interventions refine the search space , a greedy search of the sparsest permutation is still too slow in the high-dimensional regime , especially when using non-Gaussian CI tests . A graphical characterization of soft interventions with unknown targets is proposed in [ 7 ] for causally insufficient systems . This algorithm , however , relies on CI tests and it is not scalable . We focus on causally sufficient systems in this paper . 3 Problem definition . Let G , ( [ p ] , E ) be a DAG with the node set [ p ] , { 1 , . . . , p } and the edge set E ⊆ [ p ] × [ p ] . We denote the directed edge from i ∈ [ p ] to j ∈ [ p ] by i→ j . We associate a random variable Xi with i ∈ [ p ] , and accordingly , define the random vector X , ( X1 , . . . , Xp ) > . We consider a linear SEM , according to which X = B > X + , ( 1 ) where B ∈ Rp×p is the autoregressive matrix in which Bij 6= 0 if and only if i → j in G. The random vector ∈ Rp×1 has zero-mean and covariance matrix Ω , diag ( σ21 , . . . , σ2p ) . We denote the covariance matrix of X by Σ and its inverse ( the precision matrix ) by Θ , which satisfies Θ = ( I −B ) Ω−1 ( I −B ) > . For entries of Θ , we have Θij = − Bij σ2j − Bji σ2i + ∑ k∈ch ( i ) ∩ch ( j ) BikBjk σ2k , ∀i 6= j , ( 2 ) and Θii = σ−2i + ∑ j∈ch ( i ) σ−2j B 2 ij , ∀i ∈ [ p ] , ( 3 ) where ch ( i ) , pa ( i ) , de ( i ) , and an ( i ) denote children , parents , descendants , and ancestors set of node i in DAG G , respectively . From the observational data , a DAG can be learned only up to its MEC [ 20 ] . Interventions are used to increase the identifiability of a DAG by removing all causes of the intervention target ( perfect intervention ) or modifying those relationships without removing them completely ( imperfect intervention ) . We consider the following soft intervention setting , which does not remove causal effects ( from direct parents ) on intervention target nodes , and hence , is more practical . Soft intervention model . In this model , interventions correspond to disturbing the target nodes i ∈ I by changing the variances of their noise variables , while the cause weights , i.e. , the weights Bpa ( i ) , i , { Bj , i : j ∈ pa ( i ) } , can vary freely . This intervention procedure on the initial DAG results in a second DAG with new parameters . Let G ( 1 ) represent a linear SEM prior to intervention with parameters B ( 1 ) , ( 1 ) , and G ( 2 ) be the linear SEM after the intervention , with parameters B ( 2 ) , ( 2 ) . The intervention target set that relates these two DAGs is I , { i : σ ( 1 ) i 6= σ ( 2 ) i } . ( 4 ) Accordingly , denote the covariance and precision matrices of these two models by Σ ( 1 ) , Σ ( 2 ) , Θ ( 1 ) , and Θ ( 2 ) . Accordingly , denote the differences between the two models by ∆B , B ( 1 ) −B ( 2 ) and ∆Θ , Θ ( 1 ) −Θ ( 2 ) . For a subset of nodes S ⊆ [ p ] , denote the precision matrix of the random vector XS , { Xi : i ∈ S } by ΘS . We also denote the set of changed nodes by S∆ , { k : ( ∆Θ ) k , k 6= 0 } and denote its size by p∆ , |S∆| . According to ( 3 ) , this set consists of all the intervened nodes and their parents . In this paper , we estimate the intervention targets I given the data from two SEMs under the soft intervention model . Furthermore , we estimate the non-intervened parents of the targets , which update the given observational MEC into the I-MEC . Formally , we define ψ : Σ ( 1 ) × Σ ( 2 ) → ( Î , pa ( Î ) ) as the estimator that maps the covariance matrices of the observational and interventional data to an intervention target set estimate and their parents . We aim to maximize the probabilities of ψ recovers I and their parents . To this end , we define P , P ( I = Î ) and Q , P ( pa ( I ) = pa ( Î ) ) . ( 5 ) We will show that the algorithm proposed in Section 4 exactly recovers I and the non-intervened parents of the members of I . Hence , given the observational MEC , we obtain the I-MEC .

Summary: Given two datasets generated from two DAGs, where the second DAG is assumed to be created from the first DAG by varying the noise level in a subset of nodes (called intervened nodes) in the linear structure equation model, this paper proposed an efficient algorithm to identify the subset of intervened nodes and their parents.  The proposed algorithm consists of three steps. First, apply precision difference estimation (PDE) to identify the set of non-intervened nodes. This is feasible based on the established fact that the diagonal entry of the precision matrix difference is non-zero if and only if the corresponding node or its child receives intervention. Second, from the set of non-intervened nodes identified from step 1, form equivalent classes. Third, those equivalent classes are further processed in step 3.  The computational complexity of PDE is cubic w.r.t the number of nodes. In step 1, PDE nees to be run multiple times. Furthermore, in step 3, the complexity could be exponential. The algorithm is provided to be consistent in Theorem 1. The sample complexity bound is also established in Theorem 3.  In the experimental results, the proposed algorithm is shown to be able to recover the ground truth set of intervention targets with high precision given large number of samples. It's also much faster than its competing alternative of UT-IGSP algorithm. the proposed algorithm is also further evaluated on real-world datasets, including protein signaling data and perturb-seq gene expression data.