Intervention Target Estimation in the Presence of Latent Variables
Abstract
This paper considers the problem of estimating unknown intervention targets in causal directed acyclic graphs from observational and interventional data in the presence of latent variables.
The focus is on linear structural equation models with soft interventions.
The existing approaches to this problem involve performing extensive conditional independence tests, and they estimate the unknown intervention targets alongside learning the structure of the causal model in its entirety.
This joint learning approach results in algorithms that are not scalable as graph sizes grow.
This paper proposes an approach that does not necessitate learning the entire causal model and focuses on learning only the intervention targets.
The key idea of this approach is leveraging the property that interventions impose sparse changes in the precision matrix of a linear model.
The proposed framework consists of a sequence of precision di ff erence estimation steps.
Furthermore, the necessary knowledge to refine an observational Markov equivalence class (MEC) to an interventional MEC is inferred.
Simulation results are provided to illustrate the scalability of the proposed algorithm and compare it with those of the existing approaches.
INTRODUCTION
Enabling modern machine learning systems to reason involves predicting the e ff ect of an intervention and counterfactual estimation [Pearl, 2009].
Forming such predictions crucially depends on the knowledge of causal models [Pearl and Mackenzie, 2018].
One approach to represent causal knowledge is through a causal Bayesian network, which is a directed graphical model specified by a directed acyclic graph (DAG).
The nodes of a DAG represent random variables, and its directed edges represent the cause-and-e ff ect relationships among the random variables.
Such a model facilitates factorizing the observed distribution, where each factor is a conditional distribution of a variable given its causal parents.
These conditionals specify the local causal mechanisms of the variables.
However, based on purely observational data, a causal DAG is identifiable only up to an equivalence class of DAGs.
Such uncertainty is because di ff erent DAGs can encode di ff erent ways of factorizing the same observed distribution into conditionals.
The equivalence class of DAGs that can be identified from the observational data alone is called the Markov equivalence class (MEC)
[Peters et al., 2017].
To reduce the ambiguity in the MEC obtained from the observational data, interventional data can be leveraged.
Intervening on a variable refers to modifying the causal mechanism (the conditional distribution) that connects this variable and its parents in the true causal DAG while leaving the other factors unchanged.
The combination of observational and interventional data reduces the number of possible factorizations that are consistent with both data types.
In this paper, we perform soft interventions.
A soft intervention induces a change in the causal mechanism by replacing it with a di ff erent one without requiring the causal e ff ects on the target node to be removed.
While hard interventions, e.g., assigning fixed values to intervention targets, can be performed too, there are applications in which soft interventions are better suited for modeling the experiments.
For instance, soft interventions can e ff ectively model altering the gene expressions for cellular reprogramming [Zhang et al., 2021].
In a broad range of applications, when interventional data is available, the variables whose causal mechanisms have been changed, called the intervention targets , are unknown.
For instance, there is a recent growing interest in using causal discovery for fault localization in microservices systems in cloud-native applications [Bogatinovski et al., 2021, Aggarwal et al., 2020].
These systems are built as an interconnected set of loosely coupled services across various layers [Kim et al., 2013, Mariani et al., 2018].
Such systems are vulnerable to unwanted changes (e.g., equipment failure and attacks).
During the faulty operation of these systems, it is imperative to localize the faults quickly.
The root causes of the faulty operations are modeled as interventions to the system.
Hence, the data is collected under (unknown) faults, rendering fault localization a causal discovery task from interventional data of unknown intervention targets.
Furthermore, a fault in the operation of a node, e.g., a delay, is closer to soft interventions than to hard interventions since the causal parents can still a ff ect the operation of the node.
Another example is gene knockout experiments in biology.
In these experiments, a target set of genes is knocked out in an assay, and gene expressions are collected.
These are known to a ff ect o ff -target genome sites [Fu et al., 2013].
Sometimes drugs are injected into protein signaling networks, and the expression levels are measured.
In these settings, the intervention targets are unknown [Sachs et al., 2005, Ness et al., 2017].
Identifying unknown intervention targets in fully observed graphs was recently explored [Varici et al., 2021].
However, in this study, all variables of a true causal DAG are typically not observed.
This induces confounding between observed variables due to unobserved or latent variables.
A model with such confounding is called a causally insu ffi cient model.
Recent studies have characterized the interventional
MEC for causally insu ffi cient models and have provided algorithms for learning their structures.
These algorithms leverage invariance testing and conditional independence testing by using both interventional and observational data and accommodate both settings of known and unknown intervention targets [Mooij et al., 2020, Kocaoglu et al., 2019,
Jaber et al., 2020].
In these algorithms, the intervention targets are usually learned along with the interventional MEC.
In this paper, we focus on the following question: is there an e ffi cient way to learn only the intervention targets given interventional and observational data?
Our Contributions: We address the above question in linear structural equation models (SEMs) under soft interventions.
We first show that the di ff erence in the precision matrices of the interventional datasets can be used to deduce the intervention status of a node.
Next, we use the fact that these precision di ff erences have sparse support to narrow down our interest to the nodes directly a ff ected by the interventions.
Then, we show how to refine this sparse set by repeated precision di ff erence estimations to obtain the intervention targets.
In the process, we also infer the causal knowledge newly induced by the interventions.
Finally, using these elements, we propose a scalable algorithm to estimate the intervention targets.
There are two studies whose scopes are close to that of this paper.
Jaber et al.
[2020] characterize the interventional MEC for soft interventions and proves that the intervention targets can be identified only up to a superset that they graphically describe.
Noting this result, in this paper, we focus on estimating this superset, which we call the e ff ective intervention targets .
In a di ff erent study, Varici et al.
[2021] address a related problem.
However, their method is limited to only causally su ffi cient models.
We present theoretical results for causally insu ffi cient models, which are non-trivial generalizations that combine the precision di ff erence approach to the problem and the graphical characterization of the soft interventions on causally insu ffi cient models.
The existing interventional causal discovery algorithms for insu ffi cient systems jointly learn the causal structure and the intervention targets.
These approaches require performing a significant number of conditional independence and invariance tests, a major impediment to these algorithms for being scalable to large graphs [Jaber et al., 2020].
However, unlike interventional causal discovery, there exist highly e ffi cient algorithms for causal discovery with observational data.
One of the byproducts of our results is that our scalable algorithm for intervention target discovery can be used in conjunction with any observational learning algorithm for insu ffi cient systems to refine the observational MEC e ffi ciently to an interventional MEC.
Finally, we perform experiments on real and synthetic datasets to illustrate the scalability of the proposed algorithm.
RELATED WORK
At its core, this paper infers causal knowledge from interventional settings through an invariance criterion.
The existing literature on related topics is discussed next.
Interventional causal learning for causally su ffi cient systems.
There is extensive literature on interventional learning for causally su ffi cient models.
Among them, Eaton and
Murphy [2007] proposed a dynamic programming approach to interventional learning.
Hauser and Bühlmann [2012] considers the interventional MEC under hard interventions and provides a score-based algorithm for interventional learning.
Rothenhäusler et al.
[2015] learn causal cyclic graphs using shift interventions.
Ghassami et al.
[2018] consider multi-domain data without explicitly formulating the different domains via interventions.
Its method estimates the causal order by generalizing the invariance of parameters to the independence of the changes in the parameters across domains.
Huang et al.
[2020] use the distribution shifts that can be the results of interventions to determine the causal directions.
Their method works under a pseudo-causal sufficiency condition in which the values of the unobserved confounders are fixed in each domain.
Yang et al.
[2018] characterize interventional MEC under hard and soft interventions using invariance testing and provides a learning algorithm when the intervention targets are known.
The algorithm of Squires et al.
[2020] greedily searches over the space of permutations to score DAGs when the intervention targets are unknown.
Ke et al.
[2019] and Brouillard et al.
[2020] leverage di ff erentiable methods through continuous optimization to learn the causal structure from interventional data.
For linear SEMs and causally su ffi cient models, Wang et al.
[2018] propose to learn the di ff erence graph, which is the set of edge weights in the linear SEM that have been changed across two environments.
Ghoshal et al.
[2021] leverage precision di ff erence estimates to address the same problem under more stringent assumptions.
Varici et al.
[2021] use precision di ff erence estimates and achieves a higher level of scalability through a hierarchical grouping of the nodes.
Learning from observational data for causally insu fficient systems.
The fast causal inference (FCI) algorithm of Spirtes et al.
[2000] is a classic constraint-based method for learning causally insu ffi cient models from observational data.
Many e ffi cient variants such as the really fast causal inference (RFCI) algorithm of Colombo et al.
[2012] and the greedy fast causal inference (GFCI) algorithm of Ogarrio et al.
[2016] have been proposed to improve scalability.
Bernstein
[Table] extend the greedy permutation search to partially ordered sets to include the e ff ects of latent variables in ordering.
Learning from interventions on causally insu ffi cient systems.
Triantafillou and Tsamardinos [2015] consider multiple interventions for causally insu ffi cient systems.
Their algorithm applies ideal hard interventions and provides a solution based on constraint satisfaction and conditional independence testing.
Mooij et al.
[2020] propose a joint causal inference framework to pool interventional datasets to learn the causal graph.
Jaber et al.
[2020] characterize the interventional MEC and propose a variant of FCI to learn from soft interventional data in causally insu ffi cient systems.
The key shortcoming of these methods is that their runtime becomes prohibitive for large graphs.
PRELIMINARIES
We introduce some concepts and notations pertinent to causal discovery in causally insu ffi cient systems.
Let D ≜ ( W , E ) denote a causal graph in which W represents the set of nodes and E represents the set of edges.
Denote the number of nodes by p ≜ | W | .
We associate the random variable
X i to node i , for i ∈
[ p ] ≜ { 1 , . .
. , p } , and accordingly define the random vector X ≜ ( X 1 , . .
, X p )
⊤ 1 .
We consider a linear SEM, according to which
[Equation]
where B ∈ R p × p is the edge weights matrix in which
B i , j (cid:44)
if and only if X i → X j ∈ E .
The random noise vector ϵ ∈ R p × 1 has zero mean with covariance matrix Ω ≜ diag ( σ 21 , . . .
, σ 2 p ).
We denote the covariance matrix of X by Σ , and the precision matrix by Θ , which satisfies
[Equation] ⊤ .
For the entries of Θ we have
[Equation]
where ch ( i ) denotes the set of children of node
i ∈ [ p ].
In the causal graph D , we have two sets of nodes: a set of observed variables denoted by V , and a set of latent variables denoted by L .
Clearly, V ∪ L = W .
The observational data, consequently, is represented by { X i : i ∈ V } .
From the observational data alone, a DAG with only observed variables can be identified up to its MEC [Verma and
Pearl, 1992].
For causally insu ffi cient systems with latent variables L , we can only describe the MEC in terms of a family of graphs called maximal ancestral graphs (MAGs), which we formally specify later in this section.
The MAG associated with V represents the pairwise ancestral and confounding relationships among the observed variables { X i : i ∈ V } that cannot be made conditionally independent.
Therefore, for the true causal graph D , there exists a unique
MAG.
This MAG cannot be identified uniquely.
However, it is possible to recover it up to a family of equivalent MAGs that contains the true one.
Next, we describe how a MAG is obtained from a DAG and then proceed to describe the
MEC of MAGs and how they are represented.
Mixed Graphs: From a structure learning perspective, causally insu ffi cient systems are often represented by mixed graphs.
A mixed graph can contain both directed ( → ) and bi-directed ( ↔ ) edges.
In our notations, we use ←◦ to emphasize that an edge represents either a directed or a bidirected edge.
If there is a directed path from node A to node B , then A is an ancestor of B , and B is a descendant of A .
Bi-directed edges create spouses , that is, A is a spouse of B if A ↔ B is present.
A node on a path is a collider if both of its edges on the path are into the node.
A triple ⟨ X , Y , Z ⟩ is an unshielded collider if X →◦ Y ←◦ Z , and X and Z are not adjacent.
A path ⟨ X , . .
, W , Z , Y ⟩ is a discriminating path for Z if every node between X and Z is a collider on the path, and is also a parent of Y .
An inducing path relative to L is a path on D such that on this path, every non-endpoint node X (cid:60) L is a collider on the path, and every collider is an ancestor of an endpoint of the path.
Maximal Ancestral Graphs: Consider the causal graph
D =
( V ∪ L , E ).
A unique mixed graph called the MAG [Richardson and Spirtes, 2002] M D over V has the following three properties: (i) in a MAG, there exists an edge between two nodes if and only if their associated variables cannot be made conditionally independent (or d-separated) by conditioning on any subset of observed variables in the true D ; (ii) if there is an edge in the skeleton that represents the ancestral relationships among the variables in V in the true D [Zhang, 2008], then a directed edge is used to represent this edge; and (iii) if there is an edge in a MAG that connects two variables that do not have any ancestral relationship in D , then a bi-directed edge ↔ is used to represent it.
We note that the relationships between DAGs and
MAGs are many-to-one, i.e., di ff erent DAGs can have the same MAG.
Similar to the DAGs, a MAG can be identified only up to a family of MAGs that are Markov equivalent.
This Markov equivalence class is represented by a partial ancestral graph (PAG).
Markov Equivalence: Two MAGs are Markov equivalent if and only if they have (i) the same adjacencies; (ii) the same unshielded colliders; and (iii) if a path π is a discriminating path for Z in both graphs, then Z is a collider on π in one graph if
and only if it is a collider on π in the other graph as well.
A PAG represents the MEC of a MAG that can be learned from the observational data.
The skeletons of all
MAGs in the MEC are identical.
Therefore, the PAG has the same skeleton as all members of the MEC.
If an edge is oriented as → or ↔ , this orientation is fixed for that edge in all MAGs of the MEC.
If an edge in a PAG is oriented as ←◦ , this implies that there are at least two MAGs in the
MEC, such that for the first MAG, this edge is oriented as ↔ and for the second MAG, this edge is oriented as ← .
An edge with circles on both ends means there are three MAGs in the MEC with three distinct orientations ← , → , and ↔ .
We denote the MAG corresponding to the DAG D
=
( V ∪
L , E ) by M D .
Let pa ( A ), ch ( A ), sp ( A ), an ( A ), and de ( A ) denote the sets of parents, children, spouses, ancestors, and descendants of a node A .
We also create the set ps ( A ) =
pa ( A ) ∪ sp ( A ) to denote the union of parents and spouses of a node A .
We denote these relationships with respect to a graph, e.g., pa D ( A ).
The subscript is dropped if the specified graph is clear from the context.
PROBLEM STATEMENT
Interventions on causal models improve the identifiability of the underlying causal structure.
We consider a soft intervention model, which changes the conditional distributions of an intervention target node given its true parents (both observed and unobserved) in the causal DAG D without completely removing the causal e ff ects of its parents.
Soft Intervention Model.
Assume that we have n interventional settings, and denote the collection of the intervention target sets by I ≜ { I ( j ) : j ∈ [ n ] } .
In the j -th setting, the nodes in I ( j ) ⊂ V are targeted for intervention.
Soft interventions in the linear SEM specified in (1) change the conditional distributions of variables { X i : i ∈ I ( j ) } .
Under these changes (i) the variances of the noise terms { ϵ i : i ∈ I ( j ) } change, and (ii) the weights connecting the parents of the nodes associated with { X i : i ∈ I ( j ) } in the linear SEM may change.
In other words, { B pa ( i ) , i : i ∈ I ( j ) } , where B pa ( i ) , i ≜ { B u , i : X u ∈ pa ( X i ) } , may vary freely.
We also note that this formulation can readily work with mean-shift interventions that change the mean of the noise variables (see supplementary material Section D.1 for details).
Post-intervention linear SEMs have new parameters.
We denote the linear SEM parameters associated with interventions I ( j ) by B ( j ) and Ω ( j ) ≜ diag (cid:16) ( σ ( j ) 1 ) 2 , . . .
, ( σ ( j ) p ) 2 (cid:17) .
Since the noise variance terms change under soft interventions,
I ( j ) is described as follows:
[Equation]
A node can be targeted in multiple interventional settings, e.g., i ∈ I ( j )
∩ I ( l ) . We assume that each target set have different mechanisms such that upon interventions on sets I ( j ) and I ( l ) , we have σ ( j ) i (cid:44) σ ( l ) i for all i ∈ I ( j ) ∪ I ( l ) .
We note that this assumption is purely for simplicity in the notation, and can be dropped by denoting the exact mechanism applied on a node under each setting.
Identifiability conditions of the causal graphs with unknown soft interventions and the corresponding graphical characterization are established by Jaber et al.
Importantly, causal graphs with the same observed variables but di ff erent latent variables and intervention targets can still belong to the same MEC.
We follow the augmented graph construction of Kocaoglu et al. [2019] and Jaber et al. [2020] to represent the MEC’s under interventions graphically. First, we construct the augmented graph as follows: for each pair of intervention targets I ,
J ∈
I , the augmented graph Aug I ( D ) appends the causal graph D with an auxiliary node and assign directed edges from this node to each node in H
=
I ∪ J .
We denote the set of these auxiliary nodes by F , and refer to the members of F as F -nodes.
In the example in Fig.
1, we have observational setting ∅ and intervention target set { X } , so there is just one pair of target sets.
An F -node is created corresponding to this pair, and the edge F → X is drawn since X is the only node in the set ∅ ∪ { X } .
Definition 1 (Augmented Graph) Consider a causal graph D =
( V ∪ L , E ) and a set of intervention targets I .
Define the multiset H as H =
{ I ∪ J : I , J ∈ I} .
Given H , generate h ≜ |H| nodes F ≜ { F i : i ∈ [ h ] } and define the augmented graph of D as Aug I
( D ) ≜
( V ∪ L ∪ F , E ∪ E ) , where E ≜ { ( F i , V ) : i ∈
[ h ] , V ∈ H i } .
The study in citetsoftunknown20 shows that the augmented graph exactly represents the separation statements among the random variables in interventional settings.
Similar to obtaining a unique MAG from a DAG, a corresponding maximal ancestral graph for the augmented graph is constructed next.
In the example in Fig. 1, F → W and Z → W
[Figure] [Equation]
W L
Y X Z W
Y F
[Equation]
Definition 2 ( I -MAG)
Given a causal graph D =
( V ∪
L , E ) and a set of intervention targets
I , we define I -MAG to represent the maximal ancestral graph constructed over
V from Aug I
( D ) , i.e., MAG ( Aug I ( D ) , and denote its
Corresponding to every pair of intervention sets I ( j ) and
I ( l ) , define the set I jl ≜
I ( j ) ∪ I ( l ) .
Denote the single F -node associated with I ( j ) and I ( l ) by F jl ∈ F , and denote the set of nodes adjacent to F jl in I -MAG by
[Equation]
We remark that, in causally su ffi cient systems, K jl
= I jl .
However, in the presence of latent variables, one cannot distinguish between the nodes in K jl \ I jl and I jl according to the I -MAG.
Therefore, we will focus on estimating K jl , which we call the e ff ective intervention targets .
We note that the observational setting can be considered as an interventional setting with an empty target set.
When there exist more than two interventional settings, there are multiple F -nodes and intervention targets.
Accordingly, we denote the set of intervention targets by
[Equation]
Problem Statement.
We focus on two estimation problems.
In the first problem, we estimate the set of intervention targets K given the data from linear SEMs with latent variables under soft interventions.
We denote the estimate of K by ˆ K .
Our objective is to design the estimator ϕ : (cid:16) R m ×| V | (cid:17) n → (cid:16) 2 V (cid:17) n , in which | V | denotes the number of the observed variables, n denotes the interventional settings, and m denotes the number of samples in a setting.
In the second problem, based on the estimate ˆ K , for any set
K ∈ ˆ K , we consider the problem of estimating the parents and spouses of K in the augmented MAG ( I -MAG).
For any K ∈ ˆ K , we denote the set of parents and spouses of the nodes in K by ps ( K ), and denote its estimate by ˆ ps ( K ).
Therefore, our second objective is to design the estimator ϕ ps ( K ) : 2 V → (cid:16) 2 V (cid:17)
| K | .
These estimates (i.e., ˆ K and { ˆ ps ( K ) :
K ∈ ˆ K} ) are su ffi cient to refine the observational PAG to the MEC of the I -MAG.
In the rest of the paper, we denote this interventional refinement of observational PAG by ψ -PAG.
MAIN RESULTS AND ALGORITHM
Overview.
In this section, we provide our theoretical results and our Pre cision D
i ff erence-based I ntervention
T arget E stimato r (PreDITEr) algorithm.
With scalability as the central objective, we focus on estimating only the e ff ective intervention targets.
This is a computationally simpler task compared to learning the causal structure of a DAG, and, consequently, facilitates scalability.
The pivotal idea in our algorithm’s design is that soft interventions result in only sparse changes in the precision matrix of the linear SEM.
Hence, the precision matrix di ff erences have traces of the identities of the intervention sites.
We analytically establish how to use the precision matrix di ff erences between a pair of interventional settings to identify the underlying intervened sites.
Upon establishing this property, we then devise an algorithm that successively identifies pairs of intervention settings and estimates the di ff erence between their associated precision matrices.
These successive estimates are aggregated to identify the intervention targets.
Given the extensive literature on estimating precision matrix di ff erences, we can adopt any generic precision di ff erence estimation (PDE) algorithm to generate the estimates that we need in our algorithm.
Once we estimate the target intervention sites, we also provide an estimate for the set of parents and spouses of each of the nodes deemed to be an intervened node.
Theoretically, this information enables the increased identifiability of the causal structure due to the interventions.
We start describing the details by introducing the precision di ff erence estimation procedure.
Precision Di ff erence Estimation (PDE).
When the difference between two linear SEMs is sparse, the di ff erence of their respective precision matrices will also be sparse. Hence, for the two intervention target sets I ( j ) and I ( l ) , the di ff erence between their precision matrices ∆ jl ≜ Θ ( j ) − Θ ( l ) will be sparse.
In this paper, we use the algorithm of Jiang et al.
[2018] to estimate ∆ jl .
The algorithm computes sample covariance matrices ˆ Σ ( j ) and ˆ Σ ( l ) from the data.
Then, it solves the following convex optimization problem with the alternating direction method of multipliers (ADMM):
[Equation]
where λ is a tuning parameter.
We note that there exist alternative approaches to PDE [Zhao et al., 2014, Yuan et al., 2017]. Any method that is guaranteed to converge to the correct solution can be used as our PDE subroutine in a modular way. We have chosen the method of Jiang et al.
[2018] due to its significant advantage in computational complexity compared to the others ( O ( p 3 ) vs. O ( p 4 )).
Next, we define the marginal SEM over a subset of observed variables.
Definition 3 (Marginal SEM)
Corresponding to a subset of nodes S ⊆ V , we define ( B S , ϵ S ) as the marginal SEM that characterizes the relationship among the random variables
X S ≜ { i : i ∈ S } .
Accordingly, the corresponding precision matrix is denoted by Θ S .
The parametrization of a marginal
SEM is given by the following lemma.
Lemma 1 (Ghoshal et al.
Corresponding to a subset S ⊆ W , denote the removed set of nodes by U ≜ W \ S and define U i ≜
U ∩ an ( i ) , for i ∈ S .
For i , j ∈ S , we have
[Equation]
Before describing the theoretical results, we need the following faithfulness assumption.
This assumption rules out the pathological cases in which the e ff ect of an intervention is canceled by other changes in the system.
Faithfulness assumptions are generally needed for successful learning.
Assumption 1 ( I -faithfulness)
For any choice of i , j ∈
S ⊆ V , we have the following properties:
• If σ (1) i (cid:44) σ (2) i , then σ (1) S , i (cid:44) σ (2) S , i .
• If σ (1) S , i (cid:44) σ (2) S , i , then [ Θ (1) S ]
i , i (cid:44)
[ Θ (2) S ]
i , i .
If further [ B S ] j , i (cid:44) 0 in either model, then [ Θ (1) S ] i , j (cid:44)
[ Θ (2) S ] i , j .
5.1 THEORETICAL RESULTS
For the rest of the discussion, we consider a pair of interventional settings.
Without loss of generality, let them be I (1) and I (2) .
Denote the di ff erence in their precision matrices by ∆ 12 = Θ (1) − Θ (2) , and the di ff erence in marginal precision matrices for set S by ∆ 12 S =
Θ (1) S − Θ (2) S .
For simplicity in the notation, we denote the corresponding F -node F 12 by
F , K 12 by K , ∆ 12 by ∆ , and ∆ 12 S by ∆ S .
We also denote the set of a ff ected nodes among the observed variables by
S ∆ ≜ { i : [ ∆ V ] i , i (cid:44) 0 } .
Separation Property for Invariance.
For a nonintervened node i ∈ V \ K , there is no edge between F and i in I -MAG.
Therefore, there exists a set S that separates F and i , and the conditional probability distribution of
X i is invariant given S \{ X i } .
Then, the conditional mean and variance of X i , and subsequently σ S , i , are invariant.
Finally, applying the result of Wang et al.
[2018], [ Θ S ]
i , i =
σ − 2 S , i is also invariant.
Therefore, the set S that separates F and i yields [ ∆ S ]
i , i = 0 by the definition of ∆ S .
Theorem 1 Consider an F ∈ F and an observed node
V ∈ V in the augmented MAG ( I -MAG).
Then, ( F , V ) ∈
E I
if and only if ∄ S ⊆ V such that [ ∆ S ] V , V = 0 .
Theorem 1 states the existence of a conditioning set S for any non-intervened node V , that makes the corresponding diagonal entry of the precision matrix invariant.
In the following lemma, we show that the ancestors of V within the set of a ff ected nodes S ∆ su ffi ce to separate F and V .
Lemma
For a node V ∈ S ∆ \ K , consider the set
S =
S ∆ ∩ an ( V ) .
Diagonal entry corresponding to V in the precision matrix of the marginal SEM over S is invariant, i.e., [ ∆ S ] V , V = 0 .
Lemma 2 implies that we can eliminate all the nonintervened nodes (i.e., nodes not in the e ff ective intervention target set K ) in S ∆ by computing PDE for each subset of S ∆ .
Therefore, we can identify K with 2 | S ∆ | number of PDEs.
Now that we have a way to recover K , we show how to identify the parents and / or spouses of the intervened nodes.
This property will play a critical role in improving the identifiability of the MAGs under interventions.
Lemma 3 Consider K ∈ K and J ∈ V \ K .
If K ←◦ J in
I -MAG, there does not exist S ⊆ S ∆ containing { K , J } such that [ ∆ S ] K , J = 0 .
On the other hand, if K → J, or there is no edge between them in I -MAG, there exists a set S ⊆ S ∆ containing { K , J } such that [ ∆ S ] K , J = 0 .
Lemma 2 and Lemma 3 are su ffi cient to design our algorithm for learning K .
5.2 LEARNING ALGORITHM
We leverage the results in Lemma 2 and Lemma 3 to learn the intervention targets K from a tuple of interventional distributions generated by some unknown pair ⟨D , I⟩ .
Algorithm 1 presents our main learning algorithm PreDITEr
that uses the results to learn K , and subsequently ps ( K ) for K ∈ K .
We briefly describe PreDITEr
and the rationale underlying its design.
Algorithm 1 (PreDITEr) takes sample covariance matrices of interventional data as inputs.
Since estimating intervention targets K for each pair of interventional settings is independent, we investigate each pair individually. For each K K do 17: if ∄ S
S
K J S , [ ∆ S ] K , J
then 18: 19: end 20: end 21: end
Precision Di ff erence Estimation (PDE) ( ˆ Σ ( j ) , ˆ Σ ( l ) ) 1: Estimate ∆ jl
=
(ˆ Σ ( j ) ) − 1 − (ˆ Σ ( l ) )
− 1 using algorithm of
Jiang et al.
2: Symmetrize ∆ jl : set ∆ jl = ( ∆ jl + ∆ ⊤ jl ) / 2.
3: Threshold jl : set jl ] u , v = 0 if | [ ∆ jl ] u , v | < ε .
4: Return ∆ jl pair of interventional distributions (or the corresponding
F -node), we first estimate the set of a ff ected nodes S ∆ (lines 7 and 8).
Then, we estimate precision di ff erence ∆ S for each subset S of S ∆ .
If there does not exist a set S for a node
V ∈ S ∆ such that [ ∆ S ] V , V = 0, then by Lemma 2, V is an intervened node and belongs to K (lines 10-15).
After identifying K , consider a K ∈ K and J ∈ S ∆ \ K .
If there does not exist a set S such that [ ∆ S ] K , J = 0, by Lemma 3, J belongs to ps ( K ) (lines 16-20).
Algorithm 1 uses PDE as a subroutine.
Hence, the quality of the estimate formed by Algorithm 1 hinges on those of the precision di ff erence estimates.
To assess the accuracy of
Algorithm 1 in estimating the intervention targets irrespectively of the PDE subroutine used, we provide populationlevel results.
In the following theorem, we establish that
Algorithm 1 has perfect estimation if the underlying PDE subroutine performs perfectly.
This result allows decoupling the accuracy of Algorithm 1 from that of the PDE subroutine used.
In practice, however, PDE subroutines are imperfect, which is imposed by having access to only finite samples.
To address the convergence to the correct estimates, we discuss the sample complexity and convergence guarantees of the algorithm of Jiang et al.
[2018] in supplementary material Section B.
Theorem 2 When the covariance estimates are perfect and
Assumption 1 holds, Algorithm 1 perfectly estimates the set of e ff ective intervention targets K under soft interventions with probability 1 .
Furthermore, Algorithm 1 recovers non-intervened parents and / or spouses (i.e., ps ( K ) ) of an intervened node K with probability 1 .
5.3 RECOVERING ψ -MARKOV EQUIVALENCE
Next, we show how we can use the intervention target recovery of Algorithm 1 to refine the observational MEC represented by a PAG to the interventional MEC for soft interventions ψ -PAG.
We first review the interventional equivalence characterization approaches in the existing literature.
The ψ -Markov equivalence property, i.e., the conditions for two I -MAGs to be Markov equivalent, is characterized by Jaber et al.
[2020, Theorem 1].
For two MAGs M 1 and Mto be ψ -Markov equivalent:
• M 1 and M 2 must have the same skeleton.
• M 1 and M 2 must have the same unshielded colliders.
• If a path π is a discriminating path for a node V in both M 1 and M 2 , then V is a collider on the path in one graph if
and only if it is a collider on the path in the other.
The following theorem builds on the results of Theoremand Lemma 3 to obtain ψ -PAG.
Theorem 3 ( ψ -PAG) Given the PAG for the MAG M , and the results of Algorithm 1, i.e., the sets K , ps ( K ) ∀ K ∈ K , we can obtain ψ -PAG of I -MAG.
EMPIRICAL RESULTS
First, we run our PreDITEr algorithm on synthetically generated data from linear SEMs to recover intervention targets.
Next, we provide comparisons with the state-of-the-art method.
Finally, we apply our method to a biological dataset to illustrate its applicability to real data. [Figure]
6.1 SYNTHETIC DATA
We test the e ffi ciency of PreDITEr for recovering the intervention targets.
We generate 100 realizations of Erd˝os-Rényi random DAGs with the expected neighborhood size c = 2.
We consider one interventional setting in addition to the observational one, i.e., I =
⟨∅ , I ⟩ .
Therefore, we are estimating a single target set K .
For each model, we set the number of latent variables to | L | = 5, and the number of intervened nodes to | I | = 5.
The edge weights of the causal model, i.e., the entries of B , are sampled independently at random according to the uniform distribution on [ − 1 , − 0 .
25] ∪ [0 .
The additive Gaussian noise terms have distribution N (0 , I p ).
The intervention targets are selected randomly from the observed variables V .
For the intervened nodes I ∈
I , upon intervention, the variance of the noise term ϵ I changes to 2.
We run PreDITEr
with a varying number of samples on graphs with varying sizes p .
Figure 2 illustrates the target recovery performance.
Specifically, it shows that our method recovers the intervention target with high F1 scores.
We emphasize that PreDITEr
can easily process large graphs (e.g., p = 100 nodes), and have less than 1 second average runtime for the simulations shown in Figure 2.
This scalability is due to its computational complexity of O (2 | S ∆ | ).
Since the size of S ∆ is determined only by the number of intervened nodes and their parents / spouses, our method is not directly a ff ected by the graph size p .
6.2 COMPARISON TO THE RELATED WORK
We compare the scalability and accuracy of PreDITEr to those of two competing methods under various settings: the ψ -FCI algorithm of Jaber et al.
[2020] and the FCI-JCI algorithm of Mooij et al.
We note that both of these algorithms solve a more general problem than the linear
SEMs we are considering.
To the best of our knowledge, there is no algorithm specifically designed for linear SEMs, and these are the only two methods that can be applied to our setting.
Therefore, we compare our results to those of these two methods.
Jaber et al.
[2020] do not provide simulations for graphs that have more than a few nodes since ψ -FCI requires an exponentially growing number of conditional independence and invariance tests.
Mooij et al.
[2020] report experiments with larger graphs, and we compare our algorithm to their FCI-JCI123 algorithm.
We focus on scalability and provide additional experiments on small graphs and MEC refinement results in supplementary material section D.2.
To enable comparisons under soft interventions, we adopt mechanism changes of Mooij et al.
[2020], in which a constant o ff set is added to the intervention targets (see page 53, Section 5.2 for details).
We note that this is di ff erent from our model of soft interventions and results in slight degrading of the performance of our algorithm, but since we are using FCI-JCI123 as our benchmark, we adopt its setting.
We consider two environments and one intervention target for simplicity of the comparisons.
We generate 30 Erd˝osRényi random DAGs.
The probability of an edge being present in the random graphs is set to 2 / p where p is the number of observed and latent variables.
We report the precision and recall rates of both algorithms along with their runtimes in Table 1.
While both methods have similar performance, there is a significant discrepancy in their runtime.
More importantly, the runtime of FCI-JCI123 becomes prohibitive very quickly, even with graphs with as few asnodes.
In contrast, PreDITEr has a significantly lower runtime even though the considered setting (i.e., mechanism changes) is not the setting for which it is designed.
All simulations are run on a computer with i7-4960HQ, 16GB 1600MHz RAM.
[Table]
6.3 BIOLOGICAL DATA
We apply the PreDITEr algorithm to a real dataset with data from observational and multiple interventional settings.
Since PreDITEr estimates the intervention targets and their corresponding parent-spouse sets for each pair of available settings, we combine the findings from each pair and yield a mixed graph estimate of the associated causal structure.
Raf
PIP
PKA
Mek
Plcg PIP
Erk
Akt
PKC
Jnk
P
Protein signaling data.
We consider the dataset of Sachs et al.
[-We continue recursively until reaching an edge W i ←◦ W m + 
-]
[2005], which is a standard benchmark in causal inference literature.
The data is obtained from measurements of the proteins involved in T-4 cell signaling.
The protein signaling network consists of 11 nodes.
[-{ X , Y , Z , W } , and nonintervened source nodes J 
-]
In each interventional setting, various drugs are injected into the cells to inhibit or activate di ff erent signaling proteins.
The target proteins are considered sites of intervention.
Data from observational and five interventional settings are provided.
The true ground truth network is not exactly known, and the accepted ground truth has been updated over the years.
Notably, it is represented by a DAG without latent confounders.
We use the recent version of Ness et al.
[2017], which consists of 16 edges, and use the preprocessed real data provided by
Squires et al.
Figure 3 shows the output of our algorithm.
For a pair of nodes, if they are found to be in the parent-spouse sets of one another, they must be spouses, and we assign a bidirected edge.
If only one of them lies in the parent-spouse set of the other node, it must be the parent, and we assign a directed edge to them. If we do not have either of the above results, the relationship can be either parent or spouse, and we denote it by →◦ on the graph. The recovered edges that are also present in the skeleton of the ground truth DAG are marked in blue. This result illustrates that even though our algorithm is designed for linear models, it has the potential to be applied to real datasets with non-linear models.
In this paper, we have considered the problem of estimating intervention targets for causally insu ffi cient systems in linear structural equation models (SEMs). We have assumed a soft intervention model that is more realistic than hard interventions, which eradicate all causal e ff ects on targets. We have shown the usage of invariance of precision matrix entries and proposed an algorithm to identify intervention targets.
The algorithm can also be used to refine the observational MEC to interventional MEC for maximal ancestral graphs. Since there exist e ffi cient algorithms for the former, our algorithm provides scalability for the latter as well. We support our analytical results through simulations and compare them with competing methods. The limitation of our approach is that it only applies to linear SEMs.
However, we have demonstrated strong performance in real and synthetic datasets, which shows its applicability to other settings. Acknowledgements This work was supported by the Rensselaer-IBM AI Research Collaboration (http: // airc.rpi.edu), part of the IBM AI Horizons Network (http: // ibm.biz / AIHorizons).