Title: Causal Effect Identification in Uncertain Causal Networks
Venue: NeurIPS 2023
Accepted: yes
Keywords: Causal effect

identifiability

causal DAGs

probabilistic graphs
Abstract: Causal identification is at the core of the causal inference literature, where complete algorithms have been proposed to identify causal queries of interest. The validity of these algorithms hinges on the restrictive assumption of having access to a correctly specified causal structure. In this work, we study the setting where a probabilistic model of the causal structure is available. Specifically, the edges in a causal graph exist with uncertainties which may, for example, represent degree of belief from domain experts. Alternatively, the uncertainty about an edge may reflect the confidence of a particular statistical test. The question that naturally arises in this setting is: Given such a probabilistic graph and a specific causal effect of interest, what is the subgraph which has the highest plausibility and for which the causal effect is identifiable? We show that answering this question reduces to solving an NP-hard combinatorial optimization problem which we call the edge ID problem. We propose efficient algorithms to approximate this problem and evaluate them against both real-world networks and randomly generated graphs.

Causal Effect Identification in Uncertain Causal Networks

Abstract

Causal identification is at the core of the causal inference literature, where complete algorithms have been proposed to identify causal queries of interest. The validity of these algorithms hinges on the restrictive assumption of having access to a correctly specified causal structure. In this work, we study the setting where a probabilistic model of the causal structure is available. Specifically, the edges in a causal graph exist with uncertainties which may, for example, represent degree of belief from domain experts. Alternatively, the uncertainty about an edge may reflect the confidence of a particular statistical test. The question that naturally arises in this setting is: Given such a probabilistic graph and a specific causal effect of interest, what is the subgraph which has the highest plausibility and for which the causal effect is identifiable? We show that answering this question reduces to solving an NP-complete combinatorial optimization problem which we call the edge ID problem. We propose efficient algorithms to approximate this problem and evaluate them against both real-world networks and randomly generated graphs.

1 Introduction

A large proportion of questions of interest in various fields, including but not limited to psychology, social sciences, behavioral sciences, medical research, epidemiology, economy, etc., are causal in nature [33, 24, 8]. In order to estimate causal effects, the gold standard is performing controlled interventions and experiments. Unfortunately, such experiments can be prohibitively expensive, unethical, or impractical [10, 12]. In contrast, non-experimental data are comparatively abundant, and no expensive interventions are required to generate such data. This has motivated the development of numerous techniques for understanding whether a causal query can be answered using observational data. Specifically, if a particular causal query is identifiable , it means it can be expressed as a function of the observational distribution and thus estimated from observational data (at least in principle).

Asignificant body of the causal inference literature is dedicated to the identification problem [23, 6, 29, 26, 24]. In particular, [13] presented a complete algorithmic approach to decide the identifiability of a specific query and proved that Pearl's do calculus is complete. Furthermore, [26] provided graphical criteria to decide the identifiability based on the hedge criterion. The problem of identifiability was later generalized to deal with interventional data [21, 20], selection bias [1], context-specific independence relations [31, 15] and causal bandits [16]. However, all of these results hinge on the full specification of the causal structure, i.e., access to a correctly specified Acyclic Directed Mixed Graph (ADMG) that models the causal dynamics of the system. This requirement is restrictive in a number of ways. (i) The causal identification problem is concerned with inference from the

observational data, but the ADMG cannot be inferred from the observational distribution alone. (ii) Structure learning methods rely heavily on statistical tests [27, 9, 5, 4, 17], which are prone to errors arising from lack of sufficient data and method-specific limitations [25]. This can in turn result in misspecification of the causal structure.

Anotable work to address the second challenge is [22], where the authors proposed a causal discovery method that assigns a probability to the existence of each edge, as opposed to making 'hard' decisions based on a single or limited number of tests. This is achieved by updating the posterior probability of a direct link between two variables, denoted as X and Y , using conditional dependence tests that incorporate independent evidence. The authors demonstrate that the posterior probability denoted as p i , of a link between X and Y after performing i dependence tests ( d j , where j ranges from 1 to i ) is given by the following equation:

where G represents a factor ranging from 0 to 1 , and it can be interpreted as the measure of 'importance" or relevance associated with the truthfulness of each test, denoted as d i . Other closely linked work include [7] and [4], where an optimal transport (OT)-based causal discovery approach is proposed that assigns a score to each edge of the graph denoting the strength of that edge.

While previous research, such as [22], has focused on probabilistic learning of causal graphs, the problem of causal effect identification on these graphs remains unexplored. This paper introduces a novel approach to address causal effect identification in a scenario where only a probabilistic model of the causal structure is available, as opposed to a complete specification of the structure itself. For instance, an ADMG G is given along with probabilities assigned to each edge of G . An example is shown in Figure 1a. These probabilities could represent uncertainties arising from statistical tests or the strength of belief of domain experts concerning the plausibility of the existence of an edge. Under this setting, each ADMG on the set of vertices of G is assigned its own plausibility score. Since the causal structure is not deterministic anymore, answering questions such as ' is the causal effect P ( Y | do ( X )) identifiable? ' also becomes probabilistic in nature. One can compare the overall plausibility of different subgraphs in which the causal effect is identifiable and then select the graph which maximizes the plausibility. In this work, for a specific causal query P ( Y | do ( X )) , we first answer the question, 'Which graph has the highest plausibility among those compliant with the probabilistic ADMG model that renders P ( Y | do ( X )) identifiable?' The answer to this question allows us to quantify how confident we are about the causal identification task given the combination of the data at hand and the corresponding probabilistic model.

As the causal identification task is carried out through an identification formula that is based on the causal structure, our second focus is on deriving an identification formula for a given causal query that holds with the highest probability. This problem poses a greater challenge compared to the former, as a single identification formula can be valid for various graphs. Consequently, the probability that a particular identification formula is valid for a causal query becomes an aggregate probability across all graphs where this formula holds. We shall illustrate this point in more detail through Example 1 in Section 2. The intricacy of characterizing all valid graphs for a given formula makes it an intractable and open problem. To surmount this obstacle, we establish that if an identification formula is valid for a causal graph, it remains valid for all edge-induced subgraphs of that graph. This allows us to form a surrogate problem (see Problem 2 in Section 2.1) that recovers a causal graph with the highest aggregated probability of its subgraphs. The probability of the resulting graph provides a lower bound on the highest probable identification formula.

Both problems discussed in this work are aimed at evaluating the plausibility of performing causal identification for a specific query given a dataset and a non-deterministic model describing the causal structure. To sum up, our main contributions are as follows.

We study the problem of causal identifiability in probabilistic causal models, where there are uncertainties about the existence of edges and whether a given causal effect is identifiable. More precisely, we consider two problems: 1) finding the most probable graph that renders a desired causal query identifiable, and 2) finding the graph with the highest aggregate probability over its edge-induced subgraphs that renders a desired causal query identifiable.

x

z

t

y

1

.

0

1

.

0

1

.

0

0.7

0.9

0.7

1.0

(a) ADMG

G

.

x

z

t

y

1

.

0

1

.

0

1

.

0

0.7

0.7

1.0

(b)

G

1

⊆ G

.

x

z

t

y

1

.

0

1

.

0

1

.

0

0.9

1.0

(c)

G

2

⊆ G

.

We show that both aforementioned problems reduce to a special combinatorial optimization problem which we call the edge ID problem . Moreover, we prove that the edge ID problem is NP-complete, and thus, so are both of the problems.

We propose several exact and heuristic algorithms for the aforementioned problems and evaluate their performances through different experiments.

2 Preliminaries

We utilize small letters for variables and capital letters for sets of variables. Calligraphic letters are used to denote graphs. An acyclic directed mixed graph (ADMG) G = ( V G , E G d , E G b ) is defined as an acyclic graph on the vertices V G , where E G d ⊆ V G × V G and E G b ⊆ ( V G 2 ) are the set of directed and bidirected edges among the vertices, respectively. With slight abuse of notation, if e ∈ E G d ∪ E G b , we write e ∈ G . We use G ′ ⊆ G when G ′ is an edge-induced subgraph of G , i.e., G ′ = ( V G ′ , E G ′ d , E G ′ b ) , where V G ′ = V G and E G ′ i ⊆ E G i for i ∈ { b, d } . We denote by G [ X ] the vertex-induced subgraph of G over the subset of vertices X ⊆ V G . For a set of vertices X , we denote by Anc G ( X ) the set of vertices in G that have a directed path to X . Note that X ⊆ Anc G ( X ) . Let P X ( Y ) be a shorthand for P ( Y | do ( X )) , and P M ( · ) denote the distribution of variables described by the causal model M .

Definition 2.1 (Identifiability [24]) . Given a causal ADMG G = ( V G , E G d , E G b ) , and two disjoint subsets of variables X,Y ⊆ V G , the causal effect of X on Y , denoted by P X ( Y ) , is identifiable in G if P M 1 X ( Y ) = P M 2 X ( Y ) for any two models M 1 and M 2 that induce G and P M 1 ( V G ) = P M 2 ( V G ) > 0 .

Definition 2.2 (Valid identification formula) . For a causal ADMG G over variables V G and a causal query P X ( Y ) , we say a functional F defined on the probability space over V G is a valid identification formula for P X ( Y ) in G if P M 1 X ( Y ) = P M 2 X ( Y ) = F ( P M 1 ( V G )) = F ( P M 2 ( V G )) for any two models M 1 and M 2 that induce G and P M 1 ( V G ) = P M 2 ( V G ) > 0 .

For any query P X ( Y ) , let [ G ] Id ( P X ( Y )) denote the set of subgraphs of G in which P X ( Y ) is identifiable (note that if G is complete graph, [ G ] Id ( P X ( Y )) is the set of all graphs in which P X ( Y ) is identifiable.) We denote by Q [ Y ] the causal effect of V \ Y on Y , i.e., Q [ Y ] = P ( Y | do ( V \ Y )) .

Definition 2.3 (District) . For ADMG G = ( V G , E G d , E G b ) , let G ↔ denote the edge-induced subgraph of G over its bidirected edges. X ⊆ V G is a district (aka c-component) in G if G ↔ [ X ] is connected.

Definition 2.4 (Hedge [26]) . Let G be an ADMG, and Y ⊊ X be two subsets of its vertices, where Y is a district in G [ Y ] . Vertices X form a hedge for Q [ Y ] if X is a district in G [ X ] and Anc G [ X ] ( Y ) = X 1 .

Definition 2.5 (Maximal hedge [2]) . For ADMG G with set vertices Y , let X be the union of all hedges formed for Q [ Y ] . Graph G [ X ] , denoted by MH ( G , Y ) , is called the maximal hedge for Q [ Y ] .

As an example, both sets { t, x } and { z, x } form a hedge for Q [ x ] in G in Figure 1a, and G [ { x, z, t } ] is the maximal hedge for Q [ x ] .

2.1 Problem setup

Let G = ( V G , E G d , E G b ) be an ADMG, where V G is the set of vertices each representing an observed variable of the system, E G d is the set of directed edges, and E G b is the set of bidirected edges among V G . We know a priori that the true ADMG describing the system is an edge-induced subgraph of G 2 , and we are given a probability map that indicates for each subgraph of G such as G s , with what probability G s is the true causal ADMG of the system. We denote this probability as P ( G s ) . For instance, if edge probabilities p e are assumed to be mutually independent, P ( G s ) takes the form:

In what follows, we will refer to P ( G s ) simply as the probability of the ADMG G s . The first problem of our interest is formally defined as follows.

Problem 1. We consider the problem of finding the most probable edge-induced subgraph of G , in which the causal effect Q [ Y ] 3 is identifiable. That is, the goal is to find the ADMG G ∗ defined by

We will prove in Lemma 3 that if Q [ Y ] is identifiable in G , then it is also identifiable in every edge-induced subgraph of G . In other words, if G is a feasible solution to the above optimization problem, so are all its edge-induced subgraphs. Furthermore, the same identification functional that is valid w.r.t. G , is also valid w.r.t. every subgraph of G . Let us illustrate this first on an example.

Example 1. Consider the ADMG in Figure 1a. With the given edge probabilities and assuming independence among the edge probabilities, the subgraph of G illustrated in Figure 1b has probability 0 . 7 × 0 . 7 × 0 . 1 = 0 . 049 , whereas the subgraph of Figure 1c has probability 0 . 3 × 0 . 3 × 0 . 9 = 0 . 081 (see Eq. (1) ). If we were to solve Problem 1, we would choose G 2 over G 1 , as it has a higher probability. Now consider identification formulas in G 1 and G 2 , respectively:

F 1 is a valid identification formula for any edge-induced subgraph of G 1 (see Lemma 3). Analogously, F 2 is valid for all edge-induced subgraphs of G 2 . If we consider the aggregate probability of the subgraphs of G 1 and G 2 , i.e.,

then we should prefer choosing G 1 over G 2 , as its identification formula F 1 is more likely to be valid than F 2 considering the fact that for all its subgraphs, the identification functional F 1 is still valid.

Plausibility of a certain identification functional F is the sum of the probabilities of all graphs in which F is valid given the query of interest. Finding the most plausible identification formula for a given query requires computing the plausibility of all formulae. Since the space of all formulae is intractable, alternatively, we can enumerate all valid formulae for a given graph which changes the problem's search space to the space of all graphs. However, this is yet another challenging and, to the best of our knowledge, an open problem. Instead, we proposed a surrogate problem that yields a lower bound for the most plausible identification formula and identifies a causal graph where this formula is valid. To do so, we use the result of Lemma 3 that shows when an identification functional is valid in a causal graph, it is also valid in all its edge-induced subgraphs.

Problem 2. Consider the problem of finding the edge-induced subgraph H ∗ of G with a maximum aggregate probability of its subgraphs, in which Q [ Y ] is identifiable. Formally,

In other words, we are looking for a causal graph H ∗ with the maximum aggregate probability of its subgraphs among the graphs in [ G ] Id ( Q [ Y ]) , i.e., the graphs in which Q [ Y ] is identifiable. Running an identification algorithm on H ∗ yields an identification formula for Q [ Y ] , which is valid at least with the aggregate probability of the subgraphs of H ∗ . Therefore, Problem 2 is a surrogate for recovering the identification formula with the highest plausibility. In the sequel, for simplicity, we study Problems 1 and 2 under the following assumption. However, as proved in Appendix C, our results are valid in a more general setting where we allow only for perfect negative or positive correlations among the edges. An example of a perfect negative correlation between two edges is that both cannot exist simultaneously. Appendix C.1 discusses the significance of this generalization.

Assumption 2.1. The edges of G are mutually independent, i.e., the probability of a subgraph G s of G is of the form (1).

Remark 2.1. It is noteworthy that our results are not limited to causal queries of the form Q [ Y ] = P ( Y | do ( V G \ Y )) . They can be applied to general causal queries of the form P X ( Y ) if the set Anc G\ X ( Y ) is known. This is because the causal query P X ( Y ) can be expressed as ∑ Anc G\ X ( Y ) \ Y Q [ Anc G\ X ( Y )] , where Anc G\ X ( Y ) is the set of ancestors of Y in G after removing the vertices of X . Furthermore, P X ( Y ) is identifiable in G if and only if Q [ Anc G\ X ( Y )] is identifiable in G [30, 26, 14]. Note that the assumption that Anc G\ X ( Y ) is known is not equivalent to precluding uncertainty on the directed edges (as in the case of fixing the edge probabilities to 0 or 1), but it rather imposes a perfect correlation type of constraint. Consider for instance the three graphs of Figure 2, where all of them share the same set Anc G\ X ( Y ) = { z, t } . In fact, knowing this set forces a constraint of the type that if the edge z → y does not exist, the path z → t → y must.

3 Reduction to edge ID problem and establishing complexity

We begin this section with the following lemma, to which we referred before. Thereafter, we discuss the hardness of the two problems considered in this work. For any causal query P X ( Y ) and ADMG G , if F is a valid identification formula for P X ( Y ) in G (Def. 2.2), then F is a valid identification formula for P X ( Y ) in any G ′ ⊆ G . All proofs are presented in Appendix A. In what follows, we first formally define the edge ID problem and then show the equivalence of Problems 1 and 2 to the edge ID problem under Assumption 2.1.

Definition 3.1 (Edge ID problem) . For ADMG G = ( V G , E G d , E G b ) , a set of non-negative edge weights W G = { w e ≥ 0 | e ∈ G} , and a causal query Q [ Y ] for Y ⊆ V G , the objective of the edge ID problem is to find the set of edges E ∗ ⊆ E G d ∪ E G b with minimum aggregated weight (cost), such that Q [ Y ] is identifiable in the graph resulting from removing E ∗ from G . Formally,

That is, the cost of removing a set of edges from G is the sum of the weights of each individual edge.

The following result unifies the two problems considered in this work by establishing their equivalence to the edge ID problem. This is done by transforming Problems 1 and 2 with multiplicative objectives into the edge ID problem that has an additive objective.

Lemma 3.2. Under Assumption 2.1, Problem 1 is equivalent to the edge ID problem with the edge weights chosen to be the log propensity ratios, i.e., w e = max { 0 , log( p e 1 -p e ) } , ∀ e ∈ G . Moreover, Problem 2 is equivalent to the edge ID problem with the choice of weights w e = -log(1 -p e ) , ∀ e ∈ G . That is, an instance of Problems 1 and 2 can be reduced to an instance of the edge ID problem in polynomial time, and vice versa.

As we mentioned earlier, the equivalence of these three problems can be established in more general settings than what is described under Assumption 2.1. We refer the interested reader to Appendix C for a discussion on one such setting. The following result shows no polynomial-time algorithm for solving these three problems exists unless P = NP.

Theorem 3.3. The edge ID problem is NP-complete.

Theorem 3.3 is established through a reduction from another NP-complete problem, namely the minimum-cost intervention problem (MCIP) [2, 3], which we shall discuss in Section 4.3. Theorem

z

t

y

z

t

y

z

t

y

3.3 is a key result which shows the hardness of recovering the most plausible graph in which a specified causal effect of interest is identifiable.

Corollary 3.4. Problems 1 and 2 are NP-complete under Assumption 2.1.

It is noteworthy that the size of the problem depends on the number of vertices of G , i.e., | V G | , and the number of edges of G with finite weight, i.e., | E G | = | E G d | + | E G b | . Since the ID algorithm (function ID of [26]) runs in time O ( | V G | 2 ) , the brute-force algorithm that tests the identifiability of Q [ Y ] in every edge-induced subgraph of G and chooses the one with the minimum weight of deleted edges runs in time O (2 | E G | | V G | 2 ) . In the next Section, we present various algorithmic approaches for solving or approximating the solutions to these problems.

4 Algorithmic approaches

We first present a recursive approach for solving the edge ID problem in Section 4.1, described in Algorithm 1. Since the problem itself is NP-complete, Algorithm 1 runs in exponential time in the worst case. In Section 4.2, we present heuristic approximations of the edge ID problem, which runs in cubic time in the worst case. These heuristics can also be used as a pre-process to reduce the runtime of Alg. 1 by providing an upper bound that can be fed into Alg. 1 to prune the search space. Finally, in Section 4.3, we present a reduction of edge ID to yet another NP-complete problem, namely minimum-cost intervention problem [2], which allows us to use the algorithms designed for that problem to solve edge ID. Our simulations in Section 5 evaluate these approaches against each other.

4.1 Recursive exact algorithm

This approach is described in Algorithm 1. The inputs to the algorithm are an ADMG G along with edge weights W G , a set of vertices Y corresponding to the causal query Q [ Y ] , an upper bound ω ub on the aggregate weight (cost) of the optimal solution, and a threshold ω th , an upper bound on the acceptable cost of a solution. The closer ω ub is to the optimal cost, the quicker Algorithm 1 will find the solution. If no upper bound is known, the algorithm can be initiated with ω ub = ∞ . However, we shall discuss a few approaches to find a good upper bound ω ub in the following Section. Note that when ω th = 0 , Algorithm 1 will output the optimal solution. Otherwise, as soon as a feasible solution with weight less than ω th is found, the algorithm terminates (line 13).

The algorithm begins with calling subroutine MH in line 2, which constructs the maximal hedge for Q [ Y ] , denoted by H . We discuss this subroutine in detail in Appendix B. Throughout the rest of the algorithm, we only consider the edges in H , as the other edges do not alter the identifiability. If there is no hedge formed for Q [ Y ] , i.e., H = G [ Y ] , there is no need to remove any edges from G and the effect is already identified. Otherwise, we remove the edge with the lowest weight ( e ) from H and recursively call the algorithm to find the solution after removing the edge e , unless the weight of e is already higher than the upper bound ω ub , which means no feasible solutions exist for the provided upper bound. Whenever a feasible solution is found, the upper bound ω ub is updated to the lowest weight among all the solutions weights discovered so far. This, in turn, helps the algorithm prune the exponential search space during the next iterations to reduce the runtime. As soon as a solution with a weight less than the acceptable threshold, i.e., ω th , is found, the algorithm returns the solution. Otherwise, w e is updated to infinity so that it never gets removed (line 14). This is due to the fact that we have already explored all the solutions involving e .

Algorithm 1 Recursive Algorithm for edge ID.

4.2 Heuristic algorithms

In this section, we present two heuristic algorithms for approximating the solution to the edge ID problem. These algorithms can also be used to efficiently find the upper bound ω ub , which could be fed as an input to Algorithm 1.

Algorithm 2 Heuristic algorithm for edge ID. 1: function HEID ( G , Y, W G ) 2: G ′ ← MH ( G , Y ) , 3: Z ←{ z ∈ V G ′ |∃ y ∈ Y : { z, y } ∈ E G ′ b } \ Y , 4: H ← induced subgraph of G ′ on its directed edges. 5: W H ←{ w e ∈ W G | e ∈ H} , V H ← V H ∪ { y ∗ , z ∗ } 6: for z ∈ Z do E H ← E H ∪ ( z ∗ , z ) , W H ← W H ∪ { w ( z ∗ ,z ) = ∑ y w { z,y } } 7: for y ∈ Y do E H ← E H ∪ ( y, y ∗ ) , W H ← W H ∪ { w ( y,y ∗ ) = ∞} 8: E ← MinCut ( H , W H , z ∗ , y ∗ ) 9: return ( E, ∑ e ∈ E w e )

Let Z = { z ∈ V G | ∃ y ∈ Y : { z, y } ∈ E G b } \ Y denote the set of vertices that have at least one common bidirected edge with a vertex in Y . Any hedge formed for Q [ Y ] contains at least one vertex of Z . As a result, to eliminate all the hedges formed for Q [ Y ] , it suffices to ensure that none of the vertices in Z appear in such a hedge. To this end, for any z ∈ Z , it suffices to either remove all the bidirected edges between z and Y or eliminate all the directed paths from z to Y . The problem of eliminating all directed paths from Z to Y can be cast as a minimum cut problem between Z and Y in the edge-induced subgraph of G over its directed edges. To add the possibility of removing the bidirected edges between Z and Y , we add an auxiliary vertex z ∗ to the graph and draw a directed edge from z ∗ to every z ∈ Z with weight w = ∑ y ∈ Y w { z,y } , i.e., the sum of the weights of all bidirected edges between z and Y . Note that z can have bidirected edges to multiple vertices in Y . We then solve the minimum cut problem for z ∗ and Y . If an edge between z ∗ and z ∈ Z is included in the solution to this minimum cut problem, it is mapped to remove all the bidirected edges between z and Y in the main problem. Note that we can run the algorithm on the maximal hedge formed for Q [ Y ] in G rather than G itself. This heuristic is presented as Algorithm 2.

An analogous approach which goes through solving an undirected minimum cut on the edge-induced subgraph of G over its bidirected edges is presented in Algorithm 4 in Appendix D. As mentioned earlier, these algorithms can be used either as standalone algorithms to approximate the solution to the edge ID problem, or as a pre-processing step to find an upper bound ω ub for Algorithm 1.

Complexity and performance. The aforementioned heuristic algorithms can be broken down into two primary components: constructing an instance of the minimum-cut problem, and solving the latter. Both algorithms follow similar steps for constructing the minimum-cut problem. For reference,

x

1

x

2

y

1

y

2

w

d

x

2

x

1

w

d

x

1

y

1

w

b

x

1

x

2

w

b

x

2

y

2

w

b

y

1

y

2

(a) ADMG

G

,

Y

=

{

y

1

, y

2

}

.

x

1

x

2

y

1

y

2

x

d

21

w

d

x

2

x

1

y

12

1

∞

y

12

2

∞

x

b

12

w

b

x

1

x

2

z

b

22

w

b

x

2

y

2

y

b

12

w

b

y

1

y

2

y

12

12

∞

z

d

11

w

d

x

1

y

1

(b) ADMG

H

,

Y

mcip

=

{

y

1

, y

2

, y

12

2

}

.

Alg. 2 starts with identifying the maximal hedge formed for Q [ Y ] , which is cubic-time in the worst case [2], followed by forming a directed graph H over the vertices of the maximal hedge. Forming this graph, which inherits its vertices and directed edges from G , is linear in time. Lastly, there exist several minimum-cut-maximum-flow algorithms that exhibit cubic-time performance. Therefore, our heuristic algorithms have a cubic runtime in the worst case. On the other hand, given the hardness result we provided in the previous section, and in light of the fact that there is a polynomial-time reduction from the minimum hitting set problem to MCIP [3], the edge ID problem is NP-hard to approximate within a factor better than logarithmic [11]. Since our heuristic algorithms run in polynomial time, they are expected to be suboptimal by at least a logarithmic factor in the worst case. However, as we shall see in our simulations, both algorithms achieve near-optimal results on random graphs.

4.3 Alternative approach: reduction to MCIP

As an alternative approach to the algorithms discussed so far, we present a reduction of the edge ID problem to another NP-complete problem, i.e., the minimum-cost intervention problem ( MCIP ) introduced in [2]. This reduction allows us to exploit algorithms designed for MCIP to solve our problems. The formal definition of MCIP is as follows.

Definition 4.1 (MCIP) . Suppose G = ( V G , E G d , E G b ) is an ADMG, C : V G → ❘ ≥ 0 is a cost function mapping each vertex of G to a non-negative cost, and Y ⊆ V G . The objective of the minimum-cost intervention problem for identifying the causal effect Q [ Y ] is to find the subset A ⊆ V G with the minimum aggregate cost such that Q [ Y ] is identifiable after intervening on the set A .

The reduction from edge ID to MCIP is based on a transformation from ADMG G to another ADMG H , where each edge in G is represented by a vertex in H . This transformation is based on the causal query Q [ Y ] , and it maps the identifiability of Q [ Y ] in G to the identifiability of Q [ Y mcip ] in H , where Y mcip is a set of vertices in H . This transformation satisfies the following property; removing a set of edges E ∗ in G makes Q [ Y ] identifiable if and only if intervening on the corresponding vertices of E ∗ in H makes Q [ Y mcip ] identifiable. More precisely, after this transformation, solving the edge ID problem for Q [ Y ] in G is equivalent to solving MCIP for Q [ Y mcip ] in H . The complete details of this transformation can be found in Appendix A.2. An example of this reduction is shown in Figure 3, where Q [ { y 1 , y 2 } ] in G (Figure 3a) is mapped to Q [ { y 1 , y 2 , y 12 2 } ] in H (Figure 3b), where { y 1 , y 2 , y 12 2 } is a district, and the set of all vertices of H forms a hedge for it. The vertices of H corresponding to each edge in G are indicated with the same color and have the same weight (cost). We assign infinity cost to them to avoid intervening on the remaining vertices in H . It is straightforward to see that the solution to the edge ID problem in G with the query Q [ Y = { y 1 , y 2 } ] would be to remove the edge with the lowest weight. This is because after removing any edge in G , no hedge remains for Q [ Y ] . Similarly, in H , the solution to MCIP with the query Q [ Y mcip = { y 1 , y 2 , y 12 2 } ] is to intervene on the vertex with the lowest cost among Z = { z d 11 , x d 21 , x b 12 , y b 12 , z b 22 } . This is because after intervening on any vertex in Z , no hedge remains for Q [ Y mcip ] . The following result establishes the link between the edge ID problem in G and MCIP in H .

(a) Runtimes.

(b) Solution costs.

(c) Fraction runtime exceeded 3 min.

0.04

Fraction reached time-out (> 3mins)

MCIP-exact

HEID-2

HEID-2

0.6

EDGEID

Time (seconds)

0.03

0.02

MCIP-exact

1SO

MCIP-exact

0.4

EDGEID

2

EDGEID

0.01

0.2

0.00

0

50

100

150

200

250

0

50

100

150

200

250

0.0

0

50

100

150

200

250

Number of vertices before Y in topological order

Number of vertices before Y in topological order

Number of vertices before Y in topological order

Proposition 4.2. There exists a polynomial-time reduction from edge ID to MCIP and vice versa.

In particular, the time complexity of the transformation from edge ID to MCIP scales linearly in size of G , and is cubic in | Y | . More precisely, the time complexity is O ( | V G | + | E G d | + | E G b | + | Y | 3 ) . For further details, refer to the construction in Appendix A.2.

5 Experiments

We evaluate the proposed heuristic algorithms 2 (HEID-1) and 4 (HEID-2), as well as the exact algorithm 1 (EDGEID), where the upper-bound ω ub for EDGEID is set to be the minimum cost found by HEID-1 or -2. Furthermore, given the reduction of the edge ID problem to the MCIP problem described in Section 4.3, we also evaluate the two approximation and one exact algorithms from [2] (MCIP-H1, MCIP-H2, and MCIP-exact, respectively). Experimental results are provided for Problem 1, and analogous results for Problem 2 are provided in Appendix E.3. All experiments were carried out on an Intel i9-9900K CPU running at 3.6GHz 4 .

Simulations: The algorithms are evaluated for graphs with between 5 and 250 vertices. For a given number of vertices, we uniformly sample 50 ADMG structures from a library of graphs which are non-isomorphic to each other. Edges for each of these 100 graphs are sampled with probability of log( n ) /n , where n is the number of (observable) vertices, to impose sparsity (thus pragmatically reducing the search space). For each graph, we sample directed and bidirected edge probabilities p e uniformly between 0.51 and 1.0 5 . The problem is then converted into edge ID according to Lemma 3.2. The vertices in the graphs are topologically sorted and the outcome Y is selected to be the last vertex in the topological ordering. We then check whether a solution exists in principle by removing all finite cost edges and checking for identifiability. If not, a new graph is sampled to avoid evaluating the algorithms on graphs with no solution. For each of these 50 probabilistic ADMGs, we run the algorithms and record the resulting runtime and the associated cost of the solution. If the runtime exceeds 3 minutes, we abort and log that the algorithm has failed to find a solution.

Results are presented in Figure 4. Runtimes and costs are shown for the subset of graphs for which all algorithms found a solution (to facilitate comparison). Runtimes for each algorithm are shown in Fig. 4a, where it can be seen that our proposed HEID-1 and HEID-2 heuristic algorithms have negligible runtime, followed by the MCIP variants. Interestingly, the exact algorithm EDGEID outperformed the MCIP algorithms on larger graphs, for which the transformation time from the edge ID problem to the MCIP increases with the size of the graph. In contrast, EDGEID had large runtime variance which depended heavily on the specifics of the graph under evaluation, particularly for graphs with fewer vertices. The costs for each graph are shown in Fig. 4b, and here we see, as expected, the lowest cost is achieved by the two exact algorithms, EDGEID and MCIP-exact, followed closely by the heuristic algorithms. Figure 4c shows the fraction of evaluations for which the runtime exceeded 3 minutes (applicable to the exact algorithms). In general, and owing to the sparsity

penalty in our graph generation mechanism, the cost of identified solutions falls with the number of vertices. However, among the exact algorithms, EDGEID, exceeds the 3-minute runtime more often than the MCIP-Exact, regardless of the number of vertices and despite the fact that EDGEID is quicker at finding a solution when it does so. Overall, HEID-1 was the most consistent in finding a solution, having a short runtime, and achieving a close to optimal cost.

Real-world graphs: We also apply the algorithms to four real-world datasets. The first 'Psych' (22 nodes & 70 directed edges) concerns the putative structure from a causal discovery algorithm Structural Agnostic Model [18] using data collected as part of the Health and Relationships Project [32]. The other three 'Barley' (48 nodes & 84 directed edges), 'Water' (32 nodes & 66 directed edges), and 'Alarm' (37 nodes & 46 directed edges) come from the bnlearn python package [28]. For all four graphs, and as with the simulations described above, we introduce bidirected edges with a sparsity constraint of log( n ) /n , and simulate expert domain knowledge by randomly assigning directed and bidirected edge probabilities between 0.51 and 1. The outcome Y is selected to be the last vertex in the topological ordering. For these results, we provide the runtime (limited to 500 seconds) and cost, as well as the ratio of graph plausibility before and after selecting a subgraph in which the effect is identifiable P ( ˆ G ∗ ) /P ( G ) . This ratio is 1.0 if the effect is identifiable in the original graph and decreases according to the plausibility of an identified subgraph.

Results are shown in Table 1. In cases where MCIP-exact and/or EDGEID did not exceed the runtime limit, it can be seen that HEID-2 and MCIP-H2 achieved equivalent to optimal cost and ratio. Runtimes for MCIP variants exceeded the HEID variants owing to the required transformation. EDGEID timed out on all but the Alarm structure, whereas MCIP-exact only timed out on the Psych structure, indicating that the MCIP-exact is more consistent (this also corroborates Figure 4c).

6 Concluding remarks

Researchers in causal inference are often faced with graphs for which the effect of interest is not identifiable. It is common to identify a target effect by assuming ignorability. A less drastic and more reasonable approach would be to relax this assumption by identifying the most plausible subgraph, given uncertainty about the structure as we suggested in this work. We presented a number of algorithms for finding the most probable/plausible probabilistic ADMG in which the target causal effect is identifiable. We provided an analysis of the complexity of the problem, and an experimental comparison of runtimes, solution costs, and failure rates. We noted that our heuristic algorithms, Alg. 2 and Alg. 4 performed remarkably well across all metrics. In terms of limitations, we made the assumption that the edges in G are mutually independent (Assumption 2.1). Future work should explore the case where this assumption does not hold. In this work, we studied the problem of point identification of the target effect. We envision that a similar methodology can be applied to partial identification. Finally, it is worth noting that the external validity of the derived subgraph (i.e., whether or not the subgraph is correctly specified with respect to the corresponding real-world process) is not guaranteed. As such, practitioners using such approaches are encouraged to do so with caution, particularly for research involving human participants.

References

Amir Mohammad Abouei, Ehsan Mokhtarian, and Negar Kiyavash. s-id: Causal effect identification in a sub-population. AAAI , 2024.

Sina Akbari, Jalal Etesami, and Negar Kiyavash. Minimum cost intervention design for causal effect identification. In Proceedings of the 39th International Conference on Machine Learning , volume 162 of Proceedings of Machine Learning Research , pages 258-289. PMLR, 2022.

Sina Akbari, Jalal Etesami, and Negar Kiyavash. Experimental design for causal effect identification. arXiv:2205.02232 , 2023.

Sina Akbari, Luca Ganassali, and Negar Kiyavash. Causal discovery via monotone triangular transport maps. In NeurIPS 2023 Workshop Optimal Transport and Machine Learning , 2023.

Sina Akbari, Ehsan Mokhtarian, AmirEmad Ghassami, and Negar Kiyavash. Recursive causal structure learning in the presence of latent variables and selection bias. Advances in Neural Information Processing Systems , 34:10119-10130, 2021.

Joshua Angrist and Guido Imbens. Identification and estimation of local average treatment effects. Econometrica , 62(2):467-475, 1994.

Ricardo Baptista, Youssef Marzouk, Rebecca E. Morrison, and Olivier Zahm. Learning nongaussian graphical models via hessian scores and triangular transport, 2023.

Elias Bareinboim, Juan D. Correa, Duligur Ibeling, and Thomas Icard. On Pearl's hierarchy and the foundations of causal inference. ACM Special Reports , 2020.

Diego Colombo, Marloes H. Maathuis, Markus Kalisch, and Thomas S. Richardson. Learning high-dimensional directed acyclic graphs with latent and selection variables. The Annals of Statistics , pages 294-321, 2012.

Angus Deaton and Nancy Cartwright. Understanding and misundertstanding randomized controlled trials. Social Science and Medicine , 210:2-21, 2018.

Uriel Feige. A threshold of ln n for approximating set cover. Journal of the ACM (JACM) , 45(4):634-652, 1998.

Clark Glymour, Kun Zhang, and Peter Spirtes. Review of causal discovery methods based on graphical models. Frontiers in Genetics , 10, 2019.

Yimin Huang and Marco Valtorta. Pearl's calculus of intervention is complete. Proceedings of the Twenty-Second Conference on Uncertainty in Artificial Intelligence (UAI) , 2006.

Amin Jaber, Jiji Zhang, and Elias Bareinboim. Causal identification under Markov equivalence: Completeness results. In International Conference on Machine Learning , pages 2981-2989. PMLR, 2019.

Fateme Jamshidi, Sina Akbari, and Negar Kiyavash. Causal imitability under context-specific independence relations. arXiv preprint arXiv:2306.00585 , 2023.

Fateme Jamshidi, Jalal Etesami, and Negar Kiyavash. Confounded budgeted causal bandits. 3rd Conference on Causal Learning and Reasoning , 2024.

Fateme Jamshidi, Luca Ganassali, and Negar Kiyavash. On sample complexity of conditional independence testing with von mises estimator with application to causal discovery. arXiv preprint arXiv:2310.13553 , 2023.

Diviyan Kalainathan, Olivier Goudet, Isabelle Guyon, David Lopez-Paz, and Michèle Sebag. Structural agnostic modeling: Adversarial learning of causal graphs. arXiv:1803.04929v3 , 2020.

Richard M. Karp. Reducibility among combinatorial problems. In Complexity of computer computations , pages 85-103. Springer, 1972.

Yaroslav Kivva, Ehsan Mokhtarian, Jalal Etesami, and Negar Kiyavash. Revisiting the general identifiability problem. In Uncertainty in Artificial Intelligence , pages 1022-1030. PMLR, 2022.

Sanghack Lee, Juan D Correa, and Elias Bareinboim. General identifiability with arbitrary surrogate experiments. In Uncertainty in artificial intelligence , pages 389-398. PMLR, 2020.

Dimitris Margaritis and Sebastian Thrun. Bayesian network induction via local neighborhoods. Advances in neural information processing systems , 12, 1999.

Judea Pearl. Aspects of graphical models connceted with causality. Proceedings of the 49th Session of the International Statistical Institute , pages 399-401, 1993.

Judea Pearl. Causality . Cambridge University Press, Cambridge, 2009.

Rajen D. Shah and Jonas Peters. The hardness of conditional independence testing and the generalised covariance measure. The Annals of Statistics , 48(3), 2020.

Ilya Shpitser and Judea Pearl. Identification of joint interventional distributions in recursive semiMarkovian causal models. In Proceedings of the National Conference on Artificial Intelligence , volume 21, page 1219. Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999, 2006.

Peter Spirtes, Clark N. Glymour, and D. Scheines, Richardand Heckerman. Causation, prediction, and search . MIT press, 2000.

Erdogan Taskesen. bnlearn - Library for Bayesian network learning and inference. Python Library , 2020.

Jin Tian and Judea Pearl. A general identification condition for causal effects. AAAI , 2002.

Jin Tian and Judea Pearl. On the testable implications of causal models with hidden variables. In Proceedings of the Eighteenth conference on Uncertainty in artificial intelligence , pages 519-527, 2002.

Santtu Tikka, Antti Hyttinen, and Juha Karvanen. Identifying causal effects via context-specific independence relations. Advances in neural information processing systems , 32, 2019.

Debra Umberson. Health and relationships project. Inter-university consortium for political and social research , 2014-2015.

Mark J. van der Laan and Sherri Rose. Targeted Learning - Causal Inference for Observational and Experimental Data . Springer International, New York, 2011.

Appendix

The appendices are organized as follows. Formal proofs of the results stated in the main text are presented in Section A. In Section B, we describe the algorithm to recover the maximal hedge formed for a certain query (Def. 2.5), which is used as a subroutine of Algorithm 1. A generalization of Assumption 2.1 is discussed in Section C. Section D provides further details of the heuristic algorithms discussed in the main text. Further evaluations and experimental conditions for our proposed algorithms are presented in Section E.

A Formal proofs

We begin with presenting the proofs of Lemma 3 and Lemma 3.2. Proofs of Theorem 3.3 and Proposition 4.2 appear at the end of Sections A.1 and A.2, respectively.

For any causal query P X ( Y ) and ADMG G , if F is a valid identification formula for P X ( Y ) in G (Def. 2.2), then F is a valid identification formula for P X ( Y ) in any G ′ ⊆ G .

Proof. Let H ⊆ G be an arbitrary edge-induced subgraph of G . Let F be an identification formula for P X ( Y ) in G , i.e., for any model M that induces G ,

By definition, P X ( Y ) is identifiable in G . As a result, there exists and identification formula such as F ′ that can be derived for P X ( Y ) in G , using a sequence of do calculus rules and basic probability manipulations. Note that this means for any model M that induces G ,

Note that an immediate corollary of Equations 5 and 6 is that for any model M that induces G ,

Now, we first show that this sequence of actions (combination of do calculus rules and probability manipulations) is valid in H . Note that the basic probability manipulations are graph-independent. It only suffices to show that any applied do calculus rule w.r.t. G can also be applied w.r.t. H . The validity conditions of all three do calculus rules are based on certain d-separations. As a result, it suffices to show that if a d-separation relation is valid in G , it is also valid in H . To do so, it suffices to show that if all paths between Z 1 and Z 2 are blocked in G given W , they are blocked in H too, for arbitrary disjoint sets of vertices Z 1 , Z 2 , W ⊆ V G . Take an arbitrary path, p , between Z 1 and Z 2 in H . Since H ⊆ G , this path exists in G . Since Z 1 and Z 2 are d-separated given W in G , the path p is blocked by W . As a result, any path between Z 1 and Z 2 in H is blocked by W . Therefore, any do-calculus rule applied in G , can also be applied in H . Hence, F ′ is a valid identification formula for P X ( Y ) . That is, for any model M that induces H ,

Now note that any model M that induces H , i.e., is compatible with H , is also compatible with G . Also, V G = V H . As a result, from Equations 7 and 8, we know that for any model M that induces H ,

By definition, F is a valid identification formula for P X ( Y ) in H .

Lemma 3.2. Under Assumption 2.1, Problem 1 is equivalent to the edge ID problem with the edge weights chosen to be the log propensity ratios, i.e., w e = max { 0 , log( p e 1 -p e ) } , ∀ e ∈ G . Moreover, Problem 2 is equivalent to the edge ID problem with the choice of weights w e = -log(1 -p e ) , ∀ e ∈ G . That is, an instance of Problems 1 and 2 can be reduced to an instance of the edge ID problem in polynomial time, and vice versa.

Proof. Problem 1. First consider an arbitrary graph G 1 ∈ [ G ] Id ( Q [ Y ]) such that G 1 has an edge e with p e < 1 / 2 . Let G 2 denote the graph G 1 after removing e . Lemma 3 implies that G 2 ∈ [ G ] Id ( Q [ Y ]) . According to Equation 1, we have P ( G 2 ) = 1 -p e p e P ( G 1 ) > P ( G 1 ) (since p e < 1 / 2 ). As a result, the solution G ∗ to Problem 1 (Eq. 2) has no edges with probability less than 1 / 2 . We can therefore rewrite Problem 1 as:

Or equivalently, we can always assume that we start with a graph G that has no edges with probability less than 1 / 2 , as otherwise we can remove all of those edges and the problem does not change. This indeed is equivalent to choosing weight (cost) 0 for those edges in the equivalent edge ID problem. Now assuming that the edges have probability at least 1 / 2 ,

Since ∑ e/ ∈G s log(1 -p e )) + ∑ e ∈G s log(1 -p e )) is a constant value that does not depend on G s , it can be ignored in the maximization and we have:

From the formulation above, it is clear that if we assign the weight w e = log( p e 1 -p e ) to each edge e ∈ E G , we will have an instance of the edge ID problem. Note that for edges with probability higher than 1 / 2 , log( p e 1 -p e ) ≥ 0 , and this assignment of edge weights satisfies the positivity requirement. For the opposite direction, note that the procedure explained above is reversible by the choice of probabilities p e = exp ( w e ) 1+exp( w e ) , which is a value between 1 / 2 and 1 .

Problem 2. First note that under Assumption 2.1, for any graph G s ,

This is because the inner summation goes over all the possible subsets of E G s , and the summation adds up to 1. Therefore, we can rewrite Problem 2 (Eq. 3)as

With the same reasoning as before, assigning the weights w e = -log(1 -p e ) to each edge e ∈ E G , we end up with an instance of the edge ID problem. Note that again 0 ≤ -log(1 -p e ) ≤ ∞ . It is noteworthy that this procedure is also reversible with the choice of edge probabilities p e = 1 -exp( -w e ) , which reduces the edge ID problem to an instance of Problem 2. Again note that 0 ≤ 1 -exp( -w e ) ≤ 1 for any non-negative w e .

A.1 Reduction from MCIP to edge ID

Theorem 3.3. The edge ID problem is NP-complete.

First note that edge ID is in the class of NP problems, since the ID algorithm [26] can verify a solution in polynomial time. To prove Theorem 3.3, we first present a polynomial-time reduction from MCIP to the edge ID problem. It has been shown that the minimum vertex cover problem 6 can be reduced to MCIP in polynomial time [2]. Combining the two reductions, we show that there exists a polynomial-time reduction from the minimum vertex cover problem to the edge ID problem. Since the minimum vertex cover problem is known to be NP-complete [19], it follows that the edge ID problem is also NP-complete.

We propose the following reduction from MCIP to the edge ID problem. Assume we want to solve MCIP given ADMG G = ( V G , E G d , E G b ) , query Q [ Y ] , and the intervention costs C ( v ) for v ∈ V G . We construct a graph, denoted by H = T 1 ( G , Y ) , through the following steps.

For every vertex x ∈ V G \ Y , add two vertices x 1 , x 2 to V H .

For any bidirected edge { x, z } ∈ E G b where x ∈ V G \ Y and z ∈ V G , add the bidirected edge { x 2 , z 2 } to E H b .

For any directed edge ( x, z ) ∈ E G d where x ∈ V G \ Y and z ∈ V G , add the directed edge ( x 1 , z 1 ) to E H d .

For any bidirected edge { y 1 , y 2 } ∈ E G b where y 1 , y 2 ∈ Y , add the bidirected edge { y 1 , y 2 } to E H b .

For every x 1 , x 2 ∈ V G \ Y , draw the two edges { x 1 , x 2 } ∈ E H b and ( x 2 , x 1 ) ∈ E H d . Furthermore, the weight of { x 1 , x 2 } is C ( x ) .

The costs of the all other edges in H are assigned to be infinite.

With abuse of notation, for any vertex x ∈ V G \ Y , we define T 1 ( x ) = { x 2 , x 1 } ∈ E H b , where { x 2 , x 1 } is the bidirected edge in H that corresponds to x in G , and inherits the same weight (cost).

x

c

x

z

c

z

y

1

y

2

(a) The model

G

with costs

on each vertex

x

1

z

1

y

1

y

2

x

2

z

2

∞

∞

∞

c

x

c

z

∞

∞

∞

∞

(b) The model

G

with costs

on each edge

Example 2. Consider graph G in Figure 5a. Vertices x and z are mapped to x 1 , x 2 , and z 1 , z 2 , respectively. Both a directed and a bidirected edge are drawn between these pairs. The bidirected edge { x 1 , x 2 } is assigned the weight C ( x ) = c x , and the bidirected edge { z 1 , z 2 } is assigned the weight C ( z ) = c z . Infinite weights are assigned to the rest of the edges in H (Figure 5b).

Proposition A.1. Suppose G ′ is an ADMG, Y ⊆ V G ′ is a set of its vertices such that Y is a district in G ′ [ Y ] , and H ′ = T 1 ( G ′ , Y ) . Consider X ⊆ V G ′ \ Y as an arbitrary subset of vertices of G ′ , and define G = G ′ [ V G ′ \ X ] . Let E ′′ b = { e ∈ E H ′ b |∃ v ∈ X,e = T 1 ( v ) } and define E H b = E H ′ b \ E ′′ b . Let H be the edge-induced subgraph of H ′ defined as H = ( V H ′ , E H d , E H b ) . Q [ Y ] is identifiable in G if and only if Q [ Y ] is identifiable in H .

Proof. We prove the contrapositive, i.e., Q [ Y ] is not identifiable in G iff Q [ Y ] is not identifiable in H . Note that by construction, Y is a district in both G [ Y ] and H [ Y ] . That is, it suffices to show that there exists a hedge formed for Q [ Y ] in G iff there exists a hedge formed for Q [ Y ] in H .

To this end, we first prove the following claim. Let W ∈ V H form a hedge for Q [ Y ] . If x 1 ∈ W for some x ∈ V G ′ , then x 2 ∈ W and vice versa. That is, the two vertices x 1 and x 2 corresponding to the same vertex x in V G ′ appear only simultaneously in any hedge. To see this, note that by construction, x 1 is the only child of x 2 . By definition of hedge, if x 2 ∈ W , then it has a directed path to Y within H [ W ] , and this path can only go through x 1 . For the other direction, note that x 1 has only one bidirected edge, which is with x 2 . Again, by definition of hedge, if x 1 ∈ W , then it has a bidirected path to Y within H [ W ] , and this path can only go through x 2 . Hence, in the sequel, when there is a hedge W formed for Q [ Y ] in H , we will without loss of generality assume that there exists a set of variables Z ⊆ V G ′ such that W = Z 1 ∪ Z 2 ∪ Y , where Z 1 = { z 1 | z ∈ Z } and Z 2 = { z 2 | z ∈ Z } .

If part. Let W = Z 1 ∪ Z 2 ∪ Y form a hedge for Q [ Y ] in H . First note that since none of the bidirected edges between Z 1 and Z 2 are removed in H , by construction, all vertices Z are present in G , i.e., Z ⊆ V G . Now we show that Z ∪ Y forms a hedge for Q [ Y ] in G . To this end, we prove G [ Z ∪ Y ] is a district and Z ∪ Y = Anc G [ Z ∪ Y ] ( Y ) . First note that any vertex in Z 1 has only one bidirected edge to a vertex in Z 2 . That is, if we consider the edge-induced subgraph of H [ W ] over its bidirected edges, vertices of Z 1 are leaf nodes. As a result, Z 2 ∪ Y must be connected in this graph. That is, Z 2 ∪ Y is a district in H [ Z 2 ∪ Y ] . This implies by construction of H that G [ Z ∪ Y ] is a single district. With a similar reasoning, note that vertices in Z 2 have no parents. As result, Z 1 ∪ Y = Anc H [ Z 1 ∪ Y ] ( Y ) (since the directed paths cannot go through Z 2 ). Again, by construction, the edge-induced subgraph of G [ Z ∪ Y ] over its directed edges is a copy of H [ Z 1 ∪ Y ] . As a result, Z ∪ Y = Anc G [ Z ∪ Y ] ( Y ) .

Only if part. Let Z ∪ Y form a hedge for Q [ Y ] in G , where Z ⊆ V G \ Y . Define Z 1 = { z 1 | z ∈ Z } and Z 2 = { z 2 | z ∈ Z } . We show that Z 1 ∪ Z 2 ∪ Y forms a hedge for Q [ Y ] in H . First, by definition of hedge, Anc G [ Z ∪ Y ] ( Y ) = Z ∪ Y . Since the edge-induced subgraph of H [ Z 1 ∪ Y ] is a copy of G [ Z ∪ Y ] by construction, we know Anc G [ Z 1 ∪ Y ] ( Y ) = Z 1 ∪ Y . Further, each vertex z 2 ∈ Z 2 is a parent of z 1 ∈ Z 1 . As a result, Anc G [ Z 1 ∪ Z 2 ∪ Y ] ( Y ) = Z 1 ∪ Z 2 ∪ Y . Now it suffices to show that Z 1 ∪ Z 2 ∪ Y is a district in H [ Z 1 ∪ Z 2 ∪ Y ] . By definition of hedge, Z ∪ Y is a district in G [ Z ∪ Y ] . By construction of H , exactly the same bidirected edges (and therefore bidirected paths) exist in H [ Z 2 ∪ Y ] . Therefore, Z 2 ∪ Y is a district in H [ Z 2 ∪ Y ] . Now note that by construction of H ′ , each vertex z 1 ∈ Z 1 has a bidirected edge to z 2 ∈ Z 2 . And by definition of G and H , since the

vertices Z exist in G , none of these edges are removed in H . As a result, Z 1 ∪ Z 2 ∪ Y is a district in H [ Z 1 ∪ Z 2 ∪ Y ] , which completes the proof.

Proof of Theorem 3.3. A polynomial-time reduction from MCIP to the edge ID problem follows immediately from Proposition A.1. MCIP is shown to be NP-complete [2]. As a result, the edge ID problem is Np-complete.

A.2 Reduction from edge ID to MCIP

Proposition 4.2. There exists a polynomial-time reduction from edge ID to MCIP and vice versa.

To prove Proposition 4.2, we begin with presenting a transformation T 2 ( G , Y ) which is in the core of reduction from edge ID to MCIP.

Suppose we want to solve the edge ID problem given ADMG G = ( V G , E G d , E G b ) , query Q [ Y ] , and edge weights W G = { w e | e ∈ G} . Let X = V G \ Y denote the set of vertices of G excluding Y . We define the transformation ( H , Y mcip ) = T 2 ( G , Y ) where H = ( V H , E H d , E H b ) is an ADMG and Y mcip ⊆ V H as follows. Note that V H will consist of two disjoint set of vertices, namely V H top and V H bot , i.e., V H = V H top ∪ V H bot .

Begin with V H top = V H bot = ∅ , Y mcip = ∅ . For any vertex v ∈ V G , add a vertex v to V H top with cost C ( v ) = ∞ . If v ∈ Y , add v to Y mcip .

For any directed edge ( v i , v j ) ∈ E G d with weight w d ij in G , add a new vertex v d ij to V H top , with cost C ( v d ij ) = w d ij , where

Draw directed edges ( v i , v d ij ) and ( v d ij , v j ) . Further, draw a bidirected edge between v i and v d ij .

For any bidirected edge { x i , x j } ∈ E G b with weight w b ij , add a new vertex, x b ij to V H top with cost C ( x b ij ) = w b ij . Add two bidirected edges { x i , x b ij } and { x j , x b ij } . Further, draw two directed edges ( x b ij , x i ) and ( x b ij , x j ) in H .

For any bidirected edge { x i , y j } with weight w b ij , add a new vertex z b ij to V H top with cost C ( z b ij ) = w b ij . Draw bidirected edges { z b ij , x i } and { z b ij , y j } . Then draw a directed edge from z b ij to x i .

For any bidirected edge between { y i , y j } ∈ E G b with weight w b ij in G , add a new vertex, y b ij to V H top with cost C ( y b ij ) = w b ij . Draw bidirected edges { y b ij , y i } and { y b ij , y j } . Further, for any x ∈ X , draw a directed edge from y b ij to x .

Let y 1 ≺ ... ≺ y k denote a topological ordering among vertices of Y . For every pair { y i , y j } of vertices of Y , where i < j , add vertices y ij i , y ij i +1 , . . . , y ij j to V H bot . Add y ij j to Y mcip . Draw the directed edges ( y k , y ij k ) for every i ≤ k ≤ j . Draw the directed edges ( y ij k , y ij i ) for every i < k < j , and the directed edge ( y ij i , y ij j ) . Draw a bidirected edge between y j and y ij i . Further, for any bidirected edge { y k , y l } ∈ E G b where i ≤ k, l ≤ j , add a new vertex y ij kl to V H bot , draw two bidirected edges { y ij kl , y ij k } and { y ij kl , y ij l } , and a directed edge ( y ij kl , y b ij ) . The costs of the all of the vertices in V H bot are infinite.

With abuse of notation, for any bidirected edge e b ij = { v i , v j } ∈ E G b and any directed edge e d ij = ( v i , v j ) ∈ E G d , we define T 2 ( e b ij ) = v b ij and T 2 ( e d ij ) = v d ij , respectively, where v b ij , v d ij ∈ V H are the vertices representing their corresponding edges.

We will utilize the following results to prove Proposition 4.2. More precisely, Lemmas 2 through 9 are used to prove Proposition A.10, which in turn is used to prove Proposition 4.2.

Lemma A.2. Suppose G is an ADMG, Y is a set of its vertices, and ( H , Y mcip ) = T ( G , Y ) . Each vertex y ∈ Y mcip is a district in H .

Proof. It suffices to show that for every pair of v 1 , v 2 ∈ Y mcip there is no bidirected edge between them in H . Suppose first that v 1 , v 2 ∈ Y . Any bidirected edge between v 1 and v 2 in G (if it exists) is removed in step (e) of the transformation, and none of the steps (a) through (f) add a bidirected edge between them. Otherwise, at least one of v 1 , v 2 , w.l.o.g. v 1 , is in Y mcip \ Y . Suppose w.l.o.g. that v 1 = y ij j . From step (f) of the transformation T , we know that v 1 has bidirected edges only to vertices y ij kj , where none of them is a member of Y mcip .

Lemma A.3. Suppose G is an ADMG, Y is a set of its vertices, and ( H , Y mcip ) = T 2 ( G , Y ) . Suppose there is a hedge formed for Q [ y ] in H , where y ∈ Y . Let H denote the set of vertices of this hedge. H does not include any of the vertices added in the step (f) of the transformation. That is, H ∩ V H bot = ∅ .

Proof. Define V 1 = { y ij kl ∈ V H bot , ∀ i, j, k, l } , and V 2 = V H bot \ V 1 . By construction of H , the vertices of V 2 have directed edges only to vertices in V 2 . Therefore, for each vertex v ∈ V 2 , we have v / ∈ Anc H [ H ] ( y ) . As a result, V 2 ∩ H = ∅ , since by definition of hedge, any vertex of H is an ancestor of y in H [ H ] . Now, consider an arbitrary vertex v ∈ V 1 . By construction of H , if there exists a bidirected edge { v, v ′ } ∈ E H b , we must have that v ′ ∈ V 2 . Therefore, if v ∈ H , there must be at least one vertex v ′ ∈ V 2 ∩ H . Since we proved V 2 ∩ H = ∅ , v cannot be in H . Consequently, V 1 ∩ H = ∅ .

Lemma A.4. Suppose G is an ADMG, Y is a set of its vertices, and ( H , Y mcip ) = T ( G , Y ) . Suppose there is a hedge formed for Q [ y ij j ] in H , where y i , y j ∈ Y and y ij j is the vertex corresponding to the pair ( y i , y j ) added in step (f) of the transform T . Let H denote the set of vertices of this hedge. If v ∈ H ∩ V H bot , then v has the superscript ij , that is, v is either one of the vertices y ij k , or one of the vertices y ij kl , where i ≤ k, l ≤ j . In the latter case, y b kl ∈ H .

Proof. Define V 1 = { y mn kl ∈ V H bot , ∀ m,n,k,l } , and V 2 = V H bot \ V 1 . Suppose V ∗ 1 = { v ij kl ∈ V H bot , ∀ k, l } and V ∗ 2 = { v ij k ∈ V H bot , ∀ k } . Also define V ′ 1 = V 1 \ V ∗ 1 , V ′ 2 = V 2 \ V ∗ 2 . For the first part of the claim, it suffices to show that V ′ 1 ∩ H = ∅ , V ′ 2 ∩ H = ∅ . By construction of H , the vertices of V ′ 2 do not have any child out of V ′ 2 . Therefore, V ′ 2 ∩ Anc H [ H ] ( y ij j ) = ∅ . This implies that V ′ 2 ∩ H = ∅ . Now let v i ′ j ′ 1 be an arbitrary vertex in V ′ 1 . By construction of H , v i ′ j ′ 1 has bidirected edges only to vertices of V ′ 2 . This implies that if v i ′ j ′ 1 ∈ H , there must be at least one vertex of V ′ 2 in H which is in contradiction with V ′ 2 ∩ H = ∅ . Therefore, v i ′ j ′ 1 / ∈ H . Since v i ′ j ′ 1 is an arbitrary vertex in V ′ 1 , we conclude V ′ 1 ∩ H = ∅ .

Now, we prove that if v ∈ H is one of the vertices y ij kl , we have y b kl ∈ H . Since y ij kl ∈ H , there exists a directed path from y ij kl to y ij j in H [ H ] . Since y b kl is the only child of y ij kl , the aforementioned path passes through y b kl . Therefore, y b kl ∈ H .

Lemma A.5. Suppose G ′ = ( V G ′ , E G ′ d , E G ′ b ) is an ADMG, Y ⊆ V G ′ is a set of its vertices, and ( H ′ , Y mcip ) = T ( G ′ , Y ) . Let E ′′ d ⊆ E G ′ d and E ′′ b ⊆ E G ′ b be arbitrary edges of G , and define E G d = E G ′ d \ E ′′ d , E G b = E G ′ b \ E ′′ b . Define G = ( V G , E G d , E G b ) and H = H ′ [ V H ′ \ V ′ ] , where V G = V G ′ and V ′ = { v ∈ V H ′ |∃ e ∈ E ′′ b ∪ E ′′ d , v = T 2 ( e ) } . Suppose there is a hedge formed for Q [ y ij j ] in H for some i, j . Let H denote the set of vertices of this hedge in H . The set of vertices Y ∗ = { y k | y ij k ∈ H } is a district in G [ Y ] . Moreover, H top = Anc H [ H top ] ( Y ∗ ) , where H top = H ∩ V H top .

Proof. First we prove that Y ∗ is a district in G [ Y ] . Consider an arbitrary vertex y ij k in H . By definition of hedge, there exists a bidirected path, p 1 , between y ij k and y ij j in H [ H ] . Let Y ij denotes the set of vertices in H such that their superscript is ij . Lemma A.4 implies that H ⊆ V H top ∪ Y ij . Furthermore, by construction of H , there is only one bidirected edge between Y ij and H \ Y ij , which is { y j , y ij i } . Therefore, all of the vertices on the path p 1 are in Y ij . Now, we define Y ′ 1 = { y k | y ij k ∈ p 1 } and Y ′ 2 = { y b kl | y ij kl ∈ p 1 } , i.e., the V H top counterparts of the vertices in p 1 . Since the vertices on p 1 are in H , Y ′ 1 ⊆ Y ∗ . From Lemma A.4, we know that if y ij kl ∈ H , then, y b kl ∈ H . It implies that Y ′ 2 ⊆ H . As a result, Y ′ 1 and Y ′ 2 are both vertices of H . Now if we replace all the vertices in p 1 with their corresponding counterpart in Y ′ 1 ∪ Y ′ 2 , we arrive at a bidirected path p 2 between y k and y j in H [ Y ′ 1 ∪ Y ′ 2 ] (as by construction the same edges exist in V H top ). By definition of G and H , if a vertex y b kl exists in H , the corresponding edge { y k , y l } exists in G . As a result, a bidirected path between y k and y l exists in G [ Y ′ 1 ] . Noting that y k is an arbitrary vertex in Y ∗ and Y ′ 1 ⊆ Y ∗ , this implies that all of the vertices of Y ∗ are in the same district as y j in G [ Y ∗ ] , which completes the proof.

Next, we prove that H top = Anc H [ H top ] ( Y ∗ ) . To this end, it suffices to show that there is a directed path form an arbitrary vertex v ∈ H top to Y ∗ in H [ H top ] . Since H forms a hedge for Q [ y ij j ] in H , there exists a directed path from v to y ij j in H [ H ] . This path must go through the only parent of y ij j , which is y ij i . Then, the last vertex on the path is one of the parents of y ij i . If this parent is y i , we are done as we have a directed path from v to y i , where y i ∈ Y ∗ and it has no ancestors in H \ H top . Otherwise, let this parent be y ij k for some i < k < j . Now the last vertex on the path before y ij k must be y k , which is the only parent of y ij k . Note that by definition of Y ∗ , y k ∈ Y ∗ . Therefore, v has a directed path to Y ∗ in H [ H top ] .

̸

Lemma A.6. Suppose G = ( V G , E G d , E G b ) is an ADMG, Y is a set of its vertices, and ( H , Y mcip ) = T 2 ( G , Y ) . Suppose there is a hedge formed for Q [ y ] in H for some y ∈ Y mcip . Let H denote the set of vertices of this hedge. Then H ∩ X = ∅ , where X = V G \ Y .

Proof. Since H forms a hedge for Q [ y ] in H , there exists a vertex h ∈ H such that { y, h } ∈ E H b . There are two possibilities for y ∈ Y mcip :

y = y i ∈ Y . From Lemma A.4 we have h / ∈ V H bot . Therefore, by construction of H , h = y b ij for some j .

y = y ij j ∈ V H bot . By construction of H , h = y ij kj for some k . Vertex h must have a directed path to y in H by definition of hedge, which must go through the only child of h , i.e., y b kl .

In both cases, we showed that there exists a vertex v = y b ij ∈ H for some i, j . By definition of hedge, there is a bidirected path, p , from v to y in H because v ∈ Anc H ( y ) . Since all of the children of v are in X , there is at least one vertex in X on path p . Therefore, H includes at least one vertex of X .

Lemma A.7. [Inverse transform preserves hedges.] Suppose G ′ = ( V G ′ , E G ′ d , E G ′ b ) is an ADMG, Y ⊆ V G ′ is a set of its vertices, and ( H ′ , Y mcip ) = T 2 ( G ′ , Y ) . Let E ′′ d ⊆ E G ′ d and E ′′ b ⊆ E G ′ b be arbitrary edges of G , and define E G d = E G ′ d \ E ′′ d , E G b = E G ′ b \ E ′′ b . Define G = ( V G , E G d , E G b ) and H = H ′ [ V H ′ \ V ′ ] , where V G = V G ′ and V ′ = { v ∈ V H ′ |∃ e ∈ E ′′ b ∪ E ′′ d , v = T 2 ( e ) } . Let W ⊆ V H top be a set of vertices of H . Let W s ⊆ W ∩ V G be a subset of W such that W s are vertices of G as well. Consider the inverse transform of H [ W ] in the ADMG G , i.e., for any v = v b ij ∈ W , delete v and all edges incident to it and draw a bidirected edge between v i and v j , and for any v = v d ij , delete v and all edges incident to it and draw a directed edge from v i to v j . Let the resulting subgraph (which is a subgraph of G ) be denoted by G [ W -1 ] with the set of vertices W -1 ⊆ V G . If Anc H [ W ] ( W s ) = W , then Anc G [ W -1 ] ( W s ) = W -1 . Moreover, if W is a district in H [ W ] , then W -1 is a district in G [ W -1 ] .

Proof. First, we show that if Anc H [ W ] ( W s ) = W , then Anc G [ W -1 ] ( W s ) = W -1 . Let v be an arbitrary vertex in W -1 . Vertex v is in W because W -1 ⊆ W . Since v ∈ W and v ∈ Anc H [ W ] ( W s ) , v has a directed path v → . . . v i → v d ij → v j · · ·→ w , denoted by l , to a vertex w ∈ W s in H [ W ] . For each vertex v d ij on path l , we have v i , v j ∈ G [ W -1 ] and since v d ij ∈ V H , by definition of G and H , there exists ( v i , v j ) ∈ E G d s.t. i ≺ j , and consequently, ( v i , v j ) ∈ E G [ W -1 ] d . Therefore, there exists a directed path from v to w in G [ W -1 ] . Noting that v is an arbitrary vertex in W -1 , we conclude that Anc G [ W -1 ] ( W s ) = W -1 .

Now, we prove that if W is a district in H [ W ] , then W -1 is a district in G [ W -1 ] . Consider two vertices v 1 , v 2 ∈ W -1 . Since v 1 , v 2 ∈ W and W is a district, there exists a bidirected path v 1 ↔ . . . v i ↔ v b ij ↔ v j · · ·↔ v 2 , denoted by p , between v 1 and v 2 in H [ W ] . Each vertex v b ij on path p is in H and v i , v j ∈ G [ W -1 ] . By definition of G and H , we have { v i , v j } ∈ E G b . Therefore, { v i , v j } ∈ E G [ W -1 ] b . Then, there is a bidirected path between v 1 and v 2 in G [ W -1 ] . Since v 1 and v 2 are two arbitrary vertices in W -1 , it implies that W -1 is a district in G [ W -1 ] .

̸

Lemma A.8. [Transform preserves hedges.] Suppose G ′ = ( V G ′ , E G ′ d , E G ′ b ) is an ADMG, Y ⊆ V G ′ is a set of its vertices, and ( H ′ , Y mcip ) = T 2 ( G ′ , Y ) . Let E ′′ d ⊆ E G ′ d and E ′′ b ⊆ E G ′ b be arbitrary edges of G , and define E G d = E G ′ d \ E ′′ d , E G b = E G ′ b \ E ′′ b . Define G = ( V G , E G d , E G b ) and H = H ′ [ V H ′ \ V ′ ] , where V G = V G ′ and V ′ = { v ∈ V H ′ |∃ e ∈ E ′′ b ∪ E ′′ d , v = T 2 ( e ) } . Let W ⊆ V G be a set of vertices of G such that W \ Y = ∅ . Let W s ⊆ W be a subset of W . Let the transformed graph of G [ W ] under T 2 be denoted by H ′′ , where H ′′ ⊆ H . Define W ∗ = V H ′′ top . If Anc G [ W ] ( W s ) = W , then Anc H [ W ∗ ] ( W s ) = W ∗ . Moreover, if W is a district in G [ W ] , then W ∗ is a district in H [ W ∗ ] .

Proof. First, we prove that if Anc G [ W ] ( W s ) = W , then Anc H [ W ∗ ] ( W s ) = W ∗ . Take an arbitrary vertex v ∈ W ∗ . There are two possibilities for v :

v ∈ W . That is, vertex v is in G [ W ] .

v / ∈ W . This implies that v represents an edge e between two vertices v i and v j in G [ W ] . There are three possibilities for e :

-e = ( v i , v j ) . By construction of H , v is parent of v j in H [ W ∗ ] , where v j is a vertex of G [ W ] .

-e = { v i , v j } and v i ∈ X or v j ∈ X . In this case, v is parent of at least one of v i and v j in H [ W ∗ ] , w.l.o.g. v i , where v i is a vertex of G [ W ] .

̸

-e = { v i , v j } and v i , v j ∈ Y . By construction of H , v is parent of all vertices in V G \ Y . Since W \ Y = ∅ , there exists a vertex x in G [ W ] such that v is a parent of x .

In all three cases above, we proved that there exists a vertex x ∈ W such that v is a parent of x .

Therefore, we showed that any vertex v ∈ W ∗ either is itself a vertex in W or is a parent of a vertex in W . As a result, it suffices to show that every w ∈ W has a directed path to W s in H [ W ∗ ] . We know that w has a directed path to W s in G [ W ] such as p . Take an arbitrary pair of consecutive vertices on this path, such as v 1 and v 2 . The directed edge ( v 1 , v 2 ) exists in G [ W ] . As a result, the directed path v 1 → v d 12 → v 2 exists in H [ W ∗ ] . Starting at w and repeating this argument for every pair of consecutive vertices on p , we conclude that there exists a directed path from w to W s , which completes the proof.

Now, we show that if W is a district in G [ W ] , then W ∗ is a district in H [ W ∗ ] . Take an arbitrary vertex v ∈ W ∗ . There are two possibilities for v :

v ∈ W . That is, v is a vertex in G [ W ] .

v / ∈ W . In this case, at least one of the vertices v represents an edge e between two vertices v i and v j in G [ W ] . By construction of H , v is connected to at least one of v i or v j , w.l.o.g. v i , by a bidirected edge, where v i ∈ W .

We showed that any vertex v ∈ W ∗ either is in W , or is connected to a vertex in W through a bidirected edge. Therefore, it suffices to show that for any two vertices w 1 , w 2 ∈ W there exists a bidirected path between w 1 and w 2 in H [ W ∗ ] . Since w 1 , w 2 ∈ W , there is a bidirected path, p , between w 1 and w 2 in G [ W ] . Take an arbitrary pair of consecutive vertices on this path, such as v 1 and v 2 . The bidirected edge { v 1 , v 2 } exists in G [ W ] . As a result, the bidirected path v 1 ↔ v b 12 ↔ v 2 exists in H [ W ∗ ] . Starting at w and repeating this argument for every pair of consecutive vertices on p , we conclude that there exists a bidirected path from w 1 to w 2 , which completes the proof.

̸

Lemma A.9. Suppose G is an ADMG, and Y is a subset of its vertices. Also let Y ∗ be a district in G [ Y ] . If the set of vertices H form a hedge for Q [ Y ∗ ] , then H \ Y = ∅ .

̸

Proof. Assume by contradiction H \ Y = ∅ , i.e., H ⊆ Y . By definition of hedge, we know H \ Y ∗ = ∅ . Take an arbitrary vertex v ∈ H \ Y ∗ . Furthermore, v ∈ Y \ Y ∗ because H ⊆ Y . Since H forms a hedge for Q [ Y ∗ ] , H is a district in G [ H ] . Therefore, there exists a bidirected path between v and a vertex y ∗ ∈ Y ∗ in Q [ Y ] which is in contradiction with the assumption that Y ∗ is a district in G [ Y ] .

Proposition A.10. Suppose G ′ = ( V G ′ , E G ′ d , E G ′ b ) is an ADMG, Y ⊆ V G ′ is a set of its vertices, and ( H ′ , Y mcip ) = T 2 ( G ′ , Y ) . Let E ′′ d ⊆ E G ′ d and E ′′ b ⊆ E G ′ b be arbitrary edges of G , and define E G d = E G ′ d \ E ′′ d , E G b = E G ′ b \ E ′′ b . Q [ Y ] is identifiable in G = ( V G , E G d , E G b ) if and only if Q [ Y mcip ] is identifiable in H = H ′ [ V H ′ \ V ′ ] , where V G = V G ′ and V ′ = { v ∈ V H ′ |∃ e ∈ E ′′ b ∪ E ′′ d , v = T 2 ( e ) } .

Proof. We prove the contrapositive, i.e., Q [ Y ] is not identifiable in G iff Q [ Y mcip ] is not identifiable in H .

If part. Suppose Q [ Y mcip ] is not identifiable in H . That is, there exists a hedge formed for Q [ Y mcip ] in H . From Lemma A.2, this hedge is formed for Q [ y ′ ] for some y ′ ∈ Y mcip . Denote the set of vertices of this hedge by H . We consider two possibilities separately:

̸

y ′ = y i , where y i ∈ Y . From Lemma A.3, H ⊆ V H top . Taking W = H in Lemma A.7, W -1 is a set of vertices in G such that Anc G [ W -1 ] ( y ) = W -1 , and W -1 is a district in G . Now take Y ∗ to be the district of G [ Y ] that includes y i . By definition of hedge, G [ W -1 ∪ Y ∗ ] forms a hedge for Q [ Y ∗ ] in G . Note that from Lemma A.6, W -1 \ Y = ∅ . As a result, Q [ Y ] is not identifiable in G .

̸

y ′ = y ij j , where y i , y j ∈ Y and y ′ is one of the vertices added to H in the last step of the transformation T (step (f)). Define the set Y ∗ = { y k | y ij k ∈ H } . From Lemma A.5, Y ∗ is a district in G , and therefore a district in G [ Y ] . As a result, it suffices to show that there exists a hedge formed for Q [ Y ∗ ] in G . Now define H top = H ∩ V H top . By definition of hedge, H is a district in H [ H ] , i.e., it is connected over its bidirected edges. By construction of H , there is only one bidirected edge between the vertices in H top and H \ H top , which is the bidirected edge between y j and y ij i . Therefore, this edge is a cut set that partitions the graph H [ H ] into two connected components H [ H top ] and H [ H \ H top ] . That is, H [ H top ] is connected over its bidirected edges and therefore H top is a district in H [ H top ] . Further, from Lemma A.5, H top = Anc H [ H top ] ( Y ∗ ) . Noting that H top ⊆ V H top , taking W = H top in Lemma A.7, W -1 is a district in G and Anc G [ W -1 ] ( Y ∗ ) = W -1 . Note that from Lemma A.6, W -1 \ Y = ∅ . Therefore, the set of vertices W -1 form a hedge for Q [ Y ∗ ] in G . Hence, Q [ Y ] is not identifiable in G .

̸

Only if part. Suppose Q [ Y ] is not identifiable in G . It implies that there exists a district of G [ Y ] such as Y ∗ such that there is a hedge formed for Q [ Y ∗ ] in G . Let H denote the set of vertices of this hedge. From Lemma A.9, H \ Y = ∅ . Define W ∗ as in Lemma A.8, that is the transform T ( G [ H ] , Y ∗ )

without step (f) (only on the vertices of V H top ). Note that Y ∗ ⊆ W ∗ . We consider the following two cases separately:

Y ∗ = { y } , that is, Y ∗ is a single vertex. From Lemma A.8, W ∗ is a district in H [ W ∗ ] , and Anc H [ W ∗ ] ( y ) = W ∗ . By definition of hedge, the vertices W ∗ form a hedge for Q [ y ] in H . Note that y ∈ Y mcip , and from Lemma A.2 it is a district of H [ Y mcip ] . As a result, Q [ Y mcip ] is not identifiable in H .

| Y ∗ | ≥ 2 . Let y i and y j be the first and the last vertices of Y ∗ in the topological order. Define Y ij ∗ = { y ij k | y k ∈ Y ∗ } ∪ { y ij kl | y k , y l ∈ Y ∗ } . Y ij ∗ are the vertices in V H bot with superscript ij corresponding to the vertices in Y ∗ . Note that y ij i , y ij j ∈ Y ij ∗ , since y i , y j ∈ Y ∗ . Since y ij j ∈ Y mcip and from Lemma A.2 y ij j is a district in H [ Y mcip ] , it suffices to show that there is a hedge formed for y ij j in H . We show that the vertices W = W ∗ ∪ Y ij ∗ form a hedge for y ij j in H . From Lemma A.8, Anc H [ W ∗ ] ( Y ∗ ) = W ∗ , that is, all of the vertices in W ∗ are ancestors of Y ∗ in H [ W ∗ ] , and therefore in H [ W ] . Also, the vertices y ij kl in Y ij ∗ have a direct edge to their corresponding vertex in W ∗ , i.e., y b kl , and therefore are ancestors of Y ∗ in H [ W ] as well. Further, each vertex in Y ∗ such as y k is a parent of y ij k , which is in turn a parent of y ij i (or is y ij i itself if k = i .) Finally, y ij i has a directed edge to y ij j by construction. As a result, all of the vertices W have a direct path to y ij j in H [ W ] . That is, Anc H [ W ] ( y ij j ) = W . It now remains to show that W is a district in H [ W ] . From Lemma A.8, W ∗ is a district in H [ W ∗ ] . As a result, the vertices W ∗ are connected through bidirected edges in H [ W ] . There is a bidirected edge between y j and y ij i by construction of H . It suffices to show that for any v ∈ Y ij ∗ , there exists a bidirected path between v and y ij i in H [ W ] . A vertex y ij kl ∈ Y ij ∗ (with double subscript, which are due to the bidirected edges among Y ∗ ) has bidirected edges to y ij k and y ij l , which are both in Y ij ∗ by definition. Now take an arbitrary vertex y ij k ∈ Y ij ∗ (with single subscript, due to vertices in Y ∗ ). We know y k ∈ Y ∗ , as y ij k ∈ Y ij ∗ , by definition of Y ij ∗ . Y ∗ is a district in G [ Y ∗ ] . That is, there exists a bidirected path from y k to y i in G [ Y ∗ ] . From Lemma A.8 by taking W = Y ∗ , there is a bidirected path p from y k to y i in H [ Y ∗ ∪ { y lm | y l , y m ∈ Y ∗ } ] . By construction of H , if we replace each vertex v on p by v ij , we achieve a bidirected path p ′ with vertices in Y ij ∗ from y ij k to y ij i , which completes the proof.

Proof of Proposition 4.2. The reduction from the edge ID problem to MCIP was shown through the proof of Proposition A.10. The opposite direction is an immediate corollary of Proposition A.1.

Corollary A.11. The edge ID problem and MCIP are equivalent.

B Maximal hedge

Herein, we present the algorithm for recovering the maximal hedge formed for Q [ Y ] in a given ADMG G (see Definition 2.5). Maximal hedge was initially defined in [2] under the name hedge hull . We adopt the same definition, and when G [ Y ] comprises several districts, we define the maximal hedge as the union of the hedge hulls formed for each district of G [ Y ] . As a result, the complete procedure of recovering the maximal hedge for a query Q [ Y ] , summarized in Algorithm 3, finds the maximal hedge formed for each district Y i of G [ Y ] and returns the union of them. This procedure is used as a subroutine MH in Algorithm 1. The function HHull is in fact Algorithm 1 borrowed from [2]. This function is proven to recover the union of all hedges formed for Y i , where Y i is one of the districts of G [ Y ] (see Lemma 6 of [2]).

̸

C Towards generalizing Assumption 2.1

Lemma 3.2 states the equivalence of Problems 1 and 2 with the edge ID problem under Assumption 2.1. However, as mentioned in the main text, this equivalence holds in the more general setting where we allow for perfect negative correlations among edges. As an example, consider the graph of Figure 6. Suppose that the performed statistical independence tests show that the two variables z and y are independent of each other with high confidence. As a result, the expert believes that the edges ( z, x ) and ( x, y ) must not exist simultaneously, as otherwise the causal path from z to y would make them dependent. In such cases, the belief of the expert can be modeled as probabilities p and q assigned to the existence of the edges ( z, x ) and ( x, y ) , respectively, as well as a perfect negative correlation between them.

Note that the aforementioned constraint, i.e., that the edges do not exist simultaneously, can be specified for any number of edges, not limited to two edges only. For instance, the expert might believe at least one of the edges along a causal path of length n must not exist in the true ADMG describing the system. This belief can be modeled as an extra constraint in the optimization of Equations 2 and 3. We show that with the specification of such negative correlations, Problems 1 and 2 can still be cast as an instance of the edge ID problem. Therefore, the results presented in this work are valid in this setting as well.

Assumption C.1. The edges in G are assigned probabilities p e , ∀ e ∈ G , and perfect negative correlations are defined among subsets of edges. More precisely, for any subset E ⊆ E G d ∪ E G b , there is either 1) no constraint (mutually independent), or 2) the constraint that at least one of the edges in E must not exist in the true ADMG (perfect negative correlation).

Proposition C.1. Under Assumption C.1, there exists a reduction from Problems 1 and 2 to the edge ID problem and vice versa with the time complexity in the order of O ( | C | · | V G | + | E G d ∪ E G b | ) , where C is the set of perfect correlation constraints.

Proof. First note that we proved the equivalence of Problems 1 and 2 with the edge ID problem without the perfect correlation constraints in Lemma 3.2. As a result, under assumption C.1, i.e., by adding the perfect correlation constraints, Problems 1 and 2 are equivalent to a modified edge ID problem with those constraints. But we claim that there exists and instance of the original unconstrained edge ID problem which is equivalent to these problems. To see this, first note that we know from Corollary A.11 that the edge ID problem is equivalent to MCIP. Therefore, it suffices to show that there exists an instance of MCIP which is equivalent to the constrained edge ID mentioned above. To this end, consider the transform T 2 ( G , Y ) introduced in Section A.2. This transformation

x

1

x

2

x

3

x

′

1

x

′

2

x

′

3

ˆ

y

maps an instance of the edge ID problem to an instance of MCIP. Applying this transformation to the constrained edge ID problem, we can map the constrained edge ID to an instance of MCIP with extra constraints, with transforming the constraints as well. That is, if for instance, there is a perfect negative correlation among the edges e 1 , e 2 in G , this constraint is mapped to a negative perfect correlation on the corresponding vertices in H , namely T 2 ( e 1 ) , T 2 ( e 2 ) . In words, this constraint would be that at least one of T 2 ( e 1 ) and T 2 ( e 2 ) must be intervened upon. We show that such constraints can be integrated into the original definition of MCIP.

Suppose we have an MCIP problem in ADMG G with query Q [ Y ] , with the extra constraint that at least one of the vertices X ⊆ V G must be intervened upon. Consider the example of X = { x 1 , x 2 , x 3 } in Figure 7. We build a new ADMG G ′ by adding vertices { x ′ | x ∈ X } , i.e., a new vertex corresponding to each vertex in X , along with an auxiliary vertex ˆ y to G . We fix a random ordering over the vertices of X , and denote the set of vertices of X as x 1 , ..., x m . We add the directed edges ( x i , x ′ i ) to G ′ , as well as the bidirected edges { x i , x ′ i } . Further, we draw directed edges ( x ′ i , x ′ i +1 ) for every 1 ≤ i < m . Finally, we draw the directed edge ( x ′ m , ˆ y ) and the bidirected edge { x 1 , ˆ y } . Refer to the graph in Figure 7 for an example with X = { x 1 , x 2 , x 3 } . Note that the set X ∪ X ′ ∪{ ˆ y } forms a hedge for Q [ˆ y ] , where X ′ = { x ′ | x ∈ X } Now it suffices to set the cost of intervention on vertices of X ′ to infinity, and consider MCIP for the query Q [ Y ∪ { ˆ y } ] in G ′ . It is straightforward to see that the objective of this problem would be to find the minimum cost intervention for identification of Q [ Y ] , with the constraint that at least one of the vertices in X must be intervened on. Note that as soon as one vertex in X gets intervened upon, there is no hedge left for Q [ˆ y ] . Also it is noteworthy that adding this structure does not add any new hedges formed for Q [ Y ] , since the structure only includes new descendants for X which have no directed paths to Y . Also note that the vertices X ′ and ˆ y are specific to the very constraint corresponding to the set of vertices X . For any constraint, we add such a structure to G . The number of vertices (and therefore the time complexity) is at most in the order O ( | C | · | V G | ) , where C is the set of constraints.

C.1 Further applications

The relaxation provided in this Appendix allows the approaches proposed in this work to be applicable to a more general set of problems. Herein, we discuss one such application.

Let us assume we run our algorithm which returns the subgraph with the highest probability, G 1 . However, the probability that G 1 is the true causal structure describing the system might be too low. In such a case, the researcher might be interested in having a ranking of most probable graphs (for instance, the 10 most probable graphs), rather than only one most probable graph. This could be helpful for instance, when a unique identification formula is valid in a few of these graphs, or the researcher is interested in identifying more than one causal query. The methods discussed in this work along with the relaxation proposed in this appendix provide the tools to recover such a ranking (of the most probable graphs in which a query is identifiable). To see this, note that based on what we proposed in this Appendix, perfect negative correlation constraints can be added to the edge ID problem without additional computational cost. So we begin by solving the original problem, which yields a graph G 1 . We then solve it for a second time (i.e., re-run the algorithm), with the only difference that we add the perfect negative correlation constraint over the set of all edges of G 1 (i.e., we force the algorithm to exclude at least one of the edges of G 1 .) The solution to this problem (let us call it G 2 ) is the highest probability graph among all subgraphs except G 1 , i.e., it is the second highest

probability graph in which the query is identifiable. Continuing in this manner, running the algorithm n times would give us a ranking of the n highest probability graphs.

D Heuristic algorithms

Algorithm 2 was devised considering the fact that every hedge formed for Q [ Y ] must include a vertex that has a bidirected edge to Y . As mentioned in Section 4.2, an analogous approach, summarized in Algorithm 4, uses the fact that any hedge formed for Q [ Y ] must include a parent of Y .

Let Y ⊆ V G be a set of vertices of G such that G [ Y ] comprises of only one district. Let Z := { z ∈ V G |∃ y ∈ Y : ( z, y ) ∈ E G d } \ Y denote the set of vertices that have at least one directed edge to a vertex in Y , i.e., the parents of Y excluding Y . Any hedge formed for Q [ Y ] contains at least one vertex of Z . As a result, in order to eliminate all the hedges formed for Q [ Y ] , it suffices to ensure that none of the vertices in Z appear in the final hedge. To this end, for any z ∈ Z , it suffices to either remove all the directed edges between z and Y , or eliminate all the bidirected paths from z to Y . The problem of eliminating all bidirected paths from Z to Y can be cast as a minimum cut problem between Z and Y in the edge-induced subgraph of G over its bidirected edges. To add the possibility of removing the directed edges between Z and Y , we add an auxiliary vertex z ∗ to the graph and draw a bidirected edge between z ∗ and every z ∈ Z with weight w = ∑ y ∈ Y w ( z,y ) , i.e., the sum of the weights of all directed edges between z and Y . Note that z can have directed edges to multiple vertices in Y . We then solve the minimum cut problem for z ∗ and Y . If an edge between z ∗ and z ∈ Z is in the solution to this min-cut problem, it translates to removing all the directed edges from z to Y in the original problem. Note that we can run the algorithm on the maximal hedge formed for Q [ Y ] in G rather than G itself.

Algorithm 4 Heuristic algorithm 2.

1: function HEID2 ( G , Y, W G ) 2: G ′ ← MH ( G , Y ) 3: Z ←{ z ∈ V G ′ |∃ y ∈ Y : ( z, y ) ∈ E G ′ d } \ Y 4: H ← The induced subgraph of G ′ on its bidirected edges. 5: W H ←{ w e ∈ W G | e ∈ H} 6: V H ← V H ∪ { y ∗ , z ∗ } 7: for z ∈ Z do 8: E H ← E H ∪ { z ∗ , z } 9: W H ← W H ∪ { w { z ∗ ,z } = ∑ y w ( z,y ) } 10: for y ∈ Y do 11: E H ← E H ∪ { y, y ∗ } 12: W H ← W H ∪ { w { y,y ∗ } = ∞} 13: E ← MinCut ( H , W H , z ∗ , y ∗ ) 14: return ( E, ∑ e ∈ E w e )

E Experiments

Noting that the synthetic/simulation results in the main paper were for graphs with a log( n ) /n sparsity constraint, we begin this section by providing a set a results on the simulated graphs without the sparsity penalty for comparison. Then, we provide information about the causal discovery algorithm used to derive the psychology 'Psych' real-world graph. We also provide experimental results for Problem 2 formulation in Section E.3.

E.1 Additional simulation results without sparsity constraint

The simulation results for graphs generated without the sparsity constraint are shown in Figure 8. These results illustrate monotonic increases in runtime and cost as the number of nodes increases. Our proposed heuristic algorithms (HEID-1 and HEID-2) maintain runtimes less than 0.5 seconds even for 250 nodes. In contrast, the two exact algorithms (MCIP-exact and EDGEID) exceed the three

(a) Runtimes.

(b) Solution costs.

10²

200

10

150

Time (seconds)

HEID-2

10

HEID-2

00

MCIP-exact

10

50

EDGEID

MCIP-exact

EDGEID

10

0

50

100

150

200

250

0

50

100

150

200

250

Number of vertices before Y in topological order

Number of vertices beforeY in topological order

Fraction reached time-out (> 3mins)

1.0

MCIP-exact

0.8

EDGEID

0.6

0.4

0.2

0.0

0

50

100

150

200

250

Number of vertices before Y in topological order

(c) Fraction for which runtime of 3 minutes exceeded.

minute runtime limit at only 20 nodes, and the MCIP heuristic variants (MCIP-H1 and MCIP-H2) have runtimes which increase exponentially with the number of nodes. These results highlight the efficiency of our proposed heuristic algorithms to find solutions with equivalent cost with significantly faster runtimes.

E.2 Psychology graph discovery

The settings for deriving the putative structure used on the psychology real-world graph are provided in Table 3.

(a) Runtimes.

(b) Solution costs.

0.040

0.035

HEID-2

HEID-2

0.030

0.025

EDGEID

EDGEID

0.015

0.010

0.005

0.000

0

50

100

150

量

200

250

50

100

150

200

250

0.7

EDGEID

0.5

0.2

0.1

0.0

50

001

150

200

250

(c) Fraction runtime exceeded 3 min.

E.3 Simulation results for Problem 2 formulation

The experimental setup is exactly as in the main text (the results depicted in Figure 4), except that the probabilities are chosen in the range [0 . 01 , 1] instead of [0 . 51 , 1] , and we use the weight mapping corresponding to Problem 2 as described in Lemma 3.2. That is, we map each probability p e to the weight -log(1 -p e ) in the corresponding edge ID problem.

The simulation results are presented in Figure 9. Runtimes and costs are shown for the subset of graphs for which all algorithms found a solution (to facilitate comparison). Runtimes for each algorithm are shown in Fig. 9a, where it can be seen that our proposed HEID-1 and HEID-2 heuristic algorithms have negligible runtime. In contrast, EDGEID had large runtime variance which depended heavily on the specifics of the graph under evaluation, particularly for graphs with fewer vertices. The costs for each graph are shown in Fig. 9b. Figure 9c shows the fraction of evaluations for which the runtime exceeded 3 minutes (applicable to the exact algorithms). In general, and owing to the sparsity penalty in our graph generation mechanism, the cost of identified solutions falls with the number of vertices. Overall, HEID-1 was both the most consistent in terms of finding a solution, having a short runtime, and achieving a close to optimal cost.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers two problems: 1. Finding the most probable graph that makes a desired causal query identifiable. 2. Finding the graph with the highest aggregate probability over its edge-induced subgraphs makes a desired causal query identifiable. This paper shows that both problems reduce to a special combinatorial optimization problem called the edge ID problem. They prove that the edge ID problem is NP-hard, implying that both of the problems are also NP-hard. The paper presents several exact and heuristic algorithms for these problems and evaluates their performances through different experiments. The experiments note that the heuristic algorithms performed remarkably well across all metrics. The paper also discusses the application of these algorithms to four real-world datasets.\n\nSTRENGTHS:\n- The paper addresses a complex causal inference problem, specifically identifying the most probable graph that makes a desired causal query identifiable. This is a significant contribution to the field.\n\n- The authors provide a detailed analysis of the problem's complexity, demonstrating that it reduces to a special combinatorial optimization problem, the edge ID problem, which they prove to be NP-hard. This rigorous theoretical analysis is a strength of the paper.\n\n- The authors propose several exact and heuristic algorithms to solve the problem and evaluate their performance through different experiments. The heuristic algorithms, in particular, performed remarkably well across all metrics, demonstrating the practical applicability of the proposed methods.\n\n- The paper also applies the proposed algorithms to real-world datasets, further demonstrating their practical utility. The authors provide a detailed comparison of runtimes, solution costs, and failure rates, offering valuable insights into the performance of the algorithms in real-world scenarios.\n\nWEAKNESSES:\n- The authors made the assumption that the edges in the graph are mutually independent. This assumption may not be held in all cases.\n\n- The external validity of the derived subgraph is not guaranteed. This means that the subgraph may not be correctly specified with respect to the corresponding real-world process. This could limit the applicability of the results in practical scenarios.\n\n- The EDGEID algorithm, one of the exact algorithms proposed in the paper, had large runtime variance, which depended heavily on the specifics of the graph under evaluation, particularly for graphs with fewer vertices. This could limit its utility in certain scenarios.\n\n- The EDGEID algorithm also timed out on all but one of the real-world structures tested, indicating that it may not be as consistent as other algorithms in terms of runtime.\n\n- The runtimes for the MCIP variants exceeded the HEID variants due to the required transformation. This could be a potential drawback in scenarios where computational efficiency is a priority.\n\nQUESTIONS:\nCould you elaborate on the assumption of mutual independence of edges in the graph? How might the results change if this assumption does not hold?\n\nThe external validity of the derived subgraph is not guaranteed. Could you discuss potential strategies for validating these subgraphs in real-world scenarios?\n\n- The EDGEID algorithm showed large runtime variance and often timed out on real-world structures. Are there any plans to improve the performance of this algorithm in future work?\n\n- The paper mentions that the heuristic algorithms performed remarkably well across all metrics. Could you provide more details on the specific scenarios or types of data where these heuristic algorithms are most effective?\n\n- The paper suggests that future work should explore cases where the assumption of mutual independence of edges does not hold. Could you provide some insights into how this exploration might be conducted?\n\n- The paper applies the proposed algorithms to four real-world datasets. Are there plans to test these algorithms on a wider variety of datasets, particularly those with different characteristics or from different domains?\n\n- The paper presents a detailed analysis of the problem's complexity. Could you discuss the implications of this complexity for the practical application of the proposed algorithms?\n\nLIMITATIONS:\n- The authors made the assumption that the edges in the graph are mutually independent. This assumption may not hold in all cases, and the authors themselves acknowledge that future work should explore scenarios where this assumption does not hold.\n\n- The external validity of the derived subgraph is not guaranteed. This means that the subgraph may not be correctly specified with respect to the corresponding real-world process. This could limit the applicability of the results in practical scenarios.\n\n- The EDGEID algorithm, one of the exact algorithms proposed in the paper, had large runtime variance which depended heavily on the specifics of the graph under evaluation, particularly for graphs with fewer vertices. This could limit its utility in certain scenarios.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper is about causal identification in a setup where the assumption of reliable knowledge of the causal graph is relaxed. The authors assign a probabilistic weight to the possible confounders.  The combinatorial task of finding the most probable causal graph such that a given causal query is identifiable is considered. The same probable with an aggregate probability over the induced subgraph is also considered. Not surprisingly, both these problems are hard. The authors first deliver an exact solution. Two approximate procedures are obtained: (i) a recursion calling min-cut; (ii) a reduction to MCIP. Experiments on real and synthetic data are finally discussed.\n\nSTRENGTHS:\nThe idea of an uncertain causal network with distributive semantics looks very reasonable.\n\nThe hardness results are not surprising, but they appear important for properly characterising the problem.\n\nThe heuristics presented are non-trivial and effective.\n\nWEAKNESSES:\n- Both heuristic algorithms lack a worst-case complexity characterisation.\n- In the case of MCIP, it needs to be clarified whether the polynomial reduction considered by Proposition 4.2 can be implemented in practice.\n- Problems 1 and 2 are not necessarily the only/best ways to cope with uncertain causal networks. No robust discussions to advocate such a choice are reported.\n\nQUESTIONS:\nI understand that min-cut has a poly solution, but what is the overall complexity of the first heuristic algorithm?\nSimilarly, is it possible to implement the polynomial transformation in Proposition 4.2, and if so, which is the overall complexity of reducing the problem to MCIP and solving the corresponding instance?\nThe authors assume that the query should be identifiable, and in this perspective, Problems 1 and 2 are very reasonable ways to face the problem. But why should the query be identifiable? There is a lot of literature on non-identifiable queries. E.g., if the output is identifiable only in a very unlikely graph, how should we proceed?\n\nLIMITATIONS:\nI don't see relevant issues on this point.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

comments: SUMMARY:
The paper considers a setting in which we have a probabililty distribution over causal graphs. This paper considers the problem of finding the most probable subgraph in which a given query is identified, and then that subgraph with the highest sum of probabilities of its own subgraphs in which the query is identified. (The latter problem is significant since the result of the identification must agree among all these subgraphs.) A reduction is given to a known NP-hard problem for which various algorithms are evaluated empirically.

STRENGTHS:
The paper is well-written and the results are clearly set out, extensive, and correct as far as I can tell.

WEAKNESSES:
I had a little bit of difficulty understanding the problem initially. I am currently a little dubious about the paper's ultimate significance, but if the authors can make the links to structure learning more explicit then it will represent an interesting synthesis with the other approach in which ADMG encapsulates "a priori" facts about the causal structure that are known for certain.

QUESTIONS:
- Can the authors point to any other literature besides citation [12] in which probability distributions over ADMGs are considered? Or provide motivating examples perhaps involving statistical tests in structure learning.
- Line 47: "to addresses" -> "to address"

LIMITATIONS:
Yes, they have (for instance, the assumption of independence between edges).
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-06

comments: SUMMARY:
The paper studies the problem of causal identification in a setting which the structure of a causal graph (of interest) is probabilistically uncertain i.e., only known per a certain degree of belief or with a degree of confidence of a particular statistical set, asking the most likely subgraph in which causal effect identifiable. In doing so, paper reduces it the combinatorial optimisation problem i.e., edge ID (as they call it) whose computational complexity is shown to be NP-hard through a reduction from minimum-vertex cover problem. Authors also introduce exact and approximate algorithm and show empirically that the algorithms work good as well.

STRENGTHS:
- Deep theoretical paper.

- The theoretical results  are complemented with good empirical evaluation. 

 - Good exposition.

WEAKNESSES:
I did not spot any major weakness. Only very few minor issues:

- causal query better be formally mentioned in *the introduction* (i.e., how does it look like formally)
-problem To be surmountable (missing dot)
-In Figure 1 caption, also mention the longer version of Q[y] to help following it. 
-Definition 2.2: explain what you mean when you say functional (again formally). 
- (potential typo): In Example 1, I sense the direction of the subset to be other way around  Gˆ⊆G1
-Assumption 2.1: form (1) -> Equation 1
-In the next [s]ection

QUESTIONS:
-how about NP-completeness case?
-Can the work of Eiter Lukasiewicz 2001 be any useful/connect for further results in your setting?

LIMITATIONS:
I did not see any particular limitation.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-08

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors consider the problem of causal effect identification assuming the given causal graph G is probabilistic where each edge (directed or bi-directed) is associated with a measure of confidence. Under this setting, the objective is to identify a subgraph of G with the highest plausibility such that the target causal effect Q[Y] is identifiable. The authors propose an exact algorithm that is asymptotically exponential, two heuristic approaches, and a reduction to an NP-hard problem that has approximation algorithms. Finally, they evaluate the proposed methods on synthetic and real-world graphs.\n\nSTRENGTHS:\nThe formulated problem of identifying the most plausible subgraph where a target effect is identifiable is algorithmically interesting and the author highlight its complexity by showing it to be NP-hard. Moreover, the paper is well-written, for the most, with examples and discussions to illustrate the problem (i.e., Section 2.1). The experimental evaluation of the heuristic algorithms shows favourable performance.\n\nWEAKNESSES:\n1- Motivation for the problem: Whenever the target causal effect is not identifiable in the full graph G, it is not justifiable why we would drop edges from G just to make the effect identifiable; an alternative to exact identification could be bounding the target effect. The authors can provide more motivation in the introduction to justify their approach.\n\n2- Lack of performance guarantees for the heuristic approaches: There is no theoretical analysis for the heuristic algorithms discussed in Section 4.2 in terms of upper bound on the cost.\n\nQUESTIONS:\n1- \"when $w^{th} = 0$, Algorithm 1 will output the optimal solution\": Suppose we have G={Z -> X, X -> Y, X <--> Z, Z <--> Y}, $W_G$={$\\infty, \\infty, 10, 1$}, $w^{ub}=\\infty$, and $w^{th}=0$. Running this example does not lead to any solution unless I'm mistaken. It should optimally return the subgraph after removing the bi-directed between Z and Y. Unless you mean that whenever $w^{th} = 0$, then an alternative formulation of Algorithm 1 is run which discards the threshold and simply searchers for the optimal solution.\n\n2- Implication of Remark 2.1: Can you elaborate on the need for this assumption to only consider Q[Y] instead of general causal queries? Is it a necessary assumption for the problem to be defined or are general queries more challenging?\n\n3- Typos:\n  + Line 68: period after \"and open problem\"\n  + Line 224: (Line 12) --> (Line 13)\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-30"}