Title: Characterization and Learning of Causal Graphs with Small Conditioning Sets
Venue: NeurIPS 2023
Accepted: yes
Keywords: causal discovery
Abstract: Constraint-based causal discovery algorithms learn part of the causal graph structure by systematically testing conditional independences  observed in the data. These algorithms, such as the PC algorithm and its variants, rely on graphical characterizations of the so-called equivalence class of causal graphs proposed by Pearl. However, constraint-based causal discovery algorithms struggle when data is limited since conditional independence tests quickly lose their statistical power, especially when the conditioning set is large. To address this, we propose using conditional independence tests where the size of the conditioning set is upper bounded by some integer k for robust causal discovery. The existing graphical characterizations of the equivalence classes of causal graphs are not applicable when we cannot leverage all the conditional independence statements. We first define the notion of k-Markov equivalence: Two causal graphs are k-Markov equivalent if they entail the same conditional independence constraints where the conditioning set size is upper bounded by k. We propose a novel representation that allows us to graphically characterize k-Markov equivalence between two causal graphs. We propose a sound constraint-based algorithm called the k-PC algorithm for learning this equivalence class. Finally, we conduct synthetic, and semi-synthetic experiments to demonstrate that the k-PC algorithm enables more robust causal discovery in the small sample regime compared to the baseline algorithms.

Characterization and Learning of Causal Graphs with Small Conditioning Sets

Abstract

Constraint-based causal discovery algorithms learn part of the causal graph structure by systematically testing conditional independences observed in the data. These algorithms, such as the PC algorithm and its variants, rely on graphical characterizations of the so-called equivalence class of causal graphs proposed by Pearl. However, constraint-based causal discovery algorithms struggle when data is limited since conditional independence tests quickly lose their statistical power, especially when the conditioning set is large. To address this, we propose using conditional independence tests where the size of the conditioning set is upper bounded by some integer k for robust causal discovery. The existing graphical characterizations of the equivalence classes of causal graphs are not applicable when we cannot leverage all the conditional independence statements. We first define the notion of k -Markov equivalence: Two causal graphs are k -Markov equivalent if they entail the same conditional independence constraints where the conditioning set size is upper bounded by k . We propose a novel representation that allows us to graphically characterize k -Markov equivalence between two causal graphs. We propose a sound constraint-based algorithm called the k -PC algorithm for learning this equivalence class. Finally, we conduct synthetic, and semi-synthetic experiments to demonstrate that the k -PC algorithm enables more robust causal discovery in the small sample regime compared to the baseline algorithms.

1 Introduction

Causal reasoning is a critical tool for machine learning and artificial intelligence research with benefits ranging from domain adaptation to planning, explainability and fairness [27, 12, 26]. Estimating the effect of an action or an intervention from observational data is called causal inference. A very rich literature of causal inference algorithms have been developed to address this task in the literature [11, 21, 17, 1]. The function that is used to write an interventional distribution in terms of the observational distribution is called the estimand. Estimand depends on the causal relations between the system variables, which are represented in the form of a directed acyclic graph (DAG) called the causal graph. Thus, causal graph is required for solving most causal inference problems.

For small and well-studied systems, it might be possible to construct a causal graph using expert knowledge. However, in modern complex systems with changing structure, we need to learn causal graphs from data. This is called causal discovery. In most domains, we have access to plenty of observational data, but no interventional data. An important task then is to understand how much causal knowledge we can extract from observational data about a system.

The classical approach for addressing this problem is to use the conditional independence (CI) relations in the data to narrow down the space of plausible causal graphs [19, 9]. These are called constraint-based methods. Even though the number of causal graphs that entail the same set of

conditional independence relations are typically exponentially many, we can use a graphical characterization of equivalence between causal graphs to compactly represent all of them using a single mixed graph called the essential graph . This notion of equivalence is called Markov equivalence.

Even though such causal discovery algorithms are consistent, i.e., they output the correct essential graph in the large sample limit, in practice, they struggle due to finite number of samples since not all CI tests can be performed accurately [16]. For constraint-based algorithms, however, it is important for every test to be accurate since previously learned edge orientations are used to orient more edges due to their sequential nature. Thus, they may output very different causal graphs compared to the ground truth with just a few incorrect CI statements. Despite the efforts to stabilize PC output [2, 5], this fundamental issue still causes problems today. Furthermore, the existing Markov equivalence class characterization and causal discovery algorithms that rely on this characterization, such as the PC/IC algorithms [19, 22], require access to every CI relation, which is a significant practical limitation for causal discovery from data.

There are several alternatives to constraint-based causal discovery. For example, score-based approaches, such as GES [3, 4] optimize a regularized score function by greedily searching over the space of DAGs to output a graph within the Markov equivalence class. A line of works, such as NOTEARS [28] converts the graph learning problem to a continuous optimization problem by converting the acyclicity constraint to a continuous constraint via trace formulation. Note that our goal in this paper is not to beat the state-of-the-art causal discovery algorithm, but provide a theoretical basis to characterize what is learnable on a fundamental level by using conditional independence tests with restricted cardinality conditioning sets.

Variations of the causal discovery problem with limited-size conditioning sets have been considered in the literature. The special case of marginal dependence is considered in [13] and [20]. The most related existing work is [23], where the authors aim learning the graph union of all causal graphs that are consistent with a set of conditional independence statements up to a fixed cardinality conditioning set. They propose a concise algorithm that modifies the steps of PC which they show recovers the union of all equivalent graphs. Similarly in the case with latent variables, AnytimeFCI [18] shows that one can stop FCI algorithm after exhausting all conditional independence tests up to a fixed cardinality conditioning set and the output of the algorithm is still sound for learning parts of the partial ancestral graph (PAG). We propose an alternative route: First, we formally define the equivalence class of causal graphs and propose a graphical object to capture this equivalence class called the k -closure graphs. We identify a necessary and sufficient graphical condition to test equivalence between k -closure graphs. Finally, we develop a learning algorithm that leverages the representative power of partial ancestral graphs (PAGs), which are typically used in the case with latents, to obtain a provably finer characterization of the set of equivalent causal structures than [23].

In this paper, we propose learning causal graphs using CI tests with bounded conditioning set size. This allows us to ignore CI tests that become unreliable with limited data, and avoid some of the mistakes made by constraint-based causal discovery algorithms. We call CI constraints where the conditioning set size is upper bounded by k as the degreek CI constraints. We call two causal graphs k -Markov equivalent if they entail the same degreek CI constraints. We propose k -closure graphs that entail the same degreek CI relations as the causal graph, and show that we can characterize k -Markov equivalence via Markov equivalence between k -closure graphs. We then propose a constraint-based learning algorithm for learning this equivalence class from data, represented by the so-called k -essential graph. Finally, we demonstrate that our algorithm can help alleviate the finite-sample issues faced by the PC algorithm, especially while orienting arrowhead marks and for correctly learning the adjacencies in the graph. We also compare our algorithm with NOTEARS [28] and LOCI [23].

2 Background

In this section, we give some basic graph terminology as well as background on constraint-based causal discovery: An ( x ) represents the set of ancestors of x in a given graph that will be clear from context. De ( x ) similarly represents the descendants of x . Ne ( x ) represents the neighbors of x , i.e., the nodes that are adjacent to x .

Definition 2.1. A path in a causal graph is a sequence of nodes where every pair of consecutive nodes are adjacent in the graph and no node appears more than once.

Definition 2.2 (Skeleton) . The skeleton of a causal graph D is the undirected graph obtained by making every adjacent pair connected via an undirected edge.

Definition 2.3 ((Unshielded) Collider) . A path of three nodes ( a, c, b ) is a collider if the edges adjacent to c along the path are into c . A collider is called an unshielded collider if in addition the endpoints of the path a, b are non-adjacent.

Definition 2.4 (d-connecting path) . A path p is called d-connecting given a set c iff every collider along p is an ancestor of some node in c , and every non-collider is not in c .

Definition 2.5 (d-separation) . Two nodes a, b on a causal graph D are said to be d-separated given a set c of nodes iff there is no d-connecting path between a, b given c , shown by ( a ⊥ ⊥ b | c ) D , or a ⊥ ⊥ b | c if clear from context.

Definition 2.6 (Causal Markov assumption) . A distribution p is called Markov relative to a graph D =( V, E ) , if each variable is independent from its non-descendants conditioned on its parents in D .

There are local, global and pairwise Markov conditions that can all be shown as a consequence of the Causal Markov condition above [10]. This gives us the following:

Proposition 2.7 ([10]) . Let p be any joint distribution between varibles on a causal model with the graph D . If ( a ⊥ ⊥ b | c ) D then ( a ⊥ ⊥ b | c ) p , i.e., d-separation implies conditional independence under the Causal Markov condition.

Definition 2.8 (Markov equivalence) . Two causal graphs D 1 , D 2 are called Markov equivalent, shown by D 1 ∼ D 2 , if they entail the same d-separation constraints.

Definition 2.9 (Causal Faithfulness assumption) . A distribution p is called faithful to a causal graph D = ( V, E ) iff the following holds ∀ a, b ∈ V, c ⊂ V : ( a ⊥ ⊥ b | c ) p ⇒ ( a ⊥ ⊥ b | c ) D .

Constraint-based causal discovery is not possible without some form of faithfulness assumption, since otherwise CI statements do not inform us of the graph structure. We will later see that restricting ourselves to a small set of CI tests allow us to also weaken the faithfulness assumption we need for our causal discovery algorithm.

Theorem 2.10 ([22]) . Two DAGs are Markov equivalent if and only if they have the same skeleton and the same unshielded colliders.

Definition 2.11 (Degreek d-separations) . The collection of d-separation statements entailed by a DAG where the conditioning set has size of at most k is called degreek d-separation statements.

3 Markov Equivalence of Causal Graphs with Small Conditioning Sets

We are interested in characterizing the collection of causal graphs that entail the same d-separation constraints for all conditioning sets of size up to k for some k ∈ N . Wecall this k -Markov equivalence:

Definition 3.1. Two DAGs D 1 , D 2 are called k -Markov equivalent, shown by D 1 ∼ k D 2 if they entail the same d-separation constraints a ⊥ ⊥ b | c for all c ⊂ V : | c | ≤ k .

The notion of k -Markov equivalence partitions the set of causal graphs into equivalence classes:

Definition 3.2. k -Markov equivalence class of a DAG D is defined as the set of causal graphs that are k -Markov equivalent to D , shown by [ D ] k , i.e., D ′ ∼ k D, ∀ D ′ ∈ [ D ] k .

We first discuss how Verma and Pearl's characterization fails when we cannot test all d-separation statements. Note that if two DAGs have the same skeleton and unshielded colliders, then they entail the same d-separation constraints by Theorem 2.10. Therefore, they also entail the same degreek d-separation relations. However, the other direction is not true: Two graphs with different unshielded colliders or different skeletons may still entail the same degreek d-separation relations. This is expected since we are checking less constraints, which increases the size of the equivalence class: More and more graphs become indistinguishable as we do not rely on certain d-separation constraints.

For example, consider the causal graphs in Figure 1 (a, b). In D 1 , even though a, b are non-adjacent, they cannot be made conditionally independent unless both c and d are conditioned on. Similarly in D 2 , c, b cannot be made independent unless both a, d are conditioned on. In both graphs, a, d are independent, and become dependent conditioned on c or b . Therefore, D 1 , D 2 are k -Markov equivalent for k = 1 : They entail the same conditional independence relations for conditioning sets

a

c

b

d

(a)

D

1

a

c

b

d

(b)

D

2

a

c

b

d

f

e

(c)

D

3

a

c

b

d

f

e

(d)

D

4

of size up to 1 . Similarly in Figure 1 (c, d), the flipped edge between c, b induces different unshielded colliders in D 1 and D 2 . However, the endpoints of these colliders ( f, c ) , ( d, c ) and ( e, b ) , ( a, b ) are all dependent conditioned on subsets of size at most 1 . Then it is easy to verify that D 1 , D 2 entail the same degreek d-separation relations for k = 1 despite having different unshielded colliders.

These examples show that different structures may induce the same degreek d-separation relations. One might think that, if a collider actually changes the k -Markov equivalence class, then perhaps there is hope that a local characterization around all such equivalence class-changing colliders might be possible. However, this is not true. Our example in Section D.1 shows that a local characterization similar to Theorem 2.10 is not possible for k -Markov equivalence.

3.1 k -closure Graphs

Our goal is to come up with a graphical representation of k -Markov equivalence class that captures the invariant causal information across equivalent DAGs. This will later be useful for learning identifiable parts of the causal structure from data by CI tests. First, we introduce the notion of a k -covered pair.

Definition 3.3. Given a DAG D = ( V, E ) and an integer k , a pair of nodes a, b are said to be k -covered if ∄ c ⊂ V : | c | ≤ k and a ⊥ ⊥ b | c .

Therefore, k -covered pairs are pairs of variables that cannot be made independent by conditioning on subsets of size at most k . For example, a, b in Figure 1 are k -covered for k = 1 . In any graphical representation of k -Markov equivalence class, k -covered pairs should be adjacent. This is because it is not always possible to distinguish if they are adjacent or not by degreek d-separation relations.

We propose k -closure graphs as a useful representation to characterize k -Markov equivalence class.

Definition 3.4. Given a DAG D = ( V, E ) and an integer k , the k -closure of D is defined as the graph shown by C k ( D ) that satisfies the following:

If: a, b are k -covered in D i ) if a ∈ An ( b ) , then a → b in C k ( D ) , ii ) if b ∈ An ( a ) , then a ← b in C k ( D ) , iii ) else a ↔ b in C k ( D )

Else: a, b are non-adjacent in C k ( D )

The definition of k -closure trivially implies the following:

Lemma 3.5. Given a DAG D and an integer k , the k-closure graph C k ( D ) is unique.

Furthermore, there is a straightforward algorithm one can employ to recover a k -closure from the DAG: Make the non-adjacent pairs that cannot be d-separated with conditioning sets of size at most k adjacent according to the definition. Even though this may not be a poly-time operation, it is unavoidable to characterize the k -Markov equivalence class.

The following lemma shows that k -closure graphs can be used to capture all the conditional independence statements with bounded-size conditioning sets imposed by a DAG.

Lemma 3.6. k -closure C k ( D ) of a DAG D entails the same degree-k d-separation statements as the DAG, i.e., ( a ⊥ ⊥ b | c ) D ⇐⇒ ( a ⊥ ⊥ b | c ) C k ( D ) , ∀ c ⊂ V : | c | ≤ k.

Proof Sketch. The proof relies on two crucial lemmas. If a ↔ b in C k ( D ) , then in D , conditioned on any subset of size at most k , there is a d-connecting path with arrowheads on both a and b . Similarly, if a → b in C k ( D ) , then in D , conditioned on any subset of size at most k there is a d-connecting path with an arrowhead at b . Using these, we can show that any d-connecting path in C k ( D ) implies a d-connecting path in D . The other direction is straightforward as the d-connection statements in D hold in C k ( D ) since it is obtained from D by adding edges. Please see Section A.1 for the proof.

Inspired by ancestral graphs [15], k -closure graphs are mixed graphs with directed and bidirected edges, where k -covered pairs in the DAG are made adjacent in C k ( D ) based on their ancestrality.

Definition 3.7. A mixed graph is a graph that contains directed edges a → b, a ← b or bidirected edges a ↔ b where every pair of nodes is connected by at most one edge.

Due to bidirected edges, k -closure graphs are not DAGs. However, we can show that they are still acyclic. In fact, it is worth noting that k -closure graphs are a special class of ancestral graphs [15].

Lemma 3.8. For any DAG D , the k -closure graph C k ( D ) is a maximal ancestral graph (MAG).

MAGs have been successfully applied for learning causal graphs with latent variables [15]. Similar to MAGs, k -closure graphs are simply graph objects to help us compactly represent the k -Markov equivalence class of causal DAGs, rather than expressing the underlying physical system directly. They do, however, represent ancestrality relations between variables by construction.

The relation between k -closure graphs and MAGs is not an if and only if relation. In a MAG, one can have a bidirected edge between any pair of nodes as long as it does not create a (almost) directed cycle. This is because they represent latent confounders and one might have latent confounders between any pair of observed nodes. However, bidirected edges in k -closure graphs represent k -covered pairs and cannot be added arbitrarily. Accordingly, there are MAGs that are not valid k -closure graphs. An example is given in Section D.2. We have the following characterization:

Theorem 3.9. A mixed graph K = ( V, E ) is a k -closure graph if and only if it is a maximal ancestral graph and for any bidirected edge a ↔ b ∈ E the following is true:

∄ c ⊂ V : | c | ≤ k, ( a ⊥ ⊥ b | c ) K ′ , where K ′ = ( V, E -{ a ↔ b } ) .

The Markov equivalence between MAGs has been characterized in the literature. This relies on not only skeletons and unshielded colliders, but also colliders on discriminating paths being identical.

Definition 3.10. A path p = ⟨ a, z 1 , . . . z m , u, Y, v ⟩ is called a discriminating path for u, Y, v if a, v are not adjacent and every vertex { z i } i , u are colliders on p and parents of v .

Theorem 3.11. [[15]] Two MAGs M 1 , M 2 are Markov equivalent if and only if i ) They have the same skeleton, ii ) They have the same unshielded colliders, iii ) For any node Y for which there is a discriminating path p , Y has the same collider status on p in M 1 , M 2 .

One important observation for our characterization is that Markov equivalence of k -closure graphs do not rely on discriminating paths unlike arbitrary ancestral graphs.

Lemma3.12. Suppose two k -closure graphs K 1 , K 2 have the same skeleton and unshielded colliders. Then along any discriminating path p for a node Y , Y has the same collider status in K 1 and K 2 .

The above lemma shows that discriminating paths, although may exist in k -closure graphs, do not alter the equivalence class by themselves. Hence, for the graphical characterization of the equivalence between k -closure graphs, we can drop the discriminating path condition.

Corollary 3.13. Two k -closure graphs K 1 , K 2 are Markov equivalent if and only if

They have the same skeleton and

They have the same unshielded colliders.

In the next section, we will prove a k -Markov equivalence characterization based on Lemma 3.6, which will later be useful for learning, since we can employ algorithms devised for learning MAGs.

a

c

b

d

(a)

D

1

a

c

b

d

(b)

C

k

(

D

1

)

a

c

b

d

(c)

D

2

a

c

b

d

(d)

C

k

(

D

2

)

a

c

b

d

(e)

ε

k

3.2 k -Markov Equivalence

Our main result in this section is the following theorem that characterizes k -Markov equivalence:

Theorem 3.14. Two DAGs D 1 , D 2 are k-Markov equivalent if and only if C k ( D 1 ) and C k ( D 2 ) are Markov equivalent.

Thus, k -Markov equivalence of two DAGs can be reduced to checking Markov equivalence of their k -closures, which can be checked locally using the equivalence condition in Corollary 3.13. Based on this result, it is clear that, just by using conditional independence tests, we can only hope to narrow down our search up to the equivalence class of k -closure graphs.

Note that when all the CI tests can be conducted, we can learn the arrowheads and tails that consistently appear in all Markov equivalent DAGs. By operating at the k -closure graph-level, we can attain a similar objective and hope to learn the invariant arrowheads and tails.

Definition 3.15 (Edge union) . For our purposes, we define the edge union operation as follows:

Definition 3.16 ( k -essential graph) . For any DAG D , the edge union of all k -closure graphs that are Markov equivalent to C k ( D ) is called the k -essential graph 1 of D , shown by ε k ( D ) .

For example, among Markov equivalent k -closures, if a, b are adjacent only as a → b the k -essential graph will have the edge a → b . Thus, the k -essential graph will preserve the invariant arrowhead and tail marks. The difference between a -b and ao -o b is significant from a causal perspective: In the former, two variables cause each other. In the latter, there is some k -Markov equivalent DAG where a, b do not cause each other and are simply not separable by conditioning sets of size at most k .

For any edge type proposed in the edge union definition, there are relevant instances of k -essential graphs. In Figure 2, the edges a -b , ao → c , c ← b appear in the k -essential graph. Similarly, the k -essential graph of D in Figure 8 will contain the edge c o -o d since c → d, c ← d, c ↔ d are all possible edges in different Markov equivalent k -closures. Figure 9 in Section D.3 shows an instance where ↔ appears in the k -essential graph. This example also demonstrates that our representation is strictly richer than the graph union that is recovered by LOCI [23], which orients d → c, a → c , and hence cannot distinguish them from the possible edges between c ← b unlike our representation. Similarly, consider the simple graph u → v with k = 0 . Our representation yields u -v since there is no DAG where u, v are confounded in any way other than the direct edge. In a triangle graph w → u, w → v, u → v , our representation recovers uo -ov which shows there are graphs where u, v do not cause each other. LOCI cannot make such distinction as it uses u -v in both graphs.

Therefore, the variant edges that can be arrowheads or tails among different k -Markov equivalent graphs are replaced with circles and invariant arrowheads and invariant tails are preserved in the k -essential graph. Thus, k -essential graph captures the causal information that is preserved across all k -Markov equivalent causal graphs. Different from essential graphs where we can test all the

Algorithm 1 k -PC Algorithm

Input: Observational Data, V , k , CI Tester ( . ⊥ ⊥ . | . ) .

Step 0: Initiate a complete graph K between V with circle edges o -o .

Step 1: Find separating sets S a,b for every pair a, b ∈ V by conditioning on subsets of size at most k .

Step 2: Update K by removing the edges between pairs that are separable.

Step 3: Orient unshielded colliders of K : For any induced subgraph ao -o c o -o b , set ao → c ← o b for any non-adjacent pair a, b where S a,b does not contain c .

Step 4: K ← FCI_Orient ( K ) # See Algorithm 2 in Section B .

Step 5: For any node a that has no incoming edges (i.e., a ← , a ↔ , a ← o ) construct the sets B , C :

and define sets B ∗ as the set of nodes that are non-adjacent to any of the nodes in C and C ∗ as the set of nodes that are non-adjacent to other nodes in C :

̸

B ∗ = { b ∈ B : b, c are non-adjacent ∀ c ∈ C } , C ∗ = { c ′ ∈ C : c ′ , c are non-adjacent ∀ c ′ = c, c ′ ∈ C}

R 11 : Orient ao → b as a → b, ∀ b ∈ B ∗ .

R 12 : Orient ao -oc as a -c, ∀ c ∈ C ∗ .

Output: K

conditional independence relations, existence of an arrow a → b in a k -essential graph does not mean a causes b in every k -Markov equivalent DAG. Rather, it means that in any k -Markov equivalent DAG where a, b are adjacent, a causes b .

It is worth noting that k -essential graphs are in general more informative than partial ancestral graphs (PAGs), which are defined as the edge union of all Markov equivalent MAGs, where the union of ← , → is defined to give o -o instead of the undirected edge. Since every Markov equivalent k -closure graph is a MAG but not every Markov equivalent MAG is a k -closure graph, in general k -essential graphs may have invariant arrowheads and tails where PAGs only have circles. We call any graph that contains arrowheads, tails, or circles as edge marks a partial mixed graph (PMG).

Definition 3.17. For two partial mixed graphs A,B with the same skeleton, A is said to be a subset of B , shown by A ⊆ B , iff the following conditions hold for any pair a, b

Note that asterisk ∗ stands for a wild-card which can either be an arrowhead, tail, or circle. According to the definition, any circle mark in A is also a circle mark in B . We have the following lemma that relates k -essential graphs to partial ancestral graphs of k -closures.

Lemma 3.18. ε k ( D ) ⊆ PAG ( C k ( D )) .

Proof. By Theorem 3.9, every k -closure graph is a MAG whereas every MAG is not a k -closure graph. Thus the set of Markov equivalent k -closure graphs form a subset of the set of Markov equivalent MAGs. Thus, the arrowheads and tails that appear in all Markov equivalent MAGs must also appear in all the Markov equivalent k -closure graphs. The result follows from Definition 3.17.

In the next section, we propose a sound algorithm for learning k -essential graphs from data.

4 Learning k -essential Graphs

Constraint-based causal discovery algorithms use CI tests to extract all the causal knowledge that can be identified from data. In the previous section, we proposed a compact graphical characterization of what is learnable from such statistical tests. In this section, we propose a constraint-based learning algorithm. Since k -closure graphs are a special class of maximal ancestral graphs, we can use FCI algorithm that is devised for learning the invariant arrowheads and tails of a maximal ancestral graph.

FCI algorithm is sound and complete for learning PAGs: It can recover all invariant arrowheads and all invariant tails. One might think that FCI algorithm may also be sound and complete for our task. However, this is not true. Although sound, FCI is not complete for learning k -essential graphs. 2

For soundness, we need the following lemma, which shows that discriminating paths do not carry extra information about the underlying causal structure.

Lemma 4.1. In any k -closure graph, if there is a discriminating path p for ⟨ u, Y, v ⟩ and u ↔ Y ←∗ v is a collider along p , then the orientations u ∗→ Y and Y ←∗ v can be learned by first finding all unshielded colliders, and then applying the orientation rules R 1 and R 2 of FCI .

The above lemma shows that the colliders that are part of discriminating paths can be learned by simply orienting the unshielded colliders and then applying the orientation rules of FCI. This is useful since we do not need to search for colliders along discriminating paths during learning.

Our constraint-based causal discovery algorithm, called k -PC , is given in Algorithm 1 . It uses FCI Orient algorithm in Section B, with two extra rules specific for learning k -essential graph.

Corollary 4.2. k -PC without Step 5 is sound and complete for learning PAG ( C k ( D )) of any DAG D .

Proof. By Lemma A.18, any non-adjacent pair are separable by a set of size at most k . Thus, a valid separating set for any non-adjacent pair will be found in Step 1 , and will be used to learn the skeleton in Step 2 , and to orient all unshielded colliders in Step 3 . [25] proved arrowhead and tail completeness of FCI with orientation rules R 1 to R 10 . By Lemma 4.1, colliders on discriminating paths will be oriented at the end of Step 4 . Thus, R 4 that is concerned with discriminating path colliders is not applicable. Similarly, R 5 , R 6 , R 7 are only applicable in graphs with selection bias. Thus, they are not applicable. Since the rules that we omit are never applicable during the execution of the algorithm, Step 4 correctly returns the PAG ( C k ( D )) since the algorithm at that point is identical to the FCI algorithm for learning PAGs.

Definition 4.3. A distribution p is said to be k -faithful to a causal graph D = ( V, E ) , iff ( a ⊥ ⊥ b | c ) p implies ( a ⊥ ⊥ b | c ) D for all c ⊂ V : | c | ≤ k .

Theorem 4.4. k -PC algorithm is sound for learning k -essential graph given a conditional independence oracle under the causal Markov and k -faithfulness assumptions, i.e., if k -PC returns K , we have ε k ( D ) ⊆ K ⊆ PAG ( C k ( D ))

Proof Sketch. From Corollary 4.2, K obtained at the end of Step 4 is PAG ( C k ( D )) and by Lemma 3.18 we know ε k ( D ) ⊆ PAG ( C k ( D )) , i.e., every edge and tail orientation of K is consistent with ε k ( D ) . Thus, we only need to show that orientation rules R 11 , R 12 are sound, i.e., there exists no k -closure graph in the Markov equivalence class that is inconsistent with these orientation rules. By Lemma A.2, we know that if a ↔ x for some x , then conditioned on any subset of size k , there exists a d-connecting path that starts with arrow at a and x . This means that, irrespective of k , a must have some incoming edge. The rules use this fact together with the fact that if there were two incoming edges from two non-adjacent neighbors, this would create an unshielded collider, which would change the Markov equivalence class. Please see Section A.8 for the proof.

For two sample runs of the algorithm, please see Figure 5 and Figure 6 in Section C of Appendix. Note also that our algorithm can be seen as an improved version of AnytimeFCI algorithm [18] for causally sufficient systems, which the author shows can be stopped once every conditioning set of size at most k are tested. While FCI aims at learning arbitrary PAGs partially, k -PC learns k -closure graphs for causally sufficient systems. This allows extracting more causal information, evidenced by the additional orientation rules employed by k -PC . For more details on the differences and an example where AnytimeFCI is less informative than k -PC , please see Section D.7.

Computational Complexity. Suppose we are given two causal graphs. Our characterization gives an algorithmic way of testing k -Markov equivalence: Construct the k -closure graphs given the two DAGs and check whether they are Markov equivalent. The step to make two variables adjacent requires one to loop through all conditioning sets of size at most k . This would take O ( n k ) . This is the main time-consuming step. Afterward, one can test Markov equivalence using the existing

0.0

0.2

0.4

0.6

0.2

0.4

0.6

0.8

1.0

CDF of

F ar

1

, N=50

PC

kPC,k=2

(a) Arrowhead

F

1

Score

0.0

0.2

0.4

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F ta

1

, N=50

PC

kPC,k=2

(b) Tail

F

1

Score

0.2

0.4

0.6

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F sk

1

, N=50

PC

kPC,k=2

F

1

Score

0.0

0.2

0.4

0.2

0.4

0.6

0.8

1.0

CDF of

F ar

1

, N=100

PC

kPC,k=2

(d) Arrowhead

F

1

Score

0.0

0.2

0.4

0.6

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F ta

1

, N=100

PC

kPC,k=2

(e) Tail

F

1

Score

0.4

0.6

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F sk

1

, N=100

PC

kPC,k=2

F

1

Score

approaches. For example, [7] show that this can be done in O ( ne 2 + n 2 e ) time, where e is the number of edges. Thus, the overall algorithm will indeed be polynomial-time when k = O (1) .

The complexity of the learning algorithm, k -PC , will be similar to Anytime FCI, an early-stopped version of FCI [18]. Although this is more complicated and would depend on other parameters in the graph, such as primitive inducing paths, and thus the number and location of unobserved confounders, we can also roughly bound this by O ( n k +2 ) since for any pair, we will not be searching for separating sets beyond the O ( n k ) subsets of size at most k . Further runtime improvements, such as RFCI by [6] might be possible, but this requires a further understanding of the structure of k -closure graphs.

5 Experiments

5.1 Synthetic Data

In this section, we test our algorithm against the stable version of PC algorithm in causal-learn package. We randomly sample DAGs using the pyAgrum package. Variation of this experiment with a different sparsity level can be found in Section E.3 and gives similar results. All variables are binary with conditional probability tables filled randomly uniformly from the corresponding dimensional probability simplex, except the linear SCM experiments for comparing with NOTEARS. We observed similar results with a larger number of states, which are not reported. We report the F 1 scores for identifying arrowheads, tails and skeleton correctly. The results are given in Figure 3. We observe that k -PC provides significant improvement to the arrowhead F 1 score in the small-sample regime at little cost to the tail score. k -PC also helps improve skeleton discovery. Similar improvements are seen even for 10 samples whereas the advantage disappears for more than 500 samples (see Section E.3) since PC becomes reliable enough and k -PC does not use some of the informative CI tests in that high-sample regime. We report combined metrics such as the sum of all F 1 scores in Section E.4.

In Section E.5, we compare with the conservative version of PC [14] and observe similar results. In Section E.2 we generate linear SCMs randomly and compare the performance of our algorithm with NOTEARS in addition to PC. Even though NOTEARS performs slightly better than PC in general, k -PC maintains an advantage over both in the small-sample regime. In Section E.1, we compare our algorithm with LOCI [23]. As expected, the two algorithms perform similarly. We observe that our algorithm shows better arrowhead and skeleton F 1 score performance. As a side note, we show that

X1

X2

X6

X3

X4

X5

X8

X7

X1

X2

X6

X7

X8

X3

X4

X5

(b)

k

-PC ,

k

= 0

X1

X2

X6

X3

X4

X5

X8

X7

(c)

k

-PC ,

k

= 1

X1

X2

X3

X4

X6

X5

X8

X7

(d) PC Alg.

k -PC output is at least as informative as LOCI in Section D.6. See Section D for further discussions. The Python code is provided at https://github.com/CausalML-Lab/kPC .

5.2 Semi-synthetic Data

We test our algorithm on the semi-synthetic Asia dataset from bnlearn repository and compare it with PC algorithm. The dataset contains 8 binary variables. We randomly sample 500 datapoints from the observational distribution and run PC as well as k -PC for k = 0 and k = 1 . k -PC for k = 0 correctly gets some of the causal and ancestral relations (Figure 4). PC, on the other hand, recovers a very sparse graph as it utilizes unreliable conditional independence tests with conditioning size of 2 . k -PC with k = 1 recovers a graph in-between the two, still more informative about the structure than PC.

6 Discussion

A limitation of the method is that it assumes all independence statements up to degree k can be tested for some k . In some cases, the set of available, or the set of reliable CI statements might not have such a structure. Our method is not directly applicable in such scenarios. Another limitation is that we assume that we can run CI tests for the tests that are deemed reliable. This is a non-trivial problem when the data is non IID, such as time-series data. We also make some other usual assumptions that are commonly made in causal discovery, such as acyclicity.

For more remarks, such as a demonstration of incompleteness of k -PC , please see Section D.

7 Conclusion

We proposed a new notion of equivalence between causal graphs called the k -Markov equivalence. This new equivalence allows us to learn parts of the causal structure from data without relying on CI tests with large conditioning sets that tend to be unreliable. We showed that our learning algorithm is sound, and demonstrated that it can help correct some errors induced by other algorithms in practice.

References

Elias Bareinboim and Judea Pearl. Causal inference and the data-fusion problem. Proceedings of the National Academy of Sciences , 113(27):7345-7352, 2016.

Facundo Bromberg and Dimitris Margaritis. Improving the reliability of causal discovery from small data sets using argumentation. Journal of Machine Learning Research , 10(2), 2009.

David Maxwell Chickering. Optimal structure identification with greedy search. Journal of machine learning research , 3(Nov):507-554, 2002.

Max Chickering. Statistically efficient greedy equivalence search. In Conference on Uncertainty in Artificial Intelligence , pages 241-249. PMLR, 2020.

Diego Colombo and Marloes H Maathuis. A modification of the pc algorithm yielding orderindependent skeletons. CoRR, abs/1211.3295 , 2012.

Diego Colombo, Marloes H Maathuis, Markus Kalisch, and Thomas S Richardson. Learning high-dimensional directed acyclic graphs with latent and selection variables. The Annals of Statistics , pages 294-321, 2012.

Zhongyi Hu and Robin Evans. Faster algorithms for markov equivalence. In Conference on Uncertainty in Artificial Intelligence , pages 739-748. PMLR, 2020.

Steffen Lauritzen and Kayvan Sadeghi. Unifying markov properties for graphical models. The Annals of Statistics , 46(5):2251-2278, 2018.

Steffen L Lauritzen. Graphical models , volume 17. Clarendon Press, 1996.

Judea Pearl. Probabilistic reasoning in intelligent systems: networks of plausible inference . Morgan kaufmann, 1988.

Judea Pearl. Causal diagrams for empirical research. Biometrika , pages 669-688, 1995.

Judea Pearl and James M Robins. Probabilistic evaluation of sequential plans from causal models with hidden variables. arXiv preprint arXiv:1302.4977 , 2013.

Judea Pearl and Nanny Wermuth. When can association graphs admit a causal interpretation. Selecting Models from Data: Artificial Intelligence and Statistics IV , 89:205-214, 1994.

Joseph Ramsey, Peter Spirtes, and Jiji Zhang. Adjacency-faithfulness and conservative causal inference. In Proceedings of the Twenty-Second Conference on Uncertainty in Artificial Intelligence , pages 401-408, 2006.

Thomas Richardson and Peter Spirtes. Ancestral graph markov models. The Annals of Statistics , 30(4):962-1030, 2002.

Rajen D. Shah and Jonas Peters. The hardness of conditional independence testing and the generalised covariance measure. Annals of Statistics , 48(3):1514-1538, 2020.

Ilya Shpitser and Judea Pearl. Complete identification methods for the causal hierarchy. Journal of Machine Learning Research , 9:1941-1979, 2008.

Peter Spirtes. An anytime algorithm for causal inference. In International Workshop on Artificial Intelligence and Statistics , pages 278-285. PMLR, 2001.

Peter Spirtes, Clark N Glymour, Richard Scheines, and David Heckerman. Causation, prediction, and search . MIT press, 2000.

Johannes Textor, Alexander Idelberger, and Maciej Li´ skiewicz. Learning from pairwise marginal independencies. In Proceedings of the Thirty-First Conference on Uncertainty in Artificial Intelligence , pages 882-891, 2015.

Jin Tian and Judea Pearl. On the testable implications of causal models with hidden variables. In Proceedings of the Eighteenth conference on Uncertainty in artificial intelligence , pages 519-527, 2002.

Tom S Verma and Judea Pearl. Equivalence of causal models. 1990.

Marcel Wienöbst and Maciej Liskiewicz. Recovering causal structures from low-order conditional independencies. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 34, pages 10302-10309, 2020.

Jiji Zhang. Causal reasoning with ancestral graphs. Journal of Machine Learning Research , 9:1437-1474, 2008.

Jiji Zhang. On the completeness of orientation rules for causal discovery in the presence of latent confounders and selection bias. Artificial Intelligence , 172(16-17):1873-1896, 2008.

Junzhe Zhang and Elias Bareinboim. Fairness in decision-making-the causal explanation formula. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 32, 2018.

Kun Zhang, Bernhard Schölkopf, Krikamol Muandet, and Zhikun Wang. Domain adaptation under target and conditional shift. In International Conference on Machine Learning , pages 819-827. PMLR, 2013.

Xun Zheng, Bryon Aragam, Pradeep K Ravikumar, and Eric P Xing. Dags with no tears: Continuous optimization for structure learning. Advances in neural information processing systems , 31, 2018.

Appendix

A Proofs

In this section, we provide proofs for the lemmas and theorems in the paper. We also present FCI orientation rules for completeness, demonstrate why k -PC is incomplete, and give two sample runs of the k -PC algorithm. Each subsection starts from a new page to clearly separate the proofs of different lemmas/theorems.

A.1 Proof of Lemma 3.6

We will crucially use the following three lemmas to prove our main results. We say a collider ⟨ u, v, w ⟩ is closed, or blocks the path in context if no node in De ( v ) is in the conditioning set. Similarly, a path is called closed if it is not d-connecting, and open otherwise.

Lemma A.1. Consider a DAG where X / ∈ An ( Y ) . Suppose there is a d-connecting path p between X,Y given T that starts with an arrow out of X .

There is at least one collider along p .

Let K be the collider closest to X on p . Then conditioning on T ′ = T -De ( K ) instead of T does not introduce new d-connecting paths that start with an arrowhead at X .

Proof. 1 . Any path that starts with X → . . . must either be directed, or there must be at least one collider along the path. Since the path is between X,Y and X / ∈ An ( Y ) , it must be that the path has at least one collider on it.

2 . First note that without loss of generality, p has the following form for some integer m ≥ 0 ( m = 0 means X → K ):

Suppose for the sake of contradiction that conditioned on T ′ there is a new d-connecting path q that starts with an arrowhead into X . q was clearly closed conditioned on T and become open by us removing nodes from the conditioning set T . This can only happen if we removed some node from T that is a non-collider along q . Consider the non-collider we removed that was closest to X , call this M . Thus, we have the path q that looks like this:

for some W , where M is a non-collider on this path and is in De ( K ) .

We observe that the subpath between M and X cannot be directed from M to X . Because this would create the following cycle:

Thus a closer look at the path q reveals the following structure for some integer m ′ and node V :

We will consider the following two cases: The edge adjacent to M along the subpath between W m ′ and M is a tail or an arrowhead.

Suppose the edge adjacent to M along the subpath between W m ′ and M is a tail: This means there is at least one collider between W m ′ and M . Any such collider must be active since q is active given T ′ . Consider the collider that is closest to M . Since it is active, this collider must be an ancestor of T ′ . However, observe that K is an ancestor of this collider which implies that K is an ancestor of T ′ as well. However, we obtained T ′ by removing all descendants of K from T , which is a contradiction.

This establishes that the edge adjacent to M along the subpath between W m ′ and M is an arrowhead. Thus, this reveals the following structure for q :

Suppose the directed path from K to M is as follows:

Recall that M is a non-collider along q . Thus the subpath of q between M and Y must start with a tail as M → . . . Y . Now observe that if the subpath of q between M and Y had no collider, then we would have the following directed path from X to Y :

However, we know X is a non-ancestor of Y . Thus, there must be at least one collider between M and Y along p , all of which are open given T ′ . Consider the collider that is closest to M . There is a directed path from M to this collider, and a directed path from this collider to a member of T ′ since it is open conditioned on T ′ . But this means there is a directed path from K to a member of T ′ since there is a directed path from K to this collider. This is a contradiction since we obtained T ′ by removing all descendants of K from T .

This establishes the claim that removing descendants of the collider along any d-connecting path that starts with a tail at X cannot introduce a d-connecting path that starts with an arrow into X when X / ∈ An ( Y ) .

Lemma A.2. Consider a DAG D where X / ∈ An ( Y ) and Y / ∈ An ( X ) , X,Y are non-adjacent and k-covered. Then conditioned on any subset T : | T | ≤ k , there exists a d-connecting path between X,Y that has an arrowhead at both X and Y .

Proof. For the sake of contradiction suppose, conditioned on c , there is no d-connecting path with an arrow into X and an arrow into Y . Since neither X is an ancestor of Y nor Y is an ancestor of X , it must be that all d-connecting paths have colliders on them. And all such colliders must be ancestors of T .

Consider such a path p where the edge adjacent to X has a tail at X . Let K be the collider that is closest to X .

Thus we have

for some { U i } i ∈ m , V . Since the path is open it must be that K ∈ An ( T ) . Let T ′ = T -De ( K ) , where De ( K ) are all descendants of K . Clearly, q is no longer open conditioned on T ′ . Weinvestigate other open paths now, keeping in mind that X,Y being k -covered implies that X ̸ ⊥ ⊥ Y | T ′ since | T ′ | ≤ | T | ≤ k .

Claim 1: Removing the descendants of the collider closest to X from the conditioning set can only add d-connecting paths that start with a tail at X but no d-connecting paths that start with an arrowhead at X .

Proof of Claim 1: Since X is not an ancestor of Y , by Lemma A.1, we know that removing the descendants of K from T can only introduce d-connecting paths that are out of X .

Now consider the d-connecting paths under conditioning on T ′ . We know that these paths must have a tail either at X or at Y since we started with such d-connecting paths by assumption and by Claim 1 , removing De ( K ) from T can only introduce new d-connecting paths that have tails at X . Using the fact that no path that has an arrowhead into both endpoints are opened, we can use recursion and claim that we can make X,Y d-separated by removing descendants of colliders (that are closest to the endpoint that is adjacent to a tail) of active paths, which gives the following:

Claim 2: There exists a set T ∗ of size at most k such that X ⊥ ⊥ Y | T ∗ , which leads to a contradiction since X,Y are k-covered by assumption.

Proof of Claim 2: Given claim 1, we can continue removing descendants of the colliders of the active paths that are closest to the tail-end node from the set T . Either no d-connecting path is left at some point in this process, or that we end up removing all the variables from the conditioning set. If the former, this is a contradiction since X,Y are k -covered . If the latter, then this is another contradiction due to the following: This means that given empty set, paths that have a tail adjacent to one of the endpoints, i.e., the paths with colliders on them (since all paths that have a tail adjacent to one of the endpoints must have a collider because X / ∈ An ( Y ) and Y / ∈ An ( X ) ) are d-connecting, which is not possible. This proves Claim 2.

Due to the symmetry between X,Y , the supposition that the only d-connecting paths must have a tail adjacent to either endpoint must be wrong, which proves the lemma.

Lemma A.3. Consider a DAG D where X / ∈ An ( Y ) , X,Y are non-adjacent and k-covered. Then conditioned on any subset T : | T | ≤ k , there exists a d-connecting path between X,Y that starts with an arrow into X .

Proof. For the sake of contradiction, suppose otherwise. Given T , all the d-connecting paths start with a tail at X . We will show that we can find some T ′ of size at most k that d-separates X,Y , which lead to a contradiction since X,Y are assumed to be k -covered .

Consider any path q that is d-connecting given T which starts with a tail at X . Since X / ∈ An ( Y ) , by Lemma A.1 it must be that this path has at least one collider on it. Let K be the collider that is closest to X . Thus we have

for some { U i } i , V . Since the path is open, this collider cannot be blocking the path q . It must be that K ∈ An ( T ) . Let T ′ = T -De ( K ) , where De ( K ) are all descendants of K . Clearly, q is no longer open conditioned on T ′ . We investigate other open paths now, keeping in mind that X,Y being k-covered implies that X ̸ ⊥ ⊥ Y | T ′ since | T ′ | ≤ | T | ≤ k .

Claim 1: Removing the descendants of the collider closest to X from the conditioning set can only add d-connecting paths that start with a tail at X but no d-connecting paths that start with an arrowhead at X .

Proof of Claim 1: Since X is not an ancestor of Y , by Lemma A.1 we know that removing the descendants of K from T can only introduce d-connecting paths that are out of X .

Now consider the d-connecting paths under conditioning on T ′ . We know that these paths must have a tail at X since we started with only such d-connecting paths by assumption and by Claim 1 , removing De ( K ) from T can only introduce d-connecting paths that have tails at X . Using the fact that no path that has an arrowhead into X , we can use recursion and claim that we can make X,Y d-separated by removing descendants of colliders (that are closest to X ) of active paths, which gives the following:

Claim 2: There exists a set T ∗ of size at most k such that X ⊥ ⊥ Y | T ∗ , which leads to a contradiction since X,Y are k-covered by assumption.

Proof of Claim 2: Given claim 1, we can continue removing descendants of the first colliders of active paths that are closest to X from the set T . Either, no d-connecting path is left at some point in this process or that we end up removing all the variables from the conditioning set. If the former, this is a contradiction since X,Y are k-covered. If the latter, then this is another contradiction due to the following: This means that given empty set, paths that start with a tail and have colliders on them (as they cannot be directed and collider-free since X is not an ancestor of Y ) are d-connecting, which is not possible. This proves Claim 2. .

Therefore, the supposition that all the d-connecting paths must have a tail adjacent to X must be wrong, which proves the lemma.

The above lemmas will be crucial in proving Lemma 3.6. Now consider a d-connecting path between x, z given c and a d-connecting path between z, y given c . We have the following lemma:

Lemma A.4. Let p be an active path between x, z given c , and q be an active path between z, y given c , where x, y, z / ∈ c . If x and y are d-separated given c , then

Paths p, q must have no overlapping nodes and

Y must be a collider along the concatenated path and Y / ∈ An ( c ) .

Proof. We would like to allow the possibility that these paths might go through the same nodes. To address this, it helps to consider walks.

Definition A.5. A walk on a DAG is any sequence of edges.

Definition A.6. A path on a DAG is a sequence of edges where each node appears at most once.

There is a direct relation between active walks and d-connecting paths [8]. Indeed, this relation is leveraged to efficiently check dependence using paths, rather than having to search over all walks, which is a much larger space.

Definition A.7. A walk between two nodes a, b is called active given c if each collider along the walk is in c and each non-collider is not in c .

Definition A.8. A path between two nodes a, b is called open given c if each collider along the path is in An ( c ) and each non-collider is not in c .

Consider an active walk where a node t appears multiple times. Observe that t must appear with the same collider status, since otherwise the walk would not be active. If the appearance is of the form

then there must be a collider that is in c between the two appearances of t 's. We can skip the intermediate subpath between the two appearances of t 's to obtain the walk

Since there is at least one collider that is in c along the skipped subpath, we have that t ∈ An ( c ) . Therefore, t will not be blocking the path that is obtained after repeatedly applying this and other shortening steps. If the appearances is of the form:

we can similarly skip the subpath between the two appearances of t 's to obtain the shorter walk

and repeat this process until t is not repeated. The resulting walk/path is still open since t will appear in the same collider status, namely as a non-collider and if it was not blocking the walk, it will not be blocking the path either. This argument holds for any configuration where t is a non-collider. If the appearances is of the form:

then it must be that t ∈ c , and thus the walk obtained by skipping the subwalk between the two appearances of t 's, i.e.,

must be d-connecting.

This shows that each active walk corresponds to a d-connecting path and vice verse. Now we can proceed with the proof of the lemma:

1. Suppose p, q have overlapping nodes. Let w p be the walk that corresponds to p and w q be the walk that corresponds to q . Consider the concatenated walk w = w p , w q . If any repeated node has different collider status along w , then the path is not active. But this means that that node was blocking either w p or w q , which would be a contradiction. Therefore, repeated nodes cannot have different collider status along w .

Suppose a node t is repeated in w p and w q and has the same collider status. In this case, consider the walk obtained by concatenating the sub-walk of w p between x and t , with the sub-walk of w q from t to y . By repeating this process for any repeated node, we can obtain a path that corresponds to this walk, which would always be active since the repeated nodes have the same collider status along this path that they had in w p or w q and were not blocking these walks. Therefore, they cannot block the concatenated path obtained this way either, which is a contradiction since we are given that x, y are d-separated given c . Therefore if any node is repeated in w p and w q then the concatenated walk is always active. Thus, it must be the case that there is no repeated nodes.

2. Since there is no repeated nodes from 1. , we can operate at the path level instead of considering walks. Suppose Y is not a collider. Since Y / ∈ c , it would be d-connecting and thus the concatenating path would be d-connecting, a contradiction. Suppose Y is a collider but Y ∈ An ( c ) . In this case, Y would not block the concatenated path either, which is a contradiction. This establishes the result.

The next lemma shows that colliders that are closed in D must remain closed in the k -closure C k ( D ) .

Lemma A.9. If a collider is blocked in D conditioned on some c : | c | ≤ k then it must also be blocked in C k ( D ) conditioned on c .

Proof. Suppose ( X → Z ← Y ) D is a collider that is blocked given c . Thus, it must be that Z / ∈ An ( c ) in D . For the sake of contradiction, suppose that this collider is unblocked in C k ( D ) . Thus, it must be the case that Z ∈ An ( c ) in C k ( D ) . This means there is a new directed path from Z to c in C k ( D ) . If this path existed in D , the collider would be unblocked, which is a contradiction. Thus, at least one of the edges along this path must have been added during the construction of C k ( D ) . Consider the collection of edges on this path that do not exist in D . Note that by construction of C k ( D ) , a directed edge α → β is added between a k -covered pair α, β only if there is a directed path from α to β . Consider the path obtained by replacing the directed edge between any k -covered pair along this path with the corresponding directed path in D . The resulting directed path must be in D . This shows that there was at least one path already in D that implied Z ∈ An ( c ) , which is a contradiction. Therefore, any collider in p that is unblocked in C k ( D ) must also be unblocked in D .

Lemma A.10. The set of ancestors of any set c of nodes in C k ( D ) are identical to the set of ancestors of c in C k ( D ) .

Proof. Adding edges to a graph, directed or bidirected, cannot decrease the set of ancestors of any node. We only need to show that the set of ancestors in C k ( D ) is not larger than the set of ancestors in D .

Suppose otherwise: A node a ∈ An ( c ) in C k ( D ) but a / ∈ An ( c ) in D . This can only happen if a collection of edges added during the construction of C k ( D ) render a an ancestor of c . However, each such edge is added only if there is a directed path between its endpoints in D . Consider the path obtained by replacing each such added edge along the path that renders a an ancestor of c in C k ( D ) with the corresponding directed paths in D . This directed path must be in D , which means that a was an ancestor of c in D as well, which is a contradiction.

We are finally ready for the proof of Lemma 3.6.

Proof of Lemma 3.6:

Since no edge is removed during the construction of the k -closure, one direction immediately follows: If a ⊥ ⊥ b | c in C k ( D ) , then a ⊥ ⊥ b | c in D . The implication is clearly true for any set c of size at most k as well. Therefore we only need to show the other direction.

Suppose a ⊥ ⊥ b | c in D where | c | ≤ k . We will show that a ⊥ ⊥ b | c in C k ( D ) . For the sake of contradiction, suppose otherwise. Then there must be a d-connecting path p between a, b given c in C k ( D ) . The length of any such path must be greater than 1 since otherwise, whether this edge already existed in D or it was added during the construction of C k ( D ) , a, b must have been dependent given c in D , which is a contradiction. Since the orientation of the existing edges in D did not change in C k ( D ) , either this path did not exist in D or that it existed but it was blocked by some collider that is not in An ( c ) in D . The latter is not possible due to Lemma A.9, since any unblocked collider in C k ( D ) must also be unblocked in D . Thus, it must be that this d-connecting path did not exist in D .

Suppose p does not exist in D . At least one edge must have been added to form this path in C k ( D ) during the construction of C k ( D ) .

For any added edge u → v , the following is true: Since u → v was added in C k ( D ) , it must be the case that v / ∈ An ( u ) since otherwise there would be a cycle. By Lemma A.3, conditioned on c , there exists a d-connecting path between u, v where the edge adjacent to v is into v . For any added edge u ↔ v , the following is true: Since u ↔ v was added in C k ( D ) , it must be the case that u / ∈ An ( v ) and v / ∈ An ( u ) . By Lemma A.2, conditioned on c , there exists a d-connecting path between u, v where the edge adjacent to u is into u and the edge adjacent to v is into v . Call any such path implied by these lemmas a replacement path . Note that a replacement path might be directed or not.

Consider a path q in D that is obtained from p by switching the edges added during the construction of C k ( D ) with the replacement paths using the following policy: Suppose u → v in C k ( D ) for some k -covered pair u, v . If a directed path is open given c in D , use that path as the replacement path

for the edge a → b . If not, use any other path. This means that either the path that replaces an edge u → v is directed or that both the endpoints have an arrowhead and that u ∈ An ( c ) .

Observe that each replacement path is d-connecting and the subpaths of p that remain intact in q must be d-connecting since p is d-connecting. By Lemma A.4, any two paths - whether it is a pair of replacement paths or a replacement path and a subpath of p - have overlapping nodes, then their concatenation must be d-connecting. Since we assumed that q was not d-connecting, it must be that one of the endpoints of one of the added edges must be blocking q . We investigate each such node to verify that q is indeed d-connecting to arrive at a contradiction.

In other words, the collider status of some of these nodes must have changed due to replacing some edges with replacement paths. Specifically due to Lemma A.4, one of the endpoints of replacement paths must be a collider and not an ancestor of c . Since ancestrality status cannot change from D to C k ( D ) due to Lemma A.10, the only way for q to not be d-connecting is if some node that is not an ancestor of c changes status from being a non-collider along p to being a collider along q .

Note that for an edge u ↔ v , the nodes u and v are adjacent to an arrowhead in the replacement path. Thus, if some node t changes collider status in q compared to p , it cannot be due to bidirected edges along p .

Now consider the directed edges u → v . If the replacement path is directed from u to v , similarly u is adjacent to a tail and v is adjacent to an arrowhead on the replacement path. Therefore, such edges cannot alter the collider status of nodes at the junction of different paths. Finally consider the directed edges u → v where u and v are both adjacent to an arrowhead on the replacement path. Observe that this edge cannot change the collider status of v . We now focus on u . If the other edge adjacent to u along q is a tail, u remains a non-collider and cannot block q . Now suppose the other edge adjacent to u along q is an arrowhead. This makes u a collider in q whereas u was a non-collider along p since we had u → v along p . However, by construction of q , as we ended up adding a path with arrowheads at both endpoints, it must be that the directed path between u, v (which exists since u → v was added during the construction of C k ( D ) ) must be blocked via conditioning. This means u ∈ An ( c ) which means that although the status of u changes from non-collider to collider, it must be that this collider does not block q since it is an ancestor of c . Therefore, no replacement path can alter the status of a node to block the path q , and q must be d-connecting, which contradicts with the assumption that the path was not d-connecting in D .

This establishes that any d-connecting path in C k ( D ) is also d-connecting in D , which establishes that if a ̸ ⊥ ⊥ b | c in C k ( D ) , then a ̸ ⊥ ⊥ b | c in D . This establishes the lemma.

A.2 Proof of Lemma 3.8

For a mixed graph to be a maximal ancestral graph, we need to show that it does not have directed or almost directed cycles and that any non-adjacent pair of nodes can be made conditionally independent by conditioning on some subset of observed variables [24]. We first define almost directed cycle, and propose a lemma that shows that k -closure graphs do not have directed or almost directed cycles.

Definition A.11 ([24]) . A directed path p from a to b and the edge a ↔ b is called an almost directed cycle.

Lemma A.12. For any DAG D , and integer k , C k ( D ) does not have directed or almost directed cycles.

Proof. Suppose, for the sake of contradiction that there is a directed cycle in C k ( D ) . Since each edge X → Y in C k ( D ) either exists in D or for each such edge in C k ( D ) , there is a directed path from X to Y in D , existence of a directed cycle in C k ( D ) would imply a directed cycle in D , which contradicts with the DAG assumption of D .

Suppose, for the sake of contradiction that there is an almost directed cycle in C k ( D ) , i.e., we have a directed path from a to b for two nodes a ↔ b . Since a ↔ b is added during construction of C k ( D ) , it must be the case that neither a nor b are ancestors of each other. However, from the above argument, there must be a directed path from a to b in D , which is a contradiction. Thus, C k ( D ) cannot have almost directed cycles.

The other condition for a mixed graph to be a maximal ancestral graph is that for any non-adjacent pair of nodes, there exists a subset of the observed variables that make them conditionally independent. For the k -closure graphs, this simply follows by construction: Any pair of nodes that are non-adjacent in C k ( D ) can be made conditionally independent given some set of size at most k in D by construction of C k ( D ) . From Lemma 3.6, this conditional independence relation must be retained in C k ( D ) . Thus any non-adjacent pair of nodes in C k ( D ) can be d-separated in C k ( D ) by some conditioning set of size at most k . This establishes the claim.

A.3 Proof of Theorem 3.9

Our main observation is that a parallel of Lemma A.2 works for MAGs with k -covered bidirected edges. The following lemmas are for any mixed graph K that satisfies the constraints in Theorem 3.9, i.e., those that are MAGs and that satisfy the condition that for any bidirected edge a ↔ b , a, b are k -covered in the graph K -( a ↔ b ) .

Lemma A.13. Suppose X / ∈ An ( Y ) . Suppose there is a d-connecting path p between X,Y given T that starts with an arrow out of X .

There is at least one collider along p .

Let K be the collider closest to X on p . Then conditioning on T ′ = T -De ( K ) instead of T does not introduce new d-connecting paths that start with an arrowhead at X .

Proof. 1 . Any path that starts with X → . . . must either be directed, or there must be at least one collider along the path. Since the path is between X,Y and X / ∈ An ( Y ) , it must be that the path has at least one collider on it.

2 . First note that without loss of generality, p has the following form for some integer m ≥ 0 ( m = 0 means X → K ):

∗ is a wildcard representing either an arrowhead or a tail.

Suppose for the sake of contradiction that conditioned on T ′ , there is a new d-connecting path q that starts with an arrowhead into X . q was clearly closed conditioned on T and became open by us removing nodes from the conditioning set T . This can only happen if we removed some node from T that is a non-collider along q . Consider the non-collider we removed that was closest to X , call this M . Thus we have the path q that looks like this:

where M is a non-collider on this path and is in De ( K ) .

We observe that the subpath between M and X cannot be directed from M to X . Because this would create the following cycle, since K is assumed to be the first collider along p , and an ancestor of M .

Thus a closer look at the path q reveals the following structure for some integer m ′ and node V :

We will consider the following two cases: The edge mark adjacent to M along the subpath between W m ′ and M is a tail or an arrowhead.

Suppose the edge mark adjacent to M along the subpath between W m ′ and M is a tail: This means there is at least one collider between W m ′ and M . Any such collider must be active since q is active given T ′ . Consider the collider that is closest to M . Since it is active, this collider must be an ancestor of T ′ . However, observe that K is an ancestor of this collider which implies that K is an ancestor of T ′ as well. However, we obtained T ′ by removing all descendants of K from T , which is a contradiction.

This establishes that the edge mark adjacent to M along the subpath between W m ′ and M is an arrowhead. Thus, this reveals the following structure for q :

Suppose the directed path from K to M is as follows for some { θ i } i for some integer m ′′ :

Recall that M is a non-collider along q . Thus, the subpath of q between M and Y must start with a tail as M → . . . Y . Now observe that if the subpath of p between M and Y had no collider, then we would have the following directed path from X to Y :

However, we know X is a non-ancestor of Y . Thus, there must be at least one collider between M and Y along p , all of which are open given T ′ . Consider the collider that is closest to M . There is a directed path from M to this collider, and a directed path from this collider to a member of T ′ . But this means there is a directed path from K to a member of T ′ since there is a directed path from K to this collider. This is a contradiction since we obtained T ′ by removing all descendants of K from T.

This establishes the claim that removing descendants of the collider along any d-connecting path that starts with a tail at X cannot introduce a d-connecting path that starts with an arrow into X when X / ∈ An ( Y ) .

The following is the parallel lemma to Lemma A.2 for any mixed graph K that satisfies the conditions of Theorem 3.9.

Lemma A.14. Consider a bidirected edge X ↔ Y in K . Suppose conditioned on any subset T : | T | ≤ k , X ̸⊥ ⊥ Y | T in G -( X ↔ Y ) . Then conditioned on any T : | T | ≤ k , there exists a d-connecting path between X,Y that starts with an arrow into X and an arrow into Y .

Proof. For the sake of contradiction suppose, conditioned on some T : | T | ≤ k , there is no dconnecting path with an arrow into X and an arrow into Y . Since neither X is an ancestor of Y nor Y is an ancestor of X , all d-connecting paths must have colliders on them. And all such colliders must be ancestors of T .

Consider such a path p where, without loss of generality, the edge adjacent to X has a tail at X . Let K be the collider that is closest to X .

Thus we have

for some { U i } i , V and integer m . Since the path is open, this collider must be unblocked. It must be that K ∈ An ( T ) . Let T ′ = T -De ( K ) , where De ( K ) are all descendants of K . Clearly, p is no longer open. We investigate other open paths now, keeping in mind that X,Y are dependent given T ′ since | T ′ | ≤ k .

Claim 1: Removing the descendants of the collider closest to X from the conditioning set can only add d-connecting paths that start with a tail at X but no d-connecting path that starts with an arrowhead at X .

Proof of Claim 1: Since a bidirected edge exists between X,Y , and that K is a MAG, neither X nor Y are ancestors of one another, since then we would have an almost directed cycle. By Lemma A.13, we know that removing the descendants of K from T can only introduce d-connecting paths that are out of X .

Now consider the d-connecting paths under conditioning on T ′ . We know that these paths must have a tail either at X or at Y . Using the above claim that no path that has an arrowhead into both endpoints are opened, we can use recursion and claim that we can make X,Y d-separated by removing descendants of colliders (that are closest to the endpoint that is adjacent to a tail) of active paths, which gives the following:

Claim 2: There exists a set T ∗ of size at most k such that X ⊥ ⊥ Y | T ∗ , which leads to a contradiction since X,Y cannot be made independent by conditioning on sets of size at most k by the assumption.

Proof of Claim 2. Given claim 1, we can continue removing descendants of the colliders of the active paths that are closest to the tail-end node from the set T . Either no d-connecting path is left at some point in this process, or that we end up removing all the variables from the conditioning set. If former, this is a contradiction since X,Y cannot be made conditionally independent given empty set. If the latter is true, then there is another contradiction due to the following: This means that given empty set, paths that have a tail adjacent to one of the endpoints, i.e., the paths with colliders on them (since all paths that have a tail adjacent to one of the endpoints must have a collider because X / ∈ An ( Y ) and Y / ∈ An ( X ) ) are d-connecting, which is not possible. This proves Claim 2.

Therefore, the supposition that the only d-connecting paths must have a tail adjacent to either endpoint must be wrong, which proves the lemma.

Proof of Theorem 3.9. Now, we are ready to prove the main characterization theorem. We will need the following lemma:

Lemma A.15. Let K be a mixed graph that satisfies the conditions in Theorem 3.9. Let K ′ be the graph obtained by removing all the bidirected edges from K . Then

K ′ is a DAG and

K ′ ∼ k K .

Proof. Since the only difference between K ′ and K is the removal of bidirected edges, any directed cycle that exists in K ′ would also have existed in K , which contradicts with the assumption that K is a MAG. This establishes that K has no directed cycles.

Clearly, any independence statement in K holds in K ′ , since it is obtained from K by removing edges. Thus any degreek d-separation relation that holds in K also holds in K ′ . Therefore, we only need to show that for any c of size at most k ( a ̸ ⊥ ⊥ b | c ) K implies ( a ̸ ⊥ ⊥ b | c ) K ′ .

Suppose for the sake of contradiction that a ̸ ⊥ ⊥ b | c in K but a ⊥ ⊥ b | c in K ′ . Let p be a d-connecting path between a, b given c in K . This path must be closed in K ′ . Since the only difference between the two graphs is the removal of bidirected edges, ancestrality relations cannot be different. Thus, it cannot be the case that a collider that was open in K is now closed in K ′ and is closing the path p . Any collider that was open must still be open. Thus, the only way for p to be closed in K ′ is if some bidirected edge X ↔ Y along p is removed. However, by Lemma A.14, for any such bidirected edge in K , and for any conditioning set of size at most c , we have a d-connecting path called a replacement path with an incoming edge to both X and Y . Consider the path q obtained by replacing every bidirected edge along p with a corresponding replacement path. Since a, b are d-separated by assumption, this path cannot be open. As this path is a concatenation of several d-connecting paths either sub-paths of p , which must be open, or replacement paths which must be open, by Lemma A.4, they must have no overlapping nodes, and some node at the junction of these paths must be a collider and non-ancestor of c . However, since we replaced bidirected edges X ↔ Y with paths of the form X ←∗ . . . ∗→ Y , both X and Y must have the same collider status on both p and q . Thus, they cannot be blocking q since they are not blocking p . This means that q is d-connecting in K ′ , which is a contradiction. This proves the lemma that K and K ′ must entail the same degreek d-separation relations, which implies they are k -Markov equivalent.

The only if direction: Suppose a mixed graph is a k -closure graph, i.e., K = C k ( D ) for some DAG D and has the edge a ↔ b . Suppose for the sake of contradiction that a, b are not k -covered in K -( a ↔ b ) . Let K ′ be the graph obtained from K by removing all the bidirected edges. Note that K ′ is a DAG since K has no directed cycles. Also note that all edges in D must appear in K ′ by construction of k -closure graphs. D can therefore be obtained from K by removing edges. Thus, any d-separation statement in K must also hold in D . Therefore, a, b must be conditionally independent given some subset c of size at most k in D . This means K , in which a, b are adjacent, cannot be the k -closure graph of D , which is a contradiction.

If direction: Suppose a mixed graph K satisfies the conditions in Theorem 3.9. By Lemma A.15, for any such mixed graph K , there is a DAG whose k -closure is K , which shows that any such K is a valid k -closure graph, proving the theorem.

A.4 Proof of Lemma 3.12

Let K 1 = C k ( D 1 ) , K 2 = C k ( D 2 ) be two k -closure graphs with the same skeleton and unshielded colliders. Suppose for the sake of contradiction that there is a path p that is discriminating for a triple ⟨ u, Y, v ⟩ in both such that Y is a collider along p in C k ( D 1 ) and a non-collider in C k ( D 2 ) . Thus, in C k ( D 1 ) we have the path p as

where z i → v, ∀ i and u → v and a, v are non-adjacent. Note that we cannot have Y ← v instead of Y ↔ v since this would create the almost directed cycle u → v → Y ↔ u . The same path with Y as a non-collider can take two configurations in C k ( D 2 ) , either as

Other paths where Y is a non-collider would either render u a non-collider, which cannot happen by definition of a discriminating path, or create a directed or almost directed cycle. Since a, v are non-adjacent by definition of a discriminating path, there must be some S : | S | ≤ k where ( a ⊥ ⊥ v | S ) C k ( D 1 ) . Note that S must include all z i 's and u , and not include Y since otherwise there would be d-connecting paths between a, v in C k ( D 1 ) due to the discriminating path. This means that ( a ̸ ⊥ ⊥ v | S ) C k ( D 2 ) .

Since u ↔ Y in C k ( D 1 ) , by Lemma A.2, there must be a d-connecting path between u, Y in D 1 conditioned on S that has an arrowhead at Y . By construction, this path must also appear in C k ( D 1 ) . Since the path is inherited from D 1 , it does not have bidirected edges. Consider the shortest of all such d-connecting paths, call this path q . Let X be the node adjacent to Y along q . Thus, q has the form

We have that X → Y in both D 1 and C k ( D 1 ) . In C k ( D 1 ) , we have X → Y ↔ v . Since the edge between Y, v has a tail at Y in C k ( D 2 ) , this collider cannot exist in C k ( D 2 ) . Thus, it must be the case that this collider is shielded in C k ( D 1 ) , i.e., X and v are adjacent in C k ( D 1 ) . Since C k ( D 1 ) , C k ( D 2 ) have the same skeleton, they must also be adjacent in C k ( D 2 ) .

Now consider the path obtained by concatenating the subpath of p a ∗→ . . . u , and the subpath of q between u and X , and the edge between X and v in C k ( D 1 ) . Call this path r . Note that the subpath of q is d-connecting given S , as well as the subpath of p since z i 's and u are in S . Thus, unless X is a collider on it, the path r between a, v will be open, which would lead to a contradiction since a, v are d-separated given S in C k ( D 1 ) . Thus, the edge between X,v must have an arrowhead at X . Let W be the node before X along q . Thus we have W → X ↔ v in C k ( D 1 ) . Note that X ← v is not possible since this would create an almost directed cycle X → Y ↔ v → X in C k ( D 1 ) .

Suppose this collider is unshielded and appears in C k ( D 2 ) as well: W ∗→ X ←∗ v in C k ( D 2 ) . Thus in C k ( D 2 ) , we have Y → v ∗→ X ←∗ W . Since X,Y are adjacent, it must be that X ← Y or X ↔ Y to avoid a directed or almost directed cycle. Thus in C k ( D 2 ) , we have X ←∗ Y . However, this creates the collider W ∗→ X ←∗ Y in C k ( D 2 ) . Note that this collider cannot appear in C k ( D 1 ) since the edge between X,Y has a tail at X in C k ( D 1 ) . Thus the collider must be shielded, meaning that W,Y must be adjacent, and both in C k ( D 2 ) and in C k ( D 1 ) . Since we have W → X → Y in C k ( D 1 ) , the edge must be W → Y in C k ( D 1 ) . Furthermore, similar to X , W cannot be in the conditioning set since this would block the path q . This means there is a d-connecting path that has an arrowhead at Y that is shorter than q , which is a contradiction.

Thus the collider W → X ↔ v in C k ( D 1 ) must be shielded. Similar to the above argument, W must be a collider along the path constructed by concatenating the subpath a ∗→ . . . u of p , and the subpath of q between u and W , and the edge between W and v since otherwise this path would be open, which would contradict with a ⊥ ⊥ v | S . Let V be the node next to W along q . Thus we have V → W → X along q and V → W ←∗ v is a collider in C k ( D 1 ) . In fact, it must be that W ↔ v since otherwise there would be an almost directed cycle v → W → X → Y ↔ v in C k ( D 1 ) .

Suppose the collider V → W ↔ v in C k ( D 1 ) is unshielded and also appears in C k ( D 2 ) . Note that if V, X were adjacent in C k ( D 1 ) , the orientation would have to be as V ∗→ X since otherwise there or as

would be a directed cycle V → W → X → V in C k ( D 1 ) . But this would imply that there is a shorter path than q that connects u, Y and has an arrow into Y . Thus, V, X must be non-adjacent in C k ( D 1 ) and hence in C k ( D 2 ) . Thus, ⟨ V, W, X ⟩ is an unshielded non-collider in C k ( D 1 ) and must also be in C k ( D 2 ) . Thus it must be that W → X in C k ( D 2 ) . Since v ∗→ W → X in C k ( D 2 ) , it must be that v ∗→ X in C k ( D 2 ) to avoid a directed or almost directed cycle. Since Y → v ∗→ X in C k ( D 2 ) , it must be that X ←∗ Y to avoid a cycle or almost directed cycle in C k ( D 2 ) . However, now we have a collider W → X ←∗ Y in C k ( D 2 ) that is a non-collider in C k ( D 1 ) since in C k ( D 1 ) we have X → Y . Thus, this collider must be shielded, i.e, W,Y must be adjacent in C k ( D 2 ) . Thus, they must also be adjacent in C k ( D 1 ) . Since W → X → Y in C k ( D 1 ) , it must be that W → Y in C k ( D 1 ) to avoid a cycle. But this means there is a shorter d-connecting path between u, Y given S with an arrowhead at Y , which is a contradiction.

Therefore, the collider V → W ↔ v must be shielded in C k ( D 1 ) . We can repeat the above argument as many times as needed continuing from the parent of V along q . As we keep shielding more and more colliders in C k ( D 1 ) , eventually when we shield the first node along q next to u , we will end up with a directed path from u to Y . However, this is a contradiction since bidirected edge was added between u, Y which implies that u is not an ancestor of Y .

Therefore, if two k -closure graphs C k ( D 1 ) , C k ( D 2 ) have the same skeleton and unshielded colliders, then they cannot have different colliders along discriminating paths, which proves the lemma.

A.5 Proof of Corollary 3.13

( ⇒ ) If they are Markov equivalent then by Lemma 3.8 they are two Markov equivalent MAGs. Therefore by Theorem 3.11 they have the same skeleton, and the same unshielded colliders.

( ⇐ ) If they have the same skeleton and the same unshielded colliders, then by Lemma 3.12, they must have the same colliders along discriminating paths. Thus, by Theorem 3.11 they are equivalent.

A.6 Proof of Theorem 3.14

( ⇒ ) Suppose D 1 , D 2 are k -Markov equivalent. For the sake of contradiction suppose that C k ( D 1 ) and C k ( D 2 ) are not Markov equivalent. By Corollary 3.13, this happens when either they have different skeletons, or different unshielded colliders. Thus, there are two cases:

k-closures have different skeletons: C k ( D 1 ) and C k ( D 2 ) have different skeletons. Suppose without loss of generality that C k ( D 1 ) has an extra edge, i.e., a, b are adjacent in C k ( D 1 ) but not in C k ( D 2 ) . This can only happen if ∃ S ⊂ V : | S | ≤ k such that ( a ⊥ ⊥ b | S ) D 2 , while there is no such separating set in D 1 , implying that ( a ̸ ⊥ ⊥ b | S ) D 1 . This is a contradiction with the supposition that D 1 , D 2 are k-Markov equivalent. Therefore, C k ( D 1 ) and C k ( D 2 ) must have the same skeletons.

For completeness, we restate the definition of unshielded collider in k -closure graphs, which is identical to how it is defined in MAGs.

Definition A.16. A triple ⟨ a, c, b ⟩ in a k -closure graph is called an unshielded collider if a, b are non-adjacent, a, c and c, b are adjacent and the edges adjacent to c have an arrowhead mark at c .

According to the definition, a triple ⟨ a, c, b ⟩ in a k -closure graph C k ( D ) can be an unshielded collider if the induced subgraph on the nodes take either of the following configurations:

1. a → c ← b

2. a → c ↔ b

3. a ↔ c ← b

4. a ↔ c ↔ b

We use asterisk to represent either an arrowhead or a tail. ∗→ represents either → , ↔ . Similarly, ←∗ represents either ← or ↔ .

k-closures have different unshielded colliders: Without loss of generality assume that ( a ∗ → c ←∗ b ) C k ( D 1 ) but this unshielded collider does not exist in C k ( D 2 ) , i.e., ⟨ a, c, b ⟩ is an unshielded non-collider in C k ( D 2 ) .

Lemma A.17. In a k-closure graph C k ( D ) , two nodes a, b are non-adjacent iff ( a ⊥ ⊥ b | S ) C k ( D ) for some S ⊂ V .

Proof. ( ⇒ ) Suppose a, b are non-adjacent in C k ( D ) . Thus it must be the case that ( a ⊥ ⊥ b | S ) D for some S such that | S | ≤ k , since otherwise a, b would be made adjacent during the construction of C k ( D ) . By Lemma 3.6, this means ( a ⊥ ⊥ b | S ) C k ( D ) . This establishes the only if direction.

( ⇐ ) Suppose now that ( a ⊥ ⊥ b | S ) C k ( D ) . By definition of d-separation, adjacent nodes cannot be d-separated and thus a, b must be non-adjacent in C k ( D ) .

Lemma A.18. In a k-closure graph C k ( D ) , any pair of non-adjacent nodes a, b are separable by a set of size at most k , i.e., ∃ S : | S | ≤ k, ( a ⊥ ⊥ b | S ) C k ( D ) .

Proof. Suppose otherwise: For some non-adjacent pair a, b all d-separating sets in C k ( D ) have size greater than k . Let S be the smallest subset that makes a, b d-separated, i.e., ( a ⊥ ⊥ b | S ) C k ( D ) - one exists by Lemma A.17. Clearly, | S | > k . Note that non-adjacency of a, b in C k ( D ) implies that a, b are separable in D with some set T of size at most k : ( a ⊥ ⊥ b | T ) D . Since | T | ≤ k < | S | and S is the smallest subset that d-separates a, b in C k ( D ) , it must be that ( a ̸ ⊥ ⊥ b | T ) C k ( D ) . However, this contradicts with Lemma 3.6 which says that D and C k ( D ) must entail the same d-separation constraints for conditioning sets of size up to k .

Since a, b are non-adjacent in both graphs, by Lemma A.18 there are two subsets S 1 , S 2 of size at most k such that

Clearly, S 1 = c, S 2 ∋ c since c is a collider between a, b in C k ( D 1 ) and a non-collider in C k ( D 2 ) . If we switch the conditioning sets, due to the different collider status of c in both graphs the d-separation statements will switch to d-connection statements:

Since S 1 and S 2 have size of at most k , then from Lemma 3.6 we have that:

This implies that D 1 , D 2 are not k -Markov equivalent which is a contradiction.

This establishes that if D 1 , D 2 are k -Markov equivalent then C k ( D 1 ) , C k ( D 2 ) must have the same skeleton and the same unshielded colliders. By Corollary 3.13, C k ( D 1 ) , C k ( D 2 ) are Markov equivalent.

( ⇐ )

Suppose that C k ( D 1 ) and C k ( D 2 ) are Markov equivalent. Then they impose the same d-separation statements. Therefore they impose the same d-separation statements when the conditioning set is restricted to size at most k . By Lemma 3.6, this means that D 1 , D 2 must also impose the same d-separation statements for conditioning sets of size of at most k . This establishes that D 1 , D 2 are k -Markov equivalent.

A.7 Proof of Lemma 4.1

Suppose in the k -closure graph C k ( D ) for some DAG D and integer k , we have a discriminating path for Y between the nodes a, v of the form

By definition of discriminating path, u must be a collider along the path, and u → v . If Y ← v , then we would have an almost directed cycle u → v → Y ↔ u . Thus, we have u ↔ Y ↔ v .

First, we show that the arrowhead at Y of the edge Y ↔ v can be learned by first orienting unshielded colliders and then applying R 1 and R 2 . Consider the bidirected edge u ↔ Y . By definition of discriminating path a, v must be non-adjacent and thus separable by a set of size at most k by Lemma A.18. Therefore, we have a set S : | S | ≤ k such that a ⊥ ⊥ v | S . By the discriminating path definition, every collider along the path must be a parent of v , and therefore it must be the case that every collider along the discriminating path including u must be in S , since otherwise there would be a d-connecting path. From Lemma A.2, conditioned on any set of size at most k , we have a d-connecting path that starts with an edge into Y . Consider the shortest such path q . Let X be the node immediately before Y along q . Since this path exists in the DAG by the lemma, we have X → Y . If X,v are non-adjacent, then X → Y ↔ v would be an unshielded collider and we are done.

Suppose X,v are adjacent. Note that conditioned on S , a and X are d-connected. If X is a noncollider along the path obtained by concatenating the subpath of q between a, X and the edge between X,v , then a, v would be d-connected given S , which is a contradiction. Therefore, X must be a collider along this path. Thus we have X ↔ v . Note that we cannot have X ← v since this would create an almost directed cycle in C k ( D ) . Let V be the node immediately before X along q . Thus we have V → X ↔ v (not V ↔ X since this edge exists in D ).

Suppose V, v are non-adjacent. Thus, the collider V → X ←∗ v is unshielded, and therefore can be learned. Furthermore, V, Y must be non-adjacent since otherwise, we must have V → Y to avoid a cycle and there would be a path that is shorter than q , which 'jumps over" the node X along q . Thus, we can learn that X → Y from R 1 . Finally, since now we have learned v ∗→ X → Y and that Y, v are adjacent, by R 2 , we must have Y ←∗ v . Thus the arrowhead mark at Y of the edge Y ↔ v can be learned, and we are done.

Suppose V, v are adjacent. Following a similar argument, we either have some unshielded collider that can be propagated using the argument above to orient Y ←∗ v , or we can continue covering unshielded colliders, which would imply the previous nodes are always parents along q . But this implies that u has a directed path to Y , which cannot happen since we have u ↔ Y in C k ( D ) . This establishes that Y ←∗ v can be learned by orienting unshielded colliders and applying the rules R 1 and R 2 .

For the arrowhead mark at Y of the edge u ↔ Y , similarly consider the shortest d-connecting path q between Y, v in D given S that starts with an arrow into Y . The argument follows similarly that either there would be directed path from v to Y , which is a contradiction with the existence of the edge Y ↔ v in the k -closure graph, or that there exists an unshielded collider along q that can be learned by orienting unshielded colliders, which can be propagated to learn u ∗→ Y using rules R 1 , and R 2 . This establishes the lemma.

A.8 Proof of Theorem 4.4

We show soundness of the two new rules with the following two lemmas:

Lemma A.19. Let K be a mixed graph that is sandwiched between ε k ( D ) and PAG ( C k ( D )) , i.e., ε k ( D ) ⊆ K ⊆ PAG ( C k ( D )) . R 11 is sound on K for learning the k -essential graph, i.e., if K ′ = R 11( K ) , then ε k ( D ) ⊆ K ′ ⊆ K .

Proof. For the sake of contradiction, suppose otherwise: R 11 orients an edge ao → b in K as a → b , and there is a DAG D ′ with a k -closure graph C k ( D ′ ) that is Markov equivalent to C k ( D ) and is consistent with K where a ↔ b . This means a, b are k -covered in D ′ . Then from Lemma A.2, conditioned on any subset S of size at most k , there must be a d-connecting path that starts with an arrow into both a and b in D . By construction of the k -closure graph, this path must also exist in C k ( D ′ ) . Therefore, there must be some node w such that a ← w . Since a has no incoming edges, it must be the case that w ∈ C . However, b is chosen so that b is non-adjacent to any node in C . Therefore, b must be non-adjacent to w in K . However, this creates the unshielded collider w → a ↔ b . However, note that wo -oao → b in PAG ( C k ( D )) , and thus ⟨ w,a, b ⟩ is a non-collider in C k ( D ) . Therefore, C k ( D ′ ) cannot be Markov equivalent to C k ( D ) , which is a contradiction.

Lemma A.20. Let K be a mixed graph that is sandwiched between ε k ( D ) and PAG ( C k ( D )) , i.e., ε k ( D ) ⊆ K ⊆ PAG ( C k ( D )) . R 12 is sound on K for learning the k -essential graph, i.e., if K ′ = R 12( K ) , then ε k ( D ) ⊆ K ′ ⊆ K .

Proof. For the sake of contradiction, suppose otherwise: R 12 orients an edge ao -oc in K as a -c , and there is a DAG D ′ with a k -closure graph C k ( D ′ ) that is Markov equivalent to C k ( D ) and is consistent with K where a ↔ c . This means a, c are k -covered in D ′ . Then from Lemma A.2, conditioned on any subset S of size at most k , there must be a d-connecting path that starts with an arrow into both a and c in D . By construction of the k -closure graph, this path must also exist in C k ( D ′ ) . Therefore, there must be some node w such that a ← w . Since a has no incoming edges, it must be the case that w ∈ C . However, c is chosen so that c is non-adjacent to any other node in C . Therefore, c must be non-adjacent to w in K . However, note that wo -oao -oc in PAG ( C k ( D )) , and thus ⟨ w,a, c ⟩ is a non-collider in C k ( D ) . Therefore, C k ( D ′ ) cannot be Markov equivalent to C k ( D ) , which is a contradiction.

Now consider the execution of the algorithm k -PC . When the algorithm completes Step 4 , from Corollary 4.2 we have that K = PAG ( C k ( D )) . Since we start Step 5 with K = PAG ( C k ( D )) , from Lemma A.19 and A.20, any arrowhead and tail orientation of the K obtained at the end of step 5 must be consistent with the k -essential graph of D . Therefore, we have that ε k ( D ) ⊆ K .

B FCI Orientation Rules

Algorithm 2 FCI_Orient

Input: Mixed graph K

Apply the orientation rules of R 1 , R 2 , R 3 of [25] to K until none applies.

Apply the orientation rules of R 8 , R 9 , R 10 of [25].

Output: K

We restate the FCI orientation rules in detail and demonstrate how they are applicable for learning k -closure graphs. The following definitions are from [25].

Definition B.1 (Partial Mixed Graph (PMG)) . Any graph that contains the edge marks arrowhead, tail, circle is called a partial mixed graph (PMG).

Definition B.2 (Uncovered path) . In a PMG, a path ⟨ u 1 , u 2 . . . u m ⟩ is called an uncovered path if u i , u i +2 are non-adjacent for all i ∈ { 1 , 2 , . . . . m -2 } .

Definition B.3 (Potentially directed path) . In a PMG, a path ⟨ u 1 , u 2 . . . u m ⟩ is called a potentially directed (p.d.) path if the edge between u i and u i +1 does not have an arrowhead at u i for all i ∈ { 1 , 2 , . . . . m -1 } .

Definition B.4 (Circle path) . In a PMG, a path ⟨ u 1 , . . . u m ⟩ is called a circle path if u i o -ou i +1 for all i ∈ { 1 , 2 , . . . . m -1 } .

Note that circle paths are special cases of p.d. paths.

Rules 1 , 2 , 3 are straightforward extensions of the orientation rules for constraint-based learning to mixed graphs. For completeness, we restate them below. The star marks that appear both before and after the application of the rules are edge marks that remain unchanged by the rule.

R 1 : If a ∗→ bo -∗ c , and a, c are not adjacent, then orient the triple as a ∗→ b → c .

R 2 : If a → b ∗→ c or a ∗→ b → c and a ∗ -oc , then orient a ∗ -oc as a ∗→ c .

R 3 : If a ∗→ b ←∗ c , a ∗ -odo -∗ c , a, c are non-adjacent, and d ∗ -ob then orient d ∗ -ob are d ∗→ b .

We now restate FCI+ rules 3 R 8 , R 9 , R 10 and explain their relevance for learning k -closure graphs. Note that the rules are simplified since we do not have undirected edges that represent selection bias, and our undirected edges are treated as if they are circle edges.

R 8 : If a → b → c and ao → c , orient ao → c as a → c .

R 9 : If ao → c , and p = ⟨ a, b, u 1 , u 2 . . . u m , c ⟩ is an uncovered p.d. path from a to c such that b, c are non-adjacent, then orient ao → c as a → c .

R 10 : Suppose ao → c, b → c, d → c , p 1 is an uncovered p.d. path from a to d and p 2 is an uncovered p.d. path from a to b . Let t d be the node adjacent to a on p 1 ( t d can be d ) and t c be the node adjacent to a on p 2 ( t c can be c ). If t d , t c are distinct and non-adjacent, then orient ao → c as a → c .

R 11 and R 12 cannot replace any of the above rules. For example, consider Figure 5. None of the FCI rules apply to the output of Step 3, thus we can only learn of the unshielded colliders at the end of Step 4 of the algorithm. The completeness of FCI implies that any edge xo → y can be oriented as x ↔ y or x → y and give a MAG consistent with the PAG. However, not all such MAGs are valid k -closure graphs. R 11 can be applied to orient several tails, which gives the graph in ( d ) . Similarly in Figure 6, R 12 helps orient the tail edges between a, b which cannot be learned by FCI rules.

C Sample Runs of k -PC Algorithm

Consider the figures below for two sample runs of k -PC algorithm. Note that k -PC outputs the k -essential graph in these examples, i.e., it can orient every invariant arrowhead and tail mark in the k -closure graph of D .

a

b

c

u

v

(a)

D

a

b

c

u

v

(b)

C

k

(

D

)

a

b

c

u

v

(c)

K

After Step

4

a

b

c

u

v

(d)

K

5

.

R

11

applied

on

a, u, v

.

a

b

c

u

v

(e)

D

′

∼

k

D

a

b

c

u

v

(f)

C

k

(

D

′

)

is

consistent

with

the

output

of

k

-PC .

a

b

c

e

(a)

D

a

b

c

e

(b)

C

k

(

D

)

a

b

c

e

(c)

K

4

.

a

b

c

e

(d)

K

at the end

of Step 5.

a

b

c

e

(e)

D

′

a

b

c

e

(f)

D

′′

D Discussions

D.1 No Local k -Markov Equivalence Characterization on DAG Space

a

c

d

e

b

f

(a)

D

1

a

c

d

e

b

f

(b)

D

2

Consider the graphs in Figure 7. The only difference is the orientation of the edge between e, d . Due to the collider c → d ← e , we have a ⊥ ⊥ b in D 1 but not in D 2 . Therefore for k = 0 , we have that D 1 , D 2 are not k -Markov equivalent. However, this is not detectable locally: The endpoints of the collider responsible for the change of the k -Markov equivalence class cannot be d-separated. The effect of the collider can be detected only farther out in the graph between a, b . This shows that a local characterization similar to Theorem 2.10 is not possible for k -Markov equivalence.

D.2 k -closure Graphs vs. MAGs

a

b

c

d

u

1

u

2

(a)

D

a

b

c

d

u

1

u

2

(b)

K

=

C

k

(

D

)

for

k

=

1

In this section, we give an example for a MAG that is not a valid k -closure graph. Consider the graph in Figure 8. K in ( b ) is a valid k -closure graph for k = 1 since it is the k -closure graph of D . However, K is not a valid k -closure graph for k = 2 . This is because the bidirected edge c ↔ d is added between a pair that is not k -covered for k = 2 : We have c ⊥ ⊥ d | u 1 , u 2 in D . In fact, using the if and only if characterization in Theorem 3.9, we can show that there does not exist any D ′ with the given k -closure graph where c, d have a bidirected edge.

D.3 Bidirected Edge in k -essential Graphs

In Figure 9, since every endpoint is an arrowhead and is part of an unshielded collider, there is no other Markov equivalent k -closure graphs, which implies that the k -essential graph is the same as the k -closure graph. Thus, the edge c ↔ e is in ε k ( D ) . This example shows that we can learn that two nodes do not cause each other using conditional independence tests that are not even powerful enough to make them conditionally independent . It is worth noting that LOCI [23] can also infer this fact by removing this edge.

a

c

e

d

b

(a)

D

a

c

e

d

b

(b)

C

k

(

D

)

D.4 k -PC is Incomplete

One might hope that k -PC is complete and outputs the k -essential graph ε k ( D ) . This, however, is not true. We discuss an example where k -PC cannot orient an invariant tail mark.

First, observe that k -PC does not leverage the value of k . If we had an efficient way to answer the question ' Is there a k -closure graph that is consistent with K in which a, b are k -covered?" then we could leverage this to orient more o → edges as → edges. As an example, consider the causal graph in Figure 10. d may have an incoming edge that prevents us from eliminating the possibility that do → b is a bidirected edge d ↔ b . However, we can only have a single d-connecting path between d, b . Since d is a non-collider along a -d -c , one of the edges must be out of d . Suppose d ← a and d → c . For c to not block this path it has to be a non-collider, and thus c → b . This is the only way we can have two d-connecting paths between d, b in the underlying DAG. However, now we have the path d → c → b , which makes d an ancestor of b . Therefore the edge d ↔ b is inconsistent. Thus in k -essential graph of D , we must have a tail at d as d → b . This cannot be learned by k -PC .

d

a

c

b

(a)

D

1

d

a

c

b

(b)

C

k

(

D

1

)

d

a

c

b

(c)

k

-PC Output

Figure 10: A graph where k -essential graph contains an invariant tail that cannot be learned by k -PC algorithm. k = 1 ; thus d, b are k -covered but not a, c , which gives the k -closure graph on the right. k -PC algorithm outputs the graph in ( c ) . However, there is no k -closure in the Markov equivalence class where d ↔ b . Thus the edge in the k -essential graph should be d → b . Similarly, ao → b should be a → b , however this requires reasoning about the number of d-connecting paths between a, b in K that is not captured in k -PC .

We observe that a local algorithm such as k -PC cannot be used to assess if there is some k -closure graph that is consistent with the current graph in which two nodes are k -covered without the edge between them. One practical strategy would be to take the output of the k -PC algorithm and list all MAGs consistent with the circle edges, and then check if they are valid k -closure graphs by pruning every edge using Theorem 3.9. For small or sparse k -closure graphs, k -PC could be a practical way to reduce the search space efficiently, and then we can conduct an exhaustive search as the next step to obtain a sound and complete algorithm for learning the k -essential graph.

D.5 Heuristic Uses of Bounded Size Conditioning Sets

For large number of variables, and except for very sparse graphs, constraint-based algorithms take significant time to complete. This is because the progressive nature of such tests is not able to sufficiently sparsify the graph with low-degree conditional independence tests, and they have to perform many tests: If the neighbor size is O ( n ) , where n is the number of nodes, algorithm needs to check exponentially many subsets of nodes in the conditioning set.

To prevent this issue, several implementations of these algorithms have the added functionality to restrict this search by limiting the size of the conditioning set. For example, causal-learn package has this functionality. However, this is a heuristic that simply prematurely stops the search algorithms.

The results of our paper build the theoretical understanding of what is learnable in this setting with a new equivalence class and its graphical representation.

D.6 Comparison of k -PC Output to LOCI Output

Lemma D.1. Consider three nodes a, b, c . Suppose a ̸ ⊥ ⊥ b | S , b ̸ ⊥ ⊥ c | S , a ⊥ ⊥ c | S for some set S : | S | ≤ k and b / ∈ S . If b, c are k -covered , then k -PC orients the edge between b, c as b ←∗ c .

Proof. In the following, we show that given the CI pattern that LOCI uses to orient edges, k -PC also orients the same edges. Consider the CI pattern that LOCI uses between three nodes a, b, c .

Conditioned on S , there is a d-connecting path between a, b ; let us call this path p . Conditioned on S , there is a d-connecting path between b, c ; let us call this path q . Consider the path obtained by concatenating p, q . Since this path must be d-separating, b must be a collider on it and it must be the case that b / ∈ An ( S ) . Let the node that is adjacent to b along p be a ′ and the node adjacent to b along q be c ′ . Thus we have a ′ → b ← c ′ .

Case 1: Now suppose that a ′ and c are separable by some T : | T | ≤ k . k -PC would then orient a ′ ∗→ b ←∗ c since b, c remains adjacent throughout the execution of k -PC . Thus b ←∗ c would be oriented in this case.

Case 2: Suppose that a ′ , c are k -covered . Then, given S , there is a d-connecting path between a ′ , c . Now consider the path obtained by concatenating the subpath of p between a, a ′ and this d-connecting path. Since a, c are d-separated given S , a ′ must be a collider along this path and a non-ancestor of S . Thus we must have a ′′ → a ′ → b as the last three nodes of path p .

Case 2.a: Suppose a ′′ , c are separable by some T : | T | ≤ k . Since a ′ , c are k -covered , they remain adjacent throughout the execution of k -PC . Thus, k -PC would orient a ′′ ∗→ a ′ ←∗ c . Now we consider two sub-cases:

Case 2.a.i: Suppose a ′′ , b are separable by some set of size at most k . Then, a ′′ ∗→ a ′ -b is an unshielded triple and it would be oriented by the first Meek rule of k -PC as a ′′ ∗→ a ′ → b . Now we have the following edges oriented: b ←∗ a ′ ←∗ c , and b, c adjacent. By the second Meek rule, k -PC will then orient b ←∗ c . This establishes that in this sub-case, k -PC would also orient the arrowhead adjacent to b for the edge between b, c .

Case 2.a.ii: Now consider the second sub-case: Suppose a ′′ , b are k -covered . Thus, throughout the execution of k -PC , a ′′ , b are adjacent. In this case, a ′′ o -obo -oc forms an unshielded collider and the algorithm would orient them as a ′′ ∗→ b ←∗ c due to the independence statement a ′′ ⊥ ⊥ c | T . Therefore, the arrowhead at b would be oriented in this case as well.

Case 2.b: Now suppose a ′′ , c are k -covered . Thus, there must be a d-connecting path between a ′′ , c given S . Now consider the path obtained by concatenating the subpath of p between a, a ′′ and this d-connecting path. Since a ⊥ ⊥ c | S , it must be that a ′′ is a collider along this path and that a ′′ / ∈ An ( S ) . Thus, along this path we have a ′′′ → a ′′ ← . . . . Therefore, the last four nodes of the path p is a ′′′ → a ′′ → a ′ → b , and a ′ , c are k -covered , a ′′ , c are k -covered .

Case 2.b.i: Now suppose a ′′′ , c are separable by some T : | T | ≤ k .

Case 2.b.i. α : If the pair a ′′′ , b is k -covered , following the above argument, we would orient a ′′′ ∗→ b ←∗ c with the statement a ′′′ ⊥ ⊥ c | S , and the arrowhead adjacent to b along the edge between b, c would have been oriented.

Case 2.b.i. β : Suppose a ′′′ , b pair is separable.

Case 2.b.i. β .1: Suppose a ′′′ , a ′ is k -covered . We would orient a ′′′ ∗ → a ′ ←∗ c due to the statement a ′′′ ⊥ ⊥ c | S . And since a ′′′ , b are not k -covered , Meek rule one would orient the subgraph a ′′′ ∗→ a ′ o -ob as a ′′′ ∗→ a ′ ∗→ b . Since b ←∗ a ′ ←∗ c and bo -oc , second Meek rule would orient the arrowhead at b along the edge between b, c .

Case 2.b.i. β .2: Finally, suppose a ′′′ , b and a ′′′ , a ′ are both separable. Now k -PC applies Meek rule one directly to orient a ′′′ ∗→ a ′′ ∗→ a ′ . Now that we have a ′ ←∗ a ′′ ←∗ c , and that a ′ , c are adjacent, Meek rule two will orient a ′ ←∗ c . Another application of Meek rule one would give a ′′ ∗→ a ′ ∗→ b and now since we have b ←∗ a ′ ←∗ c and that b, c are adjacent, Meek rule two would orient b ←∗ c .

Case 2.b.j and beyond: Finally, if a ′′′ , c are k -covered , following a similar argument, either the node adjacent to a ′′′ along p (towards a ) is separable with c , in which case following a similar argument as above would orient b ←∗ c , or we continue until a, c become adjacent. The latter cannot happen since that contradicts with the fact that a ⊥ ⊥ c | S . Thus, there must exist some node u along p that is separable from c , and the subpath of p between u, b is directed. Following the argument above, repeated application of Meek rules one and two will result in the orientation of the edge between b, c as b ←∗ c . This establishes that k -PC orients at least as much arrowheads as LOCI.

The corollary of this lemma is that k -PC orients all arrowheads oriented by LOCI:

Corollary D.2. Any arrowhead oriented by LOCI is also oriented by k -PC .

Proof. Suppose a ̸ ⊥ ⊥ b | S , b ̸ ⊥ ⊥ c | S , a ⊥ ⊥ c | S for some set S : | S | ≤ k and b / ∈ S . Observe that if a, b and b, c are k -covered then k -PC would orient the edges between them as a ∗→ b ←∗ c due to the independence statement a ⊥ ⊥ c | S . Moreover, if a, b and b, c are both separable by some sets of size at most k , then both LOCI and k -PC would make a, b and b, c non-adjacent and thus neither algorithm orients an edge between them. Therefore, the only non-trivial case is when only one of the two pairs is k -covered . For this case, since the pre-condition of Lemma D.1 is identical to the condition of LOCI to orient any edge.

LOCI applies the three Meek rules after orienting these arrowhead marks. Since k -PC repeatedly applies a set of Meek rules that include these three rules, k -PC orients at least as many arrowheads as LOCI. Thus, the corollary follows.

Next, we show that they both carry the same adjacency information.

Corollary D.3. Any pair that is non-adjacent in LOCI output is either non-adjacent, or adjacent via a bidirected edge in k -PC output.

Proof. LOCI makes a pair non-adjacent in two ways. If LOCI makes a pair non-adjacent since they are separable, k -PC also will make them non-adjacent. Suppose LOCI makes a pair a, b non-adjacent due to the following CI pattern, which are otherwise k -covered : u ̸ ⊥ ⊥ a | S 1 , a ̸ ⊥ ⊥ b | S 1 , u ⊥ ⊥ b | S 1 and a ̸ ⊥ ⊥ b | S 2 , b ̸ ⊥ ⊥ v | S 2 , a ⊥ ⊥ v | S 2 for some u, v, S 1 , S 2 . Note that by Lemma D.1, k -PC marks both endpoints of the edge between a, b as arrowheads. This concludes the proof.

These results can be combined with the example in Figure 2 to show that the k -PC output carries strictly more information about the set of causal graphs that entail the set of degreek d-separation statements than the output of LOCI, even though k -PC is not complete for learning our equivalence class as discussed in Section D.4.

D.7 Comparison of k -PC Output to AnytimeFCI Output

The undirected edges in our representation do not represent selection bias. We use them to represent a different graph union operation. The lack of latents gives us this flexibility, which allows us to distinguish different sets of graphs by using both - and o -o edge marks for this.

For a concrete example on why AnytimeFCI is less informative than k -PC in our setting, consider the causal graph in Figure 2. Here, k -PC will learn the representation on the right: We can infer that either a causes d or d causes a , noted by the undirected edge a -d in our representation, rather than the circle edge o -o , which would allow a, d to be non-adjacent in some DAGs. Anytime FCI here will instead output circle edges between a and d , i.e., ao -od . Even if we try to incorporate the knowledge that the underlying graph is causally sufficient, it is not obvious how one can conclude that a and d must be adjacent in every DAG that induces the same degreek d-separation statements from the Anytime FCI output. One way to do this would be to remove the edge from this PAG and check whether a, d can now be d-separated by some conditioning set of size at most k . If yes, this changes the equivalence class, which means the edge must be there in any DAG. Our local rule R 12 can instead orient this as an undirected edge, signifying that a and d must be adjacent in every underlying DAG by leveraging our equivalence class representation, without having to run this type of post-processing which might be computationally intensive for larger graphs.

E Additional Experiments

E.1 Experiments vs. LOCI

0.0

0.2

0.4

0.6

0.2

0.4

0.6

0.8

1.0

CDF of

F ar

1

, N=50

kPC,k=2

LOCI,k=0

LOCI,k=1

LOCI,k=2

(a) Arrowhead

F

1

Score

0.0

0.2

0.4

0.6

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F ta

1

, N=50

kPC,k=2

LOCI,k=0

LOCI,k=1

LOCI,k=2

(b) Tail

F

1

Score

0.0

0.2

0.4

0.6

0.8

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F sk

1

, N=50

kPC,k=2

LOCI,k=0

LOCI,k=1

LOCI,k=2

F

1

Score

0.0

0.2

0.4

0.6

0.8

0.2

0.4

0.6

0.8

1.0

CDF of

F ar

1

, N=100

kPC,k=2

LOCI,k=0

LOCI,k=1

LOCI,k=2

(d) Arrowhead

F

1

Score

0.0

0.2

0.4

0.6

0.8

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F ta

1

, N=100

kPC,k=2

LOCI,k=0

LOCI,k=1

LOCI,k=2

(e) Tail

F

1

Score

0.0

0.2

0.4

0.6

0.8

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F sk

1

, N=100

kPC,k=2

LOCI,k=0

LOCI,k=1

LOCI,k=2

F

1

Score

E.2 Experiments vs. NOTEARS

0.00

0.25

0.50

0.75

1.00

0.6

0.7

0.8

0.9

1.0

CDF of

F ar

1

, N=50

PC

kPC,k=2

NOTEARS

(a) Arrowhead

F

1

Score

0.00

0.25

0.50

0.75

1.00

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F ta

1

, N=50

PC

kPC,k=2

NOTEARS

(b) Tail

F

1

Score

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F sk

1

, N=50

PC

kPC,k=2

NOTEARS

F

1

Score

0.00

0.25

0.50

0.75

1.00

0.5

0.6

0.7

0.8

0.9

1.0

CDF of

F ar

1

, N=100

PC

kPC,k=2

NOTEARS

(d) Arrowhead

F

1

Score

0.00

0.25

0.50

0.75

1.00

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F ta

1

, N=100

PC

kPC,k=2

NOTEARS

(e) Tail

F

1

Score

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F sk

1

, N=100

PC

kPC,k=2

NOTEARS

F

1

Score

E.3 More Experiments vs. PC

In this section, we show a larger range of N (number of samples). We also explore the behaivor for graphs with different edge densities and higher number of nodes ( 10 ).

0.0

0.1

0.2

0.75

0.80

0.85

0.90

0.95

1.00

CDF of

F ar

1

, N=10

PC

kPC,k=2

(a) Arrowhead

F

1

Score

0.0

0.1

0.2

0.3

0.4

0.6

0.8

1.0

CDF of

F ta

1

, N=10

PC

kPC,k=2

(b) Tail

F

1

Score

0.0

0.1

0.2

0.3

0.4

0.2

0.4

0.6

0.8

1.0

CDF of

F sk

1

, N=10

PC

kPC,k=2

F

1

Score

0.0

0.2

0.4

0.2

0.4

0.6

0.8

1.0

CDF of

F ar

1

, N=50

PC

kPC,k=2

(d) Arrowhead

F

1

Score

0.0

0.1

0.2

0.3

0.4

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F ta

1

, N=50

PC

kPC,k=2

(e) Tail

F

1

Score

0.0

0.2

0.4

0.6

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F sk

1

, N=50

PC

kPC,k=2

F

1

Score

0.0

0.2

0.4

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F ar

1

, N=100

PC

kPC,k=2

(g) Arrowhead

F

1

Score

0.0

0.2

0.4

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F ta

1

, N=100

PC

kPC,k=2

(h) Tail

F

1

Score

0.2

0.4

0.6

0.8

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F sk

1

, N=100

PC

kPC,k=2

F

1

Score

0.0

0.2

0.4

0.6

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F ar

1

, N=1000

PC

kPC,k=2

(j) Arrowhead

F

1

Score

0.0

0.2

0.4

0.6

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F ta

1

, N=1000

PC

kPC,k=2

(k) Tail

F

1

Score

0.6

0.8

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F sk

1

, N=1000

PC

kPC,k=2

F

1

Score

Next, we present combined metrics for this same setup. Namely, we show the advantage of our algorithm in terms of the sum of arrowhead and tail F 1 scores, and the sum of arrowhead, tail and skeleton F 1 scores.

E.4 Experiments of Section E.3

0.0

0.2

0.4

0.6

0.80

0.85

0.90

0.95

1.00

CDF of

F ar

1 +

F ta

1

, N=10

PC

kPC,k=2

(a) Arrowhead

+

Tail

F

1

0.00

0.25

0.50

0.75

0.80

0.85

0.90

0.95

1.00

CDF of

F sk

1 +

F ar

1 +

F ta

1

, N=10

PC

kPC,k=2

(b) Arrowhead

+

Tail

+

Skeleton

F

1

0.0

0.2

0.4

0.6

0.8

0.4

0.6

0.8

1.0

CDF of

F ar

1 +

F ta

1

, N=50

PC

kPC,k=2

(c) Arrowhead

+

Tail

F

1

0.0

0.5

1.0

0.4

0.6

0.8

1.0

CDF of

F sk

1 +

F ar

1 +

F ta

1

, N=50

PC

kPC,k=2

(d) Arrowhead

+

Tail

+

Skeleton

F

1

0.0

0.2

0.4

0.6

0.8

0.2

0.4

0.6

0.8

1.0

CDF of

F ar

1 +

F ta

1

, N=100

PC

kPC,k=2

(e) Arrowhead

+

Tail

F

1

0.0

0.5

1.0

1.5

0.2

0.4

0.6

0.8

1.0

CDF of

F sk

1 +

F ar

1 +

F ta

1

, N=100

PC

kPC,k=2

(f) Arrowhead

+

Tail

+

Skeleton

F

1

0.0

0.5

1.0

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F ar

1 +

F ta

1

, N=1000

PC

kPC,k=2

(g) Arrowhead

+

Tail

F

1

0.0

0.5

1.0

1.5

2.0

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F sk

1 +

F ar

1 +

F ta

1

, N=1000

PC

kPC,k=2

(h) Arrowhead

+

Tail

+

Skeleton

F

1

E.5 Experiments vs. Conservative PC

We use pcalg package in R as the implementation for conservative PC [14].

0.50

0.25

0.00

0.25

0.50

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F ar

1

, N=10

Cons_PC

kPC,k=2

(a) Arrowhead

F

1

Score

0.0

0.1

0.2

0.3

0.4

0.6

0.8

1.0

CDF of

F ta

1

, N=10

Cons_PC

kPC,k=2

(b) Tail

F

1

Score

0.0

0.2

0.4

0.2

0.4

0.6

0.8

1.0

CDF of

F sk

1

, N=10

Cons_PC

kPC,k=2

F

1

Score

0.0

0.2

0.4

0.2

0.4

0.6

0.8

1.0

CDF of

F ar

1

, N=50

Cons_PC

kPC,k=2

(d) Arrowhead

F

1

Score

0.0

0.2

0.4

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F ta

1

, N=50

Cons_PC

kPC,k=2

(e) Tail

F

1

Score

0.0

0.2

0.4

0.6

0.8

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F sk

1

, N=50

Cons_PC

kPC,k=2

F

1

Score

0.0

0.2

0.4

0.6

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F ar

1

, N=100

Cons_PC

kPC,k=2

(g) Arrowhead

F

1

Score

0.0

0.2

0.4

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F ta

1

, N=100

Cons_PC

kPC,k=2

(h) Tail

F

1

Score

0.2

0.4

0.6

0.8

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F sk

1

, N=100

Cons_PC

kPC,k=2

F

1

Score

0.0

0.2

0.4

0.6

0.8

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F ar

1

, N=500

Cons_PC

kPC,k=2

(j) Arrowhead

F

1

Score

0.0

0.2

0.4

0.6

0.8

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F ta

1

, N=500

Cons_PC

kPC,k=2

(k) Tail

F

1

Score

0.6

0.8

0.0

0.2

0.4

0.6

0.8

1.0

CDF of

F sk

1

, N=500

Cons_PC

kPC,k=2

F

1

Score

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nBesides computational complexity, the weak spot of constraint-based causal discovery is the assumption of having an independence oracle. Especially with large conditioning sets, independence testing is difficult. This paper aims to address this limitation by restricting the number of conditioning variables and provides a sound framework that allows us to study the guarantees one can obtain for constraint-based structure learning if one restricts the size of the conditioning sets. The authors show that the results can contain bi-directed edges and relate the obtained graph to maximal ancestral graphs. They further provide a sound algorithm to learn such graphs and evaluate it on a few synthetic and semi-synthetic examples. \n\nThe theoretical contributions of the paper are interesting and presented clearly to most parts. The main limitation of the work are the points that I mention with respect to the empirical evaluation.\n\nSTRENGTHS:\n- Rigorous definition of assumptions, goal, and solution that can be obtained under the concept of restricting the size of the conditioning set.\n- The new formulation of k-closed graphs was introduced well and discussed in the context of previous notions (ancestral graphs). Further, Markov equivalence and faithfulness are adapted for the introduced graphs.\n- The authors propose a learning algorithm k-PC to learn the Markov equivalence class of the introduced k-closure of the true DAG.\n- The proposed algorithm has been empirically evaluated to PC, LOCI and NOTEARS, and the code has been provided for reproducibility.\n\nWEAKNESSES:\n- The performance gain seems to be only evident in small sample regimes.\n- The new approach does not clearly outperform LOCI & it would be interesting to see how PC performs when restricting the conditioning set, as well.\n- It would be interesting to study a real-world example, or DAGs with non-linear relations. I could imagine that on the latter, using a smaller conditioning set can be a more substantial advantage, as independence testing with large conditioning sets is especially difficult for arbitrary functional relationships. For more details on the problem and maybe a good source to motivate the proposed algorithm please consider Shah and Peters (2020)\n\nShah, Peters, The Hardness of Conditional Independence Testing and the Generalised Covariance Measure, The Annals of Statistics, 2020\n\nMinor (suggestions):\n- For easier comparison, it might be beneficial to compute the area under the curve for the metrics shown in Figure 3.\n\nQUESTIONS:\n- What is the intuition of step 5 in the algorithm? Why do we have to exclude it for Corollary 4.2? Not much intuition is provided regarding this step, and it just occurs in Corollary 4.2.\n- I understand that if we just run the PC algorithm and restrict the conditioning set it does not inherit the same guarantees as PC-k, e.g., it would not find any bi-directed edges. What would be the other differences/failure cases of this naive approach?\n- Assume one naively restricts the maximum size of the conditioning set in the PC algorithm, which I think is an option in the pcalg package, for example. How does this version of PC perform if we restrict the conditioning size in a similar manner as for the proposed method (e.g., for the experiments shown in Fig. 3 & 4)?\n- How does the computational complexity or empirical runtime compare to PC?\n\nLIMITATIONS:\nStandard limitations occurring in constraint-based causal discovery (a variant of faithfulness, independence oracle).", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-06-27"}

comments: SUMMARY:
A new structure learning approach is proposed in limited data settings. In these settings, existing constraint-based causal discovery approaches struggle with statistical tests of independence. The main insight is that we can define an equivalence class of graphs based on an upper bound on the size of the conditioning set. Using this, a new learning algorithm is presented to learn the equivalence class.

STRENGTHS:
The paper is well-motivated and clear in its contributions. The problem of causal discovery from limited datasets seems to be a significant one. The novel contributions include introducing the formalism of K-Markov equivalence which can then be used to derive a learning algorithm. The soundness of the algorithm is guaranteed and the completeness is shown in some cases. Overall the theoretical contributions seem strong in this work.

WEAKNESSES:
On the empirical side, comparisons are performed with state of the art structure learners on synthetic and semi-synthetic datasets. Since the motivation of the work was causal discovery from limited data, using some more real-world cases could have made the experiments much stronger. Overall, perhaps the empirical evaluation is not as strong as the theoretical contributions of the paper.

QUESTIONS:
Are there real-world examples of learning from limited conditioning sets that could perhaps be used to strengthen the experimental evaluation?

LIMITATIONS:
There are no limitations that are explicitly mentioned in the paper.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-05

comments: SUMMARY:
The authors study the problem of causal learning with bounded conditioning sets. It is an important question that up to what set of equivalence graphs a learner can  infer a causal graph when it can only perform conditional independence tests with limited size of the conditioning sets. They characterize the equivalence set and propose a sound constraint-based algorithm for learning it.

STRENGTHS:
The authors study an important research question. The paper is well-organized. 

The results are quite relevant and important. It nicely establishes the connections between the existing results about Markov equivalence class and newly define k- Markov equivalence class that is the set of causal graphs that have the same conditional independences with conditioning sets at most k.

This results can be a foundation for further developments in causal graph learning.

WEAKNESSES:
A discussion about the complexity of the proposed algorithm will improve the paper. 
For instance, what is the complexity of checking whether two given DAGs are k-Markov equivalent?

QUESTIONS:
See above.

LIMITATIONS:
The authors adequately addressed the limitations.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-06

comments: SUMMARY:
While I have the background to understand this paper, I'm afraid I am not well positioned to judge its novelty and significance in the causal discovery subfield. In short, the PC algorithm struggle when conditioning sets are large, so the authors propose a modification of Markov equivalence for bounded conditioning sets, then graphically characterize this relationship, and then propose the k-PC algorithm that is more successful in synthetic small small regimes.

 Nothing in the math or presentation seems incorrect to me, though I did not closely examine the more technical parts of the paper. The experiments are on synthetic data, which is typical in this subfield, but that makes it difficult to know whether k-PC has practical advantages over PC in real-world cases, as there are likely regimes where PC is superior. 


Overall, this seems like competent well done work, but I lean towards rejection for this paper. The modification of PC and Markov equivalence for bounded conditioning sets seems like a straightforward and simple modification of existing definitions, so the contribution does not seem novel and significant to me. If there were convincing empirical results that showed this change makes causal discovery viable in new real-world contexts I might be convinced, but as it stands its unclear to me whether the synthetic experiments are good evidence of this, especially with how little prose was in the main text describing them. This might be more appropriate for a conference or workshop that is more narrowly focused on causality.  

I'm perfectly happy to be convinced otherwise, or have my opinion ignored in favor of reviewers/metareviewers with a better understanding of the subfield!

STRENGTHS:
See main

WEAKNESSES:
See main

QUESTIONS:
See main

LIMITATIONS:
See main
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07

comments: SUMMARY:
This paper aims to address the problem that in constraint-based causal discovery based on conditional independence tests, the tests become statistically unreliable with limited data when the conditioning sets are large. The proposed solution is to use only conditional independence tests for relatively small conditioning sets. To facilitate this goal, this paper provides a characterisation of k-Markov equivalence between causal DAGs, in the sense of entailing the same conditional independence statements in which the conditioning set has a cardinality not exceeding k. Since the characterisation is in terms of maximal ancestral graphs, a FCI-like algorithm is proposed to learn such equivalence classes and is empirically evaluated against some benchmarks.

STRENGTHS:
This paper is well written and very readable. The technical results are sound and the empirical evaluation is reasonable.

WEAKNESSES:
Unfortunately, as far as I can tell, this paper largely reproduces the results of an old paper by Peter Spirtes (2001), "An Anytime Algorithm for Causal Inference", Proceedings of the Eighth International Workshop on Artificial Intelligence and Statistics. It seems to me that all the main results and insights of this paper were already presented in Spirtes's paper. I may have missed something important. I will increase my score if the authors can convince me that their contributions remain sufficiently original and significant given Spirtes's paper.

QUESTIONS:
Are the main results of this paper a reproduction of Spirtes's (2001) results?

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-08