SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Case 2.a.i: Suppose  $a''$ ,  $b$  are separable by some set of size at most  $k$ . Then,  $a'' * \rightarrow a' - b$  is an unshielded triple and it would be oriented by the first Meek rule of  $k$ -PC as  $a'' * \rightarrow a' \rightarrow b$ . Now we have the following edges oriented:  $b \leftarrow *a' \leftarrow *c$ , and  $b$ ,  $c$  adjacent. By the second Meek rule,  $k$ -PC will then orient  $b \leftarrow *c$ . This establishes that in this sub-case,  $k$ -PC would also orient the arrowhead adjacent to  $b$  for the edge between  $b$ ,  $c$ .
Case 2.a.ii: Now consider the second sub-case: Suppose  $a''$ ,  $b$  are  $k$ -covered. Thus, throughout the execution of  $k$ -PC,  $a''$ ,  $b$  are adjacent. In this case,  $a''o$  -  $obo$  -  $oc$  forms an unshielded collider and the algorithm would orient them as  $a''* \rightarrow b \leftarrow *c$  due to the independence statement  $a'' \perp c|T$ . Therefore, the arrowhead at  $b$  would be oriented in this case as well.
Case 2.b: Now suppose  $a''$ ,  $c$  are  $k$ -covered. Thus, there must be a d-connecting path between  $a''$ ,  $c$  given  $S$ . Now consider the path obtained by concatenating the subpath of  $p$  between  $a$ ,  $a''$  and this d-connecting path. Since  $a \perp c|S$ , it must be that  $a''$  is a collider along this path and that  $a'' \notin An(S)$ . Thus, along this path we have  $a''' \to a'' \gets \dots$ . Therefore, the last four nodes of the path  $p$  is  $a''' \to a'' \to a' \to b$ , and  $a'$ ,  $c$  are  $k$ -covered,  $a''$ ,  $c$  are  $k$ -covered.
Case 2.b.i: Now suppose  $a'''$ ,  $c$  are separable by some  $T: |T| \leq k$ .
Case 2.b.i.  $\alpha$ : If the pair  $a'''$ ,  $b$  is  $k$ -covered, following the above argument, we would orient  $a''' * b \gets *c$  with the statement  $a''' \perp c|S$ , and the arrowhead adjacent to  $b$  along the edge between  $b, c$  would have been oriented.
Case 2.b.i.  $\beta$  : Suppose  $a^{\prime \prime \prime}$ $b$  pair is separable.
Case 2.b.i.  $\beta .1$  : Suppose  $a^{\prime \prime \prime},a^{\prime}$  is  $k$  -covered. We would orient  $a^{\prime \prime \prime}*a^{\prime}\leftarrow *c$  due to the statement  $a^{\prime \prime \prime}\perp c|S$  . And since  $a^{\prime \prime \prime}$ $b$  are not  $k$  -covered, Meek rule one would orient the subgraph  $a^{\prime \prime \prime}*a^{\prime}o\mathrm{-}ob$  as  $a^{\prime \prime \prime}*a^{\prime}*a^{\prime}\rightarrow b$  . Since  $b\gets *a^{\prime}\gets *c$  and  $bo - oc$  , second Meek rule would orient the arrowhead at  $b$  along the edge between  $b,c$
Case 2.b.i.  $\beta .2$  : Finally, suppose  $a^{\prime \prime \prime},b$  and  $a^{\prime \prime \prime}$ $a^\prime$  are both separable. Now  $k$  -PC applies Meek rule one directly to orient  $a^{\prime \prime \prime}* \rightarrow a^{\prime \prime} * \rightarrow a^{\prime}$  . Now that we have  $a^\prime \leftarrow *a^{\prime \prime}\leftarrow *c$  , and that  $a^\prime ,c$  are adjacent, Meek rule two will orient  $a^\prime \leftarrow *c$  . Another application of Meek rule one would give  $a^{\prime \prime}* \rightarrow a^{\prime} * \rightarrow b$  and now since we have  $b\gets *a^{\prime}\gets *c$  and that  $b,c$  are adjacent, Meek rule two would orient  $b\gets *c$
Case 2.b.j and beyond: Finally, if  $a'''$ ,  $c$  are  $k$ -covered, following a similar argument, either the node adjacent to  $a'''$  along  $p$  (towards  $a$ ) is separable with  $c$ , in which case following a similar argument as above would orient  $b \leftarrow *c$ , or we continue until  $a$ ,  $c$  become adjacent. The latter cannot happen since that contradicts with the fact that  $a \perp c | S$ . Thus, there must exist some node  $u$  along  $p$  that is separable from  $c$ , and the subpath of  $p$  between  $u, b$  is directed. Following the argument above, repeated application of Meek rules one and two will result in the orientation of the edge between  $b$ ,  $c$  as  $b \leftarrow *c$ . This establishes that  $k$ -PC orients at least as much arrowheads as LOCI.
The corollary of this lemma is that  $k$ -PC orients all arrowheads oriented by LOCI:
Corollary D.2. Any arrowhead oriented by LOCI is also oriented by  $k$ -PC.
Proof. Suppose  $a \nVdash b|S, b \nVdash c|S, a \bot c|S$  for some set  $S: |S| \leq k$  and  $b \notin S$ . Observe that if  $a, b$  and  $b, c$  are  $k$ -covered then  $k$ -PC would orient the edges between them as  $a * \rightarrow b \leftarrow * c$  due to the independence statement  $a \bot c|S$ . Moreover, if  $a, b$  and  $b, c$  are both separable by some sets of size at most  $k$ , then both LOCI and  $k$ -PC would make  $a, b$  and  $b, c$  non-adjacent and thus neither algorithm orients an edge between them. Therefore, the only non-trivial case is when only one of the two pairs is  $k$ -covered. For this case, since the pre-condition of Lemma D.1 is identical to the condition of LOCI to orient any edge.
LOCI applies the three Meek rules after orienting these arrowhead marks. Since  $k$ -PC repeatedly applies a set of Meek rules that include these three rules,  $k$ -PC orients at least as many arrowheads as LOCI. Thus, the corollary follows.
Next, we show that they both carry the same adjacency information.
Corollary D.3. Any pair that is non-adjacent in LOCI output is either non-adjacent, or adjacent via a bidirected edge in  $k$ -PC output.
Proof. LOCI makes a pair non-adjacent in two ways. If LOCI makes a pair non-adjacent since they are separable,  $k$ -PC also will make them non-adjacent. Suppose LOCI makes a pair  $a, b$  non-adjacent due to the following CI pattern, which are otherwise  $k$ -covered:  $u \not\perp a | S_1$ ,  $a \not\perp b | S_1$ ,  $u \perp b | S_1$  and  $a \not\perp b | S_2$ ,  $b \not\perp v | S_2$ ,  $a \perp v | S_2$  for some  $u, v, S_1, S_2$ . Note that by Lemma D.1,  $k$ -PC marks both endpoints of the edge between  $a, b$  as arrowheads. This concludes the proof.
These results can be combined with the example in Figure 2 to show that the  $k$ -PC output carries strictly more information about the set of causal graphs that entail the set of degree- $k$  d-separation statements than the output of LOCI, even though  $k$ -PC is not complete for learning our equivalence class as discussed in Section D.4.
D.7 Comparison of  $k$ -PC Output to AnytimeFCI Output
The undirected edges in our representation do not represent selection bias. We use them to represent a different graph union operation. The lack of latents gives us this flexibility, which allows us to distinguish different sets of graphs by using both — and  $o$  — edge marks for this.
For a concrete example on why AnytimeFCI is less informative than  $k$ -PC in our setting, consider the causal graph in Figure 2. Here,  $k$ -PC will learn the representation on the right: We can infer that either  $a$  causes  $d$  or  $d$  causes  $a$ , noted by the undirected edge  $a$  —  $d$  in our representation, rather than the circle edge  $o$  —  $o$ , which would allow  $a, d$  to be non-adjacent in some DAGs. Anytime FCI here will instead output circle edges between  $a$  and  $d$ , i.e.,  $ao$  —  $od$ . Even if we try to incorporate the knowledge that the underlying graph is causally sufficient, it is not obvious how one can conclude that  $a$  and  $d$  must be adjacent in every DAG that induces the same degree- $k$ $d$ -separation statements from the Anytime FCI output. One way to do this would be to remove the edge from this PAG and check whether  $a, d$  can now be  $d$ -separated by some conditioning set of size at most  $k$ . If yes, this changes the equivalence class, which means the edge must be there in any DAG. Our local rule  $\mathcal{R}12$  can instead orient this as an undirected edge, signifying that  $a$  and  $d$  must be adjacent in every underlying DAG by leveraging our equivalence class representation, without having to run this type of post-processing which might be computationally intensive for larger graphs.
E.1 Experiments vs. LOCI
(f) Skeleton  $F_{1}$  Score
Figure 11: Empirical cumulative distribution function of various  $F_{1}$  scores on 100 random DAGs on 10 nodes. We pick the parents of each node in a fixed total order randomly so that the expected value of parents is 3. The conditional distribution of every node given any configuration of its parent set is sampled independently, uniformly randomly from the corresponding dimensional probability simplex. Three datasets are sampled per instance. Performance of  $k$ -PC and LOCI [23] are similar as expected. We still observe a similar trend as PC that arrowhead score of  $k$ -PC is better. For tail accuracy, the result depends on the value of  $k$ . For small  $k$ , LOCI outperforms  $k$ -PC whereas for larger  $k$ ,  $k$ -PC outperforms LOCI in the small sample regime. Different from Figure 3, we compare the outputs to the true DAG instead of essential graph since no algorithm in this comparison can achieve essential graph.
E.2 Experiments vs. NOTEARS
(a) Arrowhead  $F_{1}$  Score
(d) Arrowhead  $F_{1}$  Score
Figure 12: Empirical cumulative distribution function of various  $F_{1}$  scores on 100 random DAGs on 5 nodes. For each DAG, a linear SCM is sampled as follows: Each coefficient is chosen randomly in the range  $[-3, 3]$ . Exogenous noise terms are jointly independent unit Gaussian. Performance of  $k$ -PC vs. NOTEARS [28]. We observe a similar trend as PC. NOTEARS is slightly better than PC consistently. Despite this,  $k$ -PC outperforms both in the low-sample regime. Metrics are computed against the true DAG.
(c) Skeleton  $F_{1}$  Score
(f) Skeleton  $F_{1}$  Score
E.3 More Experiments vs. PC
In this section, we show a larger range of  $N$  (number of samples). We also explore the behavior for graphs with different edge densities and higher number of nodes (10).
(f) Skeleton  $F_{1}$  Score
(g) Arrowhead  $F_{1}$  Score
(h) Tail  $F_{1}$  Score
(i) Skeleton  $F_{1}$  Score
(j) Arrowhead  $F_{1}$  Score
(k) Tail  $F_{1}$  Score
(1) Skeleton  $F_{1}$  Score
Figure 13: Empirical cumulative distribution function of various  $F_{1}$  scores on 100 random DAGs on 10 nodes. For each DAG, conditional probability tables are independently and uniformly randomly filled from the corresponding probability simplex. Three datasets are sampled per instance. The lower the curve the better. The maximum number of edges is 30. Even in the extreme case of just 10 samples (10 node-graphs),  $k$ -PC for  $k = 0$  provides improvement to all scores.  $k$  should be gradually increased as more samples are available to make best use of the available data. For example, for 1000 samples,  $k = 2$  provides the best arrowhead score while not giving up as much tail score as  $k = 0$ .
Next, we present combined metrics for this same setup. Namely, we show the advantage of our algorithm in terms of the sum of arrowhead and tail  $F_{1}$  scores, and the sum of arrowhead, tail and skeleton  $F_{1}$  scores.
E.4 Experiments of Section E.3
(a) Arrowhead  $+$  Tail  $F_{1}$
(c) Arrowhead  $+$  Tail  $F_{1}$
(e) Arrowhead  $+$  Tail  $F_{1}$
(g) Arrowhead  $+$  Tail  $F_{1}$
(b) Arrowhead  $+$  Tail  $^+$  Skeleton  $F_{1}$
(d) Arrowhead  $^+$  Tail  $^+$  Skeleton  $F_{1}$
(f) Arrowhead + Tail + Skeleton  $F_{1}$
(h) Arrowhead  $+$  Tail  $^+$  Skeleton  $F_{1}$
Figure 14: Results of the experiments in Section E.3 in terms of combined scores. For each instance, arrowhead and tail  $F_{1}$  scores are added before computing CDFs on the left. On the right, arrowhead, tail and skeleton  $F_{1}$  scores are added together.
E.5 Experiments vs. Conservative PC
We use pcalg package in R as the implementation for conservative PC [14].
(1) Skeleton  $F_{1}$  Score
Figure 15: Empirical cumulative distribution function of various  $F_{1}$  scores on 100 random DAGs on 10 nodes. For each DAG, conditional probability tables are independently and uniformly randomly filled from the corresponding probability simplex. One dataset is sampled per instance. The lower the curve the better. The maximum number of edges is 15.  $k$ -PC maintains an advantage against conservative PC in the arrowhead and skeleton  $F_{1}$  scores in the low-sample regime.