SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof. For the sake of contradiction suppose, conditioned on  $c$ , there is no d-connecting path with an arrow into  $X$  and an arrow into  $Y$ . Since neither  $X$  is an ancestor of  $Y$  nor  $Y$  is an ancestor of  $X$ , it must be that all d-connecting paths have colliders on them. And all such colliders must be ancestors of  $T$ .
Consider such a path  $p$  where the edge adjacent to  $X$  has a tail at  $X$ . Let  $K$  be the collider that is closest to  $X$ .
$$
X \to U _ {1} \to \dots \to U _ {m} \to K \leftarrow V \dots Y
$$
for some  $\{U_i\}_{i \in m}, V$ . Since the path is open it must be that  $K \in \mathrm{An}(T)$ . Let  $T' = T - \mathrm{De}(K)$  where  $D e(K)$  are all descendants of  $K$ . Clearly,  $q$  is no longer open conditioned on  $T'$ . We investigate other open paths now, keeping in mind that  $X, Y$  being  $k$ -covered implies that  $X \not\perp Y | T'$  since  $|T'| \leq |T| \leq k$ .
Claim 1: Removing the descendants of the collider closest to  $X$  from the conditioning set can only add  $d$ -connecting paths that start with a tail at  $X$  but no  $d$ -connecting paths that start with an arrowhead at  $X$ .
Proof of Claim 1: Since  $X$  is not an ancestor of  $Y$ , by Lemma A.1, we know that removing the descendants of  $K$  from  $T$  can only introduce d-connecting paths that are out of  $X$ .
Now consider the d-connecting paths under conditioning on  $T'$ . We know that these paths must have a tail either at  $X$  or at  $Y$  since we started with such d-connecting paths by assumption and by Claim 1, removing  $De(K)$  from  $T$  can only introduce new d-connecting paths that have tails at  $X$ . Using the fact that no path that has an arrowhead into both endpoints are opened, we can use recursion and claim that we can make  $X, Y$  d-separated by removing descendants of colliders (that are closest to the endpoint that is adjacent to a tail) of active paths, which gives the following:
Claim 2: There exists a set  $T^{*}$  of size at most  $k$  such that  $X \perp Y | T^{*}$ , which leads to a contradiction since  $X, Y$  are  $k$ -covered by assumption.
Proof of Claim 2: Given claim 1, we can continue removing descendants of the colliders of the active paths that are closest to the tail-end node from the set  $T$ . Either no d-connecting path is left at some point in this process, or that we end up removing all the variables from the conditioning set. If the former, this is a contradiction since  $X, Y$  are  $k$ -covered. If the latter, then this is another contradiction due to the following: This means that given empty set, paths that have a tail adjacent to one of the endpoints, i.e., the paths with colliders on them (since all paths that have a tail adjacent to one of the endpoints must have a collider because  $X \notin An(Y)$  and  $Y \notin An(X)$ ) are d-connecting, which is not possible. This proves Claim 2.
Due to the symmetry between  $X, Y$ , the supposition that the only d-connecting paths must have a tail adjacent to either endpoint must be wrong, which proves the lemma.  $\square$
Lemma A.3. Consider a DAG  $D$  where  $X \notin \operatorname{An}(Y)$ ,  $X, Y$  are non-adjacent and  $k$ -covered. Then conditioned on any subset  $T: |T| \leq k$ , there exists a  $d$ -connecting path between  $X, Y$  that starts with an arrow into  $X$ .
Proof. For the sake of contradiction, suppose otherwise. Given  $T$ , all the d-connecting paths start with a tail at  $X$ . We will show that we can find some  $T'$  of size at most  $k$  that d-separates  $X, Y$  which lead to a contradiction since  $X, Y$  are assumed to be  $k$ -covered.
Consider any path  $q$  that is d-connecting given  $T$  which starts with a tail at  $X$ . Since  $X \notin An(Y)$ , by Lemma A.1 it must be that this path has at least one collider on it. Let  $K$  be the collider that is closest to  $X$ . Thus we have
for some  $\{U_i\}_i, V$ . Since the path is open, this collider cannot be blocking the path  $q$ . It must be that  $K \in An(T)$ . Let  $T' = T - De(K)$ , where  $De(K)$  are all descendants of  $K$ . Clearly,  $q$  is no longer open conditioned on  $T'$ . We investigate other open paths now, keeping in mind that  $X, Y$  being k-covered implies that  $X \not\perp Y | T'$  since  $|T'| \leq |T| \leq k$ .
Proof of Claim 1: Since  $X$  is not an ancestor of  $Y$ , by Lemma A.1 we know that removing the descendants of  $K$  from  $T$  can only introduce d-connecting paths that are out of  $X$ .
Now consider the d-connecting paths under conditioning on  $T'$ . We know that these paths must have a tail at  $X$  since we started with only such d-connecting paths by assumption and by Claim 1, removing  $De(K)$  from  $T$  can only introduce d-connecting paths that have tails at  $X$ . Using the fact that no path that has an arrowhead into  $X$ , we can use recursion and claim that we can make  $X, Y$  d-separated by removing descendants of colliders (that are closest to  $X$ ) of active paths, which gives the following:
Proof of Claim 2: Given claim 1, we can continue removing descendants of the first colliders of active paths that are closest to  $X$  from the set  $T$ . Either, no d-connecting path is left at some point in this process or that we end up removing all the variables from the conditioning set. If the former, this is a contradiction since  $X, Y$  are k-covered. If the latter, then this is another contradiction due to the following: This means that given empty set, paths that start with a tail and have colliders on them (as they cannot be directed and collider-free since  $X$  is not an ancestor of  $Y$ ) are d-connecting, which is not possible. This proves Claim 2.
Therefore, the supposition that all the d-connecting paths must have a tail adjacent to  $X$  must be wrong, which proves the lemma.
The above lemmas will be crucial in proving Lemma 3.6. Now consider a d-connecting path between  $x, z$  given  $c$  and a d-connecting path between  $z, y$  given  $c$ . We have the following lemma:
Lemma A.4. Let  $p$  be an active path between  $x, z$  given  $c$ , and  $q$  be an active path between  $z, y$  given  $c$ , where  $x, y, z \notin c$ . If  $x$  and  $y$  are  $d$ -separated given  $c$ , then
1. Paths  $p, q$  must have no overlapping nodes and
2.  $Y$  must be a collider along the concatenated path and  $Y \notin \operatorname{An}(c)$ .
Proof. We would like to allow the possibility that these paths might go through the same nodes. To address this, it helps to consider walks.
Definition A.5. A walk on a DAG is any sequence of edges.
Definition A.6. A path on a DAG is a sequence of edges where each node appears at most once.
There is a direct relation between active walks and d-connecting paths [8]. Indeed, this relation is leveraged to efficiently check dependence using paths, rather than having to search over all walks, which is a much larger space.
Definition A.7. A walk between two nodes  $a, b$  is called active given  $c$  if each collider along the walk is in  $c$  and each non-collider is not in  $c$ .
Definition A.8. A path between two nodes  $a, b$  is called open given  $c$  if each collider along the path is in  $An(c)$  and each non-collider is not in  $c$ .
Consider an active walk where a node  $t$  appears multiple times. Observe that  $t$  must appear with the same collider status, since otherwise the walk would not be active. If the appearance is of the form
$$
a \dots \xrightarrow {\alpha} t \rightarrow \dots \leftarrow t \xleftarrow {\beta} \dots b, \tag {9}
$$
then there must be a collider that is in  $c$  between the two appearances of  $t$ 's. We can skip the intermediate subpath between the two appearances of  $t$ 's to obtain the walk
$$
a \dots \xrightarrow {\alpha} t \xleftarrow {\beta} \dots b, \tag {10}
$$
Since there is at least one collider that is in  $c$  along the skipped subpath, we have that  $t \in An(c)$ . Therefore,  $t$  will not be blocking the path that is obtained after repeatedly applying this and other shortening steps. If the appearances are of the form:
$$
a \dots \xrightarrow {\alpha} t \rightarrow \dots \rightarrow t \xrightarrow {\beta} \dots b, \tag {11}
$$
we can similarly skip the subpath between the two appearances of  $t$ 's to obtain the shorter walk
$$
a \dots \xrightarrow {\alpha} t \xrightarrow {\beta} \dots b, \tag {12}
$$
and repeat this process until  $t$  is not repeated. The resulting walk/path is still open since  $t$  will appear in the same collider status, namely as a non-collider and if it was not blocking the walk, it will not be blocking the path either. This argument holds for any configuration where  $t$  is a non-collider. If the appearances is of the form:
$$
a \dots \xrightarrow {\alpha} t \leftarrow \dots \rightarrow t \xleftarrow {\beta} \dots b, \tag {13}
$$
then it must be that  $t \in c$ , and thus the walk obtained by skipping the subwalk between the two appearances of  $t$ 's, i.e.,
$$
a \dots \xrightarrow {\alpha} t \xleftarrow {\beta} \dots b, \tag {14}
$$
must be d-connecting.
This shows that each active walk corresponds to a d-connecting path and vice versa. Now we can proceed with the proof of the lemma:
1. Suppose  $p, q$  have overlapping nodes. Let  $w_{p}$  be the walk that corresponds to  $p$  and  $w_{q}$  be the walk that corresponds to  $q$ . Consider the concatenated walk  $w = w_{p}, w_{q}$ . If any repeated node has different collider status along  $w$ , then the path is not active. But this means that the node was blocking either  $w_{p}$  or  $w_{q}$ , which would be a contradiction. Therefore, repeated nodes cannot have different collider status along  $w$ .
Suppose a node  $t$  is repeated in  $w_{p}$  and  $w_{q}$  and has the same collider status. In this case, consider the walk obtained by concatenating the sub-walk of  $w_{p}$  between  $x$  and  $t$ , with the sub-walk of  $w_{q}$  from  $t$  to  $y$ . By repeating this process for any repeated node, we can obtain a path that corresponds to this walk, which would always be active since the repeated nodes have the same collider status along this path that they had in  $w_{p}$  or  $w_{q}$  and were not blocking these walks. Therefore, they cannot block the concatenated path obtained this way either, which is a contradiction since we are given that  $x, y$  are d-separated given  $c$ . Therefore if any node is repeated in  $w_{p}$  and  $w_{q}$  then the concatenated walk is always active. Thus, it must be the case that there is no repeated nodes.
2. Since there is no repeated nodes from  $I$ , we can operate at the path level instead of considering walks. Suppose  $Y$  is not a collider. Since  $Y \notin c$ , it would be d-connecting and thus the concatenating path would be d-connecting, a contradiction. Suppose  $Y$  is a collider but  $Y \in An(c)$ . In this case,  $Y$  would not block the concatenated path either, which is a contradiction. This establishes the result.
The next lemma shows that colliders that are closed in  $D$  must remain closed in the  $k$ -closure  $\mathcal{C}_k(D)$ .
Lemma A.9. If a collider is blocked in  $D$  conditioned on some  $c: |c| \leq k$  then it must also be blocked in  $C_k(D)$  conditioned on  $c$ .
Proof. Suppose  $(X \to Z \leftarrow Y)_D$  is a collider that is blocked given  $c$ . Thus, it must be that  $Z \notin An(c)$  in  $D$ . For the sake of contradiction, suppose that this collider is unblocked in  $\mathcal{C}_k(D)$ . Thus, it must be the case that  $Z \in An(c)$  in  $\mathcal{C}_k(D)$ . This means there is a new directed path from  $Z$  to  $c$  in  $\mathcal{C}_k(D)$ . If this path existed in  $D$ , the collider would be unblocked, which is a contradiction. Thus, at least one of the edges along this path must have been added during the construction of  $\mathcal{C}_k(D)$ . Consider the collection of edges on this path that do not exist in  $D$ . Note that by construction of  $\mathcal{C}_k(D)$ , a directed edge  $\alpha \to \beta$  is added between a  $k$ -covered pair  $\alpha, \beta$  only if there is a directed path from  $\alpha$  to  $\beta$ . Consider the path obtained by replacing the directed edge between any  $k$ -covered pair along this path with the corresponding directed path in  $D$ . The resulting directed path must be in  $D$ . This shows that there was at least one path already in  $D$  that implied  $Z \in An(c)$ , which is a contradiction. Therefore, any collider in  $p$  that is unblocked in  $\mathcal{C}_k(D)$  must also be unblocked in  $D$ .
Lemma A.10. The set of ancestors of any set  $c$  of nodes in  $\mathcal{C}_k(D)$  are identical to the set of ancestors of  $c$  in  $\mathcal{C}_k(D)$ .
Proof. Adding edges to a graph, directed or bidirected, cannot decrease the set of ancestors of any node. We only need to show that the set of ancestors in  $\mathcal{C}_k(D)$  is not larger than the set of ancestors in  $D$ .
Suppose otherwise: A node  $a \in An(c)$  in  $\mathcal{C}_k(D)$  but  $a \notin An(c)$  in  $D$ . This can only happen if a collection of edges added during the construction of  $\mathcal{C}_k(D)$  render  $a$  an ancestor of  $c$ . However, each such edge is added only if there is a directed path between its endpoints in  $D$ . Consider the path obtained by replacing each such added edge along the path that renders  $a$  an ancestor of  $c$  in  $\mathcal{C}_k(D)$  with the corresponding directed paths in  $D$ . This directed path must be in  $D$ , which means that  $a$  was an ancestor of  $c$  in  $D$  as well, which is a contradiction.
We are finally ready for the proof of Lemma 3.6.
Proof of Lemma 3.6:
Since no edge is removed during the construction of the  $k$ -closure, one direction immediately follows: If  $a \perp b|c$  in  $\mathcal{C}_k(D)$ , then  $a \perp b|c$  in  $D$ . The implication is clearly true for any set  $c$  of size at most  $k$  as well. Therefore we only need to show the other direction.
Suppose  $a \perp b | c$  in  $D$  where  $|c| \leq k$ . We will show that  $a \perp b | c$  in  $\mathcal{C}_k(D)$ . For the sake of contradiction, suppose otherwise. Then there must be a d-connecting path  $p$  between  $a, b$  given  $c$  in  $\mathcal{C}_k(D)$ . The length of any such path must be greater than 1 since otherwise, whether this edge already existed in  $D$  or it was added during the construction of  $\mathcal{C}_k(D)$ ,  $a, b$  must have been dependent given  $c$  in  $D$ , which is a contradiction. Since the orientation of the existing edges in  $D$  did not change in  $\mathcal{C}_k(D)$ , either this path did not exist in  $D$  or that it existed but it was blocked by some collider that is not in  $An(c)$  in  $D$ . The latter is not possible due to Lemma A.9, since any unblocked collider in  $\mathcal{C}_k(D)$  must also be unblocked in  $D$ . Thus, it must be that this d-connecting path did not exist in  $D$ .
Suppose  $p$  does not exist in  $D$ . At least one edge must have been added to form this path in  $\mathcal{C}_k(D)$  during the construction of  $\mathcal{C}_k(D)$ .
For any added edge  $u \to v$ , the following is true: Since  $u \to v$  was added in  $\mathcal{C}_k(D)$ , it must be the case that  $v \notin An(u)$  since otherwise there would be a cycle. By Lemma A.3, conditioned on  $c$ , there exists a d-connecting path between  $u, v$  where the edge adjacent to  $v$  is into  $v$ . For any added edge  $u \leftrightarrow v$ , the following is true: Since  $u \leftrightarrow v$  was added in  $\mathcal{C}_k(D)$ , it must be the case that  $u \notin An(v)$  and  $v \notin An(u)$ . By Lemma A.2, conditioned on  $c$ , there exists a d-connecting path between  $u, v$  where the edge adjacent to  $u$  is into  $u$  and the edge adjacent to  $v$  is into  $v$ . Call any such path implied by these lemmas a replacement path. Note that a replacement path might be directed or not.
Consider a path  $q$  in  $D$  that is obtained from  $p$  by switching the edges added during the construction of  $\mathcal{C}_k(D)$  with the replacement paths using the following policy: Suppose  $u \to v$  in  $\mathcal{C}_k(D)$  for some  $k$ -covered pair  $u, v$ . If a directed path is open given  $c$  in  $D$ , use that path as the replacement path
for the edge  $a \to b$ . If not, use any other path. This means that either the path that replaces an edge  $u \to v$  is directed or that both the endpoints have an arrowhead and that  $u \in An(c)$ .
Observe that each replacement path is d-connecting and the subpaths of  $p$  that remain intact in  $q$  must be d-connecting since  $p$  is d-connecting. By Lemma A.4, any two paths – whether it is a pair of replacement paths or a replacement path and a subpath of  $p$  – have overlapping nodes, then their concatenation must be d-connecting. Since we assumed that  $q$  was not d-connecting, it must be that one of the endpoints of one of the added edges must be blocking  $q$ . We investigate each such node to verify that  $q$  is indeed d-connecting to arrive at a contradiction.
In other words, the collider status of some of these nodes must have changed due to replacing some edges with replacement paths. Specifically due to Lemma A.4, one of the endpoints of replacement paths must be a collider and not an ancestor of  $c$ . Since ancestrality status cannot change from  $D$  to  $\mathcal{C}_k(D)$  due to Lemma A.10, the only way for  $q$  to not be d-connecting is if some node that is not an ancestor of  $c$  changes status from being a non-collider along  $p$  to being a collider along  $q$ .
Note that for an edge  $u \leftrightarrow v$ , the nodes  $u$  and  $v$  are adjacent to an arrowhead in the replacement path. Thus, if some node  $t$  changes collider status in  $q$  compared to  $p$ , it cannot be due to bidirected edges along  $p$ .
Now consider the directed edges  $u \to v$ . If the replacement path is directed from  $u$  to  $v$ , similarly  $u$  is adjacent to a tail and  $v$  is adjacent to an arrowhead on the replacement path. Therefore, such edges cannot alter the collider status of nodes at the junction of different paths. Finally consider the directed edges  $u \to v$  where  $u$  and  $v$  are both adjacent to an arrowhead on the replacement path. Observe that this edge cannot change the collider status of  $v$ . We now focus on  $u$ . If the other edge adjacent to  $u$  along  $q$  is a tail,  $u$  remains a non-collider and cannot block  $q$ . Now suppose the other edge adjacent to  $u$  along  $q$  is an arrowhead. This makes  $u$  a collider in  $q$  whereas  $u$  was a non-collar along  $p$  since we had  $u \to v$  along  $p$ . However, by construction of  $q$ , as we ended up adding a path with arrowheads at both endpoints, it must be that the directed path between  $u, v$  (which exists since  $u \to v$  was added during the construction of  $C_k(D)$ ) must be blocked via conditioning. This means  $u \in An(c)$  which means that although the status of  $u$  changes from non-collar to collider, it must be that this collider does not block  $q$  since it is an ancestor of  $c$ . Therefore, no replacement path can alter the status of a node to block the path  $q$ , and  $q$  must be d-connecting, which contradicts with the assumption that the path was not d-connecting in  $D$ .
This establishes that any d-connecting path in  $\mathcal{C}_k(D)$  is also d-connecting in  $D$ , which establishes that if  $a \not\perp b | c$  in  $\mathcal{C}_k(D)$ , then  $a \not\perp b | c$  in  $D$ . This establishes the lemma.
A.2 Proof of Lemma 3.8
For a mixed graph to be a maximal ancestral graph, we need to show that it does not have directed or almost directed cycles and that any non-adjacent pair of nodes can be made conditionally independent by conditioning on some subset of observed variables [24]. We first define almost directed cycle, and propose a lemma that shows that  $k$ -closure graphs do not have directed or almost directed cycles.
Definition A.11 ([24]). A directed path  $p$  from  $a$  to  $b$  and the edge  $a \leftrightarrow b$  is called an almost directed cycle.
Lemma A.12. For any DAG  $D$ , and integer  $k$ ,  $\mathcal{C}_k(D)$  does not have directed or almost directed cycles.
Proof. Suppose, for the sake of contradiction that there is a directed cycle in  $\mathcal{C}_k(D)$ . Since each edge  $X \to Y$  in  $\mathcal{C}_k(D)$  either exists in  $D$  or for each such edge in  $\mathcal{C}_k(D)$ , there is a directed path from  $X$  to  $Y$  in  $D$ , existence of a directed cycle in  $\mathcal{C}_k(D)$  would imply a directed cycle in  $D$ , which contradicts with the DAG assumption of  $D$ .
Suppose, for the sake of contradiction that there is an almost directed cycle in  $\mathcal{C}_k(D)$ , i.e., we have a directed path from  $a$  to  $b$  for two nodes  $a \leftrightarrow b$ . Since  $a \leftrightarrow b$  is added during construction of  $\mathcal{C}_k(D)$ , it must be the case that neither  $a$  nor  $b$  are ancestors of each other. However, from the above argument, there must be a directed path from  $a$  to  $b$  in  $D$ , which is a contradiction. Thus,  $\mathcal{C}_k(D)$  cannot have almost directed cycles.
The other condition for a mixed graph to be a maximal ancestral graph is that for any non-adjacent pair of nodes, there exists a subset of the observed variables that make them conditionally independent. For the  $k$ -closure graphs, this simply follows by construction: Any pair of nodes that are non-adjacent in  $\mathcal{C}_k(D)$  can be made conditionally independent given some set of size at most  $k$  in  $D$  by construction of  $\mathcal{C}_k(D)$ . From Lemma 3.6, this conditional independence relation must be retained in  $\mathcal{C}_k(D)$ . Thus any non-adjacent pair of nodes in  $\mathcal{C}_k(D)$  can be d-separated in  $\mathcal{C}_k(D)$  by some conditioning set of size at most  $k$ . This establishes the claim.
A.3 Proof of Theorem 3.9
Our main observation is that a parallel of Lemma A.2 works for MAGs with  $k$ -covered bidirected edges. The following lemmas are for any mixed graph  $\mathbb{K}$  that satisfies the constraints in Theorem 3.9, i.e., those that are MAGs and that satisfy the condition that for any bidirected edge  $a \leftrightarrow b$ ,  $a, b$  are  $k$ -covered in the graph  $\mathbb{K} - (a \leftrightarrow b)$ .
Lemma A.13. Suppose  $X \notin \mathcal{A}n(Y)$ . Suppose there is a  $d$ -connecting path  $p$  between  $X, Y$  given  $T$  that starts with an arrow out of  $X$ .
$$
X \rightarrow U _ {1} \rightarrow U _ {2} \dots \rightarrow U _ {m} \rightarrow K \leftarrow * \dots Y. \tag {15}
$$
* is a wildcard representing either an arrowhead or a tail.
Suppose for the sake of contradiction that conditioned on  $T'$ , there is a new d-connecting path  $q$  that starts with an arrowhead into  $X$ .  $q$  was clearly closed conditioned on  $T$  and became open by using removing nodes from the conditioning set  $T$ . This can only happen if we removed some node from  $T$  that is a non-collider along  $q$ . Consider the non-collar we removed that was closest to  $X$ , call this  $M$ . Thus we have the path  $q$  that looks like this:
$$
X \leftarrow * W \dots M \dots Y, \tag {16}
$$
where  $M$  is a non-collider on this path and is in  $De(K)$
We observe that the subpath between  $M$  and  $X$  cannot be directed from  $M$  to  $X$ . Because this would create the following cycle, since  $K$  is assumed to be the first collider along  $p$ , and an ancestor of  $M$ .
$$
M \rightarrow \dots \rightarrow X \rightarrow U _ {1} \dots U _ {m} \rightarrow K \rightarrow \dots \rightarrow M. \tag {17}
$$
$$
X \leftarrow W _ {1} \leftarrow W _ {2} \dots \leftarrow W _ {m ^ {\prime}} * \rightarrow V \dots M \dots Y \tag {18}
$$
We will consider the following two cases: The edge mark adjacent to  $M$  along the subpath between  $W_{m'}$  and  $M$  is a tail or an arrowhead.