SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Lemma 3.6. Given an  $LDAG\mathcal{G}^{\mathcal{L}}$  and a policy space  $\Pi$ , let  $Y$  be an arbitrary variable in  $\mathcal{G}^{\mathcal{L}}$ .  $P(y)$  is imitable w.r.t.  $\langle \mathcal{G}^{\mathcal{L}},\Pi \rangle$  only if  $P(y)$  is imitable w.r.t.  $\langle \mathcal{G}_{\mathbf{w}}^{\mathcal{L}_{\mathbf{w}}},\Pi \rangle$  for every realization  $\mathbf{w}\in \mathcal{D}_{\mathbf{w}}$  of every subset of variables  $\mathbf{W}\subseteq \mathbf{C}(\mathcal{L})$ .
For instance, a necessary condition for the imitability of  $P(y)$  in the graph of Figure 2a is that  $P(y)$  is imitable in both 2b and 2c. Consider the following special case of Lemma 3.6: if  $\mathbf{W} = \mathbf{C}(\mathcal{L})$ , then  $\mathcal{G}_{\mathbf{w}}^{\mathcal{L}\mathbf{w}} = \mathcal{G}_{\mathbf{w}}$  is a DAG, as  $\mathcal{L}_{\mathbf{w}} = \emptyset$  for every  $\mathbf{w} \in \mathcal{D}_{\mathbf{W}}$ . In essence, a necessary condition of imitability under CSIs can be expressed in terms of several classic imitability instances:
Corollary 3.7. Given an  $LDAG\mathcal{G}^{\mathcal{L}}$  and a policy space  $\Pi$ , let  $Y$  be an arbitrary variable in  $\mathcal{G}^{\mathcal{L}}$ .  $P(y)$  is imitable w.r.t.  $\langle \mathcal{G}^{\mathcal{L}},\Pi \rangle$  only if  $P(y)$  is imitable w.r.t.  $\langle \mathcal{G}_{\mathbf{c}},\Pi \rangle$ , i.e., there exists a  $\pi$ -backdoor admissible set w.r.t.  $\langle \mathcal{G}_{\mathbf{c}},\Pi \rangle$ , for every  $\mathbf{c}\in \mathcal{D}_{\mathbf{C}(\mathcal{L})}$ .
It is noteworthy that although the subgraphs  $\mathcal{G}_{\mathbf{c}}$  in Corollary 3.7 are defined in terms of realizations of  $\mathbf{C}(\mathcal{L})$ , the number of such subgraphs does not exceed  $2^{|E|}$ . This is due to the fact that  $\mathcal{G}_{\mathbf{c}}$ s share the same set of vertices, and their edges are subsets of the edges of  $\mathcal{G}^{\mathcal{L}}$ .
Formally, we say that a label  $l$  is compatible with a realization  $w$  if they are consistent; in the sense that the variables at the intersection of the label and the realization take on the same values under both assignments.
Although deciding the classic imitability is straightforward, the number of instances in Corollary 3.7 can grow exponentially in the worst case. However, in view of the following hardness result, a more efficient criterion in terms of computational complexity cannot be expected.
Theorem 3.8. Given an  $LDAG\mathcal{G}^{\mathcal{L}}$  and a policy space  $\Pi$ , deciding the imitability of  $P(y)$  w.r.t.  $\langle \mathcal{G}^{\mathcal{L}},\Pi \rangle$  is NP-hard.
This result places the problem of causal imitability under CSI relations among the class of NP-hard problems in the field of causality, alongside other challenges such as devising minimum-cost interventions for query identification [3] and discovering the most plausible graph for causal effect identifiability [5]. Although Theorem 3.8 indicates that determining imitability under CSIs might be intractable in general, as we shall see in the next section taking into account only a handful of CSI relations can render previously non-imitable instances imitable. Before concluding this section, we consider a special yet important case of the general problem. Specifically, for the remainder of this section, we assume that  $\mathbf{P}\mathbf{a}(\mathbf{C}(\mathcal{L}))\subseteq \mathbf{C}(\mathcal{L})$ . That is, the context variables have parents only among the context variables. Under this assumption, the necessary criterion of Corollary 3.7 turns out to be sufficient for imitability as well. More precisely, we have the following characterization.
Proposition 3.9. Given an  $LDAG\mathcal{G}^{\mathcal{L}}$  where  $\mathbf{Pa}(\mathbf{C}(\mathcal{L}))\subseteq \mathbf{C}(\mathcal{L})$  and a policy space  $\Pi$ , let  $Y$  be an arbitrary variable in  $\mathcal{G}^{\mathcal{L}}$ .  $P(y)$  is imitable w.r.t.  $\langle \mathcal{G}^{\mathcal{L}},\Pi \rangle$  if and only if  $P(y)$  is imitable w.r.t.  $\langle \mathcal{G}_{\mathbf{c}},\Pi \rangle$ , for every  $\mathbf{c}\in \mathcal{D}_{\mathbf{C}(\mathcal{L})}$ .
Algorithm 1 Imitation w.r.t.  $\langle \mathcal{G}^{\mathcal{C}},\Pi \rangle$
1: function IMitate1  $(\mathcal{G}^{\mathcal{L}},\Pi ,X,Y)$    
2: Compute  $\mathbf{C}\coloneqq \mathbf{C}(\mathcal{L})$  using Equation (1)   
3: for  $\mathbf{c}\in \mathcal{D}_{\mathbf{C}}$  do   
4: Construct a DAG  $\mathcal{G}_{\mathrm{c}}$  using Definition 3.5   
5: if FINDSEP  $(\mathcal{G}_{\mathbf{c}},\Pi ,X,Y)$  Fails then   
6: return FAIL   
7: else   
8:  $\mathbf{Z_c}\gets \mathrm{FINDSEP}(\mathcal{G_c},\Pi ,X,Y)$    
9:  $\pi_{\mathbf{c}}(X|\mathbf{pa}^{\Pi})\gets P(X|(\mathbf{pa}^{\Pi})_{\mathbf{z_c}})$    
10:  $\pi^{*}(X|\mathbf{pa}^{\Pi})\gets \sum_{\mathbf{c}\in \mathcal{D}_{\mathbf{C}}}\mathbb{1}\{(\mathbf{pa}^{\Pi})_{\mathbf{C}} = \mathbf{c}\} \pi_{\mathbf{c}}(X|\mathbf{pa}^{\Pi})$    
11: return  $\pi^{*}$
The proof of sufficiency, which is constructive, appears in Appendix A. The key insight here is that an optimal imitation policy is constructed based on the imitation policies corresponding to the instances  $\langle \mathcal{G}_{\mathbf{c}}, \Pi \rangle$ . In view of proposition 3.9, we provide an algorithmic approach for finding an optimal imitation policy under CSIs, as described in Algorithm 1. This algorithm takes as input an LDAG  $\mathcal{G}^{\mathcal{L}}$ , a policy space  $\Pi$ , an action variable  $X$  and a latent reward  $Y$ . It begins with identifying the context variables  $\mathbf{C} (= \mathbf{C}(\mathcal{L}))$ , defined by Equation (1) (Line 2). Next, for each realization  $\mathbf{c} \in \mathcal{D}_{\mathbf{C}}$ , the corresponding context-induced subgraph (Def. 3.5) is built (which is a DAG). If  $P(y)$  is not imitable in any of these DAGs, the algorithm fails, i.e., declares  $P(y)$  is not imitable in  $\mathcal{G}^{\mathcal{L}}$ . The imitability in each DAG is checked through a d-separation based on Lemma 3.3 (for further details of the function FindSep, see Algorithm 3 in Appendix B.) Otherwise, for each realization  $\mathbf{c} \in \mathcal{D}_{\mathbf{C}}$ , an optimal policy  $\pi_c$  is learned through the application of the  $\pi$ -backdoor admissible set criterion (line 9). If such a policy exists for every realization of  $\mathbf{C}$ ,  $P(y)$  is imitable w.r.t.  $\langle \mathcal{G}^{\mathcal{L}}, \Pi \rangle$  due to Proposition 3.9. An optimal imitating policy  $\pi^*$  is computed based on the previously identified policies  $\pi_{\mathbf{c}}$ . Specifically,  $\pi^*$ , the output of the algorithm, is defined as
$$
\pi^ {*} (X | \mathbf {p a} ^ {\Pi}) \leftarrow \sum_ {\mathbf {c} \in \mathcal {D} _ {\mathbf {C}}} \mathbb {1} \{(\mathbf {p a} ^ {\Pi}) _ {\mathbf {C}} = \mathbf {c} \} \pi_ {\mathbf {c}} (X | \mathbf {p a} ^ {\Pi}).
$$
Theorem 3.10. Algorithm 1 is sound and complete for determining the imitability of  $P(y)$  w.r.t.  $\langle \mathcal{G}^{\mathcal{L}}, \Pi \rangle$  and finding the optimal imitating policy in the imitable case, under the assumption that  $\mathbf{Pa}(\mathbf{C}(\mathcal{L})) \subseteq \mathbf{C}(\mathcal{L})$ .
Remark 3.11. Algorithm 1 requires assessing the  $d$ -separation of line 5 in the context-induced subgraphs  $\mathcal{G}_{\mathbf{c}}$ . Even when the context variables  $\mathbf{C}$  are continuous, the domain of these variables can
Figure 3: Two examples of leveraging CSI relations to achieve imitability.
be partitioned into at most  $2^{m}$  equivalence classes in terms of their corresponding context-induced subgraph, where  $m$  denotes the number of labeled edges. This holds since the number of context-induced subgraphs cannot exceed  $2^{m}$ . It is noteworthy, however, that solving the equation referred to in line 10 of Algorithm 2 for continuous variables may bring additional computational challenges.
Remark 3.12. Under certain assumptions, polynomial-time algorithms can be devised for deciding imitability. One such instance is when  $\mathcal{D}_{\mathbf{C}(\mathcal{L})} = \mathcal{O}\big(\log (|V|)\big)$ . Another analogous case is when the number of context-specific edges is  $\mathcal{O}\big(\log (|V|)\big)$ . Both of these lead polynomially many context-induced subgraphs in terms of  $|V|$ , which in turn implies that Alg. 1 runs in polynomial time.
4 Leveraging causal effect identifiability for causal imitation learning
Arguably, the main challenge in imitation learning stems from the latent reward. However, in certain cases, there exist observable variables  $\mathbf{S} \subseteq \mathbf{O}$  such that  $P(\mathbf{S}|do(\pi)) = P(\mathbf{S})$  implies  $P(y|do(\pi)) = P(y)$  for any policy  $\pi \in \Pi$ . Such  $\mathbf{S}$  is said to be an imitation surrogate for  $Y$  [43]. Consider, for instance, the graph of Figure 3a, where  $X$  represents the pricing strategy of a company,  $C$  is a binary variable indicating recession ( $C = 0$ ) or expansion ( $C = 1$ ) period,  $U$  denotes factors such as demand and competition in the market,  $S$  represents the sales and  $Y$  is the overall profit of the company. Due to Proposition 3.9,  $P(y)$  is not imitable in this graph. On the other hand, the sales figure ( $S$ ) is an imitation surrogate for the profit ( $Y$ ), as it can be shown that whenever  $P(S|do(\pi)) = P(S)$  for a given policy  $\pi$ ,  $P(y|do(\pi)) = P(y)$  holds for the same policy. Yet, according to do-calculus,  $P(S|do(\pi))$  itself is not identifiable due to the common confounding  $U$ . On the other hand, we note that the company's pricing strategy ( $X$ ) becomes independent of demand ( $U$ ) during a recession ( $C = 0$ ), as the company may not have enough customers regardless of the price it sets. This CSI relation amounts to the following identification formula for the effect of an arbitrary policy  $\pi$  on sales figures:
$$
P (s \mid d o (\pi)) = \sum_ {x, c} P (s \mid x, C = 0) \pi (x \mid c) P (c), \tag {3}
$$
where all of the terms on the right-hand side are known given the observations. Note that even though  $S$  is a surrogate in Figure 3a, without the CSI  $C = 0$ , we could not have written Equation (3). Given the identification result of this equation, if the set of equations  $P(s|do(\pi)) = P(s)$  has a solution  $\pi^{*}$ , then  $\pi^{*}$  becomes an imitation policy for  $P(y)$  as well [43]. It is noteworthy that solving the aforementioned linear system of equations for  $\pi^{*}$  is straightforward, for it boils down to a matrix inversion<sup>4</sup>. In the example discussed, although the graphical criterion of Proposition 3.9 does not hold, the data-specific parameters could yield imitability in the case that these equations are solvable. We therefore say  $P(y)$  is imitable w.r.t.  $\langle \mathcal{G}^{\mathcal{L}}, \Pi, P(\mathbf{O}) \rangle$ , as opposed to 'w.r.t.  $\langle \mathcal{G}^{\mathcal{L}}, \Pi \rangle$ . Precisely, we say  $P(y)$  is imitable w.r.t.  $\langle \mathcal{G}^{\mathcal{L}}, \Pi, P(\mathbf{O}) \rangle$  if for every  $M \in \mathcal{M}_{\langle \mathcal{G}^{\mathcal{L}} \rangle}$  such that  $P^{M}(\mathbf{O}) = P(\mathbf{O})$ , there exists a policy  $\pi$  such that  $P^{M}(y|do(\pi)) = P^{M}(y)$ .
The idea of surrogates could turn out to be useful to circumvent the imitability problem when the graphical criterion does not hold. In Figure 3b, however, neither graphical criteria yields imitability nor any imitation surrogates exist. In what follows, we discuss how CSIs can help circumvent the problem of imitability in even in such instances. Given an LDAG  $\mathcal{G}^{\mathcal{L}}$  and a context  $\mathbf{C} = \mathbf{c}$ , we denote the context-specific DAG w.r.t.  $\langle \mathcal{G}^{\mathcal{L}},\mathbf{c}\rangle$  by  $\mathcal{G}^{\mathcal{L}}(\mathbf{c})$  where  $\mathcal{G}^{\mathcal{L}}(\mathbf{c})$  is the DAG obtained by deleting all the spurious edges, i.e., the edges that are absent given the context  $\mathbf{C} = \mathbf{c}$ , from  $\mathcal{G}^{\mathcal{L}}$ .
4 In the discrete case, and kernel inversion in the continuous case.
Algorithm 2 Imitation w.r.t.  $\langle \mathcal{G}^{\mathcal{L}},\Pi ,P(\mathbf{O})\rangle$
1: function IMitate2  $(\mathcal{G}^{\mathcal{L}},\Pi ,X,Y,P(\mathbf{O}))$    
2: if SUBMITATE  $(\mathcal{G}^{\mathcal{L}},\Pi ,X,Y,\emptyset ,\emptyset ,P(\mathbf{O}))$  then   
3:  $\pi^{*}\gets \mathrm{SUBMITATE}(\mathcal{G}^{\mathcal{L}},\Pi ,X,Y,\emptyset ,\emptyset ,P(\mathbf{O}))$    
4: return  $\pi^{*}$    
5: else   
6: return Fail   
1: function SUBMITATE  $(\mathcal{G}^{\mathcal{L}},\Pi ,X,Y,\mathbf{C},\mathbf{c},P(\mathbf{O}))$    
2: Construct the context-specific DAG  $\mathcal{G}^{\mathcal{L}}(\mathbf{c})$    
3: if FINDSEP  $(\mathcal{G}^{\mathcal{L}}(\mathbf{c}),\Pi ,X,Y)$  then   
4:  $Z_{c}\gets \mathrm{FINDSEP}(\mathcal{G}^{\mathcal{L}}(\mathbf{c}),\Pi ,X,Y)$    
5:  $\pi_{\mathrm{c}}(X|\mathrm{Pa}^{\Pi})\gets P(X|\mathrm{Z}_{\mathrm{c}},\mathrm{C} = \mathrm{c})$    
6: return  $\pi_{\mathrm{c}}$    
7: else if FINDMINSEP  $(\mathcal{G}^{\mathcal{L}}(\mathbf{c})\cup \Pi ,X,Y,\mathbf{C})$  then   
8:  $S_{c}\gets F i n d M I N S E P (\mathcal {G} ^ {\mathcal {L}} (\mathbf {c}) \cup \Pi , X, Y, C) \backslash C$    
9: if CSI-ID  $(P(S_{c}|do(x),C = c))$  then   
10: Solve  $P(\mathbf{S_c}|do(\pi_c),c) = P(\mathbf{S_c}|c)$  for  $\pi_{\mathrm{c}}\in \Pi$    
11: if such  $\pi_{\mathrm{c}}$  exists then   
12: return  $\pi_{\mathrm{c}}$    
13: else   
14: if  $C = C(L)\cap Pa^{\Pi}$  then   
15: return FAIL   
16: else   
17: Choose  $V\in \mathbf{C}(\mathcal{L})\cap \mathbf{Pa}^{\Pi}\setminus \mathbf{C}$  arbitrarily   
18: for  $v\in \mathcal{D}_V$  do   
19:  $C^\prime \leftarrow C\cup \{V\}$ $c^{\prime}\gets c\cup \{v\}$    
20: if SI  $(\mathcal{G}^{\mathcal{L}},\Pi ,X,Y,\mathbf{C}^{\prime},\mathbf{c}^{\prime},P(\mathbf{O}))$  then   
21:  $\pi_{\mathrm{c},v}\gets \mathrm{SI}(\mathcal{G}^{\mathcal{L}},\Pi ,X,Y,\mathbf{C}^{\prime},\mathbf{c}^{\prime},P(\mathbf{O}))$    
22: else   
23: return Fail   
24:  $\pi_{\mathrm{c}}(X|\mathrm{Pa}^{\Pi})\gets \sum v\in \mathcal{D}_{V}\mathbb{1}_{\{(P a^{\Pi})_{V} = v\}}\pi_{\mathrm{c},v}$    
25: return  $\pi_{\mathrm{c}}$
Definition 4.1 (Context-specific surrogate). Given an  $LDAG\mathcal{G}^{\mathcal{L}}$ , a policy space  $\Pi$ , and a context  $\mathbf{c}$ , a set of variables  $\mathbf{S} \subseteq \mathbf{O}$  is called context-specific surrogate w.r.t.  $\langle \mathcal{G}^{\mathcal{L}}(\mathbf{c}), \Pi \rangle$  if  $X \perp Y|\{\mathbf{S}, \mathbf{C}\}$  in  $\mathcal{G}^{\mathcal{L}}(\mathbf{c}) \cup \Pi$  where  $\mathcal{G}^{\mathcal{L}}(\mathbf{c}) \cup \Pi$  is a supergraph of  $\mathcal{G}^{\mathcal{L}}(\mathbf{c})$  by adding edges from  $\mathbf{P}\mathbf{a}^{\Pi}$  to  $X$ .
Consider the LDAG of Figure 3b for visualization. The context-specific DAGs corresponding to contexts  $C = 0$  and  $C = 1$  are shown in Figures 3c and 3d, respectively.  $S$  is a context-specific surrogate with respect to  $\langle \mathcal{G}^{\mathcal{L}}(C = 0),\Pi \rangle$ , while no context-specific surrogates exist with respect to  $\langle \mathcal{G}^{\mathcal{L}}(C = 1),\Pi \rangle$ . The following result indicates that despite the absence of a general imitation surrogate, the existence of context-specific surrogates could suffice for imitation.
Proposition 4.2. Given an  $LDAG\mathcal{G}^{\mathcal{L}}$  and a policy space  $\Pi$ , let  $\mathbf{C}$  be a subset of  $\mathbf{P}\mathbf{a}^{\Pi} \cap \mathbf{C}(\mathcal{L})$ . If for every realization  $\mathbf{c} \in \mathcal{D}_{\mathbf{C}}$  at least one of the following holds, then  $P(y)$  is imitable w.r.t.  $\langle \mathcal{G}^{\mathcal{L}}, \Pi, P(\mathbf{O}) \rangle$ .
- There exists a subset  $\mathbf{S}_{\mathbf{c}}$  such that  $X \perp Y|\{\mathbf{S}_{\mathbf{c}}, \mathbf{C}\}$  in  $\mathcal{G}^{\mathcal{L}}(\mathbf{c}) \cup \Pi$ , and  $P(\mathbf{S}_{\mathbf{c}}|do(\pi), \mathbf{C} = \mathbf{c}) = P(\mathbf{S}_{\mathbf{c}}|\mathbf{C} = \mathbf{c})$  has a solution  $\pi_{\mathbf{c}} \in \Pi$ .
- There exists  $\mathbf{Z}_c \subseteq \mathbf{Pa}^{\Pi}$  such that  $X \perp Y|\{\mathbf{Z}_{\mathbf{c}}, \mathbf{C}\}$  in  $\mathcal{G}^{\mathcal{L}}(\mathbf{c})_{\underline{X}}$ .
As a concrete example, defining  $\mathbf{Z}_1 = \emptyset$ ,  $X \perp Y|\{\mathbf{Z}_1, C\}$  holds in the graph of Figure 3d. Moreover,  $\mathbf{S}_0 = \{S\}$ , is a context-specific surrogate w.r.t.  $\langle \mathcal{G}^{\mathcal{L}}(C = 0), \Pi \rangle$ . As a consequence of Proposition 4.2,  $P(y)$  is imitable w.r.t.  $\langle \mathcal{G}^{\mathcal{L}}, \Pi, P(\mathbf{O}) \rangle$ , if a solution  $\pi_0$  exists to the linear set of equations
$$
P (S | d o (\pi), C = 0) = P (S | C = 0),
$$
where  $P(s|do(\pi), C = 0) = \sum_{x,t} P(s|x, T = 0, C = 0)\pi(x|t, C = 0)P(t|C = 0)$ , analogous to Equation (3).
Table 1: Results pertaining to the model of Figure 3a.
Metric	Algorithm
Expert	Naive 1	Naive 2	Algorithm 2
E[Y]	1.367	1.194	1.193	1.358
DKL(P(Y)||P(Y|do(πALG))	0	0.0217	0.0219	0.0007
DKL(πALG(X|T=0)||ˆALG(X|T=0))	NA	2.3 × 10-5	4.4 × 10-6	1.3 × 10-3
DKL(πALG(X|T=1)||ˆALG(X|T=1))	NA	2.3 × 10-5	4.8 × 10-4	1.3 × 10-3
To sum up, accounting for CSIs has a two-fold benefit: (a) context-specific surrogates can be leveraged to render previously non-imitable instances imitable, and (b) identification results can be derived for imitation surrogates that were previously non-identifiable.
In light of Proposition 4.2, an algorithmic approach for causal imitation learning is proposed, summarized as Algorithm 2. This algorithm calls a recursive subroutine, SubImitate, also called  $SI$  within the pseudo-code. It is noteworthy that Proposition 4.2 guarantees imitability if the two conditions are met for any arbitrary subset of  $\mathbf{C}(\mathcal{L})\cap \mathbf{P}\mathbf{a}^{\Pi}$ . As we shall see, Algorithm 2 utilizes a recursive approach for building such a subset so as to circumvent the need to test all of the possibly exponentially many subsets.
The subroutine  $SI$  is initiated with an empty set  $(\mathbf{C} = \emptyset)$  as the considered context variables at the first iteration. At each iteration, the realizations of  $\mathbf{C}$  are treated separately. For each such realization  $\mathbf{c}$ , if the second condition of Proposition 4.2 is met through a set  $\mathbf{Z}_{\mathbf{c}}$ , then  $P(X|\mathbf{Z}_{\mathbf{c}},\mathbf{C} = \mathbf{c})$  is returned as the context-specific imitating policy (lines 3-6). Otherwise, the search for a context-specific surrogate begins. We utilize the FindMinSep algorithm of [40] to identify a minimal separating set  $\mathbf{S}_{\mathbf{c}}\cup \mathbf{C}$  for  $X$  and  $Y$ , among those that necessarily include  $\mathbf{C}$  (lines 7-8). We then use the identification algorithm of [39] under CSI relations to identify the effect of an arbitrary policy on  $\mathbf{S}_{\mathbf{c}}$ , conditioned on the context  $\mathbf{c}$ . This algorithm is built upon CSI-calculus, which subsumes do-calculus<sup>5</sup>. Next, if the linear system of equations  $P(\mathbf{S}_{\mathbf{c}}|do(\pi_{\mathbf{c}}),\mathbf{c}) = P(\mathbf{S}_{\mathbf{c}}|,\mathbf{c})$  has a solution, then this solution is returned as the optimal policy (lines 9-12). Otherwise, an arbitrary variable  $V\in \mathbf{C}(\mathcal{L})\cap \mathbf{Pa}^{\Pi}\setminus \mathbf{C}$  is added to the considered context variables, and the search for context-specific policies proceeds while taking the realizations of  $V$  into account (lines 17-24). If no variables are left to add to the set of context variables (i.e.,  $\mathbf{C} = \mathbf{C}(\mathcal{L})\cap \mathbf{Pa}^{\Pi}$ ) and neither of the conditions of Proposition 4.2 are met for a realization of  $\mathbf{C}$ , then the algorithm stops with a failure (lines 14-15). Otherwise, an imitating policy  $\pi^{*}$  is returned. We finally note that if computational costs matter, the CSI-ID function of line 9 can be replaced by the ID algorithm of [33]. Further, the minimal separating sets of line 8 might not be unique, in which case all such sets can be used.
Theorem 4.3. Given an  $LDAG\mathcal{G}^{\mathcal{L}}$ , a policy space  $\Pi$  and observational distribution  $P(\mathbf{O})$ , if Algorithm 2 returns a policy  $\pi^{*}$ , then  $\pi^{*}$  is an optimal imitating policy for  $P(y)$  w.r.t.  $\langle \mathcal{G}^{\mathcal{L}},\Pi ,P(\mathbf{O})\rangle$ . That is, Algorithm 2 is sound<sup>6</sup>.
Our experimental evaluation is organized into two parts. In the first part, we address the decision problem pertaining to imitability. We evaluate the gain resulting from accounting for CSIs in rendering previously non-imitable instances imitable. In particular, we assess the classic imitability v.s. imitability under CSIs for randomly generated graphs. In the second part, we compare the performance of Alg. 2 against baseline algorithms on synthetic datasets (see Sec. D for further details of our experimental setup). Python implementation are accessible at https://github.com/SinaAkbarii/causal-imitation-learning/.
5.1 Evaluating imitability
We sampled random graphs with  $n$  vertices and maximum degree  $\Delta = \frac{n}{10}$  uniformly at random. Each variable was assumed to be latent with probability  $\frac{1}{6}$ . We chose 3 random context variables
<sup>5</sup>Figure 3a is an example where do-calculus fails to identify  $P(s|do(x))$ , whereas CSI-calculus provides an identification formula, Equation (3).
See Section C for an example where Algorithm 2 is not complete.
Figure 4: The fraction of immutable instances (in the classical sense) vs those that are immutable considering CSIs.
such that the graphical constraint  $\mathbf{Pa}(\mathbf{C}(\mathcal{L}))\subseteq \mathbf{C}(\mathcal{L})$  is satisfied. Labels on the edges were sampled with probability 0.5. We then evaluated the graphical criterion of classic imitability (existence of  $\pi$ -backdoor) and imitability with CSIs (Corollary 3.7). The results are depicted in Figure 4, where each point in the plot is an average of 100 sampled graphs. As seen in this figure, taking into account only a handful of CSI relations (in particular, 3 context variables among hundreds of variables) could significantly increase the fraction of imitable instances.
5.2 Performance evaluation
In this section, we considered the graph of Figure 3b as a generalization of the economic model of Figure 3a, where the reward variable  $Y$  can be a more complex function. As discussed earlier, this graph has neither a  $\pi$ -backdoor admissible set nor an imitation surrogate. However, given the identifiability of  $P(S|do(\pi), C = 0)$ , Algorithm 2 can achieve an optimal policy. We compared the performance of the policy returned by Algorithm 2 against two baseline algorithms: Naive algorithm 1, which mimics only the observed distribution of the action variable (by choosing  $\pi(X) = P(X)$ ), and Naive algorithm 2, which takes the causal ancestors of  $X$  into account, designing the policy  $\pi(X|T) = P(X|T)$ . Naive algorithm 2 can be thought of as a feature selection followed by a behavior cloning approach. The goal of this experiment was to demonstrate the infeasibility of imitation learning without taking CSIs into account. A model with binary observable variables and a ternary reward was generated. Let  $\pi_{ALG}$  represent the policy that the algorithm would have learned with infinite number of samples, and  $\hat{\pi}_{ALG}$  the policy it learns with the given finite sample size. As can be seen in Table 1, Algorithm 2 was able to match the expert policy both in expected reward and KL divergence of the reward distribution. The naive algorithms, on the other hand, failed to get close to the reward distribution of the expert. Since the algorithms were fed with finite observational samples, the KL divergence of the estimated policies with the nominal policy is also reported. Notably, based on the reported measures, the undesirable performance of the Naive algorithms does not stem from estimation errors.