SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We considered the causal imitation learning problem when accounting for context-specific independence relations. We proved that in contrast to the classic problem, which is equivalent to a d-separation, the decision problem of imitability under CSIs is NP-hard. We established a link between these two problems. In particular, we proved that imitability under CSIs is equivalent to several instances of the classic imitability problem for a certain class of context variables. We showed that utilizing the overlooked notion of CSIs could be a worthwhile tool in causal imitation learning as an example of a fundamental AI problem from a causal perspective. We note that while taking a few CSI relations into account could result in significant achievable results, the theory of CSIs is not yet well-developed. In particular, there exists no complete algorithm for causal identification under CSIs. Further research on the theory of CSI relations could yield considerable benefits in various domains where such relations are present.
[1] Pieter Abbeel and Andrew Y Ng. Apprenticeship learning via inverse reinforcement learning. In Proceedings of the twenty-first international conference on Machine learning, page 1, 2004.
[2] Silvia Acid and Luis M. De Campos. An algorithm for finding minimum d-separating sets in belief networks. In Proceedings of the Twelfth International Conference on Uncertainty in Artificial Intelligence, UAI'96, page 3-10, San Francisco, CA, USA, 1996. Morgan Kaufmann Publishers Inc.
[3] Sina Akbari, Jalal Etesami, and Negar Kiyavash. Minimum cost intervention design for causal effect identification. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 258-289. PMLR, 2022.
[4] Sina Akbari, Luca Ganassali, and Negar Kiyavash. Causal discovery via monotone triangular transport maps. In NeurIPS 2023 Workshop Optimal Transport and Machine Learning, 2023.
[5] Sina Akbari, Fateme Jamshidi, Ehsan Mokhtarian, Matthew James Vowels, Jalal Etesami, and Negar Kiyavash. Causal effect identification in uncertain causal networks. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[6] Sina Akbari, Ehsan Mokhtarian, AmirEmad Ghassami, and Negar Kiyavash. Recursive causal structure learning in the presence of latent variables and selection bias. Advances in Neural Information Processing Systems, 34:10119-10130, 2021.
[7] Mayank Bansal, Alex Krizhevsky, and Abhijit Ogale. Chauffeurnet: Learning to drive by imitating the best and synthesizing the worst. arXiv preprint arXiv:1812.03079, 2018.
[8] Craig Boutilier, Nir Friedman, Moises Goldszmidt, and Daphne Koller. Context-specific independence in bayesian networks. In Proceedings of the Twelfth International Conference on Uncertainty in Artificial Intelligence, UAI'96, page 115-123, San Francisco, CA, USA, 1996. Morgan Kaufmann Publishers Inc.
[9] Felipe Codevilla, Eder Santana, Antonio M López, and Adrien Gaidon. Exploring the limitations of behavior cloning for autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9329-9338, 2019.
[10] Diego Colombo, Marloes H. Maathuis, Markus Kalisch, and Thomas S. Richardson. Learning high-dimensional directed acyclic graphs with latent and selection variables. The Annals of Statistics, pages 294–321, 2012.
[11] Hal Daumé, John Langford, and Daniel Marcu. Search-based structured prediction. Machine learning, 75(3):297-325, 2009.
[12] Pim De Haan, Dinesh Jayaraman, and Sergey Levine. Causal confusion in imitation learning. Advances in Neural Information Processing Systems, 32, 2019.
[13] Jalal Etesami and Philipp Geiger. Causal transfer for imitation learning and decision making under sensor-shift. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 10118-10125, 2020.
[14] Dan Geiger, Thomas Verma, and Judea Pearl. d-separation: From theorems to algorithms. In Machine Intelligence and Pattern Recognition, volume 10, pages 139-148. Elsevier, 1990.
[15] Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning. Advances in neural information processing systems, 29, 2016.
[34] David Silver, James Bagnell, and Anthony Stentz. High performance outdoor navigation from overhead data using imitation learning. Robotics: Science and Systems IV, Zurich, Switzerland, 1, 2008.
[35] Peter Spirtes, Clark N. Glymour, and D. Scheines, Richard and Heckerman. Causation, prediction, and search. MIT press, 2000.
[36] Gokul Swamy, Sanjiban Choudhury, Drew Bagnell, and Steven Wu. Causal imitation learning under temporally correlated noise. In International Conference on Machine Learning, pages 20877-20890. PMLR, 2022.
[37] Umar Syed and Robert E Schapire. A game-theoretic approach to apprenticeship learning. Advances in neural information processing systems, 20, 2007.
[38] Jin Tian, Azaria Paz, and Judea Pearl. Finding minimal d-separators, 1998.
[39] Santtu Tikka, Antti Hyttinen, and Juha Karvanen. Identifying causal effects via context-specific independence relations. Advances in neural information processing systems, 32, 2019.
[40] Benito Van der Zander, Maciej Liskiewicz, and Johannes Textor. Constructing separators and adjustment sets in ancestral graphs. In CI@ UAI, pages 11-24, 2014.
[41] Oriol Vinyals, Igor Babuschkin, Wojciech M Czarnecki, Michael Mathieu, Andrew Dudzik, Junyoung Chung, David H Choi, Richard Powell, Timo Ewalds, Petko Georgiev, et al. Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature, 575(7782):350-354, 2019.
[42] Bernard Widrow. Pattern-recognizing control systems. Computer and Information Sciences, 1964.
[43] Junzhe Zhang, Daniel Kumor, and Elias Baireinboim. Causal imitation learning with unobserved confounders. Advances in neural information processing systems, 33:12263-12274, 2020.
[44] Brian D. Ziebart, Andrew Maas, J. Andrew Bagnell, and Anind K. Dey. Maximum entropy inverse reinforcement learning. In Proceedings of the 23rd National Conference on Artificial Intelligence - Volume 3, AAAI'08, page 1433-1438. AAAI Press, 2008.
Proofs of the main results are given in Section A. The details of FindSep subroutine are presented in Section B. In Section C provides an example where Alg. 2 is not complete, and Section D includes the details of our experiment setup.
A Technical proofs
Proof. We first claim that for any set  $\mathbf{W} \subseteq \{X, Y\}$ , the set of ancestors of  $\mathbf{W}$  in  $\mathcal{G}$  and  $\mathcal{G}_X$  coincide, i.e.,  $\mathbf{An}(\mathbf{W})_{\mathcal{G}} = An(\mathbf{W})_{\mathcal{G}_X}$ . First, note that since  $\mathcal{G}_X$  is a subgraph of  $\mathbf{G}$ ,  $An(\mathbf{W})_{\mathcal{G}_X} \subseteq An(\mathbf{W})_{\mathcal{G}}$ . For the other direction, let  $T \in An(\mathbf{W})_{\mathcal{G}}$  be an arbitrary vertex. If  $T \in An(X)$ , then  $T \in An(\mathbf{W})_{\mathcal{G}_X}$ . Otherwise, there exists a directed path from  $T$  to  $Y$  that does not pass through  $X$ . The same directed path exists in  $\mathcal{G}_X$ , and as a result,  $T \in An(\mathbf{W})_{\mathcal{G}_X}$ . Therefore,  $An(\mathbf{W})_{\mathcal{G}} \subseteq An(\mathbf{W})_{\mathcal{G}_X}$ .
The rest of the proof follows from an application of Lemma 3.4 of [40] in  $\mathcal{G}_{\underline{X}}$ , with  $I = \emptyset$  and  $R = \mathbf{P}\mathbf{a}^{\Pi}$ .
Lemma 3.6. Given an  $LDAG\mathcal{G}^{\mathcal{L}}$  and a policy space  $\Pi$ , let  $Y$  be an arbitrary variable in  $\mathcal{G}^{\mathcal{L}}$ .  $P(y)$  is imitable w.r.t.  $\langle \mathcal{G}^{\mathcal{L}}, \Pi \rangle$  only if  $P(y)$  is imitable w.r.t.  $\langle \mathcal{G}_{\mathbf{w}}^{\mathcal{L}_{\mathbf{w}}}, \Pi \rangle$  for every realization  $\mathbf{w} \in \mathcal{D}_{\mathbf{w}}$  of every subset of variables  $\mathbf{W} \subseteq \mathbf{C}(\mathcal{L})$ .
Proof. Assume the contrary, that there exists a subset of variables  $\mathbf{W} \subseteq \mathbf{V} \setminus Y$  such that for an assignment  $\mathbf{w}_0 \in \mathcal{D}_{\mathbf{W}}$ ,  $P(y)$  is not imitable w.r.t.  $\langle \mathcal{G}_{\mathbf{w}_0}^{\mathcal{L}_{\mathbf{w}_0}}, \Pi \rangle$ . It suffices to show that  $P(y)$  is not imitable w.r.t.  $\langle \mathcal{G}^{\mathcal{L}}, \Pi \rangle$ . Since  $P(y)$  is not imitable w.r.t.  $\langle \mathcal{G}_{\mathbf{w}_0}^{\mathcal{L}_{\mathbf{w}_0}}, \Pi \rangle$ , there exists an SCM  $M' \in \mathcal{M}_{\langle \mathcal{G}_{\mathbf{w}_0}^{\mathcal{L}_{\mathbf{w}_0}} \rangle}$  such that  $P^{M'}(y|do(\pi)) \neq P^{M'}(y)$  for every  $\pi \in \Pi$ . Define
$$
\epsilon := \min  _ {\pi \in \Pi} \left| P ^ {M ^ {\prime}} (y) - P ^ {M ^ {\prime}} (y | d o (\pi)) \right|. \tag {4}
$$
Also, define
$$
\delta := \min  \left\{\frac {\epsilon}{4}, \frac {1}{2} \right\}. \tag {5}
$$
Note that  $\epsilon > 0$ , and  $0 < \delta < 1$ . Next, construct an SCM  $M$  over  $\mathbf{V}$  compatible with  $\mathcal{G}_{\mathbf{w}_0}^{\mathcal{L}_{\mathbf{w}_0}}$  as follows. For variables  $\mathbf{W}$ ,  $P^M(\mathbf{W} = \mathbf{w}_0) = 1 - \delta$ , and the rest is uniformly distributed over other realizations (summing up to  $\delta$ ), such that  $P^M(\mathbf{W} \neq \mathbf{w}_0) = \delta$ . For variables  $V_i \in \mathbf{V} \setminus \mathbf{W}$ ,  $P^M(V_i|\mathbf{P}\mathbf{a}(V_i)) = P^{M'}(V_i|\mathbf{P}\mathbf{a}(V_i))$ , i.e., they follow the same law as the model  $M'$ . Note that by construction,  $\mathbf{W}$  are isolated vertices in  $\mathcal{G}_{\mathbf{w}_0}^{\mathcal{L}_{\mathbf{w}_0}}$ , and therefore independent of every other variable in both  $M'$  and  $M$ . Therefore,
$$
\begin{array}{l} P ^ {M} (y) = \sum_ {\mathbf {w} \in \mathcal {D} _ {\mathbf {W}}} P ^ {M} (y | \mathbf {w}) P ^ {M} (\mathbf {w}) = \sum_ {\mathbf {w} \in \mathcal {D} _ {\mathbf {W}}} P ^ {M ^ {\prime}} (y | \mathbf {w}) P ^ {M} (\mathbf {w}) = \sum_ {\mathbf {w} \in \mathcal {D} _ {\mathbf {W}}} P ^ {M ^ {\prime}} (y) P ^ {M} (\mathbf {w}) \tag {6} \\ = P ^ {M ^ {\prime}} (y). \\ \end{array}
$$
Moreover, for any policy  $\pi_1\in \Pi$  dependant on the values of  $\mathbf{Z}'\subseteq \mathbf{Pa}^{\Pi}$ , define a policy  $\pi_2\in \Pi$  dependant on  $\mathbf{Z} = \mathbf{Z}'\setminus \mathbf{W}$  as  $\pi_2(X|\mathbf{Z}) = (1 - \delta)\pi_1(X|\mathbf{Z},\mathbf{W} = \mathbf{w}_0) + \delta \pi_1(X|\mathbf{Z},\mathbf{W}\neq \mathbf{w}_0)$ . Note
that  $\mathbf{Z} \cap \mathbf{W} = \emptyset$ . That is,  $\pi_2$  is independent of the values of  $\mathbf{W}$ . We can write
$$
\begin{array}{l} P ^ {M} (y | d o (\pi_ {1})) = \sum_ {\mathbf {w} \in \mathcal {D} _ {\mathbf {W}}} P ^ {M} (y | d o (\pi_ {1}), \mathbf {w}) P ^ {M} (\mathbf {w} | d o (\pi)) \\ = P ^ {M} (y \mid d o (\pi_ {1}), \mathbf {W} = \mathbf {w} _ {0}) (1 - \delta) + P ^ {M} (y \mid d o (\pi_ {1}), \mathbf {W} \neq \mathbf {w} _ {0}) \delta \\ = \sum_ {x, \mathbf {z}} (1 - \delta) P ^ {M} (y | d o (x), \mathbf {z}, \mathbf {W} = \mathbf {w} _ {0}) \pi_ {1} (x | \mathbf {z}, \mathbf {W} = \mathbf {w} _ {0}) P ^ {M} (\mathbf {z} | \mathbf {W} = \mathbf {w} _ {0}) \\ + \sum_ {x, \mathbf {z}} \delta P ^ {M} (y | d o (x), \mathbf {z}, \mathbf {W} \neq \mathbf {w} _ {0}) \pi_ {1} (x | \mathbf {z}, \mathbf {W} \neq \mathbf {w} _ {0}) P ^ {M} (\mathbf {z} | \mathbf {W} \neq \mathbf {w} _ {0}) \\ = \sum_ {x, \mathbf {z}} (1 - \delta) P ^ {M ^ {\prime}} (y | d o (x), \mathbf {z}) \pi_ {1} (x | \mathbf {z}, \mathbf {W} = \mathbf {w} _ {0}) P ^ {M ^ {\prime}} (\mathbf {z}) \tag {7} \\ + \sum_ {x, \mathbf {z}} \delta P ^ {M ^ {\prime}} (y | d o (x), \mathbf {z}) \pi_ {1} (x | \mathbf {z}, \mathbf {W} \neq \mathbf {w} _ {0}) P ^ {M ^ {\prime}} (\mathbf {z}) \\ = \sum_ {x, \mathbf {z}} P ^ {M ^ {\prime}} (y | d o (x), \mathbf {z}) \pi_ {2} (x | \mathbf {z}) P ^ {M ^ {\prime}} (\mathbf {z}) \\ = P ^ {M ^ {\prime}} \left(y \mid d o \left(\pi_ {2}\right)\right). \\ \end{array}
$$
That is, for any policy  $\pi_1$  under model  $M$ , there is a policy  $\pi_2$  that results in the same reward distribution under model  $M'$ . Therefore, combining Equations (6) and (7),
$$
\min _ {\pi \in \Pi} | P ^ {M} (y) - P ^ {M} (y | d o (\pi)) | = \min _ {\pi \in \Pi} | P ^ {M ^ {\prime}} (y) - P ^ {M ^ {\prime}} (y | d o (\pi)) |,
$$
although this minimum might occur under different policies. Recalling Equation (4), we get that under Model  $M$  and for any policy  $\pi \in \Pi$ ,
$$
\epsilon \leq \left| P ^ {M} (y) - P ^ {M} (y | d o (\pi)) \right|. \tag {8}
$$
Now we construct yet another SCM  $M^{\delta}$  over  $\mathbf{V}$  as follows. Variables  $\mathbf{W}$  are distributed as in model  $M$ , i.e.,  $P^{M^{\delta}}(\mathbf{W}) = P^{M}(\mathbf{W})$ . Moreover, for each variable  $V \in \mathbf{V} \setminus \mathbf{W}$  set  $V = V^{M}$  if  $\mathbf{Pa}(V) \cap \mathbf{W} = (\mathbf{w}_0)_{\mathbf{Pa}(V) \cap \mathbf{W}}$  where  $V^{M}$  denotes the same variable  $V$  under SCM  $M$ . Otherwise, distribution of  $V$  is uniform in  $\mathcal{D}_{V}$ . Note that by definition of  $M^{\delta}$ , the values of  $\mathbf{W}$  are assigned independently, and the values of all other variables  $(\mathbf{V} \setminus \mathbf{W})$  only depend on their parents, maintaining all the CSI relations. As a result,  $M^{\delta}$  is compatible with  $\mathcal{G}^{\mathcal{L}}$ . Also, by construction,
$$
P ^ {M ^ {\delta}} (\mathbf {O} \setminus \mathbf {W} | \mathbf {W} = \mathbf {w} _ {0}) = P ^ {M} (\mathbf {O} \setminus \mathbf {W}),
$$
$$
P ^ {M ^ {\delta}} (\mathbf {O} \backslash \mathbf {W} | d o (x), \mathbf {W} = \mathbf {w} _ {0}) = P ^ {M} (\mathbf {O} \backslash \mathbf {W} | d o (x)). \tag {9}
$$
Next, we write
$$
\begin{array}{l} P ^ {M ^ {\delta}} (y) = \sum_ {\mathbf {w} \in \mathcal {D} _ {\mathbf {w}}} P ^ {M ^ {\delta}} (y | \mathbf {w}) P ^ {M ^ {\delta}} (\mathbf {w}) \\ = P ^ {M ^ {\delta}} (y | \mathbf {W} = \mathbf {w} _ {0}) P ^ {M ^ {\delta}} (\mathbf {W} = \mathbf {w} _ {0}) \\ + P ^ {M ^ {\delta}} (y | \mathbf {W} \neq \mathbf {w} _ {0}) P ^ {M ^ {\delta}} (\mathbf {W} \neq \mathbf {w} _ {0}) \\ = (1 - \delta) P ^ {M ^ {\delta}} (y | \mathbf {W} = \mathbf {w} _ {0}) + \delta P ^ {M ^ {\delta}} (y | \mathbf {W} \neq \mathbf {w} _ {0}). \\ \end{array}
$$
The second term of the right-hand side above is a positive number not larger than  $\delta$ . Moreover, by construction of  $M^{\delta}$ , we have  $P^{M^{\delta}}(y|\mathbf{W} = \mathbf{w}_0) = P^M (y)$ . Therefore, we get
$$
(1 - \delta) P ^ {M} (y) \leq P ^ {M ^ {\delta}} (y) \leq (1 - \delta) P ^ {M} (y) + \delta . \tag {10}
$$
Figure 5: Reduction from 3-SAT to imitability.
On the other hand, for an arbitrary policy  $\pi \in \Pi$ , depending on the values of  $\mathbf{Z} \subseteq \mathbf{P}\mathbf{a}^{\Pi}$
$$
\begin{array}{l} P ^ {M ^ {\delta}} (y | d o (\pi)) = P ^ {M ^ {\delta}} (y | d o (\pi), \mathbf {W} = \mathbf {w} _ {0}) P ^ {M ^ {\delta}} (\mathbf {W} = \mathbf {w} _ {0} | d o (\pi)) \\ + P ^ {M ^ {\delta}} (y | d o (\pi), \mathbf {W} \neq \mathbf {w} _ {0}) P ^ {M ^ {\delta}} (\mathbf {W} \neq \mathbf {w} _ {0} | d o (\pi)) \\ = P ^ {M ^ {\delta}} (y | d o (\pi), \mathbf {W} = \mathbf {w} _ {0}) P ^ {M} (\mathbf {W} = \mathbf {w} _ {0}) \\ + P ^ {M ^ {\delta}} (y | d o (\pi), \mathbf {W} \neq \mathbf {w} _ {0}) P ^ {M} (\mathbf {W} \neq \mathbf {w} _ {0}) \\ = \sum_ {x, \mathbf {z}} P ^ {M ^ {\delta}} (y | d o (x), \mathbf {z}, \mathbf {W} = \mathbf {w} _ {0}) \pi (x | \mathbf {z}, \mathbf {w} _ {0}) P ^ {M ^ {\delta}} (\mathbf {z} | \mathbf {W} = \mathbf {w} _ {0}) P ^ {M} (\mathbf {W} = \mathbf {w} _ {0}) \\ + \delta P ^ {M ^ {\delta}} (y | d o (\pi), \mathbf {W} \neq \mathbf {w} _ {0}) \\ = \sum_ {x, \mathbf {z}} P ^ {M} (y | d o (x), \mathbf {z}, \mathbf {W} = \mathbf {w} _ {0}) \pi (x | \mathbf {z}, \mathbf {w} _ {0}) P ^ {M} (\mathbf {z} | \mathbf {W} = \mathbf {w} _ {0}) P ^ {M} (\mathbf {W} = \mathbf {w} _ {0}) \\ + \delta P ^ {M ^ {\delta}} (y | d o (\pi), \mathbf {W} \neq \mathbf {w} _ {0}) \\ = P ^ {M} (y | d o (\pi), \mathbf {W} = \mathbf {w} _ {0}) P ^ {M} (\mathbf {W} = \mathbf {w} _ {0}) + \delta P ^ {M ^ {\delta}} (y | d o (\pi), \mathbf {W} \neq \mathbf {w} _ {0}) \\ = P ^ {M} (y \mid d o (\pi), \mathbf {W} = \mathbf {w} _ {0}) P ^ {M} (\mathbf {W} = \mathbf {w} _ {0}) + P ^ {M} (y \mid d o (\pi), \mathbf {W} \neq \mathbf {w} _ {0}) P ^ {M} (\mathbf {W} \neq \mathbf {w} _ {0}) \\ - P ^ {M} (y | d o (\pi), \mathbf {W} \neq \mathbf {w} _ {0}) P ^ {M} (\mathbf {W} \neq \mathbf {w} _ {0}) + \delta P ^ {M ^ {\delta}} (y | d o (\pi), \mathbf {W} \neq \mathbf {w} _ {0}) \\ = P ^ {M} (y | d o (\pi)) - \delta \left(P ^ {M} (y | d o (\pi), \mathbf {W} \neq \mathbf {w} _ {0}) - P ^ {M ^ {\delta}} (y | d o (\pi), \mathbf {W} \neq \mathbf {w} _ {0})\right), \\ \end{array}
$$
and as a result,
$$
P ^ {M} (y | d o (\pi)) - \delta \leq P ^ {M ^ {\delta}} (y | d o (\pi)) \leq P ^ {M} (y | d o (\pi)) + \delta . \tag {11}
$$
Combining Equations (10) and (11), we get
$$
\begin{array}{l} \left(P ^ {M} (y) - P ^ {M} (y \mid d o (\pi))\right) - \delta P ^ {M} (y) - \delta \\ \leq P ^ {M ^ {\delta}} (y) - P ^ {M ^ {\delta}} (y | d o (\pi)) \leq \\ \left(P ^ {M} (y) - P ^ {M} (y \mid d o (\pi))\right) - \delta P ^ {M} (y) + \delta , \\ \end{array}
$$
and consequently,
$$
\left(P ^ {M} (y) - P ^ {M} (y | d o (\pi))\right) - 2 \delta \leq P ^ {M ^ {\delta}} (y) - P ^ {M ^ {\delta}} (y | d o (\pi)) \leq \left(P ^ {M} (y) - P ^ {M} (y | d o (\pi))\right) + 2 \delta ,
$$
which combined with Equation (8) results in the following inequality:
$$
\left| P ^ {M ^ {\delta}} (y) - P ^ {M ^ {\delta}} (y \mid d o (\pi)) \right| \geq \epsilon - 2 \delta = \frac {\epsilon}{2} > 0.
$$
To sum up, we showed that there exists model  $M^{\delta} \in \mathcal{M}_{\langle \mathcal{G}^{\mathcal{C}} \rangle}$ , such that for every  $\pi \in \Pi$ ,  $P^{M^{\delta}}(y|do(\pi)) \neq P^{M^{\delta}}(y)$  which is in contradiction with the imitability assumption. Hence,  $P(y)$  is imitable w.r.t.  $\langle \mathcal{G}_{\mathbf{w}}^{\mathcal{L}_{\mathbf{w}}}, \Pi \rangle$  for every  $\mathbf{w} \in \mathcal{D}_{\mathbf{W}}$  of any arbitrary subset of variables  $\mathbf{W} \subseteq \mathbf{V} \setminus Y$ .
Theorem 3.8. Given an LDAG  $\mathcal{G}^{\mathcal{L}}$  and a policy space  $\Pi$ , deciding the imitability of  $P(y)$  w.r.t.  $\langle \mathcal{G}^{\mathcal{L}}, \Pi \rangle$  is NP-hard.
Proof. Our proof exploits ideas similar to the proof of hardness of causal identification under CSIs by [39]. We show a polynomial-time reduction from the 3-SAT problem, which is a well-known NP-hard problem [20], to the imitability problem. Let an instance of 3-SAT be defined as follows. A formula  $F$  in the conjunctive normal form with clauses  $S_{1},\ldots ,S_{m}$  is given, where each clause  $S_{i}$  has 3 literals from the set of binary variables  $W_{1},\ldots ,W_{k}$  and their negations,  $\neg W_{1},\ldots ,\neg W_{k}$ . The decision problem of 3-SAT is to determine whether there is a realization of the binary variables  $W_{1},\ldots ,W_{k}$  that satisfies the given formula  $F$ , i.e., makes  $F$  true. We reduce this problem to an instance of imitability problem with CSIs as follows. Define an LDAG with one vertex corresponding to each clause  $S_{i}$  for  $1\leq i\leq m$ , one vertex  $\mathbf{W}$  which corresponds to the set of binary variables  $\{W_1,\dots ,W_k\}$ , and three auxiliary vertices  $S_0,X$ , and  $Y$ . Vertex  $\mathbf{W}$  is a parent of every  $S_{i}$  for  $1\leq i\leq m$ . There is an edge from  $S_{i - 1}$  to  $S_{i}$  for  $1\leq i\leq m$ . Also,  $S_{0}$  is a parent of  $X$ , and  $X$  and  $S_{m}$  are parents of  $Y$ . As for the labels, the label on each edge  $S_{i - 1}\rightarrow S_i$  is that this edge is absent if the assignment of  $\mathbf{W}$  does not satisfy the clause  $S_{i}$ . Constructing this LDAG is clearly polynomial-time, as it only requires time in the order of the number of variables  $\{W_1,\dots ,W_k\}$  and the number of clauses  $\{S_1,\dots ,S_m\}$ . We claim that  $P(y)$  is imitable in this LDAG if and only if the 3-SAT instance is unsatisfiable.
Only if part. Suppose that the 3-SAT instance is satisfiable. There exists a realization  $\mathbf{w}^*$  of the binary variables  $\{W_1, \ldots, W_k\}$  that satisfies the formula  $F$ . Define a model  $M$  over the variables of the LDAG as follows.  $\mathbf{W}$  has the value  $\mathbf{w}^*$  with probability  $0.6(P^M(\mathbf{W} = \mathbf{w}^*) = 0.6)$ , and the rest of the probability mass is uniformly distributed over other realizations of  $\mathbf{W}$  (i.e., each with probability  $\frac{0.4}{2^k - 1}$ ).  $S_0$  is a Bernoulli random variable with parameter  $\frac{1}{2}$ .  $S_i$  for  $1 \leq i \leq m$  is equal to  $S_{i-1}$  if the edge  $S_{i-1} \to S_i$  is present (the clause  $S_i$  is satisfied by the realization of  $\mathbf{W}$ ), and  $S_i = 0$  otherwise.  $X$  is equal to  $S_0$ , and  $Y$  is defined as  $Y = \neg X \oplus S_m$ . Note that model  $M$  is compatible with the LDAG defined above. We claim that under this model, for every policy  $\pi \in \Pi$ ,  $P^M(y|do(\pi)) \neq P^M(y)$ . That is,  $P(y)$  is not imitable.
Let  $\mathcal{D}_{\mathbf{W}}^s$  and  $\mathcal{D}_{\mathbf{W}}^u$  be the partitioning of the domain of  $\mathbf{W}$  into two disjoint parts which satisfy and do not satisfy the formula  $F$ , respectively. Note that  $\mathbf{w}^{*} \in \mathcal{D}_{\mathbf{W}}^{s}$ . For any realization  $\mathbf{w} \in \mathcal{D}_{\mathbf{W}}^{s}$ , observed values of  $Y$  in  $M$  is always equal to 1 (because  $Y = \neg S_0 \oplus S_0$ ), i.e.,  $P^{M}(Y = 1|\mathbf{w}) = 1$ . On the other hand, for any realization  $\mathbf{w} \in \mathcal{D}_{\mathbf{W}}^{u}$ , there exists a clause  $S_i$  which is not satisfied, and therefore  $S_{m} = 0$ , and  $Y = \neg X$ . Therefore,  $P^{M}(Y = 1|\mathbf{w}) = P^{M}(X = 0) = \frac{1}{2}$ . From the total probability law,
$$
\begin{array}{l} P ^ {M} (Y = 1) = \sum_ {\mathbf {w} \in \mathcal {D} _ {\mathbf {w}} ^ {s}} P (Y = 1 | \mathbf {w}) P (\mathbf {w}) + \sum_ {\mathbf {w} \in \mathcal {D} _ {\mathbf {w}} ^ {u}} P (Y = 1 | \mathbf {w}) P (\mathbf {w}) \\ = \sum_ {\mathbf {w} \in \mathcal {D} _ {\mathbf {w}} ^ {s}} P (\mathbf {w}) + \frac {1}{2} \sum_ {\mathbf {w} \in \mathcal {D} _ {\mathbf {w}} ^ {u}} P (\mathbf {w}) \\ = \frac {1}{2} \left(\sum_ {\mathbf {w} \in \mathcal {D} _ {\mathbf {w}} ^ {s}} P (\mathbf {w}) + \sum_ {\mathbf {w} \in \mathcal {D} _ {\mathbf {w}} ^ {u}} P (\mathbf {w})\right) + \frac {1}{2} \sum_ {\mathbf {w} \in \mathcal {D} _ {\mathbf {w}} ^ {s}} P (\mathbf {w}) \tag {12} \\ = \frac {1}{2} + \frac {1}{2} \sum_ {\mathbf {w} \in \mathcal {D} _ {\mathbf {w}} ^ {s}} P (\mathbf {w}) \geq \frac {1}{2} + \frac {1}{2} P (\mathbf {w} ^ {*}) = 0. 8, \\ \end{array}
$$
where we dropped the superscript  $M$  for better readability. Now consider an arbitrary policy  $\pi \in \Pi$ . For any realization  $\mathbf{w} \in \mathcal{D}_{\mathbf{W}}^s$ ,  $Y = \neg X \oplus S_0$ , where  $S_0$  is a Bernoulli variable with parameter  $\frac{1}{2}$  independent of  $X$ . As a result,  $P^M(Y = 1|do(\pi), \mathbf{w}) = P^M(S = X) = \frac{1}{2}$ . On the other hand, for any realization  $\mathbf{w} \in \mathcal{D}_{\mathbf{W}}^u$ , there exists a clause  $S_i$  which is not satisfied, and therefore  $S_m = 0$ , and  $Y = \neg X$ . Therefore,  $P^M(Y = 1|do(\pi), \mathbf{w}) = P^M(X = 0|do(\pi)) = \pi(X = 0)$ . Using the total probability law again, and noting that  $P^M(\mathbf{w}|do(\pi)) = P^M(\mathbf{w})$ ,
$$
\begin{array}{l} P ^ {M} (Y = 1 | d o (\pi)) = \sum_ {\mathbf {w} \in \mathcal {D} _ {\mathbf {w}} ^ {s}} P (Y = 1 | d o (\pi), \mathbf {w}) P (\mathbf {w} | d o (\pi)) \\ + \sum_ {\mathbf {w} \in \mathcal {D} _ {\mathbf {w}} ^ {u}} P (Y = 1 | d o (\pi), \mathbf {w}) P (\mathbf {w} | d o (\pi)) \\ = \frac {1}{2} \sum_ {\mathbf {w} \in \mathcal {D} _ {\mathbf {w}} ^ {s}} P (\mathbf {w}) + \pi (X = 0) \sum_ {\mathbf {w} \in \mathcal {D} _ {\mathbf {w}} ^ {u}} P (\mathbf {w}), \\ \end{array}
$$