SlowGuess/NeurIPS_2023_OCR huggingface dataset:
As motivated in § 2.2, given data from  $\{P_X^e\}_{e\in \mathcal{E}}$  we consider candidate solutions of the form  $(h,G^{\prime})$  where  $h$  is an unmixing function, or encoder, which maps observations  $X$  to the inferred latents  $Z = h(X)$ , and  $G^{\prime}$  is causal graph capturing the relations among the  $Z_{i}$ . The corresponding distributions of the inferred latents are thus given by the push-forward
$$
Q _ {\boldsymbol {Z}} ^ {e} = h _ {*} (P _ {\boldsymbol {X}} ^ {e}) = (h \circ f) _ {*} P _ {\boldsymbol {V}} ^ {e}
$$
The multi-environment setup with unknown single-node perfect interventions is illustrated in Fig. 2.
One could also say that there exists only one environment and each dataset corresponds to an intervention.
We start by showing that identifiability up to  $\sim_{\mathrm{cRL}}$  is, in fact, the best we can hope for when learning from interventional data, without any more direct forms of supervision. To this end, we state the following result, which is presented more formally and proven in Appx. B.
Proposition 3.1 (Minimality of  $\sim_{\mathrm{CRL}}$ ; informal). Let  $Z$  be any representation that is  $\sim_{\mathrm{CRL}}$  equivalent to  $V$ , with  $G' = \pi(G)$  the associated DAG. Then for any intervention on  $V_{\mathcal{I}} \subseteq V$  in  $G$ , there exists an equally sparse intervention on  $Z_{\pi(\mathcal{I})} \subseteq Z$  in  $G'$  inducing the same observed distribution on  $X$ .
We now present our identifiability results. We first study the most fundamental bivariate case with two latent causal variables  $V_{1}$  and  $V_{2}$ . This can loosely be seen as the CRL analogue of the widely studied cause-effect problem ( $X \to Y$  or  $Y \to X$ )? in classical structure learning [88, 100].
Theorem 3.2 (Bivariate identifiability up to  $\sim_{\mathrm{CRL}}$  from one perfect stochastic intervention per node). Suppose that we have access to multiple environments  $\{P_X^e\}_{e\in \mathcal{E}}$  generated as described in § 2 under Asms. 2.2, 2.5, 2.8 and 2.9 with  $n = 2$ . Let  $(h,G')$  be any candidate solution such that the inferred latent distributions  $Q_Z^e = h_*(P_X^e)$  of  $Z = h(X)$  and the inferred mixing function  $h^{-1}$  satisfy the above assumptions w.r.t. the candidate causal graph  $G'$ . Assume additionally that
(A1) all densities  $p^e$  and  $q^e$  are continuously differentiable and fully supported on  $\mathbb{R}^n$ ;
(A2) we have access to a known observational environment  $e_0$  and one single node perfect intervention for each node, with unknown targets: there exist  $n + 1$  environments  $\mathcal{E} = \{e_i\}_{i=0}^n$  such that  $\mathcal{I}^{e_0} = \emptyset$  and for each  $i \in [n]$  we have  $\mathcal{I}^{e_i} = \{\pi(i)\}$  for an unknown permutation  $\pi$  of  $[n]$ ;
(A3) for all  $i \in [n]$ , the intervened mechanisms  $\tilde{p}_i(v_i)$  differ from the corresponding base mechanisms  $p_i(v_i \mid v_{\mathrm{pa}(i)})$  everywhere, in the sense that
$$
\forall \boldsymbol {v}: \quad \frac {\partial}{\partial v _ {i}} \frac {\tilde {p} _ {i} \left(v _ {i}\right)}{p _ {i} \left(v _ {i} \mid \boldsymbol {v} _ {\mathrm {p a} (i)}\right)} \neq 0; \tag {3.1}
$$
(A4) ("genericity") the base and intervened mechanisms are not fine-tuned to each other, in the sense that there exists a continuous function  $\varphi : \mathbb{R}^{+} \to \mathbb{R}$  for which
$$
\mathbb {E} _ {\boldsymbol {v} \sim P _ {V} ^ {e _ {0}}} \left[ \varphi \left(\frac {\tilde {p} _ {2} (v _ {2})}{p _ {2} \left(v _ {2} \mid v _ {1}\right)}\right) \right] \neq \mathbb {E} _ {\boldsymbol {v} \sim P _ {V} ^ {e _ {1}}} \left[ \varphi \left(\frac {\tilde {p} _ {2} (v _ {2})}{p _ {2} \left(v _ {2} \mid v _ {1}\right)}\right) \right] \tag {3.2}
$$
Then the ground truth is identified in the sense of Defn. 2.6, that is,  $(f^{-1}, G) \sim_{\mathrm{cRL}} (h, G')$ .
Proof sketch (full proof in Appx. C.2). Consider  $V_{1} \to V_{2}$  (the proof for  $V_{1} \nrightarrow V_{2}$  is similar). Let  $\psi = f^{-1} \circ h^{-1}: \mathbb{R}^n \to \mathbb{R}^n$  such that  $\pmb{V} = \psi(\pmb{Z})$ . By the change of variable formula, for all  $z$
$$
q ^ {e} (\boldsymbol {z}) = p ^ {e} (\psi (\boldsymbol {z})) | \det  \boldsymbol {J} _ {\psi} (\boldsymbol {z}) | \tag {3.3}
$$
where  $(J_{\psi}(z))_{ij} = \frac{\partial\psi_i}{\partial z_j} (z)$  denotes the Jacobian of  $\psi$ . We consider two separate cases, depending on whether the intervention targets in  $q^{e_i}$  for  $e_i\in \{e_1,e_2\}$  match those in  $p^{e_i}$  (Case 1) or not (Case 2).
Case 1: Aligned Intervention Targets. According to Asm. 2.8 and (A2), (3.3) applied to the known observational environment  $e_0$  and the interventional environments  $e_1, e_2$  leads to the system of equations:
$$
q _ {1} \left(z _ {1}\right) q _ {2} \left(z _ {2} \mid z _ {\mathrm {p a} \left(2; G ^ {\prime}\right)}\right) = p _ {1} \left(\psi_ {1} (\boldsymbol {z})\right) p _ {2} \left(\psi_ {2} (\boldsymbol {z}) \mid \psi_ {1} (\boldsymbol {z})\right) | \det  \boldsymbol {J} _ {\psi} (\boldsymbol {z}) | \tag {3.4}
$$
$$
\tilde {q} _ {1} \left(z _ {1}\right) q _ {2} \left(z _ {2} \mid z _ {\mathrm {p a} \left(2; G ^ {\prime}\right)}\right) = \tilde {p} _ {1} \left(\psi_ {1} (z)\right) p _ {2} \left(\psi_ {2} (z) \mid \psi_ {1} (z)\right) | \det  J _ {\psi} (z) | \tag {3.5}
$$
$$
q _ {1} \left(z _ {1}\right) \tilde {q} _ {2} \left(z _ {2}\right) = p _ {1} (\psi_ {1} (z)) \tilde {p} _ {2} (\psi_ {2} (z)) | \det  J _ {\psi} (z) | \tag {3.6}
$$
where  $z_{\mathrm{pa}(2;G')}$  denotes the parents of  $z_2$  in  $G'$ . Taking quotients of (3.5) and (3.4) yields
$$
\frac {\tilde {q} _ {1}}{q _ {1}} (z _ {1}) = \frac {\tilde {p} _ {1}}{p _ {1}} (\psi_ {1} (z)) \quad \stackrel {\frac {\partial}{\partial z _ {2}}} {\Longrightarrow} \quad 0 = \left(\frac {\tilde {p} _ {1}}{p _ {1}}\right) ^ {\prime} (\psi_ {1} (z)) \frac {\partial \psi_ {1}}{\partial z _ {2}} (z) \quad \stackrel {(A 3)} {\Longrightarrow} \quad \frac {\partial \psi_ {1}}{\partial z _ {2}} (z) = 0. \tag {3.7}
$$
Thus  $V_{1} = \psi_{1}(Z_{1})$  and  $q_{1}(z_{1}) = p_{1}(\psi_{1}(z_{1}))\left|\frac{\partial\psi_{1}}{\partial z_{1}}(z_{1})\right|$ . Substitution into (3.6) yields
$$
\tilde {q} _ {2} \left(z _ {2}\right) = \tilde {p} _ {2} \left(\psi_ {2} \left(z _ {1}, z _ {2}\right)\right) \left| \frac {\partial \psi_ {2}}{\partial z _ {2}} \left(z _ {1}, z _ {2}\right) \right| \tag {3.8}
$$
<sup>5</sup>By Asm. 2.5,  $f, h$ , and thus also  $h \circ f$  are diffeomorphisms. Hence,  $\psi$  is well-defined and also diffeomorphic.
where we have used that, according to (3.7),  $J_{\psi}$  is triangular. According to (3.8), for all  $z_{1}$ , the mapping  $z_{2} \mapsto \psi_{2}(z_{1}, z_{2})$  is measure preserving for  $\tilde{q}_{2}$  and  $\tilde{p}_{2}$ . By Lemma C.1 [18, § A.2, Lemma 2], it follows that  $\psi_{2}$  must be constant in  $z_{1}$ . Hence,  $\psi$  is an element-wise function. Finally,  $G = G'$  follows from faithfulness (Asm. 2.2), for otherwise  $(V_{1}, V_{2}) = (\psi_{1}(Z_{1}), \psi_{2}(Z_{2}))$  would be independent.
Case 2: Misaligned Intervention Targets. If  $G' \neq G$ , a similar argument to Case 1 (with the roles of  $z_1$  and  $z_2$  interchanged) also yields a contradiction to faithfulness. This leaves  $G = G'$ . Writing down the system of equations similar to (3.4)-(3.6), and then taking ratios of  $e_1$  and  $e_2$  with  $e_0$  yields
$$
\frac {\tilde {q} _ {1}}{q _ {1}} (z _ {1}) = \frac {\tilde {p} _ {2} (\psi_ {2} (z))}{p _ {2} (\psi_ {2} (z) \mid \psi_ {1} (z))} \quad \text {a n d} \quad \frac {\tilde {q} _ {2} (z _ {2})}{q _ {2} (z _ {2} \mid z _ {1})} = \frac {\tilde {p} _ {1}}{p _ {1}} (\psi_ {1} (z)). \tag {3.9}
$$
These conditions highlight the misalignment in intervention targets (see Fig. 2). Unlike in Case 1, they do not directly imply that some elements of  $J_{\psi}$  need to be zero, that is  $Z$  can be arbitrarily mixed w.r.t.  $V$ . However, (3.9) imposes constraints on the form of  $\psi$  that, by exploiting the invariance of  $q_{1}$  across  $e_{0}$  and  $e_{1}$  while  $p_{1}$  changes to  $\tilde{p}_{1}$ , can ultimately be shown to only be satisfied if the two expectations in (3.2) are equal for all continuous  $\varphi$ . However, such fine-tuning is ruled out by (A4).
Remark 3.3. The main difficulty of the proof is that (3.9) may, in principle, hold when  $(p,\tilde{p},q,\tilde{q})$  and  $\psi$  are completely unconstrained. This does not arise in prior work if the intervention targets are known [80, 81, 126] (Case 1), or the densities or mixing are parametrically constrained [3, 117, 122].
On the Genericity Condition (A4). The condition in (3.2) contrasts expectations of the same quantity w.r.t. the observational distribution  $P_{\mathbf{V}}^{e_0}$  and the interventional distribution  $P_{\mathbf{V}}^{e_1}$  corresponding to an intervention on  $V_{1}$  that turns  $p_1$  into  $\tilde{p}_1$ . The shared argument, on the other hand, is a function of the ratio between the intervened mechanism  $\tilde{p}_2$  and its base mechanism  $p_2$ . While the two expectations are always equal for linear  $\varphi$ , other choices imply non-trivial constraints. For instance,  $\varphi(y) = y^2$  yields
$$
\int \left(\tilde {p} _ {1} \left(v _ {1}\right) - p _ {1} \left(v _ {1}\right)\right) \int \frac {\tilde {p} _ {2} \left(v _ {2}\right) ^ {2}}{p _ {2} \left(v _ {2} \mid v _ {1}\right)} \mathrm {d} v _ {2} \mathrm {d} v _ {1} \neq 0.
$$
Since  $p_1 \neq \tilde{p}_1$  by assumption (A3), we argue that (A4) should generally hold for randomly chosen  $(p_1, p_2, \tilde{p}_1, \tilde{p}_2)$  and can only be violated if they are fine-tuned to each other. It can thus be viewed as encoding some notion of genericity—in line with the principle of independent causal mechanisms [15, 43, 61, 100, 109], but also involving the intervened mechanisms. Interestingly, related genericity conditions also arise in the study of nonlinear cause-effect inference from observational data, where identifiability is often obtained up to a set of pathological ("fine-tuned") spurious solutions satisfying a partial differential equation involving the original mechanisms [49, 58, 118, 137]. Further, we note that  $\varphi$  in (3.2) can also be thought of as a witness function of genericity, similar to witness functions in kernel-based two sample and independence testing [44].
Next, we provide our identifiability result for an arbitrary number of causal variables.
Theorem 3.4 (Identifiability up to  $\sim_{\mathrm{CRL}}$  from two paired perfect stochastic interventions per node). Suppose that we have access to multiple environments  $\{P_X^e\}_{e\in \varepsilon}$  generated as described in §2 under Asms. 2.2, 2.3, 2.5, 2.8 and 2.9. Let  $(h,G^{\prime})$  be any candidate solution such that the inferred latent distributions  $Q_Z^e = h_*(P_X^e)$  of  $Z = h(X)$  and the inferred mixing function  $h^{-1}$  satisfy the above assumptions w.r.t. the candidate causal graph  $G^{\prime}$ . Assume additionally that
(A1) all densities  $p^e$  and  $q^e$  are continuously differentiable and fully supported on  $\mathbb{R}^n$ ;
(A2') we have access to at least one pair of single-node perfect interventions per node, with unknown targets: there exist  $m \geq n$  known pairs of environments  $\mathcal{E} = \{(e_j, e_j')\}_{j=1}^m$  such that for each  $i \in [n]$  there exists some unknown  $j \in [m]$  for which  $\mathcal{I}^{e_j} = \mathcal{I}^{e_j'} = \{i\}$ ;
(A3') for all  $i \in [n]$ , the intervened mechanisms  $\tilde{p}_i(v_i)$  and  $\tilde{\tilde{p}}_i(v_i)$  differ everywhere, in the sense that
$$
\forall v _ {i}: \quad \left(\frac {\tilde {\tilde {p}} _ {i}}{\tilde {p} _ {i}}\right) ^ {\prime} (v _ {i}) \neq 0; \tag {3.10}
$$
Then the ground truth is identified in the sense of Defn. 2.6, that is,  $(f^{-1}, G) \sim_{\mathrm{CRL}} (h, G')$ .
<sup>6</sup>This step is where the assumption of perfect interventions (Asm. 2.9) is leveraged: the conclusion would not hold for arbitrary imperfect interventions for which (3.8) would involve  $\tilde{q}_2(z_2 \mid z_1)$  and  $p_2\left(\psi_2(z_1, z_2) \mid \psi_1(z_1)\right)$ .
Proof sketch (full proof in Appx. C.3). By considering ratios between  $e_j$  and  $e_j'$ , taking partial derivatives w.r.t.  $z_l$ , and using assumptions (A3'), we can identify a subset  $\mathcal{E}_n \subseteq \mathcal{E}$  of exactly  $n$  pairs of environments corresponding to distinct intervention targets in  $p$  (for otherwise  $\psi$  cannot be invertible). For  $(e_i, e_i') \in \mathcal{E}_n$ , w.l.o.g. fix the intervention targets in  $p$  to  $\mathcal{I}^{e_i} = \mathcal{I}^{e_i'} = \{i\}$  and let  $\pi$  be a permutation of  $[n]$  such that  $\pi(i)$  denotes the inferred intervention target in  $q$  that by (A2') is shared across  $(e_i, e_i')$ . By the same argument as before, we must have  $V_i = \psi_i(Z_{\pi(i)})$ , that is,  $\psi$  is a permutation composed with an element-wise function. It remains to show that  $\pi$  is a graph isomorphism, that is,  $V_i \to V_j$  in  $G \iff Z_{\pi(i)} \to Z_{\pi(j)}$  in  $G'$ . We prove  $\Rightarrow$ ; the other direction is analogous. Suppose for a contradiction that there exist  $(i, j)$  such that  $V_i \to V_j$  in  $G$ , but  $Z_{\pi(i)} \not\to Z_{\pi(j)}$  in  $G'$ . Consider  $e_i$  in which there are perfect interventions on  $Z_{\pi(i)}$  and  $V_i$ . For  $Z_{\pi(k)} \in Z_{\mathrm{pa}(\pi(j); G')}$ , let  $\tilde{V}_k = \psi_k(Z_{\pi(k)})$  and denote  $\tilde{V} = \cup_k \tilde{V}_k \subset V \setminus \{V_i, V_j\}$ . Since  $Z_{\pi(i)}$  and  $Z_{\pi(j)}$  are d-separated by  $Z_{\mathrm{pa}(\pi(j); G')}$  in the post-intervention graph  $G'_{\overline{Z}_{\pi(i)}}$  with arrows pointing into  $Z_{\pi(i)}$  removed [95], it follows by Markovianity of  $q$  w.r.t.  $G'$  that  $Z_{\pi(i)} \perp Z_{\pi(j)} | Z_{\mathrm{pa}(\pi(j); G')}$  in  $Q_Z^{e_i}$ . By applying the corresponding diffeomorphic functions  $\psi_i$ , it follows from Lemma C.2 in Appx. C.1 that  $V_i \perp V_j | \tilde{V}$  in  $P_V^{e_i}$ . This violates faithfulness (Asm. 2.2) of  $P_V$  to  $G$  since  $V_i$  and  $V_j$  are d-connected in  $G_{\overline{V}_i}$ .
(A2') states that we know that a pair of datasets corresponds to two distinct interventions on the same underlying variable, even though we may not know the exact target of the intervention. This situation could arise, for example, if both datasets are collected under the same experimental setup but with varying experimental parameters. We stress that this is different from data consisting of pairs of views  $(\boldsymbol{x}, \tilde{\boldsymbol{x}})$  sharing the values of some variables, which is counterfactual in nature [4, 18, 123]. One of the main challenges for our analysis (compared to a counterfactual multi-view setting) thus stems from the lack of correspondences across observations from different datasets. We also stress that a purely observational environment is not needed in this case, cf. (A2) in Thm. 3.2.
(A3') states that the intervention mechanisms are distinct in that their ratio is strictly monotonic, similar to (3.1) in (A3). This is a slightly stronger version of the assumption of interventional discrepancy proposed by Wendong et al. [126], which has been shown to be necessary for identifiability even if the graph  $G$  is known. For Gaussian  $\tilde{p}_i, \tilde{\tilde{p}}_i$ , (A3') is satisfied, for example, by a shift in mean. In the proofs, this is used to show that  $\psi$  must be an element-wise function, see (3.7). Intuitively, if  $\tilde{p}_i = \tilde{\tilde{p}}_i$  in some open set for more than one  $i$ , then the underlying variables can be nonlinearly mixed by a measure-preserving automorphism within this set without affecting the observed distributions [126].
4 Interpreting Causal Representations
Suppose that we succeed in identifying  $V$  and  $G$  up to  $\sim_{\mathrm{CRL}}$  (Defn. 2.6). How can we use or interpret such a causal representation? Since the scale of the variables is arbitrary (§ 2.2), we clearly cannot predict the exact outcomes of interventions. We therefore seek causal quantities that are preserved by the irresolvable ambiguities of  $\sim_{\mathrm{CRL}}$ . A prime candidate for this are interventional causal notions defined in terms of information theoretic quantities [28] and in particular the KL divergence  $D_{\mathrm{KL}}$ .
Definition 4.1 (Causal influence; based on Defn. 2 of Janzing et al. [62]). Let  $P_V$  be Markovian w.r.t. a DAG  $G$  with vertices  $V$ . For any  $V_i \to V_j$  in  $G$ , the causal influence of  $V_i$  on  $V_j$  is given by
$$
\mathfrak {C} _ {i \rightarrow j} ^ {P _ {V}} := D _ {\mathrm {K L}} \big (P _ {V} \parallel P _ {V} ^ {i \rightarrow j} \big), \quad \text {w h e r e} \quad p _ {j} ^ {i \rightarrow j} \big (v _ {j} \mid \boldsymbol {v} _ {\mathrm {p a} (j) \backslash \{i \}} \big) = \int p _ {j} \big (v _ {j} \mid \boldsymbol {v} _ {\mathrm {p a} (j)} \big) p _ {i} (v _ {i})   \mathrm {d} v _ {i}
$$
and  $P_{\mathbf{V}}^{i \to j}$  is the interventional distribution arising from replacing the  $j^{\text{th}}$  mechanism by  $p_j^{i \to j}$ .<sup>8</sup>
The following result, proven in Appx. C.4, states that the causal influences are invariant to reparametrisation and equivariant to permutations, the two irresolvable ambiguities of the  $\sim_{\mathrm{CRL}}$  equivalence class.
Theorem 4.2 (Preservation of causal influences under  $\sim_{\mathrm{CRL}}$ ). Let  $P_V$  be Markovian w.r.t.  $G$ , let  $\pi$  be a graph isomorphism of  $G$ , and let  $\phi$  be an element-wise diffeomorphism. Let  $\mathbf{Z} = \mathbf{P}_{\pi^{-1}} \circ \phi(\mathbf{V})$  and denote its induced distribution by  $Q_{\mathcal{Z}}$ . Then for any  $V_i \to V_j$  in  $G$  we have  $\mathfrak{C}_{i \to j}^{P_V} = \mathfrak{C}_{\pi(i) \to \pi(j)}^{Q_Z}$ .
Thm. 4.2 implies that the strength of causal relations among variables in the inferred graph carry meaning. They can thus be used to uncover changes to the latent causal mechanisms underlying different experimental datasets, for example, to gain scientific insights when combined with domain knowledge.
${}^{7}$  Cf. the interventional regularity assumption of Varici et al. [122, Asm. 2] which instead considers partial derivatives w.r.t. the parents and is related to c-faithfulness [59, Defn. 7].
Intuitively, this intervention captures the process of removing the arrow  $V_{i} \rightarrow V_{j}$  in  $G$  and "feeding" the conditional  $p(v_{j} \mid v_{\mathrm{pa}(j)})$  with an independent copy of  $V_{i}$ , distributed according to its marginal, see [62] for details.
Figure 3: Empirical Comparison of Correctly and Incorrectly Specified Normalizing Flow-Based Models. For  $n = 2$  latent causal variables with graph  $G$  given by  $V_{1} \rightarrow V_{2}$ , we compare a generative model based on the correct causal graph  $G' = G$  and intervention targets (blue) to other generative models assuming the wrong graph  $G' \neq G$  or misaligned intervention targets (yellow, red, purple). We show mean correlation coefficients (MCCs) between the learned and ground truth latents (Left) and the difference in validation model log-likelihood between the well-specified and misspecified models (Right). Each violin plot is based on 50 different ground truth data generating processes; the horizontal lines indicate the minimum, median and maximum values.
5 Learning Objectives
While our main focus is on studying identifiability, our theoretical insights also suggest approaches to learning causal representations from finite interventional datasets sampled from  $\{P_X^e\}_{e\in \mathcal{E}}$ . The main idea is to fit the data in a way that preserves the sparsity of interventions [98, 110] by employing the same (un)mixing function and sharing mechanisms across environments (Asm. 2.8). We sketch two approaches, which, according to our theory, should both asymptotically identify the ground truth up to  $\sim_{\mathrm{CRL}}$  if the set of available environments  $\mathcal{E}$  is sufficiently diverse (and the other assumptions hold).
- Autoencoder Framework: Jointly learn an encoder  $h$ , a graph  $G'$ , and intervention targets  $\mathcal{I}^e$  such that the encoded latents  $Z = h(X)$  can be used to reconstruct the observed  $X$  across all environments, while ensuring all but the intervened mechanisms are shared. Using  $E$  as an environment indicator, the latter corresponds to the constraint  $Z_i \perp E | Z_{\mathrm{pa}(i;G')}$  for  $i \notin \mathcal{I}^e$ , implementable, for example, through a suitable conditional independence test [39, 94, 138].
- Generative Modelling Approach: Fit a base generative model  $(G', p, f)$  and intervention models  $(\mathcal{I}^e, \{p_i^e\}_{i \in \mathcal{I}^e})_{e \in \mathcal{E}}$  by maximizing the likelihood of the multi-environment data. For example, given a candidate graph  $G'$  and candidate intervention targets  $\{\mathcal{I}^e\}_{e \in \mathcal{E}}$ , learn the base and intervened mechanisms and mixing; then pick the graph and intervention targets that achieve the best fit.
Setup. We experimentally pursue the second, generative approach. Specifically, we model the mixing function generating  $\mathbf{X}$  from  $\mathbf{Z}$  as a normalizing flow [30, 93] with different environment-specific base distributions, determined by the underlying causal graph, intervention targets, and (learnt) base and intervened mechanisms. Here, we focus on the bivariate case with two ground-truth causal latent variables  $V_{1} \rightarrow V_{2}$ . According to Thm. 3.2, this setting should be identifiable from three environments: an observational one and one perfect intervention on each of  $V_{1}$  and  $V_{2}$ . Our goal is to verify this empirically in light of finite data and optimization issues. We fix the observation dimension to  $d = 2$  to facilitate exact likelihood training of the normalizing flows, and fit a separate generative model for each choice of graph and intervention targets. The base and intervened mechanisms are linear Gaussian and the mixing function a three-layer MLP, see Appx. D.1 for implementation details.
Results. The results are summarized in Fig. 3. Our main findings are two-fold: First, we observe that, in the majority of cases, the well-specified model attains the highest held-out log-likelihood, as shown in Fig. 3 (Right). This suggests that the likelihood of otherwise comparable generative models can act as a reliable criterion to select the correct causal graph and intervention targets. Second, we find that the ground truth latent causal variables are approximately identified up to element-wise rescaling (MCC values close to one) by the correctly specified model and not by any other model, as shown in Fig. 3 (Left). This indicates that recovering the correct graph and targets is not only sufficient but also necessary for reliable identification of the causal representation. Taken together, these findings are consistent with our notion of  $\sim_{\mathrm{CRL}}$ -identifiability (Defn. 2.6) and Thm. 3.2.
<sup>9</sup>This can be viewed as a nested approach in which the inner loop corresponds to the method for Causal Component Analysis (CauCA) of Wendong et al. [126], and the outer loop to a search over  $G'$  and  $\{\mathcal{I}^e\}_{e \in \mathcal{E}}$ . Code to reproduce our experiments is available at: https://github.com/akekic/causal-component-analysis.
In Appx. D.2, we perform an additional experiment with  $n = 3$  variables, nonlinear latent SCMs, fixed causal order without graph search, and one interventional environment per node, thus assaying violations of assumption (A2') in the context of Thm. 3.4. Generally, we find that a correct choice of intervention targets can be selected based on model likelihood, leading to approximate identification of the causal variables, even when only the causal order is given, see Fig. 4 and Appx. D.2 for details.