SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The world is full of domain shifts and different environments. Often, we cannot pin down what exactly differs between two domains, but it may reasonably be modelled as a change in some causal mechanisms. While prior work relied on harder-to-obtain counterfactual data or parametric constraints for identifiability, our study demonstrates that interventional data can be sufficient—even in the nonparametric case where the spaces of mixing functions and mechanisms are infinite dimensional. Our results can be considered a step towards justifying the use of expressive machine learning methods for learning interpretable causal representations from high-dimensional experimental data in situations where parametric assumptions are undesirable, e.g., for complex systems in physics, biology, or medicine.
Weaker Notions of Identifiability. In this work, we have focused on the strongest notion of identifiability that is achievable in a nonparametric setting (Defn. 2.6). However, subject to the available data and assumptions, identifiability up to  $\sim_{\mathrm{CRL}}$  is not always possible. In this case, weaker notions of identifiability are of interest. For example, we may not be able to uniquely recover variables that are not targeted by interventions [5, 80, 123], or only recover groups of variables up to (non-)linear mixing [5, 122, 123] and the graph up to transitive closure if interventions are imperfect, or soft [117, 136]. A precise characterization of weaker notions of nonparametric identifiability from different types of interventions (cf. [74] for a temporal, semi-parametric setting) is an interesting direction for future work.
Known vs. Unknown Intervention Targets. When intervention targets can be considered known appears to be a more nuanced concept in CRL than in a fully observed setting, see also [126, §E] for an extended discussion. Recall that we assume w.l.o.g. that  $V_{1} \preceq \ldots \preceq V_{n}$  and only consider graphs respecting this ordering (§ 2.2), see also [117, Remark 1]. The intervention targets are then unknown w.r.t. the pre-imposed causal ordering. This is a key aspect that makes our setting more realistic, but also substantially complicates the analysis (see (3.9) and Remark 3.3). Similar to [122], for Thm. 3.2 we require a set of exactly  $n$  environments (one intervention on each node).[10] However, we relax this requirement to mere coverage ("at least 1") in Thm. 3.4 as shown for linear causal models in [22, 117].
Identifiability From One Intervention per Node for Any  $n$ . We conjecture that Thm. 3.2 can be generalized to  $n > 2$ , subject to a suitably adjusted set of genericity conditions involving several intervened and base mechanisms, akin to (3.2) in the bivariate case. The main challenge to such a generalization appears to be combinatorial, as there are  $n! - 1$  ways of misaligning intervention targets across  $p$  and  $q$ . In Thm. 3.4, we sidestep this issue by assuming pairs of environments. Thus, while two single-node perfect interventions are sufficient, we do not believe this to be necessary.
On the Assumption of Known  $n$  and Its Relation to Markovianity. Asm. 2.3 relates to causal sufficiency or Markovianity in classical causal inference, which correspond to the assumption of independent  $U_{i}$  in Defn. 2.1, implying the causal Markov factorization (2.2) [95, 100, 115]. With unobserved  $V$  and unknown  $n$ , the notion of "unobserved confounders" gets blurred, since one can, in principle, always construct a causally sufficient system by increasing  $n$  and adding any causes of two or more endogenous variables to  $V$ . Asm. 2.3 then states that the minimum number of variables required to do so is known.<sup>11</sup> However, this can lead to very large systems, which may in turn challenge the assumption of an invertible mixing (Asm. 2.5). Extensions of our analysis to unknown  $n$  [63, 68], non-Markovian, or non-invertible [131] models constitute an interesting direction for future investigations.
Practicality. The method explored in § 6 requires searching over graphs and intervention targets, which gets intractable even for moderate  $n$ . Simultaneously learning an (un)mixing function, causal graph, intervention targets, and mechanisms is challenging. Further work is needed to make methods for nonparametric CRL from multi-environment data more practical, e.g., by exploring the proposed autoencoder framework with a continuous parametrisation of graph [139, 140] and targets [60].
<sup>10</sup>By dropping the assumption of a fixed ordering and considering all DAGs, in this case one could also call  $V_{i}$  the variable intervened upon in environment  $i$ , and then consider the targets "known" in this sense.
11 Techniques for estimating the intrinsic dimensionality  $n$  of the observation manifold  $\mathcal{X} \subseteq \mathbb{R}^d$  or methods rooted in Bayesian nonparametrics could provide a means of relaxing this assumption.
We thank Simon Buchholz, Jiaqi Zhang, and the anonymous reviewers for helpful discussions, comments, and suggestions.
This work was supported by the Tübingen AI Center and by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany's Excellence Strategy, EXC number 2064/1, Project number 390727645. L.G. was supported by the VideoPredict project, FKZ: 01IS21088.
[1] J. Adams, N. Hansen, and K. Zhang. Identification of partially observed linear causal models: Graphical conditions for the non-gaussian and heterogeneous cases. Advances in Neural Information Processing Systems, 34:22822-22833, 2021. [Cited on p. 20.]
[2] K. Ahuja, J. Hartford, and Y. Bengio. Properties from mechanisms: an equivariance perspective on identifiable representation learning. In International Conference on Learning Representations, 2021. [Cited on p. 4 and 20.]
[3] K. Ahuja, D. Mahajan, V. Syrgkanis, and I. Mitliagkas. Towards efficient representation identification in supervised learning. In First Conference on Causal Learning and Reasoning, 2021. [Cited on p. 7 and 20.]
[4] K. Ahuja, J. S. Hartford, and Y. Bengio. Weakly supervised representation learning with sparse perturbations. In Advances in Neural Information Processing Systems, volume 35, pages 15516-15528, 2022. [Cited on p. 2 and 8.]
[5] K. Ahuja, D. Mahajan, Y. Wang, and Y. Bengio. Interventional causal representation learning. In International Conference on Machine Learning, pages 372-407. PMLR, 2023. [Cited on p. 3, 5, 10, and 21.]
[6] T. V. Anand, A. H. Ribeiro, J. Tian, and E. Bareinboim. Effect identification in cluster causal diagrams. In Proceedings of the 37th AAAI Conference on Artificial Intelligence, 2023. [Cited on p. 20.]
[7] J. D. Angrist and J.-S. Pischke. Mostly harmless econometrics: An empiricist's companion. Princeton University Press, 2009. [Cited on p. 1.]
[8] M. Arjovsky, L. Bottou, I. Gulrajani, and D. Lopez-Paz. Invariant risk minimization. arXiv preprint 1907.02893, 2019. [Cited on p. 5 and 20.]
[9] E. Bareinboim and J. Pearl. Causal inference and the data-fusion problem. Proceedings of the National Academy of Sciences, 113(27):7345-7352, 2016. [Cited on p. 1.]
[10] E. Baireinboim, J. D. Correa, D. Ibeling, and T. Icard. On Pearl's hierarchy and the foundations of causal inference. In Probabilistic and Causal Inference: The Works of Judea Pearl, page 507-556. Association for Computing Machinery, 2022. [Cited on p. 1 and 20.]
[11] S. Beckers and J. Y. Halpern. Abstracting causal models. In Proceedings of the AAAI conference on Artificial Intelligence, volume 33, pages 2678-2685, 2019. [Cited on p. 20.]
[12] S. Beckers, F. Eberhardt, and J. Y. Halpern. Approximate causal abstractions. In Uncertainty in Artificial Intelligence, pages 606–615. PMLR, 2020. [Cited on p. 20.]
[13] S. Beery, G. Van Horn, and P. Perona. Recognition in terra incognita. In Proceedings of the European Conference on Computer Vision (ECCV), pages 456-473, 2018. [Cited on p. 1.]
[14] Y. Bengio, A. Courville, and P. Vincent. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8):1798-1828, 2013. [Cited on p. 2.]
[15] M. Besserve, N. Shajarisales, B. Scholkopf, and D. Janzing. Group invariance principles for causal generative models. In International Conference on Artificial Intelligence and Statistics, pages 557-565. PMLR, 2018. [Cited on p. 7.]
A Extended Discussion of Related Work 20
A.1 Comparison of Related Identifiability Results 20
B Proof of Minimality of the CRL Equivalence Class  $\sim_{\mathrm{CRL}}$  22
C Identifiability Proofs 23
C.1 Auxiliary Lemmata 23
C.2 Proof of Thm. 3.2 24
C.3 Proof of Thm. 3.4 29
C.4 Proof of Thm. 4.2 31
D Experimental Details and Additional Results 33
D.1 Experimental Details for  $\S 6$  33
D.2 Additional Results: Learning Nonlinear Latent SCMs from Partial Causal Order 34
E Discussion of the Role of Our Assumptions 36
A Extended Discussion of Related Work
Prior work on causal representation learning with general nonlinear relationships (both among latents and between latents and observations) and without an explicit task or label typically relies on some form of weak supervision. One example of weak supervision is "multi-view" data consisting of tuples of related observations. von Kugelgen et al. [123] consider counterfactual pairs of observations arising from imperfect interventions through data augmentation, and prove identifiability for the invariant non-descendants of intervened-upon variables. Brehmer et al. [18] also use counterfactual pre- and post-intervention views and show that the latent SCM can be identified given all single-node perfect stochastic interventions. Another type of weak-supervision is temporal structure [2], possibly combined with nonstationarity [132, 133], interventions on known targets [80, 81], or observed actions inducing sparse mechanism shifts [74, 75, 110]. Other works use more explicit supervision in the form of annotations of the ground truth causal variables or a known causal graph [111, 126, 130].
A different line of work instead approaches causal representation learning from the perspective of causal discovery in the presence of latent variables [115]. Doing so from purely observational i.i.d. data requires additional constraints on the generative process, such as restrictions on the graph structure or particular parametric and distributional assumptions, and typically leverages the identifiability of linear ICA [27, 38, 79]. For linear, non-Gaussian models, Silva et al. [113] show that the causal DAG can be recovered up to Markov equivalence if all observed variables are "pure" in that they have a unique latent causal parent. Cai et al. [23] and Xie et al. [128, 129] extend this result to identify the full graph given two pure observed children per latent, and Adams et al. [1] provide sufficient and necessary graphical conditions for full identification. For discrete causal variables, Kivva et al. [68] introduce a similar "no twins" condition to reduce the task of learning the number and cardinality of latents and the Markov-equivalence class of the graph to mixture identifiability.
Other lines of work have investigated the relationship between causal models at different levels of coarse-graining or abstraction [6, 11, 12, 24-26, 35, 105, 134, 135], or learning invariant representations in a supervised setting [3, 8, 32, 33, 73, 86, 92, 124, 125], often for domain generalization.
Other concurrent works address, e.g., learning from soft interventions with polynomial mixing [136], or inferring both causal graph and the number of latents subject to graphical constraints [63].
A.1 Comparison of Related Identifiability Results
To complement the presentation of related multienvironment CRL works in § 1, we provide a structured overview of and comparison with existing identifiability results for causal representation learning in Tab. 1. The table categorizes work along different dimensions. First, we make a distinction based on the type of data (observational, interventional, or counterfactual) results rely on (colour coded in green, yellow, and red, respectively). These are also referred to as different rungs in the "ladder of causation" [97] or layers in the Pearl Causal Hierarchy [10]. Second, we categorize work depending on the assumptions placed on the latent causal model and the mixing function. As can be seen, works relying solely on observational data often require restrictive graphical assumptions on the mixing function. On the other hand, access to much more informative counterfactual data has allowed identification even for nonparametric causal models and mixing functions. Our work can be viewed as a step towards addressing the lack of nonparametric identifiability results in the interventional regime.
We emphasize that Tab. 1 is not exhaustive: certain relevant works did not easily fit into our categorization or the causal representation learning framework adopted in the present work. For example, not listed are works that leverage temporal structure [2, 74, 75, 80, 81], rely on heterogenous data and distribution shifts which are not directly expressed in terms of or linked to interventions [82, 132, 133], allow for edges from observed to latent variables [1], or require more direct supervision [111, 130].
Table 1: Comparison of Existing Identifiability Results for Causal Reresentation Learning. All of the listed works assume invertibility (or injectivity) of the mixing function, as well as causal sufficiency (Markovianity) for the causal latent variables. Most or all of the listed results require additional technical assumptions, and may provide additional results, which we omit for sake of readability; see the references for more details.
Work	Layer	Causal Model	Mixing Function	Main Identifiability Result
Cai et al. [23], Xie et al. [128, 129]	observational	linear, non-Gaussian	linear with non-Gaussian noise s.t. each Vi has 2 pure (obs. or unobs.) children	number of latents + G
Kivva et al. [68]	observational	discrete, nonparametric	identifiable mixture model s.t. obs. children of Vi∉ obs. children of Vj	number, cardinality & dist. of discrete latents + G up to Markov equivalence
Ahuja et al. [5, Thm. 4]	observational	nonlinear w. independent support [103, 125]	finite-degree polynomial	V up to permutation, shift, & linear scaling
Squires et al. [117, Thms. 1 & 2]	interventional	linear	linear	G and V up to partial-order preserving permutations from obs. dist. & all single-node perfect interventions
Squires et al. [117, Thm. 1]	interventional	linear	linear	G up to transitive closure from obs. dist. & all single-node imperfect interventions
Varici et al. [122, Thm. 16]	interventional	nonparametric	linear	G and V up to partial-order preserving permutations from obs. dist. & all single-node perfect interventions
Ahuja et al. [5, Thm. 2]	interventional	nonparametric	finite-degree polynomial	V up to permutation, shift, and linear scaling from all single-node perfect hard interventions
Buchholz et al. [22]	interventional	linear Gaussian	nonparametric	G and V up to permutation from obs. dist. & all single-node perfect interventions
This Work (Thm. 3.2)	interventional	nonparametric	nonparametric	for n = 2: G and V up to ~cRL from all single-node perfect interventions, subject to genericity (3.2)
This Work (Thm. 3.4)	interventional	nonparametric	nonparametric	G and V up to ~cRL from two distinct, paired single-node perfect interventions per node
von Kugelgen et al. [123]	counterfactual	nonparametric	nonparametric	block of non-descendants Vnd(T) up to invertible function from fat-hand imperfect interventions on Vz
Brehmer et al. [18]	counterfactual	nonparametric	nonparametric	G and V up to ~cRL from all single-node perfect interventions
B Proof of Minimality of the CRL Equivalence Class  $\sim_{\mathrm{CRL}}$
First, let us recall the main statements from  $\S 2.2$
$$
\left(h _ {1}, G _ {1}\right) \sim_ {\mathrm {C R L}} \left(h _ {2}, G _ {2}\right) \quad \Longleftrightarrow \quad \left(h _ {2}, G _ {2}\right) = \left(\boldsymbol {P} _ {\pi^ {- 1}} \circ \phi \circ h _ {1}, \pi \left(G _ {1}\right)\right) \tag {2.3}
$$
for some element-wise diffeomorphism  $\phi(\pmb{v}) = (\phi_1(v_1), \dots, \phi_n(v_n))$  of  $\mathbb{R}^n$  and a permutation  $\pi$  of  $[n]$  such that  $\pi : G_1 \mapsto G_2$  is a graph isomorphism and  $\pmb{P}_{\pi}$  the corresponding permutation matrix.
We now restate this result more formally.
Proposition B.1 (Minimality of  $\sim_{\mathrm{CRL}}$ ). Let  $(h,G^{\prime})\sim_{\mathrm{CRL}}(f^{-1},G)$  with  $\pi$  denoting the graph isomorphism mapping  $G$  to  $G^{\prime}$  (i.e., a permutation that preserves the partial topological order of  $G$ ). Let  $Z = h(\mathbf{X})$  be the inferred representation with distribution  $Q_{\mathbf{Z}} = h_{*}(P_{\mathbf{X}})$  Markov w.r.t.  $G^{\prime}$  and associated density  $q$ . Let  $\mathcal{I}^{e}\subseteq [n]$  denote a set of intervention targets, and consider an intervention that changes  $p_i(v_i\mid \boldsymbol{v}_{\mathrm{pa}(i)})$  to some intervened mechanism  $\tilde{p}_i(v_i\mid v_{\mathrm{pa}(i)})$  for all  $i\in \mathcal{I}^e$ , giving rise to the interventional distributions  $P_V^e$  and  $P_X^e = f_*(P_V^e)$ . Then there exist appropriately chosen  $\tilde{q}_{\pi (i)}(z_{\pi (i)}\mid z_{\mathrm{pa}(\pi (i),G')})$  for  $i\in \mathcal{I}^e$  such that the resulting interventional distribution  $Q_{\mathbf{Z}}^{e}$  gives rise to the same observed distributions, that is,  $P_{\mathbf{X}}^{e} = h_{*}^{-1}(Q_{\mathbf{Z}}^{e})$ .
Proof. Since  $(h,G^{\prime})\sim_{\mathrm{CRL}}(f^{-1},G)$ , by Defn. 2.6 we have
$$
\boldsymbol {Z} = \boldsymbol {P} _ {\pi^ {- 1}} \circ \phi (\boldsymbol {V}) \tag {B.1}
$$
for some element-wise diffeomorphism  $\phi$  with inverse  $\psi = \phi^{-1}$ . Then (B.1) implies that for all  $i\in [n]$
$$
V _ {i} = \psi_ {i} \left(Z _ {\pi (i)}\right) \tag {B.2}
$$
According to (B.2), each conditional in the Markov factorization of  $Q_{Z}$  is given in terms of  $p$  by
$$
q _ {\pi (i)} \left(z _ {\pi (i)} \mid \boldsymbol {z} _ {\mathrm {p a} (\pi (i); G ^ {\prime})}\right) = p _ {i} \left(\psi_ {i} \left(z _ {\pi (i)}\right) \mid \psi_ {\mathrm {p a} (i)} \left(\boldsymbol {z} _ {\mathrm {p a} (\pi (i); G ^ {\prime})}\right)\right) \left| \frac {\mathrm {d} \psi_ {i}}{\mathrm {d} z _ {\pi (i)}} \left(z _ {\pi (i)}\right) \right| \tag {B.3}
$$
where we have used the change of variables in (B.2) and the fact that  $\pi(\mathrm{pa}(i)) = \mathrm{pa}(\pi(i); G')$  since  $\pi: G \mapsto G'$  is a graph isomorphism.
Consider an intervention that changes  $p_i(v_i \mid v_{\mathrm{pa}(i)})$  to some intervened mechanism  $\tilde{p}_i(v_i \mid v_{\mathrm{pa}(i)})$  for all  $i \in \mathcal{I}^e$ . Denote the corresponding intervened joint distribution by  $P_V^e$  with joint density  $p^e$  given by
$$
p ^ {e} (\boldsymbol {v}) = \prod_ {i \in \mathcal {I} ^ {e}} \tilde {p} _ {i} \left(v _ {i} \mid \boldsymbol {v} _ {\mathrm {p a} (i)}\right) \prod_ {j \in [ n ] \backslash \mathcal {I} ^ {e}} p _ {j} \left(v _ {j} \mid \boldsymbol {v} _ {\mathrm {p a} (j)}\right). \tag {B.4}
$$
Denote by  $Q_{\mathbf{Z}}^{e} = (P_{\pi^{-1}} \circ \phi)_{*}(P_{\mathbf{V}}^{e})$  the corresponding distribution over  $Z$  with joint density given by  $q^{e}$
$$
\begin{array}{l} q ^ {e} (\boldsymbol {z}) = p ^ {e} \left(\psi \circ \boldsymbol {P} _ {\pi} (\boldsymbol {z})\right) | \det  \boldsymbol {J} _ {\psi \circ \boldsymbol {P} _ {\pi}} (\boldsymbol {z}) | (B.5) \\ = \prod_ {i \in \mathcal {I} ^ {e}} \tilde {p} _ {i} \left(\psi_ {i} \left(z _ {\pi (i)}\right) \mid \psi_ {\mathrm {p a} (i)} \left(\boldsymbol {z} _ {\mathrm {p a} \left(\pi (i); G ^ {\prime}\right)}\right)\right) \left| \frac {\mathrm {d} \psi_ {i}}{\mathrm {d} z _ {\pi (i)}} \left(z _ {\pi (i)}\right) \right| \prod_ {j \in [ n ] \backslash \mathcal {I} ^ {e}} q _ {\pi (j)} \left(z _ {\pi (j)} \mid \boldsymbol {z} _ {\mathrm {p a} \left(\pi (j); G ^ {\prime}\right)}\right), (B.6) \\ \end{array}
$$
where we have used (B.3), (B.4), and the fact that  $J_{\psi}$  is diagonal.
$$
\tilde {q} _ {\pi (i)} \left(z _ {\pi (i)} \mid \boldsymbol {z} _ {\mathrm {p a} (\pi (i); G ^ {\prime})}\right) := \tilde {p} _ {i} \left(\psi_ {i} \left(z _ {\pi (i)}\right) \mid \psi_ {\mathrm {p a} (i)} \left(\boldsymbol {z} _ {\mathrm {p a} (\pi (i); G ^ {\prime})}\right)\right) \left| \frac {\mathrm {d} \psi_ {i}}{\mathrm {d} z _ {\pi (i)}} \left(z _ {\pi (i)}\right) \right| \tag {B.7}
$$
we finally arrive at
$$
q ^ {e} (\boldsymbol {z}) = \prod_ {i \in \mathcal {I} ^ {e}} \tilde {q} _ {\pi (i)} \left(z _ {\pi (i)} \mid \boldsymbol {z} _ {\mathrm {p a} (\pi (i); G ^ {\prime})}\right) \prod_ {j \in [ n ] \backslash \mathcal {I} ^ {e}} q _ {\pi (j)} \left(z _ {\pi (j)} \mid \boldsymbol {z} _ {\mathrm {p a} (\pi (j); G ^ {\prime})}\right). \tag {B.8}
$$
This shows that any intervention on  $\{V_i\}_{i\in \mathcal{I}^e}\subseteq V$  which replaces
$$
\left\{p _ {i} \left(v _ {i} \mid \boldsymbol {v} _ {\mathrm {p a} (i)}\right) \right\} _ {i \in \mathcal {I} ^ {e}} \mapsto \left\{\tilde {p} _ {i} \left(v _ {i} \mid \boldsymbol {v} _ {\mathrm {p a} (i)}\right) \right\} _ {i \in \mathcal {I} ^ {e}}, \tag {B.9}
$$
can equivalently be captured by an intervention on  $\{Z_{\pi (i)}\}_{i\in \mathcal{I}^e}\subseteq Z$  which replaces
$$
\left\{q _ {\pi (i)} \left(z _ {\pi (i)} \mid z _ {\mathrm {p a} \left(\pi (i); G ^ {\prime}\right)}\right) \right\} _ {i \in \mathcal {I} ^ {e}} \mapsto \left\{\tilde {q} _ {\pi (i)} \left(z _ {\pi (i)} \mid z _ {\mathrm {p a} \left(\pi (i); G ^ {\prime}\right)}\right) \right\} _ {i \in \mathcal {I} ^ {e}}. \tag {B.10}
$$
with  $\tilde{q}_i$  defined according to (B.7).
C Identifiability Proofs
C.1 Auxiliary Lemmata
Lemma C.1 (Lemma 2 of Brehmer et al. [18]). Let  $A = C = \mathbb{R}$  and  $B = \mathbb{R}^n$ . Let  $f: A \times B \to C$  be differentiable. Define two differentiable measures  $p_A$  on  $A$  and  $p_C$  on  $C$ . Let  $\forall b \in B$ ,  $f(\cdot, b): A \to C$  be measure-preserving, in the sense that the pushforward of  $p_A$  is always  $p_C$ . Then  $f(\cdot, b)$  is constant in  $b$  on  $B$ .
Proof. See Appendix A.2 of Brehmer et al. [18].
Lemma C.2 (Preservation of conditional independence under invertible transformation.). Let  $X$  and  $Y$  be continuous real-valued random variables, and let  $Z$  be a continuous random vector taking values in  $\mathbb{R}^n$ . Suppose that  $(X, Y, Z)$  have a joint density w.r.t. the Lebesgue measure. Let  $f: \mathbb{R} \to \mathbb{R}$ ,  $g: \mathbb{R} \to \mathbb{R}$ , and  $h: \mathbb{R}^n \to \mathbb{R}^n$  be diffeomorphisms. Then  $X \perp Y \mid Z \Rightarrow f(X) \perp g(Y) \mid h(Z)$ .
Proof. Denote by  $p(x,y,z)$  the density of  $(X,Y,Z)$ . Then  $X \perp Y \mid Z$  implies that for all  $(x,y,z)$ ,  $p$  can be factorized as follows:
$$
p (x, y, z) = p _ {z} (z) p _ {x} (x \mid z) p _ {y} (y \mid z). \tag {C.1}
$$
Let  $(A,B,C) = (f(X),g(Y),h(Z))$ , and write  $\tilde{f} = f^{-1}$ ,  $\tilde{g} = g^{-1}$ , and  $\tilde{h} = h^{-1}$ .
Then  $(A,B,C)$  also has a density  $q(a,b,c)$ , which for all  $(a,b,c)$  is given by the change of variable formula:
$$
\begin{array}{l} q (a, b, \boldsymbol {c}) = p (\tilde {f} (a), \tilde {g} (b), \tilde {h} (\boldsymbol {c})) \left| \frac {\mathrm {d} \tilde {f}}{\mathrm {d} a} (a) \frac {\mathrm {d} \tilde {g}}{\mathrm {d} b} (b) \det  \boldsymbol {J} _ {\tilde {h}} (\boldsymbol {c}) \right| (C.2) \\ = p _ {z} (\tilde {h} (c)) | \det  J _ {\tilde {h}} (c) | p _ {x} (\tilde {f} (a) \mid \tilde {h} (c)) \left| \frac {\mathrm {d} \tilde {f}}{\mathrm {d} a} (a) \right| p _ {y} (\tilde {g} (b) \mid \tilde {h} (c)) \left| \frac {\mathrm {d} \tilde {g}}{\mathrm {d} b} (b) \right| (C.3) \\ \end{array}
$$
where in (C.2) we have used the fact that  $(X,Y,Z)\mapsto (f(X),g(Y),h(Z))$  has block-diagonal Jacobian, and in (C.3) that  $p$  factorises as in (C.1). Next, define
$$
q _ {\boldsymbol {c}} (\boldsymbol {c}) := p _ {\boldsymbol {z}} \left(\tilde {h} (\boldsymbol {c})\right) | \det  \boldsymbol {J} _ {\tilde {h}} (\boldsymbol {c}) |, \tag {C.4}
$$
$$
q _ {a} (a \mid c) := p _ {x} (\tilde {f} (a) \mid \tilde {h} (c)) \left| \frac {\mathrm {d} \tilde {f}}{\mathrm {d} a} (a) \right|, \tag {C.5}
$$