SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[1] K. Ahuja, J. S. Hartford, and Y. Bengio. Weakly supervised representation learning with sparse perturbations. In Advances in Neural Information Processing Systems, volume 35, pages 15516-15528, 2022. [Cited on pages 2 and 10.]
[2] K. Ahuja, D. Mahajan, Y. Wang, and Y. Bengio. Interventional causal representation learning. In International Conference on Machine Learning, pages 372-407. PMLR, 2023. [Cited on pages 2 and 10.]
[3] J. Brehmer, P. De Haan, P. Lippe, and T. S. Cohen. Weakly supervised causal representation learning. Advances in Neural Information Processing Systems, 35:38319-38331, 2022. [Cited on pages 2, 10, and 16.]
[4] S. Buchholz, M. Besserve, and B. Schölkopf. Function classes for identifiable nonlinear independent component analysis. Advances in Neural Information Processing Systems, 35: 16946-16961, 2022. [Cited on pages 1 and 4.]
[5] S. Buchholz, G. Rajendran, E. Rosenfeld, B. Aragam, B. Scholkopf, and P. Ravikumar. Learning linear causal representations from interventions under general nonlinear mixing. In Advances in Neural Information Processing Systems, 2023. [Cited on pages 2 and 10.]
[6] R. Cai, F. Xie, C. Glymour, Z. Hao, and K. Zhang. Triad constraints for learning causal structure of latent variables. In Advances in Neural Information Processing Systems, volume 32, pages 12883-12892, 2019. [Cited on page 2.]
[7] P. Comon. Independent component analysis, a new concept? Signal processing, 36(3):287-314, 1994. [Cited on page 1.]
[8] G. Darmois. Analyse des liaisons de probabilité. In Proc. Int. Stat. Conferences 1947, page 231, 1951. [Cited on page 1.]
[9] I. Daunhawer, A. Bizeul, E. Palumbo, A. Marx, and J. E. Vogt. Identifiability results for multimodal contrastive learning. In The Eleventh International Conference on Learning Representations, 2022. [Cited on page 2.]
[10] C. Durkan, A. Bekasov, I. Murray, and G. Papamakarios. Neural spline flows. Advances in Neural Information Processing Systems, 32, 2019. [Cited on page 38.]
[11] L. Gresele, P. K. Rubenstein, A. Mehrjou, F. Locatello, and B. Schölkopf. The Incomplete Rosetta Stone problem: Identifiability results for multi-view nonlinear ICA. In Uncertainty in Artificial Intelligence, pages 217–227. PMLR, 2019. [Cited on page 1.]
[12] L. Gresele, G. Fissore, A. Javaloy, B. Scholkopf, and A. Hyvarinen. Relative gradient optimization of the Jacobian term in unsupervised deep learning. Advances in neural information processing systems, 33:16567-16578, 2020. [Cited on page 38.]
[13] L. Gresele, J. von Kugelgen, V. Stimper, B. Scholkopf, and M. Besserve. Independent mechanism analysis, a new concept? In Advances in neural information processing systems, volume 34, pages 28233-28248, 2021. [Cited on page 1.]
App. A recapitulates the notation used in this paper.
- App. B contains the proofs of all theoretical statements presented in the paper.
- App. C contains a nontrivial counterexample of Thm. 4.2 (ii) when Asm. 4.1 is violated.
- App. D contains an additional result of identifiability based on Thm. 4.2 (i), when more imperfect stochastic interventions are available.
- App. E contains a general discussion on CauCA with unknown intervention targets, as well as a generalization of some of the identifiability results.
- App. F contains a technical details on the relationship between CauCA and nonlinear ICA.
- App. G contains some theoretical results which were useful in the design of the experiments.
App. H contains the details of the experiments.
A Notations
Symbol	Description
G	A directed acyclic graph with nodes V(G) = [d] and arrows E(G)
(i,j)	An ordered tuple representing an arrow in E(G), with i, j ∈ V(G)
[[i,j]]	The integers i, ..., j
[d]	The natural numbers 1, ..., d
pa(i)	Parents of i, defined as {j ∈ V(G) | (j, i) ∈ E(G)}
pa(k)(j)	Parents of j in the post-intervention graph in the intervention regime k
pa(i)	Closure of the parents of i, defined as pa(i) ∪ {i}
anc(i)	Ancestors of i, nodes j in G such that there is a directed path from j to i
anc(i)	Closure of the ancestors of i, defined as anc(i) ∪ {i}
G	Transitive closure of G defined by paG(i) := ancG(i)
X,Y,Z	Unidimensional random variables
X,Y,Z	Multidimensional random variables
x,y,z	Scalars in R
x,y,z	Vectors in Rd
z[i]	The (1, ..., i) dimensions of z
φi	The function that outputs the i-th dimension of the mapping φ
φ[i]	The mapping that outputs the (1, ..., i) dimensions of the mapping φ
τk	Intervention targets in interventional regime k
P,Q	Probability distributions
p,q	Density functions of P, Q
Pi(Zi|Zpa(i))	Causal mechanism of variable Zi
Pki(Zi|Zpa(k(i))	Intervened mechanism of variable Zi in interventional regime k
Pk(Z)	k ≠ 0: interventional distribution in interventional regime k (Defn. 2.2)
k = 0: unintervened distribution
PG	Class of latent joint probabilities that are Markov relative to G
In this paper, it is assumed to have differentiable density and to be absolutely continuous with full support in Rd
F	Function class of mixing function/decoders
In this paper, it is assumed to be all C1-diffeomorphisms
S	Indeterminacy set, defined in Defn. 3.2
f	Mixing function or decoder, a diffeomorphism Rd → Rd
f*,P	The pushforward measure of P by f
G	A directed acyclic graph without indices
G |= G	G is an indexed graph of G, i.e. V(G) = [d] and there exists an isomorphism G → G
AutG	Group of automorphisms of graph G
Gd	Group of permutations of d elements
B Proofs
B.1 Lemmata
Lemma B.1 (Lemma 2 of [3]). Let  $A = C = \mathbb{R}$  and  $B = \mathbb{R}^d$ . Let  $f: A \times B \to C$  be differentiable. Define differentiable measures  $\mathbb{P}_A$  on  $A$  and  $\mathbb{P}_C$  on  $C$ . Let  $\forall b \in B$ ,  $f(\cdot, b): A \to C$  be measure-preserving, i.e.  $\mathbb{P}_C = f(\cdot, b)_* \mathbb{P}_A$ . Then  $f$  is constant in  $b$  over  $B$ .
Lemma B.2. For any distributions  $\mathbb{P}$ ,  $\mathbb{Q}$  of full support on  $\mathbb{R}$ , with c.d.f  $F$ ,  $G$ , there are only two diffeomorphisms  $T: \mathbb{R} \to \mathbb{R}$  such that  $T_*\mathbb{P} = \mathbb{Q}$ : they are  $G^{-1} \circ F$  and  $\overline{G}^{-1} \circ F$ , where  $\overline{G}(x) := 1 - G(x)$ .
Proof.  $T$  is a diffeomorphism, then  $T'(x) \neq 0 \quad \forall x \in \mathbb{R}$ . Then the sign of  $T'(x)$  is either positive or negative everywhere.  $T$  is either strictly increasing or strictly decreasing on  $\mathbb{R}$ . Since  $\mathbb{P}$  and  $\mathbb{Q}$  are full support in  $\mathbb{R}$ ,  $F$  and  $G$  are strictly increasing in  $\mathbb{R}$ .
- If  $T$  is increasing:  $T_*\mathbb{P} = \mathbb{Q}$  implies that  $G(x) = \mathbb{Q}(X \leq x) = \mathbb{P}(T(X) \leq x)$ . Since  $T$  is strictly increasing,  $G(x) = \mathbb{P}(T(X) \leq x) = \mathbb{P}(X \leq T^{-1}(x)) = F \circ T^{-1}(x)$ . Thus  $x = G^{-1} \circ F \circ T^{-1}(x)$ .  $T = G^{-1} \circ F$ .
- If  $T$  is decreasing:  $G(x) = \mathbb{Q}(X \leq x) = \mathbb{P}(T(X) \leq x) = \mathbb{P}(X \geq T^{-1}(x)) = 1 - F(T^{-1}(x))$ . Thus  $T = \overline{G}^{-1} \circ F$ .
Lemma B.3. Suppose  $\mathbb{P}$ ,  $\mathbb{Q}$  Markov relative to  $G$ , absolutely continuous with full support in  $\mathbb{R}^d$ . Fix any functions  $\varphi_1, \ldots, \varphi_d$  diffeomorphisms strictly monotonic in  $\mathbb{R}$ . Let  $\varphi \coloneqq (\varphi_i)_{i \in [d]}$ . The following statements are equivalent:
(1)  $\mathbb{Q} = \varphi_{*}\mathbb{P}$
(2)  $\forall i\in [d],\forall z_i\in \mathbb{R},\forall \mathbf{z}_{pa(i)}\in \mathbb{R}^{|pa(i)|},p_i(z_i\mid \mathbf{z}_{pa(i)}) = q_i(\varphi_i(z_i)\mid \varphi_{pa(i)}(\mathbf{z}_{pa(i)}))|\varphi_i'(z_i)|,$  or equivalently,  $\mathbb{Q}_i(\cdot |\varphi_{pa(i)}(\mathbf{z}_{pa(i)})) = (\varphi_i)_*\mathbb{P}_i(\cdot |\mathbf{z}_{pa(i)}).$
Proof.  $\mathbb{Q}_i(\cdot |\varphi_{\mathrm{pa}(i)}(\mathbf{z}_{\mathrm{pa}(i)}))$  denotes the conditional probability of  $Z_{i}$  ..  $\mathbb{Q}_i(Z_i|\varphi_{\mathrm{pa}(i)}(\mathbf{z}_{\mathrm{pa}(i)}))$
$(2)\Rightarrow (1)$  : Multiply the equations in (2) for  $n$  indices,
$$
\prod_ {i = 1} ^ {d} p _ {i} \left(z _ {i} \mid \mathbf {z} _ {\mathrm {p a} (i)}\right) = \prod_ {i = 1} ^ {d} q _ {i} \left(\varphi_ {i} \left(z _ {i}\right) \mid \varphi_ {\mathrm {p a} (i)} \left(\mathbf {z} _ {\mathrm {p a} (i)}\right)\right) \left| \varphi_ {i} ^ {\prime} \left(z _ {i}\right) \right|.
$$
Since  $\prod_{i=1}^{d} |\varphi_i'(z_i)| = |\det D\varphi(\mathbf{z})|$ , we obtain the equation in (1).
$(1) \Rightarrow (2)$ : without loss of generality, choose a total order on  $V(G)$  that preserves the partial order of  $G$ :  $i > j$  if  $i \in \mathrm{pa}(j)$ . Since  $\varphi$  is a diffeomorphism, by the change of variables formula,
$$
p (\mathbf {z}) = q \left(\varphi (\mathbf {z})\right) | \det  D \varphi (\mathbf {z}) |. \tag {11}
$$
Since  $\mathbb{P},\mathbb{Q}$  are Markov relative to  $G$  , write  $p,q$  as the factorization according to  $G$  ..
$$
\prod_ {i = 1} ^ {d} p _ {i} \left(z _ {i} \mid \mathbf {z} _ {\mathrm {p a} (i)}\right) = \prod_ {i = 1} ^ {d} q _ {i} \left(\varphi_ {i} \left(z _ {i}\right) \mid \varphi_ {\mathrm {p a} (i)} \left(\mathbf {z} _ {\mathrm {p a} (i)}\right)\right) \left| \varphi_ {i} ^ {\prime} \left(z _ {i}\right) \right|. \tag {12}
$$
We will show by induction on the reverse order of  $[d]$  that for all  $i\in [d]$
$$
p _ {i} \left(z _ {i} \mid \mathbf {z} _ {\mathrm {p a} (i)}\right) = q _ {i} \left(\varphi_ {i} \left(z _ {i}\right) \mid \varphi_ {\mathrm {p a} (i)} \left(\mathbf {z} _ {\mathrm {p a} (i)}\right)\right) \left| \varphi_ {i} ^ {\prime} \left(z _ {i}\right) \right|.
$$
In eq. (12), marginalize over  $z_{d}$
$$
\prod_ {i = 1} ^ {n - 1} p _ {i} \left(z _ {i} \mid \mathbf {z} _ {\mathrm {p a} (i)}\right) = \int_ {\mathbb {R}} \prod_ {i = 1} ^ {d} q _ {i} \left(\varphi_ {i} \left(z _ {i}\right) \mid \varphi_ {\mathrm {p a} (i)} \left(\mathbf {z} _ {\mathrm {p a} (i)}\right)\right) \left| \varphi_ {i} ^ {\prime} \left(z _ {i}\right) \right| d z _ {d}. \tag {13}
$$
Fix  $z_{[d - 1]}$ , change of variable  $u = \varphi_d(z_d)$ ,  $du = \varphi_d'(z_d)$
$$
\prod_ {i = 1} ^ {d - 1} p _ {i} \left(z _ {i} \mid \mathbf {z} _ {\mathrm {p a} (i)}\right) = \prod_ {i = 1} ^ {d - 1} q _ {i} \left(\varphi_ {i} \left(z _ {i}\right) \mid \varphi_ {\mathrm {p a} (i)} \left(\mathbf {z} _ {\mathrm {p a} (i)}\right)\right) \left| \varphi_ {i} ^ {\prime} \left(z _ {i}\right) \right|. \tag {15}
$$
Cancel the two sides of equation (12) by (15),
$$
p _ {i} \left(z _ {d} \mid \mathbf {z} _ {\mathrm {p a} (d)}\right) = q _ {i} \left(\varphi_ {d} \left(z _ {d}\right) \mid \varphi_ {\mathrm {p a} (d)} \left(\mathbf {z} _ {\mathrm {p a} (d)}\right)\right) \left| \varphi_ {d} ^ {\prime} \left(z _ {d}\right) \right|.
$$
Suppose the property is true for  $i + 1, \dots, d$ . Then for  $i$ ,  $i$  is a leaf node in the first  $i$  nodes. We use the same proof as before, marginalize over  $z_{i}$  (same as (13) with  $d$  replaced by  $i$ ) on the joint distribution of  $i$  first variables (same as (12) with  $d$  replaced by  $i$ ), which is then divided by the obtained  $i - 1$  marginal equation (same as (15) with  $d - 1$  replaced by  $i - 1$ ).
$$
S _ {s c a l i n g}: = \left\{\mathbf {h}: \mathbb {R} ^ {d} \rightarrow \mathbb {R} ^ {d} \mid \mathbf {h} (\mathbf {z}) = \left(h _ {1} \left(z _ {1}\right), \dots , h _ {d} \left(z _ {d}\right)\right), h _ {i} \text {i s} a d i f f e o m o r p h i s m i n} \mathbb {R} \right\}, \tag {3}
$$
Proof. For any  $(G,\mathbf{f},(\mathbb{P}^k,\tau_k)_{k\in \mathbb{I}[0,K]})$  in  $(G,\mathcal{F},\mathcal{P}_G)$ , and for any  $\mathbf{h}\in S_{\mathrm{scaling}}$ , define  $\mathbf{g}\coloneqq \mathbf{h}^{-1}$ , then  $\mathbf{g}\in S_{\mathrm{scaling}}$ . Define  $\mathbb{Q}^0\coloneqq \mathbf{g}_*\mathbb{P}^0$ . By Lemma B.3, for all  $i\in [d]$ ,  $\mathbb{Q}_i\left(\cdot |\mathbf{h}_{\mathrm{pa}(i)}(\mathbf{z}_{\mathrm{pa}(i)})\right) = (g_i)_*\mathbb{P}_i\left(\cdot \mid \mathbf{z}_{\mathrm{pa}(i)}\right)$ .
For  $k\in [K]$  , define
$$
\widetilde {\mathbb {Q}} _ {j} ^ {k} \left(\cdot | \mathbf {g} _ {\mathrm {p a} ^ {k} (j)} \left(\mathbf {z} _ {\mathrm {p a} ^ {k} (j)}\right)\right) := \left(g _ {j}\right) _ {*} \widetilde {\mathbb {P}} _ {j} ^ {k} \left(\cdot \mid \mathbf {z} _ {\mathrm {p a} ^ {k} (j)}\right). \tag {16}
$$
Define  $\mathbb{Q}^k \coloneqq \prod_{j \in \tau_k} \widetilde{\mathbb{Q}}_j^k \prod_{j \notin \tau_k} \mathbb{Q}_j$ , by Lemma B.3 and (16),  $\mathbb{Q}^k = \mathbf{g}_* \mathbb{P}^k \forall k \in [K]$ . By definition of  $\mathbb{Q}^0$ ,  $\mathbb{Q}^k = \mathbf{g}_* \mathbb{P}^k \forall k \in [0, K]$ . i.e.,  $\mathbf{f}_* \mathbb{P}^k = (\mathbf{f} \circ \mathbf{h})_* \mathbb{Q}^k$ .
B.2 Proof of Thm. 4.2
Theorem 4.2. For  $\text{CauCA}$  in  $(G, \mathcal{F}, \mathcal{P}_G)$ ,
$$
\mathcal {S} _ {\overline {{G}}} = \left\{\mathbf {h}: \mathbb {R} ^ {d} \rightarrow \mathbb {R} ^ {d} | \mathbf {h} (\mathbf {z}) = \left(h _ {i} \left(\mathbf {z} _ {\bar {a n c} (i)}\right)\right) _ {i \in [ d ]}, \mathbf {h} i s \mathcal {C} ^ {1} - d i f f e o m o r p h i s m \right\}. \tag {5}
$$
(ii) Suppose for each node  $i$  in  $[d]$ , there is one perfect stochastic intervention that satisfies Asm. 4.1. Then CauCA in  $(G, \mathcal{F}, \mathcal{P}_G)$  is identifiable up to  $\mathcal{S}_{\text{scaling}}$ .
Proof. Proof of (i): Consider two latent CBNs achieving the same likelihood across all interventional regimes:  $\left(G, \mathbf{f}, \left(\mathbb{P}^i, \tau_i\right)_{i \in [0,d]}\right)$  and  $\left(G, \mathbf{f}', \left(\mathbb{Q}^i, \tau_i\right)_{i \in [0,d]}\right)$ . Since the intervention targets  $(\tau_i)_{i \in [0,d]}$  are the same on both latent CBN, by rearranging the indices of  $G$  and correspondingly the indices in  $\mathbb{P}^i$ ,  $\mathbb{Q}^i$  and  $(\tau_i)_{i \in [0,d]}$ , we can suppose without loss of generality that the index of  $G$  preserves the partial order induced by  $E(G)$ :  $i < j$  if  $(i,j) \in E(G)$ . Since  $(\tau_i)_{i \in [d]}$  covers all  $d$  nodes in  $G$ , by rearranging  $(\tau_i)_{i \in [d]}$  we can suppose without loss of generality that  $\tau_i = i \forall i \in [d]$ .
In the  $i$ -th interventional regime,
$$
\mathbb {P} ^ {i} (\mathbf {Z}) = \widetilde {\mathbb {P}} _ {i} \left(Z _ {i} \mid \mathbf {Z} _ {\mathrm {p a} ^ {i} (i)}\right) \prod_ {j \in [ d ] \backslash i} \mathbb {P} \left(Z _ {j} \mid \mathbf {Z} _ {\mathrm {p a} (j)}\right),
$$
$$
\mathbb {Q} ^ {i} (\mathbf {Z}) = \widetilde {\mathbb {Q}} _ {i} \left(Z _ {i} \mid \mathbf {Z} _ {\mathrm {p a} ^ {i} (i)}\right) \prod_ {j \in [ d ] \backslash i} \mathbb {Q} _ {j} \left(Z _ {j} \mid \mathbf {Z} _ {\mathrm {p a} (j)}\right),
$$
where  $\mathrm{pa}(j) = \mathrm{pa}^i (j)\forall j\neq i$  , since intervening on  $i$  does not change the arrows towards  $j$
Define  $\varphi \coloneqq \mathbf{f}^{\prime -1}\circ \mathbf{f}$ . Denote its  $i$ -th dimension output function as  $\varphi_{i}:\mathbb{R}^{d}\to \mathbb{R}$ . We will prove by induction that  $\forall i\in [d],\forall j\notin \overline{\mathrm{anc}} (i),\forall \mathbf{z}\in \mathbb{R}^{d},\frac{\partial\varphi_{i}}{\partial z_{j}} (\mathbf{z}) = 0$
For any  $i\in [[0,d]]$ $\mathbf{f}_{*}\mathbb{P}^{i} = \mathbf{f}_{*}^{\prime}\mathbb{Q}^{i}$  . Since  $\varphi$  is a diffeomorphism, by the change of variable formula,
$$
p ^ {i} (\mathbf {z}) = q ^ {i} \left(\varphi (\mathbf {z})\right) | \det D \varphi (\mathbf {z}) |. \tag {17}
$$
For  $i = 0$ , factorize  $p^i$  and  $q^i$  according to  $G$ , then take the logarithm on both sides:
$$
\sum_ {j = 1} ^ {d} \ln p _ {j} \left(z _ {j} \mid \mathbf {z} _ {\mathrm {p a} (j)}\right) = \sum_ {j = 1} ^ {d} \ln q _ {j} \left(\varphi_ {j} (\mathbf {z}) \mid \varphi_ {\mathrm {p a} (j)} (\mathbf {z})\right) + \ln | \det D \varphi (\mathbf {z}) |. \tag {18}
$$
For  $i = 1$ ,  $\widetilde{\mathbb{Q}}_1$  has no conditionals, thus  $q^i$  is factorized as
$$
q ^ {1} (\mathbf {z}) = \widetilde {q} _ {1} \left(z _ {1}\right) \prod_ {j = 2} ^ {d} q _ {j} \left(z _ {j} \mid \mathbf {z} _ {\mathrm {p a} (j)}\right).
$$
So the equation (17) for  $i = 1$  after taking logarithm is
$$
\begin{array}{l} \ln \widetilde {p} _ {1} \left(z _ {1}\right) + \sum_ {j = 2} ^ {d} \ln p _ {j} \left(z _ {j} \mid \mathbf {z} _ {\mathrm {p a} (j)}\right) = \ln \widetilde {q} _ {1} \left(\varphi_ {1} (\mathbf {z})\right) + \sum_ {j = 2} ^ {d} q _ {j} \left(\varphi_ {j} (\mathbf {z}) \mid \varphi_ {\mathrm {p a} (j)} (\mathbf {z})\right) \tag {19} \\ + \ln | \det  D \varphi (\mathbf {z}) |. \\ \end{array}
$$
Subtract (19) by (18),
$$
\ln \tilde {p} _ {1} (z _ {1}) - \ln p _ {1} (z _ {1}) = \ln \tilde {q} _ {1} (\varphi_ {1} (\mathbf {z})) - \ln q _ {1} (\varphi_ {1} (\mathbf {z})). \tag {20}
$$
For any  $i \neq 1$ , take the  $i$ -th partial derivative of both sides:
$$
0 = \left[ \frac {\tilde {q} _ {1} ^ {\prime} (\varphi_ {1} (\mathbf {z}))}{\tilde {q} _ {1} (\varphi_ {1} (\mathbf {z}))} - \frac {q _ {1} ^ {\prime} (\varphi_ {1} (\mathbf {z}))}{q _ {1} (\varphi_ {1} (\mathbf {z}))} \right] \frac {\partial \varphi_ {1}}{\partial z _ {i}} (\mathbf {z}).
$$
By Asm. 4.1, the term in the parenthesis is non-zero a.e. in  $\mathbb{R}^d$ . Thus  $\frac{\partial\varphi_1}{\partial z_i} (\mathbf{z}) = 0$  a.e. in  $\mathbb{R}^d$ . Since  $\varphi = \mathbf{f}'^{-1}\circ \mathbf{f}$  where  $\mathbf{f},\mathbf{f}'$  are  $\mathcal{C}^1$ -diffeomorphisms, so is  $\varphi$ .  $\frac{\partial\varphi_1}{\partial z_i}$  is continuous and thus equals zero everywhere.
Now suppose  $\forall k\in [i - 1],\forall j\notin \overline{\mathrm{anc}} (k),\forall \mathbf{z}\in \mathbb{R}^d,\frac{\partial\varphi_k}{\partial z_j} (\mathbf{z}) = 0.$  Then for interventional regime  $i$
$$
\begin{array}{l} \ln \tilde {p} _ {i} \left(z _ {i} \mid \mathbf {z} _ {\mathrm {p a} ^ {i} (i)}\right) + \sum_ {j \neq i} \ln p _ {j} \left(z _ {j} \mid \mathbf {z} _ {\mathrm {p a} (j)}\right) = \ln \tilde {q} _ {i} \left(\varphi_ {i} (\mathbf {z}) \mid \varphi_ {\mathrm {p a} ^ {i} (i)} (\mathbf {z})\right) \\ + \sum_ {j \neq i} q _ {j} \left(\varphi_ {j} (\mathbf {z}) \mid \varphi_ {\mathrm {p a} (j)} (\mathbf {z})\right) + \ln | \det D \varphi (\mathbf {z}) |, \\ \end{array}
$$
Subtracted by (18),
$$
\ln \tilde {p} _ {i} \left(z _ {i} \mid \mathbf {z} _ {\mathrm {p a} ^ {i} (i)}\right) - \ln p _ {i} \left(z _ {i} \mid \mathbf {z} _ {\mathrm {p a} (i)}\right) = \ln \tilde {q} _ {i} \left(\varphi_ {i} (\mathbf {z}) \mid \varphi_ {\mathrm {p a} ^ {i} (i)} (\mathbf {z})\right) - \ln q _ {i} \left(\varphi_ {i} (\mathbf {z}) \mid \varphi_ {\mathrm {p a} (i)} (\mathbf {z})\right). \tag {21}
$$
For any  $\forall j\notin \overline{\mathrm{anc}} (i),j\notin \mathrm{pa}(i)\supset \mathrm{pa}^i (i)$  by assumption. Take partial derivative over  $z_{j}$  ..
$$
0 = \frac {\sum_ {k \in \overline {{\mathrm {p a}}} ^ {i} (i)} \frac {\partial \tilde {q} _ {i}}{\partial x _ {k}} \left(\varphi_ {i} (\mathbf {z}) \mid \varphi_ {\mathrm {p a} ^ {i} (i)} (\mathbf {z})\right) \frac {\partial \varphi_ {k}}{\partial z _ {j}} (\mathbf {z})}{\tilde {q} _ {i} \left(\varphi_ {i} (\mathbf {z}) \mid \varphi_ {\mathrm {p a} ^ {i} (i)} (\mathbf {z})\right)} - \frac {\sum_ {k \in \overline {{\mathrm {p a}}} (i)} \frac {\partial q _ {i}}{\partial x _ {k}} \left(\varphi_ {i} (\mathbf {z}) \mid \varphi_ {\mathrm {p a} (i)} (\mathbf {z})\right) \frac {\partial \varphi_ {k}}{\partial z _ {j}} (\mathbf {z})}{q _ {i} \left(\varphi_ {i} (\mathbf {z}) \mid \varphi_ {\mathrm {p a} (i)} (\mathbf {z})\right)},
$$
where  $x_{k}$  denotes the  $k$ -th dimension of the domain of  $\tilde{q}_i$  and  $q_{i}$ .
For all  $k \in \operatorname{pa}(i)$ , since  $j \notin \overline{\operatorname{anc}}(i)$ ,  $j$  is not in  $\overline{\operatorname{anc}}(k)$  either. By the assumption of induction,  $\frac{\partial \varphi_k}{\partial z_j}(\mathbf{z}) = 0$ . Delete the partial derivatives that are zero:
$$
0 = \left[ \frac {\frac {\partial \tilde {q} _ {i}}{\partial x _ {i}} \left(\varphi_ {i} (\mathbf {z}) \mid \varphi_ {\mathrm {p a} ^ {i} (i)} (\mathbf {z})\right)}{\tilde {q} _ {i} \left(\varphi_ {i} (\mathbf {z}) \mid \varphi_ {\mathrm {p a} ^ {i} (i)} (\mathbf {z})\right)} - \frac {\frac {\partial q _ {i}}{\partial x _ {i}} \left(\varphi_ {i} (\mathbf {z}) \mid \varphi_ {\mathrm {p a} (i)} (\mathbf {z})\right)}{q _ {i} \left(\varphi_ {i} (\mathbf {z}) \mid \varphi_ {\mathrm {p a} (i)} (\mathbf {z})\right)} \right] \frac {\partial \varphi_ {i}}{\partial z _ {j}} (\mathbf {z}).
$$
By Asm. 4.1, the term in parenthesis is nonzero a.e., thus  $\frac{\partial\varphi_i}{\partial z_j} (\mathbf{z}) = 0$  a.e. Since  $\frac{\partial\varphi_i}{\partial z_j}$  is continuous, it equals zero everywhere.
The induction is finished when  $i = d$ . We have proven that  $\forall i \in [d], \forall j \notin \overline{\mathrm{anc}}(i), \forall \mathbf{z} \in \mathbb{R}^d, \frac{\partial \varphi_i}{\partial z_j}(\mathbf{z}) = 0$ . Namely,  $\varphi_i$  only depends on  $\mathbf{z}_{\overline{\mathrm{anc}}(i)}$ .
Proof of (ii):
By the result proved in  $(i)$ ,  $\varphi \coloneqq \mathbf{f}^{\prime -1}\circ \mathbf{f}\in S_{\overline{G}}$ . Thus  $D\varphi (\mathbf{z})$  is lower triangular for all  $\mathbf{z}\in \mathbb{R}^d$ . Thus  $|\operatorname *{det}D\varphi (\mathbf{z})| = \prod_{i = 1}^{d}\left|\frac{\partial\varphi_{i}}{\partial z_{i}}\left(z_{[i]}\right)\right|$ , and for all  $i$ ,  $\varphi_{i}$  only depends on  $z_{1},\dots ,z_{i}$ . We will prove that  $\varphi_{i}$  only depends on  $z_{i}$ , i.e., it is constant on other variables.
To prove the conclusion in this item, we need the following lemma:
Lemma B.4. Given any  $\varphi : \mathbb{R}^n \to \mathbb{R}^n$  diffeomorphism such that for all  $i \in [n-1]$ ,  $\frac{\partial \varphi_i}{\partial z_n}$  is zero everywhere, and given any two distributions  $\mathbb{P}, \mathbb{Q}$  that are absolutely continuous and have full support in  $\mathbb{R}^n$  such that  $\varphi_*\mathbb{P} = \mathbb{Q}$ , then the distributions of the first  $n-1$  coordinates are preserved, i.e.,
$$
\left(\boldsymbol {\varphi} _ {[ n - 1 ]}\right) _ {*} \mathbb {P} _ {[ n - 1 ]} (\mathbf {z} _ {[ n - 1 ]}) = \mathbb {Q} _ {[ n - 1 ]} (\mathbf {z} _ {[ n - 1 ]}).
$$
Proof. Fix  $\mathbf{z}_{[n-1]} \in \mathbb{R}^{n-1}$ . For all  $i \in [n-1]$ ,  $\frac{\partial \varphi_i}{\partial z_n}$  is zero everywhere, and  $\varphi$  is a diffeomorphism, so  $\frac{\partial \varphi_n}{\partial z_n}$  is nonzero everywhere, otherwise there will exist  $\mathbf{z}$  such that  $\frac{\partial \varphi_n}{\partial z_n}(\mathbf{z})$  is singular. Therefore  $\frac{\partial \varphi_n}{\partial z_n}(\mathbf{z}_{[n-1]}, \cdot)$  is continuous and nonzero, and thus  $\varphi_n(\mathbf{z}_{[n-1]}, \cdot)$  is a diffeomorphism  $\mathbb{R} \to \mathbb{R}$ . So we can apply the change of variable  $u = \varphi_n(\mathbf{z}_{[n-1]}, z_n)$ ,  $du = \left|\frac{\partial \varphi_n}{\partial z_n}(\mathbf{z}_{[n-1]}, z_n)\right| dz_n$ :
$$
\begin{array}{l} \int_ {\mathbb {R}} q \left(\varphi_ {[ n - 1 ]} \left(\mathbf {z} _ {[ n - 1 ]}\right), \varphi_ {n} \left(z _ {n}\right)\right) \left| \frac {\partial \varphi_ {n}}{\partial z _ {n}} \left(\mathbf {z} _ {[ n - 1 ]}, z _ {n}\right) \right| d z _ {n} = \int_ {\mathbb {R}} q \left(\varphi_ {[ n - 1 ]} \left(\mathbf {z} _ {[ n - 1 ]}\right), u\right) d u \tag {22} \\ = q _ {[ n - 1 ]} \left(\varphi_ {[ n - 1 ]} \left(\mathbf {z} _ {[ n - 1 ]}\right)\right). \\ \end{array}
$$
In the equation  $p(\mathbf{z}) = q(\varphi (\mathbf{z}))|\operatorname *{det}\varphi (\mathbf{z})|$ , marginalize over  $z_{n}$ :
$$
\int_ {\mathbb {R}} p \left(\mathbf {z} _ {[ n - 1 ]}, z _ {n}\right) d z _ {n} = \int_ {\mathbb {R}} q \left(\varphi_ {[ n - 1 ]} \left(\mathbf {z} _ {[ n - 1 ]}\right), \varphi_ {n} \left(z _ {n}\right)\right) \prod_ {i = 1} ^ {n} \left| \frac {\partial \varphi_ {i}}{\partial z _ {i}} \left(z _ {[ i ]}\right) \right| d z _ {n}. \tag {23}
$$
Using (22), we obtain