SlowGuess/NeurIPS_2023_OCR huggingface dataset:
By not allowing symmetric distributions in Condition (C1), we assume in particular that the distributions of the errors are non-Gaussian. Non-Gaussianity together with the assumptions of pairwise different and non-symmetric error distributions allow us to extend the results on identifiability of linear ICA to the unpaired multi-domain setup and to identify the joint distribution. In particular, the assumption of pairwise different error distributions allows for "matching" the distributions across domains to identify the ones corresponding to the shared latent space. Non-symmetry accounts for the sign-indeterminacy of linear ICA when matching the distributions. We discuss the necessity of these assumptions in Remark 3.2 and, in more detail, in Appendix C. Note that Condition (C1) is always satisfied in a generic sense, that is, randomly chosen probability distributions on the real line are pairwise different and non-symmetric with probability one. Finally, Condition (C2) requires in particular that for each shared latent node  $k \in \mathcal{L}$  there is at least one node  $v \in V_e$  in every domain  $e \in [m]$  such that  $k \in \mathrm{pa}(v)$ .
Under Conditions (C1) and (C2) we are able to derive a sufficient condition for identifiability of the joint distribution. Let  $SP(p)$  be the set of  $p \times p$  signed permutation matrices. We define the set of signed permutation block matrices:
$$
\Pi = \left\{\operatorname {d i a g} \left(\Psi_ {\mathcal {L}}, \Psi_ {I _ {1}}, \dots , \Psi_ {I _ {m}}\right): \Psi_ {\mathcal {L}} \in S P (\ell) \text {a n d} \Psi_ {I _ {e}} \in S P (| I _ {e} |) \right\}.
$$
Theorem 3.1. Let  $\mathcal{G}_m$  be an  $m$ -domain graph with shared latent nodes  $\mathcal{L} = [\ell]$ , and let  $P_X \in \mathcal{M}(\mathcal{G}_m)$  with representation  $(B, P)$ . Suppose that  $m \geq 2$  and that Conditions (C1) and (C2) are satisfied. Let  $(\widehat{\ell}, \widehat{B}, \widehat{P})$  be the output of Algorithm 1. Then  $\widehat{\ell} = \ell$  and
$$
\widehat {B} = B \cdot \Psi \quad a n d \quad \widehat {P} = \Psi^ {\top} \# P,
$$
for a signed permutation block matrix  $\Psi \in \Pi$ .
Theorem 3.1 says that the matrix  $B$  is identifiable up to signed block permutations of the columns. Under the assumptions of Theorem 3.1 it holds that  $\widehat{B} \# \widehat{P}$  is equal to  $P_{X}$ . That is, the joint distribution of the domains is identifiable.
Remark 3.2. While Theorem 3.1 is a sufficient condition for identifiability of the joint distribution, we emphasize that pairwise different error distributions are in most cases also necessary; we state the exact necessary condition in Proposition C.1 in the Appendix. Said differently, if one is willing to assume that conceptually different latent variables also follow a different distribution, then identification of these variables is possible, and otherwise (in most cases) not. Apart from pairwise different error distributions, non-symmetry is then required to fully identify the joint distribution whose dependency structure is determined by the shared latent variables. If the additional assumption on non-symmetry is not satisfied, then it is still possible to identify the shared, conceptually different latent variables, which becomes clear by inspecting the proofs of Theorem 3.1 and Proposition C.1. The non-identifiability of the joint distribution would only result in sign indeterminacy, that is, entries of the matrix  $\widehat{B}$  could have a flipped sign.
Remark 3.3. By checking the proof of Theorem 3.1, the careful reader may notice that the statement of the theorem still holds true when we relax the third condition in the definition of an  $m$ -domain
Algorithm 1 IdentifyJointDistribution
1: Input: Probability measures  $P_{X^e}$  for all  $e \in [m]$ .
2: Output: Number of shared latent variables  $\widehat{\ell}$ , matrix  $\widehat{B}$  and probability measure  $\widehat{P}$  with independent marginals.
3: for  $e \in [m]$  do
4: Linear ICA: Find the smallest value  $\widehat{s}_e$  such that  $P_{X^e} = \widehat{B}^e\# P^e$  for a matrix  $\widehat{B}^e\in \mathbb{R}^{d_e\times \widehat{s}_e}$  and an  $\widehat{s}_e$ -dimensional probability measure  $P^e$  with independent, mean-zero and unit-variance marginals  $P_i^e$ .
5: end for
6: Matching: Let  $\widehat{\ell}$  be the maximal number such that there are signed permutation matrices  $\{Q^e\}_{e\in [m]}$  satisfying
$$
d \left(\left[ \left(Q ^ {e}\right) ^ {\top} \# P ^ {e} \right] _ {i}, \left[ \left(Q ^ {f}\right) ^ {\top} \# P ^ {f} \right] _ {i}\right) = 0
$$
for all  $i = 1,\dots ,\widehat{\ell}$  and for all  $f\neq e$  .Let  $\widehat{\mathcal{L}} = \{1,\ldots ,\widehat{\ell}\}$
7: Construct the matrix  $\widehat{B}$  and the tuple of probability measures  $\widehat{P}$  given by
$$
\widehat {B} = \left( \begin{array}{c c c c} {[ \widehat {B} ^ {1} Q ^ {1} ] _ {\widehat {\mathcal {L}}}} & {[ \widehat {B} ^ {1} Q ^ {1} ] _ {[ \widehat {s} _ {1} ] \setminus \widehat {\mathcal {L}}}} & & \\ \vdots & & \ddots & \\ {[ \widehat {B} ^ {m} Q ^ {m} ] _ {\widehat {\mathcal {L}}}} & & & {[ \widehat {B} ^ {m} Q ^ {m} ] _ {[ \widehat {s} _ {m} ] \setminus \widehat {\mathcal {L}}}} \end{array} \right) \text {a n d} \widehat {P} = \left( \begin{array}{c} {[ (Q ^ {1}) ^ {\top} \# P ^ {1} ] _ {\widehat {\mathcal {L}}}} \\ {[ (Q ^ {1}) ^ {\top} \# P ^ {1} ] _ {[ \widehat {s} _ {1} ] \setminus \widehat {\mathcal {L}}}} \\ \vdots \\ {[ (Q ^ {m}) ^ {\top} \# P ^ {m} ] _ {[ \widehat {s} _ {m} ] \setminus \widehat {\mathcal {L}}}} \end{array} \right).
$$
8: return  $(\widehat{\ell},\widehat{B},\widehat{P})$
graph. That is, one may allow directed paths from shared to domain-specific latent nodes but not vice versa. For example, an additional edge  $1 \rightarrow 4$  between the latent nodes 1 and 4 would be allowed in the graph in Figure 2. In this case, the dependency structure of the domains is still determined by the shared latent space. However, the structural assumption that there are no edges between shared and domain-specific latent nodes is made for identifiability of the shared latent graph in Section 4.
Remark 3.4. The computational complexity of Algorithm 1 depends on the complexity of the chosen linear ICA algorithm, to which we make  $m$  calls. Otherwise, the dominant part is the matching in Line 6 with worst case complexity  $\mathcal{O}(m \cdot \max_{e \in [m]} d_e^2)$ , where we recall that  $m$  is the number of domains and  $d_e$  is the dimension of domain  $e$ .
In Appendix D we state a complete version of Algorithm 1 for the finite sample setting. In particular, we provide a method for the matching in Line 6 based on the two-sample Kolmogorov-Smirnov test. For finite samples, there might occur false discoveries, that is, distributions are matched that are actually not the same. With our method, we show that the probability of falsely discovering shared nodes shrinks exponentially with the number of domains.
4 Identifiability of the Causal Graph
We return to our goal of identifying the causal graph  $\mathcal{G}_{\mathcal{L}} = (\mathcal{L},D_{\mathcal{L}\mathcal{L}})$  among the shared latent nodes. By Theorem 3.1, we can identify the representation  $(B,P)$  of  $P_{X}\in \mathcal{M}(\mathcal{G}_{m})$  from the marginal distributions. In particular, we recover the matrix  $\widehat{B} = B\Psi$  for a signed permutation block matrix  $\Psi \in \Pi$ . Moreover, we know which columns correspond to the shared latent nodes. That is, we know that the submatrix  $\widehat{B}_{\mathcal{L}}$  obtained by only considering the columns indexed by  $\mathcal{L} = \widehat{\mathcal{L}} = [\ell ]$  is equal to  $B_{\mathcal{L}}\Psi_{\mathcal{L}}$ , where  $\Psi_{\mathcal{L}}\in SP(\ell)$ .
Problem 4.1. Let  $B \in \mathrm{Im}(\phi_{\mathcal{G}_m})$  for an  $m$ -domain graph  $\mathcal{G}_m$  with shared latent nodes  $\mathcal{L}$ . Given  $\widehat{B}_{\mathcal{L}} = B_{\mathcal{L}}\Psi_{\mathcal{L}}$  with  $\Psi_{\mathcal{L}}$  a signed permutation matrix, when is it possible to identify the graph  $\mathcal{G}_{\mathcal{L}}$ ?
Recently, Xie et al. (2022) and Dai et al. (2022) show that, in the one-domain setting with independent additive noise, the latent graph can be identified if each latent variable has at least two pure children. We obtain a comparable result tailored to the multi-domain setup.
Definition 4.2. Let  $\mathcal{G}_m = (\mathcal{H} \cup V, D)$  be an  $m$ -domain graph with shared latent nodes  $\mathcal{L} \subseteq \mathcal{H}$ . For  $k \in \mathcal{L}$ , we say that an observed node  $v \in V$  is a partial pure child of  $k$  if  $\mathrm{pa}(v) \cap \mathcal{L} = \{k\}$ .
Algorithm 2 IdentifySharedGraph
1: Input: Matrix  $B^{*}\in \mathbb{R}^{|V|\times \ell}$
2: Output: Parameter matrix  $\widehat{A} \in \mathbb{R}^{\ell \times \ell}$ .
3: Remove rows  $B_{i,\mathcal{L}}^{*}$  from the matrix  $B^{*}$  that are completely zero.
4: Find tuples  $(i_k,j_k)_{k\in \mathcal{L}}$  with  $i_k\neq j_k$  such that
(i)  $\mathrm{rank}(B_{\{i_k,j_k\},\mathcal{L}}^*) = 1$  for all  $k\in \mathcal{L}$  and
(ii)  $\mathrm{rank}(B_{\{i_k,i_q\} ,\mathcal{L}}^*) = 2$  for all  $k,q\in \mathcal{L}$  with  $k\neq q$
5: Let  $I = \{i_1, \ldots, i_\ell\}$  and consider  $B_{I,\mathcal{L}}^* \in \mathbb{R}^{\ell \times \ell}$ .
6: Find two permutation matrices  $R_{1}$  and  $R_{2}$  such that  $W = R_{1}B_{I,\mathcal{L}}^{*}R_{2}$  is lower triangular.
7: Multiply each column of  $\widetilde{W}$  by the sign of its corresponding diagonal element. This yields a new matrix  $\widetilde{W}$  with all diagonal elements positive.
8: Divide each row of  $\widetilde{W}$  by its corresponding diagonal element. This yields a new matrix  $\widetilde{W}'$  with all diagonal elements equal to one.
9: Compute  $\widehat{A} = I - (\widetilde{W}^{\prime})^{-1}$
10: return  $\hat{A}$ .
For a partial pure child  $v \in V$ , there may still be domain-specific latent nodes that are parents of  $v$ . Definition 4.2 only requires that there is exactly one parent that is in the set  $\mathcal{L}$ . This explains the name partial pure child; see Example B.2 in the Appendix for further elaboration.
W.l.o.g. we assume in this section that the shared latent nodes are topologically ordered such that  $i \to j \in D_{\mathcal{L}\mathcal{L}}$  implies  $i < j$  for all  $i, j \in \mathcal{L}$ . We further assume:
(C3) (Two partial pure children across domains.) For each shared latent node  $k \in \mathcal{L}$ , there exist two partial pure children.
(C4) (Rank faithfulness.) For any two subsets  $Y \subseteq V$  and  $W \subseteq \mathcal{L}$ , we assume that
$$
\operatorname {r a n k} \left(B _ {Y, W}\right) = \max  _ {B ^ {\prime} \in \operatorname {I m} \left(\phi_ {\vartheta_ {m}}\right)} \operatorname {r a n k} \left(B _ {Y, W} ^ {\prime}\right).
$$
The two partial pure children required in Condition (C3) may either be in distinct domains or in a single domain. This is a sparsity condition on the large mixing matrix  $G$ . In Appendix C we discuss that the identification of the joint latent graph is impossible without any sparsity assumptions. We conjecture that two partial pure children are not necessary, but we leave it open for future work to find a non-trivial necessary condition. Roughly speaking, we assume in Condition (C4) that no configuration of edge parameters coincidentally yields low rank. The set of matrices  $B \in \mathrm{Im}(\phi_{\mathcal{G}_m})$  that violates (C4) is a subset of measure zero of  $\mathrm{Im}(\phi_{\mathcal{G}_m})$  with respect to the Lebesgue measure. Note that our conditions do not impose constraints on the graph  $\mathcal{G}_{\mathcal{L}}$ . Our main tool to tackle Problem 4.1 will be the following lemma.
Lemma 4.3. Let  $B \in \operatorname{Im}(\phi_{\mathcal{G}_m})$  for an  $m$ -domain graph  $\mathcal{G}_m$ . Suppose that Condition (C4) is satisfied and that there are no zero-rows in  $B_{\mathcal{L}}$ . Let  $v, w \in V$ . Then  $\operatorname{rank}(B_{\{v, w\}, \mathcal{L}}) = 1$  if and only if there is a node  $k \in \mathcal{L}$  such that both  $v$  and  $w$  are partial pure children of  $k$ .
The condition on no zero-rows in Lemma 4.3 is needed since we always have  $\mathrm{rank}(B_{\{v,w\},\mathcal{L}}) \leq 1$  if one of the two rows is zero. However, this is no additional structural assumption since we allow zero-rows when identifying the latent graph; c.f. Algorithm 2. The lemma allows us to find partial pure children by testing ranks on the matrix  $\widehat{B}_{\mathcal{L}}$ . If  $(i_1,j_1)$  and  $(i_2,j_2)$  are partial pure children of two nodes in  $\mathcal{L}$ , we make sure that these two nodes are different by checking that  $\mathrm{rank}(B_{\{i_1,i_2\},\mathcal{L}}) = 2$ .
For a DAG  $G = (V, D)$ , we define  $S(G)$  to be the set of permutations on  $|V|$  elements that are consistent with the DAG, i.e.,  $\sigma \in S(G)$  if and only if  $\sigma(i) < \sigma(j)$  for all edges  $i \to j \in D$ . The following result is the main result of this section.
Theorem 4.4. Let  $\widehat{B} = B\Psi$  with  $B\in Im(\phi_{\mathcal{G}_m})$  and  $\Psi \in \Pi$ , and define  $B^{*} = \widehat{B}_{\mathcal{L}}$  to be the input of Algorithm 2. Assume that Conditions (C3) and (C4) are satisfied, and let  $\widehat{A}$  be the output of Algorithm 2. Then  $\widehat{A} = Q_{\sigma}^{\top}A_{\mathcal{L},\mathcal{L}}Q_{\sigma}$  for a signed permutation matrix  $Q_{\sigma}$  with  $\sigma \in S(\mathcal{G}_{\mathcal{L}})$ . Moreover, if  $G_{vk} > 0$  for  $G\in \mathbb{R}^{D_{V\mathcal{H}}}$  whenever  $v$  is a pure child of  $k$ , then  $Q_{\sigma}$  is a permutation matrix.
Theorem 4.4 says that the graph  $\mathcal{G}_{\mathcal{L}}$  can be recovered up to a permutation of the nodes that preserves the property that  $i \to j$  implies  $i < j$ ; see Remark 4.5. Since the columns of the matrix  $\widehat{B}$  are not only permuted but also of different signs, we solve the sign indeterminacy column-wise in Line 7 before removing the scaling indeterminacy row-wise in Line 8. In case the coefficients of partial pure children are positive, this ensures that  $Q_{\sigma}$  is a permutation matrix and we have no sign indeterminacy. In Appendix D we adapt Algorithm 2 for the empirical data setting, where we only have  $\widehat{B}_{\mathcal{L}} \approx B_{\mathcal{L}} \psi_{\mathcal{L}}$ . Remark 4.5. Let  $\widehat{A}$  be the output of Alg. 2. Then we construct the graph  $\widehat{G}_{\mathcal{L}} = (\mathcal{L}, \widehat{D}_{\mathcal{L}\mathcal{L}})$  as the graph with edges  $j \to i \in \widehat{D}_{\mathcal{L}\mathcal{L}}$  if and only if  $\widehat{A}_{ij} \neq 0$ . Condition (C4) ensures that  $\widehat{G}_{\mathcal{L}}$  is equivalent to  $\mathcal{G}_{\mathcal{L}}$  in the sense that there is a permutation  $\sigma \in S(\mathcal{G}_{\mathcal{L}})$  such that  $\widehat{D}_{\mathcal{L}\mathcal{L}} = \{\sigma(i) \to \sigma(j) : i \to j \in D_{\mathcal{L}\mathcal{L}}\}$ .
Example 4.6. As highlighted in the introduction, the unpaired multi-domain setup is motivated by applications from single-cell biology. For example, consider the domains of (i) gene expression and (ii) high-level phenotypic features extracted from imaging assays (e.g. McQuin et al., 2018). We argue that the requirement of two partial pure children is justifiable on such data as follows. The condition requires, for example, that for each shared latent variable, (i) the expression of some gene depends only upon that shared latent variable plus domain-specific latent variables, and (ii) one of the high-level phenotypic features depends only on the same latent feature plus domain-specific latent variables. Many genes have highly specialized functions, so (i) is realistic, and similarly many phenotypic features are primarily controlled by specific pathways, so (ii) is justified.
Remark 4.7. In Algorithm 2, we determine the rank of a matrix by Singular Value Decomposition, which has worst case complexity  $\mathcal{O}(mn\min \{n,m\})$  for an  $m\times n$  matrix. Since Line 4 is the dominant part, we conclude that the worst case complexity of Algorithm 2 is  $\mathcal{O}(|V|^2\cdot \ell)$
In this section we report on a small simulation study to illustrate the validity of our adapted algorithms for finite samples (detailed in Appendix D). We emphasize that this should only serve as a proof of concept as the focus of our work lies on identifiability. In future work one may develop more sophisticated methods; c.f. Appendix G. The adapted algorithms have a hyperparameter  $\gamma$ , which is a threshold on singular values to determine the rank of a matrix. In our simulations we use  $\gamma = 0.2$ .
Data Generation. In each experiment we generate 1000 random models with  $\ell = 3$  shared latent nodes. We consider different numbers of domains  $m\in \{2,3\}$  and assume that there are  $|I_e| = 2$  domain-specific latent nodes for each domain. The dimensions are given by  $d_{e} = d / m$  for all  $e\in [m]$  and  $d = 30$ . We sample the  $m$ -domain graph  $\mathcal{G}_m$  on the shared latent nodes as follows. First, we sample the graph  $\mathcal{G}_{\mathcal{L}}$  from an Erdős-Rényi model with edge probability 0.75 and assume that there are no edges between other latent nodes, that is, between  $\mathcal{L}$  and  $\mathcal{H}\backslash \mathcal{L}$  and within  $\mathcal{H}\backslash \mathcal{L}$ . Then we fix two partial pure children for each shared latent node  $k\in \mathcal{L}$  and collect them in the set  $W$ . The remaining edges from  $\mathcal{L}$  to  $V\backslash W$  and from  $\mathcal{H}$  to  $V$  are sampled from an Erdős-Rényi model with edge probability 0.9. Finally, the (nonzero) entries of  $G$  and  $A$  are sampled from  $\mathrm{Unif}(\pm [0.25,1])$ . The distributions of the error variables are specified in Appendix E. For simplicity, we assume that the sample sizes coincide, that is,  $n_e = n$  for all  $e\in [m]$ , and consider  $n\in \{1000,2500,5000,10000,25000\}$ .
Results. First, we plot the average number of shared nodes  $\widehat{\ell}$  in our experiments in Figure 3 (a). Especially for low sample sizes, we see that fewer shared nodes are detected with more domains. However, by inspecting the error bars we also see that the probability of detecting too many nodes  $\widehat{\ell} > \ell$  decreases drastically when considering 3 instead of 2 domains. This suggests that the number of falsely detected shared nodes is very low, as expected by Theorem D.3. Our findings show that more domains lead to a more conservative discovery of shared nodes, but whenever a shared node is determined this is more certain. Moreover, we measure the error in estimating  $\widehat{B}_{\widehat{\mathcal{L}}}$  in Figure 3 (b), that is, the error in the "shared" columns. We take
$$
\mathrm {s c o r e} _ {B} (\widehat {B} _ {\widehat {\mathcal {L}}}) = \left\{ \begin{array}{l l} \min  _ {\Psi \in S P (\ell)} \beta_ {\ell , \widehat {\ell}} ^ {- 1 / 2} \| \widehat {B} _ {\widehat {\mathcal {L}}} - [ B _ {\mathcal {L}} \Psi ] _ {\widehat {\mathcal {L}}} \| _ {\mathcal {F}} & \text {i f} \widehat {\ell} \leq \ell , \\ \min  _ {\Psi \in S P (\widehat {\ell})} \beta_ {\ell , \widehat {\ell}} ^ {- 1 / 2} \| [ \widehat {B} _ {\widehat {\mathcal {L}}} \Psi ] _ {\mathcal {L}} - B _ {\mathcal {L}} \| _ {\mathcal {F}} & \text {i f} \widehat {\ell} > \ell , \end{array} \right.
$$
where  $\| \cdot \|_{\mathcal{F}}$  denotes the Frobenius norm and  $\beta_{\ell, \widehat{\ell}} = \min \{\ell, \widehat{\ell}\} \cdot \sum_{e=1}^{m} d_e$  denotes the number of entries of the matrix over which the norm is taken. In the cases  $\ell = \widehat{\ell}$ , we also measure the
(b)
Figure 3: Results. Logarithmic scale on the  $x$ -axis. Error bars in (a) are one standard deviation of the mean and in (b) and (c) they are the interquartile range.
performance of recovering the shared latent graph  $\mathcal{G}_{\mathcal{L}}$  in Figure 3 (c) by taking
$$
\operatorname {s c o r e} _ {A} (\widehat {A}) = \min  _ {Q _ {\sigma} \in S P (\ell) \text {s . t .} \sigma \in S (\mathcal {G} _ {\mathcal {L}})} \frac {1}{\ell} \| Q _ {\sigma} ^ {\top} \widehat {A} Q _ {\sigma} - A _ {\mathcal {L}, \mathcal {L}} \| _ {\mathcal {F}}.
$$
As expected, the median estimation errors for  $B_{\mathcal{L}}$  and  $A_{\mathcal{L},\mathcal{L}}$  decrease with increasing sample size. In Appendix F we provide additional simulations with larger  $\ell$ . Moreover, we consider setups where we violate specific assumptions, such as pairwise different distributions (C1) and two partial pure children (C3). The results emphasize that the conditions are necessary for the algorithms provided. The computations were performed on a single thread of an Intel Xeon Gold 6242R processor (3.1 GHz), with a total computation time of 12 hours for all simulations presented in this paper (including Appendix).
This work introduces the problem of causal representation learning from unpaired multi-domain observations, in which multiple domains provide complementary information about a set of shared latent nodes that are the causal quantities of primary interest. For this problem, we laid out a setting in which we can provably identify the causal relations among the shared latent nodes. To identify the desired causal structure, we proposed a two-step approach where we first make use of linear ICA in each domain separately and match the recovered error distributions to identify shared nodes and the joint distribution of the domains. In the second step, we identify the causal structure among the shared latent variables by testing rank deficiencies in the "overall mixing matrix"  $B$ . To the best of our knowledge, our guarantees are the first principled identifiability results for shared causal representations in a general, unpaired multi-domain setting.
We proposed algorithms for recovering the joint distribution and the shared latent space making our proofs constructive. While our focus is on identifiability guarantees, we show in Appendix D how our proofs give rise to algorithms for the finite sample setting. Moreover, we propose a method to match approximate error distributions and show that the probability of falsely discovering shared nodes decreases exponentially in the number of domains. Our work opens up numerous directions for future work as we discuss in Appendix G.
This project was initiated while the first author was a visitor at the Eric and Wendy Schmidt Center of the Broad Institute of MIT and Harvard. The project has received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (grant agreement No 883818), NCCIH/NIH (1DP2AT012345), ONR (N00014-22-1-2116), DOE-ASCR (DE-SC0023187), NSF (DMS-1651995), the MIT-IBM Watson AI Lab, and a Simons Investigator Award. Nils Sturma acknowledges support by the Munich Data Science Institute at the Technical University of Munich via the Linde/MDSI PhD Fellowship program. Chandler Squires was partially supported by an NSF Graduate Research Fellowship.
Proof of Theorem 3.1. Let  $P_X \in \mathcal{M}(\mathcal{G}_m)$  for an  $m$ -domain graph  $\mathcal{G}_m = (\mathcal{H} \cup V, D)$  with shared latent nodes  $\mathcal{L} = [\ell]$  and representation  $(B, P)$ . By Condition (C1) the measure  $P$  has independent, non-degenerate, non-symmetric marginals  $P_i$ ,  $i \in \mathcal{H}$  with mean zero and variance one. Moreover, since  $B \in \mathrm{Im}(\phi_{\mathcal{G}_m})$ , we have  $B = G(I - A)^{-1}$  for matrices  $G \in \mathbb{R}^{D_V\mathcal{H}}$  and  $A \in \mathbb{R}^{D_{\mathcal{H}\mathcal{H}}}$ .
Fix one domain  $e \in [m]$ . Recall that we denote by  $S_{e} = \mathrm{pa}(V_{e}) = \mathcal{L} \cup I_{e}$  the set of latent parents in domain  $e$ . Define the matrix
$$
B ^ {e} := G _ {V _ {e}, S _ {e}} [ (I - A) ^ {- 1} ] _ {S _ {e}, S _ {e}} = G ^ {e} [ (I - A) ^ {- 1} ] _ {S _ {e}, S _ {e}},
$$
and observe that we can write  $P_{X^e} = B_{V_e, \mathcal{H}} \# P = B^e \# P_{S_e}$ . This is due to the fact that  $G_{V_e, \mathcal{H} \setminus S_e} = 0$  and  $[(I - A)^{-1}]_{S_e, \mathcal{H} \setminus S_e} = 0$  by the definition of an  $m$ -domain graph.