SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof. For each of the three cases, we will construct another  $m$ -domain graph  $\mathcal{G}_m = (\mathcal{H} \cup V, D)$  such that for suitable representations  $(\widetilde{B}, \widetilde{P})$  of distributions in  $\mathcal{M}(\widetilde{\mathcal{G}}_m)$ , the left-hand side of (9) holds, but the right-hand side is violated.
To prove the statement for case (i), let  $i, j \in \mathcal{L}$  and assume that  $P_{i} = P_{j}$ . We define the  $m$ -domain graph  $\widetilde{\mathcal{G}}_m = (\widetilde{\mathcal{H}} \cup V, \widetilde{D})$  to be the almost same graph as  $\mathcal{G}_m = (\mathcal{H} \cup V, D)$ , we only "swap" the roles of the latent nodes  $i$  and  $j$  on an arbitrary domain  $e \in [m]$ . That is, for each  $v \in V_e$ , if there was an edge  $i \rightarrow v$  in  $D$ , we remove that edge from  $\widetilde{D}$  and add the edge  $j \rightarrow v$  instead, and vice versa. Otherwise, the graph  $\widetilde{\mathcal{G}}_m$  has the same structure as  $\mathcal{G}_m$ . Now, let  $\widetilde{P} = P$  and define a the matrix  $\widetilde{B}$  to be the same matrix as  $B$ , except for the subcolumns  $\widetilde{B}_{V_e,i} := B_{V_e,j}$  and  $\widetilde{B}_{V_e,j} := B_{V_e,i}$ , that is, we swapped  $B_{V_e,i}$  and  $B_{V_e,j}$ . Then the pair  $(\widetilde{B}, \widetilde{P})$  is a representation of some distribution in  $\mathcal{M}(\widetilde{\mathcal{G}}_m)$ . Recall from the proof of Theorem 3.1 that Condition (C2) implies that the matrix  $B_{V_e,S_e}$  has full column rank. Since we only swapped columns in  $\widetilde{B}_{V_e,\widetilde{S}_e}$ , it still has full column rank. Moreover, observe that the left hand side of (9) is satisfied since  $P_i = P_j$ , that is, the marginal distributions on the single domains coincide.
However, now consider another domain  $f \in [m]$  and the submatrices
$$
B _ {V _ {e} \cup V _ {f}, \{i, j \}} = \left( \begin{array}{c c} B _ {V _ {e}, i} & B _ {V _ {e}, j} \\ B _ {V _ {f}, i} & B _ {V _ {f}, j} \end{array} \right) \quad \text {a n d} \quad \widetilde {B} _ {V _ {e} \cup V _ {f}, \{i, j \}} = \left( \begin{array}{c c} B _ {V _ {e}, j} & B _ {V _ {e}, i} \\ B _ {V _ {f}, i} & B _ {V _ {f}, j} \end{array} \right).
$$
Since all of the four subcolumns are nonzero and neither  $B_{V_e,j}$  is equal to  $B_{V_e,i}$  nor  $B_{V_f,j}$  is equal to  $B_{V_f,i}$ , there is no signed permutation matrix  $\Omega$  such that  $B_{V_e\cup V_f,\{i,j\}} = \widetilde{B}_{V_e\cup V_f,\{i,j\}}\Omega$ . Hence, there is also no larger signed permutation matrix  $\Psi$  such that  $B = \widetilde{B}\Psi$ . We conclude that the right-hand side of (9) is violated and the joint distribution is not identifiable. Finally, note that the above arguments also hold if  $P_i = -P_j$  by adding “-” signs in appropriate places.
The proof for case (ii) works with exactly the same construction. That is, for  $i \in \mathcal{L}$  and  $j \in I_e$  we swap the roles of  $i$  and  $j$  on the domain  $e$ . Then, for any other domain  $f \in [m]$ , we obtain the submatrices
$$
B _ {V _ {e} \cup V _ {f}, \{i, j \}} = \left( \begin{array}{c c} B _ {V _ {e}, i} & B _ {V _ {e}, j} \\ B _ {V _ {f}, i} & 0 \end{array} \right) \quad \text {a n d} \quad \widetilde {B} _ {V _ {e} \cup V _ {f}, \{i, j \}} = \left( \begin{array}{c c} B _ {V _ {e}, j} & B _ {V _ {e}, i} \\ B _ {V _ {f}, i} & 0 \end{array} \right).
$$
By the same arguments as before, this shows that there is no signed permutation matrix  $\Psi$  such that  $B = \widetilde{B}\Psi$  and, hence, the joint distribution is not identifiable.
To prove case (iii), we consider a slightly different construction. However, we also assume that  $P_{i_e} = P_{i_f}$  for all  $e \neq f$ , since for  $P_{i_e} = -P_{i_f}$  we only have to add some “-” signs in the following. We define the  $m$ -domain graph  $\widetilde{\mathcal{G}}_m = (\widetilde{\mathcal{H}} \cup V, \widetilde{D})$  by identifying the nodes  $i_e, e \in [m]$  with a new
node  $k$ . That is,  $\widetilde{\mathcal{L}} = \mathcal{L} \cup \{k\}$  and  $\widetilde{\mathcal{H}} = (\bigcup_{e \in [m]} I_e \setminus \{i_e\}) \cup \widetilde{\mathcal{L}}$ . For  $i \in \widetilde{\mathcal{H}} \setminus \{k\}$  and  $v \in V$ , the edge set  $\widetilde{D}$  contains an edge  $i \to v$  if and only if the edge  $i \to v$  is in  $D$ . For the node  $k \in \widetilde{\mathcal{H}}$  and  $v \in V$ , we put an edge  $k \to v$  in  $\widetilde{D}$  if and only if there is an edge  $i_e \to v$  in  $D$  for some  $e \in [m]$ .
Now, define the matrix  $\widetilde{B}$  such that  $\widetilde{B}_{V,\widetilde{\mathcal{H}}\backslash \{k\}}\coloneqq B_{V,\widetilde{\mathcal{H}}\backslash \{k\}}$  and  $\widetilde{B}_{V_e,k}\coloneqq B_{V_e,i_e}$  for all  $e\in [m]$ . Then the pair  $(\widetilde{B},\widetilde{P})$  is a representation of some distribution in  $\mathcal{M}(\widetilde{\mathcal{G}}_m)$ . Moreover, each submatrix  $\widetilde{B}_{V_e,\widetilde{S}_e}$  is equal to  $B_{V_e,S_e}$  up to relabeling of the columns. That is, the column that is labeled by  $i_e$  in  $B_{V_e,S_e}$  is now labeled by  $k$  in  $\widetilde{B}_{V_e,\widetilde{S}_e}$ . We define the measure  $\widetilde{P}$  such that  $\widetilde{P}_{\widetilde{\mathcal{H}}\backslash \{k\}} = P_{\widetilde{\mathcal{H}}\backslash \{k\}}$  and  $\widetilde{P}_k = P_{i_e}$  for all  $e\in [m]$ . Then the pair  $(\widetilde{B},\widetilde{P})$  is a representation of some distribution in  $\mathcal{M}(\widetilde{\mathcal{G}}_m)$  and, in particular, the left hand side of (9) is satisfied. That is, the marginal distributions coincide on each domain. However, the number of shared latent variables in both  $m$ -domain graphs is different since we have  $\widetilde{\ell} = \ell +1$ . We conclude that the joint distribution is not identifiable.
The proposition states that it is in most cases necessary that error distributions are pairwise different. However, in two cases the same error distributions still lead to identifiability. First, if  $i, j \in I_e$ , then the corresponding error distributions may be the same and the joint distribution is still identifiable. Similarly, if there are latent nodes  $i_e$  in a few domains  $e \in [m]$  such that the corresponding error distributions coincide, but there is at least one domain  $f \in [m]$  where there is no latent node with the same error distribution, then the joint distribution is also identifiable. Both can be seen by taking the proofs of Theorem 3.1 and Proposition C.1 together.
Gaussian Errors. Without additional assumptions to those in Section 3, it is impossible to recover the joint distribution if the distributions of the errors  $\varepsilon_{i}$  of the latent structural equation model in Equation (1) are Gaussian. In this case, the distribution of  $Z$  as well as the distribution of each observed random vector  $X^{e}$  is determined by the covariance matrix only. The observed covariance matrix in domain  $e\in [m]$  is given by  $\Sigma^e = G^e\mathrm{Cov}[Z_{\mathcal{L}\cup I_e}](G^e)^\top$ . However, knowing  $\Sigma^e$  gives no information about  $Z_{\mathcal{L}\cup I_e}$  other than  $\mathrm{rank}(\Sigma^e) = |\mathcal{L}| + |I_e|$ , that is, we cannot distinguish which latent variables are shared and which ones are domain-specific. This is formalized in the following lemma.
Lemma C.2. Let  $\Sigma$  be any  $d\times d$  symmetric positive semidefinite matrix of rank  $p$  and let  $\Xi$  be another arbitrary  $p\times p$  symmetric positive definite matrix. Then there is  $G\in \mathbb{R}^{d\times p}$  such that  $\Sigma = G\Xi G^{\top}$ .
Proof. Let  $\Sigma$  be a  $d \times d$  symmetric positive semidefinite matrix of rank  $p$ . Then,  $\Sigma$  has a decomposition similar to the Cholesky decomposition; see e.g. Gentle (1998, Section 3.2.2). That is, there exists a unique matrix  $T$ , such that  $A = TT^{\top}$ , where  $T$  is a lower triangular matrix with  $p$  positive diagonal elements and  $d - p$  columns containing all zeros. Define  $\widetilde{T}$  to be the  $d \times p$  matrix containing only the non-zero columns of  $T$ .
On the other hand, let  $\Xi$  be a symmetric positive definite  $p\times p$  matrix. By the usual Cholesky decomposition (Lyche, 2020, Section 4.2.1), there exists a unique  $p\times p$  lower triangular matrix  $L$  with positive diagonal elements such that  $\Xi = LL^{\top}$ . Now, define  $G\coloneqq \widetilde{T} L^{-1}\in \mathbb{R}^{d\times p}$ . Then,
$$
\Sigma = \widetilde {T} \widetilde {T} ^ {\top} = \widetilde {T} L ^ {- 1} L L ^ {\top} L ^ {- \top} \widetilde {T} ^ {\top} = G \Xi G ^ {\top}.
$$
Due to Lemma C.2 it is necessary to consider non-Gaussian distributions to obtain identifiability of the joint distribution.
Example C.3. In the Gaussian case we cannot distinguish whether the two observed domains in Figure 5 share a latent variable or not. Said differently, the observed marginal distributions may either be generated by the mechanism defined by graph (a) or graph (b) and there is no way to distinguish from observational distributions only.
Sparsity Assumptions. Let  $B \in \mathrm{Im}(\phi_{\mathcal{G}_m})$  for an  $m$ -domain graph  $\mathcal{G}_m = (\mathcal{H} \cup V, D)$  and suppose that we are given the matrix  $B_{\mathcal{L}} = G_{V,\mathcal{L}}(I - A_{\mathcal{L},\mathcal{L}})^{-1}$ , that is, we are given the submatrix with columns indexed by the shared latent nodes. Now, assume that the graph does not impose any sparsity restrictions on  $G_{V,\mathcal{L}}$ , which means that the set  $\mathbb{R}^{D_{V\mathcal{L}}}$  of possible matrices  $G_{V,\mathcal{L}}$  is equal to  $\mathbb{R}^{|V| \times |\mathcal{L}|}$ . Then, the set of possible matrices  $B_{\mathcal{L}}$  is also unrestricted, that is,  $B_{\mathcal{L}}$  can be any matrix in  $\mathbb{R}^{|V| \times |\mathcal{L}|}$  no matter the form of the matrix  $A_{\mathcal{L},\mathcal{L}} \in \mathbb{R}^{D_{\mathcal{L}\mathcal{C}}}$ . In other words, for arbitrary shared latent graphs
(b)
Figure 5: Compact versions of two 2-domain graphs. In both graphs, both domains have two latent parents. In setup (a) there is a shared latent parent while in setup (b) there is not.
$\mathcal{G}_{\mathcal{L}} = (\mathcal{L},D_{\mathcal{LC}})$  and arbitrary corresponding parameter matrices  $A_{\mathcal{L},\mathcal{L}}\in \mathbb{R}^{D_{\mathcal{LC}}}$ , we don't get any restrictions on the matrix  $B_{\mathcal{L}}$ . Therefore, it is impossible to infer  $A_{\mathcal{L},\mathcal{L}}$  from  $B_{\mathcal{L}}$ .
Condition (C3) requires that there are two partial pure children for every shared latent node  $k \in \mathcal{L}$ , which implies that there are  $2|\mathcal{L}|$  rows in  $G_{V,\mathcal{L}}$  in which only one entry may be nonzero. While we show in Theorem 4.4 that this condition is sufficient for identifiability of  $A_{\mathcal{L},\mathcal{L}}$ , we leave it open for future work to find a necessary condition.
D Algorithms for Finite Samples
We adjust Algorithm 1 such that it is applicable in the empirical data setting. That is, rather than the exact distribution  $P_{X^e}$ , we have a matrix of observations  $X^e \in \mathbb{R}^{d_e \times n_e}$  in each domain  $e \in [m]$ . The sample size  $n_e$  might be different across domains. We denote  $n_{\mathrm{min}} = \min_{e \in [m]} n_e$  and  $n_{\mathrm{max}} = \max_{e \in [m]} n_e$ . For implementing linear ICA on finite samples, multiple well developed algorithms are available, e.g., FastICA (Hyvärinen, 1999; Hyvärinen and Oja, 2000), Kernel ICA (Bach and Jordan, 2003) or JADE (Cardoso and Souloumiac, 1993). Applying them, we obtain a measure  $\widehat{P}_i^e$  which is an estimator of the true measure  $P_i^e$  in Algorithm 1, Line 4.
The remaining challenge is the matching in Line 6 of Algorithm 1. For finite samples, the distance between empirical distributions is almost surely not zero although the true underlying distributions might be equal. In this section, we provide a matching strategy based on the two-sample Kolmogorov-Smirnov test (van der Vaart and Wellner, 1996, Section 3.7). We match two distributions if they are not significantly different. During this process, there might occur false discoveries, that is, distributions are matched that are actually not the same. We show that the probability of falsely discovering shared nodes shrinks exponentially with the number of domains.
For two univariate Borel probability measures  $P_{i}, P_{j}$ , with corresponding cumulative distribution functions  $F_{i}, F_{j}$ , the Kolmogorov-Smirnov distance is given by the  $L^{\infty}$ -distance
$$
d _ {\mathrm {K S}} \left(P _ {i}, P _ {j}\right) = \| F _ {i} - F _ {j} \| _ {\infty} = \sup  _ {x \in \mathbb {R}} | F _ {i} (x) - F _ {j} (x) |.
$$
The two-sample Kolmogorov-Smirnov test statistic for the null hypothesis  $H_0: d_{\mathrm{KS}}(P_i^e, P_j^f) = 0$  is given by
$$
T \left(\widehat {P} _ {i} ^ {e}, \widehat {P} _ {j} ^ {f}\right) = \sqrt {\frac {n _ {e} n _ {f}}{n _ {e} + n _ {f}}} d _ {\mathrm {K S}} \left(\widehat {P} _ {i} ^ {e}, \widehat {P} _ {j} ^ {f}\right). \tag {10}
$$
It is important to note that  $\widehat{P}_i^e$  is not an empirical measure in the classical sense since it is not obtained from data sampled directly from the true distribution  $P_i^e$ . In addition to the sampling error there is the uncertainty of the ICA algorithm. However, in the analysis we present here, we will neglect this error and treat  $\widehat{P}_i^e$  as an empirical measure. In this case, under  $H_0$ , the test statistic  $T(\widehat{P}_i^e, \widehat{P}_j^f)$  converges in distribution to  $\|B\|_{\infty}$ , where  $B$  is a Brownian bridge from 0 to 1 (van der Vaart and Wellner, 1996, Section 2.1). For a given level  $\alpha \in (0,1)$ , we choose the critical value as  $c_\alpha = \inf \{t : P(\|B\|_{\infty} > t) \leq \alpha\}$  and reject  $H_0$  if  $T(\widehat{P}_i^e, \widehat{P}_j^f) > c_\alpha$ .
Definition D.1. Let  $\alpha \in (0,1)$  and suppose the distributions  $\{\widehat{P}_1^e,\dots ,\widehat{P}_{\widehat{s}_e}^e\}$  and  $\{\widehat{P}_1^f,\dots ,\widehat{P}_{\widehat{s}_f}^f\}$  are given for two domains  $e,f\in [m]$ . Define
$$
\Omega_ {\alpha} (\widehat {P} _ {i} ^ {e}, \widehat {P} _ {j} ^ {f}) = \left\{ \begin{array}{l l} 1 & \text {i f} T _ {i j} ^ {e f} \leq c _ {\alpha} \text {a n d} T _ {i j} ^ {e f} = \min  \{\min  _ {k \in [ \widehat {s} _ {f} ]} T _ {i k} ^ {e f}, \min  _ {k \in [ \widehat {s} _ {e} ]} T _ {k j} ^ {e f} \}, \\ 0 & \text {e l s e}, \end{array} \right.
$$
where  $T_{ij}^{ef} = \min \{T(\widehat{P}_i^e,\widehat{P}_j^f),T(\widehat{P}_i^e, - \widehat{P}_j^f)\}$ . We say that  $\widehat{P}_i^e$ ,  $\widehat{P}_j^f$  are matched if  $\Omega_{\alpha}(\widehat{P}_i^e,\widehat{P}_j^f) = 1$ .
Definition D.1 essentially states that two measures are matched if the test statistic (10) is not significantly large and the null hypothesis cannot be rejected. Taking the minimum of  $T(\widehat{P}_i^e, \widehat{P}_j^f)$  and  $T(\widehat{P}_i^e, -\widehat{P}_j^f)$  accounts for the sign indeterminacy of linear ICA. For two fixed domains  $e, f \in [m]$ , if it happens that the statistic  $T_{ij}^{ef}$  for multiple pairs  $(i,j)$  is small enough, then the pair with the minimal value of the statistic is matched. Note that one may use any other test than the Kolmogorov-Smirnov test to define a matching as in Definition D.1. We discover a shared latent node if it is matched consistently across domains.
Definition D.2. Let  $C = (i_1, \ldots, i_m)$  be a tuple with  $m$  elements such that  $i_e \in [\widehat{s}_e]$ . Then we say that  $C$  determines a shared node if  $\Omega_{\alpha}(\widehat{P}_{i_e}^e, \widehat{P}_{i_f}^f) = 1$  for all  $i_e, i_f \in C$ .
Inferring the existence of a shared node which does not actually exist may be considered a more serious error than inferring a shared node determined by a set  $C$ , where only some components of  $C$  are wrongly matched. In the following theorem we show that the probability of falsely discovering shared nodes shrinks exponentially with the number of wrongly matched components.
Theorem D.3. Let  $C = (i_1, \ldots, i_m)$  be a tuple with  $m$  elements such that  $i_e \in [\widehat{s}_e]$ . Let  $g: \mathbb{N} \times \mathbb{R}_{\geq 0} \to \mathbb{R}_{\geq 0}$  be a function that is monotonically decreasing in  $n \in \mathbb{N}$  and assume the following:
(i)  $P(d_{KS}(\widehat{P}_{i_e}^e,P_{i_e}^e) > x)\leq g(n_e,x)$  for all  $e\in [m]$  and for all  $x\geq 0$
(ii) There is  $E \subseteq [m]$  with  $|E| \geq 2$  and a constant  $\kappa > 0$  such that  $d_{KS}(P_{i_e}^e, P_{i_f}^f) \geq \kappa$  and  $d_{KS}(P_{i_e}^e, -P_{i_f}^f) \geq \kappa$  for all  $e \in E$ ,  $f \in [m]$  with  $e \neq f$ .
$$
P (C \text {d e t e r m i n e s a s h a r d n o d e}) \leq g \left(n _ {\min }, \max  \left\{\frac {\kappa}{2} - \frac {\sqrt {n _ {\max }}}{\sqrt {2}} n _ {\min } c _ {\alpha}, 0 \right\}\right) ^ {| E | - 1}.
$$
Proof of Theorem D.3. Let  $C = (i_1, \ldots, i_m)$  and  $g$  be as in the statement of the theorem. W.l.o.g. we assume that  $E = \{1, \ldots, |E|\}$  and that  $T(\widehat{P}_{i_e}^e, \widehat{P}_{i_f}^f) \leq T(\widehat{P}_{i_e}^e, -\widehat{P}_{i_f}^f)$ . Observe that  $C$  determines a shared node if and only if
$$
\sum_ {e <   f} \Omega_ {\alpha} (\widehat {P} _ {i _ {e}} ^ {e}, \widehat {P} _ {i _ {f}} ^ {f}) = \binom {m} {2}.
$$
$$
\begin{array}{l} P \left(\sum_ {e <   f} \Omega_ {\alpha} (\widehat {P} _ {i _ {e}} ^ {e}, \widehat {P} _ {i _ {f}} ^ {f}) = \binom {m} {2}\right) = P \left(\bigcap_ {e <   f} \left\{\Omega_ {\alpha} (\widehat {P} _ {i _ {e}} ^ {e}, \widehat {P} _ {i _ {f}} ^ {f}) = 1 \right\}\right) \\ \leq P \left(\bigcap_ {e <   f} \left\{T _ {i j} ^ {e f} \leq c _ {\alpha} \right\}\right) \\ = P \left(\bigcap_ {e <   f} \left\{T \left(\widehat {P} _ {i _ {e}} ^ {e}, \widehat {P} _ {i _ {f}} ^ {f}\right) \leq c _ {\alpha} \right\} \cup \left\{T \left(\widehat {P} _ {i _ {e}} ^ {e}, - \widehat {P} _ {i _ {f}} ^ {f}\right) \leq c _ {\alpha} \right\}\right) \\ = P \left(\bigcap_ {e <   f} \left\{T \left(\widehat {P} _ {i _ {e}} ^ {e}, \widehat {P} _ {i _ {f}} ^ {f}\right) \leq c _ {\alpha} \right\}\right) \\ \leq P \left(\bigcap_ {\substack {e \in E, f \in [ m ] \\ e <   f}} \left\{T \left(\hat {P} _ {i _ {e}} ^ {e}, \hat {P} _ {i _ {f}} ^ {f}\right) \leq c _ {\alpha} \right\}\right). \tag{11} \\ \end{array}
$$
By the triangle inequality we have
$$
d _ {\mathrm {K S}} \left(P _ {i _ {e}} ^ {e}, P _ {i _ {f}} ^ {f}\right) \leq d _ {\mathrm {K S}} \left(P _ {i _ {e}} ^ {e}, \widehat {P} _ {i _ {e}} ^ {e}\right) + d _ {\mathrm {K S}} \left(\widehat {P} _ {i _ {e}} ^ {e}, \widehat {P} _ {i _ {f}} ^ {f}\right) + d _ {\mathrm {K S}} \left(\widehat {P} _ {i _ {f}} ^ {f}, P _ {i _ {f}} ^ {f}\right). \tag {12}
$$
Moreover, if  $e \in E$  and  $f \in [m]$ , then we have by Condition (ii)
$$
d _ {\mathrm {K S}} \left(P _ {i _ {e}} ^ {e}, P _ {i _ {f}} ^ {f}\right) \geq \kappa . \tag {13}
$$
Using (12) and (13) together with the fact  $\sqrt{\frac{n_{e}n_{f}}{n_{e} + n_{f}}} \geq \frac{n_{\min}}{\sqrt{2n_{\max}}}$ , we obtain the following chain of implications for  $e \in E$  and  $f \in [m]$ :
$$
\begin{array}{l} T (\widehat {P} _ {i _ {e}} ^ {e}, \widehat {P} _ {i _ {f}} ^ {f}) \leq c _ {\alpha} \\ \Longleftrightarrow \sqrt {\frac {n _ {e} n _ {f}}{n _ {e} + n _ {f}}} d _ {\mathrm {K S}} \left(\widehat {P} _ {i _ {e}} ^ {e}, \widehat {P} _ {i _ {f}} ^ {f}\right) \leq c _ {\alpha} \\ \Longrightarrow \frac {n _ {\operatorname* {m i n}}}{\sqrt {2 n _ {\operatorname* {m a x}}}} \left(d _ {\mathrm {K S}} \left(P _ {i _ {e}} ^ {e}, P _ {i _ {f}} ^ {f}\right) - d _ {\mathrm {K S}} \left(\widehat {P} _ {i _ {e}} ^ {e}, P _ {i _ {e}} ^ {e}\right) - d _ {\mathrm {K S}} \left(\widehat {P} _ {i _ {f}} ^ {f}, P _ {i _ {f}} ^ {f}\right)\right) \leq c _ {\alpha} \\ \Rightarrow \frac {n _ {\mathrm {m i n}}}{\sqrt {2 n _ {\mathrm {m a x}}}} \left(\kappa - d _ {\mathrm {K S}} (\widehat {P} _ {i _ {e}} ^ {e}, P _ {i _ {e}} ^ {e}) - d _ {\mathrm {K S}} (\widehat {P} _ {i _ {f}} ^ {f}, P _ {i _ {f}} ^ {f})\right) \leq c _ {\alpha} \\ \Longleftrightarrow d _ {\mathrm {K S}} (\widehat {P} _ {i _ {e}} ^ {e}, P _ {i _ {e}} ^ {e}) + d _ {\mathrm {K S}} (\widehat {P} _ {i _ {f}} ^ {f}, P _ {i _ {f}} ^ {f}) \geq \kappa - \frac {\sqrt {2 n _ {\max}}}{n _ {\min}} c _ {\alpha} \\ \Longrightarrow \left\{d _ {\mathrm {K S}} (\widehat {P} _ {i _ {e}} ^ {e}, P _ {i _ {e}} ^ {e}) \geq \frac {\kappa}{2} - \frac {\sqrt {n _ {\mathrm {m a x}}}}{\sqrt {2} n _ {\mathrm {m i n}}} c _ {\alpha} \right\} \mathrm {o r} \left\{d _ {\mathrm {K S}} (\widehat {P} _ {i _ {f}} ^ {f}, P _ {i _ {f}} ^ {f}) \geq \frac {\kappa}{2} - \frac {\sqrt {n _ {\mathrm {m a x}}}}{\sqrt {2} n _ {\mathrm {m i n}}} c _ {\alpha} \right\}. \\ \end{array}
$$
Now, consider the event  $\bigcap_{\substack{e\in E,f\in [m]\\ e <   f}}\{T(\widehat{P}_{i_e}^e,\widehat{P}_{i_f}^f)\leq c_\alpha \}$ . On this event, there cannot be two elements  $e,f\in E$  such that both
$$
\left\{d _ {\mathrm {K S}} \left(\widehat {P} _ {i _ {e}} ^ {e}, P _ {i _ {e}} ^ {e}\right) <   \frac {\kappa}{2} - \frac {\sqrt {n _ {\operatorname* {m a x}}}}{\sqrt {2} n _ {\min }} c _ {\alpha} \right\} \text {a n d} \left\{d _ {\mathrm {K S}} \left(\widehat {P} _ {i _ {f}} ^ {f}, P _ {i _ {f}} ^ {f}\right) <   \frac {\kappa}{2} - \frac {\sqrt {n _ {\operatorname* {m a x}}}}{\sqrt {2} n _ {\min }} c _ {\alpha} \right\}.
$$
To see this recall that  $E \subseteq [m]$ . We conclude that it must hold  $\{d_{\mathrm{KS}}(\widehat{P}_{i_e}^e, P_{i_e}^e) \geq \frac{\kappa}{2} - \frac{\sqrt{n_{\max}}}{\sqrt{2} n_{\min}} c_\alpha \}$  for all but at most one element of  $E$ . We denote this exceptional element by  $e^* \in E$ . Taking up (11), we get the following:
$$
\begin{array}{l} P \left(\sum_ {e <   f} \Omega_ {\alpha} (\widehat {P} _ {i _ {e}} ^ {e}, \widehat {P} _ {i _ {f}} ^ {f}) = \binom {m} {2}\right) \\ \leq P\Bigg(\bigcap_{\substack{e\in E,f\in [m]\\ e <   f}}\left\{T(\widehat{P}_{i_{e}}^{e},\widehat{P}_{i_{f}}^{f})\leq c_{\alpha}\right\} \Bigg) \\ \leq P \left(\bigcap_ {e \in E \backslash \{e ^ {*} \}} \left\{d _ {\mathrm {K S}} \left(\widehat {P} _ {i _ {e}} ^ {e}, P _ {i _ {e}} ^ {e}\right) \geq \frac {\kappa}{2} - \frac {\sqrt {n _ {\operatorname* {m a x}}}}{\sqrt {2} n _ {\operatorname* {m i n}}} c _ {\alpha} \right\}\right) \\ = \prod_ {e \in E \backslash \{e ^ {*} \}} P \left(d _ {\mathrm {K S}} \left(\widehat {P} _ {i _ {e}} ^ {e}, P _ {i _ {e}} ^ {e}\right) \geq \frac {\kappa}{2} - \frac {\sqrt {n _ {\operatorname* {m a x}}}}{\sqrt {2} n _ {\min }} c _ {\alpha}\right) (14) \\ = \prod_ {e \in E \backslash \{e ^ {*} \}} P \left(d _ {\mathrm {K S}} \left(\widehat {P} _ {i _ {e}} ^ {e}, P _ {i _ {e}} ^ {e}\right) \geq \max  \left\{\frac {\kappa}{2} - \frac {\sqrt {n _ {\operatorname* {m a x}}}}{\sqrt {2} n _ {\operatorname* {m i n}}} c _ {\alpha}, 0 \right\}\right) (15) \\ \leq g \left(n _ {\min }, \max  \left\{\frac {\kappa}{2} - \frac {\sqrt {n _ {\operatorname* {m a x}}}}{\sqrt {2} n _ {\min }} c _ {\alpha}, 0 \right\}\right) ^ {| E | - 1}. (16) \\ \end{array}
$$
The last three steps need more explanation: Equality (14) follows from the fact that domains are unpaired. That is, the distances  $d_{\mathrm{KS}}(\widehat{P}_{i_e}^e,P_{i_e}^e)$  and  $d_{\mathrm{KS}}(\widehat{P}_{i_f}^f,P_{i_f}^f)$  are pairwise independent for different domains  $e,f\in [m]$ . Equality (15) is trivial since  $d_{\mathrm{KS}}(\widehat{P}_{i_e}^e,P_{i_e}^e)\geq 0$ . Finally, Inequality (16) follows from Condition (i) and that the function  $g$  is monotonically decreasing in  $n$ . We also used the fact that  $|E\setminus \{e^{*}\}| = |E| - 1$ .
If  $\widehat{P_i^e}$  were an empirical measure in the classical sense, then Condition (i) in Theorem D.3 translates to the well-known Dvoretzky-Kiefer-Wolffowitz inequality, that is, the function  $g$  is given by  $g(n,x) =$
Algorithm 3 IdentifyJointDistributionEmpirical