SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\rho_ {\mu_ {1}, \mu_ {2}, \dots , \mu_ {\ell}} (\mathbb {P}, \mathbb {Q}) = \sum_ {j = 1} ^ {\ell} \left(\left[ \mu_ {j} \mathbb {P} \right] \left(\boldsymbol {v} _ {j}\right) - \left[ \mu_ {j} \mathbb {Q} \right] \left(\boldsymbol {v} _ {j}\right)\right) ^ {2}, \tag {12}
$$
where  $[\mu_j\mathbb{P}](\pmb {v}_j) = E_{\pmb{x}\sim \mathbb{P}}[\kappa_j(\pmb {x},\pmb {v}_j)]$  and  $[\mu_j\mathbb{Q}](\pmb {v}_j) = E_{\pmb {y}\sim \mathbb{Q}}[\kappa_j(\pmb {y},\pmb {v}_j)]$
We show that the above distance metric for two probability measures is a random metric as follows:
Lemma 10. If  $\pmb{v}_1, \pmb{v}_2, \dots, \pmb{v}_\ell$  are drawn i.i.d. from a absolutely continuous distribution  $\mathcal{G}$ , then  $\rho_{\mu_1, \mu_2, \dots, \mu_\ell}(\cdot, \cdot)$  is a random metric for bounded kernels  $\{\kappa_1, \kappa_2, \dots, \kappa_\ell\}$ .
Proof. For  $j \in [\ell]$ , we first introduce a function
$$
\rho_ {\mu_ {j}} (\mathbb {P}, \mathbb {Q}) = \left(\left[ \mu_ {j} \mathbb {P} \right] \left(\boldsymbol {v} _ {j}\right) - \left[ \mu_ {j} \mathbb {Q} \right] \left(\boldsymbol {v} _ {j}\right)\right) ^ {2},
$$
and it is sufficient to prove that  $\rho_{\mu_j}(\mathbb{P},\mathbb{Q})$  is a random metric for each  $j\in [\ell ]$  from Eqn. (12).
It is well-known that Mahalanobis kernels in Eqn. (1) are characteristic and analytic from [81], and the corresponding mapping
$$
\mu_ {j}: \mathbb {P} \to \mu_ {j} \mathbb {P} \quad \text {a n d} \quad \mu_ {j}: \mathbb {Q} \to \mu_ {j} \mathbb {Q}
$$
are injective for  $\kappa_{j}$ , where  $\mu_j\mathbb{P}$  and  $\mu_j\mathbb{Q}$  denote the images of measures  $\mathbb{P}$  and  $\mathbb{Q}$ , respectively. Hence, the image of  $\mu_{j}$  is a subset of analytic functions for analytic and bounded  $\kappa_{j}$ , according to Lemma 8.
If  $\mathbb{P} = \mathbb{Q}$ , then we have
$$
f _ {j} = \mu_ {j} \mathbb {P} - \mu_ {j} \mathbb {Q} = 0 \quad \text {a n d} \quad \rho_ {\mu_ {j}} (\mathbb {P}, \mathbb {Q}) = \left(\left[ \mu_ {j} \mathbb {P} \right] (\boldsymbol {v} _ {j}) - \left[ \mu_ {j} \mathbb {Q} \right] (\boldsymbol {v} _ {j})\right) ^ {2} = 0.
$$
We now prove that if  $\mathbb{P} \neq \mathbb{Q}$  then  $f_{j} \neq 0$  almost surely, by applying Lemma 7 to analytic function  $f_{j} = \mu_{j}\mathbb{P} - \mu_{j}\mathbb{Q}$  with distribution  $\mathcal{G}$ . For injective map  $\mu_{j}$ , there exists at least one point  $a$  such that  $f_{j}(a) \neq 0$ , and there exists a ball around  $a$  with non-zero  $f_{j}$  from the continuity of  $f_{j}$ . Hence,  $f$  is almost everywhere nonzero based on Lemma 7, and this follows that
$$
\rho_ {\mu_ {j}} (\mathbb {P}, \mathbb {Q}) = \left(\left[ \mu_ {j} \mathbb {P} \right] \left(\boldsymbol {v} _ {j}\right) - \left[ \mu_ {j} \mathbb {Q} \right] \left(\boldsymbol {v} _ {j}\right)\right) ^ {2} > 0 \quad \text {a . s .} \quad \text {f o r} \quad \mathbb {P} \neq \mathbb {Q}.
$$
Hence,  $\rho_{\mu_j}$  is random metric from Definition 6 from the symmetry and triangle inequality of  $\rho_{\mu_j}$ .
Proof of Lemma 1. We first have
$$
\boldsymbol {\mu} _ {\hat {\mathbb {P}}} = E _ {\hat {\boldsymbol {x}} \sim \mathbb {P}} [ \hat {\boldsymbol {x}} ] = ([ \mu_ {1} \mathbb {P} ] (\boldsymbol {v} _ {1}), [ \mu_ {2} \mathbb {P} ] (\boldsymbol {v} _ {2}), \dots , [ \mu_ {\ell} \mathbb {P} ] (\boldsymbol {v} _ {\ell}))
$$
$$
\boldsymbol {\mu} _ {\hat {\mathbb {Q}}} = E _ {\hat {\boldsymbol {y}} \sim \mathbb {Q}} [ \hat {\boldsymbol {y}} ] = ([ \mu_ {1} \mathbb {Q} ] (\boldsymbol {v} _ {1}), [ \mu_ {2} \mathbb {Q} ] (\boldsymbol {v} _ {2}), \dots , [ \mu_ {\ell} \mathbb {Q} ] (\boldsymbol {v} _ {\ell}))
$$
and further rewrite Eqn. (12) as
$$
\rho_ {\mu_ {1}, \mu_ {2}, \dots , \mu_ {\ell}} (\mathbb {P}, \mathbb {Q}) = \sum_ {j = 1} ^ {\ell} \left(\left[ \mu_ {j} \mathbb {P} \right] (\boldsymbol {v} _ {j}) - \left[ \mu_ {j} \mathbb {Q} \right] (\boldsymbol {v} _ {j})\right) ^ {2} = \sum_ {j = 1} ^ {\ell} \left(\boldsymbol {\mu} _ {\hat {\mathbb {P}}, j} - \boldsymbol {\mu} _ {\hat {\mathbb {Q}}, j}\right) ^ {2} = \left\| \boldsymbol {\mu} _ {\hat {\mathbb {P}}} - \boldsymbol {\mu} _ {\hat {\mathbb {Q}}} \right\| _ {2} ^ {2}.
$$
From Lemma 10, we can see that  $\rho_{\mu_1,\mu_2,\dots ,\mu_\ell}(\mathbb{P},\mathbb{Q}) = 0$  if and only if  $\mathbb{P} = \mathbb{Q}$ , and this implies that  $\pmb{\mu}_{\hat{\mathbb{P}}} = \pmb{\mu}_{\hat{\mathbb{Q}}}$  if and only if  $\mathbb{P} = \mathbb{Q}$ . This completes the proof of Lemma 1.
We begin with a useful lemma from [11] as follows.
Lemma 11. For symmetric and positive definite matrix  $\Sigma$ , function  $h(\Sigma) = \Sigma^{-1/2}$  is continuous and well-defined on the positive definite space.
Based on this lemma, we present the detailed proof of Theorem 2 as follows:
Proof of Theorem 2. For embedding testing sample  $\hat{X}^{\prime} = \{\hat{x}_{1}^{\prime},\dots ,\hat{x}_{m^{\prime}}^{\prime}\}$  and  $\hat{Y}^{\prime} = \{\hat{y}_{1}^{\prime},\dots ,\hat{y}_{n^{\prime}}^{\prime}\}$ , recall the pooled covariance matrix in Eqn. (3) as
$$
\Sigma_ {\hat {X} ^ {\prime}, \hat {Y} ^ {\prime}} = \frac {(m ^ {\prime} - 1) \Sigma_ {\hat {X}} + (n ^ {\prime} - 1) \Sigma_ {\hat {Y}}}{m ^ {\prime} + n ^ {\prime} - 2} + \epsilon \boldsymbol {I} _ {d},
$$
and we also define
$$
\Sigma_ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}} = E _ {\hat {X} ^ {\prime} \sim \hat {\mathbb {P}} ^ {m ^ {\prime}}, \hat {Y} ^ {\prime} \sim \hat {\mathbb {Q}} ^ {n ^ {\prime}}} \left[ \Sigma_ {\hat {X} ^ {\prime}, \hat {Y} ^ {\prime}} \right].
$$
We first prove that testing statistic  $\mathcal{T}(\hat{X}',\hat{Y}')$  is almost surely asymptotically distributed as  $\chi_{\ell}^{2}$  with  $\ell$  degrees of freedom under the condition  $\pmb{\mu}_{\hat{\mathbb{P}}} = \pmb{\mu}_{\hat{\mathbb{Q}}}$  for small  $\epsilon$ .
For i.i.d. samples  $\hat{X}^{\prime}$  and  $\hat{Y}^{\prime}$ ,  $c_{\hat{x}}$  is independent to  $c_{\hat{y}}$ . If  $\mu_{\hat{\mathbb{P}}} = \mu_{\hat{\mathbb{Q}}}$ , then  $\bar{z} = c_{\hat{X}} - c_{\hat{Y}}$  follows a multivariate normal distribution with mean  $\pmb {\mu} = \pmb{\mu}_{\hat{\mathbb{P}}} - \pmb{\mu}_{\hat{\mathbb{Q}}} = \mathbf{0}$  and covariance matrix  $(m^{\prime} + n^{\prime})\Sigma_{\hat{\mathbb{P}},\hat{\mathbb{Q}}} / m'n'$ , from [41, 82] and the Slutsky's theorem [83], that is
$$
\bar {z} \stackrel {d} {\rightarrow} \mathcal {N} (\mathbf {0}, (m ^ {\prime} + n ^ {\prime}) \Sigma_ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}} / m ^ {\prime} n ^ {\prime}),
$$
where  $\stackrel{d}{\to}$  denotes convergence in distribution.
The matrix  $L$  is symmetric and invertible in the Schur decomposition  $\Sigma_{\hat{X}^{\prime},\hat{Y}^{\prime}} = LL$ . Our statistic can be formalized as
$$
\begin{array}{l} \mathcal {T} (\hat {X} ^ {\prime}, \hat {Y} ^ {\prime}) = m ^ {\prime} n ^ {\prime} \bar {z} ^ {\top} \Sigma_ {\hat {X}, \hat {Y}} ^ {- 1} \bar {z} / (m ^ {\prime} + n ^ {\prime}) \\ = m ^ {\prime} n ^ {\prime} \bar {z} ^ {\top} \mathbf {L} ^ {\prime - 1} \mathbf {L} ^ {\prime - 1} \bar {z} / (m ^ {\prime} + n ^ {\prime}) \\ = \left(\sqrt {m ^ {\prime} n ^ {\prime}} \left\| L ^ {\prime - 1} \bar {z} \right\| _ {2} / \sqrt {m ^ {\prime} + n ^ {\prime}}\right) ^ {2}, \\ \end{array}
$$
and by applying the Slutsky's theorem, we have
$$
\sqrt {m ^ {\prime} n ^ {\prime}} \boldsymbol {L} ^ {\prime - 1} \bar {z} / \sqrt {m ^ {\prime} + n ^ {\prime}} \xrightarrow {d} \mathcal {N} (\boldsymbol {0}, \boldsymbol {I} _ {\ell}).
$$
This follows that
$$
\mathcal {T} (\hat {X} ^ {\prime}, \hat {Y} ^ {\prime}) = \frac {m ^ {\prime} n ^ {\prime}}{m ^ {\prime} + n ^ {\prime}} \sum_ {j = 1} ^ {\ell} (\boldsymbol {L} ^ {\prime - 1} \bar {\boldsymbol {z}}) _ {j} ^ {2},
$$
where  $(\pmb{L}^{\prime -1}\bar{\pmb{z}})_j$  is the  $j$ -th dimension value of  $\pmb{L}^{\prime -1}\bar{\pmb{z}}$ , and  $\sqrt{m'n' / (m' + n')} (\pmb{L}^{\prime -1}\bar{\pmb{z}})_j$  follows the standard normal distribution. This proves the  $\chi_{\ell}^{2}$  distribution for  $\mathcal{T}(\hat{X}',\hat{Y}')$  from the sum of  $\ell$  squares of standard (i.i.d.) normal random variables.
We then prove that  $\chi_{\ell}^{2}(\mathcal{T}(\hat{X}^{\prime},\hat{Y}^{\prime}))\to 0$  as  $m'n' / (m' + n')\to \infty$  if  $\pmb{\mu}_{\hat{\mathbb{P}}}\neq \pmb{\mu}_{\hat{\mathbb{Q}}}$ , and hence our test rejects  $H_0$  almost surely.
From Lemma 11, we have
$$
\lim _ {m ^ {\prime} n ^ {\prime} / (m ^ {\prime} + n ^ {\prime}) \to \infty} L ^ {\prime - 1} = \lim _ {m ^ {\prime} n ^ {\prime} / (m ^ {\prime} + n ^ {\prime}) \to \infty} \Sigma_ {\hat {X} ^ {\prime}, \hat {Y} ^ {\prime}} ^ {- 1 / 2} = \Sigma_ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}} ^ {- 1 / 2} = L _ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}} ^ {- 1}.
$$
In a similar manner,  $\bar{z} = c_{\hat{X}} - c_{\hat{Y}}$  converges to  $\pmb{\mu}_{\mathbb{P}} - \pmb{\mu}_{\hat{\mathbb{Q}}}$  in probability. For  $\pmb{\mu}_{\mathbb{P}} \neq \pmb{\mu}_{\mathbb{Q}}$ , we have
$$
\left\| \Sigma_ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}} ^ {- \frac {1}{2}} \left(\boldsymbol {\mu} _ {\hat {\mathbb {P}}} - \boldsymbol {\mu} _ {\hat {\mathbb {Q}}}\right) \right\| _ {2} ^ {2} > 0.
$$
Then,  $\| L'^{-1}\bar{z} \|_2^2$  is a continuous function with entries  $\bar{z}$  and  $L'^{-1}$ , and it is convergent to some positive constant. We have  $(m' + n') / (m'n')\chi_{\ell,\alpha}^2 \to 0$ , and
$$
\Pr \left[ \frac {m ^ {\prime} n ^ {\prime}}{m ^ {\prime} + n ^ {\prime}} \left\| \boldsymbol {L} ^ {\prime - 1} \bar {\boldsymbol {z}} \right\| _ {2} ^ {2} > \chi_ {\ell , \alpha} ^ {2} \right] = P \left(\left\| \boldsymbol {L} ^ {\prime - 1} \bar {\boldsymbol {z}} \right\| _ {2} ^ {2} > \frac {m ^ {\prime} + n ^ {\prime}}{m ^ {\prime} n ^ {\prime}} \chi_ {\ell , \alpha} ^ {2}\right)\rightarrow 1.
$$
$$
\chi_ {\ell} ^ {2} \left(\mathcal {T} \left(\hat {X} ^ {\prime}, \hat {Y} ^ {\prime}\right)\right) = \chi_ {\ell} ^ {2} \left(\frac {m ^ {\prime} n ^ {\prime}}{m ^ {\prime} + n ^ {\prime}} \| L ^ {\prime - 1} \bar {z} \| _ {2} ^ {2}\right)\rightarrow 0,
$$
and hence  $\chi_{\ell}^{2}(\mathcal{T}(\hat{X}^{\prime},\hat{Y}^{\prime}))\to 0$  as  $m'n' / (m' + n')\to \infty$  if  $\pmb {\mu}_{\hat{\mathbb{P}}}\neq \pmb{\mu}_{\hat{\mathbb{Q}}}$
Recall the pooled covariance matrix  $\Sigma_{\hat{X}^{\prime},\hat{Y}^{\prime}}$  and  $\Sigma_{\hat{\mathbb{P}},\hat{\mathbb{Q}}} = E_{\hat{X}^{\prime}\sim \hat{\mathbb{P}}^{m^{\prime}},\hat{Y}^{\prime}\sim \hat{\mathbb{Q}}^{n^{\prime}}}[\Sigma_{\hat{X}^{\prime},\hat{Y}^{\prime}}]$  in the proof of Theorem 2, and  $\bar{z} = (c_{\hat{X}^{\prime}} - c_{\hat{Y}^{\prime}})$  follows a multivariate normal distribution with mean  $\pmb {\mu}_{\hat{\mathbb{P}}} - \pmb{\mu}_{\hat{\mathbb{Q}}}$  and covariance matrix  $(m^{\prime} + n^{\prime})\Sigma_{\hat{\mathbb{P}},\hat{\mathbb{Q}}} / m'n'$ . We first observe
$$
\boldsymbol {L} _ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}} = E _ {\hat {X} ^ {\prime} \sim \hat {\mathbb {P}} ^ {m ^ {\prime}}, \hat {Y} ^ {\prime} \sim \hat {\mathbb {Q}} ^ {n ^ {\prime}}} [ \boldsymbol {L} ^ {\prime} ] = \Sigma_ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}} ^ {1 / 2},
$$
and this follows that
$$
\boldsymbol {L} ^ {\prime - 1} \bar {\boldsymbol {z}} \sim \mathcal {N} \left(\boldsymbol {L} _ {\hat {\mathrm {P}}, \hat {\mathrm {Q}}} ^ {- 1} \left(\boldsymbol {\mu} _ {\hat {\mathrm {P}}} - \boldsymbol {\mu} _ {\hat {\mathrm {Q}}}\right), \frac {m ^ {\prime} + n ^ {\prime}}{m ^ {\prime} n ^ {\prime}} \boldsymbol {I} _ {\ell}\right).
$$
This is because  $E[L'^{-1}\bar{z}] = L_{\hat{\mathbb{P}},\hat{\mathbb{Q}}}^{-1}(\pmb{\mu}_{\hat{\mathbb{P}}} - \pmb{\mu}_{\hat{\mathbb{Q}}})$  and the covariance matrix is given by
$$
\begin{array}{l} \operatorname {C o v} \left(L ^ {\prime - 1} \bar {z}, L ^ {\prime - 1} \bar {z}\right) = L _ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}} ^ {- 1} \left(\frac {m ^ {\prime} + n ^ {\prime}}{m ^ {\prime} n ^ {\prime}} \Sigma_ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}}\right) L _ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}} ^ {- 1} \\ = \boldsymbol {L} _ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}} ^ {- 1} \left(\frac {m ^ {\prime} + n ^ {\prime}}{m ^ {\prime} n ^ {\prime}} \boldsymbol {L} _ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}} \boldsymbol {L} _ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}}\right) \boldsymbol {L} _ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}} ^ {- 1} = \frac {m ^ {\prime} + n ^ {\prime}}{m ^ {\prime} n ^ {\prime}} \boldsymbol {I} _ {\ell}. \\ \end{array}
$$
Write  $B = \sqrt{m'n' / (m' + n')L'^{-1}}\mathrm{diag}(F)\bar{z}$ , and we have
$$
\begin{array}{l} \boldsymbol {B} ^ {\top} \boldsymbol {B} = \left(\sqrt {\frac {m ^ {\prime} n ^ {\prime}}{m ^ {\prime} + n ^ {\prime}}} \boldsymbol {L} ^ {\prime - 1} \operatorname {d i a g} (\boldsymbol {F}) \bar {z}\right) ^ {\top} \left(\sqrt {\frac {m ^ {\prime} n ^ {\prime}}{m ^ {\prime} + n ^ {\prime}}} \boldsymbol {L} ^ {\prime - 1} \operatorname {d i a g} (\boldsymbol {F}) \bar {z}\right) \\ = \frac {m ^ {\prime} n ^ {\prime}}{m ^ {\prime} + n ^ {\prime}} \bar {z} ^ {\top} \operatorname {d i a g} (\boldsymbol {F}) \left(\boldsymbol {L} ^ {\prime - 1}\right) ^ {\top} \boldsymbol {L} ^ {\prime - 1} \operatorname {d i a g} (\boldsymbol {F}) \bar {z} = \frac {m ^ {\prime} n ^ {\prime}}{m ^ {\prime} + n ^ {\prime}} \bar {z} ^ {\top} \operatorname {d i a g} (\boldsymbol {F}) \Sigma_ {\hat {X} ^ {\prime}, \hat {Y} ^ {\prime}} ^ {- 1} \operatorname {d i a g} (\boldsymbol {F}) \bar {z} \\ = \frac {m ^ {\prime} n ^ {\prime}}{m ^ {\prime} + n ^ {\prime}} \bar {\boldsymbol {z}} ^ {\top} \operatorname {d i a g} (\boldsymbol {F}) \operatorname {d i a g} (\boldsymbol {F}) \Sigma_ {\hat {X} ^ {\prime}, \hat {Y} ^ {\prime}} ^ {- 1} \bar {\boldsymbol {z}} = \frac {m ^ {\prime} n ^ {\prime}}{m ^ {\prime} + n ^ {\prime}} \bar {\boldsymbol {z}} ^ {\top} \operatorname {d i a g} (\mathbf {1}) \Sigma_ {\hat {X} ^ {\prime}, \hat {Y} ^ {\prime}} ^ {- 1} \bar {\boldsymbol {z}} \\ = \frac {m ^ {\prime} n ^ {\prime}}{m ^ {\prime} + n ^ {\prime}} \bar {z} ^ {\top} \Sigma_ {\hat {X} ^ {\prime}, \hat {Y} ^ {\prime}} ^ {- 1} \bar {z} \\ \end{array}
$$
by using the symmetry of  $\Sigma_{\hat{X}^{\prime},\hat{Y}^{\prime}}^{-1}$ . It is easy to get
$$
E [ \boldsymbol {B} ] = \sqrt {m ^ {\prime} n ^ {\prime} / (m ^ {\prime} + n ^ {\prime})} \boldsymbol {L} _ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}} ^ {- 1} \operatorname {d i a g} (\boldsymbol {F}) (\boldsymbol {\mu} _ {\hat {\mathbb {P}}} - \boldsymbol {\mu} _ {\hat {\mathbb {Q}}})
$$
and covariance matrix is given by
$$
\begin{array}{l} \operatorname {C o v} (\boldsymbol {B}, \boldsymbol {B}) = \sqrt {\frac {m ^ {\prime} n ^ {\prime}}{m ^ {\prime} + n ^ {\prime}}} \boldsymbol {L} _ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}} ^ {- 1} \operatorname {d i a g} (\boldsymbol {F}) \left(\frac {m ^ {\prime} + n ^ {\prime}}{m ^ {\prime} n ^ {\prime}} \Sigma_ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}}\right) \sqrt {\frac {m ^ {\prime} n ^ {\prime}}{m ^ {\prime} + n ^ {\prime}}} \operatorname {d i a g} (\boldsymbol {F}) \boldsymbol {L} _ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}} ^ {- 1} \\ = \sqrt {\frac {m ^ {\prime} n ^ {\prime}}{m ^ {\prime} + n ^ {\prime}}} \operatorname {d i a g} (F) L _ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}} ^ {- 1} \left(\frac {m ^ {\prime} + n ^ {\prime}}{m ^ {\prime} n ^ {\prime}} L _ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}} L _ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}}\right) \sqrt {\frac {m ^ {\prime} n ^ {\prime}}{m ^ {\prime} + n ^ {\prime}}} \operatorname {d i a g} (F) L _ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}} ^ {- 1} \\ = \boldsymbol {I} _ {\ell}. \\ \end{array}
$$
$$
\boldsymbol {B} \sim \mathcal {N} \left(\sqrt {m ^ {\prime} n ^ {\prime} / (m ^ {\prime} + n ^ {\prime})} \boldsymbol {L} _ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}} ^ {- 1} \mathrm {d i a g} (\boldsymbol {F}) \left(\boldsymbol {\mu} _ {\hat {\mathbb {P}}} - \boldsymbol {\mu} _ {\hat {\mathbb {Q}}}\right), \boldsymbol {I} _ {\ell}\right),
$$
and all random variables in  $B$  are mutually independent. Define
$$
\bar {B} = \mathbf {1} ^ {\top} \boldsymbol {B} / \ell = \sum_ {i = 1} ^ {\ell} \boldsymbol {B} _ {i} / \ell \quad \text {a n d} \quad S ^ {2} = \sum_ {i = 1} ^ {\ell} (\boldsymbol {B} _ {i} - \bar {B}) ^ {2},
$$
and  $\bar{B}$  is normally distributed with mean  $\sqrt{m'n' / (m' + n')} F^{\top}L_{\hat{\mathbb{P}},\hat{\mathbb{Q}}}^{-1}(\pmb{\mu}_{\hat{\mathbb{P}}} - \pmb{\mu}_{\hat{\mathbb{Q}}}) / \ell$  and variance  $1 / \ell$ . It is easy to see that
$$
\boldsymbol {F} ^ {\top} \boldsymbol {L} ^ {\prime - 1} \left(\boldsymbol {c} _ {\hat {X} ^ {\prime}} - \boldsymbol {c} _ {\hat {Y} ^ {\prime}}\right) = \sqrt {(m ^ {\prime} + n ^ {\prime}) / m ^ {\prime} n ^ {\prime}} \boldsymbol {1} ^ {\top} \boldsymbol {B},
$$
$$
\Pr \left[ \operatorname {s g n} \left(\boldsymbol {F} ^ {\top} \boldsymbol {L} ^ {\prime - 1} \left(\boldsymbol {c} _ {\hat {X} ^ {\prime}} - \boldsymbol {c} _ {\hat {Y} ^ {\prime}}\right)\right) = 1 \right] = \Pr \left[ \sqrt {\frac {m ^ {\prime} + n ^ {\prime}}{m ^ {\prime} n ^ {\prime}}} \mathbf {1} ^ {\top} \boldsymbol {B} > 0 \right] = \Pr [ \bar {B} > 0 ]. \tag {13}
$$
$$
\Pr \left[ \operatorname {s g n} \left(\boldsymbol {F} ^ {\top} \boldsymbol {L} ^ {\prime - 1} \left(\boldsymbol {c} _ {\hat {X} ^ {\prime}} - \boldsymbol {c} _ {\hat {Y} ^ {\prime}}\right)\right) = 1 \right] = 1 - \Phi \left(- \sqrt {\frac {m ^ {\prime} n ^ {\prime}}{\left(m ^ {\prime} + n ^ {\prime}\right) \ell}} \boldsymbol {F} ^ {\top} \boldsymbol {L} _ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}} ^ {- 1} \left(\boldsymbol {\mu} _ {\hat {\mathbb {P}}} - \boldsymbol {\mu} _ {\hat {\mathbb {Q}}}\right)\right),
$$
where  $\Phi (\cdot)$  is the cumulative distribution function of standard Gaussian distribution. For continuous normal distribution, we have
$$
\Pr \left[ \operatorname {s g n} \left(\boldsymbol {F} ^ {\top} \boldsymbol {L} ^ {\prime - 1} \left(\boldsymbol {c} _ {\hat {X} ^ {\prime}} - \boldsymbol {c} _ {\hat {Y} ^ {\prime}}\right)\right) = 0 \right] = \Pr \left[ \bar {B} = 0 \right] = 0,
$$
$$
\Pr \left[ \operatorname {s g n} \left(\boldsymbol {F} ^ {\top} \boldsymbol {L} ^ {\prime - 1} \left(\boldsymbol {c} _ {\hat {X} ^ {\prime}} - \boldsymbol {c} _ {\hat {Y} ^ {\prime}}\right)\right) = - 1 \right] = \Pr \left[ \bar {B} <   0 \right] = \Phi \left(- \sqrt {\frac {m ^ {\prime} n ^ {\prime}}{\left(m ^ {\prime} + n ^ {\prime}\right) \ell}} \boldsymbol {F} ^ {\top} \boldsymbol {L} _ {\mathbb {P}, \hat {\mathbb {Q}}} ^ {- 1} \left(\boldsymbol {\mu} _ {\hat {\mathbb {P}}} - \boldsymbol {\mu} _ {\hat {\mathbb {Q}}}\right)\right).
$$
Hence, the  $\mathrm{sgn}(\pmb{F}^\top \pmb{L}^{\prime -1}(c_{\hat{X}^{\prime}} - c_{\hat{Y}^{\prime}}))$  follows a two-point distribution  $\mathcal{TP}(\xi)$  with parameter
$$
\xi = 1 - \Phi \left(- \sqrt {\frac {m ^ {\prime} n ^ {\prime}}{(m ^ {\prime} + n ^ {\prime}) \ell}} \boldsymbol {F} ^ {\top} \boldsymbol {L} _ {\mathbb {P}, \hat {\mathbb {Q}}} ^ {- 1} (\boldsymbol {\mu} _ {\mathbb {P}} - \boldsymbol {\mu} _ {\hat {\mathbb {Q}}})\right).
$$
Recall the pooled covariance matrix  $\Sigma_{\hat{X}^{\prime},\hat{Y}^{\prime}}$  and  $\Sigma_{\hat{\mathbb{P}},\hat{\mathbb{Q}}} = E_{\hat{X}^{\prime}\sim \hat{\mathbb{P}}^{m^{\prime}},\hat{Y}^{\prime}\sim \hat{\mathbb{Q}}^{n^{\prime}}}\left[\Sigma_{\hat{X}^{\prime},\hat{Y}^{\prime}}\right]$  in the proof of Theorem 2, and  $\bar{z} = (c_{\hat{X}^{\prime}} - c_{\hat{Y}^{\prime}})$  follows a multivariate normal distribution with mean  $\pmb {\mu}_{\hat{\mathbb{P}}} - \pmb {\mu}_{\hat{\mathbb{Q}}}$  and covariance matrix  $(m^{\prime} + n^{\prime})\Sigma_{\hat{\mathbb{P}},\hat{\mathbb{Q}}} / m'n'$ . We have  $L_{\hat{\mathbb{P}},\hat{\mathbb{Q}}}L_{\hat{\mathbb{P}},\hat{\mathbb{Q}}} = \Sigma_{\hat{\mathbb{P}},\hat{\mathbb{Q}}}$ .
We begin with a useful lemma and corollary as follows.
Lemma 12. If  $F^{\top}L_{\hat{\mathbb{P}},\hat{\mathbb{Q}}}^{-1}\pmb{\mu}_{\hat{\mathbb{P}}} > F^{\top}L_{\hat{\mathbb{P}},\hat{\mathbb{Q}}}^{-1}\pmb{\mu}_{\hat{\mathbb{Q}}}$  or null hypothesis  $H_0\colon \pmb{\mu}_{\hat{\mathbb{P}}} = \pmb{\mu}_{\hat{\mathbb{Q}}}$ , we have
$$
\Pr \left[ \frac {m ^ {\prime} n ^ {\prime}}{m ^ {\prime} + n ^ {\prime}} \bar {z} ^ {\top} \Sigma_ {\hat {X} ^ {\prime}, \hat {Y} ^ {\prime}} ^ {- 1} \bar {z} \geq c \mid \boldsymbol {F} ^ {\top} \boldsymbol {L} ^ {\prime - 1} \bar {z} \geq 0 \right] \geq \Pr \left[ \frac {m ^ {\prime} n ^ {\prime}}{m ^ {\prime} + n ^ {\prime}} \bar {z} ^ {\top} \Sigma_ {\hat {X} ^ {\prime}, \hat {Y} ^ {\prime}} ^ {- 1} \bar {z} \geq c \right]. \tag {14}
$$
Proof. Recall  $\pmb{B} = \sqrt{m'n' / (m' + n')} \pmb{L}'^{-1} \mathrm{diag}(\pmb{F}) \bar{z}$  and  $\pmb{F}^{\top} \pmb{L}'^{-1} \bar{z} = \sqrt{(m' + n') / m'n'} \pmb{1}^{\top} \pmb{B}$  in the proof of Lemma 3, and Eqn. (14) is equivalent to
$$
\Pr \left[ \boldsymbol {B} ^ {\top} \boldsymbol {B} \geq c \mid \mathbf {1} ^ {\top} \boldsymbol {B} \geq 0 \right] \geq \Pr \left[ \boldsymbol {B} ^ {\top} \boldsymbol {B} \geq c \right]. \tag {15}
$$
and from Eqn. (15), we have
$$
\Pr \left[ \bar {B} ^ {2} \geq (c - S ^ {2}) / \ell | \bar {B} \geq 0 \right] \geq \Pr \left[ \bar {B} ^ {2} \geq (c - S ^ {2}) / \ell \right].
$$
From the independence of  $S^2$  and  $\bar{B}$ , it is sufficient to prove that, for every  $\delta \geq 0$
$$
\Pr \left[ \bar {B} ^ {2} \geq \delta | \bar {B} \geq 0 \right] \geq \Pr \left[ \bar {B} ^ {2} \geq \delta \right]. \tag {16}
$$
It's easy to see that  $\bar{B}$  is normally distributed with mean  $\sqrt{m'n' / (m' + n')} \pmb{F}^{\top} \pmb{L}_{\hat{\mathbb{P}},\hat{\mathbb{Q}}}^{-1} (\pmb{\mu}_{\hat{\mathbb{P}}} - \pmb{\mu}_{\hat{\mathbb{Q}}}) / \ell$  and variance  $1 / \ell$ . We define
$$
a = \sqrt {\ell \delta} \quad \text {a n d} \quad b = \sqrt {\frac {m ^ {\prime} n ^ {\prime}}{(m ^ {\prime} + n ^ {\prime}) \ell}} \boldsymbol {F} ^ {\top} \boldsymbol {L} _ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}} ^ {- 1} (\boldsymbol {\mu} _ {\hat {\mathbb {P}}} - \boldsymbol {\mu} _ {\hat {\mathbb {Q}}}),
$$
and from Eqn. (16), we have
$$
\frac {\Phi (- b)}{\Phi (- b - a)} \geq \frac {\Phi (b)}{\Phi (b - a)},
$$
where the equality holds from  $b = 0$ , i.e.,  $\pmb{\mu}_{\hat{\mathbb{P}}} = \pmb{\mu}_{\hat{\mathbb{Q}}}$ . This completes the proof.
Corollary 13. If  $F^{\top}L_{\hat{\mathbb{P}},\hat{\mathbb{Q}}}^{-1}\pmb{\mu}_{\hat{\mathbb{P}}} < F^{\top}L_{\hat{\mathbb{P}},\hat{\mathbb{Q}}}^{-1}\pmb{\mu}_{\hat{\mathbb{Q}}}$ , we have
$$
\operatorname * {P r} \left[ \frac {m ^ {\prime} n ^ {\prime}}{m ^ {\prime} + n ^ {\prime}} \bar {\pmb {z}} ^ {\top} \Sigma_ {\hat {X} ^ {\prime}, \hat {Y} ^ {\prime}} ^ {- 1} \bar {\pmb {z}} \geq c \mid \pmb {F} ^ {\top} \pmb {L} ^ {\prime - 1} \bar {\pmb {z}} <   0 \right] \geq \operatorname * {P r} \left[ \frac {m ^ {\prime} n ^ {\prime}}{m ^ {\prime} + n ^ {\prime}} \bar {\pmb {z}} ^ {\top} \Sigma_ {\hat {X} ^ {\prime}, \hat {Y} ^ {\prime}} ^ {- 1} \bar {\pmb {z}} \geq c \right],
$$
Lemma 14. If  $F^{\top}L_{\mathbb{P},\hat{\mathbb{Q}}}^{-1}\pmb{\mu}_{\hat{\mathbb{P}}} > F^{\top}L_{\mathbb{P},\hat{\mathbb{Q}}}^{-1}\pmb{\mu}_{\hat{\mathbb{Q}}}$ , then the test power of our bi-directional hypothesis can be lower bounded by
$$
q (\ell , \nu - \ell , \lambda , \beta \alpha) \cdot \xi + \Phi \left(- \left(\chi_ {\ell , (2 - \beta) \alpha} ^ {2}\right) ^ {1 / 2} - (\omega / \ell) ^ {1 / 2} \boldsymbol {F} ^ {\top} \boldsymbol {L} _ {\hat {\mathbb {P}}, \hat {\mathbb {Q}}} ^ {- 1} \left(\boldsymbol {\mu} _ {\hat {\mathbb {P}}} - \boldsymbol {\mu} _ {\hat {\mathbb {Q}}}\right)\right),
$$
where  $\omega = m^{\prime}n^{\prime} / (m^{\prime} + n^{\prime})$ ,  $\nu = m^{\prime} + n^{\prime} - 1$  and  $\lambda = \omega \| L_{\hat{\mathbb{P}},\hat{\mathbb{Q}}}^{-1}(\pmb {\mu}_{\hat{\mathbb{P}}} - \pmb {\mu}_{\hat{\mathbb{Q}}})\| _2^2$
Proof. From Lemma 12, we first have, for  $F^{\top}L_{\hat{\mathbb{P}},\hat{\mathbb{Q}}}^{-1}\pmb{\mu}_{\hat{\mathbb{P}}} > F^{\top}L_{\hat{\mathbb{P}},\hat{\mathbb{Q}}}^{-1}\pmb{\mu}_{\hat{\mathbb{Q}}}$ ,