SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\mathbb {E} h (p _ {\hat {G} _ {n}}, p _ {G _ {*}}) = \int_ {0} ^ {\infty} \mathbb {P} (h (p _ {\hat {G} _ {n}}, p _ {G _ {*}}) > \delta) d \delta \leq \delta_ {n} + c \int_ {\delta_ {n}} ^ {\infty} \exp \left(- \frac {n \delta^ {2}}{c ^ {2}}\right) \leq \tilde {c} \delta_ {n},
$$
for some  $\tilde{c}$  does not depend on  $\lambda^{*}, G_{*}$ . Hence, we finally proved that
$$
\sup _ {G _ {*} \in \Xi} \mathbb {E} _ {G _ {*}} h (p _ {\widehat {G} _ {n}}, p _ {G _ {*}}) \leq C \sqrt {\log n / n}.
$$
As a consequence, we obtain the conclusion of the theorem.
B.2 Proof for Proposition 2.4
We further introduce some more notations that are required for the proof. Let  $N(\epsilon, \mathcal{P}(\Xi), \| \cdot \|_{\infty})$  be the  $\epsilon$ -covering number of  $(\mathcal{P}(\Xi), \| \cdot \|_{\infty})$  and  $N_B(\epsilon, \mathcal{P}(\Xi), h)$  be the bracketing number of  $\mathcal{P}(\Xi)$  measured by Hellinger metric  $h$ .  $H_B(\epsilon, \mathcal{P}(\Xi), h) = \log N_B(\epsilon, \mathcal{P}(\Xi), h)$  is called the bracketing entropy of  $\mathcal{P}(\Xi)$  under metric  $h$ . We want to show that
$$
\mathcal {J} _ {B} (\epsilon , \bar {\mathcal {P}} ^ {1 / 2} (\Xi , \epsilon), L ^ {2} (m)) = \left(\int_ {\epsilon^ {2} / 2 ^ {2 ^ {1 3}}} ^ {\epsilon} H _ {B} ^ {1 / 2} (\delta , \bar {\mathcal {P}} ^ {1 / 2} (\Xi , \delta), L ^ {2} (m)) d \delta \vee \delta\right) \lesssim \sqrt {n} \epsilon^ {2}, \tag {14}
$$
for all  $n > N$  large enough and  $\epsilon > (\log n / n)^{1/2}$ . We proceed to show that claim (14) will be proved if
$$
\log N (\epsilon , \mathcal {P} (\Xi), \| \cdot \| _ {\infty}) \lesssim \log (1 / \epsilon), \tag {15}
$$
$$
H _ {B} (\epsilon , \mathcal {P} (\Xi), h) \lesssim \log (1 / \epsilon), \tag {16}
$$
and then prove claim (15) and (16).
Proof of that claim (16) implies claim (14) Because  $\overline{\mathcal{P}}^{1 / 2}(\Xi ,\delta)\subset \overline{\mathcal{P}}^{1 / 2}(\Xi)$  and from the definition of Hellinger distance,
$$
H _ {B} (\delta , \overline {{\mathcal {P}}} ^ {1 / 2} (\Xi , \delta), \mu) \leq H _ {B} (\delta , \overline {{\mathcal {P}}} ^ {1 / 2} (\Xi), \mu) = H _ {B} (\frac {\delta}{\sqrt {2}}, \overline {{\mathcal {P}}} (\Xi), h).
$$
Now use the fact that for densities  $f_{*}, f_{1}, f_{2}$ , we have  $h^{2}((f_{1} + f_{*}) / 2, (f_{2} + f_{*}) / 2) \leq h^{2}(f_{1}, f_{2}) / 2$ , one can readily check that  $H_{B}(\frac{\delta}{\sqrt{2}}, \overline{\mathcal{P}}(\Xi), h) \leq H_{B}(\delta, \mathcal{P}(\Xi), h)$ . Hence, if claim (16) holds true, then
$$
H _ {B} (\delta , \overline {{\mathcal {P}}} ^ {1 / 2} (\Xi , \delta), \mu) \leq H _ {B} (\delta , \mathcal {P} (\Xi), h) \lesssim \log (1 / \delta),
$$
$$
\mathcal {J} _ {B} (\epsilon , \overline {{\mathcal {P}}} ^ {1 / 2} (\Xi , \delta), \mu) \lesssim \epsilon (\log (2 ^ {1 3} / \epsilon^ {2})) ^ {1 / 2} <   n \epsilon^ {2},
$$
for all  $\epsilon > (\log n / n)^{1/2}$ . Hence, claim (14) is proved.
Proof of claim (15) As  $\lambda \in [0,1]$ , we can choose an  $\epsilon$ -net for it with the cardinality no more than  $\frac{1}{\epsilon}$ . Similarly, because  $\Theta$  and  $\Omega$  are compact, we can cover them by hypercube  $[-a,a]^{d_1}$  and  $[-b,b]^{d_2\times d_2}$ . Hence, there exists  $\epsilon$ -nets for them with the cardinality no more than  $\left(\frac{2a}{\epsilon}\right)^{d_1}$  and  $\left(\frac{2b}{\epsilon}\right)^{d_2^2}$ . Let  $S$  be the Cartesian product of them. We have  $\log |\mathcal{S}|\lesssim \log (1 / \epsilon)$  and for every  $G = (\lambda ,\mu ,\Sigma)\in \Xi$ , there exists  $G^{\prime} = (\lambda^{\prime},\mu^{\prime},\Sigma^{\prime})\in S$  such that  $|\lambda -\lambda^{\prime}|,\| \mu -\mu^{\prime}\|$ $\| \Sigma -\Sigma^{\prime}\| \leq \epsilon$ . By triangle inequalities,
$$
\left\| p _ {G} - p _ {G ^ {\prime}} \right\| _ {\infty} \leq | \lambda - \lambda^ {\prime} | \left(\left\| h _ {0} \right\| _ {\infty} + \| f \| _ {\infty}\right) + \lambda | f (x | \mu , \Sigma) - f (x | \mu^ {\prime}, \Sigma^ {\prime}) | \lesssim \epsilon ,
$$
thanks to the uniform bounded and Lipschitz assumptions. Hence,
$$
\log N (\epsilon , \mathcal {P} (\Xi), \| \cdot \| _ {\infty}) \lesssim \log (1 / \epsilon).
$$
Proof of claim (16) Now, from the entropy number, we are going to bound the bracketing number, we let  $\eta \leq \epsilon$  which will be chosen later. Let  $f_{1},\ldots ,f_{N}$  be a  $\eta$ -net for  $\mathcal{P}(\Xi)$ , where  $f_{i}(x) = (1 - \lambda_{i})h_{0}(x) + \lambda_{i}f(x|\mu_{i},\Sigma_{i})$ . Let
$$
H (x) = \left\{ \begin{array}{l l} b _ {1} \exp \left(- b _ {2} \| x \| ^ {b _ {3}}\right), & \| x \| _ {2} \geq B _ {1}, \\ M, & \text {o t h e r w i s e} \end{array} \right. \tag {17}
$$
is an envelop for  $f(x|\mu, \Sigma)$ . We can construct brackets  $[p_i^L, p_i^U]$  as follows.
$$
p _ {i} ^ {L} (x) = \left(1 - \lambda_ {i}\right) h _ {0} (x) + \lambda_ {i} \max  \left\{f \left(x \mid \mu_ {i}, \Sigma_ {i}\right) - \eta , 0 \right\},
$$
$$
p _ {i} ^ {U} (x) = (1 - \lambda_ {i}) h _ {0} (x) + \lambda_ {i} \min \{f (x | \mu_ {i}, \Sigma_ {i}) + \eta , H (x) \}.
$$
Because for each  $f \in \mathcal{P}(\Xi)$ , there is  $f_{i}$  such that  $\| f - f_{i}\|_{\infty} < \eta$ , we have  $p_i^L \leq f \leq p_i^U$ . Moreover, for any  $\overline{B} \geq B$ ,
$$
\begin{array}{l} \int_ {\mathbb {R} ^ {d}} \left(p _ {i} ^ {U} - p _ {i} ^ {L}\right) d \mu \leq \lambda_ {i} \left(\int_ {\| x \| \leq \bar {B}} 2 \eta d x + \int_ {\| x \| \geq \bar {B}} H (x) d x\right) \\ \lesssim \eta \bar {B} ^ {d} + \bar {B} ^ {d} \exp \left(- b _ {2} \bar {B} ^ {b _ {3}}\right), \tag {18} \\ \end{array}
$$
where we use spherical coordinate to have
$$
\int_ {\| x \| \leq \overline {{B}}} d x = \frac {\pi^ {d / 2}}{\Gamma (d / 2 + 1)} \overline {{B}} ^ {d} \lesssim \overline {{B}} ^ {d},
$$
$$
\begin{array}{l} \int_ {\| x \| \geq \bar {B}} \exp \left(- b _ {2} \| x \| ^ {b _ {3}}\right) \lesssim \int_ {r \geq \bar {B}} r ^ {d - 1} \exp \left(- b _ {2} r ^ {b _ {3}}\right) d r \\ = \frac {1}{b _ {3} b _ {2} ^ {1 / b _ {3}}} \int_ {\overline {{B}} ^ {b _ {3}}} ^ {\infty} u ^ {d / b _ {3} - 1} \exp (- u) d u \quad (\text {c h a n g e o f v a r i a b l e} u = b _ {2} r ^ {b _ {3}}) \\ \leq \frac {1}{b _ {3} b _ {2} ^ {1 / b _ {3}}} \overline {{B}} ^ {d - b _ {3}} \exp (- \overline {{B}} ^ {b _ {3}}). \\ \end{array}
$$
Hence, in (18), if we choose  $\overline{B} = B\bigl (\log (1 / \eta)\bigr)^{1 / b_3}$  then
$$
\int_ {\mathbb {R} ^ {d}} \left(p _ {i} ^ {U} - p _ {i} ^ {L}\right) d \mu \lesssim \eta \left(\log \left(\frac {1}{\eta}\right)\right) ^ {d / b _ {3}}. \tag {19}
$$
Therefore, there exists a positive constant  $c$  which does not depend on  $\eta$  such that
$$
H _ {B} \left(c \eta \log \left(1 / \eta\right) ^ {d / b _ {3}}, \mathcal {P} (\Xi), \| \cdot \| _ {1}\right) \lesssim \log (1 / \eta).
$$
Let  $\epsilon = c\eta (\log (1 / \eta))^{d / b_3}$ , we have  $\log (1 / \epsilon)\asymp \log (1 / \eta)$ , which combines with inequality  $\| \cdot \| _1\leq h^2$  leads to
$$
H _ {B} (\epsilon , \mathcal {P} (\Xi), h) \leq H _ {B} (\epsilon^ {2}, \mathcal {P} (\Xi), \| \cdot \| _ {1}) \lesssim \log (1 / \epsilon^ {2}) \lesssim \log (1 / \epsilon).
$$
Thus, we have proved claim (16).
C Proofs for Geometric Inverse Bounds
C.1 Proof of Theorem 3.3
The second inequality in Theorem 3.3 is straightforward from the equivalent form of  $W_{1}(G,G_{*})$  in Lemma E.1 (see Appendix E). Therefore, we will only focus on establishing the first inequality in that theorem. We start with the following key result:
Proposition C.1. Given the assumptions in Theorem 3.5 and  $\overline{G} = (\overline{\lambda},\overline{\mu},\overline{\Sigma})$  such that  $\overline{\lambda}\in [0,1]$  and  $(\overline{\mu},\overline{\Sigma})$  can be equal to  $(\mu_0,\Sigma_0)$ . Then, we have
$$
\lim  _ {\epsilon \rightarrow 0} \inf  _ {G, G _ {*}} \left\{\frac {V (p _ {G} , p _ {G _ {*}})}{\mathcal {K} (G , G _ {*})}: \mathcal {K} (G, \overline {{G}}) \vee \mathcal {K} (G _ {*}, \overline {{G}}) \leq \epsilon \right\} > 0.
$$
Proof. The high level idea of the proof of Proposition C.3 is to utilize the Taylor expansion techniques previously employed in [9, 28, 22, 19]. Indeed, following Fatou's argument from Theorem 3.1 in [22], to obtain the conclusion of Proposition C.3 it suffices to demonstrate that
$$
\lim  _ {\epsilon \rightarrow 0} \inf  _ {G, G _ {*}} \left\{\frac {\| p _ {G} - p _ {G _ {*}} \| _ {\infty}}{\mathcal {K} (G , G _ {*})}: \mathcal {K} (G, \bar {G}) \vee \mathcal {K} (G _ {*}, \bar {G}) \leq \epsilon \right\} > 0.
$$
Assume that the above conclusion does not hold. It implies that we can find two sequences  $G_{n} = (\lambda_{n}, \mu_{n}, \Sigma_{n})$  and  $G_{*,n} = (\lambda_{n}^{*}, \mu_{n}^{*}, \Sigma_{n}^{*})$  such that  $\mathcal{K}(G_{n}, \overline{G}) \to 0$ ,  $\mathcal{K}(G_{*,n}, \overline{G}) \to 0$ , and  $\| p_{G_{n}} - p_{G_{*,n}} \|_{\infty} / \mathcal{K}(G_{n}, G_{*,n}) \to 0$  as  $n \to \infty$ . Now, we only consider the most challenging setting of  $(\mu_{n}, \Sigma_{n})$  and  $(\mu_{n}^{*}, \Sigma_{n}^{*})$  when they share the same limit point  $(\mu', \Sigma')$ . The other settings of these two components can be argued in the same fashion. Here,  $(\mu', \Sigma')$  is not necessarily equal to  $(\mu_{0}, \Sigma_{0})$  or  $(\overline{\mu}, \overline{\Sigma})$  as  $\lambda_{n}, \lambda_{n}^{*}$  can go to 0 or 1 in the limit. Under that setting, by means of Taylor expansion up to the first order we obtain
$$
\begin{array}{l} \frac {p _ {G _ {n}} (x) - p _ {G _ {* , n}} (x)}{\mathcal {K} (G _ {n} , G _ {* , n})} = \frac {(\lambda_ {n} ^ {*} - \lambda_ {n}) [ h _ {0} (x | \mu_ {0} , \Sigma_ {0}) - f (x | \mu_ {n} ^ {*} , \Sigma_ {n} ^ {*}) ] + \lambda_ {n} [ f (x | \mu_ {n} , \Sigma_ {n}) - f (x | \mu_ {n} ^ {*} , \Sigma_ {n} ^ {*}) ]}{\mathcal {K} (G _ {n} , G _ {* , n})} \\ = \frac {\left(\lambda_ {n} ^ {*} - \lambda_ {n}\right) \left[ h _ {0} \left(x \mid \mu_ {0} , \Sigma_ {0}\right) - f \left(x \mid \mu_ {n} ^ {*} , \Sigma_ {n} ^ {*}\right) \right]}{\mathcal {K} \left(G _ {n} , G _ {* , n}\right)} \\ + \frac {\lambda_ {n} \left(\sum_ {| \alpha | = 1} \frac {\left(\mu_ {n} - \mu_ {n} ^ {*}\right) ^ {\alpha_ {1}} \left(\Sigma_ {n} - \Sigma_ {n} ^ {*}\right) ^ {\alpha_ {2}}}{\alpha !} \frac {\partial^ {| \alpha |} f}{\partial \mu^ {\alpha_ {1}} \partial \Sigma^ {\alpha_ {2}}} \left(x \mid \mu_ {n} ^ {*}, \Sigma_ {n} ^ {*}\right) + R _ {1} (x)\right)}{\mathcal {K} \left(G _ {n} , G _ {* , n}\right)} \\ \end{array}
$$
where  $R_{1}(x)$  is Taylor remainder and  $\alpha = (\alpha_{1},\alpha_{2})$  in the summation of the second equality satisfies  $\alpha_{1} = (\alpha_{1}^{(1)},\ldots ,\alpha_{d_{1}}^{(1)})\in \mathbb{N}^{d_{1}},\alpha_{2} = (\alpha_{uv}^{(2)})\in \mathbb{N}^{d_{2}\times d_{2}},|\alpha | = \sum_{i = 1}^{d_{1}}\alpha_{i}^{(1)} + \sum_{1\leq u,v\leq d_{2}}\alpha_{uv}^{(2)}$ , and  $\alpha ! = \prod_{i = 1}^{d_1}\alpha_i^{(1)!}\prod_{1\leq u,v\leq d_2}\alpha_{uv}^{(2)!}$ . As  $f$  admits the first order uniform Lipschitz condition, we have  $R_{1}(x) = O(\| (\mu_{n},\Sigma_{n}) - (\mu_{n}^{*},\Sigma_{n}^{*})\|^{1 + \gamma})$  for some  $\gamma >0$ , which implies that
$$
\lambda_ {n} \left| R _ {1} (x) \right| / \mathcal {K} \left(G _ {n}, G _ {* , n}\right) = O \left(\left\|\left(\mu_ {n}, \Sigma_ {n}\right) - \left(\mu_ {n} ^ {*}, \Sigma_ {n} ^ {*}\right)\right\| ^ {\gamma}\right)\rightarrow 0
$$
as  $n\to \infty$ . Therefore, we can treat  $[p_{G_n}(x) - p_{G_{*,n}}(x)] / \mathcal{K}(G_n,G_{*,n})$  as the linear combination of  $h_0(x|\theta_0,\Sigma_0)$  and  $\frac{\partial^{|\alpha|}f}{\partial\mu^{\alpha_1}\partial\Sigma^{\alpha_2}} (x|\mu_n^*,\Sigma_n^*)$  when  $|\alpha |\leq 1$ . Assume that the coefficients of these terms go to 0. Then, by studying the coefficients of  $h_0(x|\theta_0,\Sigma_0)$ ,  $\frac{\partial f}{\partial\mu_i} (x|\mu_0,\Sigma_0)$ , and  $\frac{\partial f}{\partial\Sigma_{uv}} (x|\mu_0,\Sigma_0)$ , we achieve
$$
\begin{array}{l} \begin{array}{l} \left(\lambda_ {n} ^ {*} - \lambda_ {n}\right) / \mathcal {K} (G _ {n}, G _ {*}, n) \to 0, \lambda_ {n} \left(\mu_ {n} - \mu_ {n} ^ {*}\right) _ {i} / \mathcal {K} (G _ {n}, G _ {*}, n) \to 0, \lambda_ {n} \left(\Sigma_ {n} - \Sigma_ {n} ^ {*}\right) _ {u v} / \mathcal {K} (G _ {n}, G _ {*}, n) \to 0 \\ \text {f o r a l l} 1 \leq i \leq d _ {1} \text {a n d} 1 \leq u, v \leq d _ {2} \text {w h e r e} (a) _ {i} \text {d e n o t e s t h e} i \text {- t h e l e m e n t o f v e c t o r} a \text {a n d} A _ {u v} \\ \text {d e n o t e s t h e} (u, v) \text {- t h e l e m e n t o f m a t r i x} A. I t w i d d o w {\text {i m p l y t h a t}} \end{array} \\ \left(\lambda_ {n} + \lambda_ {n} ^ {*}\right) \| \left(\mu_ {n}, \Sigma_ {n}\right) - \left(\mu_ {n} ^ {*}, \Sigma_ {n} ^ {*}\right) \| / \mathcal {K} \left(G _ {n}, G _ {*}, n\right)\rightarrow 0. \\ \end{array}
$$
Therefore, we achieve
$$
1 = \left( \right.\left| \lambda_ {n} ^ {*} - \lambda_ {n} \right| + \left(\lambda_ {n} + \lambda_ {n} ^ {*}\right) \| \left(\mu_ {n}, \Sigma_ {n}\right) - \left(\mu_ {n} ^ {*}, \Sigma_ {n} ^ {*}\right)\left. \right|\left. \right) / \mathcal {K} \left(G _ {n}, G _ {\ast , n}\right)\rightarrow 0,
$$
a contradiction. Therefore, not all the coefficients of  $h_0(x|\theta_0,\Sigma_0)$  and  $\frac{\partial^{|\alpha|}f}{\partial\mu^{\alpha_1}\partial\Sigma^{\alpha_2}} (x|\mu_n^*,\Sigma_n^*)$  go to 0. If we denote  $m_{n}$  to be the maximum of the absolute values of the coefficients of  $h_0(x|\theta_0,\Sigma_0)$  and  $\frac{\partial^{|\alpha|}f}{\partial\mu^{\alpha_1}\partial\Sigma^{\alpha_2}} (x|\mu_n^*,\Sigma_n^*)$ , then we get  $1 / m_{n}\not\to\infty$  as  $n\to \infty$ , i.e.,  $1 / m_{n}$  is uniformly bounded. Hence, we achieve for all  $x$  that
$$
\frac {1}{m _ {n}} \frac {p _ {G _ {n}} (x) - p _ {G _ {* , n}} (x)}{K \left(G _ {n} , G _ {* , n}\right)} \rightarrow \eta f \left( \right.\left| \right. \mu_ {0}, \Sigma_ {0}\left. \right) + \sum_ {| \alpha | \leq 1} \tau_ {\alpha} \frac {\partial^ {| \alpha |} f}{\partial \mu^ {\alpha_ {1}} \partial \Sigma^ {\alpha_ {2}}} \left(x \mid \mu^ {\prime}, \Sigma^ {\prime}\right) = 0 \left. \right.
$$
for some coefficients  $\eta$  and  $\tau_{\alpha}$  such that they are not all 0. However, as  $f$  is distinguishable from  $h_0$  up to the first order, the above equation indicates that  $\eta = \tau_{\alpha} = 0$  for all  $|\alpha |\leq 1$ , a contradiction. As a consequence, we achieve the conclusion of the proposition.
Now, assume that the conclusion of Theorem (3.3) does not hold. It implies that we can find two sequences  $G_{n}^{\prime}$  and  $G_{*,n}^{\prime}$  such that  $A_{n} = \| p_{G_{n}^{\prime}} - p_{G_{*,n}^{\prime}}\|_{2} / \mathcal{K}(G_{n}^{\prime},G_{*,n}^{\prime})\to 0$  as  $n\to \infty$ . Since  $\Theta$  and  $\Omega$  are two bounded subsets, we can find subsequences of  $G_{n}^{\prime}$  and  $G_{*,n}^{\prime}$  such that  $\mathcal{K}(G_n',\overline{G}_1)$  and  $\mathcal{K}(G_{*,n}^{\prime},\overline{G}_{2})$  vanish to 0 as  $n\to \infty$  where  $\overline{G}_1,\overline{G}_2$  are some discrete measures having one component to be  $(\mu_0,\Sigma_0)$ . Because  $A_{n}\rightarrow 0$ , we obtain  $V(p_{G_n'}',p_{G_{*,n}'}')\to 0$  as  $n\to \infty$ . By means of Fatou's lemma, we have
$$
0 = \lim  _ {n \to \infty} \int \left| \left(p _ {G _ {n} ^ {\prime}} (x) - p _ {G _ {*}, n} ^ {\prime} (x)\right) \right| d x \geq \int \lim  _ {n \to \infty} \inf  _ {n \to \infty} \left| \left(p _ {G _ {n} ^ {\prime}} (x) - p _ {G _ {*}, n} ^ {\prime} (x)\right) \right| d x = V (p _ {\overline {{G}} _ {1}} (x), p _ {\overline {{G}} _ {2}} (x)).
$$
Due to the fact that  $f$  is distinguishable from  $h_0$  up to the first order, the above equation implies that  $\overline{G}_1 \equiv \overline{G}_2$ . However, from the result of Proposition C.1, regardless of the value of  $\overline{G}_1$  we would have  $A_n \nrightarrow 0$  as  $n \to \infty$ , which is a contradiction. Therefore, we obtain the conclusion of the theorem.
C.2 Proof of Theorem 3.5
Prior to presenting the proof of Theorem 3.5, we introduce the definition of second-order uniform Lipschitz:
Definition C.2 (Second-order Uniform Lipschitz). We say that  $f$  is uniformly Lipschitz up to the second order if the following holds: there are positive constants  $\delta_3, \delta_4$  such that for any  $R_4, R_5, R_6 > 0, \gamma_1 \in \mathbb{R}^{d_1}, \gamma_2 \in \mathbb{R}^{d_2 \times d_2}, R_4 \leq \sqrt{\lambda_1(\Sigma)} \leq \sqrt{\lambda_{d_2}(\Sigma)} \leq R_5, \| \theta \| \leq R_6, \theta_1, \theta_2 \in \Theta, \Sigma_1, \Sigma_2 \in \Omega,$  there are positive constants  $C_1$  depending on  $(R_4, R_5)$  and  $C_2$  depending on  $R_6$  such that for all  $x \in \mathcal{X}$ ,
$$
\begin{array}{l} \left| \gamma_ {1} ^ {\top} \left(\frac {\partial^ {2} f}{\partial \theta^ {2}} (x | \theta_ {1}, \Sigma) - \frac {\partial^ {2} f}{\partial \theta^ {2}} (x | \theta_ {2}, \Sigma)\right) \gamma_ {1} \right| \leq \| \theta_ {1} - \theta_ {2} \| _ {1} ^ {\delta_ {3}} \| \gamma_ {1} \| _ {2} ^ {2}, \\ \left. \right.\left| \operatorname {t r} \left(\left[ \frac {\partial}{\partial \Sigma} \left(\operatorname {t r} \left(\frac {\partial f}{\partial \Sigma_ {1}} (x | \theta , \Sigma) ^ {\top} \gamma_ {2}\right)\right) - \frac {\partial}{\partial \Sigma} \left(\operatorname {t r} \left(\frac {\partial f}{\partial \Sigma} (x | \theta , \Sigma_ {2}) ^ {\top} \gamma_ {2}\right)\right)\right] ^ {\top} \gamma_ {2}\right)\right| \\ \leq C _ {2} \| \Sigma_ {1} - \Sigma_ {2} \| _ {2} ^ {\delta_ {4}} \| \gamma \| _ {2} ^ {2}. \\ \end{array}
$$
Now, we are back to the main proof. Utilizing the same Fatou's argument as that of Proposition C.1, to achieve the conclusion of the first inequality in Theorem 3.5 it suffices to demonstrate the following result
Proposition C.3. Given the assumptions in Theorem 3.5 and  $\overline{G} = (\overline{\lambda},\overline{\mu},\overline{\Sigma})$  such that  $\overline{\lambda}\in [0,1]$  and  $(\overline{\mu},\overline{\Sigma})$  can be identical to  $(\mu_0,\Sigma_0)$ . Then, the following holds
(a) If  $(\mu_0,\Sigma_0)\neq (\overline{\mu},\overline{\Sigma})$  and  $\overline{\lambda} >0$  , then
$$
\lim  _ {\epsilon \rightarrow 0} \inf  _ {G, G _ {*}} \left\{\frac {\| p _ {G} - p _ {G _ {*}} \| _ {\infty}}{\mathcal {K} (G , G _ {*})}: \mathcal {K} (G, \overline {{G}}) \vee \mathcal {K} (G _ {*}, \overline {{G}}) \leq \epsilon \right\} > 0.
$$
(b) If  $(\mu_0,\Sigma_0)\equiv (\overline{\mu},\overline{\Sigma})$  or  $(\mu_0,\Sigma_0)\neq (\overline{\mu},\overline{\Sigma})$  and  $\overline{\lambda} = 0$  , then
$$
\lim  _ {\epsilon \rightarrow 0} \inf  _ {G, G _ {*}} \left\{\frac {\| p _ {G} - p _ {G _ {*}} \| _ {\infty}}{\mathcal {D} (G , G _ {*})}: \mathcal {D} (G, \overline {{G}}) \vee \mathcal {D} (G _ {*}, \overline {{G}}) \leq \epsilon \right\} > 0.
$$
Proof. The proof of part (a) is essentially similar to that of Proposition C.1; therefore, we only provide the proof for the challenging settings of part (b). Here, we only consider the setting that  $(\mu_0,\Sigma_0)\equiv (\overline{\mu},\overline{\Sigma})$  as the proof for other possibilities of  $(\mu_0,\Sigma_0)$  can be argued in the similar fashion. Under this assumption,  $(\mu_0,\Sigma_0) = (\mu_0,\Sigma_0),\overline{G} = (\overline{\lambda},\theta_0,\Sigma_0)$ , and  $h_0(x|\theta_0,\Sigma_0) = f(x|\theta_0,\Sigma_0)$  for all  $x\in \mathcal{X}$ . Assume that the conclusion of Proposition C.3 does not hold. It implies that we can find two sequences  $G_{n} = (\lambda_{n},\mu_{n},\Sigma_{n})$  and  $G_{*,n} = (\lambda_n^*,\mu_n^*,\Sigma_n^*)$  such that  $\mathcal{D}(G_n,\overline{G}) = \lambda_n\| (\Delta \mu_n,\Delta \Sigma_n)\| ^2\to 0$ $\mathcal{D}(G_{*,n},\overline{G}) = \lambda_n^*\| (\Delta \mu_n^*,\Delta \Sigma_n^*)\| ^2\to 0$ , and  $\| p_{G_n} - p_{G_{*,n}}\|_{\infty} / \mathcal{D}(G_n,G_{*,n})\to 0$  as  $n\to \infty$ . For the transparency of presentation, we denote  $A_{n} = \| (\Delta \mu_{n},\Delta \Sigma_{n})\|$ ,  $B_{n} = \| (\Delta \mu_{n}^{*},\Delta \Sigma_{n}^{*})\|$ , and  $C_n = \| (\mu_n,\Sigma_n) - (\mu_n^*,\Sigma_n^*)\| = \| (\Delta \mu_n,\Delta \Sigma_n) - (\Delta \mu_n^*,\Delta \Sigma_n^*)\|$ . Now, we have three main cases regarding the convergence behaviors of  $(\mu_n,\Sigma_n)$  and  $(\mu_n^*,\Sigma_n^*)$
Case 1: Both  $A_{n}\to 0$  and  $B_{n}\rightarrow 0$ , i.e.,  $(\mu_n,\Sigma_n)$  and  $(\mu_n^*,\Sigma_n^*)$  vanish to  $(\mu_0,\Sigma_0)$  as  $n\to \infty$ . Due to the symmetry between  $\lambda_{n}$  and  $\lambda_{n}^{*}$ , we assume without loss of generality that  $\lambda_{n}^{*}\geq \lambda_{n}$  for infinite values of  $n$ . Without loss of generality, we replace these subsequences of  $G_{n},G_{*,n}$  by the whole sequences of  $G_{n}$  and  $G_{*,n}$ . Now, the formulation of  $\mathcal{D}(G_n,G_{*,n})$  is
$$
\mathcal {D} (G _ {n}, G _ {*, n}) = (\lambda_ {n} ^ {*} - \lambda_ {n}) B _ {n} ^ {2} + \bigg (\lambda_ {n} A _ {n} + \lambda_ {n} ^ {*} B _ {n} \bigg) C _ {n}.
$$
Now, by means of Taylor expansion up to the second order, we get
$$
\begin{array}{l} \frac {p _ {G _ {n}} (x) - p _ {G _ {* , n}} (x)}{\mathcal {D} (G _ {n} , G _ {* , n})} = \frac {(\lambda_ {n} ^ {*} - \lambda_ {n}) [ f (x | \mu_ {0} , \Sigma_ {0}) - f (x | \mu_ {n} ^ {*} , \Sigma_ {n} ^ {*}) ] + \lambda_ {n} [ f (x | \mu_ {n} , \Sigma_ {n}) - f (x | \mu_ {n} ^ {*} , \Sigma_ {n} ^ {*}) ]}{\mathcal {D} (G _ {n} , G _ {* , n})} \\ = \frac {\left(\lambda_ {n} ^ {*} - \lambda_ {n}\right) \left(\sum_ {| \alpha | = 1} ^ {2} \frac {(- \Delta \mu_ {n} ^ {*}) ^ {\alpha_ {1}} (- \Delta \Sigma_ {n} ^ {*}) ^ {\alpha_ {2}}}{\alpha !} \frac {\partial^ {| \alpha |} f}{\partial \mu^ {\alpha_ {1}} \partial \Sigma^ {\alpha_ {2}}} (x | \mu_ {n} ^ {*}, \Sigma_ {n} ^ {*}) + R _ {1} (x)\right)}{\mathcal {D} \left(G _ {n} , G _ {* , n}\right)} \\ + \frac {\lambda_ {n} \left(\sum_ {| \alpha | = 1} ^ {2} \frac {\left(\Delta \mu_ {n} - \Delta \mu_ {n} ^ {*}\right) ^ {\alpha_ {1}} \left(\Delta \Sigma_ {n} - \Delta \Sigma_ {n} ^ {*}\right) ^ {\alpha_ {2}}}{\alpha !} \frac {\partial^ {| \alpha |} f}{\partial \mu^ {\alpha_ {1}} \partial \Sigma^ {\alpha_ {2}}} \left(x \mid \mu_ {n} ^ {*}, \Sigma_ {n} ^ {*}\right) + R _ {2} (x)\right)}{\mathcal {D} \left(G _ {n} , G _ {* , n}\right)} \\ \end{array}
$$
where  $R_{1}(x)$  and  $R_{2}(x)$  are Taylor remainders that satisfy  $R_{1}(x) = O(B_{n}^{2 + \gamma})$  and  $R_{2}(x) = O(C_{n}^{2 + \gamma})$  for some positive number  $\gamma$  due to the second order uniform Lipschitz condition of kernel density function  $f$ . From the formation of  $\mathcal{D}(G_n, G_{*,n})$ , since  $A_{n} + B_{n} \geq C_{n}$  (triangle inequality), as  $A_{n} \to 0$  and  $B_{n} \to 0$  it is clear that
$$
\left(\lambda_ {n} - \lambda_ {n} ^ {*}\right)\left| R _ {1} (x) \right| / \mathcal {D} \left(G _ {n}, G _ {*, n}\right) \leq \left| R _ {1} (x) \right| / B _ {n} ^ {2} = O \left(B _ {n} ^ {\gamma}\right)\rightarrow 0
$$
$$
\lambda_ {n} | R _ {2} (x) | / \mathcal {D} (G _ {n}, G _ {*}, n) \leq | R _ {2} (x) | / \left\{(A _ {n} + B _ {n}) C _ {n} \right\} = O \left(C _ {n} ^ {2 + \gamma} / C _ {n} ^ {2}\right) = O \left(C _ {n} ^ {\gamma}\right)\rightarrow 0
$$
as  $n\to \infty$  for all  $x\in \mathcal{X}$  . Therefore, we achieve for all  $x\in \mathcal{X}$  that
$$
\left(\left(\lambda_ {n} - \lambda_ {n} ^ {*}\right)\left| R _ {1} (x) \right| + \lambda_ {n} \left| R _ {2} (x) \right|\right) / \mathcal {D} \left(G _ {n}, G _ {*, n}\right)\rightarrow 0.
$$
Hence, we can treat  $[p_{G_n}(x) - p_{G_{*,n}}(x)] / \mathcal{D}(G_n,G_{*,n})$  as a linear combination of  $\frac{\partial^{|\alpha|}f}{\partial\mu^{\alpha_1}\partial\Sigma^{\alpha_2}} (x|\mu_n^*,\Sigma_n^*)$  for all  $x$  and  $\alpha = (\alpha_{1},\alpha_{2})$  such that  $1\leq |\alpha |\leq 2$ . Assume that all the coefficients of these terms go to 0 as  $n\to \infty$ . By studying the vanishing behaviors of the coefficients of  $\frac{\partial^{|\alpha|}f}{\partial\mu^{\alpha_1}\partial\Sigma^{\alpha_2}} (x|\mu_n^*,\Sigma_n^*)$  as  $|\alpha | = 1$ , we achieve the following limits