SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \mathbf {E} \left[ \left(\hat {f} _ {n} (X, S) - f _ {\mathrm {D P}} ^ {*} (X, S)\right) ^ {2} \right] \\ \leq \sum_ {s \in [ M ]} C p _ {s} \left(\frac {\sigma_ {X} ^ {2} B ^ {2}}{p _ {s} n} + \frac {\sigma_ {\xi} ^ {2} M d}{n} + \frac {\sigma_ {X} ^ {2} B ^ {2}}{n} + \frac {\sigma_ {\xi} ^ {2} \overline {{\| \beta_ {s} ^ {*} \| ^ {2}}} d}{\| \beta_ {s} ^ {*} \| ^ {2} p _ {s} n}\right) \\ + \sum_ {s \in [ M ]} C \left(\frac {p _ {s} \sigma_ {X} ^ {2}}{n} + \frac {p _ {s} \sigma_ {X} ^ {2}}{n}\right) + C \frac {B ^ {2} U ^ {2}}{n} + o \left(\frac {1}{n}\right). \\ \end{array}
$$
Consequently, there exists an universal constant  $C > 0$  such that
$$
\begin{array}{l} \mathbf {E} \left[ \left(\hat {f} _ {n} (X, S) - f _ {\mathrm {D P}} ^ {*} (X, S)\right) ^ {2} \right] \leq \\ C \left(\frac {\sigma_ {X} ^ {2} B ^ {2} M}{n} + \frac {\sigma_ {\xi} ^ {2} M d}{n} + \frac {\sigma_ {X} ^ {2} B ^ {2}}{n} + \frac {\sigma_ {\xi} ^ {2} B ^ {2} M d}{n} + \frac {\sigma_ {X} ^ {2}}{n} + \frac {\sigma_ {X} ^ {2}}{n} + \frac {B ^ {2} U ^ {2}}{n}\right) + o \left(\frac {1}{n}\right). \\ \end{array}
$$
Then, the dominating terms match the claim.
G Details of Lower Bound Analyses
This section provides the proofs of the lower bound analyses results.
G.1 Proof of Lower Bound in Theorem 1
Theorem 22. If  $M(d - 1) > 16$ , there exists an universal constant  $C > 0$  such that for any  $\alpha > 0$  and  $\delta \in (0,1)$ ,
$$
\mathcal {E} _ {n} (\alpha , \delta) \geq C \frac {\sigma_ {\xi} ^ {2} B ^ {2} d M}{n} - o \bigg (\frac {1}{n} \bigg).
$$
Proof of Theorem 22. The Varshamov-Gilbert bound guarantees that there exists a subset  $\mathcal{V}' \subseteq \mathcal{V}$  such that  $|\mathcal{V}'| \geq 2^{M(d-1)/8}$  and  $d_H(v_s, v_s') \geq (d-1)/8$  for any  $v, v' \in \mathcal{V}'$ . With the choice of  $\epsilon_s^2 = \left(\frac{d-1}{16} - \frac{1}{M}\right)\sigma_\xi^2 / 2\sigma_X^2 B_s^2 n_s$ , we confirm by Theorem 8 that  $\inf_{\pi} \frac{1}{K} \sum_{v \in \mathcal{V}'} D_{\mathrm{KL}}\left(\pi_{\theta_v|n}, \pi\right) \leq \max_{v, v' \in \mathcal{V}'} D_{\mathrm{KL}}\left(\pi_{\theta_{v'}}|n., \pi\right) \leq \ln(|\mathcal{V}'|/4)/2 \geq M(d-1)/16-1$ . From Theorem 8 and the fact  $d_H(v_s, v_s') \geq (d-1)/8$ , we can apply Theorem 6 with  $\epsilon = \sum_{s \in [M]} p_s \frac{\left(\sum_{s' \in [M]} p_s B_s\right)^2}{B_s^2} \left(\frac{d-1}{16} - \frac{1}{M}\right)\sigma_\xi^2 / 2 n_s$ . From the fact that  $\mathbf{E}\left[\frac{1}{n_s+1}\right] = \frac{1}{p_s(n+1)} (1-(1-p_s)^n)$  due to [5], there exists an universal constant  $C > 0$  such that  $\mathbf{E}\left[\frac{\epsilon}{2}\right] \geq C \left(\frac{1}{M} \sum_{s \in [M]} \frac{\left(\sum_{s' \in [M]} p_{s'} B_{s'}\right)^2}{B_s^2}\right) \frac{\sigma_\xi^2 M d}{n} - o\left(\frac{1}{n}\right)$ . We can get the claim by confirming that there exists  $B_1, ..., B_M$  such that  $\left(\frac{1}{M} \sum_{s \in [M]} \frac{\left(\sum_{s' \in [M]} p_{s'} B_{s'}\right)^2}{B_s^2}\right) = B^2$  and  $B_s \leq B$ . Because for  $B_2 = ... = B_M = B$ , tending  $B_1$  to 0 results in  $\left(\frac{1}{M} \sum_{s \in [M]} \frac{\left(\sum_{s' \in [M]} p_{s'} B_{s'}\right)^2}{B_s^2}\right)$  goes infinity, it is confirmed.
G.2 Proof of Theorem 6
Proof of Theorem 6. Since the distribution of  $n$  is invariant against  $\theta \in \Theta$ , we have
$$
\begin{array}{l} \sup  _ {\theta \in \Theta} \mathbf {E} _ {\theta} \left[ \mathcal {E} \left(\hat {f} _ {n}; \theta\right) \right] \\ = \mathbf {E} \left[ \sup  _ {\theta \in \Theta} \mathbf {E} _ {\theta} \left[ \mathcal {E} (\hat {f} _ {n}; \theta) | n. \right] \right] \\ \geq \mathbf {E} \left[ \max  _ {\theta \in \hat {\Theta}} \mathbf {E} _ {\theta} \left[ \mathcal {E} (\hat {f} _ {n}; \theta) | n. \right] \right] \\ \end{array}
$$
$$
\geq \mathbf {E} \left[ \frac {1}{| \hat {\Theta} |} \sum_ {\theta \in \hat {\Theta}} \mathbf {E} _ {\theta} \left[ \mathcal {E} (\hat {f} _ {n}; \theta) \Big | n. \right] \right].
$$
Given  $\epsilon$  possibly dependent on  $n$ , application of the Markov inequality yields
$$
\begin{array}{l} \sup _ {\theta \in \Theta} \mathbf {E} _ {\theta} \left[ \mathcal {E} (\hat {f} _ {n}; \theta) \right] \\ \geq \mathbf {E} \left[ \frac {\epsilon}{| \hat {\Theta} |} \sum_ {\theta \in \hat {\Theta}} \mathbb {P} _ {\theta} \Bigl \{\mathcal {E} (\hat {f} _ {n}; \theta) \geq \epsilon \Big | n. \Bigr \} \right]. \\ \end{array}
$$
If  $\inf_{f}\mathcal{E}(f;\theta)\vee \mathcal{E}(f;\theta^{\prime})\geq \epsilon$  for any  $\theta ,\theta^{\prime}\in \hat{\Theta},\mathcal{E}(f;\theta) <   \epsilon$  implies  $\mathcal{E}(f;\theta^{\prime})\geq \epsilon$  for any  $\theta^{\prime}\in \hat{\Theta}$  such that  $\theta \neq \theta^{\prime}$ . Hence, there exists a parton  $\{\mathcal{F}_{\theta}\}_{\theta \in \hat{\Theta}}$  of all the measurable functions  $f:\mathbb{R}^{d}\times [M]\to \mathbb{R}$  such that  $\{f:\mathcal{E}(f;\theta) <   \epsilon \} \subseteq \mathcal{F}_{\theta}$  for all  $\theta \in \hat{\Theta}$ . Consequently, we have
$$
\sup  _ {\theta \in \Theta} \mathbf {E} _ {\theta} \left[ \mathcal {E} (\hat {f} _ {n}; \theta) \right] \geq \mathbf {E} \left[ \epsilon \left(1 - \frac {1}{| \hat {\Theta} |} \sum_ {\theta \in \hat {\Theta}} \mathbb {P} _ {\theta} \left\{\hat {f} _ {n} \in \mathcal {F} _ {\theta} \mid n. \right\}\right) \right].
$$
Application of the Fano's inequality and data processing inequality yields the claim.
G.3 Proof of Theorem 7
To prove Theorem 7, we show the following more tight lower bound.
Theorem 23. Let  $\theta$  and  $\theta'$  be the parameters of the distributions such that  $\frac{1}{2\sigma_X^2}\|\mu_s - \mu_s'\|^2 \coloneqq d_s < 1$  for all  $s \in [M]$ . Then, we have
$$
\begin{array}{l} \inf  _ {f \in \mathcal {L} ^ {2}} \mathcal {E} (f; \theta) \vee \mathcal {E} (f; \theta^ {\prime}) \geq \\ \sum_ {s \in [ M ]} p _ {s} \frac {e ^ {- d _ {s}}}{4} \left(\sigma_ {X} ^ {2} \left\| \frac {\overline {{\| \beta . \|}} \beta_ {s}}{\| \beta_ {s} \|} - \frac {\overline {{\| \beta^ {\prime} \|}} \beta_ {s} ^ {\prime}}{\| \beta_ {s} ^ {\prime} \|} \right\| ^ {2} \left(1 + \frac {\| \mu_ {s} - \mu_ {s} ^ {\prime} \| ^ {2}}{4 \sigma_ {X} ^ {2}}\right) ^ {1 + \frac {d}{2}}\right) \\ + \left(- \left\langle \frac {\| \beta_ {s} \|}{\| \beta_ {s} \|} + \frac {\| \beta_ {s} ^ {\prime} \|}{\| \beta_ {s} ^ {\prime} \|}, \frac {\mu_ {s} - \mu_ {s} ^ {\prime}}{2} \right\rangle \right. \\ + \sum_ {s ^ {\prime} \in [ M ]} p _ {s ^ {\prime}} \bigg (\langle \beta_ {s ^ {\prime}} - \beta_ {s ^ {\prime}} ^ {\prime}, \bar {\mu} _ {s ^ {\prime}} \rangle + \Bigg \langle \beta_ {s ^ {\prime}} + \beta_ {s ^ {\prime}} ^ {\prime}, \frac {\mu_ {s ^ {\prime}} - \mu_ {s ^ {\prime}} ^ {\prime}}{2} \Bigg \rangle \bigg) \bigg) ^ {2} \bigg (1 + \frac {\| \mu_ {s} - \mu_ {s} ^ {\prime} \| ^ {2}}{4 \sigma_ {X} ^ {2}} \bigg) ^ {\frac {d}{2}} \bigg). \\ \end{array}
$$
Theorem 23 immediately gives Theorem 7.
We utilize the sufficient condition for the constrained optimization problem over a Banach space. Let  $Z$  be a Banach space. We say a function  $f: Z \to \mathbb{R}$  is Gateaux differentiable if the limit  $\lim_{\tau \to 0} \frac{f(z + \tau u) - f(z)}{\tau}$  exists for any open set  $U \subseteq Z$ , any  $z \in U$ , and any  $u \in Z$ . We denote the Gateaux derivative of  $f$  at  $z \in Z$ , a linear mapping from  $u \in Z$  to  $\lim_{\tau \to 0} \frac{f(z + \tau u) - f(z)}{\tau}$ , as  $D_G f(z)$ . We abuse 0 to denote the mapping that always outputs 0.
Proof of Theorem 23. Let  $q_{s}$  and  $q_{s}^{\prime}$  be the density function of  $X_{s}$  with the parameters  $(\beta_{\cdot}, \mu_{\cdot})$  and  $(\beta_{\cdot}^{\prime}, \mu_{\cdot}^{\prime})$ , respectively, regarding the base measure  $\lambda$ . Since  $X_{s}$  follows the Gaussian distribution, we can choose  $\lambda$  as the Lebesgue measure. Given  $\eta \in [0,1]$ , we have
$$
\begin{array}{l} \mathcal {E} (f; \beta , \mu) \vee \mathcal {E} (f; \beta^ {\prime}, \mu^ {\prime}) \\ \geq \eta \mathcal {E} (f; \beta , \mu .) + (1 - \eta) \mathcal {E} (f; \beta^ {\prime}, \mu^ {\prime}) \\ = \sum_ {s \in [ M ]} p _ {s} \int \left(\eta \left(f (x, s) - f _ {\beta , \mu .} (x, s)\right) ^ {2} q _ {s} (x) + (1 - \eta) \left(f (x, s) - f _ {\beta^ {\prime}, \mu^ {\prime}} (x, s)\right) ^ {2} q _ {s} ^ {\prime} (x)\right) \lambda (d x). \\ \end{array}
$$
Because  $\eta \mathcal{E}(f;\beta ,\mu_{\cdot}) + (1 - \eta)\mathcal{E}(f;\beta^{\prime},\mu^{\prime})$  is convex for  $f$ , and  $\mathcal{L}^2$  is a Banach space, it is minimized if for any  $u\in \mathcal{L}^2$
$$
\left. \frac {d}{d \gamma} \left(\eta \mathcal {E} \left(f + \gamma u; \beta_ {\cdot}, \mu_ {\cdot}\right) + \left(1 - \eta\right) \mathcal {E} \left(f + \gamma u; \beta_ {\cdot} ^ {\prime}, \mu_ {\cdot} ^ {\prime}\right)\right) \right| _ {\gamma = 0} = 0.
$$
The dominated convergence theorem gives
$$
\begin{array}{l} \frac {d}{d \gamma} (\eta \mathcal {E} (f + \gamma u; \beta_ {\cdot}, \mu_ {\cdot}) + (1 - \eta) \mathcal {E} (f + \gamma u; \beta_ {\cdot} ^ {\prime}, \mu_ {\cdot} ^ {\prime})) \bigg | _ {\gamma = 0} \\ = \sum_ {s \in [ M ]} p _ {s} \int (\eta (f (x, s) - f _ {\beta ., \mu .} (x, s)) u (x, s) q _ {s} (x) \\ + (1 - \eta) \left(f (x, s) - f _ {\beta^ {\prime}, \mu^ {\prime}} (x, s)\right) u (x, s) q _ {s} ^ {\prime} (x)) \lambda (d x) \\ = \sum_ {s \in [ M ]} p _ {s} \int (f (x, s) (\eta q _ {s} (x) + (1 - \eta) q _ {s} ^ {\prime} (x)) \\ \left. - \left(\eta f _ {\beta , \mu .} (x, s) q _ {s} (x) + (1 - \eta) f _ {\beta^ {\prime}, \mu^ {\prime}} (x, s) q _ {s} ^ {\prime} (x)\right)\right) u (x, s) \lambda (d x). \\ \end{array}
$$
Consequently,  $\eta \mathcal{E}(f;\beta ,\mu .) + (1 - \eta)\mathcal{E}(f;\beta ',\mu ')$  is minimized at
$$
f (x, s) = \frac {\eta f _ {\beta , . \mu .} (x , s) q _ {s} (x) + (1 - \eta) f _ {\beta^ {\prime} , \mu^ {\prime}} (x , s) q _ {s} ^ {\prime} (x)}{\eta q _ {s} (x) + (1 - \eta) q _ {s} ^ {\prime} (x)}.
$$
$$
\begin{array}{l} \eta \mathcal {E} (f + \gamma u; \beta ., \mu .) + (1 - \eta) \mathcal {E} (f + \gamma u; \beta^ {\prime}, \mu^ {\prime}) \\ \geq \sum_ {s \in [ M ]} p _ {s} \int \frac {\eta (1 - \eta) q _ {s} (x) q _ {s} ^ {\prime} (x)}{\eta q _ {s} (x) + (1 - \eta) q _ {s} ^ {\prime} (x)} \left(f _ {\beta , \mu .} (x, s) - f _ {\beta^ {\prime}, \mu^ {\prime}} (x, s)\right) ^ {2} \lambda (d x). \\ \end{array}
$$
With  $\eta = 1 / 2$ , we have
$$
\begin{array}{l} \frac {\eta (1 - \eta) q _ {s} (x) q _ {s} ^ {\prime} (x)}{\eta q _ {s} (x) + (1 - \eta) q _ {s} ^ {\prime} (x)} = \frac {1}{4} \frac {\sqrt {q _ {s} (x) q _ {s} ^ {\prime} (x)}}{\frac {1}{2} \sqrt {\frac {q _ {s} (x)}{q _ {s} ^ {\prime} (x)}} + \frac {1}{2} \sqrt {\frac {q _ {s} ^ {\prime} (x)}{q _ {s} (x)}}} \\ = \frac {1}{4} \frac {\sqrt {q _ {s} (x) q _ {s} ^ {\prime} (x)}}{\cosh (\frac {1}{2} \ln \frac {q _ {s} (x)}{q _ {s} ^ {\prime} (x)})}. \\ \end{array}
$$
Let  $\bar{\mu}_s = \frac{1}{2} (\mu_s + \mu_s')$ . Then, we have
$$
\begin{array}{l} \sqrt {q _ {s} (x) q _ {s} ^ {\prime} (x)} \\ = \frac {1}{\sqrt {(2 \pi) ^ {d} \sigma_ {X} ^ {2 d}}} \exp \left(- \frac {1}{4 \sigma_ {X} ^ {2}} \| x - \mu_ {s} \| ^ {2} - \frac {1}{4 \sigma_ {X} ^ {2}} \| x - \mu_ {s} ^ {\prime} \| ^ {2}\right) \\ = \frac {1}{\sqrt {(2 \pi) ^ {d} \sigma_ {X} ^ {2 d}}} \exp \left(- \frac {1}{2 \sigma_ {X} ^ {2}} \| x - \bar {\mu} _ {s} \| ^ {2} - \frac {1}{2 \sigma_ {X} ^ {2}} \left\| \frac {\mu_ {s} - \mu_ {s} ^ {\prime}}{2} \right\| ^ {2} \right. \\ \left. - \frac {1}{2 \sigma_ {X} ^ {2}} \left(\left\langle x - \bar {\mu} _ {s}, \frac {\mu_ {s} ^ {\prime} - \mu_ {s}}{2} + \frac {\mu_ {s} - \mu_ {s} ^ {\prime}}{2} \right\rangle\right)\right) \\ = \frac {1}{\sqrt {(2 \pi) ^ {d} \sigma_ {X} ^ {2 d}}} \exp \left(- \frac {1}{2 \sigma_ {X} ^ {2}} \| x - \bar {\mu} _ {s} \| ^ {2} - \frac {1}{2 \sigma_ {X} ^ {2}} \left\| \frac {\mu_ {s} - \mu_ {s} ^ {\prime}}{2} \right\| ^ {2}\right). \\ \end{array}
$$
$$
\frac {q _ {s} (x)}{q _ {s} ^ {\prime} (x)}
$$
$$
\begin{array}{l} = \exp \left(- \frac {1}{2 \sigma_ {X} ^ {2}} \| x - \mu_ {s} \| ^ {2} + \frac {1}{2 \sigma_ {X} ^ {2}} \| x - \mu_ {s} ^ {\prime} \| ^ {2}\right) \\ = \exp \left(- \frac {1}{\sigma_ {X} ^ {2}} \left\langle x - \bar {\mu} _ {s}, \frac {\mu_ {s} ^ {\prime} - \mu_ {s}}{2} - \frac {\mu_ {s} - \mu_ {s} ^ {\prime}}{2} \right\rangle\right) \\ = \exp \bigg (\frac {1}{\sigma_ {X} ^ {2}} \langle x - \bar {\mu} _ {s}, \mu_ {s} - \mu_ {s} ^ {\prime} \rangle \bigg). \\ \end{array}
$$
Let  $\bar{X}_s\sim N(\bar{\mu}_s,\sigma_X^2 I)$  . Then, we have
$$
\begin{array}{l} \mathcal {E} (f; \beta_ {\cdot}, \mu_ {\cdot}) \vee \mathcal {E} (f; \beta_ {\cdot} ^ {\prime}, \mu^ {\prime}) \\ \geq \sum_ {s \in [ M ]} p _ {s} e ^ {- \frac {1}{2 \sigma_ {X} ^ {2}} \| \frac {\mu_ {s} - \mu_ {s} ^ {\prime}}{2} \| ^ {2}} \mathbf {E} \left[ \frac {1}{4} \frac {\left(f _ {\beta , . \mu_ {.}} (\bar {X} _ {s} , s) - f _ {\beta^ {\prime} , \mu^ {\prime}} (\bar {X} _ {s} , s)\right) ^ {2}}{\cosh \left(\frac {1}{2 \sigma_ {X} ^ {2}} \left\langle \bar {X} _ {s} - \bar {\mu} _ {s} , \mu_ {s} - \mu_ {s} ^ {\prime} \right\rangle\right)} \right] \\ \geq \sum_ {s \in [ M ]} p _ {s} e ^ {- \frac {1}{2 \sigma_ {X} ^ {2}} \| \frac {\mu_ {s} - \mu_ {s} ^ {\prime}}{2} \| ^ {2}} \mathbf {E} \left[ \frac {1}{4} \frac {\left(f _ {\beta , . \mu_ {.}} \left(\bar {X} _ {s} , s\right) - f _ {\beta^ {\prime} , \mu_ {.} ^ {\prime}} \left(\bar {X} _ {s} , s\right)\right) ^ {2}}{\cosh \left(\frac {1}{2 \sigma_ {X} ^ {2}} \| \bar {X} _ {s} - \bar {\mu} _ {s} \| \| \mu_ {s} - \mu_ {s} ^ {\prime} \|\right)} \right], \tag {37} \\ \end{array}
$$
where the last line is obtained from the Cauchy-Schwarz inequality.
$$
f _ {\beta , \mu .} (x, s) - f _ {\beta^ {\prime}, \mu^ {\prime}} (x, s)
$$
$$
\begin{array}{l} = \left\langle \frac {\| \beta_ {s} \| \beta_ {s}}{\| \beta_ {s} \|} - \frac {\| \beta_ {s} ^ {\prime} \|}{\| \beta_ {s} ^ {\prime} \|}, x - \bar {\mu} _ {s} \right\rangle - \left\langle \frac {\| \beta_ {s} \| \beta_ {s}}{\| \beta_ {s} \|} + \frac {\| \beta_ {s} ^ {\prime} \| \beta_ {s} ^ {\prime}}{\| \beta_ {s} ^ {\prime} \|}, \frac {\mu_ {s} - \mu_ {s} ^ {\prime}}{2} \right\rangle \\ + \sum_ {s ^ {\prime} \in [ M ]} p _ {s ^ {\prime}} \bigg (\langle \beta_ {s ^ {\prime}} - \beta_ {s ^ {\prime}} ^ {\prime}, \bar {\mu} _ {s ^ {\prime}} \rangle + \Bigg \langle \beta_ {s ^ {\prime}} + \beta_ {s ^ {\prime}} ^ {\prime}, \frac {\mu_ {s ^ {\prime}} - \mu_ {s ^ {\prime}} ^ {\prime}}{2} \Bigg \rangle \bigg). \\ \end{array}
$$
Conditioned on  $\| \bar{X}_s - \bar{\mu}_s\| = r$  for  $r > 0$ ,  $\bar{X}_s$  follows the uniform distribution over the  $(d - 1)$ -sphere centered at  $\bar{\mu}_s$ . For a random variable  $U$  uniformly distributed over the  $(d - 1)$ -sphere centered at origin with the radius  $r$ ,  $\mathbf{E}[U] = 0$  and  $\mathbf{E}[UU^\top] = r^2 / dI$ . Hence, for a vector  $v \in \mathbb{R}^d$  and a scalar  $c \in \mathbb{R}$ , we have
$$
\begin{array}{l} \mathbf {E} \left[ \left(\left\langle v, \bar {X} _ {s} - \mu_ {s} \right\rangle + c\right) ^ {2} \mid \| \bar {X} _ {s} - \bar {\mu} _ {s} \| = r \right] \\ = \frac {r ^ {2}}{d} \| v \| ^ {2} + c ^ {2}. \\ \end{array}
$$
An elementary analysis yields that  $\| \bar{X}_s - \bar{\mu}_s\| ^2\sim \mathrm{Gamma}(\frac{d}{2},2\sigma_X^2)$ , where  $\mathrm{Gamma}(k,\theta)$  denotes the Gamma distribution with the shape parameter  $k$  and scale parameter  $\theta$ . From the upper bound of the hyperbolic cosine as  $\cosh (x)\leq e^{x^2 /2}$ , for a vector  $v\in \mathbb{R}^d$ , scalars  $c\in \mathbb{R}$  and  $c^\prime >0$ , and a random variable  $\gamma \sim \mathrm{Gamma}(k,\theta)$ , we have
$$
\begin{array}{l} \mathbf {E} \left[ \frac {1}{\cosh (c ^ {\prime} \sqrt {\gamma})} \left(\frac {\gamma}{d} \| v \| ^ {2} + c ^ {2}\right) \right] \\ \geq \mathbf {E} \left[ \left(\frac {\gamma}{d} \| v \| ^ {2} + c ^ {2}\right) e ^ {- \frac {c ^ {\prime 2} \gamma}{2}} \right] \\ = \mathbf {E} \left[ \left(\frac {\gamma}{d} \| v \| ^ {2} + c ^ {2}\right) \sum_ {m = 0} ^ {\infty} (- 1) ^ {m} \frac {c ^ {\prime 2 m} \gamma^ {m}}{2 ^ {m} m !} \right] \\ = \sum_ {m = 0} ^ {\infty} (- 1) ^ {m} \frac {c ^ {\prime 2 m}}{2 ^ {m} m !} \left(\frac {\| v \| ^ {2}}{d} \theta^ {m + 1} \frac {\Gamma (k + m + 1)}{\Gamma (k)} + c ^ {2} \theta^ {m} \frac {\Gamma (k + m)}{\Gamma (k)}\right) \\ = \sum_ {m = 0} ^ {\infty} \left(- \frac {c ^ {\prime 2} \theta}{2}\right) ^ {m} \frac {1}{m !} \left(\frac {k \| v \| ^ {2}}{d} \theta \frac {\Gamma (k + 1 + m)}{\Gamma (k + 1)} + c ^ {2} \frac {\Gamma (k + m)}{\Gamma (k)}\right) \\ = \frac {k \| v \| ^ {2}}{d} \theta \left(1 + \frac {c ^ {\prime 2} \theta}{2}\right) ^ {k + 1} + c ^ {2} \left(1 + \frac {c ^ {\prime 2} \theta}{2}\right) ^ {k}, \\ \end{array}
$$
where we use the fact that the hypergeometric function  ${}_{2}F_{1}(a,b,b;z) = \sum_{m=0}^{\infty}\frac{\Gamma(a+m)}{\Gamma(a)}\frac{z^{m}}{m!} = (1-z)^{-a}$  for some  $b$ , provided  $|z| < 1$ . By setting
$$
\begin{array}{l} v = \frac {\overline {{\| \beta_ {s} \|}} \beta_ {s}}{\| \beta_ {s} \|} - \frac {\overline {{\| \beta_ {s} ^ {\prime} \|}} \beta_ {s} ^ {\prime}}{\| \beta_ {s} ^ {\prime} \|} \\ c = - \left\langle \frac {\| \beta_ {s} \|}{\| \beta_ {s} \|} \beta_ {s} + \frac {\| \beta_ {s} ^ {\prime} \|}{\| \beta_ {s} ^ {\prime} \|}, \frac {\mu_ {s} - \mu_ {s} ^ {\prime}}{2} \right\rangle \\ + \sum_ {s ^ {\prime} \in [ M ]} p _ {s ^ {\prime}} \left(\langle \beta_ {s ^ {\prime}} - \beta_ {s ^ {\prime}} ^ {\prime}, \bar {\mu} _ {s ^ {\prime}} \rangle + \left\langle \beta_ {s ^ {\prime}} + \beta_ {s ^ {\prime}} ^ {\prime}, \frac {\mu_ {s ^ {\prime}} - \mu_ {s ^ {\prime}} ^ {\prime}}{2} \right\rangle\right) \\ \end{array}
$$
$$
c ^ {\prime} = \frac {1}{2 \sigma_ {X} ^ {2}} \| \mu_ {s} - \mu_ {s} ^ {\prime} \|
$$
$$
k = \frac {d}{2}, \text {a n d} \theta = 2 \sigma_ {X} ^ {2},
$$
$$
\begin{array}{l} \mathbf {E} \left[ \frac {\left(f _ {\beta , . \mu} . (\bar {X} _ {s} , s) - f _ {\beta^ {\prime} , \mu^ {\prime}} (\bar {X} _ {s} , s)\right) ^ {2}}{\cosh \left(\frac {1}{2 \sigma_ {X} ^ {2}} \left\| \bar {X} _ {s} - \bar {\mu} _ {s} \right\| \left\| \mu_ {s} - \mu_ {s} ^ {\prime} \right\|\right)} \right] \\ = \sigma_ {X} ^ {2} \left\| \frac {\overline {{\| \beta . \|}} \beta_ {s}}{\| \beta_ {s} \|} - \frac {\overline {{\| \beta_ {s} ^ {\prime} \|}} \beta_ {s} ^ {\prime}}{\| \beta_ {s} ^ {\prime} \|} \right\| ^ {2} \Big (1 + \frac {\| \mu_ {s} - \mu_ {s} ^ {\prime} \| ^ {2}}{4 \sigma_ {X} ^ {2}} \Big) ^ {1 + \frac {d}{2}} + \left(- \left\langle \frac {\overline {{\| \beta . \|}} \beta_ {s}}{\| \beta_ {s} \|} + \frac {\overline {{\| \beta^ {\prime} \|}} \beta_ {s} ^ {\prime}}{\| \beta_ {s} ^ {\prime} \|}, \frac {\mu_ {s} - \mu_ {s} ^ {\prime}}{2} \right\rangle \right. \\ \left. + \sum_ {s ^ {\prime} \in [ M ]} p _ {s ^ {\prime}} \left(\left\langle \beta_ {s ^ {\prime}} - \beta_ {s ^ {\prime}} ^ {\prime}, \bar {\mu} _ {s ^ {\prime}} \right\rangle + \left\langle \beta_ {s ^ {\prime}} + \beta_ {s ^ {\prime}} ^ {\prime}, \frac {\mu_ {s ^ {\prime}} - \mu_ {s ^ {\prime}} ^ {\prime}}{2} \right\rangle\right)\right) ^ {2} \left(1 + \frac {\left\| \mu_ {s} - \mu_ {s} ^ {\prime} \right\| ^ {2}}{4 \sigma_ {X} ^ {2}}\right) ^ {\frac {d}{2}}. \tag {38} \\ \end{array}
$$
Combining Eqs (37) and (38) yields the claim.
G.4 Proof of Theorem 8
Proof of Theorem 8. It is easy to check that  $d_{s} = 0$ , and for any  $v, v' \in \mathcal{V}$
$$
\begin{array}{l} \left\| \frac {\overline {{\| \beta_ {v , \cdot} \|}} \beta_ {v , s}}{\| \beta_ {v , s} \|} - \frac {\overline {{\| \beta_ {v ^ {\prime} , \cdot} \|}} \beta_ {v ^ {\prime} , s}}{\| \beta_ {v ^ {\prime} , s} \|} \right\| ^ {2} \\ = \left(\sum_ {s ^ {\prime} \in [ M ]} p _ {s ^ {\prime}} \| \beta_ {v, s} \|\right) ^ {2} \left\| \frac {\beta_ {v , s}}{\| \beta_ {v , s} \|} - \frac {\beta_ {v ^ {\prime} , s}}{\| \beta_ {v ^ {\prime} , s} \|} \right\| ^ {2} \\ = \left(\sum_ {s ^ {\prime} \in [ M ]} p _ {s ^ {\prime}} \| \beta_ {v, s} \|\right) ^ {2} \left(\sum_ {i \in [ d - 1 ]} \frac {\epsilon_ {s} ^ {2}}{d - 1} \left(v _ {s, i} - v _ {s, i} ^ {\prime}\right) ^ {2}\right) \\ = 4 \left(\sum_ {s ^ {\prime} \in [ M ]} p _ {s ^ {\prime}} B _ {s}\right) ^ {2} \frac {\epsilon_ {s} ^ {2}}{d - 1} d _ {H} \left(v _ {s}, v _ {s} ^ {\prime}\right). \\ \end{array}
$$
Since the density function of the Gaussian distribution is  $L^2$  integrable,  $\mathcal{E}(f;\theta) = \infty$  if  $f$  is not  $L^2$  integrable. Hence,  $\inf_f\mathcal{E}(f;\theta_v)\vee \mathcal{E}(f;\theta_{v'}) = \inf_{f\in \mathcal{L}^2}\mathcal{E}(f;\theta_v)\vee \mathcal{E}(f;\theta_{v'})$ , and we thus can apply Theorem 7. Then, we have
$$
\inf  _ {f} \mathcal {E} (f; \theta_ {v}) \vee \mathcal {E} (f; \theta_ {v ^ {\prime}}) \geq \sum_ {s \in [ M ]} p _ {s} \left(\sum_ {s ^ {\prime} \in [ M ]} p _ {s ^ {\prime}} B _ {s}\right) ^ {2} \frac {\sigma_ {X} ^ {2} \epsilon_ {s} ^ {2}}{d - 1} d _ {H} \left(v _ {s}, v _ {s} ^ {\prime}\right)
$$
Conditioned on  $n$ , the KL-divergence between  $\pi_{\theta_v|n}$  and  $\pi_{\theta_{v'}|n}$  is obtained as
$$
\sum_ {s \in [ M ]} n _ {s} \bigg (\frac {1}{2 \sigma_ {X} ^ {2}} \| \mu_ {v, s} - \mu_ {v ^ {\prime}, s} \| ^ {2} + \frac {\sigma_ {X} ^ {2}}{2 \sigma_ {\xi} ^ {2}} \| \beta_ {v, s} - \beta_ {v ^ {\prime}, s} \| ^ {2} + \frac {1}{2 \sigma_ {\xi} ^ {2}} \langle \mu_ {v, s}, \beta_ {v, s} - \beta_ {v ^ {\prime}, s} \rangle^ {2} \bigg).
$$
$$
D _ {\mathrm {K L}} \big (\pi_ {\theta_ {v} | n.}, \pi_ {\theta_ {v ^ {\prime}}, n.} \big) = \sum_ {s \in [ M ]} \frac {2 \sigma_ {X} ^ {2} B _ {s} ^ {2} n _ {s} \epsilon_ {s} ^ {2}}{\sigma_ {\xi} ^ {2} (d - 1)} d _ {H} (v _ {s}, v _ {s} ^ {\prime}).
$$