SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The following lemmas are needed for the proof of Theorem 4.6 and 4.7. Hereafter for notational convenience, we will use  $\widehat{h}, \widehat{m}$  for arbitrary elements in the corresponding sieve space listed in condition 4.3,  $\widehat{\nu}$  for an arbitrary element in the sieve space listed in condition 4.4, and  $\widehat{\theta}$  for an arbitrary element in  $\Theta$ .
Lemma B.1. Under condition 4.1, 4.3, 4.5, for  $(T,\delta ,Z)\in [0,\tau ]\times \{0,1\} \times [-1,1]^d$ , the following terms are bounded:
1.  $l(T,\delta ,Z;h_0,m_0,\theta_0)$  with true parameter  $(h_0,m_0,\theta_0)$
2.  $l(T,\delta ,Z;\widehat{h},\widehat{m},\widehat{\theta})$  with parameter estimates  $(\widehat{h},\widehat{m},\widehat{\theta})$  in any sieve space listed in condition 4.3.
Lemma B.2. Under condition 4.2, 4.4, 4.5, for  $(T,\delta ,Z)\in [0,\tau ]\times \{0,1\} \times [-1,1]^d$ , the following terms are bounded:
1.  $l(T,\delta ,Z;\nu_0,\theta_0)$  with true parameter  $(\nu_{0},\theta_{0})$
2.  $l(T, \delta, Z; \widehat{\nu}, \widehat{\theta})$  with parameter estimates  $(\widehat{\nu}, \widehat{\theta})$  in any sieve space listed in condition 4.4.
Lemma B.3. Under condition 4.1, 4.3, 4.5, let  $(\widehat{h},\widehat{m},\widehat{\theta})$ $(\widehat{h}_1,\widehat{m}_1,\widehat{\theta}_1)$ , and  $(\widehat{h}_2,\widehat{m}_2,\widehat{\theta}_2)$  be arbitrary three parameter triples inside the sieve space defined in condition 4.3, the following two inequalities hold.
$$
\begin{array}{l} \| l (T, \delta , Z; h _ {0}, m _ {0}, \theta_ {0}) - l (T, \delta , Z; \widehat {h}, \widehat {m}, \widehat {\theta}) \| _ {\infty} \lesssim | \theta_ {0} - \widehat {\theta} | + \| h _ {0} - \widehat {h} \| _ {\infty} + \| m _ {0} - \widehat {m} \| _ {\infty} \\ \| l (T, \delta , Z; \widehat {h} _ {1}, \widehat {m} _ {1}, \widehat {\theta} _ {1}) - l (T, \delta , Z; \widehat {h} _ {2}, \widehat {m} _ {2}, \widehat {\theta} _ {2}) \| _ {\infty} \lesssim | \widehat {\theta} _ {1} - \widehat {\theta} _ {2} | + \| \widehat {h} _ {1} - \widehat {h} _ {2} \| _ {\infty} + \| \widehat {m} _ {1} - \widehat {m} _ {2} \| _ {\infty}. \\ \end{array}
$$
Lemma B.4. Under condition 4.2, 4.4, 4.5, let  $(\widehat{\nu}, \widehat{\theta})$ ,  $(\widehat{\nu}_1, \widehat{\theta}_1)$ , and  $(\widehat{\nu}_2, \widehat{\theta}_2)$  be arbitrary three parameter tuples inside the sieve space defined in condition 4.4., the following inequalities hold.
$$
\begin{array}{l} \left\| l (T, \delta , Z; \nu_ {0}, \theta_ {0}) - l (T, \delta , Z; \widehat {\nu}, \widehat {\theta}) \right\| _ {\infty} \lesssim \left| \theta_ {0} - \widehat {\theta} \right| + \left\| \nu_ {0} - \widehat {\nu} \right\| _ {\infty} \\ \| l (T, \delta , Z; \widehat {\nu} _ {1}, \widehat {\theta} _ {1}) - l (T, \delta , Z; \widehat {\nu} _ {2}, \widehat {\theta} _ {2}) \| _ {\infty} \lesssim | \widehat {\theta} _ {1} - \widehat {\theta} _ {2} | + \| \widehat {\nu} _ {1} - \widehat {\nu} _ {2} \| _ {\infty}. \\ \end{array}
$$
Lemma B.5 (Approximating error of PF scheme). In the PF scheme, for any  $n$ , there exists an element in the corresponding sieve space  $\pi_n\phi_0 = (\pi_n h_0,\pi_n m_0,\pi_n\theta_0)$ , satisfying  $d_{PF}(\pi_n\phi_0,\phi_0) = O\left(n^{-\frac{\beta}{\beta + d}}\right)$ .
Lemma B.6 (Approximating error of FN scheme). In the FN scheme, for any  $n$ , there exists an element in the corresponding sieve space  $\pi_n\psi = (\pi_n\nu_0,\pi_n\theta_0)$  satisfying  $d_{FN}(\pi_n\psi_0,\psi_0) = O\left(n^{-\frac{\beta}{\beta + d + 1}}\right)$ .
Lemma B.7. Suppose  $\mathcal{F}$  is a class of functions satisfying that  $N(\varepsilon, \mathcal{F}, \| \cdot \|) < \infty$  for  $\forall \varepsilon > 0$ . We define  $\widetilde{N}(\varepsilon, \mathcal{F}, \| \cdot \|)$  to be the minimal number of  $\varepsilon$ -balls  $B(f, \varepsilon) = \{g : \| g - f \| < \varepsilon\}$  needed to cover  $\mathcal{F}$  with radius  $\varepsilon$  and further constrain that  $f \in \mathcal{F}$ . Then we have
$$
N (\varepsilon , \mathcal {F}, \| \cdot \|) \leq \widetilde {N} (\varepsilon , \mathcal {F}, \| \cdot \|) \leq N \left(\frac {\varepsilon}{2}, \mathcal {F}, \| \cdot \|\right).
$$
Lemma B.8. Suppose  $\mathcal{F}$  is a class of functions satisfying that  $N_{\mathbb{I}}(\varepsilon, \mathcal{F}, \| \cdot \|_{\infty}) < \infty$  for  $\forall \varepsilon > 0$ . We define  $\widetilde{N}_{\mathbb{I}}(\varepsilon, \mathcal{F}, \| \cdot \|_{\infty})$  to be the minimal number of brackets  $[l, u]$  needed to cover  $\mathcal{F}$  with  $\| l - u \|_{\infty} < \varepsilon$  and further constrain that  $f \in \mathcal{F}$ ,  $l = f - \frac{\varepsilon}{2}$  and  $u = f + \frac{\varepsilon}{2}$ . Then we have
$$
N _ {\llbracket} (\varepsilon , \mathcal {F}, \| \cdot \| _ {\infty}) \leq \widetilde {N} _ {\llbracket} (\varepsilon , \mathcal {F}, \| \cdot \| _ {\infty}) \leq N _ {\llbracket} (\frac {\varepsilon}{2}, \mathcal {F}, \| \cdot \| _ {\infty})
$$
Furthermore, we have  $\widetilde{N}_{\mathbb{I}}(\varepsilon, \mathcal{F}, \| \cdot \|_{\infty}) = \widetilde{N}(\frac{\varepsilon}{2}, \mathcal{F}, \| \cdot \|_{\infty})$ .
Lemma B.9 (Model capacity of PF scheme). Let  $\mathcal{F}_n = \{l(T, \delta, Z; \widehat{h}, \widehat{m}, \widehat{\theta}) : \widehat{h} \in \mathcal{H}_n, \widehat{m} \in \mathcal{M}_n, \widehat{\theta} \in \Theta\}$ . Under condition 4.5, with  $s_h = \frac{2\beta}{2\beta + 1}$  and  $s_m = \frac{2\beta}{2\beta + d}$ , there exist constants  $c_h$  and  $c_m > 0$  such that
$$
N _ {\mathbb {I}} (\varepsilon , \mathcal {F} _ {n}, \| \cdot \| _ {\infty}) \lesssim \frac {1}{\varepsilon} N (c _ {h} \varepsilon^ {1 / s _ {h}}, \mathcal {H} _ {n}, \| \cdot \| _ {2}) \times N (c _ {m} \varepsilon^ {1 / s _ {m}}, \mathcal {M} _ {n}, \| \cdot \| _ {2}).
$$
Lemma B.10 (Model capacity of FN scheme). Let  $\mathcal{G}_n = \{l(T, \delta, Z; \widehat{\nu}, \widehat{\theta}) : \widehat{\nu} \in \mathcal{V}_n, \widehat{\theta} \in \Theta\}$ . Under condition 4.5, with  $s_\nu = \frac{2\beta}{2\beta + d + 1}$ , there exists a constant  $c_\nu > 0$  such that
$$
N _ {\mathbb {I}} (\varepsilon , \mathcal {G} _ {n}, \| \cdot \| _ {\infty}) \lesssim \frac {1}{\varepsilon} N (c _ {\nu} \varepsilon^ {1 / s _ {\nu}}, \mathcal {V} _ {n}, \| \cdot \| _ {2}).
$$
B.3 Proofs of theorem 4.6 and 4.7
Proof of theorem 4.6. The proof is divided into four steps.
Step 1 We denote  $\phi_0 = (h_0, m_0, \theta_0)$  and  $\widehat{\phi} = (\widehat{h}, \widehat{m}, \widehat{\theta})$ , where  $\widehat{h} \in \mathcal{H}_n, \widehat{m} \in \mathcal{M}_n$  and  $\widehat{\theta} \in \Theta$ . For arbitrary small  $\varepsilon > 0$ , we have that
$$
\begin{array}{l} \inf  _ {d _ {\mathrm {P F}} (\widehat {\phi}, \phi_ {0}) \geq \varepsilon} \mathbb {E} \left[ l (T, \delta , Z; h _ {0}, m _ {0}, \theta_ {0}) - l (T, \delta , Z; \widehat {h}, \widehat {m}, \widehat {\theta}) \right] \\ = \inf  _ {d _ {\mathrm {P F}} (\widehat {\phi}, \phi_ {0}) \geq \varepsilon} \mathbb {E} _ {Z} \left[ \mathbb {E} _ {T, \delta | Z} \left[ \log p (T, \delta \mid Z; h _ {0}, m _ {0}, \theta_ {0}) - \log p (T, \delta \mid Z; \widehat {h}, \widehat {m}, \widehat {\theta}) \right] \right] \\ = \inf  _ {d _ {\mathbb {P F}} (\widehat {\phi}, \phi_ {0}) \geq \varepsilon} \mathbb {E} _ {Z} \left[ \mathrm {D} _ {\mathrm {K L}} \left(\mathbb {P} _ {\widehat {\phi}, Z} \| \mathbb {P} _ {\phi_ {0}, Z}\right) \right] \\ \end{array}
$$
Using the fact that  $\mathrm{D}_{\mathrm{KL}}\left(\mathbb{P}_{\widehat{\phi},Z}\parallel \mathbb{P}_{\phi_0,Z}\right)\geq 2H^2 (\mathbb{P}_{\widehat{\phi},Z}\parallel \mathbb{P}_{\phi_0,Z})$  . Thus, we further obtain that
$$
\begin{array}{l} \inf  _ {d _ {\mathrm {P F}} (\widehat {\phi}, \phi_ {0}) \geq \varepsilon} \mathbb {E} \left[ l (T, \delta , Z; h _ {0}, m _ {0}, \theta_ {0}) - l (T, \delta , Z; \widehat {h}, \widehat {m}, \widehat {\theta}) \right] \\ \geq \inf  _ {d _ {\mathbb {P F}} (\widehat {\phi}, \phi_ {0}) \geq \varepsilon} 2 \mathbb {E} _ {Z} \left[ H ^ {2} \left(\mathbb {P} _ {\widehat {\phi}, Z} \| \mathbb {P} _ {\phi_ {0}, Z}\right) \right] \\ = 2\inf_{d_{\mathsf{PF}}(\widehat{\phi},\phi_{0})\geq \varepsilon}d_{\mathsf{PF}}^{2}\left(\widehat{\phi},\phi_{0}\right) \\ \geq 2 \varepsilon^ {2}. \\ \end{array}
$$
Step 2 Consider the following derivation.
$$
\begin{array}{l} \sup  _ {d _ {\mathrm {P F}} (\widehat {\phi}, \phi_ {0}) \leq \varepsilon} \operatorname {V a r} \left[ l (T, \delta , Z; h _ {0}, m _ {0}, \theta_ {0}) - l (T, \delta , Z; \widehat {h}, \widehat {m}, \widehat {\theta}) \right] \\ \leq \sup  _ {d _ {\mathrm {P F}} (\widehat {\phi}, \phi_ {0}) \leq \varepsilon} \mathbb {E} \left[ \left(l (T, \delta , Z; h _ {0}, m _ {0}, \theta_ {0}) - l (T, \delta , Z; \widehat {h}, \widehat {m}, \widehat {\theta})\right) ^ {2} \right] \\ = \sup  _ {d _ {\mathrm {P F}} (\widehat {\phi}, \phi_ {0}) \leq \varepsilon} \mathbb {E} _ {Z} \mathbb {E} _ {T, \delta | Z} \left[ \log p (T, \delta , Z; h _ {0}, m _ {0}, \theta_ {0}) - \log p (T, \delta , Z; \widehat {h}, \widehat {m}, \widehat {\theta}) \right] ^ {2} \\ = 4 \sup  _ {d _ {\mathbb {P F}} (\widehat {\phi}, \phi_ {0}) \leq \varepsilon} \mathbb {E} _ {Z} \left[ \int \left(p (T, \delta , Z; h _ {0}, m _ {0}, \theta_ {0}) \left(\log \sqrt {\frac {p (T , \delta , Z ; h _ {0} , m _ {0} , \theta_ {0})}{p (T , \delta , Z ; \widehat {h} , \widehat {m} , \widehat {\theta})}}\right) ^ {2}\right) \mu (d T \times d \delta) \right] \\ \end{array}
$$
By Taylor's expansion on  $\log x$ , there exists  $\xi (T,\delta ,Z)$  between  $p^{\frac{1}{2}}(T,\delta ,Z;h_0,m_0,\theta_0)$  and  $p^{\frac{1}{2}}(T,\delta ,Z;\widehat{h},\widehat{m},\widehat{\theta})$  pointwisely such that
$$
\begin{array}{l} p (T, \delta , Z; h _ {0}, m _ {0}, \theta_ {0}) \left(\log \sqrt {\frac {p (T , \delta , Z ; h _ {0} , m _ {0} , \theta_ {0})}{p (T , \delta , Z ; \widehat {h} , \widehat {m} , \widehat {\theta})}}\right) ^ {2} \\ = p (T, \delta , Z; h _ {0}, m _ {0}, \theta_ {0}) \left(\log \sqrt {p (T , \delta , Z ; h _ {0} , m _ {0} , \theta_ {0})} - \log \sqrt {p (T , \delta , Z ; \widehat {h} , \widehat {m} , \widehat {\theta})}\right) ^ {2} \\ = \frac {p (T , \delta , Z ; h _ {0} , m _ {0} , \theta_ {0})}{\xi (T , \delta , Z) ^ {2}} \left(\sqrt {p (T , \delta , Z ; h _ {0} , m _ {0} , \theta_ {0})} - \sqrt {p (T , \delta , Z ; \widehat {h} , \widehat {m} , \widehat {\theta})}\right) ^ {2} \\ \end{array}
$$
$$
\frac {p (T , \delta , Z ; h _ {0} , m _ {0} , \theta_ {0})}{p (T , \delta , Z ; \widehat {h} , \widehat {m} , \widehat {\theta})} = e ^ {l (T, \delta , Z; h _ {0}, m _ {0}, \theta_ {0}) - l (T, \delta , Z; \widehat {h}, \widehat {m}, \widehat {\theta})}
$$
by lemma B.1,  $l(T,\delta,Z;h_0,m_0,\theta_0)$  and  $l(T,\delta,Z;\widehat{h},\widehat{m},\widehat{\theta})$  are bounded among  $[0,\tau] \times \{0,1\} \times [-1,1]^d$  uniformly on all  $\widehat{\phi} = (\widehat{h},\widehat{m},\widehat{\theta})$ . Thus, there exist constants  $C_1$  and  $C_2$  such that  $0 < C_1 \leq p(T,\delta,Z;h_0,m_0,\theta_0)/p(T,\delta,Z;\widehat{h},\widehat{m},\widehat{\theta}) \leq C_2$ . This leads to the fact that
$p(T,\delta ,Z;h_0,m_0,\theta_0)\frac{1}{\xi(T,\delta,Z)^2}$  is bounded. We further obtained that
$$
\begin{array}{l} p (T, \delta , Z; h _ {0}, m _ {0}, \theta_ {0}) \left(\log \sqrt {p (T , \delta , Z ; h _ {0} , m _ {0} , \theta_ {0})} - \log \sqrt {p (T , \delta , Z ; \widehat {h} , \widehat {m} , \widehat {\theta})}\right) ^ {2} \\ \lesssim \left| \sqrt {p (T , \delta , Z ; h _ {0} , m _ {0} , \theta_ {0})} - \sqrt {p (T , \delta , Z ; \widehat {h} , \widehat {m} , \widehat {\theta})} \right| ^ {2}. \\ \end{array}
$$
$$
\begin{array}{l} \sup_{d_{\mathsf{PF}}[\widehat{\phi},\phi_{0}]\leq \varepsilon}\mathbf{Var}(l(T,\delta ,Z;h_{0},m_{0},\theta_{0}) - l(T,\delta ,Z;\widehat{h},\widehat{m},\widehat{\theta})) \\ \lesssim \sup  _ {d _ {\mathrm {P F}} (\widehat {\phi}, \phi_ {0}) \leq \varepsilon} \mathbb {E} _ {Z} \left[ \int \left| \sqrt {p (T , \delta , Z ; h _ {0} , m _ {0} , \theta_ {0})} - \sqrt {p (T , \delta , Z ; \widehat {h} , \widehat {m} , \widehat {\theta})} \right| ^ {2} \mu (d T \times d \delta) \right] \\ = \sup  _ {d _ {\mathsf {P F}} (\widehat {\phi}, \phi_ {0}) \leq \varepsilon} d _ {\mathsf {P F}} ^ {2} (\widehat {\phi}, \phi_ {0}) \\ \leq \varepsilon^ {2}. \\ \end{array}
$$
Step 3 We define that  $\widetilde{\mathcal{F}}_n = \{l(T,\delta,Z;\widehat{h},\widehat{m},\widehat{\theta}) - l(T,\delta,Z;\pi_n h_0,\pi_n m_0,\pi_n\theta_0):\widehat{h}\in \mathcal{H}_n,\widehat{m}\in \mathcal{M}_n,\widehat{\theta}\in \Theta \}$ . Here  $(\pi_{n}h_{0},\pi_{n}m_{0},\pi_{n}\theta_{0})$  have been defined in lemma B.5. Obviously, we have that  $\log N_{\mathbb{I}}(\varepsilon ,\widetilde{\mathcal{F}}_n,\| \cdot \|_\infty) = \log N_{\mathbb{I}}(\varepsilon ,\mathcal{F}_n,\| \cdot \|_\infty)$ , where  $\mathcal{F}$  is defined in lemma B.9. By lemma B.9, we further have that
$$
\log N _ {\mathbb {I}} \left(\varepsilon , \mathcal {F} _ {n}, \| \cdot \| _ {\infty}\right) \lesssim \log \frac {1}{\varepsilon} + \log N \left(c _ {h} \varepsilon^ {1 / s _ {h}}, \mathcal {H} _ {n}, \| \cdot \| _ {2}\right) + \log N \left(c _ {m} \varepsilon^ {1 / s _ {m}}, \mathcal {M} _ {n}, \| \cdot \| _ {2}\right).
$$
According to [2, Theorem 7], under condition 4.3, we have that the VC-dimension of  $\mathcal{H}_n$  and  $\mathcal{M}_n$  satisfy that  $\mathrm{VC}(\mathcal{H}_n) \lesssim n^{\frac{1}{\beta + d}} \log^3 n$  and  $\mathrm{VC}(\mathcal{M}_n) \lesssim n^{\frac{d}{\beta + d}} \log^3 n$ . Thus, we obtain that
$$
\log N \left(c _ {h} \varepsilon^ {1 / s _ {h}}, \mathcal {H} _ {n}, \| \cdot \| _ {2}\right) \lesssim \frac {\operatorname {V C} \left(\mathcal {H} _ {n}\right)}{s _ {h}} \log \frac {1}{\varepsilon} \lesssim n ^ {\frac {1}{\beta + d}} \log^ {3} n \log \frac {1}{\varepsilon},
$$
$$
\log N \left(c _ {m} \varepsilon^ {1 / s _ {m}}, \mathcal {M} _ {n}, \| \cdot \| _ {2}\right) \lesssim \frac {\operatorname {V C} \left(\mathcal {M} _ {n}\right)}{s _ {\nu}} \log \frac {1}{\varepsilon} \lesssim n ^ {\frac {d}{\beta + d}} \log^ {3} n \log \frac {1}{\varepsilon}.
$$
Thus, we obtain that  $\log N_{[]}(\varepsilon, \widetilde{\mathcal{F}}_n, \| \cdot \|_\infty) \lesssim n^{\frac{d}{\beta + d}} \log^3 n \log \frac{1}{\varepsilon}$ .
Step 4 By the Cauchy-Schwartz inequality, we have that
$$
\begin{array}{l} \sqrt {\mathbb {E} \left[ l (T , \delta , Z ; \pi_ {n} h _ {0} , \pi_ {n} m _ {0} , \pi_ {n} \theta_ {0}) - l (T , \delta , Z ; h _ {0} , m _ {0} , \theta_ {0}) \right]} \\ \left. \right. \leq \left[ \mathbb {E} \left(l (T, \delta , Z; \pi_ {n} h _ {0}, \pi_ {n} m _ {0}, \pi_ {n} \theta_ {0}) - l (T, \delta , Z; h _ {0}, m _ {0}, \theta_ {0})\right) ^ {2} \right] ^ {\frac {1}{4}}. \\ \end{array}
$$
Similar to the second part and by lemma B.5, we further have that
$$
\sqrt {\mathbb {E} \left[ l (T , \delta , Z ; \pi_ {n} h _ {0} , \pi_ {n} m _ {0} , \pi_ {n} \theta_ {0}) - l (T , \delta , Z ; h _ {0} , m _ {0} , \theta_ {0}) \right]} \lesssim \sqrt {d _ {\mathrm {P F}} (\pi_ {n} \phi_ {0} , \phi_ {0})} \lesssim n ^ {- \frac {\beta}{2 \beta + 2 d}}.
$$
Now let
$$
\tau = \frac {\beta}{2 \beta + 2 d} - 2 \frac {\log \log n}{\log n}
$$
By Step 1,2,3 and [60, Theorem 1], we have
$$
\begin{array}{l} d _ {\mathsf {P F}} \left(\widehat {\phi} _ {n}, \phi_ {0}\right) = \max \left(n ^ {- \tau}, d _ {\mathsf {P F}} \left(\pi_ {n} \phi_ {0}, \phi_ {0}\right), \right. \\ \left. \sqrt {\mathbb {E} \left[ l (T , \delta , Z ; \pi_ {n} h _ {0} , \pi_ {n} m _ {0} , \pi_ {n} \theta_ {0}) - l (T , \delta , Z ; h _ {0} , m _ {0} , \theta_ {0}) \right]}\right) \\ \end{array}
$$
By lemma B.5,  $d_{\mathsf{PF}}(\pi_n\phi_0,\phi_0) = O(n^{-\frac{\beta}{\beta + d}})$ .
By Step 4,  $\sqrt{\mathbb{E}\left[l(T,\delta,Z;\pi_nh_0,\pi_nm_0,\pi_n\theta_0) - l(T,\delta,Z;h_0,m_0,\theta_0)\right]} = O\left(n^{-\frac{\beta}{2\beta + 2d}}\right)$ . Thus, we have  $d_{\mathsf{PF}}\left(\widehat{\phi}_n,\phi_0\right) = O(n^{-\frac{\beta}{2\beta + 2d}}\log^2 n) = \widetilde{O}(n^{-\frac{\beta}{2\beta + 2d}})$ .
Proof of theorem 4.7. The proof is divided into four steps.
Step 1 We denote  $\psi_0 = (\nu_0,\theta_0)$  and  $\widehat{\psi} = (\widehat{\nu},\widehat{\theta})$ , where  $\widehat{\nu}\in \mathcal{V}_n$  and  $\widehat{\theta}\in \Theta$ . For arbitrary  $0 < \varepsilon \leq 1$ , we have that
$$
\begin{array}{l} \inf  _ {d _ {\mathrm {F N}} (\widehat {\psi}, \psi_ {0}) \geq \varepsilon} \mathbb {E} \left[ l (T, \delta , Z; \nu_ {0}, \theta_ {0}) - l (T, \delta , Z; \widehat {\nu}, \widehat {\theta}) \right] \\ = \inf  _ {d _ {\mathbb {F N}} (\widehat {\psi}, \psi_ {0}) \geq \varepsilon} \mathbb {E} _ {Z} \left[ \mathbb {E} _ {T, \delta | Z} \left[ \log p (T, \delta \mid Z; \nu_ {0}, \theta_ {0}) - \log p (T, \delta \mid Z; \widehat {\nu}, \widehat {\theta}) \right] \right] \\ = \inf  _ {d _ {\mathbb {F N}} (\widehat {\psi}, \psi_ {0}) \geq \varepsilon} \mathbb {E} _ {Z} \left[ \mathrm {D} _ {\mathrm {K L}} \left(\mathbb {P} _ {\widehat {\psi}, Z} \| \| \mathbb {P} _ {\psi_ {0}, Z}\right) \right] \\ \end{array}
$$
Using the fact that  $KL(\mathbb{P}_{\widehat{\psi},Z}\parallel \mathbb{P}_{\psi_0,Z})\geq 2H^2 (\mathbb{P}_{\widehat{\psi},Z}\parallel \mathbb{P}_{\psi_0,Z})$  . Thus, we further obtain that
$$
\begin{array}{l} \inf  _ {d _ {\mathbb {F N}} (\widehat {\psi}, \psi_ {0}) \geq \varepsilon} \mathbb {E} \left[ l (T, \delta , Z; \nu_ {0}, \theta_ {0}) - l (T, \delta , Z; \widehat {\nu}, \widehat {\theta}) \right] \\ \geq \inf  _ {d _ {\mathbb {F N}} (\widehat {\psi}, \psi_ {0}) \geq \varepsilon} 2 \mathbb {E} _ {Z} \left[ H ^ {2} \left(\mathbb {P} _ {\widehat {\psi}, Z} \| \mathbb {P} _ {\psi_ {0}, Z}\right) \right] \\ = 2\inf_{d_{\mathsf{FN}}(\widehat{\psi},\psi_{0})\geq \varepsilon}d_{\mathsf{FN}}^{2}\left(\widehat{\psi},\psi_{0}\right) \\ \geq 2 \varepsilon^ {2}. \\ \end{array}
$$
Step 2 We consider the following derivation.
$$
\begin{array}{l} \sup  _ {d _ {\mathsf {F N}} (\widehat {\psi}, \psi_ {0}) \leq \varepsilon} \operatorname {V a r} \left[ l (T, \delta , Z; \nu_ {0}, \theta_ {0}) - l (T, \delta , Z; \widehat {\nu}, \widehat {\theta}) \right] \\ \leq \sup  _ {d _ {\mathsf {F N}} (\widehat {\psi}, \psi_ {0}) \leq \varepsilon} \mathbb {E} \left[ \left(l (T, \delta , Z; \nu_ {0}, \theta_ {0}) - l (T, \delta , Z; \widehat {\nu}, \widehat {\theta})\right) ^ {2} \right] \\ = \sup  _ {d _ {\mathbb {F N}} (\widehat {\psi}, \psi_ {0}) \leq \varepsilon} \mathbb {E} _ {Z} \left[ \mathbb {E} _ {T, \delta | Z} \left[ \left(\log p (T, \delta , Z; \nu_ {0}, \theta_ {0}) - \log p (T, \delta , Z; \widehat {\nu}, \widehat {\theta})\right) ^ {2} \right] \right] \\ = 4 \sup  _ {d _ {\mathbb {F N}} (\widehat {\psi}, \psi_ {0}) \leq \varepsilon} \mathbb {E} _ {Z} \left[ \int \left(p (T, \delta , Z; \nu_ {0}, \theta_ {0}) (\log \sqrt {\frac {p (T , \delta , Z ; \nu_ {0} , \theta_ {0})}{p (T , \delta , Z ; \widehat {\nu} , \widehat {\theta})}}) ^ {2}\right) \mu (d T \times d \delta) \right] \\ \end{array}
$$
By Taylor's expansion on  $\log x$ , there exists  $\eta(T, \delta, Z)$  between  $\sqrt{p(T, \delta, Z; \nu_0, \theta_0)}$  and  $\sqrt{p(T, \delta, Z; \widehat{\nu}, \widehat{\theta})}$  pointwisely such that
$$
\begin{array}{l} p (T, \delta , Z; \nu_ {0}, \theta_ {0}) (\log \sqrt {\frac {p (T , \delta , Z ; \nu_ {0} , \theta_ {0})}{p (T , \delta , Z ; \widehat {\nu} , \widehat {\theta})}}) ^ {2} \\ = p (T, \delta , Z; \nu_ {0}, \theta_ {0}) \left(\log \sqrt {p (T , \delta , Z ; \nu_ {0} , \theta_ {0})} - \log \sqrt {p (T , \delta , Z ; \widehat {\nu} , \widehat {\theta})}\right) ^ {2} \\ = \frac {p (T , \delta , Z ; \nu_ {0} , \theta_ {0})}{\eta (T , \delta , Z) ^ {2}} \left(\sqrt {p (T , \delta , Z ; \nu_ {0} , \theta_ {0})} - \sqrt {p (T , \delta , Z ; \widehat {\nu} , \widehat {\theta})}\right) ^ {2} \\ \end{array}
$$
Since  $p(T, \delta, Z; \nu_0, \theta_0) / p(T, \delta, Z; \widehat{\nu}, \widehat{\theta}) = e^{l(T, \delta, Z; \nu_0, \theta_0) - l(T, \delta, Z; \widehat{\nu}, \widehat{\theta})}$ , by lemma B.2,  $l(T, \delta, Z; \nu_0, \theta_0)$  and  $l(T, \delta, Z; \widehat{\nu}, \widehat{\theta})$  are bounded on  $[0, \tau] \times \{0, 1\} \times [-1, 1]^d$  uniformly for all  $\widehat{\psi} = (\widehat{\nu}, \widehat{\theta})$ . Thus there exist constants  $C_3$  and  $C_4$  such that  $0 < C_3 \leq p(T, \delta, Z; \nu_0, \theta_0) / p(T, \delta, Z; \widehat{\nu}, \widehat{\theta}) \leq C_4$ . This leads to the fact that  $p(T, \delta, Z; \nu_0, \theta_0) \frac{1}{\eta(T, \delta, Z)^2}$  is bounded. We further have that
$$
\begin{array}{l} p (T, \delta , Z; \nu_ {0}, \theta_ {0}) \left(\log \sqrt {p (T , \delta , Z ; \nu_ {0} , \theta_ {0})} - \log \sqrt {p (T , \delta , Z ; \widehat {\nu} , \widehat {\theta})}\right) ^ {2} \\ \lesssim \left| \sqrt {p (T , \delta , Z ; \nu_ {0} , \theta_ {0})} - \sqrt {p (T , \delta , Z ; \widehat {\nu} , \widehat {\theta})} \right| ^ {2}. \\ \end{array}
$$
$$
\begin{array}{l} \sup_{d_{FN}(\widehat{\psi},\psi_{0})\leq \varepsilon}\operatorname {Var}\left[l(T,\delta ,Z;\nu_{0},\theta_{0}) - l(T,\delta ,Z;\widehat{\nu},\widehat{\theta})\right] \\ \lesssim \sup  _ {d _ {\mathrm {F N}} (\widehat {\psi}, \psi_ {0}) \leq \varepsilon} \mathbb {E} _ {Z} \left[ \int \left| \sqrt {p (T , \delta , Z ; \nu_ {0} , \theta_ {0})} - \sqrt {p (T , \delta , Z ; \widehat {\nu} , \widehat {\theta})} \right| ^ {2} \mu (d T \times d \delta) \right] \\ = \sup  _ {d _ {\mathsf {F N}} (\widehat {\psi}, \psi_ {0}) \leq \varepsilon} d _ {\mathsf {F N}} ^ {2} \left(\widehat {\psi}, \psi_ {0}\right) \\ \leq \varepsilon^ {2}. \\ \end{array}
$$
Step 3 We define that  $\widetilde{\mathcal{G}}_n = \{l(T,\delta,Z;\widehat{\nu},\widehat{\theta}) - l(T,\delta,Z;\pi_n\nu_0,\pi_n\theta_0):\widehat{\nu}\in \mathcal{V}_n,\theta \in \Theta \}$ . Here  $(\pi_{n}\nu_{0},\pi_{n}\theta_{0})$  have been defined in lemma B.6. Obviously, we have that  $\log N_{\mathbb{I}}(\varepsilon ,\widetilde{\mathcal{G}}_n,\| \cdot \|_\infty) = \log N_{\mathbb{I}}(\varepsilon ,\mathcal{G}_n,\| \cdot \|_\infty)$ , where  $\mathcal{G}$  is defined in lemma B.10. By lemma B.10, we further obtain that
$$
\log N _ {\mathbb {I}} \left(\varepsilon , \mathcal {G} _ {n}, \| \cdot \| _ {\infty}\right) \lesssim \log \frac {1}{\varepsilon} + \log N \left(c _ {\nu} \varepsilon^ {1 / s _ {\nu}}, \mathcal {V} _ {n}, \| \cdot \| _ {2}\right).
$$
According to [2, Theorem 7], under condition 4.4, we have that the VC-dimension of  $\nu_{n}$  satisfies that  $\mathrm{VC}(\mathcal{V}_n)\lesssim n^{\frac{d + 1}{\beta + d + 1}}\log^3 n$ . Thus, we obtain that
$$
\log N (c _ {h} \varepsilon^ {1 / s _ {\nu}}, \mathcal {V} _ {n}, \| \cdot \| _ {2}) \lesssim \frac {\operatorname {V C} (\mathcal {V} _ {n})}{s _ {\nu}} \log \frac {1}{\varepsilon} \lesssim n ^ {\frac {d + 1}{\beta + d + 1}} \log^ {3} n \log \frac {1}{\varepsilon}.
$$
Furthermore, we get that  $\log N_{\mathbb{I}}(\varepsilon ,\widetilde{\mathcal{G}}_n,\| \cdot \|_\infty)\lesssim n^{\frac{d + 1}{\beta + d + 1}}\log^3 n\log \frac{1}{\varepsilon}$
$$
\sqrt {\mathbb {E} [ l (T , \delta , Z ; \pi_ {n} \nu_ {0} , \pi_ {n} \theta_ {0}) - l (T , \delta , Z ; \nu_ {0} , \theta_ {0}) ]} \leq \left[ \mathbb {E} \left(l (T, \delta , Z; \pi_ {n} \nu_ {0}, \pi_ {n} \theta_ {0}) - l (T, \delta , Z; \nu_ {0}, \theta_ {0})\right) ^ {2} \right] ^ {\frac {1}{4}}.
$$
Similar to the second part and by lemma B.6, we further obtain that
$$
\sqrt {\mathbb {E} [ l (T , \delta , Z ; \pi_ {n} \nu_ {0} , \pi_ {n} \theta_ {0}) - l (T , \delta , Z ; \nu_ {0} , \theta_ {0}) ]} \lesssim \sqrt {d _ {\mathrm {F N}} (\pi_ {n} \psi_ {0} , \psi_ {0})} \lesssim n ^ {- \frac {\beta}{2 \beta + 2 d + 2}}
$$
$$
\tau = \frac {\beta}{2 \beta + 2 d + 2} - 2 \frac {\log \log n}{\log n}.
$$
By step 1,2,3 and Step 1,2,3 and [60, Theorem 1],
$$
\begin{array}{l} d _ {\mathsf {F N}} \left(\widehat {\psi} _ {n}, \psi_ {0}\right) = \max  \left(n ^ {- \tau}, d _ {\mathsf {F N}} \left(\pi_ {n} \psi_ {0}, \psi_ {0}\right), \right. \\ \left. \sqrt {\mathbb {E} \left[ l (T , \delta , Z ; \pi_ {n} \nu_ {0} , \pi_ {n} \theta_ {0}) - l (T , \delta , Z ; \nu_ {0} , \theta_ {0}) \right]}\right) \\ \end{array}
$$
By lemmaB.6,  $d_{\mathsf{FN}}(\pi_n\psi_0,\psi_0) = O(n^{-\frac{\beta}{\beta + d + 1}})$