SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Following De Vito et al. [16] and Große and Schneider [24] we fix a family  $\mathcal{T} = (\mathcal{U}_l, \phi_l, \chi_l)_{l=1}^L$ , where  $L \in \mathbb{Z}_{>0}$ , the local coordinates  $\phi_l : \mathcal{U}_l \subset \mathcal{M} \to \mathcal{V}_l = \phi_l(\mathcal{U}_l) \subset \mathbb{R}^d$  are smooth diffeomorphisms, and the functions  $\chi_l$  form a partition of unity subordinate to  $\{\mathcal{U}_l\}_{l=1}^L$ , that is  $\chi_l \in \mathcal{C}^\infty(\mathcal{M})$ ,  $\operatorname{supp}(\chi_l) \subset \mathcal{U}_l$ ,  $0 \leq \chi_l \leq 1$  and  $\sum_l \chi_l = 1$ —here, we can choose  $L$  finite by compactness of  $\mathcal{M}$ . For convenience and without loss of generality, we assume that  $\mathcal{V}_l = (0,1)^d$ . With this, we can start defining function spaces on  $\mathcal{M}$ .
A.2 Hölder Spaces  $\mathcal{CH}^{\gamma}$  and the spaces of continuous functions  $\mathcal{C}$  and smooth functions  $\mathcal{C}^\infty$
Consider an arbitrary domain  $\mathcal{X} \subseteq \mathbb{R}^d$  or  $\mathcal{X} \subseteq \mathcal{M}$ . We denote the class of infinitely differentiable functions on  $\mathcal{X}$  by  $\mathcal{C}^\infty(\mathcal{X})$ . Let  $\bar{\mathcal{C}}^k(\mathcal{X})$  denote the Banach space of  $k \in \mathbb{Z}_{>0}$  times continuously differentiable functions on  $\mathcal{X}$  with finite norm
$$
\| f \| _ {\mathcal {C} ^ {k} (\mathcal {X})} = \sup  _ {x \in \mathcal {X}} | \nabla^ {k} f (x) | \tag {18}
$$
where  $\nabla^k$  is the  $k$ th covariant derivative, as in Hebey [25], Section 2.1 and Aubin [3], Definition 2.2. We also write  $\mathcal{C}(\mathcal{X}) = \mathcal{C}^0 (\mathcal{X})$  for the space of continuous functions on  $\mathcal{X}$ .
The Euclidean Hölder spaces  $\mathcal{CH}^{\gamma}(\mathbb{R}^{d})$ , where  $\gamma = k + \alpha$  with  $k \in \mathbb{Z}_{\geq 0}$  and  $0 < \alpha \leq 1$ , are defined by
$$
\mathcal {C H} ^ {\gamma} (\mathbb {R} ^ {d}) = \left\{f \in \mathcal {C} ^ {k} (\mathbb {R} ^ {d}): \| f \| _ {\mathcal {C H} ^ {\gamma} (\mathbb {R} ^ {d})} <   \infty \right\} \tag {19}
$$
$$
\| f \| _ {\mathcal {C H} ^ {\gamma} \left(\mathbb {R} ^ {d}\right)} = \| f \| _ {\mathcal {C} ^ {k} \left(\mathbb {R} ^ {d}\right)} + \sup  _ {\boldsymbol {x}, \boldsymbol {y} \in \mathbb {R} ^ {d}, \boldsymbol {x} \neq \boldsymbol {y}} \frac {| f (\boldsymbol {x}) - f (\boldsymbol {y}) |}{\| \boldsymbol {x} - \boldsymbol {y} \| _ {\mathbb {R} ^ {d}} ^ {\alpha}}. \tag {20}
$$
More information on these definitions may be found, for instance, in Giné and Nickl [21] and Triebel [51]. We now turn to the manifold versions of the Hölder spaces.
${}^{3}$  Hölder spaces are often also denoted by  ${\mathcal{C}}^{\gamma }$  as well,with  $\gamma  \in  {\mathbb{R}}_{ > 0}$  . Since,using this formulation,they do not coincide with  ${\mathcal{C}}^{k}\left( \mathcal{X}\right)$  when  $k = \gamma  \in  {\mathbb{Z}}_{ \geq  0}$  ,we use the notation  ${\mathcal{CH}}^{\gamma }$  to avoid confusion.
Definition 11 (Hölder spaces). For all  $\gamma > 0$  we define the Hölder space  $\mathcal{CH}^{\gamma}(\mathcal{M})$  on the manifold  $\mathcal{M}$  to be the space of all  $f: \mathcal{M} \to \mathbb{R}$  satisfying
$$
\| f \| _ {\mathcal {C H} ^ {\gamma} (\mathcal {M})} = \sum_ {l = 1} ^ {L} \left\| \left(\chi_ {l} f\right) \circ \phi_ {l} ^ {- 1} \right\| _ {\mathcal {C H} ^ {\gamma} \left(\mathbb {R} ^ {d}\right)} <   \infty . \tag {21}
$$
Since the charts  $\phi_l$  are smooth, Definition 11 can be easily seen to be independent of the chosen atlas, with equivalence of norms.
A.3 Sobolev and Besov Spaces
We now introduce the manifold versions of the Sobolev and Besov spaces, whose definitions in the standard Euclidean case may be found, for instance, in Triebel [51]. For Sobolev spaces we use the Bessel-potential-based definition, following De Vito et al. [16].
Definition 12 (Sobolev spaces). For any  $s > 0$  we define the Sobolev space  $H^{s}(\mathcal{M})$  on the manifold  $\mathcal{M}$  as the Hilbert space of functions  $f\in L^{2}(\mathcal{M})$  such that  $\| f\|_{H^s (\mathcal{M})}^2 = \langle f,f\rangle_{H^s (\mathcal{M})} < \infty$  where
$$
\langle f, g \rangle_ {H ^ {s} (\mathcal {M})} = \sum_ {j = 0} ^ {\infty} (1 + \lambda_ {j}) ^ {s} \langle f, f _ {j} \rangle_ {L ^ {2} (\mathcal {M})} \langle g, f _ {j} \rangle_ {L ^ {2} (\mathcal {M})}. \tag {22}
$$
Remark 13. It is easy to see that substituting  $(1 + \lambda_j)^s$  in (22) with  $\beta (\alpha +\lambda_j)^s$  or with  $(\alpha +\beta \lambda_j^s)$  for any  $\alpha ,\beta >0$  results in the same set of functions and an equivalent norm. The former follows from Borovitskiy et al. [11], eq. (109). The latter follows from the Binomial Theorem.
For Besov spaces we follow Coulhon et al. [14] and Castillo et al. [12] and define them in terms of approximations by low-frequency functions. We fix a function  $\Phi \in \mathcal{C}^{\infty}(\mathbb{R}_{\geq 0},\mathbb{R}_{\geq 0})$  such that  $K = \mathrm{supp}(\Phi)\subset [0,2]$  and  $\Phi (x) = 1$  for  $x\in [0,1]$ . We also define the functions  $\Phi_j(x) = \Phi \bigl (2^{-j}x\bigr)$ .
Coulhon et al. [14], Corollary 3.6 shows that the operators  $\Phi_j(\sqrt{\Delta})$  defined by
$$
\Phi_ {j} (\sqrt {\Delta}) f = \sum_ {j \geq 0} \Phi_ {j} (\sqrt {\lambda_ {j}}) \langle f _ {j}, f \rangle f _ {j} \tag {23}
$$
are bounded in the space  $L^p(\mathcal{M})$  for all  $1 \leq p \leq \infty$ . Moreover, the same result also shows that we can express any  $f \in L^p(\mathcal{M})$  as  $f = \lim_{j \to \infty} \Phi_j(\sqrt{\Delta}) f$  in  $L^p(\mathcal{M})$ .  $\Phi_j(\sqrt{\Delta}) f$  can intuitively be considered as a version of  $f$  filtered by a low-pass filter. The next definition introduces the Besov spaces  $B_{p,q}^s(\mathcal{M})$ , which are formulated in terms of quality-of-approximation by low-frequency functions.
Definition 14 (Besov spaces). For any  $s > 0$  and  $1 \leq p, q \leq \infty$  we define the Besov space  $B_{p,q}^{s}(\mathcal{M})$  on the manifold  $\mathcal{M}$  as the space of functions  $f \in L^{p}(\mathcal{M})$  such that  $\| f \|_{B_{p,q}^{s}(\mathcal{M})} < \infty$  where
$$
\| f \| _ {B _ {p, q} ^ {s} (\mathcal {M})} = \left\{ \begin{array}{l l} \| f \| _ {L ^ {p} (\mathcal {M})} + \left(\sum_ {j \geq 0} \left(2 ^ {j s} \| \Phi_ {j} (\sqrt {\Delta}) f - f \| _ {L ^ {p} (\mathcal {M})}\right) ^ {q}\right) ^ {1 / q} & i f q <   + \infty \\ \| f \| _ {L ^ {p} (\mathcal {M})} + \sup  _ {j \geq 0} 2 ^ {j s} \| \Phi_ {j} (\sqrt {\Delta}) f - f \| _ {L ^ {p} (\mathcal {M})} & i f q = + \infty . \end{array} \right. \tag {24}
$$
The classical Besov spaces  $B_{2,2}^{s}$  coincide with the Sobolev spaces  $H^{s}$  on  $\mathbb{R}^d$ , in the sense that they define the same set of functions and equivalent norms—see for instance Giné and Nickl [21] section 4.3.6—and even on manifolds if one follows the construction of Triebel [51], pages 7.3-7.4 for Besov spaces. Since our definition of the Besov spaces is somewhat non-standard, we present a proof.
Proposition 15. For all  $s > 0$ ,  $H^{s}(\mathcal{M}) = B_{2,2}^{s}(\mathcal{M})$  as sets and there exist two constants  $C_1, C_2 > 0$  such that for all  $f \in H^{s}(\mathcal{M}) = B_{2,2}^{s}(\mathcal{M})$  we have
$$
C _ {1} \| f \| _ {H ^ {s} (\mathcal {M})} \leq \| f \| _ {B _ {2, 2} ^ {s} (\mathcal {M})} \leq C _ {2} \| f \| _ {H ^ {s} (\mathcal {M})}. \tag {25}
$$
4The space  $L^p (\mathcal{M})$  is the Banach space of functions (or rather their equivalence classes) that are integrable when raised to the power  $p < \infty$  or essentially bounded for  $p = \infty$ . See for instance Triebel [50] for details.
Proof. It is enough to prove (25), the rest will follow automatically. The main technical tools used in the proof are Result 10 and summation by parts. First, we prove the upper bound. Denote the support set of  $\Phi$  by  $K = \mathrm{supp}(\Phi)$ . Notice that
$$
\begin{array}{l} \left(\| f \| _ {B _ {2, 2} ^ {s} (\mathcal {M})} - \| f \| _ {L ^ {2} (\mathcal {M})}\right) ^ {2} = \sum_ {j \geq 0} 2 ^ {2 j s} \left\| \Phi_ {j} \left(\sqrt {\Delta}\right) f - f \right\| _ {L ^ {2} (\mathcal {M})} ^ {2} (26) \\ = \sum_ {j \geq 0} 2 ^ {2 j s} \sum_ {l: \sqrt {\lambda_ {l}} \notin 2 ^ {j} K} \left| \left\langle f _ {l}, f \right\rangle_ {L ^ {2} (\mathcal {M})} \right| ^ {2} (27) \\ \leq \sum_ {j \geq 0} 2 ^ {2 j s} \sum_ {l: \sqrt {\lambda_ {l}} > 2 ^ {j}} \left| \left\langle f _ {l}, f \right\rangle_ {L ^ {2} (\mathcal {M})} \right| ^ {2}. (28) \\ \end{array}
$$
The last inequality results from the fact that  $[0,1] \subset K$ . By Weyl's Law (Result 10), there exists a constant  $C > 0$  such that  $\lambda_l \leq Cl^{2/d}$ . Without loss of generality, we can assume that  $C = 2^{2r}$ ,  $r \in \mathbb{Z}_{>0}$ . Hence  $\sqrt{\lambda_l} > 2^j$  implies  $l > 2^{d(j - r)}$ , thus we have
$$
\begin{array}{l} \left(\| f \| _ {B _ {2, 2} ^ {s} (\mathcal {M})} - \| f \| _ {L ^ {2} (\mathcal {M})}\right) ^ {2} \leq \sum_ {j \geq 0} 2 ^ {2 j s} \sum_ {l > 2 ^ {d (j - r)}} \left| \langle f _ {l}, f \rangle_ {L ^ {2} (\mathcal {M})} \right| ^ {2} (29) \\ = \sum_ {0 \leq j \leq r} 2 ^ {2 j s} \sum_ {l > 2 ^ {d (j - r)}} \left| \langle f _ {l}, f \rangle_ {L ^ {2} (\mathcal {M})} \right| ^ {2} + \sum_ {j > r} 2 ^ {2 j s} \sum_ {l > 2 ^ {d (j - r)}} \left| \langle f _ {l}, f \rangle_ {L ^ {2} (\mathcal {M})} \right| ^ {2} (30) \\ \leq r 2 ^ {2 r s} \| f \| _ {L ^ {2} (\mathcal {M})} ^ {2} + 2 ^ {2 r s} \sum_ {j > 0} 2 ^ {2 j s} \sum_ {l > 2 ^ {d j}} \left| \langle f _ {l}, f \rangle_ {L ^ {2} (\mathcal {M})} \right| ^ {2}. (31) \\ \end{array}
$$
Now let  $R_{j} = \sum_{l > 2^{dj}}\big|\langle f_{l},f\rangle_{L^{2}(\mathcal{M})}\big|^{2}$  and  $S_{J} = \sum_{j = 1}^{J}2^{2js}\leq \frac{2^{2s}}{2^{2s} - 1} 2^{2Js},S_{0} = 0$  . Write
$$
\begin{array}{l} \sum_ {j > 0} 2 ^ {2 j s} \sum_ {l > 2 ^ {d j}} \left| \left\langle f _ {l}, f \right\rangle_ {L ^ {2} (\mathcal {M})} \right| ^ {2} = \sum_ {j > 0} \left(S _ {j} - S _ {j - 1}\right) R _ {j} (32) \\ = \sum_ {j > 0} S _ {j} \left(R _ {j} - R _ {j + 1}\right) (33) \\ = \sum_ {j > 0} S _ {j} \sum_ {2 ^ {d j} <   l \leq 2 ^ {d (j + 1)}} \left| \langle f _ {l}, f \rangle_ {L ^ {2} (\mathcal {M})} \right| ^ {2} (34) \\ \leq \frac {2 ^ {2 s}}{2 ^ {2 s} - 1} \sum_ {j > 0} 2 ^ {2 j s} \sum_ {2 ^ {d j} <   l \leq 2 ^ {d (j + 1)}} \left| \langle f _ {l}, f \rangle_ {L ^ {2} (\mathcal {M})} \right| ^ {2} (35) \\ \leq \frac {2 ^ {2 s}}{2 ^ {2 s} - 1} \sum_ {j > 0} \sum_ {2 ^ {d j} <   l \leq 2 ^ {d (j + 1)}} l ^ {2 s / d} \left| \langle f _ {l}, f \rangle_ {L ^ {2} (\mathcal {M})} \right| ^ {2} (36) \\ \leq \frac {c ^ {s} 2 ^ {2 s}}{2 ^ {2 s} - 1} \sum_ {j > 0} \quad \sum_ {2 ^ {d j} <   l \leq 2 ^ {d (j + 1)}} \lambda_ {l} ^ {s} \left| \langle f _ {l}, f \rangle_ {L ^ {2} (\mathcal {M})} \right| ^ {2} (37) \\ = \frac {c ^ {s} 2 ^ {2 s}}{2 ^ {2 s} - 1} \sum_ {l > 2 ^ {d}} \lambda_ {l} ^ {s} \left| \langle f _ {l}, f \rangle_ {L ^ {2} (\mathcal {M})} \right| ^ {2} (38) \\ \leq \frac {c ^ {s} 2 ^ {2 s}}{2 ^ {2 s} - 1} \sum_ {l \geq 0} \lambda_ {l} ^ {s} | \langle f _ {l}, f \rangle_ {L ^ {2} (\mathcal {M})} | ^ {2} (39) \\ \end{array}
$$
where we have used Result 10 to get existence of a  $c$  such that  $l^{2 / d}\leq c\lambda_l$ . This proves the upper bound. The proof for the lower bound is similar.
Proposition 15 provides a characterization of the Sobolev spaces  $H^{s}(\mathcal{M})$ . There is yet another characterization of these spaces that will be useful later, in terms of charts. We present this characterization as part of the following result.
Theorem 16. On the Sobolev space  $H^{s}(\mathcal{M})$ , the following norms are equivalent:
$$
\left\| f \right\| _ {H ^ {s} (\mathcal {M})} = \left(\sum_ {j = 0} ^ {\infty} (1 + \lambda_ {j}) ^ {s} \langle f, f _ {j} \rangle_ {L ^ {2} (\mathcal {M})} ^ {2}\right) ^ {1 / 2} \tag {40}
$$
$$
\left\| f \right\| _ {B _ {2, 2} ^ {s} (\mathcal {M})} = \left\| f \right\| _ {L ^ {2} (\mathcal {M})} + \left(\sum_ {j \geq 0} \left(2 ^ {j s} \| \Phi_ {j} (\sqrt {\Delta}) f - f \| _ {L ^ {2} (\mathcal {M})}\right) ^ {2}\right) ^ {1 / 2} \tag {41}
$$
$$
\left\| f \right\| _ {H _ {\mathcal {T}} ^ {s} (\mathcal {M})} = \left(\sum_ {l = 1} ^ {L} \left\| \left(\chi_ {l} f\right) \circ \phi_ {l} ^ {- 1} \right\| _ {H ^ {s} \left(\mathbb {R} ^ {d}\right)} ^ {2}\right) ^ {1 / 2} \tag {42}
$$
Proof. The equivalence between  $\| \cdot \|_{H^s (\mathcal{M})}$  and  $\| f\|_{B_{2,2}^s (\mathcal{M})}$  is given by Proposition 15. The proof of equivalence between  $\| \cdot \|_{H^s (\mathcal{M})}$  and  $\| f\|_{H_T^s (\mathcal{M})}$  can be found in De Vito et al. [16].
A.4 Gaussian Random Elements
Here we recall the definition of a Gaussian process as a Banach-space-valued random variable, following for instance van Zanten and van der Vaart [59].
Definition 17 (Gaussian random element). Let  $(\mathbb{B},\| \cdot \|_{\mathbb{B}})$  be a Banach space, and  $f$  be a Borel random variable with values in  $\mathbb{B}$  almost surely. We say that  $f$  is a Gaussian random element if  $b^{*}(f)$  is a univariate Gaussian random variable for every bounded linear functional  $b^{*}:\mathbb{B}\to \mathbb{R}$ .
Random variables of this kind are also sometimes called Gaussian in the sense of duality. One should think of a Gaussian random element as a generalization of a Gaussian process, but which is better-behaved from a function-analytic point of view and in particular does not require the process to be an actual function—as opposed to, for instance, a measure or a distribution. Many connections between the usual Gaussian processes and Gaussian random elements exist, see Lifshits [32], Ghosal and van der Vaart [20], Appendix I, van der Vaart and van Zanten [57] for details. The following observation about Gaussian random elements will be useful later.
Lemma 18. A Gaussian process  $f$  on the manifold  $\mathcal{M}$  with almost surely continuous sample paths is a Gaussian random element in the Banach space  $\left(\mathcal{C}(\mathcal{M}),\| \cdot \|_{\mathcal{C}(\mathcal{M})}\right)$  of continuous functions on  $\mathcal{M}$ .
Proof. Since  $\mathcal{C}(\mathcal{M})$  is separable, this follows from Lemma I.6 in Ghosal and van der Vaart [20].
A.5 A Technical Lemma
In order to apply Bernstein's inequality when going from the error at input locations to the  $L^2(p_0)$  error, we will use the following technical extrapolation lemma.
Lemma 19. For any function  $g: \mathcal{M} \to \mathbb{R}$ , a number  $\gamma \in \mathbb{R}_{>0} \setminus \mathbb{Z}_{>0}$  and a density  $p_0: \mathcal{M} \to \mathbb{R}_{>0}$  with  $1 \lesssim p_0$ , we have
$$
\left\| g \right\| _ {L ^ {\infty} (\mathcal {M})} \lesssim \left\| g \right\| _ {\mathcal {C H} ^ {\gamma} (\mathcal {M})} ^ {\frac {d}{2 \gamma + d}} \left\| g \right\| _ {L ^ {2} \left(p _ {0}\right)} ^ {\frac {2 \gamma}{2 \gamma + d}}. \tag {43}
$$
Proof. We use Lemma 15 from van der Vaart and van Zanten [56] and push it through charts. More precisely we have, using  $B_{\infty,\infty}^{\gamma}([0,1]^D) = \mathcal{CH}^{\gamma}([0,1]^D)$  for  $\gamma \notin \mathbb{Z}_{>0}$ , that
$$
\left\| g \right\| _ {L ^ {\infty} (\mathcal {M})} \leq \sum_ {l} \left\| \left(\chi_ {l} g\right) \circ \phi_ {l} ^ {- 1} \right\| _ {L ^ {\infty} \left(\mathcal {V} _ {l}\right)} \tag {44}
$$
$$
\lesssim \max  _ {l} \| (\chi_ {\iota} g) \circ \phi_ {l} ^ {- 1} \| _ {\mathcal {C H} ^ {\gamma} (\mathcal {V} _ {l})} ^ {\frac {d}{2 \gamma + d}} \| (\chi_ {\iota} g) \circ \phi_ {l} ^ {- 1} \| _ {L ^ {2} (\mathcal {V} _ {l})} ^ {\frac {2 \gamma}{2 \gamma + d}}. \tag {45}
$$
By definition of the manifold Hölder spaces, this gives
$$
\left\| g \right\| _ {L ^ {\infty} (\mathcal {M})} \lesssim \left\| g \right\| _ {\mathcal {C H} ^ {\gamma} (\mathcal {M})} ^ {\frac {d}{2 \gamma + d}} \max  _ {l} \left\| \left(\chi_ {l} g\right) \circ \phi_ {l} ^ {- 1} \right\| _ {L ^ {2} \left(\mathcal {V} _ {l}\right)} ^ {\frac {2 \gamma}{2 \gamma + d}}. \tag {46}
$$
Finally, since  $\chi_l$  are bounded, the charts are smooth, and  $p_0$  is lower bounded, we have
$$
\left\| \left(\chi_ {l} g\right) \circ \phi_ {l} ^ {- 1} \right\| _ {L ^ {2} \left(\mathcal {V} _ {l}\right)} ^ {2} = \int_ {\mathcal {V} _ {l}} \left| \left(\chi_ {l} g\right) \circ \phi_ {l} ^ {- 1} (y) \right| ^ {2} d y \lesssim \int_ {\mathcal {U} _ {l}} g ^ {2} (x) p _ {0} (x) \mu (d x) \lesssim \| g \| _ {L ^ {2} (\mathcal {M})} ^ {2} \tag {47}
$$
which gives the result.
This concludes the necessary preliminaries. We now turn to the proofs of our main results.
Throughout this section we use the notation from Appendix A and results from Appendices C to E. We start by defining our notation for Gaussian likelihood and probability distribution of the sample.
Definition 20. For every  $\pmb{x} \in \mathcal{M}^n$  and  $f: \mathcal{M} \to \mathbb{R}$  we define  $p_{\pmb{x},\pmb{y}}^{(f)}$  to be the joint distribution corresponding to the marginal  $p_{\pmb{x}} = p_0$  and conditional  $p_{\pmb{y}|\pmb{x}}^{(f)} = \mathrm{N}(f(\pmb{x}), \sigma_{\varepsilon}^2\mathbf{I})$ , where  $f(\pmb{x})$  is the vector with entries  $f(x_i)$ . Expectations with respect to  $p_{\pmb{x},\pmb{y}}^{(f)}$ , to  $p_{\pmb{x}}$  and to  $p_{\pmb{y}|\pmb{x}}^{(f)}$  we denote by  $\mathbb{E}_{\pmb{x},\pmb{y}}^{(f)}$ , by  $\mathbb{E}_x$  and by  $\mathbb{E}_{\pmb{y}|\pmb{x}}^{(f)}$ , respectively. Sometimes, when  $f$  is clear from the context, we omit the subscript  $^{(f)}$ .
Using van der Vaart and van Zanten [56], Theorem 1, which is valid for any compact metric space and thus also for  $\mathcal{M}$ , we can deduce a posterior contraction rate at data input locactions, with respect to the empirical  $L^2$ -norm<sup>5</sup>
$$
\left\| f \right\| _ {n} = \left(\frac {1}{n} \sum_ {i = 1} ^ {n} f \left(x _ {i}\right) ^ {2}\right) ^ {1 / 2} \tag {48}
$$
by studying the concentration functions with respect to the uniform norm. This is the purpose of the following lemma, whose proof mainly follows [56], but crucially relies on two new components: (i) the small ball asymptotics for manifolds we study in Appendix E and (ii) the characterization of the RKHS corresponding to both intrinsic and extrinsic priors as manifold Sobolev spaces we obtain in Appendix C. We recall that the prior  $\Pi_n$  may depend on  $n$  if we consider a truncated intrinsic Matérn process.
Theorem 21. Let  $\Pi_n$  denote the prior in either Theorem 5, Theorem 6 or Theorem 8 with smoothness parameter  $\nu > d/2$ . Let  $\mathbb{H}_n$  denote the corresponding RKHS. Define the CONCENTRATION FUNCTION for  $f_0 \in C(\mathcal{M})$  and  $\varepsilon > 0$  by
$$
\varphi_ {f _ {0}} (\varepsilon) = - \ln \mathbb {P} _ {f \sim \Pi_ {n}} \left[ \| f \| _ {L ^ {\infty} (\mathcal {M})} <   \varepsilon \right] + \inf  _ {f \in \mathbb {H} _ {n}: \| f - f _ {0} \| _ {L ^ {\infty} (\mathcal {M})} <   \varepsilon} \| f \| _ {\mathbb {H} _ {n}} ^ {2}. \tag {49}
$$
Then if  $f_0 \in H^\beta(\mathcal{M}) \cap B_{\infty, \infty}^\beta(\mathcal{M}), \beta > 0$  we have  $\varphi_{f_0}(\varepsilon_n) \leq n \varepsilon_n^2$  for  $\varepsilon_n$  a multiple of  $n^{-\frac{\min(\nu, \beta)}{2\nu + d}}$ .
Proof. The first term on the right-hand side of Equation (49) is bounded by  $C\varepsilon^{-d / \nu}$  by Lemma 33. To bound the second term, we assume, without loss of generality, that  $\nu \geq \beta$ . Consider an approximation  $f = \Phi_j(\sqrt{\Delta})f_0$  of  $f_0$ , where  $c\varepsilon \leq 2^{-\beta j} \leq \varepsilon$  and  $c > 0$  does not depend on  $j$ . Since we assume  $f_0 \in B_{\infty,\infty}^{\beta}(\mathcal{M})$ , by definition of  $B_{\infty,\infty}^{\beta}(\mathcal{M})$  we have
$$
\left\| f _ {0} - f \right\| _ {L ^ {\infty} (\mathcal {M})} \leq \left\| f _ {0} \right\| _ {B _ {\infty , \infty} ^ {\beta} (\mathcal {M})} 2 ^ {- \beta j} \lesssim \varepsilon \tag {50}
$$
where in the last inequality the  $B_{\infty, \infty}^{\beta}(\mathcal{M})$ -norm is the constant implied by notation  $\lesssim$ . We now show that
$$
\left\| f \right\| _ {\mathbb {H} _ {n}} ^ {2} \lesssim \varepsilon^ {- \frac {2}{\beta} (\nu - \beta + d / 2)}. \tag {51}
$$
First notice that by Lemma 24 and Proposition 27, for any prior considered here we have  $\mathbb{H}_n\subseteq H^{\nu +d / 2}(\mathcal{M})$  and  $\| \cdot \|_{\mathbb{H}_n}\lesssim \| \cdot \|_{H^{\nu +d / 2}(\mathcal{M})}$  with a constant that does not depend on  $n$ . Hence using
5This is actually a seminorm, but we follow the rest of the literature in referring to it as a norm. 6If  $\beta >\nu$  then  $f_0\in H^\beta (\mathcal{M})\cap B_{\infty ,\infty}^\beta \subseteq H^\nu (\mathcal{M})\cap B_{\infty ,\infty}^\nu (\mathcal{M})$  gives  $\varepsilon_{n}\propto n^{-\frac{\nu}{2\nu + d}} = n^{-\frac{\min(\beta)}{2\nu + d}}$
Result 10 (Weyl's Law) and properties of  $\Phi$ , we have
$$
\begin{array}{l} \| f \| _ {\mathbb {H} _ {n}} ^ {2} \lesssim \| f \| _ {H ^ {\nu + d / 2} (\mathcal {M})} ^ {2} (52) \\ = \sum_ {l \geq 0} (1 + \lambda_ {l}) ^ {\nu + d / 2} \Phi^ {2} \left(2 ^ {- j} \sqrt {\lambda_ {l}}\right) \left| \langle f _ {l}, f _ {0} \rangle_ {L ^ {2} (\mathcal {M})} \right| ^ {2} (53) \\ \leq \sum_ {l: \sqrt {\lambda_ {l}} \leq 2 ^ {j + 1}} (1 + \lambda_ {l}) ^ {\nu + d / 2 - \beta} (1 + \lambda_ {l}) ^ {\beta} \left| \langle f _ {l}, f _ {0} \rangle_ {L ^ {2} (\mathcal {M})} \right| ^ {2} (54) \\ \leq 2 ^ {(j + 2) (\nu + d / 2 - \beta)} \sum_ {l: \sqrt {\lambda_ {l}} \leq 2 ^ {j + 1}} (1 + \lambda_ {l}) ^ {\beta} \left| \langle f _ {l}, f _ {0} \rangle_ {L ^ {2} (\mathcal {M})} \right| ^ {2} (55) \\ \leq 2 ^ {(j + 2) (\nu + d / 2 - \beta)} \sum_ {l \geq 0} (1 + \lambda_ {l}) ^ {\beta} \left| \langle f _ {l}, f _ {0} \rangle_ {L ^ {2} (\mathcal {M})} \right| ^ {2} (56) \\ = 2 ^ {(j + 2) (\nu + d / 2 - \beta)} \| f _ {0} \| _ {H ^ {\beta} (\mathcal {M})} ^ {2} (57) \\ \lesssim \varepsilon^ {- \frac {1}{\beta} (\nu + d / 2 - \beta)} \| f _ {0} \| _ {H ^ {\beta} (\mathcal {M})} ^ {2} \lesssim \varepsilon^ {- \frac {2}{\beta} (\nu + d / 2 - \beta)} \| f _ {0} \| _ {H ^ {\beta} (\mathcal {M})} ^ {2}. (58) \\ \end{array}
$$
Our assumption  $\nu \geq \beta$  implies that
$$
\frac {2}{\beta} (\nu - \beta + d / 2) \geq \frac {d}{\beta} \geq \frac {d}{\nu}. \tag {59}
$$
Hence, we have  $\varepsilon^{-d/\nu} \leq \varepsilon^{-\frac{2}{\beta} (\nu - \beta + d/2)}$  which gives us  $\varphi_{f_0}(\varepsilon) \lesssim \varepsilon^{-\frac{2}{\beta} (\nu - \beta + d/2)}$ . It is then easy to check that  $\varepsilon_n = Mn^{-\frac{\beta}{2\nu + d}}$  satisfies  $\varphi_{f_0}(\varepsilon_n) \leq n\varepsilon_n^2$  for  $M > 0$  large enough.
From this, we deduce an upper bound on the error in the empirical  $L^2$  norm  $\| \cdot \| _n$ , that is, on the Euclidean distance between the posterior Gaussian process  $f$  and the ground truth function  $f_{0}$  evaluated at data locations  $x_{i}$ .
Lemma 22. Let  $\Pi_n$  denote the prior in either Theorem 5, Theorem 6 or Theorem 8 with smoothness parameter  $\nu > d/2$ . Fix  $f_0 \in H^\beta(\mathcal{M}) \cap B_{\infty,\infty}^\beta(\mathcal{M})$  with  $\beta > 0$ . Then
$$
\mathbb {E} _ {f \sim \Pi_ {n} (\cdot | \boldsymbol {x}, \boldsymbol {y})} \| f - f _ {0} \| _ {n} ^ {2} \leq \varepsilon_ {n} ^ {2} \tag {60}
$$
for  $\varepsilon_{n}\propto n^{-\frac{\min(\nu,\beta)}{2\nu + d}}$  with constant depending on  $f_0,\nu$  but not on  $\pmb{x}$
Proof. By Theorem 21 for  $\varepsilon_{n}$  a multiple of  $n^{-\frac{\min(\beta,\nu)}{2\nu + d}}$ , we have  $\varphi_{f_0}(\varepsilon_n)\leq n\varepsilon_n^2$ . By virtue of this, the proof of Theorem 1 and Proposition 11 of van der Vaart and van Zanten [56] imply the result. Indeed, the proof of Theorem 1 relies solely on the fact that  $\varphi_{f_0}(\varepsilon_n / 2)\leq n\varepsilon_n^2$  and an application of van der Vaart and van Zanten [56], Proposition 11. We have  $\varphi_{f_0}(\varepsilon_n)\leq n\varepsilon_n^2\leq n(2\varepsilon_n)^2$  and hence the condition is satisfied with  $\varepsilon_{n}$  replaced by  $2\varepsilon_{n}$ .