SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where  $\mu$  is also a centered Matérn Gaussian measure with covariance operator  $1.8^{2}(-\frac{d^{2}}{dx^{2}} + 15^{2}\mathrm{Id})^{-3}$ . In the above display,  $\mathcal{F}$  maps a function to its Fourier series coefficients, and  $\mathcal{F}^{-1}$  expresses a Fourier coefficient sequence as a function expanded in the Fourier basis. The filter  $\chi$  is given by [32, Eqn. 3.6, p. 15] with  $\delta = 0.32$  and  $\beta = 0.1$ . We take  $k_{\max} = 64$ . The feature map  $\varphi$  lifts the notion of hidden neuron in neural network architectures to function space.
In Figures 2 and 3, the quantity represented on the vertical axis is an empirical approximation to the relative Bochner squared error:
$$
\frac {\frac {1}{N ^ {\prime}} \sum_ {n = 1} ^ {N ^ {\prime}} \| \mathcal {G} \left(u _ {n} ^ {\prime}\right) - \Phi \left(u _ {n} ^ {\prime}; \widehat {\alpha} , \left\{\theta_ {m} \right\}\right) \| _ {L ^ {2}} ^ {2}}{\frac {1}{N ^ {\prime}} \sum_ {n = 1} ^ {N ^ {\prime}} \| \mathcal {G} \left(u _ {n} ^ {\prime}\right) \| _ {L ^ {2}} ^ {2}} \approx \frac {\mathbb {E} _ {u \sim \nu} \| \mathcal {G} (u) - \Phi (u ; \widehat {\alpha} , \left\{\theta_ {m} \right\}) \| _ {L ^ {2}} ^ {2}}{\mathbb {E} _ {u \sim \nu} \| \mathcal {G} (u) \| _ {L ^ {2}} ^ {2}} = \frac {\mathcal {R} (\widehat {\alpha} ; \mathcal {G})}{\mathcal {R} (0 ; \mathcal {G})}. \tag {E.1}
$$
In (E.1),  $N' = 500$  is the size of the test set  $\{(u_n', \mathcal{G}(u_n'))\}_{n=1}^{N'}$ , where  $\{u_n'\}_{n=1}^{N'} \sim \nu^{\otimes N'}$  is disjoint from the training input set  $\{u_n\}_{n=1}^N$ . The input and output spaces are discretized on the same  $p$ -point equally spaced grid in  $(0, 2\pi)$ . Thus, the discretized version of any input or output function belonging to  $\mathcal{X} = \mathcal{Y} = L^2(\mathbb{T}; \mathbb{R})$  may be identified with an element of  $\mathbb{R}^p$ . In Figures 2a and 3a, the regularization factor is chosen as  $\lambda = 7 \cdot 10^{-4}/M$  and as  $\lambda = 3 \cdot 10^{-6}/\sqrt{N}$  in Figures 2b and 3b.
A priori, it is not clear whether this operator learning benchmark satisfies our theoretical assumptions because we cannot verify that the Burgers' solution operator belongs to the RKHS of  $(\varphi ,\mu)$  (or the range of some power of the RKHS kernel integral operator). At a more technical level, the feature map  $\varphi$  uses an unbounded activation function (ELU, the exponential linear unit), while our theory is only developed for bounded RFs (Assumption 3.1). Nevertheless, the empirically obtained parameter and sample complexity in Figure 2 reasonably fit the main result of our well-specified theory (Theorem 3.7).
(a) Varying  $M, \lambda, p$  for fixed  $N = 1548$ .
(b) Varying  $N, \lambda, p$  for fixed  $M = 10^4$ .
Figure 3: Squared test error—which empirically approximates the population risk  $\mathcal{R}(\widehat{\alpha};\mathcal{G})$  —versus discretized output space dimension  $p$ , where  $\mathcal{G}$  is the Burgers' equation solution operator (SM E).