SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[1] B. Adcock, S. Brugiapaglia, and C. G. Webster. Sparse Polynomial Approximation of High-Dimensional Functions. Comput. Sci. Eng. Society for Industrial and Applied Mathematics, Philadelphia, PA, 2022.
[2] B. Adcock and J. M. Cardenas. Near-optimal sampling strategies for multivariate function approximation on general domains. SIAM J. Math. Data Sci., 2(3):607-630, 2020.
[3] B. Adcock, J. M. Cardenas, and N. Dexter. CAS4DL: Christoffel adaptive sampling for function approximation via deep learning. *Sampl. Theory Signal Process. Data Anal.*, 20(21):1-29, 2022.
[4] B. Adcock, J. M. Cardenas, N. Dexter, and S. Moraga. Towards optimal sampling for learning sparse approximation in high dimensions. In A. Nikeghbali, P. Pardalos, A. Raigorodskii, and T. M. Rassias, editors, High Dimensional Optimization and Probability, volume 191. Springer Optim. Appl., 2022.
[5] B. Adcock and A. C. Hansen. Compressive Imaging: Structure, Sampling, Learning. Cambridge University Press, Cambridge, UK, 2021.
[6] B. Adcock, A. C. Hansen, C. Poon, and B. Roman. Breaking the coherence barrier: a new theory for compressed sensing. Forum Math. Sigma, 5:e4, 2017.
[7] H. K. Aggarwal, M. P. Mani, and M. Jacob. MoDL: Model-based deep learning architecture for inverse problems. IEEE transactions on medical imaging, 38(2):394-405, 2018.
[8] A. Alaoui and M. W. Mahoney. Fast randomized kernel ridge regression with statistical guarantees. In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015.
[9] A. K. Alekseev, I. M. Navon, and M. E. Zelentsov. The estimation of functional uncertainty using polynomial chaos and adjoint equations. Internat. J. Numer. Methods Fluids, 67(3):328-341, 2011.
[10] A. C. Aristotelous, E. C. Mitchell, and V. Maroulas. ADLGM: An efficient adaptive sampling deep learning Galerkin method. J. Comput. Phys, page 111944, 2023.
[11] H. Avron, M. Kapralov, C. Musco, C. Musco, A. Velingker, and A. Zandieh. Random Fourier features for kernel ridge regression: approximation bounds and statistical guarantees. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 253–262. PMLR, 2017.
[12] H. Avron, M. Kapralov, C. Musco, C. Musco, A. Velingker, and A. Zandieh. A Universal Sampling Method for Reconstructing Signals with Simple Fourier Transforms. In Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing, STOC 2019, pages 1051-1063, New York, NY, USA, 2019. Association for Computing Machinery.
[13] H. H. Barrett and K. J. Myers. Foundations of Image Science. Wiley-Interscience, Hoboken, NJ, 2004.
[14] C. Basdevant, M. Deville, P. Haldenwang, J. Lacrouix, J. Ouazzani, R. Peyret, P. Orlandi, and A. Patera. Spectral and finite difference solutions of the Burgers equation. Comput. Fluids, 14(1):23-41, 1986.
[107] S. Ravishankar, J. C. Ye, and J. A. Fessler. Image reconstruction: from sparsity to data-adaptive methods and machine learning. Proc. IEEE, 108(1):86-109, 2020.
[108] B. Recht, M. Fazel, and P. A. Parrilo. Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Rev., 52(3):471-501, 2010.
[109] S. H. Rudy, S. L. Brunton, J. L. Proctor, and J. N. Kutz. Data-driven discovery of partial differential equations. Sci. Adv., 3(4), 2017.
[110] C. M. Sandino, J. Y. Cheng, F. Chen, M. Mardani, J. M. Pauly, and S. S. Vasanawala. Compressed sensing: from research to clinical practice with deep neural networks. IEEE Signal Process. Mag., 31(1):117-127, 2020.
[111] F. Sherry, M. Benning, J. C. De los Reyes, M. J. Graves, G. Maierhofer, G. Williams, C.-B. Schonlieb, and M. J. Ehrhardt. Learning the sampling pattern for MRI. IEEE Trans. Med. Imag., 39(12):4310-4321, 2020.
[112] J. Sirignano and K. Spiliopoulos. DGM: A deep learning algorithm for solving partial differential equations. J. Comput. Phys., 375:1339-1364, 2018.
[113] K. Tang, X. Wan, and C. Yang. Das: A deep adaptive sampling method for solving partial differential equations. arXiv preprint arXiv:2112.14038, 2021.
[114] T. Tang and T. Zhou. On discrete least-squares projection in unbounded domain with random evaluations and its application to parametric uncertainty quantification. SIAM J. Sci. Comput., 36(5):A2272-A2295, 2014.
[115] J. A. Tropp. User-friendly tail bounds for sums of random matrices. Found. Comput. Math., 12:389-434, 2012.
[116] M. Uecker. Parallel Magnetic Resonance Imaging. arXiv:1501.06209, 2015.
[117] M. Vidyasagar. An Introduction to Compressed Sensing. Comput. Sci. Eng. Society for Industrial and Applied Mathematics, Philadelphia, PA, 2019.
[118] S. Wang, H. Wang, and P. Perdikaris. Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets. Sci. Adv., 7(40):eabi8605, 2021.
[119] D. P. Woodruff. Sketching as a tool for numerical linear algebra. Foundations and Trends in Theoretical Computer Science, 10(1-2):1-157, 2014.
[120] J. Yu, L. Lu, X. Meng, and G. E. Karniadakis. Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems. Comput. Methods Appl. Mech. Engrg., 393:114823, 2022.
A Further literature and discussion
In this appendix, we provide some additional discussion that relates our work to existing literature.
A.1 Classical Christoffel functions
Let  $\mathbb{F}$  be a linear subspace of the Hilbert space  $\mathbb{X} = L_{\rho}^{2}(D)$ . The Christoffel function of  $\mathbb{F}$  is the function  $\theta \mapsto 1 / K(\mathbb{F})(\theta)$ , where
$$
K (\mathbb {F}) (\theta) = \sup \left\{\frac {| f (\theta) | ^ {2}}{| | f | | _ {\mathbb {X}} ^ {2}}: f \in \mathbb {F}, v \neq 0 \right\}, \quad \theta \in D.
$$
This is clearly a special case of Definition 2.6 with  $\mathbb{X}_0 = C(\overline{D})$  and  $L: D \to \mathcal{B}(\mathbb{X}_0, \mathbb{R})$  given by  $L(\theta)(f) = f(\theta)$ . The Christoffel function is a classical object in approximation theory, most typically associated with the case where  $\mathbb{F}$  is a space of algebraic polynomials [97]. Note that  $K(\mathbb{F})$  is precisely the diagonal of the reproducing kernel of  $\mathbb{F}$  in  $\mathbb{X}$ . It also follows immediately from Lemma E.1 that
$$
K (\mathbb {F}) (\theta) = \sum_ {i = 1} ^ {n} | f _ {i} (\theta) | ^ {2},
$$
for any orthonormal basis  $\{f_i\}_{i = 1}^n$  of  $\mathbb{F}$
A.2 Relation to leverage scores and leverage score sampling
Let  $D$  be a domain with a measure  $\lambda$  (typically the Lebesgue measure),  $\mathbb{F}$  be a class of functions  $f: D \to \mathbb{C}$  and  $\rho$  be a probability measure over  $D$  with Radon-Nikodym derivative  $p = \mathrm{d}\rho/\mathrm{d}\lambda$ . Then the leverage score is defined as
$$
\tau (\theta) = \tau (\mathbb {F}, p) (\theta) = \sup  \left\{\frac {| f (\theta) | ^ {2} p (\theta)}{\| f \| _ {p} ^ {2}}: f \in \mathbb {F} \backslash \{0 \} \right\}, \tag {A.1}
$$
where  $\| f\| _p^2 = \int_D|f(\theta)|^2 p(\theta)\mathrm{d}\lambda (\theta) = \int_D|f(\theta)|^2\mathrm{d}\rho (\theta)$ . See, e.g., [46] and references therein. Like with CS, leverage score sampling is a type of importance sampling where one draws samples randomly according to the probability density proportional to  $\tau$  (or, often in practice, some easy-to-compute upper bound  $\widetilde{\tau}$ ). We remark in passing that leverage score sampling is sometimes known as effective resistance or coherence motivated sampling in the literature. Note that in the literature on leverage scores, the empirical nondegeneracy condition (2.5) is sometimes referred to as the subspace embedding condition (see, e.g., [119]).
It is readily seen that (A.1) a special case of Definition 2.6. Let  $\mathbb{X} = L_{\rho}^{2}(D)$ . Then there are several different ways to formulate this.
(i) Consider  $D$  equipped with the measure  $\lambda$  and set  $L(\theta)(f) = \sqrt{p(\theta)} f(\theta)$ . Then nondegeneracy (2.3) holds with  $\alpha = \beta = 1$  and we have
$$
K (\mathbb {F}) (\theta) = \sup  \left\{\frac {| f (\theta) | ^ {2} p (\theta)}{\| f \| _ {\mathbb {X}} ^ {2}}: f \in \mathbb {F}, f \neq 0 \right\} =: K _ {a} (\theta).
$$
(ii) Alternatively, consider  $D$  equipped with the measure  $\rho$  and set  $L(\theta)(f) = f(\theta)$ . Then nondegeneracy (2.3) holds with  $\alpha = \beta = 1$  and we have
$$
K (\mathbb {F}) (\theta) = \sup  \left\{\frac {| f (\theta) | ^ {2}}{\| f \| _ {\mathbb {X}} ^ {2}}: f \in \mathbb {F}, f \neq 0 \right\} =: K _ {b} (\theta).
$$
Notice that  $K_{a}(\theta) = \tau (\mathbb{F},p)(\theta)$  is precisely the leverage score (A.1). Conversely,  $K_{b}(\theta) = \tau (\mathbb{F},p)(\theta) / p(\theta)$  differs from the leverage score (A.1) by the factor  $p(\theta)$ . Whether to include  $p(\theta)$  in the definition is one of convention. What is important is that both (i) and (ii) lead to exactly the same Christoffel sampling sampling measure  $\mu^{\star}$  in (2.11), which is equivalent to leverage score sampling for the standard regression problem. Indeed, this measure is
$$
\mathrm {d} \mu^ {\star} (\theta) \propto K _ {a} (\theta) \mathrm {d} \lambda (\theta) = \tau (\mathbb {F}, p) (\theta) \mathrm {d} \lambda (\theta)
$$
for case (i) or
$$
\mathrm {d} \mu^ {\star} (\theta) \propto K _ {b} (\theta) \mathrm {d} \rho (\theta) = \frac {\tau (\mathbb {F} , p) (\theta)}{p (\theta)} p (\theta) \mathrm {d} \lambda (\theta) = \tau (\mathbb {F}, p) (\theta) \mathrm {d} \lambda (\theta)
$$
for case (ii).
(Statistical) leverage scores are an old concept [29] that have recently found many applications in machine learning, including randomized numerical linear algebra [119], kernel methods [8, 12, 46, 47, 94], active learning [12, 30, 41, 46, 85], data analysis [77] and beyond. Leverage scores are perhaps most commonly encountered in the discrete setting. Here the matrix leverage score of a matrix  $A \in \mathbb{R}^{N \times n}$  is defined as
$$
\tau (i) = \tau (A)(i) = \max  _ {\substack {x \in \mathbb {R} ^ {d} \\ A x \neq 0}} \frac {(A x) _ {i} ^ {2}}{\| A x \| _ {2} ^ {2}} = a _ {i} ^ {\top} (A ^ {\top} A) ^ {- 1} a _ {i}, \quad i = 1, \dots , N, \tag{A.2}
$$
where  $a_{i}$  is the  $i$ th row of  $A$ . This is also a special case of Definition 2.6. Indeed, let  $\mathbb{X} = \mathbb{R}^N$  and  $\mathbb{Y} = \mathbb{R}$ , both equipped with the Euclidean inner product,  $D = \{1,\dots ,N\}$  be equipped with the uniform measure  $\rho$ ,  $L(i)(x) = x_{i}$  for a vector  $x = (x_{i})_{i = 1}^{N}\in \mathbb{R}^{N}$  and
$$
\mathbb {F} = \left\{A x: x \in \mathbb {R} ^ {n} \right\} \subseteq \mathbb {R} ^ {N}.
$$
Then (2.3) holds with  $\alpha = \beta = 1$  (notice that  $\rho$  is the uniform measure, not the uniform probability measure) and the generalized Christoffel function satisfies
$$
K (\mathbb {F}) (i) = \sup \left\{\frac {\| L (f) (i) \| _ {\mathbb {Y}} ^ {2}}{\| f \| _ {\mathbb {X}} ^ {2}}: f \in \mathbb {F}, f \neq 0 \right\} = \sup \left\{\frac {(A x) _ {i} ^ {2}}{\| A x \| _ {2} ^ {2}}: x \in \mathbb {R} ^ {n}, A x \neq 0 \right\} = \tau (A) (i).
$$
There are a number of generalizations of leverage scores in the literature. See, e.g., [101]. However, to the best of our knowledge, none of these are similar to the setting considered in this paper (in particular, Definition 2.6).
A.3 Other related work
Our general framework and results are related to recent work [43]. Here the authors consider approximating functions in arbitrary (linear or nonlinear) approximation spaces (model classes) via an empirical least-squares regression based on a semi-norm that depends on a (random) variable  $y \in Y \subseteq \mathbb{R}^d$ . In a similar spirit to Theorem 4.2 and Theorem 4.8, [43] establishes sample complexity and generalization error bounds, and then use the former to optimize the sampling measure. Specifically, [43, Thm. 2.8 and Cor. 2.9] is similar to Theorem 4.2 with  $d = 0$  (i.e., the case of classical coverings as opposed to subspace coverings) and [43, Thm. 2.12] is similar to Theorem 4.8, albeit without the additional technical steps used in the proof of Theorem 4.8 to obtain an expected error bound involving only the  $\mathbb{X}$ -norm on the left- and right-hand sides.
This framework is related to our framework with  $C = 1$  sampling operators and with  $\mathbb{X}$  being a Banach space of functions. In fact, it is somewhat more general in this case: (i) no inner product structure is assumed and, (ii) each measurement (which are implicit in the semi-norm in [43]) could belong to a different space. Note that one could readily incorporate (ii) into our framework, by making the target space  $\mathbb{Y}_c$  depend on  $\theta_c$ . However, (i) leads to fundamentally weaker sample complexity guarantees. For example, in the case of a linear subspace  $\mathbb{F}$ , the bounds in [43] scale log-cubically in  $n = \dim(\mathbb{F})$ , i.e., like  $c \cdot n^3 \cdot \log(n)$  (see [43, Sec. 3.1]), whereas the results in this work are log-linear, i.e.,  $c \cdot n \cdot \log(n)$  (see Corollary 4.7), and therefore near-optimal. A similar suboptimal log-cubic scaling also arises in the case where  $\mathbb{F}$  is a nonlinear space of  $s$ -sparse expansions in an orthonormal basis (see [43, Sec. 3.2]). See Appendix A.4 for more on this example.
As noted, the root cause of this suboptimal scaling is (i). In our framework, we impose that  $\mathbb{X}$  is a Hilbert space and each  $\mathbb{Y}_c$  is a semi-inner product space. This allows us to use matrix concentration techniques (specifically, the matrix Chernoff bound [115]) in combination with the technique of subspace coverings (Definition 4.1) to obtain better sample complexity bounds. We remark in passing that the special case of  $d = 0$  in our main sample complexity bound Theorem 4.2 (which, as noted above, is similar to [43]) does not require an inner product structure, i.e., it holds if  $\mathbb{X}$  is a Banach space and each  $\mathbb{Y}_c$  is a semi-normed vector space. However, if applied to the case of linear subspaces, it would also lead to suboptimal sample complexity estimates much as in [43, Sec. 3.1].
A.4 The case of sparse regression
In this section, we briefly discuss the application of our general framework to sparse regression. For convenience, we shall assume that  $\mathbb{X} = L_{\rho}^{2}(D)$  is the space of weighted square-integrable functions  $f: D \to \mathbb{C}, \mathbb{X} = C(\overline{D}), C = 1, \mathbb{Y} = \mathbb{Y}_1 = \mathbb{C}$  with the Euclidean inner product and  $L = L_1$  is the pointwise evaluation operator  $L(\theta)(f) = f(\theta)$ . However, the following discussion can be readily generalized to general settings considered in Section 2.1.
Let  $\Phi = \{\phi_i\}_{i=1}^N \subset \mathbb{X}$  be a collection of elements in  $\mathbb{X}$ , where  $N \in \mathbb{N}$ . We assume that this is a Riesz system, i.e., there exist constants  $0 < a \leq b < \infty$  such that
$$
a \sum_ {i = 1} ^ {N} | c _ {i} | ^ {2} \leq \left\| \sum_ {i = 1} ^ {N} c _ {i} \phi_ {i} \right\| _ {L _ {\rho} ^ {2} (D)} ^ {2} \leq b \sum_ {i = 1} ^ {N} | c _ {i} | ^ {2}, \quad \forall c _ {i} \in \mathbb {C}.
$$
Note that  $a = b = 1$  if and only if  $\Phi$  is an orthonormal system. Now let  $1\leq s\leq N$  and define
$$
\mathbb {F} = \mathbb {F} _ {s} = \left\{\sum_ {i \in S} c _ {i} \phi_ {i}: c _ {i} \in \mathbb {C}, \forall i, S \subseteq \{1, \dots , N \}, | S | = s \right\}.
$$
Then  $\mathbb{F}$  is the set of all  $s$ -sparse expansions in the system  $\Phi$ . In this case, we refer to (2.2) as a sparse regression problem. Notice that empirical nondegeneracy Eq. (2.5) is in this case equivalent to the Restricted Isometry Property (RIP) [48] for the matrix  $A = \frac{1}{\sqrt{m}} (\phi_j(\Theta_i))_{i,j=1}^{m,N} \in \mathbb{C}^{m \times N}$ .
Several remarks are in order. First, notice that  $\mathbb{F}_s - \mathbb{F}_s = \mathbb{F}_{2s}$ . Second,  $\mathbb{F}_s$  is union-of-subspaces model. Indeed,  $\mathbb{F}_s$  is precisely a union of  $d = \binom{N}{s}$  subspaces, each corresponding to a particular choice of support set  $S \subseteq \{1, \ldots, N\}$ ,  $|S| = s$ . When  $N$  and  $s$  are large, this is a large number, which makes the bound in Corollary 4.7 meaningless.
However, in this case, we can obtain a more appealing bound. Notice that the generalized Christoffel function
$$
K(\mathbb{F}_{s})(\theta) = \max_{\substack{S\subseteq \{1,\ldots ,N\} \\ |S|\leq s}}\left\{\frac{|f(\theta)|^{2}}{\|f\|_{L^{2}_{\rho}(D)}^{2}}:f = \sum_{i\in S}c_{i}\phi_{i}, c_{i}\in \mathbb{C},\forall i\right\} .
$$
It is a simple argument to see that
$$
| f (\theta) | ^ {2} \leq s \cdot a ^ {- 1} \cdot \max _ {i = 1, \dots , N} | \phi_ {i} (\theta) | ^ {2} \cdot \| f \| _ {L _ {\rho} ^ {2} (D)} ^ {2}.
$$
$$
K (\mathbb {F} _ {s}) (\theta) \leq s \cdot a ^ {- 1} \cdot \widetilde {K} (\theta), \qquad \widetilde {K} (\theta) := \max  _ {i = 1, \dots , N} | \phi_ {i} (\theta) | ^ {2}.
$$
The function  $\widetilde{K}_c$  serves two purposes. First, it can be used to upper bound  $\kappa(\mathbb{F}_s) = \int_D K(\mathbb{F}_s)(\theta) \, \mathrm{d}\rho(\theta)$ , i.e., the term appearing in the estimate in Corollary 4.7. Second, it can also be used as a convenient surrogate from which to sample, as computing  $\widetilde{K}(\theta)$  is typically easier than computing the true Christoffel function  $K(\mathbb{F}_s)(\theta)$ .
Suppose now that  $\| \phi_i\|_{L_\rho^\infty (D)} = \operatorname *{ess}\sup_{\theta \sim D}|\phi_i(\theta)|^2\leq K$  for all  $i = 1,\ldots ,N$ . In this case,  $\Phi$  is known as a bounded Riesz system [23]. Then
$$
K (\mathbb {F} _ {s}) (\theta) \leq s \cdot a ^ {- 1} \cdot \widetilde {K} (\theta) \leq s \cdot K \cdot a ^ {- 1}.
$$
Using this and the well-known estimate  $d = \binom{N}{s} \leq (\mathrm{e}N / s)^s$ , we deduce from Corollary 4.3 the sample complexity bound
$$
m \gtrsim c _ {\epsilon / 2} \cdot a ^ {- 1} \cdot s \cdot (s \cdot \log (\mathrm {e} N / s) + \log (s / \delta)).
$$
This is a substantial improvement on the bound implied directly by Corollary 4.7, which depends linearly on  $d = \binom{N}{s}$ . It also improves the bound of [43, Sec. 3.2] from log-cubic to log-quadratic (recall the discussion in Appendix A.3). However, the reader will note that it is not optimal, since  $m$  scales log-quadratically in  $s$ . One can improve this bound to log-linear by using a more sophisticated argument specific to the sparse regression case. See, e.g., [4, 23, 48] and references therein. The
extent to which this argument can be extended to the most general setting of our framework is a problem for future work.
Unfortunately, many Riesz systems of practical interest are not bounded. In this case, one has the sample complexity bound
$$
m \gtrsim c _ {\epsilon / 2} \cdot \widetilde {\kappa} \cdot a ^ {- 1} \cdot s \cdot (s \cdot \log (\mathrm {e} N / s) + \log (s / \delta)), \tag {A.3}
$$
$$
\widetilde {\kappa} = \int_ {D} \max _ {i = 1, \dots , N} | \phi_ {i} (\theta) | ^ {2} \mathrm {d} \rho (\theta).
$$
The bound (A.3) grows log-quadratically in  $s$ , as desired (it can also be improved to log-linear as mentioned above). However, the constant  $\widetilde{\kappa}$  may depend on  $N$ . We refer to [4] for an extensive discussion on this issue. It is notable that  $\widetilde{\kappa}$  can be estimated numerically. In cases of interest such as sparse multivariate polynomial regression, it is either bounded or mildly growing in  $N$  [4].
B Polynomial regression with gradient-augmented data
In this appendix, we describe the example considered in Section 3.1. Let  $f^{*}:\mathbb{R}^{d}\to \mathbb{R}$ . In the gradient-augmented learning problem, we consider training data of the form
$$
\left. \right.\left\{\left(\theta_ {i}, (\partial_ {k} f ^ {*} (\theta_ {i})) _ {k = 0} ^ {d}\right)\right\} _ {i = 1} ^ {m},
$$
where (for convenience)  $\partial_0f^* = f^*$  and  $\partial_k f^* = \partial f^* / \partial \theta_k$  otherwise.
B.1 Formulation in terms of the general sampling framework
We now describe how to formulate the gradient-augmented learning problem as a special case of the general framework. Let  $C = 1$ ,  $D = D_{1} = (\mathbb{R}^{d},\rho)$  where  $d\geq 1$  and  $\rho$  be the tensor-product Gaussian measure on  $\mathbb{R}^d$ , i.e.  $\mathrm{d}\rho = (2\pi)^{-d / 2}\mathrm{e}^{-\frac{1}{2}\| \theta \| _2^2}\mathrm{d}\theta$ . Then we let
$$
\mathbb {X} = H _ {\rho} ^ {1} (\mathbb {R} ^ {d}) = \left\{\partial_ {k} f \in L _ {\rho} ^ {2} (\mathbb {R} ^ {d}) \right\}
$$
be the Sobolev space of weighted square-integrable functions with respect to  $\rho$  with weak first-order derivatives that are also square-integrable with respect to  $\rho$ . This is a Hilbert space with inner product
$$
\langle f, g \rangle_ {\mathbb {X}} = \sum_ {k = 0} ^ {d} \int_ {\mathbb {R} ^ {d}} \partial_ {k} f (\theta) \partial_ {k} g (\theta) d \rho (\theta).
$$
Next, we let  $\mathbb{X}_0 = C^1 (\overline{D})$  and  $\mathbb{Y}$  be the Hilbert space
$$
\mathbb {Y} = (\mathbb {C} ^ {d + 1}, \langle \cdot , \cdot \rangle_ {2}),
$$
where  $\langle \cdot, \cdot \rangle_2$  is the Euclidean inner product. Then, we define the sampling operator  $L: D \to \mathcal{B}(\mathbb{X}_0, \mathbb{Y})$  as
$$
L (\theta) (f) = \left(\partial_ {k} f (\theta)\right) _ {k = 0} ^ {d}, \tag {B.1}
$$
$$
\int_ {D} \| L (\theta) (f) \| _ {\mathbb {Y}} ^ {2} \mathrm {d} \rho (\theta) = \sum_ {k = 0} ^ {d} \int_ {\mathbb {R} ^ {d}} | \partial_ {k} f (\theta) | ^ {2} \mathrm {d} \rho (\theta) = \| f \| _ {L _ {\rho} ^ {2} (D)} ^ {2} + \sum_ {k = 1} ^ {d} \| \partial_ {k} f \| _ {L _ {\rho} ^ {2} (D)} ^ {2} = \| f \| _ {H _ {\rho} ^ {1} (\mathbb {R} ^ {d})} ^ {2}.
$$
Thus, Assumption 2.2 holds for this example with  $\alpha = \beta = 1$ .
B.2 The approximation space: multivariate Hermite polynomials
The standard Hermite polynomials  $H_{k}(x)$  are eigenfunctions of the Sturm-Liouville problem
$$
\left(w u ^ {\prime}\right) ^ {\prime} + 2 k w u = 0,
$$
where  $w(x) = \mathrm{e}^{-x^2}$ . They are orthogonal on  $\mathbb{R}$  with respect to this function, specifically,
$$
\int_ {- \infty} ^ {\infty} H _ {n} (x) H _ {m} (x) w (x) \mathrm {d} x = \sqrt {\pi} 2 ^ {n} n! \epsilon_ {n, m}.
$$
They satisfy the three term recurrence with  $H_0(x) = 1$ ,  $H_1(x) = 2x$  and
$$
H _ {n + 1} (x) = 2 x H _ {n} (x) - 2 n H _ {n - 1} (x), \quad n = 1, 2, \dots , \tag {B.2}
$$
and their derivatives satisfy the relation  $H_0' = 0$  and
$$
H _ {n} ^ {\prime} = 2 n H _ {n - 1}, \quad n = 1, 2, \dots \tag {B.3}
$$
See, e.g., [28, Sec. 2.6.2] for standard properties of Hermite polynomials.
It is more convenient for our purposes to consider the standard Gaussian (probability) measure  $\mathbb{R}$ , i.e.,
$$
\mathrm {d} \rho (\theta) = \frac {1}{\sqrt {2 \pi}} \mathrm {e} ^ {- \frac {1}{2} \theta^ {2}} \mathrm {d} \theta .
$$
Then the orthonormal Hermite polynomials (sometimes known as the "probabilists' version") are given by
$$
\psi_ {n} (\theta) = \frac {1}{\sqrt {2 ^ {n} n !}} H _ {n} (\theta / \sqrt {2}).
$$
Manipulating (B.2), we see that the orthonormal Hermite polynomials satisfy  $\psi_0(\theta) = 1$ ,  $\psi_1(\theta) = y$  and
$$
\psi_ {n + 1} (\theta) = \frac {\theta}{\sqrt {n + 1}} \psi_ {n} (\theta) - \sqrt {\frac {n}{n + 1}} \psi_ {n - 1} (\theta), \quad n = 1, 2, \dots \tag {B.4}
$$
Also, we have  $\psi_n' = \frac{1}{\sqrt{2^n n!}} \frac{1}{\sqrt{2}} H_n'(\theta / \sqrt{2}) = \frac{1}{\sqrt{2^n n!} \sqrt{2}} 2n H_{n-1}(\theta / \sqrt{2})$ , and therefore  $\psi_0' = 0$  and
$$
\psi_ {n} ^ {\prime} (\theta) = \sqrt {n} \psi_ {n - 1} (\theta), \quad n = 1, 2, \dots . \tag {B.5}
$$
Using this, we readily deduce that the functions
$$
\phi_ {n} (\theta) = \frac {\psi_ {n} (\theta)}{\sqrt {1 + n}}, \quad n = 0, 1, \ldots ,
$$
form an orthonormal basis of the space
$$
H _ {\rho} ^ {1} (\mathbb {R}) = \{f \in L _ {\rho} ^ {2} (\mathbb {R}): f ^ {\prime} \in L _ {\rho} ^ {2} (\mathbb {R}) \},
$$
i.e., the Sobolev space of square-integrable functions  $f: \mathbb{R} \to \mathbb{R}$  (with respect to  $\rho$ ) with square-integrable (weak) first derivatives.
We extend to  $d$  dimensions via tensorization. We let