SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\widehat{\boldsymbol{\alpha}}_{k}^{\lambda}(\mathcal{D}_{I}) = \operatorname *{argmin}_{\boldsymbol {\alpha}\in \mathbb{R}^{k}}\boldsymbol{\alpha}^{\top}\left(\boldsymbol{K}_{I} + k\lambda \boldsymbol{I}_{k}\right)\boldsymbol {\alpha} + \boldsymbol{\alpha}^{\top}\boldsymbol{y}_{I},
$$
where  $\pmb{K}_I = \pmb{\Phi}_I\pmb{\Phi}_I^\top \in \mathbb{R}^{k\times k}$  is the kernel matrix and  $\Phi_I = (\Phi (\pmb{x}_i))_{i\in I}\in \mathbb{R}^{n\times d}$  is the feature matrix. Simple calculation shows that  $\widehat{\beta}_k^\lambda (\mathcal{D}_I) = \Phi_I^\top \widehat{\alpha}_k^\lambda (\mathcal{D}_I)$ .
Using the same data-generating process as in the previous subsection, the results on kernel ridge regression are illustrated in Figure F8.
Figure F8: Heat map of the empirical distribution prediction risk of full-ensemble kernel ridge estimators, for varying ridge penalties  $\lambda$  and subsample aspect ratio  $\psi = p / d$  on the log-log scale. The prediction risk of the ridge ensemble ( $M = 100$ ) is computed using polynomial, Gaussian, and Laplacian kernel, using the default parameters as in Python package scikit-learn v1.2.2 [60] without the intercept terms. We scale  $\lambda$  by  $k / p$  so that we can obtain non-null estimators for the polynomial inner-product kernel (of degree 3).