Title: Random Projections for Spectral Algorithms in Mis-specified Setting: Sobolev Norm Learning Rates and Minimax Optimality
Primary area: learning theory
Keywords: Random projections; Kernel methods; Mis-specified learning; Minimax optimality; Interpolation space

Abstract: Random projections (RP) offer an effective approach to reducing computational and storage costs while preserving the geometric structure of the data. However, existing studies primarily focus on the optimal generalization performance of kernel-regularized algorithms with RP in the well-specified setting under restrictive conditions. In this paper, we provide a comprehensive and improved analysis of the generalization performance of RP-based spectral algorithms under general conditions, without increasing computational complexity. By leveraging the embedding property of the RKHS and a refined analysis of the operator similarity, we establish optimal learning rates in Sobolev norms that match the minimax lower bounds up to logarithmic factors. For both randomized sketches and Nystr\"{o}m subsampling (uniform and leverage-based), we show that the projection dimension needed for optimality is proportional to the empirical effective dimension, yielding a significant reduction of computational cost while maintaining the statistical efficiency.  Our results do not rely on the uniform boundedness assumption on the target function and hold for a broad range of source conditions, i.e., $s\geq \alpha-1/\beta$, where $s,\beta$ and $\alpha$ denote the smoothness index, capacity index, and the embedding index, respectively. In the benign case when $\alpha=1/\beta$, the optimality holds for all $s> 0$. Moreover, the derived learning rates do not exhibit saturation effect, highlighting the advantage of random projections in handling smoother regression functions. Experimental results confirm our theoretical findings and demonstrate the practical effectiveness of RP.